From commits-noreply at bitbucket.org  Tue Feb  1 00:43:43 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Tue,  1 Feb 2011 00:43:43 +0100 (CET)
Subject: [pypy-svn] pypy default: Port CPython tests about complex.__format__
Message-ID: <20110131234343.2A19536C21D@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41508:bc9095fb2698
Date: 2011-02-01 00:09 +0100
http://bitbucket.org/pypy/pypy/changeset/bc9095fb2698/

Log:	Port CPython tests about complex.__format__

diff --git a/pypy/objspace/std/test/test_complexobject.py b/pypy/objspace/std/test/test_complexobject.py
--- a/pypy/objspace/std/test/test_complexobject.py
+++ b/pypy/objspace/std/test/test_complexobject.py
@@ -438,3 +438,105 @@
 
     def test_getnewargs(self):
         assert (1+2j).__getnewargs__() == (1.0, 2.0)
+
+    def test_format(self):
+        # empty format string is same as str()
+        assert format(1+3j, '') == str(1+3j)
+        assert format(1.5+3.5j, '') == str(1.5+3.5j)
+        assert format(3j, '') == str(3j)
+        assert format(3.2j, '') == str(3.2j)
+        assert format(3+0j, '') == str(3+0j)
+        assert format(3.2+0j, '') == str(3.2+0j)
+
+        # empty presentation type should still be analogous to str,
+        # even when format string is nonempty (issue #5920).
+        assert format(3.2+0j, '-') == str(3.2+0j)
+        assert format(3.2+0j, '<') == str(3.2+0j)
+        z = 4/7. - 100j/7.
+        assert format(z, '') == str(z)
+        assert format(z, '-') == str(z)
+        assert format(z, '<') == str(z)
+        assert format(z, '10') == str(z)
+        z = complex(0.0, 3.0)
+        assert format(z, '') == str(z)
+        assert format(z, '-') == str(z)
+        assert format(z, '<') == str(z)
+        assert format(z, '2') == str(z)
+        z = complex(-0.0, 2.0)
+        assert format(z, '') == str(z)
+        assert format(z, '-') == str(z)
+        assert format(z, '<') == str(z)
+        assert format(z, '3') == str(z)
+
+        assert format(1+3j, 'g') == '1+3j'
+        assert format(3j, 'g') == '0+3j'
+        assert format(1.5+3.5j, 'g') == '1.5+3.5j'
+
+        assert format(1.5+3.5j, '+g') == '+1.5+3.5j'
+        assert format(1.5-3.5j, '+g') == '+1.5-3.5j'
+        assert format(1.5-3.5j, '-g') == '1.5-3.5j'
+        assert format(1.5+3.5j, ' g') == ' 1.5+3.5j'
+        assert format(1.5-3.5j, ' g') == ' 1.5-3.5j'
+        assert format(-1.5+3.5j, ' g') == '-1.5+3.5j'
+        assert format(-1.5-3.5j, ' g') == '-1.5-3.5j'
+
+        assert format(-1.5-3.5e-20j, 'g') == '-1.5-3.5e-20j'
+        assert format(-1.5-3.5j, 'f') == '-1.500000-3.500000j'
+        assert format(-1.5-3.5j, 'F') == '-1.500000-3.500000j'
+        assert format(-1.5-3.5j, 'e') == '-1.500000e+00-3.500000e+00j'
+        assert format(-1.5-3.5j, '.2e') == '-1.50e+00-3.50e+00j'
+        assert format(-1.5-3.5j, '.2E') == '-1.50E+00-3.50E+00j'
+        assert format(-1.5e10-3.5e5j, '.2G') == '-1.5E+10-3.5E+05j'
+
+        assert format(1.5+3j, '<20g') ==  '1.5+3j              '
+        assert format(1.5+3j, '*<20g') == '1.5+3j**************'
+        assert format(1.5+3j, '>20g') ==  '              1.5+3j'
+        assert format(1.5+3j, '^20g') ==  '       1.5+3j       '
+        assert format(1.5+3j, '<20') ==   '(1.5+3j)            '
+        assert format(1.5+3j, '>20') ==   '            (1.5+3j)'
+        assert format(1.5+3j, '^20') ==   '      (1.5+3j)      '
+        assert format(1.123-3.123j, '^20.2') == '     (1.1-3.1j)     '
+
+        assert format(1.5+3j, '20.2f') == '          1.50+3.00j'
+        assert format(1.5+3j, '>20.2f') == '          1.50+3.00j'
+        assert format(1.5+3j, '<20.2f') == '1.50+3.00j          '
+        assert format(1.5e20+3j, '<20.2f') == '150000000000000000000.00+3.00j'
+        assert format(1.5e20+3j, '>40.2f') == '          150000000000000000000.00+3.00j'
+        assert format(1.5e20+3j, '^40,.2f') == '  150,000,000,000,000,000,000.00+3.00j  '
+        assert format(1.5e21+3j, '^40,.2f') == ' 1,500,000,000,000,000,000,000.00+3.00j '
+        assert format(1.5e21+3000j, ',.2f') == '1,500,000,000,000,000,000,000.00+3,000.00j'
+
+        # alternate is invalid
+        raises(ValueError, (1.5+0.5j).__format__, '#f')
+
+        # zero padding is invalid
+        raises(ValueError, (1.5+0.5j).__format__, '010f')
+
+        # '=' alignment is invalid
+        raises(ValueError, (1.5+3j).__format__, '=20')
+
+        # integer presentation types are an error
+        for t in 'bcdoxX':
+            raises(ValueError, (1.5+0.5j).__format__, t)
+
+        # make sure everything works in ''.format()
+        assert '*{0:.3f}*'.format(3.14159+2.71828j) == '*3.142+2.718j*'
+
+        # issue 3382: 'f' and 'F' with inf's and nan's
+        assert '{0:f}'.format(INF+0j) == 'inf+0.000000j'
+        assert '{0:F}'.format(INF+0j) == 'INF+0.000000j'
+        assert '{0:f}'.format(-INF+0j) == '-inf+0.000000j'
+        assert '{0:F}'.format(-INF+0j) == '-INF+0.000000j'
+        assert '{0:f}'.format(complex(INF, INF)) == 'inf+infj'
+        assert '{0:F}'.format(complex(INF, INF)) == 'INF+INFj'
+        assert '{0:f}'.format(complex(INF, -INF)) == 'inf-infj'
+        assert '{0:F}'.format(complex(INF, -INF)) == 'INF-INFj'
+        assert '{0:f}'.format(complex(-INF, INF)) == '-inf+infj'
+        assert '{0:F}'.format(complex(-INF, INF)) == '-INF+INFj'
+        assert '{0:f}'.format(complex(-INF, -INF)) == '-inf-infj'
+        assert '{0:F}'.format(complex(-INF, -INF)) == '-INF-INFj'
+
+        assert '{0:f}'.format(complex(NAN, 0)) == 'nan+0.000000j'
+        assert '{0:F}'.format(complex(NAN, 0)) == 'NAN+0.000000j'
+        assert '{0:f}'.format(complex(NAN, NAN)) == 'nan+nanj'
+        assert '{0:F}'.format(complex(NAN, NAN)) == 'NAN+NANj'

From commits-noreply at bitbucket.org  Tue Feb  1 00:43:43 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Tue,  1 Feb 2011 00:43:43 +0100 (CET)
Subject: [pypy-svn] pypy default: Skip the test until the feature is
	developed
Message-ID: <20110131234343.C322436C21D@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41509:0d13246ddb84
Date: 2011-02-01 00:13 +0100
http://bitbucket.org/pypy/pypy/changeset/0d13246ddb84/

Log:	Skip the test until the feature is developed

diff --git a/pypy/objspace/std/test/test_complexobject.py b/pypy/objspace/std/test/test_complexobject.py
--- a/pypy/objspace/std/test/test_complexobject.py
+++ b/pypy/objspace/std/test/test_complexobject.py
@@ -440,6 +440,7 @@
         assert (1+2j).__getnewargs__() == (1.0, 2.0)
 
     def test_format(self):
+        skip("FIXME")
         # empty format string is same as str()
         assert format(1+3j, '') == str(1+3j)
         assert format(1.5+3.5j, '') == str(1.5+3.5j)

From commits-noreply at bitbucket.org  Tue Feb  1 00:43:44 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Tue,  1 Feb 2011 00:43:44 +0100 (CET)
Subject: [pypy-svn] pypy default: Merge heads
Message-ID: <20110131234344.6278136C21D@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41510:77f94f44989a
Date: 2011-02-01 00:41 +0100
http://bitbucket.org/pypy/pypy/changeset/77f94f44989a/

Log:	Merge heads

diff --git a/pypy/objspace/std/test/test_complexobject.py b/pypy/objspace/std/test/test_complexobject.py
--- a/pypy/objspace/std/test/test_complexobject.py
+++ b/pypy/objspace/std/test/test_complexobject.py
@@ -439,6 +439,15 @@
     def test_getnewargs(self):
         assert (1+2j).__getnewargs__() == (1.0, 2.0)
 
+    def test_method_not_found_on_newstyle_instance(self):
+        class A(object):
+            pass
+        a = A()
+        a.__complex__ = lambda: 5j     # ignored
+        raises(TypeError, complex, a)
+        A.__complex__ = lambda self: 42j
+        assert complex(a) == 42j
+
     def test_format(self):
         skip("FIXME")
         # empty format string is same as str()

From commits-noreply at bitbucket.org  Tue Feb  1 00:45:30 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Tue,  1 Feb 2011 00:45:30 +0100 (CET)
Subject: [pypy-svn] pypy default: Remove this test, it is not correct.
Message-ID: <20110131234530.8421D36C221@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41511:eb503d8703af
Date: 2011-01-31 18:43 -0500
http://bitbucket.org/pypy/pypy/changeset/eb503d8703af/

Log:	Remove this test, it is not correct.

diff --git a/pypy/module/_io/test/test_io.py b/pypy/module/_io/test/test_io.py
--- a/pypy/module/_io/test/test_io.py
+++ b/pypy/module/_io/test/test_io.py
@@ -265,7 +265,7 @@
         import _io
 
         with _io.open(self.tmpfile, "w+") as f:
-            f.write("abc")
+            f.write(u"abc")
 
         with _io.open(self.tmpfile, "w+") as f:
             f.truncate()
@@ -288,13 +288,13 @@
         # The BOM is not written again when appending to a non-empty file
         for charset in ["utf-8-sig", "utf-16", "utf-32"]:
             with _io.open(self.tmpfile, "w", encoding=charset) as f:
-                f.write("aaa")
+                f.write(u"aaa")
                 pos = f.tell()
             with _io.open(self.tmpfile, "rb") as f:
                 res = f.read()
                 assert res == "aaa".encode(charset)
             with _io.open(self.tmpfile, "a", encoding=charset) as f:
-                f.write("xxx")
+                f.write(u"xxx")
             with _io.open(self.tmpfile, "rb") as f:
                 res = f.read()
                 assert res == "aaaxxx".encode(charset)
@@ -315,43 +315,3 @@
             assert res == "world\n"
             assert f.newlines == "\n"
             assert type(f.newlines) is unicode
-
-    def test_custom_decoder(self):
-        import codecs
-        import _io
-
-        class WeirdDecoder(codecs.IncrementalDecoder):
-            def decode(self, input, final=False):
-                return u".".join(input)
-
-        def weird_decoder(name):
-            if name == "test_decoder":
-                latin1 = codecs.lookup("latin-1")
-                return codecs.CodecInfo(
-                    name = "test_decoder",
-                    encode =latin1.encode,
-                    decode = None,
-                    incrementalencoder = None,
-                    streamreader = None,
-                    streamwriter = None,
-                    incrementaldecoder=WeirdDecoder
-                )
-
-        codecs.register(weird_decoder)
-
-        with _io.open(self.tmpfile, "wb") as f:
-            f.write("abcd")
-
-        with _io.open(self.tmpfile, encoding="test_decoder") as f:
-            decoded = f.read()
-
-        assert decoded == "a.b.c.d"
-        with _io.open(self.tmpfile, encoding="test_decoder") as f:
-            res = f.read(1)
-            assert res == "a"
-            cookie = f.tell()
-            res = f.read(1)
-            assert res == "."
-            f.seek(cookie)
-            res = f.read()
-            assert res == ".b.c.d"

From commits-noreply at bitbucket.org  Tue Feb  1 00:45:30 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Tue,  1 Feb 2011 00:45:30 +0100 (CET)
Subject: [pypy-svn] pypy default: Merged upstream.
Message-ID: <20110131234530.A8896282BA1@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41512:88c9d8b65c47
Date: 2011-01-31 18:45 -0500
http://bitbucket.org/pypy/pypy/changeset/88c9d8b65c47/

Log:	Merged upstream.


From fijal at codespeak.net  Tue Feb  1 08:22:28 2011
From: fijal at codespeak.net (fijal at codespeak.net)
Date: Tue,  1 Feb 2011 08:22:28 +0100 (CET)
Subject: [pypy-svn] r80261 - pypy/extradoc/planning
Message-ID: <20110201072228.71B6D282BA1@codespeak.net>

Author: fijal
Date: Tue Feb  1 08:22:25 2011
New Revision: 80261

Modified:
   pypy/extradoc/planning/jit.txt
Log:
This is done in jitviewer


Modified: pypy/extradoc/planning/jit.txt
==============================================================================
--- pypy/extradoc/planning/jit.txt	(original)
+++ pypy/extradoc/planning/jit.txt	Tue Feb  1 08:22:25 2011
@@ -2,7 +2,7 @@
 ---------
 
 - have benchmarks for jit compile time and jit memory usage
-
+  
 - generators are not really fast.  DONE, with the only restriction
   that the code in generators is never inlined into some caller.
 
@@ -17,9 +17,6 @@
 - maybe refactor a bit the x86 backend, particularly the register
   allocation
 
-- work more on visualization tools for traces or any profiler (unusable outside
-  the core group)
-
 - think about having different bytecode for "xyz %s" % stuff when left side
   is a compile time constant (and call unrolled version of string formatting
   loop in this case).


From fijal at codespeak.net  Tue Feb  1 08:24:47 2011
From: fijal at codespeak.net (fijal at codespeak.net)
Date: Tue,  1 Feb 2011 08:24:47 +0100 (CET)
Subject: [pypy-svn] r80262 - pypy/extradoc/planning
Message-ID: <20110201072447.368FE282BA1@codespeak.net>

Author: fijal
Date: Tue Feb  1 08:24:45 2011
New Revision: 80262

Modified:
   pypy/extradoc/planning/jit.txt
Log:
Kill a redundant task. We have other one, more precise below


Modified: pypy/extradoc/planning/jit.txt
==============================================================================
--- pypy/extradoc/planning/jit.txt	(original)
+++ pypy/extradoc/planning/jit.txt	Tue Feb  1 08:24:45 2011
@@ -3,9 +3,6 @@
 
 - have benchmarks for jit compile time and jit memory usage
   
-- generators are not really fast.  DONE, with the only restriction
-  that the code in generators is never inlined into some caller.
-
 - kill GUARD_(NO)_EXCEPTION; replace that by LAST_EXC_VALUE to load the
   current exception from the struct in memory, followed by a regular
   GUARD_CLASS.  (Armin: Look like a simplification, but it's a bit messy too)


From fijal at codespeak.net  Tue Feb  1 08:25:00 2011
From: fijal at codespeak.net (fijal at codespeak.net)
Date: Tue,  1 Feb 2011 08:25:00 +0100 (CET)
Subject: [pypy-svn] r80263 - pypy/extradoc/planning
Message-ID: <20110201072500.D4966282BA1@codespeak.net>

Author: fijal
Date: Tue Feb  1 08:24:59 2011
New Revision: 80263

Modified:
   pypy/extradoc/planning/jit.txt
Log:
A comment


Modified: pypy/extradoc/planning/jit.txt
==============================================================================
--- pypy/extradoc/planning/jit.txt	(original)
+++ pypy/extradoc/planning/jit.txt	Tue Feb  1 08:24:59 2011
@@ -24,6 +24,7 @@
   every second instruction.
 
   there is also manipulating of valuestackdepth and such.
+  XXX find precise python code
 
 - consider how much old style classes in stdlib hurt us.
 


From commits-noreply at bitbucket.org  Tue Feb  1 08:28:10 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Tue,  1 Feb 2011 08:28:10 +0100 (CET)
Subject: [pypy-svn] pypy default: Before anyone wakes up,
	disable geninterp. Will run benchmarks and see
Message-ID: <20110201072810.5D998282BA1@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: 
Changeset: r41513:ec46f2017ef8
Date: 2011-02-01 09:26 +0200
http://bitbucket.org/pypy/pypy/changeset/ec46f2017ef8/

Log:	Before anyone wakes up, disable geninterp. Will run benchmarks and
	see

diff --git a/pypy/config/pypyoption.py b/pypy/config/pypyoption.py
--- a/pypy/config/pypyoption.py
+++ b/pypy/config/pypyoption.py
@@ -165,7 +165,7 @@
                suggests=[("objspace.allworkingmodules", False)]),
 
     BoolOption("geninterp", "specify whether geninterp should be used",
-               default=True),
+               default=False),
 
     BoolOption("logbytecodes",
                "keep track of bytecode usage",

From commits-noreply at bitbucket.org  Tue Feb  1 08:28:11 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Tue,  1 Feb 2011 08:28:11 +0100 (CET)
Subject: [pypy-svn] pypy default: I don't want to,
	but hg is making me commit this work in progress
Message-ID: <20110201072811.2E654282BA1@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: 
Changeset: r41514:2f12f4b023ab
Date: 2011-02-01 09:27 +0200
http://bitbucket.org/pypy/pypy/changeset/2f12f4b023ab/

Log:	I don't want to, but hg is making me commit this work in progress

diff --git a/pypy/module/pypyjit/test/test_pypy_c_new.py b/pypy/module/pypyjit/test/test_pypy_c_new.py
--- a/pypy/module/pypyjit/test/test_pypy_c_new.py
+++ b/pypy/module/pypyjit/test/test_pypy_c_new.py
@@ -1,16 +1,85 @@
 
-import py
-py.test.skip("DEMO")
+import py, sys, re
+import subprocess
+import disassembler
+from pypy.tool.udir import udir
+from pypy.tool import logparser
 
-class TestPyPyCNew(object):
-    def test_one(self):
+class Trace(object):
+    pass
+
+class BaseTestPyPyC(object):
+    def setup_class(cls):
+        cls.tmpdir = udir.join('test-pypy-jit')
+        cls.tmpdir.ensure(dir=True)
+
+    def setup_method(self, meth):
+        self.filepath = self.tmpdir.join(meth.im_func.func_name + '.py')
+
+    def parse_out(self, out):
+        out = out.strip("\n")
+        if out == 'None':
+            return None
+        try:
+            return int(out)
+        except ValueError:
+            return out
+
+    def parse_func(self, func):
+        # find lines such as # LOOP <name> is in a line
+        code = disassembler.dis(func)
+        result = {}
+        for i, line in enumerate(py.code.Source(func)):
+            m = re.search('# LOOP (\w+)', line)
+            if m:
+                name = m.group(1)
+                result[name] = []
+                for opcode in code.opcodes:
+                    no = opcode.lineno - func.func_code.co_firstlineno
+                    if i - 1 <= no <= i + 1:
+                        result[name].append(opcode)
+        return result
+    
+    def run(self, func):
+        with self.filepath.open("w") as f:
+            f.write(str(py.code.Source(func)) + "\n")
+            f.write("print %s()\n" % func.func_name)
+        logfile = self.filepath.new(ext='.log')
+        pipe = subprocess.Popen([sys.executable, str(self.filepath)],
+                                stdout=subprocess.PIPE, stderr=subprocess.PIPE,
+                                env={'PYPYLOG': "jit-log-opt,jit-summary:" + str(logfile)})
+        pipe.wait()
+        stderr = pipe.stderr.read()
+        assert not stderr
+        res = self.parse_out(pipe.stdout.read())
+        bytecodes = self.parse_func(func)
+        assert res == func()
+        log = logparser.parse_log_file(str(logfile))
+        parts = logparser.extract_category(log, 'jit-log-opt-')
+        log.xxx
+        return Trace()
+
+class TestInfrastructure(BaseTestPyPyC):
+    def test_parse_func(self):
+        def f():
+            i = 0
+            x = 0
+            # LOOP name
+            z = x + 3
+            return z
+
+        res = self.parse_func(f)
+        assert len(res) == 1
+        assert len(res['name']) == 6
+
+    def test_full(self):
         def f():
             i = 0
             while i < 1003:
                 # LOOP one
                 i += 1
 
-        trace = self.run(f, [])
+        trace = self.run(f)
         loop = trace.get_loops('one')
         loop.get_bytecode(3, 'LOAD_FAST').match('''
         int_add
@@ -24,3 +93,6 @@
         loo.get_bytecode(5, 'INPLACE_ADD').match_stats(
             allocs='5-10'
             )
+
+class TestPyPyCNew(BaseTestPyPyC):
+    pass

From commits-noreply at bitbucket.org  Tue Feb  1 08:28:11 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Tue,  1 Feb 2011 08:28:11 +0100 (CET)
Subject: [pypy-svn] pypy default: merge heads
Message-ID: <20110201072811.E82D5282BDC@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: 
Changeset: r41515:0ec8b9b67a09
Date: 2011-02-01 09:27 +0200
http://bitbucket.org/pypy/pypy/changeset/0ec8b9b67a09/

Log:	merge heads

diff --git a/pypy/config/pypyoption.py b/pypy/config/pypyoption.py
--- a/pypy/config/pypyoption.py
+++ b/pypy/config/pypyoption.py
@@ -92,6 +92,7 @@
     "bz2"       : ["pypy.module.bz2.interp_bz2"],
     "pyexpat"   : ["pypy.module.pyexpat.interp_pyexpat"],
     "_ssl"      : ["pypy.module._ssl.interp_ssl"],
+    "_hashlib"  : ["pypy.module._ssl.interp_ssl"],
     "_minimal_curses": ["pypy.module._minimal_curses.fficurses"],
     }
 

From commits-noreply at bitbucket.org  Tue Feb  1 12:58:15 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Tue,  1 Feb 2011 12:58:15 +0100 (CET)
Subject: [pypy-svn] pypy default: Cleanup socket. Added 10 lines,
	but it's so much cleaner by now
Message-ID: <20110201115815.1ACBE282BA1@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: 
Changeset: r41516:79a284629147
Date: 2011-02-01 13:57 +0200
http://bitbucket.org/pypy/pypy/changeset/79a284629147/

Log:	Cleanup socket. Added 10 lines, but it's so much cleaner by now

diff --git a/lib-python/modified-2.7.0/socket.py b/lib-python/modified-2.7.0/socket.py
--- a/lib-python/modified-2.7.0/socket.py
+++ b/lib-python/modified-2.7.0/socket.py
@@ -157,11 +157,6 @@
 if sys.platform == "riscos":
     _socketmethods = _socketmethods + ('sleeptaskw',)
 
-# All the method names that must be delegated to either the real socket
-# object or the _closedsocket object.
-_delegate_methods = ("recv", "recvfrom", "recv_into", "recvfrom_into",
-                     "send", "sendto")
-
 class _closedsocket(object):
     __slots__ = []
     def _dummy(*args):
@@ -178,25 +173,43 @@
 
     __doc__ = _realsocket.__doc__
 
-    __slots__ = ["_sock", "__weakref__", "_io_refs", "_closed"
-                 ] + list(_delegate_methods)
-
     def __init__(self, family=AF_INET, type=SOCK_STREAM, proto=0, _sock=None):
         if _sock is None:
             _sock = _realsocket(family, type, proto)
         self._sock = _sock
         self._io_refs = 0
         self._closed = False
-        for method in _delegate_methods:
-            setattr(self, method, getattr(_sock, method))
 
-    def close(self, _closedsocket=_closedsocket,
-              _delegate_methods=_delegate_methods, setattr=setattr):
+    def send(self, data, flags=0):
+        return self._sock.send(data, flags=flags)
+    send.__doc__ = _realsocket.send.__doc__
+
+    def recv(self, buffersize, flags=0):
+        return self._sock.recv(buffersize, flags=flags)
+    recv.__doc__ = _realsocket.recv.__doc__
+
+    def recv_into(self, buffer, nbytes=0, flags=0):
+        return self._sock.recv_into(buffer, nbytes=nbytes, flags=flags)
+    recv_into.__doc__ = _realsocket.recv_into.__doc__
+
+    def recvfrom(self, buffersize, flags=0):
+        return self._sock.recvfrom(buffersize, flags=flags)
+    recvfrom.__doc__ = _realsocket.recvfrom.__doc__
+
+    def recvfrom_into(self, buffer, nbytes=0, flags=0):
+        return self._sock.recvfrom_into(buffer, nbytes=nbytes, flags=flags)
+    recvfrom_into.__doc__ = _realsocket.recvfrom_into.__doc__
+
+    def sendto(self, data, param2, param3=None):
+        if param3 is None:
+            return self._sock.sendto(data, param2)
+        else:
+            return self._sock.sendto(data, param2, param3)
+    sendto.__doc__ = _realsocket.sendto.__doc__
+
+    def close(self):
         # This function should not reference any globals. See issue #808164.
         self._sock = _closedsocket()
-        dummy = self._sock._dummy
-        for method in _delegate_methods:
-            setattr(self, method, dummy)
     close.__doc__ = _realsocket.close.__doc__
 
     def accept(self):

From commits-noreply at bitbucket.org  Tue Feb  1 13:21:41 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Tue,  1 Feb 2011 13:21:41 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix ssl.py and test_ssl.py (by chance,
	one more test passing on nightly
Message-ID: <20110201122141.0E8F82A2002@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: 
Changeset: r41517:f7e470d7823c
Date: 2011-02-01 14:21 +0200
http://bitbucket.org/pypy/pypy/changeset/f7e470d7823c/

Log:	Fix ssl.py and test_ssl.py (by chance, one more test passing on
	nightly run as well)

diff --git a/pypy/interpreter/gateway.py b/pypy/interpreter/gateway.py
--- a/pypy/interpreter/gateway.py
+++ b/pypy/interpreter/gateway.py
@@ -344,8 +344,9 @@
                                                        self.nextarg()))
 
     def visit__ObjSpace(self, el):
-        if self.finger != 0:
+        if self.finger not in [0, 1]:
             raise FastFuncNotSupported
+        self.args.append('space')
         self.unwrap.append("space")
 
     def visit__W_Root(self, el):
@@ -391,7 +392,9 @@
         unwrap_info = UnwrapSpec_FastFunc_Unwrap()
         unwrap_info.apply_over(unwrap_spec)
         narg = unwrap_info.n
-        args = ['space'] + unwrap_info.args
+        args = unwrap_info.args
+        if 'space' not in args:
+            args = ['space'] + args
         if args == unwrap_info.unwrap:
             fastfunc = func
         else:
@@ -528,8 +531,12 @@
                     self.__class__ = BuiltinCodePassThroughArguments1
                     self.func__args__ = func
             else:
-                self.__class__ = globals()['BuiltinCode%d' % arity]
-                setattr(self, 'fastfunc_%d' % arity, fastfunc)
+                if len(unwrap_spec) >= 2 and unwrap_spec[1] is ObjSpace:
+                    self.__class__ = globals()['BuiltinCodeSpace%d' % arity]
+                    setattr(self, 'fastfunc_space_%d' % arity, fastfunc)
+                else:
+                    self.__class__ = globals()['BuiltinCode%d' % arity]
+                    setattr(self, 'fastfunc_%d' % arity, fastfunc)
 
     def descr__reduce__(self, space):
         from pypy.interpreter.mixedmodule import MixedModule
@@ -717,6 +724,78 @@
             w_result = space.w_None
         return w_result
 
+class BuiltinCodeSpace1(BuiltinCode1):
+    _immutable_ = True
+    fast_natural_arity = 1
+    
+    def fastcall_1(self, space, w_func, w1):
+        try:
+            w_result = self.fastfunc_space_1(w1, space)
+        except DescrMismatch:
+            return  w1.descr_call_mismatch(space,
+                                           self.descrmismatch_op,
+                                           self.descr_reqcls,
+                                           Arguments(space, [w1]))
+        except Exception, e:
+            raise self.handle_exception(space, e)
+        if w_result is None:
+            w_result = space.w_None
+        return w_result
+
+class BuiltinCodeSpace2(BuiltinCode2):
+    _immutable_ = True
+    fast_natural_arity = 2
+    
+    def fastcall_2(self, space, w_func, w1, w2):
+        try:
+            w_result = self.fastfunc_space_2(w1, space, w2)
+        except DescrMismatch:
+            return  w1.descr_call_mismatch(space,
+                                           self.descrmismatch_op,
+                                           self.descr_reqcls,
+                                           Arguments(space, [w1, w2]))
+        except Exception, e:
+            raise self.handle_exception(space, e)
+        if w_result is None:
+            w_result = space.w_None
+        return w_result
+
+class BuiltinCodeSpace3(BuiltinCode3):
+    _immutable_ = True
+    fast_natural_arity = 3
+    
+    def fastcall_3(self, space, func, w1, w2, w3):
+        try:
+            w_result = self.fastfunc_space_3(w1, space, w2, w3)
+        except DescrMismatch:
+            return  w1.descr_call_mismatch(space,
+                                           self.descrmismatch_op,
+                                           self.descr_reqcls,
+                                           Arguments(space, [w1, w2, w3]))
+        except Exception, e:
+            raise self.handle_exception(space, e)
+        if w_result is None:
+            w_result = space.w_None
+        return w_result
+
+class BuiltinCodeSpace4(BuiltinCode4):
+    _immutable_ = True
+    fast_natural_arity = 4
+    
+    def fastcall_4(self, space, func, w1, w2, w3, w4):
+        try:
+            w_result = self.fastfunc_space_4(w1, space, w2, w3, w4)
+        except DescrMismatch:
+            return  w1.descr_call_mismatch(space,
+                                           self.descrmismatch_op,
+                                           self.descr_reqcls,
+                                           Arguments(space,
+                                                     [w1, w2, w3, w4]))
+        except Exception, e:
+            raise self.handle_exception(space, e)
+        if w_result is None:
+            w_result = space.w_None
+        return w_result
 
 class interp2app(Wrappable):
     """Build a gateway that calls 'f' at interp-level."""

diff --git a/pypy/interpreter/test/test_gateway.py b/pypy/interpreter/test/test_gateway.py
--- a/pypy/interpreter/test/test_gateway.py
+++ b/pypy/interpreter/test/test_gateway.py
@@ -14,7 +14,7 @@
         self.name = name
         self.defs_w = []
 
-class TestBuiltinCode: 
+class TestBuiltinCode(object): 
     def test_signature(self):
         def c(space, w_x, w_y, hello_w):
             pass
@@ -545,7 +545,42 @@
         """)
 
         assert space.is_true(w_res)
-        assert called == [w_app_f, w_app_f]       
+        assert called == [w_app_f, w_app_f]
+
+    def test_interp2app_fastcall_method_space(self):
+        space = self.space
+        w = space.wrap
+        w_3 = w(3)
+
+        def f(w_self, space, w_x):
+            return w_x
+        app_f = gateway.interp2app_temp(f, unwrap_spec=[gateway.W_Root,
+                                                        gateway.ObjSpace,
+                                                        gateway.W_Root])
+        w_app_f = w(app_f)
+
+        # sanity
+        assert isinstance(w_app_f.code, gateway.BuiltinCode2)
+
+        called = []
+        fastcall_2 = w_app_f.code.fastcall_2
+        def witness_fastcall_2(space, w_func, w_a, w_b):
+            called.append(w_func)
+            return fastcall_2(space, w_func, w_a, w_b)
+
+        w_app_f.code.fastcall_2 = witness_fastcall_2    
+    
+        w_res = space.appexec([w_app_f, w_3], """(f, x):
+        class A(object):
+           m = f # not a builtin function, so works as method
+        y = A().m(x)
+        b = A().m
+        z = b(x)
+        return y is x and z is x
+        """)
+
+        assert space.is_true(w_res)
+        assert called == [w_app_f, w_app_f]        
         
     def test_plain(self):
         space = self.space

diff --git a/lib-python/modified-2.7.0/ssl.py b/lib-python/modified-2.7.0/ssl.py
new file mode 100644
--- /dev/null
+++ b/lib-python/modified-2.7.0/ssl.py
@@ -0,0 +1,437 @@
+# Wrapper module for _ssl, providing some additional facilities
+# implemented in Python.  Written by Bill Janssen.
+
+"""\
+This module provides some more Pythonic support for SSL.
+
+Object types:
+
+  SSLSocket -- subtype of socket.socket which does SSL over the socket
+
+Exceptions:
+
+  SSLError -- exception raised for I/O errors
+
+Functions:
+
+  cert_time_to_seconds -- convert time string used for certificate
+                          notBefore and notAfter functions to integer
+                          seconds past the Epoch (the time values
+                          returned from time.time())
+
+  fetch_server_certificate (HOST, PORT) -- fetch the certificate provided
+                          by the server running on HOST at port PORT.  No
+                          validation of the certificate is performed.
+
+Integer constants:
+
+SSL_ERROR_ZERO_RETURN
+SSL_ERROR_WANT_READ
+SSL_ERROR_WANT_WRITE
+SSL_ERROR_WANT_X509_LOOKUP
+SSL_ERROR_SYSCALL
+SSL_ERROR_SSL
+SSL_ERROR_WANT_CONNECT
+
+SSL_ERROR_EOF
+SSL_ERROR_INVALID_ERROR_CODE
+
+The following group define certificate requirements that one side is
+allowing/requiring from the other side:
+
+CERT_NONE - no certificates from the other side are required (or will
+            be looked at if provided)
+CERT_OPTIONAL - certificates are not required, but if provided will be
+                validated, and if validation fails, the connection will
+                also fail
+CERT_REQUIRED - certificates are required, and will be validated, and
+                if validation fails, the connection will also fail
+
+The following constants identify various SSL protocol variants:
+
+PROTOCOL_SSLv2
+PROTOCOL_SSLv3
+PROTOCOL_SSLv23
+PROTOCOL_TLSv1
+"""
+
+import textwrap
+
+import _ssl             # if we can't import it, let the error propagate
+
+from _ssl import OPENSSL_VERSION_NUMBER, OPENSSL_VERSION_INFO, OPENSSL_VERSION
+from _ssl import SSLError
+from _ssl import CERT_NONE, CERT_OPTIONAL, CERT_REQUIRED
+from _ssl import PROTOCOL_SSLv2, PROTOCOL_SSLv3, PROTOCOL_SSLv23, PROTOCOL_TLSv1
+from _ssl import RAND_status, RAND_egd, RAND_add
+from _ssl import \
+     SSL_ERROR_ZERO_RETURN, \
+     SSL_ERROR_WANT_READ, \
+     SSL_ERROR_WANT_WRITE, \
+     SSL_ERROR_WANT_X509_LOOKUP, \
+     SSL_ERROR_SYSCALL, \
+     SSL_ERROR_SSL, \
+     SSL_ERROR_WANT_CONNECT, \
+     SSL_ERROR_EOF, \
+     SSL_ERROR_INVALID_ERROR_CODE
+
+from socket import socket, _fileobject, error as socket_error
+from socket import getnameinfo as _getnameinfo
+import base64        # for DER-to-PEM translation
+import errno
+
+class SSLSocket(socket):
+
+    """This class implements a subtype of socket.socket that wraps
+    the underlying OS socket in an SSL context when necessary, and
+    provides read and write methods over that channel."""
+
+    def __init__(self, sock, keyfile=None, certfile=None,
+                 server_side=False, cert_reqs=CERT_NONE,
+                 ssl_version=PROTOCOL_SSLv23, ca_certs=None,
+                 do_handshake_on_connect=True,
+                 suppress_ragged_eofs=True, ciphers=None):
+        socket.__init__(self, _sock=sock._sock)
+
+        if certfile and not keyfile:
+            keyfile = certfile
+        # see if it's connected
+        try:
+            socket.getpeername(self)
+        except socket_error, e:
+            if e.errno != errno.ENOTCONN:
+                raise
+            # no, no connection yet
+            self._sslobj = None
+        else:
+            # yes, create the SSL object
+            self._sslobj = _ssl.sslwrap(self._sock, server_side,
+                                        keyfile, certfile,
+                                        cert_reqs, ssl_version, ca_certs,
+                                        ciphers)
+            if do_handshake_on_connect:
+                self.do_handshake()
+        self.keyfile = keyfile
+        self.certfile = certfile
+        self.cert_reqs = cert_reqs
+        self.ssl_version = ssl_version
+        self.ca_certs = ca_certs
+        self.ciphers = ciphers
+        self.do_handshake_on_connect = do_handshake_on_connect
+        self.suppress_ragged_eofs = suppress_ragged_eofs
+        self._makefile_refs = 0
+
+    def read(self, len=1024):
+
+        """Read up to LEN bytes and return them.
+        Return zero-length string on EOF."""
+
+        try:
+            return self._sslobj.read(len)
+        except SSLError, x:
+            if x.args[0] == SSL_ERROR_EOF and self.suppress_ragged_eofs:
+                return ''
+            else:
+                raise
+
+    def write(self, data):
+
+        """Write DATA to the underlying SSL channel.  Returns
+        number of bytes of DATA actually transmitted."""
+
+        return self._sslobj.write(data)
+
+    def getpeercert(self, binary_form=False):
+
+        """Returns a formatted version of the data in the
+        certificate provided by the other end of the SSL channel.
+        Return None if no certificate was provided, {} if a
+        certificate was provided, but not validated."""
+
+        return self._sslobj.peer_certificate(binary_form)
+
+    def cipher(self):
+
+        if not self._sslobj:
+            return None
+        else:
+            return self._sslobj.cipher()
+
+    def send(self, data, flags=0):
+        if self._sslobj:
+            if flags != 0:
+                raise ValueError(
+                    "non-zero flags not allowed in calls to send() on %s" %
+                    self.__class__)
+            while True:
+                try:
+                    v = self._sslobj.write(data)
+                except SSLError, x:
+                    if x.args[0] == SSL_ERROR_WANT_READ:
+                        return 0
+                    elif x.args[0] == SSL_ERROR_WANT_WRITE:
+                        return 0
+                    else:
+                        raise
+                else:
+                    return v
+        else:
+            return self._sock.send(data, flags)
+
+    def sendto(self, data, flags_or_addr, addr=None):
+        if self._sslobj:
+            raise ValueError("sendto not allowed on instances of %s" %
+                             self.__class__)
+        elif addr is None:
+            return self._sock.sendto(data, flags_or_addr)
+        else:
+            return self._sock.sendto(data, flags_or_addr, addr)
+
+    def sendall(self, data, flags=0):
+        if self._sslobj:
+            if flags != 0:
+                raise ValueError(
+                    "non-zero flags not allowed in calls to sendall() on %s" %
+                    self.__class__)
+            amount = len(data)
+            count = 0
+            while (count < amount):
+                v = self.send(data[count:])
+                count += v
+            return amount
+        else:
+            return socket.sendall(self, data, flags)
+
+    def recv(self, buflen=1024, flags=0):
+        if self._sslobj:
+            if flags != 0:
+                raise ValueError(
+                    "non-zero flags not allowed in calls to recv() on %s" %
+                    self.__class__)
+            return self.read(buflen)
+        else:
+            return self._sock.recv(buflen, flags)
+
+    def recv_into(self, buffer, nbytes=None, flags=0):
+        if buffer and (nbytes is None):
+            nbytes = len(buffer)
+        elif nbytes is None:
+            nbytes = 1024
+        if self._sslobj:
+            if flags != 0:
+                raise ValueError(
+                  "non-zero flags not allowed in calls to recv_into() on %s" %
+                  self.__class__)
+            tmp_buffer = self.read(nbytes)
+            v = len(tmp_buffer)
+            buffer[:v] = tmp_buffer
+            return v
+        else:
+            return self._sock.recv_into(buffer, nbytes, flags)
+
+    def recvfrom(self, buflen=1024, flags=0):
+        if self._sslobj:
+            raise ValueError("recvfrom not allowed on instances of %s" %
+                             self.__class__)
+        else:
+            return self._sock.recvfrom(buflen, flags)
+
+    def recvfrom_into(self, buffer, nbytes=None, flags=0):
+        if self._sslobj:
+            raise ValueError("recvfrom_into not allowed on instances of %s" %
+                             self.__class__)
+        else:
+            return self._sock.recvfrom_into(buffer, nbytes, flags)
+
+    def pending(self):
+        if self._sslobj:
+            return self._sslobj.pending()
+        else:
+            return 0
+
+    def unwrap(self):
+        if self._sslobj:
+            s = self._sslobj.shutdown()
+            self._sslobj = None
+            return s
+        else:
+            raise ValueError("No SSL wrapper around " + str(self))
+
+    def shutdown(self, how):
+        self._sslobj = None
+        socket.shutdown(self, how)
+
+    def close(self):
+        if self._makefile_refs < 1:
+            self._sslobj = None
+            socket.close(self)
+        else:
+            self._makefile_refs -= 1
+
+    def do_handshake(self):
+
+        """Perform a TLS/SSL handshake."""
+
+        self._sslobj.do_handshake()
+
+    def connect(self, addr):
+
+        """Connects to remote ADDR, and then wraps the connection in
+        an SSL channel."""
+
+        # Here we assume that the socket is client-side, and not
+        # connected at the time of the call.  We connect it, then wrap it.
+        if self._sslobj:
+            raise ValueError("attempt to connect already-connected SSLSocket!")
+        socket.connect(self, addr)
+        self._sslobj = _ssl.sslwrap(self._sock, False, self.keyfile, self.certfile,
+                                    self.cert_reqs, self.ssl_version,
+                                    self.ca_certs, self.ciphers)
+        if self.do_handshake_on_connect:
+            self.do_handshake()
+
+    def accept(self):
+
+        """Accepts a new connection from a remote client, and returns
+        a tuple containing that new connection wrapped with a server-side
+        SSL channel, and the address of the remote client."""
+
+        newsock, addr = socket.accept(self)
+        return (SSLSocket(newsock,
+                          keyfile=self.keyfile,
+                          certfile=self.certfile,
+                          server_side=True,
+                          cert_reqs=self.cert_reqs,
+                          ssl_version=self.ssl_version,
+                          ca_certs=self.ca_certs,
+                          ciphers=self.ciphers,
+                          do_handshake_on_connect=self.do_handshake_on_connect,
+                          suppress_ragged_eofs=self.suppress_ragged_eofs),
+                addr)
+
+    def makefile(self, mode='r', bufsize=-1):
+
+        """Make and return a file-like object that
+        works with the SSL connection.  Just use the code
+        from the socket module."""
+
+        self._makefile_refs += 1
+        # close=True so as to decrement the reference count when done with
+        # the file-like object.
+        return _fileobject(self, mode, bufsize, close=True)
+
+
+
+def wrap_socket(sock, keyfile=None, certfile=None,
+                server_side=False, cert_reqs=CERT_NONE,
+                ssl_version=PROTOCOL_SSLv23, ca_certs=None,
+                do_handshake_on_connect=True,
+                suppress_ragged_eofs=True, ciphers=None):
+
+    return SSLSocket(sock, keyfile=keyfile, certfile=certfile,
+                     server_side=server_side, cert_reqs=cert_reqs,
+                     ssl_version=ssl_version, ca_certs=ca_certs,
+                     do_handshake_on_connect=do_handshake_on_connect,
+                     suppress_ragged_eofs=suppress_ragged_eofs,
+                     ciphers=ciphers)
+
+
+# some utility functions
+
+def cert_time_to_seconds(cert_time):
+
+    """Takes a date-time string in standard ASN1_print form
+    ("MON DAY 24HOUR:MINUTE:SEC YEAR TIMEZONE") and return
+    a Python time value in seconds past the epoch."""
+
+    import time
+    return time.mktime(time.strptime(cert_time, "%b %d %H:%M:%S %Y GMT"))
+
+PEM_HEADER = "-----BEGIN CERTIFICATE-----"
+PEM_FOOTER = "-----END CERTIFICATE-----"
+
+def DER_cert_to_PEM_cert(der_cert_bytes):
+
+    """Takes a certificate in binary DER format and returns the
+    PEM version of it as a string."""
+
+    if hasattr(base64, 'standard_b64encode'):
+        # preferred because older API gets line-length wrong
+        f = base64.standard_b64encode(der_cert_bytes)
+        return (PEM_HEADER + '\n' +
+                textwrap.fill(f, 64) + '\n' +
+                PEM_FOOTER + '\n')
+    else:
+        return (PEM_HEADER + '\n' +
+                base64.encodestring(der_cert_bytes) +
+                PEM_FOOTER + '\n')
+
+def PEM_cert_to_DER_cert(pem_cert_string):
+
+    """Takes a certificate in ASCII PEM format and returns the
+    DER-encoded version of it as a byte sequence"""
+
+    if not pem_cert_string.startswith(PEM_HEADER):
+        raise ValueError("Invalid PEM encoding; must start with %s"
+                         % PEM_HEADER)
+    if not pem_cert_string.strip().endswith(PEM_FOOTER):
+        raise ValueError("Invalid PEM encoding; must end with %s"
+                         % PEM_FOOTER)
+    d = pem_cert_string.strip()[len(PEM_HEADER):-len(PEM_FOOTER)]
+    return base64.decodestring(d)
+
+def get_server_certificate(addr, ssl_version=PROTOCOL_SSLv3, ca_certs=None):
+
+    """Retrieve the certificate from the server at the specified address,
+    and return it as a PEM-encoded string.
+    If 'ca_certs' is specified, validate the server cert against it.
+    If 'ssl_version' is specified, use it in the connection attempt."""
+
+    host, port = addr
+    if (ca_certs is not None):
+        cert_reqs = CERT_REQUIRED
+    else:
+        cert_reqs = CERT_NONE
+    s = wrap_socket(socket(), ssl_version=ssl_version,
+                    cert_reqs=cert_reqs, ca_certs=ca_certs)
+    s.connect(addr)
+    dercert = s.getpeercert(True)
+    s.close()
+    return DER_cert_to_PEM_cert(dercert)
+
+def get_protocol_name(protocol_code):
+    if protocol_code == PROTOCOL_TLSv1:
+        return "TLSv1"
+    elif protocol_code == PROTOCOL_SSLv23:
+        return "SSLv23"
+    elif protocol_code == PROTOCOL_SSLv2:
+        return "SSLv2"
+    elif protocol_code == PROTOCOL_SSLv3:
+        return "SSLv3"
+    else:
+        return "<unknown>"
+
+
+# a replacement for the old socket.ssl function
+
+def sslwrap_simple(sock, keyfile=None, certfile=None):
+
+    """A replacement for the old socket.ssl function.  Designed
+    for compability with Python 2.5 and earlier.  Will disappear in
+    Python 3.0."""
+
+    if hasattr(sock, "_sock"):
+        sock = sock._sock
+
+    ssl_sock = _ssl.sslwrap(sock, 0, keyfile, certfile, CERT_NONE,
+                            PROTOCOL_SSLv23, None)
+    try:
+        sock.getpeername()
+    except socket_error:
+        # no, no connection yet
+        pass
+    else:
+        # yes, do the handshake
+        ssl_sock.do_handshake()
+
+    return ssl_sock

diff --git a/lib-python/modified-2.7.0/test/test_ssl.py b/lib-python/modified-2.7.0/test/test_ssl.py
new file mode 100644
--- /dev/null
+++ b/lib-python/modified-2.7.0/test/test_ssl.py
@@ -0,0 +1,1318 @@
+# Test the support for SSL and sockets
+
+import sys
+import unittest
+from test import test_support
+import asyncore
+import socket
+import select
+import time
+import gc
+import os
+import errno
+import pprint
+import urllib, urlparse
+import traceback
+import weakref
+import functools
+import platform
+
+from BaseHTTPServer import HTTPServer
+from SimpleHTTPServer import SimpleHTTPRequestHandler
+
+# Optionally test SSL support, if we have it in the tested platform
+skip_expected = False
+try:
+    import ssl
+except ImportError:
+    skip_expected = True
+
+HOST = test_support.HOST
+CERTFILE = None
+SVN_PYTHON_ORG_ROOT_CERT = None
+
+def handle_error(prefix):
+    exc_format = ' '.join(traceback.format_exception(*sys.exc_info()))
+    if test_support.verbose:
+        sys.stdout.write(prefix + exc_format)
+
+
+class BasicTests(unittest.TestCase):
+
+    def test_sslwrap_simple(self):
+        # A crude test for the legacy API
+        try:
+            ssl.sslwrap_simple(socket.socket(socket.AF_INET))
+        except IOError, e:
+            if e.errno == 32: # broken pipe when ssl_sock.do_handshake(), this test doesn't care about that
+                pass
+            else:
+                raise
+        try:
+            ssl.sslwrap_simple(socket.socket(socket.AF_INET)._sock)
+        except IOError, e:
+            if e.errno == 32: # broken pipe when ssl_sock.do_handshake(), this test doesn't care about that
+                pass
+            else:
+                raise
+
+# Issue #9415: Ubuntu hijacks their OpenSSL and forcefully disables SSLv2
+def skip_if_broken_ubuntu_ssl(func):
+    # We need to access the lower-level wrapper in order to create an
+    # implicit SSL context without trying to connect or listen.
+    try:
+        import _ssl
+    except ImportError:
+        # The returned function won't get executed, just ignore the error
+        pass
+    @functools.wraps(func)
+    def f(*args, **kwargs):
+        try:
+            s = socket.socket(socket.AF_INET)
+            _ssl.sslwrap(s._sock, 0, None, None,
+                         ssl.CERT_NONE, ssl.PROTOCOL_SSLv2, None, None)
+        except ssl.SSLError as e:
+            if (ssl.OPENSSL_VERSION_INFO == (0, 9, 8, 15, 15) and
+                platform.linux_distribution() == ('debian', 'squeeze/sid', '')
+                and 'Invalid SSL protocol variant specified' in str(e)):
+                raise unittest.SkipTest("Patched Ubuntu OpenSSL breaks behaviour")
+        return func(*args, **kwargs)
+    return f
+
+
+class BasicSocketTests(unittest.TestCase):
+
+    def test_constants(self):
+        ssl.PROTOCOL_SSLv2
+        ssl.PROTOCOL_SSLv23
+        ssl.PROTOCOL_SSLv3
+        ssl.PROTOCOL_TLSv1
+        ssl.CERT_NONE
+        ssl.CERT_OPTIONAL
+        ssl.CERT_REQUIRED
+
+    def test_random(self):
+        v = ssl.RAND_status()
+        if test_support.verbose:
+            sys.stdout.write("\n RAND_status is %d (%s)\n"
+                             % (v, (v and "sufficient randomness") or
+                                "insufficient randomness"))
+        try:
+            ssl.RAND_egd(1)
+        except TypeError:
+            pass
+        else:
+            print "didn't raise TypeError"
+        ssl.RAND_add("this is a random string", 75.0)
+
+    @test_support.impl_detail("obscure test")
+    def test_parse_cert(self):
+        # note that this uses an 'unofficial' function in _ssl.c,
+        # provided solely for this test, to exercise the certificate
+        # parsing code
+        p = ssl._ssl._test_decode_cert(CERTFILE, False)
+        if test_support.verbose:
+            sys.stdout.write("\n" + pprint.pformat(p) + "\n")
+
+    def test_DER_to_PEM(self):
+        with open(SVN_PYTHON_ORG_ROOT_CERT, 'r') as f:
+            pem = f.read()
+        d1 = ssl.PEM_cert_to_DER_cert(pem)
+        p2 = ssl.DER_cert_to_PEM_cert(d1)
+        d2 = ssl.PEM_cert_to_DER_cert(p2)
+        self.assertEqual(d1, d2)
+        if not p2.startswith(ssl.PEM_HEADER + '\n'):
+            self.fail("DER-to-PEM didn't include correct header:\n%r\n" % p2)
+        if not p2.endswith('\n' + ssl.PEM_FOOTER + '\n'):
+            self.fail("DER-to-PEM didn't include correct footer:\n%r\n" % p2)
+
+    def test_openssl_version(self):
+        n = ssl.OPENSSL_VERSION_NUMBER
+        t = ssl.OPENSSL_VERSION_INFO
+        s = ssl.OPENSSL_VERSION
+        self.assertIsInstance(n, (int, long))
+        self.assertIsInstance(t, tuple)
+        self.assertIsInstance(s, str)
+        # Some sanity checks follow
+        # >= 0.9
+        self.assertGreaterEqual(n, 0x900000)
+        # < 2.0
+        self.assertLess(n, 0x20000000)
+        major, minor, fix, patch, status = t
+        self.assertGreaterEqual(major, 0)
+        self.assertLess(major, 2)
+        self.assertGreaterEqual(minor, 0)
+        self.assertLess(minor, 256)
+        self.assertGreaterEqual(fix, 0)
+        self.assertLess(fix, 256)
+        self.assertGreaterEqual(patch, 0)
+        self.assertLessEqual(patch, 26)
+        self.assertGreaterEqual(status, 0)
+        self.assertLessEqual(status, 15)
+        # Version string as returned by OpenSSL, the format might change
+        self.assertTrue(s.startswith("OpenSSL {:d}.{:d}.{:d}".format(major, minor, fix)),
+                        (s, t))
+
+    def test_ciphers(self):
+        if not test_support.is_resource_enabled('network'):
+            return
+        remote = ("svn.python.org", 443)
+        with test_support.transient_internet(remote[0]):
+            s = ssl.wrap_socket(socket.socket(socket.AF_INET),
+                                cert_reqs=ssl.CERT_NONE, ciphers="ALL")
+            s.connect(remote)
+            s = ssl.wrap_socket(socket.socket(socket.AF_INET),
+                                cert_reqs=ssl.CERT_NONE, ciphers="DEFAULT")
+            s.connect(remote)
+            # Error checking occurs when connecting, because the SSL context
+            # isn't created before.
+            s = ssl.wrap_socket(socket.socket(socket.AF_INET),
+                                cert_reqs=ssl.CERT_NONE, ciphers="^$:,;?*'dorothyx")
+            with self.assertRaisesRegexp(ssl.SSLError, "No cipher can be selected"):
+                s.connect(remote)
+
+    @test_support.cpython_only
+    def test_refcycle(self):
+        # Issue #7943: an SSL object doesn't create reference cycles with
+        # itself.
+        s = socket.socket(socket.AF_INET)
+        ss = ssl.wrap_socket(s)
+        wr = weakref.ref(ss)
+        del ss
+        self.assertEqual(wr(), None)
+
+    def test_wrapped_unconnected(self):
+        # The _delegate_methods in socket.py are correctly delegated to by an
+        # unconnected SSLSocket, so they will raise a socket.error rather than
+        # something unexpected like TypeError.
+        s = socket.socket(socket.AF_INET)
+        ss = ssl.wrap_socket(s)
+        self.assertRaises(socket.error, ss.recv, 1)
+        self.assertRaises(socket.error, ss.recv_into, bytearray(b'x'))
+        self.assertRaises(socket.error, ss.recvfrom, 1)
+        self.assertRaises(socket.error, ss.recvfrom_into, bytearray(b'x'), 1)
+        self.assertRaises(socket.error, ss.send, b'x')
+        self.assertRaises(socket.error, ss.sendto, b'x', ('0.0.0.0', 0))
+
+
+class NetworkedTests(unittest.TestCase):
+
+    def test_connect(self):
+        with test_support.transient_internet("svn.python.org"):
+            s = ssl.wrap_socket(socket.socket(socket.AF_INET),
+                                cert_reqs=ssl.CERT_NONE)
+            s.connect(("svn.python.org", 443))
+            c = s.getpeercert()
+            if c:
+                self.fail("Peer cert %s shouldn't be here!")
+            s.close()
+
+            # this should fail because we have no verification certs
+            s = ssl.wrap_socket(socket.socket(socket.AF_INET),
+                                cert_reqs=ssl.CERT_REQUIRED)
+            try:
+                s.connect(("svn.python.org", 443))
+            except ssl.SSLError:
+                pass
+            finally:
+                s.close()
+
+            # this should succeed because we specify the root cert
+            s = ssl.wrap_socket(socket.socket(socket.AF_INET),
+                                cert_reqs=ssl.CERT_REQUIRED,
+                                ca_certs=SVN_PYTHON_ORG_ROOT_CERT)
+            try:
+                s.connect(("svn.python.org", 443))
+            finally:
+                s.close()
+
+    @unittest.skipIf(os.name == "nt", "Can't use a socket as a file under Windows")
+    def test_makefile_close(self):
+        # Issue #5238: creating a file-like object with makefile() shouldn't
+        # delay closing the underlying "real socket" (here tested with its
+        # file descriptor, hence skipping the test under Windows).
+        with test_support.transient_internet("svn.python.org"):
+            ss = ssl.wrap_socket(socket.socket(socket.AF_INET))
+            ss.connect(("svn.python.org", 443))
+            fd = ss.fileno()
+            f = ss.makefile()
+            f.close()
+            # The fd is still open
+            os.read(fd, 0)
+            # Closing the SSL socket should close the fd too
+            ss.close()
+            gc.collect()
+            with self.assertRaises(OSError) as e:
+                os.read(fd, 0)
+            self.assertEqual(e.exception.errno, errno.EBADF)
+
+    def test_non_blocking_handshake(self):
+        with test_support.transient_internet("svn.python.org"):
+            s = socket.socket(socket.AF_INET)
+            s.connect(("svn.python.org", 443))
+            s.setblocking(False)
+            s = ssl.wrap_socket(s,
+                                cert_reqs=ssl.CERT_NONE,
+                                do_handshake_on_connect=False)
+            count = 0
+            while True:
+                try:
+                    count += 1
+                    s.do_handshake()
+                    break
+                except ssl.SSLError, err:
+                    if err.args[0] == ssl.SSL_ERROR_WANT_READ:
+                        select.select([s], [], [])
+                    elif err.args[0] == ssl.SSL_ERROR_WANT_WRITE:
+                        select.select([], [s], [])
+                    else:
+                        raise
+            s.close()
+            if test_support.verbose:
+                sys.stdout.write("\nNeeded %d calls to do_handshake() to establish session.\n" % count)
+
+    def test_get_server_certificate(self):
+        with test_support.transient_internet("svn.python.org"):
+            pem = ssl.get_server_certificate(("svn.python.org", 443))
+            if not pem:
+                self.fail("No server certificate on svn.python.org:443!")
+
+            try:
+                pem = ssl.get_server_certificate(("svn.python.org", 443), ca_certs=CERTFILE)
+            except ssl.SSLError:
+                #should fail
+                pass
+            else:
+                self.fail("Got server certificate %s for svn.python.org!" % pem)
+
+            pem = ssl.get_server_certificate(("svn.python.org", 443), ca_certs=SVN_PYTHON_ORG_ROOT_CERT)
+            if not pem:
+                self.fail("No server certificate on svn.python.org:443!")
+            if test_support.verbose:
+                sys.stdout.write("\nVerified certificate for svn.python.org:443 is\n%s\n" % pem)
+
+    def test_algorithms(self):
+        # Issue #8484: all algorithms should be available when verifying a
+        # certificate.
+        # SHA256 was added in OpenSSL 0.9.8
+        if ssl.OPENSSL_VERSION_INFO < (0, 9, 8, 0, 15):
+            self.skipTest("SHA256 not available on %r" % ssl.OPENSSL_VERSION)
+        # NOTE: https://sha256.tbs-internet.com is another possible test host
+        remote = ("sha2.hboeck.de", 443)
+        sha256_cert = os.path.join(os.path.dirname(__file__), "sha256.pem")
+        with test_support.transient_internet("sha2.hboeck.de"):
+            s = ssl.wrap_socket(socket.socket(socket.AF_INET),
+                                cert_reqs=ssl.CERT_REQUIRED,
+                                ca_certs=sha256_cert,)
+            try:
+                s.connect(remote)
+                if test_support.verbose:
+                    sys.stdout.write("\nCipher with %r is %r\n" %
+                                     (remote, s.cipher()))
+                    sys.stdout.write("Certificate is:\n%s\n" %
+                                     pprint.pformat(s.getpeercert()))
+            finally:
+                s.close()
+
+
+try:
+    import threading
+except ImportError:
+    _have_threads = False
+else:
+    _have_threads = True
+
+    class ThreadedEchoServer(threading.Thread):
+
+        class ConnectionHandler(threading.Thread):
+
+            """A mildly complicated class, because we want it to work both
+            with and without the SSL wrapper around the socket connection, so
+            that we can test the STARTTLS functionality."""
+
+            def __init__(self, server, connsock):
+                self.server = server
+                self.running = False
+                self.sock = connsock
+                self.sock.setblocking(1)
+                self.sslconn = None
+                threading.Thread.__init__(self)
+                self.daemon = True
+
+            def show_conn_details(self):
+                if self.server.certreqs == ssl.CERT_REQUIRED:
+                    cert = self.sslconn.getpeercert()
+                    if test_support.verbose and self.server.chatty:
+                        sys.stdout.write(" client cert is " + pprint.pformat(cert) + "\n")
+                    cert_binary = self.sslconn.getpeercert(True)
+                    if test_support.verbose and self.server.chatty:
+                        sys.stdout.write(" cert binary is " + str(len(cert_binary)) + " bytes\n")
+                cipher = self.sslconn.cipher()
+                if test_support.verbose and self.server.chatty:
+                    sys.stdout.write(" server: connection cipher is now " + str(cipher) + "\n")
+
+            def wrap_conn(self):
+                try:
+                    self.sslconn = ssl.wrap_socket(self.sock, server_side=True,
+                                                   certfile=self.server.certificate,
+                                                   ssl_version=self.server.protocol,
+                                                   ca_certs=self.server.cacerts,
+                                                   cert_reqs=self.server.certreqs,
+                                                   ciphers=self.server.ciphers)
+                except ssl.SSLError:
+                    # XXX Various errors can have happened here, for example
+                    # a mismatching protocol version, an invalid certificate,
+                    # or a low-level bug. This should be made more discriminating.
+                    if self.server.chatty:
+                        handle_error("\n server:  bad connection attempt from " +
+                                     str(self.sock.getpeername()) + ":\n")
+                    self.close()
+                    self.running = False
+                    self.server.stop()
+                    return False
+                else:
+                    return True
+
+            def read(self):
+                if self.sslconn:
+                    return self.sslconn.read()
+                else:
+                    return self.sock.recv(1024)
+
+            def write(self, bytes):
+                if self.sslconn:
+                    return self.sslconn.write(bytes)
+                else:
+                    return self.sock.send(bytes)
+
+            def close(self):
+                if self.sslconn:
+                    self.sslconn.close()
+                else:
+                    self.sock._sock.close()
+
+            def run(self):
+                self.running = True
+                if not self.server.starttls_server:
+                    if isinstance(self.sock, ssl.SSLSocket):
+                        self.sslconn = self.sock
+                    elif not self.wrap_conn():
+                        return
+                    self.show_conn_details()
+                while self.running:
+                    try:
+                        msg = self.read()
+                        if not msg:
+                            # eof, so quit this handler
+                            self.running = False
+                            self.close()
+                        elif msg.strip() == 'over':
+                            if test_support.verbose and self.server.connectionchatty:
+                                sys.stdout.write(" server: client closed connection\n")
+                            self.close()
+                            return
+                        elif self.server.starttls_server and msg.strip() == 'STARTTLS':
+                            if test_support.verbose and self.server.connectionchatty:
+                                sys.stdout.write(" server: read STARTTLS from client, sending OK...\n")
+                            self.write("OK\n")
+                            if not self.wrap_conn():
+                                return
+                        elif self.server.starttls_server and self.sslconn and msg.strip() == 'ENDTLS':
+                            if test_support.verbose and self.server.connectionchatty:
+                                sys.stdout.write(" server: read ENDTLS from client, sending OK...\n")
+                            self.write("OK\n")
+                            self.sslconn.unwrap()
+                            self.sslconn = None
+                            if test_support.verbose and self.server.connectionchatty:
+                                sys.stdout.write(" server: connection is now unencrypted...\n")
+                        else:
+                            if (test_support.verbose and
+                                self.server.connectionchatty):
+                                ctype = (self.sslconn and "encrypted") or "unencrypted"
+                                sys.stdout.write(" server: read %s (%s), sending back %s (%s)...\n"
+                                                 % (repr(msg), ctype, repr(msg.lower()), ctype))
+                            self.write(msg.lower())
+                    except ssl.SSLError:
+                        if self.server.chatty:
+                            handle_error("Test server failure:\n")
+                        self.close()
+                        self.running = False
+                        # normally, we'd just stop here, but for the test
+                        # harness, we want to stop the server
+                        self.server.stop()
+
+        def __init__(self, certificate, ssl_version=None,
+                     certreqs=None, cacerts=None,
+                     chatty=True, connectionchatty=False, starttls_server=False,
+                     wrap_accepting_socket=False, ciphers=None):
+
+            if ssl_version is None:
+                ssl_version = ssl.PROTOCOL_TLSv1
+            if certreqs is None:
+                certreqs = ssl.CERT_NONE
+            self.certificate = certificate
+            self.protocol = ssl_version
+            self.certreqs = certreqs
+            self.cacerts = cacerts
+            self.ciphers = ciphers
+            self.chatty = chatty
+            self.connectionchatty = connectionchatty
+            self.starttls_server = starttls_server
+            self.sock = socket.socket()
+            self.flag = None
+            if wrap_accepting_socket:
+                self.sock = ssl.wrap_socket(self.sock, server_side=True,
+                                            certfile=self.certificate,
+                                            cert_reqs = self.certreqs,
+                                            ca_certs = self.cacerts,
+                                            ssl_version = self.protocol,
+                                            ciphers = self.ciphers)
+                if test_support.verbose and self.chatty:
+                    sys.stdout.write(' server:  wrapped server socket as %s\n' % str(self.sock))
+            self.port = test_support.bind_port(self.sock)
+            self.active = False
+            threading.Thread.__init__(self)
+            self.daemon = True
+
+        def start(self, flag=None):
+            self.flag = flag
+            threading.Thread.start(self)
+
+        def run(self):
+            self.sock.settimeout(0.05)
+            self.sock.listen(5)
+            self.active = True
+            if self.flag:
+                # signal an event
+                self.flag.set()
+            while self.active:
+                try:
+                    newconn, connaddr = self.sock.accept()
+                    if test_support.verbose and self.chatty:
+                        sys.stdout.write(' server:  new connection from '
+                                         + str(connaddr) + '\n')
+                    handler = self.ConnectionHandler(self, newconn)
+                    handler.start()
+                except socket.timeout:
+                    pass
+                except KeyboardInterrupt:
+                    self.stop()
+            self.sock.close()
+
+        def stop(self):
+            self.active = False
+
+    class AsyncoreEchoServer(threading.Thread):
+
+        class EchoServer(asyncore.dispatcher):
+
+            class ConnectionHandler(asyncore.dispatcher_with_send):
+
+                def __init__(self, conn, certfile):
+                    asyncore.dispatcher_with_send.__init__(self, conn)
+                    self.socket = ssl.wrap_socket(conn, server_side=True,
+                                                  certfile=certfile,
+                                                  do_handshake_on_connect=False)
+                    self._ssl_accepting = True
+
+                def readable(self):
+                    if isinstance(self.socket, ssl.SSLSocket):
+                        while self.socket.pending() > 0:
+                            self.handle_read_event()
+                    return True
+
+                def _do_ssl_handshake(self):
+                    try:
+                        self.socket.do_handshake()
+                    except ssl.SSLError, err:
+                        if err.args[0] in (ssl.SSL_ERROR_WANT_READ,
+                                           ssl.SSL_ERROR_WANT_WRITE):
+                            return
+                        elif err.args[0] == ssl.SSL_ERROR_EOF:
+                            return self.handle_close()
+                        raise
+                    except socket.error, err:
+                        if err.args[0] == errno.ECONNABORTED:
+                            return self.handle_close()
+                    else:
+                        self._ssl_accepting = False
+
+                def handle_read(self):
+                    if self._ssl_accepting:
+                        self._do_ssl_handshake()
+                    else:
+                        data = self.recv(1024)
+                        if data and data.strip() != 'over':
+                            self.send(data.lower())
+
+                def handle_close(self):
+                    self.close()
+                    if test_support.verbose:
+                        sys.stdout.write(" server:  closed connection %s\n" % self.socket)
+
+                def handle_error(self):
+                    raise
+
+            def __init__(self, certfile):
+                self.certfile = certfile
+                asyncore.dispatcher.__init__(self)
+                self.create_socket(socket.AF_INET, socket.SOCK_STREAM)
+                self.port = test_support.bind_port(self.socket)
+                self.listen(5)
+
+            def handle_accept(self):
+                sock_obj, addr = self.accept()
+                if test_support.verbose:
+                    sys.stdout.write(" server:  new connection from %s:%s\n" %addr)
+                self.ConnectionHandler(sock_obj, self.certfile)
+
+            def handle_error(self):
+                raise
+
+        def __init__(self, certfile):
+            self.flag = None
+            self.active = False
+            self.server = self.EchoServer(certfile)
+            self.port = self.server.port
+            threading.Thread.__init__(self)
+            self.daemon = True
+
+        def __str__(self):
+            return "<%s %s>" % (self.__class__.__name__, self.server)
+
+        def start(self, flag=None):
+            self.flag = flag
+            threading.Thread.start(self)
+
+        def run(self):
+            self.active = True
+            if self.flag:
+                self.flag.set()
+            while self.active:
+                asyncore.loop(0.05)
+
+        def stop(self):
+            self.active = False
+            self.server.close()
+
+    class SocketServerHTTPSServer(threading.Thread):
+
+        class HTTPSServer(HTTPServer):
+
+            def __init__(self, server_address, RequestHandlerClass, certfile):
+                HTTPServer.__init__(self, server_address, RequestHandlerClass)
+                # we assume the certfile contains both private key and certificate
+                self.certfile = certfile
+                self.allow_reuse_address = True
+
+            def __str__(self):
+                return ('<%s %s:%s>' %
+                        (self.__class__.__name__,
+                         self.server_name,
+                         self.server_port))
+
+            def get_request(self):
+                # override this to wrap socket with SSL
+                sock, addr = self.socket.accept()
+                sslconn = ssl.wrap_socket(sock, server_side=True,
+                                          certfile=self.certfile)
+                return sslconn, addr
+
+        class RootedHTTPRequestHandler(SimpleHTTPRequestHandler):
+            # need to override translate_path to get a known root,
+            # instead of using os.curdir, since the test could be
+            # run from anywhere
+
+            server_version = "TestHTTPS/1.0"
+
+            root = None
+
+            def translate_path(self, path):
+                """Translate a /-separated PATH to the local filename syntax.
+
+                Components that mean special things to the local file system
+                (e.g. drive or directory names) are ignored.  (XXX They should
+                probably be diagnosed.)
+
+                """
+                # abandon query parameters
+                path = urlparse.urlparse(path)[2]
+                path = os.path.normpath(urllib.unquote(path))
+                words = path.split('/')
+                words = filter(None, words)
+                path = self.root
+                for word in words:
+                    drive, word = os.path.splitdrive(word)
+                    head, word = os.path.split(word)
+                    if word in self.root: continue
+                    path = os.path.join(path, word)
+                return path
+
+            def log_message(self, format, *args):
+
+                # we override this to suppress logging unless "verbose"
+
+                if test_support.verbose:
+                    sys.stdout.write(" server (%s:%d %s):\n   [%s] %s\n" %
+                                     (self.server.server_address,
+                                      self.server.server_port,
+                                      self.request.cipher(),
+                                      self.log_date_time_string(),
+                                      format%args))
+
+
+        def __init__(self, certfile):
+            self.flag = None
+            self.RootedHTTPRequestHandler.root = os.path.split(CERTFILE)[0]
+            self.server = self.HTTPSServer(
+                (HOST, 0), self.RootedHTTPRequestHandler, certfile)
+            self.port = self.server.server_port
+            threading.Thread.__init__(self)
+            self.daemon = True
+
+        def __str__(self):
+            return "<%s %s>" % (self.__class__.__name__, self.server)
+
+        def start(self, flag=None):
+            self.flag = flag
+            threading.Thread.start(self)
+
+        def run(self):
+            if self.flag:
+                self.flag.set()
+            self.server.serve_forever(0.05)
+
+        def stop(self):
+            self.server.shutdown()
+
+
+    def bad_cert_test(certfile):
+        """
+        Launch a server with CERT_REQUIRED, and check that trying to
+        connect to it with the given client certificate fails.
+        """
+        server = ThreadedEchoServer(CERTFILE,
+                                    certreqs=ssl.CERT_REQUIRED,
+                                    cacerts=CERTFILE, chatty=False)
+        flag = threading.Event()
+        server.start(flag)
+        # wait for it to start
+        flag.wait()
+        # try to connect
+        try:
+            try:
+                s = ssl.wrap_socket(socket.socket(),
+                                    certfile=certfile,
+                                    ssl_version=ssl.PROTOCOL_TLSv1)
+                s.connect((HOST, server.port))
+            except ssl.SSLError, x:
+                if test_support.verbose:
+                    sys.stdout.write("\nSSLError is %s\n" % x[1])
+            except socket.error, x:
+                if test_support.verbose:
+                    sys.stdout.write("\nsocket.error is %s\n" % x[1])
+            else:
+                raise AssertionError("Use of invalid cert should have failed!")
+        finally:
+            server.stop()
+            server.join()
+
+    def server_params_test(certfile, protocol, certreqs, cacertsfile,
+                           client_certfile, client_protocol=None, indata="FOO\n",
+                           ciphers=None, chatty=True, connectionchatty=False,
+                           wrap_accepting_socket=False):
+        """
+        Launch a server, connect a client to it and try various reads
+        and writes.
+        """
+        server = ThreadedEchoServer(certfile,
+                                    certreqs=certreqs,
+                                    ssl_version=protocol,
+                                    cacerts=cacertsfile,
+                                    ciphers=ciphers,
+                                    chatty=chatty,
+                                    connectionchatty=connectionchatty,
+                                    wrap_accepting_socket=wrap_accepting_socket)
+        flag = threading.Event()
+        server.start(flag)
+        # wait for it to start
+        flag.wait()
+        # try to connect
+        if client_protocol is None:
+            client_protocol = protocol
+        try:
+            s = ssl.wrap_socket(socket.socket(),
+                                certfile=client_certfile,
+                                ca_certs=cacertsfile,
+                                ciphers=ciphers,
+                                cert_reqs=certreqs,
+                                ssl_version=client_protocol)
+            s.connect((HOST, server.port))
+            for arg in [indata, bytearray(indata), memoryview(indata)]:
+                if connectionchatty:
+                    if test_support.verbose:
+                        sys.stdout.write(
+                            " client:  sending %s...\n" % (repr(arg)))
+                s.write(arg)
+                outdata = s.read()
+                if connectionchatty:
+                    if test_support.verbose:
+                        sys.stdout.write(" client:  read %s\n" % repr(outdata))
+                if outdata != indata.lower():
+                    raise AssertionError(
+                        "bad data <<%s>> (%d) received; expected <<%s>> (%d)\n"
+                        % (outdata[:min(len(outdata),20)], len(outdata),
+                           indata[:min(len(indata),20)].lower(), len(indata)))
+            s.write("over\n")
+            if connectionchatty:
+                if test_support.verbose:
+                    sys.stdout.write(" client:  closing connection.\n")
+            s.close()
+        finally:
+            server.stop()
+            server.join()
+
+    def try_protocol_combo(server_protocol,
+                           client_protocol,
+                           expect_success,
+                           certsreqs=None):
+        if certsreqs is None:
+            certsreqs = ssl.CERT_NONE
+        certtype = {
+            ssl.CERT_NONE: "CERT_NONE",
+            ssl.CERT_OPTIONAL: "CERT_OPTIONAL",
+            ssl.CERT_REQUIRED: "CERT_REQUIRED",
+        }[certsreqs]
+        if test_support.verbose:
+            formatstr = (expect_success and " %s->%s %s\n") or " {%s->%s} %s\n"
+            sys.stdout.write(formatstr %
+                             (ssl.get_protocol_name(client_protocol),
+                              ssl.get_protocol_name(server_protocol),
+                              certtype))
+        try:
+            # NOTE: we must enable "ALL" ciphers, otherwise an SSLv23 client
+            # will send an SSLv3 hello (rather than SSLv2) starting from
+            # OpenSSL 1.0.0 (see issue #8322).
+            server_params_test(CERTFILE, server_protocol, certsreqs,
+                               CERTFILE, CERTFILE, client_protocol,
+                               ciphers="ALL", chatty=False)
+        # Protocol mismatch can result in either an SSLError, or a
+        # "Connection reset by peer" error.
+        except ssl.SSLError:
+            if expect_success:
+                raise
+        except socket.error as e:
+            if expect_success or e.errno != errno.ECONNRESET:
+                raise
+        else:
+            if not expect_success:
+                raise AssertionError(
+                    "Client protocol %s succeeded with server protocol %s!"
+                    % (ssl.get_protocol_name(client_protocol),
+                       ssl.get_protocol_name(server_protocol)))
+
+
+    class ThreadedTests(unittest.TestCase):
+
+        def test_rude_shutdown(self):
+            """A brutal shutdown of an SSL server should raise an IOError
+            in the client when attempting handshake.
+            """
+            listener_ready = threading.Event()
+            listener_gone = threading.Event()
+
+            s = socket.socket()
+            port = test_support.bind_port(s, HOST)
+
+            # `listener` runs in a thread.  It sits in an accept() until
+            # the main thread connects.  Then it rudely closes the socket,
+            # and sets Event `listener_gone` to let the main thread know
+            # the socket is gone.
+            def listener():
+                s.listen(5)
+                listener_ready.set()
+                s.accept()
+                s.close()
+                listener_gone.set()
+
+            def connector():
+                listener_ready.wait()
+                c = socket.socket()
+                c.connect((HOST, port))
+                listener_gone.wait()
+                try:
+                    ssl_sock = ssl.wrap_socket(c)
+                except IOError:
+                    pass
+                else:
+                    self.fail('connecting to closed SSL socket should have failed')
+
+            t = threading.Thread(target=listener)
+            t.start()
+            try:
+                connector()
+            finally:
+                t.join()
+
+        @skip_if_broken_ubuntu_ssl
+        def test_echo(self):
+            """Basic test of an SSL client connecting to a server"""
+            if test_support.verbose:
+                sys.stdout.write("\n")
+            server_params_test(CERTFILE, ssl.PROTOCOL_TLSv1, ssl.CERT_NONE,
+                               CERTFILE, CERTFILE, ssl.PROTOCOL_TLSv1,
+                               chatty=True, connectionchatty=True)
+
+        def test_getpeercert(self):
+            if test_support.verbose:
+                sys.stdout.write("\n")
+            s2 = socket.socket()
+            server = ThreadedEchoServer(CERTFILE,
+                                        certreqs=ssl.CERT_NONE,
+                                        ssl_version=ssl.PROTOCOL_SSLv23,
+                                        cacerts=CERTFILE,
+                                        chatty=False)
+            flag = threading.Event()
+            server.start(flag)
+            # wait for it to start
+            flag.wait()
+            # try to connect
+            try:
+                s = ssl.wrap_socket(socket.socket(),
+                                    certfile=CERTFILE,
+                                    ca_certs=CERTFILE,
+                                    cert_reqs=ssl.CERT_REQUIRED,
+                                    ssl_version=ssl.PROTOCOL_SSLv23)
+                s.connect((HOST, server.port))
+                cert = s.getpeercert()
+                self.assertTrue(cert, "Can't get peer certificate.")
+                cipher = s.cipher()
+                if test_support.verbose:
+                    sys.stdout.write(pprint.pformat(cert) + '\n')
+                    sys.stdout.write("Connection cipher is " + str(cipher) + '.\n')
+                if 'subject' not in cert:
+                    self.fail("No subject field in certificate: %s." %
+                              pprint.pformat(cert))
+                if ((('organizationName', 'Python Software Foundation'),)
+                    not in cert['subject']):
+                    self.fail(
+                        "Missing or invalid 'organizationName' field in certificate subject; "
+                        "should be 'Python Software Foundation'.")
+                s.close()
+            finally:
+                server.stop()
+                server.join()
+
+        def test_empty_cert(self):
+            """Connecting with an empty cert file"""
+            bad_cert_test(os.path.join(os.path.dirname(__file__) or os.curdir,
+                                      "nullcert.pem"))
+        def test_malformed_cert(self):
+            """Connecting with a badly formatted certificate (syntax error)"""
+            bad_cert_test(os.path.join(os.path.dirname(__file__) or os.curdir,
+                                       "badcert.pem"))
+        def test_nonexisting_cert(self):
+            """Connecting with a non-existing cert file"""
+            bad_cert_test(os.path.join(os.path.dirname(__file__) or os.curdir,
+                                       "wrongcert.pem"))
+        def test_malformed_key(self):
+            """Connecting with a badly formatted key (syntax error)"""
+            bad_cert_test(os.path.join(os.path.dirname(__file__) or os.curdir,
+                                       "badkey.pem"))
+
+        @skip_if_broken_ubuntu_ssl
+        def test_protocol_sslv2(self):
+            """Connecting to an SSLv2 server with various client options"""
+            if test_support.verbose:
+                sys.stdout.write("\n")
+            try_protocol_combo(ssl.PROTOCOL_SSLv2, ssl.PROTOCOL_SSLv2, True)
+            try_protocol_combo(ssl.PROTOCOL_SSLv2, ssl.PROTOCOL_SSLv2, True, ssl.CERT_OPTIONAL)
+            try_protocol_combo(ssl.PROTOCOL_SSLv2, ssl.PROTOCOL_SSLv2, True, ssl.CERT_REQUIRED)
+            try_protocol_combo(ssl.PROTOCOL_SSLv2, ssl.PROTOCOL_SSLv23, True)
+            try_protocol_combo(ssl.PROTOCOL_SSLv2, ssl.PROTOCOL_SSLv3, False)
+            try_protocol_combo(ssl.PROTOCOL_SSLv2, ssl.PROTOCOL_TLSv1, False)
+
+        @skip_if_broken_ubuntu_ssl
+        def test_protocol_sslv23(self):
+            """Connecting to an SSLv23 server with various client options"""
+            if test_support.verbose:
+                sys.stdout.write("\n")
+            try:
+                try_protocol_combo(ssl.PROTOCOL_SSLv23, ssl.PROTOCOL_SSLv2, True)
+            except (ssl.SSLError, socket.error), x:
+                # this fails on some older versions of OpenSSL (0.9.7l, for instance)
+                if test_support.verbose:
+                    sys.stdout.write(
+                        " SSL2 client to SSL23 server test unexpectedly failed:\n %s\n"
+                        % str(x))
+            try_protocol_combo(ssl.PROTOCOL_SSLv23, ssl.PROTOCOL_SSLv3, True)
+            try_protocol_combo(ssl.PROTOCOL_SSLv23, ssl.PROTOCOL_SSLv23, True)
+            try_protocol_combo(ssl.PROTOCOL_SSLv23, ssl.PROTOCOL_TLSv1, True)
+
+            try_protocol_combo(ssl.PROTOCOL_SSLv23, ssl.PROTOCOL_SSLv3, True, ssl.CERT_OPTIONAL)
+            try_protocol_combo(ssl.PROTOCOL_SSLv23, ssl.PROTOCOL_SSLv23, True, ssl.CERT_OPTIONAL)
+            try_protocol_combo(ssl.PROTOCOL_SSLv23, ssl.PROTOCOL_TLSv1, True, ssl.CERT_OPTIONAL)
+
+            try_protocol_combo(ssl.PROTOCOL_SSLv23, ssl.PROTOCOL_SSLv3, True, ssl.CERT_REQUIRED)
+            try_protocol_combo(ssl.PROTOCOL_SSLv23, ssl.PROTOCOL_SSLv23, True, ssl.CERT_REQUIRED)
+            try_protocol_combo(ssl.PROTOCOL_SSLv23, ssl.PROTOCOL_TLSv1, True, ssl.CERT_REQUIRED)
+
+        @skip_if_broken_ubuntu_ssl
+        def test_protocol_sslv3(self):
+            """Connecting to an SSLv3 server with various client options"""
+            if test_support.verbose:
+                sys.stdout.write("\n")
+            try_protocol_combo(ssl.PROTOCOL_SSLv3, ssl.PROTOCOL_SSLv3, True)
+            try_protocol_combo(ssl.PROTOCOL_SSLv3, ssl.PROTOCOL_SSLv3, True, ssl.CERT_OPTIONAL)
+            try_protocol_combo(ssl.PROTOCOL_SSLv3, ssl.PROTOCOL_SSLv3, True, ssl.CERT_REQUIRED)
+            try_protocol_combo(ssl.PROTOCOL_SSLv3, ssl.PROTOCOL_SSLv2, False)
+            try_protocol_combo(ssl.PROTOCOL_SSLv3, ssl.PROTOCOL_SSLv23, False)
+            try_protocol_combo(ssl.PROTOCOL_SSLv3, ssl.PROTOCOL_TLSv1, False)
+
+        @skip_if_broken_ubuntu_ssl
+        def test_protocol_tlsv1(self):
+            """Connecting to a TLSv1 server with various client options"""
+            if test_support.verbose:
+                sys.stdout.write("\n")
+            try_protocol_combo(ssl.PROTOCOL_TLSv1, ssl.PROTOCOL_TLSv1, True)
+            try_protocol_combo(ssl.PROTOCOL_TLSv1, ssl.PROTOCOL_TLSv1, True, ssl.CERT_OPTIONAL)
+            try_protocol_combo(ssl.PROTOCOL_TLSv1, ssl.PROTOCOL_TLSv1, True, ssl.CERT_REQUIRED)
+            try_protocol_combo(ssl.PROTOCOL_TLSv1, ssl.PROTOCOL_SSLv2, False)
+            try_protocol_combo(ssl.PROTOCOL_TLSv1, ssl.PROTOCOL_SSLv3, False)
+            try_protocol_combo(ssl.PROTOCOL_TLSv1, ssl.PROTOCOL_SSLv23, False)
+
+        def test_starttls(self):
+            """Switching from clear text to encrypted and back again."""
+            msgs = ("msg 1", "MSG 2", "STARTTLS", "MSG 3", "msg 4", "ENDTLS", "msg 5", "msg 6")
+
+            server = ThreadedEchoServer(CERTFILE,
+                                        ssl_version=ssl.PROTOCOL_TLSv1,
+                                        starttls_server=True,
+                                        chatty=True,
+                                        connectionchatty=True)
+            flag = threading.Event()
+            server.start(flag)
+            # wait for it to start
+            flag.wait()
+            # try to connect
+            wrapped = False
+            try:
+                s = socket.socket()
+                s.setblocking(1)
+                s.connect((HOST, server.port))
+                if test_support.verbose:
+                    sys.stdout.write("\n")
+                for indata in msgs:
+                    if test_support.verbose:
+                        sys.stdout.write(
+                            " client:  sending %s...\n" % repr(indata))
+                    if wrapped:
+                        conn.write(indata)
+                        outdata = conn.read()
+                    else:
+                        s.send(indata)
+                        outdata = s.recv(1024)
+                    if (indata == "STARTTLS" and
+                        outdata.strip().lower().startswith("ok")):
+                        # STARTTLS ok, switch to secure mode
+                        if test_support.verbose:
+                            sys.stdout.write(
+                                " client:  read %s from server, starting TLS...\n"
+                                % repr(outdata))
+                        conn = ssl.wrap_socket(s, ssl_version=ssl.PROTOCOL_TLSv1)
+                        wrapped = True
+                    elif (indata == "ENDTLS" and
+                        outdata.strip().lower().startswith("ok")):
+                        # ENDTLS ok, switch back to clear text
+                        if test_support.verbose:
+                            sys.stdout.write(
+                                " client:  read %s from server, ending TLS...\n"
+                                % repr(outdata))
+                        s = conn.unwrap()
+                        wrapped = False
+                    else:
+                        if test_support.verbose:
+                            sys.stdout.write(
+                                " client:  read %s from server\n" % repr(outdata))
+                if test_support.verbose:
+                    sys.stdout.write(" client:  closing connection.\n")
+                if wrapped:
+                    conn.write("over\n")
+                else:
+                    s.send("over\n")
+                s.close()
+            finally:
+                server.stop()
+                server.join()
+
+        def test_socketserver(self):
+            """Using a SocketServer to create and manage SSL connections."""
+            server = SocketServerHTTPSServer(CERTFILE)
+            flag = threading.Event()
+            server.start(flag)
+            # wait for it to start
+            flag.wait()
+            # try to connect
+            try:
+                if test_support.verbose:
+                    sys.stdout.write('\n')
+                with open(CERTFILE, 'rb') as f:
+                    d1 = f.read()
+                d2 = ''
+                # now fetch the same data from the HTTPS server
+                url = 'https://127.0.0.1:%d/%s' % (
+                    server.port, os.path.split(CERTFILE)[1])
+                with test_support.check_py3k_warnings():
+                    f = urllib.urlopen(url)
+                dlen = f.info().getheader("content-length")
+                if dlen and (int(dlen) > 0):
+                    d2 = f.read(int(dlen))
+                    if test_support.verbose:
+                        sys.stdout.write(
+                            " client: read %d bytes from remote server '%s'\n"
+                            % (len(d2), server))
+                f.close()
+                self.assertEqual(d1, d2)
+            finally:
+                server.stop()
+                server.join()
+
+        def test_wrapped_accept(self):
+            """Check the accept() method on SSL sockets."""
+            if test_support.verbose:
+                sys.stdout.write("\n")
+            server_params_test(CERTFILE, ssl.PROTOCOL_SSLv23, ssl.CERT_REQUIRED,
+                               CERTFILE, CERTFILE, ssl.PROTOCOL_SSLv23,
+                               chatty=True, connectionchatty=True,
+                               wrap_accepting_socket=True)
+
+        def test_asyncore_server(self):
+            """Check the example asyncore integration."""
+            indata = "TEST MESSAGE of mixed case\n"
+
+            if test_support.verbose:
+                sys.stdout.write("\n")
+            server = AsyncoreEchoServer(CERTFILE)
+            flag = threading.Event()
+            server.start(flag)
+            # wait for it to start
+            flag.wait()
+            # try to connect
+            try:
+                s = ssl.wrap_socket(socket.socket())
+                s.connect(('127.0.0.1', server.port))
+                if test_support.verbose:
+                    sys.stdout.write(
+                        " client:  sending %s...\n" % (repr(indata)))
+                s.write(indata)
+                outdata = s.read()
+                if test_support.verbose:
+                    sys.stdout.write(" client:  read %s\n" % repr(outdata))
+                if outdata != indata.lower():
+                    self.fail(
+                        "bad data <<%s>> (%d) received; expected <<%s>> (%d)\n"
+                        % (outdata[:min(len(outdata),20)], len(outdata),
+                           indata[:min(len(indata),20)].lower(), len(indata)))
+                s.write("over\n")
+                if test_support.verbose:
+                    sys.stdout.write(" client:  closing connection.\n")
+                s.close()
+            finally:
+                server.stop()
+                # wait for server thread to end
+                server.join()
+
+        def test_recv_send(self):
+            """Test recv(), send() and friends."""
+            if test_support.verbose:
+                sys.stdout.write("\n")
+
+            server = ThreadedEchoServer(CERTFILE,
+                                        certreqs=ssl.CERT_NONE,
+                                        ssl_version=ssl.PROTOCOL_TLSv1,
+                                        cacerts=CERTFILE,
+                                        chatty=True,
+                                        connectionchatty=False)
+            flag = threading.Event()
+            server.start(flag)
+            # wait for it to start
+            flag.wait()
+            # try to connect
+            s = ssl.wrap_socket(socket.socket(),
+                                server_side=False,
+                                certfile=CERTFILE,
+                                ca_certs=CERTFILE,
+                                cert_reqs=ssl.CERT_NONE,
+                                ssl_version=ssl.PROTOCOL_TLSv1)
+            s.connect((HOST, server.port))
+            try:
+                # helper methods for standardising recv* method signatures
+                def _recv_into():
+                    b = bytearray("\0"*100)
+                    count = s.recv_into(b)
+                    return b[:count]
+
+                def _recvfrom_into():
+                    b = bytearray("\0"*100)
+                    count, addr = s.recvfrom_into(b)
+                    return b[:count]
+
+                # (name, method, whether to expect success, *args)
+                send_methods = [
+                    ('send', s.send, True, []),
+                    ('sendto', s.sendto, False, ["some.address"]),
+                    ('sendall', s.sendall, True, []),
+                ]
+                recv_methods = [
+                    ('recv', s.recv, True, []),
+                    ('recvfrom', s.recvfrom, False, ["some.address"]),
+                    ('recv_into', _recv_into, True, []),
+                    ('recvfrom_into', _recvfrom_into, False, []),
+                ]
+                data_prefix = u"PREFIX_"
+
+                for meth_name, send_meth, expect_success, args in send_methods:
+                    indata = data_prefix + meth_name
+                    try:
+                        send_meth(indata.encode('ASCII', 'strict'), *args)
+                        outdata = s.read()
+                        outdata = outdata.decode('ASCII', 'strict')
+                        if outdata != indata.lower():
+                            self.fail(
+                                "While sending with <<%s>> bad data "
+                                "<<%r>> (%d) received; "
+                                "expected <<%r>> (%d)\n" % (
+                                    meth_name, outdata[:20], len(outdata),
+                                    indata[:20], len(indata)
+                                )
+                            )
+                    except ValueError as e:
+                        if expect_success:
+                            self.fail(
+                                "Failed to send with method <<%s>>; "
+                                "expected to succeed.\n" % (meth_name,)
+                            )
+                        if not str(e).startswith(meth_name):
+                            self.fail(
+                                "Method <<%s>> failed with unexpected "
+                                "exception message: %s\n" % (
+                                    meth_name, e
+                                )
+                            )
+
+                for meth_name, recv_meth, expect_success, args in recv_methods:
+                    indata = data_prefix + meth_name
+                    try:
+                        s.send(indata.encode('ASCII', 'strict'))
+                        outdata = recv_meth(*args)
+                        outdata = outdata.decode('ASCII', 'strict')
+                        if outdata != indata.lower():
+                            self.fail(
+                                "While receiving with <<%s>> bad data "
+                                "<<%r>> (%d) received; "
+                                "expected <<%r>> (%d)\n" % (
+                                    meth_name, outdata[:20], len(outdata),
+                                    indata[:20], len(indata)
+                                )
+                            )
+                    except ValueError as e:
+                        if expect_success:
+                            self.fail(
+                                "Failed to receive with method <<%s>>; "
+                                "expected to succeed.\n" % (meth_name,)
+                            )
+                        if not str(e).startswith(meth_name):
+                            self.fail(
+                                "Method <<%s>> failed with unexpected "
+                                "exception message: %s\n" % (
+                                    meth_name, e
+                                )
+                            )
+                        # consume data
+                        s.read()
+
+                s.write("over\n".encode("ASCII", "strict"))
+                s.close()
+            finally:
+                server.stop()
+                server.join()
+
+        def test_handshake_timeout(self):
+            # Issue #5103: SSL handshake must respect the socket timeout
+            server = socket.socket(socket.AF_INET)
+            host = "127.0.0.1"
+            port = test_support.bind_port(server)
+            started = threading.Event()
+            finish = False
+
+            def serve():
+                server.listen(5)
+                started.set()
+                conns = []
+                while not finish:
+                    r, w, e = select.select([server], [], [], 0.1)
+                    if server in r:
+                        # Let the socket hang around rather than having
+                        # it closed by garbage collection.
+                        conns.append(server.accept()[0])
+
+            t = threading.Thread(target=serve)
+            t.start()
+            started.wait()
+
+            try:
+                try:
+                    c = socket.socket(socket.AF_INET)
+                    c.settimeout(0.2)
+                    c.connect((host, port))
+                    # Will attempt handshake and time out
+                    self.assertRaisesRegexp(ssl.SSLError, "timed out",
+                                            ssl.wrap_socket, c)
+                finally:
+                    c.close()
+                try:
+                    c = socket.socket(socket.AF_INET)
+                    c.settimeout(0.2)
+                    c = ssl.wrap_socket(c)
+                    # Will attempt handshake and time out
+                    self.assertRaisesRegexp(ssl.SSLError, "timed out",
+                                            c.connect, (host, port))
+                finally:
+                    c.close()
+            finally:
+                finish = True
+                t.join()
+                server.close()
+
+
+def test_main(verbose=False):
+    if skip_expected:
+        raise unittest.SkipTest("No SSL support")
+
+    global CERTFILE, SVN_PYTHON_ORG_ROOT_CERT
+    CERTFILE = test_support.findfile("keycert.pem")
+    SVN_PYTHON_ORG_ROOT_CERT =  test_support.findfile(
+        "https_svn_python_org_root.pem")
+
+    if (not os.path.exists(CERTFILE) or
+        not os.path.exists(SVN_PYTHON_ORG_ROOT_CERT)):
+        raise test_support.TestFailed("Can't read certificate files!")
+
+    tests = [BasicTests, BasicSocketTests]
+
+    if test_support.is_resource_enabled('network'):
+        tests.append(NetworkedTests)
+
+    if _have_threads:
+        thread_info = test_support.threading_setup()
+        if thread_info and test_support.is_resource_enabled('network'):
+            tests.append(ThreadedTests)
+
+    try:
+        test_support.run_unittest(*tests)
+    finally:
+        if _have_threads:
+            test_support.threading_cleanup(*thread_info)
+
+if __name__ == "__main__":
+    test_main()

diff --git a/pypy/jit/tl/pypyjit_demo.py b/pypy/jit/tl/pypyjit_demo.py
--- a/pypy/jit/tl/pypyjit_demo.py
+++ b/pypy/jit/tl/pypyjit_demo.py
@@ -1,19 +1,16 @@
 
 try:
-    import pypyjit
-    pypyjit.set_param(threshold=3, inlining=True)
+    class C:
+        def __call__(self):
+            pass
 
-    def sqrt(y, n=10000):
-        x = y / 2
-        while n > 0:
-            #assert y > 0 and x > 0
-            if y > 0 and x > 0: pass
-            n -= 1
-            x = (x + y/x) / 2
-        return x
-
-    print sqrt(1234, 4)
-    
+    def f():
+        i = 0
+        c = C()
+        while i < 10:
+            c()
+            i += 1
+    f()
 except Exception, e:
     print "Exception: ", type(e)
     print e

From commits-noreply at bitbucket.org  Tue Feb  1 13:26:29 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Tue,  1 Feb 2011 13:26:29 +0100 (CET)
Subject: [pypy-svn] pypy default: Backed out changeset f7e470d7823c
Message-ID: <20110201122629.8B0FD282BD8@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: 
Changeset: r41518:0872e654e4d7
Date: 2011-02-01 14:25 +0200
http://bitbucket.org/pypy/pypy/changeset/0872e654e4d7/

Log:	Backed out changeset f7e470d7823c

diff --git a/pypy/interpreter/gateway.py b/pypy/interpreter/gateway.py
--- a/pypy/interpreter/gateway.py
+++ b/pypy/interpreter/gateway.py
@@ -344,9 +344,8 @@
                                                        self.nextarg()))
 
     def visit__ObjSpace(self, el):
-        if self.finger not in [0, 1]:
+        if self.finger != 0:
             raise FastFuncNotSupported
-        self.args.append('space')
         self.unwrap.append("space")
 
     def visit__W_Root(self, el):
@@ -392,9 +391,7 @@
         unwrap_info = UnwrapSpec_FastFunc_Unwrap()
         unwrap_info.apply_over(unwrap_spec)
         narg = unwrap_info.n
-        args = unwrap_info.args
-        if 'space' not in args:
-            args = ['space'] + args
+        args = ['space'] + unwrap_info.args
         if args == unwrap_info.unwrap:
             fastfunc = func
         else:
@@ -531,12 +528,8 @@
                     self.__class__ = BuiltinCodePassThroughArguments1
                     self.func__args__ = func
             else:
-                if len(unwrap_spec) >= 2 and unwrap_spec[1] is ObjSpace:
-                    self.__class__ = globals()['BuiltinCodeSpace%d' % arity]
-                    setattr(self, 'fastfunc_space_%d' % arity, fastfunc)
-                else:
-                    self.__class__ = globals()['BuiltinCode%d' % arity]
-                    setattr(self, 'fastfunc_%d' % arity, fastfunc)
+                self.__class__ = globals()['BuiltinCode%d' % arity]
+                setattr(self, 'fastfunc_%d' % arity, fastfunc)
 
     def descr__reduce__(self, space):
         from pypy.interpreter.mixedmodule import MixedModule
@@ -724,78 +717,6 @@
             w_result = space.w_None
         return w_result
 
-class BuiltinCodeSpace1(BuiltinCode1):
-    _immutable_ = True
-    fast_natural_arity = 1
-    
-    def fastcall_1(self, space, w_func, w1):
-        try:
-            w_result = self.fastfunc_space_1(w1, space)
-        except DescrMismatch:
-            return  w1.descr_call_mismatch(space,
-                                           self.descrmismatch_op,
-                                           self.descr_reqcls,
-                                           Arguments(space, [w1]))
-        except Exception, e:
-            raise self.handle_exception(space, e)
-        if w_result is None:
-            w_result = space.w_None
-        return w_result
-
-class BuiltinCodeSpace2(BuiltinCode2):
-    _immutable_ = True
-    fast_natural_arity = 2
-    
-    def fastcall_2(self, space, w_func, w1, w2):
-        try:
-            w_result = self.fastfunc_space_2(w1, space, w2)
-        except DescrMismatch:
-            return  w1.descr_call_mismatch(space,
-                                           self.descrmismatch_op,
-                                           self.descr_reqcls,
-                                           Arguments(space, [w1, w2]))
-        except Exception, e:
-            raise self.handle_exception(space, e)
-        if w_result is None:
-            w_result = space.w_None
-        return w_result
-
-class BuiltinCodeSpace3(BuiltinCode3):
-    _immutable_ = True
-    fast_natural_arity = 3
-    
-    def fastcall_3(self, space, func, w1, w2, w3):
-        try:
-            w_result = self.fastfunc_space_3(w1, space, w2, w3)
-        except DescrMismatch:
-            return  w1.descr_call_mismatch(space,
-                                           self.descrmismatch_op,
-                                           self.descr_reqcls,
-                                           Arguments(space, [w1, w2, w3]))
-        except Exception, e:
-            raise self.handle_exception(space, e)
-        if w_result is None:
-            w_result = space.w_None
-        return w_result
-
-class BuiltinCodeSpace4(BuiltinCode4):
-    _immutable_ = True
-    fast_natural_arity = 4
-    
-    def fastcall_4(self, space, func, w1, w2, w3, w4):
-        try:
-            w_result = self.fastfunc_space_4(w1, space, w2, w3, w4)
-        except DescrMismatch:
-            return  w1.descr_call_mismatch(space,
-                                           self.descrmismatch_op,
-                                           self.descr_reqcls,
-                                           Arguments(space,
-                                                     [w1, w2, w3, w4]))
-        except Exception, e:
-            raise self.handle_exception(space, e)
-        if w_result is None:
-            w_result = space.w_None
-        return w_result
 
 class interp2app(Wrappable):
     """Build a gateway that calls 'f' at interp-level."""

diff --git a/pypy/interpreter/test/test_gateway.py b/pypy/interpreter/test/test_gateway.py
--- a/pypy/interpreter/test/test_gateway.py
+++ b/pypy/interpreter/test/test_gateway.py
@@ -14,7 +14,7 @@
         self.name = name
         self.defs_w = []
 
-class TestBuiltinCode(object): 
+class TestBuiltinCode: 
     def test_signature(self):
         def c(space, w_x, w_y, hello_w):
             pass
@@ -545,42 +545,7 @@
         """)
 
         assert space.is_true(w_res)
-        assert called == [w_app_f, w_app_f]
-
-    def test_interp2app_fastcall_method_space(self):
-        space = self.space
-        w = space.wrap
-        w_3 = w(3)
-
-        def f(w_self, space, w_x):
-            return w_x
-        app_f = gateway.interp2app_temp(f, unwrap_spec=[gateway.W_Root,
-                                                        gateway.ObjSpace,
-                                                        gateway.W_Root])
-        w_app_f = w(app_f)
-
-        # sanity
-        assert isinstance(w_app_f.code, gateway.BuiltinCode2)
-
-        called = []
-        fastcall_2 = w_app_f.code.fastcall_2
-        def witness_fastcall_2(space, w_func, w_a, w_b):
-            called.append(w_func)
-            return fastcall_2(space, w_func, w_a, w_b)
-
-        w_app_f.code.fastcall_2 = witness_fastcall_2    
-    
-        w_res = space.appexec([w_app_f, w_3], """(f, x):
-        class A(object):
-           m = f # not a builtin function, so works as method
-        y = A().m(x)
-        b = A().m
-        z = b(x)
-        return y is x and z is x
-        """)
-
-        assert space.is_true(w_res)
-        assert called == [w_app_f, w_app_f]        
+        assert called == [w_app_f, w_app_f]       
         
     def test_plain(self):
         space = self.space

diff --git a/lib-python/modified-2.7.0/ssl.py b/lib-python/modified-2.7.0/ssl.py
deleted file mode 100644
--- a/lib-python/modified-2.7.0/ssl.py
+++ /dev/null
@@ -1,437 +0,0 @@
-# Wrapper module for _ssl, providing some additional facilities
-# implemented in Python.  Written by Bill Janssen.
-
-"""\
-This module provides some more Pythonic support for SSL.
-
-Object types:
-
-  SSLSocket -- subtype of socket.socket which does SSL over the socket
-
-Exceptions:
-
-  SSLError -- exception raised for I/O errors
-
-Functions:
-
-  cert_time_to_seconds -- convert time string used for certificate
-                          notBefore and notAfter functions to integer
-                          seconds past the Epoch (the time values
-                          returned from time.time())
-
-  fetch_server_certificate (HOST, PORT) -- fetch the certificate provided
-                          by the server running on HOST at port PORT.  No
-                          validation of the certificate is performed.
-
-Integer constants:
-
-SSL_ERROR_ZERO_RETURN
-SSL_ERROR_WANT_READ
-SSL_ERROR_WANT_WRITE
-SSL_ERROR_WANT_X509_LOOKUP
-SSL_ERROR_SYSCALL
-SSL_ERROR_SSL
-SSL_ERROR_WANT_CONNECT
-
-SSL_ERROR_EOF
-SSL_ERROR_INVALID_ERROR_CODE
-
-The following group define certificate requirements that one side is
-allowing/requiring from the other side:
-
-CERT_NONE - no certificates from the other side are required (or will
-            be looked at if provided)
-CERT_OPTIONAL - certificates are not required, but if provided will be
-                validated, and if validation fails, the connection will
-                also fail
-CERT_REQUIRED - certificates are required, and will be validated, and
-                if validation fails, the connection will also fail
-
-The following constants identify various SSL protocol variants:
-
-PROTOCOL_SSLv2
-PROTOCOL_SSLv3
-PROTOCOL_SSLv23
-PROTOCOL_TLSv1
-"""
-
-import textwrap
-
-import _ssl             # if we can't import it, let the error propagate
-
-from _ssl import OPENSSL_VERSION_NUMBER, OPENSSL_VERSION_INFO, OPENSSL_VERSION
-from _ssl import SSLError
-from _ssl import CERT_NONE, CERT_OPTIONAL, CERT_REQUIRED
-from _ssl import PROTOCOL_SSLv2, PROTOCOL_SSLv3, PROTOCOL_SSLv23, PROTOCOL_TLSv1
-from _ssl import RAND_status, RAND_egd, RAND_add
-from _ssl import \
-     SSL_ERROR_ZERO_RETURN, \
-     SSL_ERROR_WANT_READ, \
-     SSL_ERROR_WANT_WRITE, \
-     SSL_ERROR_WANT_X509_LOOKUP, \
-     SSL_ERROR_SYSCALL, \
-     SSL_ERROR_SSL, \
-     SSL_ERROR_WANT_CONNECT, \
-     SSL_ERROR_EOF, \
-     SSL_ERROR_INVALID_ERROR_CODE
-
-from socket import socket, _fileobject, error as socket_error
-from socket import getnameinfo as _getnameinfo
-import base64        # for DER-to-PEM translation
-import errno
-
-class SSLSocket(socket):
-
-    """This class implements a subtype of socket.socket that wraps
-    the underlying OS socket in an SSL context when necessary, and
-    provides read and write methods over that channel."""
-
-    def __init__(self, sock, keyfile=None, certfile=None,
-                 server_side=False, cert_reqs=CERT_NONE,
-                 ssl_version=PROTOCOL_SSLv23, ca_certs=None,
-                 do_handshake_on_connect=True,
-                 suppress_ragged_eofs=True, ciphers=None):
-        socket.__init__(self, _sock=sock._sock)
-
-        if certfile and not keyfile:
-            keyfile = certfile
-        # see if it's connected
-        try:
-            socket.getpeername(self)
-        except socket_error, e:
-            if e.errno != errno.ENOTCONN:
-                raise
-            # no, no connection yet
-            self._sslobj = None
-        else:
-            # yes, create the SSL object
-            self._sslobj = _ssl.sslwrap(self._sock, server_side,
-                                        keyfile, certfile,
-                                        cert_reqs, ssl_version, ca_certs,
-                                        ciphers)
-            if do_handshake_on_connect:
-                self.do_handshake()
-        self.keyfile = keyfile
-        self.certfile = certfile
-        self.cert_reqs = cert_reqs
-        self.ssl_version = ssl_version
-        self.ca_certs = ca_certs
-        self.ciphers = ciphers
-        self.do_handshake_on_connect = do_handshake_on_connect
-        self.suppress_ragged_eofs = suppress_ragged_eofs
-        self._makefile_refs = 0
-
-    def read(self, len=1024):
-
-        """Read up to LEN bytes and return them.
-        Return zero-length string on EOF."""
-
-        try:
-            return self._sslobj.read(len)
-        except SSLError, x:
-            if x.args[0] == SSL_ERROR_EOF and self.suppress_ragged_eofs:
-                return ''
-            else:
-                raise
-
-    def write(self, data):
-
-        """Write DATA to the underlying SSL channel.  Returns
-        number of bytes of DATA actually transmitted."""
-
-        return self._sslobj.write(data)
-
-    def getpeercert(self, binary_form=False):
-
-        """Returns a formatted version of the data in the
-        certificate provided by the other end of the SSL channel.
-        Return None if no certificate was provided, {} if a
-        certificate was provided, but not validated."""
-
-        return self._sslobj.peer_certificate(binary_form)
-
-    def cipher(self):
-
-        if not self._sslobj:
-            return None
-        else:
-            return self._sslobj.cipher()
-
-    def send(self, data, flags=0):
-        if self._sslobj:
-            if flags != 0:
-                raise ValueError(
-                    "non-zero flags not allowed in calls to send() on %s" %
-                    self.__class__)
-            while True:
-                try:
-                    v = self._sslobj.write(data)
-                except SSLError, x:
-                    if x.args[0] == SSL_ERROR_WANT_READ:
-                        return 0
-                    elif x.args[0] == SSL_ERROR_WANT_WRITE:
-                        return 0
-                    else:
-                        raise
-                else:
-                    return v
-        else:
-            return self._sock.send(data, flags)
-
-    def sendto(self, data, flags_or_addr, addr=None):
-        if self._sslobj:
-            raise ValueError("sendto not allowed on instances of %s" %
-                             self.__class__)
-        elif addr is None:
-            return self._sock.sendto(data, flags_or_addr)
-        else:
-            return self._sock.sendto(data, flags_or_addr, addr)
-
-    def sendall(self, data, flags=0):
-        if self._sslobj:
-            if flags != 0:
-                raise ValueError(
-                    "non-zero flags not allowed in calls to sendall() on %s" %
-                    self.__class__)
-            amount = len(data)
-            count = 0
-            while (count < amount):
-                v = self.send(data[count:])
-                count += v
-            return amount
-        else:
-            return socket.sendall(self, data, flags)
-
-    def recv(self, buflen=1024, flags=0):
-        if self._sslobj:
-            if flags != 0:
-                raise ValueError(
-                    "non-zero flags not allowed in calls to recv() on %s" %
-                    self.__class__)
-            return self.read(buflen)
-        else:
-            return self._sock.recv(buflen, flags)
-
-    def recv_into(self, buffer, nbytes=None, flags=0):
-        if buffer and (nbytes is None):
-            nbytes = len(buffer)
-        elif nbytes is None:
-            nbytes = 1024
-        if self._sslobj:
-            if flags != 0:
-                raise ValueError(
-                  "non-zero flags not allowed in calls to recv_into() on %s" %
-                  self.__class__)
-            tmp_buffer = self.read(nbytes)
-            v = len(tmp_buffer)
-            buffer[:v] = tmp_buffer
-            return v
-        else:
-            return self._sock.recv_into(buffer, nbytes, flags)
-
-    def recvfrom(self, buflen=1024, flags=0):
-        if self._sslobj:
-            raise ValueError("recvfrom not allowed on instances of %s" %
-                             self.__class__)
-        else:
-            return self._sock.recvfrom(buflen, flags)
-
-    def recvfrom_into(self, buffer, nbytes=None, flags=0):
-        if self._sslobj:
-            raise ValueError("recvfrom_into not allowed on instances of %s" %
-                             self.__class__)
-        else:
-            return self._sock.recvfrom_into(buffer, nbytes, flags)
-
-    def pending(self):
-        if self._sslobj:
-            return self._sslobj.pending()
-        else:
-            return 0
-
-    def unwrap(self):
-        if self._sslobj:
-            s = self._sslobj.shutdown()
-            self._sslobj = None
-            return s
-        else:
-            raise ValueError("No SSL wrapper around " + str(self))
-
-    def shutdown(self, how):
-        self._sslobj = None
-        socket.shutdown(self, how)
-
-    def close(self):
-        if self._makefile_refs < 1:
-            self._sslobj = None
-            socket.close(self)
-        else:
-            self._makefile_refs -= 1
-
-    def do_handshake(self):
-
-        """Perform a TLS/SSL handshake."""
-
-        self._sslobj.do_handshake()
-
-    def connect(self, addr):
-
-        """Connects to remote ADDR, and then wraps the connection in
-        an SSL channel."""
-
-        # Here we assume that the socket is client-side, and not
-        # connected at the time of the call.  We connect it, then wrap it.
-        if self._sslobj:
-            raise ValueError("attempt to connect already-connected SSLSocket!")
-        socket.connect(self, addr)
-        self._sslobj = _ssl.sslwrap(self._sock, False, self.keyfile, self.certfile,
-                                    self.cert_reqs, self.ssl_version,
-                                    self.ca_certs, self.ciphers)
-        if self.do_handshake_on_connect:
-            self.do_handshake()
-
-    def accept(self):
-
-        """Accepts a new connection from a remote client, and returns
-        a tuple containing that new connection wrapped with a server-side
-        SSL channel, and the address of the remote client."""
-
-        newsock, addr = socket.accept(self)
-        return (SSLSocket(newsock,
-                          keyfile=self.keyfile,
-                          certfile=self.certfile,
-                          server_side=True,
-                          cert_reqs=self.cert_reqs,
-                          ssl_version=self.ssl_version,
-                          ca_certs=self.ca_certs,
-                          ciphers=self.ciphers,
-                          do_handshake_on_connect=self.do_handshake_on_connect,
-                          suppress_ragged_eofs=self.suppress_ragged_eofs),
-                addr)
-
-    def makefile(self, mode='r', bufsize=-1):
-
-        """Make and return a file-like object that
-        works with the SSL connection.  Just use the code
-        from the socket module."""
-
-        self._makefile_refs += 1
-        # close=True so as to decrement the reference count when done with
-        # the file-like object.
-        return _fileobject(self, mode, bufsize, close=True)
-
-
-
-def wrap_socket(sock, keyfile=None, certfile=None,
-                server_side=False, cert_reqs=CERT_NONE,
-                ssl_version=PROTOCOL_SSLv23, ca_certs=None,
-                do_handshake_on_connect=True,
-                suppress_ragged_eofs=True, ciphers=None):
-
-    return SSLSocket(sock, keyfile=keyfile, certfile=certfile,
-                     server_side=server_side, cert_reqs=cert_reqs,
-                     ssl_version=ssl_version, ca_certs=ca_certs,
-                     do_handshake_on_connect=do_handshake_on_connect,
-                     suppress_ragged_eofs=suppress_ragged_eofs,
-                     ciphers=ciphers)
-
-
-# some utility functions
-
-def cert_time_to_seconds(cert_time):
-
-    """Takes a date-time string in standard ASN1_print form
-    ("MON DAY 24HOUR:MINUTE:SEC YEAR TIMEZONE") and return
-    a Python time value in seconds past the epoch."""
-
-    import time
-    return time.mktime(time.strptime(cert_time, "%b %d %H:%M:%S %Y GMT"))
-
-PEM_HEADER = "-----BEGIN CERTIFICATE-----"
-PEM_FOOTER = "-----END CERTIFICATE-----"
-
-def DER_cert_to_PEM_cert(der_cert_bytes):
-
-    """Takes a certificate in binary DER format and returns the
-    PEM version of it as a string."""
-
-    if hasattr(base64, 'standard_b64encode'):
-        # preferred because older API gets line-length wrong
-        f = base64.standard_b64encode(der_cert_bytes)
-        return (PEM_HEADER + '\n' +
-                textwrap.fill(f, 64) + '\n' +
-                PEM_FOOTER + '\n')
-    else:
-        return (PEM_HEADER + '\n' +
-                base64.encodestring(der_cert_bytes) +
-                PEM_FOOTER + '\n')
-
-def PEM_cert_to_DER_cert(pem_cert_string):
-
-    """Takes a certificate in ASCII PEM format and returns the
-    DER-encoded version of it as a byte sequence"""
-
-    if not pem_cert_string.startswith(PEM_HEADER):
-        raise ValueError("Invalid PEM encoding; must start with %s"
-                         % PEM_HEADER)
-    if not pem_cert_string.strip().endswith(PEM_FOOTER):
-        raise ValueError("Invalid PEM encoding; must end with %s"
-                         % PEM_FOOTER)
-    d = pem_cert_string.strip()[len(PEM_HEADER):-len(PEM_FOOTER)]
-    return base64.decodestring(d)
-
-def get_server_certificate(addr, ssl_version=PROTOCOL_SSLv3, ca_certs=None):
-
-    """Retrieve the certificate from the server at the specified address,
-    and return it as a PEM-encoded string.
-    If 'ca_certs' is specified, validate the server cert against it.
-    If 'ssl_version' is specified, use it in the connection attempt."""
-
-    host, port = addr
-    if (ca_certs is not None):
-        cert_reqs = CERT_REQUIRED
-    else:
-        cert_reqs = CERT_NONE
-    s = wrap_socket(socket(), ssl_version=ssl_version,
-                    cert_reqs=cert_reqs, ca_certs=ca_certs)
-    s.connect(addr)
-    dercert = s.getpeercert(True)
-    s.close()
-    return DER_cert_to_PEM_cert(dercert)
-
-def get_protocol_name(protocol_code):
-    if protocol_code == PROTOCOL_TLSv1:
-        return "TLSv1"
-    elif protocol_code == PROTOCOL_SSLv23:
-        return "SSLv23"
-    elif protocol_code == PROTOCOL_SSLv2:
-        return "SSLv2"
-    elif protocol_code == PROTOCOL_SSLv3:
-        return "SSLv3"
-    else:
-        return "<unknown>"
-
-
-# a replacement for the old socket.ssl function
-
-def sslwrap_simple(sock, keyfile=None, certfile=None):
-
-    """A replacement for the old socket.ssl function.  Designed
-    for compability with Python 2.5 and earlier.  Will disappear in
-    Python 3.0."""
-
-    if hasattr(sock, "_sock"):
-        sock = sock._sock
-
-    ssl_sock = _ssl.sslwrap(sock, 0, keyfile, certfile, CERT_NONE,
-                            PROTOCOL_SSLv23, None)
-    try:
-        sock.getpeername()
-    except socket_error:
-        # no, no connection yet
-        pass
-    else:
-        # yes, do the handshake
-        ssl_sock.do_handshake()
-
-    return ssl_sock

diff --git a/lib-python/modified-2.7.0/test/test_ssl.py b/lib-python/modified-2.7.0/test/test_ssl.py
deleted file mode 100644
--- a/lib-python/modified-2.7.0/test/test_ssl.py
+++ /dev/null
@@ -1,1318 +0,0 @@
-# Test the support for SSL and sockets
-
-import sys
-import unittest
-from test import test_support
-import asyncore
-import socket
-import select
-import time
-import gc
-import os
-import errno
-import pprint
-import urllib, urlparse
-import traceback
-import weakref
-import functools
-import platform
-
-from BaseHTTPServer import HTTPServer
-from SimpleHTTPServer import SimpleHTTPRequestHandler
-
-# Optionally test SSL support, if we have it in the tested platform
-skip_expected = False
-try:
-    import ssl
-except ImportError:
-    skip_expected = True
-
-HOST = test_support.HOST
-CERTFILE = None
-SVN_PYTHON_ORG_ROOT_CERT = None
-
-def handle_error(prefix):
-    exc_format = ' '.join(traceback.format_exception(*sys.exc_info()))
-    if test_support.verbose:
-        sys.stdout.write(prefix + exc_format)
-
-
-class BasicTests(unittest.TestCase):
-
-    def test_sslwrap_simple(self):
-        # A crude test for the legacy API
-        try:
-            ssl.sslwrap_simple(socket.socket(socket.AF_INET))
-        except IOError, e:
-            if e.errno == 32: # broken pipe when ssl_sock.do_handshake(), this test doesn't care about that
-                pass
-            else:
-                raise
-        try:
-            ssl.sslwrap_simple(socket.socket(socket.AF_INET)._sock)
-        except IOError, e:
-            if e.errno == 32: # broken pipe when ssl_sock.do_handshake(), this test doesn't care about that
-                pass
-            else:
-                raise
-
-# Issue #9415: Ubuntu hijacks their OpenSSL and forcefully disables SSLv2
-def skip_if_broken_ubuntu_ssl(func):
-    # We need to access the lower-level wrapper in order to create an
-    # implicit SSL context without trying to connect or listen.
-    try:
-        import _ssl
-    except ImportError:
-        # The returned function won't get executed, just ignore the error
-        pass
-    @functools.wraps(func)
-    def f(*args, **kwargs):
-        try:
-            s = socket.socket(socket.AF_INET)
-            _ssl.sslwrap(s._sock, 0, None, None,
-                         ssl.CERT_NONE, ssl.PROTOCOL_SSLv2, None, None)
-        except ssl.SSLError as e:
-            if (ssl.OPENSSL_VERSION_INFO == (0, 9, 8, 15, 15) and
-                platform.linux_distribution() == ('debian', 'squeeze/sid', '')
-                and 'Invalid SSL protocol variant specified' in str(e)):
-                raise unittest.SkipTest("Patched Ubuntu OpenSSL breaks behaviour")
-        return func(*args, **kwargs)
-    return f
-
-
-class BasicSocketTests(unittest.TestCase):
-
-    def test_constants(self):
-        ssl.PROTOCOL_SSLv2
-        ssl.PROTOCOL_SSLv23
-        ssl.PROTOCOL_SSLv3
-        ssl.PROTOCOL_TLSv1
-        ssl.CERT_NONE
-        ssl.CERT_OPTIONAL
-        ssl.CERT_REQUIRED
-
-    def test_random(self):
-        v = ssl.RAND_status()
-        if test_support.verbose:
-            sys.stdout.write("\n RAND_status is %d (%s)\n"
-                             % (v, (v and "sufficient randomness") or
-                                "insufficient randomness"))
-        try:
-            ssl.RAND_egd(1)
-        except TypeError:
-            pass
-        else:
-            print "didn't raise TypeError"
-        ssl.RAND_add("this is a random string", 75.0)
-
-    @test_support.impl_detail("obscure test")
-    def test_parse_cert(self):
-        # note that this uses an 'unofficial' function in _ssl.c,
-        # provided solely for this test, to exercise the certificate
-        # parsing code
-        p = ssl._ssl._test_decode_cert(CERTFILE, False)
-        if test_support.verbose:
-            sys.stdout.write("\n" + pprint.pformat(p) + "\n")
-
-    def test_DER_to_PEM(self):
-        with open(SVN_PYTHON_ORG_ROOT_CERT, 'r') as f:
-            pem = f.read()
-        d1 = ssl.PEM_cert_to_DER_cert(pem)
-        p2 = ssl.DER_cert_to_PEM_cert(d1)
-        d2 = ssl.PEM_cert_to_DER_cert(p2)
-        self.assertEqual(d1, d2)
-        if not p2.startswith(ssl.PEM_HEADER + '\n'):
-            self.fail("DER-to-PEM didn't include correct header:\n%r\n" % p2)
-        if not p2.endswith('\n' + ssl.PEM_FOOTER + '\n'):
-            self.fail("DER-to-PEM didn't include correct footer:\n%r\n" % p2)
-
-    def test_openssl_version(self):
-        n = ssl.OPENSSL_VERSION_NUMBER
-        t = ssl.OPENSSL_VERSION_INFO
-        s = ssl.OPENSSL_VERSION
-        self.assertIsInstance(n, (int, long))
-        self.assertIsInstance(t, tuple)
-        self.assertIsInstance(s, str)
-        # Some sanity checks follow
-        # >= 0.9
-        self.assertGreaterEqual(n, 0x900000)
-        # < 2.0
-        self.assertLess(n, 0x20000000)
-        major, minor, fix, patch, status = t
-        self.assertGreaterEqual(major, 0)
-        self.assertLess(major, 2)
-        self.assertGreaterEqual(minor, 0)
-        self.assertLess(minor, 256)
-        self.assertGreaterEqual(fix, 0)
-        self.assertLess(fix, 256)
-        self.assertGreaterEqual(patch, 0)
-        self.assertLessEqual(patch, 26)
-        self.assertGreaterEqual(status, 0)
-        self.assertLessEqual(status, 15)
-        # Version string as returned by OpenSSL, the format might change
-        self.assertTrue(s.startswith("OpenSSL {:d}.{:d}.{:d}".format(major, minor, fix)),
-                        (s, t))
-
-    def test_ciphers(self):
-        if not test_support.is_resource_enabled('network'):
-            return
-        remote = ("svn.python.org", 443)
-        with test_support.transient_internet(remote[0]):
-            s = ssl.wrap_socket(socket.socket(socket.AF_INET),
-                                cert_reqs=ssl.CERT_NONE, ciphers="ALL")
-            s.connect(remote)
-            s = ssl.wrap_socket(socket.socket(socket.AF_INET),
-                                cert_reqs=ssl.CERT_NONE, ciphers="DEFAULT")
-            s.connect(remote)
-            # Error checking occurs when connecting, because the SSL context
-            # isn't created before.
-            s = ssl.wrap_socket(socket.socket(socket.AF_INET),
-                                cert_reqs=ssl.CERT_NONE, ciphers="^$:,;?*'dorothyx")
-            with self.assertRaisesRegexp(ssl.SSLError, "No cipher can be selected"):
-                s.connect(remote)
-
-    @test_support.cpython_only
-    def test_refcycle(self):
-        # Issue #7943: an SSL object doesn't create reference cycles with
-        # itself.
-        s = socket.socket(socket.AF_INET)
-        ss = ssl.wrap_socket(s)
-        wr = weakref.ref(ss)
-        del ss
-        self.assertEqual(wr(), None)
-
-    def test_wrapped_unconnected(self):
-        # The _delegate_methods in socket.py are correctly delegated to by an
-        # unconnected SSLSocket, so they will raise a socket.error rather than
-        # something unexpected like TypeError.
-        s = socket.socket(socket.AF_INET)
-        ss = ssl.wrap_socket(s)
-        self.assertRaises(socket.error, ss.recv, 1)
-        self.assertRaises(socket.error, ss.recv_into, bytearray(b'x'))
-        self.assertRaises(socket.error, ss.recvfrom, 1)
-        self.assertRaises(socket.error, ss.recvfrom_into, bytearray(b'x'), 1)
-        self.assertRaises(socket.error, ss.send, b'x')
-        self.assertRaises(socket.error, ss.sendto, b'x', ('0.0.0.0', 0))
-
-
-class NetworkedTests(unittest.TestCase):
-
-    def test_connect(self):
-        with test_support.transient_internet("svn.python.org"):
-            s = ssl.wrap_socket(socket.socket(socket.AF_INET),
-                                cert_reqs=ssl.CERT_NONE)
-            s.connect(("svn.python.org", 443))
-            c = s.getpeercert()
-            if c:
-                self.fail("Peer cert %s shouldn't be here!")
-            s.close()
-
-            # this should fail because we have no verification certs
-            s = ssl.wrap_socket(socket.socket(socket.AF_INET),
-                                cert_reqs=ssl.CERT_REQUIRED)
-            try:
-                s.connect(("svn.python.org", 443))
-            except ssl.SSLError:
-                pass
-            finally:
-                s.close()
-
-            # this should succeed because we specify the root cert
-            s = ssl.wrap_socket(socket.socket(socket.AF_INET),
-                                cert_reqs=ssl.CERT_REQUIRED,
-                                ca_certs=SVN_PYTHON_ORG_ROOT_CERT)
-            try:
-                s.connect(("svn.python.org", 443))
-            finally:
-                s.close()
-
-    @unittest.skipIf(os.name == "nt", "Can't use a socket as a file under Windows")
-    def test_makefile_close(self):
-        # Issue #5238: creating a file-like object with makefile() shouldn't
-        # delay closing the underlying "real socket" (here tested with its
-        # file descriptor, hence skipping the test under Windows).
-        with test_support.transient_internet("svn.python.org"):
-            ss = ssl.wrap_socket(socket.socket(socket.AF_INET))
-            ss.connect(("svn.python.org", 443))
-            fd = ss.fileno()
-            f = ss.makefile()
-            f.close()
-            # The fd is still open
-            os.read(fd, 0)
-            # Closing the SSL socket should close the fd too
-            ss.close()
-            gc.collect()
-            with self.assertRaises(OSError) as e:
-                os.read(fd, 0)
-            self.assertEqual(e.exception.errno, errno.EBADF)
-
-    def test_non_blocking_handshake(self):
-        with test_support.transient_internet("svn.python.org"):
-            s = socket.socket(socket.AF_INET)
-            s.connect(("svn.python.org", 443))
-            s.setblocking(False)
-            s = ssl.wrap_socket(s,
-                                cert_reqs=ssl.CERT_NONE,
-                                do_handshake_on_connect=False)
-            count = 0
-            while True:
-                try:
-                    count += 1
-                    s.do_handshake()
-                    break
-                except ssl.SSLError, err:
-                    if err.args[0] == ssl.SSL_ERROR_WANT_READ:
-                        select.select([s], [], [])
-                    elif err.args[0] == ssl.SSL_ERROR_WANT_WRITE:
-                        select.select([], [s], [])
-                    else:
-                        raise
-            s.close()
-            if test_support.verbose:
-                sys.stdout.write("\nNeeded %d calls to do_handshake() to establish session.\n" % count)
-
-    def test_get_server_certificate(self):
-        with test_support.transient_internet("svn.python.org"):
-            pem = ssl.get_server_certificate(("svn.python.org", 443))
-            if not pem:
-                self.fail("No server certificate on svn.python.org:443!")
-
-            try:
-                pem = ssl.get_server_certificate(("svn.python.org", 443), ca_certs=CERTFILE)
-            except ssl.SSLError:
-                #should fail
-                pass
-            else:
-                self.fail("Got server certificate %s for svn.python.org!" % pem)
-
-            pem = ssl.get_server_certificate(("svn.python.org", 443), ca_certs=SVN_PYTHON_ORG_ROOT_CERT)
-            if not pem:
-                self.fail("No server certificate on svn.python.org:443!")
-            if test_support.verbose:
-                sys.stdout.write("\nVerified certificate for svn.python.org:443 is\n%s\n" % pem)
-
-    def test_algorithms(self):
-        # Issue #8484: all algorithms should be available when verifying a
-        # certificate.
-        # SHA256 was added in OpenSSL 0.9.8
-        if ssl.OPENSSL_VERSION_INFO < (0, 9, 8, 0, 15):
-            self.skipTest("SHA256 not available on %r" % ssl.OPENSSL_VERSION)
-        # NOTE: https://sha256.tbs-internet.com is another possible test host
-        remote = ("sha2.hboeck.de", 443)
-        sha256_cert = os.path.join(os.path.dirname(__file__), "sha256.pem")
-        with test_support.transient_internet("sha2.hboeck.de"):
-            s = ssl.wrap_socket(socket.socket(socket.AF_INET),
-                                cert_reqs=ssl.CERT_REQUIRED,
-                                ca_certs=sha256_cert,)
-            try:
-                s.connect(remote)
-                if test_support.verbose:
-                    sys.stdout.write("\nCipher with %r is %r\n" %
-                                     (remote, s.cipher()))
-                    sys.stdout.write("Certificate is:\n%s\n" %
-                                     pprint.pformat(s.getpeercert()))
-            finally:
-                s.close()
-
-
-try:
-    import threading
-except ImportError:
-    _have_threads = False
-else:
-    _have_threads = True
-
-    class ThreadedEchoServer(threading.Thread):
-
-        class ConnectionHandler(threading.Thread):
-
-            """A mildly complicated class, because we want it to work both
-            with and without the SSL wrapper around the socket connection, so
-            that we can test the STARTTLS functionality."""
-
-            def __init__(self, server, connsock):
-                self.server = server
-                self.running = False
-                self.sock = connsock
-                self.sock.setblocking(1)
-                self.sslconn = None
-                threading.Thread.__init__(self)
-                self.daemon = True
-
-            def show_conn_details(self):
-                if self.server.certreqs == ssl.CERT_REQUIRED:
-                    cert = self.sslconn.getpeercert()
-                    if test_support.verbose and self.server.chatty:
-                        sys.stdout.write(" client cert is " + pprint.pformat(cert) + "\n")
-                    cert_binary = self.sslconn.getpeercert(True)
-                    if test_support.verbose and self.server.chatty:
-                        sys.stdout.write(" cert binary is " + str(len(cert_binary)) + " bytes\n")
-                cipher = self.sslconn.cipher()
-                if test_support.verbose and self.server.chatty:
-                    sys.stdout.write(" server: connection cipher is now " + str(cipher) + "\n")
-
-            def wrap_conn(self):
-                try:
-                    self.sslconn = ssl.wrap_socket(self.sock, server_side=True,
-                                                   certfile=self.server.certificate,
-                                                   ssl_version=self.server.protocol,
-                                                   ca_certs=self.server.cacerts,
-                                                   cert_reqs=self.server.certreqs,
-                                                   ciphers=self.server.ciphers)
-                except ssl.SSLError:
-                    # XXX Various errors can have happened here, for example
-                    # a mismatching protocol version, an invalid certificate,
-                    # or a low-level bug. This should be made more discriminating.
-                    if self.server.chatty:
-                        handle_error("\n server:  bad connection attempt from " +
-                                     str(self.sock.getpeername()) + ":\n")
-                    self.close()
-                    self.running = False
-                    self.server.stop()
-                    return False
-                else:
-                    return True
-
-            def read(self):
-                if self.sslconn:
-                    return self.sslconn.read()
-                else:
-                    return self.sock.recv(1024)
-
-            def write(self, bytes):
-                if self.sslconn:
-                    return self.sslconn.write(bytes)
-                else:
-                    return self.sock.send(bytes)
-
-            def close(self):
-                if self.sslconn:
-                    self.sslconn.close()
-                else:
-                    self.sock._sock.close()
-
-            def run(self):
-                self.running = True
-                if not self.server.starttls_server:
-                    if isinstance(self.sock, ssl.SSLSocket):
-                        self.sslconn = self.sock
-                    elif not self.wrap_conn():
-                        return
-                    self.show_conn_details()
-                while self.running:
-                    try:
-                        msg = self.read()
-                        if not msg:
-                            # eof, so quit this handler
-                            self.running = False
-                            self.close()
-                        elif msg.strip() == 'over':
-                            if test_support.verbose and self.server.connectionchatty:
-                                sys.stdout.write(" server: client closed connection\n")
-                            self.close()
-                            return
-                        elif self.server.starttls_server and msg.strip() == 'STARTTLS':
-                            if test_support.verbose and self.server.connectionchatty:
-                                sys.stdout.write(" server: read STARTTLS from client, sending OK...\n")
-                            self.write("OK\n")
-                            if not self.wrap_conn():
-                                return
-                        elif self.server.starttls_server and self.sslconn and msg.strip() == 'ENDTLS':
-                            if test_support.verbose and self.server.connectionchatty:
-                                sys.stdout.write(" server: read ENDTLS from client, sending OK...\n")
-                            self.write("OK\n")
-                            self.sslconn.unwrap()
-                            self.sslconn = None
-                            if test_support.verbose and self.server.connectionchatty:
-                                sys.stdout.write(" server: connection is now unencrypted...\n")
-                        else:
-                            if (test_support.verbose and
-                                self.server.connectionchatty):
-                                ctype = (self.sslconn and "encrypted") or "unencrypted"
-                                sys.stdout.write(" server: read %s (%s), sending back %s (%s)...\n"
-                                                 % (repr(msg), ctype, repr(msg.lower()), ctype))
-                            self.write(msg.lower())
-                    except ssl.SSLError:
-                        if self.server.chatty:
-                            handle_error("Test server failure:\n")
-                        self.close()
-                        self.running = False
-                        # normally, we'd just stop here, but for the test
-                        # harness, we want to stop the server
-                        self.server.stop()
-
-        def __init__(self, certificate, ssl_version=None,
-                     certreqs=None, cacerts=None,
-                     chatty=True, connectionchatty=False, starttls_server=False,
-                     wrap_accepting_socket=False, ciphers=None):
-
-            if ssl_version is None:
-                ssl_version = ssl.PROTOCOL_TLSv1
-            if certreqs is None:
-                certreqs = ssl.CERT_NONE
-            self.certificate = certificate
-            self.protocol = ssl_version
-            self.certreqs = certreqs
-            self.cacerts = cacerts
-            self.ciphers = ciphers
-            self.chatty = chatty
-            self.connectionchatty = connectionchatty
-            self.starttls_server = starttls_server
-            self.sock = socket.socket()
-            self.flag = None
-            if wrap_accepting_socket:
-                self.sock = ssl.wrap_socket(self.sock, server_side=True,
-                                            certfile=self.certificate,
-                                            cert_reqs = self.certreqs,
-                                            ca_certs = self.cacerts,
-                                            ssl_version = self.protocol,
-                                            ciphers = self.ciphers)
-                if test_support.verbose and self.chatty:
-                    sys.stdout.write(' server:  wrapped server socket as %s\n' % str(self.sock))
-            self.port = test_support.bind_port(self.sock)
-            self.active = False
-            threading.Thread.__init__(self)
-            self.daemon = True
-
-        def start(self, flag=None):
-            self.flag = flag
-            threading.Thread.start(self)
-
-        def run(self):
-            self.sock.settimeout(0.05)
-            self.sock.listen(5)
-            self.active = True
-            if self.flag:
-                # signal an event
-                self.flag.set()
-            while self.active:
-                try:
-                    newconn, connaddr = self.sock.accept()
-                    if test_support.verbose and self.chatty:
-                        sys.stdout.write(' server:  new connection from '
-                                         + str(connaddr) + '\n')
-                    handler = self.ConnectionHandler(self, newconn)
-                    handler.start()
-                except socket.timeout:
-                    pass
-                except KeyboardInterrupt:
-                    self.stop()
-            self.sock.close()
-
-        def stop(self):
-            self.active = False
-
-    class AsyncoreEchoServer(threading.Thread):
-
-        class EchoServer(asyncore.dispatcher):
-
-            class ConnectionHandler(asyncore.dispatcher_with_send):
-
-                def __init__(self, conn, certfile):
-                    asyncore.dispatcher_with_send.__init__(self, conn)
-                    self.socket = ssl.wrap_socket(conn, server_side=True,
-                                                  certfile=certfile,
-                                                  do_handshake_on_connect=False)
-                    self._ssl_accepting = True
-
-                def readable(self):
-                    if isinstance(self.socket, ssl.SSLSocket):
-                        while self.socket.pending() > 0:
-                            self.handle_read_event()
-                    return True
-
-                def _do_ssl_handshake(self):
-                    try:
-                        self.socket.do_handshake()
-                    except ssl.SSLError, err:
-                        if err.args[0] in (ssl.SSL_ERROR_WANT_READ,
-                                           ssl.SSL_ERROR_WANT_WRITE):
-                            return
-                        elif err.args[0] == ssl.SSL_ERROR_EOF:
-                            return self.handle_close()
-                        raise
-                    except socket.error, err:
-                        if err.args[0] == errno.ECONNABORTED:
-                            return self.handle_close()
-                    else:
-                        self._ssl_accepting = False
-
-                def handle_read(self):
-                    if self._ssl_accepting:
-                        self._do_ssl_handshake()
-                    else:
-                        data = self.recv(1024)
-                        if data and data.strip() != 'over':
-                            self.send(data.lower())
-
-                def handle_close(self):
-                    self.close()
-                    if test_support.verbose:
-                        sys.stdout.write(" server:  closed connection %s\n" % self.socket)
-
-                def handle_error(self):
-                    raise
-
-            def __init__(self, certfile):
-                self.certfile = certfile
-                asyncore.dispatcher.__init__(self)
-                self.create_socket(socket.AF_INET, socket.SOCK_STREAM)
-                self.port = test_support.bind_port(self.socket)
-                self.listen(5)
-
-            def handle_accept(self):
-                sock_obj, addr = self.accept()
-                if test_support.verbose:
-                    sys.stdout.write(" server:  new connection from %s:%s\n" %addr)
-                self.ConnectionHandler(sock_obj, self.certfile)
-
-            def handle_error(self):
-                raise
-
-        def __init__(self, certfile):
-            self.flag = None
-            self.active = False
-            self.server = self.EchoServer(certfile)
-            self.port = self.server.port
-            threading.Thread.__init__(self)
-            self.daemon = True
-
-        def __str__(self):
-            return "<%s %s>" % (self.__class__.__name__, self.server)
-
-        def start(self, flag=None):
-            self.flag = flag
-            threading.Thread.start(self)
-
-        def run(self):
-            self.active = True
-            if self.flag:
-                self.flag.set()
-            while self.active:
-                asyncore.loop(0.05)
-
-        def stop(self):
-            self.active = False
-            self.server.close()
-
-    class SocketServerHTTPSServer(threading.Thread):
-
-        class HTTPSServer(HTTPServer):
-
-            def __init__(self, server_address, RequestHandlerClass, certfile):
-                HTTPServer.__init__(self, server_address, RequestHandlerClass)
-                # we assume the certfile contains both private key and certificate
-                self.certfile = certfile
-                self.allow_reuse_address = True
-
-            def __str__(self):
-                return ('<%s %s:%s>' %
-                        (self.__class__.__name__,
-                         self.server_name,
-                         self.server_port))
-
-            def get_request(self):
-                # override this to wrap socket with SSL
-                sock, addr = self.socket.accept()
-                sslconn = ssl.wrap_socket(sock, server_side=True,
-                                          certfile=self.certfile)
-                return sslconn, addr
-
-        class RootedHTTPRequestHandler(SimpleHTTPRequestHandler):
-            # need to override translate_path to get a known root,
-            # instead of using os.curdir, since the test could be
-            # run from anywhere
-
-            server_version = "TestHTTPS/1.0"
-
-            root = None
-
-            def translate_path(self, path):
-                """Translate a /-separated PATH to the local filename syntax.
-
-                Components that mean special things to the local file system
-                (e.g. drive or directory names) are ignored.  (XXX They should
-                probably be diagnosed.)
-
-                """
-                # abandon query parameters
-                path = urlparse.urlparse(path)[2]
-                path = os.path.normpath(urllib.unquote(path))
-                words = path.split('/')
-                words = filter(None, words)
-                path = self.root
-                for word in words:
-                    drive, word = os.path.splitdrive(word)
-                    head, word = os.path.split(word)
-                    if word in self.root: continue
-                    path = os.path.join(path, word)
-                return path
-
-            def log_message(self, format, *args):
-
-                # we override this to suppress logging unless "verbose"
-
-                if test_support.verbose:
-                    sys.stdout.write(" server (%s:%d %s):\n   [%s] %s\n" %
-                                     (self.server.server_address,
-                                      self.server.server_port,
-                                      self.request.cipher(),
-                                      self.log_date_time_string(),
-                                      format%args))
-
-
-        def __init__(self, certfile):
-            self.flag = None
-            self.RootedHTTPRequestHandler.root = os.path.split(CERTFILE)[0]
-            self.server = self.HTTPSServer(
-                (HOST, 0), self.RootedHTTPRequestHandler, certfile)
-            self.port = self.server.server_port
-            threading.Thread.__init__(self)
-            self.daemon = True
-
-        def __str__(self):
-            return "<%s %s>" % (self.__class__.__name__, self.server)
-
-        def start(self, flag=None):
-            self.flag = flag
-            threading.Thread.start(self)
-
-        def run(self):
-            if self.flag:
-                self.flag.set()
-            self.server.serve_forever(0.05)
-
-        def stop(self):
-            self.server.shutdown()
-
-
-    def bad_cert_test(certfile):
-        """
-        Launch a server with CERT_REQUIRED, and check that trying to
-        connect to it with the given client certificate fails.
-        """
-        server = ThreadedEchoServer(CERTFILE,
-                                    certreqs=ssl.CERT_REQUIRED,
-                                    cacerts=CERTFILE, chatty=False)
-        flag = threading.Event()
-        server.start(flag)
-        # wait for it to start
-        flag.wait()
-        # try to connect
-        try:
-            try:
-                s = ssl.wrap_socket(socket.socket(),
-                                    certfile=certfile,
-                                    ssl_version=ssl.PROTOCOL_TLSv1)
-                s.connect((HOST, server.port))
-            except ssl.SSLError, x:
-                if test_support.verbose:
-                    sys.stdout.write("\nSSLError is %s\n" % x[1])
-            except socket.error, x:
-                if test_support.verbose:
-                    sys.stdout.write("\nsocket.error is %s\n" % x[1])
-            else:
-                raise AssertionError("Use of invalid cert should have failed!")
-        finally:
-            server.stop()
-            server.join()
-
-    def server_params_test(certfile, protocol, certreqs, cacertsfile,
-                           client_certfile, client_protocol=None, indata="FOO\n",
-                           ciphers=None, chatty=True, connectionchatty=False,
-                           wrap_accepting_socket=False):
-        """
-        Launch a server, connect a client to it and try various reads
-        and writes.
-        """
-        server = ThreadedEchoServer(certfile,
-                                    certreqs=certreqs,
-                                    ssl_version=protocol,
-                                    cacerts=cacertsfile,
-                                    ciphers=ciphers,
-                                    chatty=chatty,
-                                    connectionchatty=connectionchatty,
-                                    wrap_accepting_socket=wrap_accepting_socket)
-        flag = threading.Event()
-        server.start(flag)
-        # wait for it to start
-        flag.wait()
-        # try to connect
-        if client_protocol is None:
-            client_protocol = protocol
-        try:
-            s = ssl.wrap_socket(socket.socket(),
-                                certfile=client_certfile,
-                                ca_certs=cacertsfile,
-                                ciphers=ciphers,
-                                cert_reqs=certreqs,
-                                ssl_version=client_protocol)
-            s.connect((HOST, server.port))
-            for arg in [indata, bytearray(indata), memoryview(indata)]:
-                if connectionchatty:
-                    if test_support.verbose:
-                        sys.stdout.write(
-                            " client:  sending %s...\n" % (repr(arg)))
-                s.write(arg)
-                outdata = s.read()
-                if connectionchatty:
-                    if test_support.verbose:
-                        sys.stdout.write(" client:  read %s\n" % repr(outdata))
-                if outdata != indata.lower():
-                    raise AssertionError(
-                        "bad data <<%s>> (%d) received; expected <<%s>> (%d)\n"
-                        % (outdata[:min(len(outdata),20)], len(outdata),
-                           indata[:min(len(indata),20)].lower(), len(indata)))
-            s.write("over\n")
-            if connectionchatty:
-                if test_support.verbose:
-                    sys.stdout.write(" client:  closing connection.\n")
-            s.close()
-        finally:
-            server.stop()
-            server.join()
-
-    def try_protocol_combo(server_protocol,
-                           client_protocol,
-                           expect_success,
-                           certsreqs=None):
-        if certsreqs is None:
-            certsreqs = ssl.CERT_NONE
-        certtype = {
-            ssl.CERT_NONE: "CERT_NONE",
-            ssl.CERT_OPTIONAL: "CERT_OPTIONAL",
-            ssl.CERT_REQUIRED: "CERT_REQUIRED",
-        }[certsreqs]
-        if test_support.verbose:
-            formatstr = (expect_success and " %s->%s %s\n") or " {%s->%s} %s\n"
-            sys.stdout.write(formatstr %
-                             (ssl.get_protocol_name(client_protocol),
-                              ssl.get_protocol_name(server_protocol),
-                              certtype))
-        try:
-            # NOTE: we must enable "ALL" ciphers, otherwise an SSLv23 client
-            # will send an SSLv3 hello (rather than SSLv2) starting from
-            # OpenSSL 1.0.0 (see issue #8322).
-            server_params_test(CERTFILE, server_protocol, certsreqs,
-                               CERTFILE, CERTFILE, client_protocol,
-                               ciphers="ALL", chatty=False)
-        # Protocol mismatch can result in either an SSLError, or a
-        # "Connection reset by peer" error.
-        except ssl.SSLError:
-            if expect_success:
-                raise
-        except socket.error as e:
-            if expect_success or e.errno != errno.ECONNRESET:
-                raise
-        else:
-            if not expect_success:
-                raise AssertionError(
-                    "Client protocol %s succeeded with server protocol %s!"
-                    % (ssl.get_protocol_name(client_protocol),
-                       ssl.get_protocol_name(server_protocol)))
-
-
-    class ThreadedTests(unittest.TestCase):
-
-        def test_rude_shutdown(self):
-            """A brutal shutdown of an SSL server should raise an IOError
-            in the client when attempting handshake.
-            """
-            listener_ready = threading.Event()
-            listener_gone = threading.Event()
-
-            s = socket.socket()
-            port = test_support.bind_port(s, HOST)
-
-            # `listener` runs in a thread.  It sits in an accept() until
-            # the main thread connects.  Then it rudely closes the socket,
-            # and sets Event `listener_gone` to let the main thread know
-            # the socket is gone.
-            def listener():
-                s.listen(5)
-                listener_ready.set()
-                s.accept()
-                s.close()
-                listener_gone.set()
-
-            def connector():
-                listener_ready.wait()
-                c = socket.socket()
-                c.connect((HOST, port))
-                listener_gone.wait()
-                try:
-                    ssl_sock = ssl.wrap_socket(c)
-                except IOError:
-                    pass
-                else:
-                    self.fail('connecting to closed SSL socket should have failed')
-
-            t = threading.Thread(target=listener)
-            t.start()
-            try:
-                connector()
-            finally:
-                t.join()
-
-        @skip_if_broken_ubuntu_ssl
-        def test_echo(self):
-            """Basic test of an SSL client connecting to a server"""
-            if test_support.verbose:
-                sys.stdout.write("\n")
-            server_params_test(CERTFILE, ssl.PROTOCOL_TLSv1, ssl.CERT_NONE,
-                               CERTFILE, CERTFILE, ssl.PROTOCOL_TLSv1,
-                               chatty=True, connectionchatty=True)
-
-        def test_getpeercert(self):
-            if test_support.verbose:
-                sys.stdout.write("\n")
-            s2 = socket.socket()
-            server = ThreadedEchoServer(CERTFILE,
-                                        certreqs=ssl.CERT_NONE,
-                                        ssl_version=ssl.PROTOCOL_SSLv23,
-                                        cacerts=CERTFILE,
-                                        chatty=False)
-            flag = threading.Event()
-            server.start(flag)
-            # wait for it to start
-            flag.wait()
-            # try to connect
-            try:
-                s = ssl.wrap_socket(socket.socket(),
-                                    certfile=CERTFILE,
-                                    ca_certs=CERTFILE,
-                                    cert_reqs=ssl.CERT_REQUIRED,
-                                    ssl_version=ssl.PROTOCOL_SSLv23)
-                s.connect((HOST, server.port))
-                cert = s.getpeercert()
-                self.assertTrue(cert, "Can't get peer certificate.")
-                cipher = s.cipher()
-                if test_support.verbose:
-                    sys.stdout.write(pprint.pformat(cert) + '\n')
-                    sys.stdout.write("Connection cipher is " + str(cipher) + '.\n')
-                if 'subject' not in cert:
-                    self.fail("No subject field in certificate: %s." %
-                              pprint.pformat(cert))
-                if ((('organizationName', 'Python Software Foundation'),)
-                    not in cert['subject']):
-                    self.fail(
-                        "Missing or invalid 'organizationName' field in certificate subject; "
-                        "should be 'Python Software Foundation'.")
-                s.close()
-            finally:
-                server.stop()
-                server.join()
-
-        def test_empty_cert(self):
-            """Connecting with an empty cert file"""
-            bad_cert_test(os.path.join(os.path.dirname(__file__) or os.curdir,
-                                      "nullcert.pem"))
-        def test_malformed_cert(self):
-            """Connecting with a badly formatted certificate (syntax error)"""
-            bad_cert_test(os.path.join(os.path.dirname(__file__) or os.curdir,
-                                       "badcert.pem"))
-        def test_nonexisting_cert(self):
-            """Connecting with a non-existing cert file"""
-            bad_cert_test(os.path.join(os.path.dirname(__file__) or os.curdir,
-                                       "wrongcert.pem"))
-        def test_malformed_key(self):
-            """Connecting with a badly formatted key (syntax error)"""
-            bad_cert_test(os.path.join(os.path.dirname(__file__) or os.curdir,
-                                       "badkey.pem"))
-
-        @skip_if_broken_ubuntu_ssl
-        def test_protocol_sslv2(self):
-            """Connecting to an SSLv2 server with various client options"""
-            if test_support.verbose:
-                sys.stdout.write("\n")
-            try_protocol_combo(ssl.PROTOCOL_SSLv2, ssl.PROTOCOL_SSLv2, True)
-            try_protocol_combo(ssl.PROTOCOL_SSLv2, ssl.PROTOCOL_SSLv2, True, ssl.CERT_OPTIONAL)
-            try_protocol_combo(ssl.PROTOCOL_SSLv2, ssl.PROTOCOL_SSLv2, True, ssl.CERT_REQUIRED)
-            try_protocol_combo(ssl.PROTOCOL_SSLv2, ssl.PROTOCOL_SSLv23, True)
-            try_protocol_combo(ssl.PROTOCOL_SSLv2, ssl.PROTOCOL_SSLv3, False)
-            try_protocol_combo(ssl.PROTOCOL_SSLv2, ssl.PROTOCOL_TLSv1, False)
-
-        @skip_if_broken_ubuntu_ssl
-        def test_protocol_sslv23(self):
-            """Connecting to an SSLv23 server with various client options"""
-            if test_support.verbose:
-                sys.stdout.write("\n")
-            try:
-                try_protocol_combo(ssl.PROTOCOL_SSLv23, ssl.PROTOCOL_SSLv2, True)
-            except (ssl.SSLError, socket.error), x:
-                # this fails on some older versions of OpenSSL (0.9.7l, for instance)
-                if test_support.verbose:
-                    sys.stdout.write(
-                        " SSL2 client to SSL23 server test unexpectedly failed:\n %s\n"
-                        % str(x))
-            try_protocol_combo(ssl.PROTOCOL_SSLv23, ssl.PROTOCOL_SSLv3, True)
-            try_protocol_combo(ssl.PROTOCOL_SSLv23, ssl.PROTOCOL_SSLv23, True)
-            try_protocol_combo(ssl.PROTOCOL_SSLv23, ssl.PROTOCOL_TLSv1, True)
-
-            try_protocol_combo(ssl.PROTOCOL_SSLv23, ssl.PROTOCOL_SSLv3, True, ssl.CERT_OPTIONAL)
-            try_protocol_combo(ssl.PROTOCOL_SSLv23, ssl.PROTOCOL_SSLv23, True, ssl.CERT_OPTIONAL)
-            try_protocol_combo(ssl.PROTOCOL_SSLv23, ssl.PROTOCOL_TLSv1, True, ssl.CERT_OPTIONAL)
-
-            try_protocol_combo(ssl.PROTOCOL_SSLv23, ssl.PROTOCOL_SSLv3, True, ssl.CERT_REQUIRED)
-            try_protocol_combo(ssl.PROTOCOL_SSLv23, ssl.PROTOCOL_SSLv23, True, ssl.CERT_REQUIRED)
-            try_protocol_combo(ssl.PROTOCOL_SSLv23, ssl.PROTOCOL_TLSv1, True, ssl.CERT_REQUIRED)
-
-        @skip_if_broken_ubuntu_ssl
-        def test_protocol_sslv3(self):
-            """Connecting to an SSLv3 server with various client options"""
-            if test_support.verbose:
-                sys.stdout.write("\n")
-            try_protocol_combo(ssl.PROTOCOL_SSLv3, ssl.PROTOCOL_SSLv3, True)
-            try_protocol_combo(ssl.PROTOCOL_SSLv3, ssl.PROTOCOL_SSLv3, True, ssl.CERT_OPTIONAL)
-            try_protocol_combo(ssl.PROTOCOL_SSLv3, ssl.PROTOCOL_SSLv3, True, ssl.CERT_REQUIRED)
-            try_protocol_combo(ssl.PROTOCOL_SSLv3, ssl.PROTOCOL_SSLv2, False)
-            try_protocol_combo(ssl.PROTOCOL_SSLv3, ssl.PROTOCOL_SSLv23, False)
-            try_protocol_combo(ssl.PROTOCOL_SSLv3, ssl.PROTOCOL_TLSv1, False)
-
-        @skip_if_broken_ubuntu_ssl
-        def test_protocol_tlsv1(self):
-            """Connecting to a TLSv1 server with various client options"""
-            if test_support.verbose:
-                sys.stdout.write("\n")
-            try_protocol_combo(ssl.PROTOCOL_TLSv1, ssl.PROTOCOL_TLSv1, True)
-            try_protocol_combo(ssl.PROTOCOL_TLSv1, ssl.PROTOCOL_TLSv1, True, ssl.CERT_OPTIONAL)
-            try_protocol_combo(ssl.PROTOCOL_TLSv1, ssl.PROTOCOL_TLSv1, True, ssl.CERT_REQUIRED)
-            try_protocol_combo(ssl.PROTOCOL_TLSv1, ssl.PROTOCOL_SSLv2, False)
-            try_protocol_combo(ssl.PROTOCOL_TLSv1, ssl.PROTOCOL_SSLv3, False)
-            try_protocol_combo(ssl.PROTOCOL_TLSv1, ssl.PROTOCOL_SSLv23, False)
-
-        def test_starttls(self):
-            """Switching from clear text to encrypted and back again."""
-            msgs = ("msg 1", "MSG 2", "STARTTLS", "MSG 3", "msg 4", "ENDTLS", "msg 5", "msg 6")
-
-            server = ThreadedEchoServer(CERTFILE,
-                                        ssl_version=ssl.PROTOCOL_TLSv1,
-                                        starttls_server=True,
-                                        chatty=True,
-                                        connectionchatty=True)
-            flag = threading.Event()
-            server.start(flag)
-            # wait for it to start
-            flag.wait()
-            # try to connect
-            wrapped = False
-            try:
-                s = socket.socket()
-                s.setblocking(1)
-                s.connect((HOST, server.port))
-                if test_support.verbose:
-                    sys.stdout.write("\n")
-                for indata in msgs:
-                    if test_support.verbose:
-                        sys.stdout.write(
-                            " client:  sending %s...\n" % repr(indata))
-                    if wrapped:
-                        conn.write(indata)
-                        outdata = conn.read()
-                    else:
-                        s.send(indata)
-                        outdata = s.recv(1024)
-                    if (indata == "STARTTLS" and
-                        outdata.strip().lower().startswith("ok")):
-                        # STARTTLS ok, switch to secure mode
-                        if test_support.verbose:
-                            sys.stdout.write(
-                                " client:  read %s from server, starting TLS...\n"
-                                % repr(outdata))
-                        conn = ssl.wrap_socket(s, ssl_version=ssl.PROTOCOL_TLSv1)
-                        wrapped = True
-                    elif (indata == "ENDTLS" and
-                        outdata.strip().lower().startswith("ok")):
-                        # ENDTLS ok, switch back to clear text
-                        if test_support.verbose:
-                            sys.stdout.write(
-                                " client:  read %s from server, ending TLS...\n"
-                                % repr(outdata))
-                        s = conn.unwrap()
-                        wrapped = False
-                    else:
-                        if test_support.verbose:
-                            sys.stdout.write(
-                                " client:  read %s from server\n" % repr(outdata))
-                if test_support.verbose:
-                    sys.stdout.write(" client:  closing connection.\n")
-                if wrapped:
-                    conn.write("over\n")
-                else:
-                    s.send("over\n")
-                s.close()
-            finally:
-                server.stop()
-                server.join()
-
-        def test_socketserver(self):
-            """Using a SocketServer to create and manage SSL connections."""
-            server = SocketServerHTTPSServer(CERTFILE)
-            flag = threading.Event()
-            server.start(flag)
-            # wait for it to start
-            flag.wait()
-            # try to connect
-            try:
-                if test_support.verbose:
-                    sys.stdout.write('\n')
-                with open(CERTFILE, 'rb') as f:
-                    d1 = f.read()
-                d2 = ''
-                # now fetch the same data from the HTTPS server
-                url = 'https://127.0.0.1:%d/%s' % (
-                    server.port, os.path.split(CERTFILE)[1])
-                with test_support.check_py3k_warnings():
-                    f = urllib.urlopen(url)
-                dlen = f.info().getheader("content-length")
-                if dlen and (int(dlen) > 0):
-                    d2 = f.read(int(dlen))
-                    if test_support.verbose:
-                        sys.stdout.write(
-                            " client: read %d bytes from remote server '%s'\n"
-                            % (len(d2), server))
-                f.close()
-                self.assertEqual(d1, d2)
-            finally:
-                server.stop()
-                server.join()
-
-        def test_wrapped_accept(self):
-            """Check the accept() method on SSL sockets."""
-            if test_support.verbose:
-                sys.stdout.write("\n")
-            server_params_test(CERTFILE, ssl.PROTOCOL_SSLv23, ssl.CERT_REQUIRED,
-                               CERTFILE, CERTFILE, ssl.PROTOCOL_SSLv23,
-                               chatty=True, connectionchatty=True,
-                               wrap_accepting_socket=True)
-
-        def test_asyncore_server(self):
-            """Check the example asyncore integration."""
-            indata = "TEST MESSAGE of mixed case\n"
-
-            if test_support.verbose:
-                sys.stdout.write("\n")
-            server = AsyncoreEchoServer(CERTFILE)
-            flag = threading.Event()
-            server.start(flag)
-            # wait for it to start
-            flag.wait()
-            # try to connect
-            try:
-                s = ssl.wrap_socket(socket.socket())
-                s.connect(('127.0.0.1', server.port))
-                if test_support.verbose:
-                    sys.stdout.write(
-                        " client:  sending %s...\n" % (repr(indata)))
-                s.write(indata)
-                outdata = s.read()
-                if test_support.verbose:
-                    sys.stdout.write(" client:  read %s\n" % repr(outdata))
-                if outdata != indata.lower():
-                    self.fail(
-                        "bad data <<%s>> (%d) received; expected <<%s>> (%d)\n"
-                        % (outdata[:min(len(outdata),20)], len(outdata),
-                           indata[:min(len(indata),20)].lower(), len(indata)))
-                s.write("over\n")
-                if test_support.verbose:
-                    sys.stdout.write(" client:  closing connection.\n")
-                s.close()
-            finally:
-                server.stop()
-                # wait for server thread to end
-                server.join()
-
-        def test_recv_send(self):
-            """Test recv(), send() and friends."""
-            if test_support.verbose:
-                sys.stdout.write("\n")
-
-            server = ThreadedEchoServer(CERTFILE,
-                                        certreqs=ssl.CERT_NONE,
-                                        ssl_version=ssl.PROTOCOL_TLSv1,
-                                        cacerts=CERTFILE,
-                                        chatty=True,
-                                        connectionchatty=False)
-            flag = threading.Event()
-            server.start(flag)
-            # wait for it to start
-            flag.wait()
-            # try to connect
-            s = ssl.wrap_socket(socket.socket(),
-                                server_side=False,
-                                certfile=CERTFILE,
-                                ca_certs=CERTFILE,
-                                cert_reqs=ssl.CERT_NONE,
-                                ssl_version=ssl.PROTOCOL_TLSv1)
-            s.connect((HOST, server.port))
-            try:
-                # helper methods for standardising recv* method signatures
-                def _recv_into():
-                    b = bytearray("\0"*100)
-                    count = s.recv_into(b)
-                    return b[:count]
-
-                def _recvfrom_into():
-                    b = bytearray("\0"*100)
-                    count, addr = s.recvfrom_into(b)
-                    return b[:count]
-
-                # (name, method, whether to expect success, *args)
-                send_methods = [
-                    ('send', s.send, True, []),
-                    ('sendto', s.sendto, False, ["some.address"]),
-                    ('sendall', s.sendall, True, []),
-                ]
-                recv_methods = [
-                    ('recv', s.recv, True, []),
-                    ('recvfrom', s.recvfrom, False, ["some.address"]),
-                    ('recv_into', _recv_into, True, []),
-                    ('recvfrom_into', _recvfrom_into, False, []),
-                ]
-                data_prefix = u"PREFIX_"
-
-                for meth_name, send_meth, expect_success, args in send_methods:
-                    indata = data_prefix + meth_name
-                    try:
-                        send_meth(indata.encode('ASCII', 'strict'), *args)
-                        outdata = s.read()
-                        outdata = outdata.decode('ASCII', 'strict')
-                        if outdata != indata.lower():
-                            self.fail(
-                                "While sending with <<%s>> bad data "
-                                "<<%r>> (%d) received; "
-                                "expected <<%r>> (%d)\n" % (
-                                    meth_name, outdata[:20], len(outdata),
-                                    indata[:20], len(indata)
-                                )
-                            )
-                    except ValueError as e:
-                        if expect_success:
-                            self.fail(
-                                "Failed to send with method <<%s>>; "
-                                "expected to succeed.\n" % (meth_name,)
-                            )
-                        if not str(e).startswith(meth_name):
-                            self.fail(
-                                "Method <<%s>> failed with unexpected "
-                                "exception message: %s\n" % (
-                                    meth_name, e
-                                )
-                            )
-
-                for meth_name, recv_meth, expect_success, args in recv_methods:
-                    indata = data_prefix + meth_name
-                    try:
-                        s.send(indata.encode('ASCII', 'strict'))
-                        outdata = recv_meth(*args)
-                        outdata = outdata.decode('ASCII', 'strict')
-                        if outdata != indata.lower():
-                            self.fail(
-                                "While receiving with <<%s>> bad data "
-                                "<<%r>> (%d) received; "
-                                "expected <<%r>> (%d)\n" % (
-                                    meth_name, outdata[:20], len(outdata),
-                                    indata[:20], len(indata)
-                                )
-                            )
-                    except ValueError as e:
-                        if expect_success:
-                            self.fail(
-                                "Failed to receive with method <<%s>>; "
-                                "expected to succeed.\n" % (meth_name,)
-                            )
-                        if not str(e).startswith(meth_name):
-                            self.fail(
-                                "Method <<%s>> failed with unexpected "
-                                "exception message: %s\n" % (
-                                    meth_name, e
-                                )
-                            )
-                        # consume data
-                        s.read()
-
-                s.write("over\n".encode("ASCII", "strict"))
-                s.close()
-            finally:
-                server.stop()
-                server.join()
-
-        def test_handshake_timeout(self):
-            # Issue #5103: SSL handshake must respect the socket timeout
-            server = socket.socket(socket.AF_INET)
-            host = "127.0.0.1"
-            port = test_support.bind_port(server)
-            started = threading.Event()
-            finish = False
-
-            def serve():
-                server.listen(5)
-                started.set()
-                conns = []
-                while not finish:
-                    r, w, e = select.select([server], [], [], 0.1)
-                    if server in r:
-                        # Let the socket hang around rather than having
-                        # it closed by garbage collection.
-                        conns.append(server.accept()[0])
-
-            t = threading.Thread(target=serve)
-            t.start()
-            started.wait()
-
-            try:
-                try:
-                    c = socket.socket(socket.AF_INET)
-                    c.settimeout(0.2)
-                    c.connect((host, port))
-                    # Will attempt handshake and time out
-                    self.assertRaisesRegexp(ssl.SSLError, "timed out",
-                                            ssl.wrap_socket, c)
-                finally:
-                    c.close()
-                try:
-                    c = socket.socket(socket.AF_INET)
-                    c.settimeout(0.2)
-                    c = ssl.wrap_socket(c)
-                    # Will attempt handshake and time out
-                    self.assertRaisesRegexp(ssl.SSLError, "timed out",
-                                            c.connect, (host, port))
-                finally:
-                    c.close()
-            finally:
-                finish = True
-                t.join()
-                server.close()
-
-
-def test_main(verbose=False):
-    if skip_expected:
-        raise unittest.SkipTest("No SSL support")
-
-    global CERTFILE, SVN_PYTHON_ORG_ROOT_CERT
-    CERTFILE = test_support.findfile("keycert.pem")
-    SVN_PYTHON_ORG_ROOT_CERT =  test_support.findfile(
-        "https_svn_python_org_root.pem")
-
-    if (not os.path.exists(CERTFILE) or
-        not os.path.exists(SVN_PYTHON_ORG_ROOT_CERT)):
-        raise test_support.TestFailed("Can't read certificate files!")
-
-    tests = [BasicTests, BasicSocketTests]
-
-    if test_support.is_resource_enabled('network'):
-        tests.append(NetworkedTests)
-
-    if _have_threads:
-        thread_info = test_support.threading_setup()
-        if thread_info and test_support.is_resource_enabled('network'):
-            tests.append(ThreadedTests)
-
-    try:
-        test_support.run_unittest(*tests)
-    finally:
-        if _have_threads:
-            test_support.threading_cleanup(*thread_info)
-
-if __name__ == "__main__":
-    test_main()

diff --git a/pypy/jit/tl/pypyjit_demo.py b/pypy/jit/tl/pypyjit_demo.py
--- a/pypy/jit/tl/pypyjit_demo.py
+++ b/pypy/jit/tl/pypyjit_demo.py
@@ -1,16 +1,19 @@
 
 try:
-    class C:
-        def __call__(self):
-            pass
+    import pypyjit
+    pypyjit.set_param(threshold=3, inlining=True)
 
-    def f():
-        i = 0
-        c = C()
-        while i < 10:
-            c()
-            i += 1
-    f()
+    def sqrt(y, n=10000):
+        x = y / 2
+        while n > 0:
+            #assert y > 0 and x > 0
+            if y > 0 and x > 0: pass
+            n -= 1
+            x = (x + y/x) / 2
+        return x
+
+    print sqrt(1234, 4)
+    
 except Exception, e:
     print "Exception: ", type(e)
     print e

From commits-noreply at bitbucket.org  Tue Feb  1 13:26:31 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Tue,  1 Feb 2011 13:26:31 +0100 (CET)
Subject: [pypy-svn] pypy default: Backed out changeset 0872e654e4d7
Message-ID: <20110201122631.4D8AE282BD8@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: 
Changeset: r41519:177524b5d89d
Date: 2011-02-01 14:26 +0200
http://bitbucket.org/pypy/pypy/changeset/177524b5d89d/

Log:	Backed out changeset 0872e654e4d7

diff --git a/pypy/interpreter/gateway.py b/pypy/interpreter/gateway.py
--- a/pypy/interpreter/gateway.py
+++ b/pypy/interpreter/gateway.py
@@ -344,8 +344,9 @@
                                                        self.nextarg()))
 
     def visit__ObjSpace(self, el):
-        if self.finger != 0:
+        if self.finger not in [0, 1]:
             raise FastFuncNotSupported
+        self.args.append('space')
         self.unwrap.append("space")
 
     def visit__W_Root(self, el):
@@ -391,7 +392,9 @@
         unwrap_info = UnwrapSpec_FastFunc_Unwrap()
         unwrap_info.apply_over(unwrap_spec)
         narg = unwrap_info.n
-        args = ['space'] + unwrap_info.args
+        args = unwrap_info.args
+        if 'space' not in args:
+            args = ['space'] + args
         if args == unwrap_info.unwrap:
             fastfunc = func
         else:
@@ -528,8 +531,12 @@
                     self.__class__ = BuiltinCodePassThroughArguments1
                     self.func__args__ = func
             else:
-                self.__class__ = globals()['BuiltinCode%d' % arity]
-                setattr(self, 'fastfunc_%d' % arity, fastfunc)
+                if len(unwrap_spec) >= 2 and unwrap_spec[1] is ObjSpace:
+                    self.__class__ = globals()['BuiltinCodeSpace%d' % arity]
+                    setattr(self, 'fastfunc_space_%d' % arity, fastfunc)
+                else:
+                    self.__class__ = globals()['BuiltinCode%d' % arity]
+                    setattr(self, 'fastfunc_%d' % arity, fastfunc)
 
     def descr__reduce__(self, space):
         from pypy.interpreter.mixedmodule import MixedModule
@@ -717,6 +724,78 @@
             w_result = space.w_None
         return w_result
 
+class BuiltinCodeSpace1(BuiltinCode1):
+    _immutable_ = True
+    fast_natural_arity = 1
+    
+    def fastcall_1(self, space, w_func, w1):
+        try:
+            w_result = self.fastfunc_space_1(w1, space)
+        except DescrMismatch:
+            return  w1.descr_call_mismatch(space,
+                                           self.descrmismatch_op,
+                                           self.descr_reqcls,
+                                           Arguments(space, [w1]))
+        except Exception, e:
+            raise self.handle_exception(space, e)
+        if w_result is None:
+            w_result = space.w_None
+        return w_result
+
+class BuiltinCodeSpace2(BuiltinCode2):
+    _immutable_ = True
+    fast_natural_arity = 2
+    
+    def fastcall_2(self, space, w_func, w1, w2):
+        try:
+            w_result = self.fastfunc_space_2(w1, space, w2)
+        except DescrMismatch:
+            return  w1.descr_call_mismatch(space,
+                                           self.descrmismatch_op,
+                                           self.descr_reqcls,
+                                           Arguments(space, [w1, w2]))
+        except Exception, e:
+            raise self.handle_exception(space, e)
+        if w_result is None:
+            w_result = space.w_None
+        return w_result
+
+class BuiltinCodeSpace3(BuiltinCode3):
+    _immutable_ = True
+    fast_natural_arity = 3
+    
+    def fastcall_3(self, space, func, w1, w2, w3):
+        try:
+            w_result = self.fastfunc_space_3(w1, space, w2, w3)
+        except DescrMismatch:
+            return  w1.descr_call_mismatch(space,
+                                           self.descrmismatch_op,
+                                           self.descr_reqcls,
+                                           Arguments(space, [w1, w2, w3]))
+        except Exception, e:
+            raise self.handle_exception(space, e)
+        if w_result is None:
+            w_result = space.w_None
+        return w_result
+
+class BuiltinCodeSpace4(BuiltinCode4):
+    _immutable_ = True
+    fast_natural_arity = 4
+    
+    def fastcall_4(self, space, func, w1, w2, w3, w4):
+        try:
+            w_result = self.fastfunc_space_4(w1, space, w2, w3, w4)
+        except DescrMismatch:
+            return  w1.descr_call_mismatch(space,
+                                           self.descrmismatch_op,
+                                           self.descr_reqcls,
+                                           Arguments(space,
+                                                     [w1, w2, w3, w4]))
+        except Exception, e:
+            raise self.handle_exception(space, e)
+        if w_result is None:
+            w_result = space.w_None
+        return w_result
 
 class interp2app(Wrappable):
     """Build a gateway that calls 'f' at interp-level."""

diff --git a/pypy/interpreter/test/test_gateway.py b/pypy/interpreter/test/test_gateway.py
--- a/pypy/interpreter/test/test_gateway.py
+++ b/pypy/interpreter/test/test_gateway.py
@@ -14,7 +14,7 @@
         self.name = name
         self.defs_w = []
 
-class TestBuiltinCode: 
+class TestBuiltinCode(object): 
     def test_signature(self):
         def c(space, w_x, w_y, hello_w):
             pass
@@ -545,7 +545,42 @@
         """)
 
         assert space.is_true(w_res)
-        assert called == [w_app_f, w_app_f]       
+        assert called == [w_app_f, w_app_f]
+
+    def test_interp2app_fastcall_method_space(self):
+        space = self.space
+        w = space.wrap
+        w_3 = w(3)
+
+        def f(w_self, space, w_x):
+            return w_x
+        app_f = gateway.interp2app_temp(f, unwrap_spec=[gateway.W_Root,
+                                                        gateway.ObjSpace,
+                                                        gateway.W_Root])
+        w_app_f = w(app_f)
+
+        # sanity
+        assert isinstance(w_app_f.code, gateway.BuiltinCode2)
+
+        called = []
+        fastcall_2 = w_app_f.code.fastcall_2
+        def witness_fastcall_2(space, w_func, w_a, w_b):
+            called.append(w_func)
+            return fastcall_2(space, w_func, w_a, w_b)
+
+        w_app_f.code.fastcall_2 = witness_fastcall_2    
+    
+        w_res = space.appexec([w_app_f, w_3], """(f, x):
+        class A(object):
+           m = f # not a builtin function, so works as method
+        y = A().m(x)
+        b = A().m
+        z = b(x)
+        return y is x and z is x
+        """)
+
+        assert space.is_true(w_res)
+        assert called == [w_app_f, w_app_f]        
         
     def test_plain(self):
         space = self.space

diff --git a/lib-python/modified-2.7.0/ssl.py b/lib-python/modified-2.7.0/ssl.py
new file mode 100644
--- /dev/null
+++ b/lib-python/modified-2.7.0/ssl.py
@@ -0,0 +1,437 @@
+# Wrapper module for _ssl, providing some additional facilities
+# implemented in Python.  Written by Bill Janssen.
+
+"""\
+This module provides some more Pythonic support for SSL.
+
+Object types:
+
+  SSLSocket -- subtype of socket.socket which does SSL over the socket
+
+Exceptions:
+
+  SSLError -- exception raised for I/O errors
+
+Functions:
+
+  cert_time_to_seconds -- convert time string used for certificate
+                          notBefore and notAfter functions to integer
+                          seconds past the Epoch (the time values
+                          returned from time.time())
+
+  fetch_server_certificate (HOST, PORT) -- fetch the certificate provided
+                          by the server running on HOST at port PORT.  No
+                          validation of the certificate is performed.
+
+Integer constants:
+
+SSL_ERROR_ZERO_RETURN
+SSL_ERROR_WANT_READ
+SSL_ERROR_WANT_WRITE
+SSL_ERROR_WANT_X509_LOOKUP
+SSL_ERROR_SYSCALL
+SSL_ERROR_SSL
+SSL_ERROR_WANT_CONNECT
+
+SSL_ERROR_EOF
+SSL_ERROR_INVALID_ERROR_CODE
+
+The following group define certificate requirements that one side is
+allowing/requiring from the other side:
+
+CERT_NONE - no certificates from the other side are required (or will
+            be looked at if provided)
+CERT_OPTIONAL - certificates are not required, but if provided will be
+                validated, and if validation fails, the connection will
+                also fail
+CERT_REQUIRED - certificates are required, and will be validated, and
+                if validation fails, the connection will also fail
+
+The following constants identify various SSL protocol variants:
+
+PROTOCOL_SSLv2
+PROTOCOL_SSLv3
+PROTOCOL_SSLv23
+PROTOCOL_TLSv1
+"""
+
+import textwrap
+
+import _ssl             # if we can't import it, let the error propagate
+
+from _ssl import OPENSSL_VERSION_NUMBER, OPENSSL_VERSION_INFO, OPENSSL_VERSION
+from _ssl import SSLError
+from _ssl import CERT_NONE, CERT_OPTIONAL, CERT_REQUIRED
+from _ssl import PROTOCOL_SSLv2, PROTOCOL_SSLv3, PROTOCOL_SSLv23, PROTOCOL_TLSv1
+from _ssl import RAND_status, RAND_egd, RAND_add
+from _ssl import \
+     SSL_ERROR_ZERO_RETURN, \
+     SSL_ERROR_WANT_READ, \
+     SSL_ERROR_WANT_WRITE, \
+     SSL_ERROR_WANT_X509_LOOKUP, \
+     SSL_ERROR_SYSCALL, \
+     SSL_ERROR_SSL, \
+     SSL_ERROR_WANT_CONNECT, \
+     SSL_ERROR_EOF, \
+     SSL_ERROR_INVALID_ERROR_CODE
+
+from socket import socket, _fileobject, error as socket_error
+from socket import getnameinfo as _getnameinfo
+import base64        # for DER-to-PEM translation
+import errno
+
+class SSLSocket(socket):
+
+    """This class implements a subtype of socket.socket that wraps
+    the underlying OS socket in an SSL context when necessary, and
+    provides read and write methods over that channel."""
+
+    def __init__(self, sock, keyfile=None, certfile=None,
+                 server_side=False, cert_reqs=CERT_NONE,
+                 ssl_version=PROTOCOL_SSLv23, ca_certs=None,
+                 do_handshake_on_connect=True,
+                 suppress_ragged_eofs=True, ciphers=None):
+        socket.__init__(self, _sock=sock._sock)
+
+        if certfile and not keyfile:
+            keyfile = certfile
+        # see if it's connected
+        try:
+            socket.getpeername(self)
+        except socket_error, e:
+            if e.errno != errno.ENOTCONN:
+                raise
+            # no, no connection yet
+            self._sslobj = None
+        else:
+            # yes, create the SSL object
+            self._sslobj = _ssl.sslwrap(self._sock, server_side,
+                                        keyfile, certfile,
+                                        cert_reqs, ssl_version, ca_certs,
+                                        ciphers)
+            if do_handshake_on_connect:
+                self.do_handshake()
+        self.keyfile = keyfile
+        self.certfile = certfile
+        self.cert_reqs = cert_reqs
+        self.ssl_version = ssl_version
+        self.ca_certs = ca_certs
+        self.ciphers = ciphers
+        self.do_handshake_on_connect = do_handshake_on_connect
+        self.suppress_ragged_eofs = suppress_ragged_eofs
+        self._makefile_refs = 0
+
+    def read(self, len=1024):
+
+        """Read up to LEN bytes and return them.
+        Return zero-length string on EOF."""
+
+        try:
+            return self._sslobj.read(len)
+        except SSLError, x:
+            if x.args[0] == SSL_ERROR_EOF and self.suppress_ragged_eofs:
+                return ''
+            else:
+                raise
+
+    def write(self, data):
+
+        """Write DATA to the underlying SSL channel.  Returns
+        number of bytes of DATA actually transmitted."""
+
+        return self._sslobj.write(data)
+
+    def getpeercert(self, binary_form=False):
+
+        """Returns a formatted version of the data in the
+        certificate provided by the other end of the SSL channel.
+        Return None if no certificate was provided, {} if a
+        certificate was provided, but not validated."""
+
+        return self._sslobj.peer_certificate(binary_form)
+
+    def cipher(self):
+
+        if not self._sslobj:
+            return None
+        else:
+            return self._sslobj.cipher()
+
+    def send(self, data, flags=0):
+        if self._sslobj:
+            if flags != 0:
+                raise ValueError(
+                    "non-zero flags not allowed in calls to send() on %s" %
+                    self.__class__)
+            while True:
+                try:
+                    v = self._sslobj.write(data)
+                except SSLError, x:
+                    if x.args[0] == SSL_ERROR_WANT_READ:
+                        return 0
+                    elif x.args[0] == SSL_ERROR_WANT_WRITE:
+                        return 0
+                    else:
+                        raise
+                else:
+                    return v
+        else:
+            return self._sock.send(data, flags)
+
+    def sendto(self, data, flags_or_addr, addr=None):
+        if self._sslobj:
+            raise ValueError("sendto not allowed on instances of %s" %
+                             self.__class__)
+        elif addr is None:
+            return self._sock.sendto(data, flags_or_addr)
+        else:
+            return self._sock.sendto(data, flags_or_addr, addr)
+
+    def sendall(self, data, flags=0):
+        if self._sslobj:
+            if flags != 0:
+                raise ValueError(
+                    "non-zero flags not allowed in calls to sendall() on %s" %
+                    self.__class__)
+            amount = len(data)
+            count = 0
+            while (count < amount):
+                v = self.send(data[count:])
+                count += v
+            return amount
+        else:
+            return socket.sendall(self, data, flags)
+
+    def recv(self, buflen=1024, flags=0):
+        if self._sslobj:
+            if flags != 0:
+                raise ValueError(
+                    "non-zero flags not allowed in calls to recv() on %s" %
+                    self.__class__)
+            return self.read(buflen)
+        else:
+            return self._sock.recv(buflen, flags)
+
+    def recv_into(self, buffer, nbytes=None, flags=0):
+        if buffer and (nbytes is None):
+            nbytes = len(buffer)
+        elif nbytes is None:
+            nbytes = 1024
+        if self._sslobj:
+            if flags != 0:
+                raise ValueError(
+                  "non-zero flags not allowed in calls to recv_into() on %s" %
+                  self.__class__)
+            tmp_buffer = self.read(nbytes)
+            v = len(tmp_buffer)
+            buffer[:v] = tmp_buffer
+            return v
+        else:
+            return self._sock.recv_into(buffer, nbytes, flags)
+
+    def recvfrom(self, buflen=1024, flags=0):
+        if self._sslobj:
+            raise ValueError("recvfrom not allowed on instances of %s" %
+                             self.__class__)
+        else:
+            return self._sock.recvfrom(buflen, flags)
+
+    def recvfrom_into(self, buffer, nbytes=None, flags=0):
+        if self._sslobj:
+            raise ValueError("recvfrom_into not allowed on instances of %s" %
+                             self.__class__)
+        else:
+            return self._sock.recvfrom_into(buffer, nbytes, flags)
+
+    def pending(self):
+        if self._sslobj:
+            return self._sslobj.pending()
+        else:
+            return 0
+
+    def unwrap(self):
+        if self._sslobj:
+            s = self._sslobj.shutdown()
+            self._sslobj = None
+            return s
+        else:
+            raise ValueError("No SSL wrapper around " + str(self))
+
+    def shutdown(self, how):
+        self._sslobj = None
+        socket.shutdown(self, how)
+
+    def close(self):
+        if self._makefile_refs < 1:
+            self._sslobj = None
+            socket.close(self)
+        else:
+            self._makefile_refs -= 1
+
+    def do_handshake(self):
+
+        """Perform a TLS/SSL handshake."""
+
+        self._sslobj.do_handshake()
+
+    def connect(self, addr):
+
+        """Connects to remote ADDR, and then wraps the connection in
+        an SSL channel."""
+
+        # Here we assume that the socket is client-side, and not
+        # connected at the time of the call.  We connect it, then wrap it.
+        if self._sslobj:
+            raise ValueError("attempt to connect already-connected SSLSocket!")
+        socket.connect(self, addr)
+        self._sslobj = _ssl.sslwrap(self._sock, False, self.keyfile, self.certfile,
+                                    self.cert_reqs, self.ssl_version,
+                                    self.ca_certs, self.ciphers)
+        if self.do_handshake_on_connect:
+            self.do_handshake()
+
+    def accept(self):
+
+        """Accepts a new connection from a remote client, and returns
+        a tuple containing that new connection wrapped with a server-side
+        SSL channel, and the address of the remote client."""
+
+        newsock, addr = socket.accept(self)
+        return (SSLSocket(newsock,
+                          keyfile=self.keyfile,
+                          certfile=self.certfile,
+                          server_side=True,
+                          cert_reqs=self.cert_reqs,
+                          ssl_version=self.ssl_version,
+                          ca_certs=self.ca_certs,
+                          ciphers=self.ciphers,
+                          do_handshake_on_connect=self.do_handshake_on_connect,
+                          suppress_ragged_eofs=self.suppress_ragged_eofs),
+                addr)
+
+    def makefile(self, mode='r', bufsize=-1):
+
+        """Make and return a file-like object that
+        works with the SSL connection.  Just use the code
+        from the socket module."""
+
+        self._makefile_refs += 1
+        # close=True so as to decrement the reference count when done with
+        # the file-like object.
+        return _fileobject(self, mode, bufsize, close=True)
+
+
+
+def wrap_socket(sock, keyfile=None, certfile=None,
+                server_side=False, cert_reqs=CERT_NONE,
+                ssl_version=PROTOCOL_SSLv23, ca_certs=None,
+                do_handshake_on_connect=True,
+                suppress_ragged_eofs=True, ciphers=None):
+
+    return SSLSocket(sock, keyfile=keyfile, certfile=certfile,
+                     server_side=server_side, cert_reqs=cert_reqs,
+                     ssl_version=ssl_version, ca_certs=ca_certs,
+                     do_handshake_on_connect=do_handshake_on_connect,
+                     suppress_ragged_eofs=suppress_ragged_eofs,
+                     ciphers=ciphers)
+
+
+# some utility functions
+
+def cert_time_to_seconds(cert_time):
+
+    """Takes a date-time string in standard ASN1_print form
+    ("MON DAY 24HOUR:MINUTE:SEC YEAR TIMEZONE") and return
+    a Python time value in seconds past the epoch."""
+
+    import time
+    return time.mktime(time.strptime(cert_time, "%b %d %H:%M:%S %Y GMT"))
+
+PEM_HEADER = "-----BEGIN CERTIFICATE-----"
+PEM_FOOTER = "-----END CERTIFICATE-----"
+
+def DER_cert_to_PEM_cert(der_cert_bytes):
+
+    """Takes a certificate in binary DER format and returns the
+    PEM version of it as a string."""
+
+    if hasattr(base64, 'standard_b64encode'):
+        # preferred because older API gets line-length wrong
+        f = base64.standard_b64encode(der_cert_bytes)
+        return (PEM_HEADER + '\n' +
+                textwrap.fill(f, 64) + '\n' +
+                PEM_FOOTER + '\n')
+    else:
+        return (PEM_HEADER + '\n' +
+                base64.encodestring(der_cert_bytes) +
+                PEM_FOOTER + '\n')
+
+def PEM_cert_to_DER_cert(pem_cert_string):
+
+    """Takes a certificate in ASCII PEM format and returns the
+    DER-encoded version of it as a byte sequence"""
+
+    if not pem_cert_string.startswith(PEM_HEADER):
+        raise ValueError("Invalid PEM encoding; must start with %s"
+                         % PEM_HEADER)
+    if not pem_cert_string.strip().endswith(PEM_FOOTER):
+        raise ValueError("Invalid PEM encoding; must end with %s"
+                         % PEM_FOOTER)
+    d = pem_cert_string.strip()[len(PEM_HEADER):-len(PEM_FOOTER)]
+    return base64.decodestring(d)
+
+def get_server_certificate(addr, ssl_version=PROTOCOL_SSLv3, ca_certs=None):
+
+    """Retrieve the certificate from the server at the specified address,
+    and return it as a PEM-encoded string.
+    If 'ca_certs' is specified, validate the server cert against it.
+    If 'ssl_version' is specified, use it in the connection attempt."""
+
+    host, port = addr
+    if (ca_certs is not None):
+        cert_reqs = CERT_REQUIRED
+    else:
+        cert_reqs = CERT_NONE
+    s = wrap_socket(socket(), ssl_version=ssl_version,
+                    cert_reqs=cert_reqs, ca_certs=ca_certs)
+    s.connect(addr)
+    dercert = s.getpeercert(True)
+    s.close()
+    return DER_cert_to_PEM_cert(dercert)
+
+def get_protocol_name(protocol_code):
+    if protocol_code == PROTOCOL_TLSv1:
+        return "TLSv1"
+    elif protocol_code == PROTOCOL_SSLv23:
+        return "SSLv23"
+    elif protocol_code == PROTOCOL_SSLv2:
+        return "SSLv2"
+    elif protocol_code == PROTOCOL_SSLv3:
+        return "SSLv3"
+    else:
+        return "<unknown>"
+
+
+# a replacement for the old socket.ssl function
+
+def sslwrap_simple(sock, keyfile=None, certfile=None):
+
+    """A replacement for the old socket.ssl function.  Designed
+    for compability with Python 2.5 and earlier.  Will disappear in
+    Python 3.0."""
+
+    if hasattr(sock, "_sock"):
+        sock = sock._sock
+
+    ssl_sock = _ssl.sslwrap(sock, 0, keyfile, certfile, CERT_NONE,
+                            PROTOCOL_SSLv23, None)
+    try:
+        sock.getpeername()
+    except socket_error:
+        # no, no connection yet
+        pass
+    else:
+        # yes, do the handshake
+        ssl_sock.do_handshake()
+
+    return ssl_sock

diff --git a/lib-python/modified-2.7.0/test/test_ssl.py b/lib-python/modified-2.7.0/test/test_ssl.py
new file mode 100644
--- /dev/null
+++ b/lib-python/modified-2.7.0/test/test_ssl.py
@@ -0,0 +1,1318 @@
+# Test the support for SSL and sockets
+
+import sys
+import unittest
+from test import test_support
+import asyncore
+import socket
+import select
+import time
+import gc
+import os
+import errno
+import pprint
+import urllib, urlparse
+import traceback
+import weakref
+import functools
+import platform
+
+from BaseHTTPServer import HTTPServer
+from SimpleHTTPServer import SimpleHTTPRequestHandler
+
+# Optionally test SSL support, if we have it in the tested platform
+skip_expected = False
+try:
+    import ssl
+except ImportError:
+    skip_expected = True
+
+HOST = test_support.HOST
+CERTFILE = None
+SVN_PYTHON_ORG_ROOT_CERT = None
+
+def handle_error(prefix):
+    exc_format = ' '.join(traceback.format_exception(*sys.exc_info()))
+    if test_support.verbose:
+        sys.stdout.write(prefix + exc_format)
+
+
+class BasicTests(unittest.TestCase):
+
+    def test_sslwrap_simple(self):
+        # A crude test for the legacy API
+        try:
+            ssl.sslwrap_simple(socket.socket(socket.AF_INET))
+        except IOError, e:
+            if e.errno == 32: # broken pipe when ssl_sock.do_handshake(), this test doesn't care about that
+                pass
+            else:
+                raise
+        try:
+            ssl.sslwrap_simple(socket.socket(socket.AF_INET)._sock)
+        except IOError, e:
+            if e.errno == 32: # broken pipe when ssl_sock.do_handshake(), this test doesn't care about that
+                pass
+            else:
+                raise
+
+# Issue #9415: Ubuntu hijacks their OpenSSL and forcefully disables SSLv2
+def skip_if_broken_ubuntu_ssl(func):
+    # We need to access the lower-level wrapper in order to create an
+    # implicit SSL context without trying to connect or listen.
+    try:
+        import _ssl
+    except ImportError:
+        # The returned function won't get executed, just ignore the error
+        pass
+    @functools.wraps(func)
+    def f(*args, **kwargs):
+        try:
+            s = socket.socket(socket.AF_INET)
+            _ssl.sslwrap(s._sock, 0, None, None,
+                         ssl.CERT_NONE, ssl.PROTOCOL_SSLv2, None, None)
+        except ssl.SSLError as e:
+            if (ssl.OPENSSL_VERSION_INFO == (0, 9, 8, 15, 15) and
+                platform.linux_distribution() == ('debian', 'squeeze/sid', '')
+                and 'Invalid SSL protocol variant specified' in str(e)):
+                raise unittest.SkipTest("Patched Ubuntu OpenSSL breaks behaviour")
+        return func(*args, **kwargs)
+    return f
+
+
+class BasicSocketTests(unittest.TestCase):
+
+    def test_constants(self):
+        ssl.PROTOCOL_SSLv2
+        ssl.PROTOCOL_SSLv23
+        ssl.PROTOCOL_SSLv3
+        ssl.PROTOCOL_TLSv1
+        ssl.CERT_NONE
+        ssl.CERT_OPTIONAL
+        ssl.CERT_REQUIRED
+
+    def test_random(self):
+        v = ssl.RAND_status()
+        if test_support.verbose:
+            sys.stdout.write("\n RAND_status is %d (%s)\n"
+                             % (v, (v and "sufficient randomness") or
+                                "insufficient randomness"))
+        try:
+            ssl.RAND_egd(1)
+        except TypeError:
+            pass
+        else:
+            print "didn't raise TypeError"
+        ssl.RAND_add("this is a random string", 75.0)
+
+    @test_support.impl_detail("obscure test")
+    def test_parse_cert(self):
+        # note that this uses an 'unofficial' function in _ssl.c,
+        # provided solely for this test, to exercise the certificate
+        # parsing code
+        p = ssl._ssl._test_decode_cert(CERTFILE, False)
+        if test_support.verbose:
+            sys.stdout.write("\n" + pprint.pformat(p) + "\n")
+
+    def test_DER_to_PEM(self):
+        with open(SVN_PYTHON_ORG_ROOT_CERT, 'r') as f:
+            pem = f.read()
+        d1 = ssl.PEM_cert_to_DER_cert(pem)
+        p2 = ssl.DER_cert_to_PEM_cert(d1)
+        d2 = ssl.PEM_cert_to_DER_cert(p2)
+        self.assertEqual(d1, d2)
+        if not p2.startswith(ssl.PEM_HEADER + '\n'):
+            self.fail("DER-to-PEM didn't include correct header:\n%r\n" % p2)
+        if not p2.endswith('\n' + ssl.PEM_FOOTER + '\n'):
+            self.fail("DER-to-PEM didn't include correct footer:\n%r\n" % p2)
+
+    def test_openssl_version(self):
+        n = ssl.OPENSSL_VERSION_NUMBER
+        t = ssl.OPENSSL_VERSION_INFO
+        s = ssl.OPENSSL_VERSION
+        self.assertIsInstance(n, (int, long))
+        self.assertIsInstance(t, tuple)
+        self.assertIsInstance(s, str)
+        # Some sanity checks follow
+        # >= 0.9
+        self.assertGreaterEqual(n, 0x900000)
+        # < 2.0
+        self.assertLess(n, 0x20000000)
+        major, minor, fix, patch, status = t
+        self.assertGreaterEqual(major, 0)
+        self.assertLess(major, 2)
+        self.assertGreaterEqual(minor, 0)
+        self.assertLess(minor, 256)
+        self.assertGreaterEqual(fix, 0)
+        self.assertLess(fix, 256)
+        self.assertGreaterEqual(patch, 0)
+        self.assertLessEqual(patch, 26)
+        self.assertGreaterEqual(status, 0)
+        self.assertLessEqual(status, 15)
+        # Version string as returned by OpenSSL, the format might change
+        self.assertTrue(s.startswith("OpenSSL {:d}.{:d}.{:d}".format(major, minor, fix)),
+                        (s, t))
+
+    def test_ciphers(self):
+        if not test_support.is_resource_enabled('network'):
+            return
+        remote = ("svn.python.org", 443)
+        with test_support.transient_internet(remote[0]):
+            s = ssl.wrap_socket(socket.socket(socket.AF_INET),
+                                cert_reqs=ssl.CERT_NONE, ciphers="ALL")
+            s.connect(remote)
+            s = ssl.wrap_socket(socket.socket(socket.AF_INET),
+                                cert_reqs=ssl.CERT_NONE, ciphers="DEFAULT")
+            s.connect(remote)
+            # Error checking occurs when connecting, because the SSL context
+            # isn't created before.
+            s = ssl.wrap_socket(socket.socket(socket.AF_INET),
+                                cert_reqs=ssl.CERT_NONE, ciphers="^$:,;?*'dorothyx")
+            with self.assertRaisesRegexp(ssl.SSLError, "No cipher can be selected"):
+                s.connect(remote)
+
+    @test_support.cpython_only
+    def test_refcycle(self):
+        # Issue #7943: an SSL object doesn't create reference cycles with
+        # itself.
+        s = socket.socket(socket.AF_INET)
+        ss = ssl.wrap_socket(s)
+        wr = weakref.ref(ss)
+        del ss
+        self.assertEqual(wr(), None)
+
+    def test_wrapped_unconnected(self):
+        # The _delegate_methods in socket.py are correctly delegated to by an
+        # unconnected SSLSocket, so they will raise a socket.error rather than
+        # something unexpected like TypeError.
+        s = socket.socket(socket.AF_INET)
+        ss = ssl.wrap_socket(s)
+        self.assertRaises(socket.error, ss.recv, 1)
+        self.assertRaises(socket.error, ss.recv_into, bytearray(b'x'))
+        self.assertRaises(socket.error, ss.recvfrom, 1)
+        self.assertRaises(socket.error, ss.recvfrom_into, bytearray(b'x'), 1)
+        self.assertRaises(socket.error, ss.send, b'x')
+        self.assertRaises(socket.error, ss.sendto, b'x', ('0.0.0.0', 0))
+
+
+class NetworkedTests(unittest.TestCase):
+
+    def test_connect(self):
+        with test_support.transient_internet("svn.python.org"):
+            s = ssl.wrap_socket(socket.socket(socket.AF_INET),
+                                cert_reqs=ssl.CERT_NONE)
+            s.connect(("svn.python.org", 443))
+            c = s.getpeercert()
+            if c:
+                self.fail("Peer cert %s shouldn't be here!")
+            s.close()
+
+            # this should fail because we have no verification certs
+            s = ssl.wrap_socket(socket.socket(socket.AF_INET),
+                                cert_reqs=ssl.CERT_REQUIRED)
+            try:
+                s.connect(("svn.python.org", 443))
+            except ssl.SSLError:
+                pass
+            finally:
+                s.close()
+
+            # this should succeed because we specify the root cert
+            s = ssl.wrap_socket(socket.socket(socket.AF_INET),
+                                cert_reqs=ssl.CERT_REQUIRED,
+                                ca_certs=SVN_PYTHON_ORG_ROOT_CERT)
+            try:
+                s.connect(("svn.python.org", 443))
+            finally:
+                s.close()
+
+    @unittest.skipIf(os.name == "nt", "Can't use a socket as a file under Windows")
+    def test_makefile_close(self):
+        # Issue #5238: creating a file-like object with makefile() shouldn't
+        # delay closing the underlying "real socket" (here tested with its
+        # file descriptor, hence skipping the test under Windows).
+        with test_support.transient_internet("svn.python.org"):
+            ss = ssl.wrap_socket(socket.socket(socket.AF_INET))
+            ss.connect(("svn.python.org", 443))
+            fd = ss.fileno()
+            f = ss.makefile()
+            f.close()
+            # The fd is still open
+            os.read(fd, 0)
+            # Closing the SSL socket should close the fd too
+            ss.close()
+            gc.collect()
+            with self.assertRaises(OSError) as e:
+                os.read(fd, 0)
+            self.assertEqual(e.exception.errno, errno.EBADF)
+
+    def test_non_blocking_handshake(self):
+        with test_support.transient_internet("svn.python.org"):
+            s = socket.socket(socket.AF_INET)
+            s.connect(("svn.python.org", 443))
+            s.setblocking(False)
+            s = ssl.wrap_socket(s,
+                                cert_reqs=ssl.CERT_NONE,
+                                do_handshake_on_connect=False)
+            count = 0
+            while True:
+                try:
+                    count += 1
+                    s.do_handshake()
+                    break
+                except ssl.SSLError, err:
+                    if err.args[0] == ssl.SSL_ERROR_WANT_READ:
+                        select.select([s], [], [])
+                    elif err.args[0] == ssl.SSL_ERROR_WANT_WRITE:
+                        select.select([], [s], [])
+                    else:
+                        raise
+            s.close()
+            if test_support.verbose:
+                sys.stdout.write("\nNeeded %d calls to do_handshake() to establish session.\n" % count)
+
+    def test_get_server_certificate(self):
+        with test_support.transient_internet("svn.python.org"):
+            pem = ssl.get_server_certificate(("svn.python.org", 443))
+            if not pem:
+                self.fail("No server certificate on svn.python.org:443!")
+
+            try:
+                pem = ssl.get_server_certificate(("svn.python.org", 443), ca_certs=CERTFILE)
+            except ssl.SSLError:
+                #should fail
+                pass
+            else:
+                self.fail("Got server certificate %s for svn.python.org!" % pem)
+
+            pem = ssl.get_server_certificate(("svn.python.org", 443), ca_certs=SVN_PYTHON_ORG_ROOT_CERT)
+            if not pem:
+                self.fail("No server certificate on svn.python.org:443!")
+            if test_support.verbose:
+                sys.stdout.write("\nVerified certificate for svn.python.org:443 is\n%s\n" % pem)
+
+    def test_algorithms(self):
+        # Issue #8484: all algorithms should be available when verifying a
+        # certificate.
+        # SHA256 was added in OpenSSL 0.9.8
+        if ssl.OPENSSL_VERSION_INFO < (0, 9, 8, 0, 15):
+            self.skipTest("SHA256 not available on %r" % ssl.OPENSSL_VERSION)
+        # NOTE: https://sha256.tbs-internet.com is another possible test host
+        remote = ("sha2.hboeck.de", 443)
+        sha256_cert = os.path.join(os.path.dirname(__file__), "sha256.pem")
+        with test_support.transient_internet("sha2.hboeck.de"):
+            s = ssl.wrap_socket(socket.socket(socket.AF_INET),
+                                cert_reqs=ssl.CERT_REQUIRED,
+                                ca_certs=sha256_cert,)
+            try:
+                s.connect(remote)
+                if test_support.verbose:
+                    sys.stdout.write("\nCipher with %r is %r\n" %
+                                     (remote, s.cipher()))
+                    sys.stdout.write("Certificate is:\n%s\n" %
+                                     pprint.pformat(s.getpeercert()))
+            finally:
+                s.close()
+
+
+try:
+    import threading
+except ImportError:
+    _have_threads = False
+else:
+    _have_threads = True
+
+    class ThreadedEchoServer(threading.Thread):
+
+        class ConnectionHandler(threading.Thread):
+
+            """A mildly complicated class, because we want it to work both
+            with and without the SSL wrapper around the socket connection, so
+            that we can test the STARTTLS functionality."""
+
+            def __init__(self, server, connsock):
+                self.server = server
+                self.running = False
+                self.sock = connsock
+                self.sock.setblocking(1)
+                self.sslconn = None
+                threading.Thread.__init__(self)
+                self.daemon = True
+
+            def show_conn_details(self):
+                if self.server.certreqs == ssl.CERT_REQUIRED:
+                    cert = self.sslconn.getpeercert()
+                    if test_support.verbose and self.server.chatty:
+                        sys.stdout.write(" client cert is " + pprint.pformat(cert) + "\n")
+                    cert_binary = self.sslconn.getpeercert(True)
+                    if test_support.verbose and self.server.chatty:
+                        sys.stdout.write(" cert binary is " + str(len(cert_binary)) + " bytes\n")
+                cipher = self.sslconn.cipher()
+                if test_support.verbose and self.server.chatty:
+                    sys.stdout.write(" server: connection cipher is now " + str(cipher) + "\n")
+
+            def wrap_conn(self):
+                try:
+                    self.sslconn = ssl.wrap_socket(self.sock, server_side=True,
+                                                   certfile=self.server.certificate,
+                                                   ssl_version=self.server.protocol,
+                                                   ca_certs=self.server.cacerts,
+                                                   cert_reqs=self.server.certreqs,
+                                                   ciphers=self.server.ciphers)
+                except ssl.SSLError:
+                    # XXX Various errors can have happened here, for example
+                    # a mismatching protocol version, an invalid certificate,
+                    # or a low-level bug. This should be made more discriminating.
+                    if self.server.chatty:
+                        handle_error("\n server:  bad connection attempt from " +
+                                     str(self.sock.getpeername()) + ":\n")
+                    self.close()
+                    self.running = False
+                    self.server.stop()
+                    return False
+                else:
+                    return True
+
+            def read(self):
+                if self.sslconn:
+                    return self.sslconn.read()
+                else:
+                    return self.sock.recv(1024)
+
+            def write(self, bytes):
+                if self.sslconn:
+                    return self.sslconn.write(bytes)
+                else:
+                    return self.sock.send(bytes)
+
+            def close(self):
+                if self.sslconn:
+                    self.sslconn.close()
+                else:
+                    self.sock._sock.close()
+
+            def run(self):
+                self.running = True
+                if not self.server.starttls_server:
+                    if isinstance(self.sock, ssl.SSLSocket):
+                        self.sslconn = self.sock
+                    elif not self.wrap_conn():
+                        return
+                    self.show_conn_details()
+                while self.running:
+                    try:
+                        msg = self.read()
+                        if not msg:
+                            # eof, so quit this handler
+                            self.running = False
+                            self.close()
+                        elif msg.strip() == 'over':
+                            if test_support.verbose and self.server.connectionchatty:
+                                sys.stdout.write(" server: client closed connection\n")
+                            self.close()
+                            return
+                        elif self.server.starttls_server and msg.strip() == 'STARTTLS':
+                            if test_support.verbose and self.server.connectionchatty:
+                                sys.stdout.write(" server: read STARTTLS from client, sending OK...\n")
+                            self.write("OK\n")
+                            if not self.wrap_conn():
+                                return
+                        elif self.server.starttls_server and self.sslconn and msg.strip() == 'ENDTLS':
+                            if test_support.verbose and self.server.connectionchatty:
+                                sys.stdout.write(" server: read ENDTLS from client, sending OK...\n")
+                            self.write("OK\n")
+                            self.sslconn.unwrap()
+                            self.sslconn = None
+                            if test_support.verbose and self.server.connectionchatty:
+                                sys.stdout.write(" server: connection is now unencrypted...\n")
+                        else:
+                            if (test_support.verbose and
+                                self.server.connectionchatty):
+                                ctype = (self.sslconn and "encrypted") or "unencrypted"
+                                sys.stdout.write(" server: read %s (%s), sending back %s (%s)...\n"
+                                                 % (repr(msg), ctype, repr(msg.lower()), ctype))
+                            self.write(msg.lower())
+                    except ssl.SSLError:
+                        if self.server.chatty:
+                            handle_error("Test server failure:\n")
+                        self.close()
+                        self.running = False
+                        # normally, we'd just stop here, but for the test
+                        # harness, we want to stop the server
+                        self.server.stop()
+
+        def __init__(self, certificate, ssl_version=None,
+                     certreqs=None, cacerts=None,
+                     chatty=True, connectionchatty=False, starttls_server=False,
+                     wrap_accepting_socket=False, ciphers=None):
+
+            if ssl_version is None:
+                ssl_version = ssl.PROTOCOL_TLSv1
+            if certreqs is None:
+                certreqs = ssl.CERT_NONE
+            self.certificate = certificate
+            self.protocol = ssl_version
+            self.certreqs = certreqs
+            self.cacerts = cacerts
+            self.ciphers = ciphers
+            self.chatty = chatty
+            self.connectionchatty = connectionchatty
+            self.starttls_server = starttls_server
+            self.sock = socket.socket()
+            self.flag = None
+            if wrap_accepting_socket:
+                self.sock = ssl.wrap_socket(self.sock, server_side=True,
+                                            certfile=self.certificate,
+                                            cert_reqs = self.certreqs,
+                                            ca_certs = self.cacerts,
+                                            ssl_version = self.protocol,
+                                            ciphers = self.ciphers)
+                if test_support.verbose and self.chatty:
+                    sys.stdout.write(' server:  wrapped server socket as %s\n' % str(self.sock))
+            self.port = test_support.bind_port(self.sock)
+            self.active = False
+            threading.Thread.__init__(self)
+            self.daemon = True
+
+        def start(self, flag=None):
+            self.flag = flag
+            threading.Thread.start(self)
+
+        def run(self):
+            self.sock.settimeout(0.05)
+            self.sock.listen(5)
+            self.active = True
+            if self.flag:
+                # signal an event
+                self.flag.set()
+            while self.active:
+                try:
+                    newconn, connaddr = self.sock.accept()
+                    if test_support.verbose and self.chatty:
+                        sys.stdout.write(' server:  new connection from '
+                                         + str(connaddr) + '\n')
+                    handler = self.ConnectionHandler(self, newconn)
+                    handler.start()
+                except socket.timeout:
+                    pass
+                except KeyboardInterrupt:
+                    self.stop()
+            self.sock.close()
+
+        def stop(self):
+            self.active = False
+
+    class AsyncoreEchoServer(threading.Thread):
+
+        class EchoServer(asyncore.dispatcher):
+
+            class ConnectionHandler(asyncore.dispatcher_with_send):
+
+                def __init__(self, conn, certfile):
+                    asyncore.dispatcher_with_send.__init__(self, conn)
+                    self.socket = ssl.wrap_socket(conn, server_side=True,
+                                                  certfile=certfile,
+                                                  do_handshake_on_connect=False)
+                    self._ssl_accepting = True
+
+                def readable(self):
+                    if isinstance(self.socket, ssl.SSLSocket):
+                        while self.socket.pending() > 0:
+                            self.handle_read_event()
+                    return True
+
+                def _do_ssl_handshake(self):
+                    try:
+                        self.socket.do_handshake()
+                    except ssl.SSLError, err:
+                        if err.args[0] in (ssl.SSL_ERROR_WANT_READ,
+                                           ssl.SSL_ERROR_WANT_WRITE):
+                            return
+                        elif err.args[0] == ssl.SSL_ERROR_EOF:
+                            return self.handle_close()
+                        raise
+                    except socket.error, err:
+                        if err.args[0] == errno.ECONNABORTED:
+                            return self.handle_close()
+                    else:
+                        self._ssl_accepting = False
+
+                def handle_read(self):
+                    if self._ssl_accepting:
+                        self._do_ssl_handshake()
+                    else:
+                        data = self.recv(1024)
+                        if data and data.strip() != 'over':
+                            self.send(data.lower())
+
+                def handle_close(self):
+                    self.close()
+                    if test_support.verbose:
+                        sys.stdout.write(" server:  closed connection %s\n" % self.socket)
+
+                def handle_error(self):
+                    raise
+
+            def __init__(self, certfile):
+                self.certfile = certfile
+                asyncore.dispatcher.__init__(self)
+                self.create_socket(socket.AF_INET, socket.SOCK_STREAM)
+                self.port = test_support.bind_port(self.socket)
+                self.listen(5)
+
+            def handle_accept(self):
+                sock_obj, addr = self.accept()
+                if test_support.verbose:
+                    sys.stdout.write(" server:  new connection from %s:%s\n" %addr)
+                self.ConnectionHandler(sock_obj, self.certfile)
+
+            def handle_error(self):
+                raise
+
+        def __init__(self, certfile):
+            self.flag = None
+            self.active = False
+            self.server = self.EchoServer(certfile)
+            self.port = self.server.port
+            threading.Thread.__init__(self)
+            self.daemon = True
+
+        def __str__(self):
+            return "<%s %s>" % (self.__class__.__name__, self.server)
+
+        def start(self, flag=None):
+            self.flag = flag
+            threading.Thread.start(self)
+
+        def run(self):
+            self.active = True
+            if self.flag:
+                self.flag.set()
+            while self.active:
+                asyncore.loop(0.05)
+
+        def stop(self):
+            self.active = False
+            self.server.close()
+
+    class SocketServerHTTPSServer(threading.Thread):
+
+        class HTTPSServer(HTTPServer):
+
+            def __init__(self, server_address, RequestHandlerClass, certfile):
+                HTTPServer.__init__(self, server_address, RequestHandlerClass)
+                # we assume the certfile contains both private key and certificate
+                self.certfile = certfile
+                self.allow_reuse_address = True
+
+            def __str__(self):
+                return ('<%s %s:%s>' %
+                        (self.__class__.__name__,
+                         self.server_name,
+                         self.server_port))
+
+            def get_request(self):
+                # override this to wrap socket with SSL
+                sock, addr = self.socket.accept()
+                sslconn = ssl.wrap_socket(sock, server_side=True,
+                                          certfile=self.certfile)
+                return sslconn, addr
+
+        class RootedHTTPRequestHandler(SimpleHTTPRequestHandler):
+            # need to override translate_path to get a known root,
+            # instead of using os.curdir, since the test could be
+            # run from anywhere
+
+            server_version = "TestHTTPS/1.0"
+
+            root = None
+
+            def translate_path(self, path):
+                """Translate a /-separated PATH to the local filename syntax.
+
+                Components that mean special things to the local file system
+                (e.g. drive or directory names) are ignored.  (XXX They should
+                probably be diagnosed.)
+
+                """
+                # abandon query parameters
+                path = urlparse.urlparse(path)[2]
+                path = os.path.normpath(urllib.unquote(path))
+                words = path.split('/')
+                words = filter(None, words)
+                path = self.root
+                for word in words:
+                    drive, word = os.path.splitdrive(word)
+                    head, word = os.path.split(word)
+                    if word in self.root: continue
+                    path = os.path.join(path, word)
+                return path
+
+            def log_message(self, format, *args):
+
+                # we override this to suppress logging unless "verbose"
+
+                if test_support.verbose:
+                    sys.stdout.write(" server (%s:%d %s):\n   [%s] %s\n" %
+                                     (self.server.server_address,
+                                      self.server.server_port,
+                                      self.request.cipher(),
+                                      self.log_date_time_string(),
+                                      format%args))
+
+
+        def __init__(self, certfile):
+            self.flag = None
+            self.RootedHTTPRequestHandler.root = os.path.split(CERTFILE)[0]
+            self.server = self.HTTPSServer(
+                (HOST, 0), self.RootedHTTPRequestHandler, certfile)
+            self.port = self.server.server_port
+            threading.Thread.__init__(self)
+            self.daemon = True
+
+        def __str__(self):
+            return "<%s %s>" % (self.__class__.__name__, self.server)
+
+        def start(self, flag=None):
+            self.flag = flag
+            threading.Thread.start(self)
+
+        def run(self):
+            if self.flag:
+                self.flag.set()
+            self.server.serve_forever(0.05)
+
+        def stop(self):
+            self.server.shutdown()
+
+
+    def bad_cert_test(certfile):
+        """
+        Launch a server with CERT_REQUIRED, and check that trying to
+        connect to it with the given client certificate fails.
+        """
+        server = ThreadedEchoServer(CERTFILE,
+                                    certreqs=ssl.CERT_REQUIRED,
+                                    cacerts=CERTFILE, chatty=False)
+        flag = threading.Event()
+        server.start(flag)
+        # wait for it to start
+        flag.wait()
+        # try to connect
+        try:
+            try:
+                s = ssl.wrap_socket(socket.socket(),
+                                    certfile=certfile,
+                                    ssl_version=ssl.PROTOCOL_TLSv1)
+                s.connect((HOST, server.port))
+            except ssl.SSLError, x:
+                if test_support.verbose:
+                    sys.stdout.write("\nSSLError is %s\n" % x[1])
+            except socket.error, x:
+                if test_support.verbose:
+                    sys.stdout.write("\nsocket.error is %s\n" % x[1])
+            else:
+                raise AssertionError("Use of invalid cert should have failed!")
+        finally:
+            server.stop()
+            server.join()
+
+    def server_params_test(certfile, protocol, certreqs, cacertsfile,
+                           client_certfile, client_protocol=None, indata="FOO\n",
+                           ciphers=None, chatty=True, connectionchatty=False,
+                           wrap_accepting_socket=False):
+        """
+        Launch a server, connect a client to it and try various reads
+        and writes.
+        """
+        server = ThreadedEchoServer(certfile,
+                                    certreqs=certreqs,
+                                    ssl_version=protocol,
+                                    cacerts=cacertsfile,
+                                    ciphers=ciphers,
+                                    chatty=chatty,
+                                    connectionchatty=connectionchatty,
+                                    wrap_accepting_socket=wrap_accepting_socket)
+        flag = threading.Event()
+        server.start(flag)
+        # wait for it to start
+        flag.wait()
+        # try to connect
+        if client_protocol is None:
+            client_protocol = protocol
+        try:
+            s = ssl.wrap_socket(socket.socket(),
+                                certfile=client_certfile,
+                                ca_certs=cacertsfile,
+                                ciphers=ciphers,
+                                cert_reqs=certreqs,
+                                ssl_version=client_protocol)
+            s.connect((HOST, server.port))
+            for arg in [indata, bytearray(indata), memoryview(indata)]:
+                if connectionchatty:
+                    if test_support.verbose:
+                        sys.stdout.write(
+                            " client:  sending %s...\n" % (repr(arg)))
+                s.write(arg)
+                outdata = s.read()
+                if connectionchatty:
+                    if test_support.verbose:
+                        sys.stdout.write(" client:  read %s\n" % repr(outdata))
+                if outdata != indata.lower():
+                    raise AssertionError(
+                        "bad data <<%s>> (%d) received; expected <<%s>> (%d)\n"
+                        % (outdata[:min(len(outdata),20)], len(outdata),
+                           indata[:min(len(indata),20)].lower(), len(indata)))
+            s.write("over\n")
+            if connectionchatty:
+                if test_support.verbose:
+                    sys.stdout.write(" client:  closing connection.\n")
+            s.close()
+        finally:
+            server.stop()
+            server.join()
+
+    def try_protocol_combo(server_protocol,
+                           client_protocol,
+                           expect_success,
+                           certsreqs=None):
+        if certsreqs is None:
+            certsreqs = ssl.CERT_NONE
+        certtype = {
+            ssl.CERT_NONE: "CERT_NONE",
+            ssl.CERT_OPTIONAL: "CERT_OPTIONAL",
+            ssl.CERT_REQUIRED: "CERT_REQUIRED",
+        }[certsreqs]
+        if test_support.verbose:
+            formatstr = (expect_success and " %s->%s %s\n") or " {%s->%s} %s\n"
+            sys.stdout.write(formatstr %
+                             (ssl.get_protocol_name(client_protocol),
+                              ssl.get_protocol_name(server_protocol),
+                              certtype))
+        try:
+            # NOTE: we must enable "ALL" ciphers, otherwise an SSLv23 client
+            # will send an SSLv3 hello (rather than SSLv2) starting from
+            # OpenSSL 1.0.0 (see issue #8322).
+            server_params_test(CERTFILE, server_protocol, certsreqs,
+                               CERTFILE, CERTFILE, client_protocol,
+                               ciphers="ALL", chatty=False)
+        # Protocol mismatch can result in either an SSLError, or a
+        # "Connection reset by peer" error.
+        except ssl.SSLError:
+            if expect_success:
+                raise
+        except socket.error as e:
+            if expect_success or e.errno != errno.ECONNRESET:
+                raise
+        else:
+            if not expect_success:
+                raise AssertionError(
+                    "Client protocol %s succeeded with server protocol %s!"
+                    % (ssl.get_protocol_name(client_protocol),
+                       ssl.get_protocol_name(server_protocol)))
+
+
+    class ThreadedTests(unittest.TestCase):
+
+        def test_rude_shutdown(self):
+            """A brutal shutdown of an SSL server should raise an IOError
+            in the client when attempting handshake.
+            """
+            listener_ready = threading.Event()
+            listener_gone = threading.Event()
+
+            s = socket.socket()
+            port = test_support.bind_port(s, HOST)
+
+            # `listener` runs in a thread.  It sits in an accept() until
+            # the main thread connects.  Then it rudely closes the socket,
+            # and sets Event `listener_gone` to let the main thread know
+            # the socket is gone.
+            def listener():
+                s.listen(5)
+                listener_ready.set()
+                s.accept()
+                s.close()
+                listener_gone.set()
+
+            def connector():
+                listener_ready.wait()
+                c = socket.socket()
+                c.connect((HOST, port))
+                listener_gone.wait()
+                try:
+                    ssl_sock = ssl.wrap_socket(c)
+                except IOError:
+                    pass
+                else:
+                    self.fail('connecting to closed SSL socket should have failed')
+
+            t = threading.Thread(target=listener)
+            t.start()
+            try:
+                connector()
+            finally:
+                t.join()
+
+        @skip_if_broken_ubuntu_ssl
+        def test_echo(self):
+            """Basic test of an SSL client connecting to a server"""
+            if test_support.verbose:
+                sys.stdout.write("\n")
+            server_params_test(CERTFILE, ssl.PROTOCOL_TLSv1, ssl.CERT_NONE,
+                               CERTFILE, CERTFILE, ssl.PROTOCOL_TLSv1,
+                               chatty=True, connectionchatty=True)
+
+        def test_getpeercert(self):
+            if test_support.verbose:
+                sys.stdout.write("\n")
+            s2 = socket.socket()
+            server = ThreadedEchoServer(CERTFILE,
+                                        certreqs=ssl.CERT_NONE,
+                                        ssl_version=ssl.PROTOCOL_SSLv23,
+                                        cacerts=CERTFILE,
+                                        chatty=False)
+            flag = threading.Event()
+            server.start(flag)
+            # wait for it to start
+            flag.wait()
+            # try to connect
+            try:
+                s = ssl.wrap_socket(socket.socket(),
+                                    certfile=CERTFILE,
+                                    ca_certs=CERTFILE,
+                                    cert_reqs=ssl.CERT_REQUIRED,
+                                    ssl_version=ssl.PROTOCOL_SSLv23)
+                s.connect((HOST, server.port))
+                cert = s.getpeercert()
+                self.assertTrue(cert, "Can't get peer certificate.")
+                cipher = s.cipher()
+                if test_support.verbose:
+                    sys.stdout.write(pprint.pformat(cert) + '\n')
+                    sys.stdout.write("Connection cipher is " + str(cipher) + '.\n')
+                if 'subject' not in cert:
+                    self.fail("No subject field in certificate: %s." %
+                              pprint.pformat(cert))
+                if ((('organizationName', 'Python Software Foundation'),)
+                    not in cert['subject']):
+                    self.fail(
+                        "Missing or invalid 'organizationName' field in certificate subject; "
+                        "should be 'Python Software Foundation'.")
+                s.close()
+            finally:
+                server.stop()
+                server.join()
+
+        def test_empty_cert(self):
+            """Connecting with an empty cert file"""
+            bad_cert_test(os.path.join(os.path.dirname(__file__) or os.curdir,
+                                      "nullcert.pem"))
+        def test_malformed_cert(self):
+            """Connecting with a badly formatted certificate (syntax error)"""
+            bad_cert_test(os.path.join(os.path.dirname(__file__) or os.curdir,
+                                       "badcert.pem"))
+        def test_nonexisting_cert(self):
+            """Connecting with a non-existing cert file"""
+            bad_cert_test(os.path.join(os.path.dirname(__file__) or os.curdir,
+                                       "wrongcert.pem"))
+        def test_malformed_key(self):
+            """Connecting with a badly formatted key (syntax error)"""
+            bad_cert_test(os.path.join(os.path.dirname(__file__) or os.curdir,
+                                       "badkey.pem"))
+
+        @skip_if_broken_ubuntu_ssl
+        def test_protocol_sslv2(self):
+            """Connecting to an SSLv2 server with various client options"""
+            if test_support.verbose:
+                sys.stdout.write("\n")
+            try_protocol_combo(ssl.PROTOCOL_SSLv2, ssl.PROTOCOL_SSLv2, True)
+            try_protocol_combo(ssl.PROTOCOL_SSLv2, ssl.PROTOCOL_SSLv2, True, ssl.CERT_OPTIONAL)
+            try_protocol_combo(ssl.PROTOCOL_SSLv2, ssl.PROTOCOL_SSLv2, True, ssl.CERT_REQUIRED)
+            try_protocol_combo(ssl.PROTOCOL_SSLv2, ssl.PROTOCOL_SSLv23, True)
+            try_protocol_combo(ssl.PROTOCOL_SSLv2, ssl.PROTOCOL_SSLv3, False)
+            try_protocol_combo(ssl.PROTOCOL_SSLv2, ssl.PROTOCOL_TLSv1, False)
+
+        @skip_if_broken_ubuntu_ssl
+        def test_protocol_sslv23(self):
+            """Connecting to an SSLv23 server with various client options"""
+            if test_support.verbose:
+                sys.stdout.write("\n")
+            try:
+                try_protocol_combo(ssl.PROTOCOL_SSLv23, ssl.PROTOCOL_SSLv2, True)
+            except (ssl.SSLError, socket.error), x:
+                # this fails on some older versions of OpenSSL (0.9.7l, for instance)
+                if test_support.verbose:
+                    sys.stdout.write(
+                        " SSL2 client to SSL23 server test unexpectedly failed:\n %s\n"
+                        % str(x))
+            try_protocol_combo(ssl.PROTOCOL_SSLv23, ssl.PROTOCOL_SSLv3, True)
+            try_protocol_combo(ssl.PROTOCOL_SSLv23, ssl.PROTOCOL_SSLv23, True)
+            try_protocol_combo(ssl.PROTOCOL_SSLv23, ssl.PROTOCOL_TLSv1, True)
+
+            try_protocol_combo(ssl.PROTOCOL_SSLv23, ssl.PROTOCOL_SSLv3, True, ssl.CERT_OPTIONAL)
+            try_protocol_combo(ssl.PROTOCOL_SSLv23, ssl.PROTOCOL_SSLv23, True, ssl.CERT_OPTIONAL)
+            try_protocol_combo(ssl.PROTOCOL_SSLv23, ssl.PROTOCOL_TLSv1, True, ssl.CERT_OPTIONAL)
+
+            try_protocol_combo(ssl.PROTOCOL_SSLv23, ssl.PROTOCOL_SSLv3, True, ssl.CERT_REQUIRED)
+            try_protocol_combo(ssl.PROTOCOL_SSLv23, ssl.PROTOCOL_SSLv23, True, ssl.CERT_REQUIRED)
+            try_protocol_combo(ssl.PROTOCOL_SSLv23, ssl.PROTOCOL_TLSv1, True, ssl.CERT_REQUIRED)
+
+        @skip_if_broken_ubuntu_ssl
+        def test_protocol_sslv3(self):
+            """Connecting to an SSLv3 server with various client options"""
+            if test_support.verbose:
+                sys.stdout.write("\n")
+            try_protocol_combo(ssl.PROTOCOL_SSLv3, ssl.PROTOCOL_SSLv3, True)
+            try_protocol_combo(ssl.PROTOCOL_SSLv3, ssl.PROTOCOL_SSLv3, True, ssl.CERT_OPTIONAL)
+            try_protocol_combo(ssl.PROTOCOL_SSLv3, ssl.PROTOCOL_SSLv3, True, ssl.CERT_REQUIRED)
+            try_protocol_combo(ssl.PROTOCOL_SSLv3, ssl.PROTOCOL_SSLv2, False)
+            try_protocol_combo(ssl.PROTOCOL_SSLv3, ssl.PROTOCOL_SSLv23, False)
+            try_protocol_combo(ssl.PROTOCOL_SSLv3, ssl.PROTOCOL_TLSv1, False)
+
+        @skip_if_broken_ubuntu_ssl
+        def test_protocol_tlsv1(self):
+            """Connecting to a TLSv1 server with various client options"""
+            if test_support.verbose:
+                sys.stdout.write("\n")
+            try_protocol_combo(ssl.PROTOCOL_TLSv1, ssl.PROTOCOL_TLSv1, True)
+            try_protocol_combo(ssl.PROTOCOL_TLSv1, ssl.PROTOCOL_TLSv1, True, ssl.CERT_OPTIONAL)
+            try_protocol_combo(ssl.PROTOCOL_TLSv1, ssl.PROTOCOL_TLSv1, True, ssl.CERT_REQUIRED)
+            try_protocol_combo(ssl.PROTOCOL_TLSv1, ssl.PROTOCOL_SSLv2, False)
+            try_protocol_combo(ssl.PROTOCOL_TLSv1, ssl.PROTOCOL_SSLv3, False)
+            try_protocol_combo(ssl.PROTOCOL_TLSv1, ssl.PROTOCOL_SSLv23, False)
+
+        def test_starttls(self):
+            """Switching from clear text to encrypted and back again."""
+            msgs = ("msg 1", "MSG 2", "STARTTLS", "MSG 3", "msg 4", "ENDTLS", "msg 5", "msg 6")
+
+            server = ThreadedEchoServer(CERTFILE,
+                                        ssl_version=ssl.PROTOCOL_TLSv1,
+                                        starttls_server=True,
+                                        chatty=True,
+                                        connectionchatty=True)
+            flag = threading.Event()
+            server.start(flag)
+            # wait for it to start
+            flag.wait()
+            # try to connect
+            wrapped = False
+            try:
+                s = socket.socket()
+                s.setblocking(1)
+                s.connect((HOST, server.port))
+                if test_support.verbose:
+                    sys.stdout.write("\n")
+                for indata in msgs:
+                    if test_support.verbose:
+                        sys.stdout.write(
+                            " client:  sending %s...\n" % repr(indata))
+                    if wrapped:
+                        conn.write(indata)
+                        outdata = conn.read()
+                    else:
+                        s.send(indata)
+                        outdata = s.recv(1024)
+                    if (indata == "STARTTLS" and
+                        outdata.strip().lower().startswith("ok")):
+                        # STARTTLS ok, switch to secure mode
+                        if test_support.verbose:
+                            sys.stdout.write(
+                                " client:  read %s from server, starting TLS...\n"
+                                % repr(outdata))
+                        conn = ssl.wrap_socket(s, ssl_version=ssl.PROTOCOL_TLSv1)
+                        wrapped = True
+                    elif (indata == "ENDTLS" and
+                        outdata.strip().lower().startswith("ok")):
+                        # ENDTLS ok, switch back to clear text
+                        if test_support.verbose:
+                            sys.stdout.write(
+                                " client:  read %s from server, ending TLS...\n"
+                                % repr(outdata))
+                        s = conn.unwrap()
+                        wrapped = False
+                    else:
+                        if test_support.verbose:
+                            sys.stdout.write(
+                                " client:  read %s from server\n" % repr(outdata))
+                if test_support.verbose:
+                    sys.stdout.write(" client:  closing connection.\n")
+                if wrapped:
+                    conn.write("over\n")
+                else:
+                    s.send("over\n")
+                s.close()
+            finally:
+                server.stop()
+                server.join()
+
+        def test_socketserver(self):
+            """Using a SocketServer to create and manage SSL connections."""
+            server = SocketServerHTTPSServer(CERTFILE)
+            flag = threading.Event()
+            server.start(flag)
+            # wait for it to start
+            flag.wait()
+            # try to connect
+            try:
+                if test_support.verbose:
+                    sys.stdout.write('\n')
+                with open(CERTFILE, 'rb') as f:
+                    d1 = f.read()
+                d2 = ''
+                # now fetch the same data from the HTTPS server
+                url = 'https://127.0.0.1:%d/%s' % (
+                    server.port, os.path.split(CERTFILE)[1])
+                with test_support.check_py3k_warnings():
+                    f = urllib.urlopen(url)
+                dlen = f.info().getheader("content-length")
+                if dlen and (int(dlen) > 0):
+                    d2 = f.read(int(dlen))
+                    if test_support.verbose:
+                        sys.stdout.write(
+                            " client: read %d bytes from remote server '%s'\n"
+                            % (len(d2), server))
+                f.close()
+                self.assertEqual(d1, d2)
+            finally:
+                server.stop()
+                server.join()
+
+        def test_wrapped_accept(self):
+            """Check the accept() method on SSL sockets."""
+            if test_support.verbose:
+                sys.stdout.write("\n")
+            server_params_test(CERTFILE, ssl.PROTOCOL_SSLv23, ssl.CERT_REQUIRED,
+                               CERTFILE, CERTFILE, ssl.PROTOCOL_SSLv23,
+                               chatty=True, connectionchatty=True,
+                               wrap_accepting_socket=True)
+
+        def test_asyncore_server(self):
+            """Check the example asyncore integration."""
+            indata = "TEST MESSAGE of mixed case\n"
+
+            if test_support.verbose:
+                sys.stdout.write("\n")
+            server = AsyncoreEchoServer(CERTFILE)
+            flag = threading.Event()
+            server.start(flag)
+            # wait for it to start
+            flag.wait()
+            # try to connect
+            try:
+                s = ssl.wrap_socket(socket.socket())
+                s.connect(('127.0.0.1', server.port))
+                if test_support.verbose:
+                    sys.stdout.write(
+                        " client:  sending %s...\n" % (repr(indata)))
+                s.write(indata)
+                outdata = s.read()
+                if test_support.verbose:
+                    sys.stdout.write(" client:  read %s\n" % repr(outdata))
+                if outdata != indata.lower():
+                    self.fail(
+                        "bad data <<%s>> (%d) received; expected <<%s>> (%d)\n"
+                        % (outdata[:min(len(outdata),20)], len(outdata),
+                           indata[:min(len(indata),20)].lower(), len(indata)))
+                s.write("over\n")
+                if test_support.verbose:
+                    sys.stdout.write(" client:  closing connection.\n")
+                s.close()
+            finally:
+                server.stop()
+                # wait for server thread to end
+                server.join()
+
+        def test_recv_send(self):
+            """Test recv(), send() and friends."""
+            if test_support.verbose:
+                sys.stdout.write("\n")
+
+            server = ThreadedEchoServer(CERTFILE,
+                                        certreqs=ssl.CERT_NONE,
+                                        ssl_version=ssl.PROTOCOL_TLSv1,
+                                        cacerts=CERTFILE,
+                                        chatty=True,
+                                        connectionchatty=False)
+            flag = threading.Event()
+            server.start(flag)
+            # wait for it to start
+            flag.wait()
+            # try to connect
+            s = ssl.wrap_socket(socket.socket(),
+                                server_side=False,
+                                certfile=CERTFILE,
+                                ca_certs=CERTFILE,
+                                cert_reqs=ssl.CERT_NONE,
+                                ssl_version=ssl.PROTOCOL_TLSv1)
+            s.connect((HOST, server.port))
+            try:
+                # helper methods for standardising recv* method signatures
+                def _recv_into():
+                    b = bytearray("\0"*100)
+                    count = s.recv_into(b)
+                    return b[:count]
+
+                def _recvfrom_into():
+                    b = bytearray("\0"*100)
+                    count, addr = s.recvfrom_into(b)
+                    return b[:count]
+
+                # (name, method, whether to expect success, *args)
+                send_methods = [
+                    ('send', s.send, True, []),
+                    ('sendto', s.sendto, False, ["some.address"]),
+                    ('sendall', s.sendall, True, []),
+                ]
+                recv_methods = [
+                    ('recv', s.recv, True, []),
+                    ('recvfrom', s.recvfrom, False, ["some.address"]),
+                    ('recv_into', _recv_into, True, []),
+                    ('recvfrom_into', _recvfrom_into, False, []),
+                ]
+                data_prefix = u"PREFIX_"
+
+                for meth_name, send_meth, expect_success, args in send_methods:
+                    indata = data_prefix + meth_name
+                    try:
+                        send_meth(indata.encode('ASCII', 'strict'), *args)
+                        outdata = s.read()
+                        outdata = outdata.decode('ASCII', 'strict')
+                        if outdata != indata.lower():
+                            self.fail(
+                                "While sending with <<%s>> bad data "
+                                "<<%r>> (%d) received; "
+                                "expected <<%r>> (%d)\n" % (
+                                    meth_name, outdata[:20], len(outdata),
+                                    indata[:20], len(indata)
+                                )
+                            )
+                    except ValueError as e:
+                        if expect_success:
+                            self.fail(
+                                "Failed to send with method <<%s>>; "
+                                "expected to succeed.\n" % (meth_name,)
+                            )
+                        if not str(e).startswith(meth_name):
+                            self.fail(
+                                "Method <<%s>> failed with unexpected "
+                                "exception message: %s\n" % (
+                                    meth_name, e
+                                )
+                            )
+
+                for meth_name, recv_meth, expect_success, args in recv_methods:
+                    indata = data_prefix + meth_name
+                    try:
+                        s.send(indata.encode('ASCII', 'strict'))
+                        outdata = recv_meth(*args)
+                        outdata = outdata.decode('ASCII', 'strict')
+                        if outdata != indata.lower():
+                            self.fail(
+                                "While receiving with <<%s>> bad data "
+                                "<<%r>> (%d) received; "
+                                "expected <<%r>> (%d)\n" % (
+                                    meth_name, outdata[:20], len(outdata),
+                                    indata[:20], len(indata)
+                                )
+                            )
+                    except ValueError as e:
+                        if expect_success:
+                            self.fail(
+                                "Failed to receive with method <<%s>>; "
+                                "expected to succeed.\n" % (meth_name,)
+                            )
+                        if not str(e).startswith(meth_name):
+                            self.fail(
+                                "Method <<%s>> failed with unexpected "
+                                "exception message: %s\n" % (
+                                    meth_name, e
+                                )
+                            )
+                        # consume data
+                        s.read()
+
+                s.write("over\n".encode("ASCII", "strict"))
+                s.close()
+            finally:
+                server.stop()
+                server.join()
+
+        def test_handshake_timeout(self):
+            # Issue #5103: SSL handshake must respect the socket timeout
+            server = socket.socket(socket.AF_INET)
+            host = "127.0.0.1"
+            port = test_support.bind_port(server)
+            started = threading.Event()
+            finish = False
+
+            def serve():
+                server.listen(5)
+                started.set()
+                conns = []
+                while not finish:
+                    r, w, e = select.select([server], [], [], 0.1)
+                    if server in r:
+                        # Let the socket hang around rather than having
+                        # it closed by garbage collection.
+                        conns.append(server.accept()[0])
+
+            t = threading.Thread(target=serve)
+            t.start()
+            started.wait()
+
+            try:
+                try:
+                    c = socket.socket(socket.AF_INET)
+                    c.settimeout(0.2)
+                    c.connect((host, port))
+                    # Will attempt handshake and time out
+                    self.assertRaisesRegexp(ssl.SSLError, "timed out",
+                                            ssl.wrap_socket, c)
+                finally:
+                    c.close()
+                try:
+                    c = socket.socket(socket.AF_INET)
+                    c.settimeout(0.2)
+                    c = ssl.wrap_socket(c)
+                    # Will attempt handshake and time out
+                    self.assertRaisesRegexp(ssl.SSLError, "timed out",
+                                            c.connect, (host, port))
+                finally:
+                    c.close()
+            finally:
+                finish = True
+                t.join()
+                server.close()
+
+
+def test_main(verbose=False):
+    if skip_expected:
+        raise unittest.SkipTest("No SSL support")
+
+    global CERTFILE, SVN_PYTHON_ORG_ROOT_CERT
+    CERTFILE = test_support.findfile("keycert.pem")
+    SVN_PYTHON_ORG_ROOT_CERT =  test_support.findfile(
+        "https_svn_python_org_root.pem")
+
+    if (not os.path.exists(CERTFILE) or
+        not os.path.exists(SVN_PYTHON_ORG_ROOT_CERT)):
+        raise test_support.TestFailed("Can't read certificate files!")
+
+    tests = [BasicTests, BasicSocketTests]
+
+    if test_support.is_resource_enabled('network'):
+        tests.append(NetworkedTests)
+
+    if _have_threads:
+        thread_info = test_support.threading_setup()
+        if thread_info and test_support.is_resource_enabled('network'):
+            tests.append(ThreadedTests)
+
+    try:
+        test_support.run_unittest(*tests)
+    finally:
+        if _have_threads:
+            test_support.threading_cleanup(*thread_info)
+
+if __name__ == "__main__":
+    test_main()

diff --git a/pypy/jit/tl/pypyjit_demo.py b/pypy/jit/tl/pypyjit_demo.py
--- a/pypy/jit/tl/pypyjit_demo.py
+++ b/pypy/jit/tl/pypyjit_demo.py
@@ -1,19 +1,16 @@
 
 try:
-    import pypyjit
-    pypyjit.set_param(threshold=3, inlining=True)
+    class C:
+        def __call__(self):
+            pass
 
-    def sqrt(y, n=10000):
-        x = y / 2
-        while n > 0:
-            #assert y > 0 and x > 0
-            if y > 0 and x > 0: pass
-            n -= 1
-            x = (x + y/x) / 2
-        return x
-
-    print sqrt(1234, 4)
-    
+    def f():
+        i = 0
+        c = C()
+        while i < 10:
+            c()
+            i += 1
+    f()
 except Exception, e:
     print "Exception: ", type(e)
     print e

From commits-noreply at bitbucket.org  Tue Feb  1 13:29:49 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Tue,  1 Feb 2011 13:29:49 +0100 (CET)
Subject: [pypy-svn] pypy default: Proper backout
Message-ID: <20110201122949.02245282BD8@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: 
Changeset: r41520:815cadb9214e
Date: 2011-02-01 14:28 +0200
http://bitbucket.org/pypy/pypy/changeset/815cadb9214e/

Log:	Proper backout

diff --git a/pypy/interpreter/gateway.py b/pypy/interpreter/gateway.py
--- a/pypy/interpreter/gateway.py
+++ b/pypy/interpreter/gateway.py
@@ -344,9 +344,8 @@
                                                        self.nextarg()))
 
     def visit__ObjSpace(self, el):
-        if self.finger not in [0, 1]:
+        if self.finger != 0:
             raise FastFuncNotSupported
-        self.args.append('space')
         self.unwrap.append("space")
 
     def visit__W_Root(self, el):
@@ -392,9 +391,7 @@
         unwrap_info = UnwrapSpec_FastFunc_Unwrap()
         unwrap_info.apply_over(unwrap_spec)
         narg = unwrap_info.n
-        args = unwrap_info.args
-        if 'space' not in args:
-            args = ['space'] + args
+        args = ['space'] + unwrap_info.args
         if args == unwrap_info.unwrap:
             fastfunc = func
         else:
@@ -531,12 +528,8 @@
                     self.__class__ = BuiltinCodePassThroughArguments1
                     self.func__args__ = func
             else:
-                if len(unwrap_spec) >= 2 and unwrap_spec[1] is ObjSpace:
-                    self.__class__ = globals()['BuiltinCodeSpace%d' % arity]
-                    setattr(self, 'fastfunc_space_%d' % arity, fastfunc)
-                else:
-                    self.__class__ = globals()['BuiltinCode%d' % arity]
-                    setattr(self, 'fastfunc_%d' % arity, fastfunc)
+                self.__class__ = globals()['BuiltinCode%d' % arity]
+                setattr(self, 'fastfunc_%d' % arity, fastfunc)
 
     def descr__reduce__(self, space):
         from pypy.interpreter.mixedmodule import MixedModule
@@ -724,78 +717,6 @@
             w_result = space.w_None
         return w_result
 
-class BuiltinCodeSpace1(BuiltinCode1):
-    _immutable_ = True
-    fast_natural_arity = 1
-    
-    def fastcall_1(self, space, w_func, w1):
-        try:
-            w_result = self.fastfunc_space_1(w1, space)
-        except DescrMismatch:
-            return  w1.descr_call_mismatch(space,
-                                           self.descrmismatch_op,
-                                           self.descr_reqcls,
-                                           Arguments(space, [w1]))
-        except Exception, e:
-            raise self.handle_exception(space, e)
-        if w_result is None:
-            w_result = space.w_None
-        return w_result
-
-class BuiltinCodeSpace2(BuiltinCode2):
-    _immutable_ = True
-    fast_natural_arity = 2
-    
-    def fastcall_2(self, space, w_func, w1, w2):
-        try:
-            w_result = self.fastfunc_space_2(w1, space, w2)
-        except DescrMismatch:
-            return  w1.descr_call_mismatch(space,
-                                           self.descrmismatch_op,
-                                           self.descr_reqcls,
-                                           Arguments(space, [w1, w2]))
-        except Exception, e:
-            raise self.handle_exception(space, e)
-        if w_result is None:
-            w_result = space.w_None
-        return w_result
-
-class BuiltinCodeSpace3(BuiltinCode3):
-    _immutable_ = True
-    fast_natural_arity = 3
-    
-    def fastcall_3(self, space, func, w1, w2, w3):
-        try:
-            w_result = self.fastfunc_space_3(w1, space, w2, w3)
-        except DescrMismatch:
-            return  w1.descr_call_mismatch(space,
-                                           self.descrmismatch_op,
-                                           self.descr_reqcls,
-                                           Arguments(space, [w1, w2, w3]))
-        except Exception, e:
-            raise self.handle_exception(space, e)
-        if w_result is None:
-            w_result = space.w_None
-        return w_result
-
-class BuiltinCodeSpace4(BuiltinCode4):
-    _immutable_ = True
-    fast_natural_arity = 4
-    
-    def fastcall_4(self, space, func, w1, w2, w3, w4):
-        try:
-            w_result = self.fastfunc_space_4(w1, space, w2, w3, w4)
-        except DescrMismatch:
-            return  w1.descr_call_mismatch(space,
-                                           self.descrmismatch_op,
-                                           self.descr_reqcls,
-                                           Arguments(space,
-                                                     [w1, w2, w3, w4]))
-        except Exception, e:
-            raise self.handle_exception(space, e)
-        if w_result is None:
-            w_result = space.w_None
-        return w_result
 
 class interp2app(Wrappable):
     """Build a gateway that calls 'f' at interp-level."""

diff --git a/pypy/interpreter/test/test_gateway.py b/pypy/interpreter/test/test_gateway.py
--- a/pypy/interpreter/test/test_gateway.py
+++ b/pypy/interpreter/test/test_gateway.py
@@ -14,7 +14,7 @@
         self.name = name
         self.defs_w = []
 
-class TestBuiltinCode(object): 
+class TestBuiltinCode: 
     def test_signature(self):
         def c(space, w_x, w_y, hello_w):
             pass
@@ -545,42 +545,7 @@
         """)
 
         assert space.is_true(w_res)
-        assert called == [w_app_f, w_app_f]
-
-    def test_interp2app_fastcall_method_space(self):
-        space = self.space
-        w = space.wrap
-        w_3 = w(3)
-
-        def f(w_self, space, w_x):
-            return w_x
-        app_f = gateway.interp2app_temp(f, unwrap_spec=[gateway.W_Root,
-                                                        gateway.ObjSpace,
-                                                        gateway.W_Root])
-        w_app_f = w(app_f)
-
-        # sanity
-        assert isinstance(w_app_f.code, gateway.BuiltinCode2)
-
-        called = []
-        fastcall_2 = w_app_f.code.fastcall_2
-        def witness_fastcall_2(space, w_func, w_a, w_b):
-            called.append(w_func)
-            return fastcall_2(space, w_func, w_a, w_b)
-
-        w_app_f.code.fastcall_2 = witness_fastcall_2    
-    
-        w_res = space.appexec([w_app_f, w_3], """(f, x):
-        class A(object):
-           m = f # not a builtin function, so works as method
-        y = A().m(x)
-        b = A().m
-        z = b(x)
-        return y is x and z is x
-        """)
-
-        assert space.is_true(w_res)
-        assert called == [w_app_f, w_app_f]        
+        assert called == [w_app_f, w_app_f]       
         
     def test_plain(self):
         space = self.space

diff --git a/pypy/jit/tl/pypyjit_demo.py b/pypy/jit/tl/pypyjit_demo.py
--- a/pypy/jit/tl/pypyjit_demo.py
+++ b/pypy/jit/tl/pypyjit_demo.py
@@ -1,16 +1,19 @@
 
 try:
-    class C:
-        def __call__(self):
-            pass
+    import pypyjit
+    pypyjit.set_param(threshold=3, inlining=True)
 
-    def f():
-        i = 0
-        c = C()
-        while i < 10:
-            c()
-            i += 1
-    f()
+    def sqrt(y, n=10000):
+        x = y / 2
+        while n > 0:
+            #assert y > 0 and x > 0
+            if y > 0 and x > 0: pass
+            n -= 1
+            x = (x + y/x) / 2
+        return x
+
+    print sqrt(1234, 4)
+    
 except Exception, e:
     print "Exception: ", type(e)
     print e

From commits-noreply at bitbucket.org  Tue Feb  1 13:29:51 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Tue,  1 Feb 2011 13:29:51 +0100 (CET)
Subject: [pypy-svn] pypy default: merge
Message-ID: <20110201122951.2650A282BD8@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: 
Changeset: r41521:59399209047b
Date: 2011-02-01 14:29 +0200
http://bitbucket.org/pypy/pypy/changeset/59399209047b/

Log:	merge




From commits-noreply at bitbucket.org  Tue Feb  1 15:07:43 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue,  1 Feb 2011 15:07:43 +0100 (CET)
Subject: [pypy-svn] pypy default: Re-enable geninterp.  For me,
	running "translate.py -O2" crashes
Message-ID: <20110201140743.6A61E282BD8@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41522:cf0bdb323fec
Date: 2011-02-01 15:07 +0100
http://bitbucket.org/pypy/pypy/changeset/cf0bdb323fec/

Log:	Re-enable geninterp. For me, running "translate.py -O2" crashes
	without it.

diff --git a/pypy/config/pypyoption.py b/pypy/config/pypyoption.py
--- a/pypy/config/pypyoption.py
+++ b/pypy/config/pypyoption.py
@@ -166,7 +166,7 @@
                suggests=[("objspace.allworkingmodules", False)]),
 
     BoolOption("geninterp", "specify whether geninterp should be used",
-               default=False),
+               default=True),
 
     BoolOption("logbytecodes",
                "keep track of bytecode usage",

From commits-noreply at bitbucket.org  Tue Feb  1 15:32:11 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Tue,  1 Feb 2011 15:32:11 +0100 (CET)
Subject: [pypy-svn] pypy default: Remove filestub. nobody uses it
Message-ID: <20110201143211.0FFB9282BAD@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: 
Changeset: r41523:a7a3f5f337c4
Date: 2011-02-01 16:31 +0200
http://bitbucket.org/pypy/pypy/changeset/a7a3f5f337c4/

Log:	Remove filestub. nobody uses it

diff --git a/pypy/module/__builtin__/__init__.py b/pypy/module/__builtin__/__init__.py
--- a/pypy/module/__builtin__/__init__.py
+++ b/pypy/module/__builtin__/__init__.py
@@ -36,7 +36,6 @@
 
         'bin'           : 'app_operation.bin',
 
-        '__filestub'    : 'app_file_stub.file',
     }
 
     interpleveldefs = {

From commits-noreply at bitbucket.org  Tue Feb  1 15:32:11 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Tue,  1 Feb 2011 15:32:11 +0100 (CET)
Subject: [pypy-svn] pypy default: fixed issue, redisable geninterp
Message-ID: <20110201143211.8C2A5282BAD@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: 
Changeset: r41524:aa1a6d5a1ef1
Date: 2011-02-01 16:31 +0200
http://bitbucket.org/pypy/pypy/changeset/aa1a6d5a1ef1/

Log:	fixed issue, redisable geninterp

diff --git a/pypy/config/pypyoption.py b/pypy/config/pypyoption.py
--- a/pypy/config/pypyoption.py
+++ b/pypy/config/pypyoption.py
@@ -166,7 +166,7 @@
                suggests=[("objspace.allworkingmodules", False)]),
 
     BoolOption("geninterp", "specify whether geninterp should be used",
-               default=True),
+               default=False),
 
     BoolOption("logbytecodes",
                "keep track of bytecode usage",

From commits-noreply at bitbucket.org  Tue Feb  1 15:34:45 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Tue,  1 Feb 2011 15:34:45 +0100 (CET)
Subject: [pypy-svn] pypy default: Remove app_file_stub.py
Message-ID: <20110201143445.3DBE6282BAD@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: 
Changeset: r41525:a46f6fc27ded
Date: 2011-02-01 16:34 +0200
http://bitbucket.org/pypy/pypy/changeset/a46f6fc27ded/

Log:	Remove app_file_stub.py

diff --git a/pypy/module/__builtin__/app_file_stub.py b/pypy/module/__builtin__/app_file_stub.py
deleted file mode 100644
--- a/pypy/module/__builtin__/app_file_stub.py
+++ /dev/null
@@ -1,20 +0,0 @@
-# NOT_RPYTHON
-
-class file(object): 
-    """file(name[, mode[, buffering]]) -> file object
-
-Open a file.  The mode can be 'r', 'w' or 'a' for reading (default),
-writing or appending.  The file will be created if it doesn't exist
-when opened for writing or appending; it will be truncated when
-opened for writing.  Add a 'b' to the mode for binary files.
-Add a '+' to the mode to allow simultaneous reading and writing.
-If the buffering argument is given, 0 means unbuffered, 1 means line
-buffered, and larger numbers specify the buffer size.
-Add a 'U' to mode to open the file for input with universal newline
-support.  Any line ending in the input file will be seen as a '\n'
-in Python.  Also, a file so opened gains the attribute 'newlines';
-the value for this attribute is one of None (no newline read yet),
-'\r', '\n', '\r\n' or a tuple containing all the newline types seen.
-
-Note:  open() is an alias for file().
-"""

From commits-noreply at bitbucket.org  Tue Feb  1 16:00:55 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Tue,  1 Feb 2011 16:00:55 +0100 (CET)
Subject: [pypy-svn] pypy default: Negative values can be raised to the nan
	power.
Message-ID: <20110201150055.6BC59282BAD@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41526:57363f9b7479
Date: 2011-02-01 09:59 -0500
http://bitbucket.org/pypy/pypy/changeset/57363f9b7479/

Log:	Negative values can be raised to the nan power.

diff --git a/pypy/objspace/std/floatobject.py b/pypy/objspace/std/floatobject.py
--- a/pypy/objspace/std/floatobject.py
+++ b/pypy/objspace/std/floatobject.py
@@ -10,12 +10,13 @@
 from pypy.objspace.std.longobject import W_LongObject
 from pypy.rlib.rarithmetic import ovfcheck_float_to_int, intmask, isinf, isnan
 from pypy.rlib.rarithmetic import (LONG_BIT, INFINITY, copysign,
-    formatd, DTSF_ADD_DOT_0, DTSF_STR_PRECISION)
+    formatd, DTSF_ADD_DOT_0, DTSF_STR_PRECISION, NAN)
 from pypy.rlib.rbigint import rbigint
 from pypy.rlib.objectmodel import we_are_translated
 from pypy.rlib import rfloat
 from pypy.tool.sourcetools import func_with_new_name
 
+
 import math
 from pypy.objspace.std.intobject import W_IntObject
 
@@ -340,7 +341,7 @@
     x = w_float1.floatval
     y = w_float2.floatval
     if y == 0.0:
-        raise FailedToImplementArgs(space.w_ZeroDivisionError, space.wrap("float division"))    
+        raise FailedToImplementArgs(space.w_ZeroDivisionError, space.wrap("float division"))
     return W_FloatObject(x / y)
 
 truediv__Float_Float = div__Float_Float
@@ -424,6 +425,8 @@
     # unlike "math.pow(-1.0, bignum)".  See http://mail.python.org/
     # -           pipermail/python-bugs-list/2003-March/016795.html
     if x < 0.0:
+        if math.isnan(y):
+            return W_FloatObject(NAN)
         if math.floor(y) != y:
             raise OperationError(space.w_ValueError,
                                  space.wrap("negative number cannot be "

diff --git a/pypy/objspace/std/test/test_floatobject.py b/pypy/objspace/std/test/test_floatobject.py
--- a/pypy/objspace/std/test/test_floatobject.py
+++ b/pypy/objspace/std/test/test_floatobject.py
@@ -170,21 +170,23 @@
 
     def test_special_float_method(self):
         class a(object):
-            def __float__(self): 
-                self.ar = True 
+            def __float__(self):
+                self.ar = True
                 return None
         inst = a()
-        raises(TypeError, float, inst) 
-        assert inst.ar 
+        raises(TypeError, float, inst)
+        assert inst.ar
 
-        class b(object): 
-            pass 
-        raises((AttributeError, TypeError), float, b()) 
+        class b(object):
+            pass
+        raises((AttributeError, TypeError), float, b())
 
     def test_getnewargs(self):
         assert  0.0 .__getnewargs__() == (0.0,)
 
     def test_pow(self):
+        import math
+
         def pw(x, y):
             return x ** y
         def espeq(x, y):
@@ -200,6 +202,8 @@
         assert pw(-1.0, 1e200) == 1.0
         if self.py26:
             assert pw(0.0, float("-inf")) == float("inf")
+            assert math.isnan(pw(-3, float("nan")))
+            assert math.isnan(pw(-3., float("nan")))
 
     def test_pow_neg_base(self):
         import math

From commits-noreply at bitbucket.org  Tue Feb  1 16:00:56 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Tue,  1 Feb 2011 16:00:56 +0100 (CET)
Subject: [pypy-svn] pypy default: Merged upstream.
Message-ID: <20110201150056.1997F282BAD@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41527:13c0dc295d74
Date: 2011-02-01 10:00 -0500
http://bitbucket.org/pypy/pypy/changeset/13c0dc295d74/

Log:	Merged upstream.

diff --git a/pypy/module/__builtin__/app_file_stub.py b/pypy/module/__builtin__/app_file_stub.py
deleted file mode 100644
--- a/pypy/module/__builtin__/app_file_stub.py
+++ /dev/null
@@ -1,20 +0,0 @@
-# NOT_RPYTHON
-
-class file(object): 
-    """file(name[, mode[, buffering]]) -> file object
-
-Open a file.  The mode can be 'r', 'w' or 'a' for reading (default),
-writing or appending.  The file will be created if it doesn't exist
-when opened for writing or appending; it will be truncated when
-opened for writing.  Add a 'b' to the mode for binary files.
-Add a '+' to the mode to allow simultaneous reading and writing.
-If the buffering argument is given, 0 means unbuffered, 1 means line
-buffered, and larger numbers specify the buffer size.
-Add a 'U' to mode to open the file for input with universal newline
-support.  Any line ending in the input file will be seen as a '\n'
-in Python.  Also, a file so opened gains the attribute 'newlines';
-the value for this attribute is one of None (no newline read yet),
-'\r', '\n', '\r\n' or a tuple containing all the newline types seen.
-
-Note:  open() is an alias for file().
-"""

From commits-noreply at bitbucket.org  Tue Feb  1 17:42:23 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Tue,  1 Feb 2011 17:42:23 +0100 (CET)
Subject: [pypy-svn] pypy default: Try to get test_seek_and_tell from
	lib-python test_io	passing.
Message-ID: <20110201164223.1DED1282BAD@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41528:3f6b33b21cc9
Date: 2011-02-01 11:42 -0500
http://bitbucket.org/pypy/pypy/changeset/3f6b33b21cc9/

Log:	Try to get test_seek_and_tell from lib-python test_io passing.

diff --git a/pypy/objspace/std/formatting.py b/pypy/objspace/std/formatting.py
--- a/pypy/objspace/std/formatting.py
+++ b/pypy/objspace/std/formatting.py
@@ -548,5 +548,3 @@
 int_num_helper = format_num_helper_generator('%d', '0123456789')
 oct_num_helper = format_num_helper_generator('%o', '01234567')
 hex_num_helper = format_num_helper_generator('%x', '0123456789abcdef')
-
-

diff --git a/pypy/module/_io/interp_textio.py b/pypy/module/_io/interp_textio.py
--- a/pypy/module/_io/interp_textio.py
+++ b/pypy/module/_io/interp_textio.py
@@ -5,8 +5,9 @@
 from pypy.interpreter.gateway import interp2app, unwrap_spec
 from pypy.interpreter.baseobjspace import ObjSpace, Wrappable, W_Root
 from pypy.interpreter.error import OperationError, operationerrfmt
+from pypy.rlib.rarithmetic import r_ulonglong
+from pypy.rlib.rbigint import rbigint
 from pypy.rlib.rstring import UnicodeBuilder
-from pypy.rlib.rarithmetic import r_ulonglong
 from pypy.module._codecs import interp_codecs
 from pypy.module._io.interp_iobase import convert_size
 import sys
@@ -234,14 +235,17 @@
     errors = GetSetProperty(W_TextIOBase.errors_get_w),
 )
 
-class PositionCookie:
+class PositionCookie(object):
     def __init__(self, bigint):
         self.start_pos = bigint.ulonglongmask()
         bigint = bigint.rshift(r_ulonglong.BITS)
-        self.dec_flags = 0
-        self.bytes_to_feed = 0
-        self.chars_to_skip = 0
-        self.need_eof = 0
+        self.dec_flags = bigint.ulonglongmask()
+        bigint = bigint.rshift(r_ulonglong.BITS)
+        self.bytes_to_feed = bigint.ulonglongmask()
+        bigint = bigint.rshift(r_ulonglong.BITS)
+        self.chars_to_skip = bigint.ulonglongmask()
+        bigint = bigint.rshift(r_ulonglong.BITS)
+        self.need_eof = bigint.ulonglongmask()
 
     def pack(self):
         # The meaning of a tell() cookie is: seek to position, set the
@@ -249,11 +253,13 @@
         # into the decoder with need_eof as the EOF flag, then skip
         # chars_to_skip characters of the decoded result.  For most simple
         # decoders, tell() will often just give a byte offset in the file.
-        return (self.start_pos |
-                (self.dec_flags<<64) |    # XXX fixme! does not work in RPython
-                (self.bytes_to_feed<<128) |
-                (self.chars_to_skip<<192) |
-                bool(self.need_eof)<<256)
+        rb = rbigint.fromint
+
+        res = rb(self.start_pos)
+        res = res.or_(rb(self.dec_flags).lshift(1 * r_ulonglong.BITS))
+        res = res.or_(rb(self.bytes_to_feed).lshift(2 * r_ulonglong.BITS))
+        res = res.or_(rb(self.chars_to_skip).lshift(3 * r_ulonglong.BITS))
+        return res.or_(rb(self.need_eof).lshift(4 * r_ulonglong.BITS))
 
 class PositionSnapshot:
     def __init__(self, flags, input):
@@ -363,6 +369,7 @@
         self.seekable = space.is_true(space.call_method(w_buffer, "seekable"))
         self.telling = self.seekable
 
+        self.encoding_start_of_stream = False
         if self.seekable and self.w_encoder:
             self.encoding_start_of_stream = True
             w_cookie = space.call_method(self.w_buffer, "tell")
@@ -538,10 +545,11 @@
             raise OperationError(space.w_IOError, space.wrap("not readable"))
 
         size = convert_size(space, w_size)
+        self._writeflush(space)
         if size < 0:
             # Read everything
             w_bytes = space.call_method(self.w_buffer, "read")
-            w_decoded = space.call_method(self.w_decoder, "decode", w_bytes)
+            w_decoded = space.call_method(self.w_decoder, "decode", w_bytes, space.w_True)
             w_result = space.wrap(self._get_decoded_chars(-1))
             w_final = space.add(w_result, w_decoded)
             self.snapshot = None
@@ -779,7 +787,6 @@
         # utf-16, that we are expecting a BOM).
         if cookie.start_pos == 0 and cookie.dec_flags == 0:
             space.call_method(self.w_decoder, "reset")
-            self.encoding_start_of_stream = True
         else:
             space.call_method(self.w_decoder, "setstate",
                               space.newtuple([space.wrap(""),
@@ -904,7 +911,7 @@
         # How many decoded characters have been used up since the snapshot?
         if not self.decoded_chars_used:
             # We haven't moved from the snapshot point.
-            return space.wrap(cookie.pack())
+            return space.newlong_from_rbigint(cookie.pack())
 
         chars_to_skip = self.decoded_chars_used
 
@@ -961,7 +968,7 @@
 
         # The returned cookie corresponds to the last safe start point.
         cookie.chars_to_skip = chars_to_skip
-        return space.wrap(cookie.pack())
+        return space.newlong_from_rbigint(cookie.pack())
 
     def chunk_size_get_w(space, self):
         self._check_init(space)

From commits-noreply at bitbucket.org  Tue Feb  1 20:08:54 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Tue,  1 Feb 2011 20:08:54 +0100 (CET)
Subject: [pypy-svn] pypy jit-fromstart: Allow the entire optimizer and its
	state to be	cloned to allow the optimization to be resumed at
	any point	(work in progress). This is used by the unroller to
	optimize	the loop twice. The second time around the full set
	of	inputargs are known from the start and these can be passed
	on	to the bridges through the failargs.
Message-ID: <20110201190854.8E542282BD8@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-fromstart
Changeset: r41529:9d324e858411
Date: 2011-02-01 20:08 +0100
http://bitbucket.org/pypy/pypy/changeset/9d324e858411/

Log:	Allow the entire optimizer and its state to be cloned to allow the
	optimization to be resumed at any point (work in progress). This is
	used by the unroller to optimize the loop twice. The second time
	around the full set of inputargs are known from the start and these
	can be passed on to the bridges through the failargs.

diff --git a/pypy/jit/metainterp/optimizeopt/fficall.py b/pypy/jit/metainterp/optimizeopt/fficall.py
--- a/pypy/jit/metainterp/optimizeopt/fficall.py
+++ b/pypy/jit/metainterp/optimizeopt/fficall.py
@@ -67,7 +67,7 @@
     def __init__(self):
         self.funcinfo = None
 
-    def reconstruct_for_next_iteration(self, optimizer, valuemap):
+    def clone_for_next_iteration(self, optimizer, valuemap):
         return OptFfiCall()
         # FIXME: Should any status be saved for next iteration?
 

diff --git a/pypy/jit/metainterp/optimizeopt/string.py b/pypy/jit/metainterp/optimizeopt/string.py
--- a/pypy/jit/metainterp/optimizeopt/string.py
+++ b/pypy/jit/metainterp/optimizeopt/string.py
@@ -367,9 +367,8 @@
     "Handling of strings and unicodes."
     enabled = True
 
-    def reconstruct_for_next_iteration(self, optimizer, valuemap):
-        self.enabled = True
-        return self
+    def clone_for_next_iteration(self, optimizer, valuemap):
+        return OptString()
     
     def make_vstring_plain(self, box, source_op, mode):
         vvalue = VStringPlainValue(self.optimizer, box, source_op, mode)

diff --git a/pypy/jit/metainterp/optimizeopt/unroll.py b/pypy/jit/metainterp/optimizeopt/unroll.py
--- a/pypy/jit/metainterp/optimizeopt/unroll.py
+++ b/pypy/jit/metainterp/optimizeopt/unroll.py
@@ -155,61 +155,29 @@
         if jumpop:
             assert jumpop.getdescr() is loop.token
             loop.preamble.operations = self.optimizer.newoperations
-            self.optimizer = self.optimizer.reconstruct_for_next_iteration()
+            jump_args = jumpop.getarglist()
+            jumpop.initarglist([])            
 
-            jump_args = jumpop.getarglist()
-            #for i in range(len(jump_args)):
-            #    self.getvalue(jump_args[i]).start_index = i
-
-            jumpop.initarglist([])            
+            preamble_optmizer = self.optimizer
+            self.optimizer = preamble_optmizer.clone_for_next_iteration()
             inputargs = self.inline(self.cloned_operations,
                                     loop.inputargs, jump_args)
+
+            loop_inputargs = loop.inputargs
             loop.inputargs = inputargs
             jmp = ResOperation(rop.JUMP, loop.inputargs[:], None)
             jmp.setdescr(loop.token)
             loop.preamble.operations.append(jmp)
-
             loop.operations = self.optimizer.newoperations
 
-            start_resumedescr = loop.preamble.start_resumedescr.clone_if_mutable()
-            assert isinstance(start_resumedescr, ResumeGuardDescr)
-            snapshot = start_resumedescr.rd_snapshot
-            while snapshot is not None:
-                snapshot_args = snapshot.boxes 
-                new_snapshot_args = []
-                for a in snapshot_args:
-                    if not isinstance(a, Const):
-                        a = loop.preamble.inputargs[jump_args.index(a)]
-                    new_snapshot_args.append(a)
-                snapshot.boxes = new_snapshot_args
-                snapshot = snapshot.prev
-
-            short = self.create_short_preamble(loop.preamble, loop)
-            if short:
+            short_loop = self.create_short_preamble(loop.preamble, loop,
+                                                    jump_args)
+            if short_loop:
                 if False:
                     # FIXME: This should save some memory but requires
-                    # a lot of tests to be fixed...
-                    loop.preamble.operations = short[:]
-
-                # Turn guards into conditional jumps to the preamble
-                for i in range(len(short)):
-                    op = short[i]
-                    if op.is_guard():
-                        op = op.clone()
-                        op.setfailargs(None)
-                        op.setdescr(start_resumedescr.clone_if_mutable())
-                        short[i] = op
-
-                short_loop = TreeLoop('short preamble')
-                short_loop.inputargs = loop.preamble.inputargs[:]
-                short_loop.operations = short
-
-                # Clone ops and boxes to get private versions and 
-                newargs = [a.clonebox() for a in short_loop.inputargs]
-                inliner = Inliner(short_loop.inputargs, newargs)
-                short_loop.inputargs = newargs
-                ops = [inliner.inline_op(op) for op in short_loop.operations]
-                short_loop.operations = ops
+                    # a lot of tests to be fixed, and the guards to be
+                    # replaced with ResumeGuardDescr
+                    loop.preamble.operations = short_loop.operations[:]
 
                 assert isinstance(loop.preamble.token, LoopToken)
                 if loop.preamble.token.short_preamble:
@@ -217,12 +185,14 @@
                 else:
                     loop.preamble.token.short_preamble = [short_loop]
 
-                # Forget the values to allow them to be freed
-                for box in short_loop.inputargs:
-                    box.forget_value()
-                for op in short_loop.operations:
-                    if op.result:
-                        op.result.forget_value()
+                if True:
+                    self.optimizer = \
+                            preamble_optmizer.clone_for_next_iteration()
+                    self.optimizer.extraargs = loop.inputargs
+                    loop.inputargs = self.inline(self.cloned_operations,
+                                             loop_inputargs, jump_args)
+                    assert loop.inputargs == inputargs
+                    loop.operations = self.optimizer.newoperations
 
                 for op in loop.operations:
                     if op.is_guard():
@@ -235,6 +205,21 @@
                                 i = -1
                             descr.start_indexes.append(i)
                         descr.parent_short_preamble = short_loop
+                        
+                # Clone ops and boxes to get private versions  
+                newargs = [a.clonebox() for a in short_loop.inputargs]
+                inliner = Inliner(short_loop.inputargs, newargs)
+                short_loop.inputargs = newargs
+                ops = [inliner.inline_op(op) for op in short_loop.operations]
+                short_loop.operations = ops
+
+                # Forget the values to allow them to be freed
+                for box in short_loop.inputargs:
+                    box.forget_value()
+                for op in short_loop.operations:
+                    if op.result:
+                        op.result.forget_value()
+                        
                             
                                 
                 
@@ -242,10 +227,7 @@
     def inline(self, loop_operations, loop_args, jump_args):
         self.inliner = inliner = Inliner(loop_args, jump_args)
            
-        for v in self.optimizer.values.values():
-            v.last_guard_index = -1 # FIXME: Are there any more indexes stored?
-
-        self.optimizer.extraargs = inputargs = []
+        inputargs = []
         seen_inputargs = {}
         for arg in jump_args:
             boxes = []
@@ -257,30 +239,13 @@
         # This loop is equivalent to the main optimization loop in
         # Optimizer.propagate_all_forward
         boxes_created_this_iteration = {}
-        last_emitted_len = -1
         for newop in loop_operations:
             if newop.getopnum() == rop.JUMP:
                 newop.initarglist(inputargs)
-            newop = inliner.inline_op(newop, clone=False)
+            newop = inliner.inline_op(newop) #, clone=False)
             
             self.optimizer.first_optimization.propagate_forward(newop)
 
-            # This should take care of updating inputargs in most cases i.e.
-            # when newoperations is not reorder too much. The rest should be
-            # pacthed below. Bridges will only be able to inherit the part of
-            # the short preamble that produces the boxes that are placed in 
-            # inputargs before the guard producing the bridge is emitted.
-            i = max(last_emitted_len - 2, 0)
-            while i < len(self.optimizer.newoperations):
-                op = self.optimizer.newoperations[i]
-                boxes_created_this_iteration[op.result] = True
-                for a in op.getarglist():
-                    if not isinstance(a, Const) and not a in boxes_created_this_iteration:
-                        if a not in inputargs:
-                            inputargs.append(a)
-                i += 1
-            last_emitted_len = len(self.optimizer.newoperations)
-
         # Remove jump to make sure forced code are placed before it
         newoperations = self.optimizer.newoperations
         jmp = newoperations[-1]
@@ -335,7 +300,7 @@
 
         return True
 
-    def create_short_preamble(self, preamble, loop):
+    def create_short_preamble(self, preamble, loop, original_jump_args):
         #return None # Dissable
 
         preamble_ops = preamble.operations
@@ -417,8 +382,36 @@
                     return None
             if op.result:
                 seen[op.result] = True
-        
-        return short_preamble
+
+        # Make all guards resume at the top of the preamble
+        start_resumedescr = loop.preamble.start_resumedescr
+        start_resumedescr = start_resumedescr.clone_if_mutable()
+        assert isinstance(start_resumedescr, ResumeGuardDescr)
+        snapshot = start_resumedescr.rd_snapshot
+        while snapshot is not None:
+            snapshot_args = snapshot.boxes 
+            new_snapshot_args = []
+            for a in snapshot_args:
+                if not isinstance(a, Const):
+                    a = loop.preamble.inputargs[original_jump_args.index(a)]
+                new_snapshot_args.append(a)
+            snapshot.boxes = new_snapshot_args
+            snapshot = snapshot.prev
+
+        for i in range(len(short_preamble)):
+            op = short_preamble[i]
+            if op.is_guard():
+                op = op.clone()
+                op.setfailargs(None)
+                op.setdescr(start_resumedescr.clone_if_mutable())
+                short_preamble[i] = op
+
+        # Contruct the TreeLoop object
+        short_loop = TreeLoop('short preamble')
+        short_loop.inputargs = loop.preamble.inputargs[:]
+        short_loop.operations = short_preamble
+                
+        return short_loop
 
 class ExeState(object):
     def __init__(self, optimizer):
@@ -543,8 +536,8 @@
         return self.map[loopbox]
 
 class OptInlineShortPreamble(Optimization):
-    def reconstruct_for_next_iteration(self, optimizer, valuemap):
-        return self
+    def clone_for_next_iteration(self, optimizer, valuemap):
+        return OptInlineShortPreamble()
     
     def propagate_forward(self, op):
         if op.getopnum() == rop.JUMP:

diff --git a/pypy/jit/metainterp/optimizeopt/heap.py b/pypy/jit/metainterp/optimizeopt/heap.py
--- a/pypy/jit/metainterp/optimizeopt/heap.py
+++ b/pypy/jit/metainterp/optimizeopt/heap.py
@@ -24,8 +24,10 @@
         self.lazy_setfields = {}
         self.lazy_setfields_descrs = []     # keys (at least) of previous dict
 
-    def reconstruct_for_next_iteration(self, optimizer, valuemap):
+    def clone_for_next_iteration(self, optimizer, valuemap):
         new = OptHeap()
+        return new
+        # FIXME:
 
         if True:
             self.force_all_lazy_setfields()

diff --git a/pypy/jit/metainterp/optimizeopt/rewrite.py b/pypy/jit/metainterp/optimizeopt/rewrite.py
--- a/pypy/jit/metainterp/optimizeopt/rewrite.py
+++ b/pypy/jit/metainterp/optimizeopt/rewrite.py
@@ -13,8 +13,8 @@
        This includes already executed operations and constants.
     """
 
-    def reconstruct_for_next_iteration(self, optimizer, valuemap):
-        return self
+    def clone_for_next_iteration(self, optimizer, valuemap):
+        return OptRewrite()
     
     def propagate_forward(self, op):
         args = self.optimizer.make_args_key(op)

diff --git a/pypy/jit/metainterp/optimizeopt/optimizer.py b/pypy/jit/metainterp/optimizeopt/optimizer.py
--- a/pypy/jit/metainterp/optimizeopt/optimizer.py
+++ b/pypy/jit/metainterp/optimizeopt/optimizer.py
@@ -27,15 +27,19 @@
     __metaclass__ = extendabletype
     _attrs_ = ('box', 'known_class', 'last_guard_index', 'level', 'intbound')
     last_guard_index = -1
-    #start_index = -1
 
     level = LEVEL_UNKNOWN
     known_class = None
     intbound = None
 
-    def __init__(self, box):
+    def __init__(self, box, level=None, known_class=None, intbound=None):
         self.box = box
-        self.intbound = IntBound(MININT, MAXINT) #IntUnbounded()
+        self.level = level
+        self.known_class = known_class
+        if intbound:
+            self.intbound = intbound
+        else:
+            self.intbound = IntBound(MININT, MAXINT) #IntUnbounded()
         if isinstance(box, Const):
             self.make_constant(box)
         # invariant: box is a Const if and only if level == LEVEL_CONSTANT
@@ -55,13 +59,17 @@
     def get_reconstructed(self, optimizer, valuemap):
         if self in valuemap:
             return valuemap[self]
-        new = self.reconstruct_for_next_iteration(optimizer)
+        new = self.clone_for_next_iteration(optimizer)
+        assert new.__class__ == self.__class__
         valuemap[self] = new
         self.reconstruct_childs(new, valuemap)
         return new
 
-    def reconstruct_for_next_iteration(self, optimizer):
-        return self
+    def clone_for_next_iteration(self, optimizer):
+        # last_guard_index should not be propagated here as the list into
+        # which it is an index is cleared
+        return OptValue(self.box, self.level, self.known_class,
+                        self.intbound.clone())
 
     def reconstruct_childs(self, new, valuemap):
         pass
@@ -224,8 +232,7 @@
     def turned_constant(self, value):
         pass
 
-    def reconstruct_for_next_iteration(self, optimizer=None, valuemap=None):
-        #return self.__class__()
+    def clone_for_next_iteration(self, optimizer=None, valuemap=None):
         raise NotImplementedError
     
 
@@ -304,31 +311,39 @@
         for o in self.optimizations:
             o.force_at_end_of_preamble()
             
-    def reconstruct_for_next_iteration(self, optimizer=None, valuemap=None):
+    def clone_for_next_iteration(self, optimizer=None, valuemap=None):
         assert optimizer is None
         assert valuemap is None
         valuemap = {}
         new = Optimizer(self.metainterp_sd, self.loop)
-        optimizations = [o.reconstruct_for_next_iteration(new, valuemap) for o in 
+        new.values = {}
+        optimizations = [o.clone_for_next_iteration(new, valuemap) for o in 
                          self.optimizations]
         new.set_optimizations(optimizations)
 
-        new.values = {}
-        for box, value in self.values.items():
-            new.values[box] = value.get_reconstructed(new, valuemap)
-        new.interned_refs = self.interned_refs
-        new.bool_boxes = {}
-        for value in new.bool_boxes.keys():
-            new.bool_boxes[value.get_reconstructed(new, valuemap)] = None
+        # FIXME: new.interned_refs = self.interned_refs
+        # FIXME:
+        #new.bool_boxes = {}
+        #for value in new.bool_boxes.keys():
+        #    new.bool_boxes[value.get_reconstructed(new, valuemap)] = None
 
         # FIXME: Move to rewrite.py
         new.loop_invariant_results = {}
         for key, value in self.loop_invariant_results.items():
             new.loop_invariant_results[key] = \
                                  value.get_reconstructed(new, valuemap)
+
+        for args, op in self.pure_operations.items():
+            newargs = args[:]
+            for i in range(len(newargs)):
+                if isinstance(newargs[i], OptValue):
+                    newargs[i] = newargs[i].get_reconstructed(new, valuemap)
+            v = self.getvalue(op.result)
+            new.values[op.result] = v.get_reconstructed(new, valuemap)
+            new.pure_operations[newargs] = op
+            # FIXME: This will not work for ops with mutable descr
             
-        new.pure_operations = self.pure_operations
-        new.producer = self.producer
+        # FIXME: Any point in propagating these? new.producer = self.producer
         assert self.posponedop is None
 
         return new
@@ -539,14 +554,15 @@
                 # all constant arguments: constant-fold away
                 argboxes = [self.get_constant_box(op.getarg(i))
                             for i in range(op.numargs())]
-                resbox = execute_nonspec(self.cpu, None,
-                                         op.getopnum(), argboxes, op.getdescr())
+                resbox = execute_nonspec(self.cpu, None, op.getopnum(),
+                                         argboxes, op.getdescr())
                 # FIXME: Don't we need to check for an overflow here?
                 self.make_constant(op.result, resbox.constbox())
                 return
 
             # did we do the exact same operation already?
             args = self.make_args_key(op)
+            
             oldop = self.pure_operations.get(args, None)
             if oldop is not None and oldop.getdescr() is op.getdescr():
                 assert oldop.getopnum() == op.getopnum()

diff --git a/pypy/jit/metainterp/test/test_basic.py b/pypy/jit/metainterp/test/test_basic.py
--- a/pypy/jit/metainterp/test/test_basic.py
+++ b/pypy/jit/metainterp/test/test_basic.py
@@ -403,6 +403,26 @@
                           'jump': 2,
                           'int_gt': 1, 'guard_true': 1})
 
+    def test_loop_invariant_mul_bridge_maintaining3(self):
+        myjitdriver = JitDriver(greens = [], reds = ['y', 'res', 'x'])
+        def f(x, y):
+            res = 0
+            while y > 0:
+                myjitdriver.can_enter_jit(x=x, y=y, res=res)
+                myjitdriver.jit_merge_point(x=x, y=y, res=res)
+                if y<16:
+                    res += 1
+                res += x * x
+                y -= 1
+            return res
+        res = self.meta_interp(f, [6, 32])
+        assert res == 1167
+        self.check_loop_count(3)
+        self.check_loops({'int_add': 3, 'int_lt': 1,
+                          'int_sub': 2, 'guard_false': 1,
+                          'jump': 2,
+                          'int_gt': 1, 'guard_true': 1})
+
     def test_loop_invariant_intbox(self):
         myjitdriver = JitDriver(greens = [], reds = ['y', 'res', 'x'])
         class I:

diff --git a/pypy/jit/metainterp/optimizeopt/intbounds.py b/pypy/jit/metainterp/optimizeopt/intbounds.py
--- a/pypy/jit/metainterp/optimizeopt/intbounds.py
+++ b/pypy/jit/metainterp/optimizeopt/intbounds.py
@@ -13,9 +13,9 @@
         self.posponedop = None
         self.nextop = None
 
-    def reconstruct_for_next_iteration(self, optimizer, valuemap):
+    def clone_for_next_iteration(self, optimizer, valuemap):
         assert self.posponedop is None
-        return self 
+        return OptIntBounds()
 
     def propagate_forward(self, op):
         if op.is_ovf():

diff --git a/pypy/jit/metainterp/optimizeopt/intutils.py b/pypy/jit/metainterp/optimizeopt/intutils.py
--- a/pypy/jit/metainterp/optimizeopt/intutils.py
+++ b/pypy/jit/metainterp/optimizeopt/intutils.py
@@ -76,7 +76,7 @@
         return r
     
     def add(self, offset):
-        res = self.copy()
+        res = self.clone()
         try:
             res.lower = ovfcheck(res.lower + offset)
         except OverflowError:
@@ -91,7 +91,7 @@
         return self.mul_bound(IntBound(value, value))
     
     def add_bound(self, other):
-        res = self.copy()
+        res = self.clone()
         if other.has_upper:
             try:
                 res.upper = ovfcheck(res.upper + other.upper)
@@ -109,7 +109,7 @@
         return res
 
     def sub_bound(self, other):
-        res = self.copy()
+        res = self.clone()
         if other.has_lower:
             try:
                 res.upper = ovfcheck(res.upper - other.lower)
@@ -204,7 +204,7 @@
             u = 'Inf'
         return '%s <= x <= %s' % (l, u)
 
-    def copy(self):
+    def clone(self):
         res = IntBound(self.lower, self.upper)
         res.has_lower = self.has_lower
         res.has_upper = self.has_upper

diff --git a/pypy/jit/metainterp/optimizeopt/virtualize.py b/pypy/jit/metainterp/optimizeopt/virtualize.py
--- a/pypy/jit/metainterp/optimizeopt/virtualize.py
+++ b/pypy/jit/metainterp/optimizeopt/virtualize.py
@@ -252,8 +252,8 @@
 class OptVirtualize(optimizer.Optimization):
     "Virtualize objects until they escape."
 
-    def reconstruct_for_next_iteration(self, optimizer, valuemap):
-        return self
+    def clone_for_next_iteration(self, optimizer, valuemap):
+        return OptVirtualize()
 
     def make_virtual(self, known_class, box, source_op=None):
         vvalue = VirtualValue(self.optimizer, known_class, box, source_op)

From commits-noreply at bitbucket.org  Tue Feb  1 22:48:14 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Tue,  1 Feb 2011 22:48:14 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix translation by python 2.5 (and pypy
	1.4)
Message-ID: <20110201214814.19769282B8B@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41530:64d00b0b042a
Date: 2011-02-01 22:41 +0100
http://bitbucket.org/pypy/pypy/changeset/64d00b0b042a/

Log:	Fix translation by python 2.5 (and pypy 1.4)

diff --git a/pypy/objspace/std/floatobject.py b/pypy/objspace/std/floatobject.py
--- a/pypy/objspace/std/floatobject.py
+++ b/pypy/objspace/std/floatobject.py
@@ -425,7 +425,7 @@
     # unlike "math.pow(-1.0, bignum)".  See http://mail.python.org/
     # -           pipermail/python-bugs-list/2003-March/016795.html
     if x < 0.0:
-        if math.isnan(y):
+        if isnan(y):
             return W_FloatObject(NAN)
         if math.floor(y) != y:
             raise OperationError(space.w_ValueError,

From commits-noreply at bitbucket.org  Wed Feb  2 00:05:16 2011
From: commits-noreply at bitbucket.org (wlav)
Date: Wed,  2 Feb 2011 00:05:16 +0100 (CET)
Subject: [pypy-svn] pypy reflex-support: check for negative values on
	unsigned types
Message-ID: <20110201230516.267A2282B8B@codespeak.net>

Author: Wim Lavrijsen <WLavrijsen at lbl.gov>
Branch: reflex-support
Changeset: r41531:0cefe1fd53e7
Date: 2011-02-01 15:03 -0800
http://bitbucket.org/pypy/pypy/changeset/0cefe1fd53e7/

Log:	check for negative values on unsigned types

diff --git a/pypy/module/cppyy/test/test_datatypes.py b/pypy/module/cppyy/test/test_datatypes.py
--- a/pypy/module/cppyy/test/test_datatypes.py
+++ b/pypy/module/cppyy/test/test_datatypes.py
@@ -31,7 +31,7 @@
         lib2 = cppyy.load_lib(self.shared_lib)
         assert self.datatypes is lib2
 
-    def test2WriteAccess( self ):
+    def test2WriteAccess(self):
         """Test write access to instance public data and verify values"""
 
         import cppyy, sys
@@ -111,3 +111,17 @@
 #                assert eval('c.m_%s_array2[i]' % names[j]) == b[i]
 
         c.destruct()
+
+    def test3RangeAccess(self):
+        """Test the ranges of integer types"""
+
+        import cppyy, sys
+        cppyy_test_data = cppyy.gbl.cppyy_test_data
+
+        c = cppyy_test_data()
+        assert isinstance(c, cppyy_test_data)
+
+        raises(ValueError, setattr, c, 'm_uint',  -1)
+        raises(ValueError, setattr, c, 'm_ulong', -1)
+
+        c.destruct()

diff --git a/pypy/module/cppyy/converter.py b/pypy/module/cppyy/converter.py
--- a/pypy/module/cppyy/converter.py
+++ b/pypy/module/cppyy/converter.py
@@ -133,13 +133,14 @@
 
 class LongConverter(TypeConverter):
     libffitype = libffi.types.slong
+    rffiptrtype = rffi.LONGP
 
     def _unwrap_object(self, space, w_obj):
         return space.c_int_w(w_obj)
 
     def convert_argument(self, space, w_obj):
         arg = self._unwrap_object(space, w_obj)
-        x = lltype.malloc(rffi.LONGP.TO, 1, flavor='raw')
+        x = lltype.malloc(self.rffiptrtype.TO, 1, flavor='raw')
         x[0] = arg
         return rffi.cast(rffi.VOIDP, x)
 
@@ -148,13 +149,20 @@
 
     def from_memory(self, space, w_obj, offset):
         fieldptr = self._get_fieldptr(space, w_obj, offset)
-        longptr = rffi.cast(rffi.LONGP, fieldptr)
+        longptr = rffi.cast(self.rffiptrtype, fieldptr)
         return space.wrap(longptr[0])
 
     def to_memory(self, space, w_obj, w_value, offset):
         fieldptr = self._get_fieldptr(space, w_obj, offset)
-        longptr = rffi.cast(rffi.LONGP, fieldptr)
-        longptr[0] = space.c_int_w(w_value)
+        longptr = rffi.cast(self.rffiptrtype, fieldptr)
+        longptr[0] = self._unwrap_object(space, w_value)
+
+class UnsignedLongConverter(LongConverter):
+    libffitype = libffi.types.ulong
+    rffiptrtype = rffi.ULONGP
+
+    def _unwrap_object(self, space, w_obj):
+        return space.c_uint_w(w_obj)
 
 class ShortConverter(LongConverter):
     libffitype = libffi.types.sshort
@@ -353,13 +361,13 @@
 _converters["int"]                      = LongConverter
 _converters["int*"]                     = LongPtrConverter
 _converters["int[]"]                    = LongArrayConverter
-_converters["unsigned int"]             = LongConverter
+_converters["unsigned int"]             = UnsignedLongConverter
 _converters["unsigned int*"]            = LongPtrConverter
 _converters["unsigned int[]"]           = LongArrayConverter
 _converters["long int"]                 = LongConverter
 _converters["long int*"]                = LongPtrConverter
 _converters["long int[]"]               = LongArrayConverter
-_converters["unsigned long int"]        = LongConverter
+_converters["unsigned long int"]        = UnsignedLongConverter
 _converters["unsigned long int*"]       = LongPtrConverter
 _converters["unsigned long int[]"]      = LongArrayConverter
 _converters["float"]                    = FloatConverter

From commits-noreply at bitbucket.org  Wed Feb  2 00:51:39 2011
From: commits-noreply at bitbucket.org (wlav)
Date: Wed,  2 Feb 2011 00:51:39 +0100 (CET)
Subject: [pypy-svn] pypy reflex-support: added test for builtin type
	conversions
Message-ID: <20110201235139.55276282B8B@codespeak.net>

Author: Wim Lavrijsen <WLavrijsen at lbl.gov>
Branch: reflex-support
Changeset: r41532:99ad86908454
Date: 2011-02-01 15:50 -0800
http://bitbucket.org/pypy/pypy/changeset/99ad86908454/

Log:	added test for builtin type conversions

diff --git a/pypy/module/cppyy/test/test_datatypes.py b/pypy/module/cppyy/test/test_datatypes.py
--- a/pypy/module/cppyy/test/test_datatypes.py
+++ b/pypy/module/cppyy/test/test_datatypes.py
@@ -105,6 +105,7 @@
             for i in range(self.N):
                 assert eval('c.m_%s_array[i]' % names[j]) == b[i]
 
+# TODO: be able to dissect array.array for its pointers
 #            exec 'c.m_%s_array2 = b' % names[j]  # pointer copies
 #            b[i] = 28
 #            for i in range(self.N):
@@ -125,3 +126,20 @@
         raises(ValueError, setattr, c, 'm_ulong', -1)
 
         c.destruct()
+
+    def test4TypeConversions(self):
+        """Test conversions between builtin types"""
+
+        import cppyy, sys
+        cppyy_test_data = cppyy.gbl.cppyy_test_data
+
+        c = cppyy_test_data()
+        assert isinstance(c, cppyy_test_data)
+
+
+        c.m_double = -1
+        assert round(c.m_double + 1.0, 8) == 0
+
+        raises(TypeError, c.m_double,  'c')
+        raises(TypeError, c.m_int,     -1.)
+        raises(TypeError, c.m_int,      1.)

From commits-noreply at bitbucket.org  Wed Feb  2 08:20:50 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Wed,  2 Feb 2011 08:20:50 +0100 (CET)
Subject: [pypy-svn] pypy jit-fromstart: interface to specify which boxes
	will survive	into next iteration
Message-ID: <20110202072050.31C052A2006@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-fromstart
Changeset: r41533:a2aeb3bd859a
Date: 2011-02-01 20:44 +0100
http://bitbucket.org/pypy/pypy/changeset/a2aeb3bd859a/

Log:	interface to specify which boxes will survive into next iteration

diff --git a/pypy/jit/metainterp/optimizeopt/fficall.py b/pypy/jit/metainterp/optimizeopt/fficall.py
--- a/pypy/jit/metainterp/optimizeopt/fficall.py
+++ b/pypy/jit/metainterp/optimizeopt/fficall.py
@@ -67,7 +67,7 @@
     def __init__(self):
         self.funcinfo = None
 
-    def clone_for_next_iteration(self, optimizer, valuemap):
+    def clone_for_next_iteration(self, surviving_boxes, optimizer, valuemap):
         return OptFfiCall()
         # FIXME: Should any status be saved for next iteration?
 

diff --git a/pypy/jit/metainterp/optimizeopt/string.py b/pypy/jit/metainterp/optimizeopt/string.py
--- a/pypy/jit/metainterp/optimizeopt/string.py
+++ b/pypy/jit/metainterp/optimizeopt/string.py
@@ -367,7 +367,7 @@
     "Handling of strings and unicodes."
     enabled = True
 
-    def clone_for_next_iteration(self, optimizer, valuemap):
+    def clone_for_next_iteration(self, surviving_boxes, optimizer, valuemap):
         return OptString()
     
     def make_vstring_plain(self, box, source_op, mode):

diff --git a/pypy/jit/metainterp/optimizeopt/unroll.py b/pypy/jit/metainterp/optimizeopt/unroll.py
--- a/pypy/jit/metainterp/optimizeopt/unroll.py
+++ b/pypy/jit/metainterp/optimizeopt/unroll.py
@@ -159,7 +159,8 @@
             jumpop.initarglist([])            
 
             preamble_optmizer = self.optimizer
-            self.optimizer = preamble_optmizer.clone_for_next_iteration()
+            self.optimizer = \
+                        preamble_optmizer.clone_for_next_iteration(jump_args)
             inputargs = self.inline(self.cloned_operations,
                                     loop.inputargs, jump_args)
 
@@ -187,7 +188,7 @@
 
                 if True:
                     self.optimizer = \
-                            preamble_optmizer.clone_for_next_iteration()
+                          preamble_optmizer.clone_for_next_iteration(jump_args)
                     self.optimizer.extraargs = loop.inputargs
                     loop.inputargs = self.inline(self.cloned_operations,
                                              loop_inputargs, jump_args)
@@ -536,7 +537,7 @@
         return self.map[loopbox]
 
 class OptInlineShortPreamble(Optimization):
-    def clone_for_next_iteration(self, optimizer, valuemap):
+    def clone_for_next_iteration(self, surviving_boxes, optimizer, valuemap):
         return OptInlineShortPreamble()
     
     def propagate_forward(self, op):

diff --git a/pypy/jit/metainterp/optimizeopt/heap.py b/pypy/jit/metainterp/optimizeopt/heap.py
--- a/pypy/jit/metainterp/optimizeopt/heap.py
+++ b/pypy/jit/metainterp/optimizeopt/heap.py
@@ -24,33 +24,27 @@
         self.lazy_setfields = {}
         self.lazy_setfields_descrs = []     # keys (at least) of previous dict
 
-    def clone_for_next_iteration(self, optimizer, valuemap):
+    def clone_for_next_iteration(self, surviving_boxes, optimizer, valuemap):
         new = OptHeap()
-        return new
-        # FIXME:
 
-        if True:
-            self.force_all_lazy_setfields()
-            assert not self.lazy_setfields_descrs
-            assert not self.lazy_setfields
-        else:
-            new.lazy_setfields_descrs = self.lazy_setfields_descrs
-            new.lazy_setfields = self.lazy_setfields
+        self.force_all_lazy_setfields()
+        assert not self.lazy_setfields_descrs
+        assert not self.lazy_setfields
         
         for descr, d in self.cached_fields.items():
             newd = {}
             new.cached_fields[descr] = newd
             for value, fieldvalue in d.items():
                 newd[value.get_reconstructed(optimizer, valuemap)] = \
-                                       fieldvalue.get_reconstructed(optimizer, valuemap)
+                             fieldvalue.get_reconstructed(optimizer, valuemap)
             
         for descr, d in self.known_heap_fields.items():
             newd = {}
             new.known_heap_fields[descr] = newd
             for value, fieldvalue in d.items():
                 newd[value.get_reconstructed(optimizer, valuemap)] = \
-                                       fieldvalue.get_reconstructed(optimizer, valuemap)
-            
+                             fieldvalue.get_reconstructed(optimizer, valuemap)
+
         new.cached_arrayitems = {}
         for descr, d in self.cached_arrayitems.items():
             newd = {}
@@ -60,10 +54,12 @@
                 newd[value.get_reconstructed(optimizer, valuemap)] = newcache
                 if cache.var_index_item:
                     newcache.var_index_item = \
-                          cache.var_index_item.get_reconstructed(optimizer, valuemap)
+                          cache.var_index_item.get_reconstructed(optimizer,
+                                                                 valuemap)
                 if cache.var_index_indexvalue:
                     newcache.var_index_indexvalue = \
-                          cache.var_index_indexvalue.get_reconstructed(optimizer, valuemap)
+                        cache.var_index_indexvalue.get_reconstructed(optimizer,
+                                                                     valuemap)
                 for index, fieldvalue in cache.fixed_index_items.items():
                     newcache.fixed_index_items[index] = \
                            fieldvalue.get_reconstructed(optimizer, valuemap)

diff --git a/pypy/jit/metainterp/optimizeopt/rewrite.py b/pypy/jit/metainterp/optimizeopt/rewrite.py
--- a/pypy/jit/metainterp/optimizeopt/rewrite.py
+++ b/pypy/jit/metainterp/optimizeopt/rewrite.py
@@ -13,7 +13,7 @@
        This includes already executed operations and constants.
     """
 
-    def clone_for_next_iteration(self, optimizer, valuemap):
+    def clone_for_next_iteration(self, surviving_boxes, optimizer, valuemap):
         return OptRewrite()
     
     def propagate_forward(self, op):

diff --git a/pypy/jit/metainterp/optimizeopt/optimizer.py b/pypy/jit/metainterp/optimizeopt/optimizer.py
--- a/pypy/jit/metainterp/optimizeopt/optimizer.py
+++ b/pypy/jit/metainterp/optimizeopt/optimizer.py
@@ -232,7 +232,8 @@
     def turned_constant(self, value):
         pass
 
-    def clone_for_next_iteration(self, optimizer=None, valuemap=None):
+    def clone_for_next_iteration(self, surviving_boxes=None,
+                                 optimizer=None, valuemap=None):
         raise NotImplementedError
     
 
@@ -311,14 +312,20 @@
         for o in self.optimizations:
             o.force_at_end_of_preamble()
             
-    def clone_for_next_iteration(self, optimizer=None, valuemap=None):
+    def clone_for_next_iteration(self, surviving_boxes=None,
+                                 optimizer=None, valuemap=None):
         assert optimizer is None
         assert valuemap is None
+        if surviving_boxes is None:
+            surviving_boxes = []
+        else:
+            surviving_boxes = surviving_boxes[:]
         valuemap = {}
         new = Optimizer(self.metainterp_sd, self.loop)
         new.values = {}
-        optimizations = [o.clone_for_next_iteration(new, valuemap) for o in 
-                         self.optimizations]
+        optimizations = [o.clone_for_next_iteration(surviving_boxes, 
+                                                    new, valuemap)
+                         for o in self.optimizations]
         new.set_optimizations(optimizations)
 
         # FIXME: new.interned_refs = self.interned_refs
@@ -338,14 +345,18 @@
             for i in range(len(newargs)):
                 if isinstance(newargs[i], OptValue):
                     newargs[i] = newargs[i].get_reconstructed(new, valuemap)
-            v = self.getvalue(op.result)
-            new.values[op.result] = v.get_reconstructed(new, valuemap)
+            surviving_boxes.append(op.result)
             new.pure_operations[newargs] = op
             # FIXME: This will not work for ops with mutable descr
             
         # FIXME: Any point in propagating these? new.producer = self.producer
+
         assert self.posponedop is None
 
+        for box in surviving_boxes:
+            v = self.getvalue(box)
+            new.values[box] = v.get_reconstructed(new, valuemap)
+
         return new
 
     def turned_constant(self, value):

diff --git a/pypy/jit/metainterp/test/test_optimizeopt.py b/pypy/jit/metainterp/test/test_optimizeopt.py
--- a/pypy/jit/metainterp/test/test_optimizeopt.py
+++ b/pypy/jit/metainterp/test/test_optimizeopt.py
@@ -694,9 +694,9 @@
         i1 = ptr_eq(p0, NULL)
         guard_false(i1) []
         i2 = ptr_ne(NULL, p0)
-        guard_true(i0) []
+        guard_true(i2) []
         i3 = ptr_eq(NULL, p0)
-        guard_false(i1) []
+        guard_false(i3) []
         guard_nonnull(p0) []
         jump(p0)
         """

diff --git a/pypy/jit/metainterp/optimizeopt/intbounds.py b/pypy/jit/metainterp/optimizeopt/intbounds.py
--- a/pypy/jit/metainterp/optimizeopt/intbounds.py
+++ b/pypy/jit/metainterp/optimizeopt/intbounds.py
@@ -13,7 +13,7 @@
         self.posponedop = None
         self.nextop = None
 
-    def clone_for_next_iteration(self, optimizer, valuemap):
+    def clone_for_next_iteration(self, surviving_boxes, optimizer, valuemap):
         assert self.posponedop is None
         return OptIntBounds()
 

diff --git a/pypy/jit/metainterp/optimizeopt/virtualize.py b/pypy/jit/metainterp/optimizeopt/virtualize.py
--- a/pypy/jit/metainterp/optimizeopt/virtualize.py
+++ b/pypy/jit/metainterp/optimizeopt/virtualize.py
@@ -252,7 +252,7 @@
 class OptVirtualize(optimizer.Optimization):
     "Virtualize objects until they escape."
 
-    def clone_for_next_iteration(self, optimizer, valuemap):
+    def clone_for_next_iteration(self, surviving_boxes, optimizer, valuemap):
         return OptVirtualize()
 
     def make_virtual(self, known_class, box, source_op=None):

From commits-noreply at bitbucket.org  Wed Feb  2 08:20:50 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Wed,  2 Feb 2011 08:20:50 +0100 (CET)
Subject: [pypy-svn] pypy jit-fromstart: whitespace
Message-ID: <20110202072050.C09472A2006@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-fromstart
Changeset: r41534:6cc9547bba28
Date: 2011-02-01 20:47 +0100
http://bitbucket.org/pypy/pypy/changeset/6cc9547bba28/

Log:	whitespace

diff --git a/pypy/jit/metainterp/optimizeopt/heap.py b/pypy/jit/metainterp/optimizeopt/heap.py
--- a/pypy/jit/metainterp/optimizeopt/heap.py
+++ b/pypy/jit/metainterp/optimizeopt/heap.py
@@ -44,7 +44,7 @@
             for value, fieldvalue in d.items():
                 newd[value.get_reconstructed(optimizer, valuemap)] = \
                              fieldvalue.get_reconstructed(optimizer, valuemap)
-
+        
         new.cached_arrayitems = {}
         for descr, d in self.cached_arrayitems.items():
             newd = {}

From commits-noreply at bitbucket.org  Wed Feb  2 08:20:51 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Wed,  2 Feb 2011 08:20:51 +0100 (CET)
Subject: [pypy-svn] pypy jit-fromstart: some interned support
Message-ID: <20110202072051.3FD2C2A2006@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-fromstart
Changeset: r41535:2a2796f9b086
Date: 2011-02-02 06:59 +0100
http://bitbucket.org/pypy/pypy/changeset/2a2796f9b086/

Log:	some interned support

diff --git a/pypy/jit/metainterp/optimizeopt/optimizer.py b/pypy/jit/metainterp/optimizeopt/optimizer.py
--- a/pypy/jit/metainterp/optimizeopt/optimizer.py
+++ b/pypy/jit/metainterp/optimizeopt/optimizer.py
@@ -353,8 +353,17 @@
 
         assert self.posponedop is None
 
+        for value in valuemap.values():
+            box = value.get_key_box()
+            assert box is new.getinterned(box)
+            try:
+                assert new.values[box] == value
+            except KeyError:
+                new.values[box] = value
+            
         for box in surviving_boxes:
             v = self.getvalue(box)
+            box = new.getinterned(box)
             new.values[box] = v.get_reconstructed(new, valuemap)
 
         return new

From commits-noreply at bitbucket.org  Wed Feb  2 08:20:52 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Wed,  2 Feb 2011 08:20:52 +0100 (CET)
Subject: [pypy-svn] pypy default: 2 failing and 1 passing test where
	inlining the short	preamble causes wrong behaviour
Message-ID: <20110202072052.AC0732A200E@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: 
Changeset: r41536:3318fd48d87b
Date: 2011-02-02 08:20 +0100
http://bitbucket.org/pypy/pypy/changeset/3318fd48d87b/

Log:	2 failing and 1 passing test where inlining the short preamble
	causes wrong behaviour

diff --git a/pypy/jit/metainterp/test/test_virtual.py b/pypy/jit/metainterp/test/test_virtual.py
--- a/pypy/jit/metainterp/test/test_virtual.py
+++ b/pypy/jit/metainterp/test/test_virtual.py
@@ -394,6 +394,89 @@
         #    ENTER             - compile the leaving path
         self.check_enter_count(2)
 
+    def test_stored_reference_with_bridge1(self):
+        class RefNode(object):
+            def __init__(self, ref):
+                self.ref = ref
+        myjitdriver = JitDriver(greens = [], reds = ['n', 'sa', 'node1', 'node2'])
+        def f(n):
+            node1 = self._new()
+            node1.value = 1
+            node2 = RefNode(node1)
+            sa = 0
+            while n > 0:
+                myjitdriver.can_enter_jit(n=n, sa=sa, node1=node1, node2=node2)
+                myjitdriver.jit_merge_point(n=n, sa=sa, node1=node1, node2=node2)
+                if n>10:
+                    next = self._new()
+                    next.value = 2
+                    node1 = next
+                else:
+                    node2.ref.value = 3
+                sa += node1.value
+                n -= 1
+            return sa
+        def g():
+            return  f(20) * 100 + f(10)
+        assert f(20) == 20 * 2
+        assert self.meta_interp(f, [20]) == 20 * 2
+        assert f(10) == 10 * 3
+        assert self.meta_interp(f, [10]) == 10 * 3
+        assert g() == 4030
+        assert self.meta_interp(g, []) == 4030
+
+    def test_stored_reference_with_bridge2(self):
+        class RefNode(object):
+            def __init__(self, ref):
+                self.ref = ref
+        myjitdriver = JitDriver(greens = [], reds = ['n', 'sa', 'node1', 'node2'])
+        def f(n):
+            node1 = self._new()
+            node1.value = 1
+            node2 = RefNode(node1)
+            sa = 0
+            while n > 0:
+                myjitdriver.can_enter_jit(n=n, sa=sa, node1=node1, node2=node2)
+                myjitdriver.jit_merge_point(n=n, sa=sa, node1=node1, node2=node2)
+                if n>10:
+                    next = self._new()
+                    next.value += 2
+                    node1 = next
+                else:
+                    node2.ref.value += 3
+                sa += node1.value
+                n -= 1
+            return sa
+        def g():
+            return  f(20) * 100 + f(10)
+        assert self.meta_interp(g, []) == g()
+
+    def test_stored_reference_with_bridge3(self):
+        class RefNode(object):
+            def __init__(self, ref):
+                self.ref = ref
+        myjitdriver = JitDriver(greens = [], reds = ['n', 'sa', 'node1', 'node2'])
+        def f(n):
+            node1 = self._new()
+            node1.value = 1
+            node2 = RefNode(node1)
+            sa = 0
+            while n > 0:
+                myjitdriver.can_enter_jit(n=n, sa=sa, node1=node1, node2=node2)
+                myjitdriver.jit_merge_point(n=n, sa=sa, node1=node1, node2=node2)
+                node2.ref.value += n
+                sa += node1.value
+                if n>10:
+                    next = self._new()
+                    next.value = node1.value + 1
+                    node1 = next
+                else:
+                    node1 = node2.ref
+                n -= 1
+            return sa
+        assert self.meta_interp(f, [20]) == f(20)
+
+
 class VirtualMiscTests:
 
     def test_guards_around_forcing(self):

From commits-noreply at bitbucket.org  Wed Feb  2 08:37:14 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Wed,  2 Feb 2011 08:37:14 +0100 (CET)
Subject: [pypy-svn] pypy default: typo
Message-ID: <20110202073714.43AE22A2006@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: 
Changeset: r41537:974e5556eb09
Date: 2011-02-02 08:36 +0100
http://bitbucket.org/pypy/pypy/changeset/974e5556eb09/

Log:	typo

diff --git a/pypy/jit/metainterp/test/test_virtual.py b/pypy/jit/metainterp/test/test_virtual.py
--- a/pypy/jit/metainterp/test/test_virtual.py
+++ b/pypy/jit/metainterp/test/test_virtual.py
@@ -440,7 +440,7 @@
                 myjitdriver.jit_merge_point(n=n, sa=sa, node1=node1, node2=node2)
                 if n>10:
                     next = self._new()
-                    next.value += 2
+                    next.value = node1.value + 2
                     node1 = next
                 else:
                     node2.ref.value += 3

From commits-noreply at bitbucket.org  Wed Feb  2 12:04:20 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed,  2 Feb 2011 12:04:20 +0100 (CET)
Subject: [pypy-svn] pypy default: Add rlib.debug.check_regular_int(),
	used in rbigint.fromint()
Message-ID: <20110202110420.6B08B2A2006@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41538:3db61138bdbd
Date: 2011-02-02 12:03 +0100
http://bitbucket.org/pypy/pypy/changeset/3db61138bdbd/

Log:	Add rlib.debug.check_regular_int(), used in rbigint.fromint() to
	check that we are really getting a regular integer.

diff --git a/pypy/rlib/debug.py b/pypy/rlib/debug.py
--- a/pypy/rlib/debug.py
+++ b/pypy/rlib/debug.py
@@ -233,6 +233,9 @@
 class UnexpectedRUInt(Exception):
     pass
 
+class ExpectedRegularInt(Exception):
+    pass
+
 def check_nonneg(x):
     """Give a translation-time error if 'x' is not known to be non-negative.
     To help debugging, this also gives a translation-time error if 'x' is
@@ -259,3 +262,23 @@
     def specialize_call(self, hop):
         hop.exception_cannot_occur()
         return hop.inputarg(hop.args_r[0], arg=0)
+
+def check_regular_int(x):
+    """Give a translation-time error if 'x' is not a plain int
+    (e.g. if it's a r_longlong or an r_uint).
+    """
+    assert type(x) is int
+    return x
+
+class Entry(ExtRegistryEntry):
+    _about_ = check_regular_int
+
+    def compute_result_annotation(self, s_arg):
+        from pypy.annotation.model import SomeInteger
+        if not SomeInteger().contains(s_arg):
+            raise ExpectedRegularInt(s_arg)
+        return s_arg
+
+    def specialize_call(self, hop):
+        hop.exception_cannot_occur()
+        return hop.inputarg(hop.args_r[0], arg=0)

diff --git a/pypy/rlib/rbigint.py b/pypy/rlib/rbigint.py
--- a/pypy/rlib/rbigint.py
+++ b/pypy/rlib/rbigint.py
@@ -1,7 +1,7 @@
 from pypy.rlib.rarithmetic import LONG_BIT, intmask, r_uint, r_ulonglong
 from pypy.rlib.rarithmetic import ovfcheck, r_longlong, widen, isinf, isnan
 from pypy.rlib.rarithmetic import most_neg_value_of_same_type
-from pypy.rlib.debug import make_sure_not_resized
+from pypy.rlib.debug import make_sure_not_resized, check_regular_int
 from pypy.rlib.objectmodel import we_are_translated
 
 import math, sys
@@ -71,6 +71,7 @@
         return len(self.digits)
 
     def fromint(intval):
+        check_regular_int(intval)
         if intval < 0:
             sign = -1
             ival = r_uint(-intval)

From commits-noreply at bitbucket.org  Wed Feb  2 13:49:37 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Wed,  2 Feb 2011 13:49:37 +0100 (CET)
Subject: [pypy-svn] pypy arm-backed-float: Implement FLOAT_MUL operation
Message-ID: <20110202124937.C84222A2006@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backed-float
Changeset: r41539:2a270aeaa3dc
Date: 2011-01-17 17:46 +0100
http://bitbucket.org/pypy/pypy/changeset/2a270aeaa3dc/

Log:	Implement FLOAT_MUL operation

diff --git a/pypy/jit/backend/arm/instructions.py b/pypy/jit/backend/arm/instructions.py
--- a/pypy/jit/backend/arm/instructions.py
+++ b/pypy/jit/backend/arm/instructions.py
@@ -129,6 +129,7 @@
 # based on encoding from A7.5	VFP data-processing instructions
 # opc2 is one of the parameters and therefore ignored here
 float64_data_proc_instructions = {
-    'VADD': {'opc1':0x3, 'opc3':0},
-    'VSUB': {'opc1':0x3, 'opc3':1},
+    'VADD' : {'opc1':0x3, 'opc3':0},
+    'VSUB' : {'opc1':0x3, 'opc3':1},
+    'VMUL' : {'opc1':0x2, 'opc3':0},
 }

diff --git a/pypy/jit/backend/arm/opassembler.py b/pypy/jit/backend/arm/opassembler.py
--- a/pypy/jit/backend/arm/opassembler.py
+++ b/pypy/jit/backend/arm/opassembler.py
@@ -8,7 +8,9 @@
 
 from pypy.jit.backend.arm.helper.assembler import (gen_emit_op_by_helper_call,
                                                     gen_emit_op_unary_cmp,
-                                                    gen_emit_op_ri, gen_emit_cmp_op)
+                                                    gen_emit_op_ri,
+                                                    gen_emit_cmp_op,
+                                                    gen_emit_float_op)
 from pypy.jit.backend.arm.codebuilder import ARMv7Builder, OverwritingBuilder
 from pypy.jit.backend.arm.jump import remap_frame_layout
 from pypy.jit.backend.arm.regalloc import Regalloc
@@ -772,13 +774,9 @@
 class FloatOpAssemlber(object):
     _mixin_ = True
 
-    def emit_op_float_add(self, op, arglocs, regalloc, fcon):
-        arg1, arg2, result = arglocs
-        self.mc.VADD(result.value, arg1.value, arg2.value)
-
-    def emit_op_float_sub(self, op, arglocs, regalloc, fcon):
-        arg1, arg2, result = arglocs
-        self.mc.VSUB(result.value, arg1.value, arg2.value)
+    emit_op_float_add = gen_emit_float_op('VADD')
+    emit_op_float_sub = gen_emit_float_op('VSUB')
+    emit_op_float_mul = gen_emit_float_op('VMUL')
 
 class ResOpAssembler(GuardOpAssembler, IntOpAsslember,
                     OpAssembler, UnaryIntOpAssembler,

diff --git a/pypy/jit/backend/arm/helper/assembler.py b/pypy/jit/backend/arm/helper/assembler.py
--- a/pypy/jit/backend/arm/helper/assembler.py
+++ b/pypy/jit/backend/arm/helper/assembler.py
@@ -49,3 +49,10 @@
         self.mc.MOV_ri(res.value, 0, cond=inv)
         return fcond
     return f
+
+def gen_emit_float_op(opname):
+    op_rr = getattr(AbstractARMv7Builder, opname)
+    def f(self, op, arglocs, regalloc, fcon):
+        arg1, arg2, result = arglocs
+        op_rr(self.mc, result.value, arg1.value, arg2.value)
+    return f

diff --git a/pypy/jit/backend/arm/regalloc.py b/pypy/jit/backend/arm/regalloc.py
--- a/pypy/jit/backend/arm/regalloc.py
+++ b/pypy/jit/backend/arm/regalloc.py
@@ -784,6 +784,7 @@
 
     prepare_op_float_add = prepare_float_op()
     prepare_op_float_sub = prepare_float_op()
+    prepare_op_float_mul = prepare_float_op()
 
 def make_operation_list():
     def notimplemented(self, op, fcond):

From commits-noreply at bitbucket.org  Wed Feb  2 13:49:38 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Wed,  2 Feb 2011 13:49:38 +0100 (CET)
Subject: [pypy-svn] pypy arm-backed-float: Implement FLOAT_TRUEDIV operation
Message-ID: <20110202124938.E20432A2006@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backed-float
Changeset: r41540:9c673870557c
Date: 2011-01-17 17:54 +0100
http://bitbucket.org/pypy/pypy/changeset/9c673870557c/

Log:	Implement FLOAT_TRUEDIV operation

diff --git a/pypy/jit/backend/arm/instructions.py b/pypy/jit/backend/arm/instructions.py
--- a/pypy/jit/backend/arm/instructions.py
+++ b/pypy/jit/backend/arm/instructions.py
@@ -132,4 +132,5 @@
     'VADD' : {'opc1':0x3, 'opc3':0},
     'VSUB' : {'opc1':0x3, 'opc3':1},
     'VMUL' : {'opc1':0x2, 'opc3':0},
+    'VDIV' : {'opc1':0x8, 'opc3':0},
 }

diff --git a/pypy/jit/backend/arm/opassembler.py b/pypy/jit/backend/arm/opassembler.py
--- a/pypy/jit/backend/arm/opassembler.py
+++ b/pypy/jit/backend/arm/opassembler.py
@@ -777,6 +777,7 @@
     emit_op_float_add = gen_emit_float_op('VADD')
     emit_op_float_sub = gen_emit_float_op('VSUB')
     emit_op_float_mul = gen_emit_float_op('VMUL')
+    emit_op_float_truediv = gen_emit_float_op('VDIV')
 
 class ResOpAssembler(GuardOpAssembler, IntOpAsslember,
                     OpAssembler, UnaryIntOpAssembler,

diff --git a/pypy/jit/backend/arm/regalloc.py b/pypy/jit/backend/arm/regalloc.py
--- a/pypy/jit/backend/arm/regalloc.py
+++ b/pypy/jit/backend/arm/regalloc.py
@@ -785,6 +785,7 @@
     prepare_op_float_add = prepare_float_op()
     prepare_op_float_sub = prepare_float_op()
     prepare_op_float_mul = prepare_float_op()
+    prepare_op_float_truediv = prepare_float_op()
 
 def make_operation_list():
     def notimplemented(self, op, fcond):

From commits-noreply at bitbucket.org  Wed Feb  2 13:49:39 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Wed,  2 Feb 2011 13:49:39 +0100 (CET)
Subject: [pypy-svn] pypy arm-backed-float: mege
Message-ID: <20110202124939.3632F2A2008@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backed-float
Changeset: r41541:3796ea4bca06
Date: 2011-01-17 18:00 +0100
http://bitbucket.org/pypy/pypy/changeset/3796ea4bca06/

Log:	mege


From commits-noreply at bitbucket.org  Wed Feb  2 13:49:40 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Wed,  2 Feb 2011 13:49:40 +0100 (CET)
Subject: [pypy-svn] pypy arm-backed-float: merge
Message-ID: <20110202124940.A4BD82A2017@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backed-float
Changeset: r41542:0a490c4f0725
Date: 2010-10-08 01:38 +0100
http://bitbucket.org/pypy/pypy/changeset/0a490c4f0725/

Log:	merge

diff --git a/pypy/jit/backend/arm/assembler.py b/pypy/jit/backend/arm/assembler.py
--- a/pypy/jit/backend/arm/assembler.py
+++ b/pypy/jit/backend/arm/assembler.py
@@ -273,7 +273,7 @@
                     mem[j] = chr(loc.value)
                     j += 1
                 elif loc.is_imm():
-                    assert arg.type == INT
+                    assert arg.type == INT or arg.type == REF
                     mem[j] = self.IMM_LOC
                     self.encode32(mem, j+1, loc.getint())
                     j += 5

From commits-noreply at bitbucket.org  Wed Feb  2 13:49:42 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Wed,  2 Feb 2011 13:49:42 +0100 (CET)
Subject: [pypy-svn] pypy arm-backed-float: Add test case for casting a
	negative number to	a float
Message-ID: <20110202124942.411862A2017@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backed-float
Changeset: r41543:45e0d4ab7b4a
Date: 2011-01-18 18:07 +0100
http://bitbucket.org/pypy/pypy/changeset/45e0d4ab7b4a/

Log:	Add test case for casting a negative number to a float

diff --git a/pypy/jit/metainterp/test/test_executor.py b/pypy/jit/metainterp/test/test_executor.py
--- a/pypy/jit/metainterp/test/test_executor.py
+++ b/pypy/jit/metainterp/test/test_executor.py
@@ -277,6 +277,7 @@
     yield (rop.CAST_FLOAT_TO_INT, [-5.9], 'int', -5)
     yield (rop.CAST_FLOAT_TO_INT, [5.9], 'int', 5)
     yield (rop.CAST_INT_TO_FLOAT, [123], 'float', 123.0)
+    yield (rop.CAST_INT_TO_FLOAT, [-123], 'float', -123.0)
 
 def get_float_tests(cpu):
     if not cpu.supports_floats:

From commits-noreply at bitbucket.org  Wed Feb  2 13:49:47 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Wed,  2 Feb 2011 13:49:47 +0100 (CET)
Subject: [pypy-svn] pypy arm-backed-float: Finish implementing float
	operations
Message-ID: <20110202124947.B35252A2008@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backed-float
Changeset: r41544:5c8e9dd21de5
Date: 2011-01-18 18:08 +0100
http://bitbucket.org/pypy/pypy/changeset/5c8e9dd21de5/

Log:	Finish implementing float operations

diff --git a/pypy/jit/backend/arm/instruction_builder.py b/pypy/jit/backend/arm/instruction_builder.py
--- a/pypy/jit/backend/arm/instruction_builder.py
+++ b/pypy/jit/backend/arm/instruction_builder.py
@@ -303,15 +303,36 @@
     n = (0xE << 24
         | 0x5 << 9
         | 0x1 << 8 # 64 bit flag
-        | (table['opc1'] & 0xF) << 20
         | (table['opc3'] & 0x3) << 6)
-    def f(self, dd, dn, dm, cond=cond.AL):
-        instr = (n
-                | (cond & 0xF) << 28
-                | (dn & 0xF) << 16
-                | (dd & 0xF) << 12
-                | (dm & 0xF))
-        self.write32(instr)
+
+    if 'opc1' in table:
+        n |= (table['opc1'] & 0xF) << 20
+    if 'opc2' in table:
+        n |= (table['opc2'] & 0xF) << 16
+
+    if 'result' in table and not table['result']:
+        def f(self, dd, dm, cond=cond.AL):
+            instr = (n
+                    | (cond & 0xF) << 28
+                    | 0x4 << 16
+                    | (dd & 0xF) << 12
+                    | (dm & 0xF))
+            self.write32(instr)
+    elif 'base' in table and not table['base']:
+        def f(self, dd, dm, cond=cond.AL):
+            instr = (n
+                    | (cond & 0xF) << 28
+                    | (dd & 0xF) << 12
+                    | (dm & 0xF))
+            self.write32(instr)
+    else:
+        def f(self, dd, dn, dm, cond=cond.AL):
+            instr = (n
+                    | (cond & 0xF) << 28
+                    | (dn & 0xF) << 16
+                    | (dd & 0xF) << 12
+                    | (dm & 0xF))
+            self.write32(instr)
     return f
 
 def imm_operation(rt, rn, imm):

diff --git a/pypy/jit/backend/arm/instructions.py b/pypy/jit/backend/arm/instructions.py
--- a/pypy/jit/backend/arm/instructions.py
+++ b/pypy/jit/backend/arm/instructions.py
@@ -129,8 +129,12 @@
 # based on encoding from A7.5	VFP data-processing instructions
 # opc2 is one of the parameters and therefore ignored here
 float64_data_proc_instructions = {
-    'VADD' : {'opc1':0x3, 'opc3':0},
-    'VSUB' : {'opc1':0x3, 'opc3':1},
-    'VMUL' : {'opc1':0x2, 'opc3':0},
-    'VDIV' : {'opc1':0x8, 'opc3':0},
+    'VADD' : {'opc1':0x3, 'opc3':0x0},
+    'VSUB' : {'opc1':0x3, 'opc3':0x1},
+    'VMUL' : {'opc1':0x2, 'opc3':0x0},
+    'VDIV' : {'opc1':0x8, 'opc3':0x0},
+    'VCMP' : {'opc1':0xB, 'opc2':0x4, 'opc3':0x1, 'result': False},
+    'VNEG' : {'opc1':0xB, 'opc2':0x1, 'opc3':0x1, 'base': False},
+    'VABS' : {'opc1':0xB, 'opc2':0x0, 'opc3':0x3, 'base': False},
+    #'VCVT' : {'opc1':0xB, 'opc2':0xE, 'opc3':0x1, 'base': False},
 }

diff --git a/pypy/jit/backend/arm/test/test_instr_codebuilder.py b/pypy/jit/backend/arm/test/test_instr_codebuilder.py
--- a/pypy/jit/backend/arm/test/test_instr_codebuilder.py
+++ b/pypy/jit/backend/arm/test/test_instr_codebuilder.py
@@ -152,6 +152,10 @@
     def test_vstr_offset(self):
         assert py.test.raises(AssertionError, 'self.cb.VSTR(r.d1, r.r4, 3)')
 
+    def test_vmrs(self):
+        self.cb.VMRS(conditions.AL)
+        self.assert_equal("vmrs APSR_nzcv, fpscr")
+
     def test_pop_raises_on_lr(self):
         assert py.test.raises(AssertionError, 'self.cb.POP([r.lr.value])')
 
@@ -175,8 +179,15 @@
     tests = []
     for c,v in [('EQ', conditions.EQ), ('LE', conditions.LE), ('AL', conditions.AL)]:
         for reg in range(16):
-            asm = 'd%d, d1, d2' % reg
-            tests.append((asm, (reg, r.d1.value, r.d2.value), {}, '.F64'))
+            if 'result' in table and not table['result']:
+                asm = 'd%d, d2' % reg
+                tests.append((asm, (reg, r.d2.value), {}, '.F64'))
+            elif 'base' in table and not table['base']:
+                asm = 'd%d, d2' % reg
+                tests.append((asm, (reg, r.d2.value), {}, '.F64'))
+            else:
+                asm = 'd%d, d1, d2' % reg
+                tests.append((asm, (reg, r.d1.value, r.d2.value), {}, '.F64'))
     return tests
 
 def gen_test_data_proc_imm_func(name, table):

diff --git a/pypy/jit/backend/arm/codebuilder.py b/pypy/jit/backend/arm/codebuilder.py
--- a/pypy/jit/backend/arm/codebuilder.py
+++ b/pypy/jit/backend/arm/codebuilder.py
@@ -44,20 +44,60 @@
         self.write32(instr)
 
     def VPUSH(self, regs, cond=cond.AL):
-        nregs = len(regs) 
+        nregs = len(regs)
         assert nregs > 0 and nregs <= 16
         freg = regs[0]
         D = (freg & 0x10) >> 4
         Dd = (freg & 0xF)
-        nregs *= 2 
-        instr = (cond << 28 
-                | 0xD2D << 16 
-                | D << 22 
+        nregs *= 2
+        instr = (cond << 28
+                | 0xD2D << 16
+                | D << 22
                 | Dd << 12
                 | 0xB << 8
                 | nregs)
         self.write32(instr)
 
+    def VPOP(self, regs, cond=cond.AL):
+        nregs = len(regs)
+        assert nregs > 0 and nregs <= 16
+        freg = regs[0]
+        D = (freg & 0x10) >> 4
+        Dd = (freg & 0xF)
+        nregs *= 2
+        instr = (cond << 28
+                | 0xCBD << 16
+                | D << 22
+                | Dd << 12
+                | 0xB << 8
+                | nregs)
+        self.write32(instr)
+
+    def VCVT_float_to_int(self, target, source, cond=cond.AL):
+        opc2 = 0x5
+        sz = 1
+        self._VCVT(target, source, cond, opc2, sz)
+
+    def VCVT_int_to_float(self, target, source, cond=cond.AL):
+        self._VCVT(target, source, cond, 0, 1)
+
+    def _VCVT(self, target, source, cond, opc2, sz):
+        D = 0x0
+        M = 0
+        op = 1
+        instr = (cond << 28
+                | 0xEB8 << 16
+                | D << 22
+                | opc2 << 16
+                | (target & 0xF) << 12
+                | 0x5 << 9
+                | sz << 8
+                | op << 7
+                | 1 << 6
+                | M << 5
+                | (source & 0xF))
+        self.write32(instr)
+
     def POP(self, regs, cond=cond.AL):
         assert reg.lr.value not in regs
         instr = self._encode_reg_list(cond << 28 | 0x8BD << 16, regs)
@@ -66,6 +106,10 @@
     def BKPT(self, cond=cond.AL):
         self.write32(cond << 28 | 0x1200070)
 
+    # corresponds to the instruction vmrs APSR_nzcv, fpscr
+    def VMRS(self, cond=cond.AL):
+        self.write32(cond << 28 | 0xEF1FA10)
+
     def B(self, target, c=cond.AL):
         if c == cond.AL:
             self.LDR_ri(reg.pc.value, reg.pc.value, -arch.PC_OFFSET/2)

diff --git a/pypy/jit/backend/arm/registers.py b/pypy/jit/backend/arm/registers.py
--- a/pypy/jit/backend/arm/registers.py
+++ b/pypy/jit/backend/arm/registers.py
@@ -1,10 +1,11 @@
-from pypy.jit.backend.arm.locations import RegisterLocation
+from pypy.jit.backend.arm.locations import RegisterLocation, VFPRegisterLocation
 
 registers = [RegisterLocation(i) for i in range(16)]
+vfpregisters = [VFPRegisterLocation(i) for i in range(16)]
 r0, r1, r2, r3, r4, r5, r6, r7, r8, r9, r10, r11, r12, r13, r14, r15 = registers
 
 #vfp registers interpreted as 64-bit registers
-d0, d1, d2, d3, d4, d5, d6, d7, d8, d9, d10, d11, d12, d13, d14, d15 = registers
+d0, d1, d2, d3, d4, d5, d6, d7, d8, d9, d10, d11, d12, d13, d14, d15 = vfpregisters
 
 # aliases for registers
 fp = r11
@@ -14,7 +15,7 @@
 pc = r15
 
 all_regs = [r0, r1, r2, r3, r4, r5, r6, r7, r8, r9, r10]
-all_vfp_regs = registers
+all_vfp_regs = vfpregisters
 
 caller_resp = [r0, r1, r2, r3]
 callee_resp = [r4, r5, r6, r7, r8, r9, r10, fp]

diff --git a/pypy/jit/backend/arm/assembler.py b/pypy/jit/backend/arm/assembler.py
--- a/pypy/jit/backend/arm/assembler.py
+++ b/pypy/jit/backend/arm/assembler.py
@@ -139,7 +139,7 @@
             i += 1
             res = enc[i]
             if res == self.IMM_LOC:
-                assert group == self.INT_TYPE
+                assert group == self.INT_TYPE or group == self.REF_TYPE
                 # imm value
                 value = self.decode32(enc, i+1)
                 i += 4
@@ -210,7 +210,7 @@
         low = self.decode32(mem, index)
         index += 4
         high = self.decode32(mem, index)
-        return r_longlong(r_uint(low) | (r_longlong(high << 32)))
+        return r_longlong(high << 32) | r_longlong(r_uint(low))
 
     def encode32(self, mem, i, n):
         mem[i] = chr(n & 0xFF)
@@ -273,7 +273,9 @@
                     mem[j] = chr(loc.value)
                     j += 1
                 elif loc.is_imm():
-                    assert arg.type == INT or arg.type == REF
+                    if not arg.type == INT or arg.type == REF:
+                        print "Expected INT or REF values"
+                        assert 0
                     mem[j] = self.IMM_LOC
                     self.encode32(mem, j+1, loc.getint())
                     j += 5

diff --git a/pypy/jit/backend/arm/opassembler.py b/pypy/jit/backend/arm/opassembler.py
--- a/pypy/jit/backend/arm/opassembler.py
+++ b/pypy/jit/backend/arm/opassembler.py
@@ -10,7 +10,9 @@
                                                     gen_emit_op_unary_cmp,
                                                     gen_emit_op_ri,
                                                     gen_emit_cmp_op,
-                                                    gen_emit_float_op)
+                                                    gen_emit_float_op,
+                                                    gen_emit_float_cmp_op,
+                                                    gen_emit_unary_float_op)
 from pypy.jit.backend.arm.codebuilder import ARMv7Builder, OverwritingBuilder
 from pypy.jit.backend.arm.jump import remap_frame_layout
 from pypy.jit.backend.arm.regalloc import Regalloc
@@ -779,6 +781,29 @@
     emit_op_float_mul = gen_emit_float_op('VMUL')
     emit_op_float_truediv = gen_emit_float_op('VDIV')
 
+    emit_op_float_neg = gen_emit_unary_float_op('VNEG')
+    emit_op_float_abs = gen_emit_unary_float_op('VABS')
+
+    emit_op_float_lt = gen_emit_float_cmp_op(c.LT)
+    emit_op_float_le = gen_emit_float_cmp_op(c.LE)
+    emit_op_float_eq = gen_emit_float_cmp_op(c.EQ)
+    emit_op_float_ne = gen_emit_float_cmp_op(c.NE)
+    emit_op_float_gt = gen_emit_float_cmp_op(c.GT)
+    emit_op_float_ge = gen_emit_float_cmp_op(c.GE)
+
+    def emit_op_cast_float_to_int(self, op, arglocs, regalloc, fcond):
+        arg, temp, res = arglocs
+        self.mc.VCVT_float_to_int(temp.value, arg.value)
+        self.mc.VPUSH([temp.value])
+        # res is lower register than r.ip
+        self.mc.POP([res.value, r.ip.value])
+
+    def emit_op_cast_int_to_float(self, op, arglocs, regalloc, fcond):
+        arg, temp, res = arglocs
+        self.mc.PUSH([arg.value, r.ip.value])
+        self.mc.VPOP([temp.value])
+        self.mc.VCVT_int_to_float(res.value, temp.value)
+
 class ResOpAssembler(GuardOpAssembler, IntOpAsslember,
                     OpAssembler, UnaryIntOpAssembler,
                     FieldOpAssembler, ArrayOpAssember,

diff --git a/pypy/jit/backend/arm/helper/assembler.py b/pypy/jit/backend/arm/helper/assembler.py
--- a/pypy/jit/backend/arm/helper/assembler.py
+++ b/pypy/jit/backend/arm/helper/assembler.py
@@ -56,3 +56,20 @@
         arg1, arg2, result = arglocs
         op_rr(self.mc, result.value, arg1.value, arg2.value)
     return f
+def gen_emit_unary_float_op(opname):
+    op_rr = getattr(AbstractARMv7Builder, opname)
+    def f(self, op, arglocs, regalloc, fcon):
+        arg1, result = arglocs
+        op_rr(self.mc, result.value, arg1.value)
+    return f
+
+def gen_emit_float_cmp_op(cond):
+    def f(self, op, arglocs, regalloc, fcond):
+        arg1, arg2, res = arglocs
+        inv = c.get_opposite_of(cond)
+        self.mc.VCMP(arg1.value, arg2.value)
+        self.mc.VMRS(cond=fcond)
+        self.mc.MOV_ri(res.value, 1, cond=cond)
+        self.mc.MOV_ri(res.value, 0, cond=inv)
+        return fcond
+    return f

diff --git a/pypy/jit/backend/arm/helper/regalloc.py b/pypy/jit/backend/arm/helper/regalloc.py
--- a/pypy/jit/backend/arm/helper/regalloc.py
+++ b/pypy/jit/backend/arm/helper/regalloc.py
@@ -51,15 +51,24 @@
         return [l0, l1, res]
     return f
 
-def prepare_float_op():
+def prepare_float_op(base=True, float_result=True):
     def f(self, op, fcond):
+        locs = []
         loc1, box1 = self._ensure_value_is_boxed(op.getarg(0))
-        loc2, box2 = self._ensure_value_is_boxed(op.getarg(1))
+        locs.append(loc1)
         self.vfprm.possibly_free_var(box1)
-        self.vfprm.possibly_free_var(box2)
-        res  = self.vfprm.force_allocate_reg(op.result)
-        self.vfprm.possibly_free_var(op.result)
-        return [loc1, loc2, res]
+        if base:
+            loc2, box2 = self._ensure_value_is_boxed(op.getarg(1))
+            locs.append(loc2)
+            self.vfprm.possibly_free_var(box2)
+        if float_result:
+            res  = self.vfprm.force_allocate_reg(op.result)
+            self.vfprm.possibly_free_var(op.result)
+        else:
+            res  = self.rm.force_allocate_reg(op.result)
+            self.rm.possibly_free_var(op.result)
+        locs.append(res)
+        return locs
     return f
 
 def prepare_op_by_helper_call():

diff --git a/pypy/jit/backend/arm/locations.py b/pypy/jit/backend/arm/locations.py
--- a/pypy/jit/backend/arm/locations.py
+++ b/pypy/jit/backend/arm/locations.py
@@ -33,6 +33,15 @@
     def as_key(self):
         return self.value
 
+class VFPRegisterLocation(RegisterLocation):
+    _immutable_ = True
+
+    def get_single_precision_regs(self):
+        return [VFPRegisterLocation(i) for i in [self.value*2, self.value*2+1]]
+
+    def __repr__(self):
+        return 'f%d' % self.value
+
 class ImmLocation(AssemblerLocation):
     _immutable_ = True
 

diff --git a/pypy/jit/backend/arm/regalloc.py b/pypy/jit/backend/arm/regalloc.py
--- a/pypy/jit/backend/arm/regalloc.py
+++ b/pypy/jit/backend/arm/regalloc.py
@@ -53,7 +53,7 @@
 class VFPRegisterManager(RegisterManager):
     all_regs = r.all_vfp_regs
     box_types = [FLOAT]
-    save_around_call_regs = all_regs
+    save_around_call_regs = r.all_vfp_regs
 
     def convert_to_imm(self, c):
         adr = self.assembler.datablockwrapper.malloc_aligned(8, 8)
@@ -786,6 +786,50 @@
     prepare_op_float_sub = prepare_float_op()
     prepare_op_float_mul = prepare_float_op()
     prepare_op_float_truediv = prepare_float_op()
+    prepare_op_float_lt = prepare_float_op(float_result=False)
+    prepare_op_float_le = prepare_float_op(float_result=False)
+    prepare_op_float_eq = prepare_float_op(float_result=False)
+    prepare_op_float_ne = prepare_float_op(float_result=False)
+    prepare_op_float_gt = prepare_float_op(float_result=False)
+    prepare_op_float_ge = prepare_float_op(float_result=False)
+    prepare_op_float_neg = prepare_float_op(base=False)
+    prepare_op_float_abs = prepare_float_op(base=False)
+
+    def prepare_op_cast_float_to_int(self, op, fcond):
+        locs = []
+
+        loc1, box1 = self._ensure_value_is_boxed(op.getarg(0))
+        locs.append(loc1)
+        self.possibly_free_var(box1)
+
+        t = TempFloat()
+        temp_loc = self.vfprm.force_allocate_reg(t)
+        locs.append(temp_loc)
+        self.possibly_free_var(t)
+
+        res  = self.rm.force_allocate_reg(op.result)
+        self.possibly_free_var(op.result)
+        locs.append(res)
+
+        return locs
+
+    def prepare_op_cast_int_to_float(self, op, fcond):
+        locs = []
+
+        loc1, box1 = self._ensure_value_is_boxed(op.getarg(0))
+        locs.append(loc1)
+        self.possibly_free_var(box1)
+
+        t = TempFloat()
+        temp_loc = self.vfprm.force_allocate_reg(t)
+        locs.append(temp_loc)
+        self.possibly_free_var(t)
+
+        res  = self.vfprm.force_allocate_reg(op.result)
+        self.possibly_free_var(op.result)
+        locs.append(res)
+
+        return locs
 
 def make_operation_list():
     def notimplemented(self, op, fcond):

From commits-noreply at bitbucket.org  Wed Feb  2 13:49:48 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Wed,  2 Feb 2011 13:49:48 +0100 (CET)
Subject: [pypy-svn] pypy arm-backed-float: return condition
Message-ID: <20110202124948.B31BD2A2008@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backed-float
Changeset: r41545:b966311db125
Date: 2011-01-18 18:26 +0100
http://bitbucket.org/pypy/pypy/changeset/b966311db125/

Log:	return condition

diff --git a/pypy/jit/backend/arm/assembler.py b/pypy/jit/backend/arm/assembler.py
--- a/pypy/jit/backend/arm/assembler.py
+++ b/pypy/jit/backend/arm/assembler.py
@@ -181,7 +181,8 @@
                 j += 1
                 res = enc[j]
 
-            assert res in [self.INT_TYPE, self.REF_TYPE], 'location type is not supported'
+            assert res in [self.FLOAT_TYPE, self.INT_TYPE, self.REF_TYPE], 'location type is not supported'
+            res_type = res
             j += 1
             res = enc[j]
             if res == self.IMM_LOC:
@@ -192,7 +193,10 @@
                 loc = regalloc.frame_manager.frame_pos(stack_loc, INT)
                 j += 4
             else: # REG_LOC
-                loc = r.all_regs[ord(res)]
+                if res_type == self.FLOAT_TYPE:
+                    loc = r.all_vfp_regs[ord(res)]
+                else:
+                    loc = r.all_regs[ord(res)]
             j += 1
             locs.append(loc)
         return locs

diff --git a/pypy/jit/backend/arm/opassembler.py b/pypy/jit/backend/arm/opassembler.py
--- a/pypy/jit/backend/arm/opassembler.py
+++ b/pypy/jit/backend/arm/opassembler.py
@@ -797,12 +797,14 @@
         self.mc.VPUSH([temp.value])
         # res is lower register than r.ip
         self.mc.POP([res.value, r.ip.value])
+        return fcond
 
     def emit_op_cast_int_to_float(self, op, arglocs, regalloc, fcond):
         arg, temp, res = arglocs
         self.mc.PUSH([arg.value, r.ip.value])
         self.mc.VPOP([temp.value])
         self.mc.VCVT_int_to_float(res.value, temp.value)
+        return fcond
 
 class ResOpAssembler(GuardOpAssembler, IntOpAsslember,
                     OpAssembler, UnaryIntOpAssembler,

From commits-noreply at bitbucket.org  Wed Feb  2 13:49:56 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Wed,  2 Feb 2011 13:49:56 +0100 (CET)
Subject: [pypy-svn] pypy arm-backend-2: merge default
Message-ID: <20110202124956.954522A2011@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backend-2
Changeset: r41546:1e9a43942d1d
Date: 2011-01-20 17:05 +0100
http://bitbucket.org/pypy/pypy/changeset/1e9a43942d1d/

Log:	merge default

diff --git a/pypy/config/translationoption.py b/pypy/config/translationoption.py
--- a/pypy/config/translationoption.py
+++ b/pypy/config/translationoption.py
@@ -388,8 +388,9 @@
         else:
             raise ValueError(word)
 
-    hasbackendopts = 'nobackendopt' not in words
-    config.translation.suggest(list_comprehension_operations=hasbackendopts)
+    # list_comprehension_operations is needed for translation, because
+    # make_sure_not_resized often relies on it, so we always enable them
+    config.translation.suggest(list_comprehension_operations=True)
 
 # ----------------------------------------------------------------
 

diff --git a/pypy/objspace/std/test/test_dictmultiobject.py b/pypy/objspace/std/test/test_dictmultiobject.py
--- a/pypy/objspace/std/test/test_dictmultiobject.py
+++ b/pypy/objspace/std/test/test_dictmultiobject.py
@@ -667,13 +667,10 @@
 class AppTestModuleDict(object):
     def setup_class(cls):
         cls.space = gettestobjspace(**{"objspace.std.withcelldict": True})
-        cls.w_impl_used = cls.space.appexec([], """():
-            import __pypy__
-            def impl_used(obj):
-                assert "ModuleDictImplementation" in __pypy__.internal_repr(obj)
-            return impl_used
-        """)
 
+    def w_impl_used(self, obj):
+        import __pypy__
+        assert "ModuleDictImplementation" in __pypy__.internal_repr(obj)
 
     def test_check_module_uses_module_dict(self):
         m = type(__builtins__)("abc")

diff --git a/pypy/conftest.py b/pypy/conftest.py
--- a/pypy/conftest.py
+++ b/pypy/conftest.py
@@ -128,7 +128,7 @@
 
         for name in ('int', 'long', 'str', 'unicode'):
             setattr(self, 'w_' + name, eval(name))
-        
+
 
     def appexec(self, args, body):
         body = body.lstrip()
@@ -294,16 +294,6 @@
     py.test.skip("need translated pypy with: %s, got %s"
                  %(ropts,options))
 
-def getwithoutbinding(x, name):
-    try:
-        return x.__dict__[name]
-    except (AttributeError, KeyError):
-        for cls in getmro(x.__class__):
-            if name in cls.__dict__:
-                return cls.__dict__[name]
-        # uh? not found anywhere, fall back (which might raise AttributeError)
-        return getattr(x, name)
-
 class LazyObjSpaceGetter(object):
     def __get__(self, obj, cls=None):
         space = gettestobjspace()
@@ -429,10 +419,7 @@
         for name in dir(instance):
             if name.startswith('w_'):
                 if option.runappdirect:
-                    # if the value is a function living on the class,
-                    # don't turn it into a bound method here
-                    obj = getwithoutbinding(instance, name)
-                    setattr(instance, name[2:], obj)
+                    setattr(instance, name[2:], getattr(instance, name))
                 else:
                     obj = getattr(instance, name)
                     if isinstance(obj, types.MethodType):

diff --git a/pypy/translator/platform/__init__.py b/pypy/translator/platform/__init__.py
--- a/pypy/translator/platform/__init__.py
+++ b/pypy/translator/platform/__init__.py
@@ -176,8 +176,11 @@
     def _finish_linking(self, ofiles, eci, outputfilename, standalone):
         if outputfilename is None:
             outputfilename = ofiles[0].purebasename
-        exe_name = py.path.local(os.path.join(str(ofiles[0].dirpath()),
-                                              outputfilename))
+        if ofiles:
+            dirname = ofiles[0].dirpath()
+        else:
+            dirname = udir.join('module_cache')
+        exe_name = dirname.join(outputfilename, abs=True)
         if standalone:
             if self.exe_ext:
                 exe_name += '.' + self.exe_ext

diff --git a/pypy/module/array/interp_array.py b/pypy/module/array/interp_array.py
--- a/pypy/module/array/interp_array.py
+++ b/pypy/module/array/interp_array.py
@@ -362,10 +362,19 @@
             self.setlen(0)
             self.fromsequence(w_lst)
         else:
-            j = 0
-            for i in range(start, stop, step):
-                self.buffer[i] = w_item.buffer[j]
-                j += 1
+            if self is w_item:
+                with lltype.scoped_alloc(mytype.arraytype, self.allocated) as new_buffer:
+                    for i in range(self.len):
+                        new_buffer[i] = w_item.buffer[i]
+                    j = 0
+                    for i in range(start, stop, step):
+                        self.buffer[i] = new_buffer[j]
+                        j += 1
+            else:
+                j = 0
+                for i in range(start, stop, step):
+                    self.buffer[i] = w_item.buffer[j]
+                    j += 1
 
     def setslice__Array_ANY_ANY_ANY(space, self, w_i, w_j, w_x):
         space.setitem(self, space.newslice(w_i, w_j, space.w_None), w_x)

diff --git a/pypy/interpreter/baseobjspace.py b/pypy/interpreter/baseobjspace.py
--- a/pypy/interpreter/baseobjspace.py
+++ b/pypy/interpreter/baseobjspace.py
@@ -1156,38 +1156,66 @@
         # This is here mostly just for gateway.int_unwrapping_space_method().
         return bool(self.int_w(w_obj))
 
-    def nonnegint_w(self, w_obj):
-        # Like space.int_w(), but raises an app-level ValueError if
-        # the integer is negative.  Mostly here for gateway.py.
-        value = self.int_w(w_obj)
+    # This is all interface for gateway.py.
+    def gateway_int_w(self, w_obj):
+        if self.is_true(self.isinstance(w_obj, self.w_float)):
+            raise OperationError(self.w_TypeError,
+                            self.wrap("integer argument expected, got float"))
+        return self.int_w(self.int(w_obj))
+
+    def gateway_float_w(self, w_obj):
+        return self.float_w(self.float(w_obj))
+
+    def gateway_r_longlong_w(self, w_obj):
+        if self.is_true(self.isinstance(w_obj, self.w_float)):
+            raise OperationError(self.w_TypeError,
+                            self.wrap("integer argument expected, got float"))
+        return self.r_longlong_w(self.int(w_obj))
+
+    def gateway_r_uint_w(self, w_obj):
+        if self.is_true(self.isinstance(w_obj, self.w_float)):
+            raise OperationError(self.w_TypeError,
+                            self.wrap("integer argument expected, got float"))
+        return self.uint_w(self.int(w_obj))
+
+    def gateway_r_ulonglong_w(self, w_obj):
+        if self.is_true(self.isinstance(w_obj, self.w_float)):
+            raise OperationError(self.w_TypeError,
+                            self.wrap("integer argument expected, got float"))
+        return self.r_ulonglong_w(self.int(w_obj))
+
+    def gateway_nonnegint_w(self, w_obj):
+        # Like space.gateway_int_w(), but raises an app-level ValueError if
+        # the integer is negative.  Here for gateway.py.
+        value = self.gateway_int_w(w_obj)
         if value < 0:
             raise OperationError(self.w_ValueError,
                                  self.wrap("expected a non-negative integer"))
         return value
 
     def c_int_w(self, w_obj):
-        # Like space.int_w(), but raises an app-level OverflowError if
-        # the integer does not fit in 32 bits.  Mostly here for gateway.py.
-        value = self.int_w(w_obj)
+        # Like space.gateway_int_w(), but raises an app-level OverflowError if
+        # the integer does not fit in 32 bits.  Here for gateway.py.
+        value = self.gateway_int_w(w_obj)
         if value < -2147483647-1 or value > 2147483647:
             raise OperationError(self.w_OverflowError,
                                  self.wrap("expected a 32-bit integer"))
         return value
 
     def c_uint_w(self, w_obj):
-        # Like space.uint_w(), but raises an app-level OverflowError if
-        # the integer does not fit in 32 bits.  Mostly here for gateway.py.
-        value = self.uint_w(w_obj)
+        # Like space.gateway_uint_w(), but raises an app-level OverflowError if
+        # the integer does not fit in 32 bits.  Here for gateway.py.
+        value = self.gateway_r_uint_w(w_obj)
         if value > UINT_MAX_32_BITS:
             raise OperationError(self.w_OverflowError,
                               self.wrap("expected an unsigned 32-bit integer"))
         return value
 
     def c_nonnegint_w(self, w_obj):
-        # Like space.int_w(), but raises an app-level ValueError if
-        # the integer is negative or does not fit in 32 bits.  Mostly here
+        # Like space.gateway_int_w(), but raises an app-level ValueError if
+        # the integer is negative or does not fit in 32 bits.  Here
         # for gateway.py.
-        value = self.int_w(w_obj)
+        value = self.gateway_int_w(w_obj)
         if value < 0:
             raise OperationError(self.w_ValueError,
                                  self.wrap("expected a non-negative integer"))
@@ -1197,6 +1225,10 @@
         return value
 
     def c_filedescriptor_w(self, w_fd):
+        # This is only used sometimes in CPython, e.g. for os.fsync() but
+        # not os.close().  It's likely designed for 'select'.  It's irregular
+        # in the sense that it expects either a real int/long or an object
+        # with a fileno(), but not an object with an __int__().
         if (not self.isinstance_w(w_fd, self.w_int) and
             not self.isinstance_w(w_fd, self.w_long)):
             try:

diff --git a/lib-python/conftest.py b/lib-python/conftest.py
--- a/lib-python/conftest.py
+++ b/lib-python/conftest.py
@@ -36,7 +36,9 @@
     group.addoption('--pypy', action="store", type="string",
        dest="pypy",  help="use given pypy executable to run lib-python tests. "
                           "This will run the tests directly (i.e. not through py.py)")
-   
+    group.addoption('--filter', action="store", type="string", default=None,
+                    dest="unittest_filter",  help="Similar to -k, XXX")
+
 option = py.test.config.option 
 
 def gettimeout(): 
@@ -688,7 +690,14 @@
             else:
                 status = 'abnormal termination 0x%x' % status
         else:
-            status = os.system("%s >>%s 2>>%s" %(cmd, stdout, stderr))
+            if self.config.option.unittest_filter is not None:
+                cmd += ' --filter %s' % self.config.option.unittest_filter
+            if self.config.option.usepdb:
+                cmd += ' --pdb'
+            if self.config.option.capture == 'no':
+                status = os.system(cmd)
+            else:
+                status = os.system("%s >>%s 2>>%s" %(cmd, stdout, stderr))
             if os.WIFEXITED(status):
                 status = os.WEXITSTATUS(status)
             else:

diff --git a/pypy/module/signal/interp_signal.py b/pypy/module/signal/interp_signal.py
--- a/pypy/module/signal/interp_signal.py
+++ b/pypy/module/signal/interp_signal.py
@@ -1,11 +1,15 @@
+from __future__ import with_statement
 from pypy.interpreter.error import OperationError
 from pypy.interpreter.baseobjspace import W_Root, ObjSpace
 from pypy.interpreter.executioncontext import AsyncAction, AbstractActionFlag
 from pypy.interpreter.executioncontext import PeriodicAsyncAction
+from pypy.interpreter.gateway import unwrap_spec
 import signal as cpy_signal
 from pypy.rpython.lltypesystem import lltype, rffi
+from pypy.rpython.tool import rffi_platform
 from pypy.translator.tool.cbuild import ExternalCompilationInfo
 import py
+import sys
 from pypy.tool import autopath
 from pypy.rlib import jit, rposix
 from pypy.rlib.rarithmetic import intmask
@@ -21,8 +25,12 @@
 SIG_IGN = cpy_signal.SIG_IGN
 signal_names = list(setup())
 
+includes = ['stdlib.h', 'src/signals.h']
+if sys.platform != 'win32':
+    includes.append('sys/time.h')
+
 eci = ExternalCompilationInfo(
-    includes = ['stdlib.h', 'src/signals.h'],
+    includes = includes,
     separate_module_sources = ['#include <src/signals.h>'],
     include_dirs = [str(py.path.local(autopath.pypydir).join('translator', 'c'))],
     export_symbols = ['pypysig_poll', 'pypysig_default',
@@ -31,6 +39,26 @@
                       'pypysig_getaddr_occurred'],
 )
 
+class CConfig:
+    _compilation_info_ = eci
+
+if sys.platform != 'win32':
+    for name in """ITIMER_REAL ITIMER_VIRTUAL ITIMER_PROF""".split():
+        setattr(CConfig, name, rffi_platform.DefinedConstantInteger(name))
+
+    CConfig.timeval = rffi_platform.Struct(
+        'struct timeval',
+        [('tv_sec', rffi.LONG),
+         ('tv_usec', rffi.LONG)])
+
+    CConfig.itimerval = rffi_platform.Struct(
+        'struct itimerval',
+        [('it_value', CConfig.timeval),
+         ('it_interval', CConfig.timeval)])
+
+for k, v in rffi_platform.configure(CConfig).items():
+    globals()[k] = v
+
 def external(name, args, result, **kwds):
     return rffi.llexternal(name, args, result, compilation_info=eci, **kwds)
 
@@ -55,6 +83,12 @@
 c_pause = external('pause', [], rffi.INT)
 c_siginterrupt = external('siginterrupt', [rffi.INT, rffi.INT], rffi.INT)
 
+if sys.platform != 'win32':
+    itimervalP = rffi.CArrayPtr(itimerval)
+    c_setitimer = external('setitimer',
+                           [rffi.INT, itimervalP, itimervalP], rffi.INT)
+    c_getitimer = external('getitimer', [rffi.INT, itimervalP], rffi.INT)
+
 
 class SignalActionFlag(AbstractActionFlag):
     # This class uses the C-level pypysig_counter variable as the tick
@@ -252,3 +286,41 @@
         errno = rposix.get_errno()
         raise OperationError(space.w_RuntimeError, space.wrap(errno))
 siginterrupt.unwrap_spec = [ObjSpace, int, int]
+
+
+#__________________________________________________________
+
+def timeval_from_double(d, timeval):
+    timeval.c_tv_sec = int(d)
+    timeval.c_tv_usec = int((d - int(d)) * 1000000)
+
+def double_from_timeval(tv):
+    return tv.c_tv_sec + (tv.c_tv_usec / 1000000.0)
+
+def itimer_retval(space, val):
+    w_value = space.wrap(double_from_timeval(val.c_it_value))
+    w_interval = space.wrap(double_from_timeval(val.c_it_interval))
+    return space.newtuple([w_value, w_interval])
+
+ at jit.dont_look_inside
+ at unwrap_spec(ObjSpace, int, float, float)
+def setitimer(space, which, first, interval=0):
+    with lltype.scoped_alloc(itimervalP.TO, 1) as new:
+
+        timeval_from_double(first, new[0].c_it_value)
+        timeval_from_double(interval, new[0].c_it_interval)
+
+        with lltype.scoped_alloc(itimervalP.TO, 1) as old:
+
+            c_setitimer(which, new, old)
+
+            return itimer_retval(space, old[0])
+
+ at jit.dont_look_inside
+ at unwrap_spec(ObjSpace, int)
+def getitimer(space, which):
+    with lltype.scoped_alloc(itimervalP.TO, 1) as old:
+
+        c_getitimer(which, old)
+
+        return itimer_retval(space, old[0])

diff --git a/pypy/jit/metainterp/test/test_optimizeopt.py b/pypy/jit/metainterp/test/test_optimizeopt.py
--- a/pypy/jit/metainterp/test/test_optimizeopt.py
+++ b/pypy/jit/metainterp/test/test_optimizeopt.py
@@ -552,21 +552,47 @@
         """
         self.optimize_loop(ops, expected, preamble)
 
+    def test_bound_int_is_true(self):
+        ops = """
+        [i0]
+        i1 = int_add(i0, 1)
+        i2 = int_gt(i1, 0)
+        guard_true(i2) []
+        i3 = int_is_true(i1)
+        guard_true(i3) []
+        jump(i1)
+        """
+        expected = """
+        [i0]
+        i1 = int_add(i0, 1)
+        jump(i1)
+        """
+        preamble = """
+        [i0]
+        i1 = int_add(i0, 1)
+        i2 = int_gt(i1, 0)
+        guard_true(i2) []
+        jump(i1)
+        """
+        self.optimize_loop(ops, expected, preamble)
+
     def test_int_is_true_is_zero(self):
-        py.test.skip("XXX implement me")
+        py.test.skip("in-progress")
         ops = """
         [i0]
-        i1 = int_is_true(i0)
-        guard_true(i1) []
-        i2 = int_is_zero(i0)
-        guard_false(i2) []
-        jump(i0)
+        i1 = int_add(i0, 1)
+        i2 = int_is_true(i1)
+        guard_true(i2) []
+        i3 = int_is_zero(i1)
+        guard_false(i3) []
+        jump(i1)
         """
         expected = """
         [i0]
-        i1 = int_is_true(i0)
-        guard_true(i1) []
-        jump(i0)
+        i1 = int_add(i0, 1)
+        i2 = int_is_true(i1)
+        guard_true(i2) []
+        jump(i1)
         """
         self.optimize_loop(ops, expected)
 

diff --git a/pypy/objspace/std/objspace.py b/pypy/objspace/std/objspace.py
--- a/pypy/objspace/std/objspace.py
+++ b/pypy/objspace/std/objspace.py
@@ -12,6 +12,7 @@
 from pypy.rlib.rarithmetic import base_int, widen
 from pypy.rlib.objectmodel import we_are_translated
 from pypy.rlib.jit import hint
+from pypy.rlib.rbigint import rbigint
 from pypy.tool.sourcetools import func_with_new_name
 
 # Object imports
@@ -181,8 +182,12 @@
                 return self.newint(x)
             else:
                 return W_LongObject.fromrarith_int(x)
+        return self._wrap_not_rpython(x)
+    wrap._annspecialcase_ = "specialize:wrap"
 
-        # _____ below here is where the annotator should not get _____
+    def _wrap_not_rpython(self, x):
+        "NOT_RPYTHON"
+        # _____ this code is here to support testing only _____
 
         # wrap() of a container works on CPython, but the code is
         # not RPython.  Don't use -- it is kept around mostly for tests.
@@ -226,9 +231,6 @@
             return self.w_Ellipsis
 
         if self.config.objspace.nofaking:
-            # annotation should actually not get here.  If it does, you get
-            # an error during rtyping because '%r' is not supported.  It tells
-            # you that there was a space.wrap() on a strange object.
             raise OperationError(self.w_RuntimeError,
                                  self.wrap("nofaking enabled: refusing "
                                            "to wrap cpython value %r" %(x,)))
@@ -239,8 +241,6 @@
         from fake import fake_object
         return fake_object(self, x)
 
-    wrap._annspecialcase_ = "specialize:wrap"
-
     def wrap_exception_cls(self, x):
         """NOT_RPYTHON"""
         if hasattr(self, 'w_' + x.__name__):
@@ -264,9 +264,16 @@
     def newcomplex(self, realval, imagval):
         return W_ComplexObject(realval, imagval)
 
+    def unpackcomplex(self, w_complex):
+        from pypy.objspace.std.complextype import unpackcomplex
+        return unpackcomplex(self, w_complex)
+
     def newlong(self, val): # val is an int
         return W_LongObject.fromint(self, val)
 
+    def newlong_from_rbigint(self, val):
+        return W_LongObject(val)
+
     def newtuple(self, list_w):
         assert isinstance(list_w, list)
         make_sure_not_resized(list_w)

diff --git a/pypy/translator/tool/cbuild.py b/pypy/translator/tool/cbuild.py
--- a/pypy/translator/tool/cbuild.py
+++ b/pypy/translator/tool/cbuild.py
@@ -1,4 +1,5 @@
 import py
+import sys
 
 from pypy.tool.autopath import pypydir
 from pypy.translator.platform import host
@@ -274,10 +275,15 @@
     def compile_shared_lib(self, outputfilename=None):
         self = self.convert_sources_to_files()
         if not self.separate_module_files:
-            return self
+            if sys.platform != 'win32':
+                return self
+            if not self.export_symbols:
+                return self
+            basepath = udir.join('module_cache')
+        else:
+            basepath = py.path.local(self.separate_module_files[0]).dirpath()
         if outputfilename is None:
             # find more or less unique name there
-            basepath = py.path.local(self.separate_module_files[0]).dirpath()
             pth = basepath.join('externmod').new(ext=host.so_ext)
             num = 0
             while pth.check():

diff --git a/lib_pypy/cmath.py b/lib_pypy/cmath.py
deleted file mode 100644
--- a/lib_pypy/cmath.py
+++ /dev/null
@@ -1,288 +0,0 @@
-"""This module is always available. It provides access to mathematical
-functions for complex numbers."""
-
-# Complex math module
-
-# much code borrowed from mathmodule.c
-
-import math
-from math import e, pi
-
-try: from __pypy__ import builtinify
-except ImportError: builtinify = lambda f: f
-
-
-# constants
-_one = complex(1., 0.)
-_half = complex(0.5, 0.)
-_i = complex(0., 1.)
-_halfi = complex(0., 0.5)
-
-
-
-# internal functions not available from Python
-def _to_complex(x):
-    if isinstance(x, complex):
-        return x
-    if isinstance(x, (str, unicode)):
-        raise TypeError('float or complex required')
-    return complex(x)
-
-def _prodi(x):
-    x = _to_complex(x)
-    real = -x.imag
-    imag = x.real
-    return complex(real, imag)
-
-
- at builtinify
-def phase(x):
-    x = _to_complex(x)
-    return math.atan2(x.imag, x.real)
-
-
- at builtinify
-def polar(x):
-    x = _to_complex(x)
-    phi = math.atan2(x.imag, x.real)
-    r = abs(x)
-    return r, phi
-
-
- at builtinify
-def rect(r, phi):
-    return complex(r * math.cos(phi), r * math.sin(phi))
-
-
- at builtinify
-def acos(x):
-    """acos(x)
-
-    Return the arc cosine of x."""
-
-    x = _to_complex(x)
-    return -(_prodi(log((x+(_i*sqrt((_one-(x*x))))))))
-
-
- at builtinify
-def acosh(x):
-    """acosh(x)
-
-    Return the hyperbolic arccosine of x."""
-
-    x = _to_complex(x)
-    z = log(_sqrt_half*(sqrt(x+_one)+sqrt(x-_one)))
-    return z+z
-
-
- at builtinify
-def asin(x):
-    """asin(x)
-
-    Return the arc sine of x."""
-
-    x = _to_complex(x)
-    # -i * log[(sqrt(1-x**2) + i*x]
-    squared = x*x
-    sqrt_1_minus_x_sq = sqrt(_one-squared)
-    return -(_prodi(log((sqrt_1_minus_x_sq+_prodi(x)))))
-
-
- at builtinify
-def asinh(x):
-    """asinh(x)
-
-    Return the hyperbolic arc sine of x."""
-
-    x = _to_complex(x)
-    z = log((_sqrt_half * (sqrt(x+_i)+sqrt((x-_i))) ))
-    return z+z
-
-
- at builtinify
-def atan(x):
-    """atan(x)
-    
-    Return the arc tangent of x."""
-
-    x = _to_complex(x)
-    return _halfi*log(((_i+x)/(_i-x)))
-
-
- at builtinify
-def atanh(x):
-    """atanh(x)
-
-    Return the hyperbolic arc tangent of x."""
-
-    x = _to_complex(x)
-    return _half*log((_one+x)/(_one-x))
-
-
- at builtinify
-def cos(x):
-    """cos(x)
-
-    Return the cosine of x."""
-
-    x = _to_complex(x)
-    real = math.cos(x.real) * math.cosh(x.imag)
-    imag = -math.sin(x.real) * math.sinh(x.imag)
-    return complex(real, imag)
-
-
- at builtinify
-def cosh(x):
-    """cosh(x)
-    
-    Return the hyperbolic cosine of x."""
-
-    x = _to_complex(x)
-    real = math.cos(x.imag) * math.cosh(x.real)
-    imag = math.sin(x.imag) * math.sinh(x.real)
-    return complex(real, imag)
-
-
- at builtinify
-def exp(x):
-    """exp(x)
-    
-    Return the exponential value e**x."""
-
-    x = _to_complex(x)
-    l = math.exp(x.real)
-    real = l * math.cos(x.imag)
-    imag = l * math.sin(x.imag)
-    return complex(real, imag)
-
-
- at builtinify
-def log(x, base=None):
-    """log(x)
-
-    Return the natural logarithm of x."""
-    
-    if base is not None:
-        return log(x) / log(base)
-    x = _to_complex(x)
-    l = math.hypot(x.real,x.imag)
-    imag = math.atan2(x.imag, x.real)
-    real = math.log(l)
-    return complex(real, imag)
-
-
- at builtinify
-def log10(x):
-    """log10(x)
-
-    Return the base-10 logarithm of x."""
-    
-    x = _to_complex(x)
-    l = math.hypot(x.real, x.imag)
-    imag = math.atan2(x.imag, x.real)/math.log(10.)
-    real = math.log10(l)
-    return complex(real, imag)
-
-
- at builtinify
-def sin(x):
-    """sin(x)
-
-    Return the sine of x."""
-    
-    x = _to_complex(x)
-    real = math.sin(x.real) * math.cosh(x.imag)
-    imag = math.cos(x.real) * math.sinh(x.imag)
-    return complex(real, imag)
-
-
- at builtinify
-def sinh(x):
-    """sinh(x)
-
-    Return the hyperbolic sine of x."""
-    
-    x = _to_complex(x)
-    real = math.cos(x.imag) * math.sinh(x.real)
-    imag = math.sin(x.imag) * math.cosh(x.real)
-    return complex(real, imag)
-
-
- at builtinify
-def sqrt(x):
-    """sqrt(x)
-
-    Return the square root of x."""
-    
-    x = _to_complex(x)
-    if x.real == 0. and x.imag == 0.:
-        real, imag = 0, 0
-    else:
-        s = math.sqrt(0.5*(math.fabs(x.real) + math.hypot(x.real,x.imag)))
-        d = 0.5*x.imag/s
-        if x.real > 0.:
-            real = s
-            imag = d
-        elif x.imag >= 0.:
-            real = d
-            imag = s
-        else:
-            real = -d
-            imag = -s
-    return complex(real, imag)
-
-_sqrt_half = sqrt(_half)
-
-
- at builtinify
-def tan(x):
-    """tan(x)
-
-    Return the tangent of x."""
-
-    x = _to_complex(x)
-    sr = math.sin(x.real)
-    cr = math.cos(x.real)
-    shi = math.sinh(x.imag)
-    chi = math.cosh(x.imag)
-    rs = sr * chi
-    is_ = cr * shi
-    rc = cr * chi
-    ic = -sr * shi
-    d = rc*rc + ic * ic
-    real = (rs*rc + is_*ic) / d
-    imag = (is_*rc - rs*ic) / d
-    return complex(real, imag)
-
-
- at builtinify
-def tanh(x):
-    """tanh(x)
-
-    Return the hyperbolic tangent of x."""
-    
-    x = _to_complex(x)
-    si = math.sin(x.imag)
-    ci = math.cos(x.imag)
-    shr = math.sinh(x.real)
-    chr = math.cosh(x.real)
-    rs = ci * shr
-    is_ = si * chr
-    rc = ci * chr
-    ic = si * shr
-    d = rc*rc + ic*ic
-    real = (rs*rc + is_*ic) / d
-    imag = (is_*rc - rs*ic) / d
-    return complex(real, imag)
-
-def isnan(x):
-    """isnan(z) -> bool
-    Checks if the real or imaginary part of z not a number (NaN)"""
-    x = _to_complex(x)
-    return math.isnan(x.real) or math.isnan(x.imag)
-
-def isinf(x):
-    """isnan(z) -> bool
-    Checks if the real or imaginary part of z is infinite"""
-    x = _to_complex(x)
-    return math.isinf(x.real) or math.isinf(x.imag)

diff --git a/pypy/config/pypyoption.py b/pypy/config/pypyoption.py
--- a/pypy/config/pypyoption.py
+++ b/pypy/config/pypyoption.py
@@ -18,7 +18,7 @@
 default_modules = essential_modules.copy()
 default_modules.update(dict.fromkeys(
     ["_codecs", "gc", "_weakref", "marshal", "errno", "imp",
-     "math", "_sre", "_pickle_support", "operator",
+     "math", "cmath", "_sre", "_pickle_support", "operator",
      "parser", "symbol", "token", "_ast",  "_io", "_random", "__pypy__",
      "_testing"]))
 

From commits-noreply at bitbucket.org  Wed Feb  2 13:49:57 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Wed,  2 Feb 2011 13:49:57 +0100 (CET)
Subject: [pypy-svn] pypy arm-backed-float: merge arm-backend-2
Message-ID: <20110202124957.3587A2A2011@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backed-float
Changeset: r41547:7aeb31f6ea89
Date: 2011-01-20 17:09 +0100
http://bitbucket.org/pypy/pypy/changeset/7aeb31f6ea89/

Log:	merge arm-backend-2

diff --git a/lib_pypy/cmath.py b/lib_pypy/cmath.py
deleted file mode 100644
--- a/lib_pypy/cmath.py
+++ /dev/null
@@ -1,288 +0,0 @@
-"""This module is always available. It provides access to mathematical
-functions for complex numbers."""
-
-# Complex math module
-
-# much code borrowed from mathmodule.c
-
-import math
-from math import e, pi
-
-try: from __pypy__ import builtinify
-except ImportError: builtinify = lambda f: f
-
-
-# constants
-_one = complex(1., 0.)
-_half = complex(0.5, 0.)
-_i = complex(0., 1.)
-_halfi = complex(0., 0.5)
-
-
-
-# internal functions not available from Python
-def _to_complex(x):
-    if isinstance(x, complex):
-        return x
-    if isinstance(x, (str, unicode)):
-        raise TypeError('float or complex required')
-    return complex(x)
-
-def _prodi(x):
-    x = _to_complex(x)
-    real = -x.imag
-    imag = x.real
-    return complex(real, imag)
-
-
- at builtinify
-def phase(x):
-    x = _to_complex(x)
-    return math.atan2(x.imag, x.real)
-
-
- at builtinify
-def polar(x):
-    x = _to_complex(x)
-    phi = math.atan2(x.imag, x.real)
-    r = abs(x)
-    return r, phi
-
-
- at builtinify
-def rect(r, phi):
-    return complex(r * math.cos(phi), r * math.sin(phi))
-
-
- at builtinify
-def acos(x):
-    """acos(x)
-
-    Return the arc cosine of x."""
-
-    x = _to_complex(x)
-    return -(_prodi(log((x+(_i*sqrt((_one-(x*x))))))))
-
-
- at builtinify
-def acosh(x):
-    """acosh(x)
-
-    Return the hyperbolic arccosine of x."""
-
-    x = _to_complex(x)
-    z = log(_sqrt_half*(sqrt(x+_one)+sqrt(x-_one)))
-    return z+z
-
-
- at builtinify
-def asin(x):
-    """asin(x)
-
-    Return the arc sine of x."""
-
-    x = _to_complex(x)
-    # -i * log[(sqrt(1-x**2) + i*x]
-    squared = x*x
-    sqrt_1_minus_x_sq = sqrt(_one-squared)
-    return -(_prodi(log((sqrt_1_minus_x_sq+_prodi(x)))))
-
-
- at builtinify
-def asinh(x):
-    """asinh(x)
-
-    Return the hyperbolic arc sine of x."""
-
-    x = _to_complex(x)
-    z = log((_sqrt_half * (sqrt(x+_i)+sqrt((x-_i))) ))
-    return z+z
-
-
- at builtinify
-def atan(x):
-    """atan(x)
-    
-    Return the arc tangent of x."""
-
-    x = _to_complex(x)
-    return _halfi*log(((_i+x)/(_i-x)))
-
-
- at builtinify
-def atanh(x):
-    """atanh(x)
-
-    Return the hyperbolic arc tangent of x."""
-
-    x = _to_complex(x)
-    return _half*log((_one+x)/(_one-x))
-
-
- at builtinify
-def cos(x):
-    """cos(x)
-
-    Return the cosine of x."""
-
-    x = _to_complex(x)
-    real = math.cos(x.real) * math.cosh(x.imag)
-    imag = -math.sin(x.real) * math.sinh(x.imag)
-    return complex(real, imag)
-
-
- at builtinify
-def cosh(x):
-    """cosh(x)
-    
-    Return the hyperbolic cosine of x."""
-
-    x = _to_complex(x)
-    real = math.cos(x.imag) * math.cosh(x.real)
-    imag = math.sin(x.imag) * math.sinh(x.real)
-    return complex(real, imag)
-
-
- at builtinify
-def exp(x):
-    """exp(x)
-    
-    Return the exponential value e**x."""
-
-    x = _to_complex(x)
-    l = math.exp(x.real)
-    real = l * math.cos(x.imag)
-    imag = l * math.sin(x.imag)
-    return complex(real, imag)
-
-
- at builtinify
-def log(x, base=None):
-    """log(x)
-
-    Return the natural logarithm of x."""
-    
-    if base is not None:
-        return log(x) / log(base)
-    x = _to_complex(x)
-    l = math.hypot(x.real,x.imag)
-    imag = math.atan2(x.imag, x.real)
-    real = math.log(l)
-    return complex(real, imag)
-
-
- at builtinify
-def log10(x):
-    """log10(x)
-
-    Return the base-10 logarithm of x."""
-    
-    x = _to_complex(x)
-    l = math.hypot(x.real, x.imag)
-    imag = math.atan2(x.imag, x.real)/math.log(10.)
-    real = math.log10(l)
-    return complex(real, imag)
-
-
- at builtinify
-def sin(x):
-    """sin(x)
-
-    Return the sine of x."""
-    
-    x = _to_complex(x)
-    real = math.sin(x.real) * math.cosh(x.imag)
-    imag = math.cos(x.real) * math.sinh(x.imag)
-    return complex(real, imag)
-
-
- at builtinify
-def sinh(x):
-    """sinh(x)
-
-    Return the hyperbolic sine of x."""
-    
-    x = _to_complex(x)
-    real = math.cos(x.imag) * math.sinh(x.real)
-    imag = math.sin(x.imag) * math.cosh(x.real)
-    return complex(real, imag)
-
-
- at builtinify
-def sqrt(x):
-    """sqrt(x)
-
-    Return the square root of x."""
-    
-    x = _to_complex(x)
-    if x.real == 0. and x.imag == 0.:
-        real, imag = 0, 0
-    else:
-        s = math.sqrt(0.5*(math.fabs(x.real) + math.hypot(x.real,x.imag)))
-        d = 0.5*x.imag/s
-        if x.real > 0.:
-            real = s
-            imag = d
-        elif x.imag >= 0.:
-            real = d
-            imag = s
-        else:
-            real = -d
-            imag = -s
-    return complex(real, imag)
-
-_sqrt_half = sqrt(_half)
-
-
- at builtinify
-def tan(x):
-    """tan(x)
-
-    Return the tangent of x."""
-
-    x = _to_complex(x)
-    sr = math.sin(x.real)
-    cr = math.cos(x.real)
-    shi = math.sinh(x.imag)
-    chi = math.cosh(x.imag)
-    rs = sr * chi
-    is_ = cr * shi
-    rc = cr * chi
-    ic = -sr * shi
-    d = rc*rc + ic * ic
-    real = (rs*rc + is_*ic) / d
-    imag = (is_*rc - rs*ic) / d
-    return complex(real, imag)
-
-
- at builtinify
-def tanh(x):
-    """tanh(x)
-
-    Return the hyperbolic tangent of x."""
-    
-    x = _to_complex(x)
-    si = math.sin(x.imag)
-    ci = math.cos(x.imag)
-    shr = math.sinh(x.real)
-    chr = math.cosh(x.real)
-    rs = ci * shr
-    is_ = si * chr
-    rc = ci * chr
-    ic = si * shr
-    d = rc*rc + ic*ic
-    real = (rs*rc + is_*ic) / d
-    imag = (is_*rc - rs*ic) / d
-    return complex(real, imag)
-
-def isnan(x):
-    """isnan(z) -> bool
-    Checks if the real or imaginary part of z not a number (NaN)"""
-    x = _to_complex(x)
-    return math.isnan(x.real) or math.isnan(x.imag)
-
-def isinf(x):
-    """isnan(z) -> bool
-    Checks if the real or imaginary part of z is infinite"""
-    x = _to_complex(x)
-    return math.isinf(x.real) or math.isinf(x.imag)

From commits-noreply at bitbucket.org  Wed Feb  2 13:50:01 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Wed,  2 Feb 2011 13:50:01 +0100 (CET)
Subject: [pypy-svn] pypy arm-backend-2: merge default (again)
Message-ID: <20110202125001.4016C2A2006@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backend-2
Changeset: r41548:87ea43a8585d
Date: 2011-01-21 12:38 +0100
http://bitbucket.org/pypy/pypy/changeset/87ea43a8585d/

Log:	merge default (again)

diff --git a/pypy/objspace/std/stringobject.py b/pypy/objspace/std/stringobject.py
--- a/pypy/objspace/std/stringobject.py
+++ b/pypy/objspace/std/stringobject.py
@@ -541,11 +541,11 @@
     if left:
         #print "while %d < %d and -%s- in -%s-:"%(lpos, rpos, u_self[lpos],w_chars)
         while lpos < rpos and u_self[lpos] in u_chars:
-           lpos += 1
+            lpos += 1
 
     if right:
         while rpos > lpos and u_self[rpos - 1] in u_chars:
-           rpos -= 1
+            rpos -= 1
 
     assert rpos >= lpos    # annotator hint, don't remove
     return sliced(space, u_self, lpos, rpos, w_self)
@@ -950,11 +950,10 @@
     remaining characters have been mapped through the given translation table,
     which must be a string of length 256"""
 
-    # XXX CPython accepts buffers, too, not sure what we should do
     if space.is_w(w_table, space.w_None):
         table = DEFAULT_NOOP_TABLE
     else:
-        table = space.str_w(w_table)
+        table = space.bufferstr_w(w_table)
         if len(table) != 256:
             raise OperationError(
                 space.w_ValueError,

diff --git a/pypy/module/sys/__init__.py b/pypy/module/sys/__init__.py
--- a/pypy/module/sys/__init__.py
+++ b/pypy/module/sys/__init__.py
@@ -26,7 +26,6 @@
         'maxsize'               : 'space.wrap(sys.maxint)',
         'byteorder'             : 'space.wrap(sys.byteorder)', 
         'maxunicode'            : 'space.wrap(vm.MAXUNICODE)',
-        'maxint'                : 'space.wrap(sys.maxint)',
         'stdin'                 : 'state.getio(space).w_stdin',
         '__stdin__'             : 'state.getio(space).w_stdin',
         'stdout'                : 'state.getio(space).w_stdout',
@@ -56,6 +55,7 @@
         'setprofile'            : 'vm.setprofile',
         'getprofile'            : 'vm.getprofile',
         'call_tracing'          : 'vm.call_tracing',
+        'getsizeof'             : 'vm.getsizeof',
         
         'executable'            : 'space.wrap("py.py")', 
         'api_version'           : 'version.get_api_version(space)',

diff --git a/pypy/module/cpyext/test/test_cpyext.py b/pypy/module/cpyext/test/test_cpyext.py
--- a/pypy/module/cpyext/test/test_cpyext.py
+++ b/pypy/module/cpyext/test/test_cpyext.py
@@ -46,8 +46,8 @@
 
     def test_dllhandle(self):
         import sys
-        if sys.version_info < (2, 6):
-            skip("Python >= 2.6 only")
+        if sys.platform != "win32" or sys.version_info < (2, 6):
+            skip("Windows Python >= 2.6 only")
         assert sys.dllhandle
         assert sys.dllhandle.getaddressindll('PyPyErr_NewException')
         import ctypes # slow

diff --git a/pypy/module/array/interp_array.py b/pypy/module/array/interp_array.py
--- a/pypy/module/array/interp_array.py
+++ b/pypy/module/array/interp_array.py
@@ -1,3 +1,5 @@
+from __future__ import with_statement
+
 from pypy.interpreter.error import OperationError
 from pypy.interpreter.typedef import TypeDef, GetSetProperty, make_weakref_descr
 from pypy.rpython.lltypesystem import lltype, rffi

diff --git a/pypy/module/itertools/test/errors.txt b/pypy/module/itertools/test/errors.txt
deleted file mode 100644
--- a/pypy/module/itertools/test/errors.txt
+++ /dev/null
@@ -1,67 +0,0 @@
-
-
-Here are the remaining errors of CPython 2.5's test_itertools.  FWIW I
-consider them all as obscure undocumented implementation details.
-
-
-======================================================================
-ERROR: test_islice (__main__.TestBasicOps)
-----------------------------------------------------------------------
-Traceback (most recent call last):
-  File "test_itertools.py", line 285, in test_islice
-    self.assertRaises(ValueError, islice, xrange(10), 'a')
-  File "/home/arigo/pypysrc/lib-python/2.4.1/unittest.py", line 322, in failUnlessRaises
-    return
-  File "/home/arigo/pypysrc/lib-python/2.4.1/unittest.py", line 320, in failUnlessRaises
-    callableObj(*args, **kwargs)
-TypeError: expected integer, got str object
-
-======================================================================
-ERROR: test_tee (__main__.TestBasicOps)
-----------------------------------------------------------------------
-Traceback (most recent call last):
-  File "itest25.py", line 376, in test_tee
-    c = type(a)('def')
-TypeError: default __new__ takes no parameters
-
-======================================================================
-ERROR: test_repeat (__main__.LengthTransparency)
-----------------------------------------------------------------------
-Traceback (most recent call last):
-  File "itest25.py", line 690, in test_repeat
-    from test.test_iterlen import len
-ImportError: cannot import name 'len'
-
-======================================================================
-ERROR: test_keywords_in_subclass (__main__.SubclassWithKwargsTest)
-----------------------------------------------------------------------
-Traceback (most recent call last):
-  File "itest25.py", line 760, in test_keywords_in_subclass
-    class Subclass(cls):
-TypeError: type 'repeat' is not an acceptable base class
-
-======================================================================
-FAIL: test_count (__main__.TestBasicOps)
-----------------------------------------------------------------------
-Traceback (most recent call last):
-  File "itest25.py", line 59, in test_count
-    self.assertEqual(repr(c), 'count(3)')
-AssertionError: '<count object at 0x08fcdcac>' != 'count(3)'
-
-======================================================================
-FAIL: test_izip (__main__.TestBasicOps)
-----------------------------------------------------------------------
-Traceback (most recent call last):
-  File "itest25.py", line 199, in test_izip
-    self.assertEqual(min(ids), max(ids))
-AssertionError: 149283404 != 150789644
-
-======================================================================
-FAIL: test_repeat (__main__.TestBasicOps)
-----------------------------------------------------------------------
-Traceback (most recent call last):
-  File "itest25.py", line 214, in test_repeat
-    self.assertEqual(repr(r), 'repeat((1+0j))')
-AssertionError: '<repeat object at 0x09124a2c>' != 'repeat((1+0j))'
-
-----------------------------------------------------------------------

diff --git a/lib-python/conftest.py b/lib-python/conftest.py
--- a/lib-python/conftest.py
+++ b/lib-python/conftest.py
@@ -422,7 +422,7 @@
     RegrTest('test_sundry.py'),
     RegrTest('test_symtable.py', skip="implementation detail"),
     RegrTest('test_syntax.py', core=True),
-    RegrTest('test_sys.py', core=True),
+    RegrTest('test_sys.py', core=True, usemodules='struct'),
     RegrTest('test_sys_settrace.py', core=True),
     RegrTest('test_sys_setprofile.py', core=True),
     RegrTest('test_sysconfig.py'),

diff --git a/pypy/translator/goal/app_main.py b/pypy/translator/goal/app_main.py
--- a/pypy/translator/goal/app_main.py
+++ b/pypy/translator/goal/app_main.py
@@ -227,6 +227,11 @@
                     executable = fn
                     break
     sys.executable = os.path.abspath(executable)
+    #
+    # 'sys.executable' should not end up being an non-existing file;
+    # just use '' in this case. (CPython issue #7774)
+    if not os.path.isfile(sys.executable):
+        sys.executable = ''
 
 def setup_initial_paths(ignore_environment=False, **extra):
     newpath = get_library_path(sys.executable)

From commits-noreply at bitbucket.org  Wed Feb  2 13:50:12 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Wed,  2 Feb 2011 13:50:12 +0100 (CET)
Subject: [pypy-svn] pypy arm-backend-2: merge default
Message-ID: <20110202125012.663FA2A2008@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backend-2
Changeset: r41549:7c31324be2de
Date: 2011-01-25 10:13 +0100
http://bitbucket.org/pypy/pypy/changeset/7c31324be2de/

Log:	merge default

diff --git a/pypy/jit/metainterp/compile.py b/pypy/jit/metainterp/compile.py
--- a/pypy/jit/metainterp/compile.py
+++ b/pypy/jit/metainterp/compile.py
@@ -80,11 +80,11 @@
 
 # ____________________________________________________________
 
-def compile_new_loop(metainterp, old_loop_tokens, greenkey, start,
-                     full_preamble_needed=True):
+def compile_new_loop(metainterp, old_loop_tokens, greenkey, start, start_resumedescr):
     """Try to compile a new loop by closing the current history back
     to the first operation.
     """
+    full_preamble_needed=True
     history = metainterp.history
     loop = create_empty_loop(metainterp)
     loop.inputargs = history.inputargs
@@ -102,6 +102,7 @@
     loop.preamble = create_empty_loop(metainterp, 'Preamble ')
     loop.preamble.inputargs = loop.inputargs
     loop.preamble.token = make_loop_token(len(loop.inputargs), jitdriver_sd)
+    loop.preamble.start_resumedescr = start_resumedescr
 
     try:
         old_loop_token = jitdriver_sd.warmstate.optimize_loop(
@@ -390,6 +391,12 @@
         self.copy_all_attrbutes_into(res)
         return res
 
+class ResumeAtPositionDescr(ResumeGuardDescr):
+    def _clone_if_mutable(self):
+        res = ResumeAtPositionDescr()
+        self.copy_all_attrbutes_into(res)
+        return res
+
 class ResumeGuardForcedDescr(ResumeGuardDescr):
 
     def __init__(self, metainterp_sd, jitdriver_sd):
@@ -574,10 +581,14 @@
     new_loop.operations = [op.clone() for op in metainterp.history.operations]
     metainterp_sd = metainterp.staticdata
     state = metainterp.jitdriver_sd.warmstate
+    if isinstance(resumekey, ResumeAtPositionDescr):
+        inline_short_preamble = False
+    else:
+        inline_short_preamble = True
     try:
         target_loop_token = state.optimize_bridge(metainterp_sd,
                                                   old_loop_tokens,
-                                                  new_loop)
+                                                  new_loop, inline_short_preamble)
     except InvalidLoop:
         # XXX I am fairly convinced that optimize_bridge cannot actually raise
         # InvalidLoop
@@ -588,34 +599,9 @@
         # know exactly what we must do (ResumeGuardDescr/ResumeFromInterpDescr)
         prepare_last_operation(new_loop, target_loop_token)
         resumekey.compile_and_attach(metainterp, new_loop)
-        compile_known_target_bridges(metainterp, new_loop)
         record_loop_or_bridge(metainterp_sd, new_loop)
     return target_loop_token
 
-# For backends that not supports emitting guards with preset jump
-# targets, emit mini-bridges containing the jump
-def compile_known_target_bridges(metainterp, bridge):
-    for op in bridge.operations:
-        if op.is_guard():
-            target = op.getjumptarget()
-            if target:
-                mini = create_empty_loop(metainterp, 'fallback')
-                mini.inputargs = op.getfailargs()[:]
-                jmp = ResOperation(rop.JUMP, mini.inputargs[:], None, target)
-                mini.operations = [jmp]
-                descr = op.getdescr()
-                assert isinstance(descr, ResumeGuardDescr)
-                mini.token = bridge.token
-                
-                #descr.compile_and_attach(metainterp, mini)
-                if not we_are_translated():
-                    descr._debug_suboperations = mini.operations
-                send_bridge_to_backend(metainterp.staticdata, descr,
-                                       mini.inputargs, mini.operations,
-                                       bridge.token)
-                record_loop_or_bridge(metainterp.staticdata, mini)
-
-
 def prepare_last_operation(new_loop, target_loop_token):
     op = new_loop.operations[-1]
     if not isinstance(target_loop_token, TerminatingLoopToken):

diff --git a/pypy/jit/codewriter/effectinfo.py b/pypy/jit/codewriter/effectinfo.py
--- a/pypy/jit/codewriter/effectinfo.py
+++ b/pypy/jit/codewriter/effectinfo.py
@@ -10,9 +10,9 @@
 
     # the 'extraeffect' field is one of the following values:
     EF_PURE                            = 0 #pure function (and cannot raise)
-    EF_CANNOT_RAISE                    = 1 #a function which cannot raise
-    EF_CAN_RAISE                       = 2 #normal function (can raise)
-    EF_LOOPINVARIANT                   = 3 #special: call it only once per loop
+    EF_LOOPINVARIANT                   = 1 #special: call it only once per loop
+    EF_CANNOT_RAISE                    = 2 #a function which cannot raise
+    EF_CAN_RAISE                       = 3 #normal function (can raise)
     EF_FORCES_VIRTUAL_OR_VIRTUALIZABLE = 4 #can raise and force virtualizables
 
     # the 'oopspecindex' field is one of the following values:
@@ -60,8 +60,13 @@
             return cls._cache[key]
         result = object.__new__(cls)
         result.readonly_descrs_fields = readonly_descrs_fields
-        result.write_descrs_fields = write_descrs_fields
-        result.write_descrs_arrays = write_descrs_arrays
+        if extraeffect == EffectInfo.EF_LOOPINVARIANT or \
+           extraeffect == EffectInfo.EF_PURE:            
+            result.write_descrs_fields = []
+            result.write_descrs_arrays = []
+        else:
+            result.write_descrs_fields = write_descrs_fields
+            result.write_descrs_arrays = write_descrs_arrays
         result.extraeffect = extraeffect
         result.oopspecindex = oopspecindex
         cls._cache[key] = result

diff --git a/pypy/translator/c/test/test_dtoa.py b/pypy/translator/c/test/test_dtoa.py
deleted file mode 100644
--- a/pypy/translator/c/test/test_dtoa.py
+++ /dev/null
@@ -1,92 +0,0 @@
-from __future__ import with_statement
-from pypy.translator.tool.cbuild import ExternalCompilationInfo
-from pypy.tool.autopath import pypydir
-from pypy.rpython.lltypesystem import lltype, rffi
-from pypy.rlib.rstring import StringBuilder
-import py
-
-includes = []
-libraries = []
-
-cdir = py.path.local(pypydir) / 'translator' / 'c'
-files = [cdir / 'src' / 'dtoa.c']
-include_dirs = [cdir]
-
-eci = ExternalCompilationInfo(
-    include_dirs = include_dirs,
-    libraries = libraries,
-    separate_module_files = files,
-    separate_module_sources = ['''
-        #include <stdlib.h>
-        #include <assert.h>
-        #define WITH_PYMALLOC
-        #include "src/obmalloc.c"
-    '''],
-    export_symbols = ['_Py_dg_strtod',
-                      '_Py_dg_dtoa',
-                      '_Py_dg_freedtoa',
-                      ],
-)
-
-dg_strtod = rffi.llexternal(
-    '_Py_dg_strtod', [rffi.CCHARP, rffi.CCHARPP], rffi.DOUBLE,
-    compilation_info=eci)
-
-dg_dtoa = rffi.llexternal(
-    '_Py_dg_dtoa', [rffi.DOUBLE, rffi.INT, rffi.INT,
-                    rffi.INTP, rffi.INTP, rffi.CCHARPP], rffi.CCHARP,
-    compilation_info=eci)
-
-dg_freedtoa = rffi.llexternal(
-    '_Py_dg_freedtoa', [rffi.CCHARP], lltype.Void,
-    compilation_info=eci)
-
-def strtod(input):
-    with lltype.scoped_alloc(rffi.CCHARPP.TO, 1) as end_ptr:
-        with rffi.scoped_str2charp(input) as ll_input:
-            result = dg_strtod(ll_input, end_ptr)
-            if end_ptr[0] and ord(end_ptr[0][0]):
-                offset = (rffi.cast(rffi.LONG, end_ptr[0]) -
-                          rffi.cast(rffi.LONG, ll_input))
-                raise ValueError("invalid input at position %d" % (offset,))
-            return result
-
-def dtoa(value, mode=0, precision=0):
-    builder = StringBuilder(20)
-    with lltype.scoped_alloc(rffi.INTP.TO, 1) as decpt_ptr:
-        with lltype.scoped_alloc(rffi.INTP.TO, 1) as sign_ptr:
-            with lltype.scoped_alloc(rffi.CCHARPP.TO, 1) as end_ptr:
-                output_ptr = dg_dtoa(value, mode, precision,
-                                     decpt_ptr, sign_ptr, end_ptr)
-                try:
-                    buflen = (rffi.cast(rffi.LONG, end_ptr[0]) -
-                              rffi.cast(rffi.LONG, output_ptr))
-                    intpart = rffi.cast(lltype.Signed, decpt_ptr[0])
-                    if intpart <= buflen:
-                        builder.append(rffi.charpsize2str(output_ptr, intpart))
-                    else:
-                        builder.append(rffi.charpsize2str(output_ptr, buflen))
-                        while buflen < intpart:
-                            builder.append('0')
-                            intpart -= 1
-                    builder.append('.')
-                    fracpart = buflen - intpart
-                    if fracpart > 0:
-                        ptr = rffi.ptradd(output_ptr, intpart)
-                        builder.append(rffi.charpsize2str(ptr, fracpart))
-                finally:
-                    dg_freedtoa(output_ptr)
-    return builder.build()
-
-def test_strtod():
-    assert strtod("12345") == 12345.0
-    assert strtod("1.1") == 1.1
-    assert strtod("3.47") == 3.47
-    raises(ValueError, strtod, "123A")
-
-def test_dtoa():
-    assert dtoa(3.47) == "3.47"
-    assert dtoa(1.1) == "1.1"
-    assert dtoa(12.3577) == "12.3577"
-    assert dtoa(10) == "10."
-    assert dtoa(1e100) == "1" + "0" * 100 + "."

diff --git a/pypy/module/pypyjit/test/test_pypy_c.py b/pypy/module/pypyjit/test/test_pypy_c.py
--- a/pypy/module/pypyjit/test/test_pypy_c.py
+++ b/pypy/module/pypyjit/test/test_pypy_c.py
@@ -150,6 +150,14 @@
         _, self.sliced_entrybridge, _ = \
                                     self.parse_rawloops(self.rawentrybridges)
 
+        from pypy.jit.tool.jitoutput import parse_prof
+        summaries  = logparser.extract_category(log, 'jit-summary')
+        if len(summaries) > 0:
+            self.jit_summary = parse_prof(summaries[-1])
+        else:
+            self.jit_summary = None
+        
+
     def parse_rawloops(self, rawloops):
         from pypy.jit.tool.oparser import parse
         loops = [parse(part, no_namespace=True) for part in rawloops]
@@ -1437,7 +1445,6 @@
                          count_debug_merge_point=False)
         
     def test_mod(self):
-        py.test.skip('Results are correct, but traces 1902 times (on trunk too).')
         avalues = ('a', 'b', 7, -42, 8)
         bvalues = ['b'] + range(-10, 0) + range(1,10)
         code = ''
@@ -1466,11 +1473,25 @@
 %s
                 i += 1
             return sa
-        ''' % code, 0, ([a1, b1], 2000 * res1),
+        ''' % code, 150, ([a1, b1], 2000 * res1),
                          ([a2, b2], 2000 * res2),
                          ([a3, b3], 2000 * res3),
                          count_debug_merge_point=False)
         
+    def test_dont_trace_every_iteration(self):
+        self.run_source('''
+        def main(a, b):
+            i = sa = 0
+            while i < 200:
+                if a > 0: pass
+                if 1 < b < 2: pass
+                sa += a % b
+                i += 1
+            return sa
+        ''', 11,  ([10, 20], 200 * (10 % 20)),
+                 ([-10, -20], 200 * (-10 % -20)),
+                        count_debug_merge_point=False)
+        assert self.jit_summary.tracing_no == 2
     def test_id_compare_optimization(self):
         # XXX: lower the instruction count, 35 is the old value.
         self.run_source("""

diff --git a/pypy/jit/metainterp/pyjitpl.py b/pypy/jit/metainterp/pyjitpl.py
--- a/pypy/jit/metainterp/pyjitpl.py
+++ b/pypy/jit/metainterp/pyjitpl.py
@@ -19,7 +19,7 @@
 from pypy.jit.metainterp.jitexc import JitException, get_llexception
 from pypy.rlib.rarithmetic import intmask
 from pypy.rlib.objectmodel import specialize
-from pypy.jit.codewriter.jitcode import JitCode, SwitchDictDescr
+from pypy.jit.codewriter.jitcode import JitCode, SwitchDictDescr, MissingLiveness
 from pypy.jit.codewriter import heaptracker
 from pypy.jit.metainterp.optimizeutil import RetraceLoop
 
@@ -819,8 +819,8 @@
         self.generate_guard(rop.GUARD_CLASS, box, [clsbox], resumepc=orgpc)
         return clsbox
 
-    @arguments("int")
-    def opimpl_loop_header(self, jdindex):
+    @arguments("int", "orgpc")
+    def opimpl_loop_header(self, jdindex, orgpc):
         self.metainterp.seen_loop_header_for_jdindex = jdindex
 
     def verify_green_args(self, jitdriver_sd, varargs):
@@ -832,12 +832,16 @@
     @arguments("orgpc", "int", "boxes3", "jitcode_position", "boxes3")
     def opimpl_jit_merge_point(self, orgpc, jdindex, greenboxes,
                                jcposition, redboxes):
+        resumedescr = compile.ResumeAtPositionDescr()
+        self.capture_resumedata(resumedescr, orgpc)
+        
         any_operation = len(self.metainterp.history.operations) > 0
         jitdriver_sd = self.metainterp.staticdata.jitdrivers_sd[jdindex]
         self.verify_green_args(jitdriver_sd, greenboxes)
         # xxx we may disable the following line in some context later
         self.debug_merge_point(jitdriver_sd, self.metainterp.in_recursion,
                                greenboxes)
+
         if self.metainterp.seen_loop_header_for_jdindex < 0:
             if not jitdriver_sd.no_loop_header or not any_operation:
                 return
@@ -848,6 +852,7 @@
             "found a loop_header for a JitDriver that does not match "
             "the following jit_merge_point's")
         self.metainterp.seen_loop_header_for_jdindex = -1
+        
         #
         if not self.metainterp.in_recursion:
             assert jitdriver_sd is self.metainterp.jitdriver_sd
@@ -857,7 +862,7 @@
             # much less expensive to blackhole out of.
             saved_pc = self.pc
             self.pc = orgpc
-            self.metainterp.reached_loop_header(greenboxes, redboxes)
+            self.metainterp.reached_loop_header(greenboxes, redboxes, resumedescr)
             self.pc = saved_pc
             # no exception, which means that the jit_merge_point did not
             # close the loop.  We have to put the possibly-modified list
@@ -1075,20 +1080,26 @@
             resumedescr = compile.ResumeGuardDescr()
         guard_op = metainterp.history.record(opnum, moreargs, None,
                                              descr=resumedescr)
+        self.capture_resumedata(resumedescr, resumepc)
+        self.metainterp.staticdata.profiler.count_ops(opnum, GUARDS)
+        # count
+        metainterp.attach_debug_info(guard_op)
+        return guard_op
+
+    def capture_resumedata(self, resumedescr, resumepc=-1):
+        metainterp = self.metainterp
         virtualizable_boxes = None
         if (metainterp.jitdriver_sd.virtualizable_info is not None or
             metainterp.jitdriver_sd.greenfield_info is not None):
             virtualizable_boxes = metainterp.virtualizable_boxes
         saved_pc = self.pc
-        if resumepc >= 0:
-            self.pc = resumepc
-        resume.capture_resumedata(metainterp.framestack, virtualizable_boxes,
-                                  metainterp.virtualref_boxes, resumedescr)
-        self.pc = saved_pc
-        self.metainterp.staticdata.profiler.count_ops(opnum, GUARDS)
-        # count
-        metainterp.attach_debug_info(guard_op)
-        return guard_op
+        try:
+            if resumepc >= 0:
+                self.pc = resumepc
+            resume.capture_resumedata(metainterp.framestack, virtualizable_boxes,
+                                      metainterp.virtualref_boxes, resumedescr)
+        finally:
+            self.pc = saved_pc
 
     def implement_guard_value(self, orgpc, box):
         """Promote the given Box into a Const.  Note: be careful, it's a
@@ -1192,7 +1203,7 @@
                     self.execute_varargs(rop.CALL, allboxes, descr, False))
             elif effect == effectinfo.EF_LOOPINVARIANT:
                 return self.execute_varargs(rop.CALL_LOOPINVARIANT, allboxes,
-                                            descr, True)
+                                            descr, False)
             else:
                 return self.execute_varargs(rop.CALL, allboxes, descr, True)
 
@@ -1703,8 +1714,13 @@
         self.current_merge_points = []
         self.resumekey = key
         self.seen_loop_header_for_jdindex = -1
-        try:
-            self.prepare_resume_from_failure(key.guard_opnum)
+        if isinstance(key, compile.ResumeAtPositionDescr):
+            self.seen_loop_header_for_jdindex = self.jitdriver_sd.index
+            dont_change_position = True
+        else:
+            dont_change_position = False
+        try:            
+            self.prepare_resume_from_failure(key.guard_opnum, dont_change_position)
             if self.resumekey_original_loop_token is None:   # very rare case
                 raise SwitchToBlackhole(ABORT_BRIDGE)
             self.interpret()
@@ -1734,7 +1750,7 @@
             else:
                 duplicates[box] = None
 
-    def reached_loop_header(self, greenboxes, redboxes):
+    def reached_loop_header(self, greenboxes, redboxes, resumedescr):
         duplicates = {}
         self.remove_consts_and_duplicates(redboxes, len(redboxes),
                                           duplicates)
@@ -1789,9 +1805,9 @@
                     bridge_arg_boxes = self.retracing_loop_from.live_arg_boxes
                     self.compile_bridge_and_loop(original_boxes, \
                                                  live_arg_boxes, start,
-                                                 bridge_arg_boxes)
+                                                 bridge_arg_boxes, resumedescr)
                 else:
-                    self.compile(original_boxes, live_arg_boxes, start)
+                    self.compile(original_boxes, live_arg_boxes, start, resumedescr)
                 # creation of the loop was cancelled!
                 #self.staticdata.log('cancelled, tracing more...')
                 self.staticdata.log('cancelled, stopping tracing')
@@ -1808,10 +1824,11 @@
         history.set_future_values(self.cpu, residual_args)
         return loop_token
 
-    def prepare_resume_from_failure(self, opnum):
+    def prepare_resume_from_failure(self, opnum, dont_change_position=False):
         frame = self.framestack[-1]
         if opnum == rop.GUARD_TRUE:     # a goto_if_not that jumps only now
-            frame.pc = frame.jitcode.follow_jump(frame.pc)
+            if not dont_change_position:
+                frame.pc = frame.jitcode.follow_jump(frame.pc)
         elif opnum == rop.GUARD_FALSE:     # a goto_if_not that stops jumping
             pass
         elif opnum == rop.GUARD_VALUE or opnum == rop.GUARD_CLASS:
@@ -1854,14 +1871,14 @@
         cell = self.jitdriver_sd.warmstate.jit_cell_at_key(greenkey)
         cell.set_compiled_merge_points(looptokens)
 
-    def compile(self, original_boxes, live_arg_boxes, start):
+    def compile(self, original_boxes, live_arg_boxes, start, start_resumedescr):
         num_green_args = self.jitdriver_sd.num_green_args
         self.history.inputargs = original_boxes[num_green_args:]
         greenkey = original_boxes[:num_green_args]
         old_loop_tokens = self.get_compiled_merge_points(greenkey)
         self.history.record(rop.JUMP, live_arg_boxes[num_green_args:], None)
         loop_token = compile.compile_new_loop(self, old_loop_tokens,
-                                              greenkey, start)
+                                              greenkey, start, start_resumedescr)
         if loop_token is not None: # raise if it *worked* correctly
             self.set_compiled_merge_points(greenkey, old_loop_tokens)
             raise GenerateMergePoint(live_arg_boxes, loop_token)
@@ -1887,7 +1904,7 @@
             self.history.operations.pop()     # remove the JUMP
 
     def compile_bridge_and_loop(self, original_boxes, live_arg_boxes, start,
-                                bridge_arg_boxes):
+                                bridge_arg_boxes, start_resumedescr):
         num_green_args = self.jitdriver_sd.num_green_args
         original_inputargs = self.history.inputargs
         greenkey = original_boxes[:num_green_args]
@@ -1896,7 +1913,7 @@
         self.history.inputargs = original_boxes[num_green_args:]
         greenkey = original_boxes[:num_green_args]
         self.history.record(rop.JUMP, live_arg_boxes[num_green_args:], None)
-        loop_token = compile.compile_new_loop(self, [], greenkey, start)
+        loop_token = compile.compile_new_loop(self, [], greenkey, start, start_resumedescr)
         self.history.operations.pop()     # remove the JUMP
         if loop_token is None:
             return

diff --git a/pypy/jit/metainterp/simple_optimize.py b/pypy/jit/metainterp/simple_optimize.py
--- a/pypy/jit/metainterp/simple_optimize.py
+++ b/pypy/jit/metainterp/simple_optimize.py
@@ -47,7 +47,7 @@
             jumpop.setdescr(loop.token)
         return None
 
-def optimize_bridge(metainterp_sd, old_loops, loop):
+def optimize_bridge(metainterp_sd, old_loops, loop, inline_short_preamble):
     optimize_loop(metainterp_sd, [], loop)
     jumpop = loop.operations[-1]
     if jumpop.getopnum() == rop.JUMP:

diff --git a/pypy/jit/codewriter/test/test_jtransform.py b/pypy/jit/codewriter/test/test_jtransform.py
--- a/pypy/jit/codewriter/test/test_jtransform.py
+++ b/pypy/jit/codewriter/test/test_jtransform.py
@@ -722,18 +722,19 @@
     tr = Transformer()
     tr.portal_jd = jd
     oplist = tr.rewrite_operation(op)
-    assert len(oplist) == 6
+    assert len(oplist) == 7
     assert oplist[0].opname == '-live-'
     assert oplist[1].opname == 'int_guard_value'
     assert oplist[1].args   == [v1]
     assert oplist[2].opname == '-live-'
     assert oplist[3].opname == 'int_guard_value'
     assert oplist[3].args   == [v2]
-    assert oplist[4].opname == 'jit_merge_point'
-    assert oplist[4].args[0].value == 42
-    assert list(oplist[4].args[1]) == [v1, v2]
-    assert list(oplist[4].args[4]) == [v3, v4]
-    assert oplist[5].opname == '-live-'
+    assert oplist[4].opname == '-live-'
+    assert oplist[5].opname == 'jit_merge_point'
+    assert oplist[5].args[0].value == 42
+    assert list(oplist[5].args[1]) == [v1, v2]
+    assert list(oplist[5].args[4]) == [v3, v4]
+    assert oplist[6].opname == '-live-'
 
 def test_getfield_gc():
     S = lltype.GcStruct('S', ('x', lltype.Char))

diff --git a/pypy/jit/metainterp/test/test_compile.py b/pypy/jit/metainterp/test/test_compile.py
--- a/pypy/jit/metainterp/test/test_compile.py
+++ b/pypy/jit/metainterp/test/test_compile.py
@@ -86,7 +86,7 @@
     metainterp.history.inputargs = loop.inputargs[:]
     #
     loop_tokens = []
-    loop_token = compile_new_loop(metainterp, loop_tokens, [], 0)
+    loop_token = compile_new_loop(metainterp, loop_tokens, [], 0, None)
     assert loop_tokens == [loop_token]
     assert loop_token.number == 1
     assert staticdata.globaldata.loopnumbering == 2
@@ -102,7 +102,7 @@
     metainterp.history.operations = loop.operations[:]
     metainterp.history.inputargs = loop.inputargs[:]
     #
-    loop_token_2 = compile_new_loop(metainterp, loop_tokens, [], 0)
+    loop_token_2 = compile_new_loop(metainterp, loop_tokens, [], 0, None)
     assert loop_token_2 is loop_token
     assert loop_tokens == [loop_token]
     assert len(cpu.seen) == 0

diff --git a/pypy/module/signal/interp_signal.py b/pypy/module/signal/interp_signal.py
--- a/pypy/module/signal/interp_signal.py
+++ b/pypy/module/signal/interp_signal.py
@@ -291,11 +291,12 @@
 #__________________________________________________________
 
 def timeval_from_double(d, timeval):
-    timeval.c_tv_sec = int(d)
-    timeval.c_tv_usec = int((d - int(d)) * 1000000)
+    rffi.setintfield(timeval, 'c_tv_sec', int(d))
+    rffi.setintfield(timeval, 'c_tv_usec', int((d - int(d)) * 1000000))
 
 def double_from_timeval(tv):
-    return tv.c_tv_sec + (tv.c_tv_usec / 1000000.0)
+    return rffi.getintfield(tv, 'c_tv_sec') + (
+        rffi.getintfield(tv, 'c_tv_usec') / 1000000.0)
 
 def itimer_retval(space, val):
     w_value = space.wrap(double_from_timeval(val.c_it_value))

diff --git a/pypy/jit/metainterp/blackhole.py b/pypy/jit/metainterp/blackhole.py
--- a/pypy/jit/metainterp/blackhole.py
+++ b/pypy/jit/metainterp/blackhole.py
@@ -9,7 +9,7 @@
 from pypy.jit.codewriter.jitcode import JitCode, SwitchDictDescr
 from pypy.jit.codewriter import heaptracker
 from pypy.jit.metainterp.jitexc import JitException, get_llexception, reraise
-
+from pypy.jit.metainterp.compile import ResumeAtPositionDescr
 
 def arguments(*argtypes, **kwds):
     resulttype = kwds.pop('returns', None)
@@ -1210,13 +1210,14 @@
             assert kind == 'v'
         return lltype.nullptr(rclass.OBJECTPTR.TO)
 
-    def _prepare_resume_from_failure(self, opnum):
+    def _prepare_resume_from_failure(self, opnum, dont_change_position=False):
         from pypy.jit.metainterp.resoperation import rop
         #
         if opnum == rop.GUARD_TRUE:
             # Produced directly by some goto_if_not_xxx() opcode that did not
             # jump, but which must now jump.  The pc is just after the opcode.
-            self.position = self.jitcode.follow_jump(self.position)
+            if not dont_change_position:
+                self.position = self.jitcode.follow_jump(self.position)
         #
         elif opnum == rop.GUARD_FALSE:
             # Produced directly by some goto_if_not_xxx() opcode that jumped,
@@ -1372,8 +1373,14 @@
         jitdriver_sd,
         resumedescr,
         all_virtuals)
+    if isinstance(resumedescr, ResumeAtPositionDescr):
+        dont_change_position = True
+    else:
+        dont_change_position = False
+
     current_exc = blackholeinterp._prepare_resume_from_failure(
-        resumedescr.guard_opnum)
+        resumedescr.guard_opnum, dont_change_position)
+        
     try:
         _run_forever(blackholeinterp, current_exc)
     finally:

diff --git a/pypy/interpreter/baseobjspace.py b/pypy/interpreter/baseobjspace.py
--- a/pypy/interpreter/baseobjspace.py
+++ b/pypy/interpreter/baseobjspace.py
@@ -1114,6 +1114,13 @@
                                  self.wrap('read-write buffer expected'))
         return buffer
 
+    def bufferstr_new_w(self, w_obj):
+        # Implement the "new buffer interface" (new in Python 2.7)
+        # returning an unwrapped string. It doesn't accept unicode
+        # strings
+        buffer = self.buffer_w(w_obj)
+        return buffer.as_str()
+
     def bufferstr_w(self, w_obj):
         # Directly returns an interp-level str.  Note that if w_obj is a
         # unicode string, this is different from str_w(buffer(w_obj)):

diff --git a/pypy/jit/codewriter/jtransform.py b/pypy/jit/codewriter/jtransform.py
--- a/pypy/jit/codewriter/jtransform.py
+++ b/pypy/jit/codewriter/jtransform.py
@@ -881,7 +881,9 @@
         op1 = SpaceOperation('jit_merge_point', args, None)
         op2 = SpaceOperation('-live-', [], None)
         # ^^^ we need a -live- for the case of do_recursive_call()
-        return ops + [op1, op2]
+        op3 = SpaceOperation('-live-', [], None)
+        # and one for inlined short preambles
+        return ops + [op3, op1, op2]
 
     def handle_jit_marker__loop_header(self, op, jitdriver):
         jd = self.callcontrol.jitdriver_sd_from_jitdriver(jitdriver)

diff --git a/pypy/jit/metainterp/optimizeopt/optimizer.py b/pypy/jit/metainterp/optimizeopt/optimizer.py
--- a/pypy/jit/metainterp/optimizeopt/optimizer.py
+++ b/pypy/jit/metainterp/optimizeopt/optimizer.py
@@ -436,9 +436,6 @@
         self.newoperations.append(op)
 
     def store_final_boxes_in_guard(self, op):
-        ###pendingfields = self.heap_op_optimizer.force_lazy_setfields_for_guard()
-        if op.getjumptarget():
-            return op
         descr = op.getdescr()
         assert isinstance(descr, compile.ResumeGuardDescr)
         modifier = resume.ResumeDataVirtualAdder(descr, self.resumedata_memo)

diff --git a/pypy/jit/metainterp/test/test_optimizeopt.py b/pypy/jit/metainterp/test/test_optimizeopt.py
--- a/pypy/jit/metainterp/test/test_optimizeopt.py
+++ b/pypy/jit/metainterp/test/test_optimizeopt.py
@@ -176,6 +176,15 @@
             metainterp_sd.virtualref_info = self.vrefinfo
         if hasattr(self, 'callinfocollection'):
             metainterp_sd.callinfocollection = self.callinfocollection
+        class FakeDescr(compile.ResumeGuardDescr):
+            class rd_snapshot:
+                class prev:
+                    prev = None
+                    boxes = []
+                boxes = []
+            def clone_if_mutable(self):
+                return self
+        loop.preamble.start_resumedescr = FakeDescr()
         optimize_loop_1(metainterp_sd, loop)
         #
 
@@ -691,12 +700,16 @@
         guard_nonnull(p0) []
         jump(p0)
         """
-        expected = """
+        preamble = """
         [p0]
         setfield_gc(p0, 5, descr=valuedescr)
         jump(p0)
         """
-        self.optimize_loop(ops, expected)
+        expected = """
+        [p0]
+        jump(p0)
+        """
+        self.optimize_loop(ops, expected, preamble)
 
     def test_const_guard_value(self):
         ops = """
@@ -2838,31 +2851,59 @@
 
     def test_call_assembler_invalidates_caches(self):
         ops = '''
-        [p1, i1]
+        [p1, i1, i4]
         setfield_gc(p1, i1, descr=valuedescr)
         i3 = call_assembler(i1, descr=asmdescr)
         setfield_gc(p1, i3, descr=valuedescr)
-        jump(p1, i3)
+        jump(p1, i4, i3)
         '''
-        self.optimize_loop(ops, ops)
+        self.optimize_loop(ops, ops, ops)
+
+    def test_call_assembler_invalidates_heap_knowledge(self):
+        ops = '''
+        [p1, i1, i4]
+        setfield_gc(p1, i1, descr=valuedescr)
+        i3 = call_assembler(i1, descr=asmdescr)
+        setfield_gc(p1, i1, descr=valuedescr)
+        jump(p1, i4, i3)
+        '''
+        self.optimize_loop(ops, ops, ops)
 
     def test_call_pure_invalidates_caches(self):
         # CALL_PURE should still force the setfield_gc() to occur before it
         ops = '''
-        [p1, i1]
+        [p1, i1, i4]
         setfield_gc(p1, i1, descr=valuedescr)
         i3 = call_pure(42, p1, descr=plaincalldescr)
         setfield_gc(p1, i3, descr=valuedescr)
-        jump(p1, i3)
+        jump(p1, i4, i3)
         '''
         expected = '''
-        [p1, i1]
+        [p1, i1, i4]
         setfield_gc(p1, i1, descr=valuedescr)
         i3 = call(p1, descr=plaincalldescr)
         setfield_gc(p1, i3, descr=valuedescr)
-        jump(p1, i3)
+        jump(p1, i4, i3)
         '''
-        self.optimize_loop(ops, expected)
+        self.optimize_loop(ops, expected, expected)
+
+    def test_call_pure_invalidates_heap_knowledge(self):
+        # CALL_PURE should still force the setfield_gc() to occur before it
+        ops = '''
+        [p1, i1, i4]
+        setfield_gc(p1, i1, descr=valuedescr)
+        i3 = call_pure(42, p1, descr=plaincalldescr)
+        setfield_gc(p1, i1, descr=valuedescr)
+        jump(p1, i4, i3)
+        '''
+        expected = '''
+        [p1, i1, i4]
+        setfield_gc(p1, i1, descr=valuedescr)
+        i3 = call(p1, descr=plaincalldescr)
+        setfield_gc(p1, i1, descr=valuedescr)
+        jump(p1, i4, i3)
+        '''
+        self.optimize_loop(ops, expected, expected)
 
     def test_call_pure_constant_folding(self):
         # CALL_PURE is not marked as is_always_pure(), because it is wrong
@@ -3384,7 +3425,6 @@
         """
         expected = """
         [i0]
-        i2 = int_add(i0, 10)
         jump(i0)
         """
         self.optimize_loop(ops, expected, preamble)
@@ -3675,10 +3715,9 @@
         jump(p0, i22)
         """
         expected = """
-        [p0, i22, i1]
+        [p0, i22]
         i331 = force_token()
-        setfield_gc(p0, i1, descr=valuedescr)
-        jump(p0, i22, i1)
+        jump(p0, i22)
         """
         self.optimize_loop(ops, expected)
 
@@ -4184,7 +4223,6 @@
         guard_no_overflow() []
         i2 = int_gt(i1, 1)
         guard_true(i2) []
-        i3 = int_sub(1, i0)
         jump(i0)
         """
         expected = """
@@ -4519,16 +4557,11 @@
         guard_true(i4) []
         i5 = int_gt(i3, 2)
         guard_true(i5) []
-        jump(i0, i1, i22)
-        """
-        expected = """
-        [i0, i1, i22]
-        i3 = int_mul(i22, i1)
-        i4 = int_lt(i3, 10)
-        guard_true(i4) []
-        i5 = int_gt(i3, 2)
-        guard_true(i5) []
-        jump(i0, i1, i22)
+        jump(i0, i1)
+        """
+        expected = """
+        [i0, i1]
+        jump(i0, i1)
         """
         self.optimize_loop(ops, expected, preamble)
 
@@ -4557,19 +4590,43 @@
         guard_true(i4) []
         i5 = int_ge(i3, 2)
         guard_true(i5) []
-        jump(i0, i1, i2)
-        """
-        expected = """
-        [i0, i1, i2]
-        i3 = int_sub(i2, i1)
-        i4 = int_le(i3, 10)
-        guard_true(i4) []
-        i5 = int_ge(i3, 2)
-        guard_true(i5) []
-        jump(i0, i1, i2)
+        jump(i0, i1)
+        """
+        expected = """
+        [i0, i1]
+        jump(i0, i1)
         """
         self.optimize_loop(ops, expected, preamble)
 
+    def test_invariant_ovf(self):
+        ops = """
+        [i0, i1, i10, i11, i12]
+        i2 = int_add_ovf(i0, i1)
+        guard_no_overflow() []
+        i3 = int_sub_ovf(i0, i1)
+        guard_no_overflow() []
+        i4 = int_mul_ovf(i0, i1)
+        guard_no_overflow() []
+        i24 = int_mul_ovf(i10, i11)
+        guard_no_overflow() []
+        i23 = int_sub_ovf(i10, i11)
+        guard_no_overflow() []
+        i22 = int_add_ovf(i10, i11)
+        guard_no_overflow() []
+        jump(i0, i1, i2, i3, i4)
+        """
+        expected = """
+        [i0, i1, i10, i11, i12]
+        i24 = int_mul_ovf(i10, i11)
+        guard_no_overflow() []
+        i23 = int_sub_ovf(i10, i11)
+        guard_no_overflow() []
+        i22 = int_add_ovf(i10, i11)
+        guard_no_overflow() []
+        jump(i0, i1, i10, i11, i12)
+        """
+        self.optimize_loop(ops, expected, ops)
+
     def test_value_proven_to_be_constant_after_two_iterations(self):
         class FakeDescr(AbstractDescr):
             def __init__(self, name):
@@ -4633,6 +4690,47 @@
         """
         self.optimize_loop(ops, expected)
 
+    def test_let_getfield_kill_setfields(self):
+        ops = """
+        [p0]
+        p1 = getfield_gc(p0, descr=valuedescr)
+        setfield_gc(p0, p1, descr=valuedescr)
+        setfield_gc(p0, p1, descr=valuedescr)
+        setfield_gc(p0, p0, descr=valuedescr)        
+        jump(p0)
+        """
+        preamble = """
+        [p0]
+        p1 = getfield_gc(p0, descr=valuedescr)
+        setfield_gc(p0, p0, descr=valuedescr)                
+        jump(p0)
+        """
+        expected = """
+        [p0]
+        jump(p0)
+        """
+        self.optimize_loop(ops, expected, preamble)
+
+    def test_let_getfield_kill_chained_setfields(self):
+        ops = """
+        [p0]
+        p1 = getfield_gc(p0, descr=valuedescr)
+        setfield_gc(p0, p0, descr=valuedescr)        
+        setfield_gc(p0, p1, descr=valuedescr)
+        setfield_gc(p0, p1, descr=valuedescr)
+        jump(p0)
+        """
+        preamble = """
+        [p0]
+        p1 = getfield_gc(p0, descr=valuedescr)
+        jump(p0)
+        """
+        expected = """
+        [p0]
+        jump(p0)
+        """
+        self.optimize_loop(ops, expected, preamble)
+
     def test_inputargs_added_by_forcing_jumpargs(self):
         # FXIME: Can this occur?
         ops = """
@@ -5282,6 +5380,7 @@
         # more generally, supporting non-constant but virtual cases is
         # not obvious, because of the exception UnicodeDecodeError that
         # can be raised by ll_str2unicode()
+        
 
 
 

diff --git a/pypy/jit/metainterp/test/test_basic.py b/pypy/jit/metainterp/test/test_basic.py
--- a/pypy/jit/metainterp/test/test_basic.py
+++ b/pypy/jit/metainterp/test/test_basic.py
@@ -1843,6 +1843,47 @@
                           'int_add': 1, 'int_mul': 1, 'int_sub': 2,
                           'int_gt': 2, 'jump': 2})
 
+    def test_multiple_specialied_versions_array(self):
+        myjitdriver = JitDriver(greens = [], reds = ['idx', 'y', 'x', 'res',
+                                                     'array'])
+        class Base:
+            def __init__(self, val):
+                self.val = val
+        class A(Base):
+            def binop(self, other):
+                return A(self.val + other.val)
+        class B(Base):
+            def binop(self, other):
+                return B(self.val - other.val)
+        def f(x, y):
+            res = x
+            array = [1, 2, 3]
+            array[1] = 7
+            idx = 0
+            while y > 0:
+                myjitdriver.can_enter_jit(idx=idx, y=y, x=x, res=res,
+                                          array=array)
+                myjitdriver.jit_merge_point(idx=idx, y=y, x=x, res=res,
+                                            array=array)
+                res = res.binop(x)
+                res.val += array[idx] + array[1]
+                if y < 7:
+                    idx = 2
+                y -= 1
+            return res
+        def g(x, y):
+            a1 = f(A(x), y)
+            a2 = f(A(x), y)
+            b1 = f(B(x), y)
+            b2 = f(B(x), y)
+            assert a1.val == a2.val
+            assert b1.val == b2.val
+            return a1.val + b1.val
+        res = self.meta_interp(g, [6, 14])
+        assert res == g(6, 14)
+        self.check_loop_count(8)
+        self.check_loops(getarrayitem_gc=16, everywhere=True)
+
     def test_multiple_specialied_versions_bridge(self):
         myjitdriver = JitDriver(greens = [], reds = ['y', 'x', 'z', 'res'])
         class Base:
@@ -1869,20 +1910,78 @@
         def g(x, y):
             a1 = f(A(x), y, A(x))
             a2 = f(A(x), y, A(x))
+            assert a1.val == a2.val
             b1 = f(B(x), y, B(x))
             b2 = f(B(x), y, B(x))
+            assert b1.val == b2.val
             c1 = f(B(x), y, A(x))
             c2 = f(B(x), y, A(x))
+            assert c1.val == c2.val
             d1 = f(A(x), y, B(x))
             d2 = f(A(x), y, B(x))
-            assert a1.val == a2.val
-            assert b1.val == b2.val
-            assert c1.val == c2.val
             assert d1.val == d2.val
             return a1.val + b1.val + c1.val + d1.val
         res = self.meta_interp(g, [3, 14])
         assert res == g(3, 14)
 
+    def test_failing_inlined_guard(self):
+        myjitdriver = JitDriver(greens = [], reds = ['y', 'x', 'z', 'res'])
+        class Base:
+            def __init__(self, val):
+                self.val = val
+            def getval(self):
+                return self.val
+        class A(Base):
+            def binop(self, other):
+                return A(self.getval() + other.getval())
+        class B(Base):
+            def binop(self, other):
+                return B(self.getval() * other.getval())
+        def f(x, y, z):
+            res = x
+            while y > 0:
+                myjitdriver.can_enter_jit(y=y, x=x, z=z, res=res)
+                myjitdriver.jit_merge_point(y=y, x=x, z=z, res=res)
+                res = res.binop(x)
+                y -= 1
+                if y < 8:
+                    x = z
+            return res
+        def g(x, y):
+            c1 = f(A(x), y, B(x))
+            c2 = f(A(x), y, B(x))
+            assert c1.val == c2.val
+            return c1.val
+        res = self.meta_interp(g, [3, 16])
+        assert res == g(3, 16)
+
+    def test_inlined_guard_in_short_preamble(self):
+        myjitdriver = JitDriver(greens = [], reds = ['y', 'x', 'z', 'res'])
+        class A:
+            def __init__(self, val):
+                self.val = val
+            def getval(self):
+                return self.val
+            def binop(self, other):
+                return A(self.getval() + other.getval())
+        def f(x, y, z):
+            res = x
+            while y > 0:
+                myjitdriver.can_enter_jit(y=y, x=x, z=z, res=res)
+                myjitdriver.jit_merge_point(y=y, x=x, z=z, res=res)
+                res = res.binop(x)
+                y -= 1
+                if y < 7:
+                    x = z
+            return res
+        def g(x, y):
+            a1 = f(A(x), y, A(x))
+            a2 = f(A(x), y, A(x))
+            assert a1.val == a2.val
+            return a1.val
+        res = self.meta_interp(g, [3, 14])
+        assert res == g(3, 14)
+
     def test_specialied_bridge(self):
         myjitdriver = JitDriver(greens = [], reds = ['y', 'x', 'res'])
         class A:
@@ -1974,6 +2073,26 @@
         self.check_loops(guard_true=4, guard_class=0, int_add=2, int_mul=2,
                          guard_false=2)
 
+    def test_dont_trace_every_iteration(self):
+        myjitdriver = JitDriver(greens = [], reds = ['a', 'b', 'i', 'sa'])
+        
+        def main(a, b):
+            i = sa = 0
+            #while i < 200:
+            while i < 200:
+                myjitdriver.can_enter_jit(a=a, b=b, i=i, sa=sa)
+                myjitdriver.jit_merge_point(a=a, b=b, i=i, sa=sa)
+                if a > 0: pass
+                if b < 2: pass
+                sa += a % b
+                i += 1
+            return sa
+        def g():
+            return main(10, 20) + main(-10, -20)
+        res = self.meta_interp(g, [])
+        assert res == g()
+        self.check_enter_count(3)
+
     def test_current_trace_length(self):
         myjitdriver = JitDriver(greens = ['g'], reds = ['x'])
         @dont_look_inside

diff --git a/pypy/jit/metainterp/resoperation.py b/pypy/jit/metainterp/resoperation.py
--- a/pypy/jit/metainterp/resoperation.py
+++ b/pypy/jit/metainterp/resoperation.py
@@ -198,7 +198,6 @@
 class GuardResOp(ResOpWithDescr):
 
     _fail_args = None
-    _jump_target = None
 
     def getfailargs(self):
         return self._fail_args
@@ -206,22 +205,14 @@
     def setfailargs(self, fail_args):
         self._fail_args = fail_args
 
-    def getjumptarget(self):
-        return self._jump_target
-
-    def setjumptarget(self, jump_target):
-        self._jump_target = jump_target
-
     def copy_and_change(self, opnum, args=None, result=None, descr=None):
         newop = AbstractResOp.copy_and_change(self, opnum, args, result, descr)
         newop.setfailargs(self.getfailargs())
-        newop.setjumptarget(self.getjumptarget())
         return newop
 
     def clone(self):
         newop = AbstractResOp.clone(self)
         newop.setfailargs(self.getfailargs())
-        newop.setjumptarget(self.getjumptarget())        
         return newop
 
 

diff --git a/pypy/jit/metainterp/optimizeopt/__init__.py b/pypy/jit/metainterp/optimizeopt/__init__.py
--- a/pypy/jit/metainterp/optimizeopt/__init__.py
+++ b/pypy/jit/metainterp/optimizeopt/__init__.py
@@ -6,17 +6,19 @@
 from pypy.jit.metainterp.optimizeopt.string import OptString
 from pypy.jit.metainterp.optimizeopt.unroll import optimize_unroll, OptInlineShortPreamble
 
-def optimize_loop_1(metainterp_sd, loop, unroll=True):
+def optimize_loop_1(metainterp_sd, loop, unroll=True, inline_short_preamble=True):
     """Optimize loop.operations to remove internal overheadish operations. 
     """
     opt_str = OptString()
-    optimizations = [OptInlineShortPreamble(),
-                     OptIntBounds(),
+    optimizations = [OptIntBounds(),
                      OptRewrite(),
                      OptVirtualize(),
                      opt_str,
                      OptHeap(),
                     ]
+    if inline_short_preamble:
+        optimizations = [OptInlineShortPreamble()] +  optimizations
+        
     if metainterp_sd.jit_ffi:
         from pypy.jit.metainterp.optimizeopt.fficall import OptFfiCall
         optimizations = optimizations + [
@@ -31,6 +33,6 @@
         optimizer = Optimizer(metainterp_sd, loop, optimizations)
         optimizer.propagate_all_forward()
 
-def optimize_bridge_1(metainterp_sd, bridge):
+def optimize_bridge_1(metainterp_sd, bridge, inline_short_preamble=True):
     """The same, but for a bridge. """
-    optimize_loop_1(metainterp_sd, bridge, False)
+    optimize_loop_1(metainterp_sd, bridge, False, inline_short_preamble)

From commits-noreply at bitbucket.org  Wed Feb  2 13:50:13 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Wed,  2 Feb 2011 13:50:13 +0100 (CET)
Subject: [pypy-svn] pypy arm-backend-2: merge default
Message-ID: <20110202125013.43C692A2008@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backend-2
Changeset: r41550:e7af2d652594
Date: 2011-01-25 16:27 +0100
http://bitbucket.org/pypy/pypy/changeset/e7af2d652594/

Log:	merge default

diff --git a/pypy/jit/tl/pypyjit.py b/pypy/jit/tl/pypyjit.py
--- a/pypy/jit/tl/pypyjit.py
+++ b/pypy/jit/tl/pypyjit.py
@@ -30,7 +30,7 @@
     BACKEND = 'c'
 
 config = get_pypy_config(translating=True)
-config.translation.backendopt.inline_threshold = 0
+config.translation.backendopt.inline_threshold = 0.1
 config.translation.gc = 'boehm'
 config.objspace.nofaking = True
 config.translating = True

diff --git a/.hgignore b/.hgignore
--- a/.hgignore
+++ b/.hgignore
@@ -60,3 +60,4 @@
 ^pypy/doc/image/lattice3\.png$
 ^pypy/doc/image/stackless_informal\.png$
 ^pypy/doc/image/parsing_example.+\.png$
+^compiled
\ No newline at end of file

From commits-noreply at bitbucket.org  Wed Feb  2 13:50:14 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Wed,  2 Feb 2011 13:50:14 +0100 (CET)
Subject: [pypy-svn] pypy arm-backend-2: merge default
Message-ID: <20110202125014.D44BA2A2017@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backend-2
Changeset: r41551:710630f93423
Date: 2011-01-26 10:31 +0100
http://bitbucket.org/pypy/pypy/changeset/710630f93423/

Log:	merge default

diff --git a/pypy/jit/codewriter/support.py b/pypy/jit/codewriter/support.py
--- a/pypy/jit/codewriter/support.py
+++ b/pypy/jit/codewriter/support.py
@@ -190,7 +190,7 @@
 def _ll_2_int_floordiv_zer(x, y):
     if y == 0:
         raise ZeroDivisionError
-    return llop.int_floordiv_zer(lltype.Signed, x, y)
+    return llop.int_floordiv(lltype.Signed, x, y)
 
 def _ll_2_int_mod_ovf_zer(x, y):
     if y == 0:

From commits-noreply at bitbucket.org  Wed Feb  2 13:50:15 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Wed,  2 Feb 2011 13:50:15 +0100 (CET)
Subject: [pypy-svn] pypy arm-backend-2: merge default
Message-ID: <20110202125015.AC3662A2018@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backend-2
Changeset: r41552:331986dc1e48
Date: 2011-01-26 14:44 +0100
http://bitbucket.org/pypy/pypy/changeset/331986dc1e48/

Log:	merge default


From commits-noreply at bitbucket.org  Wed Feb  2 13:50:28 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Wed,  2 Feb 2011 13:50:28 +0100 (CET)
Subject: [pypy-svn] pypy arm-backend-2: merge default
Message-ID: <20110202125028.E52C52A2006@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backend-2
Changeset: r41553:ac3fe5c04e9b
Date: 2011-01-29 17:30 +0100
http://bitbucket.org/pypy/pypy/changeset/ac3fe5c04e9b/

Log:	merge default

diff --git a/pypy/rpython/llinterp.py b/pypy/rpython/llinterp.py
--- a/pypy/rpython/llinterp.py
+++ b/pypy/rpython/llinterp.py
@@ -1045,20 +1045,6 @@
         except OverflowError:
             self.make_llexception()
 
-    def op_llong_neg_ovf(self, x):
-        assert type(x) is r_longlong
-        try:
-            return ovfcheck(-x)
-        except OverflowError:
-            self.make_llexception()
-
-    def op_llong_abs_ovf(self, x):
-        assert type(x) is r_longlong
-        try:
-            return ovfcheck(abs(x))
-        except OverflowError:
-            self.make_llexception()
-
     def op_int_lshift_ovf(self, x, y):
         assert isinstance(x, int)
         assert isinstance(y, int)

diff --git a/pypy/interpreter/argument.py b/pypy/interpreter/argument.py
--- a/pypy/interpreter/argument.py
+++ b/pypy/interpreter/argument.py
@@ -154,6 +154,8 @@
         # unpack the ** arguments
         space = self.space
         if space.isinstance_w(w_starstararg, space.w_dict):
+            if not space.is_true(w_starstararg):
+                return False # don't call unpackiterable - it's jit-opaque
             keys_w = space.unpackiterable(w_starstararg)
         else:
             try:

diff --git a/pypy/module/cpyext/api.py b/pypy/module/cpyext/api.py
--- a/pypy/module/cpyext/api.py
+++ b/pypy/module/cpyext/api.py
@@ -199,6 +199,8 @@
             error = CANNOT_FAIL
     if type(error) is int:
         error = rffi.cast(restype, error)
+    expect_integer = (isinstance(restype, lltype.Primitive) and
+                      rffi.cast(restype, 0) == 0)
 
     def decorate(func):
         func_name = func.func_name
@@ -268,6 +270,9 @@
                             return None
                         else:
                             return api_function.error_value
+                    if not we_are_translated():
+                        got_integer = isinstance(res, (int, long, float))
+                        assert got_integer == expect_integer
                     if res is None:
                         return None
                     elif isinstance(res, Reference):


diff --git a/pypy/objspace/std/callmethod.py b/pypy/objspace/std/callmethod.py
--- a/pypy/objspace/std/callmethod.py
+++ b/pypy/objspace/std/callmethod.py
@@ -104,7 +104,10 @@
         if w_self is None:
             f.popvalue()    # removes w_self, which is None
         w_callable = f.popvalue()
-        w_result = f.space.call_args(w_callable, args)
+        if f.is_being_profiled and function.is_builtin_code(w_callable):
+            w_result = f.space.call_args_and_c_profile(f, w_callable, args)
+        else:
+            w_result = f.space.call_args(w_callable, args)
         rstack.resume_point("CALL_METHOD_KW", f, returns=w_result)
     f.pushvalue(w_result)
 

diff --git a/pypy/jit/codewriter/jtransform.py b/pypy/jit/codewriter/jtransform.py
--- a/pypy/jit/codewriter/jtransform.py
+++ b/pypy/jit/codewriter/jtransform.py
@@ -39,6 +39,7 @@
     def optimize_block(self, block):
         if block.operations == ():
             return
+        self.remove_longlong_constants(block)
         self.vable_array_vars = {}
         self.vable_flags = {}
         renamings = {}
@@ -134,6 +135,55 @@
         block.exits = block.exits[:1]
         block.exitswitch = None
 
+    def remove_longlong_constants(self, block):
+        # remove all Constant({Un}signedLongLong), and replace them with
+        # cast_int_to_longlong(Constant(Signed)) or
+        # two_ints_to_longlong(Constant(Signed), Constant(Signed)).
+        operations = []
+        all_constants = {}
+        #
+        def _get_const_as_var(c):
+            v = all_constants.get(c)
+            if v is None:
+                from pypy.rlib.rarithmetic import intmask
+                v = varoftype(c.concretetype)
+                value = int(c.value)
+                c_hi = Constant(intmask(value >> 32), lltype.Signed)
+                c_lo = Constant(intmask(value), lltype.Signed)
+                if c_lo.value == value:
+                    # a long long constant, but it fits in 32 bits
+                    op1 = SpaceOperation('cast_int_to_longlong', [c_lo], v)
+                else:
+                    # a 64-bit long long constant, requires two ints
+                    op1 = SpaceOperation('two_ints_to_longlong', [c_lo, c_hi],
+                                         v)
+                operations.append(op1)
+                all_constants[c] = v
+            return v
+        #
+        for op in block.operations:
+            for i, v in enumerate(op.args):
+                if (isinstance(v, Constant) and
+                        self._is_longlong(v.concretetype)):
+                    args = op.args[:]
+                    args[i] = _get_const_as_var(v)
+                    op = SpaceOperation(op.opname, args, op.result)
+            operations.append(op)
+        #
+        last_op = None
+        if block.exitswitch == c_last_exception:
+            last_op = operations.pop()
+        for link in block.exits:
+            for i, v in enumerate(link.args):
+                if (isinstance(v, Constant) and
+                        self._is_longlong(v.concretetype)):
+                    args = link.args[:]
+                    args[i] = _get_const_as_var(v)
+                    link.args = args
+        if last_op is not None:
+            operations.append(last_op)
+        block.operations = operations
+
     # ----------
 
     def follow_constant_exit(self, block):
@@ -199,7 +249,6 @@
 
     rewrite_op_cast_pointer = rewrite_op_same_as
     rewrite_op_cast_opaque_ptr = rewrite_op_same_as   # rlib.rerased
-    def rewrite_op_cast_primitive(self, op): pass
     def rewrite_op_cast_bool_to_int(self, op): pass
     def rewrite_op_cast_bool_to_uint(self, op): pass
     def rewrite_op_cast_char_to_int(self, op): pass
@@ -424,6 +473,15 @@
     rewrite_op_int_mod_zer     = _do_builtin_call
     rewrite_op_int_lshift_ovf  = _do_builtin_call
     rewrite_op_int_abs         = _do_builtin_call
+    rewrite_op_llong_abs          = _do_builtin_call
+    rewrite_op_llong_floordiv     = _do_builtin_call
+    rewrite_op_llong_floordiv_zer = _do_builtin_call
+    rewrite_op_llong_mod          = _do_builtin_call
+    rewrite_op_llong_mod_zer      = _do_builtin_call
+    rewrite_op_ullong_floordiv     = _do_builtin_call
+    rewrite_op_ullong_floordiv_zer = _do_builtin_call
+    rewrite_op_ullong_mod          = _do_builtin_call
+    rewrite_op_ullong_mod_zer      = _do_builtin_call
     rewrite_op_gc_identityhash = _do_builtin_call
     rewrite_op_gc_id           = _do_builtin_call
 
@@ -779,6 +837,116 @@
         return result
 
     # ----------
+    # Long longs, for 32-bit only.  Supported operations are left unmodified,
+    # and unsupported ones are turned into a call to a function from
+    # jit.codewriter.support.
+
+    if lltype.SignedLongLong != lltype.Signed:
+        @staticmethod
+        def _is_longlong(TYPE):
+            return (TYPE == lltype.SignedLongLong or
+                    TYPE == lltype.UnsignedLongLong)
+    else:
+        # on 64-bit, _is_longlong() returns always False
+        @staticmethod
+        def _is_longlong(TYPE):
+            return False
+
+    for _op, _oopspec in [('llong_invert',  'INVERT'),
+                          ('ullong_invert', 'INVERT'),
+                          ('llong_lt',      'LT'),
+                          ('llong_le',      'LE'),
+                          ('llong_eq',      'EQ'),
+                          ('llong_ne',      'NE'),
+                          ('llong_gt',      'GT'),
+                          ('llong_ge',      'GE'),
+                          ('ullong_lt',     'ULT'),
+                          ('ullong_le',     'ULE'),
+                          ('ullong_eq',     'EQ'),
+                          ('ullong_ne',     'NE'),
+                          ('ullong_gt',     'UGT'),
+                          ('ullong_ge',     'UGE'),
+                          ('llong_add',     'ADD'),
+                          ('llong_sub',     'SUB'),
+                          ('llong_mul',     'MUL'),
+                          ('llong_and',     'AND'),
+                          ('llong_or',      'OR'),
+                          ('llong_xor',     'XOR'),
+                          ('ullong_add',    'ADD'),
+                          ('ullong_sub',    'SUB'),
+                          ('ullong_mul',    'MUL'),
+                          ('ullong_and',    'AND'),
+                          ('ullong_or',     'OR'),
+                          ('ullong_xor',    'XOR'),
+                          ('llong_lshift',  'LSHIFT'),
+                          ('llong_rshift',  'RSHIFT'),
+                          ('ullong_lshift', 'LSHIFT'),
+                          ('ullong_rshift', 'URSHIFT'),
+                          ('cast_int_to_longlong',     'FROM_INT'),
+                          ('truncate_longlong_to_int', 'TO_INT'),
+                          ('cast_float_to_longlong',   'FROM_FLOAT'),
+                          ('cast_longlong_to_float',   'TO_FLOAT'),
+                          # internal pseuso-operation:
+                          ('two_ints_to_longlong',     'FROM_TWO_INTS'),
+                          ]:
+        exec py.code.Source('''
+            def rewrite_op_%s(self, op):
+                args = op.args
+                op1 = self.prepare_builtin_call(op, "llong_%s", args)
+                op2 = self._handle_oopspec_call(op1, args,
+                                                EffectInfo.OS_LLONG_%s,
+                                                EffectInfo.EF_PURE)
+                return op2
+        ''' % (_op, _oopspec.lower(), _oopspec)).compile()
+
+    def _normalize(self, oplist):
+        if isinstance(oplist, SpaceOperation):
+            return [oplist]
+        else:
+            assert type(oplist) is list
+            return oplist
+
+    def rewrite_op_llong_neg(self, op):
+        v = varoftype(lltype.SignedLongLong)
+        op0 = SpaceOperation('cast_int_to_longlong',
+                             [Constant(0, lltype.Signed)],
+                             v)
+        args = [v, op.args[0]]
+        op1 = SpaceOperation('llong_sub', args, op.result)
+        return (self._normalize(self.rewrite_operation(op0)) +
+                self._normalize(self.rewrite_operation(op1)))
+
+    def rewrite_op_llong_is_true(self, op):
+        v = varoftype(lltype.SignedLongLong)
+        op0 = SpaceOperation('cast_int_to_longlong',
+                             [Constant(0, lltype.Signed)],
+                             v)
+        args = [op.args[0], v]
+        op1 = SpaceOperation('llong_ne', args, op.result)
+        return (self._normalize(self.rewrite_operation(op0)) +
+                self._normalize(self.rewrite_operation(op1)))
+
+    rewrite_op_ullong_is_true = rewrite_op_llong_is_true
+
+    def rewrite_op_cast_primitive(self, op):
+        fromll = self._is_longlong(op.args[0].concretetype)
+        toll   = self._is_longlong(op.result.concretetype)
+        if fromll != toll:
+            args = op.args
+            if fromll:
+                opname = 'truncate_longlong_to_int'
+            else:
+                from pypy.rpython.lltypesystem import rffi
+                if rffi.cast(op.args[0].concretetype, -1) < 0:
+                    opname = 'cast_int_to_longlong'
+                else:
+                    opname = 'two_ints_to_longlong'
+                    c_hi = Constant(0, lltype.Signed)
+                    args = [args[0], c_hi]
+            op1 = SpaceOperation(opname, args, op.result)
+            return self.rewrite_operation(op1)
+
+    # ----------
     # Renames, from the _old opname to the _new one.
     # The new operation is optionally further processed by rewrite_operation().
     for _old, _new in [('bool_not', 'int_is_zero'),
@@ -1107,9 +1275,11 @@
     # Strings and Unicodes.
 
     def _handle_oopspec_call(self, op, args, oopspecindex, extraeffect=None):
-        calldescr = self.callcontrol.getcalldescr(op, oopspecindex)
-        if extraeffect:
-            calldescr.get_extra_info().extraeffect = extraeffect
+        calldescr = self.callcontrol.getcalldescr(op, oopspecindex,
+                                                  extraeffect)
+        if extraeffect is not None:
+            assert (type(calldescr) is str      # for tests
+                    or calldescr.get_extra_info().extraeffect == extraeffect)
         if isinstance(op.args[0].value, str):
             pass  # for tests only
         else:

diff --git a/pypy/jit/metainterp/test/test_basic.py b/pypy/jit/metainterp/test/test_basic.py
--- a/pypy/jit/metainterp/test/test_basic.py
+++ b/pypy/jit/metainterp/test/test_basic.py
@@ -7,6 +7,7 @@
 from pypy.jit.metainterp.warmspot import ll_meta_interp, get_stats
 from pypy.jit.backend.llgraph import runner
 from pypy.jit.metainterp import pyjitpl, history
+from pypy.jit.metainterp.warmstate import set_future_value
 from pypy.jit.codewriter.policy import JitPolicy, StopAtXPolicy
 from pypy import conftest
 from pypy.rlib.rarithmetic import ovfcheck
@@ -14,7 +15,8 @@
 from pypy.rpython.lltypesystem import lltype, llmemory
 from pypy.rpython.ootypesystem import ootype
 
-def _get_jitcodes(testself, CPUClass, func, values, type_system):
+def _get_jitcodes(testself, CPUClass, func, values, type_system,
+                  supports_longlong=False, **kwds):
     from pypy.jit.codewriter import support, codewriter
     from pypy.jit.metainterp import simple_optimize
 
@@ -57,7 +59,9 @@
     cpu = CPUClass(rtyper, stats, None, False)
     cw = codewriter.CodeWriter(cpu, [FakeJitDriverSD()])
     testself.cw = cw
-    cw.find_all_graphs(JitPolicy())
+    policy = JitPolicy()
+    policy.set_supports_longlong(supports_longlong)
+    cw.find_all_graphs(policy)
     #
     testself.warmrunnerstate = FakeWarmRunnerState()
     testself.warmrunnerstate.cpu = cpu
@@ -119,6 +123,29 @@
     else:
         raise Exception("FAILED")
 
+def _run_with_machine_code(testself, args):
+    metainterp = testself.metainterp
+    num_green_args = metainterp.jitdriver_sd.num_green_args
+    loop_tokens = metainterp.get_compiled_merge_points(args[:num_green_args])
+    if len(loop_tokens) != 1:
+        return NotImplemented
+    # a loop was successfully created by _run_with_pyjitpl(); call it
+    cpu = metainterp.cpu
+    for i in range(len(args) - num_green_args):
+        x = args[num_green_args + i]
+        typecode = history.getkind(lltype.typeOf(x))
+        set_future_value(cpu, i, x, typecode)
+    faildescr = cpu.execute_token(loop_tokens[0])
+    assert faildescr.__class__.__name__.startswith('DoneWithThisFrameDescr')
+    if metainterp.jitdriver_sd.result_type == history.INT:
+        return cpu.get_latest_value_int(0)
+    elif metainterp.jitdriver_sd.result_type == history.REF:
+        return cpu.get_latest_value_ref(0)
+    elif metainterp.jitdriver_sd.result_type == history.FLOAT:
+        return cpu.get_latest_value_float(0)
+    else:
+        return None
+
 
 class JitMixin:
     basic = True
@@ -156,12 +183,15 @@
 
     def interp_operations(self, f, args, **kwds):
         # get the JitCodes for the function f
-        _get_jitcodes(self, self.CPUClass, f, args, self.type_system)
+        _get_jitcodes(self, self.CPUClass, f, args, self.type_system, **kwds)
         # try to run it with blackhole.py
         result1 = _run_with_blackhole(self, args)
         # try to run it with pyjitpl.py
         result2 = _run_with_pyjitpl(self, args)
         assert result1 == result2
+        # try to run it by running the code compiled just before
+        result3 = _run_with_machine_code(self, args)
+        assert result1 == result3 or result3 == NotImplemented
         return result1
 
     def check_history(self, expected=None, **isns):
@@ -1261,20 +1291,6 @@
         res = self.interp_operations(f, [5])
         assert res == f(5)
 
-    def test_long_long(self):
-        from pypy.rlib.rarithmetic import r_longlong, intmask
-        def g(n, m, o):
-            # This function should be completely marked as residual by
-            # codewriter.py on 32-bit platforms.  On 64-bit platforms,
-            # this function should be JITted and the test should pass too.
-            n = r_longlong(n)
-            m = r_longlong(m)
-            return intmask((n*m) // o)
-        def f(n, m, o):
-            return g(n, m, o) // 3
-        res = self.interp_operations(f, [1000000000, 90, 91])
-        assert res == (1000000000 * 90 // 91) // 3
-
     def test_free_object(self):
         import weakref
         from pypy.rlib import rgc

diff --git a/pypy/jit/backend/llgraph/runner.py b/pypy/jit/backend/llgraph/runner.py
--- a/pypy/jit/backend/llgraph/runner.py
+++ b/pypy/jit/backend/llgraph/runner.py
@@ -86,6 +86,7 @@
 
 class BaseCPU(model.AbstractCPU):
     supports_floats = True
+    supports_longlong = True
 
     def __init__(self, rtyper, stats=None, opts=None,
                  translate_support_code=False,

diff --git a/pypy/translator/c/src/math.c b/pypy/translator/c/src/math.c
deleted file mode 100644
--- a/pypy/translator/c/src/math.c
+++ /dev/null
@@ -1,256 +0,0 @@
-/* Definitions of some C99 math library functions, for those platforms
-   that don't implement these functions already. */
-
-#include <errno.h>
-
-/* The following macros are copied from CPython header files */
-
-#ifdef _MSC_VER
-#include <float.h>
-#define PyPy_IS_NAN _isnan
-#define PyPy_IS_INFINITY(X) (!_finite(X) && !_isnan(X))
-#define copysign _copysign
-#else
-#define PyPy_IS_NAN(X) ((X) != (X))
-#define PyPy_IS_INFINITY(X) ((X) &&                                   \
-                             (Py_FORCE_DOUBLE(X)*0.5 == Py_FORCE_DOUBLE(X)))
-#endif
-
-#undef PyPy_NAN
-
-int
-_pypy_math_isinf(double x)
-{
-    return PyPy_IS_INFINITY(x);
-}
-
-int
-_pypy_math_isnan(double x)
-{
-    return PyPy_IS_NAN(x);
-}
-
-/* The following copyright notice applies to the original
-   implementations of acosh, asinh and atanh. */
-
-/*
- * ====================================================
- * Copyright (C) 1993 by Sun Microsystems, Inc. All rights reserved.
- *
- * Developed at SunPro, a Sun Microsystems, Inc. business.
- * Permission to use, copy, modify, and distribute this
- * software is freely granted, provided that this notice
- * is preserved.
- * ====================================================
- */
-
-double _pypy_math_log1p(double x);
-
-static const double ln2 = 6.93147180559945286227E-01;
-static const double two_pow_m28 = 3.7252902984619141E-09; /* 2**-28 */
-static const double two_pow_p28 = 268435456.0; /* 2**28 */
-static const double zero = 0.0;
-
-/* acosh(x)
- * Method :
- *      Based on
- *            acosh(x) = log [ x + sqrt(x*x-1) ]
- *      we have
- *            acosh(x) := log(x)+ln2, if x is large; else
- *            acosh(x) := log(2x-1/(sqrt(x*x-1)+x)) if x>2; else
- *            acosh(x) := log1p(t+sqrt(2.0*t+t*t)); where t=x-1.
- *
- * Special cases:
- *      acosh(x) is NaN with signal if x<1.
- *      acosh(NaN) is NaN without signal.
- */
-
-double
-_pypy_math_acosh(double x)
-{
-    if (PyPy_IS_NAN(x)) {
-        return x+x;
-    }
-    if (x < 1.) {                       /* x < 1;  return a signaling NaN */
-        errno = EDOM;
-#ifdef PyPy_NAN
-        return PyPy_NAN;
-#else
-        return (x-x)/(x-x);
-#endif
-    }
-    else if (x >= two_pow_p28) {        /* x > 2**28 */
-        if (PyPy_IS_INFINITY(x)) {
-            return x+x;
-        } else {
-            return log(x)+ln2;                  /* acosh(huge)=log(2x) */
-        }
-    }
-    else if (x == 1.) {
-        return 0.0;                             /* acosh(1) = 0 */
-    }
-    else if (x > 2.) {                          /* 2 < x < 2**28 */
-        double t = x*x;
-        return log(2.0*x - 1.0 / (x + sqrt(t - 1.0)));
-    }
-    else {                              /* 1 < x <= 2 */
-        double t = x - 1.0;
-        return _pypy_math_log1p(t + sqrt(2.0*t + t*t));
-    }
-}
-
-
-/* asinh(x)
- * Method :
- *      Based on
- *              asinh(x) = sign(x) * log [ |x| + sqrt(x*x+1) ]
- *      we have
- *      asinh(x) := x  if  1+x*x=1,
- *               := sign(x)*(log(x)+ln2)) for large |x|, else
- *               := sign(x)*log(2|x|+1/(|x|+sqrt(x*x+1))) if|x|>2, else
- *               := sign(x)*log1p(|x| + x^2/(1 + sqrt(1+x^2)))
- */
-
-double
-_pypy_math_asinh(double x)
-{
-    double w;
-    double absx = fabs(x);
-
-    if (PyPy_IS_NAN(x) || PyPy_IS_INFINITY(x)) {
-        return x+x;
-    }
-    if (absx < two_pow_m28) {           /* |x| < 2**-28 */
-        return x;               /* return x inexact except 0 */
-    }
-    if (absx > two_pow_p28) {           /* |x| > 2**28 */
-        w = log(absx)+ln2;
-    }
-    else if (absx > 2.0) {              /* 2 < |x| < 2**28 */
-        w = log(2.0*absx + 1.0 / (sqrt(x*x + 1.0) + absx));
-    }
-    else {                              /* 2**-28 <= |x| < 2= */
-        double t = x*x;
-        w = _pypy_math_log1p(absx + t / (1.0 + sqrt(1.0 + t)));
-    }
-    return copysign(w, x);
-
-}
-
-/* atanh(x)
- * Method :
- *    1.Reduced x to positive by atanh(-x) = -atanh(x)
- *    2.For x>=0.5
- *                1           2x                          x
- *      atanh(x) = --- * log(1 + -------) = 0.5 * log1p(2 * --------)
- *                2          1 - x                    1 - x
- *
- *      For x<0.5
- *      atanh(x) = 0.5*log1p(2x+2x*x/(1-x))
- *
- * Special cases:
- *      atanh(x) is NaN if |x| >= 1 with signal;
- *      atanh(NaN) is that NaN with no signal;
- *
- */
-
-double
-_pypy_math_atanh(double x)
-{
-    double absx;
-    double t;
-
-    if (PyPy_IS_NAN(x)) {
-        return x+x;
-    }
-    absx = fabs(x);
-    if (absx >= 1.) {                   /* |x| >= 1 */
-        errno = EDOM;
-#ifdef PyPy_NAN
-        return PyPy_NAN;
-#else
-        return x/zero;
-#endif
-    }
-    if (absx < two_pow_m28) {           /* |x| < 2**-28 */
-        return x;
-    }
-    if (absx < 0.5) {                   /* |x| < 0.5 */
-        t = absx+absx;
-        t = 0.5 * _pypy_math_log1p(t + t*absx / (1.0 - absx));
-    }
-    else {                              /* 0.5 <= |x| <= 1.0 */
-        t = 0.5 * _pypy_math_log1p((absx + absx) / (1.0 - absx));
-    }
-    return copysign(t, x);
-}
-
-/* Mathematically, expm1(x) = exp(x) - 1.  The expm1 function is designed
-   to avoid the significant loss of precision that arises from direct
-   evaluation of the expression exp(x) - 1, for x near 0. */
-
-double
-_pypy_math_expm1(double x)
-{
-    /* For abs(x) >= log(2), it's safe to evaluate exp(x) - 1 directly; this
-       also works fine for infinities and nans.
-
-       For smaller x, we can use a method due to Kahan that achieves close to
-       full accuracy.
-    */
-
-    if (fabs(x) < 0.7) {
-        double u;
-        u  = exp(x);
-        if (u == 1.0)
-            return x;
-        else
-            return (u - 1.0) * x / log(u);
-    }
-    else
-        return exp(x) - 1.0;
-}
-
-/* log1p(x) = log(1+x).  The log1p function is designed to avoid the
-   significant loss of precision that arises from direct evaluation when x is
-   small. */
-
-double
-_pypy_math_log1p(double x)
-{
-    /* For x small, we use the following approach.  Let y be the nearest float
-       to 1+x, then
-
-      1+x = y * (1 - (y-1-x)/y)
-
-       so log(1+x) = log(y) + log(1-(y-1-x)/y).  Since (y-1-x)/y is tiny, the
-       second term is well approximated by (y-1-x)/y.  If abs(x) >=
-       DBL_EPSILON/2 or the rounding-mode is some form of round-to-nearest
-       then y-1-x will be exactly representable, and is computed exactly by
-       (y-1)-x.
-
-       If abs(x) < DBL_EPSILON/2 and the rounding mode is not known to be
-       round-to-nearest then this method is slightly dangerous: 1+x could be
-       rounded up to 1+DBL_EPSILON instead of down to 1, and in that case
-       y-1-x will not be exactly representable any more and the result can be
-       off by many ulps.  But this is easily fixed: for a floating-point
-       number |x| < DBL_EPSILON/2., the closest floating-point number to
-       log(1+x) is exactly x.
-    */
-
-    double y;
-    if (fabs(x) < DBL_EPSILON/2.) {
-        return x;
-    } else if (-0.5 <= x && x <= 1.) {
-    /* WARNING: it's possible than an overeager compiler
-       will incorrectly optimize the following two lines
-       to the equivalent of "return log(1.+x)". If this
-       happens, then results from log1p will be inaccurate
-       for small x. */
-        y = 1.+x;
-        return log(y)-((y-1.)-x)/y;
-    } else {
-    /* NaNs and infinities should end up here */
-        return log(1.+x);
-    }
-}

diff --git a/pypy/jit/tl/pypyjit.py b/pypy/jit/tl/pypyjit.py
--- a/pypy/jit/tl/pypyjit.py
+++ b/pypy/jit/tl/pypyjit.py
@@ -30,7 +30,6 @@
     BACKEND = 'c'
 
 config = get_pypy_config(translating=True)
-config.translation.backendopt.inline_threshold = 0.1
 config.translation.gc = 'boehm'
 config.objspace.nofaking = True
 config.translating = True

diff --git a/pypy/jit/backend/x86/regalloc.py b/pypy/jit/backend/x86/regalloc.py
--- a/pypy/jit/backend/x86/regalloc.py
+++ b/pypy/jit/backend/x86/regalloc.py
@@ -5,7 +5,7 @@
 import os
 from pypy.jit.metainterp.history import (Box, Const, ConstInt, ConstPtr,
                                          ResOperation, BoxPtr, ConstFloat,
-                                         LoopToken, INT, REF, FLOAT)
+                                         BoxFloat, LoopToken, INT, REF, FLOAT)
 from pypy.jit.backend.x86.regloc import *
 from pypy.rpython.lltypesystem import lltype, ll2ctypes, rffi, rstr
 from pypy.rlib.objectmodel import we_are_translated
@@ -13,12 +13,14 @@
 from pypy.jit.backend.llsupport import symbolic
 from pypy.jit.backend.x86.jump import remap_frame_layout
 from pypy.jit.codewriter import heaptracker
+from pypy.jit.codewriter.effectinfo import EffectInfo
 from pypy.jit.metainterp.resoperation import rop
 from pypy.jit.backend.llsupport.descr import BaseFieldDescr, BaseArrayDescr
 from pypy.jit.backend.llsupport.descr import BaseCallDescr, BaseSizeDescr
 from pypy.jit.backend.llsupport.regalloc import FrameManager, RegisterManager,\
      TempBox, compute_vars_longevity, compute_loop_consts
 from pypy.jit.backend.x86.arch import WORD, FRAME_FIXED_SIZE, IS_X86_32, IS_X86_64
+from pypy.rlib.rarithmetic import r_longlong, r_uint
 
 class X86RegisterManager(RegisterManager):
 
@@ -72,6 +74,12 @@
         rffi.cast(rffi.CArrayPtr(rffi.DOUBLE), adr)[0] = c.getfloat()
         return ConstFloatLoc(adr)
 
+    def convert_to_imm_16bytes_align(self, c):
+        adr = self.assembler.datablockwrapper.malloc_aligned(16, 16)
+        rffi.cast(rffi.CArrayPtr(rffi.DOUBLE), adr)[0] = c.getfloat()
+        rffi.cast(rffi.CArrayPtr(rffi.DOUBLE), adr)[1] = 0.0
+        return ConstFloatLoc(adr)
+
     def after_call(self, v):
         # the result is stored in st0, but we don't have this around,
         # so genop_call will move it to some frame location immediately
@@ -229,6 +237,14 @@
             return self.rm.force_allocate_reg(var, forbidden_vars,
                                               selected_reg, need_lower_byte)
 
+    def load_xmm_aligned_16_bytes(self, var, forbidden_vars=[]):
+        # Load 'var' in a register; but if it is a constant, we can return
+        # a 16-bytes-aligned ConstFloatLoc.
+        if isinstance(var, Const):
+            return self.xrm.convert_to_imm_16bytes_align(var)
+        else:
+            return self.xrm.make_sure_var_in_reg(var, forbidden_vars)
+
     def _update_bindings(self, locs, inputargs):
         # XXX this should probably go to llsupport/regalloc.py
         used = {}
@@ -269,6 +285,11 @@
             self.assembler.dump('%s <- %s(%s)' % (result_loc, op, arglocs))
         self.assembler.regalloc_perform(op, arglocs, result_loc)
 
+    def PerformLLong(self, op, arglocs, result_loc):
+        if not we_are_translated():
+            self.assembler.dump('%s <- %s(%s)' % (result_loc, op, arglocs))
+        self.assembler.regalloc_perform_llong(op, arglocs, result_loc)
+
     def locs_for_fail(self, guard_op):
         return [self.loc(v) for v in guard_op.getfailargs()]
 
@@ -583,6 +604,110 @@
         self.Perform(op, [loc0], loc1)
         self.rm.possibly_free_var(op.getarg(0))
 
+    def _consider_llong_binop_xx(self, op):
+        # must force both arguments into xmm registers, because we don't
+        # know if they will be suitably aligned.  Exception: if the second
+        # argument is a constant, we can ask it to be aligned to 16 bytes.
+        args = [op.getarg(1), op.getarg(2)]
+        loc1 = self.load_xmm_aligned_16_bytes(args[1])
+        loc0 = self.xrm.force_result_in_reg(op.result, args[0], args)
+        self.PerformLLong(op, [loc0, loc1], loc0)
+        self.xrm.possibly_free_vars(args)
+
+    def _consider_llong_eq_ne_xx(self, op):
+        # must force both arguments into xmm registers, because we don't
+        # know if they will be suitably aligned.  Exception: if they are
+        # constants, we can ask them to be aligned to 16 bytes.
+        args = [op.getarg(1), op.getarg(2)]
+        loc1 = self.load_xmm_aligned_16_bytes(args[0])
+        loc2 = self.load_xmm_aligned_16_bytes(args[1], args)
+        tmpxvar = TempBox()
+        loc3 = self.xrm.force_allocate_reg(tmpxvar, args)
+        self.xrm.possibly_free_var(tmpxvar)
+        loc0 = self.rm.force_allocate_reg(op.result, need_lower_byte=True)
+        self.PerformLLong(op, [loc1, loc2, loc3], loc0)
+        self.xrm.possibly_free_vars(args)
+
+    def _maybe_consider_llong_lt(self, op):
+        # XXX just a special case for now
+        from pypy.rlib.longlong2float import longlong2float
+        box = op.getarg(2)
+        if not isinstance(box, ConstFloat):
+            return False
+        if not (box.value == longlong2float(r_longlong(0))):
+            return False
+        # "x < 0"
+        box = op.getarg(1)
+        assert isinstance(box, BoxFloat)
+        loc1 = self.xrm.make_sure_var_in_reg(box)
+        loc0 = self.rm.force_allocate_reg(op.result)
+        self.PerformLLong(op, [loc1], loc0)
+        self.xrm.possibly_free_var(box)
+        return True
+
+    def _consider_llong_to_int(self, op):
+        # accept an argument in a xmm register or in the stack
+        loc1 = self.xrm.loc(op.getarg(1))
+        loc0 = self.rm.force_allocate_reg(op.result)
+        self.PerformLLong(op, [loc1], loc0)
+        self.xrm.possibly_free_var(op.getarg(1))
+
+    def _loc_of_const_longlong(self, value64):
+        from pypy.rlib.longlong2float import longlong2float
+        c = ConstFloat(longlong2float(value64))
+        return self.xrm.convert_to_imm(c)
+
+    def _consider_llong_from_int(self, op):
+        assert IS_X86_32
+        loc0 = self.xrm.force_allocate_reg(op.result)
+        box = op.getarg(1)
+        if isinstance(box, ConstInt):
+            loc1 = self._loc_of_const_longlong(r_longlong(box.value))
+            loc2 = None    # unused
+        else:
+            # requires the argument to be in eax, and trash edx.
+            loc1 = self.rm.make_sure_var_in_reg(box, selected_reg=eax)
+            tmpvar = TempBox()
+            self.rm.force_allocate_reg(tmpvar, [box], selected_reg=edx)
+            self.rm.possibly_free_var(tmpvar)
+            tmpxvar = TempBox()
+            loc2 = self.xrm.force_allocate_reg(tmpxvar, [op.result])
+            self.xrm.possibly_free_var(tmpxvar)
+        self.PerformLLong(op, [loc1, loc2], loc0)
+        self.rm.possibly_free_var(box)
+
+    def _consider_llong_from_two_ints(self, op):
+        assert IS_X86_32
+        box1 = op.getarg(1)
+        box2 = op.getarg(2)
+        loc0 = self.xrm.force_allocate_reg(op.result)
+        #
+        if isinstance(box1, ConstInt) and isinstance(box2, ConstInt):
+            # all-constant arguments: load the result value in a single step
+            value64 = r_longlong(box2.value) << 32
+            value64 |= r_longlong(r_uint(box1.value))
+            loc1 = self._loc_of_const_longlong(value64)
+            loc2 = None    # unused
+            loc3 = None    # unused
+        #
+        else:
+            tmpxvar = TempBox()
+            loc3 = self.xrm.force_allocate_reg(tmpxvar, [op.result])
+            self.xrm.possibly_free_var(tmpxvar)
+            #
+            if isinstance(box1, ConstInt):
+                loc1 = self._loc_of_const_longlong(r_longlong(box1.value))
+            else:
+                loc1 = self.rm.make_sure_var_in_reg(box1)
+            #
+            if isinstance(box2, ConstInt):
+                loc2 = self._loc_of_const_longlong(r_longlong(box2.value))
+            else:
+                loc2 = self.rm.make_sure_var_in_reg(box2, [box1])
+        #
+        self.PerformLLong(op, [loc1, loc2, loc3], loc0)
+        self.rm.possibly_free_vars_for_op(op)
+
     def _call(self, op, arglocs, force_store=[], guard_not_forced_op=None):
         save_all_regs = guard_not_forced_op is not None
         self.rm.before_call(force_store, save_all_regs=save_all_regs)
@@ -614,6 +739,31 @@
                    guard_not_forced_op=guard_not_forced_op)
 
     def consider_call(self, op):
+        if IS_X86_32:
+            # support for some of the llong operations,
+            # which only exist on x86-32
+            effectinfo = op.getdescr().get_extra_info()
+            if effectinfo is not None:
+                oopspecindex = effectinfo.oopspecindex
+                if oopspecindex in (EffectInfo.OS_LLONG_ADD,
+                                    EffectInfo.OS_LLONG_SUB,
+                                    EffectInfo.OS_LLONG_AND,
+                                    EffectInfo.OS_LLONG_OR,
+                                    EffectInfo.OS_LLONG_XOR):
+                    return self._consider_llong_binop_xx(op)
+                if oopspecindex == EffectInfo.OS_LLONG_TO_INT:
+                    return self._consider_llong_to_int(op)
+                if oopspecindex == EffectInfo.OS_LLONG_FROM_INT:
+                    return self._consider_llong_from_int(op)
+                if oopspecindex == EffectInfo.OS_LLONG_FROM_TWO_INTS:
+                    return self._consider_llong_from_two_ints(op)
+                if (oopspecindex == EffectInfo.OS_LLONG_EQ or
+                    oopspecindex == EffectInfo.OS_LLONG_NE):
+                    return self._consider_llong_eq_ne_xx(op)
+                if oopspecindex == EffectInfo.OS_LLONG_LT:
+                    if self._maybe_consider_llong_lt(op):
+                        return
+        #
         self._consider_call(op)
 
     def consider_call_may_force(self, op, guard_op):

diff --git a/pypy/interpreter/test/test_executioncontext.py b/pypy/interpreter/test/test_executioncontext.py
--- a/pypy/interpreter/test/test_executioncontext.py
+++ b/pypy/interpreter/test/test_executioncontext.py
@@ -164,6 +164,25 @@
         events = space.unwrap(w_events)
         assert events == ['return', 'c_call', 'c_return', 'return', 'c_call']
 
+    def test_c_call_setprofile_kwargs(self):
+        space = self.space
+        w_events = space.appexec([], """():
+        import sys
+        l = []
+        def profile(frame, event, arg):
+            l.append(event)
+
+        def bar():
+            sys.setprofile(profile)
+            [].sort(reverse=True)
+            sys.setprofile(None)
+
+        bar()
+        return l
+        """)
+        events = space.unwrap(w_events)
+        assert events == ['c_call', 'c_return', 'c_call']
+
     def test_c_call_setprofile_strange_method(self):
         space = self.space
         w_events = space.appexec([], """():

diff --git a/pypy/module/imp/test/test_import.py b/pypy/module/imp/test/test_import.py
--- a/pypy/module/imp/test/test_import.py
+++ b/pypy/module/imp/test/test_import.py
@@ -49,6 +49,7 @@
              absolute   = "from __future__ import absolute_import\nimport string",
              relative_b = "from __future__ import absolute_import\nfrom . import string",
              relative_c = "from __future__ import absolute_import\nfrom .string import inpackage",
+             relative_f = "from .os import sep",
              )
     setuppkg("pkg.pkg1", 
              a          = '',
@@ -189,6 +190,13 @@
     def test_import_keywords(self):
         __import__(name='sys', level=0)
 
+    def test_import_by_filename(self):
+        import pkg.a
+        filename = pkg.a.__file__
+        assert filename.endswith('.py')
+        exc = raises(ImportError, __import__, filename[:-3])
+        assert exc.value.message == "Import by filename is not supported."
+
     def test_import_badcase(self):
         def missing(name):
             try:
@@ -269,7 +277,7 @@
     def test_import_relative_partial_success(self):
         def imp():
             import pkg_r.inpkg
-        raises(ImportError,imp)
+        raises(ImportError, imp)
 
     def test_import_builtin_inpackage(self):
         def imp():
@@ -347,6 +355,12 @@
         from pkg import relative_b
         assert relative_b.string.inpackage == 1
 
+    def test_no_relative_import(self):
+        def imp():
+            from pkg import relative_f
+        exc = raises(ImportError, imp)
+        assert exc.value.message == "No module named pkg.os"
+
     def test_future_relative_import_level_1(self):
         from pkg import relative_c
         assert relative_c.inpackage == 1
@@ -722,6 +736,7 @@
     def test_write_compiled_module(self):
         space = self.space
         pathname = _testfilesource()
+        os.chmod(pathname, 0777)
         stream = streamio.open_file_as_stream(pathname, "r")
         try:
             w_ret = importing.parse_source_module(space,
@@ -733,10 +748,12 @@
         assert type(pycode) is pypy.interpreter.pycode.PyCode
 
         cpathname = str(udir.join('cpathname.pyc'))
+        mode = 0777
         mtime = 12345
         importing.write_compiled_module(space,
                                         pycode,
                                         cpathname,
+                                        mode,
                                         mtime)
 
         # check
@@ -746,6 +763,9 @@
         assert ret is not None
         ret.close()
 
+        # Check that the executable bit was removed
+        assert os.stat(cpathname).st_mode & 0111 == 0
+
         # read compiled module
         stream = streamio.open_file_as_stream(cpathname, "rb")
         try:

diff --git a/pypy/translator/c/src/signals.h b/pypy/translator/c/src/signals.h
--- a/pypy/translator/c/src/signals.h
+++ b/pypy/translator/c/src/signals.h
@@ -117,8 +117,11 @@
         pypysig_counter.value = -1;
       }
 
-    if (wakeup_fd != -1)
-      write(wakeup_fd, "\0", 1);
+    if (wakeup_fd != -1) 
+      {
+        write(wakeup_fd, "\0", 1);
+        /* the return value is ignored here */
+      }
 }
 
 void pypysig_setflag(int signum)

diff --git a/pypy/jit/backend/llgraph/llimpl.py b/pypy/jit/backend/llgraph/llimpl.py
--- a/pypy/jit/backend/llgraph/llimpl.py
+++ b/pypy/jit/backend/llgraph/llimpl.py
@@ -23,12 +23,16 @@
 
 from pypy.rlib.objectmodel import ComputedIntSymbolic, we_are_translated
 from pypy.rlib.rarithmetic import ovfcheck
+from pypy.rlib.rarithmetic import r_longlong, r_ulonglong, r_uint
+from pypy.rlib.longlong2float import longlong2float, float2longlong
 
 import py
 from pypy.tool.ansi_print import ansi_log
 log = py.log.Producer('runner')
 py.log.setconsumer('runner', ansi_log)
 
+IS_32_BIT = r_ulonglong is not r_uint
+
 
 def _from_opaque(opq):
     return opq._obj.externalobj
@@ -1071,14 +1075,26 @@
 def cast_from_ptr(TYPE, x):
     return lltype.cast_opaque_ptr(TYPE, x)
 
-def cast_to_float(x):      # not really a cast, just a type check
+def cast_to_float(x):
+    if isinstance(x, float):
+        return x      # common case
+    if IS_32_BIT:
+        if isinstance(x, r_longlong):
+            return longlong2float(x)
+        if isinstance(x, r_ulonglong):
+            return longlong2float(rffi.cast(lltype.SignedLongLong, x))
+    raise TypeError(type(x))
+
+def cast_from_float(TYPE, x):
     assert isinstance(x, float)
-    return x
-
-def cast_from_float(TYPE, x):   # not really a cast, just a type check
-    assert TYPE is lltype.Float
-    assert isinstance(x, float)
-    return x
+    if TYPE is lltype.Float:
+        return x
+    if IS_32_BIT:
+        if TYPE is lltype.SignedLongLong:
+            return float2longlong(x)
+        if TYPE is lltype.UnsignedLongLong:
+            return r_ulonglong(float2longlong(x))
+    raise TypeError(TYPE)
 
 
 def new_frame(is_oo, cpu):
@@ -1518,7 +1534,9 @@
                     assert n == 'r'
                 x = argsiter_r.next()
                 x = cast_from_ptr(TYPE, x)
-            elif TYPE is lltype.Float:
+            elif TYPE is lltype.Float or (
+                    IS_32_BIT and TYPE in (lltype.SignedLongLong,
+                                           lltype.UnsignedLongLong)):
                 if args_in_order is not None:
                     n = orderiter.next()
                     assert n == 'f'

diff --git a/pypy/jit/codewriter/test/test_jtransform.py b/pypy/jit/codewriter/test/test_jtransform.py
--- a/pypy/jit/codewriter/test/test_jtransform.py
+++ b/pypy/jit/codewriter/test/test_jtransform.py
@@ -90,7 +90,7 @@
         self.callinfocollection = FakeCallInfoCollection()
     def guess_call_kind(self, op):
         return 'builtin'
-    def getcalldescr(self, op, oopspecindex=None):
+    def getcalldescr(self, op, oopspecindex=None, extraeffect=None):
         assert oopspecindex is not None    # in this test
         EI = effectinfo.EffectInfo
         if oopspecindex != EI.OS_ARRAYCOPY:

diff --git a/pypy/jit/codewriter/effectinfo.py b/pypy/jit/codewriter/effectinfo.py
--- a/pypy/jit/codewriter/effectinfo.py
+++ b/pypy/jit/codewriter/effectinfo.py
@@ -46,6 +46,32 @@
     OS_LIBFFI_PREPARE           = 60
     OS_LIBFFI_PUSH_ARG          = 61
     OS_LIBFFI_CALL              = 62
+    #
+    OS_LLONG_INVERT             = 69
+    OS_LLONG_ADD                = 70
+    OS_LLONG_SUB                = 71
+    OS_LLONG_MUL                = 72
+    OS_LLONG_LT                 = 73
+    OS_LLONG_LE                 = 74
+    OS_LLONG_EQ                 = 75
+    OS_LLONG_NE                 = 76
+    OS_LLONG_GT                 = 77
+    OS_LLONG_GE                 = 78
+    OS_LLONG_AND                = 79
+    OS_LLONG_OR                 = 80
+    OS_LLONG_LSHIFT             = 81
+    OS_LLONG_RSHIFT             = 82
+    OS_LLONG_XOR                = 83
+    OS_LLONG_FROM_INT           = 84
+    OS_LLONG_TO_INT             = 85
+    OS_LLONG_FROM_FLOAT         = 86
+    OS_LLONG_TO_FLOAT           = 87
+    OS_LLONG_ULT                = 88
+    OS_LLONG_ULE                = 89
+    OS_LLONG_UGT                = 90
+    OS_LLONG_UGE                = 91
+    OS_LLONG_URSHIFT            = 92
+    OS_LLONG_FROM_TWO_INTS      = 93
 
     def __new__(cls, readonly_descrs_fields,
                 write_descrs_fields, write_descrs_arrays,

diff --git a/pypy/jit/metainterp/pyjitpl.py b/pypy/jit/metainterp/pyjitpl.py
--- a/pypy/jit/metainterp/pyjitpl.py
+++ b/pypy/jit/metainterp/pyjitpl.py
@@ -1163,7 +1163,7 @@
                     src_r += 1
                     if box.type == history.REF:
                         break
-            elif kind == history.FLOAT:
+            elif kind == history.FLOAT or kind == 'L':    # long long
                 while True:
                     box = argboxes[src_f]
                     src_f += 1

diff --git a/pypy/module/imp/importing.py b/pypy/module/imp/importing.py
--- a/pypy/module/imp/importing.py
+++ b/pypy/module/imp/importing.py
@@ -10,7 +10,7 @@
 from pypy.interpreter.error import OperationError, operationerrfmt
 from pypy.interpreter.baseobjspace import W_Root, ObjSpace, Wrappable
 from pypy.interpreter.eval import Code
-from pypy.rlib import streamio, jit
+from pypy.rlib import streamio, jit, rposix
 from pypy.rlib.streamio import StreamErrors
 from pypy.rlib.rarithmetic import intmask
 from pypy.rlib.objectmodel import we_are_translated, specialize
@@ -127,42 +127,58 @@
     if (level != 0 and
         w_globals is not None and
         space.isinstance_w(w_globals, space.w_dict)):
+
         ctxt_w_name = space.finditem(w_globals, w('__name__'))
         ctxt_w_path = space.finditem(w_globals, w('__path__'))
+
+        ctxt_name = None
         if ctxt_w_name is not None:
             try:
                 ctxt_name = space.str_w(ctxt_w_name)
             except OperationError, e:
                 if not e.match(space, space.w_TypeError):
                     raise
-            else:
-                ctxt_name_prefix_parts = ctxt_name.split('.')
-                if level > 0:
-                    n = len(ctxt_name_prefix_parts)-level+1
-                    assert n>=0
-                    ctxt_name_prefix_parts = ctxt_name_prefix_parts[:n]
-                if ctxt_name_prefix_parts and ctxt_w_path is None: # plain module
-                    ctxt_name_prefix_parts.pop()
-                if ctxt_name_prefix_parts:
-                    rel_modulename = '.'.join(ctxt_name_prefix_parts)
-                    if modulename:
-                        rel_modulename += '.' + modulename
-                baselevel = len(ctxt_name_prefix_parts)
-                if rel_modulename is not None:
-                    w_mod = check_sys_modules(space, w(rel_modulename))
-                    if (w_mod is None or
-                        not space.is_w(w_mod, space.w_None)):
-                        w_mod = absolute_import(space, rel_modulename,
-                                                baselevel,
-                                                fromlist_w, tentative=1)
-                        if w_mod is not None:
-                            space.timer.stop_name("importhook", modulename)
-                            return w_mod
+
+        if ctxt_name is not None:
+            ctxt_name_prefix_parts = ctxt_name.split('.')
+            if level > 0:
+                n = len(ctxt_name_prefix_parts)-level+1
+                assert n>=0
+                ctxt_name_prefix_parts = ctxt_name_prefix_parts[:n]
+            if ctxt_name_prefix_parts and ctxt_w_path is None: # plain module
+                ctxt_name_prefix_parts.pop()
+            if ctxt_name_prefix_parts:
+                rel_modulename = '.'.join(ctxt_name_prefix_parts)
+                if modulename:
+                    rel_modulename += '.' + modulename
+            baselevel = len(ctxt_name_prefix_parts)
+            if rel_modulename is not None:
+                # XXX What is this check about? There is no test for it
+                w_mod = check_sys_modules(space, w(rel_modulename))
+
+                if (w_mod is None or
+                    not space.is_w(w_mod, space.w_None)):
+
+                    # if no level was set, ignore import errors, and
+                    # fall back to absolute import at the end of the
+                    # function.
+                    if level == -1:
+                        tentative = True
                     else:
-                        rel_modulename = None
-    if level > 0:
-        msg = "Attempted relative import in non-package"
-        raise OperationError(space.w_ValueError, w(msg))
+                        tentative = False
+
+                    w_mod = absolute_import(space, rel_modulename,
+                                            baselevel, fromlist_w,
+                                            tentative=tentative)
+                    if w_mod is not None:
+                        space.timer.stop_name("importhook", modulename)
+                        return w_mod
+                else:
+                    rel_modulename = None
+
+        if level > 0:
+            msg = "Attempted relative import in non-package"
+            raise OperationError(space.w_ValueError, w(msg))
     w_mod = absolute_import_try(space, modulename, 0, fromlist_w)
     if w_mod is None or space.is_w(w_mod, space.w_None):
         w_mod = absolute_import(space, modulename, 0, fromlist_w, tentative=0)
@@ -226,6 +242,10 @@
 def _absolute_import(space, modulename, baselevel, fromlist_w, tentative):
     w = space.wrap
 
+    if '/' in modulename or '\\' in modulename:
+        raise OperationError(space.w_ImportError, space.wrap(
+            "Import by filename is not supported."))
+
     w_mod = None
     parts = modulename.split('.')
     prefix = []
@@ -720,11 +740,14 @@
 
     if space.config.objspace.usepycfiles:
         cpathname = pathname + 'c'
-        mtime = int(os.stat(pathname)[stat.ST_MTIME])
+        src_stat = os.stat(pathname)
+        mtime = int(src_stat[stat.ST_MTIME])
+        mode = src_stat[stat.ST_MODE]
         stream = check_compiled_module(space, cpathname, mtime)
     else:
         cpathname = None
         mtime = 0
+        mode = 0
         stream = None
 
     if stream:
@@ -738,7 +761,7 @@
         code_w = parse_source_module(space, pathname, source)
 
         if space.config.objspace.usepycfiles and write_pyc:
-            write_compiled_module(space, code_w, cpathname, mtime)
+            write_compiled_module(space, code_w, cpathname, mode, mtime)
 
     exec_code_module(space, w_mod, code_w)
 
@@ -825,8 +848,18 @@
 
     return w_mod
 
+def open_exclusive(space, cpathname, mode):
+    try:
+        os.unlink(cpathname)
+    except OSError:
+        pass
 
-def write_compiled_module(space, co, cpathname, mtime):
+    flags = (os.O_EXCL|os.O_CREAT|os.O_WRONLY|os.O_TRUNC|
+             streamio.O_BINARY)
+    fd = os.open(cpathname, flags, mode)
+    return streamio.fdopen_as_stream(fd, "wb")
+
+def write_compiled_module(space, co, cpathname, src_mode, src_mtime):
     """
     Write a compiled module to a file, placing the time of last
     modification of its source into the header.
@@ -847,10 +880,16 @@
     # Careful here: we must not crash nor leave behind something that looks
     # too much like a valid pyc file but really isn't one.
     #
+    mode = src_mode & ~0111
     try:
-        stream = streamio.open_file_as_stream(cpathname, "wb")
-    except StreamErrors:
-        return    # cannot create file
+        stream = open_exclusive(space, cpathname, mode)
+    except (OSError, StreamErrors):
+        try:
+            os.unlink(cpathname)
+        except OSError:
+            pass
+        return
+
     try:
         try:
             # will patch the header later; write zeroes until we are sure that
@@ -862,7 +901,7 @@
             # should be ok (XXX or should call os.fsync() to be sure?)
             stream.seek(0, 0)
             _w_long(stream, get_pyc_magic(space))
-            _w_long(stream, mtime)
+            _w_long(stream, src_mtime)
         finally:
             stream.close()
     except StreamErrors:

diff --git a/pypy/jit/backend/x86/runner.py b/pypy/jit/backend/x86/runner.py
--- a/pypy/jit/backend/x86/runner.py
+++ b/pypy/jit/backend/x86/runner.py
@@ -13,6 +13,7 @@
 class AbstractX86CPU(AbstractLLCPU):
     debug = True
     supports_floats = True
+    supports_longlong = True
 
     BOOTSTRAP_TP = lltype.FuncType([], lltype.Signed)
     dont_keepalive_stuff = False # for tests


diff --git a/pypy/rpython/lltypesystem/lloperation.py b/pypy/rpython/lltypesystem/lloperation.py
--- a/pypy/rpython/lltypesystem/lloperation.py
+++ b/pypy/rpython/lltypesystem/lloperation.py
@@ -265,8 +265,8 @@
     'uint_ge':              LLOp(canfold=True),
     'uint_and':             LLOp(canfold=True),
     'uint_or':              LLOp(canfold=True),
-    'uint_lshift':          LLOp(canfold=True),
-    'uint_rshift':          LLOp(canfold=True),
+    'uint_lshift':          LLOp(canfold=True),  # args (r_uint, int)
+    'uint_rshift':          LLOp(canfold=True),  # args (r_uint, int)
     'uint_xor':             LLOp(canfold=True),
 
     'float_is_true':        LLOp(canfold=True),  # it really means "x != 0.0"
@@ -288,9 +288,7 @@
 
     'llong_is_true':        LLOp(canfold=True),
     'llong_neg':            LLOp(canfold=True),
-    'llong_neg_ovf':        LLOp(canraise=(OverflowError,), tryfold=True),
     'llong_abs':            LLOp(canfold=True),
-    'llong_abs_ovf':        LLOp(canraise=(OverflowError,), tryfold=True),
     'llong_invert':         LLOp(canfold=True),
 
     'llong_add':            LLOp(canfold=True),
@@ -308,8 +306,8 @@
     'llong_ge':             LLOp(canfold=True),
     'llong_and':            LLOp(canfold=True),
     'llong_or':             LLOp(canfold=True),
-    'llong_lshift':         LLOp(canfold=True),
-    'llong_rshift':         LLOp(canfold=True),
+    'llong_lshift':         LLOp(canfold=True),  # args (r_longlong, int)
+    'llong_rshift':         LLOp(canfold=True),  # args (r_longlong, int)
     'llong_xor':            LLOp(canfold=True),
 
     'ullong_is_true':       LLOp(canfold=True),
@@ -330,8 +328,8 @@
     'ullong_ge':            LLOp(canfold=True),
     'ullong_and':           LLOp(canfold=True),
     'ullong_or':            LLOp(canfold=True),
-    'ullong_lshift':        LLOp(canfold=True),
-    'ullong_rshift':        LLOp(canfold=True),
+    'ullong_lshift':        LLOp(canfold=True),  # args (r_ulonglong, int)
+    'ullong_rshift':        LLOp(canfold=True),  # args (r_ulonglong, int)
     'ullong_xor':           LLOp(canfold=True),
 
     'cast_primitive':       LLOp(canfold=True),

diff --git a/pypy/jit/backend/llsupport/test/test_descr.py b/pypy/jit/backend/llsupport/test/test_descr.py
--- a/pypy/jit/backend/llsupport/test/test_descr.py
+++ b/pypy/jit/backend/llsupport/test/test_descr.py
@@ -5,7 +5,7 @@
 from pypy.rpython.annlowlevel import llhelper
 from pypy.jit.metainterp.history import BoxInt, BoxFloat, BoxPtr
 from pypy.jit.metainterp import history
-import struct
+import sys, struct, py
 
 def test_get_size_descr():
     c0 = GcCache(False)
@@ -95,6 +95,16 @@
             descr_x = get_field_descr(c2, S, 'x')
             assert descr_x.is_field_signed() == signed
 
+def test_get_field_descr_longlong():
+    if sys.maxint > 2147483647:
+        py.test.skip("long long: for 32-bit only")
+    c0 = GcCache(False)
+    S = lltype.GcStruct('S', ('y', lltype.UnsignedLongLong))
+    descr = get_field_descr(c0, S, 'y')
+    assert not descr.is_pointer_field()
+    assert descr.is_float_field()
+    assert descr.get_field_size(False) == 8
+
 
 def test_get_array_descr():
     U = lltype.Struct('U')
@@ -226,6 +236,21 @@
     assert descr4.get_return_type() == history.FLOAT
     assert descr4.arg_classes == "ff"
 
+def test_get_call_descr_not_translated_longlong():
+    if sys.maxint > 2147483647:
+        py.test.skip("long long: for 32-bit only")
+    c0 = GcCache(False)
+    #
+    descr5 = get_call_descr(c0, [lltype.SignedLongLong], lltype.Signed)
+    assert descr5.get_result_size(False) == 4
+    assert descr5.get_return_type() == history.INT
+    assert descr5.arg_classes == "L"
+    #
+    descr6 = get_call_descr(c0, [lltype.Signed], lltype.SignedLongLong)
+    assert descr6.get_result_size(False) == 8
+    assert descr6.get_return_type() == history.FLOAT
+    assert descr6.arg_classes == "i"
+
 def test_get_call_descr_translated():
     c1 = GcCache(True)
     T = lltype.GcStruct('T')

diff --git a/pypy/jit/backend/llsupport/descr.py b/pypy/jit/backend/llsupport/descr.py
--- a/pypy/jit/backend/llsupport/descr.py
+++ b/pypy/jit/backend/llsupport/descr.py
@@ -6,6 +6,8 @@
 from pypy.jit.metainterp import history
 from pypy.jit.metainterp.resoperation import ResOperation, rop
 from pypy.jit.codewriter import heaptracker
+from pypy.rlib.rarithmetic import r_longlong, r_ulonglong
+from pypy.rlib.longlong2float import longlong2float, float2longlong
 
 # The point of the class organization in this file is to make instances
 # as compact as possible.  This is done by not storing the field size or
@@ -29,6 +31,14 @@
         assert isinstance(ARRAY, lltype.GcArray)
 
 
+if lltype.SignedLongLong is lltype.Signed:
+    def is_longlong(TYPE):
+        return False
+else:
+    assert rffi.sizeof(lltype.SignedLongLong) == rffi.sizeof(lltype.Float)
+    def is_longlong(TYPE):
+        return TYPE in (lltype.SignedLongLong, lltype.UnsignedLongLong)
+
 # ____________________________________________________________
 # SizeDescrs
 
@@ -264,6 +274,8 @@
 
     def create_call_stub(self, rtyper, RESULT):
         def process(c):
+            if c == 'L':
+                return 'float2longlong(%s)' % (process('f'),)
             arg = 'args_%s[%d]' % (c, seen[c])
             seen[c] += 1
             return arg
@@ -277,6 +289,10 @@
                 return llmemory.GCREF
             elif arg == 'v':
                 return lltype.Void
+            elif arg == 'L':
+                return lltype.SignedLongLong
+            else:
+                raise AssertionError(arg)
 
         seen = {'i': 0, 'r': 0, 'f': 0}
         args = ", ".join([process(c) for c in self.arg_classes])
@@ -286,7 +302,7 @@
         elif self.get_return_type() == history.REF:
             result = 'lltype.cast_opaque_ptr(llmemory.GCREF, res)'
         elif self.get_return_type() == history.FLOAT:
-            result = 'res'
+            result = 'cast_to_float(res)'
         elif self.get_return_type() == history.VOID:
             result = 'None'
         else:
@@ -308,11 +324,21 @@
         assert self._return_type == return_type
         assert self.arg_classes.count('i') == len(args_i or ())
         assert self.arg_classes.count('r') == len(args_r or ())
-        assert self.arg_classes.count('f') == len(args_f or ())
+        assert (self.arg_classes.count('f') +
+                self.arg_classes.count('L')) == len(args_f or ())
 
     def repr_of_descr(self):
         return '<%s>' % self._clsname
 
+def cast_to_float(x):
+    if isinstance(x, r_longlong):
+        return longlong2float(x)
+    if isinstance(x, r_ulonglong):
+        return longlong2float(rffi.cast(lltype.SignedLongLong, x))
+    assert isinstance(x, float)
+    return x
+cast_to_float._annspecialcase_ = 'specialize:argtype(0)'
+
 
 class BaseIntCallDescr(BaseCallDescr):
     # Base class of the various subclasses of descrs corresponding to
@@ -371,6 +397,9 @@
     def get_result_size(self, translate_support_code):
         return symbolic.get_size(lltype.Float, translate_support_code)
 
+class LongLongCallDescr(FloatCallDescr):
+    _clsname = 'LongLongCallDescr'
+
 class VoidCallDescr(BaseCallDescr):
     _clsname = 'VoidCallDescr'
     _return_type = history.VOID
@@ -383,6 +412,8 @@
         return VoidCallDescr
     if RESULT is lltype.Float:
         return FloatCallDescr
+    if is_longlong(RESULT):
+        return LongLongCallDescr
     return getDescrClass(RESULT, BaseIntCallDescr, GcPtrCallDescr,
                          NonGcPtrCallDescr, 'Call', 'get_result_size',
                          Ellipsis,  # <= floatattrname should not be used here
@@ -394,7 +425,11 @@
         kind = getkind(ARG)
         if   kind == 'int': arg_classes.append('i')
         elif kind == 'ref': arg_classes.append('r')
-        elif kind == 'float': arg_classes.append('f')
+        elif kind == 'float':
+            if is_longlong(ARG):
+                arg_classes.append('L')
+            else:
+                arg_classes.append('f')
         else:
             raise NotImplementedError('ARG = %r' % (ARG,))
     arg_classes = ''.join(arg_classes)
@@ -432,7 +467,7 @@
             return symbolic.get_size(TYPE, translate_support_code)
         setattr(Descr, methodname, method)
         #
-        if TYPE is lltype.Float:
+        if TYPE is lltype.Float or is_longlong(TYPE):
             setattr(Descr, floatattrname, True)
         elif TYPE is not lltype.Bool and rffi.cast(TYPE, -1) == -1:
             setattr(Descr, signedattrname, True)

diff --git a/pypy/rlib/rsre/rsre_core.py b/pypy/rlib/rsre/rsre_core.py
--- a/pypy/rlib/rsre/rsre_core.py
+++ b/pypy/rlib/rsre/rsre_core.py
@@ -385,10 +385,13 @@
                 marks = p.marks
                 enum = p.enum.move_to_next_result(ctx)
             #
+            # zero-width match protection
             min = ctx.pat(ppos+1)
-            if (enum is not None and
-                (ptr != ctx.match_end or self.num_pending < min)):
-                #               ^^^^^^^^^^ zero-width match protection
+            if self.num_pending >= min:
+                while enum is not None and ptr == ctx.match_end:
+                    enum = enum.move_to_next_result(ctx)
+            #
+            if enum is not None:
                 # matched one more 'item'.  record it and continue.
                 self.pending = Pending(ptr, marks, enum, self.pending)
                 self.num_pending += 1
@@ -436,12 +439,15 @@
             if max == 65535 or self.num_pending < max:
                 # try to match one more 'item'
                 enum = sre_match(ctx, ppos + 3, ptr, marks)
+                #
+                # zero-width match protection
+                if self.num_pending >= min:
+                    while enum is not None and ptr == ctx.match_end:
+                        enum = enum.move_to_next_result(ctx)
             else:
                 enum = None    # 'max' reached, no more matches
 
-            while (enum is None or
-                   (ptr == ctx.match_end and self.num_pending >= min)):
-                #                   ^^^^^^^^^^ zero-width match protection
+            while enum is None:
                 # 'item' does not match; try to get further results from
                 # the 'pending' list.
                 p = self.pending

diff --git a/pypy/objspace/std/objspace.py b/pypy/objspace/std/objspace.py
--- a/pypy/objspace/std/objspace.py
+++ b/pypy/objspace/std/objspace.py
@@ -177,6 +177,13 @@
             #print 'wrapping', x, '->', w_result
             return w_result
         if isinstance(x, base_int):
+            if self.config.objspace.std.withsmalllong:
+                from pypy.objspace.std.smalllongobject import W_SmallLongObject
+                from pypy.rlib.rarithmetic import r_longlong, r_ulonglong
+                from pypy.rlib.rarithmetic import longlongmax
+                if (not isinstance(x, r_ulonglong)
+                    or x <= r_ulonglong(longlongmax)):
+                    return W_SmallLongObject(r_longlong(x))
             x = widen(x)
             if isinstance(x, int):
                 return self.newint(x)
@@ -207,6 +214,16 @@
         # The following cases are even stranger.
         # Really really only for tests.
         if type(x) is long:
+            if self.config.objspace.std.withsmalllong:
+                from pypy.rlib.rarithmetic import r_longlong
+                try:
+                    rx = r_longlong(x)
+                except OverflowError:
+                    pass
+                else:
+                    from pypy.objspace.std.smalllongobject import \
+                                                   W_SmallLongObject
+                    return W_SmallLongObject(rx)
             return W_LongObject.fromlong(x)
         if isinstance(x, slice):
             return W_SliceObject(self.wrap(x.start),
@@ -269,6 +286,9 @@
         return unpackcomplex(self, w_complex)
 
     def newlong(self, val): # val is an int
+        if self.config.objspace.std.withsmalllong:
+            from pypy.objspace.std.smalllongobject import W_SmallLongObject
+            return W_SmallLongObject.fromint(val)
         return W_LongObject.fromint(self, val)
 
     def newlong_from_rbigint(self, val):

diff --git a/pypy/jit/metainterp/warmspot.py b/pypy/jit/metainterp/warmspot.py
--- a/pypy/jit/metainterp/warmspot.py
+++ b/pypy/jit/metainterp/warmspot.py
@@ -155,6 +155,7 @@
         if policy is None:
             policy = JitPolicy()
         policy.set_supports_floats(self.cpu.supports_floats)
+        policy.set_supports_longlong(self.cpu.supports_longlong)
         graphs = self.codewriter.find_all_graphs(policy)
         policy.dump_unsafe_loops()
         self.check_access_directly_sanity(graphs)

diff --git a/pypy/jit/backend/model.py b/pypy/jit/backend/model.py
--- a/pypy/jit/backend/model.py
+++ b/pypy/jit/backend/model.py
@@ -4,6 +4,11 @@
 
 class AbstractCPU(object):
     supports_floats = False
+    supports_longlong = False
+    # ^^^ This is only useful on 32-bit platforms.  If True,
+    # longlongs are supported by the JIT, but stored as doubles.
+    # Boxes and Consts are BoxFloats and ConstFloats.
+
     done_with_this_frame_void_v = -1
     done_with_this_frame_int_v = -1
     done_with_this_frame_ref_v = -1

diff --git a/pypy/module/cpyext/include/modsupport.inl b/pypy/module/cpyext/include/modsupport.inl
deleted file mode 100644
--- a/pypy/module/cpyext/include/modsupport.inl
+++ /dev/null
@@ -1,29 +0,0 @@
-/* -*- C -*- */
-/* Module support interface */
-
-#ifndef Py_MODSUPPORT_INL
-#define Py_MODSUPPORT_INL
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-#ifdef PYPY_STANDALONE
-/* XXX1 On translation, forwarddecl.h is included after this file */
-/* XXX2 genc.py transforms "const char*" into "char*" */
-extern PyObject *_Py_InitPyPyModule(char *, PyMethodDef *, char *, PyObject *, int);
-#endif
-
-Py_LOCAL_INLINE(PyObject *) Py_InitModule4(
-        const char* name, PyMethodDef* methods,
-        const char* doc, PyObject *self,
-        int api_version)
-{
-    return _Py_InitPyPyModule((char*)name, methods,
-                              (char*)doc, self,
-                              api_version);
-}
-
-#ifdef __cplusplus
-}
-#endif
-#endif /* !Py_MODSUPPORT_INL */

diff --git a/pypy/jit/codewriter/call.py b/pypy/jit/codewriter/call.py
--- a/pypy/jit/codewriter/call.py
+++ b/pypy/jit/codewriter/call.py
@@ -188,7 +188,8 @@
                                          FUNC.RESULT)
         return (fnaddr, calldescr)
 
-    def getcalldescr(self, op, oopspecindex=EffectInfo.OS_NONE):
+    def getcalldescr(self, op, oopspecindex=EffectInfo.OS_NONE,
+                     extraeffect=None):
         """Return the calldescr that describes all calls done by 'op'.
         This returns a calldescr that we can put in the corresponding
         call operation in the calling jitcode.  It gets an effectinfo
@@ -216,17 +217,18 @@
                 assert not NON_VOID_ARGS, ("arguments not supported for "
                                            "loop-invariant function!")
         # build the extraeffect
-        if self.virtualizable_analyzer.analyze(op):
-            extraeffect = EffectInfo.EF_FORCES_VIRTUAL_OR_VIRTUALIZABLE
-        elif loopinvariant:
-            extraeffect = EffectInfo.EF_LOOPINVARIANT
-        elif pure:
-            # XXX check what to do about exceptions (also MemoryError?)
-            extraeffect = EffectInfo.EF_PURE
-        elif self._canraise(op):
-            extraeffect = EffectInfo.EF_CAN_RAISE
-        else:
-            extraeffect = EffectInfo.EF_CANNOT_RAISE
+        if extraeffect is None:
+            if self.virtualizable_analyzer.analyze(op):
+                extraeffect = EffectInfo.EF_FORCES_VIRTUAL_OR_VIRTUALIZABLE
+            elif loopinvariant:
+                extraeffect = EffectInfo.EF_LOOPINVARIANT
+            elif pure:
+                # XXX check what to do about exceptions (also MemoryError?)
+                extraeffect = EffectInfo.EF_PURE
+            elif self._canraise(op):
+                extraeffect = EffectInfo.EF_CAN_RAISE
+            else:
+                extraeffect = EffectInfo.EF_CANNOT_RAISE
         #
         effectinfo = effectinfo_from_writeanalyze(
             self.readwrite_analyzer.analyze(op), self.cpu, extraeffect,

diff --git a/pypy/jit/backend/x86/assembler.py b/pypy/jit/backend/x86/assembler.py
--- a/pypy/jit/backend/x86/assembler.py
+++ b/pypy/jit/backend/x86/assembler.py
@@ -37,6 +37,7 @@
 from pypy.rlib import rgc
 from pypy.jit.backend.x86.jump import remap_frame_layout
 from pypy.jit.metainterp.history import ConstInt, BoxInt
+from pypy.jit.codewriter.effectinfo import EffectInfo
 
 # darwin requires the stack to be 16 bytes aligned on calls. Same for gcc 4.5.0,
 # better safe than sorry
@@ -769,6 +770,11 @@
     def regalloc_perform_discard(self, op, arglocs):
         genop_discard_list[op.getopnum()](self, op, arglocs)
 
+    def regalloc_perform_llong(self, op, arglocs, resloc):
+        effectinfo = op.getdescr().get_extra_info()
+        oopspecindex = effectinfo.oopspecindex
+        genop_llong_list[oopspecindex](self, op, arglocs, resloc)
+
     def regalloc_perform_with_guard(self, op, guard_op, faillocs,
                                     arglocs, resloc, current_depths):
         faildescr = guard_op.getdescr()
@@ -1115,6 +1121,88 @@
         self.mc.XOR_rr(edx.value, edx.value)
         self.mc.DIV_r(ecx.value)
 
+    genop_llong_add = _binaryop("PADDQ", True)
+    genop_llong_sub = _binaryop("PSUBQ")
+    genop_llong_and = _binaryop("PAND",  True)
+    genop_llong_or  = _binaryop("POR",   True)
+    genop_llong_xor = _binaryop("PXOR",  True)
+
+    def genop_llong_to_int(self, op, arglocs, resloc):
+        loc = arglocs[0]
+        assert isinstance(resloc, RegLoc)
+        if isinstance(loc, RegLoc):
+            self.mc.MOVD_rx(resloc.value, loc.value)
+        elif isinstance(loc, StackLoc):
+            self.mc.MOV_rb(resloc.value, loc.value)
+        else:
+            not_implemented("llong_to_int: %s" % (loc,))
+
+    def genop_llong_from_int(self, op, arglocs, resloc):
+        loc = arglocs[0]
+        if isinstance(loc, ConstFloatLoc):
+            self.mc.MOVSD(resloc, loc)
+        else:
+            assert loc is eax
+            assert isinstance(resloc, RegLoc)
+            loc2 = arglocs[1]
+            assert isinstance(loc2, RegLoc)
+            self.mc.CDQ()       # eax -> eax:edx
+            self.mc.MOVD_xr(resloc.value, eax.value)
+            self.mc.MOVD_xr(loc2.value, edx.value)
+            self.mc.PUNPCKLDQ_xx(resloc.value, loc2.value)
+
+    def genop_llong_from_two_ints(self, op, arglocs, resloc):
+        assert isinstance(resloc, RegLoc)
+        loc1, loc2, loc3 = arglocs
+        #
+        if isinstance(loc1, ConstFloatLoc):
+            self.mc.MOVSD(resloc, loc1)
+        else:
+            assert isinstance(loc1, RegLoc)
+            self.mc.MOVD_xr(resloc.value, loc1.value)
+        #
+        if loc2 is not None:
+            assert isinstance(loc3, RegLoc)
+            if isinstance(loc2, ConstFloatLoc):
+                self.mc.MOVSD(loc3, loc2)
+            else:
+                assert isinstance(loc2, RegLoc)
+                self.mc.MOVD_xr(loc3.value, loc2.value)
+            self.mc.PUNPCKLDQ_xx(resloc.value, loc3.value)
+
+    def genop_llong_eq(self, op, arglocs, resloc):
+        loc1, loc2, locxtmp = arglocs
+        self.mc.MOVSD(locxtmp, loc1)
+        self.mc.PCMPEQD(locxtmp, loc2)
+        self.mc.PMOVMSKB_rx(resloc.value, locxtmp.value)
+        # Now the lower 8 bits of resloc contain 0x00, 0x0F, 0xF0 or 0xFF
+        # depending on the result of the comparison of each of the two
+        # double-words of loc1 and loc2.  The higher 8 bits contain random
+        # results.  We want to map 0xFF to 1, and 0x00, 0x0F and 0xF0 to 0.
+        self.mc.CMP8_ri(resloc.value | rx86.BYTE_REG_FLAG, -1)
+        self.mc.SBB_rr(resloc.value, resloc.value)
+        self.mc.ADD_ri(resloc.value, 1)
+
+    def genop_llong_ne(self, op, arglocs, resloc):
+        loc1, loc2, locxtmp = arglocs
+        self.mc.MOVSD(locxtmp, loc1)
+        self.mc.PCMPEQD(locxtmp, loc2)
+        self.mc.PMOVMSKB_rx(resloc.value, locxtmp.value)
+        # Now the lower 8 bits of resloc contain 0x00, 0x0F, 0xF0 or 0xFF
+        # depending on the result of the comparison of each of the two
+        # double-words of loc1 and loc2.  The higher 8 bits contain random
+        # results.  We want to map 0xFF to 0, and 0x00, 0x0F and 0xF0 to 1.
+        self.mc.CMP8_ri(resloc.value | rx86.BYTE_REG_FLAG, -1)
+        self.mc.SBB_rr(resloc.value, resloc.value)
+        self.mc.NEG_r(resloc.value)
+
+    def genop_llong_lt(self, op, arglocs, resloc):
+        # XXX just a special case for now: "x < 0"
+        loc1, = arglocs
+        self.mc.PMOVMSKB_rx(resloc.value, loc1.value)
+        self.mc.SHR_ri(resloc.value, 7)
+        self.mc.AND_ri(resloc.value, 1)
+
     def genop_new_with_vtable(self, op, arglocs, result_loc):
         assert result_loc is eax
         loc_vtable = arglocs[-1]
@@ -1725,7 +1813,14 @@
         self._emit_call(x, arglocs, 3, tmp=tmp)
 
         if IS_X86_32 and isinstance(resloc, StackLoc) and resloc.width == 8:
-            self.mc.FSTP_b(resloc.value)   # float return
+            # a float or a long long return
+            from pypy.jit.backend.llsupport.descr import LongLongCallDescr
+            if isinstance(op.getdescr(), LongLongCallDescr):
+                self.mc.MOV_br(resloc.value, eax.value)      # long long
+                self.mc.MOV_br(resloc.value + 4, edx.value)
+                # XXX should ideally not move the result on the stack
+            else:
+                self.mc.FSTP_b(resloc.value)   # float return
         elif size == WORD:
             assert resloc is eax or resloc is xmm0    # a full word
         elif size == 0:
@@ -1959,6 +2054,7 @@
         
 genop_discard_list = [Assembler386.not_implemented_op_discard] * rop._LAST
 genop_list = [Assembler386.not_implemented_op] * rop._LAST
+genop_llong_list = {}
 genop_guard_list = [Assembler386.not_implemented_op_guard] * rop._LAST
 
 for name, value in Assembler386.__dict__.iteritems():
@@ -1970,6 +2066,10 @@
         opname = name[len('genop_guard_'):]
         num = getattr(rop, opname.upper())
         genop_guard_list[num] = value
+    elif name.startswith('genop_llong_'):
+        opname = name[len('genop_llong_'):]
+        num = getattr(EffectInfo, 'OS_LLONG_' + opname.upper())
+        genop_llong_list[num] = value
     elif name.startswith('genop_'):
         opname = name[len('genop_'):]
         num = getattr(rop, opname.upper())

diff --git a/pypy/jit/codewriter/support.py b/pypy/jit/codewriter/support.py
--- a/pypy/jit/codewriter/support.py
+++ b/pypy/jit/codewriter/support.py
@@ -19,6 +19,7 @@
 from pypy.rpython.annlowlevel import MixLevelHelperAnnotator
 from pypy.jit.metainterp.typesystem import deref
 from pypy.rlib import rgc
+from pypy.rlib.rarithmetic import r_longlong, r_ulonglong, r_uint, intmask
 
 def getargtypes(annotator, values):
     if values is None:    # for backend tests producing stand-alone exe's
@@ -222,6 +223,136 @@
         return x
 
 
+# long long support
+# -----------------
+
+def u_to_longlong(x):
+    return rffi.cast(lltype.SignedLongLong, x)
+
+def _ll_1_llong_invert(xll):
+    y = ~r_ulonglong(xll)
+    return u_to_longlong(y)
+
+def _ll_2_llong_lt(xll, yll):
+    return xll < yll
+
+def _ll_2_llong_le(xll, yll):
+    return xll <= yll
+
+def _ll_2_llong_eq(xll, yll):
+    return xll == yll
+
+def _ll_2_llong_ne(xll, yll):
+    return xll != yll
+
+def _ll_2_llong_gt(xll, yll):
+    return xll > yll
+
+def _ll_2_llong_ge(xll, yll):
+    return xll >= yll
+
+def _ll_2_llong_ult(xull, yull):
+    return xull < yull
+
+def _ll_2_llong_ule(xull, yull):
+    return xull <= yull
+
+def _ll_2_llong_ugt(xull, yull):
+    return xull > yull
+
+def _ll_2_llong_uge(xull, yull):
+    return xull >= yull
+
+def _ll_2_llong_add(xll, yll):
+    z = r_ulonglong(xll) + r_ulonglong(yll)
+    return u_to_longlong(z)
+
+def _ll_2_llong_sub(xll, yll):
+    z = r_ulonglong(xll) - r_ulonglong(yll)
+    return u_to_longlong(z)
+
+def _ll_2_llong_mul(xll, yll):
+    z = r_ulonglong(xll) * r_ulonglong(yll)
+    return u_to_longlong(z)
+
+def _ll_2_llong_and(xll, yll):
+    z = r_ulonglong(xll) & r_ulonglong(yll)
+    return u_to_longlong(z)
+
+def _ll_2_llong_or(xll, yll):
+    z = r_ulonglong(xll) | r_ulonglong(yll)
+    return u_to_longlong(z)
+
+def _ll_2_llong_xor(xll, yll):
+    z = r_ulonglong(xll) ^ r_ulonglong(yll)
+    return u_to_longlong(z)
+
+def _ll_2_llong_lshift(xll, y):
+    z = r_ulonglong(xll) << y
+    return u_to_longlong(z)
+
+def _ll_2_llong_rshift(xll, y):
+    return xll >> y
+
+def _ll_2_llong_urshift(xull, y):
+    return xull >> y
+
+def _ll_1_llong_from_int(x):
+    return r_longlong(intmask(x))
+
+def _ll_2_llong_from_two_ints(x_lo, x_hi):
+    z = (r_ulonglong(r_uint(x_hi)) << 32) | r_ulonglong(r_uint(x_lo))
+    return u_to_longlong(z)
+
+def _ll_1_llong_to_int(xll):
+    return intmask(xll)
+
+def _ll_1_llong_from_float(xf):
+    return r_longlong(xf)
+
+def _ll_1_llong_to_float(xll):
+    return float(rffi.cast(lltype.SignedLongLong, xll))
+
+
+def _ll_1_llong_abs(xll):
+    if xll < 0:
+        return -xll
+    else:
+        return xll
+
+def _ll_2_llong_floordiv(xll, yll):
+    return llop.llong_floordiv(lltype.SignedLongLong, xll, yll)
+
+def _ll_2_llong_floordiv_zer(xll, yll):
+    if yll == 0:
+        raise ZeroDivisionError
+    return llop.llong_floordiv(lltype.SignedLongLong, xll, yll)
+
+def _ll_2_llong_mod(xll, yll):
+    return llop.llong_mod(lltype.SignedLongLong, xll, yll)
+
+def _ll_2_llong_mod_zer(xll, yll):
+    if yll == 0:
+        raise ZeroDivisionError
+    return llop.llong_mod(lltype.SignedLongLong, xll, yll)
+
+def _ll_2_ullong_floordiv(xll, yll):
+    return llop.ullong_floordiv(lltype.SignedLongLong, xll, yll)
+
+def _ll_2_ullong_floordiv_zer(xll, yll):
+    if yll == 0:
+        raise ZeroDivisionError
+    return llop.ullong_floordiv(lltype.SignedLongLong, xll, yll)
+
+def _ll_2_ullong_mod(xll, yll):
+    return llop.ullong_mod(lltype.SignedLongLong, xll, yll)
+
+def _ll_2_ullong_mod_zer(xll, yll):
+    if yll == 0:
+        raise ZeroDivisionError
+    return llop.ullong_mod(lltype.SignedLongLong, xll, yll)
+
+
 # libffi support
 # --------------
 

diff --git a/pypy/config/pypyoption.py b/pypy/config/pypyoption.py
--- a/pypy/config/pypyoption.py
+++ b/pypy/config/pypyoption.py
@@ -4,6 +4,7 @@
 from pypy.config.config import OptionDescription, BoolOption, IntOption, ArbitraryOption
 from pypy.config.config import ChoiceOption, StrOption, to_optparse, Config
 from pypy.config.config import ConflictConfigError
+from pypy.config.translationoption import IS_64_BITS
 
 modulepath = py.path.local(__file__).dirpath().dirpath().join("module")
 all_modules = [p.basename for p in modulepath.listdir()
@@ -212,6 +213,11 @@
         IntOption("prebuiltintto", "highest integer which is prebuilt",
                   default=100, cmdline="--prebuiltintto"),
 
+        BoolOption("withsmalllong", "use a version of 'long' in a C long long",
+                   default=False,
+                   requires=[("objspace.std.withsmallint", False)]),
+                             #  ^^^ because of missing delegate_xx2yy
+
         BoolOption("withstrjoin", "use strings optimized for addition",
                    default=False),
 
@@ -345,6 +351,8 @@
         config.objspace.std.suggest(optimized_list_getitem=True)
         config.objspace.std.suggest(getattributeshortcut=True)
         config.objspace.std.suggest(newshortcut=True)        
+        if not IS_64_BITS:
+            config.objspace.std.suggest(withsmalllong=True)
 
     # extra costly optimizations only go in level 3
     if level == '3':
@@ -360,6 +368,8 @@
         config.objspace.std.suggest(withmapdict=True)
         config.objspace.std.suggest(withstrslice=True)
         config.objspace.std.suggest(withstrjoin=True)
+        if not IS_64_BITS:
+            config.objspace.std.suggest(withsmalllong=True)
         # xxx other options? ropes maybe?
 
     # completely disable geninterp in a level 0 translation

From commits-noreply at bitbucket.org  Wed Feb  2 13:50:29 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Wed,  2 Feb 2011 13:50:29 +0100 (CET)
Subject: [pypy-svn] pypy arm-backend-2: support refs as imm values in
	failargs
Message-ID: <20110202125029.B744C2A2006@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backend-2
Changeset: r41554:2a82d42e0a77
Date: 2011-01-30 15:47 +0100
http://bitbucket.org/pypy/pypy/changeset/2a82d42e0a77/

Log:	support refs as imm values in failargs

diff --git a/pypy/jit/backend/arm/assembler.py b/pypy/jit/backend/arm/assembler.py
--- a/pypy/jit/backend/arm/assembler.py
+++ b/pypy/jit/backend/arm/assembler.py
@@ -128,7 +128,7 @@
             i += 1
             res = enc[i]
             if res == self.IMM_LOC:
-                assert group == self.INT_TYPE
+                assert group == self.INT_TYPE or group == self.REF_TYPE
                 # imm value
                 value = self.decode32(enc, i+1)
                 i += 4

diff --git a/pypy/jit/backend/arm/test/test_trace_operations.py b/pypy/jit/backend/arm/test/test_trace_operations.py
new file mode 100644
--- /dev/null
+++ b/pypy/jit/backend/arm/test/test_trace_operations.py
@@ -0,0 +1,39 @@
+from pypy.jit.backend.x86.test.test_regalloc import BaseTestRegalloc
+from pypy.jit.backend.detect_cpu import getcpuclass
+from pypy.rpython.lltypesystem import lltype, llmemory
+CPU = getcpuclass()
+
+class TestConstPtr(BaseTestRegalloc):
+
+    cpu = CPU(None, None)
+    #cpu.gc_ll_descr = MockGcDescr(False)
+    cpu.setup_once()
+
+    S = lltype.GcForwardReference()
+    S.become(lltype.GcStruct('S', ('field', lltype.Ptr(S)),
+                             ('int', lltype.Signed)))
+
+    fielddescr = cpu.fielddescrof(S, 'field')
+
+    struct_ptr = lltype.malloc(S)
+    struct_ref = lltype.cast_opaque_ptr(llmemory.GCREF, struct_ptr)
+    #child_ptr = lltype.nullptr(S)
+    #struct_ptr.field = child_ptr
+
+
+    #descr0 = cpu.fielddescrof(S, 'int')
+    ptr0 = struct_ref
+
+    namespace = locals().copy()
+
+    def test_finish_failargs_constptr(self):
+        ops = '''
+        [i0]
+        i1 = int_add(i0, 1)
+        finish(i1, ConstPtr(ptr0))
+        '''
+        self.interpret(ops, [99])
+        assert self.getint(0) == 100
+        ptr = self.cpu.get_latest_value_ref(1)
+        assert self.ptr0 == ptr
+

From commits-noreply at bitbucket.org  Wed Feb  2 13:50:30 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Wed,  2 Feb 2011 13:50:30 +0100 (CET)
Subject: [pypy-svn] pypy arm-backend-2: use a block of memory that is freed
	with the loop	to store the encoding of the failargs
Message-ID: <20110202125030.367A32A2006@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backend-2
Changeset: r41555:41ea330316d6
Date: 2011-01-30 15:54 +0100
http://bitbucket.org/pypy/pypy/changeset/41ea330316d6/

Log:	use a block of memory that is freed with the loop to store the
	encoding of the failargs

diff --git a/pypy/jit/backend/arm/assembler.py b/pypy/jit/backend/arm/assembler.py
--- a/pypy/jit/backend/arm/assembler.py
+++ b/pypy/jit/backend/arm/assembler.py
@@ -6,6 +6,7 @@
 from pypy.jit.backend.arm.regalloc import (ARMRegisterManager, ARMFrameManager,
                                                     _check_imm_arg, TempInt, TempPtr)
 from pypy.jit.backend.llsupport.regalloc import compute_vars_longevity, TempBox
+from pypy.jit.backend.llsupport.asmmemmgr import MachineDataBlockWrapper
 from pypy.jit.backend.model import CompiledLoopToken
 from pypy.jit.metainterp.history import (Const, ConstInt, ConstPtr,
                                         BoxInt, BoxPtr, AbstractFailDescr,
@@ -226,9 +227,11 @@
         # 1 byte for the location
         # 1 separator byte
         # 4 bytes for the faildescr
-        # XXX free this memory
-        mem = lltype.malloc(rffi.CArray(lltype.Char), (len(arglocs)-1)*6+5,
-                                    flavor='raw', track_allocation=False)
+        memsize = (len(arglocs)-1)*6+5
+        datablockwrapper = MachineDataBlockWrapper(self.cpu.asmmemmgr, [])
+        memaddr = datablockwrapper.malloc_aligned(memsize, alignment=WORD)
+        datablockwrapper.done()
+        mem = rffi.cast(rffi.CArrayPtr(lltype.Char), memaddr)
         i = 0
         j = 0
         while i < len(args):
@@ -262,7 +265,6 @@
             i += 1
 
         mem[j] = chr(0xFF)
-        memaddr = rffi.cast(lltype.Signed, mem)
 
         n = self.cpu.get_fail_descr_number(descr)
         self.encode32(mem, j+1, n)

From commits-noreply at bitbucket.org  Wed Feb  2 13:50:32 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Wed,  2 Feb 2011 13:50:32 +0100 (CET)
Subject: [pypy-svn] pypy arm-backend-2: implement redirect_call_assembler
Message-ID: <20110202125032.E4FF22A201C@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backend-2
Changeset: r41556:1743ab839eb9
Date: 2011-01-30 17:24 +0100
http://bitbucket.org/pypy/pypy/changeset/1743ab839eb9/

Log:	implement redirect_call_assembler

diff --git a/pypy/jit/backend/arm/runner.py b/pypy/jit/backend/arm/runner.py
--- a/pypy/jit/backend/arm/runner.py
+++ b/pypy/jit/backend/arm/runner.py
@@ -105,3 +105,6 @@
         # end of "no gc operation!" block
         assert fail_index == fail_index_2
         return faildescr
+
+    def redirect_call_assembler(self, oldlooptoken, newlooptoken):
+        self.assembler.redirect_call_assembler(oldlooptoken, newlooptoken)

diff --git a/pypy/jit/backend/arm/opassembler.py b/pypy/jit/backend/arm/opassembler.py
--- a/pypy/jit/backend/arm/opassembler.py
+++ b/pypy/jit/backend/arm/opassembler.py
@@ -719,6 +719,17 @@
             regalloc.possibly_free_var(op.result)
         return fcond
 
+    # ../x86/assembler.py:668
+    def redirect_call_assembler(self, oldlooptoken, newlooptoken):
+        # we overwrite the instructions at the old _x86_direct_bootstrap_code
+        # to start with a JMP to the new _x86_direct_bootstrap_code.
+        # Ideally we should rather patch all existing CALLs, but well.
+        oldadr = oldlooptoken._arm_direct_bootstrap_code
+        target = newlooptoken._arm_direct_bootstrap_code
+        mc = ARMv7Builder()
+        mc.B(target)
+        mc.copy_to_raw_memory(oldadr)
+
     def emit_guard_call_may_force(self, op, guard_op, arglocs, regalloc, fcond):
         self.mc.LDR_ri(r.ip.value, r.fp.value)
         self.mc.CMP_ri(r.ip.value, 0)

diff --git a/pypy/jit/backend/arm/test/test_runner.py b/pypy/jit/backend/arm/test/test_runner.py
--- a/pypy/jit/backend/arm/test/test_runner.py
+++ b/pypy/jit/backend/arm/test/test_runner.py
@@ -11,6 +11,9 @@
                                          BoxObj, Const,
                                          ConstObj, BoxFloat, ConstFloat)
 from pypy.jit.metainterp.resoperation import ResOperation, rop
+from pypy.jit.tool.oparser import parse
+from pypy.rpython.lltypesystem import lltype, llmemory
+from pypy.rpython.annlowlevel import llhelper
 
 skip_unless_arm()
 
@@ -55,3 +58,46 @@
 
     def test_cond_call_gc_wb(self, *args):
         py.test.skip('needs gc support')
+
+    def test_redirect_call_assember(self):
+        called = []
+        def assembler_helper(failindex, virtualizable):
+            return self.cpu.get_latest_value_int(0)
+
+        FUNCPTR = lltype.Ptr(lltype.FuncType([lltype.Signed, llmemory.GCREF],
+                                             lltype.Signed))
+        class FakeJitDriverSD:
+            index_of_virtualizable = -1
+            _assembler_helper_ptr = llhelper(FUNCPTR, assembler_helper)
+            assembler_helper_adr = llmemory.cast_ptr_to_adr(
+                _assembler_helper_ptr)
+
+        lt1, lt2, lt3 = [LoopToken() for x in range(3)]
+        lt2.outermost_jitdriver_sd = FakeJitDriverSD()
+        loop1 = parse('''
+        [i0]
+        i1 = call_assembler(i0, descr=lt2)
+        guard_not_forced()[]
+        finish(i1)
+        ''', namespace=locals())
+        loop2 = parse('''
+        [i0]
+        i1 = int_add(i0, 1)
+        finish(i1)
+        ''')
+        loop3 = parse('''
+        [i0]
+        i1 = int_sub(i0, 1)
+        finish(i1)
+        ''')
+        self.cpu.compile_loop(loop2.inputargs, loop2.operations, lt2)
+        self.cpu.compile_loop(loop3.inputargs, loop3.operations, lt3)
+        self.cpu.compile_loop(loop1.inputargs, loop1.operations, lt1)
+        self.cpu.set_future_value_int(0, 11)
+        res = self.cpu.execute_token(lt1)
+        assert self.cpu.get_latest_value_int(0) == 12
+
+        self.cpu.redirect_call_assembler(lt2, lt3)
+        self.cpu.set_future_value_int(0, 11)
+        res = self.cpu.execute_token(lt1)
+        assert self.cpu.get_latest_value_int(0) == 10

From commits-noreply at bitbucket.org  Wed Feb  2 13:50:34 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Wed,  2 Feb 2011 13:50:34 +0100 (CET)
Subject: [pypy-svn] pypy arm-backend-2: in getfield and setfield operations
	use a	register to store the offset of a field when it does
	not fit in	the 12 bits for imm values
Message-ID: <20110202125034.16A412A2014@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backend-2
Changeset: r41557:d8b64e66d3d4
Date: 2011-01-31 16:17 +0100
http://bitbucket.org/pypy/pypy/changeset/d8b64e66d3d4/

Log:	in getfield and setfield operations use a register to store the
	offset of a field when it does not fit in the 12 bits for imm values

diff --git a/pypy/jit/backend/arm/opassembler.py b/pypy/jit/backend/arm/opassembler.py
--- a/pypy/jit/backend/arm/opassembler.py
+++ b/pypy/jit/backend/arm/opassembler.py
@@ -347,11 +347,20 @@
     def emit_op_setfield_gc(self, op, arglocs, regalloc, fcond):
         value_loc, base_loc, ofs, size = arglocs
         if size.value == 4:
-            self.mc.STR_ri(value_loc.value, base_loc.value, ofs.value)
+            if ofs.is_imm():
+                self.mc.STR_ri(value_loc.value, base_loc.value, ofs.value)
+            else:
+                self.mc.STR_rr(value_loc.value, base_loc.value, ofs.value)
         elif size.value == 2:
-            self.mc.STRH_ri(value_loc.value, base_loc.value, ofs.value)
+            if ofs.is_imm():
+                self.mc.STRH_ri(value_loc.value, base_loc.value, ofs.value)
+            else:
+                self.mc.STRH_rr(value_loc.value, base_loc.value, ofs.value)
         elif size.value == 1:
-            self.mc.STRB_ri(value_loc.value, base_loc.value, ofs.value)
+            if ofs.is_imm():
+                self.mc.STRB_ri(value_loc.value, base_loc.value, ofs.value)
+            else:
+                self.mc.STRB_rr(value_loc.value, base_loc.value, ofs.value)
         else:
             assert 0
         return fcond
@@ -361,11 +370,20 @@
     def emit_op_getfield_gc(self, op, arglocs, regalloc, fcond):
         base_loc, ofs, res, size = arglocs
         if size.value == 4:
-            self.mc.LDR_ri(res.value, base_loc.value, ofs.value)
+            if ofs.is_imm():
+                self.mc.LDR_ri(res.value, base_loc.value, ofs.value)
+            else:
+                self.mc.LDR_rr(res.value, base_loc.value, ofs.value)
         elif size.value == 2:
-            self.mc.LDRH_ri(res.value, base_loc.value, ofs.value)
+            if ofs.is_imm():
+                self.mc.LDRH_ri(res.value, base_loc.value, ofs.value)
+            else:
+                self.mc.LDRH_rr(res.value, base_loc.value, ofs.value)
         elif size.value == 1:
-            self.mc.LDRB_ri(res.value, base_loc.value, ofs.value)
+            if ofs.is_imm():
+                self.mc.LDRB_ri(res.value, base_loc.value, ofs.value)
+            else:
+                self.mc.LDRB_rr(res.value, base_loc.value, ofs.value)
         else:
             assert 0
 

diff --git a/pypy/jit/backend/arm/test/test_trace_operations.py b/pypy/jit/backend/arm/test/test_trace_operations.py
--- a/pypy/jit/backend/arm/test/test_trace_operations.py
+++ b/pypy/jit/backend/arm/test/test_trace_operations.py
@@ -10,18 +10,14 @@
     cpu.setup_once()
 
     S = lltype.GcForwardReference()
-    S.become(lltype.GcStruct('S', ('field', lltype.Ptr(S)),
-                             ('int', lltype.Signed)))
+    fields = [('int%d' % i, lltype.Signed) for i in range(1050)]
+    S.become(lltype.GcStruct('S', *fields))
 
-    fielddescr = cpu.fielddescrof(S, 'field')
+    fielddescr = cpu.fielddescrof(S, 'int1049')
 
     struct_ptr = lltype.malloc(S)
     struct_ref = lltype.cast_opaque_ptr(llmemory.GCREF, struct_ptr)
-    #child_ptr = lltype.nullptr(S)
-    #struct_ptr.field = child_ptr
 
-
-    #descr0 = cpu.fielddescrof(S, 'int')
     ptr0 = struct_ref
 
     namespace = locals().copy()
@@ -37,3 +33,21 @@
         ptr = self.cpu.get_latest_value_ref(1)
         assert self.ptr0 == ptr
 
+    def test_getfield_with_offset_gt_one_byte(self):
+        self.struct_ptr.int1049 = 666
+        ops = '''
+        [p0]
+        i0 = getfield_gc(p0, descr=fielddescr)
+        finish(i0)
+        '''
+        self.interpret(ops, [self.struct_ptr])
+        assert self.getint(0) == 666
+
+    def test_setfield_with_offset_gt_one_byte(self):
+        ops = '''
+        [p0]
+        setfield_gc(p0, 777, descr=fielddescr)
+        finish()
+        '''
+        self.interpret(ops, [self.struct_ptr])
+        assert self.struct_ptr.int1049 == 777

diff --git a/pypy/jit/backend/arm/regalloc.py b/pypy/jit/backend/arm/regalloc.py
--- a/pypy/jit/backend/arm/regalloc.py
+++ b/pypy/jit/backend/arm/regalloc.py
@@ -379,8 +379,14 @@
         boxes.append(base_box)
         value_loc, value_box = self._ensure_value_is_boxed(a1, boxes)
         boxes.append(value_box)
+        c_ofs = ConstInt(ofs)
+        if _check_imm_arg(c_ofs):
+            ofs_loc = imm(ofs)
+        else:
+            ofs_loc, ofs_box = self._ensure_value_is_boxed(c_ofs, boxes)
+            boxes.append(ofs_box)
         self.possibly_free_vars(boxes)
-        return [value_loc, base_loc, imm(ofs), imm(size)]
+        return [value_loc, base_loc, ofs_loc, imm(size)]
 
     prepare_op_setfield_raw = prepare_op_setfield_gc
 
@@ -388,11 +394,17 @@
         a0 = op.getarg(0)
         ofs, size, ptr = self._unpack_fielddescr(op.getdescr())
         base_loc, base_box = self._ensure_value_is_boxed(a0)
+        c_ofs = ConstInt(ofs)
+        if _check_imm_arg(c_ofs):
+            ofs_loc = imm(ofs)
+        else:
+            ofs_loc, ofs_box = self._ensure_value_is_boxed(c_ofs, [base_box])
+            self.possibly_free_var(ofs_box)
         self.possibly_free_var(a0)
         self.possibly_free_var(base_box)
         res = self.force_allocate_reg(op.result)
         self.possibly_free_var(op.result)
-        return [base_loc, imm(ofs), res, imm(size)]
+        return [base_loc, ofs_loc, res, imm(size)]
 
     prepare_op_getfield_raw = prepare_op_getfield_gc
     prepare_op_getfield_raw_pure = prepare_op_getfield_gc

From commits-noreply at bitbucket.org  Wed Feb  2 13:50:38 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Wed,  2 Feb 2011 13:50:38 +0100 (CET)
Subject: [pypy-svn] pypy arm-backend-2: import some tests from the x86
	backend
Message-ID: <20110202125038.8FCC32A2015@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backend-2
Changeset: r41558:65b41cb0797a
Date: 2011-02-01 02:11 +0100
http://bitbucket.org/pypy/pypy/changeset/65b41cb0797a/

Log:	import some tests from the x86 backend

diff --git a/pypy/jit/backend/arm/test/test_regalloc2.py b/pypy/jit/backend/arm/test/test_regalloc2.py
new file mode 100644
--- /dev/null
+++ b/pypy/jit/backend/arm/test/test_regalloc2.py
@@ -0,0 +1,294 @@
+import py
+from pypy.jit.metainterp.history import ResOperation, BoxInt, ConstInt,\
+     BoxPtr, ConstPtr, BasicFailDescr, LoopToken
+from pypy.jit.metainterp.resoperation import rop
+from pypy.jit.backend.detect_cpu import getcpuclass
+from pypy.jit.backend.arm.arch import WORD
+CPU = getcpuclass()
+
+def test_bug_rshift():
+    v1 = BoxInt()
+    v2 = BoxInt()
+    v3 = BoxInt()
+    v4 = BoxInt()
+    inputargs = [v1]
+    operations = [
+        ResOperation(rop.INT_ADD, [v1, v1], v2),
+        ResOperation(rop.INT_INVERT, [v2], v3),
+        ResOperation(rop.UINT_RSHIFT, [v1, ConstInt(3)], v4),
+        ResOperation(rop.FINISH, [v4, v3], None, descr=BasicFailDescr()),
+        ]
+    cpu = CPU(None, None)
+    cpu.setup_once()
+    looptoken = LoopToken()
+    cpu.compile_loop(inputargs, operations, looptoken)
+    cpu.set_future_value_int(0, 9)
+    cpu.execute_token(looptoken)
+    assert cpu.get_latest_value_int(0) == (9 >> 3)
+    assert cpu.get_latest_value_int(1) == (~18)
+
+def test_bug_int_is_true_1():
+    v1 = BoxInt()
+    v2 = BoxInt()
+    v3 = BoxInt()
+    v4 = BoxInt()
+    tmp5 = BoxInt()
+    inputargs = [v1]
+    operations = [
+        ResOperation(rop.INT_MUL, [v1, v1], v2),
+        ResOperation(rop.INT_MUL, [v2, v1], v3),
+        ResOperation(rop.INT_IS_TRUE, [v2], tmp5),
+        ResOperation(rop.INT_IS_ZERO, [tmp5], v4),
+        ResOperation(rop.FINISH, [v4, v3, tmp5], None, descr=BasicFailDescr()),
+            ]
+    cpu = CPU(None, None)
+    cpu.setup_once()
+    looptoken = LoopToken()
+    cpu.compile_loop(inputargs, operations, looptoken)
+    cpu.set_future_value_int(0, -10)
+    cpu.execute_token(looptoken)
+    assert cpu.get_latest_value_int(0) == 0
+    assert cpu.get_latest_value_int(1) == -1000
+    assert cpu.get_latest_value_int(2) == 1
+
+def test_bug_0():
+    v1 = BoxInt()
+    v2 = BoxInt()
+    v3 = BoxInt()
+    v4 = BoxInt()
+    v5 = BoxInt()
+    v6 = BoxInt()
+    v7 = BoxInt()
+    v8 = BoxInt()
+    v9 = BoxInt()
+    v10 = BoxInt()
+    v11 = BoxInt()
+    v12 = BoxInt()
+    v13 = BoxInt()
+    v14 = BoxInt()
+    v15 = BoxInt()
+    v16 = BoxInt()
+    v17 = BoxInt()
+    v18 = BoxInt()
+    v19 = BoxInt()
+    v20 = BoxInt()
+    v21 = BoxInt()
+    v22 = BoxInt()
+    v23 = BoxInt()
+    v24 = BoxInt()
+    v25 = BoxInt()
+    v26 = BoxInt()
+    v27 = BoxInt()
+    v28 = BoxInt()
+    v29 = BoxInt()
+    v30 = BoxInt()
+    v31 = BoxInt()
+    v32 = BoxInt()
+    v33 = BoxInt()
+    v34 = BoxInt()
+    v35 = BoxInt()
+    v36 = BoxInt()
+    v37 = BoxInt()
+    v38 = BoxInt()
+    v39 = BoxInt()
+    v40 = BoxInt()
+    tmp41 = BoxInt()
+    tmp42 = BoxInt()
+    tmp43 = BoxInt()
+    tmp44 = BoxInt()
+    tmp45 = BoxInt()
+    tmp46 = BoxInt()
+    inputargs = [v1, v2, v3, v4, v5, v6, v7, v8, v9, v10]
+    operations = [
+        ResOperation(rop.UINT_GT, [v3, ConstInt(-48)], v11),
+        ResOperation(rop.INT_XOR, [v8, v1], v12),
+        ResOperation(rop.INT_GT, [v6, ConstInt(-9)], v13),
+        ResOperation(rop.INT_LE, [v13, v2], v14),
+        ResOperation(rop.INT_LE, [v11, v5], v15),
+        ResOperation(rop.UINT_GE, [v13, v13], v16),
+        ResOperation(rop.INT_OR, [v9, ConstInt(-23)], v17),
+        ResOperation(rop.INT_LT, [v10, v13], v18),
+        ResOperation(rop.INT_OR, [v15, v5], v19),
+        ResOperation(rop.INT_XOR, [v17, ConstInt(54)], v20),
+        ResOperation(rop.INT_MUL, [v8, v10], v21),
+        ResOperation(rop.INT_OR, [v3, v9], v22),
+        ResOperation(rop.INT_AND, [v11, ConstInt(-4)], tmp41),
+        ResOperation(rop.INT_OR, [tmp41, ConstInt(1)], tmp42),
+        ResOperation(rop.INT_MOD, [v12, tmp42], v23),
+        ResOperation(rop.INT_IS_TRUE, [v6], v24),
+        ResOperation(rop.UINT_RSHIFT, [v15, ConstInt(6)], v25),
+        ResOperation(rop.INT_OR, [ConstInt(-4), v25], v26),
+        ResOperation(rop.INT_INVERT, [v8], v27),
+        ResOperation(rop.INT_SUB, [ConstInt(-113), v11], v28),
+        ResOperation(rop.INT_NEG, [v7], v29),
+        ResOperation(rop.INT_NEG, [v24], v30),
+        ResOperation(rop.INT_FLOORDIV, [v3, ConstInt(53)], v31),
+        ResOperation(rop.INT_MUL, [v28, v27], v32),
+        ResOperation(rop.INT_AND, [v18, ConstInt(-4)], tmp43),
+        ResOperation(rop.INT_OR, [tmp43, ConstInt(1)], tmp44),
+        ResOperation(rop.INT_MOD, [v26, tmp44], v33),
+        ResOperation(rop.INT_OR, [v27, v19], v34),
+        ResOperation(rop.UINT_LT, [v13, ConstInt(1)], v35),
+        ResOperation(rop.INT_AND, [v21, ConstInt(31)], tmp45),
+        ResOperation(rop.INT_RSHIFT, [v21, tmp45], v36),
+        ResOperation(rop.INT_AND, [v20, ConstInt(31)], tmp46),
+        ResOperation(rop.UINT_RSHIFT, [v4, tmp46], v37),
+        ResOperation(rop.UINT_GT, [v33, ConstInt(-11)], v38),
+        ResOperation(rop.INT_NEG, [v7], v39),
+        ResOperation(rop.INT_GT, [v24, v32], v40),
+        ResOperation(rop.FINISH, [v40, v36, v37, v31, v16, v34, v35, v23, v22, v29, v14, v39, v30, v38], None, descr=BasicFailDescr()),
+            ]
+    cpu = CPU(None, None)
+    cpu.setup_once()
+    looptoken = LoopToken()
+    cpu.compile_loop(inputargs, operations, looptoken)
+    cpu.set_future_value_int(0, -13)
+    cpu.set_future_value_int(1, 10)
+    cpu.set_future_value_int(2, 10)
+    cpu.set_future_value_int(3, 8)
+    cpu.set_future_value_int(4, -8)
+    cpu.set_future_value_int(5, -16)
+    cpu.set_future_value_int(6, -18)
+    cpu.set_future_value_int(7, 46)
+    cpu.set_future_value_int(8, -12)
+    cpu.set_future_value_int(9, 26)
+    cpu.execute_token(looptoken)
+    assert cpu.get_latest_value_int(0) == 0
+    assert cpu.get_latest_value_int(1) == 0
+    assert cpu.get_latest_value_int(2) == 0
+    assert cpu.get_latest_value_int(3) == 0
+    assert cpu.get_latest_value_int(4) == 1
+    assert cpu.get_latest_value_int(5) == -7
+    assert cpu.get_latest_value_int(6) == 1
+    assert cpu.get_latest_value_int(7) == 0
+    assert cpu.get_latest_value_int(8) == -2
+    assert cpu.get_latest_value_int(9) == 18
+    assert cpu.get_latest_value_int(10) == 1
+    assert cpu.get_latest_value_int(11) == 18
+    assert cpu.get_latest_value_int(12) == -1
+    assert cpu.get_latest_value_int(13) == 0
+
+def test_bug_1():
+    v1 = BoxInt()
+    v2 = BoxInt()
+    v3 = BoxInt()
+    v4 = BoxInt()
+    v5 = BoxInt()
+    v6 = BoxInt()
+    v7 = BoxInt()
+    v8 = BoxInt()
+    v9 = BoxInt()
+    v10 = BoxInt()
+    v11 = BoxInt()
+    v12 = BoxInt()
+    v13 = BoxInt()
+    v14 = BoxInt()
+    v15 = BoxInt()
+    v16 = BoxInt()
+    v17 = BoxInt()
+    v18 = BoxInt()
+    v19 = BoxInt()
+    v20 = BoxInt()
+    v21 = BoxInt()
+    v22 = BoxInt()
+    v23 = BoxInt()
+    v24 = BoxInt()
+    v25 = BoxInt()
+    v26 = BoxInt()
+    v27 = BoxInt()
+    v28 = BoxInt()
+    v29 = BoxInt()
+    v30 = BoxInt()
+    v31 = BoxInt()
+    v32 = BoxInt()
+    v33 = BoxInt()
+    v34 = BoxInt()
+    v35 = BoxInt()
+    v36 = BoxInt()
+    v37 = BoxInt()
+    v38 = BoxInt()
+    v39 = BoxInt()
+    v40 = BoxInt()
+    tmp41 = BoxInt()
+    tmp42 = BoxInt()
+    tmp43 = BoxInt()
+    tmp44 = BoxInt()
+    tmp45 = BoxInt()
+    inputargs = [v1, v2, v3, v4, v5, v6, v7, v8, v9, v10]
+    operations = [
+        ResOperation(rop.UINT_LT, [v6, ConstInt(0)], v11),
+        ResOperation(rop.INT_AND, [v3, ConstInt(31)], tmp41),
+        ResOperation(rop.INT_RSHIFT, [v3, tmp41], v12),
+        ResOperation(rop.INT_NEG, [v2], v13),
+        ResOperation(rop.INT_ADD, [v11, v7], v14),
+        ResOperation(rop.INT_OR, [v3, v2], v15),
+        ResOperation(rop.INT_OR, [v12, v12], v16),
+        ResOperation(rop.INT_NE, [v2, v5], v17),
+        ResOperation(rop.INT_AND, [v5, ConstInt(31)], tmp42),
+        ResOperation(rop.UINT_RSHIFT, [v14, tmp42], v18),
+        ResOperation(rop.INT_AND, [v14, ConstInt(31)], tmp43),
+        ResOperation(rop.INT_LSHIFT, [ConstInt(7), tmp43], v19),
+        ResOperation(rop.INT_NEG, [v19], v20),
+        ResOperation(rop.INT_MOD, [v3, ConstInt(1)], v21),
+        ResOperation(rop.UINT_GE, [v15, v1], v22),
+        ResOperation(rop.INT_AND, [v16, ConstInt(31)], tmp44),
+        ResOperation(rop.INT_LSHIFT, [v8, tmp44], v23),
+        ResOperation(rop.INT_IS_TRUE, [v17], v24),
+        ResOperation(rop.INT_AND, [v5, ConstInt(31)], tmp45),
+        ResOperation(rop.INT_LSHIFT, [v14, tmp45], v25),
+        ResOperation(rop.INT_LSHIFT, [v5, ConstInt(17)], v26),
+        ResOperation(rop.INT_EQ, [v9, v15], v27),
+        ResOperation(rop.INT_GE, [ConstInt(0), v6], v28),
+        ResOperation(rop.INT_NEG, [v15], v29),
+        ResOperation(rop.INT_NEG, [v22], v30),
+        ResOperation(rop.INT_ADD, [v7, v16], v31),
+        ResOperation(rop.UINT_LT, [v19, v19], v32),
+        ResOperation(rop.INT_ADD, [v2, ConstInt(1)], v33),
+        ResOperation(rop.INT_NEG, [v5], v34),
+        ResOperation(rop.INT_ADD, [v17, v24], v35),
+        ResOperation(rop.UINT_LT, [ConstInt(2), v16], v36),
+        ResOperation(rop.INT_NEG, [v9], v37),
+        ResOperation(rop.INT_GT, [v4, v11], v38),
+        ResOperation(rop.INT_LT, [v27, v22], v39),
+        ResOperation(rop.INT_NEG, [v27], v40),
+        ResOperation(rop.FINISH, [v40, v10, v36, v26, v13, v30, v21, v33, v18, v25, v31, v32, v28, v29, v35, v38, v20, v39, v34, v23, v37], None, descr=BasicFailDescr()),
+            ]
+    cpu = CPU(None, None)
+    cpu.setup_once()
+    looptoken = LoopToken()
+    cpu.compile_loop(inputargs, operations, looptoken)
+    cpu.set_future_value_int(0, 17)
+    cpu.set_future_value_int(1, -20)
+    cpu.set_future_value_int(2, -6)
+    cpu.set_future_value_int(3, 6)
+    cpu.set_future_value_int(4, 1)
+    cpu.set_future_value_int(5, 13)
+    cpu.set_future_value_int(6, 13)
+    cpu.set_future_value_int(7, 9)
+    cpu.set_future_value_int(8, 49)
+    cpu.set_future_value_int(9, 8)
+    cpu.execute_token(looptoken)
+    assert cpu.get_latest_value_int(0) == 0
+    assert cpu.get_latest_value_int(1) == 8
+    assert cpu.get_latest_value_int(2) == 1
+    assert cpu.get_latest_value_int(3) == 131072
+    assert cpu.get_latest_value_int(4) == 20
+    assert cpu.get_latest_value_int(5) == -1
+    assert cpu.get_latest_value_int(6) == 0
+    assert cpu.get_latest_value_int(7) == -19
+    assert cpu.get_latest_value_int(8) == 6
+    assert cpu.get_latest_value_int(9) == 26
+    assert cpu.get_latest_value_int(10) == 12
+    assert cpu.get_latest_value_int(11) == 0
+    assert cpu.get_latest_value_int(12) == 0
+    assert cpu.get_latest_value_int(13) == 2
+    assert cpu.get_latest_value_int(14) == 2
+    assert cpu.get_latest_value_int(15) == 1
+    assert cpu.get_latest_value_int(16) == -57344
+    assert cpu.get_latest_value_int(17) == 1
+    assert cpu.get_latest_value_int(18) == -1
+    if WORD == 4:
+        assert cpu.get_latest_value_int(19) == -2147483648
+    elif WORD == 8:
+        assert cpu.get_latest_value_int(19) == 19327352832
+    assert cpu.get_latest_value_int(20) == -49

diff --git a/pypy/jit/backend/arm/test/test_recompilation.py b/pypy/jit/backend/arm/test/test_recompilation.py
new file mode 100644
--- /dev/null
+++ b/pypy/jit/backend/arm/test/test_recompilation.py
@@ -0,0 +1,142 @@
+from pypy.jit.backend.arm.test.test_regalloc import BaseTestRegalloc
+
+class TestRecompilation(BaseTestRegalloc):
+    def test_compile_bridge_not_deeper(self):
+        ops = '''
+        [i0]
+        i1 = int_add(i0, 1)
+        i2 = int_lt(i1, 20)
+        guard_true(i2, descr=fdescr1) [i1]
+        jump(i1)
+        '''
+        loop = self.interpret(ops, [0])
+        assert self.getint(0) == 20
+        ops = '''
+        [i1]
+        i3 = int_add(i1, 1)
+        finish(i3, descr=fdescr2)
+        '''
+        bridge = self.attach_bridge(ops, loop, -2)
+        self.cpu.set_future_value_int(0, 0)
+        fail = self.run(loop)
+        assert fail.identifier == 2
+        assert self.getint(0) == 21
+    
+    def test_compile_bridge_deeper(self):
+        ops = '''
+        [i0]
+        i1 = int_add(i0, 1)
+        i2 = int_lt(i1, 20)
+        guard_true(i2, descr=fdescr1) [i1]
+        jump(i1)
+        '''
+        loop = self.interpret(ops, [0])
+        assert self.getint(0) == 20
+        ops = '''
+        [i1]
+        i3 = int_add(i1, 1)
+        i4 = int_add(i3, 1)
+        i5 = int_add(i4, 1)
+        i6 = int_add(i5, 1)
+        i7 = int_add(i5, i4)
+        i8 = int_add(i7, 1)
+        i9 = int_add(i8, 1)
+        finish(i3, i4, i5, i6, i7, i8, i9, descr=fdescr2)
+        '''
+        bridge = self.attach_bridge(ops, loop, -2)
+        descr = loop.operations[2].getdescr()
+        self.cpu.set_future_value_int(0, 0)
+        fail = self.run(loop)
+        assert fail.identifier == 2
+        assert self.getint(0) == 21
+        assert self.getint(1) == 22
+        assert self.getint(2) == 23
+        assert self.getint(3) == 24
+
+    def test_bridge_jump_to_other_loop(self):
+        loop = self.interpret('''
+        [i0, i10, i11, i12, i13, i14, i15, i16]
+        i1 = int_add(i0, 1)
+        i2 = int_lt(i1, 20)
+        guard_true(i2, descr=fdescr1) [i1]
+        jump(i1, i10, i11, i12, i13, i14, i15, i16)
+        ''', [0])
+        other_loop = self.interpret('''
+        [i3]
+        guard_false(i3, descr=fdescr2) [i3]
+        jump(i3)
+        ''', [1])
+        ops = '''
+        [i3]
+        jump(i3, 1, 2, 3, 4, 5, 6, 7, descr=looptoken)
+        '''
+        bridge = self.attach_bridge(ops, other_loop, 0, looptoken=loop.token)
+        self.cpu.set_future_value_int(0, 1)
+        fail = self.run(other_loop)
+        assert fail.identifier == 1
+
+    def test_bridge_jumps_to_self_deeper(self):
+        loop = self.interpret('''
+        [i0, i1, i2, i31, i32, i33]
+        i98 = same_as(0)
+        i99 = same_as(1)
+        i30 = int_add(i1, i2)
+        i3 = int_add(i0, 1)
+        i4 = int_and(i3, 1)
+        guard_false(i4) [i98, i3]
+        i5 = int_lt(i3, 20)
+        guard_true(i5) [i99, i3]
+        jump(i3, i30, 1, i30, i30, i30)
+        ''', [0])
+        assert self.getint(0) == 0
+        assert self.getint(1) == 1
+        ops = '''
+        [i97, i3]
+        i10 = int_mul(i3, 2)
+        i8 = int_add(i3, 1)
+        i6 = int_add(i8, i10)
+        i7 = int_add(i3, i6)
+        i12 = int_add(i7, i8)
+        i11 = int_add(i12, i6)
+        jump(i3, i12, i11, i10, i6, i7, descr=looptoken)
+        '''
+        bridge = self.attach_bridge(ops, loop, 5, looptoken=loop.token)
+        guard_op = loop.operations[5]
+        #loop_frame_depth = loop.token._x86_frame_depth
+        #assert loop.token._x86_param_depth == 0
+        ## XXX: Maybe add enough ops to force stack on 64-bit as well?
+        #    assert guard_op.getdescr()._arm_bridge_frame_depth > loop_frame_depth
+        #assert guard_op.getdescr()._x86_bridge_param_depth == 0
+        self.cpu.set_future_value_int(0, 0)
+        self.cpu.set_future_value_int(1, 0)
+        self.cpu.set_future_value_int(2, 0)
+        self.run(loop)
+        assert self.getint(0) == 1
+        assert self.getint(1) == 20
+
+    def test_bridge_jumps_to_self_shallower(self):
+        loop = self.interpret('''
+        [i0, i1, i2]
+        i98 = same_as(0)
+        i99 = same_as(1)
+        i3 = int_add(i0, 1)
+        i4 = int_and(i3, 1)
+        guard_false(i4) [i98, i3]
+        i5 = int_lt(i3, 20)
+        guard_true(i5) [i99, i3]
+        jump(i3, i1, i2)
+        ''', [0])
+        assert self.getint(0) == 0
+        assert self.getint(1) == 1
+        ops = '''
+        [i97, i3]
+        jump(i3, 0, 1, descr=looptoken)
+        '''
+        bridge = self.attach_bridge(ops, loop, 4, looptoken=loop.token)
+        self.cpu.set_future_value_int(0, 0)
+        self.cpu.set_future_value_int(1, 0)
+        self.cpu.set_future_value_int(2, 0)
+        self.run(loop)
+        assert self.getint(0) == 1
+        assert self.getint(1) == 20
+        

diff --git a/pypy/jit/backend/arm/test/test_gc_integration.py b/pypy/jit/backend/arm/test/test_gc_integration.py
new file mode 100644
--- /dev/null
+++ b/pypy/jit/backend/arm/test/test_gc_integration.py
@@ -0,0 +1,259 @@
+
+""" Tests for register allocation for common constructs
+"""
+
+import py
+from pypy.jit.metainterp.history import BoxInt, ConstInt,\
+     BoxPtr, ConstPtr, TreeLoop
+from pypy.jit.metainterp.resoperation import rop, ResOperation
+from pypy.jit.codewriter import heaptracker
+from pypy.jit.backend.llsupport.descr import GcCache
+from pypy.jit.backend.llsupport.gc import GcLLDescription
+from pypy.jit.backend.detect_cpu import getcpuclass
+from pypy.jit.backend.arm.regalloc import ARMRegisterManager
+from pypy.jit.backend.arm.arch import WORD
+from pypy.jit.tool.oparser import parse
+from pypy.rpython.lltypesystem import lltype, llmemory, rffi
+from pypy.rpython.annlowlevel import llhelper
+from pypy.rpython.lltypesystem import rclass, rstr
+from pypy.jit.backend.llsupport.gc import GcLLDescr_framework, GcRefList, GcPtrFieldDescr
+
+from pypy.jit.backend.arm.test.test_regalloc import MockAssembler
+from pypy.jit.backend.arm.test.test_regalloc import BaseTestRegalloc
+
+CPU = getcpuclass()
+
+class MockGcRootMap(object):
+    def get_basic_shape(self, is_64_bit):
+        return ['shape']
+    def add_ebp_offset(self, shape, offset):
+        shape.append(offset)
+    def add_callee_save_reg(self, shape, reg_index):
+        index_to_name = { 1: 'ebx', 2: 'esi', 3: 'edi' }
+        shape.append(index_to_name[reg_index])
+    def compress_callshape(self, shape, datablockwrapper):
+        assert datablockwrapper == 'fakedatablockwrapper'
+        assert shape[0] == 'shape'
+        return ['compressed'] + shape[1:]
+
+class MockGcDescr(GcCache):
+    def get_funcptr_for_new(self):
+        return 123
+    get_funcptr_for_newarray = get_funcptr_for_new
+    get_funcptr_for_newstr = get_funcptr_for_new
+    get_funcptr_for_newunicode = get_funcptr_for_new
+    
+    moving_gc = True
+    gcrootmap = MockGcRootMap()
+
+    def initialize(self):
+        self.gcrefs = GcRefList()
+        self.gcrefs.initialize()
+        self.single_gcref_descr = GcPtrFieldDescr('', 0)
+        
+    rewrite_assembler = GcLLDescr_framework.rewrite_assembler.im_func
+
+
+class TestRegallocGcIntegration(BaseTestRegalloc):
+    
+    cpu = CPU(None, None)
+    cpu.gc_ll_descr = MockGcDescr(False)
+    cpu.setup_once()
+    
+    S = lltype.GcForwardReference()
+    S.become(lltype.GcStruct('S', ('field', lltype.Ptr(S)),
+                             ('int', lltype.Signed)))
+
+    fielddescr = cpu.fielddescrof(S, 'field')
+
+    struct_ptr = lltype.malloc(S)
+    struct_ref = lltype.cast_opaque_ptr(llmemory.GCREF, struct_ptr)
+    child_ptr = lltype.nullptr(S)
+    struct_ptr.field = child_ptr
+
+
+    descr0 = cpu.fielddescrof(S, 'int')
+    ptr0 = struct_ref
+
+    namespace = locals().copy()
+
+    def test_basic(self):
+        ops = '''
+        [p0]
+        p1 = getfield_gc(p0, descr=fielddescr)
+        finish(p1)
+        '''
+        self.interpret(ops, [self.struct_ptr])
+        assert not self.getptr(0, lltype.Ptr(self.S))
+
+    def test_rewrite_constptr(self):
+        ops = '''
+        []
+        p1 = getfield_gc(ConstPtr(struct_ref), descr=fielddescr)
+        finish(p1)
+        '''
+        self.interpret(ops, [])
+        assert not self.getptr(0, lltype.Ptr(self.S))
+
+    def test_bug_0(self):
+        ops = '''
+        [i0, i1, i2, i3, i4, i5, i6, i7, i8]
+        guard_value(i2, 1) [i2, i3, i4, i5, i6, i7, i0, i1, i8]
+        guard_class(i4, 138998336) [i4, i5, i6, i7, i0, i1, i8]
+        i11 = getfield_gc(i4, descr=descr0)
+        guard_nonnull(i11) [i4, i5, i6, i7, i0, i1, i11, i8]
+        i13 = getfield_gc(i11, descr=descr0)
+        guard_isnull(i13) [i4, i5, i6, i7, i0, i1, i11, i8]
+        i15 = getfield_gc(i4, descr=descr0)
+        i17 = int_lt(i15, 0)
+        guard_false(i17) [i4, i5, i6, i7, i0, i1, i11, i15, i8]
+        i18 = getfield_gc(i11, descr=descr0)
+        i19 = int_ge(i15, i18)
+        guard_false(i19) [i4, i5, i6, i7, i0, i1, i11, i15, i8]
+        i20 = int_lt(i15, 0)
+        guard_false(i20) [i4, i5, i6, i7, i0, i1, i11, i15, i8]
+        i21 = getfield_gc(i11, descr=descr0)
+        i22 = getfield_gc(i11, descr=descr0)
+        i23 = int_mul(i15, i22)
+        i24 = int_add(i21, i23)
+        i25 = getfield_gc(i4, descr=descr0)
+        i27 = int_add(i25, 1)
+        setfield_gc(i4, i27, descr=descr0)
+        i29 = getfield_raw(144839744, descr=descr0)
+        i31 = int_and(i29, -2141192192)
+        i32 = int_is_true(i31)
+        guard_false(i32) [i4, i6, i7, i0, i1, i24]
+        i33 = getfield_gc(i0, descr=descr0)
+        guard_value(i33, ConstPtr(ptr0)) [i4, i6, i7, i0, i1, i33, i24]
+        jump(i0, i1, 1, 17, i4, ConstPtr(ptr0), i6, i7, i24)
+        '''
+        self.interpret(ops, [0, 0, 0, 0, 0, 0, 0, 0, 0], run=False)
+
+class GCDescrFastpathMalloc(GcLLDescription):
+    gcrootmap = None
+    
+    def __init__(self):
+        GcCache.__init__(self, False)
+        # create a nursery
+        NTP = rffi.CArray(lltype.Signed)
+        self.nursery = lltype.malloc(NTP, 16, flavor='raw')
+        self.addrs = lltype.malloc(rffi.CArray(lltype.Signed), 2,
+                                   flavor='raw')
+        self.addrs[0] = rffi.cast(lltype.Signed, self.nursery)
+        self.addrs[1] = self.addrs[0] + 64
+        # 64 bytes
+        def malloc_slowpath(size):
+            assert size == WORD*2
+            nadr = rffi.cast(lltype.Signed, self.nursery)
+            self.addrs[0] = nadr + size
+            return nadr
+        self.malloc_slowpath = malloc_slowpath
+        self.MALLOC_SLOWPATH = lltype.FuncType([lltype.Signed],
+                                               lltype.Signed)
+        self._counter = 123
+
+    def can_inline_malloc(self, descr):
+        return True
+
+    def get_funcptr_for_new(self):
+        return 42
+#        return llhelper(lltype.Ptr(self.NEW_TP), self.new)
+
+    def init_size_descr(self, S, descr):
+        descr.tid = self._counter
+        self._counter += 1
+
+    def get_nursery_free_addr(self):
+        return rffi.cast(lltype.Signed, self.addrs)
+
+    def get_nursery_top_addr(self):
+        return rffi.cast(lltype.Signed, self.addrs) + WORD
+
+    def get_malloc_fixedsize_slowpath_addr(self):
+        fptr = llhelper(lltype.Ptr(self.MALLOC_SLOWPATH), self.malloc_slowpath)
+        return rffi.cast(lltype.Signed, fptr)
+
+    get_funcptr_for_newarray = None
+    get_funcptr_for_newstr = None
+    get_funcptr_for_newunicode = None
+
+class TestMallocFastpath(BaseTestRegalloc):
+
+    def setup_method(self, method):
+        cpu = CPU(None, None)
+        cpu.vtable_offset = WORD
+        cpu.gc_ll_descr = GCDescrFastpathMalloc()
+        cpu.setup_once()
+
+        NODE = lltype.Struct('node', ('tid', lltype.Signed),
+                                     ('value', lltype.Signed))
+        nodedescr = cpu.sizeof(NODE)     # xxx hack: NODE is not a GcStruct
+        valuedescr = cpu.fielddescrof(NODE, 'value')
+
+        self.cpu = cpu
+        self.nodedescr = nodedescr
+        vtable = lltype.malloc(rclass.OBJECT_VTABLE, immortal=True)
+        vtable_int = cpu.cast_adr_to_int(llmemory.cast_ptr_to_adr(vtable))
+        NODE2 = lltype.GcStruct('node2',
+                                  ('parent', rclass.OBJECT),
+                                  ('tid', lltype.Signed),
+                                  ('vtable', lltype.Ptr(rclass.OBJECT_VTABLE)))
+        descrsize = cpu.sizeof(NODE2)
+        heaptracker.register_known_gctype(cpu, vtable, NODE2)
+        self.descrsize = descrsize
+        self.vtable_int = vtable_int
+
+        self.namespace = locals().copy()
+        
+    def test_malloc_fastpath(self):
+        py.test.skip()
+        ops = '''
+        [i0]
+        p0 = new(descr=nodedescr)
+        setfield_gc(p0, i0, descr=valuedescr)
+        finish(p0)
+        '''
+        self.interpret(ops, [42])
+        # check the nursery
+        gc_ll_descr = self.cpu.gc_ll_descr
+        assert gc_ll_descr.nursery[0] == self.nodedescr.tid
+        assert gc_ll_descr.nursery[1] == 42
+        nurs_adr = rffi.cast(lltype.Signed, gc_ll_descr.nursery)
+        assert gc_ll_descr.addrs[0] == nurs_adr + (WORD*2)
+
+    def test_malloc_slowpath(self):
+        py.test.skip()
+        ops = '''
+        []
+        p0 = new(descr=nodedescr)
+        p1 = new(descr=nodedescr)
+        p2 = new(descr=nodedescr)
+        p3 = new(descr=nodedescr)
+        p4 = new(descr=nodedescr)
+        p5 = new(descr=nodedescr)
+        p6 = new(descr=nodedescr)
+        p7 = new(descr=nodedescr)
+        p8 = new(descr=nodedescr)
+        finish(p0, p1, p2, p3, p4, p5, p6, p7, p8)
+        '''
+        self.interpret(ops, [])
+        # this should call slow path once
+        gc_ll_descr = self.cpu.gc_ll_descr
+        nadr = rffi.cast(lltype.Signed, gc_ll_descr.nursery)
+        assert gc_ll_descr.addrs[0] == nadr + (WORD*2)
+
+    def test_new_with_vtable(self):
+        py.test.skip()
+        ops = '''
+        [i0, i1]
+        p0 = new_with_vtable(ConstClass(vtable))
+        guard_class(p0, ConstClass(vtable)) [i0]
+        finish(i1)
+        '''
+        self.interpret(ops, [0, 1])
+        assert self.getint(0) == 1
+        gc_ll_descr = self.cpu.gc_ll_descr
+        assert gc_ll_descr.nursery[0] == self.descrsize.tid
+        assert gc_ll_descr.nursery[1] == self.vtable_int
+        nurs_adr = rffi.cast(lltype.Signed, gc_ll_descr.nursery)
+        assert gc_ll_descr.addrs[0] == nurs_adr + (WORD*3)

diff --git a/pypy/jit/backend/arm/test/support.py b/pypy/jit/backend/arm/test/support.py
--- a/pypy/jit/backend/arm/test/support.py
+++ b/pypy/jit/backend/arm/test/support.py
@@ -2,6 +2,16 @@
 import py
 
 from pypy.rpython.lltypesystem import lltype, rffi
+from pypy.jit.backend.detect_cpu import getcpuclass
+from pypy.jit.metainterp.test import test_basic
+from pypy.rlib.jit import JitDriver
+
+class JitARMMixin(test_basic.LLJitMixin):
+    type_system = 'lltype'
+    CPUClass = getcpuclass()
+
+    def check_jumps(self, maxcount):
+        pass
 
 if os.uname()[1] == 'llaima.local':
     AS = '~/Code/arm-jit/android/android-ndk-r4b//build/prebuilt/darwin-x86/arm-eabi-4.4.0/arm-eabi/bin/as'

diff --git a/pypy/jit/backend/arm/test/test_loop_unroll.py b/pypy/jit/backend/arm/test/test_loop_unroll.py
new file mode 100644
--- /dev/null
+++ b/pypy/jit/backend/arm/test/test_loop_unroll.py
@@ -0,0 +1,8 @@
+import py
+from pypy.jit.backend.x86.test.test_basic import Jit386Mixin
+from pypy.jit.metainterp.test import test_loop_unroll
+
+class TestLoopSpec(Jit386Mixin, test_loop_unroll.LoopUnrollTest):
+    # for the individual tests see
+    # ====> ../../../metainterp/test/test_loop.py
+    pass

diff --git a/pypy/jit/backend/arm/test/test_regalloc.py b/pypy/jit/backend/arm/test/test_regalloc.py
new file mode 100644
--- /dev/null
+++ b/pypy/jit/backend/arm/test/test_regalloc.py
@@ -0,0 +1,633 @@
+
+""" Tests for register allocation for common constructs
+"""
+
+import py
+from pypy.jit.metainterp.history import BoxInt, ConstInt,\
+     BoxPtr, ConstPtr, LoopToken, BasicFailDescr
+from pypy.jit.metainterp.resoperation import rop, ResOperation
+from pypy.jit.backend.llsupport.descr import GcCache
+from pypy.jit.backend.detect_cpu import getcpuclass
+from pypy.jit.backend.arm.regalloc import ARMRegisterManager
+from pypy.jit.tool.oparser import parse
+from pypy.rpython.lltypesystem import lltype, llmemory, rffi
+from pypy.rpython.annlowlevel import llhelper
+from pypy.rpython.lltypesystem import rclass, rstr
+
+
+CPU = getcpuclass()
+class MockGcDescr(GcCache):
+    def get_funcptr_for_new(self):
+        return 123
+    get_funcptr_for_newarray = get_funcptr_for_new
+    get_funcptr_for_newstr = get_funcptr_for_new
+    get_funcptr_for_newunicode = get_funcptr_for_new
+ 
+    def rewrite_assembler(self, cpu, operations):
+        pass
+
+class MockAssembler(object):
+    gcrefs = None
+    _float_constants = None
+
+    def __init__(self, cpu=None, gc_ll_descr=None):
+        self.movs = []
+        self.performs = []
+        self.lea = []
+        if cpu is None:
+            cpu = CPU(None, None)
+            cpu.setup_once()
+        self.cpu = cpu
+        if gc_ll_descr is None:
+            gc_ll_descr = MockGcDescr(False)
+        self.cpu.gc_ll_descr = gc_ll_descr
+
+    def dump(self, *args):
+        pass
+
+    def regalloc_mov(self, from_loc, to_loc):
+        self.movs.append((from_loc, to_loc))
+
+    def regalloc_perform(self, op, arglocs, resloc):
+        self.performs.append((op, arglocs, resloc))
+
+    def regalloc_perform_discard(self, op, arglocs):
+        self.performs.append((op, arglocs))
+
+    def load_effective_addr(self, *args):
+        self.lea.append(args)
+
+class RegAllocForTests(ARMRegisterManager):
+    position = 0
+    def _compute_next_usage(self, v, _):
+        return -1
+
+class BaseTestRegalloc(object):
+    cpu = CPU(None, None)
+    cpu.setup_once()
+
+    def raising_func(i):
+        if i:
+            raise LLException(zero_division_error,
+                              zero_division_value)
+    FPTR = lltype.Ptr(lltype.FuncType([lltype.Signed], lltype.Void))
+    raising_fptr = llhelper(FPTR, raising_func)
+    zero_division_tp, zero_division_value = cpu.get_zero_division_error()
+    zd_addr = cpu.cast_int_to_adr(zero_division_tp)
+    zero_division_error = llmemory.cast_adr_to_ptr(zd_addr,
+                                            lltype.Ptr(rclass.OBJECT_VTABLE))
+    raising_calldescr = cpu.calldescrof(FPTR.TO, FPTR.TO.ARGS, FPTR.TO.RESULT)
+
+    fdescr1 = BasicFailDescr(1)
+    fdescr2 = BasicFailDescr(2)
+    fdescr3 = BasicFailDescr(3)
+
+    def f1(x):
+        return x+1
+
+    def f2(x, y):
+        return x*y
+
+    def f10(*args):
+        assert len(args) == 10
+        return sum(args)
+
+    F1PTR = lltype.Ptr(lltype.FuncType([lltype.Signed], lltype.Signed))
+    F2PTR = lltype.Ptr(lltype.FuncType([lltype.Signed]*2, lltype.Signed))
+    F10PTR = lltype.Ptr(lltype.FuncType([lltype.Signed]*10, lltype.Signed))
+    f1ptr = llhelper(F1PTR, f1)
+    f2ptr = llhelper(F2PTR, f2)
+    f10ptr = llhelper(F10PTR, f10)
+
+    f1_calldescr = cpu.calldescrof(F1PTR.TO, F1PTR.TO.ARGS, F1PTR.TO.RESULT)
+    f2_calldescr = cpu.calldescrof(F2PTR.TO, F2PTR.TO.ARGS, F2PTR.TO.RESULT)
+    f10_calldescr = cpu.calldescrof(F10PTR.TO, F10PTR.TO.ARGS, F10PTR.TO.RESULT)
+
+    namespace = locals().copy()
+    type_system = 'lltype'
+
+    def parse(self, s, boxkinds=None):
+        return parse(s, self.cpu, self.namespace,
+                     type_system=self.type_system,
+                     boxkinds=boxkinds)
+
+    def interpret(self, ops, args, run=True):
+        loop = self.parse(ops)
+        self.cpu.compile_loop(loop.inputargs, loop.operations, loop.token)
+        for i, arg in enumerate(args):
+            if isinstance(arg, int):
+                self.cpu.set_future_value_int(i, arg)
+            elif isinstance(arg, float):
+                self.cpu.set_future_value_float(i, arg)
+            else:
+                assert isinstance(lltype.typeOf(arg), lltype.Ptr)
+                llgcref = lltype.cast_opaque_ptr(llmemory.GCREF, arg)
+                self.cpu.set_future_value_ref(i, llgcref)
+        if run:
+            self.cpu.execute_token(loop.token)
+        return loop
+
+    def getint(self, index):
+        return self.cpu.get_latest_value_int(index)
+
+    def getfloat(self, index):
+        return self.cpu.get_latest_value_float(index)
+
+    def getints(self, end):
+        return [self.cpu.get_latest_value_int(index) for
+                index in range(0, end)]
+
+    def getfloats(self, end):
+        return [self.cpu.get_latest_value_float(index) for
+                index in range(0, end)]
+
+    def getptr(self, index, T):
+        gcref = self.cpu.get_latest_value_ref(index)
+        return lltype.cast_opaque_ptr(T, gcref)
+
+    def attach_bridge(self, ops, loop, guard_op_index, looptoken=None, **kwds):
+        if looptoken is not None:
+            self.namespace = self.namespace.copy()
+            self.namespace['looptoken'] = looptoken
+        guard_op = loop.operations[guard_op_index]
+        assert guard_op.is_guard()
+        bridge = self.parse(ops, **kwds)
+        assert ([box.type for box in bridge.inputargs] ==
+                [box.type for box in guard_op.getfailargs()])
+        faildescr = guard_op.getdescr()
+        self.cpu.compile_bridge(faildescr, bridge.inputargs, bridge.operations,
+                                loop.token)
+        return bridge
+
+    def run(self, loop):
+        return self.cpu.execute_token(loop.token)
+
+class TestRegallocSimple(BaseTestRegalloc):
+    def test_simple_loop(self):
+        ops = '''
+        [i0]
+        i1 = int_add(i0, 1)
+        i2 = int_lt(i1, 20)
+        guard_true(i2) [i1]
+        jump(i1)
+        '''
+        self.interpret(ops, [0])
+        assert self.getint(0) == 20
+
+    def test_two_loops_and_a_bridge(self):
+        ops = '''
+        [i0, i1, i2, i3]
+        i4 = int_add(i0, 1)
+        i5 = int_lt(i4, 20)
+        guard_true(i5) [i4, i1, i2, i3]
+        jump(i4, i1, i2, i3)
+        '''
+        loop = self.interpret(ops, [0, 0, 0, 0])
+        ops2 = '''
+        [i5]
+        i1 = int_add(i5, 1)
+        i3 = int_add(i1, 1)
+        i4 = int_add(i3, 1)
+        i2 = int_lt(i4, 30)
+        guard_true(i2) [i4]
+        jump(i4)
+        '''
+        loop2 = self.interpret(ops2, [0])
+        bridge_ops = '''
+        [i4]
+        jump(i4, i4, i4, i4, descr=looptoken)
+        '''
+        bridge = self.attach_bridge(bridge_ops, loop2, 4, looptoken=loop.token)
+        self.cpu.set_future_value_int(0, 0)
+        self.run(loop2)
+        assert self.getint(0) == 31
+        assert self.getint(1) == 30
+        assert self.getint(2) == 30
+        assert self.getint(3) == 30
+
+    def test_pointer_arg(self):
+        ops = '''
+        [i0, p0]
+        i1 = int_add(i0, 1)
+        i2 = int_lt(i1, 10)
+        guard_true(i2) [p0]
+        jump(i1, p0)
+        '''
+        S = lltype.GcStruct('S')
+        ptr = lltype.malloc(S)
+        self.cpu.clear_latest_values(2)
+        self.interpret(ops, [0, ptr])
+        assert self.getptr(0, lltype.Ptr(S)) == ptr
+
+    def test_exception_bridge_no_exception(self):
+        ops = '''
+        [i0]
+        i1 = same_as(1)
+        call(ConstClass(raising_fptr), i0, descr=raising_calldescr)
+        guard_exception(ConstClass(zero_division_error)) [i1]
+        finish(0)
+        '''
+        bridge_ops = '''
+        [i3]
+        i2 = same_as(2)
+        guard_no_exception() [i2]
+        finish(1)
+        '''
+        loop = self.interpret(ops, [0])
+        assert self.getint(0) == 1
+        bridge = self.attach_bridge(bridge_ops, loop, 2)
+        self.cpu.set_future_value_int(0, 0)
+        self.run(loop)
+        assert self.getint(0) == 1
+
+    def test_inputarg_unused(self):
+        ops = '''
+        [i0]
+        finish(1)
+        '''
+        self.interpret(ops, [0])
+        # assert did not explode
+
+    def test_nested_guards(self):
+        ops = '''
+        [i0, i1]
+        guard_true(i0) [i0, i1]
+        finish(4)
+        '''
+        bridge_ops = '''
+        [i0, i1]
+        guard_true(i0) [i0, i1]
+        finish(3)
+        '''
+        loop = self.interpret(ops, [0, 10])
+        assert self.getint(0) == 0
+        assert self.getint(1) == 10
+        bridge = self.attach_bridge(bridge_ops, loop, 0)
+        self.cpu.set_future_value_int(0, 0)
+        self.cpu.set_future_value_int(1, 10)
+        self.run(loop)
+        assert self.getint(0) == 0
+        assert self.getint(1) == 10
+
+    def test_nested_unused_arg(self):
+        ops = '''
+        [i0, i1]
+        guard_true(i0) [i0, i1]
+        finish(1)
+        '''
+        loop = self.interpret(ops, [0, 1])
+        assert self.getint(0) == 0
+        bridge_ops = '''
+        [i0, i1]
+        finish(1, 2)
+        '''
+        self.attach_bridge(bridge_ops, loop, 0)
+        self.cpu.set_future_value_int(0, 0)
+        self.cpu.set_future_value_int(1, 1)
+        self.run(loop)
+
+    def test_spill_for_constant(self):
+        ops = '''
+        [i0, i1, i2, i3]
+        i4 = int_add(3, i1)
+        i5 = int_lt(i4, 30)
+        guard_true(i5) [i0, i4, i2, i3]
+        jump(1, i4, 3, 4)
+        '''
+        self.interpret(ops, [0, 0, 0, 0])
+        assert self.getints(4) == [1, 30, 3, 4]
+
+    def test_spill_for_constant_lshift(self):
+        ops = '''
+        [i0, i2, i1, i3]
+        i4 = int_lshift(1, i1)
+        i5 = int_add(1, i1)
+        i6 = int_lt(i5, 30)
+        guard_true(i6) [i4, i5, i2, i3]
+        jump(i4, 3, i5, 4)
+        '''
+        self.interpret(ops, [0, 0, 0, 0])
+        assert self.getints(4) == [1<<29, 30, 3, 4]
+        ops = '''
+        [i0, i1, i2, i3]
+        i4 = int_lshift(1, i1)
+        i5 = int_add(1, i1)
+        i6 = int_lt(i5, 30)
+        guard_true(i6) [i4, i5, i2, i3]
+        jump(i4, i5, 3, 4)
+        '''
+        self.interpret(ops, [0, 0, 0, 0])
+        assert self.getints(4) == [1<<29, 30, 3, 4]
+        ops = '''
+        [i0, i3, i1, i2]
+        i4 = int_lshift(1, i1)
+        i5 = int_add(1, i1)
+        i6 = int_lt(i5, 30)
+        guard_true(i6) [i4, i5, i2, i3]
+        jump(i4, 4, i5, 3)
+        '''
+        self.interpret(ops, [0, 0, 0, 0])
+        assert self.getints(4) == [1<<29, 30, 3, 4]
+
+    def test_result_selected_reg_via_neg(self):
+        ops = '''
+        [i0, i1, i2, i3]
+        i6 = int_neg(i2)
+        i7 = int_add(1, i1)
+        i4 = int_lt(i7, 10)
+        guard_true(i4) [i0, i6, i7]
+        jump(1, i7, i2, i6)
+        '''
+        self.interpret(ops, [0, 0, 3, 0])
+        assert self.getints(3) == [1, -3, 10]
+        
+    def test_compare_memory_result_survives(self):
+        ops = '''
+        [i0, i1, i2, i3]
+        i4 = int_lt(i0, i1)
+        i5 = int_add(i3, 1)
+        i6 = int_lt(i5, 30)
+        guard_true(i6) [i4]
+        jump(i0, i1, i4, i5)
+        '''
+        self.interpret(ops, [0, 10, 0, 0])
+        assert self.getint(0) == 1
+
+    def test_jump_different_args(self):
+        ops = '''
+        [i0, i15, i16, i18, i1, i2, i3]
+        i4 = int_add(i3, 1)
+        i5 = int_lt(i4, 20)
+        guard_true(i5) [i2, i1]
+        jump(i0, i18, i15, i16, i2, i1, i4)
+        '''
+        self.interpret(ops, [0, 1, 2, 3])
+
+    def test_op_result_unused(self):
+        ops = '''
+        [i0, i1]
+        i2 = int_add(i0, i1)
+        finish(0)
+        '''
+        self.interpret(ops, [0, 0])
+
+    def test_guard_value_two_boxes(self):
+        ops = '''
+        [i0, i1, i2, i3, i4, i5, i6, i7]
+        guard_value(i6, i1) [i0, i2, i3, i4, i5, i6]
+        finish(i0, i2, i3, i4, i5, i6)
+        '''
+        self.interpret(ops, [0, 0, 0, 0, 0, 0, 0, 0])
+        assert self.getint(0) == 0
+
+    def test_bug_wrong_stack_adj(self):
+        ops = '''
+        [i0, i1, i2, i3, i4, i5, i6, i7, i8]
+        i9 = same_as(0)
+        guard_true(i0) [i9, i0, i1, i2, i3, i4, i5, i6, i7, i8]
+        finish(1, i0, i1, i2, i3, i4, i5, i6, i7, i8)
+        '''
+        loop = self.interpret(ops, [0, 1, 2, 3, 4, 5, 6, 7, 8])
+        assert self.getint(0) == 0
+        bridge_ops = '''
+        [i9, i0, i1, i2, i3, i4, i5, i6, i7, i8]
+        call(ConstClass(raising_fptr), 0, descr=raising_calldescr)
+        finish(i0, i1, i2, i3, i4, i5, i6, i7, i8)
+        '''
+        self.attach_bridge(bridge_ops, loop, 1)
+        for i in range(9):
+            self.cpu.set_future_value_int(i, i)
+        self.run(loop)
+        assert self.getints(9) == range(9)
+
+class TestRegallocCompOps(BaseTestRegalloc):
+    
+    def test_cmp_op_0(self):
+        ops = '''
+        [i0, i3]
+        i1 = same_as(1)
+        i2 = int_lt(i0, 100)
+        guard_true(i3) [i1, i2]
+        finish(0, i2)
+        '''
+        self.interpret(ops, [0, 1])
+        assert self.getint(0) == 0
+
+class TestRegallocMoreRegisters(BaseTestRegalloc):
+
+    cpu = BaseTestRegalloc.cpu
+
+    S = lltype.GcStruct('S', ('field', lltype.Char))
+    fielddescr = cpu.fielddescrof(S, 'field')
+
+    A = lltype.GcArray(lltype.Char)
+    arraydescr = cpu.arraydescrof(A)
+
+    namespace = locals().copy()
+
+    def test_int_is_true(self):
+        ops = '''
+        [i0, i1, i2, i3, i4, i5, i6, i7]
+        i10 = int_is_true(i0)
+        i11 = int_is_true(i1)
+        i12 = int_is_true(i2)
+        i13 = int_is_true(i3)
+        i14 = int_is_true(i4)
+        i15 = int_is_true(i5)
+        i16 = int_is_true(i6)
+        i17 = int_is_true(i7)
+        finish(i10, i11, i12, i13, i14, i15, i16, i17)
+        '''
+        self.interpret(ops, [0, 42, 12, 0, 13, 0, 0, 3333])
+        assert self.getints(8) == [0, 1, 1, 0, 1, 0, 0, 1]
+
+    def test_comparison_ops(self):
+        ops = '''
+        [i0, i1, i2, i3, i4, i5, i6]
+        i10 = int_lt(i0, i1)
+        i11 = int_le(i2, i3)
+        i12 = int_ge(i4, i5)
+        i13 = int_eq(i5, i6)
+        i14 = int_gt(i6, i2)
+        i15 = int_ne(i2, i6)
+        finish(i10, i11, i12, i13, i14, i15)
+        '''
+        self.interpret(ops, [0, 1, 2, 3, 4, 5, 6])
+        assert self.getints(6) == [1, 1, 0, 0, 1, 1]
+
+    def test_strsetitem(self):
+        ops = '''
+        [p0, i]
+        strsetitem(p0, 1, i)
+        finish()
+        '''
+        llstr  = rstr.mallocstr(10)
+        self.interpret(ops, [llstr, ord('a')])
+        assert llstr.chars[1] == 'a'
+
+    def test_setfield_char(self):
+        ops = '''
+        [p0, i]
+        setfield_gc(p0, i, descr=fielddescr)
+        finish()
+        '''
+        s = lltype.malloc(self.S)
+        self.interpret(ops, [s, ord('a')])
+        assert s.field == 'a'
+
+    def test_setarrayitem_gc(self):
+        ops = '''
+        [p0, i]
+        setarrayitem_gc(p0, 1, i, descr=arraydescr)
+        finish()
+        '''
+        s = lltype.malloc(self.A, 3)
+        self.interpret(ops, [s, ord('a')])
+        assert s[1] == 'a'
+
+    def test_division_optimized(self):
+        ops = '''
+        [i7, i6]
+        i18 = int_floordiv(i7, i6)
+        i19 = int_xor(i7, i6)
+        i21 = int_lt(i19, 0)
+        i22 = int_mod(i7, i6)
+        i23 = int_is_true(i22)
+        i24 = int_eq(i6, 4)
+        guard_false(i24) [i18]
+        jump(i18, i6)
+        '''
+        self.interpret(ops, [10, 4])
+        assert self.getint(0) == 2
+        # FIXME: Verify that i19 - i23 are removed
+
+class TestRegallocFloats(BaseTestRegalloc):
+    def test_float_add(self):
+        py.test.skip('need floats')
+        ops = '''
+        [f0, f1]
+        f2 = float_add(f0, f1)
+        finish(f2, f0, f1)
+        '''
+        self.interpret(ops, [3.0, 1.5])
+        assert self.getfloats(3) == [4.5, 3.0, 1.5]
+
+    def test_float_adds_stack(self):
+        py.test.skip('need floats')
+        ops = '''
+        [f0, f1, f2, f3, f4, f5, f6, f7, f8]
+        f9 = float_add(f0, f1)
+        f10 = float_add(f8, 3.5)
+        finish(f9, f10, f2, f3, f4, f5, f6, f7, f8)
+        '''
+        self.interpret(ops, [0.1, .2, .3, .4, .5, .6, .7, .8, .9])
+        assert self.getfloats(9) == [.1+.2, .9+3.5, .3, .4, .5, .6, .7, .8, .9]
+
+    def test_lt_const(self):
+        py.test.skip('need floats')
+        ops = '''
+        [f0]
+        i1 = float_lt(3.5, f0)
+        finish(i1)
+        '''
+        self.interpret(ops, [0.1])
+        assert self.getint(0) == 0
+
+    def test_bug_float_is_true_stack(self):
+        py.test.skip('need floats')
+        # NB. float_is_true no longer exists.  Unsure if keeping this test
+        # makes sense any more.
+        ops = '''
+        [f0, f1, f2, f3, f4, f5, f6, f7, f8, f9]
+        i0 = float_ne(f0, 0.0)
+        i1 = float_ne(f1, 0.0)
+        i2 = float_ne(f2, 0.0)
+        i3 = float_ne(f3, 0.0)
+        i4 = float_ne(f4, 0.0)
+        i5 = float_ne(f5, 0.0)
+        i6 = float_ne(f6, 0.0)
+        i7 = float_ne(f7, 0.0)
+        i8 = float_ne(f8, 0.0)
+        i9 = float_ne(f9, 0.0)
+        finish(i0, i1, i2, i3, i4, i5, i6, i7, i8, i9)
+        '''
+        loop = self.interpret(ops, [0.0, .1, .2, .3, .4, .5, .6, .7, .8, .9])
+        assert self.getints(9) == [0, 1, 1, 1, 1, 1, 1, 1, 1]
+
+class TestRegAllocCallAndStackDepth(BaseTestRegalloc):
+    def expected_param_depth(self, num_args):
+        # Assumes the arguments are all non-float
+        return num_args
+
+    def test_one_call(self):
+        ops = '''
+        [i0, i1, i2, i3, i4, i5, i6, i7, i8, i9]
+        i10 = call(ConstClass(f1ptr), i0, descr=f1_calldescr)
+        finish(i10, i1, i2, i3, i4, i5, i6, i7, i8, i9)
+        '''
+        loop = self.interpret(ops, [4, 7, 9, 9 ,9, 9, 9, 9, 9, 9, 9])
+        assert self.getints(11) == [5, 7, 9, 9, 9, 9, 9, 9, 9, 9, 9]
+
+    def test_two_calls(self):
+        ops = '''
+        [i0, i1,  i2, i3, i4, i5, i6, i7, i8, i9]
+        i10 = call(ConstClass(f1ptr), i0, descr=f1_calldescr)
+        i11 = call(ConstClass(f2ptr), i10, i1, descr=f2_calldescr)        
+        finish(i11, i1,  i2, i3, i4, i5, i6, i7, i8, i9)
+        '''
+        loop = self.interpret(ops, [4, 7, 9, 9 ,9, 9, 9, 9, 9, 9, 9])
+        assert self.getints(11) == [5*7, 7, 9, 9, 9, 9, 9, 9, 9, 9, 9]
+
+    def test_call_many_arguments(self):
+        ops = '''
+        [i0, i1, i2, i3, i4, i5, i6, i7]
+        i8 = call(ConstClass(f10ptr), 1, i0, i1, i2, i3, i4, i5, i6, i7, 10, descr=f10_calldescr)
+        finish(i8)
+        '''
+        loop = self.interpret(ops, [2, 3, 4, 5, 6, 7, 8, 9])
+        assert self.getint(0) == 55
+
+    def test_bridge_calls_1(self):
+        ops = '''
+        [i0, i1]
+        i2 = call(ConstClass(f1ptr), i0, descr=f1_calldescr)
+        guard_value(i2, 0, descr=fdescr1) [i2, i1]
+        finish(i1)
+        '''
+        loop = self.interpret(ops, [4, 7])
+        assert self.getint(0) == 5
+        ops = '''
+        [i2, i1]
+        i3 = call(ConstClass(f2ptr), i2, i1, descr=f2_calldescr)        
+        finish(i3, descr=fdescr2)        
+        '''
+        bridge = self.attach_bridge(ops, loop, -2)
+
+
+        self.cpu.set_future_value_int(0, 4)
+        self.cpu.set_future_value_int(1, 7)        
+        self.run(loop)
+        assert self.getint(0) == 5*7
+
+    def test_bridge_calls_2(self):
+        ops = '''
+        [i0, i1]
+        i2 = call(ConstClass(f2ptr), i0, i1, descr=f2_calldescr)
+        guard_value(i2, 0, descr=fdescr1) [i2]
+        finish(i1)
+        '''
+        loop = self.interpret(ops, [4, 7])
+        assert self.getint(0) == 4*7
+        ops = '''
+        [i2]
+        i3 = call(ConstClass(f1ptr), i2, descr=f1_calldescr)        
+        finish(i3, descr=fdescr2)        
+        '''
+        bridge = self.attach_bridge(ops, loop, -2)
+
+
+        self.cpu.set_future_value_int(0, 4)
+        self.cpu.set_future_value_int(1, 7)        
+        self.run(loop)
+        assert self.getint(0) == 29
+

diff --git a/pypy/jit/backend/arm/test/test_list.py b/pypy/jit/backend/arm/test/test_list.py
new file mode 100644
--- /dev/null
+++ b/pypy/jit/backend/arm/test/test_list.py
@@ -0,0 +1,8 @@
+
+from pypy.jit.metainterp.test.test_list import ListTests
+from pypy.jit.backend.arm.test.support import JitARMMixin
+
+class TestList(JitARMMixin, ListTests):
+    # for individual tests see
+    # ====> ../../../metainterp/test/test_list.py
+    pass

diff --git a/pypy/jit/backend/arm/test/test_recursive.py b/pypy/jit/backend/arm/test/test_recursive.py
new file mode 100644
--- /dev/null
+++ b/pypy/jit/backend/arm/test/test_recursive.py
@@ -0,0 +1,8 @@
+
+from pypy.jit.metainterp.test.test_recursive import RecursiveTests
+from pypy.jit.backend.arm.test.support import JitARMMixin
+
+class TestRecursive(JitARMMixin, RecursiveTests):
+    # for the individual tests see
+    # ====> ../../../metainterp/test/test_recursive.py
+    pass

diff --git a/pypy/jit/backend/arm/test/test_string.py b/pypy/jit/backend/arm/test/test_string.py
new file mode 100644
--- /dev/null
+++ b/pypy/jit/backend/arm/test/test_string.py
@@ -0,0 +1,13 @@
+import py
+from pypy.jit.metainterp.test import test_string
+from pypy.jit.backend.arm.test.support import JitARMMixin
+
+class TestString(JitARMMixin, test_string.TestLLtype):
+    # for the individual tests see
+    # ====> ../../../metainterp/test/test_string.py
+    pass
+
+class TestUnicode(JitARMMixin, test_string.TestLLtypeUnicode):
+    # for the individual tests see
+    # ====> ../../../metainterp/test/test_string.py
+    pass

From commits-noreply at bitbucket.org  Wed Feb  2 13:50:42 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Wed,  2 Feb 2011 13:50:42 +0100 (CET)
Subject: [pypy-svn] pypy arm-backend-2: merge default
Message-ID: <20110202125042.042602A2027@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backend-2
Changeset: r41559:903857f83c41
Date: 2011-02-02 13:48 +0100
http://bitbucket.org/pypy/pypy/changeset/903857f83c41/

Log:	merge default

diff --git a/pypy/module/__builtin__/app_file_stub.py b/pypy/module/__builtin__/app_file_stub.py
deleted file mode 100644
--- a/pypy/module/__builtin__/app_file_stub.py
+++ /dev/null
@@ -1,20 +0,0 @@
-# NOT_RPYTHON
-
-class file(object): 
-    """file(name[, mode[, buffering]]) -> file object
-
-Open a file.  The mode can be 'r', 'w' or 'a' for reading (default),
-writing or appending.  The file will be created if it doesn't exist
-when opened for writing or appending; it will be truncated when
-opened for writing.  Add a 'b' to the mode for binary files.
-Add a '+' to the mode to allow simultaneous reading and writing.
-If the buffering argument is given, 0 means unbuffered, 1 means line
-buffered, and larger numbers specify the buffer size.
-Add a 'U' to mode to open the file for input with universal newline
-support.  Any line ending in the input file will be seen as a '\n'
-in Python.  Also, a file so opened gains the attribute 'newlines';
-the value for this attribute is one of None (no newline read yet),
-'\r', '\n', '\r\n' or a tuple containing all the newline types seen.
-
-Note:  open() is an alias for file().
-"""

diff --git a/pypy/module/imp/importing.py b/pypy/module/imp/importing.py
--- a/pypy/module/imp/importing.py
+++ b/pypy/module/imp/importing.py
@@ -112,7 +112,7 @@
                w_locals=None, w_fromlist=None, level=-1):
     modulename = name
     space.timer.start_name("importhook", modulename)
-    if not modulename and level < 0: 
+    if not modulename and level < 0:
         raise OperationError(
             space.w_ValueError,
             space.wrap("Empty module name"))
@@ -305,7 +305,14 @@
             else:
                 break
         if w_importer is None:
-            w_importer = space.wrap(W_NullImporter(space))
+            try:
+                w_importer = space.call_function(
+                    space.gettypefor(W_NullImporter), w_pathitem
+                )
+            except OperationError, e:
+                if e.match(space, space.w_ImportError):
+                    return
+                raise
         if space.is_true(w_importer):
             space.setitem(w_path_importer_cache, w_pathitem, w_importer)
     if space.is_true(w_importer):
@@ -822,7 +829,7 @@
 
 def read_compiled_module(space, cpathname, strbuf):
     """ Read a code object from a file and check it for validity """
-    
+
     w_marshal = space.getbuiltinmodule('marshal')
     w_code = space.call_method(w_marshal, 'loads', space.wrap(strbuf))
     pycode = space.interpclass_w(w_code)
@@ -909,4 +916,3 @@
             os.unlink(cpathname)
         except OSError:
             pass
-

diff --git a/pypy/objspace/std/test/test_dictmultiobject.py b/pypy/objspace/std/test/test_dictmultiobject.py
--- a/pypy/objspace/std/test/test_dictmultiobject.py
+++ b/pypy/objspace/std/test/test_dictmultiobject.py
@@ -501,6 +501,27 @@
         iterable = {}
         raises(TypeError, len, iter(iterable))
 
+    def test_missing(self):
+        class X(dict):
+            def __missing__(self, x):
+                assert x == 'hi'
+                return 42
+        assert X()['hi'] == 42
+
+    def test_missing_more(self):
+        def missing(self, x):
+            assert x == 'hi'
+            return 42
+        class SpecialDescr(object):
+            def __init__(self, impl):
+                self.impl = impl
+            def __get__(self, obj, owner):
+                return self.impl.__get__(obj, owner)
+        class X(dict):
+            __missing__ = SpecialDescr(missing)
+        assert X()['hi'] == 42
+
+
 class AppTest_DictMultiObject(AppTest_DictObject):
 
     def test_emptydict_unhashable(self):

diff --git a/pypy/translator/c/src/signals.h b/pypy/translator/c/src/signals.h
--- a/pypy/translator/c/src/signals.h
+++ b/pypy/translator/c/src/signals.h
@@ -119,7 +119,7 @@
 
     if (wakeup_fd != -1) 
       {
-        write(wakeup_fd, "\0", 1);
+        ssize_t res = write(wakeup_fd, "\0", 1);
         /* the return value is ignored here */
       }
 }

diff --git a/pypy/objspace/std/dictmultiobject.py b/pypy/objspace/std/dictmultiobject.py
--- a/pypy/objspace/std/dictmultiobject.py
+++ b/pypy/objspace/std/dictmultiobject.py
@@ -90,7 +90,7 @@
             w_missing = space.lookup(w_dict, "__missing__")
             if w_missing is None:
                 return None
-            return space.call_function(w_missing, w_dict, w_key)
+            return space.get_and_call_function(w_missing, w_dict, w_key)
         else:
             return None
 

diff --git a/pypy/module/pypyjit/policy.py b/pypy/module/pypyjit/policy.py
--- a/pypy/module/pypyjit/policy.py
+++ b/pypy/module/pypyjit/policy.py
@@ -13,14 +13,15 @@
         if '.' in modname:
             modname, _ = modname.split('.', 1)
         if modname in ['pypyjit', 'signal', 'micronumpy', 'math', 'exceptions',
-                       'imp', 'sys', 'array', '_ffi', 'itertools', 'operator']:
+                       'imp', 'sys', 'array', '_ffi', 'itertools', 'operator',
+                       '_socket']:
             return True
         return False
 
     def look_inside_function(self, func):
         mod = func.__module__ or '?'
 
-        if mod == 'pypy.rlib.rbigint' or mod == 'pypy.rlib.rlocale':
+        if mod == 'pypy.rlib.rbigint' or mod == 'pypy.rlib.rlocale' or mod == 'pypy.rlib.rsocket':
             return False
         if '_geninterp_' in func.func_globals: # skip all geninterped stuff
             return False

diff --git a/pypy/config/pypyoption.py b/pypy/config/pypyoption.py
--- a/pypy/config/pypyoption.py
+++ b/pypy/config/pypyoption.py
@@ -92,6 +92,7 @@
     "bz2"       : ["pypy.module.bz2.interp_bz2"],
     "pyexpat"   : ["pypy.module.pyexpat.interp_pyexpat"],
     "_ssl"      : ["pypy.module._ssl.interp_ssl"],
+    "_hashlib"  : ["pypy.module._ssl.interp_ssl"],
     "_minimal_curses": ["pypy.module._minimal_curses.fficurses"],
     }
 
@@ -165,7 +166,7 @@
                suggests=[("objspace.allworkingmodules", False)]),
 
     BoolOption("geninterp", "specify whether geninterp should be used",
-               default=True),
+               default=False),
 
     BoolOption("logbytecodes",
                "keep track of bytecode usage",

From commits-noreply at bitbucket.org  Wed Feb  2 14:10:33 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Wed,  2 Feb 2011 14:10:33 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix translation.
Message-ID: <20110202131033.883A22A2006@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41560:ee06cacb0be6
Date: 2011-02-02 08:10 -0500
http://bitbucket.org/pypy/pypy/changeset/ee06cacb0be6/

Log:	Fix translation.

diff --git a/pypy/module/_io/interp_textio.py b/pypy/module/_io/interp_textio.py
--- a/pypy/module/_io/interp_textio.py
+++ b/pypy/module/_io/interp_textio.py
@@ -253,7 +253,7 @@
         # into the decoder with need_eof as the EOF flag, then skip
         # chars_to_skip characters of the decoded result.  For most simple
         # decoders, tell() will often just give a byte offset in the file.
-        rb = rbigint.fromint
+        rb = rbigint.fromrarith_int
 
         res = rb(self.start_pos)
         res = res.or_(rb(self.dec_flags).lshift(1 * r_ulonglong.BITS))

From commits-noreply at bitbucket.org  Wed Feb  2 16:00:51 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Wed,  2 Feb 2011 16:00:51 +0100 (CET)
Subject: [pypy-svn] pypy default: Added a test that crashes on pypy-c.
Message-ID: <20110202150051.4EAFC2A2008@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41561:0a31526bc07b
Date: 2011-02-02 10:00 -0500
http://bitbucket.org/pypy/pypy/changeset/0a31526bc07b/

Log:	Added a test that crashes on pypy-c.

diff --git a/pypy/module/_io/test/test_textio.py b/pypy/module/_io/test/test_textio.py
--- a/pypy/module/_io/test/test_textio.py
+++ b/pypy/module/_io/test/test_textio.py
@@ -166,6 +166,21 @@
                         assert got_line == exp_line
                     assert len(got_lines) == len(exp_lines)
 
+    def test_readline(self):
+        import _io
+
+        s = "AAA\r\nBBB\rCCC\r\nDDD\nEEE\r\n"
+        r = "AAA\nBBB\nCCC\nDDD\nEEE\n".decode("ascii")
+        txt = _io.TextIOWrapper(_io.BytesIO(s), encoding="ascii")
+        txt._CHUNK_SIZE = 4
+
+        reads = txt.read(4)
+        reads += txt.read(4)
+        reads += txt.readline()
+        reads += txt.readline()
+        reads += txt.readline()
+        assert reads == r
+
     def test_name(self):
         import _io
 

From commits-noreply at bitbucket.org  Wed Feb  2 16:01:28 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Wed,  2 Feb 2011 16:01:28 +0100 (CET)
Subject: [pypy-svn] pypy default: Rename this test to somethign which makes
	sense.o
Message-ID: <20110202150128.607AA2A200D@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41562:be3d50435359
Date: 2011-02-02 10:01 -0500
http://bitbucket.org/pypy/pypy/changeset/be3d50435359/

Log:	Rename this test to somethign which makes sense.o

diff --git a/pypy/module/_io/test/test_io.py b/pypy/module/_io/test/test_io.py
--- a/pypy/module/_io/test/test_io.py
+++ b/pypy/module/_io/test/test_io.py
@@ -261,7 +261,7 @@
             assert f._CHUNK_SIZE == 4096
             raises(ValueError, setattr, f, "_CHUNK_SIZE", 0)
 
-    def test_multi_line(self):
+    def test_truncate(self):
         import _io
 
         with _io.open(self.tmpfile, "w+") as f:

From commits-noreply at bitbucket.org  Wed Feb  2 16:33:32 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed,  2 Feb 2011 16:33:32 +0100 (CET)
Subject: [pypy-svn] pypy default: Free memory allocated to shadow stacks
	after a fork().
Message-ID: <20110202153332.D8BFF2A2010@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41563:d2d23acf30a9
Date: 2011-02-01 15:00 +0100
http://bitbucket.org/pypy/pypy/changeset/d2d23acf30a9/

Log:	Free memory allocated to shadow stacks after a fork(). (not tested
	so far...)

diff --git a/pypy/module/thread/ll_thread.py b/pypy/module/thread/ll_thread.py
--- a/pypy/module/thread/ll_thread.py
+++ b/pypy/module/thread/ll_thread.py
@@ -1,6 +1,6 @@
 
 from pypy.rpython.lltypesystem import rffi
-from pypy.rpython.lltypesystem import lltype
+from pypy.rpython.lltypesystem import lltype, llmemory
 from pypy.rpython.tool import rffi_platform as platform
 from pypy.translator.tool.cbuild import ExternalCompilationInfo
 import py, os
@@ -152,7 +152,7 @@
 # ____________________________________________________________
 #
 # Thread integration.
-# These are three completely ad-hoc operations at the moment.
+# These are five completely ad-hoc operations at the moment.
 
 def gc_thread_prepare():
     """To call just before thread.start_new_thread().  This
@@ -179,3 +179,20 @@
     if we_are_translated():
         llop.gc_thread_die(lltype.Void)
 gc_thread_die._always_inline_ = True
+
+def gc_thread_before_fork():
+    """To call just before fork().  Prepares for forking, after
+    which only the current thread will be alive.
+    """
+    if we_are_translated():
+        return llop.gc_thread_before_fork(llmemory.Address)
+    else:
+        return llmemory.NULL
+
+def gc_thread_after_fork(result_of_fork, opaqueaddr):
+    """To call just after fork().
+    """
+    if we_are_translated():
+        llop.gc_thread_after_fork(lltype.Void, result_of_fork, opaqueaddr)
+    else:
+        assert opaqueaddr == llmemory.NULL

diff --git a/pypy/rpython/lltypesystem/lloperation.py b/pypy/rpython/lltypesystem/lloperation.py
--- a/pypy/rpython/lltypesystem/lloperation.py
+++ b/pypy/rpython/lltypesystem/lloperation.py
@@ -466,6 +466,8 @@
                                  # allocating non-GC structures only
     'gc_thread_run'       : LLOp(),
     'gc_thread_die'       : LLOp(),
+    'gc_thread_before_fork':LLOp(),   # returns an opaque address
+    'gc_thread_after_fork': LLOp(),   # arguments: (result_of_fork, opaqueaddr)
     'gc_assume_young_pointers': LLOp(canrun=True),
     'gc_writebarrier_before_copy': LLOp(canrun=True),
     'gc_heap_stats'       : LLOp(canunwindgc=True),

diff --git a/pypy/rpython/module/ll_os.py b/pypy/rpython/module/ll_os.py
--- a/pypy/rpython/module/ll_os.py
+++ b/pypy/rpython/module/ll_os.py
@@ -1531,14 +1531,17 @@
 
     @registering_if(os, 'fork')
     def register_os_fork(self):
+        from pypy.module.thread import ll_thread
         eci = self.gcc_profiling_bug_workaround('pid_t _noprof_fork(void)',
                                                 'return fork();')
         os_fork = self.llexternal('_noprof_fork', [], rffi.PID_T,
                                   compilation_info = eci,
-                                  threadsafe = False)
+                                  _nowrapper = True)
 
         def fork_llimpl():
+            opaqueaddr = ll_thread.gc_thread_before_fork()
             childpid = rffi.cast(lltype.Signed, os_fork())
+            ll_thread.gc_thread_after_fork(childpid, opaqueaddr)
             if childpid == -1:
                 raise OSError(rposix.get_errno(), "os_fork failed")
             return rffi.cast(lltype.Signed, childpid)

diff --git a/pypy/translator/c/gc.py b/pypy/translator/c/gc.py
--- a/pypy/translator/c/gc.py
+++ b/pypy/translator/c/gc.py
@@ -92,6 +92,12 @@
     def OP_GC_THREAD_DIE(self, funcgen, op):
         return ''
 
+    def OP_GC_THREAD_BEFORE_FORK(self, funcgen, op):
+        return '%s = NULL;' % funcgen.expr(op.result)
+
+    def OP_GC_THREAD_AFTER_FORK(self, funcgen, op):
+        return ''
+
     def OP_GC_ASSUME_YOUNG_POINTERS(self, funcgen, op):
         return ''
 

diff --git a/pypy/rpython/memory/gctransform/framework.py b/pypy/rpython/memory/gctransform/framework.py
--- a/pypy/rpython/memory/gctransform/framework.py
+++ b/pypy/rpython/memory/gctransform/framework.py
@@ -960,6 +960,22 @@
         if hasattr(self.root_walker, 'thread_die_ptr'):
             hop.genop("direct_call", [self.root_walker.thread_die_ptr])
 
+    def gct_gc_thread_before_fork(self, hop):
+        assert self.translator.config.translation.thread
+        if hasattr(self.root_walker, 'thread_before_fork_ptr'):
+            hop.genop("direct_call", [self.root_walker.thread_before_fork_ptr],
+                      resultvar=hop.spaceop.result)
+        else:
+            c_null = rmodel.inputconst(llmemory.Address, llmemory.NULL)
+            hop.genop("same_as", [c_null],
+                      resultvar=hop.spaceop.result)
+
+    def gct_gc_thread_after_fork(self, hop):
+        assert self.translator.config.translation.thread
+        if hasattr(self.root_walker, 'thread_after_fork_ptr'):
+            hop.genop("direct_call", [self.root_walker.thread_after_fork_ptr]
+                                     + hop.spaceop.args)
+
     def gct_gc_get_type_info_group(self, hop):
         return hop.cast_result(self.c_type_info_group)
 
@@ -1435,7 +1451,7 @@
             gcdata.active_thread = new_aid
 
         def collect_stack(aid, stacktop, callback):
-            if stacktop != llmemory.NULL and aid != get_aid():
+            if stacktop != llmemory.NULL and aid != gcdata.active_thread:
                 # collect all valid stacks from the dict (the entry
                 # corresponding to the current thread is not valid)
                 gc = self.gc
@@ -1447,11 +1463,41 @@
                     addr += sizeofaddr
 
         def collect_more_stacks(callback):
+            ll_assert(get_aid() == gcdata.active_thread,
+                      "collect_more_stacks(): invalid active_thread")
             gcdata.thread_stacks.foreach(collect_stack, callback)
 
+        def _free_if_not_current(aid, stacktop, _):
+            if stacktop != llmemory.NULL and aid != gcdata.active_thread:
+                end = stacktop - sizeofaddr
+                base = end.address[0]
+                llmemory.raw_free(base)
+
+        def thread_after_fork(result_of_fork, opaqueaddr):
+            # we don't need a thread_before_fork in this case, so
+            # opaqueaddr == NULL.  This is called after fork().
+            if result_of_fork == 0:
+                # We are in the child process.  Assumes that only the
+                # current thread survived, so frees the shadow stacks
+                # of all the other ones.
+                gcdata.thread_stacks.foreach(_free_if_not_current, None)
+                # Clears the dict (including the current thread, which
+                # was an invalid entry anyway and will be recreated by
+                # the next call to save_away_current_stack()).
+                gcdata.thread_stacks.clear()
+                # Finally, reset the stored thread IDs, in case it
+                # changed because of fork().
+                aid = get_aid()
+                gcdata.main_thread = aid
+                gcdata.active_thread = aid
+
         self.thread_setup = thread_setup
         self.thread_prepare_ptr = getfn(thread_prepare, [], annmodel.s_None)
         self.thread_run_ptr = getfn(thread_run, [], annmodel.s_None,
                                     inline=True)
         self.thread_die_ptr = getfn(thread_die, [], annmodel.s_None)
+        self.thread_after_fork_ptr = getfn(thread_after_fork,
+                                           [annmodel.SomeInteger(),
+                                            annmodel.SomeAddress()],
+                                           annmodel.s_None)
         self.collect_stacks_from_other_threads = collect_more_stacks

From commits-noreply at bitbucket.org  Wed Feb  2 16:33:33 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed,  2 Feb 2011 16:33:33 +0100 (CET)
Subject: [pypy-svn] pypy default: Revert this part of the change.
Message-ID: <20110202153333.668E52A2010@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41564:e16ea8c81bc0
Date: 2011-02-01 15:15 +0100
http://bitbucket.org/pypy/pypy/changeset/e16ea8c81bc0/

Log:	Revert this part of the change.

diff --git a/pypy/rpython/memory/gctransform/framework.py b/pypy/rpython/memory/gctransform/framework.py
--- a/pypy/rpython/memory/gctransform/framework.py
+++ b/pypy/rpython/memory/gctransform/framework.py
@@ -1451,7 +1451,7 @@
             gcdata.active_thread = new_aid
 
         def collect_stack(aid, stacktop, callback):
-            if stacktop != llmemory.NULL and aid != gcdata.active_thread:
+            if stacktop != llmemory.NULL and aid != get_aid():
                 # collect all valid stacks from the dict (the entry
                 # corresponding to the current thread is not valid)
                 gc = self.gc
@@ -1463,8 +1463,6 @@
                     addr += sizeofaddr
 
         def collect_more_stacks(callback):
-            ll_assert(get_aid() == gcdata.active_thread,
-                      "collect_more_stacks(): invalid active_thread")
             gcdata.thread_stacks.foreach(collect_stack, callback)
 
         def _free_if_not_current(aid, stacktop, _):

From commits-noreply at bitbucket.org  Wed Feb  2 16:33:34 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed,  2 Feb 2011 16:33:34 +0100 (CET)
Subject: [pypy-svn] pypy default: Wrong,
	un-revert the previous checkin.  It was correct, and
Message-ID: <20110202153334.D44BD2A2017@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41565:a5669fad99a4
Date: 2011-02-01 15:30 +0100
http://bitbucket.org/pypy/pypy/changeset/a5669fad99a4/

Log:	Wrong, un-revert the previous checkin. It was correct, and the
	error was actually in the test, which missed calls to the correct
	ll_thread.gc_thread_xxx() operations.

diff --git a/pypy/translator/c/test/test_standalone.py b/pypy/translator/c/test/test_standalone.py
--- a/pypy/translator/c/test/test_standalone.py
+++ b/pypy/translator/c/test/test_standalone.py
@@ -832,6 +832,7 @@
             ll_thread.release_NOAUTO(state.ll_lock)
         def after():
             ll_thread.acquire_NOAUTO(state.ll_lock, True)
+            ll_thread.gc_thread_run()
 
         class Cons:
             def __init__(self, head, tail):
@@ -841,6 +842,7 @@
         def bootstrap():
             state.xlist.append(Cons(123, Cons(456, None)))
             gc.collect()
+            ll_thread.gc_thread_die()
 
         def entry_point(argv):
             os.write(1, "hello world\n")
@@ -850,13 +852,18 @@
             state.ll_lock = ll_thread.allocate_ll_lock()
             after()
             invoke_around_extcall(before, after)
+            ll_thread.gc_thread_prepare()
             ident1 = ll_thread.start_new_thread(bootstrap, ())
+            ll_thread.gc_thread_prepare()
             ident2 = ll_thread.start_new_thread(bootstrap, ())
             #
             gc.collect()
             #
+            ll_thread.gc_thread_prepare()
             ident3 = ll_thread.start_new_thread(bootstrap, ())
+            ll_thread.gc_thread_prepare()
             ident4 = ll_thread.start_new_thread(bootstrap, ())
+            ll_thread.gc_thread_prepare()
             ident5 = ll_thread.start_new_thread(bootstrap, ())
             # wait for the 5 threads to finish
             while True:

diff --git a/pypy/rpython/memory/gctransform/framework.py b/pypy/rpython/memory/gctransform/framework.py
--- a/pypy/rpython/memory/gctransform/framework.py
+++ b/pypy/rpython/memory/gctransform/framework.py
@@ -1451,7 +1451,7 @@
             gcdata.active_thread = new_aid
 
         def collect_stack(aid, stacktop, callback):
-            if stacktop != llmemory.NULL and aid != get_aid():
+            if stacktop != llmemory.NULL and aid != gcdata.active_thread:
                 # collect all valid stacks from the dict (the entry
                 # corresponding to the current thread is not valid)
                 gc = self.gc
@@ -1463,6 +1463,8 @@
                     addr += sizeofaddr
 
         def collect_more_stacks(callback):
+            ll_assert(get_aid() == gcdata.active_thread,
+                      "collect_more_stacks(): invalid active_thread")
             gcdata.thread_stacks.foreach(collect_stack, callback)
 
         def _free_if_not_current(aid, stacktop, _):

From commits-noreply at bitbucket.org  Wed Feb  2 16:33:36 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed,  2 Feb 2011 16:33:36 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix test_thread_and_gc,
	and add another test_thread_and_gc_with_fork.
Message-ID: <20110202153336.690602A2017@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41566:a63129a046bc
Date: 2011-02-01 15:56 +0100
http://bitbucket.org/pypy/pypy/changeset/a63129a046bc/

Log:	Fix test_thread_and_gc, and add another
	test_thread_and_gc_with_fork. It passes here (meaning without
	asmgcc).

diff --git a/pypy/translator/c/test/test_standalone.py b/pypy/translator/c/test/test_standalone.py
--- a/pypy/translator/c/test/test_standalone.py
+++ b/pypy/translator/c/test/test_standalone.py
@@ -844,6 +844,12 @@
             gc.collect()
             ll_thread.gc_thread_die()
 
+        def new_thread():
+            ll_thread.gc_thread_prepare()
+            ident = ll_thread.start_new_thread(bootstrap, ())
+            time.sleep(0.5)    # enough time to start, hopefully
+            return ident
+
         def entry_point(argv):
             os.write(1, "hello world\n")
             state.xlist = []
@@ -852,19 +858,14 @@
             state.ll_lock = ll_thread.allocate_ll_lock()
             after()
             invoke_around_extcall(before, after)
-            ll_thread.gc_thread_prepare()
-            ident1 = ll_thread.start_new_thread(bootstrap, ())
-            ll_thread.gc_thread_prepare()
-            ident2 = ll_thread.start_new_thread(bootstrap, ())
+            ident1 = new_thread()
+            ident2 = new_thread()
             #
             gc.collect()
             #
-            ll_thread.gc_thread_prepare()
-            ident3 = ll_thread.start_new_thread(bootstrap, ())
-            ll_thread.gc_thread_prepare()
-            ident4 = ll_thread.start_new_thread(bootstrap, ())
-            ll_thread.gc_thread_prepare()
-            ident5 = ll_thread.start_new_thread(bootstrap, ())
+            ident3 = new_thread()
+            ident4 = new_thread()
+            ident5 = new_thread()
             # wait for the 5 threads to finish
             while True:
                 gc.collect()
@@ -892,3 +893,85 @@
                                      '3 ok',
                                      '4 ok',
                                      '5 ok']
+
+
+    def test_thread_and_gc_with_fork(self):
+        # Ideally, it should test that memory allocated for the shadow
+        # stacks of the other threads is really released when doing a
+        # fork(), but it's hard.  Instead it's a "does not crash" kind
+        # of test.
+        import time, gc, os
+        from pypy.module.thread import ll_thread
+        from pypy.rlib.objectmodel import invoke_around_extcall
+        if not hasattr(os, 'fork'):
+            py.test.skip("requires fork()")
+
+        class State:
+            pass
+        state = State()
+
+        def before():
+            ll_assert(not ll_thread.acquire_NOAUTO(state.ll_lock, False),
+                      "lock not held!")
+            ll_thread.release_NOAUTO(state.ll_lock)
+        def after():
+            ll_thread.acquire_NOAUTO(state.ll_lock, True)
+            ll_thread.gc_thread_run()
+
+        class Cons:
+            def __init__(self, head, tail):
+                self.head = head
+                self.tail = tail
+
+        def bootstrap():
+            for i in range(10):
+                state.xlist.append(Cons(123, Cons(456, None)))
+                time.sleep(0.01)
+            childpid = os.fork()
+            gc.collect()        # collect both in the child and in the parent
+            if childpid == 0:
+                os.write(state.write_end, 'c')   # "I did not die!" from child
+            else:
+                os.write(state.write_end, 'p')   # "I did not die!" from parent
+            ll_thread.gc_thread_die()
+
+        def new_thread():
+            ll_thread.gc_thread_prepare()
+            ident = ll_thread.start_new_thread(bootstrap, ())
+            time.sleep(0.5)    # enough time to start, hopefully
+            return ident
+
+        def entry_point(argv):
+            os.write(1, "hello world\n")
+            state.xlist = []
+            state.read_end, state.write_end = os.pipe()
+            x2 = Cons(51, Cons(62, Cons(74, None)))
+            # start 5 new threads
+            state.ll_lock = ll_thread.allocate_ll_lock()
+            after()
+            invoke_around_extcall(before, after)
+            ident1 = new_thread()
+            ident2 = new_thread()
+            ident3 = new_thread()
+            ident4 = new_thread()
+            ident5 = new_thread()
+            # wait for 4 more seconds, which should be plenty of time
+            time.sleep(4)
+            # return everything that was written to the pipe so far,
+            # followed by the final dot.
+            os.write(state.write_end, '.')
+            result = os.read(state.read_end, 256)
+            os.write(1, "got: %s\n" % result)
+            return 0
+
+        t, cbuilder = self.compile(entry_point)
+        data = cbuilder.cmdexec('')
+        print repr(data)
+        header, footer = data.splitlines()
+        assert header == 'hello world'
+        assert footer.startswith('got: ')
+        result = footer[5:]
+        # check that all 5 threads and 5 forked processes
+        # finished successfully
+        assert (len(result) == 11 and result[10] == '.'
+                and result.count('c') == result.count('p') == 5)

From commits-noreply at bitbucket.org  Wed Feb  2 16:33:37 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed,  2 Feb 2011 16:33:37 +0100 (CET)
Subject: [pypy-svn] pypy default: Improve the test to really check that
	object 's' is	freed in
Message-ID: <20110202153337.AD1932A2017@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41567:e851faac2492
Date: 2011-02-01 16:27 +0100
http://bitbucket.org/pypy/pypy/changeset/e851faac2492/

Log:	Improve the test to really check that object 's' is freed in every
	forked subprocess, because it is only referenced from the stack of
	the main thread. (This is actually going further than CPython,
	where objects referenced from other threads just become dead garbage
	after doing a fork().)

diff --git a/pypy/translator/c/test/test_standalone.py b/pypy/translator/c/test/test_standalone.py
--- a/pypy/translator/c/test/test_standalone.py
+++ b/pypy/translator/c/test/test_standalone.py
@@ -1,6 +1,7 @@
 import py
 import sys, os, re
 
+from pypy.rlib.objectmodel import keepalive_until_here
 from pypy.rlib.rarithmetic import r_longlong
 from pypy.rlib.debug import ll_assert, have_debug_prints
 from pypy.rlib.debug import debug_print, debug_start, debug_stop
@@ -896,10 +897,10 @@
 
 
     def test_thread_and_gc_with_fork(self):
-        # Ideally, it should test that memory allocated for the shadow
-        # stacks of the other threads is really released when doing a
-        # fork(), but it's hard.  Instead it's a "does not crash" kind
-        # of test.
+        # This checks that memory allocated for the shadow stacks of the
+        # other threads is really released when doing a fork() -- or at
+        # least that the object referenced from stacks that are no longer
+        # alive are really freed.
         import time, gc, os
         from pypy.module.thread import ll_thread
         from pypy.rlib.objectmodel import invoke_around_extcall
@@ -923,12 +924,27 @@
                 self.head = head
                 self.tail = tail
 
-        def bootstrap():
+        class Stuff:
+            def __del__(self):
+                os.write(state.write_end, 'd')
+
+        def allocate_stuff():
+            s = Stuff()
+            os.write(state.write_end, 'a')
+            return s
+
+        def run_in_thread():
             for i in range(10):
                 state.xlist.append(Cons(123, Cons(456, None)))
                 time.sleep(0.01)
             childpid = os.fork()
+            return childpid
+
+        def bootstrap():
+            childpid = run_in_thread()
             gc.collect()        # collect both in the child and in the parent
+            gc.collect()
+            gc.collect()
             if childpid == 0:
                 os.write(state.write_end, 'c')   # "I did not die!" from child
             else:
@@ -941,22 +957,32 @@
             time.sleep(0.5)    # enough time to start, hopefully
             return ident
 
+        def start_all_threads():
+            s = allocate_stuff()
+            ident1 = new_thread()
+            ident2 = new_thread()
+            ident3 = new_thread()
+            ident4 = new_thread()
+            ident5 = new_thread()
+            keepalive_until_here(s)
+
         def entry_point(argv):
             os.write(1, "hello world\n")
             state.xlist = []
+            state.deleted = 0
             state.read_end, state.write_end = os.pipe()
             x2 = Cons(51, Cons(62, Cons(74, None)))
             # start 5 new threads
             state.ll_lock = ll_thread.allocate_ll_lock()
             after()
             invoke_around_extcall(before, after)
-            ident1 = new_thread()
-            ident2 = new_thread()
-            ident3 = new_thread()
-            ident4 = new_thread()
-            ident5 = new_thread()
+            start_all_threads()
             # wait for 4 more seconds, which should be plenty of time
             time.sleep(4)
+            # force freeing
+            gc.collect()
+            gc.collect()
+            gc.collect()
             # return everything that was written to the pipe so far,
             # followed by the final dot.
             os.write(state.write_end, '.')
@@ -972,6 +998,10 @@
         assert footer.startswith('got: ')
         result = footer[5:]
         # check that all 5 threads and 5 forked processes
-        # finished successfully
-        assert (len(result) == 11 and result[10] == '.'
-                and result.count('c') == result.count('p') == 5)
+        # finished successfully, that we did 1 allocation,
+        # and that it was freed 6 times -- once in the parent
+        # process and once in every child process.
+        assert (result[-1] == '.'
+                and result.count('c') == result.count('p') == 5
+                and result.count('a') == 1
+                and result.count('d') == 6)

From commits-noreply at bitbucket.org  Wed Feb  2 16:33:38 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed,  2 Feb 2011 16:33:38 +0100 (CET)
Subject: [pypy-svn] pypy default: After much thinking,
	add a condition.  Really hard to test :-(
Message-ID: <20110202153338.8DAB32A201B@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41568:dd6382d7a599
Date: 2011-02-01 16:43 +0100
http://bitbucket.org/pypy/pypy/changeset/dd6382d7a599/

Log:	After much thinking, add a condition. Really hard to test :-(

diff --git a/pypy/rpython/memory/gctransform/framework.py b/pypy/rpython/memory/gctransform/framework.py
--- a/pypy/rpython/memory/gctransform/framework.py
+++ b/pypy/rpython/memory/gctransform/framework.py
@@ -1406,6 +1406,9 @@
             occur in this thread.
             """
             aid = get_aid()
+            if aid == gcdata.main_thread:
+                return   # ignore calls to thread_die() in the main thread
+                         # (which can occur after a fork()).
             gcdata.thread_stacks.setitem(aid, llmemory.NULL)
             old = gcdata.root_stack_base
             if gcdata._fresh_rootstack == llmemory.NULL:
@@ -1486,7 +1489,9 @@
                 # the next call to save_away_current_stack()).
                 gcdata.thread_stacks.clear()
                 # Finally, reset the stored thread IDs, in case it
-                # changed because of fork().
+                # changed because of fork().  Also change the main
+                # thread to the current one (because there is not any
+                # other left).
                 aid = get_aid()
                 gcdata.main_thread = aid
                 gcdata.active_thread = aid

From commits-noreply at bitbucket.org  Wed Feb  2 16:33:40 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed,  2 Feb 2011 16:33:40 +0100 (CET)
Subject: [pypy-svn] pypy default: Sleep with a reference to 's' still around.
Message-ID: <20110202153340.E06162A2024@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41569:aaa22145926c
Date: 2011-02-01 19:03 +0100
http://bitbucket.org/pypy/pypy/changeset/aaa22145926c/

Log:	Sleep with a reference to 's' still around.

diff --git a/pypy/translator/c/test/test_standalone.py b/pypy/translator/c/test/test_standalone.py
--- a/pypy/translator/c/test/test_standalone.py
+++ b/pypy/translator/c/test/test_standalone.py
@@ -964,6 +964,8 @@
             ident3 = new_thread()
             ident4 = new_thread()
             ident5 = new_thread()
+            # wait for 4 more seconds, which should be plenty of time
+            time.sleep(4)
             keepalive_until_here(s)
 
         def entry_point(argv):
@@ -977,8 +979,6 @@
             after()
             invoke_around_extcall(before, after)
             start_all_threads()
-            # wait for 4 more seconds, which should be plenty of time
-            time.sleep(4)
             # force freeing
             gc.collect()
             gc.collect()

From commits-noreply at bitbucket.org  Wed Feb  2 16:33:42 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed,  2 Feb 2011 16:33:42 +0100 (CET)
Subject: [pypy-svn] pypy default: Add call_initial_function(),
	similar to call_final_function().
Message-ID: <20110202153342.6D20B2A2021@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41570:e512a5378402
Date: 2011-02-02 14:51 +0100
http://bitbucket.org/pypy/pypy/changeset/e512a5378402/

Log:	Add call_initial_function(), similar to call_final_function().

diff --git a/pypy/translator/unsimplify.py b/pypy/translator/unsimplify.py
--- a/pypy/translator/unsimplify.py
+++ b/pypy/translator/unsimplify.py
@@ -155,6 +155,31 @@
     if links_to_start_block:
         insert_empty_startblock(None, graph)
 
+def call_initial_function(translator, initial_func, annhelper=None):
+    """Before the program starts, call 'initial_func()'."""
+    from pypy.annotation import model as annmodel
+    from pypy.rpython.lltypesystem import lltype
+    from pypy.rpython.annlowlevel import MixLevelHelperAnnotator
+
+    own_annhelper = (annhelper is None)
+    if own_annhelper:
+        annhelper = MixLevelHelperAnnotator(translator.rtyper)
+    c_initial_func = annhelper.constfunc(initial_func, [], annmodel.s_None)
+    if own_annhelper:
+        annhelper.finish()
+
+    entry_point = translator.entry_point_graph
+    args = [copyvar(translator.annotator, v) for v in entry_point.getargs()]
+    extrablock = Block(args)
+    v_none = varoftype(lltype.Void)
+    newop = SpaceOperation('direct_call', [c_initial_func], v_none)
+    extrablock.operations = [newop]
+    extrablock.closeblock(Link(args, entry_point.startblock))
+    entry_point.startblock.isstartblock = False
+    entry_point.startblock = extrablock
+    entry_point.startblock.isstartblock = True
+    checkgraph(entry_point)
+
 def call_final_function(translator, final_func, annhelper=None):
     """When the program finishes normally, call 'final_func()'."""
     from pypy.annotation import model as annmodel

diff --git a/pypy/translator/test/test_unsimplify.py b/pypy/translator/test/test_unsimplify.py
--- a/pypy/translator/test/test_unsimplify.py
+++ b/pypy/translator/test/test_unsimplify.py
@@ -1,6 +1,7 @@
 import os
 from pypy.translator.translator import TranslationContext, graphof
 from pypy.translator.unsimplify import split_block, call_final_function
+from pypy.translator.unsimplify import call_initial_function
 from pypy.rpython.llinterp import LLInterpreter
 from pypy.objspace.flow.model import checkgraph
 from pypy.rlib.objectmodel import we_are_translated
@@ -69,7 +70,26 @@
         assert result == 1
         result = interp.eval_graph(graph, [2])
         assert result == 2
-    
+
+def test_call_initial_function():
+    tmpfile = str(udir.join('test_call_initial_function'))
+    for type_system in ['lltype', 'ootype']:
+        def f(x):
+            return x * 6
+        def hello_world():
+            if we_are_translated():
+                fd = os.open(tmpfile, os.O_WRONLY | os.O_CREAT, 0)
+                os.close(fd)
+        graph, t = translate(f, [int], type_system)
+        call_initial_function(t, hello_world)
+        #
+        if os.path.exists(tmpfile):
+            os.unlink(tmpfile)
+        interp = LLInterpreter(t.rtyper)
+        result = interp.eval_graph(graph, [7])
+        assert result == 42
+        assert os.path.isfile(tmpfile)
+
 def test_call_final_function():
     tmpfile = str(udir.join('test_call_final_function'))
     for type_system in ['lltype', 'ootype']:

From commits-noreply at bitbucket.org  Wed Feb  2 16:33:48 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed,  2 Feb 2011 16:33:48 +0100 (CET)
Subject: [pypy-svn] pypy default: Port the thread+fork() logic to asmgcc,
	painfully.
Message-ID: <20110202153348.0525F2A202B@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41571:78912da6a09a
Date: 2011-02-02 15:31 +0100
http://bitbucket.org/pypy/pypy/changeset/78912da6a09a/

Log:	Port the thread+fork() logic to asmgcc, painfully.

diff --git a/pypy/rpython/llinterp.py b/pypy/rpython/llinterp.py
--- a/pypy/rpython/llinterp.py
+++ b/pypy/rpython/llinterp.py
@@ -824,6 +824,9 @@
     def op_gc_thread_run(self):
         self.heap.thread_run()
 
+    def op_gc_thread_start(self):
+        self.heap.thread_start()
+
     def op_gc_thread_die(self):
         self.heap.thread_die()
 

diff --git a/pypy/rpython/memory/gctransform/asmgcroot.py b/pypy/rpython/memory/gctransform/asmgcroot.py
--- a/pypy/rpython/memory/gctransform/asmgcroot.py
+++ b/pypy/rpython/memory/gctransform/asmgcroot.py
@@ -149,9 +149,110 @@
 
     def need_thread_support(self, gctransformer, getfn):
         # Threads supported "out of the box" by the rest of the code.
-        # In particular, we can ignore the gc_thread_prepare,
-        # gc_thread_run and gc_thread_die operations.
-        pass
+        # The whole code in this function is only there to support
+        # fork()ing in a multithreaded process :-(
+        # For this, we need to handle gc_thread_start and gc_thread_die
+        # to record the mapping {thread_id: stack_start}, and
+        # gc_thread_before_fork and gc_thread_after_fork to get rid of
+        # all ASM_FRAMEDATA structures that do no belong to the current
+        # thread after a fork().
+        from pypy.module.thread import ll_thread
+        from pypy.rpython.memory.support import AddressDict
+        from pypy.rpython.memory.support import copy_without_null_values
+        from pypy.annotation import model as annmodel
+        gcdata = self.gcdata
+
+        def get_aid():
+            """Return the thread identifier, cast to an (opaque) address."""
+            return llmemory.cast_int_to_adr(ll_thread.get_ident())
+
+        def thread_start():
+            value = llop.stack_current(llmemory.Address)
+            gcdata.aid2stack.setitem(get_aid(), value)
+        thread_start._always_inline_ = True
+
+        def thread_setup():
+            gcdata.aid2stack = AddressDict()
+            gcdata.dead_threads_count = 0
+            # to also register the main thread's stack
+            thread_start()
+        thread_setup._always_inline_ = True
+
+        def thread_die():
+            gcdata.aid2stack.setitem(get_aid(), llmemory.NULL)
+            # from time to time, rehash the dictionary to remove
+            # old NULL entries
+            gcdata.dead_threads_count += 1
+            if (gcdata.dead_threads_count & 511) == 0:
+                gcdata.aid2stack = copy_without_null_values(gcdata.aid2stack)
+
+        def belongs_to_current_thread(framedata):
+            # xxx obscure: the answer is Yes if, as a pointer, framedata
+            # lies between the start of the current stack and the top of it.
+            stack_start = gcdata.aid2stack.get(get_aid(), llmemory.NULL)
+            ll_assert(stack_start != llmemory.NULL,
+                      "current thread not found in gcdata.aid2stack!")
+            stack_stop  = llop.stack_current(llmemory.Address)
+            return (stack_start <= framedata <= stack_stop or
+                    stack_start >= framedata >= stack_stop)
+
+        def thread_before_fork():
+            # before fork(): collect all ASM_FRAMEDATA structures that do
+            # not belong to the current thread, and move them out of the
+            # way, i.e. out of the main circular doubly linked list.
+            detached_pieces = llmemory.NULL
+            anchor = llmemory.cast_ptr_to_adr(gcrootanchor)
+            initialframedata = anchor.address[1]
+            while initialframedata != anchor:   # while we have not looped back
+                if not belongs_to_current_thread(initialframedata):
+                    # Unlink it
+                    prev = initialframedata.address[0]
+                    next = initialframedata.address[1]
+                    prev.address[1] = next
+                    next.address[0] = prev
+                    # Link it to the singly linked list 'detached_pieces'
+                    initialframedata.address[0] = detached_pieces
+                    detached_pieces = initialframedata
+                    rffi.stackcounter.stacks_counter -= 1
+                # Then proceed to the next piece of stack
+                initialframedata = initialframedata.address[1]
+            return detached_pieces
+
+        def thread_after_fork(result_of_fork, detached_pieces):
+            if result_of_fork == 0:
+                # We are in the child process.  Assumes that only the
+                # current thread survived.  All the detached_pieces
+                # are pointers in other stacks, so have likely been
+                # freed already by the multithreaded library.
+                # Nothing more for us to do.
+                pass
+            else:
+                # We are still in the parent process.  The fork() may
+                # have succeeded or not, but that's irrelevant here.
+                # We need to reattach the detached_pieces now, to the
+                # circular doubly linked list at 'gcrootanchor'.  The
+                # order is not important.
+                anchor = llmemory.cast_ptr_to_adr(gcrootanchor)
+                while detached_pieces != llmemory.NULL:
+                    reattach = detached_pieces
+                    detached_pieces = detached_pieces.address[0]
+                    a_next = anchor.address[1]
+                    reattach.address[0] = anchor
+                    reattach.address[1] = a_next
+                    anchor.address[1] = reattach
+                    a_next.address[0] = reattach
+                    rffi.stackcounter.stacks_counter += 1
+
+        self.thread_setup = thread_setup
+        self.thread_start_ptr = getfn(thread_start, [], annmodel.s_None,
+                                      inline=True)
+        self.thread_die_ptr = getfn(thread_die, [], annmodel.s_None)
+        self.thread_before_fork_ptr = getfn(thread_before_fork, [],
+                                            annmodel.SomeAddress())
+        self.thread_after_fork_ptr = getfn(thread_after_fork,
+                                           [annmodel.SomeInteger(),
+                                            annmodel.SomeAddress()],
+                                           annmodel.s_None)
 
     def walk_stack_roots(self, collect_stack_root):
         gcdata = self.gcdata

diff --git a/pypy/module/thread/ll_thread.py b/pypy/module/thread/ll_thread.py
--- a/pypy/module/thread/ll_thread.py
+++ b/pypy/module/thread/ll_thread.py
@@ -152,7 +152,7 @@
 # ____________________________________________________________
 #
 # Thread integration.
-# These are five completely ad-hoc operations at the moment.
+# These are six completely ad-hoc operations at the moment.
 
 def gc_thread_prepare():
     """To call just before thread.start_new_thread().  This
@@ -171,6 +171,12 @@
         llop.gc_thread_run(lltype.Void)
 gc_thread_run._always_inline_ = True
 
+def gc_thread_start():
+    """To call at the beginning of a new thread.
+    """
+    if we_are_translated():
+        llop.gc_thread_start(lltype.Void)
+
 def gc_thread_die():
     """To call just before the final GIL release done by a dying
     thread.  After a thread_die(), no more gc operation should

diff --git a/pypy/translator/c/test/test_standalone.py b/pypy/translator/c/test/test_standalone.py
--- a/pypy/translator/c/test/test_standalone.py
+++ b/pypy/translator/c/test/test_standalone.py
@@ -841,6 +841,7 @@
                 self.tail = tail
 
         def bootstrap():
+            ll_thread.gc_thread_start()
             state.xlist.append(Cons(123, Cons(456, None)))
             gc.collect()
             ll_thread.gc_thread_die()
@@ -941,6 +942,7 @@
             return childpid
 
         def bootstrap():
+            ll_thread.gc_thread_start()
             childpid = run_in_thread()
             gc.collect()        # collect both in the child and in the parent
             gc.collect()

diff --git a/pypy/rpython/memory/gctransform/framework.py b/pypy/rpython/memory/gctransform/framework.py
--- a/pypy/rpython/memory/gctransform/framework.py
+++ b/pypy/rpython/memory/gctransform/framework.py
@@ -955,6 +955,11 @@
         if hasattr(self.root_walker, 'thread_run_ptr'):
             hop.genop("direct_call", [self.root_walker.thread_run_ptr])
 
+    def gct_gc_thread_start(self, hop):
+        assert self.translator.config.translation.thread
+        if hasattr(self.root_walker, 'thread_start_ptr'):
+            hop.genop("direct_call", [self.root_walker.thread_start_ptr])
+
     def gct_gc_thread_die(self, hop):
         assert self.translator.config.translation.thread
         if hasattr(self.root_walker, 'thread_die_ptr'):
@@ -1258,6 +1263,7 @@
 
 class BaseRootWalker(object):
     need_root_stack = False
+    thread_setup = None
 
     def __init__(self, gctransformer):
         self.gcdata = gctransformer.gcdata
@@ -1267,7 +1273,8 @@
         return True
 
     def setup_root_walker(self):
-        pass
+        if self.thread_setup is not None:
+            self.thread_setup()
 
     def walk_roots(self, collect_stack_root,
                    collect_static_in_prebuilt_nongc,
@@ -1300,7 +1307,6 @@
 
 class ShadowStackRootWalker(BaseRootWalker):
     need_root_stack = True
-    thread_setup = None
     collect_stacks_from_other_threads = None
 
     def __init__(self, gctransformer):
@@ -1340,8 +1346,7 @@
         ll_assert(bool(stackbase), "could not allocate root stack")
         self.gcdata.root_stack_top  = stackbase
         self.gcdata.root_stack_base = stackbase
-        if self.thread_setup is not None:
-            self.thread_setup()
+        BaseRootWalker.setup_root_walker(self)
 
     def walk_stack_roots(self, collect_stack_root):
         gcdata = self.gcdata
@@ -1500,7 +1505,9 @@
         self.thread_prepare_ptr = getfn(thread_prepare, [], annmodel.s_None)
         self.thread_run_ptr = getfn(thread_run, [], annmodel.s_None,
                                     inline=True)
+        # no thread_start_ptr here
         self.thread_die_ptr = getfn(thread_die, [], annmodel.s_None)
+        # no thread_before_fork_ptr here
         self.thread_after_fork_ptr = getfn(thread_after_fork,
                                            [annmodel.SomeInteger(),
                                             annmodel.SomeAddress()],

diff --git a/pypy/translator/c/src/mem.h b/pypy/translator/c/src/mem.h
--- a/pypy/translator/c/src/mem.h
+++ b/pypy/translator/c/src/mem.h
@@ -84,6 +84,9 @@
 #endif
 
 
+/* used by pypy.rlib.rstack, but also by asmgcc */
+#define OP_STACK_CURRENT(r)  r = (long)&r
+
 
 #define RAW_MALLOC_ZERO_FILLED 0
 

diff --git a/pypy/translator/c/src/stack.h b/pypy/translator/c/src/stack.h
--- a/pypy/translator/c/src/stack.h
+++ b/pypy/translator/c/src/stack.h
@@ -17,7 +17,6 @@
 char LL_stack_too_big_slowpath(long);    /* returns 0 (ok) or 1 (too big) */
 
 /* some macros referenced from pypy.rlib.rstack */
-#define OP_STACK_CURRENT(r)  r = (long)&r
 #define LL_stack_get_start() ((long)_LLstacktoobig_stack_start)
 #define LL_stack_get_length() MAX_STACK_SIZE
 #define LL_stack_get_start_adr() ((long)&_LLstacktoobig_stack_start)  /* JIT */

diff --git a/pypy/rpython/lltypesystem/lloperation.py b/pypy/rpython/lltypesystem/lloperation.py
--- a/pypy/rpython/lltypesystem/lloperation.py
+++ b/pypy/rpython/lltypesystem/lloperation.py
@@ -465,6 +465,7 @@
                                  # ^^^ but canunwindgc=False, as it is
                                  # allocating non-GC structures only
     'gc_thread_run'       : LLOp(),
+    'gc_thread_start'     : LLOp(),
     'gc_thread_die'       : LLOp(),
     'gc_thread_before_fork':LLOp(),   # returns an opaque address
     'gc_thread_after_fork': LLOp(),   # arguments: (result_of_fork, opaqueaddr)

diff --git a/pypy/rpython/lltypesystem/llheap.py b/pypy/rpython/lltypesystem/llheap.py
--- a/pypy/rpython/lltypesystem/llheap.py
+++ b/pypy/rpython/lltypesystem/llheap.py
@@ -31,5 +31,8 @@
 def thread_run():
     pass
 
+def thread_start():
+    pass
+
 def thread_die():
     pass

diff --git a/pypy/module/thread/os_thread.py b/pypy/module/thread/os_thread.py
--- a/pypy/module/thread/os_thread.py
+++ b/pypy/module/thread/os_thread.py
@@ -83,6 +83,7 @@
         # Note that when this runs, we already hold the GIL.  This is ensured
         # by rffi's callback mecanism: we are a callback for the
         # c_thread_start() external function.
+        thread.gc_thread_start()
         space = bootstrapper.space
         w_callable = bootstrapper.w_callable
         args = bootstrapper.args

From commits-noreply at bitbucket.org  Wed Feb  2 16:33:48 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed,  2 Feb 2011 16:33:48 +0100 (CET)
Subject: [pypy-svn] pypy default: merge heads
Message-ID: <20110202153348.ABFDA2A202B@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41572:7fcf95245cd0
Date: 2011-02-02 16:31 +0100
http://bitbucket.org/pypy/pypy/changeset/7fcf95245cd0/

Log:	merge heads

diff --git a/pypy/module/__builtin__/app_file_stub.py b/pypy/module/__builtin__/app_file_stub.py
deleted file mode 100644
--- a/pypy/module/__builtin__/app_file_stub.py
+++ /dev/null
@@ -1,20 +0,0 @@
-# NOT_RPYTHON
-
-class file(object): 
-    """file(name[, mode[, buffering]]) -> file object
-
-Open a file.  The mode can be 'r', 'w' or 'a' for reading (default),
-writing or appending.  The file will be created if it doesn't exist
-when opened for writing or appending; it will be truncated when
-opened for writing.  Add a 'b' to the mode for binary files.
-Add a '+' to the mode to allow simultaneous reading and writing.
-If the buffering argument is given, 0 means unbuffered, 1 means line
-buffered, and larger numbers specify the buffer size.
-Add a 'U' to mode to open the file for input with universal newline
-support.  Any line ending in the input file will be seen as a '\n'
-in Python.  Also, a file so opened gains the attribute 'newlines';
-the value for this attribute is one of None (no newline read yet),
-'\r', '\n', '\r\n' or a tuple containing all the newline types seen.
-
-Note:  open() is an alias for file().
-"""

From commits-noreply at bitbucket.org  Wed Feb  2 17:13:34 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed,  2 Feb 2011 17:13:34 +0100 (CET)
Subject: [pypy-svn] pypy default: (agaynor, arigo)
Message-ID: <20110202161334.7DAD22A200A@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41573:60a46baccf3c
Date: 2011-02-02 17:11 +0100
http://bitbucket.org/pypy/pypy/changeset/60a46baccf3c/

Log:	(agaynor, arigo)

	Fix the PositionCookie, to not have the type of all decoded fields
	be r_ulonglong.

diff --git a/pypy/module/_io/interp_textio.py b/pypy/module/_io/interp_textio.py
--- a/pypy/module/_io/interp_textio.py
+++ b/pypy/module/_io/interp_textio.py
@@ -5,7 +5,7 @@
 from pypy.interpreter.gateway import interp2app, unwrap_spec
 from pypy.interpreter.baseobjspace import ObjSpace, Wrappable, W_Root
 from pypy.interpreter.error import OperationError, operationerrfmt
-from pypy.rlib.rarithmetic import r_ulonglong
+from pypy.rlib.rarithmetic import intmask, r_ulonglong, r_uint
 from pypy.rlib.rbigint import rbigint
 from pypy.rlib.rstring import UnicodeBuilder
 from pypy.module._codecs import interp_codecs
@@ -239,13 +239,13 @@
     def __init__(self, bigint):
         self.start_pos = bigint.ulonglongmask()
         bigint = bigint.rshift(r_ulonglong.BITS)
-        self.dec_flags = bigint.ulonglongmask()
-        bigint = bigint.rshift(r_ulonglong.BITS)
-        self.bytes_to_feed = bigint.ulonglongmask()
-        bigint = bigint.rshift(r_ulonglong.BITS)
-        self.chars_to_skip = bigint.ulonglongmask()
-        bigint = bigint.rshift(r_ulonglong.BITS)
-        self.need_eof = bigint.ulonglongmask()
+        self.dec_flags = intmask(bigint.uintmask())
+        bigint = bigint.rshift(r_uint.BITS)
+        self.bytes_to_feed = intmask(bigint.uintmask())
+        bigint = bigint.rshift(r_uint.BITS)
+        self.chars_to_skip = intmask(bigint.uintmask())
+        bigint = bigint.rshift(r_uint.BITS)
+        self.need_eof = int(bigint.uintmask())
 
     def pack(self):
         # The meaning of a tell() cookie is: seek to position, set the
@@ -256,10 +256,14 @@
         rb = rbigint.fromrarith_int
 
         res = rb(self.start_pos)
-        res = res.or_(rb(self.dec_flags).lshift(1 * r_ulonglong.BITS))
-        res = res.or_(rb(self.bytes_to_feed).lshift(2 * r_ulonglong.BITS))
-        res = res.or_(rb(self.chars_to_skip).lshift(3 * r_ulonglong.BITS))
-        return res.or_(rb(self.need_eof).lshift(4 * r_ulonglong.BITS))
+        bits = r_ulonglong.BITS
+        res = res.or_(rb(r_uint(self.dec_flags)).lshift(bits))
+        bits += r_uint.BITS
+        res = res.or_(rb(r_uint(self.bytes_to_feed)).lshift(bits))
+        bits += r_uint.BITS
+        res = res.or_(rb(r_uint(self.chars_to_skip)).lshift(bits))
+        bits += r_uint.BITS
+        return res.or_(rb(r_uint(self.need_eof)).lshift(bits))
 
 class PositionSnapshot:
     def __init__(self, flags, input):

From commits-noreply at bitbucket.org  Wed Feb  2 18:16:25 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed,  2 Feb 2011 18:16:25 +0100 (CET)
Subject: [pypy-svn] pypy default: (agaynor, arigo)
Message-ID: <20110202171625.53ADE2A2006@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41574:53a482379b4f
Date: 2011-02-02 18:14 +0100
http://bitbucket.org/pypy/pypy/changeset/53a482379b4f/

Log:	(agaynor, arigo)

	Fix translation by adding asserts in the PositionCookie.

diff --git a/pypy/module/_io/interp_textio.py b/pypy/module/_io/interp_textio.py
--- a/pypy/module/_io/interp_textio.py
+++ b/pypy/module/_io/interp_textio.py
@@ -239,13 +239,19 @@
     def __init__(self, bigint):
         self.start_pos = bigint.ulonglongmask()
         bigint = bigint.rshift(r_ulonglong.BITS)
-        self.dec_flags = intmask(bigint.uintmask())
+        x = intmask(bigint.uintmask())
+        assert x >= 0
+        self.dec_flags = x
         bigint = bigint.rshift(r_uint.BITS)
-        self.bytes_to_feed = intmask(bigint.uintmask())
+        x = intmask(bigint.uintmask())
+        assert x >= 0
+        self.bytes_to_feed = x
         bigint = bigint.rshift(r_uint.BITS)
-        self.chars_to_skip = intmask(bigint.uintmask())
+        x = intmask(bigint.uintmask())
+        assert x >= 0
+        self.chars_to_skip = x
         bigint = bigint.rshift(r_uint.BITS)
-        self.need_eof = int(bigint.uintmask())
+        self.need_eof = bigint.tobool()
 
     def pack(self):
         # The meaning of a tell() cookie is: seek to position, set the

From fijal at codespeak.net  Wed Feb  2 19:13:26 2011
From: fijal at codespeak.net (fijal at codespeak.net)
Date: Wed,  2 Feb 2011 19:13:26 +0100 (CET)
Subject: [pypy-svn] r80264 - pypy/extradoc/planning
Message-ID: <20110202181326.CCC5F2A2006@codespeak.net>

Author: fijal
Date: Wed Feb  2 19:13:23 2011
New Revision: 80264

Modified:
   pypy/extradoc/planning/jit.txt
Log:
before we forget


Modified: pypy/extradoc/planning/jit.txt
==============================================================================
--- pypy/extradoc/planning/jit.txt	(original)
+++ pypy/extradoc/planning/jit.txt	Wed Feb  2 19:13:23 2011
@@ -1,3 +1,10 @@
+INVESTIGATIONS
+--------------
+
+* 25% slowdown on pyflate fast (Jan 29)
+
+* 10% slowdown on spitfire (Feb 01)
+
 NEW TASKS
 ---------
 
@@ -40,7 +47,7 @@
 
 Things we can do mostly by editing optimizeopt.py:
 
-- getfields which result is never used never get removed (probably cause -
+- getfields which result is never used never get removed (probable cause -
   they used to be as livevars in removed guards). also getfields which result
   is only used as a livevar in a guard should be removed and encoded in
   the guard recovert code (only if we are sure that the stored field cannot
@@ -79,7 +86,7 @@
 - cProfile should be supported (right now it prevents JITting completely):
   the calls to get the time should be done with the single assembler
   instruction "read high-perf time stamp".  The dict lookups done by
-  cProfile should be folded away.
+  cProfile should be folded away. IN PROGRESS
 
 - let super() work with the method cache.
 


From fijal at codespeak.net  Wed Feb  2 20:02:34 2011
From: fijal at codespeak.net (fijal at codespeak.net)
Date: Wed,  2 Feb 2011 20:02:34 +0100 (CET)
Subject: [pypy-svn] r80265 - pypy/extradoc/planning
Message-ID: <20110202190234.E07462A2008@codespeak.net>

Author: fijal
Date: Wed Feb  2 20:02:31 2011
New Revision: 80265

Modified:
   pypy/extradoc/planning/jit.txt
Log:
Write down what I know


Modified: pypy/extradoc/planning/jit.txt
==============================================================================
--- pypy/extradoc/planning/jit.txt	(original)
+++ pypy/extradoc/planning/jit.txt	Wed Feb  2 20:02:31 2011
@@ -2,6 +2,9 @@
 --------------
 
 * 25% slowdown on pyflate fast (Jan 29)
+  - pyflate_fast uses python longs on 32bit
+  - some places don't create SmallLongs even when they should (like conts)
+  - we end up with comparison of Longs and SmallLongs
 
 * 10% slowdown on spitfire (Feb 01)
 


From commits-noreply at bitbucket.org  Wed Feb  2 20:48:15 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Wed,  2 Feb 2011 20:48:15 +0100 (CET)
Subject: [pypy-svn] pypy default: constant propagating tests
Message-ID: <20110202194815.EB96D2A2008@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: 
Changeset: r41575:ab615286e628
Date: 2011-02-02 18:48 +0100
http://bitbucket.org/pypy/pypy/changeset/ab615286e628/

Log:	constant propagating tests

diff --git a/pypy/jit/metainterp/test/test_virtual.py b/pypy/jit/metainterp/test/test_virtual.py
--- a/pypy/jit/metainterp/test/test_virtual.py
+++ b/pypy/jit/metainterp/test/test_virtual.py
@@ -394,6 +394,50 @@
         #    ENTER             - compile the leaving path
         self.check_enter_count(2)
 
+    def test_constant_virtual1(self):
+        myjitdriver = JitDriver(greens = [], reds = ['n', 'sa', 'node'])
+        def f(n):
+            node = self._new()
+            node.value = 1
+            sa = 0
+            while n > 0:
+                myjitdriver.can_enter_jit(n=n, sa=sa, node=node)
+                myjitdriver.jit_merge_point(n=n, sa=sa, node=node)
+                if n>20:
+                    next = self._new()
+                    next.value = 2
+                    node = next
+                elif n>10:
+                    next = self._new()
+                    next.value = 3
+                    node = next
+                sa += node.value
+                n -= 1
+            return sa
+        assert self.meta_interp(f, [30]) == f(30)
+        
+    def test_constant_virtual2(self):
+        myjitdriver = JitDriver(greens = [], reds = ['n', 'sa', 'node'])
+        def f(n):
+            node = self._new()
+            node.value = 1
+            sa = 0
+            while n > 0:
+                myjitdriver.can_enter_jit(n=n, sa=sa, node=node)
+                myjitdriver.jit_merge_point(n=n, sa=sa, node=node)
+                sa += node.value
+                if n&15 > 7:
+                    next = self._new()
+                    next.value = 2
+                    node = next
+                else:
+                    next = self._new()
+                    next.value = 3
+                    node = next
+                n -= 1
+            return sa
+        assert self.meta_interp(f, [31]) == f(31)
+        
     def test_stored_reference_with_bridge1(self):
         class RefNode(object):
             def __init__(self, ref):

From commits-noreply at bitbucket.org  Wed Feb  2 20:48:18 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Wed,  2 Feb 2011 20:48:18 +0100 (CET)
Subject: [pypy-svn] pypy default: This should prevent the short preamble
	from beeing	inlined if the virtuals needed to call the loop
	cannot be	virtuals (ie some pointer to them have escaped).
	Instead the	loop will be retraced and a new specialized
	version generated.	If this too fails,
	jump to the preamble instead. Also short preambles are
	nolonger	generated for cases with virtuals in the arguments
	with members	that are constants. To handle that case we need
	to also check	that the values of those constants are the same
	at the end of	the bridge.
Message-ID: <20110202194818.598152A2008@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: 
Changeset: r41576:6c956838e89c
Date: 2011-02-02 20:36 +0100
http://bitbucket.org/pypy/pypy/changeset/6c956838e89c/

Log:	This should prevent the short preamble from beeing inlined if the
	virtuals needed to call the loop cannot be virtuals (ie some pointer
	to them have escaped). Instead the loop will be retraced and a new
	specialized version generated. If this too fails, jump to the
	preamble instead. Also short preambles are nolonger generated for
	cases with virtuals in the arguments with members that are
	constants. To handle that case we need to also check that the values
	of those constants are the same at the end of the bridge.

diff --git a/pypy/jit/metainterp/optimizeopt/unroll.py b/pypy/jit/metainterp/optimizeopt/unroll.py
--- a/pypy/jit/metainterp/optimizeopt/unroll.py
+++ b/pypy/jit/metainterp/optimizeopt/unroll.py
@@ -139,6 +139,7 @@
             self.cloned_operations.append(newop)
             
     def propagate_all_forward(self):
+        self.make_short_preamble = True
         loop = self.optimizer.loop
         jumpop = loop.operations[-1]
         if jumpop.getopnum() == rop.JUMP:
@@ -151,11 +152,12 @@
 
         if jumpop:
             assert jumpop.getdescr() is loop.token
+            jump_args = jumpop.getarglist()
+            jumpop.initarglist([])
+            virtual_state = [self.getvalue(a).is_virtual() for a in jump_args]
+
             loop.preamble.operations = self.optimizer.newoperations
             self.optimizer = self.optimizer.reconstruct_for_next_iteration()
-
-            jump_args = jumpop.getarglist()
-            jumpop.initarglist([])            
             inputargs = self.inline(self.cloned_operations,
                                     loop.inputargs, jump_args)
             loop.inputargs = inputargs
@@ -210,6 +212,7 @@
                     loop.preamble.token.short_preamble.append(short_loop)
                 else:
                     loop.preamble.token.short_preamble = [short_loop]
+                short_loop.virtual_state = virtual_state
 
                 # Forget the values to allow them to be freed
                 for box in short_loop.inputargs:
@@ -233,6 +236,8 @@
             for a in boxes:
                 if not isinstance(a, Const):
                     inputargs.append(a)
+                else:
+                    self.make_short_preamble = False
 
         # This loop is equivalent to the main optimization loop in
         # Optimizer.propagate_all_forward
@@ -299,6 +304,8 @@
         return True
 
     def create_short_preamble(self, preamble, loop):
+        if not self.make_short_preamble:
+            return None
         #return None # Dissable
 
         preamble_ops = preamble.operations
@@ -506,6 +513,10 @@
         return self.map[loopbox]
 
 class OptInlineShortPreamble(Optimization):
+    def __init__(self, retraced):
+        self.retraced = retraced
+        
+    
     def reconstruct_for_next_iteration(self, optimizer, valuemap):
         return self
     
@@ -520,19 +531,34 @@
             # handle and the inlining fails unexpectedly belwo.
             short = descr.short_preamble
             if short:
+                args = op.getarglist()
+                virtual_state = [self.getvalue(a).is_virtual() for a in args]
                 for sh in short:
-                    if self.inline(sh.operations, sh.inputargs,
-                                   op.getarglist(), dryrun=True):
-                        try:
-                            self.inline(sh.operations, sh.inputargs,
-                                        op.getarglist())
-                        except InvalidLoop:
-                            debug_print("Inlining failed unexpectedly",
-                                        "jumping to preamble instead")
-                            self.emit_operation(op)
-                        return
-                    
-                raise RetraceLoop
+                    assert len(virtual_state) == len(sh.virtual_state)
+                    for i in range(len(virtual_state)):
+                        if sh.virtual_state[i] and not virtual_state[i]:
+                            break
+                        elif not sh.virtual_state[i] and virtual_state[i]:
+                            # XXX Here, this bridge has made some box virtual
+                            # that is not virtual in the original loop. These
+                            # will be forced below. However we could choose
+                            # to raise RetraceLoop here to create a new 
+                            # specialized version of the loop where more
+                            # boxes will be virtual.
+                            pass
+                    else:
+                        if self.inline(sh.operations, sh.inputargs,
+                                       op.getarglist(), dryrun=True):
+                            try:
+                                self.inline(sh.operations, sh.inputargs,
+                                            op.getarglist())
+                            except InvalidLoop:
+                                debug_print("Inlining failed unexpectedly",
+                                            "jumping to preamble instead")
+                                self.emit_operation(op)
+                            return
+                if not self.retraced:    
+                    raise RetraceLoop
         self.emit_operation(op)
                 
         

diff --git a/pypy/jit/metainterp/compile.py b/pypy/jit/metainterp/compile.py
--- a/pypy/jit/metainterp/compile.py
+++ b/pypy/jit/metainterp/compile.py
@@ -566,7 +566,7 @@
         pass
 
 
-def compile_new_bridge(metainterp, old_loop_tokens, resumekey):
+def compile_new_bridge(metainterp, old_loop_tokens, resumekey, retraced=False):
     """Try to compile a new bridge leading from the beginning of the history
     to some existing place.
     """
@@ -587,8 +587,9 @@
         inline_short_preamble = True
     try:
         target_loop_token = state.optimize_bridge(metainterp_sd,
-                                                  old_loop_tokens,
-                                                  new_loop, inline_short_preamble)
+                                                  old_loop_tokens, new_loop,
+                                                  inline_short_preamble,
+                                                  retraced)
     except InvalidLoop:
         # XXX I am fairly convinced that optimize_bridge cannot actually raise
         # InvalidLoop

diff --git a/pypy/jit/metainterp/pyjitpl.py b/pypy/jit/metainterp/pyjitpl.py
--- a/pypy/jit/metainterp/pyjitpl.py
+++ b/pypy/jit/metainterp/pyjitpl.py
@@ -1929,7 +1929,8 @@
         try:
             target_loop_token = compile.compile_new_bridge(self,
                                                            [loop_token],
-                                                           self.resumekey)
+                                                           self.resumekey,
+                                                           True)
         except RetraceLoop:
             assert False
         assert target_loop_token is not None

diff --git a/pypy/jit/metainterp/simple_optimize.py b/pypy/jit/metainterp/simple_optimize.py
--- a/pypy/jit/metainterp/simple_optimize.py
+++ b/pypy/jit/metainterp/simple_optimize.py
@@ -47,7 +47,8 @@
             jumpop.setdescr(loop.token)
         return None
 
-def optimize_bridge(metainterp_sd, old_loops, loop, inline_short_preamble):
+def optimize_bridge(metainterp_sd, old_loops, loop, inline_short_preamble,
+                    retraced):
     optimize_loop(metainterp_sd, [], loop)
     jumpop = loop.operations[-1]
     if jumpop.getopnum() == rop.JUMP:

diff --git a/pypy/jit/metainterp/nounroll_optimize.py b/pypy/jit/metainterp/nounroll_optimize.py
--- a/pypy/jit/metainterp/nounroll_optimize.py
+++ b/pypy/jit/metainterp/nounroll_optimize.py
@@ -17,7 +17,8 @@
     optimize_loop_1(metainterp_sd, loop, False)
     return None
 
-def optimize_bridge(metainterp_sd, old_loop_tokens, bridge, inline_short_preamble):
+def optimize_bridge(metainterp_sd, old_loop_tokens, bridge,
+                    inline_short_preamble, retraced=False):
     debug_start("jit-optimize")
     try:
         return _optimize_bridge(metainterp_sd, old_loop_tokens, bridge)

diff --git a/pypy/jit/metainterp/optimize.py b/pypy/jit/metainterp/optimize.py
--- a/pypy/jit/metainterp/optimize.py
+++ b/pypy/jit/metainterp/optimize.py
@@ -22,20 +22,24 @@
 
 # ____________________________________________________________
 
-def optimize_bridge(metainterp_sd, old_loop_tokens, bridge, inline_short_preamble=True):
+def optimize_bridge(metainterp_sd, old_loop_tokens, bridge,
+                    inline_short_preamble=True, retraced=False):
     debug_start("jit-optimize")
     try:
-        return _optimize_bridge(metainterp_sd, old_loop_tokens, bridge, inline_short_preamble)
+        return _optimize_bridge(metainterp_sd, old_loop_tokens, bridge,
+                                inline_short_preamble, retraced)
     finally:
         debug_stop("jit-optimize")
 
-def _optimize_bridge(metainterp_sd, old_loop_tokens, bridge, inline_short_preamble):
+def _optimize_bridge(metainterp_sd, old_loop_tokens, bridge,
+                     inline_short_preamble, retraced=False):
     cpu = metainterp_sd.cpu
     metainterp_sd.logger_noopt.log_loop(bridge.inputargs, bridge.operations)
     if old_loop_tokens:
         old_loop_token = old_loop_tokens[0]
         bridge.operations[-1].setdescr(old_loop_token)   # patch jump target
-        optimize_bridge_1(metainterp_sd, bridge, inline_short_preamble)
+        optimize_bridge_1(metainterp_sd, bridge, inline_short_preamble,
+                          retraced)
         return old_loop_tokens[0]
         #return bridge.operations[-1].getdescr()
     return None

diff --git a/pypy/jit/metainterp/optimizeopt/__init__.py b/pypy/jit/metainterp/optimizeopt/__init__.py
--- a/pypy/jit/metainterp/optimizeopt/__init__.py
+++ b/pypy/jit/metainterp/optimizeopt/__init__.py
@@ -6,7 +6,8 @@
 from pypy.jit.metainterp.optimizeopt.string import OptString
 from pypy.jit.metainterp.optimizeopt.unroll import optimize_unroll, OptInlineShortPreamble
 
-def optimize_loop_1(metainterp_sd, loop, unroll=True, inline_short_preamble=True):
+def optimize_loop_1(metainterp_sd, loop, unroll=True,
+                    inline_short_preamble=True, retraced=False):
     """Optimize loop.operations to remove internal overheadish operations. 
     """
     opt_str = OptString()
@@ -17,7 +18,7 @@
                      OptHeap(),
                     ]
     if inline_short_preamble:
-        optimizations = [OptInlineShortPreamble()] +  optimizations
+        optimizations = [OptInlineShortPreamble(retraced)] +  optimizations
         
     if metainterp_sd.jit_ffi:
         from pypy.jit.metainterp.optimizeopt.fficall import OptFfiCall
@@ -33,6 +34,8 @@
         optimizer = Optimizer(metainterp_sd, loop, optimizations)
         optimizer.propagate_all_forward()
 
-def optimize_bridge_1(metainterp_sd, bridge, inline_short_preamble=True):
+def optimize_bridge_1(metainterp_sd, bridge, inline_short_preamble=True,
+                      retraced=False):
     """The same, but for a bridge. """
-    optimize_loop_1(metainterp_sd, bridge, False, inline_short_preamble)
+    optimize_loop_1(metainterp_sd, bridge, False, inline_short_preamble,
+                    retraced)

diff --git a/pypy/jit/metainterp/test/test_virtual.py b/pypy/jit/metainterp/test/test_virtual.py
--- a/pypy/jit/metainterp/test/test_virtual.py
+++ b/pypy/jit/metainterp/test/test_virtual.py
@@ -211,7 +211,7 @@
             return node.value
         res = self.meta_interp(f, [20], policy=StopAtXPolicy(externfn))
         assert res == f(20)
-        self.check_loop_count(2)
+        self.check_loop_count(3)
         self.check_loops(**{self._new_op: 1})
         self.check_loops(int_mul=0, call=1)
 

From commits-noreply at bitbucket.org  Wed Feb  2 20:48:18 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Wed,  2 Feb 2011 20:48:18 +0100 (CET)
Subject: [pypy-svn] pypy default: hg merge
Message-ID: <20110202194818.CD5272A200C@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: 
Changeset: r41577:107195875588
Date: 2011-02-02 20:47 +0100
http://bitbucket.org/pypy/pypy/changeset/107195875588/

Log:	hg merge


From commits-noreply at bitbucket.org  Wed Feb  2 21:12:58 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Wed,  2 Feb 2011 21:12:58 +0100 (CET)
Subject: [pypy-svn] pypy default: StringIO.line_buffering is always false.
Message-ID: <20110202201258.32D6D2A2008@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41578:0207326e3669
Date: 2011-02-02 15:06 -0500
http://bitbucket.org/pypy/pypy/changeset/0207326e3669/

Log:	StringIO.line_buffering is always false.

diff --git a/pypy/module/_io/test/test_stringio.py b/pypy/module/_io/test/test_stringio.py
--- a/pypy/module/_io/test/test_stringio.py
+++ b/pypy/module/_io/test/test_stringio.py
@@ -16,6 +16,7 @@
         assert sio.seekable()
         assert not sio.isatty()
         assert not sio.closed
+        assert not sio.line_buffering
         sio.close()
         assert sio.readable()
         assert sio.writable()

diff --git a/pypy/module/_io/interp_stringio.py b/pypy/module/_io/interp_stringio.py
--- a/pypy/module/_io/interp_stringio.py
+++ b/pypy/module/_io/interp_stringio.py
@@ -146,6 +146,10 @@
     def closed_get_w(space, self):
         return space.wrap(self.buf is None)
 
+    def line_buffering_get_w(space, self):
+        self._check_closed(self)
+        return space.w_False
+
 W_StringIO.typedef = TypeDef(
     'StringIO', W_TextIOBase.typedef,
     __module__ = "_io",
@@ -161,4 +165,5 @@
     seekable = interp2app(W_StringIO.seekable_w),
     close = interp2app(W_StringIO.close_w),
     closed = GetSetProperty(W_StringIO.closed_get_w),
+    line_buffering = GetSetProperty(W_StringIO.line_buffering_get_w),
 )

From commits-noreply at bitbucket.org  Thu Feb  3 02:53:00 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Thu,  3 Feb 2011 02:53:00 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix a stupid typo.
Message-ID: <20110203015300.3A5112A2006@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41579:3a39da00a0d6
Date: 2011-02-02 20:52 -0500
http://bitbucket.org/pypy/pypy/changeset/3a39da00a0d6/

Log:	Fix a stupid typo.

diff --git a/pypy/module/_io/interp_stringio.py b/pypy/module/_io/interp_stringio.py
--- a/pypy/module/_io/interp_stringio.py
+++ b/pypy/module/_io/interp_stringio.py
@@ -147,7 +147,6 @@
         return space.wrap(self.buf is None)
 
     def line_buffering_get_w(space, self):
-        self._check_closed(self)
         return space.w_False
 
 W_StringIO.typedef = TypeDef(

From commits-noreply at bitbucket.org  Thu Feb  3 05:55:55 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Thu,  3 Feb 2011 05:55:55 +0100 (CET)
Subject: [pypy-svn] pypy default: Support None for indexes.
Message-ID: <20110203045555.A9F882A2007@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41580:368285d0d54f
Date: 2011-02-02 23:55 -0500
http://bitbucket.org/pypy/pypy/changeset/368285d0d54f/

Log:	Support None for indexes.

diff --git a/pypy/objspace/std/test/test_unicodeobject.py b/pypy/objspace/std/test/test_unicodeobject.py
--- a/pypy/objspace/std/test/test_unicodeobject.py
+++ b/pypy/objspace/std/test/test_unicodeobject.py
@@ -579,6 +579,9 @@
                 assert u[j+2] == u'3'
             assert u'123' * i == i * u'123'
 
+    def test_index(self):
+        assert u"rrarrrrrrrrra".index(u'a', 4, None) == 12
+
     def test_rindex(self):
         from sys import maxint
         assert u'abcdefghiabc'.rindex(u'') == 12
@@ -586,6 +589,8 @@
         assert u'abcdefghiabc'.rindex(u'abc') == 9
         assert u'abcdefghiabc'.rindex(u'abc', 0, -1) == 0
         assert u'abcdefghiabc'.rindex(u'abc', -4*maxint, 4*maxint) == 9
+        assert u'rrarrrrrrrrra'.rindex(u'a', 4, None) == 12
+
         raises(ValueError, u'abcdefghiabc'.rindex, u'hib')
         raises(ValueError, u'defghiabc'.rindex, u'def', 1)
         raises(ValueError, u'defghiabc'.rindex, u'abc', 0, -1)

diff --git a/pypy/objspace/std/unicodeobject.py b/pypy/objspace/std/unicodeobject.py
--- a/pypy/objspace/std/unicodeobject.py
+++ b/pypy/objspace/std/unicodeobject.py
@@ -476,6 +476,10 @@
     assert isinstance(w_sub, W_UnicodeObject)
     self = w_self._value
     sub = w_sub._value
+
+    if space.is_w(w_end, space.w_None):
+        w_end = space.len(w_self)
+
     if upper_bound:
         start = slicetype.adapt_bound(space, len(self), w_start)
         end = slicetype.adapt_bound(space, len(self), w_end)

From commits-noreply at bitbucket.org  Thu Feb  3 06:19:55 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Thu,  3 Feb 2011 06:19:55 +0100 (CET)
Subject: [pypy-svn] pypy default: fix upercase float formatting with
	exponent.
Message-ID: <20110203051955.906412A2006@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41581:b4116b8b83a2
Date: 2011-02-03 00:19 -0500
http://bitbucket.org/pypy/pypy/changeset/b4116b8b83a2/

Log:	fix upercase float formatting with exponent.

diff --git a/pypy/objspace/std/test/test_floatobject.py b/pypy/objspace/std/test/test_floatobject.py
--- a/pypy/objspace/std/test/test_floatobject.py
+++ b/pypy/objspace/std/test/test_floatobject.py
@@ -398,6 +398,10 @@
     def test_from_string(self):
         raises(ValueError, float, "\0")
 
+    def test_format(self):
+        f = 1.1234e200
+        assert f.__format__("G") == "1.1234E+200"
+
 
 class AppTestFloatHex:
     def w_identical(self, x, y):

diff --git a/pypy/rlib/rdtoa.py b/pypy/rlib/rdtoa.py
--- a/pypy/rlib/rdtoa.py
+++ b/pypy/rlib/rdtoa.py
@@ -77,7 +77,7 @@
         raise ValueError
 
 @jit.dont_look_inside
-def format_number(digits, buflen, sign, decpt, code, precision, flags):
+def format_number(digits, buflen, sign, decpt, code, precision, flags, upper):
     # We got digits back, format them.  We may need to pad 'digits'
     # either on the left or right (or both) with extra zeros, so in
     # general the resulting string has the form
@@ -198,23 +198,28 @@
 
     # Now that we've done zero padding, add an exponent if needed.
     if use_exp:
+        if upper:
+            e = 'E'
+        else:
+            e = 'e'
+
         if exp >= 0:
             exp_str = str(exp)
             if len(exp_str) < 2:
-                s += 'e+0' + exp_str
+                s += e + '+0' + exp_str
             else:
-                s += 'e+' + exp_str
+                s += e + '+' + exp_str
         else:
             exp_str = str(-exp)
             if len(exp_str) < 2:
-                s += 'e-0' + exp_str
+                s += e + '-0' + exp_str
             else:
-                s += 'e-' + exp_str
+                s += e + '-' + exp_str
 
     return s
 
 def dtoa(value, code='r', mode=0, precision=0, flags=0,
-         special_strings=lower_special_strings):
+         special_strings=lower_special_strings, upper=False):
     decpt_ptr = lltype.malloc(rffi.INTP.TO, 1, flavor='raw')
     try:
         sign_ptr = lltype.malloc(rffi.INTP.TO, 1, flavor='raw')
@@ -239,7 +244,7 @@
                     decpt = rffi.cast(lltype.Signed, decpt_ptr[0])
 
                     return format_number(digits, buflen, sign, decpt,
-                                         code, precision, flags)
+                                         code, precision, flags, upper)
 
                 finally:
                     dg_freedtoa(digits)
@@ -254,8 +259,10 @@
     if code in 'EFG':
         code = code.lower()
         special_strings = upper_special_strings
+        upper = True
     else:
         special_strings = lower_special_strings
+        upper = False
 
     if code == 'e':
         mode = 2
@@ -275,4 +282,4 @@
         raise ValueError('Invalid mode')
 
     return dtoa(value, code, mode=mode, precision=precision, flags=flags,
-                special_strings=special_strings)
+                special_strings=special_strings, upper=upper)

From commits-noreply at bitbucket.org  Thu Feb  3 07:42:16 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Thu,  3 Feb 2011 07:42:16 +0100 (CET)
Subject: [pypy-svn] pypy default: Failing test that shows that we need to
	check the	complete state of all the virtuals before inlining
	the short preamble
Message-ID: <20110203064216.F147B2A2006@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: 
Changeset: r41582:4689e8f0cb05
Date: 2011-02-03 07:41 +0100
http://bitbucket.org/pypy/pypy/changeset/4689e8f0cb05/

Log:	Failing test that shows that we need to check the complete state of
	all the virtuals before inlining the short preamble

diff --git a/pypy/jit/metainterp/test/test_virtual.py b/pypy/jit/metainterp/test/test_virtual.py
--- a/pypy/jit/metainterp/test/test_virtual.py
+++ b/pypy/jit/metainterp/test/test_virtual.py
@@ -394,6 +394,27 @@
         #    ENTER             - compile the leaving path
         self.check_enter_count(2)
 
+    def test_new_virtual_member_in_bridge(self):
+        myjitdriver = JitDriver(greens = [], reds = ['n', 'sa', 'node'])
+        def f(n):
+            node = self._new()
+            node.value = 1
+            node.extra = 2
+            sa = 0
+            while n > 0:
+                myjitdriver.can_enter_jit(n=n, sa=sa, node=node)
+                myjitdriver.jit_merge_point(n=n, sa=sa, node=node)
+                if n&30 > 0:
+                    sa += node.value
+                    next = self._new()
+                    next.value = n
+                    node = next
+                    if n<10:
+                        node.extra = sa
+                n -= 1
+            return node.extra
+        assert self.meta_interp(f, [20]) == f(20)
+
     def test_constant_virtual1(self):
         myjitdriver = JitDriver(greens = [], reds = ['n', 'sa', 'node'])
         def f(n):

From commits-noreply at bitbucket.org  Thu Feb  3 07:42:17 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Thu,  3 Feb 2011 07:42:17 +0100 (CET)
Subject: [pypy-svn] pypy default: hg merge
Message-ID: <20110203064217.41FD92A2007@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: 
Changeset: r41583:98e2ae9bcd3c
Date: 2011-02-03 07:41 +0100
http://bitbucket.org/pypy/pypy/changeset/98e2ae9bcd3c/

Log:	hg merge


From arigo at codespeak.net  Thu Feb  3 14:21:47 2011
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Thu,  3 Feb 2011 14:21:47 +0100 (CET)
Subject: [pypy-svn] r80268 - pypy/extradoc/talk/ustour2011
Message-ID: <20110203132147.0040F2A2022@codespeak.net>

Author: arigo
Date: Thu Feb  3 14:21:44 2011
New Revision: 80268

Added:
   pypy/extradoc/talk/ustour2011/
   pypy/extradoc/talk/ustour2011/google-abstract.txt
Log:
(fijal, arigo)
The google abstract, first version.


Added: pypy/extradoc/talk/ustour2011/google-abstract.txt
==============================================================================
--- (empty file)
+++ pypy/extradoc/talk/ustour2011/google-abstract.txt	Thu Feb  3 14:21:44 2011
@@ -0,0 +1,24 @@
+
+Authors: Armin Rigo, Maciej Fijalkowski
+# XXX get bios from laura
+
+The PyPy project has gathered recently a lot of attention for its progress
+on speeding up the python language. In this talk we would like to discuss
+what the PyPy project is, what we achieved in past two years in the area
+of dynamic language interpreters and where we'll go from now.
+
+With our recent round of optimizations, we got most benchmarks
+running faster than CPython, the original Python interpreter -- and
+faster than Psyco too, the existing Python specializing JIT
+compiler. Our goal is to reach the time when rewriting parts of a
+Python program to C for performance would not be necessary. One good
+example of our approach is our own translation toolchain, which a
+large (200K lines of code) Python program not originally written for
+PyPy, which runs faster by a factor of two.
+
+PyPy is the most compatible and stable alternative python interpreter
+out there.  It already passes the CPython test suite, with minor
+changes that have been agreed to be implementation-specific issues by
+the Python community.  Recently, a lot of work has been put into
+integrating PyPy with C-based extensions, but a lot more can be
+done about it (e.g. compatibility with SWIG or Cython).


From hpk at codespeak.net  Thu Feb  3 14:54:37 2011
From: hpk at codespeak.net (hpk at codespeak.net)
Date: Thu,  3 Feb 2011 14:54:37 +0100 (CET)
Subject: [pypy-svn] r80269 - pypy/extradoc/talk/ustour2011
Message-ID: <20110203135437.2AB322A2006@codespeak.net>

Author: hpk
Date: Thu Feb  3 14:54:35 2011
New Revision: 80269

Added:
   pypy/extradoc/talk/ustour2011/google-abstract2.txt
Log:
more points for the Google abstract. idea is to interest more people by dropping more keywords and areas than is possible when writing longer sentences.

checked in as a separate suggestion because it's a different style.



Added: pypy/extradoc/talk/ustour2011/google-abstract2.txt
==============================================================================
--- (empty file)
+++ pypy/extradoc/talk/ustour2011/google-abstract2.txt	Thu Feb  3 14:54:35 2011
@@ -0,0 +1,20 @@
+
+Authors: Armin Rigo, Maciej Fijalkowski
+# XXX get bios from laura
+
+The PyPy project has gathered recently a lot of attention for its progress
+on speeding up the Python language. In this talk we would like to discuss
+what the PyPy project achieved particularly in the past two years in the area
+of dynamic language interpreters:
+
+* most Python benchmarks run much faster than with CPython or Psyco
+* the realworld PyPy compiler toolchain itself (200 KLocs) runs twice as fast
+* 64bit and ARM are now supported
+* full compatibility to CPython (much deeper and more than Jython/IronPython)
+* fully pluggable Garbage Collection
+* ctypes and new "cpyext" layer to integrate existing CPython C Extensions
+* experimental super-fast JIT-compilation of calls from Python to C++ libraries
+
+We'll also see to briefly highlight project backgrounds like number of
+active developers and funding background to round up the view
+on the project.


From arigo at codespeak.net  Thu Feb  3 15:23:39 2011
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Thu,  3 Feb 2011 15:23:39 +0100 (CET)
Subject: [pypy-svn] r80270 - pypy/extradoc/talk/ustour2011
Message-ID: <20110203142339.0FC61282B90@codespeak.net>

Author: arigo
Date: Thu Feb  3 15:23:38 2011
New Revision: 80270

Modified:
   pypy/extradoc/talk/ustour2011/google-abstract2.txt
Log:
Tweak this version.  Notably, import what I think is one
valid point of google-abstract.txt.


Modified: pypy/extradoc/talk/ustour2011/google-abstract2.txt
==============================================================================
--- pypy/extradoc/talk/ustour2011/google-abstract2.txt	(original)
+++ pypy/extradoc/talk/ustour2011/google-abstract2.txt	Thu Feb  3 15:23:38 2011
@@ -2,19 +2,21 @@
 Authors: Armin Rigo, Maciej Fijalkowski
 # XXX get bios from laura
 
-The PyPy project has gathered recently a lot of attention for its progress
-on speeding up the Python language. In this talk we would like to discuss
-what the PyPy project achieved particularly in the past two years in the area
-of dynamic language interpreters:
+The PyPy project has gathered recently a lot of attention for its
+progress on speeding up the Python language -- it is the fastest,
+most compatible and stable alternative Python interpreter.  We
+will discuss what the PyPy project achieved, particularly in the
+past two years in the area of dynamic (Just-in-Time) interpreters:
 
 * most Python benchmarks run much faster than with CPython or Psyco
-* the realworld PyPy compiler toolchain itself (200 KLocs) runs twice as fast
-* 64bit and ARM are now supported
-* full compatibility to CPython (much deeper and more than Jython/IronPython)
+* the real-world PyPy compiler toolchain itself (200 KLocs) runs twice as fast
+* supports 64bit and (in-progress) ARM
+* full compatibility to CPython (more than Jython/IronPython)
+* full (and fast) ctypes support to call C libraries from Python
+* new "cpyext" layer to integrate existing CPython C extensions
 * fully pluggable Garbage Collection
-* ctypes and new "cpyext" layer to integrate existing CPython C Extensions
-* experimental super-fast JIT-compilation of calls from Python to C++ libraries
+* experimental super-fast JIT-compilation of calls to C++ libraries
+* future work now possible: SWIG and Cython compatibility
 
-We'll also see to briefly highlight project backgrounds like number of
-active developers and funding background to round up the view
-on the project.
+We will also briefly highlight who the active developers are and
+their funding background.


From arigo at codespeak.net  Thu Feb  3 15:36:39 2011
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Thu,  3 Feb 2011 15:36:39 +0100 (CET)
Subject: [pypy-svn] r80271 - pypy/extradoc/talk/ustour2011
Message-ID: <20110203143639.7AC9E282B90@codespeak.net>

Author: arigo
Date: Thu Feb  3 15:36:37 2011
New Revision: 80271

Modified:
   pypy/extradoc/talk/ustour2011/google-abstract2.txt
Log:
What about this?

Modified: pypy/extradoc/talk/ustour2011/google-abstract2.txt
==============================================================================
--- pypy/extradoc/talk/ustour2011/google-abstract2.txt	(original)
+++ pypy/extradoc/talk/ustour2011/google-abstract2.txt	Thu Feb  3 15:36:37 2011
@@ -12,11 +12,11 @@
 * the real-world PyPy compiler toolchain itself (200 KLocs) runs twice as fast
 * supports 64bit and (in-progress) ARM
 * full compatibility to CPython (more than Jython/IronPython)
-* full (and fast) ctypes support to call C libraries from Python
-* new "cpyext" layer to integrate existing CPython C extensions
+* full (and JIT-ed) ctypes support to call C libraries from Python
 * fully pluggable Garbage Collection
+* new "cpyext" layer to integrate existing CPython C extensions
+* (also, future work: SWIG and/or Cython compatibility)
 * experimental super-fast JIT-compilation of calls to C++ libraries
-* future work now possible: SWIG and Cython compatibility
 
 We will also briefly highlight who the active developers are and
 their funding background.


From hpk at codespeak.net  Thu Feb  3 15:39:37 2011
From: hpk at codespeak.net (hpk at codespeak.net)
Date: Thu,  3 Feb 2011 15:39:37 +0100 (CET)
Subject: [pypy-svn] r80272 - pypy/extradoc/talk/ustour2011
Message-ID: <20110203143937.73927282B90@codespeak.net>

Author: hpk
Date: Thu Feb  3 15:39:36 2011
New Revision: 80272

Modified:
   pypy/extradoc/talk/ustour2011/google-abstract2.txt
Log:
shifting "potential future areas" right next to the sentence
about funding :)



Modified: pypy/extradoc/talk/ustour2011/google-abstract2.txt
==============================================================================
--- pypy/extradoc/talk/ustour2011/google-abstract2.txt	(original)
+++ pypy/extradoc/talk/ustour2011/google-abstract2.txt	Thu Feb  3 15:39:36 2011
@@ -15,8 +15,9 @@
 * full (and JIT-ed) ctypes support to call C libraries from Python
 * fully pluggable Garbage Collection
 * new "cpyext" layer to integrate existing CPython C extensions
-* (also, future work: SWIG and/or Cython compatibility)
 * experimental super-fast JIT-compilation of calls to C++ libraries
 
-We will also briefly highlight who the active developers are and
-their funding background.
+We want to reserve time for discussing potential future work like SWIG
+and/or Cython compatibility and other areas brought up by the audience.
+We will also briefly highlight who the active developers are and their
+funding background.


From arigo at codespeak.net  Thu Feb  3 15:51:30 2011
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Thu,  3 Feb 2011 15:51:30 +0100 (CET)
Subject: [pypy-svn] r80273 - pypy/extradoc/talk/ustour2011
Message-ID: <20110203145130.98667282B90@codespeak.net>

Author: arigo
Date: Thu Feb  3 15:51:29 2011
New Revision: 80273

Modified:
   pypy/extradoc/talk/ustour2011/google-abstract2.txt
Log:
Mention Stackless.

Modified: pypy/extradoc/talk/ustour2011/google-abstract2.txt
==============================================================================
--- pypy/extradoc/talk/ustour2011/google-abstract2.txt	(original)
+++ pypy/extradoc/talk/ustour2011/google-abstract2.txt	Thu Feb  3 15:51:29 2011
@@ -14,6 +14,7 @@
 * full compatibility to CPython (more than Jython/IronPython)
 * full (and JIT-ed) ctypes support to call C libraries from Python
 * fully pluggable Garbage Collection
+* supports Stackless Python (in-progress)
 * new "cpyext" layer to integrate existing CPython C extensions
 * experimental super-fast JIT-compilation of calls to C++ libraries
 


From arigo at codespeak.net  Thu Feb  3 17:20:23 2011
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Thu,  3 Feb 2011 17:20:23 +0100 (CET)
Subject: [pypy-svn] r80274 - pypy/extradoc/talk/ustour2011
Message-ID: <20110203162023.25D2A2A2007@codespeak.net>

Author: arigo
Date: Thu Feb  3 17:20:21 2011
New Revision: 80274

Removed:
   pypy/extradoc/talk/ustour2011/google-abstract.txt
Log:
Remove this one, to avoid confusion.



From commits-noreply at bitbucket.org  Thu Feb  3 18:36:53 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Thu,  3 Feb 2011 18:36:53 +0100 (CET)
Subject: [pypy-svn] pypy 32ptr-on-64bit: Fix.
Message-ID: <20110203173653.6B9B52A2008@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 32ptr-on-64bit
Changeset: r41584:1c8aa0b7d1b4
Date: 2011-01-29 12:58 +0100
http://bitbucket.org/pypy/pypy/changeset/1c8aa0b7d1b4/

Log:	Fix.

diff --git a/pypy/jit/metainterp/resume.py b/pypy/jit/metainterp/resume.py
--- a/pypy/jit/metainterp/resume.py
+++ b/pypy/jit/metainterp/resume.py
@@ -75,7 +75,7 @@
 # important because this is often the biggest single consumer of memory
 # in a pypy-c-jit.
 #
-NUMBERINGP = lltype.Ptr(lltype.GcForwardReference())
+NUMBERINGP = lltype.Ptr(lltype.GcForwardReference(will_be_varsize=True))
 NUMBERING = lltype.GcStruct('Numbering',
                             ('prev', NUMBERINGP),
                             ('nums', lltype.Array(rffi.SHORT)))

From commits-noreply at bitbucket.org  Thu Feb  3 18:36:54 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Thu,  3 Feb 2011 18:36:54 +0100 (CET)
Subject: [pypy-svn] pypy default: Port long_true_divide() from CPython 2.7
	to PyPy.
Message-ID: <20110203173654.42E822A2008@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41585:bdcdaa2c9fc6
Date: 2011-02-03 18:27 +0100
http://bitbucket.org/pypy/pypy/changeset/bdcdaa2c9fc6/

Log:	Port long_true_divide() from CPython 2.7 to PyPy.

diff --git a/pypy/rlib/test/test_rbigint.py b/pypy/rlib/test/test_rbigint.py
--- a/pypy/rlib/test/test_rbigint.py
+++ b/pypy/rlib/test/test_rbigint.py
@@ -36,6 +36,12 @@
                 r2 = op1 / op2
                 assert r1 == r2
 
+    def test_truediv_precision(self):
+        op1 = rbigint.fromlong(12345*2**30)
+        op2 = rbigint.fromlong(98765*7**81)
+        f = op1.truediv(op2)
+        assert f == 4.7298422347492634e-61      # exactly
+
     def test_mod(self):
         for op1 in [-50, -12, -2, -1, 1, 2, 50, 52]:
             for op2 in [-4, -2, -1, 1, 2, 8]:

diff --git a/pypy/rlib/rbigint.py b/pypy/rlib/rbigint.py
--- a/pypy/rlib/rbigint.py
+++ b/pypy/rlib/rbigint.py
@@ -1356,25 +1356,155 @@
     return func(x) + (e * float(SHIFT) * func(2.0))
 _loghelper._annspecialcase_ = 'specialize:arg(0)'
 
+# ____________________________________________________________
+
+BASE_AS_FLOAT = float(1 << SHIFT)     # note that it may not fit an int
+
+BitLengthTable = ''.join(map(chr, [
+    0, 1, 2, 2, 3, 3, 3, 3, 4, 4, 4, 4, 4, 4, 4, 4,
+    5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5]))
+
+def bits_in_digit(d):
+    # returns the unique integer k such that 2**(k-1) <= d <
+    # 2**k if d is nonzero, else 0.
+    d_bits = 0
+    while d >= 32:
+        d_bits += 6
+        d >>= 6
+    d_bits += ord(BitLengthTable[d])
+    return d_bits
+
+def _truediv_result(result, negate):
+    if negate:
+        result = -result
+    return result
+
+def _truediv_overflow():
+    raise OverflowError("integer division result too large for a float")
+
 def _bigint_true_divide(a, b):
-    ad, aexp = _AsScaledDouble(a)
-    bd, bexp = _AsScaledDouble(b)
-    if bd == 0.0:
+    # A longish method to obtain the floating-point result with as much
+    # precision as theoretically possible.  The code is almost directly
+    # copied from CPython.  See there (Objects/longobject.c,
+    # long_true_divide) for detailled comments.  Method in a nutshell:
+    #
+    #    0. reduce to case a, b > 0; filter out obvious underflow/overflow
+    #    1. choose a suitable integer 'shift'
+    #    2. use integer arithmetic to compute x = floor(2**-shift*a/b)
+    #    3. adjust x for correct rounding
+    #    4. convert x to a double dx with the same value
+    #    5. return ldexp(dx, shift).
+
+    from pypy.rlib import rfloat
+    DBL_MANT_DIG = rfloat.DBL_MANT_DIG  # 53 for IEEE 754 binary64
+    DBL_MAX_EXP = rfloat.DBL_MAX_EXP    # 1024 for IEEE 754 binary64
+    DBL_MIN_EXP = rfloat.DBL_MIN_EXP
+    MANT_DIG_DIGITS = DBL_MANT_DIG // SHIFT
+    MANT_DIG_BITS = DBL_MANT_DIG % SHIFT
+
+    # Reduce to case where a and b are both positive.
+    negate = (a.sign < 0) ^ (b.sign < 0)
+    if not b.tobool():
         raise ZeroDivisionError("long division or modulo by zero")
+    if not a.tobool():
+        return _truediv_result(0.0, negate)
 
-    # True value is very close to ad/bd * 2**(SHIFT*(aexp-bexp))
-    ad /= bd   # overflow/underflow impossible here
-    aexp -= bexp
-    if aexp > sys.maxint / SHIFT:
-        raise OverflowError
-    elif aexp < -(sys.maxint / SHIFT):
-        return 0.0 # underflow to 0
-    ad = math.ldexp(ad, aexp * SHIFT)
-    ##if isinf(ad):   # ignore underflow to 0.0
-    ##    raise OverflowError
-    # math.ldexp checks and raises
-    return ad
+    a_size = a._numdigits()
+    b_size = b._numdigits()
 
+    # Fast path for a and b small (exactly representable in a double).
+    # Relies on floating-point division being correctly rounded; results
+    # may be subject to double rounding on x86 machines that operate with
+    # the x87 FPU set to 64-bit precision.
+    a_is_small = (a_size <= MANT_DIG_DIGITS or
+                  (a_size == MANT_DIG_DIGITS+1 and
+                   a.digits[MANT_DIG_DIGITS] >> MANT_DIG_BITS == 0))
+    b_is_small = (b_size <= MANT_DIG_DIGITS or
+                  (b_size == MANT_DIG_DIGITS+1 and
+                   b.digits[MANT_DIG_DIGITS] >> MANT_DIG_BITS == 0))
+    if a_is_small and b_is_small:
+        a_size -= 1
+        da = float(a.digits[a_size])
+        while True:
+            a_size -= 1
+            if a_size < 0: break
+            da = da * BASE_AS_FLOAT + a.digits[a_size]
+
+        b_size -= 1
+        db = float(b.digits[b_size])
+        while True:
+            b_size -= 1
+            if b_size < 0: break
+            db = db * BASE_AS_FLOAT + b.digits[b_size]
+
+        return _truediv_result(da / db, negate)
+
+    # Catch obvious cases of underflow and overflow
+    diff = a_size - b_size
+    if diff > sys.maxint/SHIFT - 1:
+        return _truediv_overflow()           # Extreme overflow
+    elif diff < 1 - sys.maxint/SHIFT:
+        return _truediv_result(0.0, negate)  # Extreme underflow
+    # Next line is now safe from overflowing integers
+    diff = (diff * SHIFT + bits_in_digit(a.digits[a_size - 1]) -
+                           bits_in_digit(b.digits[b_size - 1]))
+    # Now diff = a_bits - b_bits.
+    if diff > DBL_MAX_EXP:
+        return _truediv_overflow()
+    elif diff < DBL_MIN_EXP - DBL_MANT_DIG - 1:
+        return _truediv_result(0.0, negate)
+
+    # Choose value for shift; see comments for step 1 in CPython.
+    shift = max(diff, DBL_MIN_EXP) - DBL_MANT_DIG - 2
+
+    inexact = False
+
+    # x = abs(a * 2**-shift)
+    if shift <= 0:
+        x = a.lshift(-shift)
+    else:
+        x = a.rshift(shift)
+        # set inexact if any of the bits shifted out is nonzero
+        if not a.eq(x.lshift(shift)):
+            inexact = True
+
+    # x //= b. If the remainder is nonzero, set inexact.
+    x, rem = x.divmod(b)
+    if rem.tobool():
+        inexact = True
+
+    assert x.tobool()    # result of division is never zero
+    x_size = x._numdigits()
+    x_bits = (x_size-1)*SHIFT + bits_in_digit(x.digits[x_size-1])
+
+    # The number of extra bits that have to be rounded away.
+    extra_bits = max(x_bits, DBL_MIN_EXP - shift) - DBL_MANT_DIG
+    assert extra_bits == 2 or extra_bits == 3
+
+    # Round by remembering a modified copy of the low digit of x
+    mask = 1 << (extra_bits - 1)
+    low = x.digits[0] | inexact
+    if (low & mask) != 0 and (low & (3*mask-1)) != 0:
+        low += mask
+    x_digit_0 = low & ~(mask-1)
+
+    # Convert x to a double dx; the conversion is exact.
+    x_size -= 1
+    dx = 0.0
+    while x_size > 0:
+        dx += x.digits[x_size]
+        dx *= BASE_AS_FLOAT
+        x_size -= 1
+    dx += x_digit_0
+
+    # Check whether ldexp result will overflow a double.
+    if (shift + x_bits >= DBL_MAX_EXP and
+        (shift + x_bits > DBL_MAX_EXP or dx == math.ldexp(1.0, x_bits))):
+        return _truediv_overflow()
+
+    return _truediv_result(math.ldexp(dx, shift), negate)
+
+# ____________________________________________________________
 
 BASE8  = '01234567'
 BASE10 = '0123456789'

From commits-noreply at bitbucket.org  Thu Feb  3 21:39:38 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Thu,  3 Feb 2011 21:39:38 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix for rope unicode objects.
Message-ID: <20110203203938.F01532A2009@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41586:7b4c8677e64b
Date: 2011-02-03 15:36 -0500
http://bitbucket.org/pypy/pypy/changeset/7b4c8677e64b/

Log:	Fix for rope unicode objects.

diff --git a/pypy/objspace/std/ropeunicodeobject.py b/pypy/objspace/std/ropeunicodeobject.py
--- a/pypy/objspace/std/ropeunicodeobject.py
+++ b/pypy/objspace/std/ropeunicodeobject.py
@@ -482,6 +482,8 @@
 def _convert_idx_params(space, w_self, w_start, w_end):
     self = w_self._node
     length = w_self._node.length()
+    if space.is_w(w_end, space.w_None):
+        w_end = space.len(w_self)
     start = slicetype.adapt_bound(space, length, w_start)
     end = slicetype.adapt_bound(space, length, w_end)
 

From commits-noreply at bitbucket.org  Thu Feb  3 21:42:15 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Thu,  3 Feb 2011 21:42:15 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: Stores the OptValue states of
	all arguments	to the loop and use them to deciede which
	specialized version	of the loop to jump to (work in progress,
	only non virtuals fully supoprted so far). The intbounds creates a
	lot	of (most of the time) identical specialized version. We
	need to	generalize that somehow.
Message-ID: <20110203204215.CC1552A2009@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r41587:057bd6a641f3
Date: 2011-02-03 21:40 +0100
http://bitbucket.org/pypy/pypy/changeset/057bd6a641f3/

Log:	Stores the OptValue states of all arguments to the loop and use them
	to deciede which specialized version of the loop to jump to (work in
	progress, only non virtuals fully supoprted so far). The intbounds
	creates a lot of (most of the time) identical specialized version.
	We need to generalize that somehow.

diff --git a/pypy/jit/metainterp/optimizeopt/unroll.py b/pypy/jit/metainterp/optimizeopt/unroll.py
--- a/pypy/jit/metainterp/optimizeopt/unroll.py
+++ b/pypy/jit/metainterp/optimizeopt/unroll.py
@@ -125,6 +125,64 @@
         self.snapshot_map[snapshot] = new_snapshot
         return new_snapshot
 
+class VirtualStateAdder(resume.ResumeDataVirtualAdder):
+    def __init__(self, optimizer):
+        self.fieldboxes = {}
+        self.optimizer = optimizer
+        self.info = {}
+
+    def register_virtual_fields(self, keybox, fieldboxes):
+        self.fieldboxes[keybox] = fieldboxes
+        
+    def already_seen_virtual(self, keybox):
+        return keybox in self.fieldboxes
+
+    def getvalue(self, box):
+        return self.optimizer.getvalue(box)
+
+    def state(self, box):
+        try:
+            info = self.info[box]
+        except KeyError:
+            value = self.getvalue(box)
+            self.info[box] = info = value.make_virtual_info(self, [])
+            if box in self.fieldboxes:
+                info.fieldstate = [self.state(b) for b in self.fieldboxes[box]]
+                # FIXME: Do we realy want to add fields to the VirtualInfo's?
+        return info
+
+    def get_virtual_state(self, jump_args):
+        for box in jump_args:
+            value = self.getvalue(box)
+            value.get_args_for_fail(self)
+        return [self.state(box) for box in jump_args]
+
+
+    def make_not_virtual(self, value):
+        return NotVirtualInfo(value)
+
+class NotVirtualInfo(resume.AbstractVirtualInfo):
+    def __init__(self, value):
+        self.known_class = value.known_class
+        self.level = value.level
+        self.intbound = value.intbound.clone()
+        if value.is_constant():
+            self.constbox = value.box.clonebox()
+        else:
+            self.constbox = None
+
+    def more_general_than(self, other):
+        # XXX This will always retrace instead of forcing anything which
+        # might be what we want sometimes?
+        if not isinstance(other, NotVirtualInfo):
+            return False
+        if self.constbox:
+            if not self.constbox.same_const(other):
+                return False
+        return (self.known_class == other.known_class and
+                self.level == other.level and
+                self.intbound.contains_bound(other.intbound))
+            
 
 class UnrollOptimizer(Optimization):
     """Unroll the loop into two iterations. The first one will
@@ -139,7 +197,6 @@
             self.cloned_operations.append(newop)
             
     def propagate_all_forward(self):
-        self.make_short_preamble = True
         loop = self.optimizer.loop
         jumpop = loop.operations[-1]
         if jumpop.getopnum() == rop.JUMP:
@@ -154,7 +211,9 @@
             assert jumpop.getdescr() is loop.token
             jump_args = jumpop.getarglist()
             jumpop.initarglist([])
-            virtual_state = [self.getvalue(a).is_virtual() for a in jump_args]
+            #virtual_state = [self.getvalue(a).is_virtual() for a in jump_args]
+            modifier = VirtualStateAdder(self.optimizer)
+            virtual_state = modifier.get_virtual_state(jump_args)
 
             loop.preamble.operations = self.optimizer.newoperations
             self.optimizer = self.optimizer.reconstruct_for_next_iteration()
@@ -221,7 +280,6 @@
                     if op.result:
                         op.result.forget_value()
                 
-
     def inline(self, loop_operations, loop_args, jump_args):
         self.inliner = inliner = Inliner(loop_args, jump_args)
            
@@ -236,8 +294,6 @@
             for a in boxes:
                 if not isinstance(a, Const):
                     inputargs.append(a)
-                else:
-                    self.make_short_preamble = False
 
         # This loop is equivalent to the main optimization loop in
         # Optimizer.propagate_all_forward
@@ -304,8 +360,6 @@
         return True
 
     def create_short_preamble(self, preamble, loop):
-        if not self.make_short_preamble:
-            return None
         #return None # Dissable
 
         preamble_ops = preamble.operations
@@ -532,20 +586,15 @@
             short = descr.short_preamble
             if short:
                 args = op.getarglist()
-                virtual_state = [self.getvalue(a).is_virtual() for a in args]
+                modifier = VirtualStateAdder(self.optimizer)
+                virtual_state = modifier.get_virtual_state(args)
+                print 'len', len(short)
                 for sh in short:
                     assert len(virtual_state) == len(sh.virtual_state)
+                    
                     for i in range(len(virtual_state)):
-                        if sh.virtual_state[i] and not virtual_state[i]:
+                        if not sh.virtual_state[i].more_general_than(virtual_state[i]):
                             break
-                        elif not sh.virtual_state[i] and virtual_state[i]:
-                            # XXX Here, this bridge has made some box virtual
-                            # that is not virtual in the original loop. These
-                            # will be forced below. However we could choose
-                            # to raise RetraceLoop here to create a new 
-                            # specialized version of the loop where more
-                            # boxes will be virtual.
-                            pass
                     else:
                         if self.inline(sh.operations, sh.inputargs,
                                        op.getarglist(), dryrun=True):

diff --git a/pypy/jit/metainterp/optimizeopt/intbounds.py b/pypy/jit/metainterp/optimizeopt/intbounds.py
--- a/pypy/jit/metainterp/optimizeopt/intbounds.py
+++ b/pypy/jit/metainterp/optimizeopt/intbounds.py
@@ -94,21 +94,27 @@
         v2 = self.getvalue(op.getarg(1))
         self.emit_operation(op)
         r = self.getvalue(op.result)
-        r.intbound.intersect(v1.intbound.sub_bound(v2.intbound))
+        b = v1.intbound.sub_bound(v2.intbound)
+        if b.bounded():
+            r.intbound.intersect(b)
 
     def optimize_INT_ADD(self, op):
         v1 = self.getvalue(op.getarg(0))
         v2 = self.getvalue(op.getarg(1))
         self.emit_operation(op)
         r = self.getvalue(op.result)
-        r.intbound.intersect(v1.intbound.add_bound(v2.intbound))
+        b = v1.intbound.add_bound(v2.intbound)
+        if b.bounded():
+            r.intbound.intersect(b)
 
     def optimize_INT_MUL(self, op):
         v1 = self.getvalue(op.getarg(0))
         v2 = self.getvalue(op.getarg(1))
         self.emit_operation(op)
         r = self.getvalue(op.result)
-        r.intbound.intersect(v1.intbound.mul_bound(v2.intbound))
+        b = v1.intbound.mul_bound(v2.intbound)
+        if b.bounded():
+            r.intbound.intersect(b)
 
     def optimize_INT_FLOORDIV(self, op):
         v1 = self.getvalue(op.getarg(0))
@@ -300,7 +306,7 @@
             if r.box.same_constant(CONST_1):
                 self.make_int_gt(op.getarg(0), op.getarg(1))
             else:
-                self.make_int_le(op.getarg(0), op.getarg(1))
+                self.make_int_le(op.getarg(0), op.getarg(1))        
 
     def propagate_bounds_INT_LE(self, op):
         r = self.getvalue(op.result)

diff --git a/pypy/jit/metainterp/optimizeopt/intutils.py b/pypy/jit/metainterp/optimizeopt/intutils.py
--- a/pypy/jit/metainterp/optimizeopt/intutils.py
+++ b/pypy/jit/metainterp/optimizeopt/intutils.py
@@ -42,6 +42,9 @@
         self.has_lower = False
         self.has_upper = False
 
+    def bounded(self):
+        return self.has_lower and self.has_upper
+
     def known_lt(self, other):
         if self.has_upper and other.has_lower and self.upper < other.lower:
             return True
@@ -76,7 +79,7 @@
         return r
     
     def add(self, offset):
-        res = self.copy()
+        res = self.clone()
         try:
             res.lower = ovfcheck(res.lower + offset)
         except OverflowError:
@@ -91,7 +94,7 @@
         return self.mul_bound(IntBound(value, value))
     
     def add_bound(self, other):
-        res = self.copy()
+        res = self.clone()
         if other.has_upper:
             try:
                 res.upper = ovfcheck(res.upper + other.upper)
@@ -109,7 +112,7 @@
         return res
 
     def sub_bound(self, other):
-        res = self.copy()
+        res = self.clone()
         if other.has_lower:
             try:
                 res.upper = ovfcheck(res.upper - other.lower)
@@ -192,6 +195,19 @@
         if self.has_upper and val > self.upper:
             return False
         return True
+
+    def contains_bound(self, other):
+        if other.has_lower:
+            if not self.contains(other.lower):
+                return False
+        elif self.has_lower:
+            return False
+        if other.has_upper:
+            if not self.contains(other.upper):
+                return False
+        elif self.has_upper:
+            return False
+        return True
         
     def __repr__(self):
         if self.has_lower:
@@ -204,7 +220,7 @@
             u = 'Inf'
         return '%s <= x <= %s' % (l, u)
 
-    def copy(self):
+    def clone(self):
         res = IntBound(self.lower, self.upper)
         res.has_lower = self.has_lower
         res.has_upper = self.has_upper

diff --git a/pypy/jit/metainterp/optimizeopt/optimizer.py b/pypy/jit/metainterp/optimizeopt/optimizer.py
--- a/pypy/jit/metainterp/optimizeopt/optimizer.py
+++ b/pypy/jit/metainterp/optimizeopt/optimizer.py
@@ -69,7 +69,8 @@
         pass
 
     def make_virtual_info(self, modifier, fieldnums):
-        raise NotImplementedError # should not be called on this level
+        #raise NotImplementedError # should not be called on this level
+        return modifier.make_not_virtual(self)
 
     def is_constant(self):
         return self.level == LEVEL_CONSTANT

From commits-noreply at bitbucket.org  Thu Feb  3 23:13:03 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Thu,  3 Feb 2011 23:13:03 +0100 (CET)
Subject: [pypy-svn] pypy default: Added newline support to string io.
Message-ID: <20110203221303.236042A2009@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41588:b1399d25389c
Date: 2011-02-03 17:12 -0500
http://bitbucket.org/pypy/pypy/changeset/b1399d25389c/

Log:	Added newline support to string io.

diff --git a/pypy/module/_io/interp_textio.py b/pypy/module/_io/interp_textio.py
--- a/pypy/module/_io/interp_textio.py
+++ b/pypy/module/_io/interp_textio.py
@@ -182,7 +182,7 @@
             space.call_method(self.w_decoder, "setstate", w_state)
 
 W_IncrementalNewlineDecoder.typedef = TypeDef(
-    'TextIOWrapper',
+    'IncrementalNewlineDecoder',
     __new__ = generic_new_descr(W_IncrementalNewlineDecoder),
     __init__  = interp2app(W_IncrementalNewlineDecoder.descr_init),
 
@@ -224,6 +224,49 @@
     def errors_get_w(space, self):
         return space.w_None
 
+
+    def _find_line_ending(self, line, start, end):
+        size = end - start
+        if self.readtranslate:
+
+            # Newlines are already translated, only search for \n
+            pos = line.find(u'\n', start, end)
+            if pos >= 0:
+                return pos - start + 1, 0
+            else:
+                return -1, size
+        elif self.readuniversal:
+            # Universal newline search. Find any of \r, \r\n, \n
+            # The decoder ensures that \r\n are not split in two pieces
+            i = 0
+            while True:
+                # Fast path for non-control chars. The loop always ends
+                # since the Py_UNICODE storage is NUL-terminated.
+                while i < size and line[start + i] > '\r':
+                    i += 1
+                if i >= size:
+                    return -1, size
+                ch = line[start + i]
+                i += 1
+                if ch == '\n':
+                    return i, 0
+                if ch == '\r':
+                    if line[start + i] == '\n':
+                        return i + 1, 0
+                    else:
+                        return i, 0
+        else:
+            # Non-universal mode.
+            pos = line.find(self.readnl, start, end)
+            if pos >= 0:
+                return pos - start + len(self.readnl), 0
+            else:
+                pos = line.find(self.readnl[0], start, end)
+                if pos >= 0:
+                    return -1, pos - start
+                return -1, size
+
+
 W_TextIOBase.typedef = TypeDef(
     '_TextIOBase', W_IOBase.typedef,
     __new__ = generic_new_descr(W_TextIOBase),
@@ -583,48 +626,6 @@
 
         return space.wrap(builder.build())
 
-    def _find_line_ending(self, line, start):
-        end = len(line)
-        size = end - start
-        if self.readtranslate:
-
-            # Newlines are already translated, only search for \n
-            pos = line.find(u'\n', start, end)
-            if pos >= 0:
-                return pos - start + 1, 0
-            else:
-                return -1, size
-        elif self.readuniversal:
-            # Universal newline search. Find any of \r, \r\n, \n
-            # The decoder ensures that \r\n are not split in two pieces
-            i = 0
-            while True:
-                # Fast path for non-control chars. The loop always ends
-                # since the Py_UNICODE storage is NUL-terminated.
-                while i < size and line[start + i] > '\r':
-                    i += 1
-                if i >= size:
-                    return -1, size
-                ch = line[start + i]
-                i += 1
-                if ch == '\n':
-                    return i, 0
-                if ch == '\r':
-                    if line[start + i] == '\n':
-                        return i + 1, 0
-                    else:
-                        return i, 0
-        else:
-            # Non-universal mode.
-            pos = line.find(self.readnl, start, end)
-            if pos >= 0:
-                return pos - start + len(self.readnl), 0
-            else:
-                pos = line.find(self.readnl[0], start, end)
-                if pos >= 0:
-                    return -1, pos - start
-                return -1, size
-
     @unwrap_spec('self', ObjSpace, W_Root)
     def readline_w(self, space, w_limit=None):
         self._check_closed(space)
@@ -663,7 +664,7 @@
                 remaining = None
 
             line_len = len(line)
-            endpos, consumed = self._find_line_ending(line, start)
+            endpos, consumed = self._find_line_ending(line, start, line_len)
             if endpos >= 0:
                 endpos += start
                 if limit >= 0 and endpos >= start + limit - chunked:

diff --git a/pypy/module/_io/test/test_stringio.py b/pypy/module/_io/test/test_stringio.py
--- a/pypy/module/_io/test/test_stringio.py
+++ b/pypy/module/_io/test/test_stringio.py
@@ -143,3 +143,87 @@
         import io
 
         assert io.StringIO.__module__ == "_io"
+
+    def test_newline_none(self):
+        import io
+
+        sio = io.StringIO(u"a\nb\r\nc\rd", newline=None)
+        res = list(sio)
+        assert res == [u"a\n", u"b\n", u"c\n", u"d"]
+        sio.seek(0)
+        res = sio.read(1)
+        assert res == u"a"
+        res = sio.read(2)
+        assert res == u"\nb"
+        res = sio.read(2)
+        assert res == u"\nc"
+        res = sio.read(1)
+        assert res == u"\n"
+
+        sio = io.StringIO(newline=None)
+        res = sio.write(u"a\n")
+        assert res == 2
+        res = sio.write(u"b\r\n")
+        assert res == 3
+        res = sio.write(u"c\rd")
+        assert res == 3
+        sio.seek(0)
+        res = sio.read()
+        assert res == u"a\nb\nc\nd"
+        sio = io.StringIO(u"a\r\nb", newline=None)
+        res = sio.read(3)
+        assert res == u"a\nb"
+
+    def test_newline_empty(self):
+        import io
+
+        sio = io.StringIO(u"a\nb\r\nc\rd", newline="")
+        res = list(sio)
+        assert res == [u"a\n", u"b\r\n", u"c\r", u"d"]
+        sio.seek(0)
+        res = sio.read(4)
+        assert res == u"a\nb\r"
+        res = sio.read(2)
+        assert res == u"\nc"
+        res = sio.read(1)
+        assert res == u"\r"
+
+        sio = io.StringIO(newline="")
+        res = sio.write(u"a\n")
+        assert res == 2
+        res = sio.write(u"b\r")
+        assert res == 2
+        res = sio.write(u"\nc")
+        assert res == 2
+        res = sio.write(u"\rd")
+        assert res == 2
+        sio.seek(0)
+        res = list(sio)
+        assert res == [u"a\n", u"b\r\n", u"c\r", u"d"]
+
+    def test_newline_lf(self):
+        import io
+
+        sio = io.StringIO(u"a\nb\r\nc\rd")
+        res = list(sio)
+        assert res == [u"a\n", u"b\r\n", u"c\rd"]
+
+    def test_newline_cr(self):
+        import io
+
+        sio = io.StringIO(u"a\nb\r\nc\rd", newline="\r")
+        res = sio.read()
+        assert res == u"a\rb\r\rc\rd"
+        sio.seek(0)
+        res = list(sio)
+        assert res == [u"a\r", u"b\r", u"\r", u"c\r", u"d"]
+
+    def test_newline_crlf(self):
+        import io
+
+        sio = io.StringIO(u"a\nb\r\nc\rd", newline="\r\n")
+        res = sio.read()
+        assert res == u"a\r\nb\r\r\nc\rd"
+        sio.seek(0)
+        res = list(sio)
+        assert res == [u"a\r\n", u"b\r\r\n", u"c\rd"]

diff --git a/pypy/module/_io/interp_stringio.py b/pypy/module/_io/interp_stringio.py
--- a/pypy/module/_io/interp_stringio.py
+++ b/pypy/module/_io/interp_stringio.py
@@ -3,7 +3,7 @@
 from pypy.interpreter.gateway import interp2app, unwrap_spec
 from pypy.interpreter.error import OperationError, operationerrfmt
 from pypy.interpreter.baseobjspace import ObjSpace, W_Root
-from pypy.module._io.interp_textio import W_TextIOBase
+from pypy.module._io.interp_textio import W_TextIOBase, W_IncrementalNewlineDecoder
 from pypy.module._io.interp_iobase import convert_size
 
 
@@ -13,11 +13,32 @@
         self.buf = []
         self.pos = 0
 
-    @unwrap_spec('self', ObjSpace, W_Root)
-    def descr_init(self, space, w_initvalue=None):
+    @unwrap_spec('self', ObjSpace, W_Root, "str_or_None")
+    def descr_init(self, space, w_initvalue=None, newline="\n"):
         # In case __init__ is called multiple times
         self.buf = []
         self.pos = 0
+        self.w_decoder = None
+        self.readnl = None
+        self.writenl = None
+
+        if (newline is not None and newline != "" and newline != "\n" and
+            newline != "\r" and newline != "\r\n"):
+            raise operationerrfmt(space.w_ValueError,
+                "illegal newline value: %s", newline
+            )
+        if newline is not None:
+            self.readnl = newline
+        self.readuniversal = newline is None or newline == ""
+        self.readtranslate = newline is None
+        if newline and newline[0] == "\r":
+            self.writenl = newline
+        if self.readuniversal:
+            self.w_decoder = space.call_function(
+                space.gettypefor(W_IncrementalNewlineDecoder),
+                space.w_None,
+                space.wrap(int(self.readtranslate))
+            )
 
         if not space.is_w(w_initvalue, space.w_None):
             self.write_w(space, w_initvalue)
@@ -55,11 +76,27 @@
                                   "string argument expected, got '%s'",
                                   space.type(w_obj).getname(space, '?'))
         self._check_closed(space)
-        string = space.unicode_w(w_obj)
+
+        orig_size = space.int_w(space.len(w_obj))
+
+        if self.w_decoder is not None:
+            w_decoded = space.call_method(
+                self.w_decoder, "decode", w_obj, space.w_True
+            )
+        else:
+            w_decoded = w_obj
+
+        if self.writenl:
+            w_decoded = space.call_method(
+                w_decoded, "replace", space.wrap("\n"), space.wrap(self.writenl)
+            )
+
+        string = space.unicode_w(w_decoded)
         size = len(string)
+
         if size:
             self.write(string)
-        return space.wrap(size)
+        return space.wrap(orig_size)
 
     @unwrap_spec('self', ObjSpace, W_Root)
     def read_w(self, space, w_size=None):
@@ -77,6 +114,30 @@
         self.pos = end
         return space.wrap(u''.join(self.buf[start:end]))
 
+    @unwrap_spec('self', ObjSpace, int)
+    def readline_w(self, space, limit=-1):
+        if self.pos >= len(self.buf):
+            return space.wrap(u"")
+
+        start = self.pos
+        if limit < 0 or limit > len(self.buf) - self.pos:
+            limit = len(self.buf) - self.pos
+
+        end = start + limit
+
+        endpos, consumed = self._find_line_ending(
+            # XXX: super inefficient, makes a copy of the entire contents.
+            "".join(self.buf),
+            start,
+            end
+        )
+        if endpos >= 0:
+            endpos += start
+        else:
+            endpos = end
+        self.pos = endpos
+        return space.wrap("".join(self.buf[start:endpos]))
+
     @unwrap_spec('self', ObjSpace, int, int)
     def seek_w(self, space, pos, mode=0):
         self._check_closed(space)
@@ -149,6 +210,7 @@
     def line_buffering_get_w(space, self):
         return space.w_False
 
+
 W_StringIO.typedef = TypeDef(
     'StringIO', W_TextIOBase.typedef,
     __module__ = "_io",
@@ -156,6 +218,7 @@
     __init__ = interp2app(W_StringIO.descr_init),
     write = interp2app(W_StringIO.write_w),
     read = interp2app(W_StringIO.read_w),
+    readline = interp2app(W_StringIO.readline_w),
     seek = interp2app(W_StringIO.seek_w),
     truncate = interp2app(W_StringIO.truncate_w),
     getvalue = interp2app(W_StringIO.getvalue_w),

From commits-noreply at bitbucket.org  Thu Feb  3 23:17:11 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Thu,  3 Feb 2011 23:17:11 +0100 (CET)
Subject: [pypy-svn] pypy default: Expose newlines property on stringio.
Message-ID: <20110203221711.0647C2A2009@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41589:8708a496628e
Date: 2011-02-03 17:16 -0500
http://bitbucket.org/pypy/pypy/changeset/8708a496628e/

Log:	Expose newlines property on stringio.

diff --git a/pypy/module/_io/test/test_stringio.py b/pypy/module/_io/test/test_stringio.py
--- a/pypy/module/_io/test/test_stringio.py
+++ b/pypy/module/_io/test/test_stringio.py
@@ -227,3 +227,15 @@
         sio.seek(0)
         res = list(sio)
         assert res == [u"a\r\n", u"b\r\r\n", u"c\rd"]
+
+    def test_newline_property(self):
+        import io
+
+        sio = io.StringIO(newline=None)
+        assert sio.newlines is None
+        sio.write(u"a\n")
+        assert sio.newlines == "\n"
+        sio.write(u"b\r\n")
+        assert sio.newlines == ("\n", "\r\n")
+        sio.write(u"c\rd")
+        assert sio.newlines == ("\r", "\n", "\r\n")

diff --git a/pypy/module/_io/interp_stringio.py b/pypy/module/_io/interp_stringio.py
--- a/pypy/module/_io/interp_stringio.py
+++ b/pypy/module/_io/interp_stringio.py
@@ -210,6 +210,11 @@
     def line_buffering_get_w(space, self):
         return space.w_False
 
+    def newlines_get_w(space, self):
+        if self.w_decoder is None:
+            return space.w_None
+        return space.getattr(self.w_decoder, space.wrap("newlines"))
+
 
 W_StringIO.typedef = TypeDef(
     'StringIO', W_TextIOBase.typedef,
@@ -228,4 +233,5 @@
     close = interp2app(W_StringIO.close_w),
     closed = GetSetProperty(W_StringIO.closed_get_w),
     line_buffering = GetSetProperty(W_StringIO.line_buffering_get_w),
+    newlines = GetSetProperty(W_StringIO.newlines_get_w),
 )

From commits-noreply at bitbucket.org  Thu Feb  3 23:25:39 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Thu,  3 Feb 2011 23:25:39 +0100 (CET)
Subject: [pypy-svn] pypy default: Remove a few things from TODO.
Message-ID: <20110203222539.D6EEF2A2009@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41590:8d78e590934f
Date: 2011-02-03 17:25 -0500
http://bitbucket.org/pypy/pypy/changeset/8d78e590934f/

Log:	Remove a few things from TODO.

diff --git a/lib-python/TODO b/lib-python/TODO
--- a/lib-python/TODO
+++ b/lib-python/TODO
@@ -8,19 +8,11 @@
 Probably easy tasks
 -------------------
 
-- New complex syntax (see test_complex.py)::
-
-      assert complex("(1+2j)") == (1+2j)
-      assert complex("(1-2j)") == (1-2j)
-      assert complex("1e500") == complex(INF, 0.0)
-
 - (unicode|bytearray).(index|find) should accept None as indices (see
   test_unicode.py)
 
 - Fix fcntl.fcntl(fd, fcntl.F_NOTIFY, fcntl.DN_MULTISHOT) on 32bit platform.
 
-- missing functions in itertools: combinations, product...
-
 - in test_os.py, fix posix.setregid(-1, -1), posix.setreuid(-1, -1).  This
   proably requires to use the git_t typedef instead of rffi.INT.
 
@@ -45,8 +37,6 @@
 
 - socket module has a couple of changes (including AF_TIPC packet range)
 
-- implement _io.open() (currently it delegates to _pyio.open)
-
 - module/unicodedata/generate_unicodedb.py should parse LineBreaks.txt
   see http://svn.python.org/view?view=rev&revision=79494
 

From commits-noreply at bitbucket.org  Fri Feb  4 01:25:39 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Fri,  4 Feb 2011 01:25:39 +0100 (CET)
Subject: [pypy-svn] pypy default: Attempted translation fix.
Message-ID: <20110204002539.67F9E2A2002@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41591:a54aa9270f2e
Date: 2011-02-03 19:25 -0500
http://bitbucket.org/pypy/pypy/changeset/a54aa9270f2e/

Log:	Attempted translation fix.

diff --git a/pypy/module/_io/interp_stringio.py b/pypy/module/_io/interp_stringio.py
--- a/pypy/module/_io/interp_stringio.py
+++ b/pypy/module/_io/interp_stringio.py
@@ -135,6 +135,7 @@
             endpos += start
         else:
             endpos = end
+        assert endpos >= 0
         self.pos = endpos
         return space.wrap("".join(self.buf[start:endpos]))
 

From commits-noreply at bitbucket.org  Fri Feb  4 01:40:39 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri,  4 Feb 2011 01:40:39 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix a crash in pow() with nonfinite values
Message-ID: <20110204004039.B3D082A2002@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41592:c5f9e9eb2e26
Date: 2011-02-04 01:38 +0100
http://bitbucket.org/pypy/pypy/changeset/c5f9e9eb2e26/

Log:	Fix a crash in pow() with nonfinite values

diff --git a/pypy/objspace/std/floatobject.py b/pypy/objspace/std/floatobject.py
--- a/pypy/objspace/std/floatobject.py
+++ b/pypy/objspace/std/floatobject.py
@@ -412,6 +412,48 @@
     x = w_float1.floatval
     y = w_float2.floatval
 
+    # Sort out special cases here instead of relying on pow()
+    if y == 0.0:
+        # x**0 is 1, even 0**0
+        return W_FloatObject(1.0)
+    if isnan(x):
+        # nan**y = nan, unless y == 0
+        return W_FloatObject(x)
+    if isnan(y):
+        # x**nan = nan, unless x == 1; x**nan = x
+        if x == 1.0:
+            return W_FloatObject(1.0)
+        else:
+            return W_FloatObject(y)
+    if isinf(y):
+        # x**inf is: 0.0 if abs(x) < 1; 1.0 if abs(x) == 1; inf if
+        # abs(x) > 1 (including case where x infinite)
+        #
+        # x**-inf is: inf if abs(x) < 1; 1.0 if abs(x) == 1; 0.0 if
+        # abs(x) > 1 (including case where v infinite)
+        x = abs(x)
+        if x == 1.0:
+            return W_FloatObject(1.0)
+        elif (y > 0.0) == (x > 1.0):
+            return W_FloatObject(INFINITY)
+        else:
+            return W_FloatObject(0.0)
+    if isinf(x):
+        # (+-inf)**w is: inf for w positive, 0 for w negative; in oth
+        # cases, we need to add the appropriate sign if w is an odd
+        # integer.
+        y_is_odd = math.fmod(abs(y), 2.0) == 1.0
+        if y > 0.0:
+            if y_is_odd:
+                return W_FloatObject(x)
+            else:
+                return W_FloatObject(abs(x))
+        else:
+            if y_is_odd:
+                return W_FloatObject(copysign(0.0, x))
+            else:
+                return W_FloatObject(0.0)
+
     if x == 0.0:
         if y < 0.0:
             if isinf(y):

diff --git a/pypy/objspace/std/test/test_floatobject.py b/pypy/objspace/std/test/test_floatobject.py
--- a/pypy/objspace/std/test/test_floatobject.py
+++ b/pypy/objspace/std/test/test_floatobject.py
@@ -204,6 +204,21 @@
             assert pw(0.0, float("-inf")) == float("inf")
             assert math.isnan(pw(-3, float("nan")))
             assert math.isnan(pw(-3., float("nan")))
+            assert pw(-1.0, -float('inf')) == 1.0
+            assert pw(-1.0, float('inf')) == 1.0
+            assert pw(float('inf'), 0) == 1.0
+            assert pw(float('nan'), 0) == 1.0
+
+            assert math.isinf(pw(-0.5, float('-inf')))
+            assert math.isinf(pw(+0.5, float('-inf')))
+            assert pw(-1.5, float('-inf')) == 0.0
+            assert pw(+1.5, float('-inf')) == 0.0
+
+            assert str(pw(float('-inf'), -0.5)) == '0.0'
+            assert str(pw(float('-inf'), -2.0)) == '0.0'
+            assert str(pw(float('-inf'), -1.0)) == '-0.0'
+            assert str(pw(float('-inf'), 1.0)) == '-inf'
+            assert str(pw(float('-inf'), 2.0)) == 'inf'
 
     def test_pow_neg_base(self):
         import math

From commits-noreply at bitbucket.org  Fri Feb  4 01:40:40 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri,  4 Feb 2011 01:40:40 +0100 (CET)
Subject: [pypy-svn] pypy default: Merge heads
Message-ID: <20110204004040.0A4952A2004@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41593:ff167ff59038
Date: 2011-02-04 01:39 +0100
http://bitbucket.org/pypy/pypy/changeset/ff167ff59038/

Log:	Merge heads


From commits-noreply at bitbucket.org  Fri Feb  4 02:08:58 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Fri,  4 Feb 2011 02:08:58 +0100 (CET)
Subject: [pypy-svn] pypy default: Ensure StringIO.readline returns unicode.
Message-ID: <20110204010858.AFF282A2002@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41594:98cb945c7f74
Date: 2011-02-03 20:07 -0500
http://bitbucket.org/pypy/pypy/changeset/98cb945c7f74/

Log:	Ensure StringIO.readline returns unicode.

diff --git a/pypy/module/_io/interp_stringio.py b/pypy/module/_io/interp_stringio.py
--- a/pypy/module/_io/interp_stringio.py
+++ b/pypy/module/_io/interp_stringio.py
@@ -137,7 +137,7 @@
             endpos = end
         assert endpos >= 0
         self.pos = endpos
-        return space.wrap("".join(self.buf[start:endpos]))
+        return space.wrap(u"".join(self.buf[start:endpos]))
 
     @unwrap_spec('self', ObjSpace, int, int)
     def seek_w(self, space, pos, mode=0):

From commits-noreply at bitbucket.org  Fri Feb  4 02:08:59 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Fri,  4 Feb 2011 02:08:59 +0100 (CET)
Subject: [pypy-svn] pypy default: Merged upstream.
Message-ID: <20110204010859.39A8E2A2004@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41595:0f343e70340b
Date: 2011-02-03 20:08 -0500
http://bitbucket.org/pypy/pypy/changeset/0f343e70340b/

Log:	Merged upstream.


From commits-noreply at bitbucket.org  Fri Feb  4 03:47:06 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Fri,  4 Feb 2011 03:47:06 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix translation.
Message-ID: <20110204024706.5FCEC282B9E@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41596:693b3462fc4d
Date: 2011-02-03 21:46 -0500
http://bitbucket.org/pypy/pypy/changeset/693b3462fc4d/

Log:	Fix translation.

diff --git a/pypy/module/_io/interp_stringio.py b/pypy/module/_io/interp_stringio.py
--- a/pypy/module/_io/interp_stringio.py
+++ b/pypy/module/_io/interp_stringio.py
@@ -13,8 +13,8 @@
         self.buf = []
         self.pos = 0
 
-    @unwrap_spec('self', ObjSpace, W_Root, "str_or_None")
-    def descr_init(self, space, w_initvalue=None, newline="\n"):
+    @unwrap_spec('self', ObjSpace, W_Root, W_Root)
+    def descr_init(self, space, w_initvalue=None, w_newline="\n"):
         # In case __init__ is called multiple times
         self.buf = []
         self.pos = 0
@@ -22,16 +22,25 @@
         self.readnl = None
         self.writenl = None
 
-        if (newline is not None and newline != "" and newline != "\n" and
-            newline != "\r" and newline != "\r\n"):
-            raise operationerrfmt(space.w_ValueError,
-                "illegal newline value: %s", newline
+        if space.is_w(w_newline, space.w_None):
+            newline = None
+        else:
+            newline = space.unicode_w(w_newline)
+
+        if (newline is not None and newline != u"" and newline != u"\n" and
+            newline != u"\r" and newline != u"\r\n"):
+            # Not using operationerrfmt() because I don't know how to ues it
+            # with unicode
+            raise OperationError(space.w_ValueError,
+                space.mod(
+                    space.wrap("illegal newline value: %s"), space.wrap(newline)
+                )
             )
         if newline is not None:
             self.readnl = newline
-        self.readuniversal = newline is None or newline == ""
+        self.readuniversal = newline is None or newline == u""
         self.readtranslate = newline is None
-        if newline and newline[0] == "\r":
+        if newline and newline[0] == u"\r":
             self.writenl = newline
         if self.readuniversal:
             self.w_decoder = space.call_function(
@@ -123,11 +132,12 @@
         if limit < 0 or limit > len(self.buf) - self.pos:
             limit = len(self.buf) - self.pos
 
+        assert limit >= 0
         end = start + limit
 
         endpos, consumed = self._find_line_ending(
             # XXX: super inefficient, makes a copy of the entire contents.
-            "".join(self.buf),
+            u"".join(self.buf),
             start,
             end
         )

From commits-noreply at bitbucket.org  Fri Feb  4 04:48:47 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Fri,  4 Feb 2011 04:48:47 +0100 (CET)
Subject: [pypy-svn] pypy default: Remove some XXX.
Message-ID: <20110204034847.72965282B9E@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41597:ecef9c6c9d3f
Date: 2011-02-03 21:59 -0500
http://bitbucket.org/pypy/pypy/changeset/ecef9c6c9d3f/

Log:	Remove some XXX.

diff --git a/pypy/module/_io/interp_stringio.py b/pypy/module/_io/interp_stringio.py
--- a/pypy/module/_io/interp_stringio.py
+++ b/pypy/module/_io/interp_stringio.py
@@ -66,11 +66,7 @@
             self.buf.extend([u'\0'] * (newlength - len(self.buf)))
 
     def write(self, string):
-        # XXX self.decoder
-        decoded = string
-        # XXX writenl
-
-        length = len(decoded)
+        length = len(string)
         if self.pos + length > len(self.buf):
             self.resize_buffer(self.pos + length)
 

From commits-noreply at bitbucket.org  Fri Feb  4 04:48:48 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Fri,  4 Feb 2011 04:48:48 +0100 (CET)
Subject: [pypy-svn] pypy default: Fixed a segfault in StringIO.
Message-ID: <20110204034848.56ABB282B9E@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41598:5fcda453df94
Date: 2011-02-03 22:48 -0500
http://bitbucket.org/pypy/pypy/changeset/5fcda453df94/

Log:	Fixed a segfault in StringIO.

diff --git a/pypy/module/_io/test/test_stringio.py b/pypy/module/_io/test/test_stringio.py
--- a/pypy/module/_io/test/test_stringio.py
+++ b/pypy/module/_io/test/test_stringio.py
@@ -239,3 +239,28 @@
         assert sio.newlines == ("\n", "\r\n")
         sio.write(u"c\rd")
         assert sio.newlines == ("\r", "\n", "\r\n")
+
+    def test_iterator(self):
+        import io
+
+        s = u"1234567890\n"
+        sio = io.StringIO(s * 10)
+
+        assert iter(sio) is sio
+        assert hasattr(sio, "__iter__")
+        assert hasattr(sio, "next")
+
+        i = 0
+        for line in sio:
+            assert line == s
+            i += 1
+        assert i == 10
+        sio.seek(0)
+        i = 0
+        for line in sio:
+            assert line == s
+            i += 1
+        assert i == 10
+        sio = io.StringIO(s * 2)
+        sio.close()
+        raises(ValueError, next, sio)

diff --git a/pypy/module/_io/interp_stringio.py b/pypy/module/_io/interp_stringio.py
--- a/pypy/module/_io/interp_stringio.py
+++ b/pypy/module/_io/interp_stringio.py
@@ -121,6 +121,8 @@
 
     @unwrap_spec('self', ObjSpace, int)
     def readline_w(self, space, limit=-1):
+        self._check_closed(space)
+
         if self.pos >= len(self.buf):
             return space.wrap(u"")
 

From commits-noreply at bitbucket.org  Fri Feb  4 05:26:49 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Fri,  4 Feb 2011 05:26:49 +0100 (CET)
Subject: [pypy-svn] pypy default: Allow None for start index in unicode.index
Message-ID: <20110204042649.37FFE2A2002@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41599:f4961caf91a7
Date: 2011-02-03 23:26 -0500
http://bitbucket.org/pypy/pypy/changeset/f4961caf91a7/

Log:	Allow None for start index in unicode.index

diff --git a/pypy/objspace/std/test/test_unicodeobject.py b/pypy/objspace/std/test/test_unicodeobject.py
--- a/pypy/objspace/std/test/test_unicodeobject.py
+++ b/pypy/objspace/std/test/test_unicodeobject.py
@@ -581,6 +581,7 @@
 
     def test_index(self):
         assert u"rrarrrrrrrrra".index(u'a', 4, None) == 12
+        assert u"rrarrrrrrrrra".index(u'a', None, 6) == 2
 
     def test_rindex(self):
         from sys import maxint

diff --git a/pypy/objspace/std/unicodeobject.py b/pypy/objspace/std/unicodeobject.py
--- a/pypy/objspace/std/unicodeobject.py
+++ b/pypy/objspace/std/unicodeobject.py
@@ -477,6 +477,8 @@
     self = w_self._value
     sub = w_sub._value
 
+    if space.is_w(w_start, space.w_None):
+        w_start = space.wrap(0)
     if space.is_w(w_end, space.w_None):
         w_end = space.len(w_self)
 

diff --git a/pypy/objspace/std/ropeunicodeobject.py b/pypy/objspace/std/ropeunicodeobject.py
--- a/pypy/objspace/std/ropeunicodeobject.py
+++ b/pypy/objspace/std/ropeunicodeobject.py
@@ -482,6 +482,8 @@
 def _convert_idx_params(space, w_self, w_start, w_end):
     self = w_self._node
     length = w_self._node.length()
+    if space.is_w(w_start, space.w_None):
+        w_start = space.wrap(0)
     if space.is_w(w_end, space.w_None):
         w_end = space.len(w_self)
     start = slicetype.adapt_bound(space, length, w_start)

From fijal at codespeak.net  Fri Feb  4 07:56:24 2011
From: fijal at codespeak.net (fijal at codespeak.net)
Date: Fri,  4 Feb 2011 07:56:24 +0100 (CET)
Subject: [pypy-svn] r80275 - pypy/extradoc/talk/ustour2011
Message-ID: <20110204065624.1B4CC282B90@codespeak.net>

Author: fijal
Date: Fri Feb  4 07:56:22 2011
New Revision: 80275

Modified:
   pypy/extradoc/talk/ustour2011/google-abstract2.txt
Log:
Strike pluggable GCs. I don't think anyone cares about pluggability as long
as it serves their purpose (ie it's fast). This term is very abused.


Modified: pypy/extradoc/talk/ustour2011/google-abstract2.txt
==============================================================================
--- pypy/extradoc/talk/ustour2011/google-abstract2.txt	(original)
+++ pypy/extradoc/talk/ustour2011/google-abstract2.txt	Fri Feb  4 07:56:22 2011
@@ -13,7 +13,6 @@
 * supports 64bit and (in-progress) ARM
 * full compatibility to CPython (more than Jython/IronPython)
 * full (and JIT-ed) ctypes support to call C libraries from Python
-* fully pluggable Garbage Collection
 * supports Stackless Python (in-progress)
 * new "cpyext" layer to integrate existing CPython C extensions
 * experimental super-fast JIT-compilation of calls to C++ libraries


From commits-noreply at bitbucket.org  Fri Feb  4 08:24:28 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Fri,  4 Feb 2011 08:24:28 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: supporting VirtualStruct
Message-ID: <20110204072428.343CF282B90@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r41600:76a3cb0c3800
Date: 2011-02-04 08:24 +0100
http://bitbucket.org/pypy/pypy/changeset/76a3cb0c3800/

Log:	supporting VirtualStruct

diff --git a/pypy/jit/metainterp/optimizeopt/unroll.py b/pypy/jit/metainterp/optimizeopt/unroll.py
--- a/pypy/jit/metainterp/optimizeopt/unroll.py
+++ b/pypy/jit/metainterp/optimizeopt/unroll.py
@@ -125,6 +125,17 @@
         self.snapshot_map[snapshot] = new_snapshot
         return new_snapshot
 
+class VirtualState(object):
+    def __init__(self, state):
+        self.state = state
+
+    def more_general_than(self, other):
+        assert len(self.state) == len(other.state)
+        for i in range(len(self.state)):
+            if not self.state[i].more_general_than(other.state[i]):
+                return False
+        return True
+
 class VirtualStateAdder(resume.ResumeDataVirtualAdder):
     def __init__(self, optimizer):
         self.fieldboxes = {}
@@ -155,7 +166,7 @@
         for box in jump_args:
             value = self.getvalue(box)
             value.get_args_for_fail(self)
-        return [self.state(box) for box in jump_args]
+        return VirtualState([self.state(box) for box in jump_args])
 
 
     def make_not_virtual(self, value):
@@ -167,7 +178,7 @@
         self.level = value.level
         self.intbound = value.intbound.clone()
         if value.is_constant():
-            self.constbox = value.box.clonebox()
+            self.constbox = value.box
         else:
             self.constbox = None
 
@@ -176,12 +187,15 @@
         # might be what we want sometimes?
         if not isinstance(other, NotVirtualInfo):
             return False
-        if self.constbox:
-            if not self.constbox.same_const(other):
+        if other.level < self.level:
+            return False
+        if self.level == LEVEL_CONSTANT:
+            if not self.constbox.same_constant(other.constbox):
                 return False
-        return (self.known_class == other.known_class and
-                self.level == other.level and
-                self.intbound.contains_bound(other.intbound))
+        elif self.level == LEVEL_KNOWNCLASS:
+            if self.known_class != other.known_class: # FIXME: use issubclass?
+                return False
+        return self.intbound.contains_bound(other.intbound)
             
 
 class UnrollOptimizer(Optimization):
@@ -590,12 +604,8 @@
                 virtual_state = modifier.get_virtual_state(args)
                 print 'len', len(short)
                 for sh in short:
-                    assert len(virtual_state) == len(sh.virtual_state)
-                    
-                    for i in range(len(virtual_state)):
-                        if not sh.virtual_state[i].more_general_than(virtual_state[i]):
-                            break
-                    else:
+                    if sh.virtual_state.more_general_than(virtual_state):
+                        # FIXME: Do we still need the dry run
                         if self.inline(sh.operations, sh.inputargs,
                                        op.getarglist(), dryrun=True):
                             try:

diff --git a/pypy/jit/metainterp/resume.py b/pypy/jit/metainterp/resume.py
--- a/pypy/jit/metainterp/resume.py
+++ b/pypy/jit/metainterp/resume.py
@@ -452,6 +452,23 @@
                         str(self.fielddescrs[i]),
                         str(untag(self.fieldnums[i])))
 
+    def more_general_than(self, other):
+        if not isinstance(other, AbstractVirtualStructInfo):
+            return False
+        assert len(self.fielddescrs) == len(self.fieldstate)
+        assert len(other.fielddescrs) == len(other.fieldstate)
+        if len(self.fielddescrs) != len(other.fielddescrs):
+            return False
+        
+        for i in range(len(self.fielddescrs)):
+            if other.fielddescrs[i] is not self.fielddescrs[i]:
+                return False
+            if not self.fieldstate[i].more_general_than(other.fieldstate[i]):
+                return False
+        return True
+            
+        
+
 class VirtualInfo(AbstractVirtualStructInfo):
     def __init__(self, known_class, fielddescrs):
         AbstractVirtualStructInfo.__init__(self, fielddescrs)

diff --git a/pypy/jit/metainterp/test/test_virtual.py b/pypy/jit/metainterp/test/test_virtual.py
--- a/pypy/jit/metainterp/test/test_virtual.py
+++ b/pypy/jit/metainterp/test/test_virtual.py
@@ -186,7 +186,7 @@
         self.check_loop_count(2)
         self.check_loops(new=0, new_with_vtable=0,
                                 getfield_gc=0, setfield_gc=0)
-
+        
     def test_two_loops_with_escaping_virtual(self):
         myjitdriver = JitDriver(greens = [], reds = ['n', 'node'])
         def externfn(node):

From commits-noreply at bitbucket.org  Fri Feb  4 12:02:15 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Fri,  4 Feb 2011 12:02:15 +0100 (CET)
Subject: [pypy-svn] pypy default: Remove old operation left behind.
Message-ID: <20110204110215.D2D3A2A2002@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41601:68741da9a537
Date: 2011-02-04 12:01 +0100
http://bitbucket.org/pypy/pypy/changeset/68741da9a537/

Log:	Remove old operation left behind.

diff --git a/pypy/jit/backend/llgraph/llimpl.py b/pypy/jit/backend/llgraph/llimpl.py
--- a/pypy/jit/backend/llgraph/llimpl.py
+++ b/pypy/jit/backend/llgraph/llimpl.py
@@ -852,9 +852,6 @@
     def op_cast_ptr_to_int(self, descr, ptr):
         return cast_to_int(ptr)
 
-    def op_uint_xor(self, descr, arg1, arg2):
-        return arg1 ^ arg2
-
     def op_force_token(self, descr):
         opaque_frame = _to_opaque(self)
         return llmemory.cast_ptr_to_adr(opaque_frame)

From commits-noreply at bitbucket.org  Fri Feb  4 12:04:04 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri,  4 Feb 2011 12:04:04 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix compilation on Windows
Message-ID: <20110204110404.996DE2A2002@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41602:3a51b7723c9c
Date: 2011-02-04 11:56 +0100
http://bitbucket.org/pypy/pypy/changeset/3a51b7723c9c/

Log:	Fix compilation on Windows

diff --git a/pypy/translator/c/src/signals.h b/pypy/translator/c/src/signals.h
--- a/pypy/translator/c/src/signals.h
+++ b/pypy/translator/c/src/signals.h
@@ -119,7 +119,12 @@
 
     if (wakeup_fd != -1) 
       {
-        ssize_t res = write(wakeup_fd, "\0", 1);
+#ifndef _WIN32
+        ssize_t res;
+#else
+        int res;
+#endif
+        res = write(wakeup_fd, "\0", 1);
         /* the return value is ignored here */
       }
 }

From commits-noreply at bitbucket.org  Fri Feb  4 12:04:05 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri,  4 Feb 2011 12:04:05 +0100 (CET)
Subject: [pypy-svn] pypy default: Add modifiable copy of test_cpickle.py
Message-ID: <20110204110405.550922A2002@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41603:06a5dc59d0f2
Date: 2011-02-04 11:57 +0100
http://bitbucket.org/pypy/pypy/changeset/06a5dc59d0f2/

Log:	Add modifiable copy of test_cpickle.py

diff --git a/lib-python/2.7.0/test/test_cpickle.py b/lib-python/modified-2.7.0/test/test_cpickle.py
copy from lib-python/2.7.0/test/test_cpickle.py
copy to lib-python/modified-2.7.0/test/test_cpickle.py

From commits-noreply at bitbucket.org  Fri Feb  4 12:04:06 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri,  4 Feb 2011 12:04:06 +0100 (CET)
Subject: [pypy-svn] pypy default: "Fast" pickling has no memo,
	so it raises an exception with recursive structure.
Message-ID: <20110204110406.D02D02A2002@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41604:3ddf0b03e02f
Date: 2011-02-04 12:01 +0100
http://bitbucket.org/pypy/pypy/changeset/3ddf0b03e02f/

Log:	"Fast" pickling has no memo, so it raises an exception with
	recursive structure. CPython raises ValueError, PyPy raises
	RuntimeError.

diff --git a/lib-python/modified-2.7.0/test/test_cpickle.py b/lib-python/modified-2.7.0/test/test_cpickle.py
--- a/lib-python/modified-2.7.0/test/test_cpickle.py
+++ b/lib-python/modified-2.7.0/test/test_cpickle.py
@@ -61,27 +61,27 @@
     error = cPickle.BadPickleGet
 
     def test_recursive_list(self):
-        self.assertRaises(ValueError,
+        self.assertRaises((ValueError, RuntimeError),
                           AbstractPickleTests.test_recursive_list,
                           self)
 
     def test_recursive_tuple(self):
-        self.assertRaises(ValueError,
+        self.assertRaises((ValueError, RuntimeError),
                           AbstractPickleTests.test_recursive_tuple,
                           self)
 
     def test_recursive_inst(self):
-        self.assertRaises(ValueError,
+        self.assertRaises((ValueError, RuntimeError),
                           AbstractPickleTests.test_recursive_inst,
                           self)
 
     def test_recursive_dict(self):
-        self.assertRaises(ValueError,
+        self.assertRaises((ValueError, RuntimeError),
                           AbstractPickleTests.test_recursive_dict,
                           self)
 
     def test_recursive_multi(self):
-        self.assertRaises(ValueError,
+        self.assertRaises((ValueError, RuntimeError),
                           AbstractPickleTests.test_recursive_multi,
                           self)
 

From commits-noreply at bitbucket.org  Fri Feb  4 12:04:07 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri,  4 Feb 2011 12:04:07 +0100 (CET)
Subject: [pypy-svn] pypy default: merge heads
Message-ID: <20110204110407.9107C2A2009@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41605:ef22897351e2
Date: 2011-02-04 12:02 +0100
http://bitbucket.org/pypy/pypy/changeset/ef22897351e2/

Log:	merge heads


From commits-noreply at bitbucket.org  Fri Feb  4 13:25:17 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Fri,  4 Feb 2011 13:25:17 +0100 (CET)
Subject: [pypy-svn] pypy jitypes2: Fix for tests.
Message-ID: <20110204122517.AE6722A2002@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: jitypes2
Changeset: r41606:96964ba376b4
Date: 2011-02-04 09:57 +0100
http://bitbucket.org/pypy/pypy/changeset/96964ba376b4/

Log:	Fix for tests.

diff --git a/pypy/jit/metainterp/optimizeopt/fficall.py b/pypy/jit/metainterp/optimizeopt/fficall.py
--- a/pypy/jit/metainterp/optimizeopt/fficall.py
+++ b/pypy/jit/metainterp/optimizeopt/fficall.py
@@ -73,7 +73,7 @@
 
     def setup(self):
         self.funcinfo = None
-        self.logger = self.optimizer.metainterp_sd.logger_noopt
+        self.logger = self.optimizer.metainterp_sd.logger_ops
 
     def propagate_begin_forward(self):
         debug_start('jit-log-ffiopt')

From commits-noreply at bitbucket.org  Fri Feb  4 13:28:39 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Fri,  4 Feb 2011 13:28:39 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix for test_lloperation.
Message-ID: <20110204122839.CC1322A2002@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41607:4128bf56b110
Date: 2011-02-04 13:28 +0100
http://bitbucket.org/pypy/pypy/changeset/4128bf56b110/

Log:	Fix for test_lloperation.

diff --git a/pypy/rpython/llinterp.py b/pypy/rpython/llinterp.py
--- a/pypy/rpython/llinterp.py
+++ b/pypy/rpython/llinterp.py
@@ -830,6 +830,12 @@
     def op_gc_thread_die(self):
         self.heap.thread_die()
 
+    def op_gc_thread_before_fork(self):
+        raise NotImplementedError
+
+    def op_gc_thread_after_fork(self):
+        raise NotImplementedError
+
     def op_gc_free(self, addr):
         # what can you do?
         pass

From commits-noreply at bitbucket.org  Fri Feb  4 13:35:29 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri,  4 Feb 2011 13:35:29 +0100 (CET)
Subject: [pypy-svn] pypy default: Support for :format_spec in
	TemplateFormatter
Message-ID: <20110204123529.700782A2002@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41608:903e2eabfd76
Date: 2011-02-04 13:33 +0100
http://bitbucket.org/pypy/pypy/changeset/903e2eabfd76/

Log:	Support for :format_spec in TemplateFormatter

diff --git a/pypy/objspace/std/newformat.py b/pypy/objspace/std/newformat.py
--- a/pypy/objspace/std/newformat.py
+++ b/pypy/objspace/std/newformat.py
@@ -126,9 +126,12 @@
                         w_msg = self.space.wrap("expected conversion")
                         raise OperationError(self.space.w_ValueError, w_msg)
                     conversion = s[i]
+                    i += 1
+                    c = s[i]
                 else:
                     conversion = self.empty
-                i += 1
+                if c == ':':
+                    i += 1
                 return s[start:end_name], conversion, i
             i += 1
         return s[start:end], self.empty, end

diff --git a/pypy/objspace/std/test/test_newformat.py b/pypy/objspace/std/test/test_newformat.py
--- a/pypy/objspace/std/test/test_newformat.py
+++ b/pypy/objspace/std/test/test_newformat.py
@@ -72,6 +72,13 @@
         assert self.s("{!r}").format(x()) == self.s("32")
         assert self.s("{!s}").format(x()) == self.s("32")
 
+    def test_format_spec(self):
+        assert self.s('{0!s:}').format('Hello') == 'Hello'
+        assert self.s('{0!s:15}').format('Hello') == 'Hello          '
+        assert self.s('{0!s:15s}').format('Hello') == 'Hello          '
+        assert self.s('{0!r}').format('Hello') == "'Hello'"
+        assert self.s('{0!r:}').format('Hello') == "'Hello'"
+
     def test_invalid_conversion(self):
         raises(ValueError, self.s("{!x}").format, 3)
         raises(ValueError, self.s("{!}").format)

From commits-noreply at bitbucket.org  Fri Feb  4 13:35:29 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri,  4 Feb 2011 13:35:29 +0100 (CET)
Subject: [pypy-svn] pypy default: Merge heads
Message-ID: <20110204123529.BE5922A2006@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41609:b6fef6d2d210
Date: 2011-02-04 13:34 +0100
http://bitbucket.org/pypy/pypy/changeset/b6fef6d2d210/

Log:	Merge heads


From antocuni at codespeak.net  Fri Feb  4 13:42:25 2011
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Fri,  4 Feb 2011 13:42:25 +0100 (CET)
Subject: [pypy-svn] r80282 - pypy/pysqlite2
Message-ID: <20110204124225.133162A2002@codespeak.net>

Author: antocuni
Date: Fri Feb  4 13:42:23 2011
New Revision: 80282

Modified:
   pypy/pysqlite2/dbapi2.py
Log:
revert 96964ba376b4, and fix the test in another way: it's better to use logger_noopt because this way the name of the variables are preserved




Modified: pypy/pysqlite2/dbapi2.py
==============================================================================
--- pypy/pysqlite2/dbapi2.py	(original)
+++ pypy/pysqlite2/dbapi2.py	Fri Feb  4 13:42:23 2011
@@ -205,6 +205,46 @@
 sqlite.sqlite3_result_error.argtypes = [c_void_p, c_char_p, c_int]
 sqlite.sqlite3_result_text.argtypes = [c_void_p, c_char_p, c_int, c_void_p]
 
+# new
+# ===================================
+
+# safe
+
+sqlite.sqlite3_libversion.argtypes = []
+sqlite.sqlite3_column_blob.argtypes = [c_void_p, c_int]
+sqlite.sqlite3_column_bytes.argtypes = [c_void_p, c_int]
+sqlite.sqlite3_column_double.argtypes = [c_void_p, c_int]
+sqlite.sqlite3_column_int64.argtypes = [c_void_p, c_int]
+sqlite.sqlite3_column_name.argtypes = [c_void_p, c_int]
+sqlite.sqlite3_column_text.argtypes = [c_void_p, c_int]
+sqlite.sqlite3_errcode.argtypes = [c_void_p]
+sqlite.sqlite3_errmsg.argtypes = [c_void_p]
+sqlite.sqlite3_column_count.argtypes = [c_void_p]
+sqlite.sqlite3_column_count.restype = c_int
+sqlite.sqlite3_reset.argtypes = [c_void_p]
+sqlite.sqlite3_reset.restype = c_int
+
+# unsafe
+
+
+sqlite.sqlite3_step.argtypes = [c_void_p]
+sqlite.sqlite3_step.restype = c_int
+sqlite.sqlite3_finalize.argtypes = [c_void_p]
+sqlite.sqlite3_finalize.restype = c_int
+
+## sqlite.sqlite3_column_type.argtypes = [c_void_p, c_int]
+## sqlite.sqlite3_column_type.restype = c_int
+## sqlite.sqlite3_total_changes.argtypes = [c_void_p]
+## sqlite.sqlite3_total_changes.restype = c_int
+## sqlite.sqlite3_bind_null.argtypes = [c_void_p, c_int]
+## sqlite.sqlite3_bind_null.restype = c_int
+## sqlite.sqlite3_busy_timeout.argtypes = [c_void_p, c_int]
+## sqlite.sqlite3_busy_timeout.restype = c_int
+## sqlite.sqlite3_value_type.argtypes = [c_void_p]
+## sqlite.sqlite3_value_type.restype = c_int
+
+
+
 ##########################################
 # END Wrapped SQLite C API and constants
 ##########################################
@@ -369,9 +409,13 @@
                 sql = "BEGIN " + self._isolation_level
                 statement = c_void_p()
                 next_char = c_char_p()
-                ret = sqlite.sqlite3_prepare_v2(self.db, sql, -1, byref(statement), next_char)
+                psql = c_char_p(sql)
+                ret = sqlite.sqlite3_prepare_v2(self.db, psql, -1, byref(statement), next_char)
                 if ret != SQLITE_OK:
-                    raise self._get_exception(ret)
+                    xx = self._get_exception(ret)
+                    if 'syntax error' in str(xx):
+                        import pdb;pdb.set_trace()
+                    raise xx
                 ret = sqlite.sqlite3_step(statement)
                 if ret != SQLITE_DONE:
                     raise self._get_exception(ret)
@@ -387,9 +431,13 @@
             sql = "COMMIT"
             statement = c_void_p()
             next_char = c_char_p()
-            ret = sqlite.sqlite3_prepare_v2(self.db, sql, -1, byref(statement), next_char)
+            psql = c_char_p(sql)
+            ret = sqlite.sqlite3_prepare_v2(self.db, psql, -1, byref(statement), next_char)
             if ret != SQLITE_OK:
-                raise self._get_exception(ret)
+                xx = self._get_exception(ret)
+                if 'syntax error' in str(xx) or 'token' in str(xx):
+                    import pdb;pdb.set_trace()
+                raise xx
             ret = sqlite.sqlite3_step(statement)
             if ret != SQLITE_DONE:
                 raise self._get_exception(ret)
@@ -630,14 +678,18 @@
 
         self.statement = c_void_p()
         next_char = c_char_p()
-        ret = sqlite.sqlite3_prepare_v2(self.con.db, sql, -1, byref(self.statement), byref(next_char))
+        psql = c_char_p(sql)
+        ret = sqlite.sqlite3_prepare_v2(self.con.db, psql, -1, byref(self.statement), byref(next_char))
         if ret == SQLITE_OK and self.statement.value is None:
             # an empty statement, we work around that, as it's the least trouble
             ret = sqlite.sqlite3_prepare_v2(self.con.db, "select 42", -1, byref(self.statement), byref(next_char))
             self.kind = "DQL"
 
         if ret != SQLITE_OK:
-            raise self.con._get_exception(ret)
+            xx =  self.con._get_exception(ret)
+            if 'syntax error' in str(xx):
+                import pdb;pdb.set_trace()
+            raise xx
         self.con._remember_statement(self)
         if _check_remaining_sql(next_char.value):
             raise Warning, "One and only one statement required"


From commits-noreply at bitbucket.org  Fri Feb  4 13:42:39 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Fri,  4 Feb 2011 13:42:39 +0100 (CET)
Subject: [pypy-svn] pypy jitypes2: revert 96964ba376b4,
	and fix the test in another way: it's better to use
	logger_noopt	because this way the name of the variables are
	preserved
Message-ID: <20110204124239.4F56B2A2002@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: jitypes2
Changeset: r41610:5fbf8ac37f9e
Date: 2011-02-04 13:42 +0100
http://bitbucket.org/pypy/pypy/changeset/5fbf8ac37f9e/

Log:	revert 96964ba376b4, and fix the test in another way: it's better to
	use logger_noopt because this way the name of the variables are
	preserved

diff --git a/pypy/jit/metainterp/optimizeopt/fficall.py b/pypy/jit/metainterp/optimizeopt/fficall.py
--- a/pypy/jit/metainterp/optimizeopt/fficall.py
+++ b/pypy/jit/metainterp/optimizeopt/fficall.py
@@ -73,7 +73,7 @@
 
     def setup(self):
         self.funcinfo = None
-        self.logger = self.optimizer.metainterp_sd.logger_ops
+        self.logger = self.optimizer.metainterp_sd.logger_noopt
 
     def propagate_begin_forward(self):
         debug_start('jit-log-ffiopt')

diff --git a/pypy/jit/metainterp/test/test_optimizebasic.py b/pypy/jit/metainterp/test/test_optimizebasic.py
--- a/pypy/jit/metainterp/test/test_optimizebasic.py
+++ b/pypy/jit/metainterp/test/test_optimizebasic.py
@@ -33,6 +33,7 @@
         self.options = Fake()
         self.globaldata = Fake()
         self.logger_ops = FakeLogger()
+        self.logger_noopt = FakeLogger()
 
 def test_store_final_boxes_in_guard():
     from pypy.jit.metainterp.compile import ResumeGuardDescr

diff --git a/.hgsubstate b/.hgsubstate
--- a/.hgsubstate
+++ b/.hgsubstate
@@ -1,4 +1,4 @@
 80037 greenlet
 80037 lib_pypy/pyrepl
-80037 lib_pypy/sqlite3
+80282 lib_pypy/sqlite3
 80037 testrunner

From antocuni at codespeak.net  Fri Feb  4 13:44:14 2011
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Fri,  4 Feb 2011 13:44:14 +0100 (CET)
Subject: [pypy-svn] r80283 - pypy/pysqlite2
Message-ID: <20110204124414.529552A2002@codespeak.net>

Author: antocuni
Date: Fri Feb  4 13:44:12 2011
New Revision: 80283

Modified:
   pypy/pysqlite2/dbapi2.py
Log:
revert r80282, I checked it in by mistake

Modified: pypy/pysqlite2/dbapi2.py
==============================================================================
--- pypy/pysqlite2/dbapi2.py	(original)
+++ pypy/pysqlite2/dbapi2.py	Fri Feb  4 13:44:12 2011
@@ -205,46 +205,6 @@
 sqlite.sqlite3_result_error.argtypes = [c_void_p, c_char_p, c_int]
 sqlite.sqlite3_result_text.argtypes = [c_void_p, c_char_p, c_int, c_void_p]
 
-# new
-# ===================================
-
-# safe
-
-sqlite.sqlite3_libversion.argtypes = []
-sqlite.sqlite3_column_blob.argtypes = [c_void_p, c_int]
-sqlite.sqlite3_column_bytes.argtypes = [c_void_p, c_int]
-sqlite.sqlite3_column_double.argtypes = [c_void_p, c_int]
-sqlite.sqlite3_column_int64.argtypes = [c_void_p, c_int]
-sqlite.sqlite3_column_name.argtypes = [c_void_p, c_int]
-sqlite.sqlite3_column_text.argtypes = [c_void_p, c_int]
-sqlite.sqlite3_errcode.argtypes = [c_void_p]
-sqlite.sqlite3_errmsg.argtypes = [c_void_p]
-sqlite.sqlite3_column_count.argtypes = [c_void_p]
-sqlite.sqlite3_column_count.restype = c_int
-sqlite.sqlite3_reset.argtypes = [c_void_p]
-sqlite.sqlite3_reset.restype = c_int
-
-# unsafe
-
-
-sqlite.sqlite3_step.argtypes = [c_void_p]
-sqlite.sqlite3_step.restype = c_int
-sqlite.sqlite3_finalize.argtypes = [c_void_p]
-sqlite.sqlite3_finalize.restype = c_int
-
-## sqlite.sqlite3_column_type.argtypes = [c_void_p, c_int]
-## sqlite.sqlite3_column_type.restype = c_int
-## sqlite.sqlite3_total_changes.argtypes = [c_void_p]
-## sqlite.sqlite3_total_changes.restype = c_int
-## sqlite.sqlite3_bind_null.argtypes = [c_void_p, c_int]
-## sqlite.sqlite3_bind_null.restype = c_int
-## sqlite.sqlite3_busy_timeout.argtypes = [c_void_p, c_int]
-## sqlite.sqlite3_busy_timeout.restype = c_int
-## sqlite.sqlite3_value_type.argtypes = [c_void_p]
-## sqlite.sqlite3_value_type.restype = c_int
-
-
-
 ##########################################
 # END Wrapped SQLite C API and constants
 ##########################################
@@ -409,13 +369,9 @@
                 sql = "BEGIN " + self._isolation_level
                 statement = c_void_p()
                 next_char = c_char_p()
-                psql = c_char_p(sql)
-                ret = sqlite.sqlite3_prepare_v2(self.db, psql, -1, byref(statement), next_char)
+                ret = sqlite.sqlite3_prepare_v2(self.db, sql, -1, byref(statement), next_char)
                 if ret != SQLITE_OK:
-                    xx = self._get_exception(ret)
-                    if 'syntax error' in str(xx):
-                        import pdb;pdb.set_trace()
-                    raise xx
+                    raise self._get_exception(ret)
                 ret = sqlite.sqlite3_step(statement)
                 if ret != SQLITE_DONE:
                     raise self._get_exception(ret)
@@ -431,13 +387,9 @@
             sql = "COMMIT"
             statement = c_void_p()
             next_char = c_char_p()
-            psql = c_char_p(sql)
-            ret = sqlite.sqlite3_prepare_v2(self.db, psql, -1, byref(statement), next_char)
+            ret = sqlite.sqlite3_prepare_v2(self.db, sql, -1, byref(statement), next_char)
             if ret != SQLITE_OK:
-                xx = self._get_exception(ret)
-                if 'syntax error' in str(xx) or 'token' in str(xx):
-                    import pdb;pdb.set_trace()
-                raise xx
+                raise self._get_exception(ret)
             ret = sqlite.sqlite3_step(statement)
             if ret != SQLITE_DONE:
                 raise self._get_exception(ret)
@@ -678,18 +630,14 @@
 
         self.statement = c_void_p()
         next_char = c_char_p()
-        psql = c_char_p(sql)
-        ret = sqlite.sqlite3_prepare_v2(self.con.db, psql, -1, byref(self.statement), byref(next_char))
+        ret = sqlite.sqlite3_prepare_v2(self.con.db, sql, -1, byref(self.statement), byref(next_char))
         if ret == SQLITE_OK and self.statement.value is None:
             # an empty statement, we work around that, as it's the least trouble
             ret = sqlite.sqlite3_prepare_v2(self.con.db, "select 42", -1, byref(self.statement), byref(next_char))
             self.kind = "DQL"
 
         if ret != SQLITE_OK:
-            xx =  self.con._get_exception(ret)
-            if 'syntax error' in str(xx):
-                import pdb;pdb.set_trace()
-            raise xx
+            raise self.con._get_exception(ret)
         self.con._remember_statement(self)
         if _check_remaining_sql(next_char.value):
             raise Warning, "One and only one statement required"


From commits-noreply at bitbucket.org  Fri Feb  4 13:56:18 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Fri,  4 Feb 2011 13:56:18 +0100 (CET)
Subject: [pypy-svn] pypy jitypes2: update to the latest svn revision
Message-ID: <20110204125618.C75B02A2002@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: jitypes2
Changeset: r41611:dd7f53f3fcb7
Date: 2011-02-04 13:56 +0100
http://bitbucket.org/pypy/pypy/changeset/dd7f53f3fcb7/

Log:	update to the latest svn revision


From fijal at codespeak.net  Fri Feb  4 14:06:13 2011
From: fijal at codespeak.net (fijal at codespeak.net)
Date: Fri,  4 Feb 2011 14:06:13 +0100 (CET)
Subject: [pypy-svn] r80284 - pypy/extradoc/planning
Message-ID: <20110204130613.9262B2A2002@codespeak.net>

Author: fijal
Date: Fri Feb  4 14:06:12 2011
New Revision: 80284

Modified:
   pypy/extradoc/planning/jit.txt
Log:
typo

Modified: pypy/extradoc/planning/jit.txt
==============================================================================
--- pypy/extradoc/planning/jit.txt	(original)
+++ pypy/extradoc/planning/jit.txt	Fri Feb  4 14:06:12 2011
@@ -3,7 +3,7 @@
 
 * 25% slowdown on pyflate fast (Jan 29)
   - pyflate_fast uses python longs on 32bit
-  - some places don't create SmallLongs even when they should (like conts)
+  - some places don't create SmallLongs even when they should (like consts)
   - we end up with comparison of Longs and SmallLongs
 
 * 10% slowdown on spitfire (Feb 01)


From commits-noreply at bitbucket.org  Fri Feb  4 17:40:16 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Fri,  4 Feb 2011 17:40:16 +0100 (CET)
Subject: [pypy-svn] pypy default: Try not to crash for non-thread
	translations.
Message-ID: <20110204164016.1DF132A2002@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41612:9d323f73fa71
Date: 2011-02-04 17:38 +0100
http://bitbucket.org/pypy/pypy/changeset/9d323f73fa71/

Log:	Try not to crash for non-thread translations.

diff --git a/pypy/rpython/memory/gctransform/framework.py b/pypy/rpython/memory/gctransform/framework.py
--- a/pypy/rpython/memory/gctransform/framework.py
+++ b/pypy/rpython/memory/gctransform/framework.py
@@ -966,8 +966,8 @@
             hop.genop("direct_call", [self.root_walker.thread_die_ptr])
 
     def gct_gc_thread_before_fork(self, hop):
-        assert self.translator.config.translation.thread
-        if hasattr(self.root_walker, 'thread_before_fork_ptr'):
+        if (self.translator.config.translation.thread
+            and hasattr(self.root_walker, 'thread_before_fork_ptr')):
             hop.genop("direct_call", [self.root_walker.thread_before_fork_ptr],
                       resultvar=hop.spaceop.result)
         else:
@@ -976,8 +976,8 @@
                       resultvar=hop.spaceop.result)
 
     def gct_gc_thread_after_fork(self, hop):
-        assert self.translator.config.translation.thread
-        if hasattr(self.root_walker, 'thread_after_fork_ptr'):
+        if (self.translator.config.translation.thread
+            and hasattr(self.root_walker, 'thread_after_fork_ptr')):
             hop.genop("direct_call", [self.root_walker.thread_after_fork_ptr]
                                      + hop.spaceop.args)
 

From commits-noreply at bitbucket.org  Fri Feb  4 17:40:16 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Fri,  4 Feb 2011 17:40:16 +0100 (CET)
Subject: [pypy-svn] pypy default: merge heads
Message-ID: <20110204164016.6BFE12A2006@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41613:748a2a99f14f
Date: 2011-02-04 17:39 +0100
http://bitbucket.org/pypy/pypy/changeset/748a2a99f14f/

Log:	merge heads


From commits-noreply at bitbucket.org  Fri Feb  4 18:57:23 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Fri,  4 Feb 2011 18:57:23 +0100 (CET)
Subject: [pypy-svn] pypy default: Implemented StringIO.__{get, set}setate__.
Message-ID: <20110204175723.DC14B2A2002@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41614:5a4e944400d5
Date: 2011-02-04 12:41 -0500
http://bitbucket.org/pypy/pypy/changeset/5a4e944400d5/

Log:	Implemented StringIO.__{get,set}setate__.

diff --git a/pypy/module/_io/test/test_stringio.py b/pypy/module/_io/test/test_stringio.py
--- a/pypy/module/_io/test/test_stringio.py
+++ b/pypy/module/_io/test/test_stringio.py
@@ -264,3 +264,33 @@
         sio = io.StringIO(s * 2)
         sio.close()
         raises(ValueError, next, sio)
+
+    def test_getstate(self):
+        import io
+
+        sio = io.StringIO()
+        state = sio.__getstate__()
+        assert len(state) == 4
+        assert isinstance(state[0], unicode)
+        assert isinstance(state[1], str)
+        assert isinstance(state[2], int)
+        assert isinstance(state[3], dict)
+        sio.close()
+        raises(ValueError, sio.__getstate__)
+
+    def test_setstate(self):
+        import io
+
+        sio = io.StringIO()
+        sio.__setstate__((u"no error", u"\n", 0, None))
+        sio.__setstate__((u"no error", u"", 0, {"spam": 3}))
+        raises(ValueError, sio.__setstate__, (u"", u"f", 0, None))
+        raises(ValueError, sio.__setstate__, (u"", u"", -1, None))
+        raises(TypeError, sio.__setstate__, ("", u"", 0, None))
+        raises(TypeError, sio.__setstate__, (u"", u"", 0.0, None))
+        raises(TypeError, sio.__setstate__, (u"", u"", 0, 0))
+        raises(TypeError, sio.__setstate__, (u"len-test", 0))
+        raises(TypeError, sio.__setstate__)
+        raises(TypeError, sio.__setstate__, 0)
+        sio.close()
+        raises(ValueError, sio.__setstate__, (u"closed", u"", 0, None))

diff --git a/pypy/module/_io/interp_stringio.py b/pypy/module/_io/interp_stringio.py
--- a/pypy/module/_io/interp_stringio.py
+++ b/pypy/module/_io/interp_stringio.py
@@ -53,6 +53,61 @@
             self.write_w(space, w_initvalue)
             self.pos = 0
 
+    @unwrap_spec('self', ObjSpace)
+    def descr_getstate(self, space):
+        w_initialval = self.getvalue_w(space)
+        w_dict = space.call_method(self.w_dict, "copy")
+        if self.readnl is None:
+            w_readnl = space.w_None
+        else:
+            w_readnl = space.str(space.wrap(self.readnl))
+        return space.newtuple([
+            w_initialval, w_readnl, space.wrap(self.pos), w_dict
+        ])
+
+    @unwrap_spec('self', ObjSpace, W_Root)
+    def descr_setstate(self, space, w_state):
+        self._check_closed(space)
+
+        # We allow the state tuple to be longer than 4, because we may need
+        # someday to extend the object's state without breaking
+        # backwards-compatibility
+        if not space.isinstance_w(w_state, space.w_tuple) or space.int_w(space.len(w_state)) < 4:
+            raise operationerrfmt(space.w_TypeError,
+                "%s.__setstate__ argument should be a 4-tuple, got %s",
+                space.type(self).getname(space),
+                space.type(w_state).getname(space)
+            )
+        w_initval, w_readnl, w_pos, w_dict = space.unpackiterable(w_state, 4)
+        # Initialize state
+        self.descr_init(space, w_initval, w_readnl)
+
+        # Restore the buffer state. Even if __init__ did initialize the buffer,
+        # we have to initialize it again since __init__ may translates the
+        # newlines in the inital_value string. We clearly do not want that
+        # because the string value in the state tuple has already been
+        # translated once by __init__. So we do not take any chance and replace
+        # object's buffer completely
+        initval = space.unicode_w(w_initval)
+        size = len(initval)
+        self.resize_buffer(size)
+        self.buf = list(initval)
+        pos = space.getindex_w(w_pos, space.w_TypeError)
+        if pos < 0:
+            raise OperationError(space.w_ValueError,
+                space.wrap("position value cannot be negative")
+            )
+        self.pos = pos
+        if not space.is_w(w_dict, space.w_None):
+            if not space.isinstance_w(w_dict, space.w_dict):
+                raise operationerrfmt(space.w_TypeError,
+                    "fourth item of state should be a dict, got a %s",
+                    space.type(w_dict).getname(space)
+                )
+            # Alternatively, we could replace the internal dictionary
+            # completely. However, it seems more practical to just update it.
+            space.call_method(self.w_dict, "update", w_dict)
+
     def _check_closed(self, space, message=None):
         if self.buf is None:
             if message is None:
@@ -230,6 +285,8 @@
     __module__ = "_io",
     __new__  = generic_new_descr(W_StringIO),
     __init__ = interp2app(W_StringIO.descr_init),
+    __getstate__ = interp2app(W_StringIO.descr_getstate),
+    __setstate__ = interp2app(W_StringIO.descr_setstate),
     write = interp2app(W_StringIO.write_w),
     read = interp2app(W_StringIO.read_w),
     readline = interp2app(W_StringIO.readline_w),

From commits-noreply at bitbucket.org  Fri Feb  4 18:57:24 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Fri,  4 Feb 2011 18:57:24 +0100 (CET)
Subject: [pypy-svn] pypy default: Make sure BytesIO.__{get,
	set}state__ raise the right exceptions.
Message-ID: <20110204175724.AAA8C2A2002@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41615:9d967a690db0
Date: 2011-02-04 12:52 -0500
http://bitbucket.org/pypy/pypy/changeset/9d967a690db0/

Log:	Make sure BytesIO.__{get,set}state__ raise the right exceptions.

diff --git a/pypy/module/_io/test/test_bytesio.py b/pypy/module/_io/test/test_bytesio.py
--- a/pypy/module/_io/test/test_bytesio.py
+++ b/pypy/module/_io/test/test_bytesio.py
@@ -67,6 +67,10 @@
         assert f.read() == "ld"
         assert f.a == 1
         assert f.__getstate__() == ("world", 5, {"a": 1})
+        raises(TypeError, f.__setstate__, ("", 0))
+        f.close()
+        raises(ValueError, f.__getstate__)
+        raises(ValueError, f.__setstate__, ("world", 3, {"a": 1}))
 
     def test_readinto(self):
         import _io

diff --git a/pypy/module/_io/interp_bytesio.py b/pypy/module/_io/interp_bytesio.py
--- a/pypy/module/_io/interp_bytesio.py
+++ b/pypy/module/_io/interp_bytesio.py
@@ -189,6 +189,7 @@
 
     @unwrap_spec('self', ObjSpace)
     def getstate_w(self, space):
+        self._check_closed(space)
         w_content = space.wrap(buffer2string(self.buf, 0, self.string_size))
         return space.newtuple([
             w_content,
@@ -197,6 +198,14 @@
 
     @unwrap_spec('self', ObjSpace, W_Root)
     def setstate_w(self, space, w_state):
+        self._check_closed(space)
+
+        if space.int_w(space.len(w_state)) != 3:
+            raise operationerrfmt(space.w_TypeError,
+                "%s.__setstate__ argument should be 3-tuple, got %s",
+                space.type(self).getname(space),
+                space.type(w_state).getname(space)
+            )
         w_content, w_pos, w_dict = space.unpackiterable(w_state, 3)
         pos = space.int_w(w_pos)
         self.buf = []

From commits-noreply at bitbucket.org  Fri Feb  4 18:57:24 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Fri,  4 Feb 2011 18:57:24 +0100 (CET)
Subject: [pypy-svn] pypy default: Merged upstream.
Message-ID: <20110204175724.008732A2006@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41616:66f379bffeb3
Date: 2011-02-04 12:57 -0500
http://bitbucket.org/pypy/pypy/changeset/66f379bffeb3/

Log:	Merged upstream.


From commits-noreply at bitbucket.org  Fri Feb  4 19:16:27 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Fri,  4 Feb 2011 19:16:27 +0100 (CET)
Subject: [pypy-svn] pypy default: Added space.len_w which is a shortcut
	for	space.int_w(space.len(w_obj))
Message-ID: <20110204181627.C3FA82A2002@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41617:a5ead4a17c44
Date: 2011-02-04 13:16 -0500
http://bitbucket.org/pypy/pypy/changeset/a5ead4a17c44/

Log:	Added space.len_w which is a shortcut for
	space.int_w(space.len(w_obj))

diff --git a/pypy/module/cpyext/sequence.py b/pypy/module/cpyext/sequence.py
--- a/pypy/module/cpyext/sequence.py
+++ b/pypy/module/cpyext/sequence.py
@@ -30,11 +30,11 @@
     Returns the number of objects in sequence o on success, and -1 on failure.
     For objects that do not provide sequence protocol, this is equivalent to the
     Python expression len(o)."""
-    return space.int_w(space.len(w_obj))
+    return space.len_w(w_obj)
 
 @cpython_api([PyObject], Py_ssize_t, error=-1)
 def PySequence_Length(space, w_obj):
-    return space.int_w(space.len(w_obj))
+    return space.len_w(w_obj)
 
 
 @cpython_api([PyObject, CONST_STRING], PyObject)
@@ -142,7 +142,7 @@
     """Assign object v to the ith element of o.  Returns -1 on failure.  This
     is the equivalent of the Python statement o[i] = v.  This function does
     not steal a reference to v.
-    
+
     This function used an int type for i. This might require
     changes in your code for properly supporting 64-bit systems."""
     if PyDict_Check(space, w_o) or not PySequence_Check(space, w_o):
@@ -155,7 +155,7 @@
 def PySequence_DelItem(space, w_o, i):
     """Delete the ith element of object o.  Returns -1 on failure.  This is the
     equivalent of the Python statement del o[i].
-    
+
     This function used an int type for i. This might require
     changes in your code for properly supporting 64-bit systems."""
     space.delitem(w_o, space.wrap(i))

diff --git a/pypy/module/cpyext/typeobject.py b/pypy/module/cpyext/typeobject.py
--- a/pypy/module/cpyext/typeobject.py
+++ b/pypy/module/cpyext/typeobject.py
@@ -356,7 +356,7 @@
              error=CANNOT_FAIL)
 def str_segcount(space, w_obj, ref):
     if ref:
-        ref[0] = rffi.cast(rffi.INT, space.int_w(space.len(w_obj)))
+        ref[0] = rffi.cast(rffi.INT, space.len_w(w_obj))
     return 1
 
 @cpython_api([PyObject, lltype.Signed, rffi.VOIDPP], lltype.Signed,
@@ -370,7 +370,7 @@
     ref[0] = PyString_AsString(space, pyref)
     # Stolen reference: the object has better exist somewhere else
     Py_DecRef(space, pyref)
-    return space.int_w(space.len(w_str))
+    return space.len_w(w_str)
 
 def setup_string_buffer_procs(space, pto):
     c_buf = lltype.malloc(PyBufferProcs, flavor='raw', zero=True)
@@ -617,6 +617,3 @@
     name = space.str_w(w_name)
     w_obj = w_type.lookup(name)
     return borrow_from(w_type, w_obj)
-
-
-

diff --git a/pypy/module/cpyext/object.py b/pypy/module/cpyext/object.py
--- a/pypy/module/cpyext/object.py
+++ b/pypy/module/cpyext/object.py
@@ -156,7 +156,7 @@
 
 @cpython_api([PyObject], Py_ssize_t, error=-1)
 def PyObject_Size(space, w_obj):
-    return space.int_w(space.len(w_obj))
+    return space.len_w(w_obj)
 
 @cpython_api([PyObject], rffi.INT_real, error=CANNOT_FAIL)
 def PyCallable_Check(space, w_obj):
@@ -420,7 +420,7 @@
     else:
         w_str = space.repr(w_obj)
 
-    count = space.int_w(space.len(w_str))
+    count = space.len_w(w_str)
     data = space.str_w(w_str)
     buf = rffi.get_nonmovingbuffer(data)
     try:

diff --git a/pypy/module/cpyext/dictobject.py b/pypy/module/cpyext/dictobject.py
--- a/pypy/module/cpyext/dictobject.py
+++ b/pypy/module/cpyext/dictobject.py
@@ -65,7 +65,7 @@
     """
     Return the number of items in the dictionary.  This is equivalent to
     len(p) on a dictionary."""
-    return space.int_w(space.len(w_obj))
+    return space.len_w(w_obj)
 
 @cpython_api([PyObject, PyObject], rffi.INT_real, error=-1)
 def PyDict_Contains(space, w_obj, w_value):
@@ -125,25 +125,25 @@
     them are borrowed.  ppos should not be altered during iteration. Its
     value represents offsets within the internal dictionary structure, and
     since the structure is sparse, the offsets are not consecutive.
-    
+
     For example:
-    
+
     PyObject *key, *value;
     Py_ssize_t pos = 0;
-    
+
     while (PyDict_Next(self->dict, &pos, &key, &value)) {
         /* do something interesting with the values... */
         ...
     }
-    
+
     The dictionary p should not be mutated during iteration.  It is safe
     (since Python 2.1) to modify the values of the keys as you iterate over the
     dictionary, but only so long as the set of keys does not change.  For
     example:
-    
+
     PyObject *key, *value;
     Py_ssize_t pos = 0;
-    
+
     while (PyDict_Next(self->dict, &pos, &key, &value)) {
         int i = PyInt_AS_LONG(value) + 1;
         PyObject *o = PyInt_FromLong(i);
@@ -180,5 +180,3 @@
             raise
         return 0
     return 1
-
-

diff --git a/pypy/module/__builtin__/interp_classobj.py b/pypy/module/__builtin__/interp_classobj.py
--- a/pypy/module/__builtin__/interp_classobj.py
+++ b/pypy/module/__builtin__/interp_classobj.py
@@ -464,7 +464,7 @@
                 w_as_str = self.descr_unicode(space)
             else:
                 w_as_str = self.descr_str(space)
-            if space.int_w(space.len(w_format_spec)) > 0:
+            if space.len_w(w_format_spec) > 0:
                 space.warn(
                     ("object.__format__ with a non-empty format string is "
                         "deprecated"),

diff --git a/pypy/module/_codecs/interp_codecs.py b/pypy/module/_codecs/interp_codecs.py
--- a/pypy/module/_codecs/interp_codecs.py
+++ b/pypy/module/_codecs/interp_codecs.py
@@ -19,7 +19,7 @@
     def make_errorhandler(self, space, decode):
         def unicode_call_errorhandler(errors,  encoding, reason, input,
                                       startpos, endpos):
-            
+
             w_errorhandler = lookup_error(space, errors)
             if decode:
                 w_cls = space.w_UnicodeDecodeError
@@ -34,7 +34,7 @@
                 space.wrap(reason))
             w_res = space.call_function(w_errorhandler, w_exc)
             if (not space.is_true(space.isinstance(w_res, space.w_tuple))
-                or space.int_w(space.len(w_res)) != 2):
+                or space.len_w(w_res) != 2):
                 raise operationerrfmt(
                     space.w_TypeError,
                     "encoding error handler must return "
@@ -77,7 +77,7 @@
 
 def register_codec(space, w_search_function):
     """register(search_function)
-    
+
     Register a codec search function. Search functions are expected to take
     one argument, the encoding name in all lower case letters, and return
     a tuple of functions (encoder, decoder, stream_reader, stream_writer).
@@ -100,7 +100,7 @@
     assert not (space.config.translating and not we_are_translated()), \
         "lookup_codec() should not be called during translation"
     state = space.fromcache(CodecState)
-    normalized_encoding = encoding.replace(" ", "-").lower()    
+    normalized_encoding = encoding.replace(" ", "-").lower()
     w_result = state.codec_search_cache.get(normalized_encoding, None)
     if w_result is not None:
         return w_result
@@ -118,14 +118,14 @@
         w_result = space.call_function(w_search,
                                        space.wrap(normalized_encoding))
         if not space.is_w(w_result, space.w_None):
-            if not (space.is_true(space.isinstance(w_result,  
+            if not (space.is_true(space.isinstance(w_result,
                                             space.w_tuple)) and
-                    space.int_w(space.len(w_result)) == 4):
+                    space.len_w(w_result) == 4):
                 raise OperationError(
                     space.w_TypeError,
                     space.wrap("codec search functions must return 4-tuples"))
             else:
-                state.codec_search_cache[normalized_encoding] = w_result 
+                state.codec_search_cache[normalized_encoding] = w_result
                 return w_result
     raise operationerrfmt(
         space.w_LookupError,
@@ -259,7 +259,7 @@
     Return the error handler for the specified error handling name
     or raise a LookupError, if no handler exists under this name.
     """
-    
+
     state = space.fromcache(CodecState)
     try:
         w_err_handler = state.codec_error_registry[errors]
@@ -273,7 +273,7 @@
 
 def encode(space, w_obj, w_encoding=NoneNotWrapped, errors='strict'):
     """encode(obj, [encoding[,errors]]) -> object
-    
+
     Encodes obj using the codec registered for encoding. encoding defaults
     to the default encoding. errors may be given to set a different error
     handling scheme. Default is 'strict' meaning that encoding errors raise
@@ -312,7 +312,7 @@
     if space.is_true(w_decoder):
         w_res = space.call_function(w_decoder, w_obj, space.wrap(errors))
         if (not space.is_true(space.isinstance(w_res, space.w_tuple))
-            or space.int_w(space.len(w_res)) != 2):
+            or space.len_w(w_res) != 2):
             raise OperationError(
                 space.w_TypeError,
                 space.wrap("encoder must return a tuple (object, integer)"))

diff --git a/pypy/module/unicodedata/interp_ucd.py b/pypy/module/unicodedata/interp_ucd.py
--- a/pypy/module/unicodedata/interp_ucd.py
+++ b/pypy/module/unicodedata/interp_ucd.py
@@ -65,7 +65,7 @@
                 raise OperationError(space.w_TypeError, space.wrap(
                     'need a single Unicode character as parameter'))
         else:
-            if not space.int_w(space.len(w_unichr)) == 1:
+            if not space.len_w(w_unichr) == 1:
                 raise OperationError(space.w_TypeError, space.wrap(
                     'need a single Unicode character as parameter'))
             return space.int_w(space.ord(w_unichr))
@@ -85,7 +85,7 @@
 
         if not we_are_translated() and sys.maxunicode > 0xFFFF:
             # Host CPython is wide build, forbid surrogates
-            if not space.int_w(space.len(w_unichr)) == 1:
+            if not space.len_w(w_unichr) == 1:
                 raise OperationError(space.w_TypeError, space.wrap(
                     'need a single Unicode character as parameter'))
             return space.int_w(space.ord(w_unichr))
@@ -119,9 +119,9 @@
         self._canon_decomposition = unicodedb.canon_decomposition
         self._compat_decomposition = unicodedb.compat_decomposition
         self._composition = unicodedb._composition
-        
+
         self.version = unicodedb.version
-        
+
     def _get_code(self, space, name):
         try:
             code = self._lookup(name.upper())
@@ -130,7 +130,7 @@
             raise OperationError(space.w_KeyError, msg)
         return space.wrap(code)
     _get_code.unwrap_spec = ['self', ObjSpace, str]
-    
+
     def lookup(self, space, name):
         try:
             code = self._lookup(name.upper())
@@ -226,7 +226,7 @@
         elif form == 'NFD':
             composed = False
             decomposition = self._canon_decomposition
-        elif form == 'NFKC': 
+        elif form == 'NFKC':
             composed = True
             decomposition = self._compat_decomposition
         elif form == 'NFKD':
@@ -236,7 +236,7 @@
             raise OperationError(space.w_ValueError,
                                  space.wrap('invalid normalization form'))
 
-        strlen = space.int_w(space.len(w_unistr))
+        strlen = space.len_w(w_unistr)
         result = [0] * (strlen + strlen / 10 + 10)
         j = 0
         resultlen = len(result)
@@ -350,7 +350,7 @@
 
         return space.wrap(u''.join([unichr(i) for i in result[:next_insert]]))
     normalize.unwrap_spec = ['self', ObjSpace, str, W_Root]
-    
+
 
 methods = {}
 for methodname in UCD.__dict__:
@@ -359,7 +359,7 @@
         continue
     assert method.im_func.func_code.co_argcount == len(method.unwrap_spec), methodname
     methods[methodname] = interp2app(method, unwrap_spec=method.unwrap_spec)
-    
+
 
 UCD.typedef = TypeDef("unicodedata.UCD",
                       __doc__ = "",
@@ -369,4 +369,3 @@
 ucd_3_2_0 = UCD(unicodedb_3_2_0)
 ucd_5_2_0 = UCD(unicodedb_5_2_0)
 ucd = ucd_5_2_0
-

diff --git a/pypy/module/_warnings/interp_warnings.py b/pypy/module/_warnings/interp_warnings.py
--- a/pypy/module/_warnings/interp_warnings.py
+++ b/pypy/module/_warnings/interp_warnings.py
@@ -107,7 +107,7 @@
             raise
         if space.str_w(w_module) == '__main__':
             w_argv = space.sys.getdictvalue(space, 'argv')
-            if w_argv and space.int_w(space.len(w_argv)) > 0:
+            if w_argv and space.len_w(w_argv) > 0:
                 w_filename = space.getitem(w_argv, space.wrap(0))
                 if not space.is_true(w_filename):
                     w_filename = space.wrap('__main__')

diff --git a/pypy/module/exceptions/interp_exceptions.py b/pypy/module/exceptions/interp_exceptions.py
--- a/pypy/module/exceptions/interp_exceptions.py
+++ b/pypy/module/exceptions/interp_exceptions.py
@@ -91,7 +91,7 @@
     """Superclass representing the base of the exception hierarchy.
 
     The __getitem__ method is provided for backwards-compatibility
-    and will be deprecated at some point. 
+    and will be deprecated at some point.
     """
     w_dict = None
     args_w = []
@@ -291,7 +291,7 @@
     start = None
     end = None
     reason = None
-    
+
     def descr_init(self, space, w_object, w_start, w_end, w_reason):
         # typechecking
         space.realunicode_w(w_object)
@@ -346,7 +346,7 @@
     else:
         return space.str(space.newtuple(self.args_w))
 key_error_str.unwrap_spec = ['self', ObjSpace]
-    
+
 W_KeyError = _new_exception('KeyError', W_LookupError,
                             """Mapping key not found.""",
                             __str__ = key_error_str)
@@ -367,7 +367,7 @@
     def __init__(self, space):
         self.w_errno = space.w_None
         self.w_strerror = space.w_None
-        self.w_filename = space.w_None        
+        self.w_filename = space.w_None
         W_BaseException.__init__(self, space)
 
     def descr_init(self, space, args_w):
@@ -425,7 +425,7 @@
 
 class W_WindowsError(W_OSError):
     """MS-Windows OS system call failed."""
-    
+
     def __init__(self, space):
         self.w_winerror = space.w_None
         W_OSError.__init__(self, space)
@@ -557,7 +557,7 @@
     def descr_repr(self, space):
         if (len(self.args_w) == 2
             and not space.is_w(self.w_lastlineno, space.w_None)
-            and space.int_w(space.len(self.args_w[1])) == 4):
+            and space.len_w(self.args_w[1]) == 4):
             # fake a 5-element tuple in the repr, suitable for calling
             # __init__ again
             values_w = space.fixedview(self.args_w[1])
@@ -594,7 +594,7 @@
 
 class W_SystemExit(W_BaseException):
     """Request to exit from the interpreter."""
-    
+
     def __init__(self, space):
         self.w_code = space.w_None
         W_BaseException.__init__(self, space)

diff --git a/pypy/module/_winreg/interp_winreg.py b/pypy/module/_winreg/interp_winreg.py
--- a/pypy/module/_winreg/interp_winreg.py
+++ b/pypy/module/_winreg/interp_winreg.py
@@ -85,7 +85,7 @@
 the object is destroyed.  To guarantee cleanup, you can call either
 the Close() method on the PyHKEY, or the CloseKey() method.
 
-All functions which accept a handle object also accept an integer - 
+All functions which accept a handle object also accept an integer -
 however, use of the handle object is encouraged.
 
 Functions:
@@ -294,7 +294,7 @@
                 w_value = space.call_method(w_value, 'encode',
                                             space.wrap('mbcs'))
             buf = rffi.str2charp(space.str_w(w_value))
-            buflen = space.int_w(space.len(w_value)) + 1
+            buflen = space.len_w(w_value) + 1
 
     elif typ == rwinreg.REG_MULTI_SZ:
         if space.is_w(w_value, space.w_None):

diff --git a/pypy/interpreter/buffer.py b/pypy/interpreter/buffer.py
--- a/pypy/interpreter/buffer.py
+++ b/pypy/interpreter/buffer.py
@@ -254,7 +254,7 @@
 
     def getlength(self):
         space = self.space
-        return space.int_w(space.len(self.w_obj))
+        return space.len_w(self.w_obj)
 
     def getitem(self, index):
         space = self.space

diff --git a/pypy/module/oracle/interp_cursor.py b/pypy/module/oracle/interp_cursor.py
--- a/pypy/module/oracle/interp_cursor.py
+++ b/pypy/module/oracle/interp_cursor.py
@@ -196,7 +196,7 @@
     def _call(self, space, name, retvar, w_args):
         # determine the number of arguments passed
         if w_args:
-            numArguments = space.int_w(space.len(w_args))
+            numArguments = space.len_w(w_args)
         else:
             numArguments = 0
 
@@ -994,7 +994,7 @@
 
         # determine the number of elements to create
         if space.is_true(space.isinstance(w_value, space.w_list)):
-            numElements = space.int_w(space.len(w_value))
+            numElements = space.len_w(w_value)
         elif space.is_true(space.isinstance(w_value, space.w_int)):
             numElements = space.int_w(w_value)
         else:

diff --git a/pypy/module/_random/interp_random.py b/pypy/module/_random/interp_random.py
--- a/pypy/module/_random/interp_random.py
+++ b/pypy/module/_random/interp_random.py
@@ -65,7 +65,7 @@
         if not space.is_true(space.isinstance(w_state, space.w_tuple)):
             errstring = space.wrap("state vector must be tuple")
             raise OperationError(space.w_TypeError, errstring)
-        if space.int_w(space.len(w_state)) != rrandom.N + 1:
+        if space.len_w(w_state) != rrandom.N + 1:
             errstring = space.wrap("state vector is the wrong size")
             raise OperationError(space.w_ValueError, errstring)
         w_zero = space.newint(0)

diff --git a/pypy/module/cpyext/test/test_eval.py b/pypy/module/cpyext/test/test_eval.py
--- a/pypy/module/cpyext/test/test_eval.py
+++ b/pypy/module/cpyext/test/test_eval.py
@@ -22,7 +22,7 @@
         w_t = space.newtuple([space.wrap(1), space.wrap(2)])
         w_res = api.PyEval_CallObjectWithKeywords(w_f, w_t, None)
         assert space.int_w(w_res) == 2
-        assert space.int_w(space.len(w_l)) == 2
+        assert space.len_w(w_l) == 2
         w_f = space.appexec([], """():
             def f(*args, **kwds):
                 assert isinstance(kwds, dict)
@@ -35,7 +35,7 @@
         space.setitem(w_d, space.wrap("xyz"), space.wrap(3))
         w_res = api.PyEval_CallObjectWithKeywords(w_f, w_t, w_d)
         assert space.int_w(w_res) == 21
-    
+
     def test_call_object(self, space, api):
         w_l, w_f = space.fixedview(space.appexec([], """():
         l = []
@@ -49,8 +49,8 @@
         w_t = space.newtuple([space.wrap(1), space.wrap(2)])
         w_res = api.PyObject_CallObject(w_f, w_t)
         assert space.int_w(w_res) == 2
-        assert space.int_w(space.len(w_l)) == 2
-        
+        assert space.len_w(w_l) == 2
+
         w_f = space.appexec([], """():
             def f(*args):
                 assert isinstance(args, tuple)
@@ -60,7 +60,7 @@
 
         w_t = space.newtuple([space.wrap(1), space.wrap(2)])
         w_res = api.PyObject_CallObject(w_f, w_t)
-        
+
         assert space.int_w(w_res) == 10
 
     def test_run_string(self, space, api):
@@ -118,7 +118,7 @@
             d = dict(__builtins__={'len':len}, cpybuiltins=cpybuiltins)
             return eval("cpybuiltins()", d, d)
         """)
-        assert space.int_w(space.len(w_result)) == 1
+        assert space.len_w(w_result) == 1
 
     def test_getglobals(self, space, api):
         assert api.PyEval_GetLocals() is None

diff --git a/pypy/objspace/descroperation.py b/pypy/objspace/descroperation.py
--- a/pypy/objspace/descroperation.py
+++ b/pypy/objspace/descroperation.py
@@ -69,7 +69,7 @@
             return w_value
         if w_descr is not None:
             return space.get(w_descr, w_obj)
-        raiseattrerror(space, w_obj, name) 
+        raiseattrerror(space, w_obj, name)
 
     def descr__setattr__(space, w_obj, w_name, w_value):
         name = space.str_w(w_name)
@@ -357,7 +357,7 @@
                w_res = space.get_and_call_function(w_right_impl, w_obj2, w_obj1)
            else:
                w_res = space.get_and_call_function(w_right_impl, w_obj2, w_obj1,
-                                                   w_obj3)               
+                                                   w_obj3)
            if _check_notimplemented(space, w_res):
                return w_res
 
@@ -386,7 +386,7 @@
                 return space.w_False
             if space.eq_w(w_next, w_item):
                 return space.w_True
-    
+
     def hash(space, w_obj):
         w_hash = space.lookup(w_obj, '__hash__')
         if w_hash is None:
@@ -412,8 +412,8 @@
             # be careful about subclasses of 'long'...
             bigint = space.bigint_w(w_result)
             return space.wrap(bigint.hash())
-        else: 
-            raise OperationError(space.w_TypeError, 
+        else:
+            raise OperationError(space.w_TypeError,
                     space.wrap("__hash__() should return an int or long"))
 
     def userdel(space, w_obj):
@@ -460,12 +460,12 @@
                 raise OperationError(space.w_TypeError,
                                      space.wrap("coercion failed"))
             if (not space.is_true(space.isinstance(w_res, space.w_tuple)) or
-                space.int_w(space.len(w_res)) != 2):
+                space.len_w(w_res) != 2):
                 raise OperationError(space.w_TypeError,
                                      space.wrap("coercion should return None or 2-tuple"))
             w_res = space.newtuple([space.getitem(w_res, space.wrap(1)), space.getitem(w_res, space.wrap(0))])
         elif (not space.is_true(space.isinstance(w_res, space.w_tuple)) or
-            space.int_w(space.len(w_res)) != 2):
+            space.len_w(w_res) != 2):
             raise OperationError(space.w_TypeError,
                                  space.wrap("coercion should return None or 2-tuple"))
         return w_res
@@ -675,7 +675,7 @@
         typename2 = w_typ2.getname(space)
         raise operationerrfmt(space.w_TypeError, errormsg,
                               typename1, typename2)
-    
+
     return func_with_new_name(binop_impl, "binop_%s_impl"%left.strip('_'))
 
 def _make_comparison_impl(symbol, specialnames):
@@ -740,17 +740,17 @@
 
 # the following seven operations are really better to generate with
 # string-templating (and maybe we should consider this for
-# more of the above manually-coded operations as well) 
+# more of the above manually-coded operations as well)
 
 for targetname, specialname, checkerspec in [
-    ('int', '__int__', ("space.w_int", "space.w_long")), 
+    ('int', '__int__', ("space.w_int", "space.w_long")),
     ('index', '__index__', ("space.w_int", "space.w_long")),
-    ('long', '__long__', ("space.w_int", "space.w_long")), 
+    ('long', '__long__', ("space.w_int", "space.w_long")),
     ('float', '__float__', ("space.w_float",))]:
 
-    l = ["space.is_true(space.isinstance(w_result, %s))" % x 
+    l = ["space.is_true(space.isinstance(w_result, %s))" % x
                 for x in checkerspec]
-    checker = " or ".join(l) 
+    checker = " or ".join(l)
     source = """if 1:
         def %(targetname)s(space, w_obj):
             w_impl = space.lookup(w_obj, %(specialname)r)
@@ -761,22 +761,22 @@
                                       typename)
             w_result = space.get_and_call_function(w_impl, w_obj)
 
-            if %(checker)s: 
+            if %(checker)s:
                 return w_result
             typename = space.type(w_result).getname(space)
             msg = "%(specialname)s returned non-%(targetname)s (type '%%s')"
             raise operationerrfmt(space.w_TypeError, msg, typename)
         assert not hasattr(DescrOperation, %(targetname)r)
         DescrOperation.%(targetname)s = %(targetname)s
-        del %(targetname)s 
+        del %(targetname)s
         \n""" % locals()
-    exec compile2(source) 
+    exec compile2(source)
 
 for targetname, specialname in [
-    ('str', '__str__'), 
-    ('repr', '__repr__'), 
-    ('oct', '__oct__'), 
-    ('hex', '__hex__')]: 
+    ('str', '__str__'),
+    ('repr', '__repr__'),
+    ('oct', '__oct__'),
+    ('hex', '__hex__')]:
 
     source = """if 1:
         def %(targetname)s(space, w_obj):
@@ -803,11 +803,11 @@
                 return space.wrap(result)
         assert not hasattr(DescrOperation, %(targetname)r)
         DescrOperation.%(targetname)s = %(targetname)s
-        del %(targetname)s 
-        \n""" % locals() 
-    exec compile2(source) 
+        del %(targetname)s
+        \n""" % locals()
+    exec compile2(source)
 
-# add default operation implementations for all still missing ops 
+# add default operation implementations for all still missing ops
 
 for _name, _symbol, _arity, _specialnames in ObjSpace.MethodTable:
     if not hasattr(DescrOperation, _name):
@@ -820,10 +820,10 @@
             _impl_maker = _make_inplace_impl
         elif _arity == 2 and len(_specialnames) == 2:
             #print "binop", _specialnames
-            _impl_maker = _make_binop_impl     
+            _impl_maker = _make_binop_impl
         elif _arity == 1 and len(_specialnames) == 1:
             #print "unaryop", _specialnames
-            _impl_maker = _make_unaryop_impl    
+            _impl_maker = _make_unaryop_impl
         if _impl_maker:
             setattr(DescrOperation,_name,_impl_maker(_symbol,_specialnames))
         elif _name not in ['is_', 'id','type','issubtype',

diff --git a/pypy/objspace/std/objecttype.py b/pypy/objspace/std/objecttype.py
--- a/pypy/objspace/std/objecttype.py
+++ b/pypy/objspace/std/objecttype.py
@@ -110,7 +110,7 @@
     else:
         msg = "format_spec must be a string"
         raise OperationError(space.w_TypeError, space.wrap(msg))
-    if space.int_w(space.len(w_format_spec)) > 0:
+    if space.len_w(w_format_spec) > 0:
         space.warn(
             ("object.__format__ with a non-empty format string is "
                 "deprecated"),

diff --git a/pypy/module/_stackless/test/test_coroutine.py b/pypy/module/_stackless/test/test_coroutine.py
--- a/pypy/module/_stackless/test/test_coroutine.py
+++ b/pypy/module/_stackless/test/test_coroutine.py
@@ -176,7 +176,7 @@
 class TestRandomThings:
     def setup_class(cls):
         cls.space = gettestobjspace(usemodules=('_stackless',))
-    
+
     def test___del___handling(self):
         space = self.space
         w_l = space.newlist([])
@@ -190,7 +190,6 @@
         coro.__del__()
         space.user_del_action.perform(space.getexecutioncontext(), None)
         coro._kill_finally()
-        assert space.int_w(space.len(w_l)) == 1
+        assert space.len_w(w_l) == 1
         res = space.is_true(space.getitem(w_l, space.wrap(0)))
         assert res
-

diff --git a/pypy/module/__builtin__/functional.py b/pypy/module/__builtin__/functional.py
--- a/pypy/module/__builtin__/functional.py
+++ b/pypy/module/__builtin__/functional.py
@@ -206,7 +206,7 @@
 def max(space, __args__):
     """max(iterable[, key=func]) -> value
     max(a, b, c, ...[, key=func]) -> value
-    
+
     With a single iterable argument, return its largest item.
     With two or more arguments, return the largest argument.
     """
@@ -430,7 +430,7 @@
 
 def _filter_tuple(space, w_func, w_tuple):
     none_func = space.is_w(w_func, space.w_None)
-    length = space.int_w(space.len(w_tuple))
+    length = space.len_w(w_tuple)
     result_w = []
     for i in range(length):
         w_item = space.getitem(w_tuple, space.wrap(i))
@@ -446,7 +446,7 @@
     none_func = space.is_w(w_func, space.w_None)
     if none_func and space.is_w(space.type(w_string), w_str_type):
         return w_string
-    length = space.int_w(space.len(w_string))
+    length = space.len_w(w_string)
     result_w = []
     for i in range(length):
         w_item = space.getitem(w_string, space.wrap(i))
@@ -558,7 +558,7 @@
 class W_ReversedIterator(Wrappable):
 
     def __init__(self, space, w_sequence):
-        self.remaining = space.int_w(space.len(w_sequence)) - 1
+        self.remaining = space.len_w(w_sequence) - 1
         if space.lookup(w_sequence, "__getitem__") is None:
             msg = "reversed() argument must be a sequence"
             raise OperationError(space.w_TypeError, space.wrap(msg))
@@ -631,12 +631,12 @@
         return space.wrap(obj)
 
     def descr_repr(self):
-        stop = self.start + self.len * self.step 
-        if self.start == 0 and self.step == 1: 
-            s = "xrange(%d)" % (stop,) 
-        elif self.step == 1: 
-            s = "xrange(%d, %d)" % (self.start, stop) 
-        else: 
+        stop = self.start + self.len * self.step
+        if self.start == 0 and self.step == 1:
+            s = "xrange(%d)" % (stop,)
+        elif self.step == 1:
+            s = "xrange(%d, %d)" % (self.start, stop)
+        else:
             s = "xrange(%d, %d, %d)" %(self.start, stop, self.step)
         return self.space.wrap(s)
 
@@ -646,7 +646,7 @@
     def descr_getitem(self, i):
         # xrange does NOT support slicing
         space = self.space
-        len = self.len 
+        len = self.len
         if i < 0:
             i += len
         if 0 <= i < len:
@@ -680,7 +680,7 @@
 W_XRange.typedef = TypeDef("xrange",
     __new__          = interp2app(W_XRange.descr_new.im_func),
     __repr__         = interp2app(W_XRange.descr_repr),
-    __getitem__      = interp2app(W_XRange.descr_getitem, 
+    __getitem__      = interp2app(W_XRange.descr_getitem,
                                   unwrap_spec=['self', 'index']),
     __iter__         = interp2app(W_XRange.descr_iter),
     __len__          = interp2app(W_XRange.descr_len),

diff --git a/pypy/module/_io/interp_textio.py b/pypy/module/_io/interp_textio.py
--- a/pypy/module/_io/interp_textio.py
+++ b/pypy/module/_io/interp_textio.py
@@ -566,11 +566,11 @@
         # Read a chunk, decode it, and put the result in self._decoded_chars
         w_input = space.call_method(self.w_buffer, "read1",
                                     space.wrap(self.chunk_size))
-        eof = space.int_w(space.len(w_input)) == 0
+        eof = space.len_w(w_input) == 0
         w_decoded = space.call_method(self.w_decoder, "decode",
                                       w_input, space.wrap(eof))
         self._set_decoded_chars(space.unicode_w(w_decoded))
-        if space.int_w(space.len(w_decoded)) > 0:
+        if space.len_w(w_decoded) > 0:
             eof = False
 
         if self.telling:

diff --git a/pypy/module/_io/interp_bytesio.py b/pypy/module/_io/interp_bytesio.py
--- a/pypy/module/_io/interp_bytesio.py
+++ b/pypy/module/_io/interp_bytesio.py
@@ -200,7 +200,7 @@
     def setstate_w(self, space, w_state):
         self._check_closed(space)
 
-        if space.int_w(space.len(w_state)) != 3:
+        if space.len_w(w_state) != 3:
             raise operationerrfmt(space.w_TypeError,
                 "%s.__setstate__ argument should be 3-tuple, got %s",
                 space.type(self).getname(space),

diff --git a/pypy/module/_sre/interp_sre.py b/pypy/module/_sre/interp_sre.py
--- a/pypy/module/_sre/interp_sre.py
+++ b/pypy/module/_sre/interp_sre.py
@@ -289,7 +289,7 @@
 
 def SRE_Pattern__new__(space, w_subtype, w_pattern, flags, w_code,
               groups=0, w_groupindex=None, w_indexgroup=None):
-    n = space.int_w(space.len(w_code))
+    n = space.len_w(w_code)
     code = [intmask(space.uint_w(space.getitem(w_code, space.wrap(i))))
             for i in range(n)]
     #

diff --git a/pypy/module/cpyext/state.py b/pypy/module/cpyext/state.py
--- a/pypy/module/cpyext/state.py
+++ b/pypy/module/cpyext/state.py
@@ -93,7 +93,7 @@
         if not self.programname:
             space = self.space
             argv = space.sys.get('argv')
-            if space.int_w(space.len(argv)):
+            if space.len_w(argv):
                 argv0 = space.getitem(argv, space.wrap(0))
                 progname = space.str_w(argv0)
             else:

diff --git a/pypy/module/array/interp_array.py b/pypy/module/array/interp_array.py
--- a/pypy/module/array/interp_array.py
+++ b/pypy/module/array/interp_array.py
@@ -33,7 +33,7 @@
         if w_args.keywords: # XXX this might be forbidden fishing
             msg = 'array.array() does not take keyword arguments'
             raise OperationError(space.w_TypeError, space.wrap(msg))
-        
+
     for tc in unroll_typecodes:
         if typecode == tc:
             a = space.allocate_instance(types[tc].w_class, w_cls)
@@ -253,7 +253,7 @@
             space = self.space
             oldlen = self.len
             try:
-                new = space.int_w(space.len(w_seq))
+                new = space.len_w(w_seq)
                 self.setlen(self.len + new)
             except OperationError:
                 pass
@@ -534,7 +534,7 @@
         cbuf = self.charbuf()
         s = ''.join([cbuf[i] for i in xrange(self.len * mytype.bytes)])
         return self.space.wrap(s)
-##         
+##
 ##         s = ''
 ##         i = 0
 ##         while i < self.len * mytype.bytes:

diff --git a/pypy/interpreter/baseobjspace.py b/pypy/interpreter/baseobjspace.py
--- a/pypy/interpreter/baseobjspace.py
+++ b/pypy/interpreter/baseobjspace.py
@@ -655,6 +655,10 @@
         """shortcut for space.int_w(space.hash(w_obj))"""
         return self.int_w(self.hash(w_obj))
 
+    def len_w(self, w_obj):
+        """shotcut for space.int_w(space.len(w_obj))"""
+        return self.int_w(self.len(w_obj))
+
     def setitem_str(self, w_obj, key, w_value):
         return self.setitem(w_obj, self.wrap(key), w_value)
 

diff --git a/pypy/module/_io/interp_iobase.py b/pypy/module/_io/interp_iobase.py
--- a/pypy/module/_io/interp_iobase.py
+++ b/pypy/module/_io/interp_iobase.py
@@ -126,7 +126,7 @@
     @unwrap_spec('self', ObjSpace)
     def next_w(self, space):
         w_line = space.call_method(self, "readline")
-        if space.int_w(space.len(w_line)) == 0:
+        if space.len_w(w_line) == 0:
             raise OperationError(space.w_StopIteration, space.w_None)
         return w_line
 
@@ -171,7 +171,7 @@
                         space.w_IOError,
                         "peek() should have returned a bytes object, "
                         "not '%s'", space.type(w_readahead).getname(space, '?'))
-                length = space.int_w(space.len(w_readahead))
+                length = space.len_w(w_readahead)
                 if length > 0:
                     n = 0
                     buf = space.str_w(w_readahead)
@@ -220,7 +220,7 @@
         length = 0
         while True:
             w_line = space.call_method(self, "readline")
-            line_length = space.int_w(space.len(w_line))
+            line_length = space.len_w(w_line)
             if line_length == 0: # done
                 break
 

diff --git a/pypy/module/_io/interp_stringio.py b/pypy/module/_io/interp_stringio.py
--- a/pypy/module/_io/interp_stringio.py
+++ b/pypy/module/_io/interp_stringio.py
@@ -72,7 +72,7 @@
         # We allow the state tuple to be longer than 4, because we may need
         # someday to extend the object's state without breaking
         # backwards-compatibility
-        if not space.isinstance_w(w_state, space.w_tuple) or space.int_w(space.len(w_state)) < 4:
+        if not space.isinstance_w(w_state, space.w_tuple) or space.len_w(w_state) < 4:
             raise operationerrfmt(space.w_TypeError,
                 "%s.__setstate__ argument should be a 4-tuple, got %s",
                 space.type(self).getname(space),
@@ -137,7 +137,7 @@
                                   space.type(w_obj).getname(space, '?'))
         self._check_closed(space)
 
-        orig_size = space.int_w(space.len(w_obj))
+        orig_size = space.len_w(w_obj)
 
         if self.w_decoder is not None:
             w_decoded = space.call_method(

diff --git a/pypy/interpreter/astcompiler/assemble.py b/pypy/interpreter/astcompiler/assemble.py
--- a/pypy/interpreter/astcompiler/assemble.py
+++ b/pypy/interpreter/astcompiler/assemble.py
@@ -323,7 +323,7 @@
         """Turn the applevel constants dictionary into a list."""
         w_consts = self.w_consts
         space = self.space
-        consts_w = [space.w_None] * space.int_w(space.len(w_consts))
+        consts_w = [space.w_None] * space.len_w(w_consts)
         w_iter = space.iter(w_consts)
         first = space.wrap(0)
         while True:

diff --git a/pypy/module/cpyext/unicodeobject.py b/pypy/module/cpyext/unicodeobject.py
--- a/pypy/module/cpyext/unicodeobject.py
+++ b/pypy/module/cpyext/unicodeobject.py
@@ -108,7 +108,7 @@
     """Return the size of the object.  o has to be a PyUnicodeObject (not
     checked)."""
     assert isinstance(w_obj, unicodeobject.W_UnicodeObject)
-    return space.int_w(space.len(w_obj))
+    return space.len_w(w_obj)
 
 @cpython_api([PyObject], rffi.CWCHARP, error=CANNOT_FAIL)
 def PyUnicode_AS_UNICODE(space, ref):
@@ -138,7 +138,7 @@
         return ref.c_size
     else:
         w_obj = from_ref(space, ref)
-        return space.int_w(space.len(w_obj))
+        return space.len_w(w_obj)
 
 @cpython_api([PyUnicodeObject, rffi.CWCHARP, Py_ssize_t], Py_ssize_t, error=-1)
 def PyUnicode_AsWideChar(space, ref, buf, size):
@@ -266,12 +266,12 @@
 def PyUnicode_FromEncodedObject(space, w_obj, encoding, errors):
     """Coerce an encoded object obj to an Unicode object and return a reference with
     incremented refcount.
-    
+
     String and other char buffer compatible objects are decoded according to the
     given encoding and using the error handling defined by errors.  Both can be
     NULL to have the interface use the default values (see the next section for
     details).
-    
+
     All other objects, including Unicode objects, cause a TypeError to be
     set."""
     w_encoding = space.wrap(rffi.charp2str(encoding))
@@ -341,27 +341,27 @@
     """Decode length bytes from a UTF-16 encoded buffer string and return the
     corresponding Unicode object.  errors (if non-NULL) defines the error
     handling. It defaults to "strict".
-    
+
     If byteorder is non-NULL, the decoder starts decoding using the given byte
     order:
-    
+
     *byteorder == -1: little endian
     *byteorder == 0:  native order
     *byteorder == 1:  big endian
-    
+
     If *byteorder is zero, and the first two bytes of the input data are a
     byte order mark (BOM), the decoder switches to this byte order and the BOM is
     not copied into the resulting Unicode string.  If *byteorder is -1 or
     1, any byte order mark is copied to the output (where it will result in
     either a \ufeff or a \ufffe character).
-    
+
     After completion, *byteorder is set to the current byte order at the end
     of input data.
-    
+
     If byteorder is NULL, the codec starts in native order mode.
-    
+
     Return NULL if an exception was raised by the codec.
-    
+
     This function used an int type for size. This might require
     changes in your code for properly supporting 64-bit systems."""
 
@@ -439,4 +439,3 @@
     """Compare two strings and return -1, 0, 1 for less than, equal, and greater
     than, respectively."""
     return space.int_w(space.cmp(w_left, w_right))
-

diff --git a/pypy/module/_bisect/interp_bisect.py b/pypy/module/_bisect/interp_bisect.py
--- a/pypy/module/_bisect/interp_bisect.py
+++ b/pypy/module/_bisect/interp_bisect.py
@@ -15,7 +15,7 @@
         raise OperationError(space.w_ValueError,
                              space.wrap("lo must be non-negative"))
     if hi == -1:
-        hi = space.int_w(space.len(w_a))
+        hi = space.len_w(w_a)
     while lo < hi:
         mid = (lo + hi) >> 1
         w_litem = space.getitem(w_a, space.wrap(mid))
@@ -40,7 +40,7 @@
         raise OperationError(space.w_ValueError,
                              space.wrap("lo must be non-negative"))
     if hi == -1:
-        hi = space.int_w(space.len(w_a))
+        hi = space.len_w(w_a)
     while lo < hi:
         mid = (lo + hi) >> 1
         w_litem = space.getitem(w_a, space.wrap(mid))

diff --git a/pypy/module/oracle/interp_variable.py b/pypy/module/oracle/interp_variable.py
--- a/pypy/module/oracle/interp_variable.py
+++ b/pypy/module/oracle/interp_variable.py
@@ -148,7 +148,7 @@
 
         # allocate the data for the variable
         self.allocateData(self.environment.space)
-    
+
         # allocate the indicator for the variable
         self.indicator = lltype.malloc(rffi.CArrayPtr(roci.sb2).TO,
                                        self.allocatedElements,
@@ -1424,14 +1424,14 @@
         return VT_String, 1, numElements
 
     if space.is_true(space.isinstance(w_value, space.w_str)):
-        size = space.int_w(space.len(w_value))
+        size = space.len_w(w_value)
         if size > config.MAX_STRING_CHARS:
             return VT_LongString, size, numElements
         else:
             return VT_String, size, numElements
 
     if space.is_true(space.isinstance(w_value, space.w_unicode)):
-        size = space.int_w(space.len(w_value))
+        size = space.len_w(w_value)
         return VT_NationalCharString, size, numElements
 
     if space.is_true(space.isinstance(w_value, space.w_int)):

diff --git a/pypy/module/cpyext/stringobject.py b/pypy/module/cpyext/stringobject.py
--- a/pypy/module/cpyext/stringobject.py
+++ b/pypy/module/cpyext/stringobject.py
@@ -167,7 +167,7 @@
         return ref.c_size
     else:
         w_obj = from_ref(space, ref)
-        return space.int_w(space.len(w_obj))
+        return space.len_w(w_obj)
 
 @cpython_api([PyObjectP, Py_ssize_t], rffi.INT_real, error=-1)
 def _PyString_Resize(space, ref, newsize):
@@ -209,10 +209,10 @@
     the old value of string will be stolen.  If the new string cannot be created,
     the old reference to string will still be discarded and the value of
     *string will be set to NULL; the appropriate exception will be set."""
-    
-    if not ref[0]: 
+
+    if not ref[0]:
         return
-    
+
     if w_newpart is None or not PyString_Check(space, ref[0]) or \
             not PyString_Check(space, w_newpart):
          Py_DecRef(space, ref[0])
@@ -244,4 +244,3 @@
     object with the same value."""
     s = rffi.charp2str(string)
     return space.new_interned_str(s)
-

diff --git a/pypy/module/cpyext/mapping.py b/pypy/module/cpyext/mapping.py
--- a/pypy/module/cpyext/mapping.py
+++ b/pypy/module/cpyext/mapping.py
@@ -12,11 +12,11 @@
 
 @cpython_api([PyObject], Py_ssize_t, error=-1)
 def PyMapping_Size(space, w_obj):
-    return space.int_w(space.len(w_obj))
+    return space.len_w(w_obj)
 
 @cpython_api([PyObject], Py_ssize_t, error=-1)
 def PyMapping_Length(space, w_obj):
-    return space.int_w(space.len(w_obj))
+    return space.len_w(w_obj)
 
 @cpython_api([PyObject], PyObject)
 def PyMapping_Keys(space, w_obj):
@@ -74,4 +74,3 @@
         return 1
     except:
         return 0
-

diff --git a/pypy/module/cpyext/test/test_sequence.py b/pypy/module/cpyext/test/test_sequence.py
--- a/pypy/module/cpyext/test/test_sequence.py
+++ b/pypy/module/cpyext/test/test_sequence.py
@@ -17,7 +17,7 @@
         w_set = space.wrap(set((1, 2, 3, 4)))
         w_seq = api.PySequence_Fast(w_set, "message")
         assert space.type(w_seq) is space.w_tuple
-        assert space.int_w(space.len(w_seq)) == 4
+        assert space.len_w(w_seq) == 4
 
         w_seq = api.PySequence_Tuple(w_set)
         assert space.type(w_seq) is space.w_tuple
@@ -52,7 +52,7 @@
         assert exc.value.match(space, space.w_TypeError)
         assert space.str_w(exc.value.get_w_value(space)) == "message"
         rffi.free_charp(message)
-    
+
     def test_get_slice(self, space, api):
         w_t = space.wrap([1, 2, 3, 4, 5])
         assert space.unwrap(api.PySequence_GetSlice(w_t, 2, 4)) == [3, 4]
@@ -105,4 +105,3 @@
 
         self.raises(space, api, IndexError, api.PySequence_DelItem,
                     w_l, 3)
-

From arigo at codespeak.net  Fri Feb  4 21:02:12 2011
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri,  4 Feb 2011 21:02:12 +0100 (CET)
Subject: [pypy-svn] r80285 - pypy/extradoc/talk/ustour2011
Message-ID: <20110204200212.27CEA2A2002@codespeak.net>

Author: arigo
Date: Fri Feb  4 21:02:03 2011
New Revision: 80285

Added:
   pypy/extradoc/talk/ustour2011/mozilla-abstract.txt
Log:
Add the abstract for Mozilla.


Added: pypy/extradoc/talk/ustour2011/mozilla-abstract.txt
==============================================================================
--- (empty file)
+++ pypy/extradoc/talk/ustour2011/mozilla-abstract.txt	Fri Feb  4 21:02:03 2011
@@ -0,0 +1,23 @@
+
+Authors: Armin Rigo, Maciej Fijalkowski
+# XXX get bios from laura
+
+The PyPy project has gathered recently a lot of attention for its
+progress on speeding up the Python language -- it is the fastest,
+most compatible and stable alternative Python interpreter:
+
+* most Python benchmarks run much faster than with CPython or Psyco
+* the real-world PyPy compiler toolchain itself (200 KLocs) runs twice as fast
+* full compatibility to CPython (more than Jython/IronPython)
+
+In this talk we will focus on how we reached this goal:
+
+* "RPython", a subset of Python that our toolchain analyzes and compiles to C
+* our Python interpreter is written in RPython
+* pluggable GCs, also written in RPython
+* JIT support automatically generated from the interpreter source code
+* the JIT itself is also written in RPython
+
+There are many interesting details that can be explored further;
+we will focus on the points the audiance is most interested in.
+


From hpk at codespeak.net  Fri Feb  4 21:59:16 2011
From: hpk at codespeak.net (hpk at codespeak.net)
Date: Fri,  4 Feb 2011 21:59:16 +0100 (CET)
Subject: [pypy-svn] r80286 - pypy/extradoc/talk/ustour2011
Message-ID: <20110204205916.9DBFA2A2002@codespeak.net>

Author: hpk
Date: Fri Feb  4 21:59:13 2011
New Revision: 80286

Modified:
   pypy/extradoc/talk/ustour2011/mozilla-abstract.txt
Log:
* factoring out "RPython" into a leading sentence
* focusing a bit more on the jit-generation for interpreters
  bit as it also represents the main research result
  (which is now a practical one)


Modified: pypy/extradoc/talk/ustour2011/mozilla-abstract.txt
==============================================================================
--- pypy/extradoc/talk/ustour2011/mozilla-abstract.txt	(original)
+++ pypy/extradoc/talk/ustour2011/mozilla-abstract.txt	Fri Feb  4 21:59:13 2011
@@ -10,13 +10,17 @@
 * the real-world PyPy compiler toolchain itself (200 KLocs) runs twice as fast
 * full compatibility to CPython (more than Jython/IronPython)
 
-In this talk we will focus on how we reached this goal:
+In this talk we will focus on how we reached this goal in a truly
+ouroboros manner by implementing many pieces in RPython, a subset
+of Python that our toolchain analyzes and compiles to C:
 
-* "RPython", a subset of Python that our toolchain analyzes and compiles to C
-* our Python interpreter is written in RPython
-* pluggable GCs, also written in RPython
-* JIT support automatically generated from the interpreter source code
-* the JIT itself is also written in RPython
+* our Python interpreter
+* pluggable GCs
+* the JIT itself, including x86/ARM backends
+
+The translation toolchain itself is written in full Python (not RPython!)
+and generates JIT support automatically from the interpreter source code,
+thereby reducing complexity for writing state-of-the-art interpreters.
 
 There are many interesting details that can be explored further;
 we will focus on the points the audiance is most interested in.


From commits-noreply at bitbucket.org  Fri Feb  4 22:23:58 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Fri,  4 Feb 2011 22:23:58 +0100 (CET)
Subject: [pypy-svn] pypy default: list(u"") is not valid.
Message-ID: <20110204212358.CC38A2A2002@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41618:65b796deaab8
Date: 2011-02-04 16:23 -0500
http://bitbucket.org/pypy/pypy/changeset/65b796deaab8/

Log:	list(u"") is not valid.

diff --git a/pypy/module/_io/interp_stringio.py b/pypy/module/_io/interp_stringio.py
--- a/pypy/module/_io/interp_stringio.py
+++ b/pypy/module/_io/interp_stringio.py
@@ -91,7 +91,7 @@
         initval = space.unicode_w(w_initval)
         size = len(initval)
         self.resize_buffer(size)
-        self.buf = list(initval)
+        self.buf = [c for c in initval]
         pos = space.getindex_w(w_pos, space.w_TypeError)
         if pos < 0:
             raise OperationError(space.w_ValueError,

From commits-noreply at bitbucket.org  Sat Feb  5 00:06:36 2011
From: commits-noreply at bitbucket.org (wlav)
Date: Sat,  5 Feb 2011 00:06:36 +0100 (CET)
Subject: [pypy-svn] pypy reflex-support: short array returns and short ptr
	data member	access
Message-ID: <20110204230636.5356B2A2002@codespeak.net>

Author: Wim Lavrijsen <WLavrijsen at lbl.gov>
Branch: reflex-support
Changeset: r41619:34969598fd06
Date: 2011-02-04 14:35 -0800
http://bitbucket.org/pypy/pypy/changeset/34969598fd06/

Log:	short array returns and short ptr data member access

diff --git a/pypy/module/cppyy/executor.py b/pypy/module/cppyy/executor.py
--- a/pypy/module/cppyy/executor.py
+++ b/pypy/module/cppyy/executor.py
@@ -1,6 +1,10 @@
+import sys
 from pypy.rpython.lltypesystem import rffi, lltype
 from pypy.rlib import libffi
 
+from pypy.module._rawffi.interp_rawffi import unpack_simple_shape
+from pypy.module._rawffi.array import W_Array
+
 from pypy.module.cppyy import helper, capi
 
 
@@ -21,6 +25,10 @@
         raise FastCallNotPossible
 
 
+class ArrayExecutor(FunctionExecutor):
+    _immutable_ = True
+
+
 class VoidExecutor(FunctionExecutor):
     _immutable_ = True
 
@@ -82,6 +90,7 @@
     def execute_libffi(self, space, libffifunc, argchain):
         return space.wrap(libffifunc.call(argchain, rffi.DOUBLE))
 
+
 class CStringExecutor(FunctionExecutor):
     _immutable_ = True
     def execute(self, space, func, cppthis, num_args, args):
@@ -91,6 +100,15 @@
         return space.wrap(result)
 
 
+class ShortArrayExecutor(ArrayExecutor):
+    _immutable_ = True
+    def execute(self, space, func, cppthis, num_args, args):
+        lresult = capi.c_call_l(func.cpptype.handle, func.method_index, cppthis, num_args, args)
+        spresult = rffi.cast(rffi.SHORTP, lresult)
+        arr = space.interp_w(W_Array, unpack_simple_shape(space, space.wrap('h')))
+        return arr.fromaddress(space, spresult, sys.maxint)
+
+
 class InstancePtrExecutor(FunctionExecutor):
     _immutable_ = True
     def __init__(self, space, name, cpptype):
@@ -128,6 +146,7 @@
 _executors["char"]                = CharExecutor
 _executors["unsigned char"]       = CharExecutor
 _executors["short int"]           = ShortExecutor
+_executors["short int*"]          = ShortArrayExecutor
 _executors["unsigned short int"]  = ShortExecutor
 _executors["int"]                 = LongExecutor
 _executors["unsigned int"]        = LongExecutor

diff --git a/pypy/module/cppyy/test/test_datatypes.py b/pypy/module/cppyy/test/test_datatypes.py
--- a/pypy/module/cppyy/test/test_datatypes.py
+++ b/pypy/module/cppyy/test/test_datatypes.py
@@ -31,6 +31,84 @@
         lib2 = cppyy.load_lib(self.shared_lib)
         assert self.datatypes is lib2
 
+    def test1ReadAccess( self ):
+        """Test read access to instance public data and verify values"""
+
+        import cppyy, sys
+        cppyy_test_data = cppyy.gbl.cppyy_test_data
+
+        c = cppyy_test_data()
+        assert isinstance(c, cppyy_test_data)
+
+        # reading boolean type
+        assert c.m_bool == False
+
+        # reading char types
+        assert c.m_char  == 'a'
+        assert c.m_uchar == 'c'
+
+        # reading integer types
+        assert c.m_short  == -11
+        assert c.m_ushort ==  11
+        assert c.m_int    == -22
+        assert c.m_uint   ==  22
+        assert c.m_long   == -33
+        assert c.m_ulong  ==  33
+
+        # reading floating point types
+        assert round(c.m_float  + 44., 5) == 0
+        assert round(c.m_double + 55., 8) == 0
+
+        # reding of array types
+        for i in range(self.N):
+            # reading of integer array types
+            assert c.m_short_array[i]       ==  -1*i
+            assert c.get_short_array()[i]   ==  -1*i
+            assert c.m_short_array2[i]      ==  -2*i
+            assert c.get_short_array2()[i]  ==  -2*i
+            """
+            assert c.m_ushort_array[i]      ==   3*i
+            assert c.get_ushort_array()[i]  ==   3*i
+            assert c.m_ushort_array2[i]     ==   4*i
+            assert c.get_ushort_array2()[i] ==   4*i
+            assert c.m_int_array[i]         ==  -5*i
+            assert c.get_int_array()[i]     ==  -5*i
+            assert c.m_int_array2[i]        ==  -6*i
+            assert c.get_int_array2()[i]    ==  -6*i
+            assert c.m_uint_array[i]        ==   7*i
+            assert c.get_uint_array()[i]    ==   7*i
+            assert c.m_uint_array2[i]       ==   8*i
+            assert c.get_uint_array2()[i]   ==   8*i
+
+            assert c.m_long_array[i]        ==  -9*i
+            assert c.get_long_array()[i]    ==  -9*i
+            assert c.m_long_array2[i]       == -10*i
+            assert c.get_long_array2()[i]   == -10*i
+            assert c.m_ulong_array[i]       ==  11*i
+            assert c.get_ulong_array()[i]   ==  11*i
+            assert c.m_ulong_array2[i]      ==  12*i
+            assert c.get_ulong_array2()[i]  ==  12*i
+
+            assert round(c.m_float_array[i]   + 13.*i, 5) == 0
+            assert round(c.m_float_array2[i]  + 14.*i, 5) == 0
+            assert round(c.m_double_array[i]  + 15.*i, 8) == 0
+            assert round(c.m_double_array2[i] + 16.*i, 8) == 0
+            """
+
+        """
+        # out-of-bounds checks
+        raises(IndexError, c.m_short_array.__getitem__,  self.N)
+        raises(IndexError, c.m_ushort_array.__getitem__, self.N)
+        raises(IndexError, c.m_int_array.__getitem__,    self.N)
+        raises(IndexError, c.m_uint_array.__getitem__,   self.N)
+        raises(IndexError, c.m_long_array.__getitem__,   self.N)
+        raises(IndexError, c.m_ulong_array.__getitem__,  self.N)
+        raises(IndexError, c.m_float_array.__getitem__,  self.N)
+        raises(IndexError, c.m_double_array.__getitem__, self.N)
+        """
+
+        c.destruct()
+
     def test2WriteAccess(self):
         """Test write access to instance public data and verify values"""
 
@@ -143,3 +221,5 @@
         raises(TypeError, c.m_double,  'c')
         raises(TypeError, c.m_int,     -1.)
         raises(TypeError, c.m_int,      1.)
+
+        c.destruct()

diff --git a/pypy/module/cppyy/converter.py b/pypy/module/cppyy/converter.py
--- a/pypy/module/cppyy/converter.py
+++ b/pypy/module/cppyy/converter.py
@@ -228,46 +228,61 @@
         return rffi.cast(rffi.VOIDP, x)
 
 
-class ShortPtrConverter(ArrayTypeConverter):
+class ShortArrayConverter(ArrayTypeConverter):
     _immutable_ = True
-    def convert_argument(self, space, w_obj):
-        assert 0, "not yet implemented"
-
     def from_memory(self, space, w_obj, offset):
         # read access, so no copy needed
         fieldptr = self._get_fieldptr(space, w_obj, offset)
         ptrval = rffi.cast(rffi.UINT, fieldptr)
         arr = space.interp_w(W_Array, unpack_simple_shape(space, space.wrap('h')))
+        return arr.fromaddress(space, fieldptr, self.size)
+
+    def to_memory(self, space, w_obj, w_value, offset):
+        # copy the full array (uses byte copy for now)
+        fieldptr = self._get_fieldptr(space, w_obj, offset)
+        buf = space.interp_w(Buffer, w_value.getslotvalue(2))
+        # TODO: get sizeof(short) from system
+        for i in range(min(self.size*2, buf.getlength())):
+            fieldptr[i] = buf.getitem(i)
+
+class LongArrayConverter(ArrayTypeConverter):
+    _immutable_ = True
+    def from_memory(self, space, w_obj, offset):
+        # read access, so no copy needed
+        fieldptr = self._get_fieldptr(space, w_obj, offset)
+        ptrval = rffi.cast(rffi.UINT, fieldptr)
+        arr = space.interp_w(W_Array, unpack_simple_shape(space, space.wrap('l')))
         return arr.fromaddress(space, ptrval, self.size)
 
     def to_memory(self, space, w_obj, w_value, offset):
+        # copy the full array (uses byte copy for now)
+        fieldptr = self._get_fieldptr(space, w_obj, offset)
+        buf = space.interp_w(Buffer, w_value.getslotvalue(2))
+        # TODO: get sizeof(long) from system
+        for i in range(min(self.size*4, buf.getlength())):
+            fieldptr[i] = buf.getitem(i)
+
+
+class ShortPtrConverter(ShortArrayConverter):
+    _immutable_ = True
+    def _get_fieldptr(self, space, w_obj, offset):
+        fieldptr = TypeConverter._get_fieldptr(self, space, w_obj, offset)
+        ptrptr = rffi.cast(rffi.LONGP, fieldptr)
+        return ptrptr[0]
+
+    def to_memory(self, space, w_obj, w_value, offset):
         # copy only the pointer value
         rawobject = get_rawobject(space, w_obj)
         byteptr = rffi.cast(rffi.LONGP, rawobject[offset])
         # TODO: now what ... ?? AFAICS, w_value is a pure python list, not an array?
 #        byteptr[0] = space.unwrap(space.id(w_value.getslotvalue(2)))
 
-class ShortArrayConverter(ShortPtrConverter):
+class LongPtrConverter(LongArrayConverter):
     _immutable_ = True
-    def to_memory(self, space, w_obj, w_value, offset):
-        # copy the full array (uses byte copy for now)
-        fieldptr = self._get_fieldptr(space, w_obj, offset)
-        buf = space.interp_w(Buffer, w_value.getslotvalue(2))
-        # TODO: get sizeof(short) from system
-        for i in range(min(self.size*2, buf.getlength())):
-            fieldptr[i] = buf.getitem(i)
-
-class LongPtrConverter(ArrayTypeConverter):
-    _immutable_ = True
-    def convert_argument(self, space, w_obj):
-        assert 0, "not yet implemented"
-
-    def from_memory(self, space, w_obj, offset):
-        # read access, so no copy needed
-        fieldptr = self._get_fieldptr(space, w_obj, offset)
-        ptrval = rffi.cast(rffi.UINT, fieldptr)
-        arr = space.interp_w(W_Array, unpack_simple_shape(space, space.wrap('l')))
-        return arr.fromaddress(space, ptrval, self.size)
+    def _get_fieldptr(self, space, w_obj, offset):
+        fieldptr = TypeConverter._get_fieldptr(self, space, w_obj, offset)
+        ptrptr = rffi.cast(rffi.LONGP, fieldptr)
+        return ptrptr[0]
 
     def to_memory(self, space, w_obj, w_value, offset):
         # copy only the pointer value
@@ -276,15 +291,6 @@
         # TODO: now what ... ?? AFAICS, w_value is a pure python list, not an array?
 #        byteptr[0] = space.unwrap(space.id(w_value.getslotvalue(2)))
 
-class LongArrayConverter(LongPtrConverter):
-    _immutable_ = True
-    def to_memory(self, space, w_obj, w_value, offset):
-        # copy the full array (uses byte copy for now)
-        fieldptr = self._get_fieldptr(space, w_obj, offset)
-        buf = space.interp_w(Buffer, w_value.getslotvalue(2))
-        # TODO: get sizeof(long) from system
-        for i in range(min(self.size*4, buf.getlength())):
-            fieldptr[i] = buf.getitem(i)
 
 
 class InstancePtrConverter(TypeConverter):

From commits-noreply at bitbucket.org  Sat Feb  5 00:06:36 2011
From: commits-noreply at bitbucket.org (wlav)
Date: Sat,  5 Feb 2011 00:06:36 +0100 (CET)
Subject: [pypy-svn] pypy reflex-support: refactoring of array handling
Message-ID: <20110204230636.D496A2A2002@codespeak.net>

Author: Wim Lavrijsen <WLavrijsen at lbl.gov>
Branch: reflex-support
Changeset: r41620:d300522715d1
Date: 2011-02-04 14:49 -0800
http://bitbucket.org/pypy/pypy/changeset/d300522715d1/

Log:	refactoring of array handling

diff --git a/pypy/module/cppyy/converter.py b/pypy/module/cppyy/converter.py
--- a/pypy/module/cppyy/converter.py
+++ b/pypy/module/cppyy/converter.py
@@ -52,13 +52,36 @@
 
 
 class ArrayTypeConverter(TypeConverter):
-    _immutable_ = True
+    typecode = ''
+    typesize = 0         # TODO: get sizeof(type) from system
+
     def __init__(self, space, array_size):
         if array_size <= 0:
             self.size = sys.maxint
         else:
             self.size = array_size
 
+    def from_memory(self, space, w_obj, offset):
+        # read access, so no copy needed
+        fieldptr = self._get_fieldptr(space, w_obj, offset)
+        ptrval = rffi.cast(rffi.UINT, fieldptr)
+        arr = space.interp_w(W_Array, unpack_simple_shape(space, space.wrap(self.typecode)))
+        return arr.fromaddress(space, fieldptr, self.size)
+
+    def to_memory(self, space, w_obj, w_value, offset):
+        # copy the full array (uses byte copy for now)
+        fieldptr = self._get_fieldptr(space, w_obj, offset)
+        buf = space.interp_w(Buffer, w_value.getslotvalue(2))
+        for i in range(min(self.size*self.typesize, buf.getlength())):
+            fieldptr[i] = buf.getitem(i)
+
+
+class PtrTypeConverter(ArrayTypeConverter):
+    def _get_fieldptr(self, space, w_obj, offset):
+        fieldptr = TypeConverter._get_fieldptr(self, space, w_obj, offset)
+        ptrptr = rffi.cast(rffi.LONGP, fieldptr)
+        return ptrptr[0]
+
 
 class VoidConverter(TypeConverter):
     libffitype = libffi.types.void
@@ -230,45 +253,19 @@
 
 class ShortArrayConverter(ArrayTypeConverter):
     _immutable_ = True
-    def from_memory(self, space, w_obj, offset):
-        # read access, so no copy needed
-        fieldptr = self._get_fieldptr(space, w_obj, offset)
-        ptrval = rffi.cast(rffi.UINT, fieldptr)
-        arr = space.interp_w(W_Array, unpack_simple_shape(space, space.wrap('h')))
-        return arr.fromaddress(space, fieldptr, self.size)
-
-    def to_memory(self, space, w_obj, w_value, offset):
-        # copy the full array (uses byte copy for now)
-        fieldptr = self._get_fieldptr(space, w_obj, offset)
-        buf = space.interp_w(Buffer, w_value.getslotvalue(2))
-        # TODO: get sizeof(short) from system
-        for i in range(min(self.size*2, buf.getlength())):
-            fieldptr[i] = buf.getitem(i)
+    typecode = 'h'
+    typesize = 2
 
 class LongArrayConverter(ArrayTypeConverter):
     _immutable_ = True
-    def from_memory(self, space, w_obj, offset):
-        # read access, so no copy needed
-        fieldptr = self._get_fieldptr(space, w_obj, offset)
-        ptrval = rffi.cast(rffi.UINT, fieldptr)
-        arr = space.interp_w(W_Array, unpack_simple_shape(space, space.wrap('l')))
-        return arr.fromaddress(space, ptrval, self.size)
+    typecode = 'l'
+    typesize = 4
 
-    def to_memory(self, space, w_obj, w_value, offset):
-        # copy the full array (uses byte copy for now)
-        fieldptr = self._get_fieldptr(space, w_obj, offset)
-        buf = space.interp_w(Buffer, w_value.getslotvalue(2))
-        # TODO: get sizeof(long) from system
-        for i in range(min(self.size*4, buf.getlength())):
-            fieldptr[i] = buf.getitem(i)
 
-
-class ShortPtrConverter(ShortArrayConverter):
+class ShortPtrConverter(PtrTypeConverter):
     _immutable_ = True
-    def _get_fieldptr(self, space, w_obj, offset):
-        fieldptr = TypeConverter._get_fieldptr(self, space, w_obj, offset)
-        ptrptr = rffi.cast(rffi.LONGP, fieldptr)
-        return ptrptr[0]
+    typecode = 'h'
+    typesize = 2
 
     def to_memory(self, space, w_obj, w_value, offset):
         # copy only the pointer value
@@ -277,20 +274,10 @@
         # TODO: now what ... ?? AFAICS, w_value is a pure python list, not an array?
 #        byteptr[0] = space.unwrap(space.id(w_value.getslotvalue(2)))
 
-class LongPtrConverter(LongArrayConverter):
+class LongPtrConverter(PtrTypeConverter):
     _immutable_ = True
-    def _get_fieldptr(self, space, w_obj, offset):
-        fieldptr = TypeConverter._get_fieldptr(self, space, w_obj, offset)
-        ptrptr = rffi.cast(rffi.LONGP, fieldptr)
-        return ptrptr[0]
-
-    def to_memory(self, space, w_obj, w_value, offset):
-        # copy only the pointer value
-        rawobject = get_rawobject(space, w_obj)
-        byteptr = rffi.cast(rffi.LONGP, rawobject[offset])
-        # TODO: now what ... ?? AFAICS, w_value is a pure python list, not an array?
-#        byteptr[0] = space.unwrap(space.id(w_value.getslotvalue(2)))
-
+    typecode = 'l'
+    typesize = 4
 
 
 class InstancePtrConverter(TypeConverter):

From commits-noreply at bitbucket.org  Sat Feb  5 00:06:37 2011
From: commits-noreply at bitbucket.org (wlav)
Date: Sat,  5 Feb 2011 00:06:37 +0100 (CET)
Subject: [pypy-svn] pypy reflex-support: integer and floating point array
	return types	and data member access
Message-ID: <20110204230637.C31562A2002@codespeak.net>

Author: Wim Lavrijsen <WLavrijsen at lbl.gov>
Branch: reflex-support
Changeset: r41621:2d45870d5299
Date: 2011-02-04 15:05 -0800
http://bitbucket.org/pypy/pypy/changeset/2d45870d5299/

Log:	integer and floating point array return types and data member access

diff --git a/pypy/module/cppyy/executor.py b/pypy/module/cppyy/executor.py
--- a/pypy/module/cppyy/executor.py
+++ b/pypy/module/cppyy/executor.py
@@ -25,8 +25,15 @@
         raise FastCallNotPossible
 
 
-class ArrayExecutor(FunctionExecutor):
+class PtrTypeExecutor(FunctionExecutor):
     _immutable_ = True
+    typecode = ''
+
+    def execute(self, space, func, cppthis, num_args, args):
+        lresult = capi.c_call_l(func.cpptype.handle, func.method_index, cppthis, num_args, args)
+        spresult = rffi.cast(rffi.SHORTP, lresult)
+        arr = space.interp_w(W_Array, unpack_simple_shape(space, space.wrap(self.typecode)))
+        return arr.fromaddress(space, spresult, sys.maxint)
 
 
 class VoidExecutor(FunctionExecutor):
@@ -40,6 +47,7 @@
         libffifunc.call(argchain, lltype.Void)
         return space.w_None
 
+
 class BoolExecutor(FunctionExecutor):
     _immutable_ = True
     def execute(self, space, func, cppthis, num_args, args):
@@ -100,13 +108,21 @@
         return space.wrap(result)
 
 
-class ShortArrayExecutor(ArrayExecutor):
+class ShortPtrExecutor(PtrTypeExecutor):
     _immutable_ = True
-    def execute(self, space, func, cppthis, num_args, args):
-        lresult = capi.c_call_l(func.cpptype.handle, func.method_index, cppthis, num_args, args)
-        spresult = rffi.cast(rffi.SHORTP, lresult)
-        arr = space.interp_w(W_Array, unpack_simple_shape(space, space.wrap('h')))
-        return arr.fromaddress(space, spresult, sys.maxint)
+    typecode = 'h'
+
+class LongPtrExecutor(PtrTypeExecutor):
+    _immutable_ = True
+    typecode = 'l'
+
+class FloatPtrExecutor(PtrTypeExecutor):
+    _immutable_ = True
+    typecode = 'f'
+
+class DoublePtrExecutor(PtrTypeExecutor):
+    _immutable_ = True
+    typecode = 'd'
 
 
 class InstancePtrExecutor(FunctionExecutor):
@@ -146,12 +162,19 @@
 _executors["char"]                = CharExecutor
 _executors["unsigned char"]       = CharExecutor
 _executors["short int"]           = ShortExecutor
-_executors["short int*"]          = ShortArrayExecutor
+_executors["short int*"]          = ShortPtrExecutor
 _executors["unsigned short int"]  = ShortExecutor
+_executors["unsigned short int*"] = ShortPtrExecutor
 _executors["int"]                 = LongExecutor
+_executors["int*"]                = LongPtrExecutor
 _executors["unsigned int"]        = LongExecutor
+_executors["unsigned int*"]       = LongPtrExecutor
 _executors["long int"]            = LongExecutor
+_executors["long int*"]           = LongPtrExecutor
 _executors["unsigned long int"]   = LongExecutor
+_executors["unsigned long int*"]  = LongPtrExecutor
 _executors["float"]               = FloatExecutor
+_executors["float*"]              = FloatPtrExecutor
 _executors["double"]              = DoubleExecutor
+_executors["double*"]             = DoublePtrExecutor
 _executors["char*"]               = CStringExecutor

diff --git a/pypy/module/cppyy/test/test_datatypes.py b/pypy/module/cppyy/test/test_datatypes.py
--- a/pypy/module/cppyy/test/test_datatypes.py
+++ b/pypy/module/cppyy/test/test_datatypes.py
@@ -66,7 +66,6 @@
             assert c.get_short_array()[i]   ==  -1*i
             assert c.m_short_array2[i]      ==  -2*i
             assert c.get_short_array2()[i]  ==  -2*i
-            """
             assert c.m_ushort_array[i]      ==   3*i
             assert c.get_ushort_array()[i]  ==   3*i
             assert c.m_ushort_array2[i]     ==   4*i
@@ -93,9 +92,7 @@
             assert round(c.m_float_array2[i]  + 14.*i, 5) == 0
             assert round(c.m_double_array[i]  + 15.*i, 8) == 0
             assert round(c.m_double_array2[i] + 16.*i, 8) == 0
-            """
 
-        """
         # out-of-bounds checks
         raises(IndexError, c.m_short_array.__getitem__,  self.N)
         raises(IndexError, c.m_ushort_array.__getitem__, self.N)
@@ -105,7 +102,6 @@
         raises(IndexError, c.m_ulong_array.__getitem__,  self.N)
         raises(IndexError, c.m_float_array.__getitem__,  self.N)
         raises(IndexError, c.m_double_array.__getitem__, self.N)
-        """
 
         c.destruct()
 

diff --git a/pypy/module/cppyy/converter.py b/pypy/module/cppyy/converter.py
--- a/pypy/module/cppyy/converter.py
+++ b/pypy/module/cppyy/converter.py
@@ -261,6 +261,16 @@
     typecode = 'l'
     typesize = 4
 
+class FloatArrayConverter(ArrayTypeConverter):
+    _immutable_ = True
+    typecode = 'f'
+    typesize = 4
+
+class DoubleArrayConverter(ArrayTypeConverter):
+    _immutable_ = True
+    typecode = 'd'
+    typesize = 8
+
 
 class ShortPtrConverter(PtrTypeConverter):
     _immutable_ = True
@@ -279,6 +289,16 @@
     typecode = 'l'
     typesize = 4
 
+class FloatPtrConverter(PtrTypeConverter):
+    _immutable_ = True
+    typecode = 'f'
+    typesize = 4
+
+class DoublePtrConverter(PtrTypeConverter):
+    _immutable_ = True
+    typecode = 'd'
+    typesize = 8
+
 
 class InstancePtrConverter(TypeConverter):
     _immutable_ = True
@@ -364,5 +384,9 @@
 _converters["unsigned long int*"]       = LongPtrConverter
 _converters["unsigned long int[]"]      = LongArrayConverter
 _converters["float"]                    = FloatConverter
+_converters["float*"]                   = FloatPtrConverter
+_converters["float[]"]                  = FloatArrayConverter
 _converters["double"]                   = DoubleConverter
+_converters["double*"]                  = DoublePtrConverter
+_converters["double[]"]                 = DoubleArrayConverter
 _converters["const char*"]              = CStringConverter

From commits-noreply at bitbucket.org  Sat Feb  5 02:50:39 2011
From: commits-noreply at bitbucket.org (wlav)
Date: Sat,  5 Feb 2011 02:50:39 +0100 (CET)
Subject: [pypy-svn] pypy reflex-support: read support for static
	(class-level) data	members
Message-ID: <20110205015039.ABC8A2A2002@codespeak.net>

Author: Wim Lavrijsen <WLavrijsen at lbl.gov>
Branch: reflex-support
Changeset: r41622:ac289a3e9f8b
Date: 2011-02-04 17:50 -0800
http://bitbucket.org/pypy/pypy/changeset/ac289a3e9f8b/

Log:	read support for static (class-level) data members

diff --git a/pypy/module/cppyy/test/test_datatypes.py b/pypy/module/cppyy/test/test_datatypes.py
--- a/pypy/module/cppyy/test/test_datatypes.py
+++ b/pypy/module/cppyy/test/test_datatypes.py
@@ -31,7 +31,7 @@
         lib2 = cppyy.load_lib(self.shared_lib)
         assert self.datatypes is lib2
 
-    def test1ReadAccess( self ):
+    def test1InstanceDataReadAccess( self ):
         """Test read access to instance public data and verify values"""
 
         import cppyy, sys
@@ -105,7 +105,7 @@
 
         c.destruct()
 
-    def test2WriteAccess(self):
+    def test2InstanceDataWriteAccess(self):
         """Test write access to instance public data and verify values"""
 
         import cppyy, sys
@@ -187,7 +187,49 @@
 
         c.destruct()
 
-    def test3RangeAccess(self):
+    def test3ClassReadAccess(self):
+        """Test read access to class public data and verify values"""
+
+        import cppyy, sys
+        cppyy_test_data = cppyy.gbl.cppyy_test_data
+
+        c = cppyy_test_data()
+        assert isinstance(c, cppyy_test_data)
+
+        # char types
+        assert cppyy_test_data.s_char  == 's'
+        assert c.s_char                == 's'
+        assert c.s_uchar               == 'u'
+        assert cppyy_test_data.s_uchar == 'u'
+
+        # integer types
+        assert cppyy_test_data.s_short  == -101
+        assert c.s_short                == -101
+        assert c.s_ushort               ==  255
+        assert cppyy_test_data.s_ushort ==  255
+        assert cppyy_test_data.s_int    == -202
+        assert c.s_int                  == -202
+        assert c.s_uint                 ==  202
+        assert cppyy_test_data.s_uint   ==  202
+        assert cppyy_test_data.s_long   == -303
+        assert c.s_long                 == -303
+        assert c.s_ulong                ==  303
+        assert cppyy_test_data.s_ulong  ==  303
+
+        # floating point types
+        assert round(cppyy_test_data.s_float  + 404., 5) == 0
+        assert round(c.s_float                + 404., 5) == 0
+        assert round(cppyy_test_data.s_double + 505., 8) == 0
+        assert round(c.s_double               + 505., 8) == 0
+
+        c.destruct()
+
+    def test4ClassDataWriteAccess(self):
+        """Test write access to class public data and verify values"""
+
+        pass
+
+    def test5RangeAccess(self):
         """Test the ranges of integer types"""
 
         import cppyy, sys
@@ -201,7 +243,7 @@
 
         c.destruct()
 
-    def test4TypeConversions(self):
+    def test6TypeConversions(self):
         """Test conversions between builtin types"""
 
         import cppyy, sys
@@ -210,7 +252,6 @@
         c = cppyy_test_data()
         assert isinstance(c, cppyy_test_data)
 
-
         c.m_double = -1
         assert round(c.m_double + 1.0, 8) == 0
 

diff --git a/pypy/module/cppyy/include/reflexcwrapper.h b/pypy/module/cppyy/include/reflexcwrapper.h
--- a/pypy/module/cppyy/include/reflexcwrapper.h
+++ b/pypy/module/cppyy/include/reflexcwrapper.h
@@ -26,6 +26,9 @@
     void cppyy_destruct(cppyy_typehandle_t handle, cppyy_object_t self);
     cppyy_methptrgetter_t cppyy_get_methptr_getter(cppyy_typehandle_t handle, int method_index);
 
+    /* type properties */
+    int cppyy_is_subtype(cppyy_typehandle_t h1, cppyy_typehandle_t h2);
+
     /* method/function reflection information */
     int cppyy_num_methods(cppyy_typehandle_t handle);
     char* cppyy_method_name(cppyy_typehandle_t handle, int method_index);
@@ -33,16 +36,20 @@
     int cppyy_method_num_args(cppyy_typehandle_t handle, int method_index);
     char* cppyy_method_arg_type(cppyy_typehandle_t handle, int method_index, int index);
 
+    /* method properties */
+    int cppyy_is_constructor(cppyy_typehandle_t handle, int method_index);
+    int cppyy_is_staticmethod(cppyy_typehandle_t handle, int method_index);
+
     /* data member reflection information */
     int cppyy_num_data_members(cppyy_typehandle_t handle);
     char* cppyy_data_member_name(cppyy_typehandle_t handle, int data_member_index);
     char* cppyy_data_member_type(cppyy_typehandle_t handle, int data_member_index);
     size_t cppyy_data_member_offset(cppyy_typehandle_t handle, int data_member_index);
 
-    int cppyy_is_constructor(cppyy_typehandle_t handle, int method_index);
-    int cppyy_is_static(cppyy_typehandle_t handle, int method_index);
-    int cppyy_is_subtype(cppyy_typehandle_t h1, cppyy_typehandle_t h2);
+    /* data member properties */
+    int cppyy_is_staticdata(cppyy_typehandle_t handle, int data_member_index);
 
+    /* misc helper */
     void cppyy_free(void* ptr);
 
 #ifdef __cplusplus

diff --git a/pypy/module/cppyy/src/reflexcwrapper.cxx b/pypy/module/cppyy/src/reflexcwrapper.cxx
--- a/pypy/module/cppyy/src/reflexcwrapper.cxx
+++ b/pypy/module/cppyy/src/reflexcwrapper.cxx
@@ -102,6 +102,16 @@
 }
 
 
+/* type/class reflection information -------------------------------------- */
+int cppyy_is_subtype(cppyy_typehandle_t h1, cppyy_typehandle_t h2) {
+    if (h1 == h2)
+        return 1;
+    Reflex::Type t1((Reflex::TypeName*)h1);
+    Reflex::Type t2((Reflex::TypeName*)h2);
+    return (int)t2.HasBase(t1);
+}
+
+
 /* method/function reflection information --------------------------------- */
 int cppyy_num_methods(cppyy_typehandle_t handle) {
     Reflex::Type t((Reflex::TypeName*)handle);
@@ -154,6 +164,19 @@
 }
 
 
+int cppyy_is_constructor(cppyy_typehandle_t handle, int method_index) {
+    Reflex::Type t((Reflex::TypeName*)handle);
+    Reflex::Member m = t.FunctionMemberAt(method_index);
+    return m.IsConstructor();
+}
+
+int cppyy_is_staticmethod(cppyy_typehandle_t handle, int method_index) {
+    Reflex::Type t((Reflex::TypeName*)handle);
+    Reflex::Member m = t.FunctionMemberAt(method_index);
+    return m.IsStatic();
+}
+
+
 /* data member reflection information ------------------------------------- */
 int cppyy_num_data_members(cppyy_typehandle_t handle) {
     Reflex::Type t((Reflex::TypeName*)handle);
@@ -185,26 +208,14 @@
 }
 
 
-int cppyy_is_constructor(cppyy_typehandle_t handle, int method_index) {
+int cppyy_is_staticdata(cppyy_typehandle_t handle, int data_member_index) {
     Reflex::Type t((Reflex::TypeName*)handle);
-    Reflex::Member m = t.FunctionMemberAt(method_index);
-    return m.IsConstructor();
-}
-
-int cppyy_is_static(cppyy_typehandle_t handle, int method_index) {
-    Reflex::Type t((Reflex::TypeName*)handle);
-    Reflex::Member m = t.FunctionMemberAt(method_index);
+    Reflex::Member m = t.DataMemberAt(data_member_index);
     return m.IsStatic();
 }
 
-int cppyy_is_subtype(cppyy_typehandle_t h1, cppyy_typehandle_t h2) {
-    if (h1 == h2)
-        return 1;
-    Reflex::Type t1((Reflex::TypeName*)h1);
-    Reflex::Type t2((Reflex::TypeName*)h2);
-    return (int)t2.HasBase(t1);
-}
 
+/* misc helper ------------------------------------------------------------ */
 void cppyy_free(void* ptr) {
     free(ptr);
 }

diff --git a/pypy/module/cppyy/interp_cppyy.py b/pypy/module/cppyy/interp_cppyy.py
--- a/pypy/module/cppyy/interp_cppyy.py
+++ b/pypy/module/cppyy/interp_cppyy.py
@@ -261,6 +261,21 @@
 )
 
 
+class W_CPPStaticDataMember(W_CPPDataMember):
+    def __get__(self, args_w):
+        return self.converter.from_memory(self.space, self.space.w_None, self.offset)
+
+    def __set__(self, args_w):
+        self.converter.to_memory(self.space, self.space_w_None, args_w[1], self.offset)
+        return None
+
+W_CPPStaticDataMember.typedef = TypeDef(
+    'CPPStaticDataMember',
+    __get__ = interp2app(W_CPPStaticDataMember.__get__, unwrap_spec=['self', 'args_w']),
+    __set__ = interp2app(W_CPPStaticDataMember.__set__, unwrap_spec=['self', 'args_w']),
+)
+
+
 class W_CPPType(Wrappable):
     _immutable_fields_ = ["name", "handle"]
     def __init__(self, space, name, handle):
@@ -297,7 +312,7 @@
             argtypes.append(argtype)
         if capi.c_is_constructor(self.handle, method_index):
             cls = CPPConstructor
-        elif capi.c_is_static(self.handle, method_index):
+        elif capi.c_is_staticmethod(self.handle, method_index):
             cls = CPPFunction
         else:
             cls = CPPMethod
@@ -309,7 +324,10 @@
             data_member_name = capi.charp2str_free(capi.c_data_member_name(self.handle, i))
             cpptype = capi.charp2str_free(capi.c_data_member_type(self.handle, i))
             offset = capi.c_data_member_offset(self.handle, i)
-            data_member = W_CPPDataMember(self.space, cpptype, offset)
+            if capi.c_is_staticdata(self.handle, i):
+                data_member = W_CPPStaticDataMember(self.space, cpptype, offset)
+            else:
+                data_member = W_CPPDataMember(self.space, cpptype, offset)
             self.data_members[data_member_name] = data_member
 
     def get_method_names(self):

diff --git a/pypy/module/cppyy/converter.py b/pypy/module/cppyy/converter.py
--- a/pypy/module/cppyy/converter.py
+++ b/pypy/module/cppyy/converter.py
@@ -5,6 +5,8 @@
 from pypy.rlib.rarithmetic import r_singlefloat
 from pypy.rlib import jit, libffi
 
+from pypy.objspace.std.noneobject import W_NoneObject
+
 from pypy.module._rawffi.interp_rawffi import unpack_simple_shape
 from pypy.module._rawffi.array import W_Array
 
@@ -30,6 +32,13 @@
         rawobject = get_rawobject(space, w_obj)
         return lltype.direct_ptradd(rawobject, offset)
 
+    def _get_address(self, space, w_obj, offset):
+        if space.eq_w(w_obj, space.w_None): # meaning, static/global data
+            address = rffi.cast(rffi.CCHARP, offset)
+        else:
+            address = self._get_fieldptr(space, w_obj, offset)
+        return address
+
     def _is_abstract(self):
         raise NotImplementedError(
             "abstract base class" ) # more detailed part is not rpython: (actual: %s)" % type(self).__name__)
@@ -63,17 +72,16 @@
 
     def from_memory(self, space, w_obj, offset):
         # read access, so no copy needed
-        fieldptr = self._get_fieldptr(space, w_obj, offset)
-        ptrval = rffi.cast(rffi.UINT, fieldptr)
+        address = self._get_address(space, w_obj, offset)
         arr = space.interp_w(W_Array, unpack_simple_shape(space, space.wrap(self.typecode)))
-        return arr.fromaddress(space, fieldptr, self.size)
+        return arr.fromaddress(space, address, self.size)
 
     def to_memory(self, space, w_obj, w_value, offset):
         # copy the full array (uses byte copy for now)
-        fieldptr = self._get_fieldptr(space, w_obj, offset)
+        address = self._get_address(space, w_obj, offset)
         buf = space.interp_w(Buffer, w_value.getslotvalue(2))
         for i in range(min(self.size*self.typesize, buf.getlength())):
-            fieldptr[i] = buf.getitem(i)
+            address[i] = buf.getitem(i)
 
 
 class PtrTypeConverter(ArrayTypeConverter):
@@ -105,21 +113,21 @@
         return rffi.cast(rffi.VOIDP, x)
 
     def from_memory(self, space, w_obj, offset):
-        fieldptr = self._get_fieldptr(space, w_obj, offset)
-        if fieldptr[0] == '\x01':
+        address = self._get_address(space, w_obj, offset)
+        if address[0] == '\x01':
             return space.wrap(True)
         return space.wrap(False)
 
     def to_memory(self, space, w_obj, w_value, offset):
-        fieldptr = self._get_fieldptr(space, w_obj, offset)
+        address = self._get_address(space, w_obj, offset)
         arg = space.c_int_w(w_value)
         if arg != False and arg != True:
             raise OperationError(space.w_TypeError,
                                  space.wrap("boolean value should be bool, or integer 1 or 0"))
         if arg:
-           fieldptr[0] = '\x01'
+           address[0] = '\x01'
         else:
-           fieldptr[0] = '\x00'
+           address[0] = '\x00'
 
 class CharConverter(TypeConverter):
     libffitype = libffi.types.schar
@@ -147,12 +155,12 @@
         return rffi.cast(rffi.VOIDP, x)
 
     def from_memory(self, space, w_obj, offset):
-        fieldptr = self._get_fieldptr(space, w_obj, offset)
-        return space.wrap(fieldptr[0])
+        address = self._get_address(space, w_obj, offset)
+        return space.wrap(address[0])
 
     def to_memory(self, space, w_obj, w_value, offset):
-        fieldptr = self._get_fieldptr(space, w_obj, offset)
-        fieldptr[0] = self._from_space(space, w_value)
+        address = self._get_address(space, w_obj, offset)
+        address[0] = self._from_space(space, w_value)
 
 class LongConverter(TypeConverter):
     libffitype = libffi.types.slong
@@ -171,13 +179,13 @@
         argchain.arg(self._unwrap_object(space, w_obj))
 
     def from_memory(self, space, w_obj, offset):
-        fieldptr = self._get_fieldptr(space, w_obj, offset)
-        longptr = rffi.cast(self.rffiptrtype, fieldptr)
+        address = self._get_address(space, w_obj, offset)
+        longptr = rffi.cast(self.rffiptrtype, address)
         return space.wrap(longptr[0])
 
     def to_memory(self, space, w_obj, w_value, offset):
-        fieldptr = self._get_fieldptr(space, w_obj, offset)
-        longptr = rffi.cast(self.rffiptrtype, fieldptr)
+        address = self._get_address(space, w_obj, offset)
+        longptr = rffi.cast(self.rffiptrtype, address)
         longptr[0] = self._unwrap_object(space, w_value)
 
 class UnsignedLongConverter(LongConverter):
@@ -191,13 +199,13 @@
     libffitype = libffi.types.sshort
 
     def from_memory(self, space, w_obj, offset):
-        fieldptr = self._get_fieldptr(space, w_obj, offset)
-        shortptr = rffi.cast(rffi.SHORTP, fieldptr)
+        address = self._get_address(space, w_obj, offset)
+        shortptr = rffi.cast(rffi.SHORTP, address)
         return space.wrap(shortptr[0])
 
     def to_memory(self, space, w_obj, w_value, offset):
-        fieldptr = self._get_fieldptr(space, w_obj, offset)
-        shortptr = rffi.cast(rffi.SHORTP, fieldptr)
+        address = self._get_address(space, w_obj, offset)
+        shortptr = rffi.cast(rffi.SHORTP, address)
         shortptr[0] = rffi.cast(rffi.SHORT, space.c_int_w(w_value))
 
 class FloatConverter(TypeConverter):
@@ -210,13 +218,13 @@
         return rffi.cast(rffi.VOIDP, x)
 
     def from_memory(self, space, w_obj, offset):
-        fieldptr = self._get_fieldptr(space, w_obj, offset)
-        floatptr = rffi.cast(rffi.FLOATP, fieldptr)
+        address = self._get_address(space, w_obj, offset)
+        floatptr = rffi.cast(rffi.FLOATP, address)
         return space.wrap(float(floatptr[0]))
 
     def to_memory(self, space, w_obj, w_value, offset):
-        fieldptr = self._get_fieldptr(space, w_obj, offset)
-        floatptr = rffi.cast(rffi.FLOATP, fieldptr)
+        address = self._get_address(space, w_obj, offset)
+        floatptr = rffi.cast(rffi.FLOATP, address)
         floatptr[0] = r_singlefloat(space.float_w(w_value))
 
 class DoubleConverter(TypeConverter):
@@ -234,13 +242,13 @@
         argchain.arg(self._unwrap_object(space, w_obj))
 
     def from_memory(self, space, w_obj, offset):
-        fieldptr = self._get_fieldptr(space, w_obj, offset)
-        doubleptr = rffi.cast(rffi.DOUBLEP, fieldptr)
+        address = self._get_address(space, w_obj, offset)
+        doubleptr = rffi.cast(rffi.DOUBLEP, address)
         return space.wrap(doubleptr[0])
 
     def to_memory(self, space, w_obj, w_value, offset):
-        fieldptr = self._get_fieldptr(space, w_obj, offset)
-        doubleptr = rffi.cast(rffi.DOUBLEP, fieldptr)
+        address = self._get_address(space, w_obj, offset)
+        doubleptr = rffi.cast(rffi.DOUBLEP, address)
         doubleptr[0] = space.float_w(w_value)
 
 

diff --git a/pypy/module/cppyy/capi.py b/pypy/module/cppyy/capi.py
--- a/pypy/module/cppyy/capi.py
+++ b/pypy/module/cppyy/capi.py
@@ -81,6 +81,11 @@
     [C_TYPEHANDLE, rffi.INT], C_METHPTRGETTER_PTR,
     compilation_info=eci)
 
+c_is_subtype = rffi.llexternal(
+    "cppyy_is_subtype",
+    [C_TYPEHANDLE, C_TYPEHANDLE], rffi.INT,
+    compilation_info=eci)
+
 c_num_methods = rffi.llexternal(
     "cppyy_num_methods",
     [C_TYPEHANDLE], rffi.INT,
@@ -102,6 +107,15 @@
     [C_TYPEHANDLE, rffi.INT, rffi.INT], rffi.CCHARP,
     compilation_info=eci)
 
+c_is_constructor = rffi.llexternal(
+    "cppyy_is_constructor",
+    [C_TYPEHANDLE, rffi.INT], rffi.INT,
+    compilation_info=eci)
+c_is_staticmethod = rffi.llexternal(
+    "cppyy_is_staticmethod",
+    [C_TYPEHANDLE, rffi.INT], rffi.INT,
+    compilation_info=eci)
+
 c_num_data_members = rffi.llexternal(
     "cppyy_num_data_members",
     [C_TYPEHANDLE], rffi.INT,
@@ -119,18 +133,10 @@
     [C_TYPEHANDLE, rffi.INT], rffi.INT,
     compilation_info=eci)
 
-c_is_constructor = rffi.llexternal(
-    "cppyy_is_constructor",
+c_is_staticdata = rffi.llexternal(
+    "cppyy_is_staticdata",
     [C_TYPEHANDLE, rffi.INT], rffi.INT,
     compilation_info=eci)
-c_is_static = rffi.llexternal(
-    "cppyy_is_static",
-    [C_TYPEHANDLE, rffi.INT], rffi.INT,
-    compilation_info=eci)
-c_is_subtype = rffi.llexternal(
-    "cppyy_is_subtype",
-    [C_TYPEHANDLE, C_TYPEHANDLE], rffi.INT,
-    compilation_info=eci)
 
 c_free = rffi.llexternal(
     "cppyy_free",

From cfbolz at codespeak.net  Sat Feb  5 04:29:27 2011
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Sat,  5 Feb 2011 04:29:27 +0100 (CET)
Subject: [pypy-svn] r80287 - in pypy/extradoc/talk/microsoft-jan-2011: .
	figures
Message-ID: <20110205032927.7D2B52A2002@codespeak.net>

Author: cfbolz
Date: Sat Feb  5 04:29:25 2011
New Revision: 80287

Added:
   pypy/extradoc/talk/microsoft-jan-2011/
   pypy/extradoc/talk/microsoft-jan-2011/figures/
   pypy/extradoc/talk/microsoft-jan-2011/figures/add.dot
   pypy/extradoc/talk/microsoft-jan-2011/figures/add.pdf   (contents, props changed)
   pypy/extradoc/talk/microsoft-jan-2011/talk.pdf   (contents, props changed)
   pypy/extradoc/talk/microsoft-jan-2011/talk.tex
Log:
talk that I gave at Microsoft Research on Monday


Added: pypy/extradoc/talk/microsoft-jan-2011/figures/add.dot
==============================================================================
--- (empty file)
+++ pypy/extradoc/talk/microsoft-jan-2011/figures/add.dot	Sat Feb  5 04:29:25 2011
@@ -0,0 +1,31 @@
+digraph G {
+    a -> b_int [label="Integer"];
+    a -> b_float [label="Float"];
+    a -> b_long [label="Long"];
+    a -> b_string [label="String"];
+    a -> b_list [label="List"];
+    a -> b_tuple [label="Tuple"];
+    a -> b_user [label="User defined"];
+
+    a [label="a", shape=box];
+    b_int [label="b", shape=box];
+    b_float [label="b", shape=box];
+    b_long [label="b", shape=box];
+    b_string [label="b", shape=box];
+    b_list [label="b", shape=box];
+    b_tuple [label="b", shape=box];
+    b_user [label="b", shape=box];
+
+    b_int -> int [label="Integer"];
+    b_int -> float [label="Float"];
+    b_int -> long [label="Long"];
+    b_int -> user [label="User defined"];
+
+    b_float -> float [label="Integer"];
+    b_long -> long [label="Integer"];
+
+    int [label="Integer"];
+    float [label="Float"];
+    long [label="Long"];
+    user [label="User defined"];
+}

Added: pypy/extradoc/talk/microsoft-jan-2011/figures/add.pdf
==============================================================================
Binary file. No diff available.

Added: pypy/extradoc/talk/microsoft-jan-2011/talk.pdf
==============================================================================
Binary file. No diff available.

Added: pypy/extradoc/talk/microsoft-jan-2011/talk.tex
==============================================================================
--- (empty file)
+++ pypy/extradoc/talk/microsoft-jan-2011/talk.tex	Sat Feb  5 04:29:25 2011
@@ -0,0 +1,1067 @@
+\documentclass[utf8x]{beamer}
+
+\mode<presentation>
+{
+  \usetheme{Warsaw}
+
+  %\setbeamercovered{transparent}
+}
+
+
+\usepackage[english]{babel}
+
+\usepackage[utf8x]{inputenc}
+
+\usepackage{times}
+\usepackage[T1]{fontenc}
+\usepackage{ifthen}
+\usepackage{fancyvrb}
+\usepackage{color}
+\usepackage{ulem}
+\usepackage{listings}
+\usepackage{amssymb}
+\usepackage{booktabs}
+\usepackage{wrapfig}
+\usepackage{url}
+\usepackage{alltt}
+\usepackage{tikz}
+\usetikzlibrary{arrows,positioning,shadows,shapes,calc,backgrounds,fit}
+
+%\input{pygmentheader.tex}
+
+\title[PyPy Implements Dynamic Languages Using a Tracing JIT]{
+PyPy's Approach to \\
+Implementing Dynamic Languages \\
+Using a Tracing JIT Compiler}
+
+\author{Carl Friedrich Bolz}
+
+\institute[Heinrich-Heine-Universit?t D?sseldorf]
+{
+  Institut f?r Informatik\\
+  Heinrich-Heine-Universit?t D?sseldorf
+}
+
+\date{
+%\includegraphics[scale=0.25]{figures/hpi_logo_kl.jpg}\\
+%IBM Watson Research Center, February 9th, 2011
+Microsoft Research, January 31st, 2011
+}
+
+%\pgfdeclareimage[height=0.5cm]{pypy-logo}{image/py-web.png}
+%\logo{\pgfuseimage{pypy-logo}}
+
+
+
+% Delete this, if you do not want the table of contents to pop up at
+% the beginning of each subsection:
+%\AtBeginSubsection[]
+%{
+%  \begin{frame}<beamer>
+%    \frametitle{Outline}
+%    \tableofcontents[currentsection,currentsubsection]
+%  \end{frame}
+%}
+
+
+% If you wish to uncover everything in a step-wise fashion, uncomment
+% the following command: 
+
+%\beamerdefaultoverlayspecification{<+->}
+
+
+\begin{document}
+
+\begin{frame}
+  \titlepage
+\end{frame}
+
+
+\begin{frame}
+  \frametitle{Scope}
+  This talk is about:
+
+  \begin{itemize}
+  \item implementing dynamic languages \par(with a focus on complicated ones)
+  \item in a context of limited resources \par(academic, open source, or
+    domain-specific)
+  \item imperative, object-oriented languages
+  \item single-threaded implementations
+  \end{itemize}
+  \pause
+  \begin{block}{Goals}
+    Reconciling:
+    \begin{itemize}
+    \item flexibility, maintainability (because languages evolve)
+    \item simplicity (because teams are small)
+    \item performance
+    \end{itemize}
+  \end{block}
+\end{frame}
+
+\begin{frame}
+  \frametitle{Outline}
+  \tableofcontents[pausesections]
+  % You might wish to add the option [pausesections]
+\end{frame}
+
+\section{The Difficulties of Implementing Dynamic Languages}
+
+\subsection{Technical Factors}
+
+\begin{frame}
+  \frametitle{What is Needed Anyway}
+  A lot of things are not really different from other languages:
+  \begin{itemize}
+      \item lexer, parser
+      \item (bytecode) compiler
+      \item garbage collector
+      \item object system
+  \end{itemize}
+\end{frame}
+
+% say, but don't write:
+% from pypy perspective, but trying to do a general analysis
+% feedback/discussion welcome
+%  XXX random thought: a lot of semantics implemented on VM level, operations
+%  XXX other direction: PyPy has granularity issues in opposite direction
+
+
+\begin{frame}
+  \frametitle{Control Flow}
+    \begin{itemize}
+        \item every language implementation needs a way to implement the control flow of the language
+        \item trivially and slowly done in interpreters with AST or bytecode
+        \item technically very well understood
+        \item sometimes small difficulties, like generators in Python
+        \pause
+        \item some languages have more complex demands\\
+        but this is rare
+        \item examples: Prolog
+    \end{itemize}
+\end{frame}
+
+\begin{frame}
+  \frametitle{Late Binding}
+  \begin{itemize}
+      \item lookups can be done only at runtime
+      \item historically, dynamic languages have moved to ever later binding times
+      \item a large variety of mechanisms exist in various languages
+      \item mechanism are often very ad-hoc because \\
+      "it was easy to do in an interpreter"
+  \end{itemize}
+\end{frame}
+
+\begin{frame}
+  \frametitle{Late Binding in Python}
+  In Python, the following things are late bound
+  \begin{itemize}
+      \item global names
+      \item modules
+      \item instance variables
+      \item methods
+      \pause
+      \item the class of objects
+      \item class hierarchy
+  \end{itemize}
+\end{frame}
+
+
+\begin{frame}
+  \frametitle{Dispatching}
+  \begin{itemize}
+      \item dispatching is a very important special case of late binding
+      \item how are the operations on objects implemented?
+      \item this is usually very complex, and different between languages
+      \pause
+      \item operations are internally split up into one or several lookup and call steps
+      \item a huge space of paths ensues
+      \item most of the paths are uncommon
+  \end{itemize}
+\end{frame}
+
+\begin{frame}
+  \frametitle{Example: Attribute Reads in Python}
+  What happens when an attribute \texttt{x.m} is read? (simplified)
+  \begin{itemize}
+      \item check for the presence of \texttt{x.\_\_getattribute\_\_}, if there, call it
+      \pause
+      \item look for the name of the attribute in the object's dictionary, if it's there, return it
+      \pause
+      \item walk up the MRO of the object and look in each class' dictionary for the attribute
+      \pause
+      \item if the attribute is found, call its \texttt{\_\_get\_\_} attribute and return the result
+      \pause
+      \item if the attribute is not found, look for \texttt{x.\_\_getattr\_\_}, if there, call it
+      \pause
+      \item raise an \texttt{AttributeError}
+  \end{itemize}
+\end{frame}
+
+\begin{frame}
+  \frametitle{Example: Addition in Python}
+  \includegraphics[scale=0.5]{figures/add.pdf}
+\end{frame}
+
+\begin{frame}
+  \frametitle{Dependencies Between Subsequent Dispatches}
+  \begin{itemize}
+      \item one dispatch operation is complex
+      \item many in a sequence are worse
+      \item take \texttt{(a + b) + c}
+      \item the dispatch decision of the first operation influences the second
+  \end{itemize}
+\end{frame}
+
+
+
+
+\begin{frame}
+  \frametitle{Boxing of Primitive Values}
+  \begin{itemize}
+      \item primitive values often need to be boxed,\\
+      to ensure uniform access
+      \item a lot of pressure is put on the GC by arithmetic
+      \item need a good GC (clear anyway)
+      \item in arithmetic, lifetime of boxes is known
+  \end{itemize}
+%  \pause
+%  \begin{block}{Escape Analysis?}
+%    \begin{itemize}
+%        \item should help in theory
+%        \item nearly always there are some unlikely paths,\\
+%        that lead to escapes
+%        \item this is often due to user-overriding of operations
+%    \end{itemize}
+%  \end{block}
+\end{frame}
+
+\begin{frame}
+  \frametitle{Escaping Paths}
+  \begin{itemize}
+      \item considering again \texttt{(a + b) + c}
+      \item assume \texttt{a} and \texttt{b} are ints
+      \item then the result should not be allocated
+      \item escaping path: if \texttt{c} has a user-defined class
+  \end{itemize}
+\end{frame}
+
+
+
+\begin{frame}
+  \frametitle{(Frames)}
+  \begin{itemize}
+      \item side problem:
+      \item many languages have reified frame access
+      \item e.g. Python, Smalltalk, Ruby, ...
+      \item support for in-language debuggers
+      \item in an interpreter these are trivial,\\
+      because the interpreter needs them anyway
+      \item how should reified frames work efficiently when a compiler is used?
+  \end{itemize}
+\end{frame}
+
+%\begin{frame}
+%  \frametitle{(Fast Access to Native Libraries)}
+%  \begin{itemize}
+%      \item another side problem
+%      \item how can native libraries be accessed efficiently?
+%      \item important in dynamic languages, because they are often used as glue
+%      \item even more important in browser, to interact with the DOM
+%      \item can be solved either by compiling extensions, or via an FFI
+%      \item both not very efficient
+%  \end{itemize}
+%\end{frame}
+
+\subsection{Requirements}
+
+\begin{frame}
+  \frametitle{Summarizing the Requirements}
+  \begin{enumerate}
+      \item control flow
+      \item late binding
+      \item \textbf{dispatching}
+      \item \textbf{dependencies between subsequent dispatches}
+      \item boxing
+      \item (reified frames)
+%      \item (access to native libraries)
+  \end{enumerate}
+\end{frame}
+
+
+\section{Approaches For Dynamic Language Implementation}
+
+\begin{frame}
+  \frametitle{Common Approaches to Language Implementation}
+  \begin{itemize}
+    \item Using C/C++
+    \begin{itemize}
+      \item for an interpreter
+      \item for a static compiler
+      \item for a method-based JIT
+      \item for a tracing JIT
+    \end{itemize}
+    \item Building on top of a general-purpose OO VM
+  \end{itemize}
+\end{frame}
+
+\begin{frame}
+  \frametitle{Common Approaches to Language Implementation}
+  \begin{block}{
+    Using C/C++}
+    \begin{itemize}
+    \item CPython (interpreter)
+    \item Ruby (interpreter)
+    \item V8 (method-based JIT)
+    \item TraceMonkey (tracing JIT)
+    \item ...
+    %\item but also: Scheme48, Squeak (interpreters)
+    \end{itemize}
+  \end{block}
+  \pause
+  \begin{block}{Building on top of a general-purpose OO VM}
+    \begin{itemize}
+    \item Jython, IronPython
+    \item JRuby, IronRuby
+    \item various Prolog, Lisp, even Smalltalk implementations
+    \end{itemize}
+  \end{block}
+\end{frame}
+
+\subsection{Implementing VMs in C/C++}
+
+\begin{frame}
+  \frametitle{Implementing VMs in C}
+  When writing a VM in C it is hard to reconcile our goals
+  \begin{itemize}
+  \item flexibility, maintainability
+  \item simplicity
+  \item performance
+  \end{itemize}
+  \pause
+  \begin{block}{Python Case}
+    \begin{itemize}
+    \item \alert{CPython} is a very simple bytecode VM, performance not great
+    \item \alert{Psyco} is a just-in-time-specializer, very complex, hard to
+      maintain, but good performance
+    \item \alert{Stackless} is a fork of CPython adding microthreads. It was
+    never incorporated into CPython for complexity reasons
+    \end{itemize}
+  \end{block}
+\end{frame}
+
+\begin{frame}
+  \frametitle{Interpreters in C/C++}
+  \begin{itemize}
+      \item mostly very easy
+      \item well understood problem
+      \item portable, maintainable
+      \item slow
+  \end{itemize}
+
+\end{frame}
+
+
+%\begin{frame}
+%  \frametitle{Fixing of Early Design Decisions}
+%  \begin{itemize}
+%  \item when starting a VM in C, many design decisions need to be made upfront
+%  \item examples: memory management technique, threading model
+%  \item such decisions are manifested throughout the VM source
+%  \item very hard to change later
+%  \item low level details mixed with language semantics
+%  \end{itemize}
+%  \pause
+%  \begin{block}{Python Case}
+%    \begin{itemize}
+%    \item CPython uses reference counting, increfs and decrefs everywhere
+%    \item CPython uses OS threads with one global lock, hard to change to
+%      lightweight threads or finer locking
+%    \end{itemize}
+%  \end{block}
+%\end{frame}
+
+
+\begin{frame}
+  \frametitle{How do Interpreters Meet the Requirements?}
+  \begin{enumerate}
+      \item \alert{control flow slowed by bytecode or AST dispatch overhead}
+      \item \alert{late binding works but slow}
+      \item \alert{dispatching works but slow}
+      \item \alert{no special dependencies support}
+      \item \alert{everything is boxed}
+      \item \alert{reified frames are easy but slow}
+%      \item \alert{access to native libraries easy but slow}
+  \end{enumerate}
+\end{frame}
+
+
+
+\begin{frame}
+  \frametitle{Static Compilers to C/C++}
+  \begin{itemize}
+      \item first reflex of many people is to blame it all on bytecode dispatch overhead
+      \item thus static compilers are implemented that reuse the object model of an interpreter
+      \item gets rid of interpretation overhead only
+      \item seems to give about 2x speedup
+      \pause
+      \item dispatch, late-binding and boxing only marginally improved
+      \item static analysis mostly never works
+%      \item often easy access to native libraries
+  \end{itemize}
+  \pause
+  \begin{block}{Python Case}
+    \begin{itemize}
+    \item \alert{Cython}, \alert{Pyrex} are compilers\\
+    from large subsets of Python to C
+    \item lots of older experiments, most discontinued
+    \end{itemize}
+  \end{block}
+\end{frame}
+
+
+\begin{frame}
+  \frametitle{How do Static Compilers Meet the Requirements?}
+  \begin{enumerate}
+      \item control flow works well
+      \item \alert{late binding not improved}
+      \item \alert{dispatching not improved}
+      \item \alert{dependencies not improved}
+      \item \alert{everything is boxed}
+      \item \alert{reified frames often not supported}
+%      \item direct access to native libraries
+  \end{enumerate}
+\end{frame}
+
+
+\subsection{Method-Based JIT Compilers}
+
+\begin{frame}
+  \frametitle{Method-Based JIT Compilers}
+  \begin{itemize}
+      \item to fundamentally attack some of the problems, \\
+      a dynamic compiler is needed
+      \item a whole new can of worms
+      \pause
+      \begin{itemize}
+          \item type profiling
+          \item inlining based on that
+          \item general optimizations
+          \item complex backends
+      \end{itemize}
+      \item very hard to pull off for a volunteer team
+  \end{itemize}
+  \pause
+  \begin{block}{Examples}
+      \begin{itemize}
+          \item Smalltalk and SELF JITs
+          \item V8 and J?gerMonkey
+          \item Psyco, sort of
+      \end{itemize}
+  \end{block}
+\end{frame}
+
+\begin{frame}
+  \frametitle{Compilers are a bad encoding of Semantics}
+  \begin{itemize}
+  \item to improve all complex corner cases of the language, \\
+  a huge effort is needed
+  \item often needs a big "bag of tricks"
+  \item the interactions between all tricks is hard to foresee
+  \item the encoding of language semantics in the compiler is thus often obscure and hard to change
+  \end{itemize}
+  \pause
+  \begin{block}{
+    Python Case}
+    \begin{itemize}
+    \item Psyco is a dynamic compiler for Python
+    \item synchronizing with CPython's development is a lot of effort
+    \item many of CPython's new features not supported well
+    \item not ported to 64-bit machines, and probably never will
+    \end{itemize}
+  \end{block}
+\end{frame}
+
+\begin{frame}[containsverbatim]
+  \frametitle{Method-Based JITs and Dispatching Dependencies}
+\begin{verbatim}
+    x = add(a, b)
+    r = add(x, c)
+\end{verbatim}
+\end{frame}
+
+\begin{frame}[containsverbatim, plain]
+  \frametitle{Method-Based JITs and Dispatching Dependencies}
+\begin{verbatim}
+    if isinstance(a, Integer):
+        if isinstance(b, Integer):
+            x = Integer(<perform int addition>)
+        else:
+            x = Float(<perform float addition>)
+    else:
+        x = Float(<perform float addition>)
+    # x can be Float or Integer here
+    if isinstance(x, Integer):
+        if isinstance(c, Integer):
+            r = Integer(<perform int addition>)
+        else:
+            r = Float(<perform float addition>)
+    else:
+        r = Float(<perform float addition>)
+\end{verbatim}
+\end{frame}
+
+\begin{frame}
+  \frametitle{How do Method-Based JIT Compilers Meet the Requirements?}
+  \begin{enumerate}
+      \item control flow works well
+      \item late binding can be handled
+      \item dispatching can be handled
+      \item \alert{dependencies not necessarily improved}
+      \item boxing hard to support
+      \item \alert{reified frames hard to support}
+%      \item \alert{access to native libraries not improved}
+  \end{enumerate}
+\end{frame}
+
+
+
+\subsection{Tracing JIT Compilers}
+
+\begin{frame}
+  \frametitle{Tracing JIT Compilers}
+  \begin{itemize}
+      \item relatively recent approach to JIT compilers
+      \item pioneered by Michael Franz and Andreas Gal for Java
+      \item turned out to be well-suited for dynamic languages
+  \end{itemize}
+  \pause
+  \begin{block}{Examples}
+      \begin{itemize}
+          \item \alert{TraceMonkey}
+          \item \alert{LuaJIT}
+          \item \alert{SPUR}, sort of
+          \item \alert{PyPy}, sort of
+      \end{itemize}
+  \end{block}
+\end{frame}
+
+\begin{frame}
+    \frametitle{Tracing JIT Compilers}
+    \begin{itemize}
+    \item idea from Dynamo project: \\
+    dynamic rewriting of machine code
+    \item conceptually simpler than type profiling
+    \end{itemize}
+    \pause
+    \begin{block}{Basic Assumption of a Tracing JIT}
+        \begin{itemize}
+        \item programs spend most of their time executing loops
+        \item several iterations of a loop are likely to take similar code paths
+        \end{itemize}
+    \end{block}
+\end{frame}
+
+\begin{frame}
+    \frametitle{Tracing VMs}
+    \begin{itemize}
+    \item mixed-mode execution environment
+    \item at first, everything is interpreted
+    \item lightweight profiling to discover hot loops
+    \item code generation only for common paths of hot loops
+    \item when a hot loop is discovered, start to produce a trace
+    \end{itemize}
+\end{frame}
+
+\begin{frame}
+    \frametitle{Tracing}
+    \begin{itemize}
+    \item a \emph{trace} is a sequential list of operations
+    \item a trace is produced by recording every operation the interpreter executes
+    \item tracing ends when the tracer sees a position in the program it has seen before
+    \item a trace thus corresponds to exactly one loop
+    \item that means it ends with a jump to its beginning
+    \end{itemize}
+    \pause
+    \begin{block}{Guards}
+        \begin{itemize}
+        \item the trace is only one of the possible code paths through the loop
+        \item at places where the path \emph{could} diverge, a guard is placed
+        \end{itemize}
+    \end{block}
+\end{frame}
+
+\begin{frame}
+    \frametitle{Code Generation and Execution}
+    \begin{itemize}
+    \item being linear, the trace can easily be turned into machine code
+    \item execution stops when a guard fails
+    \item after a guard failure, go back to interpreting program
+    \end{itemize}
+\end{frame}
+
+\begin{frame}
+  \frametitle{Dealing With Control Flow}
+  \begin{itemize}
+      \item an if statement in a loop is turned into a guard
+      \item if that guard fails often, things are inefficient
+      \item solution: attach a new trace to a guard, if it fails often enough
+      \item new trace can lead back to same loop
+      \item or to some other loop
+  \end{itemize}
+\end{frame}
+
+%\begin{frame}
+%  \frametitle{Stages of Execution}
+%  \includegraphics[scale=0.5]{figures/tracing.pdf}
+%\end{frame}
+
+\begin{frame}
+  \frametitle{Dispatching in a Tracing JIT}
+  \begin{itemize}
+      \item trace contains bytecode operations
+      \item bytecodes often have complex semantics
+      \item optimizer often type-specializes the bytecodes
+      \item according to the concrete types seen during tracing
+      \item need to duplicate language semantics in optimizer for that
+  \end{itemize}
+\end{frame}
+
+
+\begin{frame}
+  \frametitle{Dispatching Dependencies in a Tracing JIT}
+  \begin{itemize}
+      \item one consequence of the tracing approach:
+      \item paths are split aggressively
+      \item control flow merging happens at beginning of loop only
+      \item after a type check, the rest of the trace can assume that type
+      \item only deal with paths that are actually seen
+  \end{itemize}
+\end{frame}
+
+\begin{frame}
+  \frametitle{Boxing Optimizations in a Tracing JIT}
+  \begin{itemize}
+      \item possibility to do escape analysis within the trace
+      \item only optimize common path
+      \item i.e. the one where the object doesn't escape
+  \end{itemize}
+\end{frame}
+
+
+\begin{frame}
+    \frametitle{Advantages of Tracing JITs}
+    \begin{itemize}
+    \item can be added to an existing interpreter unobtrusively
+    \item interpreter does most of the work
+    \item automatic inlining
+    \item deals well with finding the few common paths through the large space
+    \end{itemize}
+\end{frame}
+
+\begin{frame}
+    \frametitle{Bad Points of the Approach}
+    \begin{itemize}
+        \item switching between interpretation and machine code execution takes time
+        \item problems with really complex control flow
+        \item granularity issues: often interpreter bytecode is too coarse
+        \item if this is the case, the optimizer needs to carefully re-add the decision tree
+    \end{itemize}
+\end{frame}
+
+\begin{frame}
+  \frametitle{How do Tracing JITs Meet the Requirements?}
+  \begin{enumerate}
+      \item control flow works rather well
+      \item late binding can be handled
+      \item dispatching can be handled
+      \item dependencies improved by path splitting
+      \item unboxing optimization much simpler
+      \item reified frames can be implemented \\
+      by falling back to the interpreter
+%      \item \alert{access to native libraries not improved}
+  \end{enumerate}
+\end{frame}
+
+
+\subsection{Building on Top of an OO VM}
+
+\begin{frame}
+  \frametitle{Implementing Languages on Top of OO VMs}
+  \begin{itemize}
+  \item approach: implement on top of the JVM or the CLR
+  \item usually by compiling to the target bytecode
+  \item plus an object model implementation
+  \item brings its own set of benefits of problems
+  \end{itemize}
+  \pause
+  \begin{block}{
+    Python Case}
+    \begin{itemize}
+    \item \alert{Jython} is a Python-to-Java-bytecode compiler
+    \item \alert{IronPython} is a Python-to-CLR-bytecode compiler
+    \end{itemize}
+  \end{block}
+\end{frame}
+
+\begin{frame}
+  \frametitle{Benefits of Implementing on Top of OO VMs}
+  \begin{itemize}
+  \item higher level of implementation
+  \item the VM supplies a GC and a JIT
+  \item better interoperability than what the C level provides
+  \end{itemize}
+  \pause
+  \begin{block}{
+    Python Case}
+    \begin{itemize}
+    \item both Jython and IronPython integrate well with their host OO VM
+    \item both have proper threading
+    \end{itemize}
+  \end{block}
+\end{frame}
+
+\begin{frame}
+  \frametitle{The Problems of OO VMs}
+  \begin{itemize}
+      \item often hard to map concepts of the dynamic language
+      \item performance not improved because of the semantic mismatch
+      \item untypical code in most object models
+      \item object model typically has many megamorphic call sites
+      \pause
+      \item escape analysis cannot help with boxing,\\
+      due to escaping paths
+      \item to improve, very careful manual tuning is needed
+      \item VM does not provide enough customization/feedback
+  \end{itemize}
+\end{frame}
+
+\begin{frame}
+  \frametitle{Examples of Problems}
+  \begin{itemize}
+      \item both Jython and IronPython are quite a bit slower than CPython
+      \item IronPython misses reified frames
+      \pause
+      \item for languages like Prolog it is even harder to map the concepts
+  \end{itemize}
+\end{frame}
+
+ 
+\begin{frame}
+  \frametitle{The Future of OO VMs?}
+  \begin{itemize}
+  \item the problems described might improve in the future
+  \item JVM will add extra support for more languages
+  \item i.e. tail calls, \texttt{InvokeDynamic}, ...
+  \item has not really landed yet
+  \item good performance needs a huge amount of tweaking
+  \item controlling the VM's behaviour is brittle:\\
+  VMs not meant for people who care about exact shape of assembler
+  \end{itemize}
+  \pause
+  \begin{block}{Ruby Case}
+    \begin{itemize}
+    \item JRuby tries really hard to be a very good implementations
+    \item took an enormous amount of effort
+    \item tweaking is essentially Hotspot-specific
+    \end{itemize}
+  \end{block}
+\end{frame}
+
+\begin{frame}
+  \frametitle{How do OO VMs Meet the Requirements?}
+  \begin{enumerate}
+      \item control flow works well
+      \item late binding can be handled with a lot of effort
+      \item dispatching can be handled with a lot of effort
+      \item \alert{dependencies not improved}
+      \item \alert{boxing not necessarily improved}
+      \item \alert{reified frames are inefficient}
+%      \item access to native libraries not improved, but VM libraries help
+  \end{enumerate}
+\end{frame}
+
+\section{PyPy's Approach to VM Construction}
+
+
+\begin{frame}
+  \frametitle{The PyPy Project}
+  \begin{itemize}
+      \item started in 2003, received funding from the EU, Google, Nokia and some smaller companies
+      \item goal: "The PyPy project aims at producing a flexible and fast Python implementation."
+      \item technology should be reusable for other dynamic languages
+  \end{itemize}
+  \pause
+  \begin{block}{Language Status}
+    \begin{itemize}
+        \item the fastest Python implementation, very complete
+        \item contains a reasonably good Prolog
+        \item full Squeak, but no JIT for that yet
+        \item various smaller experiments (JavaScript, Scheme, Haskell)
+    \end{itemize}
+      
+  \end{block}
+\end{frame}
+
+\begin{frame}
+  \frametitle{PyPy's Approach to VM Construction}
+  \emph{Goal: achieve flexibility, simplicity and performance together}
+
+  \begin{itemize}
+  \item Approach: auto-generate VMs from high-level descriptions of the language
+  \item ... using meta-programming techniques and \emph{aspects}
+  \item high-level description: an interpreter written in a high-level language
+  \item ... which we translate (i.e.\ compile) to a VM running in various target
+    environments, like C/Posix\pause, CLR, JVM
+  \end{itemize}
+\end{frame}
+
+\begin{frame}
+  \frametitle{PyPy}
+  \begin{itemize}
+  \item PyPy = Python interpreter written in RPython + translation toolchain
+    for RPython
+  \end{itemize}
+  \pause
+  \begin{block}{What is RPython}
+    \begin{itemize}
+    \item RPython is a (large) subset of Python
+    \item subset chosen in such a way that type-inference can be performed
+    \item still a high-level language (unlike SLang or PreScheme)
+    \end{itemize}
+  \end{block}
+\end{frame}
+
+\begin{frame}
+  \frametitle{Auto-generating VMs}
+  \begin{itemize}
+  \item we need a custom \emph{translation toolchain} to compile the interpreter
+    to a full VM
+  \item many aspects of the final VM are orthogonal from the interpreter source:
+    they are inserted during translation
+  \end{itemize}
+  \pause
+  \begin{block}{
+    Examples}
+    \begin{itemize}
+    \item Garbage Collection strategy
+%    \item Threading models (e.g.\ coroutines with CPS...)
+    \item non-trivial translation aspect: auto-generating a tracing JIT compiler from
+      the interpreter
+    \end{itemize}
+  \end{block}
+\end{frame}
+
+
+\begin{frame}
+  \frametitle{Good Points of the Approach}
+  {\bf Simplicity:} separation of language semantics from low-level details
+  \pause
+
+  {\bf Flexibility} high-level implementation language eases things (meta-programming)
+  \pause
+
+  {\bf Performance:} ``reasonable'' baseline performance, can be very good with JIT
+\end{frame}
+
+
+\begin{frame}[plain]
+  \includegraphics[scale=0.3]{figures/all_numbers.png}
+\end{frame}
+
+\subsection{PyPy's Meta-Tracing JIT Compiler}
+
+%\begin{frame}
+%  \frametitle{Example}
+%XXX the idea of this section is to have a running example of a small
+%interpreter that is extended with language constructs to provide examples for
+%all the technologies
+%
+%\end{frame}
+
+\begin{frame}
+  \frametitle{Meta-Tracing}
+  Problems of Tracing JITs:
+  \begin{itemize}
+      \item specific to one language's bytecode
+      \item bytecode has wrong granularity
+      \item internals of an operation not visible in trace
+  \end{itemize}
+  \pause
+  \begin{block}{PyPy's Idea:}
+      \begin{itemize}
+          \item write interpreters in RPython
+          \item trace the execution of the RPython code
+          \item using one generic RPython tracer
+          \item the process is customized via hints in the interpreter
+          \item no language-specific bugs
+      \end{itemize}
+  \end{block}
+\end{frame}
+
+\begin{frame}
+  \frametitle{Interpreter Overhead}
+  \begin{itemize}
+      \item most immediate problem with meta-tracing
+      \item interpreter typically has a bytecode dispatch loop
+      \item not a good idea to trace that
+      \pause
+      \item solved by a simple trick:
+      \item \emph{unroll} the bytecode dispatch loop
+      \item control flow then taken care of
+  \end{itemize}
+\end{frame}
+
+\begin{frame}
+  \frametitle{Optimizing Late Binding and Dispatching}
+  \begin{itemize}
+      \item late binding and dispatching code in the interpreter is traced
+      \item as in a normal tracing JIT, the meta-tracer is good at picking common paths
+      \item a number of hints to fine-tune the process
+  \end{itemize}
+\end{frame}
+
+\begin{frame}
+  \frametitle{Optimizing Boxing Overhead}
+  \begin{itemize}
+      \item boxing optimized by a powerful general optimization on traces
+      \item tries to defer allocations for as long as possible
+      \item allocations only happen in those (rare) paths where they are needed
+  \end{itemize}
+  \pause
+  \begin{block}{Use Cases}
+      \begin{itemize}
+          \item arithmetic
+          \item argument holder objects
+          \item frames of inlined functions
+      \end{itemize}
+  \end{block}
+\end{frame}
+
+\begin{frame}
+  \frametitle{Dealing With Reified Frames}
+  \begin{itemize}
+      \item interpreter needs a frame object to store its data anyway
+      \item those frame objects are specially marked
+      \item JIT special-cases them
+      \item their attributes can live in CPU registers/stack
+      \item on reflective access, machine code is left, interpreter continues
+      \pause
+      \item nothing deep, but a lot of engineering
+  \end{itemize}
+\end{frame}
+
+\begin{frame}
+  \frametitle{Feedback from the VM}
+  \begin{itemize}
+      \item in the beginning the hints are often not optimal yet
+      \item to understand how to improve them, the traces must be read
+      \item traces are in a machine-level intermediate representation
+      \item not machine code
+      \item corresponds quite closely to RPython interpreter code
+      \item visualization and profiling tools
+  \end{itemize}
+\end{frame}
+
+
+%\subsection{Customizing the Tracing of the Interpreter}
+%
+%\begin{frame}
+%  \frametitle{Customizing the Tracing of the Interpreter}
+%  - general building blocks for implementing many different dynamic languages well
+%
+%  - more expedient than minimal
+%
+%  without hints, behaviour is correct but not very fast
+%\end{frame}
+%
+%
+%\begin{frame}
+%  \frametitle{The Extent of Tracing}
+%  \begin{itemize}
+%      \item tracer always starts at bytecode dispatch loop
+%      \item at any interpreter-level function call, the tracer can decide to trace into call, or not
+%      \item tracing should not continue arbitrarily deep into implementation of operations
+%      \pause
+%      \item basic mechanism: stop at functions that contain loops
+%      \item hint to stop tracing explicitly
+%      \item hint to force tracing into function with loops, unrolling them
+%  \end{itemize}
+%\end{frame}
+%
+%\begin{frame}
+%  \frametitle{Example}
+%  XXX
+%\end{frame}
+%
+%
+%\begin{frame}
+%  \frametitle{Mechanisms for Dispatching and Late-Binding}
+%  - tracing time computation
+%
+%  - promotion, turn a variable into a constant to insert a guard
+%
+%  - immutability, to fold away reads out of constants
+%
+%  - pure function hints, remove function calls if the arguments are pure
+%
+%\end{frame}
+%
+%\begin{frame}
+%  \frametitle{Example Promotion}
+%  XXX maps
+%\end{frame}
+%
+%\begin{frame}
+%  \frametitle{Example Immutability}
+%  XXX code objects
+%\end{frame}
+%
+%
+%\begin{frame}
+%  \frametitle{Example Pure Function}
+%  XXX version tags
+%\end{frame}
+%
+%\begin{frame}
+%  \frametitle{Out-of-Line Guards}
+%  - soon: slow-changing fields into out-of-line guards
+%\end{frame}
+%
+%\begin{frame}
+%  \frametitle{Example Out-of-Line Guards}
+%  XXX version tags
+%\end{frame}
+
+\begin{frame}
+  \frametitle{Drawbacks / Open Issues / Further Work}
+  \begin{itemize}
+  \item writing the translation toolchain in the first place takes lots of effort
+    (but it can be reused)
+  \item writing a good GC was still necessary, not perfect yet
+  \item dynamic compiler generation seems to work now, but took \emph{very} long to get right
+  \item granularity of tracing is sometimes not optimal, very low level
+  \end{itemize}
+\end{frame}
+
+\begin{frame}
+  \frametitle{Conclusion}
+  \begin{itemize}
+      \item PyPy solves many of the problems of dynamic language implementations
+      \item it uses a high-level language
+      \begin{itemize}
+          \item to ease implementation
+          \item for better analyzability
+      \end{itemize}
+      \item it gives good feedback to the language implementor
+      \item and provides various mechanisms to express deeply different language semantics
+      \pause
+      \item only one solution in this design space (SPUR is another)
+      \item more experiments needed
+  \end{itemize}
+\end{frame}
+
+
+\end{document}
+
+


From commits-noreply at bitbucket.org  Sat Feb  5 11:39:57 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Sat,  5 Feb 2011 11:39:57 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: intbound optimizations should
	not prevent	wrapping
Message-ID: <20110205103957.B910D2A2002@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r41623:9e43517161bf
Date: 2011-02-05 09:02 +0100
http://bitbucket.org/pypy/pypy/changeset/9e43517161bf/

Log:	intbound optimizations should not prevent wrapping

diff --git a/pypy/jit/metainterp/test/test_basic.py b/pypy/jit/metainterp/test/test_basic.py
--- a/pypy/jit/metainterp/test/test_basic.py
+++ b/pypy/jit/metainterp/test/test_basic.py
@@ -2157,6 +2157,55 @@
         res = self.interp_operations(f, [])
         assert res
 
+    def test_wrap_around_add(self):
+        myjitdriver = JitDriver(greens = [], reds = ['x', 'n'])
+        class A:
+            pass
+        def f(x):
+            n = 0
+            while x > 0:
+                myjitdriver.can_enter_jit(x=x, n=n)
+                myjitdriver.jit_merge_point(x=x, n=n)
+                x += 1
+                n += 1
+            return n
+        res = self.meta_interp(f, [sys.maxint-10])
+        assert res == 11
+        self.check_tree_loop_count(2)
+
+    def test_wrap_around_mul(self):
+        myjitdriver = JitDriver(greens = [], reds = ['x', 'n'])
+        class A:
+            pass
+        def f(x):
+            n = 0
+            while x > 0:
+                myjitdriver.can_enter_jit(x=x, n=n)
+                myjitdriver.jit_merge_point(x=x, n=n)
+                x *= 2
+                n += 1
+            return n
+        res = self.meta_interp(f, [sys.maxint>>10])
+        assert res == 11
+        self.check_tree_loop_count(2)        
+
+    def test_wrap_around_sub(self):
+        myjitdriver = JitDriver(greens = [], reds = ['x', 'n'])
+        class A:
+            pass
+        def f(x):
+            n = 0
+            while x < 0:
+                myjitdriver.can_enter_jit(x=x, n=n)
+                myjitdriver.jit_merge_point(x=x, n=n)
+                x -= 1
+                n += 1
+            return n
+        res = self.meta_interp(f, [10-sys.maxint])
+        assert res == 12
+        self.check_tree_loop_count(2)        
+
+
 
 class TestOOtype(BasicTests, OOJitMixin):
 
@@ -2238,9 +2287,6 @@
                                optimizer=OPTIMIZER_SIMPLE)
         assert res
 
-
-
-
 class BaseLLtypeTests(BasicTests):
 
     def test_identityhash(self):

From commits-noreply at bitbucket.org  Sat Feb  5 11:39:58 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Sat,  5 Feb 2011 11:39:58 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: support for forced virtuals
Message-ID: <20110205103958.957A42A2002@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r41624:a333f1a9c94e
Date: 2011-02-05 09:21 +0100
http://bitbucket.org/pypy/pypy/changeset/a333f1a9c94e/

Log:	support for forced virtuals

diff --git a/pypy/jit/metainterp/optimizeopt/unroll.py b/pypy/jit/metainterp/optimizeopt/unroll.py
--- a/pypy/jit/metainterp/optimizeopt/unroll.py
+++ b/pypy/jit/metainterp/optimizeopt/unroll.py
@@ -1,4 +1,5 @@
 from pypy.jit.metainterp.optimizeopt.optimizer import *
+from pypy.jit.metainterp.optimizeopt.virtualize import AbstractVirtualValue
 from pypy.jit.metainterp.resoperation import rop, ResOperation
 from pypy.jit.metainterp.compile import ResumeGuardDescr
 from pypy.jit.metainterp.resume import Snapshot
@@ -160,6 +161,8 @@
             if box in self.fieldboxes:
                 info.fieldstate = [self.state(b) for b in self.fieldboxes[box]]
                 # FIXME: Do we realy want to add fields to the VirtualInfo's?
+            elif isinstance(value, AbstractVirtualValue):
+                info.fieldstate = []
         return info
 
     def get_virtual_state(self, jump_args):

diff --git a/pypy/jit/metainterp/optimizeopt/virtualize.py b/pypy/jit/metainterp/optimizeopt/virtualize.py
--- a/pypy/jit/metainterp/optimizeopt/virtualize.py
+++ b/pypy/jit/metainterp/optimizeopt/virtualize.py
@@ -99,7 +99,10 @@
             len(self._fields) == len(_cached_sorted_fields)):
             lst = self._cached_sorted_fields
         else:
-            lst = self._fields.keys()
+            if self._fields is None:
+                lst = []
+            else:
+                lst = self._fields.keys()
             sort_descrs(lst)
             cache = get_fielddescrlist_cache(self.optimizer.cpu)
             result = cache.get(lst, None)

From commits-noreply at bitbucket.org  Sat Feb  5 11:39:59 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Sat,  5 Feb 2011 11:39:59 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: retraces in more situations now
Message-ID: <20110205103959.2CA932A2002@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r41625:dc8a3dd2bff0
Date: 2011-02-05 10:06 +0100
http://bitbucket.org/pypy/pypy/changeset/dc8a3dd2bff0/

Log:	retraces in more situations now

diff --git a/pypy/jit/metainterp/test/test_basic.py b/pypy/jit/metainterp/test/test_basic.py
--- a/pypy/jit/metainterp/test/test_basic.py
+++ b/pypy/jit/metainterp/test/test_basic.py
@@ -406,10 +406,10 @@
         res = self.meta_interp(f, [6, 32])
         assert res == 1167
         self.check_loop_count(3)
-        self.check_loops({'int_add': 2, 'int_lt': 1,
+        self.check_loops({'int_add': 3, 'int_lt': 2,
                           'int_sub': 2, 'guard_false': 1,
                           'jump': 2,
-                          'int_gt': 1, 'guard_true': 1, 'int_mul': 1})
+                          'int_gt': 1, 'guard_true': 2})
 
 
     def test_loop_invariant_mul_bridge_maintaining2(self):
@@ -428,10 +428,31 @@
         res = self.meta_interp(f, [6, 32])
         assert res == 1692
         self.check_loop_count(3)
+        self.check_loops({'int_add': 3, 'int_lt': 2,
+                          'int_sub': 2, 'guard_false': 1,
+                          'jump': 2,
+                          'int_gt': 1, 'guard_true': 2})
+
+    def test_loop_invariant_mul_bridge_maintaining3(self):
+        myjitdriver = JitDriver(greens = [], reds = ['y', 'res', 'x', 'm'])
+        def f(x, y, m):
+            res = 0
+            while y > 0:
+                myjitdriver.can_enter_jit(x=x, y=y, res=res, m=m)
+                myjitdriver.jit_merge_point(x=x, y=y, res=res, m=m)
+                z = x * x
+                res += z
+                if y<m:
+                    res += z
+                y -= 1
+            return res
+        res = self.meta_interp(f, [6, 32, 16])
+        assert res == 1692
+        self.check_loop_count(3)
         self.check_loops({'int_add': 2, 'int_lt': 1,
                           'int_sub': 2, 'guard_false': 1,
-                          'jump': 2,
-                          'int_gt': 1, 'guard_true': 1, 'int_mul': 1})
+                          'jump': 2, 'int_mul': 1,
+                          'int_gt': 2, 'guard_true': 2})
 
     def test_loop_invariant_intbox(self):
         myjitdriver = JitDriver(greens = [], reds = ['y', 'res', 'x'])
@@ -1478,7 +1499,7 @@
         res = self.meta_interp(f, [299], listops=True)
         assert res == f(299)
         self.check_loops(guard_class=0, guard_value=3)        
-        self.check_loops(guard_class=0, guard_value=6, everywhere=True)
+        self.check_loops(guard_class=0, guard_value=8, everywhere=True)
 
     def test_merge_guardnonnull_guardclass(self):
         from pypy.rlib.objectmodel import instantiate
@@ -1509,7 +1530,7 @@
         self.check_loops(guard_class=0, guard_nonnull=0,
                          guard_nonnull_class=2, guard_isnull=1)
         self.check_loops(guard_class=0, guard_nonnull=0,
-                         guard_nonnull_class=4, guard_isnull=2,
+                         guard_nonnull_class=5, guard_isnull=3,
                          everywhere=True)
 
     def test_merge_guardnonnull_guardvalue(self):
@@ -1539,8 +1560,8 @@
         assert res == f(299)
         self.check_loops(guard_class=0, guard_nonnull=0, guard_value=2,
                          guard_nonnull_class=0, guard_isnull=1)
-        self.check_loops(guard_class=0, guard_nonnull=0, guard_value=4,
-                         guard_nonnull_class=0, guard_isnull=2,
+        self.check_loops(guard_class=0, guard_nonnull=0, guard_value=5,
+                         guard_nonnull_class=0, guard_isnull=3,
                          everywhere=True)
 
     def test_merge_guardnonnull_guardvalue_2(self):
@@ -1570,8 +1591,8 @@
         assert res == f(299)
         self.check_loops(guard_class=0, guard_nonnull=0, guard_value=2,
                          guard_nonnull_class=0, guard_isnull=1)
-        self.check_loops(guard_class=0, guard_nonnull=0, guard_value=4,
-                         guard_nonnull_class=0, guard_isnull=2,
+        self.check_loops(guard_class=0, guard_nonnull=0, guard_value=5,
+                         guard_nonnull_class=0, guard_isnull=3,
                          everywhere=True)
 
     def test_merge_guardnonnull_guardclass_guardvalue(self):
@@ -1604,8 +1625,8 @@
         assert res == f(399)
         self.check_loops(guard_class=0, guard_nonnull=0, guard_value=3,
                          guard_nonnull_class=0, guard_isnull=1)
-        self.check_loops(guard_class=0, guard_nonnull=0, guard_value=6,
-                         guard_nonnull_class=0, guard_isnull=2,
+        self.check_loops(guard_class=0, guard_nonnull=0, guard_value=8,
+                         guard_nonnull_class=0, guard_isnull=3,
                          everywhere=True)
 
     def test_residual_call_doesnt_lose_info(self):
@@ -1897,8 +1918,8 @@
             return a1.val + b1.val
         res = self.meta_interp(g, [6, 14])
         assert res == g(6, 14)
-        self.check_loop_count(8)
-        self.check_loops(getarrayitem_gc=16, everywhere=True)
+        self.check_loop_count(11)
+        self.check_loops(getarrayitem_gc=17, everywhere=True)
 
     def test_multiple_specialied_versions_bridge(self):
         myjitdriver = JitDriver(greens = [], reds = ['y', 'x', 'z', 'res'])
@@ -2107,7 +2128,7 @@
             return main(10, 20) + main(-10, -20)
         res = self.meta_interp(g, [])
         assert res == g()
-        self.check_enter_count(3)
+        self.check_enter_count(2)
 
     def test_current_trace_length(self):
         myjitdriver = JitDriver(greens = ['g'], reds = ['x'])

From commits-noreply at bitbucket.org  Sat Feb  5 11:39:59 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Sat,  5 Feb 2011 11:39:59 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: non ovf int opps might wrap
Message-ID: <20110205103959.BC7632A2002@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r41626:242b80516c16
Date: 2011-02-05 10:15 +0100
http://bitbucket.org/pypy/pypy/changeset/242b80516c16/

Log:	non ovf int opps might wrap

diff --git a/pypy/jit/metainterp/test/test_optimizeopt.py b/pypy/jit/metainterp/test/test_optimizeopt.py
--- a/pypy/jit/metainterp/test/test_optimizeopt.py
+++ b/pypy/jit/metainterp/test/test_optimizeopt.py
@@ -574,16 +574,11 @@
         expected = """
         [i0]
         i1 = int_add(i0, 1)
-        jump(i1)
-        """
-        preamble = """
-        [i0]
-        i1 = int_add(i0, 1)
         i2 = int_gt(i1, 0)
         guard_true(i2) []
         jump(i1)
         """
-        self.optimize_loop(ops, expected, preamble)
+        self.optimize_loop(ops, expected, expected)
 
     def test_int_is_true_is_zero(self):
         py.test.skip("in-progress")
@@ -3429,7 +3424,7 @@
         """
         self.optimize_loop(ops, expected, preamble)
 
-    def test_bound_lt_sub(self):
+    def test_bound_lt_sub1(self):
         ops = """
         [i0]
         i1 = int_lt(i0, 4)
@@ -3444,6 +3439,35 @@
         i1 = int_lt(i0, 4)
         guard_true(i1) []
         i2 = int_sub(i0, 10)
+        i3 = int_lt(i2, -5)
+        guard_true(i3) []        
+        jump(i0)
+        """
+        expected = """
+        [i0]
+        jump(i0)
+        """
+        self.optimize_loop(ops, expected, preamble)
+
+    def test_bound_lt_sub2(self):
+        ops = """
+        [i0]
+        i1 = int_lt(i0, 4)
+        guard_true(i1) []
+        i1p = int_gt(i0, -4)
+        guard_true(i1p) []
+        i2 = int_sub(i0, 10)
+        i3 = int_lt(i2, -5)
+        guard_true(i3) []
+        jump(i0)
+        """
+        preamble = """
+        [i0]
+        i1 = int_lt(i0, 4)
+        guard_true(i1) []
+        i1p = int_gt(i0, -4)
+        guard_true(i1p) []        
+        i2 = int_sub(i0, 10)
         jump(i0)
         """
         expected = """

From commits-noreply at bitbucket.org  Sat Feb  5 11:40:00 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Sat,  5 Feb 2011 11:40:00 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: hg merge default
Message-ID: <20110205104000.9B9CB2A200C@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r41627:1877ab947e2e
Date: 2011-02-05 10:16 +0100
http://bitbucket.org/pypy/pypy/changeset/1877ab947e2e/

Log:	hg merge default


From commits-noreply at bitbucket.org  Sat Feb  5 11:40:03 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Sat,  5 Feb 2011 11:40:03 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: fixed forced case
Message-ID: <20110205104003.DC09A2A2002@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r41628:6fd1e52a389a
Date: 2011-02-05 11:38 +0100
http://bitbucket.org/pypy/pypy/changeset/6fd1e52a389a/

Log:	fixed forced case

diff --git a/pypy/jit/metainterp/optimizeopt/unroll.py b/pypy/jit/metainterp/optimizeopt/unroll.py
--- a/pypy/jit/metainterp/optimizeopt/unroll.py
+++ b/pypy/jit/metainterp/optimizeopt/unroll.py
@@ -199,7 +199,6 @@
             if self.known_class != other.known_class: # FIXME: use issubclass?
                 return False
         return self.intbound.contains_bound(other.intbound)
-            
 
 class UnrollOptimizer(Optimization):
     """Unroll the loop into two iterations. The first one will
@@ -605,8 +604,7 @@
                 args = op.getarglist()
                 modifier = VirtualStateAdder(self.optimizer)
                 virtual_state = modifier.get_virtual_state(args)
-                print 'len', len(short)
-                for sh in short:
+                for sh in short:                                        
                     if sh.virtual_state.more_general_than(virtual_state):
                         # FIXME: Do we still need the dry run
                         if self.inline(sh.operations, sh.inputargs,

diff --git a/pypy/jit/metainterp/test/test_send.py b/pypy/jit/metainterp/test/test_send.py
--- a/pypy/jit/metainterp/test/test_send.py
+++ b/pypy/jit/metainterp/test/test_send.py
@@ -334,6 +334,10 @@
         class State:
             count = 0
             def externfn(self, n):
+                print
+                print
+                print n, 198 - self.count
+                print
                 assert n == 198 - self.count
                 self.count += 1
                 if n % 5:
@@ -439,7 +443,7 @@
         if self.optimizer != OPTIMIZER_FULL:
             self.check_tree_loop_count(1)
         else:
-            self.check_tree_loop_count(2)
+            self.check_tree_loop_count(6)
 
     def test_three_classes(self):
         class Base:

diff --git a/pypy/jit/metainterp/optimizeopt/virtualize.py b/pypy/jit/metainterp/optimizeopt/virtualize.py
--- a/pypy/jit/metainterp/optimizeopt/virtualize.py
+++ b/pypy/jit/metainterp/optimizeopt/virtualize.py
@@ -95,8 +95,12 @@
 
     def _get_field_descr_list(self):
         _cached_sorted_fields = self._cached_sorted_fields
+        if self._fields is None:
+            nfields = 0
+        else:
+            nfields = len(self._fields)
         if (_cached_sorted_fields is not None and
-            len(self._fields) == len(_cached_sorted_fields)):
+            nfields == len(_cached_sorted_fields)):
             lst = self._cached_sorted_fields
         else:
             if self._fields is None:

From lac at codespeak.net  Sat Feb  5 13:46:59 2011
From: lac at codespeak.net (lac at codespeak.net)
Date: Sat,  5 Feb 2011 13:46:59 +0100 (CET)
Subject: [pypy-svn] r80289 - pypy/extradoc/talk/ustour2011
Message-ID: <20110205124659.AFB982A2002@codespeak.net>

Author: lac
Date: Sat Feb  5 13:46:57 2011
New Revision: 80289

Modified:
   pypy/extradoc/talk/ustour2011/google-abstract2.txt
Log:
Add bios, fix prepositions


Modified: pypy/extradoc/talk/ustour2011/google-abstract2.txt
==============================================================================
--- pypy/extradoc/talk/ustour2011/google-abstract2.txt	(original)
+++ pypy/extradoc/talk/ustour2011/google-abstract2.txt	Sat Feb  5 13:46:57 2011
@@ -1,21 +1,49 @@
 
-Authors: Armin Rigo, Maciej Fijalkowski
-# XXX get bios from laura
+Google Tech Talk: Speakers: Armin Rigo, Maciej Fijalkowski
 
-The PyPy project has gathered recently a lot of attention for its
-progress on speeding up the Python language -- it is the fastest,
-most compatible and stable alternative Python interpreter.  We
-will discuss what the PyPy project achieved, particularly in the
-past two years in the area of dynamic (Just-in-Time) interpreters:
+Bios:
+Armin Rigo is a researcher at Heinrich-Heine-Universit?t in D?sseldorf,
+Germany.  His academic interests include Programming Languages and
+Implementation Techniques.
+
+He is the lead designer of the PyPy project and one of its
+original founders.  He is also the author of Psyco, a hand-coded
+Just-in-Time specializing compiler for Python, which can be used
+transparently with 32-bit x86 versions of CPython.  Since 2003 he has
+worked on all aspects of PyPy: its Python interpreter (written in
+Python), its translation toolchain (which produces C code), its
+garbage collectors, and its Tracing Just-in-Time compiler generator.
+Since the end of 2010, the Just in Time compiler generated by PyPy
+has outperformed Psyco, while being much more general and robust.
+
+Maciej Fijalkowski is a core developer of the PyPy project. He
+started contributing to it with the help of the Google Summer of Code
+Project in 2005, when he developed the Javascript backend. Since then,
+he has worked on the core of PyPy, including ctypes, the Just-In-Time
+compiler, garbage collectors and more.
+
+At present he is employed by SKA South Africa to build infrastructure
+for MeerKAT, a very large radio telescope under construction in the
+Karoo desert. The infrastructure is largely python-based and employs
+Twisted and PyPy among other technologies.
+
+Abstract:
+The PyPy project has recently gathered a lot of attention for its
+progress in speeding up the Python language -- it is the fastest,
+most compatible and most stable 'alternative? Python interpreter.  No
+longer merely a research curiosity, PyPy is now suitablefor production
+use.  We will discuss what the PyPy project has achieved, with a particular
+focus on the past two years' work  in the area of dynamic (Just-in-Time) 
+interpreters:
 
 * most Python benchmarks run much faster than with CPython or Psyco
 * the real-world PyPy compiler toolchain itself (200 KLocs) runs twice as fast
-* supports 64bit and (in-progress) ARM
-* full compatibility to CPython (more than Jython/IronPython)
+* already supports 64bit and is in the process of supporting ARM
+* full compatibility with  CPython (more than Jython/IronPython)
 * full (and JIT-ed) ctypes support to call C libraries from Python
 * supports Stackless Python (in-progress)
-* new "cpyext" layer to integrate existing CPython C extensions
-* experimental super-fast JIT-compilation of calls to C++ libraries
+* new "cpyext" layer which integrates existing CPython C extensions
+* an experimental super-fast JIT-compilation of calls to C++ libraries
 
 We want to reserve time for discussing potential future work like SWIG
 and/or Cython compatibility and other areas brought up by the audience.


From commits-noreply at bitbucket.org  Sat Feb  5 14:07:52 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sat,  5 Feb 2011 14:07:52 +0100 (CET)
Subject: [pypy-svn] pypy default: Rewrite nanos.py: instead of directly
	importing from	the os module
Message-ID: <20110205130752.8F4F12A2002@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41629:d5bac79d645e
Date: 2011-02-05 13:28 +0100
http://bitbucket.org/pypy/pypy/changeset/d5bac79d645e/

Log:	Rewrite nanos.py: instead of directly importing from the os module
	and expecting geninterp to do the right thing (i.e. filter out the
	unneeded parts and keep only the declaration of the functions
	actually used), we have to copy explicitly parts of posixpath.py or
	ntpath.py. This is a bit saner anyway, even though it means more
	code. Fixes test_nanos.py at least.

diff --git a/pypy/translator/goal/nanos.py b/pypy/translator/goal/nanos.py
--- a/pypy/translator/goal/nanos.py
+++ b/pypy/translator/goal/nanos.py
@@ -28,17 +28,169 @@
 from pypy.interpreter.gateway import applevel, ObjSpace, W_Root, interp2app
 import os, py
 
-app_os_path = applevel(r'''
-    from os.path import dirname, join, abspath, isfile, islink
-''', filename=__file__)
+if os.name == 'posix':
+    # code copied from posixpath.py
+    app_os_path = applevel("""
+        from posix import getcwd, stat, lstat, error
 
-app_os = applevel(r'''
-    from os import sep, pathsep, getenv, name, fdopen
-    try:
-        from os import readlink
-    except ImportError:
-        pass
-''', filename=__file__)
+        def dirname(p):
+            i = p.rfind('/') + 1
+            head = p[:i]
+            if head and head != '/'*len(head):
+                head = head.rstrip('/')
+            return head
+
+        def join(path, b):
+            if b.startswith('/'):
+                path = b
+            elif path == '' or path.endswith('/'):
+                path +=  b
+            else:
+                path += '/' + b
+            return path
+
+        def abspath(path):
+            if not path.startswith('/'):
+                cwd = getcwd()
+                path = join(cwd, path)
+            return path       # this version does not call normpath()!
+
+        def isfile(path):
+            try:
+                st = stat(path)
+            except error:
+                return False
+            return (st.st_mode & 0170000) == 0100000      # S_ISREG
+
+        def islink(path):
+            try:
+                st = lstat(path)
+            except error:
+                return False
+            return (st.st_mode & 0170000) == 0120000      # S_ISLNK
+
+    """, filename=__file__)
+
+    app_os = applevel("""
+        sep = '/'
+        pathsep = ':'
+        name = 'posix'
+        from posix import fdopen, readlink
+    """, filename=__file__)
+
+elif os.name == 'nt':
+    # code copied from ntpath.py
+    app_os_path = applevel("""
+        from nt import _getfullpathname, getcwd, stat, lstat, error
+
+        def splitdrive(p):
+            if p[1:2] == ':':
+                return p[0:2], p[2:]
+            return '', p
+
+        def isabs(s):
+            s = splitdrive(s)[1]
+            return s != '' and s[:1] in '/\\'
+
+        def dirname(p):
+            d, p = splitdrive(p)
+            # set i to index beyond p's last slash
+            i = len(p)
+            while i and p[i-1] not in '/\\':
+                i = i - 1
+            head = p[:i]
+            # remove trailing slashes from head, unless it's all slashes
+            head2 = head
+            while head2 and head2[-1] in '/\\':
+                head2 = head2[:-1]
+            head = head2 or head
+            return d + head
+
+        def join(path, b):
+            b_wins = 0  # set to 1 iff b makes path irrelevant
+            if path == "":
+                b_wins = 1
+
+            elif isabs(b):
+                # This probably wipes out path so far.  However, it's more
+                # complicated if path begins with a drive letter:
+                #     1. join('c:', '/a') == 'c:/a'
+                #     2. join('c:/', '/a') == 'c:/a'
+                # But
+                #     3. join('c:/a', '/b') == '/b'
+                #     4. join('c:', 'd:/') = 'd:/'
+                #     5. join('c:/', 'd:/') = 'd:/'
+                if path[1:2] != ":" or b[1:2] == ":":
+                    # Path doesn't start with a drive letter, or cases 4 and 5.
+                    b_wins = 1
+
+                # Else path has a drive letter, and b doesn't but is absolute.
+                elif len(path) > 3 or (len(path) == 3 and
+                                       path[-1] not in "/\\"):
+                    # case 3
+                    b_wins = 1
+
+            if b_wins:
+                path = b
+            else:
+                # Join, and ensure there's a separator.
+                assert len(path) > 0
+                if path[-1] in "/\\":
+                    if b and b[0] in "/\\":
+                        path += b[1:]
+                    else:
+                        path += b
+                elif path[-1] == ":":
+                    path += b
+                elif b:
+                    if b[0] in "/\\":
+                        path += b
+                    else:
+                        path += "\\" + b
+                else:
+                    # path is not empty and does not end with a backslash,
+                    # but b is empty; since, e.g., split('a/') produces
+                    # ('a', ''), it's best if join() adds a backslash in
+                    # this case.
+                    path += '\\'
+
+            return path
+
+        def abspath(path):
+            if path: # Empty path must return current working directory.
+                try:
+                    path = _getfullpathname(path)
+                except WindowsError:
+                    pass # Bad path - return unchanged.
+            else:
+                path = getcwd()
+            return path       # this version does not call normpath()!
+
+        def isfile(path):
+            try:
+                st = stat(path)
+            except error:
+                return False
+            return (st.st_mode & 0170000) == 0100000      # S_ISREG
+
+        def islink(path):
+            try:
+                st = lstat(path)
+            except error:
+                return False
+            return (st.st_mode & 0170000) == 0120000      # S_ISLNK
+
+    """, filename=__file__)
+
+    app_os = applevel("""
+        sep = '\\'
+        pathsep = ';'
+        name = 'nt'
+        from posix import fdopen
+    """, filename=__file__)
+
+else:
+    raise NotImplementedError("os.name == %r" % (os.name,))
 
 def getenv(space, w_name):
     name = space.str_w(w_name)

From commits-noreply at bitbucket.org  Sat Feb  5 14:07:53 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sat,  5 Feb 2011 14:07:53 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix for tests.
Message-ID: <20110205130753.241802A2002@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41630:8d93957d6dc5
Date: 2011-02-05 13:32 +0100
http://bitbucket.org/pypy/pypy/changeset/8d93957d6dc5/

Log:	Fix for tests.

diff --git a/pypy/translator/goal/nanos.py b/pypy/translator/goal/nanos.py
--- a/pypy/translator/goal/nanos.py
+++ b/pypy/translator/goal/nanos.py
@@ -211,6 +211,7 @@
 path_module_for_testing = type(os)("os.path")
 os_module_for_testing = type(os)("os")
 os_module_for_testing.path = path_module_for_testing
+os_module_for_testing.getenv = os.getenv
 eval(py.code.Source(app_os_path.source).compile(), path_module_for_testing.__dict__)
 eval(py.code.Source(app_os.source).compile(), os_module_for_testing.__dict__)
 

From commits-noreply at bitbucket.org  Sat Feb  5 14:07:53 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sat,  5 Feb 2011 14:07:53 +0100 (CET)
Subject: [pypy-svn] pypy default: Simplify.
Message-ID: <20110205130753.A7DD32A2002@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41631:b0e2ac3eb998
Date: 2011-02-05 14:07 +0100
http://bitbucket.org/pypy/pypy/changeset/b0e2ac3eb998/

Log:	Simplify.

diff --git a/pypy/translator/goal/nanos.py b/pypy/translator/goal/nanos.py
--- a/pypy/translator/goal/nanos.py
+++ b/pypy/translator/goal/nanos.py
@@ -174,11 +174,7 @@
             return (st.st_mode & 0170000) == 0100000      # S_ISREG
 
         def islink(path):
-            try:
-                st = lstat(path)
-            except error:
-                return False
-            return (st.st_mode & 0170000) == 0120000      # S_ISLNK
+            return False
 
     """, filename=__file__)
 

From commits-noreply at bitbucket.org  Sat Feb  5 14:07:54 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sat,  5 Feb 2011 14:07:54 +0100 (CET)
Subject: [pypy-svn] pypy default: Write some tests.
Message-ID: <20110205130754.9392B2A2009@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41632:eebd24f2941b
Date: 2011-02-05 14:07 +0100
http://bitbucket.org/pypy/pypy/changeset/eebd24f2941b/

Log:	Write some tests.

diff --git a/pypy/translator/goal/test2/test_nanos.py b/pypy/translator/goal/test2/test_nanos.py
--- a/pypy/translator/goal/test2/test_nanos.py
+++ b/pypy/translator/goal/test2/test_nanos.py
@@ -9,6 +9,47 @@
 
 from pypy.objspace.std import Space
 from pypy.translator.goal.targetpypystandalone import create_entry_point
+from pypy.tool.udir import udir
+
+
+class TestNanos:
+    def getnanos(self):
+        from pypy.translator.goal.nanos import os_module_for_testing
+        return os_module_for_testing
+
+    def test_exists(self):
+        os1 = self.getnanos()
+        assert os1.name == os.name
+        assert os1.sep == os.sep
+        assert os1.pathsep == os.pathsep
+        # check that the following attrs exist
+        os1.fdopen
+        os1.path.dirname
+
+    def test_dirname(self):
+        p1 = os.path
+        p2 = self.getnanos().path
+        path = str(udir.join('baz'))
+        assert p1.dirname(path) == p2.dirname(path)
+        assert p1.dirname(path + os.sep) == p2.dirname(path + os.sep)
+        assert p1.dirname(path + 2*os.sep) == p2.dirname(path + 2*os.sep)
+
+    def test_join(self):
+        p1 = os.path
+        p2 = self.getnanos().path
+        base = str(udir)
+        assert p1.join(base, '') == p2.join(base, '')
+        assert p1.join(base, 'baz') == p2.join(base, 'baz')
+        assert p1.join(base + os.sep, 'baz') == p2.join(base + os.sep, 'baz')
+        assert p1.join(base, 'baz' + os.sep) == p2.join(base, 'baz' + os.sep)
+        assert p1.join(base, base) == p2.join(base, base)
+
+    def test_abspath(self):
+        p2 = self.getnanos().path
+        base = str(udir)
+        assert p2.abspath(base) == base
+        assert p2.abspath('x') == os.path.join(os.getcwd(), 'x')
+
 
 def test_nanos():
     space = Space()

From lac at codespeak.net  Sat Feb  5 14:08:39 2011
From: lac at codespeak.net (lac at codespeak.net)
Date: Sat,  5 Feb 2011 14:08:39 +0100 (CET)
Subject: [pypy-svn] r80290 - pypy/extradoc/talk/ustour2011
Message-ID: <20110205130839.E9AEA2A2002@codespeak.net>

Author: lac
Date: Sat Feb  5 14:08:38 2011
New Revision: 80290

Modified:
   pypy/extradoc/talk/ustour2011/google-abstract2.txt
Log:
Add mention of Eurostars, and our blog.  Without being too blatant about it,
we want the message of 'pay us for full time work, get miraculous results'
to be there.


Modified: pypy/extradoc/talk/ustour2011/google-abstract2.txt
==============================================================================
--- pypy/extradoc/talk/ustour2011/google-abstract2.txt	(original)
+++ pypy/extradoc/talk/ustour2011/google-abstract2.txt	Sat Feb  5 14:08:38 2011
@@ -31,8 +31,12 @@
 The PyPy project has recently gathered a lot of attention for its
 progress in speeding up the Python language -- it is the fastest,
 most compatible and most stable 'alternative? Python interpreter.  No
-longer merely a research curiosity, PyPy is now suitablefor production
-use.  We will discuss what the PyPy project has achieved, with a particular
+longer merely a research curiosity, PyPy is now suitable for production
+use.  Since 2009, the PyPy project has received funding from the Eurostars 
+Eureka program, which targets small European firms which produce research.
+(See: http://www.eurostars-eureka.eu/)
+
+We will discuss what the PyPy project has achieved, with a particular
 focus on the past two years' work  in the area of dynamic (Just-in-Time) 
 interpreters:
 
@@ -48,4 +52,5 @@
 We want to reserve time for discussing potential future work like SWIG
 and/or Cython compatibility and other areas brought up by the audience.
 We will also briefly highlight who the active developers are and their
-funding background.
+funding background.  The PyPy project has a blog which you can read at
+http://morepypy.blogspot.com/


From commits-noreply at bitbucket.org  Sat Feb  5 14:09:09 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sat,  5 Feb 2011 14:09:09 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix for smalllong: make more
	systematically sure that	the created
Message-ID: <20110205130909.36DC62A2002@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41633:1bae24f77155
Date: 2011-02-05 14:08 +0100
http://bitbucket.org/pypy/pypy/changeset/1bae24f77155/

Log:	Fix for smalllong: make more systematically sure that the created
	long objects are W_SmallLongs if possible, by using systematically
	newlong() instead of directly W_LongObject().

diff --git a/pypy/objspace/std/longobject.py b/pypy/objspace/std/longobject.py
--- a/pypy/objspace/std/longobject.py
+++ b/pypy/objspace/std/longobject.py
@@ -32,8 +32,8 @@
     def toint(self):
         return self.num.toint()
 
-    def fromfloat(f):
-        return W_LongObject(rbigint.fromfloat(f))
+    def fromfloat(space, f):
+        return newlong(space, rbigint.fromfloat(f))
     fromfloat = staticmethod(fromfloat)
 
     def fromlong(l):

diff --git a/pypy/objspace/std/inttype.py b/pypy/objspace/std/inttype.py
--- a/pypy/objspace/std/inttype.py
+++ b/pypy/objspace/std/inttype.py
@@ -82,8 +82,8 @@
     except ParseStringError, e:
         raise OperationError(space.w_ValueError,
                              space.wrap(e.msg))
-    from pypy.objspace.std.longobject import W_LongObject
-    return W_LongObject(bigint)
+    from pypy.objspace.std.longobject import newlong
+    return newlong(space, bigint)
 
 def descr__new__(space, w_inttype, w_x=0, w_base=gateway.NoneNotWrapped):
     from pypy.objspace.std.intobject import W_IntObject

diff --git a/pypy/objspace/std/test/test_floatobject.py b/pypy/objspace/std/test/test_floatobject.py
--- a/pypy/objspace/std/test/test_floatobject.py
+++ b/pypy/objspace/std/test/test_floatobject.py
@@ -39,7 +39,7 @@
         from pypy.objspace.std.longobject import W_LongObject
         space = self.space
         saved = W_LongObject.__dict__['fromfloat']
-        W_LongObject.fromfloat = lambda x: disabled
+        W_LongObject.fromfloat = lambda space, x: disabled
         try:
             w_i = space.wrap(12)
             w_f = space.wrap(12.3)

diff --git a/pypy/objspace/std/marshal_impl.py b/pypy/objspace/std/marshal_impl.py
--- a/pypy/objspace/std/marshal_impl.py
+++ b/pypy/objspace/std/marshal_impl.py
@@ -28,7 +28,7 @@
 from pypy.objspace.std.stringobject  import W_StringObject
 from pypy.objspace.std.ropeobject    import W_RopeObject
 from pypy.objspace.std.typeobject    import W_TypeObject
-from pypy.objspace.std.longobject    import W_LongObject
+from pypy.objspace.std.longobject    import W_LongObject, newlong
 from pypy.objspace.std.noneobject    import W_NoneObject
 from pypy.objspace.std.unicodeobject import W_UnicodeObject
 
@@ -265,7 +265,7 @@
         if digits[-1] == 0:
             raise_exception(space, 'bad marshal data')
         result = rbigint(digits, sign)
-    w_long = W_LongObject(result)
+    w_long = newlong(space, result)
     return w_long
 register(TYPE_LONG, unmarshal_Long)
 

diff --git a/pypy/module/marshal/test/test_marshal.py b/pypy/module/marshal/test/test_marshal.py
--- a/pypy/module/marshal/test/test_marshal.py
+++ b/pypy/module/marshal/test/test_marshal.py
@@ -17,6 +17,7 @@
         f.seek(0)
         x = marshal.load(f)
         assert x == case and type(x) is type(case)
+        return x
 
     def test_None(self):
         case = None
@@ -179,3 +180,18 @@
         from pypy.conftest import gettestobjspace
         cls.space = gettestobjspace(**{"objspace.std.withrope": True})
         AppTestMarshal.setup_class.im_func(cls)
+
+class AppTestSmallLong(AppTestMarshal):
+    def setup_class(cls):
+        from pypy.conftest import gettestobjspace
+        cls.space = gettestobjspace(**{"objspace.std.withsmalllong": True})
+        AppTestMarshal.setup_class.im_func(cls)
+
+    def test_smalllong(self):
+        import __pypy__
+        x = -123456789012345L
+        assert 'SmallLong' in __pypy__.internal_repr(x)
+        y = self.marshal_check(x)
+        assert y == x
+        # must be unpickled as a small long
+        assert 'SmallLong' in __pypy__.internal_repr(y)

diff --git a/pypy/objspace/std/floatobject.py b/pypy/objspace/std/floatobject.py
--- a/pypy/objspace/std/floatobject.py
+++ b/pypy/objspace/std/floatobject.py
@@ -74,7 +74,7 @@
 
 def long__Float(space, w_floatobj):
     try:
-        return W_LongObject.fromfloat(w_floatobj.floatval)
+        return W_LongObject.fromfloat(space, w_floatobj.floatval)
     except OverflowError:
         if isnan(w_floatobj.floatval):
             raise OperationError(
@@ -288,7 +288,7 @@
         except OverflowError:
             # Convert to long and use its hash.
             try:
-                w_lval = W_LongObject.fromfloat(v)
+                w_lval = W_LongObject.fromfloat(space, v)
             except OverflowError:
                 # can't convert to long int -- arbitrary
                 if v < 0:
@@ -530,7 +530,7 @@
             break
         float_part *= 2.0
         exp -= 1
-    w_num = W_LongObject.fromfloat(float_part)
+    w_num = W_LongObject.fromfloat(space, float_part)
     w_den = space.newlong(1)
     w_exp = space.newlong(abs(exp))
     w_exp = space.lshift(w_den, w_exp)

diff --git a/pypy/objspace/std/objspace.py b/pypy/objspace/std/objspace.py
--- a/pypy/objspace/std/objspace.py
+++ b/pypy/objspace/std/objspace.py
@@ -22,7 +22,7 @@
 from pypy.objspace.std.floatobject import W_FloatObject
 from pypy.objspace.std.intobject import W_IntObject
 from pypy.objspace.std.listobject import W_ListObject
-from pypy.objspace.std.longobject import W_LongObject
+from pypy.objspace.std.longobject import W_LongObject, newlong
 from pypy.objspace.std.noneobject import W_NoneObject
 from pypy.objspace.std.objectobject import W_ObjectObject
 from pypy.objspace.std.ropeobject import W_RopeObject
@@ -292,7 +292,7 @@
         return W_LongObject.fromint(self, val)
 
     def newlong_from_rbigint(self, val):
-        return W_LongObject(val)
+        return newlong(self, val)
 
     def newtuple(self, list_w):
         assert isinstance(list_w, list)

From lac at codespeak.net  Sat Feb  5 14:10:35 2011
From: lac at codespeak.net (lac at codespeak.net)
Date: Sat,  5 Feb 2011 14:10:35 +0100 (CET)
Subject: [pypy-svn] r80291 - pypy/extradoc/talk/ustour2011
Message-ID: <20110205131035.BE1E72A2002@codespeak.net>

Author: lac
Date: Sat Feb  5 14:10:34 2011
New Revision: 80291

Added:
   pypy/extradoc/talk/ustour2011/noisebridge.txt   (contents, props changed)
Log:
Original doc by Armin


Added: pypy/extradoc/talk/ustour2011/noisebridge.txt
==============================================================================
--- (empty file)
+++ pypy/extradoc/talk/ustour2011/noisebridge.txt	Sat Feb  5 14:10:34 2011
@@ -0,0 +1,8 @@
+*PyPy mini-sprint*
+
+Programming Language topic: this is a "sprint" (intensive
+collaborative work) on PyPy, the Python interpreter written in Python.
+ Everybody is welcome.  The goal is to show off and work on whatever
+parts of PyPy people are interested in: the Just-in-Time Compiler,
+Garbage Collectors, Stackless Python, or the Python 2.7 language
+itself.
\ No newline at end of file


From lac at codespeak.net  Sat Feb  5 14:36:14 2011
From: lac at codespeak.net (lac at codespeak.net)
Date: Sat,  5 Feb 2011 14:36:14 +0100 (CET)
Subject: [pypy-svn] r80292 - pypy/extradoc/talk/ustour2011
Message-ID: <20110205133614.E88D22A2002@codespeak.net>

Author: lac
Date: Sat Feb  5 14:36:13 2011
New Revision: 80292

Modified:
   pypy/extradoc/talk/ustour2011/noisebridge.txt
Log:
Add the stuff Jim asked for.  Explain what we are going to do a bit for those
who have never heard of a sprint.


Modified: pypy/extradoc/talk/ustour2011/noisebridge.txt
==============================================================================
--- pypy/extradoc/talk/ustour2011/noisebridge.txt	(original)
+++ pypy/extradoc/talk/ustour2011/noisebridge.txt	Sat Feb  5 14:36:13 2011
@@ -1,8 +1,30 @@
 *PyPy mini-sprint*
 
-Programming Language topic: this is a "sprint" (intensive
-collaborative work) on PyPy, the Python interpreter written in Python.
- Everybody is welcome.  The goal is to show off and work on whatever
-parts of PyPy people are interested in: the Just-in-Time Compiler,
-Garbage Collectors, Stackless Python, or the Python 2.7 language
-itself.
\ No newline at end of file
+On Satuday the 5th or March, and Sunday the 6th, from 10 a.m. until
+whenever, there will be a PyPy mini-sprint hosted at Noisebridge.  For
+those unfamiliar with the term Sprint -- a Sprint is a time when
+develoeprs of a project get together to collaborate intensely.
+Sometimes the focus is to 'break the back' of some hard problem that
+the project faces, but in this case the focus will be on introducing
+people to the project in a hands-on, practical way.  PyPy is the
+Python interpreter written in Python which you can read about here:
+http://morepypy.blogspot.com/ Everybody is welcome.  The goal is to
+show off and work on whatever parts of PyPy people are interested in.
+Possible areas include, but are not limited to the Just-in-Time
+Compiler, Garbage Collectors, Stackless Python, or the Python 2.7
+language compatibility.  A sprint is the best way for would-be contributors
+to get involved with the project.  Come and bring your laptop.
+
+Note:  On Wednesday March the 2nd, at 16:15 Armin Rigo will be giving a talk
+about Pypy at the Stanford EE Computer Systems Colloquium 
+http://www.stanford.edu/class/ee380/winter-schedule-20102011.html , and
+on Thursay March the 3rd at 18:00 he will give another public talk at Yelp,
+706 Mission St 9th Floor, San Francisco, California.  While attending
+these talks is in no way mandatory, they might be of interest to those
+planning on attending the Mini Sprint.
+
+We expect to have between 6 and 12 people at this Sprint, but it is
+possible that we will have many more.  We need wireless access,
+electricity, chairs and tables which are set up so that we can
+pair-program, a whiteboard, and some flat surface where we can use a
+projector.  We have our own projector.


From lac at codespeak.net  Sat Feb  5 14:43:37 2011
From: lac at codespeak.net (lac at codespeak.net)
Date: Sat,  5 Feb 2011 14:43:37 +0100 (CET)
Subject: [pypy-svn] r80293 - pypy/extradoc/talk/ustour2011
Message-ID: <20110205134337.940EA2A2002@codespeak.net>

Author: lac
Date: Sat Feb  5 14:43:36 2011
New Revision: 80293

Modified:
   pypy/extradoc/talk/ustour2011/mozilla-abstract.txt
Log:
Add bios.


Modified: pypy/extradoc/talk/ustour2011/mozilla-abstract.txt
==============================================================================
--- pypy/extradoc/talk/ustour2011/mozilla-abstract.txt	(original)
+++ pypy/extradoc/talk/ustour2011/mozilla-abstract.txt	Sat Feb  5 14:43:36 2011
@@ -1,10 +1,37 @@
+XXX Need title here
+Speakers: Armin Rigo, Maciej Fijalkowski
 
-Authors: Armin Rigo, Maciej Fijalkowski
-# XXX get bios from laura
+Bios:
+Armin Rigo is a researcher at Heinrich-Heine-Universit?t in D?sseldorf,
+Germany.  His academic interests include Programming Languages and
+Implementation Techniques.
+
+He is the lead designer of the PyPy project and one of its
+original founders.  He is also the author of Psyco, a hand-coded
+Just-in-Time specializing compiler for Python, which can be used
+transparently with 32-bit x86 versions of CPython.  Since 2003 he has
+worked on all aspects of PyPy: its Python interpreter (written in
+Python), its translation toolchain (which produces C code), its
+garbage collectors, and its Tracing Just-in-Time compiler generator.
+Since the end of 2010, the Just in Time compiler generated by PyPy
+has outperformed Psyco, while being much more general and robust.
+
+Maciej Fijalkowski is a core developer of the PyPy project. He
+started contributing to it with the help of the Google Summer of Code
+Project in 2005, when he developed the Javascript backend. Since then,
+he has worked on the core of PyPy, including ctypes, the Just-In-Time
+compiler, garbage collectors and more.
+
+At present he is employed by SKA South Africa to build infrastructure
+for MeerKAT, a very large radio telescope under construction in the
+Karoo desert. The infrastructure is largely python-based and employs
+Twisted and PyPy among other technologies.
 
-The PyPy project has gathered recently a lot of attention for its
+Abstract:
+
+The PyPy project has recently gathered a lot of attention for its
 progress on speeding up the Python language -- it is the fastest,
-most compatible and stable alternative Python interpreter:
+most compatible and most stable 'alternative? Python interpreter:
 
 * most Python benchmarks run much faster than with CPython or Psyco
 * the real-world PyPy compiler toolchain itself (200 KLocs) runs twice as fast


From lac at codespeak.net  Sat Feb  5 15:16:17 2011
From: lac at codespeak.net (lac at codespeak.net)
Date: Sat,  5 Feb 2011 15:16:17 +0100 (CET)
Subject: [pypy-svn] r80294 - pypy/extradoc/talk/ustour2011
Message-ID: <20110205141617.0D0792A2002@codespeak.net>

Author: lac
Date: Sat Feb  5 15:16:11 2011
New Revision: 80294

Modified:
   pypy/extradoc/talk/ustour2011/google-abstract2.txt
   pypy/extradoc/talk/ustour2011/mozilla-abstract.txt
Log:
Add information about contacting us to let us know ahead of time what
it is that you are interested in having us speak about.  We'll see if
it works.


Modified: pypy/extradoc/talk/ustour2011/google-abstract2.txt
==============================================================================
--- pypy/extradoc/talk/ustour2011/google-abstract2.txt	(original)
+++ pypy/extradoc/talk/ustour2011/google-abstract2.txt	Sat Feb  5 15:16:11 2011
@@ -51,6 +51,12 @@
 
 We want to reserve time for discussing potential future work like SWIG
 and/or Cython compatibility and other areas brought up by the audience.
-We will also briefly highlight who the active developers are and their
-funding background.  The PyPy project has a blog which you can read at
-http://morepypy.blogspot.com/
+There are many interesting details that can be explored further;
+we will focus on the points the audience is most interested in.  If
+you already know what you would like us to speak about, could you
+please send a note to lac at openend.se and we will make sure that we
+cover this in our presentation.
+
+The PyPy project has a blog which make be of interest.  You can read
+it at http://morepypy.blogspot.com/ .
+

Modified: pypy/extradoc/talk/ustour2011/mozilla-abstract.txt
==============================================================================
--- pypy/extradoc/talk/ustour2011/mozilla-abstract.txt	(original)
+++ pypy/extradoc/talk/ustour2011/mozilla-abstract.txt	Sat Feb  5 15:16:11 2011
@@ -50,5 +50,10 @@
 thereby reducing complexity for writing state-of-the-art interpreters.
 
 There are many interesting details that can be explored further;
-we will focus on the points the audiance is most interested in.
+we will focus on the points the audience is most interested in.  If
+you already know what you would like us to speak about, could you
+please send a note to lac at openend.se and we will make sure that we
+cover this in our presentation.
 
+The PyPy project has a blog which make be of interest.  You can read
+it at http://morepypy.blogspot.com/ .


From arigo at codespeak.net  Sat Feb  5 16:47:39 2011
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sat,  5 Feb 2011 16:47:39 +0100 (CET)
Subject: [pypy-svn] r80295 - pypy/extradoc/talk/ustour2011
Message-ID: <20110205154739.02C0536C536@codespeak.net>

Author: arigo
Date: Sat Feb  5 16:47:38 2011
New Revision: 80295

Modified:
   pypy/extradoc/talk/ustour2011/google-abstract2.txt
   pypy/extradoc/talk/ustour2011/noisebridge.txt
Log:
One-character typos only.


Modified: pypy/extradoc/talk/ustour2011/google-abstract2.txt
==============================================================================
--- pypy/extradoc/talk/ustour2011/google-abstract2.txt	(original)
+++ pypy/extradoc/talk/ustour2011/google-abstract2.txt	Sat Feb  5 16:47:38 2011
@@ -57,6 +57,6 @@
 please send a note to lac at openend.se and we will make sure that we
 cover this in our presentation.
 
-The PyPy project has a blog which make be of interest.  You can read
+The PyPy project has a blog which may be of interest.  You can read
 it at http://morepypy.blogspot.com/ .
 

Modified: pypy/extradoc/talk/ustour2011/noisebridge.txt
==============================================================================
--- pypy/extradoc/talk/ustour2011/noisebridge.txt	(original)
+++ pypy/extradoc/talk/ustour2011/noisebridge.txt	Sat Feb  5 16:47:38 2011
@@ -3,12 +3,12 @@
 On Satuday the 5th or March, and Sunday the 6th, from 10 a.m. until
 whenever, there will be a PyPy mini-sprint hosted at Noisebridge.  For
 those unfamiliar with the term Sprint -- a Sprint is a time when
-develoeprs of a project get together to collaborate intensely.
+developers of a project get together to collaborate intensely.
 Sometimes the focus is to 'break the back' of some hard problem that
 the project faces, but in this case the focus will be on introducing
 people to the project in a hands-on, practical way.  PyPy is the
 Python interpreter written in Python which you can read about here:
-http://morepypy.blogspot.com/ Everybody is welcome.  The goal is to
+http://morepypy.blogspot.com/  Everybody is welcome.  The goal is to
 show off and work on whatever parts of PyPy people are interested in.
 Possible areas include, but are not limited to the Just-in-Time
 Compiler, Garbage Collectors, Stackless Python, or the Python 2.7
@@ -16,7 +16,7 @@
 to get involved with the project.  Come and bring your laptop.
 
 Note:  On Wednesday March the 2nd, at 16:15 Armin Rigo will be giving a talk
-about Pypy at the Stanford EE Computer Systems Colloquium 
+about PyPy at the Stanford EE Computer Systems Colloquium 
 http://www.stanford.edu/class/ee380/winter-schedule-20102011.html , and
 on Thursay March the 3rd at 18:00 he will give another public talk at Yelp,
 706 Mission St 9th Floor, San Francisco, California.  While attending


From arigo at codespeak.net  Sat Feb  5 17:26:46 2011
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sat,  5 Feb 2011 17:26:46 +0100 (CET)
Subject: [pypy-svn] r80296 - pypy/extradoc/talk/ustour2011
Message-ID: <20110205162646.E459D282B9C@codespeak.net>

Author: arigo
Date: Sat Feb  5 17:26:45 2011
New Revision: 80296

Added:
   pypy/extradoc/talk/ustour2011/yelp-abstract.txt
      - copied, changed from r80295, pypy/extradoc/talk/ustour2011/google-abstract2.txt
Log:
Copy the google abstract.


Copied: pypy/extradoc/talk/ustour2011/yelp-abstract.txt (from r80295, pypy/extradoc/talk/ustour2011/google-abstract2.txt)
==============================================================================
--- pypy/extradoc/talk/ustour2011/google-abstract2.txt	(original)
+++ pypy/extradoc/talk/ustour2011/yelp-abstract.txt	Sat Feb  5 17:26:45 2011
@@ -1,5 +1,5 @@
 
-Google Tech Talk: Speakers: Armin Rigo, Maciej Fijalkowski
+BayPIGgies Talk at Yelp: Speakers: Armin Rigo, Maciej Fijalkowski
 
 Bios:
 Armin Rigo is a researcher at Heinrich-Heine-Universit?t in D?sseldorf,
@@ -43,7 +43,7 @@
 * most Python benchmarks run much faster than with CPython or Psyco
 * the real-world PyPy compiler toolchain itself (200 KLocs) runs twice as fast
 * already supports 64bit and is in the process of supporting ARM
-* full compatibility with  CPython (more than Jython/IronPython)
+* full compatibility with CPython (more than Jython/IronPython)
 * full (and JIT-ed) ctypes support to call C libraries from Python
 * supports Stackless Python (in-progress)
 * new "cpyext" layer which integrates existing CPython C extensions
@@ -59,4 +59,3 @@
 
 The PyPy project has a blog which may be of interest.  You can read
 it at http://morepypy.blogspot.com/ .
-


From lac at codespeak.net  Sat Feb  5 18:18:52 2011
From: lac at codespeak.net (lac at codespeak.net)
Date: Sat,  5 Feb 2011 18:18:52 +0100 (CET)
Subject: [pypy-svn] r80297 - pypy/extradoc/talk/ustour2011
Message-ID: <20110205171852.169482A2006@codespeak.net>

Author: lac
Date: Sat Feb  5 18:18:51 2011
New Revision: 80297

Modified:
   pypy/extradoc/talk/ustour2011/yelp-abstract.txt
Log:
Maciej won't be giving the Yelp talk -- he won't have arrived yet.


Modified: pypy/extradoc/talk/ustour2011/yelp-abstract.txt
==============================================================================
--- pypy/extradoc/talk/ustour2011/yelp-abstract.txt	(original)
+++ pypy/extradoc/talk/ustour2011/yelp-abstract.txt	Sat Feb  5 18:18:51 2011
@@ -1,7 +1,7 @@
 
 BayPIGgies Talk at Yelp: Speakers: Armin Rigo, Maciej Fijalkowski
 
-Bios:
+Bio:
 Armin Rigo is a researcher at Heinrich-Heine-Universit?t in D?sseldorf,
 Germany.  His academic interests include Programming Languages and
 Implementation Techniques.
@@ -16,17 +16,6 @@
 Since the end of 2010, the Just in Time compiler generated by PyPy
 has outperformed Psyco, while being much more general and robust.
 
-Maciej Fijalkowski is a core developer of the PyPy project. He
-started contributing to it with the help of the Google Summer of Code
-Project in 2005, when he developed the Javascript backend. Since then,
-he has worked on the core of PyPy, including ctypes, the Just-In-Time
-compiler, garbage collectors and more.
-
-At present he is employed by SKA South Africa to build infrastructure
-for MeerKAT, a very large radio telescope under construction in the
-Karoo desert. The infrastructure is largely python-based and employs
-Twisted and PyPy among other technologies.
-
 Abstract:
 The PyPy project has recently gathered a lot of attention for its
 progress in speeding up the Python language -- it is the fastest,


From lac at codespeak.net  Sat Feb  5 18:19:57 2011
From: lac at codespeak.net (lac at codespeak.net)
Date: Sat,  5 Feb 2011 18:19:57 +0100 (CET)
Subject: [pypy-svn] r80298 - pypy/extradoc/talk/ustour2011
Message-ID: <20110205171957.7D959282B9C@codespeak.net>

Author: lac
Date: Sat Feb  5 18:19:56 2011
New Revision: 80298

Modified:
   pypy/extradoc/talk/ustour2011/yelp-abstract.txt
Log:
oops, and this line needs changing, too.


Modified: pypy/extradoc/talk/ustour2011/yelp-abstract.txt
==============================================================================
--- pypy/extradoc/talk/ustour2011/yelp-abstract.txt	(original)
+++ pypy/extradoc/talk/ustour2011/yelp-abstract.txt	Sat Feb  5 18:19:56 2011
@@ -1,5 +1,5 @@
 
-BayPIGgies Talk at Yelp: Speakers: Armin Rigo, Maciej Fijalkowski
+BayPIGgies Talk at Yelp: Speaker: Armin Rigo
 
 Bio:
 Armin Rigo is a researcher at Heinrich-Heine-Universit?t in D?sseldorf,


From lac at codespeak.net  Sat Feb  5 18:30:41 2011
From: lac at codespeak.net (lac at codespeak.net)
Date: Sat,  5 Feb 2011 18:30:41 +0100 (CET)
Subject: [pypy-svn] r80299 - pypy/extradoc/talk/ustour2011
Message-ID: <20110205173041.DD1242A2006@codespeak.net>

Author: lac
Date: Sat Feb  5 18:30:40 2011
New Revision: 80299

Modified:
   pypy/extradoc/talk/ustour2011/mozilla-abstract.txt
Log:
Give the talk a title.


Modified: pypy/extradoc/talk/ustour2011/mozilla-abstract.txt
==============================================================================
--- pypy/extradoc/talk/ustour2011/mozilla-abstract.txt	(original)
+++ pypy/extradoc/talk/ustour2011/mozilla-abstract.txt	Sat Feb  5 18:30:40 2011
@@ -1,4 +1,4 @@
-XXX Need title here
+PyPy Mozilla Talk:  Monday March 7th, 2011 at 2:30
 Speakers: Armin Rigo, Maciej Fijalkowski
 
 Bios:


From commits-noreply at bitbucket.org  Sat Feb  5 19:50:27 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sat,  5 Feb 2011 19:50:27 +0100 (CET)
Subject: [pypy-svn] pypy default: More tweaks to try to make the translated
	pypy-c more	happy.
Message-ID: <20110205185027.2B13636C394@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41634:98c9d9cfcfbe
Date: 2011-02-05 19:28 +0100
http://bitbucket.org/pypy/pypy/changeset/98c9d9cfcfbe/

Log:	More tweaks to try to make the translated pypy-c more happy.

diff --git a/pypy/translator/goal/test2/test_nanos.py b/pypy/translator/goal/test2/test_nanos.py
--- a/pypy/translator/goal/test2/test_nanos.py
+++ b/pypy/translator/goal/test2/test_nanos.py
@@ -22,9 +22,6 @@
         assert os1.name == os.name
         assert os1.sep == os.sep
         assert os1.pathsep == os.pathsep
-        # check that the following attrs exist
-        os1.fdopen
-        os1.path.dirname
 
     def test_dirname(self):
         p1 = os.path
@@ -33,6 +30,7 @@
         assert p1.dirname(path) == p2.dirname(path)
         assert p1.dirname(path + os.sep) == p2.dirname(path + os.sep)
         assert p1.dirname(path + 2*os.sep) == p2.dirname(path + 2*os.sep)
+        assert p1.dirname(p1.dirname(path)) == p2.dirname(p2.dirname(path))
 
     def test_join(self):
         p1 = os.path
@@ -50,6 +48,14 @@
         assert p2.abspath(base) == base
         assert p2.abspath('x') == os.path.join(os.getcwd(), 'x')
 
+    def test_isfile(self):
+        p2 = self.getnanos().path
+        udir.join('test_isfile').write('\n')
+        base = str(udir)
+        assert p2.isfile(p2.join(base, 'test_isfile'))
+        assert not p2.isfile(p2.join(base, 'test_isfile.DOES.NOT.EXIST'))
+        assert not p2.isfile(base)
+
 
 def test_nanos():
     space = Space()

diff --git a/pypy/translator/goal/nanos.py b/pypy/translator/goal/nanos.py
--- a/pypy/translator/goal/nanos.py
+++ b/pypy/translator/goal/nanos.py
@@ -31,8 +31,6 @@
 if os.name == 'posix':
     # code copied from posixpath.py
     app_os_path = applevel("""
-        from posix import getcwd, stat, lstat, error
-
         def dirname(p):
             i = p.rfind('/') + 1
             head = p[:i]
@@ -51,21 +49,24 @@
 
         def abspath(path):
             if not path.startswith('/'):
-                cwd = getcwd()
+                import posix
+                cwd = posix.getcwd()
                 path = join(cwd, path)
             return path       # this version does not call normpath()!
 
         def isfile(path):
+            import posix
             try:
-                st = stat(path)
-            except error:
+                st = posix.stat(path)
+            except posix.error:
                 return False
             return (st.st_mode & 0170000) == 0100000      # S_ISREG
 
         def islink(path):
+            import posix
             try:
-                st = lstat(path)
-            except error:
+                st = posix.lstat(path)
+            except posix.error:
                 return False
             return (st.st_mode & 0170000) == 0120000      # S_ISLNK
 
@@ -75,14 +76,15 @@
         sep = '/'
         pathsep = ':'
         name = 'posix'
-        from posix import fdopen, readlink
+
+        def readlink(fn):
+            import posix
+            return posix.readlink(fn)
     """, filename=__file__)
 
 elif os.name == 'nt':
     # code copied from ntpath.py
     app_os_path = applevel("""
-        from nt import _getfullpathname, getcwd, stat, lstat, error
-
         def splitdrive(p):
             if p[1:2] == ':':
                 return p[0:2], p[2:]
@@ -157,19 +159,21 @@
             return path
 
         def abspath(path):
+            import nt
             if path: # Empty path must return current working directory.
                 try:
-                    path = _getfullpathname(path)
+                    path = nt._getfullpathname(path)
                 except WindowsError:
                     pass # Bad path - return unchanged.
             else:
-                path = getcwd()
+                path = nt.getcwd()
             return path       # this version does not call normpath()!
 
         def isfile(path):
+            import nt
             try:
-                st = stat(path)
-            except error:
+                st = nt.stat(path)
+            except nt.error:
                 return False
             return (st.st_mode & 0170000) == 0100000      # S_ISREG
 
@@ -182,7 +186,6 @@
         sep = '\\'
         pathsep = ';'
         name = 'nt'
-        from posix import fdopen
     """, filename=__file__)
 
 else:

diff --git a/pypy/translator/goal/app_main.py b/pypy/translator/goal/app_main.py
--- a/pypy/translator/goal/app_main.py
+++ b/pypy/translator/goal/app_main.py
@@ -164,13 +164,20 @@
     print >> sys.stderr, 'usage: %s [options]' % (sys.executable,)
     print >> sys.stderr, 'Try `%s -h` for more information.' % (sys.executable,)
 
+def fdopen(fd, mode, bufsize=-1):
+    try:
+        fdopen = file.fdopen
+    except AttributeError:     # only on top of CPython, running tests
+        from os import fdopen
+    return fdopen(fd, mode, bufsize)
+
 def set_unbuffered_io():
-    sys.stdin  = sys.__stdin__  = os.fdopen(0, 'rb', 0)
-    sys.stdout = sys.__stdout__ = os.fdopen(1, 'wb', 0)
-    sys.stderr = sys.__stderr__ = os.fdopen(2, 'wb', 0)
+    sys.stdin  = sys.__stdin__  = fdopen(0, 'rb', 0)
+    sys.stdout = sys.__stdout__ = fdopen(1, 'wb', 0)
+    sys.stderr = sys.__stderr__ = fdopen(2, 'wb', 0)
 
 def set_fully_buffered_io():
-    sys.stdout = sys.__stdout__ = os.fdopen(1, 'w')
+    sys.stdout = sys.__stdout__ = fdopen(1, 'w')
 
 # ____________________________________________________________
 # Main entry point

From commits-noreply at bitbucket.org  Sat Feb  5 20:34:10 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sat,  5 Feb 2011 20:34:10 +0100 (CET)
Subject: [pypy-svn] pypy default: Skip these tests about c_longdouble,
	because it's not supported
Message-ID: <20110205193410.46A4736C394@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41635:3edf2a432212
Date: 2011-02-05 20:33 +0100
http://bitbucket.org/pypy/pypy/changeset/3edf2a432212/

Log:	Skip these tests about c_longdouble, because it's not supported by
	the struct or array modules.

diff --git a/pypy/module/test_lib_pypy/ctypes_tests/test_numbers.py b/pypy/module/test_lib_pypy/ctypes_tests/test_numbers.py
--- a/pypy/module/test_lib_pypy/ctypes_tests/test_numbers.py
+++ b/pypy/module/test_lib_pypy/ctypes_tests/test_numbers.py
@@ -112,6 +112,8 @@
 
     def test_sizes(self):
         for t in signed_types + unsigned_types + float_types:
+            if t is c_longdouble:   # no support for 'g' in the struct module
+                continue
             size = struct.calcsize(t._type_)
             # sizeof of the type...
             assert sizeof(t) == size
@@ -120,6 +122,8 @@
 
     def test_alignments(self):
         for t in signed_types + unsigned_types + float_types:
+            if t is c_longdouble:   # no support for 'g' in the struct module
+                continue
             code = t._type_ # the typecode
             align = struct.calcsize("c%c" % code) - struct.calcsize(code)
 
@@ -152,6 +156,8 @@
     def test_float_from_address(self):
         from array import array
         for t in float_types:
+            if t is c_longdouble:   # no support for 'g' in the array module
+                continue
             a = array(t._type_, [3.14])
             v = t.from_address(a.buffer_info()[0])
             assert v.value == a[0]

From commits-noreply at bitbucket.org  Sat Feb  5 20:55:54 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sat,  5 Feb 2011 20:55:54 +0100 (CET)
Subject: [pypy-svn] pypy default: Oups, broke translation.
Message-ID: <20110205195554.EC85036C394@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41636:9779792196c6
Date: 2011-02-05 20:55 +0100
http://bitbucket.org/pypy/pypy/changeset/9779792196c6/

Log:	Oups, broke translation.

diff --git a/pypy/objspace/std/floatobject.py b/pypy/objspace/std/floatobject.py
--- a/pypy/objspace/std/floatobject.py
+++ b/pypy/objspace/std/floatobject.py
@@ -295,7 +295,7 @@
                     return -271828
                 else:
                     return 314159
-            return space.int_w(hash__Long(space, w_lval))
+            return space.int_w(space.hash(w_lval))
 
     # The fractional part is non-zero, so we don't have to worry about
     # making this match the hash of some other type.

From commits-noreply at bitbucket.org  Sun Feb  6 00:14:09 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Sun,  6 Feb 2011 00:14:09 +0100 (CET)
Subject: [pypy-svn] pypy default: Make a modifiable copy of test_memoryio.
Message-ID: <20110205231409.E690B2A2006@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41637:5eaac0a2b9c3
Date: 2011-02-05 18:12 -0500
http://bitbucket.org/pypy/pypy/changeset/5eaac0a2b9c3/

Log:	Make a modifiable copy of test_memoryio.

diff --git a/lib-python/2.7.0/test/test_memoryio.py b/lib-python/modified-2.7.0/test/test_memoryio.py
copy from lib-python/2.7.0/test/test_memoryio.py
copy to lib-python/modified-2.7.0/test/test_memoryio.py

From commits-noreply at bitbucket.org  Sun Feb  6 00:14:10 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Sun,  6 Feb 2011 00:14:10 +0100 (CET)
Subject: [pypy-svn] pypy default: longs are acceptable here.
Message-ID: <20110205231410.8783F2A2006@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41638:542a9a6f3372
Date: 2011-02-05 18:13 -0500
http://bitbucket.org/pypy/pypy/changeset/542a9a6f3372/

Log:	longs are acceptable here.

diff --git a/lib-python/modified-2.7.0/test/test_memoryio.py b/lib-python/modified-2.7.0/test/test_memoryio.py
--- a/lib-python/modified-2.7.0/test/test_memoryio.py
+++ b/lib-python/modified-2.7.0/test/test_memoryio.py
@@ -612,7 +612,7 @@
         state = memio.__getstate__()
         self.assertEqual(len(state), 3)
         bytearray(state[0]) # Check if state[0] supports the buffer interface.
-        self.assertIsInstance(state[1], int)
+        self.assertIsInstance(state[1], (int, long))
         self.assert_(isinstance(state[2], dict) or state[2] is None)
         memio.close()
         self.assertRaises(ValueError, memio.__getstate__)

From commits-noreply at bitbucket.org  Sun Feb  6 10:28:47 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Sun,  6 Feb 2011 10:28:47 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: Resume with correct live_boxes
	after retraced	loop
Message-ID: <20110206092847.39FE3282B9C@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r41639:ec93501c0a52
Date: 2011-02-06 10:28 +0100
http://bitbucket.org/pypy/pypy/changeset/ec93501c0a52/

Log:	Resume with correct live_boxes after retraced loop

diff --git a/pypy/jit/metainterp/test/test_send.py b/pypy/jit/metainterp/test/test_send.py
--- a/pypy/jit/metainterp/test/test_send.py
+++ b/pypy/jit/metainterp/test/test_send.py
@@ -334,10 +334,6 @@
         class State:
             count = 0
             def externfn(self, n):
-                print
-                print
-                print n, 198 - self.count
-                print
                 assert n == 198 - self.count
                 self.count += 1
                 if n % 5:

diff --git a/pypy/jit/metainterp/pyjitpl.py b/pypy/jit/metainterp/pyjitpl.py
--- a/pypy/jit/metainterp/pyjitpl.py
+++ b/pypy/jit/metainterp/pyjitpl.py
@@ -1913,7 +1913,8 @@
         self.history.inputargs = original_boxes[num_green_args:]
         greenkey = original_boxes[:num_green_args]
         self.history.record(rop.JUMP, live_arg_boxes[num_green_args:], None)
-        loop_token = compile.compile_new_loop(self, [], greenkey, start, start_resumedescr)
+        loop_token = compile.compile_new_loop(self, [], greenkey, start,
+                                              start_resumedescr)
         self.history.operations.pop()     # remove the JUMP
         if loop_token is None:
             return
@@ -1923,9 +1924,8 @@
 
         self.history.inputargs = original_inputargs
         self.history.operations = self.history.operations[:start]
-        live_arg_boxes = bridge_arg_boxes
         
-        self.history.record(rop.JUMP, live_arg_boxes[num_green_args:], None)
+        self.history.record(rop.JUMP, bridge_arg_boxes[num_green_args:], None)
         try:
             target_loop_token = compile.compile_new_bridge(self,
                                                            [loop_token],

From commits-noreply at bitbucket.org  Sun Feb  6 12:58:27 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Sun,  6 Feb 2011 12:58:27 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: rename
Message-ID: <20110206115827.15F7D282BD4@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r41640:1f02b3de5e1b
Date: 2011-02-06 10:46 +0100
http://bitbucket.org/pypy/pypy/changeset/1f02b3de5e1b/

Log:	rename

diff --git a/pypy/jit/metainterp/optimizeopt/unroll.py b/pypy/jit/metainterp/optimizeopt/unroll.py
--- a/pypy/jit/metainterp/optimizeopt/unroll.py
+++ b/pypy/jit/metainterp/optimizeopt/unroll.py
@@ -130,10 +130,10 @@
     def __init__(self, state):
         self.state = state
 
-    def more_general_than(self, other):
+    def generalization_of(self, other):
         assert len(self.state) == len(other.state)
         for i in range(len(self.state)):
-            if not self.state[i].more_general_than(other.state[i]):
+            if not self.state[i].generalization_of(other.state[i]):
                 return False
         return True
 
@@ -185,7 +185,7 @@
         else:
             self.constbox = None
 
-    def more_general_than(self, other):
+    def generalization_of(self, other):
         # XXX This will always retrace instead of forcing anything which
         # might be what we want sometimes?
         if not isinstance(other, NotVirtualInfo):
@@ -605,7 +605,7 @@
                 modifier = VirtualStateAdder(self.optimizer)
                 virtual_state = modifier.get_virtual_state(args)
                 for sh in short:                                        
-                    if sh.virtual_state.more_general_than(virtual_state):
+                    if sh.virtual_state.generalization_of(virtual_state):
                         # FIXME: Do we still need the dry run
                         if self.inline(sh.operations, sh.inputargs,
                                        op.getarglist(), dryrun=True):

diff --git a/pypy/jit/metainterp/resume.py b/pypy/jit/metainterp/resume.py
--- a/pypy/jit/metainterp/resume.py
+++ b/pypy/jit/metainterp/resume.py
@@ -452,7 +452,7 @@
                         str(self.fielddescrs[i]),
                         str(untag(self.fieldnums[i])))
 
-    def more_general_than(self, other):
+    def generalization_of(self, other):
         if not isinstance(other, AbstractVirtualStructInfo):
             return False
         assert len(self.fielddescrs) == len(self.fieldstate)
@@ -463,7 +463,7 @@
         for i in range(len(self.fielddescrs)):
             if other.fielddescrs[i] is not self.fielddescrs[i]:
                 return False
-            if not self.fieldstate[i].more_general_than(other.fieldstate[i]):
+            if not self.fieldstate[i].generalization_of(other.fieldstate[i]):
                 return False
         return True
             

From commits-noreply at bitbucket.org  Sun Feb  6 12:58:27 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Sun,  6 Feb 2011 12:58:27 +0100 (CET)
Subject: [pypy-svn] pypy default: unsafe case
Message-ID: <20110206115827.A720B282BD4@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: 
Changeset: r41641:1c988271728b
Date: 2011-02-06 10:57 +0100
http://bitbucket.org/pypy/pypy/changeset/1c988271728b/

Log:	unsafe case

diff --git a/pypy/jit/metainterp/test/test_send.py b/pypy/jit/metainterp/test/test_send.py
--- a/pypy/jit/metainterp/test/test_send.py
+++ b/pypy/jit/metainterp/test/test_send.py
@@ -361,6 +361,46 @@
         else:
             self.check_tree_loop_count(2)
 
+    def test_indirect_call_unknown_object_3(self):
+        myjitdriver = JitDriver(greens = [], reds = ['x', 'y', 'z', 'state'])
+        def getvalue2():
+            return 2
+        def getvalue25():
+            return 25
+        def getvalue1001():
+            return -1001
+
+        class State:
+            count = 0
+            def externfn(self, n):
+                assert n == 198 - self.count
+                self.count += 1
+                if n % 5:
+                    return getvalue2
+                elif n % 7:
+                    return getvalue25
+                else:
+                    return getvalue1001
+        def f(y):
+            state = State()
+            x = z = 0
+            while y > 0:
+                myjitdriver.can_enter_jit(x=x, y=y, z=z, state=state)
+                myjitdriver.jit_merge_point(x=x, y=y, z=z, state=state)
+                x += z
+                z = state.externfn(y)()
+                y -= 1
+            return x
+        res = self.meta_interp(f, [198],
+                               policy=StopAtXPolicy(State.externfn.im_func))
+        assert res == f(198)
+        # we get two TreeLoops: an initial one, and one entering from
+        # the interpreter
+        if self.optimizer != OPTIMIZER_FULL:
+            self.check_tree_loop_count(1)
+        else:
+            self.check_tree_loop_count(2)
+
     def test_two_behaviors(self):
         py.test.skip("XXX fix me!!!!!!! problem in optimize.py")
         myjitdriver = JitDriver(greens = [], reds = ['x', 'y'])

From commits-noreply at bitbucket.org  Sun Feb  6 12:58:28 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Sun,  6 Feb 2011 12:58:28 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: hg merge default
Message-ID: <20110206115828.438A8282BD4@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r41642:05253e99cbb3
Date: 2011-02-06 10:57 +0100
http://bitbucket.org/pypy/pypy/changeset/05253e99cbb3/

Log:	hg merge default

diff --git a/pypy/jit/metainterp/test/test_send.py b/pypy/jit/metainterp/test/test_send.py
--- a/pypy/jit/metainterp/test/test_send.py
+++ b/pypy/jit/metainterp/test/test_send.py
@@ -361,6 +361,46 @@
         else:
             self.check_tree_loop_count(2)
 
+    def test_indirect_call_unknown_object_3(self):
+        myjitdriver = JitDriver(greens = [], reds = ['x', 'y', 'z', 'state'])
+        def getvalue2():
+            return 2
+        def getvalue25():
+            return 25
+        def getvalue1001():
+            return -1001
+
+        class State:
+            count = 0
+            def externfn(self, n):
+                assert n == 198 - self.count
+                self.count += 1
+                if n % 5:
+                    return getvalue2
+                elif n % 7:
+                    return getvalue25
+                else:
+                    return getvalue1001
+        def f(y):
+            state = State()
+            x = z = 0
+            while y > 0:
+                myjitdriver.can_enter_jit(x=x, y=y, z=z, state=state)
+                myjitdriver.jit_merge_point(x=x, y=y, z=z, state=state)
+                x += z
+                z = state.externfn(y)()
+                y -= 1
+            return x
+        res = self.meta_interp(f, [198],
+                               policy=StopAtXPolicy(State.externfn.im_func))
+        assert res == f(198)
+        # we get two TreeLoops: an initial one, and one entering from
+        # the interpreter
+        if self.optimizer != OPTIMIZER_FULL:
+            self.check_tree_loop_count(1)
+        else:
+            self.check_tree_loop_count(2)
+
     def test_two_behaviors(self):
         py.test.skip("XXX fix me!!!!!!! problem in optimize.py")
         myjitdriver = JitDriver(greens = [], reds = ['x', 'y'])

From commits-noreply at bitbucket.org  Sun Feb  6 12:58:29 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Sun,  6 Feb 2011 12:58:29 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: make sure we have the same class
Message-ID: <20110206115829.DBBA6282BD4@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r41643:fdc5adf84129
Date: 2011-02-06 11:16 +0100
http://bitbucket.org/pypy/pypy/changeset/fdc5adf84129/

Log:	make sure we have the same class

diff --git a/pypy/jit/metainterp/optimizeopt/unroll.py b/pypy/jit/metainterp/optimizeopt/unroll.py
--- a/pypy/jit/metainterp/optimizeopt/unroll.py
+++ b/pypy/jit/metainterp/optimizeopt/unroll.py
@@ -607,16 +607,16 @@
                 for sh in short:                                        
                     if sh.virtual_state.generalization_of(virtual_state):
                         # FIXME: Do we still need the dry run
-                        if self.inline(sh.operations, sh.inputargs,
-                                       op.getarglist(), dryrun=True):
-                            try:
-                                self.inline(sh.operations, sh.inputargs,
-                                            op.getarglist())
-                            except InvalidLoop:
-                                debug_print("Inlining failed unexpectedly",
-                                            "jumping to preamble instead")
-                                self.emit_operation(op)
-                            return
+                        #if self.inline(sh.operations, sh.inputargs,
+                        #               op.getarglist(), dryrun=True):
+                        try:
+                            self.inline(sh.operations, sh.inputargs,
+                                        op.getarglist())
+                        except InvalidLoop:
+                            debug_print("Inlining failed unexpectedly",
+                                        "jumping to preamble instead")
+                            self.emit_operation(op)
+                        return
                 if not self.retraced:    
                     raise RetraceLoop
         self.emit_operation(op)

diff --git a/pypy/jit/metainterp/resume.py b/pypy/jit/metainterp/resume.py
--- a/pypy/jit/metainterp/resume.py
+++ b/pypy/jit/metainterp/resume.py
@@ -455,6 +455,9 @@
     def generalization_of(self, other):
         if not isinstance(other, AbstractVirtualStructInfo):
             return False
+        if not self.known_class.same_constant(other.known_class):
+            return False
+        
         assert len(self.fielddescrs) == len(self.fieldstate)
         assert len(other.fielddescrs) == len(other.fieldstate)
         if len(self.fielddescrs) != len(other.fielddescrs):

From commits-noreply at bitbucket.org  Sun Feb  6 12:58:30 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Sun,  6 Feb 2011 12:58:30 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: make sure we have the same class
	or type
Message-ID: <20110206115830.A859B282BD4@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r41644:94c4b6e6d04c
Date: 2011-02-06 11:30 +0100
http://bitbucket.org/pypy/pypy/changeset/94c4b6e6d04c/

Log:	make sure we have the same class or type

diff --git a/pypy/jit/metainterp/resume.py b/pypy/jit/metainterp/resume.py
--- a/pypy/jit/metainterp/resume.py
+++ b/pypy/jit/metainterp/resume.py
@@ -453,11 +453,6 @@
                         str(untag(self.fieldnums[i])))
 
     def generalization_of(self, other):
-        if not isinstance(other, AbstractVirtualStructInfo):
-            return False
-        if not self.known_class.same_constant(other.known_class):
-            return False
-        
         assert len(self.fielddescrs) == len(self.fieldstate)
         assert len(other.fielddescrs) == len(other.fieldstate)
         if len(self.fielddescrs) != len(other.fielddescrs):
@@ -468,7 +463,11 @@
                 return False
             if not self.fieldstate[i].generalization_of(other.fieldstate[i]):
                 return False
-        return True
+
+        return self._generalization_of(other)
+
+    def _generalization_of(self, other):
+        raise NotImplementedError
             
         
 
@@ -487,6 +486,14 @@
         debug_print("\tvirtualinfo", self.known_class.repr_rpython())
         AbstractVirtualStructInfo.debug_prints(self)
 
+    def _generalization_of(self, other):        
+        if not isinstance(other, VirtualInfo):
+            return False
+        if not self.known_class.same_constant(other.known_class):
+            return False
+        return True
+        
+
 class VStructInfo(AbstractVirtualStructInfo):
     def __init__(self, typedescr, fielddescrs):
         AbstractVirtualStructInfo.__init__(self, fielddescrs)
@@ -502,6 +509,14 @@
         debug_print("\tvstructinfo", self.typedescr.repr_rpython())
         AbstractVirtualStructInfo.debug_prints(self)
 
+    def _generalization_of(self, other):        
+        if not isinstance(other, VStructInfo):
+            return False
+        if self.typedescr is not other.typedescr:
+            return False
+        return True
+        
+
 class VArrayInfo(AbstractVirtualInfo):
     def __init__(self, arraydescr):
         self.arraydescr = arraydescr

From commits-noreply at bitbucket.org  Sun Feb  6 12:58:33 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Sun,  6 Feb 2011 12:58:33 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: Generate guards instead of
	retracing when it	is likely it will pass and thereby make the
	excisting compiled	loop reusable
Message-ID: <20110206115833.7EF05282BEA@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r41645:22bd566e537d
Date: 2011-02-06 12:56 +0100
http://bitbucket.org/pypy/pypy/changeset/22bd566e537d/

Log:	Generate guards instead of retracing when it is likely it will pass
	and thereby make the excisting compiled loop reusable

diff --git a/pypy/jit/metainterp/optimizeopt/unroll.py b/pypy/jit/metainterp/optimizeopt/unroll.py
--- a/pypy/jit/metainterp/optimizeopt/unroll.py
+++ b/pypy/jit/metainterp/optimizeopt/unroll.py
@@ -105,12 +105,14 @@
             newop.result = newop.result.clonebox()
             self.argmap[old_result] = newop.result
 
-        descr = newop.getdescr()
+        self.inline_descr_inplace(newop.getdescr())
+
+        return newop
+
+    def inline_descr_inplace(self, descr):
         if isinstance(descr, ResumeGuardDescr):
             descr.rd_snapshot = self.inline_snapshot(descr.rd_snapshot)
-
-        return newop
-    
+            
     def inline_arg(self, arg):
         if arg is None:
             return None
@@ -137,6 +139,12 @@
                 return False
         return True
 
+    def generate_guards(self, other, args, cpu, extra_guards):        
+        assert len(self.state) == len(other.state) == len(args)
+        for i in range(len(self.state)):
+            self.state[i].generate_guards(other.state[i], args[i],
+                                          cpu, extra_guards)
+
 class VirtualStateAdder(resume.ResumeDataVirtualAdder):
     def __init__(self, optimizer):
         self.fieldboxes = {}
@@ -200,6 +208,29 @@
                 return False
         return self.intbound.contains_bound(other.intbound)
 
+    def _generate_guards(self, other, box, cpu, extra_guards):
+        if not isinstance(other, NotVirtualInfo):
+            raise InvalidLoop
+        if self.level == LEVEL_KNOWNCLASS and \
+           box.value and \
+           self.known_class.same_constant(cpu.ts.cls_of_box(box)):
+            # Note: This is only a hint on what the class of box was
+            # during the trace. There are actually no guarentees that this
+            # box realy comes from a trace. The hint is used here to choose
+            # between either eimtting a guard_class and jumping to an
+            # excisting compiled loop or retracing the loop. Both
+            # alternatives will always generate correct behaviour, but
+            # performace will differ.
+            op = ResOperation(rop.GUARD_CLASS, [box, self.known_class], None)
+            extra_guards.append(op)
+            return
+        # Remaining cases are probably not interesting
+        raise InvalidLoop
+        if self.level == LEVEL_CONSTANT:
+            import pdb; pdb.set_trace()
+            raise NotImplementedError
+        
+
 class UnrollOptimizer(Optimization):
     """Unroll the loop into two iterations. The first one will
     become the preamble or entry bridge (don't think there is a
@@ -281,6 +312,9 @@
                 short_loop.inputargs = newargs
                 ops = [inliner.inline_op(op) for op in short_loop.operations]
                 short_loop.operations = ops
+                descr = start_resumedescr.clone_if_mutable()
+                inliner.inline_descr_inplace(descr)
+                short_loop.start_resumedescr = descr
 
                 assert isinstance(loop.preamble.token, LoopToken)
                 if loop.preamble.token.short_preamble:
@@ -585,6 +619,7 @@
 class OptInlineShortPreamble(Optimization):
     def __init__(self, retraced):
         self.retraced = retraced
+        self.inliner = None
         
     
     def reconstruct_for_next_iteration(self, optimizer, valuemap):
@@ -604,14 +639,32 @@
                 args = op.getarglist()
                 modifier = VirtualStateAdder(self.optimizer)
                 virtual_state = modifier.get_virtual_state(args)
-                for sh in short:                                        
+                for sh in short:
+                    ok = False
+                    extra_guards = []
                     if sh.virtual_state.generalization_of(virtual_state):
+                        ok = True
+                    else:
+                        try:
+                            cpu = self.optimizer.cpu
+                            sh.virtual_state.generate_guards(virtual_state,
+                                                             args, cpu,
+                                                             extra_guards)
+                            ok = True
+                        except InvalidLoop:
+                            pass
+                    if ok:
                         # FIXME: Do we still need the dry run
                         #if self.inline(sh.operations, sh.inputargs,
                         #               op.getarglist(), dryrun=True):
                         try:
                             self.inline(sh.operations, sh.inputargs,
                                         op.getarglist())
+                            for guard in extra_guards:
+                                descr = sh.start_resumedescr.clone_if_mutable()
+                                self.inliner.inline_descr_inplace(descr)
+                                guard.setdescr(descr)
+                                self.emit_operation(guard)
                         except InvalidLoop:
                             debug_print("Inlining failed unexpectedly",
                                         "jumping to preamble instead")
@@ -624,7 +677,7 @@
         
         
     def inline(self, loop_operations, loop_args, jump_args, dryrun=False):
-        inliner = Inliner(loop_args, jump_args)
+        self.inliner = inliner = Inliner(loop_args, jump_args)
 
         for op in loop_operations:
             newop = inliner.inline_op(op)
@@ -637,7 +690,7 @@
         
         return True
 
-    def inline_arg(self, arg):
-        if isinstance(arg, Const):
-            return arg
-        return self.argmap[arg]
+    #def inline_arg(self, arg):
+    #    if isinstance(arg, Const):
+    #        return arg
+    #    return self.argmap[arg]

diff --git a/pypy/jit/metainterp/resume.py b/pypy/jit/metainterp/resume.py
--- a/pypy/jit/metainterp/resume.py
+++ b/pypy/jit/metainterp/resume.py
@@ -10,6 +10,7 @@
 from pypy.rlib.objectmodel import we_are_translated, specialize
 from pypy.rlib.debug import have_debug_prints, ll_assert
 from pypy.rlib.debug import debug_start, debug_stop, debug_print
+from pypy.jit.metainterp.optimizeutil import InvalidLoop
 
 # Logic to encode the chain of frames and the state of the boxes at a
 # guard operation, and to decode it again.  This is a bit advanced,
@@ -427,12 +428,24 @@
     #    raise NotImplementedError
     def equals(self, fieldnums):
         return tagged_list_eq(self.fieldnums, fieldnums)
+
     def set_content(self, fieldnums):
         self.fieldnums = fieldnums
 
     def debug_prints(self):
         raise NotImplementedError
 
+    def generalization_of(self, other):
+        raise NotImplementedError
+
+    def generate_guards(self, other, box, cpu, extra_guards):
+        if self.generalization_of(other):
+            return
+        self._generate_guards(other, box, cpu, extra_guards)
+
+    def _generate_guards(self, other, box, cpu, extra_guards):
+        raise NotImplementedError
+        
 class AbstractVirtualStructInfo(AbstractVirtualInfo):
     def __init__(self, fielddescrs):
         self.fielddescrs = fielddescrs
@@ -468,6 +481,9 @@
 
     def _generalization_of(self, other):
         raise NotImplementedError
+
+    def _generate_guards(self, other, box, cpu, extra_guards):
+        raise InvalidLoop
             
         
 

From commits-noreply at bitbucket.org  Sun Feb  6 12:58:34 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Sun,  6 Feb 2011 12:58:34 +0100 (CET)
Subject: [pypy-svn] pypy default: hg merge
Message-ID: <20110206115834.08C44282BDD@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: 
Changeset: r41646:d893efb1b2c4
Date: 2011-02-06 12:57 +0100
http://bitbucket.org/pypy/pypy/changeset/d893efb1b2c4/

Log:	hg merge


From commits-noreply at bitbucket.org  Sun Feb  6 13:48:00 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sun,  6 Feb 2011 13:48:00 +0100 (CET)
Subject: [pypy-svn] pypy default: Refactor rbigint to use the proper types
	more	explicitly:
Message-ID: <20110206124800.11CF436C21F@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41647:04682c4565b6
Date: 2011-02-06 12:21 +0100
http://bitbucket.org/pypy/pypy/changeset/04682c4565b6/

Log:	Refactor rbigint to use the proper types more explicitly:

	* the digits are now a list of rffi.INT, so that they are 32-bits
	even on 64-bit machines

	* to read a digit out of the list, there are now three functions:
	digit() which returns an int, udigit() which returns an unsigned
	int, and widedigit() which returns a longlong. Trying to use
	carefully the right one at each place...

diff --git a/pypy/rlib/test/test_rbigint.py b/pypy/rlib/test/test_rbigint.py
--- a/pypy/rlib/test/test_rbigint.py
+++ b/pypy/rlib/test/test_rbigint.py
@@ -3,6 +3,7 @@
 import operator, sys
 from random import random, randint, sample
 from pypy.rlib.rbigint import rbigint, SHIFT, MASK, KARATSUBA_CUTOFF
+from pypy.rlib.rbigint import _store_digit
 from pypy.rlib import rbigint as lobj
 from pypy.rlib.rarithmetic import r_uint, r_longlong, r_ulonglong, intmask
 from pypy.rpython.test.test_llinterp import interpret
@@ -17,7 +18,15 @@
                     r1 = getattr(rl_op1, op)(rl_op2)
                     r2 = getattr(operator, op)(op1, op2)
                     assert r1.tolong() == r2
-            
+
+    def test_str(self):
+        for i in range(100):
+            n = 3 ** i
+            r1 = rbigint.fromlong(n)
+            assert r1.str() == str(n)
+            r2 = rbigint.fromlong(-n)
+            assert r2.str() == str(-n)
+
     def test_floordiv(self):
         for op1 in [-12, -2, -1, 1, 2, 50]:
             for op2 in [-4, -2, -1, 1, 2, 8]:
@@ -67,47 +76,52 @@
             yield s
             yield -s
 
+def bigint(lst, sign):
+    for digit in lst:
+        assert digit & MASK == digit    # wrongly written test!
+    return rbigint(map(_store_digit, lst), sign)
+
 
 class Test_rbigint(object):
 
     def test_args_from_long(self):
         BASE = 1 << SHIFT
-        assert rbigint.fromlong(0).eq(rbigint([0], 0))
-        assert rbigint.fromlong(17).eq(rbigint([17], 1))
-        assert rbigint.fromlong(BASE-1).eq(rbigint([intmask(BASE-1)], 1))
-        assert rbigint.fromlong(BASE).eq(rbigint([0, 1], 1))
-        assert rbigint.fromlong(BASE**2).eq(rbigint([0, 0, 1], 1))
-        assert rbigint.fromlong(-17).eq(rbigint([17], -1))
-        assert rbigint.fromlong(-(BASE-1)).eq(rbigint([intmask(BASE-1)], -1))
-        assert rbigint.fromlong(-BASE).eq(rbigint([0, 1], -1))
-        assert rbigint.fromlong(-(BASE**2)).eq(rbigint([0, 0, 1], -1))
+        assert rbigint.fromlong(0).eq(bigint([0], 0))
+        assert rbigint.fromlong(17).eq(bigint([17], 1))
+        assert rbigint.fromlong(BASE-1).eq(bigint([intmask(BASE-1)], 1))
+        assert rbigint.fromlong(BASE).eq(bigint([0, 1], 1))
+        assert rbigint.fromlong(BASE**2).eq(bigint([0, 0, 1], 1))
+        assert rbigint.fromlong(-17).eq(bigint([17], -1))
+        assert rbigint.fromlong(-(BASE-1)).eq(bigint([intmask(BASE-1)], -1))
+        assert rbigint.fromlong(-BASE).eq(bigint([0, 1], -1))
+        assert rbigint.fromlong(-(BASE**2)).eq(bigint([0, 0, 1], -1))
 #        assert rbigint.fromlong(-sys.maxint-1).eq(
 #            rbigint.digits_for_most_neg_long(-sys.maxint-1), -1)
 
     def test_args_from_int(self):
         BASE = 1 << SHIFT
         MAX = int(BASE-1)
-        assert rbigint.fromrarith_int(0).eq(rbigint([0], 0))
-        assert rbigint.fromrarith_int(17).eq(rbigint([17], 1))
-        assert rbigint.fromrarith_int(MAX).eq(rbigint([MAX], 1))
-        assert rbigint.fromrarith_int(r_longlong(BASE)).eq(rbigint([0, 1], 1))
+        assert rbigint.fromrarith_int(0).eq(bigint([0], 0))
+        assert rbigint.fromrarith_int(17).eq(bigint([17], 1))
+        assert rbigint.fromrarith_int(MAX).eq(bigint([MAX], 1))
+        assert rbigint.fromrarith_int(r_longlong(BASE)).eq(bigint([0, 1], 1))
         assert rbigint.fromrarith_int(r_longlong(BASE**2)).eq(
-            rbigint([0, 0, 1], 1))
-        assert rbigint.fromrarith_int(-17).eq(rbigint([17], -1))
-        assert rbigint.fromrarith_int(-MAX).eq(rbigint([MAX], -1))
-        assert rbigint.fromrarith_int(-MAX-1).eq(rbigint([0, 1], -1))
+            bigint([0, 0, 1], 1))
+        assert rbigint.fromrarith_int(-17).eq(bigint([17], -1))
+        assert rbigint.fromrarith_int(-MAX).eq(bigint([MAX], -1))
+        assert rbigint.fromrarith_int(-MAX-1).eq(bigint([0, 1], -1))
         assert rbigint.fromrarith_int(r_longlong(-(BASE**2))).eq(
-            rbigint([0, 0, 1], -1))
+            bigint([0, 0, 1], -1))
 #        assert rbigint.fromrarith_int(-sys.maxint-1).eq((
 #            rbigint.digits_for_most_neg_long(-sys.maxint-1), -1)
 
     def test_args_from_uint(self):
         BASE = 1 << SHIFT
-        assert rbigint.fromrarith_int(r_uint(0)).eq(rbigint([0], 0))
-        assert rbigint.fromrarith_int(r_uint(17)).eq(rbigint([17], 1))
-        assert rbigint.fromrarith_int(r_uint(BASE-1)).eq(rbigint([intmask(BASE-1)], 1))
-        assert rbigint.fromrarith_int(r_uint(BASE)).eq(rbigint([0, 1], 1))
-        #assert rbigint.fromrarith_int(r_uint(BASE**2)).eq(rbigint([0], 0))
+        assert rbigint.fromrarith_int(r_uint(0)).eq(bigint([0], 0))
+        assert rbigint.fromrarith_int(r_uint(17)).eq(bigint([17], 1))
+        assert rbigint.fromrarith_int(r_uint(BASE-1)).eq(bigint([intmask(BASE-1)], 1))
+        assert rbigint.fromrarith_int(r_uint(BASE)).eq(bigint([0, 1], 1))
+        #assert rbigint.fromrarith_int(r_uint(BASE**2)).eq(bigint([0], 0))
         assert rbigint.fromrarith_int(r_uint(sys.maxint)).eq(
             rbigint.fromint(sys.maxint))
         assert rbigint.fromrarith_int(r_uint(sys.maxint+1)).eq(
@@ -175,7 +189,7 @@
         x = x ** 100
         f1 = rbigint.fromlong(x)
         assert raises(OverflowError, f1.tofloat)
-        f2 = rbigint([0, 2097152], 1)
+        f2 = rbigint.fromlong(2097152 << SHIFT)
         d = f2.tofloat()
         assert d == float(2097152 << SHIFT)
 
@@ -290,12 +304,30 @@
         f3 = rbigint.fromlong(z)
         v = f1.pow(f2, f3)
         assert v.tolong() == pow(x, y, z)
+        f3n = f3.neg()
+        v = f1.pow(f2, f3n)
+        assert v.tolong() == pow(x, y, -z)
+        #
         f1, f2, f3 = [rbigint.fromlong(i)
                       for i in (10L, -1L, 42L)]
         py.test.raises(TypeError, f1.pow, f2, f3)
         f1, f2, f3 = [rbigint.fromlong(i)
                       for i in (10L, 5L, 0L)]
         py.test.raises(ValueError, f1.pow, f2, f3)
+        #
+        MAX = 1E40
+        x = long(random() * MAX) + 1
+        y = long(random() * MAX) + 1
+        z = long(random() * MAX) + 1
+        f1 = rbigint.fromlong(x)
+        f2 = rbigint.fromlong(y)
+        f3 = rbigint.fromlong(z)
+        print f1
+        print f2
+        print f3
+        v = f1.pow(f2, f3)
+        print '--->', v
+        assert v.tolong() == pow(x, y, z)
 
     def test_pow_lln(self):
         x = 10L
@@ -306,10 +338,10 @@
         assert v.tolong() == x ** y
 
     def test_normalize(self):
-        f1 = rbigint([1, 0], 1)
+        f1 = bigint([1, 0], 1)
         f1._normalize()
-        assert len(f1.digits) == 1
-        f0 = rbigint([0], 0)
+        assert len(f1._digits) == 1
+        f0 = bigint([0], 0)
         assert f1.sub(f1).eq(f0)
 
     def test_invert(self):
@@ -394,11 +426,13 @@
     def test__inplace_divrem1(self):
         # signs are not handled in the helpers!
         for x, y in [(1238585838347L, 3), (1234123412311231L, 1231231), (99, 100)]:
+            if y > MASK:
+                continue
             f1 = rbigint.fromlong(x)
             f2 = y
             remainder = lobj._inplace_divrem1(f1, f1, f2)
             assert (f1.tolong(), remainder) == divmod(x, y)
-        out = rbigint([99, 99], 1)
+        out = bigint([99, 99], 1)
         remainder = lobj._inplace_divrem1(out, out, 100)
 
     def test__divrem1(self):
@@ -447,16 +481,16 @@
 
     # testing Karatsuba stuff
     def test__v_iadd(self):
-        f1 = rbigint([lobj.MASK] * 10, 1)
-        f2 = rbigint([1], 1)
-        carry = lobj._v_iadd(f1, 1, len(f1.digits)-1, f2, 1)
+        f1 = bigint([lobj.MASK] * 10, 1)
+        f2 = bigint([1], 1)
+        carry = lobj._v_iadd(f1, 1, len(f1._digits)-1, f2, 1)
         assert carry == 1
         assert f1.tolong() == lobj.MASK
 
     def test__v_isub(self):
-        f1 = rbigint([lobj.MASK] + [0] * 9 + [1], 1)
-        f2 = rbigint([1], 1)
-        borrow = lobj._v_isub(f1, 1, len(f1.digits)-1, f2, 1)
+        f1 = bigint([lobj.MASK] + [0] * 9 + [1], 1)
+        f2 = bigint([1], 1)
+        borrow = lobj._v_isub(f1, 1, len(f1._digits)-1, f2, 1)
         assert borrow == 0
         assert f1.tolong() == (1 << lobj.SHIFT) ** 10 - 1
 
@@ -464,23 +498,23 @@
         split = 5
         diglo = [0] * split
         dighi = [lobj.MASK] * split
-        f1 = rbigint(diglo + dighi, 1)
+        f1 = bigint(diglo + dighi, 1)
         hi, lo = lobj._kmul_split(f1, split)
-        assert lo.digits == [0]
-        assert hi.digits == dighi
+        assert lo._digits == [_store_digit(0)]
+        assert hi._digits == map(_store_digit, dighi)
 
     def test__k_mul(self):
         digs = KARATSUBA_CUTOFF * 5
-        f1 = rbigint([lobj.MASK] * digs, 1)
-        f2 = lobj._x_add(f1,rbigint([1], 1))
+        f1 = bigint([lobj.MASK] * digs, 1)
+        f2 = lobj._x_add(f1, bigint([1], 1))
         ret = lobj._k_mul(f1, f2)
         assert ret.tolong() == f1.tolong() * f2.tolong()
 
     def test__k_lopsided_mul(self):
         digs_a = KARATSUBA_CUTOFF + 3
         digs_b = 3 * digs_a
-        f1 = rbigint([lobj.MASK] * digs_a, 1)
-        f2 = rbigint([lobj.MASK] * digs_b, 1)
+        f1 = bigint([lobj.MASK] * digs_a, 1)
+        f2 = bigint([lobj.MASK] * digs_b, 1)
         ret = lobj._k_lopsided_mul(f1, f2)
         assert ret.tolong() == f1.tolong() * f2.tolong()
 
@@ -547,7 +581,9 @@
 class TestTranslatable(object):
     def test_square(self):
         def test():
-            x = rbigint([1410065408, 4], 1)
+            xlo = rbigint.fromint(1410065408)
+            xhi = rbigint.fromint(4)
+            x = xlo.or_(xhi.lshift(31))
             y = x.mul(x)
             return y.str()
         res = interpret(test, [])

diff --git a/pypy/rlib/rbigint.py b/pypy/rlib/rbigint.py
--- a/pypy/rlib/rbigint.py
+++ b/pypy/rlib/rbigint.py
@@ -3,6 +3,8 @@
 from pypy.rlib.rarithmetic import most_neg_value_of_same_type
 from pypy.rlib.debug import make_sure_not_resized, check_regular_int
 from pypy.rlib.objectmodel import we_are_translated
+from pypy.rpython.lltypesystem import lltype, rffi
+from pypy.rpython import extregistry
 
 import math, sys
 
@@ -39,36 +41,84 @@
 FIVEARY_CUTOFF = 8
 
 
-def mask_digit(x):
+def _mask_digit(x):
+    if not we_are_translated():
+        assert type(x) is not long, "overflow occurred!"
     return intmask(x & MASK)
-mask_digit._annspecialcase_ = 'specialize:argtype(0)'
+_mask_digit._annspecialcase_ = 'specialize:argtype(0)'
 
-def widen_digit(x):
+def _widen_digit(x):
+    if not we_are_translated():
+        assert type(x) is int, "widen_digit() takes an int, got a %r" % type(x)
     if SHIFT <= 15:
         return int(x)
     return r_longlong(x)
 
+def _store_digit(x):
+    if not we_are_translated():
+        assert type(x) is int, "store_digit() takes an int, got a %r" % type(x)
+    if SHIFT <= 15:
+        return rffi.cast(rffi.SHORT, x)
+    elif SHIFT <= 31:
+        return rffi.cast(rffi.INT, x)
+    else:
+        raise ValueError("SHIFT too large!")
+
+def _load_digit(x):
+    return rffi.cast(lltype.Signed, x)
+
+def _load_unsigned_digit(x):
+    return rffi.cast(lltype.Unsigned, x)
+
+NULLDIGIT = _store_digit(0)
+
+def _check_digits(l):
+    for x in l:
+        assert type(x) is type(NULLDIGIT)
+        assert intmask(x) & MASK == intmask(x)
+class Entry(extregistry.ExtRegistryEntry):
+    _about_ = _check_digits
+    def compute_result_annotation(self, s_list):
+        from pypy.annotation import model as annmodel
+        assert isinstance(s_list, annmodel.SomeList)
+        s_DIGIT = self.bookkeeper.valueoftype(type(NULLDIGIT))
+        assert s_DIGIT.contains(s_list.listdef.listitem.s_value)
+    def specialize_call(self, hop):
+        pass
+
 
 class rbigint(object):
     """This is a reimplementation of longs using a list of digits."""
-    # XXX relace the list of ints with a list of rffi.INTs, maybe
-    
-    def __init__(self, digits=None, sign=0):
-        if digits is None or len(digits) == 0:
-            digits = [0]
+
+    def __init__(self, digits=[], sign=0):
+        if len(digits) == 0:
+            digits = [NULLDIGIT]
+        _check_digits(digits)
         make_sure_not_resized(digits)
-        self.digits = digits
+        self._digits = digits
         self.sign = sign
 
-    def _digit(self, x):
-        return widen_digit(self.digits[x])
-    def _setdigit(self, x, val):
-        val = mask_digit(val)
+    def digit(self, x):
+        """Return the x'th digit, as an int."""
+        return _load_digit(self._digits[x])
+
+    def widedigit(self, x):
+        """Return the x'th digit, as a long long int if needed
+        to have enough room to contain two digits."""
+        return _widen_digit(_load_digit(self._digits[x]))
+
+    def udigit(self, x):
+        """Return the x'th digit, as an unsigned int."""
+        return _load_unsigned_digit(self._digits[x])
+
+    def setdigit(self, x, val):
+        val = _mask_digit(val)
         assert val >= 0
-        self.digits[x] = int(val)
-    _setdigit._annspecialcase_ = 'specialize:argtype(2)'
-    def _numdigits(self):
-        return len(self.digits)
+        self._digits[x] = _store_digit(val)
+    setdigit._annspecialcase_ = 'specialize:argtype(2)'
+
+    def numdigits(self):
+        return len(self._digits)
 
     def fromint(intval):
         check_regular_int(intval)
@@ -89,11 +139,11 @@
         while t:
             ndigits += 1
             t >>= SHIFT
-        v = rbigint([0] * ndigits, sign)
+        v = rbigint([NULLDIGIT] * ndigits, sign)
         t = ival
         p = 0
         while t:
-            v._setdigit(p, t)
+            v.setdigit(p, t)
             t >>= SHIFT
             p += 1
         return v
@@ -121,11 +171,11 @@
         if expo <= 0:
             return rbigint()
         ndig = (expo-1) // SHIFT + 1 # Number of 'digits' in result
-        v = rbigint([0] * ndig, 1)
+        v = rbigint([NULLDIGIT] * ndig, 1)
         frac = math.ldexp(frac, (expo-1) % SHIFT + 1)
         for i in range(ndig-1, -1, -1):
-            bits = mask_digit(int(frac))
-            v._setdigit(i, bits)
+            bits = int(frac)
+            v.setdigit(i, bits)
             frac -= float(bits)
             frac = math.ldexp(frac, SHIFT)
         if neg:
@@ -170,10 +220,10 @@
 
     def _touint_helper(self):
         x = r_uint(0)
-        i = self._numdigits() - 1
+        i = self.numdigits() - 1
         while i >= 0:
             prev = x
-            x = r_uint((x << SHIFT) + self.digits[i])
+            x = (x << SHIFT) + self.udigit(i)
             if (x >> SHIFT) != prev:
                 raise OverflowError(
                         "long int too large to convert to unsigned int")
@@ -195,26 +245,6 @@
     def tofloat(self):
         return _AsDouble(self)
 
-    def _count_bits(self):
-        # return the number of bits in the digits
-        if self.sign == 0:
-            return 0
-        p = self._numdigits() - 1
-        bits = SHIFT * p
-        digit = self.digits[p]
-        while digit:
-            digit >>= 1
-            bits += 1
-        return bits
-
-    def is_odd(self):
-        # Note: this is a tiny optimization.
-        # Instead of implementing a general "get_bit" operation,
-        # which would be expensive for negative numbers,
-        # get_odd has the nice feature that it is always correct,
-        # no matter what the sign is (two's complement)
-        return self.digits[0] & 1
-
     def format(self, digits, prefix='', suffix=''):
         # 'digits' is a string whose length is the base to use,
         # and where each character is the corresponding digit.
@@ -228,12 +258,12 @@
 
     def eq(self, other):
         if (self.sign != other.sign or
-            self._numdigits() != other._numdigits()):
+            self.numdigits() != other.numdigits()):
             return False
         i = 0
-        ld = self._numdigits()
+        ld = self.numdigits()
         while i < ld:
-            if self.digits[i] != other.digits[i]:
+            if self.digit(i) != other.digit(i):
                 return False
             i += 1
         return True
@@ -246,8 +276,8 @@
             return False
         if self.sign < other.sign:
             return True
-        ld1 = self._numdigits()
-        ld2 = other._numdigits()
+        ld1 = self.numdigits()
+        ld2 = other.numdigits()
         if ld1 > ld2:
             if other.sign > 0:
                 return False
@@ -260,8 +290,8 @@
                 return False
         i = ld1 - 1
         while i >= 0:
-            d1 = self.digits[i]
-            d2 = other.digits[i]
+            d1 = self.digit(i)
+            d2 = other.digit(i)
             if d1 < d2:
                 if other.sign > 0:
                     return True
@@ -303,7 +333,7 @@
         if other.sign == 0:
             return self
         if self.sign == 0:
-            return rbigint(other.digits[:], -other.sign)
+            return rbigint(other._digits[:], -other.sign)
         if self.sign == other.sign:
             result = _x_sub(self, other)
         else:
@@ -355,7 +385,7 @@
         div, mod = _divrem(v, w)
         if mod.sign * w.sign == -1:
             mod = mod.add(w)
-            div = div.sub(rbigint([1], 1))
+            div = div.sub(rbigint([_store_digit(1)], 1))
         return div, mod
 
     def pow(a, b, c=None):
@@ -382,11 +412,11 @@
             #     modulus = -modulus
             if c.sign < 0:
                 negativeOutput = True
-                c = rbigint(c.digits, -c.sign)
+                c = c.neg()
 
             # if modulus == 1:
             #     return 0
-            if c._numdigits() == 1 and c.digits[0] == 1:
+            if c.numdigits() == 1 and c.digit(0) == 1:
                 return rbigint()
 
             # if base < 0:
@@ -399,16 +429,16 @@
         # At this point a, b, and c are guaranteed non-negative UNLESS
         # c is NULL, in which case a may be negative. */
 
-        z = rbigint([1], 1)
+        z = rbigint([_store_digit(1)], 1)
 
         # python adaptation: moved macros REDUCE(X) and MULT(X, Y, result)
         # into helper function result = _help_mult(x, y, c)
-        if b._numdigits() <= FIVEARY_CUTOFF:
+        if b.numdigits() <= FIVEARY_CUTOFF:
             # Left-to-right binary exponentiation (HAC Algorithm 14.79)
             # http://www.cacr.math.uwaterloo.ca/hac/about/chap14.pdf
-            i = b._numdigits() - 1
+            i = b.numdigits() - 1
             while i >= 0:
-                bi = b.digits[i]
+                bi = b.digit(i)
                 j = 1 << (SHIFT-1)
                 while j != 0:
                     z = _help_mult(z, z, c)
@@ -418,14 +448,15 @@
                 i -= 1
         else:
             # Left-to-right 5-ary exponentiation (HAC Algorithm 14.82)
+            # This is only useful in the case where c != None.
             # z still holds 1L
             table = [z] * 32
-            table[0] = z;
+            table[0] = z
             for i in range(1, 32):
                 table[i] = _help_mult(table[i-1], a, c)
-            i = b._numdigits() - 1
+            i = b.numdigits() - 1
             while i >= 0:
-                bi = b.digits[i]
+                bi = b.digit(i)
                 j = SHIFT - 5
                 while j >= 0:
                     index = (bi >> j) & 0x1f
@@ -441,13 +472,13 @@
         return z
 
     def neg(self):
-        return rbigint(self.digits, -self.sign)
+        return rbigint(self._digits, -self.sign)
 
     def abs(self):
-        return rbigint(self.digits, abs(self.sign))
+        return rbigint(self._digits, abs(self.sign))
 
     def invert(self): #Implement ~x as -(x + 1)
-        return self.add(rbigint([1], 1)).neg()
+        return self.add(rbigint([_store_digit(1)], 1)).neg()
 
     def lshift(self, int_other):
         if int_other < 0:
@@ -459,22 +490,22 @@
         wordshift = int_other // SHIFT
         remshift  = int_other - wordshift * SHIFT
 
-        oldsize = self._numdigits()
+        oldsize = self.numdigits()
         newsize = oldsize + wordshift
         if remshift:
             newsize += 1
-        z = rbigint([0] * newsize, self.sign)
-        accum = 0
+        z = rbigint([NULLDIGIT] * newsize, self.sign)
+        accum = _widen_digit(0)
         i = wordshift
         j = 0
         while j < oldsize:
-            accum |= self._digit(j) << remshift
-            z._setdigit(i, accum)
+            accum |= self.widedigit(j) << remshift
+            z.setdigit(i, accum)
             accum >>= SHIFT
             i += 1
             j += 1
         if remshift:
-            z._setdigit(newsize - 1, accum)
+            z.setdigit(newsize - 1, accum)
         else:
             assert not accum
         z._normalize()
@@ -491,21 +522,22 @@
             return a2.invert()
 
         wordshift = int_other // SHIFT
-        newsize = self._numdigits() - wordshift
+        newsize = self.numdigits() - wordshift
         if newsize <= 0:
             return rbigint()
 
         loshift = int_other % SHIFT
         hishift = SHIFT - loshift
-        lomask = (1 << hishift) - 1
+        lomask = intmask((r_uint(1) << hishift) - 1)
         himask = MASK ^ lomask
-        z = rbigint([0] * newsize, self.sign)
+        z = rbigint([NULLDIGIT] * newsize, self.sign)
         i = 0
         j = wordshift
         while i < newsize:
-            z.digits[i] = (self.digits[j] >> loshift) & lomask
+            newdigit = (self.digit(j) >> loshift) & lomask
             if i+1 < newsize:
-                z.digits[i] |= (self.digits[j+1] << hishift) & himask
+                newdigit |= intmask(self.digit(j+1) << hishift) & himask
+            z.setdigit(i, newdigit)
             i += 1
             j += 1
         z._normalize()
@@ -538,34 +570,35 @@
             ret /= math.log(base)
         return ret
 
-    def tolong(self): #YYYYYY
-        l = 0
-        digits = list(self.digits)
+    def tolong(self):
+        "NOT_RPYTHON"
+        l = 0L
+        digits = list(self._digits)
         digits.reverse()
         for d in digits:
             l = l << SHIFT
-            l += long(d)
+            l += intmask(d)
         return l * self.sign
 
     def _normalize(self):
-        if self._numdigits() == 0:
+        if self.numdigits() == 0:
             self.sign = 0
-            self.digits = [0]
+            self._digits = [NULLDIGIT]
             return
-        i = self._numdigits()
-        while i > 1 and self.digits[i - 1] == 0:
+        i = self.numdigits()
+        while i > 1 and self.digit(i - 1) == 0:
             i -= 1
         assert i >= 1
-        if i != self._numdigits():
-            self.digits = self.digits[:i]
-        if self._numdigits() == 1 and self.digits[0] == 0:
+        if i != self.numdigits():
+            self._digits = self._digits[:i]
+        if self.numdigits() == 1 and self.digit(0) == 0:
             self.sign = 0
 
     def bit_length(self):
-        i = self._numdigits()
-        if i == 1 and self.digits[0] == 0:
+        i = self.numdigits()
+        if i == 1 and self.digit(0) == 0:
             return 0
-        msd = self.digits[i - 1]
+        msd = self.digit(i - 1)
         msd_bits = 0
         while msd >= 32:
             msd_bits += 6
@@ -580,7 +613,8 @@
         return bits
 
     def __repr__(self):
-        return "<rbigint digits=%s, sign=%s, %s>" % (self.digits, self.sign, self.str())
+        return "<rbigint digits=%s, sign=%s, %s>" % (self._digits,
+                                                     self.sign, self.str())
 
 #_________________________________________________________________
 
@@ -605,7 +639,7 @@
 def digits_from_nonneg_long(l):
     digits = []
     while True:
-        digits.append(mask_digit(l))
+        digits.append(_store_digit(intmask(l & MASK)))
         l = l >> SHIFT
         if not l:
             return digits[:] # to make it non-resizable
@@ -615,15 +649,15 @@
     # This helper only works if 'l' is the most negative integer of its
     # type, which in base 2 looks like: 1000000..0000
     digits = []
-    while (mask_digit(l)) == 0:
-        digits.append(0)
+    while _mask_digit(l) == 0:
+        digits.append(NULLDIGIT)
         l = l >> SHIFT
     # now 'l' looks like: ...111100000
     # turn it into:       ...000100000
     # to drop the extra unwanted 1's introduced by the signed right shift
     l = -intmask(l)
-    assert l >= 0
-    digits.append(l)
+    assert l & MASK == l
+    digits.append(_store_digit(l))
     return digits[:] # to make it non-resizable
 digits_for_most_neg_long._annspecialcase_ = "specialize:argtype(0)"
 
@@ -631,7 +665,7 @@
     if x > 0:
         return digits_from_nonneg_long(x), 1
     elif x == 0:
-        return [0], 0
+        return [NULLDIGIT], 0
     elif x != most_neg_value_of_same_type(x):
         # normal case
         return digits_from_nonneg_long(-x), -1
@@ -650,42 +684,42 @@
     "NOT_RPYTHON"
     if x >= 0:
         if x == 0:
-            return [0], 0
+            return [NULLDIGIT], 0
         else:
             return digits_from_nonneg_long(x), 1
     else:
-        return digits_from_nonneg_long(-long(x)), -1
+        return digits_from_nonneg_long(-x), -1
 
 def _x_add(a, b):
     """ Add the absolute values of two bigint integers. """
-    size_a = a._numdigits()
-    size_b = b._numdigits()
+    size_a = a.numdigits()
+    size_b = b.numdigits()
 
     # Ensure a is the larger of the two:
     if size_a < size_b:
         a, b = b, a
         size_a, size_b = size_b, size_a
-    z = rbigint([0] * (a._numdigits() + 1), 1)
+    z = rbigint([NULLDIGIT] * (a.numdigits() + 1), 1)
     i = 0
-    carry = 0
+    carry = r_uint(0)
     while i < size_b:
-        carry += a._digit(i) + b._digit(i)
-        z._setdigit(i, carry)
+        carry += a.udigit(i) + b.udigit(i)
+        z.setdigit(i, carry)
         carry >>= SHIFT
         i += 1
     while i < size_a:
-        carry += a._digit(i)
-        z._setdigit(i, carry)
+        carry += a.udigit(i)
+        z.setdigit(i, carry)
         carry >>= SHIFT
         i += 1
-    z._setdigit(i, carry)
+    z.setdigit(i, carry)
     z._normalize()
     return z
 
 def _x_sub(a, b):
     """ Subtract the absolute values of two integers. """
-    size_a = a._numdigits()
-    size_b = b._numdigits()
+    size_a = a.numdigits()
+    size_b = b.numdigits()
     sign = 1
 
     # Ensure a is the larger of the two:
@@ -696,34 +730,32 @@
     elif size_a == size_b:
         # Find highest digit where a and b differ:
         i = size_a - 1
-        while i >= 0 and a.digits[i] == b.digits[i]:
+        while i >= 0 and a.digit(i) == b.digit(i):
             i -= 1
         if i < 0:
-            return rbigint([0], 0)
-        if a.digits[i] < b.digits[i]:
+            return rbigint()
+        if a.digit(i) < b.digit(i):
             sign = -1
             a, b = b, a
         size_a = size_b = i+1
-    borrow = 0
-    z = rbigint([0] * size_a, 1)
+    z = rbigint([NULLDIGIT] * size_a, sign)
+    borrow = r_uint(0)
     i = 0
     while i < size_b:
         # The following assumes unsigned arithmetic
         # works modulo 2**N for some N>SHIFT.
-        borrow = a._digit(i) - b._digit(i) - borrow
-        z._setdigit(i, borrow)
+        borrow = a.udigit(i) - b.udigit(i) - borrow
+        z.setdigit(i, borrow)
         borrow >>= SHIFT
         borrow &= 1 # Keep only one sign bit
         i += 1
     while i < size_a:
-        borrow = a._digit(i) - borrow
-        z._setdigit(i, borrow)
+        borrow = a.udigit(i) - borrow
+        z.setdigit(i, borrow)
         borrow >>= SHIFT
         borrow &= 1 # Keep only one sign bit
         i += 1
     assert borrow == 0
-    if sign < 0:
-        z.sign = -1
     z._normalize()
     return z
 
@@ -734,9 +766,9 @@
     Returns the absolute value of the product, or None if error.
     """
 
-    size_a = a._numdigits()
-    size_b = b._numdigits()
-    z = rbigint([0] * (size_a + size_b), 1)
+    size_a = a.numdigits()
+    size_b = b.numdigits()
+    z = rbigint([NULLDIGIT] * (size_a + size_b), 1)
     if a is b:
         # Efficient squaring per HAC, Algorithm 14.16:
         # http://www.cacr.math.uwaterloo.ca/hac/about/chap14.pdf
@@ -745,13 +777,13 @@
         # pyramid appears twice (except for the size_a squares).
         i = 0
         while i < size_a:
-            f = a._digit(i)
+            f = a.widedigit(i)
             pz = i << 1
             pa = i + 1
             paend = size_a
 
-            carry = z._digit(pz) + f * f
-            z._setdigit(pz, carry)
+            carry = z.widedigit(pz) + f * f
+            z.setdigit(pz, carry)
             pz += 1
             carry >>= SHIFT
             assert carry <= MASK
@@ -760,19 +792,19 @@
             # pyramid it appears.  Same as adding f<<1 once.
             f <<= 1
             while pa < paend:
-                carry += z._digit(pz) + a._digit(pa) * f
+                carry += z.widedigit(pz) + a.widedigit(pa) * f
                 pa += 1
-                z._setdigit(pz, carry)
+                z.setdigit(pz, carry)
                 pz += 1
                 carry >>= SHIFT
-                assert carry <= (widen_digit(MASK) << 1)
+                assert carry <= (_widen_digit(MASK) << 1)
             if carry:
-                carry += z._digit(pz)
-                z._setdigit(pz, carry)
+                carry += z.widedigit(pz)
+                z.setdigit(pz, carry)
                 pz += 1
                 carry >>= SHIFT
             if carry:
-                z._setdigit(pz, z._digit(pz) + carry)
+                z.setdigit(pz, z.widedigit(pz) + carry)
             assert (carry >> SHIFT) == 0
             i += 1
     else:
@@ -780,19 +812,19 @@
         i = 0
         while i < size_a:
             carry = 0
-            f = a._digit(i)
+            f = a.widedigit(i)
             pz = i
             pb = 0
             pbend = size_b
             while pb < pbend:
-                carry += z._digit(pz) + b._digit(pb) * f
+                carry += z.widedigit(pz) + b.widedigit(pb) * f
                 pb += 1
-                z._setdigit(pz, carry)
+                z.setdigit(pz, carry)
                 pz += 1
                 carry >>= SHIFT
                 assert carry <= MASK
             if carry:
-                z._setdigit(pz, z._digit(pz) + carry)
+                z.setdigit(pz, z.widedigit(pz) + carry)
             assert (carry >> SHIFT) == 0
             i += 1
     z._normalize()
@@ -807,11 +839,11 @@
     viewing the shift as being by digits.  The sign bit is ignored, and
     the return values are >= 0.
     """
-    size_n = n._numdigits()
+    size_n = n.numdigits()
     size_lo = min(size_n, size)
 
-    lo = rbigint(n.digits[:size_lo], 1)
-    hi = rbigint(n.digits[size_lo:], 1)
+    lo = rbigint(n._digits[:size_lo], 1)
+    hi = rbigint(n._digits[size_lo:], 1)
     lo._normalize()
     hi._normalize()
     return hi, lo
@@ -822,8 +854,8 @@
     absolute value of the product (or raises if error).
     See Knuth Vol. 2 Chapter 4.3.3 (Pp. 294-295).
     """
-    asize = a._numdigits()
-    bsize = b._numdigits()
+    asize = a.numdigits()
+    bsize = b.numdigits()
     # (ah*X+al)(bh*X+bl) = ah*bh*X*X + (ah*bl + al*bh)*X + al*bl
     # Let k = (ah+al)*(bh+bl) = ah*bl + al*bh  + ah*bh + al*bl
     # Then the original product is
@@ -843,7 +875,7 @@
         i = KARATSUBA_CUTOFF
     if asize <= i:
         if a.sign == 0:
-            return rbigint([0], 0)
+            return rbigint()     # zero
         else:
             return _x_mul(a, b)
 
@@ -882,13 +914,13 @@
     #    at shift.
 
     # 1. Allocate result space.
-    ret = rbigint([0] * (asize + bsize), 1)
+    ret = rbigint([NULLDIGIT] * (asize + bsize), 1)
 
     # 2. t1 <- ah*bh, and copy into high digits of result.
     t1 = _k_mul(ah, bh)
     assert t1.sign >= 0
-    assert 2*shift + t1._numdigits() <= ret._numdigits()
-    ret.digits[2*shift : 2*shift + t1._numdigits()] = t1.digits
+    assert 2*shift + t1.numdigits() <= ret.numdigits()
+    ret._digits[2*shift : 2*shift + t1.numdigits()] = t1._digits
 
     # Zero-out the digits higher than the ah*bh copy. */
     ## ignored, assuming that we initialize to zero
@@ -900,8 +932,8 @@
     # 3. t2 <- al*bl, and copy into the low digits.
     t2 = _k_mul(al, bl)
     assert t2.sign >= 0
-    assert t2._numdigits() <= 2*shift # no overlap with high digits
-    ret.digits[:t2._numdigits()] = t2.digits
+    assert t2.numdigits() <= 2*shift # no overlap with high digits
+    ret._digits[:t2.numdigits()] = t2._digits
 
     # Zero out remaining digits.
     ## ignored, assuming that we initialize to zero
@@ -911,9 +943,9 @@
 
     # 4 & 5. Subtract ah*bh (t1) and al*bl (t2).  We do al*bl first
     # because it's fresher in cache.
-    i = ret._numdigits() - shift  # # digits after shift
-    _v_isub(ret, shift, i, t2, t2._numdigits())
-    _v_isub(ret, shift, i, t1, t1._numdigits())
+    i = ret.numdigits() - shift  # # digits after shift
+    _v_isub(ret, shift, i, t2, t2.numdigits())
+    _v_isub(ret, shift, i, t1, t1.numdigits())
     del t1, t2
 
     # 6. t3 <- (ah+al)(bh+bl), and add into result.
@@ -932,7 +964,7 @@
 
     # Add t3.  It's not obvious why we can't run out of room here.
     # See the (*) comment after this function.
-    _v_iadd(ret, shift, i, t3, t3._numdigits())
+    _v_iadd(ret, shift, i, t3, t3.numdigits())
     del t3
 
     ret._normalize()
@@ -993,20 +1025,20 @@
     at a time, then move on, never bactracking except for the helpful
     single-width slice overlap between successive partial sums).
     """
-    asize = a._numdigits()
-    bsize = b._numdigits()
+    asize = a.numdigits()
+    bsize = b.numdigits()
     # nbdone is # of b digits already multiplied
 
     assert asize > KARATSUBA_CUTOFF
     assert 2 * asize <= bsize
 
     # Allocate result space, and zero it out.
-    ret = rbigint([0] * (asize + bsize), 1)
+    ret = rbigint([NULLDIGIT] * (asize + bsize), 1)
 
     # Successive slices of b are copied into bslice.
     #bslice = rbigint([0] * asize, 1)
     # XXX we cannot pre-allocate, see comments below!
-    bslice = rbigint([0], 1)
+    bslice = rbigint([NULLDIGIT], 1)
 
     nbdone = 0;
     while bsize > 0:
@@ -1018,12 +1050,12 @@
         # XXX: this would be more efficient if we adopted CPython's
         # way to store the size, instead of resizing the list!
         # XXX change the implementation, encoding length via the sign.
-        bslice.digits = b.digits[nbdone : nbdone + nbtouse]
+        bslice._digits = b._digits[nbdone : nbdone + nbtouse]
         product = _k_mul(a, bslice)
 
         # Add into result.
-        _v_iadd(ret, nbdone, ret._numdigits() - nbdone,
-                 product, product._numdigits())
+        _v_iadd(ret, nbdone, ret.numdigits() - nbdone,
+                 product, product.numdigits())
         del product
 
         bsize -= nbtouse
@@ -1038,18 +1070,18 @@
     Divide bigint pin by non-zero digit n, storing quotient
     in pout, and returning the remainder. It's OK for pin == pout on entry.
     """
-    rem = widen_digit(0)
+    rem = _widen_digit(0)
     assert n > 0 and n <= MASK
     if not size:
-        size = pin._numdigits()
+        size = pin.numdigits()
     size -= 1
     while size >= 0:
-        rem = (rem << SHIFT) + pin._digit(size)
+        rem = (rem << SHIFT) + pin.widedigit(size)
         hi = rem // n
-        pout._setdigit(size, hi)
+        pout.setdigit(size, hi)
         rem -= hi * n
         size -= 1
-    return mask_digit(rem)
+    return _mask_digit(rem)
 
 def _divrem1(a, n):
     """
@@ -1058,8 +1090,8 @@
     The sign of a is ignored; n should not be zero.
     """
     assert n > 0 and n <= MASK
-    size = a._numdigits()
-    z = rbigint([0] * size, 1)
+    size = a.numdigits()
+    z = rbigint([NULLDIGIT] * size, 1)
     rem = _inplace_divrem1(z, a, n)
     z._normalize()
     return z, rem
@@ -1071,21 +1103,21 @@
     x[m-1], and the remaining carry (0 or 1) is returned.
     Python adaptation: x is addressed relative to xofs!
     """
-    carry = 0;
+    carry = r_uint(0)
 
     assert m >= n
     i = xofs
     iend = xofs + n
     while i < iend:
-        carry += x._digit(i) + y._digit(i-xofs)
-        x._setdigit(i, carry)
+        carry += x.udigit(i) + y.udigit(i-xofs)
+        x.setdigit(i, carry)
         carry >>= SHIFT
         assert (carry & 1) == carry
         i += 1
     iend = xofs + m
     while carry and i < iend:
-        carry += x._digit(i)
-        x._setdigit(i, carry)
+        carry += x.udigit(i)
+        x.setdigit(i, carry)
         carry >>= SHIFT
         assert (carry & 1) == carry
         i += 1
@@ -1098,57 +1130,59 @@
     far as x[m-1], and the remaining borrow (0 or 1) is returned.
     Python adaptation: x is addressed relative to xofs!
     """
-    borrow = 0
+    borrow = r_uint(0)
 
     assert m >= n
     i = xofs
     iend = xofs + n
     while i < iend:
-        borrow = x._digit(i) - y._digit(i-xofs) - borrow
-        x._setdigit(i, borrow)
+        borrow = x.udigit(i) - y.udigit(i-xofs) - borrow
+        x.setdigit(i, borrow)
         borrow >>= SHIFT
         borrow &= 1    # keep only 1 sign bit
         i += 1
     iend = xofs + m
     while borrow and i < iend:
-        borrow = x._digit(i) - borrow
-        x._setdigit(i, borrow)
+        borrow = x.udigit(i) - borrow
+        x.setdigit(i, borrow)
         borrow >>= SHIFT
         borrow &= 1
         i += 1
     return borrow
 
 
-def _muladd1(a, n, extra=widen_digit(0)):
+def _muladd1(a, n, extra=0):
     """Multiply by a single digit and add a single digit, ignoring the sign.
     """
-    size_a = a._numdigits()
-    z = rbigint([0] * (size_a+1), 1)
-    carry = extra
-    assert carry & MASK == carry
+    size_a = a.numdigits()
+    z = rbigint([NULLDIGIT] * (size_a+1), 1)
+    assert extra & MASK == extra
+    carry = _widen_digit(extra)
     i = 0
     while i < size_a:
-        carry += a._digit(i) * n
-        z._setdigit(i, carry)
+        carry += a.widedigit(i) * n
+        z.setdigit(i, carry)
         carry >>= SHIFT
         i += 1
-    z._setdigit(i, carry)
+    z.setdigit(i, carry)
     z._normalize()
     return z
 
 
 def _x_divrem(v1, w1):
     """ Unsigned bigint division with remainder -- the algorithm """
-    size_w = w1._numdigits()
-    d = (widen_digit(MASK)+1) // (w1._digit(size_w-1) + 1)
+    size_w = w1.numdigits()
+    d = (r_uint(MASK)+1) // (w1.udigit(size_w-1) + 1)
+    assert d <= MASK    # because the first digit of w1 is not zero
+    d = intmask(d)
     v = _muladd1(v1, d)
     w = _muladd1(w1, d)
-    size_v = v._numdigits()
-    size_w = w._numdigits()
+    size_v = v.numdigits()
+    size_w = w.numdigits()
     assert size_v >= size_w and size_w > 1 # Assert checks by div()
 
     size_a = size_v - size_w + 1
-    a = rbigint([0] * size_a, 1)
+    a = rbigint([NULLDIGIT] * size_a, 1)
 
     j = size_v
     k = size_a - 1
@@ -1156,50 +1190,50 @@
         if j >= size_v:
             vj = 0
         else:
-            vj = v._digit(j)
+            vj = v.widedigit(j)
         carry = 0
 
-        if vj == w._digit(size_w-1):
+        if vj == w.widedigit(size_w-1):
             q = MASK
         else:
-            q = ((vj << SHIFT) + v._digit(j-1)) // w._digit(size_w-1)
+            q = ((vj << SHIFT) + v.widedigit(j-1)) // w.widedigit(size_w-1)
 
-        while (w._digit(size_w-2) * q >
+        while (w.widedigit(size_w-2) * q >
                 ((
                     (vj << SHIFT)
-                    + v._digit(j-1)
-                    - q * w._digit(size_w-1)
+                    + v.widedigit(j-1)
+                    - q * w.widedigit(size_w-1)
                                 ) << SHIFT)
-                + v._digit(j-2)):
+                + v.widedigit(j-2)):
             q -= 1
         i = 0
         while i < size_w and i+k < size_v:
-            z = w._digit(i) * q
+            z = w.widedigit(i) * q
             zz = z >> SHIFT
-            carry += v._digit(i+k) - z + (zz << SHIFT)
-            v._setdigit(i+k, carry)
+            carry += v.widedigit(i+k) - z + (zz << SHIFT)
+            v.setdigit(i+k, carry)
             carry >>= SHIFT
             carry -= zz
             i += 1
 
         if i+k < size_v:
-            carry += v._digit(i+k)
-            v._setdigit(i+k, 0)
+            carry += v.widedigit(i+k)
+            v.setdigit(i+k, 0)
 
         if carry == 0:
-            a._setdigit(k, q)
+            a.setdigit(k, q)
             assert not q >> SHIFT
         else:
             assert carry == -1
             q -= 1
-            a._setdigit(k, q)
+            a.setdigit(k, q)
             assert not q >> SHIFT
 
             carry = 0
             i = 0
             while i < size_w and i+k < size_v:
-                carry += v._digit(i+k) + w._digit(i)
-                v._setdigit(i+k, carry)
+                carry += v.udigit(i+k) + w.udigit(i)
+                v.setdigit(i+k, carry)
                 carry >>= SHIFT
                 i += 1
         j -= 1
@@ -1212,22 +1246,22 @@
 
 def _divrem(a, b):
     """ Long division with remainder, top-level routine """
-    size_a = a._numdigits()
-    size_b = b._numdigits()
+    size_a = a.numdigits()
+    size_b = b.numdigits()
 
     if b.sign == 0:
         raise ZeroDivisionError("long division or modulo by zero")
 
     if (size_a < size_b or
         (size_a == size_b and
-         a.digits[size_a-1] < b.digits[size_b-1])):
+         a.digit(size_a-1) < b.digit(size_b-1))):
         # |a| < |b|
-        z = rbigint([0], 0)
+        z = rbigint()   # result is 0
         rem = a
         return z, rem
     if size_b == 1:
-        z, urem = _divrem1(a, b._digit(0))
-        rem = rbigint([urem], int(urem != 0))
+        z, urem = _divrem1(a, b.digit(0))
+        rem = rbigint([_store_digit(urem)], int(urem != 0))
     else:
         z, rem = _x_divrem(a, b)
     # Set the signs.
@@ -1257,14 +1291,14 @@
     NBITS_WANTED = 57
     if v.sign == 0:
         return 0.0, 0
-    i = v._numdigits() - 1
+    i = v.numdigits() - 1
     sign = v.sign
-    x = float(v.digits[i])
+    x = float(v.digit(i))
     nbitsneeded = NBITS_WANTED - 1
     # Invariant:  i Python digits remain unaccounted for.
     while i > 0 and nbitsneeded > 0:
         i -= 1
-        x = x * FLOAT_MULTIPLIER + float(v.digits[i])
+        x = x * FLOAT_MULTIPLIER + float(v.digit(i))
         nbitsneeded -= SHIFT
     # There are i digits we didn't shift in.  Pretending they're all
     # zeroes, the true value is x * 2**(i*SHIFT).
@@ -1385,7 +1419,7 @@
     Convert a bigint object to a string, using a given conversion base.
     Return a string object.
     """
-    size_a = a._numdigits()
+    size_a = a.numdigits()
 
     base = len(digits)
     assert base >= 2 and base <= 36
@@ -1421,11 +1455,11 @@
             basebits += 1
 
         for i in range(size_a):
-            accum |= a._digit(i) << accumbits
+            accum |= a.widedigit(i) << accumbits
             accumbits += SHIFT
             assert accumbits >= basebits
             while 1:
-                cdigit = accum & (base - 1)
+                cdigit = intmask(accum & (base - 1))
                 p -= 1
                 assert p >= 0
                 s[p] = digits[cdigit]
@@ -1444,7 +1478,7 @@
         size = size_a
         pin = a # just for similarity to C source which uses the array
         # powbase <- largest power of base that fits in a digit.
-        powbase = widen_digit(base)  # powbase == base ** power
+        powbase = _widen_digit(base)  # powbase == base ** power
         power = 1
         while 1:
             newpow = powbase * base
@@ -1454,14 +1488,14 @@
             power += 1
 
         # Get a scratch area for repeated division.
-        scratch = rbigint([0] * size, 1)
+        scratch = rbigint([NULLDIGIT] * size, 1)
 
         # Repeatedly divide by powbase.
         while 1:
             ntostore = power
             rem = _inplace_divrem1(scratch, pin, powbase, size)
             pin = scratch  # no need to use a again
-            if pin.digits[size - 1] == 0:
+            if pin.digit(size - 1) == 0:
                 size -= 1
 
             # Break rem into digits.
@@ -1538,8 +1572,8 @@
     # iff one of these cases applies, and mask will be non-0 for operands
     # whose length should be ignored.
 
-    size_a = a._numdigits()
-    size_b = b._numdigits()
+    size_a = a.numdigits()
+    size_b = b.numdigits()
     if op == '&':
         if maska:
             size_z = size_b
@@ -1551,23 +1585,23 @@
     else:
         size_z = max(size_a, size_b)
 
-    z = rbigint([0] * size_z, 1)
+    z = rbigint([NULLDIGIT] * size_z, 1)
 
     for i in range(size_z):
         if i < size_a:
-            diga = a.digits[i] ^ maska
+            diga = a.digit(i) ^ maska
         else:
             diga = maska
         if i < size_b:
-            digb = b.digits[i] ^ maskb
+            digb = b.digit(i) ^ maskb
         else:
             digb = maskb
         if op == '&':
-            z.digits[i] = diga & digb
+            z.setdigit(i, diga & digb)
         elif op == '|':
-            z.digits[i] = diga | digb
+            z.setdigit(i, diga | digb)
         elif op == '^':
-            z.digits[i] = diga ^ digb
+            z.setdigit(i, diga ^ digb)
 
     z._normalize()
     if negz == 0:
@@ -1599,10 +1633,10 @@
 
 def _AsULonglong_ignore_sign(v):
     x = r_ulonglong(0)
-    i = v._numdigits() - 1
+    i = v.numdigits() - 1
     while i >= 0:
         prev = x
-        x = (x << SHIFT) + v._digit(i)
+        x = (x << SHIFT) + v.widedigit(i)
         if (x >> SHIFT) != prev:
                 raise OverflowError(
                     "long int too large to convert to unsigned long long int")
@@ -1612,10 +1646,9 @@
 def make_unsigned_mask_conversion(T):
     def _As_unsigned_mask(v):
         x = T(0)
-        i = v._numdigits() - 1
+        i = v.numdigits() - 1
         while i >= 0:
-            prev = x
-            x = (x << SHIFT) + T(v._digit(i))
+            x = (x << SHIFT) + T(v.digit(i))
             i -= 1
         if v.sign < 0:
             x = -x
@@ -1632,18 +1665,18 @@
     # to please decimal.py, we return a hash that satisfies
     # hash(x) == hash(x % ULONG_MAX).  In particular, this
     # implies that hash(x) == hash(x % (2**64-1)).
-    i = v._numdigits() - 1
+    i = v.numdigits() - 1
     sign = v.sign
     x = r_uint(0)
     LONG_BIT_SHIFT = LONG_BIT - SHIFT
     while i >= 0:
         # Force a native long #-bits (32 or 64) circular shift
         x = (x << SHIFT) | (x >> LONG_BIT_SHIFT)
-        x += r_uint(v.digits[i])
+        x += v.udigit(i)
         # If the addition above overflowed we compensate by
         # incrementing.  This preserves the value modulo
         # ULONG_MAX.
-        if x < r_uint(v.digits[i]):
+        if x < v.udigit(i):
             x += 1
         i -= 1
     x = intmask(x * sign)

From commits-noreply at bitbucket.org  Sun Feb  6 13:48:00 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sun,  6 Feb 2011 13:48:00 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix this call to ovfcheck(). I know
	RPython is not	completely obvious.
Message-ID: <20110206124800.A644136C21F@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41648:98560d7bb148
Date: 2011-02-06 10:47 +0100
http://bitbucket.org/pypy/pypy/changeset/98560d7bb148/

Log:	Fix this call to ovfcheck(). I know RPython is not completely
	obvious.

diff --git a/pypy/rlib/rbigint.py b/pypy/rlib/rbigint.py
--- a/pypy/rlib/rbigint.py
+++ b/pypy/rlib/rbigint.py
@@ -576,7 +576,7 @@
             ][msd]
         # yes, this can overflow: a huge number which fits 3 gigabytes of
         # memory has around 24 gigabits!
-        bits = ovfcheck((i-1) * SHIFT + msd_bits)
+        bits = ovfcheck((i-1) * SHIFT) + msd_bits
         return bits
 
     def __repr__(self):

From commits-noreply at bitbucket.org  Sun Feb  6 13:48:01 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sun,  6 Feb 2011 13:48:01 +0100 (CET)
Subject: [pypy-svn] pypy default: merge heads
Message-ID: <20110206124801.AC5F636C21F@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41649:8ec34df8ed8a
Date: 2011-02-06 12:36 +0100
http://bitbucket.org/pypy/pypy/changeset/8ec34df8ed8a/

Log:	merge heads



From commits-noreply at bitbucket.org  Sun Feb  6 13:48:03 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sun,  6 Feb 2011 13:48:03 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix the new _bigint_true_divide() too.
Message-ID: <20110206124803.7A84F36C21F@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41650:946bfdb56371
Date: 2011-02-06 12:59 +0100
http://bitbucket.org/pypy/pypy/changeset/946bfdb56371/

Log:	Fix the new _bigint_true_divide() too.

diff --git a/pypy/rlib/rbigint.py b/pypy/rlib/rbigint.py
--- a/pypy/rlib/rbigint.py
+++ b/pypy/rlib/rbigint.py
@@ -1409,8 +1409,8 @@
     if not a.tobool():
         return _truediv_result(0.0, negate)
 
-    a_size = a._numdigits()
-    b_size = b._numdigits()
+    a_size = a.numdigits()
+    b_size = b.numdigits()
 
     # Fast path for a and b small (exactly representable in a double).
     # Relies on floating-point division being correctly rounded; results
@@ -1418,24 +1418,24 @@
     # the x87 FPU set to 64-bit precision.
     a_is_small = (a_size <= MANT_DIG_DIGITS or
                   (a_size == MANT_DIG_DIGITS+1 and
-                   a.digits[MANT_DIG_DIGITS] >> MANT_DIG_BITS == 0))
+                   a.digit(MANT_DIG_DIGITS) >> MANT_DIG_BITS == 0))
     b_is_small = (b_size <= MANT_DIG_DIGITS or
                   (b_size == MANT_DIG_DIGITS+1 and
-                   b.digits[MANT_DIG_DIGITS] >> MANT_DIG_BITS == 0))
+                   b.digit(MANT_DIG_DIGITS) >> MANT_DIG_BITS == 0))
     if a_is_small and b_is_small:
         a_size -= 1
-        da = float(a.digits[a_size])
+        da = float(a.digit(a_size))
         while True:
             a_size -= 1
             if a_size < 0: break
-            da = da * BASE_AS_FLOAT + a.digits[a_size]
+            da = da * BASE_AS_FLOAT + a.digit(a_size)
 
         b_size -= 1
-        db = float(b.digits[b_size])
+        db = float(b.digit(b_size))
         while True:
             b_size -= 1
             if b_size < 0: break
-            db = db * BASE_AS_FLOAT + b.digits[b_size]
+            db = db * BASE_AS_FLOAT + b.digit(b_size)
 
         return _truediv_result(da / db, negate)
 
@@ -1446,8 +1446,8 @@
     elif diff < 1 - sys.maxint/SHIFT:
         return _truediv_result(0.0, negate)  # Extreme underflow
     # Next line is now safe from overflowing integers
-    diff = (diff * SHIFT + bits_in_digit(a.digits[a_size - 1]) -
-                           bits_in_digit(b.digits[b_size - 1]))
+    diff = (diff * SHIFT + bits_in_digit(a.digit(a_size - 1)) -
+                           bits_in_digit(b.digit(b_size - 1)))
     # Now diff = a_bits - b_bits.
     if diff > DBL_MAX_EXP:
         return _truediv_overflow()
@@ -1474,8 +1474,8 @@
         inexact = True
 
     assert x.tobool()    # result of division is never zero
-    x_size = x._numdigits()
-    x_bits = (x_size-1)*SHIFT + bits_in_digit(x.digits[x_size-1])
+    x_size = x.numdigits()
+    x_bits = (x_size-1)*SHIFT + bits_in_digit(x.digit(x_size-1))
 
     # The number of extra bits that have to be rounded away.
     extra_bits = max(x_bits, DBL_MIN_EXP - shift) - DBL_MANT_DIG
@@ -1483,7 +1483,7 @@
 
     # Round by remembering a modified copy of the low digit of x
     mask = 1 << (extra_bits - 1)
-    low = x.digits[0] | inexact
+    low = x.digit(0) | inexact
     if (low & mask) != 0 and (low & (3*mask-1)) != 0:
         low += mask
     x_digit_0 = low & ~(mask-1)
@@ -1492,7 +1492,7 @@
     x_size -= 1
     dx = 0.0
     while x_size > 0:
-        dx += x.digits[x_size]
+        dx += x.digit(x_size)
         dx *= BASE_AS_FLOAT
         x_size -= 1
     dx += x_digit_0

From commits-noreply at bitbucket.org  Sun Feb  6 13:48:04 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sun,  6 Feb 2011 13:48:04 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix this call to ovfcheck(). I know
	RPython is not	completely obvious.
Message-ID: <20110206124804.90EF636C21F@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41651:8924cbf5038e
Date: 2011-02-06 10:47 +0100
http://bitbucket.org/pypy/pypy/changeset/8924cbf5038e/

Log:	Fix this call to ovfcheck(). I know RPython is not completely
	obvious.

diff --git a/pypy/rlib/rbigint.py b/pypy/rlib/rbigint.py
--- a/pypy/rlib/rbigint.py
+++ b/pypy/rlib/rbigint.py
@@ -576,7 +576,7 @@
             ][msd]
         # yes, this can overflow: a huge number which fits 3 gigabytes of
         # memory has around 24 gigabits!
-        bits = ovfcheck((i-1) * SHIFT + msd_bits)
+        bits = ovfcheck((i-1) * SHIFT) + msd_bits
         return bits
 
     def __repr__(self):

From commits-noreply at bitbucket.org  Sun Feb  6 13:48:05 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sun,  6 Feb 2011 13:48:05 +0100 (CET)
Subject: [pypy-svn] pypy default: merge heads
Message-ID: <20110206124805.E9ECC36C21F@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41652:5d43e0a91661
Date: 2011-02-06 12:36 +0100
http://bitbucket.org/pypy/pypy/changeset/5d43e0a91661/

Log:	merge heads

diff --git a/pypy/rlib/test/test_rbigint.py b/pypy/rlib/test/test_rbigint.py
--- a/pypy/rlib/test/test_rbigint.py
+++ b/pypy/rlib/test/test_rbigint.py
@@ -45,6 +45,12 @@
                 r2 = op1 / op2
                 assert r1 == r2
 
+    def test_truediv_precision(self):
+        op1 = rbigint.fromlong(12345*2**30)
+        op2 = rbigint.fromlong(98765*7**81)
+        f = op1.truediv(op2)
+        assert f == 4.7298422347492634e-61      # exactly
+
     def test_mod(self):
         for op1 in [-50, -12, -2, -1, 1, 2, 50, 52]:
             for op2 in [-4, -2, -1, 1, 2, 8]:

diff --git a/pypy/rlib/rbigint.py b/pypy/rlib/rbigint.py
--- a/pypy/rlib/rbigint.py
+++ b/pypy/rlib/rbigint.py
@@ -609,7 +609,7 @@
             ][msd]
         # yes, this can overflow: a huge number which fits 3 gigabytes of
         # memory has around 24 gigabits!
-        bits = ovfcheck((i-1) * SHIFT + msd_bits)
+        bits = ovfcheck((i-1) * SHIFT) + msd_bits
         return bits
 
     def __repr__(self):
@@ -1390,25 +1390,155 @@
     return func(x) + (e * float(SHIFT) * func(2.0))
 _loghelper._annspecialcase_ = 'specialize:arg(0)'
 
+# ____________________________________________________________
+
+BASE_AS_FLOAT = float(1 << SHIFT)     # note that it may not fit an int
+
+BitLengthTable = ''.join(map(chr, [
+    0, 1, 2, 2, 3, 3, 3, 3, 4, 4, 4, 4, 4, 4, 4, 4,
+    5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5]))
+
+def bits_in_digit(d):
+    # returns the unique integer k such that 2**(k-1) <= d <
+    # 2**k if d is nonzero, else 0.
+    d_bits = 0
+    while d >= 32:
+        d_bits += 6
+        d >>= 6
+    d_bits += ord(BitLengthTable[d])
+    return d_bits
+
+def _truediv_result(result, negate):
+    if negate:
+        result = -result
+    return result
+
+def _truediv_overflow():
+    raise OverflowError("integer division result too large for a float")
+
 def _bigint_true_divide(a, b):
-    ad, aexp = _AsScaledDouble(a)
-    bd, bexp = _AsScaledDouble(b)
-    if bd == 0.0:
+    # A longish method to obtain the floating-point result with as much
+    # precision as theoretically possible.  The code is almost directly
+    # copied from CPython.  See there (Objects/longobject.c,
+    # long_true_divide) for detailled comments.  Method in a nutshell:
+    #
+    #    0. reduce to case a, b > 0; filter out obvious underflow/overflow
+    #    1. choose a suitable integer 'shift'
+    #    2. use integer arithmetic to compute x = floor(2**-shift*a/b)
+    #    3. adjust x for correct rounding
+    #    4. convert x to a double dx with the same value
+    #    5. return ldexp(dx, shift).
+
+    from pypy.rlib import rfloat
+    DBL_MANT_DIG = rfloat.DBL_MANT_DIG  # 53 for IEEE 754 binary64
+    DBL_MAX_EXP = rfloat.DBL_MAX_EXP    # 1024 for IEEE 754 binary64
+    DBL_MIN_EXP = rfloat.DBL_MIN_EXP
+    MANT_DIG_DIGITS = DBL_MANT_DIG // SHIFT
+    MANT_DIG_BITS = DBL_MANT_DIG % SHIFT
+
+    # Reduce to case where a and b are both positive.
+    negate = (a.sign < 0) ^ (b.sign < 0)
+    if not b.tobool():
         raise ZeroDivisionError("long division or modulo by zero")
+    if not a.tobool():
+        return _truediv_result(0.0, negate)
 
-    # True value is very close to ad/bd * 2**(SHIFT*(aexp-bexp))
-    ad /= bd   # overflow/underflow impossible here
-    aexp -= bexp
-    if aexp > sys.maxint / SHIFT:
-        raise OverflowError
-    elif aexp < -(sys.maxint / SHIFT):
-        return 0.0 # underflow to 0
-    ad = math.ldexp(ad, aexp * SHIFT)
-    ##if isinf(ad):   # ignore underflow to 0.0
-    ##    raise OverflowError
-    # math.ldexp checks and raises
-    return ad
+    a_size = a._numdigits()
+    b_size = b._numdigits()
 
+    # Fast path for a and b small (exactly representable in a double).
+    # Relies on floating-point division being correctly rounded; results
+    # may be subject to double rounding on x86 machines that operate with
+    # the x87 FPU set to 64-bit precision.
+    a_is_small = (a_size <= MANT_DIG_DIGITS or
+                  (a_size == MANT_DIG_DIGITS+1 and
+                   a.digits[MANT_DIG_DIGITS] >> MANT_DIG_BITS == 0))
+    b_is_small = (b_size <= MANT_DIG_DIGITS or
+                  (b_size == MANT_DIG_DIGITS+1 and
+                   b.digits[MANT_DIG_DIGITS] >> MANT_DIG_BITS == 0))
+    if a_is_small and b_is_small:
+        a_size -= 1
+        da = float(a.digits[a_size])
+        while True:
+            a_size -= 1
+            if a_size < 0: break
+            da = da * BASE_AS_FLOAT + a.digits[a_size]
+
+        b_size -= 1
+        db = float(b.digits[b_size])
+        while True:
+            b_size -= 1
+            if b_size < 0: break
+            db = db * BASE_AS_FLOAT + b.digits[b_size]
+
+        return _truediv_result(da / db, negate)
+
+    # Catch obvious cases of underflow and overflow
+    diff = a_size - b_size
+    if diff > sys.maxint/SHIFT - 1:
+        return _truediv_overflow()           # Extreme overflow
+    elif diff < 1 - sys.maxint/SHIFT:
+        return _truediv_result(0.0, negate)  # Extreme underflow
+    # Next line is now safe from overflowing integers
+    diff = (diff * SHIFT + bits_in_digit(a.digits[a_size - 1]) -
+                           bits_in_digit(b.digits[b_size - 1]))
+    # Now diff = a_bits - b_bits.
+    if diff > DBL_MAX_EXP:
+        return _truediv_overflow()
+    elif diff < DBL_MIN_EXP - DBL_MANT_DIG - 1:
+        return _truediv_result(0.0, negate)
+
+    # Choose value for shift; see comments for step 1 in CPython.
+    shift = max(diff, DBL_MIN_EXP) - DBL_MANT_DIG - 2
+
+    inexact = False
+
+    # x = abs(a * 2**-shift)
+    if shift <= 0:
+        x = a.lshift(-shift)
+    else:
+        x = a.rshift(shift)
+        # set inexact if any of the bits shifted out is nonzero
+        if not a.eq(x.lshift(shift)):
+            inexact = True
+
+    # x //= b. If the remainder is nonzero, set inexact.
+    x, rem = x.divmod(b)
+    if rem.tobool():
+        inexact = True
+
+    assert x.tobool()    # result of division is never zero
+    x_size = x._numdigits()
+    x_bits = (x_size-1)*SHIFT + bits_in_digit(x.digits[x_size-1])
+
+    # The number of extra bits that have to be rounded away.
+    extra_bits = max(x_bits, DBL_MIN_EXP - shift) - DBL_MANT_DIG
+    assert extra_bits == 2 or extra_bits == 3
+
+    # Round by remembering a modified copy of the low digit of x
+    mask = 1 << (extra_bits - 1)
+    low = x.digits[0] | inexact
+    if (low & mask) != 0 and (low & (3*mask-1)) != 0:
+        low += mask
+    x_digit_0 = low & ~(mask-1)
+
+    # Convert x to a double dx; the conversion is exact.
+    x_size -= 1
+    dx = 0.0
+    while x_size > 0:
+        dx += x.digits[x_size]
+        dx *= BASE_AS_FLOAT
+        x_size -= 1
+    dx += x_digit_0
+
+    # Check whether ldexp result will overflow a double.
+    if (shift + x_bits >= DBL_MAX_EXP and
+        (shift + x_bits > DBL_MAX_EXP or dx == math.ldexp(1.0, x_bits))):
+        return _truediv_overflow()
+
+    return _truediv_result(math.ldexp(dx, shift), negate)
+
+# ____________________________________________________________
 
 BASE8  = '01234567'
 BASE10 = '0123456789'

From commits-noreply at bitbucket.org  Sun Feb  6 13:48:08 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sun,  6 Feb 2011 13:48:08 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix the new _bigint_true_divide() too.
Message-ID: <20110206124808.3E3422A2010@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41653:71f5106973cf
Date: 2011-02-06 12:59 +0100
http://bitbucket.org/pypy/pypy/changeset/71f5106973cf/

Log:	Fix the new _bigint_true_divide() too.

diff --git a/pypy/rlib/rbigint.py b/pypy/rlib/rbigint.py
--- a/pypy/rlib/rbigint.py
+++ b/pypy/rlib/rbigint.py
@@ -1443,8 +1443,8 @@
     if not a.tobool():
         return _truediv_result(0.0, negate)
 
-    a_size = a._numdigits()
-    b_size = b._numdigits()
+    a_size = a.numdigits()
+    b_size = b.numdigits()
 
     # Fast path for a and b small (exactly representable in a double).
     # Relies on floating-point division being correctly rounded; results
@@ -1452,24 +1452,24 @@
     # the x87 FPU set to 64-bit precision.
     a_is_small = (a_size <= MANT_DIG_DIGITS or
                   (a_size == MANT_DIG_DIGITS+1 and
-                   a.digits[MANT_DIG_DIGITS] >> MANT_DIG_BITS == 0))
+                   a.digit(MANT_DIG_DIGITS) >> MANT_DIG_BITS == 0))
     b_is_small = (b_size <= MANT_DIG_DIGITS or
                   (b_size == MANT_DIG_DIGITS+1 and
-                   b.digits[MANT_DIG_DIGITS] >> MANT_DIG_BITS == 0))
+                   b.digit(MANT_DIG_DIGITS) >> MANT_DIG_BITS == 0))
     if a_is_small and b_is_small:
         a_size -= 1
-        da = float(a.digits[a_size])
+        da = float(a.digit(a_size))
         while True:
             a_size -= 1
             if a_size < 0: break
-            da = da * BASE_AS_FLOAT + a.digits[a_size]
+            da = da * BASE_AS_FLOAT + a.digit(a_size)
 
         b_size -= 1
-        db = float(b.digits[b_size])
+        db = float(b.digit(b_size))
         while True:
             b_size -= 1
             if b_size < 0: break
-            db = db * BASE_AS_FLOAT + b.digits[b_size]
+            db = db * BASE_AS_FLOAT + b.digit(b_size)
 
         return _truediv_result(da / db, negate)
 
@@ -1480,8 +1480,8 @@
     elif diff < 1 - sys.maxint/SHIFT:
         return _truediv_result(0.0, negate)  # Extreme underflow
     # Next line is now safe from overflowing integers
-    diff = (diff * SHIFT + bits_in_digit(a.digits[a_size - 1]) -
-                           bits_in_digit(b.digits[b_size - 1]))
+    diff = (diff * SHIFT + bits_in_digit(a.digit(a_size - 1)) -
+                           bits_in_digit(b.digit(b_size - 1)))
     # Now diff = a_bits - b_bits.
     if diff > DBL_MAX_EXP:
         return _truediv_overflow()
@@ -1508,8 +1508,8 @@
         inexact = True
 
     assert x.tobool()    # result of division is never zero
-    x_size = x._numdigits()
-    x_bits = (x_size-1)*SHIFT + bits_in_digit(x.digits[x_size-1])
+    x_size = x.numdigits()
+    x_bits = (x_size-1)*SHIFT + bits_in_digit(x.digit(x_size-1))
 
     # The number of extra bits that have to be rounded away.
     extra_bits = max(x_bits, DBL_MIN_EXP - shift) - DBL_MANT_DIG
@@ -1517,7 +1517,7 @@
 
     # Round by remembering a modified copy of the low digit of x
     mask = 1 << (extra_bits - 1)
-    low = x.digits[0] | inexact
+    low = x.digit(0) | inexact
     if (low & mask) != 0 and (low & (3*mask-1)) != 0:
         low += mask
     x_digit_0 = low & ~(mask-1)
@@ -1526,7 +1526,7 @@
     x_size -= 1
     dx = 0.0
     while x_size > 0:
-        dx += x.digits[x_size]
+        dx += x.digit(x_size)
         dx *= BASE_AS_FLOAT
         x_size -= 1
     dx += x_digit_0

From commits-noreply at bitbucket.org  Sun Feb  6 13:48:10 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sun,  6 Feb 2011 13:48:10 +0100 (CET)
Subject: [pypy-svn] pypy default: merge heads
Message-ID: <20110206124810.45FAB282B9C@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41654:f252b80b9c9f
Date: 2011-02-06 13:45 +0100
http://bitbucket.org/pypy/pypy/changeset/f252b80b9c9f/

Log:	merge heads

diff --git a/pypy/rlib/test/test_rbigint.py b/pypy/rlib/test/test_rbigint.py
--- a/pypy/rlib/test/test_rbigint.py
+++ b/pypy/rlib/test/test_rbigint.py
@@ -3,6 +3,7 @@
 import operator, sys
 from random import random, randint, sample
 from pypy.rlib.rbigint import rbigint, SHIFT, MASK, KARATSUBA_CUTOFF
+from pypy.rlib.rbigint import _store_digit
 from pypy.rlib import rbigint as lobj
 from pypy.rlib.rarithmetic import r_uint, r_longlong, r_ulonglong, intmask
 from pypy.rpython.test.test_llinterp import interpret
@@ -17,7 +18,15 @@
                     r1 = getattr(rl_op1, op)(rl_op2)
                     r2 = getattr(operator, op)(op1, op2)
                     assert r1.tolong() == r2
-            
+
+    def test_str(self):
+        for i in range(100):
+            n = 3 ** i
+            r1 = rbigint.fromlong(n)
+            assert r1.str() == str(n)
+            r2 = rbigint.fromlong(-n)
+            assert r2.str() == str(-n)
+
     def test_floordiv(self):
         for op1 in [-12, -2, -1, 1, 2, 50]:
             for op2 in [-4, -2, -1, 1, 2, 8]:
@@ -73,47 +82,52 @@
             yield s
             yield -s
 
+def bigint(lst, sign):
+    for digit in lst:
+        assert digit & MASK == digit    # wrongly written test!
+    return rbigint(map(_store_digit, lst), sign)
+
 
 class Test_rbigint(object):
 
     def test_args_from_long(self):
         BASE = 1 << SHIFT
-        assert rbigint.fromlong(0).eq(rbigint([0], 0))
-        assert rbigint.fromlong(17).eq(rbigint([17], 1))
-        assert rbigint.fromlong(BASE-1).eq(rbigint([intmask(BASE-1)], 1))
-        assert rbigint.fromlong(BASE).eq(rbigint([0, 1], 1))
-        assert rbigint.fromlong(BASE**2).eq(rbigint([0, 0, 1], 1))
-        assert rbigint.fromlong(-17).eq(rbigint([17], -1))
-        assert rbigint.fromlong(-(BASE-1)).eq(rbigint([intmask(BASE-1)], -1))
-        assert rbigint.fromlong(-BASE).eq(rbigint([0, 1], -1))
-        assert rbigint.fromlong(-(BASE**2)).eq(rbigint([0, 0, 1], -1))
+        assert rbigint.fromlong(0).eq(bigint([0], 0))
+        assert rbigint.fromlong(17).eq(bigint([17], 1))
+        assert rbigint.fromlong(BASE-1).eq(bigint([intmask(BASE-1)], 1))
+        assert rbigint.fromlong(BASE).eq(bigint([0, 1], 1))
+        assert rbigint.fromlong(BASE**2).eq(bigint([0, 0, 1], 1))
+        assert rbigint.fromlong(-17).eq(bigint([17], -1))
+        assert rbigint.fromlong(-(BASE-1)).eq(bigint([intmask(BASE-1)], -1))
+        assert rbigint.fromlong(-BASE).eq(bigint([0, 1], -1))
+        assert rbigint.fromlong(-(BASE**2)).eq(bigint([0, 0, 1], -1))
 #        assert rbigint.fromlong(-sys.maxint-1).eq(
 #            rbigint.digits_for_most_neg_long(-sys.maxint-1), -1)
 
     def test_args_from_int(self):
         BASE = 1 << SHIFT
         MAX = int(BASE-1)
-        assert rbigint.fromrarith_int(0).eq(rbigint([0], 0))
-        assert rbigint.fromrarith_int(17).eq(rbigint([17], 1))
-        assert rbigint.fromrarith_int(MAX).eq(rbigint([MAX], 1))
-        assert rbigint.fromrarith_int(r_longlong(BASE)).eq(rbigint([0, 1], 1))
+        assert rbigint.fromrarith_int(0).eq(bigint([0], 0))
+        assert rbigint.fromrarith_int(17).eq(bigint([17], 1))
+        assert rbigint.fromrarith_int(MAX).eq(bigint([MAX], 1))
+        assert rbigint.fromrarith_int(r_longlong(BASE)).eq(bigint([0, 1], 1))
         assert rbigint.fromrarith_int(r_longlong(BASE**2)).eq(
-            rbigint([0, 0, 1], 1))
-        assert rbigint.fromrarith_int(-17).eq(rbigint([17], -1))
-        assert rbigint.fromrarith_int(-MAX).eq(rbigint([MAX], -1))
-        assert rbigint.fromrarith_int(-MAX-1).eq(rbigint([0, 1], -1))
+            bigint([0, 0, 1], 1))
+        assert rbigint.fromrarith_int(-17).eq(bigint([17], -1))
+        assert rbigint.fromrarith_int(-MAX).eq(bigint([MAX], -1))
+        assert rbigint.fromrarith_int(-MAX-1).eq(bigint([0, 1], -1))
         assert rbigint.fromrarith_int(r_longlong(-(BASE**2))).eq(
-            rbigint([0, 0, 1], -1))
+            bigint([0, 0, 1], -1))
 #        assert rbigint.fromrarith_int(-sys.maxint-1).eq((
 #            rbigint.digits_for_most_neg_long(-sys.maxint-1), -1)
 
     def test_args_from_uint(self):
         BASE = 1 << SHIFT
-        assert rbigint.fromrarith_int(r_uint(0)).eq(rbigint([0], 0))
-        assert rbigint.fromrarith_int(r_uint(17)).eq(rbigint([17], 1))
-        assert rbigint.fromrarith_int(r_uint(BASE-1)).eq(rbigint([intmask(BASE-1)], 1))
-        assert rbigint.fromrarith_int(r_uint(BASE)).eq(rbigint([0, 1], 1))
-        #assert rbigint.fromrarith_int(r_uint(BASE**2)).eq(rbigint([0], 0))
+        assert rbigint.fromrarith_int(r_uint(0)).eq(bigint([0], 0))
+        assert rbigint.fromrarith_int(r_uint(17)).eq(bigint([17], 1))
+        assert rbigint.fromrarith_int(r_uint(BASE-1)).eq(bigint([intmask(BASE-1)], 1))
+        assert rbigint.fromrarith_int(r_uint(BASE)).eq(bigint([0, 1], 1))
+        #assert rbigint.fromrarith_int(r_uint(BASE**2)).eq(bigint([0], 0))
         assert rbigint.fromrarith_int(r_uint(sys.maxint)).eq(
             rbigint.fromint(sys.maxint))
         assert rbigint.fromrarith_int(r_uint(sys.maxint+1)).eq(
@@ -181,7 +195,7 @@
         x = x ** 100
         f1 = rbigint.fromlong(x)
         assert raises(OverflowError, f1.tofloat)
-        f2 = rbigint([0, 2097152], 1)
+        f2 = rbigint.fromlong(2097152 << SHIFT)
         d = f2.tofloat()
         assert d == float(2097152 << SHIFT)
 
@@ -296,12 +310,30 @@
         f3 = rbigint.fromlong(z)
         v = f1.pow(f2, f3)
         assert v.tolong() == pow(x, y, z)
+        f3n = f3.neg()
+        v = f1.pow(f2, f3n)
+        assert v.tolong() == pow(x, y, -z)
+        #
         f1, f2, f3 = [rbigint.fromlong(i)
                       for i in (10L, -1L, 42L)]
         py.test.raises(TypeError, f1.pow, f2, f3)
         f1, f2, f3 = [rbigint.fromlong(i)
                       for i in (10L, 5L, 0L)]
         py.test.raises(ValueError, f1.pow, f2, f3)
+        #
+        MAX = 1E40
+        x = long(random() * MAX) + 1
+        y = long(random() * MAX) + 1
+        z = long(random() * MAX) + 1
+        f1 = rbigint.fromlong(x)
+        f2 = rbigint.fromlong(y)
+        f3 = rbigint.fromlong(z)
+        print f1
+        print f2
+        print f3
+        v = f1.pow(f2, f3)
+        print '--->', v
+        assert v.tolong() == pow(x, y, z)
 
     def test_pow_lln(self):
         x = 10L
@@ -312,10 +344,10 @@
         assert v.tolong() == x ** y
 
     def test_normalize(self):
-        f1 = rbigint([1, 0], 1)
+        f1 = bigint([1, 0], 1)
         f1._normalize()
-        assert len(f1.digits) == 1
-        f0 = rbigint([0], 0)
+        assert len(f1._digits) == 1
+        f0 = bigint([0], 0)
         assert f1.sub(f1).eq(f0)
 
     def test_invert(self):
@@ -400,11 +432,13 @@
     def test__inplace_divrem1(self):
         # signs are not handled in the helpers!
         for x, y in [(1238585838347L, 3), (1234123412311231L, 1231231), (99, 100)]:
+            if y > MASK:
+                continue
             f1 = rbigint.fromlong(x)
             f2 = y
             remainder = lobj._inplace_divrem1(f1, f1, f2)
             assert (f1.tolong(), remainder) == divmod(x, y)
-        out = rbigint([99, 99], 1)
+        out = bigint([99, 99], 1)
         remainder = lobj._inplace_divrem1(out, out, 100)
 
     def test__divrem1(self):
@@ -453,16 +487,16 @@
 
     # testing Karatsuba stuff
     def test__v_iadd(self):
-        f1 = rbigint([lobj.MASK] * 10, 1)
-        f2 = rbigint([1], 1)
-        carry = lobj._v_iadd(f1, 1, len(f1.digits)-1, f2, 1)
+        f1 = bigint([lobj.MASK] * 10, 1)
+        f2 = bigint([1], 1)
+        carry = lobj._v_iadd(f1, 1, len(f1._digits)-1, f2, 1)
         assert carry == 1
         assert f1.tolong() == lobj.MASK
 
     def test__v_isub(self):
-        f1 = rbigint([lobj.MASK] + [0] * 9 + [1], 1)
-        f2 = rbigint([1], 1)
-        borrow = lobj._v_isub(f1, 1, len(f1.digits)-1, f2, 1)
+        f1 = bigint([lobj.MASK] + [0] * 9 + [1], 1)
+        f2 = bigint([1], 1)
+        borrow = lobj._v_isub(f1, 1, len(f1._digits)-1, f2, 1)
         assert borrow == 0
         assert f1.tolong() == (1 << lobj.SHIFT) ** 10 - 1
 
@@ -470,23 +504,23 @@
         split = 5
         diglo = [0] * split
         dighi = [lobj.MASK] * split
-        f1 = rbigint(diglo + dighi, 1)
+        f1 = bigint(diglo + dighi, 1)
         hi, lo = lobj._kmul_split(f1, split)
-        assert lo.digits == [0]
-        assert hi.digits == dighi
+        assert lo._digits == [_store_digit(0)]
+        assert hi._digits == map(_store_digit, dighi)
 
     def test__k_mul(self):
         digs = KARATSUBA_CUTOFF * 5
-        f1 = rbigint([lobj.MASK] * digs, 1)
-        f2 = lobj._x_add(f1,rbigint([1], 1))
+        f1 = bigint([lobj.MASK] * digs, 1)
+        f2 = lobj._x_add(f1, bigint([1], 1))
         ret = lobj._k_mul(f1, f2)
         assert ret.tolong() == f1.tolong() * f2.tolong()
 
     def test__k_lopsided_mul(self):
         digs_a = KARATSUBA_CUTOFF + 3
         digs_b = 3 * digs_a
-        f1 = rbigint([lobj.MASK] * digs_a, 1)
-        f2 = rbigint([lobj.MASK] * digs_b, 1)
+        f1 = bigint([lobj.MASK] * digs_a, 1)
+        f2 = bigint([lobj.MASK] * digs_b, 1)
         ret = lobj._k_lopsided_mul(f1, f2)
         assert ret.tolong() == f1.tolong() * f2.tolong()
 
@@ -553,7 +587,9 @@
 class TestTranslatable(object):
     def test_square(self):
         def test():
-            x = rbigint([1410065408, 4], 1)
+            xlo = rbigint.fromint(1410065408)
+            xhi = rbigint.fromint(4)
+            x = xlo.or_(xhi.lshift(31))
             y = x.mul(x)
             return y.str()
         res = interpret(test, [])

diff --git a/pypy/rlib/rbigint.py b/pypy/rlib/rbigint.py
--- a/pypy/rlib/rbigint.py
+++ b/pypy/rlib/rbigint.py
@@ -3,6 +3,8 @@
 from pypy.rlib.rarithmetic import most_neg_value_of_same_type
 from pypy.rlib.debug import make_sure_not_resized, check_regular_int
 from pypy.rlib.objectmodel import we_are_translated
+from pypy.rpython.lltypesystem import lltype, rffi
+from pypy.rpython import extregistry
 
 import math, sys
 
@@ -39,36 +41,84 @@
 FIVEARY_CUTOFF = 8
 
 
-def mask_digit(x):
+def _mask_digit(x):
+    if not we_are_translated():
+        assert type(x) is not long, "overflow occurred!"
     return intmask(x & MASK)
-mask_digit._annspecialcase_ = 'specialize:argtype(0)'
+_mask_digit._annspecialcase_ = 'specialize:argtype(0)'
 
-def widen_digit(x):
+def _widen_digit(x):
+    if not we_are_translated():
+        assert type(x) is int, "widen_digit() takes an int, got a %r" % type(x)
     if SHIFT <= 15:
         return int(x)
     return r_longlong(x)
 
+def _store_digit(x):
+    if not we_are_translated():
+        assert type(x) is int, "store_digit() takes an int, got a %r" % type(x)
+    if SHIFT <= 15:
+        return rffi.cast(rffi.SHORT, x)
+    elif SHIFT <= 31:
+        return rffi.cast(rffi.INT, x)
+    else:
+        raise ValueError("SHIFT too large!")
+
+def _load_digit(x):
+    return rffi.cast(lltype.Signed, x)
+
+def _load_unsigned_digit(x):
+    return rffi.cast(lltype.Unsigned, x)
+
+NULLDIGIT = _store_digit(0)
+
+def _check_digits(l):
+    for x in l:
+        assert type(x) is type(NULLDIGIT)
+        assert intmask(x) & MASK == intmask(x)
+class Entry(extregistry.ExtRegistryEntry):
+    _about_ = _check_digits
+    def compute_result_annotation(self, s_list):
+        from pypy.annotation import model as annmodel
+        assert isinstance(s_list, annmodel.SomeList)
+        s_DIGIT = self.bookkeeper.valueoftype(type(NULLDIGIT))
+        assert s_DIGIT.contains(s_list.listdef.listitem.s_value)
+    def specialize_call(self, hop):
+        pass
+
 
 class rbigint(object):
     """This is a reimplementation of longs using a list of digits."""
-    # XXX relace the list of ints with a list of rffi.INTs, maybe
-    
-    def __init__(self, digits=None, sign=0):
-        if digits is None or len(digits) == 0:
-            digits = [0]
+
+    def __init__(self, digits=[], sign=0):
+        if len(digits) == 0:
+            digits = [NULLDIGIT]
+        _check_digits(digits)
         make_sure_not_resized(digits)
-        self.digits = digits
+        self._digits = digits
         self.sign = sign
 
-    def _digit(self, x):
-        return widen_digit(self.digits[x])
-    def _setdigit(self, x, val):
-        val = mask_digit(val)
+    def digit(self, x):
+        """Return the x'th digit, as an int."""
+        return _load_digit(self._digits[x])
+
+    def widedigit(self, x):
+        """Return the x'th digit, as a long long int if needed
+        to have enough room to contain two digits."""
+        return _widen_digit(_load_digit(self._digits[x]))
+
+    def udigit(self, x):
+        """Return the x'th digit, as an unsigned int."""
+        return _load_unsigned_digit(self._digits[x])
+
+    def setdigit(self, x, val):
+        val = _mask_digit(val)
         assert val >= 0
-        self.digits[x] = int(val)
-    _setdigit._annspecialcase_ = 'specialize:argtype(2)'
-    def _numdigits(self):
-        return len(self.digits)
+        self._digits[x] = _store_digit(val)
+    setdigit._annspecialcase_ = 'specialize:argtype(2)'
+
+    def numdigits(self):
+        return len(self._digits)
 
     def fromint(intval):
         check_regular_int(intval)
@@ -89,11 +139,11 @@
         while t:
             ndigits += 1
             t >>= SHIFT
-        v = rbigint([0] * ndigits, sign)
+        v = rbigint([NULLDIGIT] * ndigits, sign)
         t = ival
         p = 0
         while t:
-            v._setdigit(p, t)
+            v.setdigit(p, t)
             t >>= SHIFT
             p += 1
         return v
@@ -121,11 +171,11 @@
         if expo <= 0:
             return rbigint()
         ndig = (expo-1) // SHIFT + 1 # Number of 'digits' in result
-        v = rbigint([0] * ndig, 1)
+        v = rbigint([NULLDIGIT] * ndig, 1)
         frac = math.ldexp(frac, (expo-1) % SHIFT + 1)
         for i in range(ndig-1, -1, -1):
-            bits = mask_digit(int(frac))
-            v._setdigit(i, bits)
+            bits = int(frac)
+            v.setdigit(i, bits)
             frac -= float(bits)
             frac = math.ldexp(frac, SHIFT)
         if neg:
@@ -170,10 +220,10 @@
 
     def _touint_helper(self):
         x = r_uint(0)
-        i = self._numdigits() - 1
+        i = self.numdigits() - 1
         while i >= 0:
             prev = x
-            x = r_uint((x << SHIFT) + self.digits[i])
+            x = (x << SHIFT) + self.udigit(i)
             if (x >> SHIFT) != prev:
                 raise OverflowError(
                         "long int too large to convert to unsigned int")
@@ -195,26 +245,6 @@
     def tofloat(self):
         return _AsDouble(self)
 
-    def _count_bits(self):
-        # return the number of bits in the digits
-        if self.sign == 0:
-            return 0
-        p = self._numdigits() - 1
-        bits = SHIFT * p
-        digit = self.digits[p]
-        while digit:
-            digit >>= 1
-            bits += 1
-        return bits
-
-    def is_odd(self):
-        # Note: this is a tiny optimization.
-        # Instead of implementing a general "get_bit" operation,
-        # which would be expensive for negative numbers,
-        # get_odd has the nice feature that it is always correct,
-        # no matter what the sign is (two's complement)
-        return self.digits[0] & 1
-
     def format(self, digits, prefix='', suffix=''):
         # 'digits' is a string whose length is the base to use,
         # and where each character is the corresponding digit.
@@ -228,12 +258,12 @@
 
     def eq(self, other):
         if (self.sign != other.sign or
-            self._numdigits() != other._numdigits()):
+            self.numdigits() != other.numdigits()):
             return False
         i = 0
-        ld = self._numdigits()
+        ld = self.numdigits()
         while i < ld:
-            if self.digits[i] != other.digits[i]:
+            if self.digit(i) != other.digit(i):
                 return False
             i += 1
         return True
@@ -246,8 +276,8 @@
             return False
         if self.sign < other.sign:
             return True
-        ld1 = self._numdigits()
-        ld2 = other._numdigits()
+        ld1 = self.numdigits()
+        ld2 = other.numdigits()
         if ld1 > ld2:
             if other.sign > 0:
                 return False
@@ -260,8 +290,8 @@
                 return False
         i = ld1 - 1
         while i >= 0:
-            d1 = self.digits[i]
-            d2 = other.digits[i]
+            d1 = self.digit(i)
+            d2 = other.digit(i)
             if d1 < d2:
                 if other.sign > 0:
                     return True
@@ -303,7 +333,7 @@
         if other.sign == 0:
             return self
         if self.sign == 0:
-            return rbigint(other.digits[:], -other.sign)
+            return rbigint(other._digits[:], -other.sign)
         if self.sign == other.sign:
             result = _x_sub(self, other)
         else:
@@ -355,7 +385,7 @@
         div, mod = _divrem(v, w)
         if mod.sign * w.sign == -1:
             mod = mod.add(w)
-            div = div.sub(rbigint([1], 1))
+            div = div.sub(rbigint([_store_digit(1)], 1))
         return div, mod
 
     def pow(a, b, c=None):
@@ -382,11 +412,11 @@
             #     modulus = -modulus
             if c.sign < 0:
                 negativeOutput = True
-                c = rbigint(c.digits, -c.sign)
+                c = c.neg()
 
             # if modulus == 1:
             #     return 0
-            if c._numdigits() == 1 and c.digits[0] == 1:
+            if c.numdigits() == 1 and c.digit(0) == 1:
                 return rbigint()
 
             # if base < 0:
@@ -399,16 +429,16 @@
         # At this point a, b, and c are guaranteed non-negative UNLESS
         # c is NULL, in which case a may be negative. */
 
-        z = rbigint([1], 1)
+        z = rbigint([_store_digit(1)], 1)
 
         # python adaptation: moved macros REDUCE(X) and MULT(X, Y, result)
         # into helper function result = _help_mult(x, y, c)
-        if b._numdigits() <= FIVEARY_CUTOFF:
+        if b.numdigits() <= FIVEARY_CUTOFF:
             # Left-to-right binary exponentiation (HAC Algorithm 14.79)
             # http://www.cacr.math.uwaterloo.ca/hac/about/chap14.pdf
-            i = b._numdigits() - 1
+            i = b.numdigits() - 1
             while i >= 0:
-                bi = b.digits[i]
+                bi = b.digit(i)
                 j = 1 << (SHIFT-1)
                 while j != 0:
                     z = _help_mult(z, z, c)
@@ -418,14 +448,15 @@
                 i -= 1
         else:
             # Left-to-right 5-ary exponentiation (HAC Algorithm 14.82)
+            # This is only useful in the case where c != None.
             # z still holds 1L
             table = [z] * 32
-            table[0] = z;
+            table[0] = z
             for i in range(1, 32):
                 table[i] = _help_mult(table[i-1], a, c)
-            i = b._numdigits() - 1
+            i = b.numdigits() - 1
             while i >= 0:
-                bi = b.digits[i]
+                bi = b.digit(i)
                 j = SHIFT - 5
                 while j >= 0:
                     index = (bi >> j) & 0x1f
@@ -441,13 +472,13 @@
         return z
 
     def neg(self):
-        return rbigint(self.digits, -self.sign)
+        return rbigint(self._digits, -self.sign)
 
     def abs(self):
-        return rbigint(self.digits, abs(self.sign))
+        return rbigint(self._digits, abs(self.sign))
 
     def invert(self): #Implement ~x as -(x + 1)
-        return self.add(rbigint([1], 1)).neg()
+        return self.add(rbigint([_store_digit(1)], 1)).neg()
 
     def lshift(self, int_other):
         if int_other < 0:
@@ -459,22 +490,22 @@
         wordshift = int_other // SHIFT
         remshift  = int_other - wordshift * SHIFT
 
-        oldsize = self._numdigits()
+        oldsize = self.numdigits()
         newsize = oldsize + wordshift
         if remshift:
             newsize += 1
-        z = rbigint([0] * newsize, self.sign)
-        accum = 0
+        z = rbigint([NULLDIGIT] * newsize, self.sign)
+        accum = _widen_digit(0)
         i = wordshift
         j = 0
         while j < oldsize:
-            accum |= self._digit(j) << remshift
-            z._setdigit(i, accum)
+            accum |= self.widedigit(j) << remshift
+            z.setdigit(i, accum)
             accum >>= SHIFT
             i += 1
             j += 1
         if remshift:
-            z._setdigit(newsize - 1, accum)
+            z.setdigit(newsize - 1, accum)
         else:
             assert not accum
         z._normalize()
@@ -491,21 +522,22 @@
             return a2.invert()
 
         wordshift = int_other // SHIFT
-        newsize = self._numdigits() - wordshift
+        newsize = self.numdigits() - wordshift
         if newsize <= 0:
             return rbigint()
 
         loshift = int_other % SHIFT
         hishift = SHIFT - loshift
-        lomask = (1 << hishift) - 1
+        lomask = intmask((r_uint(1) << hishift) - 1)
         himask = MASK ^ lomask
-        z = rbigint([0] * newsize, self.sign)
+        z = rbigint([NULLDIGIT] * newsize, self.sign)
         i = 0
         j = wordshift
         while i < newsize:
-            z.digits[i] = (self.digits[j] >> loshift) & lomask
+            newdigit = (self.digit(j) >> loshift) & lomask
             if i+1 < newsize:
-                z.digits[i] |= (self.digits[j+1] << hishift) & himask
+                newdigit |= intmask(self.digit(j+1) << hishift) & himask
+            z.setdigit(i, newdigit)
             i += 1
             j += 1
         z._normalize()
@@ -538,34 +570,35 @@
             ret /= math.log(base)
         return ret
 
-    def tolong(self): #YYYYYY
-        l = 0
-        digits = list(self.digits)
+    def tolong(self):
+        "NOT_RPYTHON"
+        l = 0L
+        digits = list(self._digits)
         digits.reverse()
         for d in digits:
             l = l << SHIFT
-            l += long(d)
+            l += intmask(d)
         return l * self.sign
 
     def _normalize(self):
-        if self._numdigits() == 0:
+        if self.numdigits() == 0:
             self.sign = 0
-            self.digits = [0]
+            self._digits = [NULLDIGIT]
             return
-        i = self._numdigits()
-        while i > 1 and self.digits[i - 1] == 0:
+        i = self.numdigits()
+        while i > 1 and self.digit(i - 1) == 0:
             i -= 1
         assert i >= 1
-        if i != self._numdigits():
-            self.digits = self.digits[:i]
-        if self._numdigits() == 1 and self.digits[0] == 0:
+        if i != self.numdigits():
+            self._digits = self._digits[:i]
+        if self.numdigits() == 1 and self.digit(0) == 0:
             self.sign = 0
 
     def bit_length(self):
-        i = self._numdigits()
-        if i == 1 and self.digits[0] == 0:
+        i = self.numdigits()
+        if i == 1 and self.digit(0) == 0:
             return 0
-        msd = self.digits[i - 1]
+        msd = self.digit(i - 1)
         msd_bits = 0
         while msd >= 32:
             msd_bits += 6
@@ -580,7 +613,8 @@
         return bits
 
     def __repr__(self):
-        return "<rbigint digits=%s, sign=%s, %s>" % (self.digits, self.sign, self.str())
+        return "<rbigint digits=%s, sign=%s, %s>" % (self._digits,
+                                                     self.sign, self.str())
 
 #_________________________________________________________________
 
@@ -605,7 +639,7 @@
 def digits_from_nonneg_long(l):
     digits = []
     while True:
-        digits.append(mask_digit(l))
+        digits.append(_store_digit(intmask(l & MASK)))
         l = l >> SHIFT
         if not l:
             return digits[:] # to make it non-resizable
@@ -615,15 +649,15 @@
     # This helper only works if 'l' is the most negative integer of its
     # type, which in base 2 looks like: 1000000..0000
     digits = []
-    while (mask_digit(l)) == 0:
-        digits.append(0)
+    while _mask_digit(l) == 0:
+        digits.append(NULLDIGIT)
         l = l >> SHIFT
     # now 'l' looks like: ...111100000
     # turn it into:       ...000100000
     # to drop the extra unwanted 1's introduced by the signed right shift
     l = -intmask(l)
-    assert l >= 0
-    digits.append(l)
+    assert l & MASK == l
+    digits.append(_store_digit(l))
     return digits[:] # to make it non-resizable
 digits_for_most_neg_long._annspecialcase_ = "specialize:argtype(0)"
 
@@ -631,7 +665,7 @@
     if x > 0:
         return digits_from_nonneg_long(x), 1
     elif x == 0:
-        return [0], 0
+        return [NULLDIGIT], 0
     elif x != most_neg_value_of_same_type(x):
         # normal case
         return digits_from_nonneg_long(-x), -1
@@ -650,42 +684,42 @@
     "NOT_RPYTHON"
     if x >= 0:
         if x == 0:
-            return [0], 0
+            return [NULLDIGIT], 0
         else:
             return digits_from_nonneg_long(x), 1
     else:
-        return digits_from_nonneg_long(-long(x)), -1
+        return digits_from_nonneg_long(-x), -1
 
 def _x_add(a, b):
     """ Add the absolute values of two bigint integers. """
-    size_a = a._numdigits()
-    size_b = b._numdigits()
+    size_a = a.numdigits()
+    size_b = b.numdigits()
 
     # Ensure a is the larger of the two:
     if size_a < size_b:
         a, b = b, a
         size_a, size_b = size_b, size_a
-    z = rbigint([0] * (a._numdigits() + 1), 1)
+    z = rbigint([NULLDIGIT] * (a.numdigits() + 1), 1)
     i = 0
-    carry = 0
+    carry = r_uint(0)
     while i < size_b:
-        carry += a._digit(i) + b._digit(i)
-        z._setdigit(i, carry)
+        carry += a.udigit(i) + b.udigit(i)
+        z.setdigit(i, carry)
         carry >>= SHIFT
         i += 1
     while i < size_a:
-        carry += a._digit(i)
-        z._setdigit(i, carry)
+        carry += a.udigit(i)
+        z.setdigit(i, carry)
         carry >>= SHIFT
         i += 1
-    z._setdigit(i, carry)
+    z.setdigit(i, carry)
     z._normalize()
     return z
 
 def _x_sub(a, b):
     """ Subtract the absolute values of two integers. """
-    size_a = a._numdigits()
-    size_b = b._numdigits()
+    size_a = a.numdigits()
+    size_b = b.numdigits()
     sign = 1
 
     # Ensure a is the larger of the two:
@@ -696,34 +730,32 @@
     elif size_a == size_b:
         # Find highest digit where a and b differ:
         i = size_a - 1
-        while i >= 0 and a.digits[i] == b.digits[i]:
+        while i >= 0 and a.digit(i) == b.digit(i):
             i -= 1
         if i < 0:
-            return rbigint([0], 0)
-        if a.digits[i] < b.digits[i]:
+            return rbigint()
+        if a.digit(i) < b.digit(i):
             sign = -1
             a, b = b, a
         size_a = size_b = i+1
-    borrow = 0
-    z = rbigint([0] * size_a, 1)
+    z = rbigint([NULLDIGIT] * size_a, sign)
+    borrow = r_uint(0)
     i = 0
     while i < size_b:
         # The following assumes unsigned arithmetic
         # works modulo 2**N for some N>SHIFT.
-        borrow = a._digit(i) - b._digit(i) - borrow
-        z._setdigit(i, borrow)
+        borrow = a.udigit(i) - b.udigit(i) - borrow
+        z.setdigit(i, borrow)
         borrow >>= SHIFT
         borrow &= 1 # Keep only one sign bit
         i += 1
     while i < size_a:
-        borrow = a._digit(i) - borrow
-        z._setdigit(i, borrow)
+        borrow = a.udigit(i) - borrow
+        z.setdigit(i, borrow)
         borrow >>= SHIFT
         borrow &= 1 # Keep only one sign bit
         i += 1
     assert borrow == 0
-    if sign < 0:
-        z.sign = -1
     z._normalize()
     return z
 
@@ -734,9 +766,9 @@
     Returns the absolute value of the product, or None if error.
     """
 
-    size_a = a._numdigits()
-    size_b = b._numdigits()
-    z = rbigint([0] * (size_a + size_b), 1)
+    size_a = a.numdigits()
+    size_b = b.numdigits()
+    z = rbigint([NULLDIGIT] * (size_a + size_b), 1)
     if a is b:
         # Efficient squaring per HAC, Algorithm 14.16:
         # http://www.cacr.math.uwaterloo.ca/hac/about/chap14.pdf
@@ -745,13 +777,13 @@
         # pyramid appears twice (except for the size_a squares).
         i = 0
         while i < size_a:
-            f = a._digit(i)
+            f = a.widedigit(i)
             pz = i << 1
             pa = i + 1
             paend = size_a
 
-            carry = z._digit(pz) + f * f
-            z._setdigit(pz, carry)
+            carry = z.widedigit(pz) + f * f
+            z.setdigit(pz, carry)
             pz += 1
             carry >>= SHIFT
             assert carry <= MASK
@@ -760,19 +792,19 @@
             # pyramid it appears.  Same as adding f<<1 once.
             f <<= 1
             while pa < paend:
-                carry += z._digit(pz) + a._digit(pa) * f
+                carry += z.widedigit(pz) + a.widedigit(pa) * f
                 pa += 1
-                z._setdigit(pz, carry)
+                z.setdigit(pz, carry)
                 pz += 1
                 carry >>= SHIFT
-                assert carry <= (widen_digit(MASK) << 1)
+                assert carry <= (_widen_digit(MASK) << 1)
             if carry:
-                carry += z._digit(pz)
-                z._setdigit(pz, carry)
+                carry += z.widedigit(pz)
+                z.setdigit(pz, carry)
                 pz += 1
                 carry >>= SHIFT
             if carry:
-                z._setdigit(pz, z._digit(pz) + carry)
+                z.setdigit(pz, z.widedigit(pz) + carry)
             assert (carry >> SHIFT) == 0
             i += 1
     else:
@@ -780,19 +812,19 @@
         i = 0
         while i < size_a:
             carry = 0
-            f = a._digit(i)
+            f = a.widedigit(i)
             pz = i
             pb = 0
             pbend = size_b
             while pb < pbend:
-                carry += z._digit(pz) + b._digit(pb) * f
+                carry += z.widedigit(pz) + b.widedigit(pb) * f
                 pb += 1
-                z._setdigit(pz, carry)
+                z.setdigit(pz, carry)
                 pz += 1
                 carry >>= SHIFT
                 assert carry <= MASK
             if carry:
-                z._setdigit(pz, z._digit(pz) + carry)
+                z.setdigit(pz, z.widedigit(pz) + carry)
             assert (carry >> SHIFT) == 0
             i += 1
     z._normalize()
@@ -807,11 +839,11 @@
     viewing the shift as being by digits.  The sign bit is ignored, and
     the return values are >= 0.
     """
-    size_n = n._numdigits()
+    size_n = n.numdigits()
     size_lo = min(size_n, size)
 
-    lo = rbigint(n.digits[:size_lo], 1)
-    hi = rbigint(n.digits[size_lo:], 1)
+    lo = rbigint(n._digits[:size_lo], 1)
+    hi = rbigint(n._digits[size_lo:], 1)
     lo._normalize()
     hi._normalize()
     return hi, lo
@@ -822,8 +854,8 @@
     absolute value of the product (or raises if error).
     See Knuth Vol. 2 Chapter 4.3.3 (Pp. 294-295).
     """
-    asize = a._numdigits()
-    bsize = b._numdigits()
+    asize = a.numdigits()
+    bsize = b.numdigits()
     # (ah*X+al)(bh*X+bl) = ah*bh*X*X + (ah*bl + al*bh)*X + al*bl
     # Let k = (ah+al)*(bh+bl) = ah*bl + al*bh  + ah*bh + al*bl
     # Then the original product is
@@ -843,7 +875,7 @@
         i = KARATSUBA_CUTOFF
     if asize <= i:
         if a.sign == 0:
-            return rbigint([0], 0)
+            return rbigint()     # zero
         else:
             return _x_mul(a, b)
 
@@ -882,13 +914,13 @@
     #    at shift.
 
     # 1. Allocate result space.
-    ret = rbigint([0] * (asize + bsize), 1)
+    ret = rbigint([NULLDIGIT] * (asize + bsize), 1)
 
     # 2. t1 <- ah*bh, and copy into high digits of result.
     t1 = _k_mul(ah, bh)
     assert t1.sign >= 0
-    assert 2*shift + t1._numdigits() <= ret._numdigits()
-    ret.digits[2*shift : 2*shift + t1._numdigits()] = t1.digits
+    assert 2*shift + t1.numdigits() <= ret.numdigits()
+    ret._digits[2*shift : 2*shift + t1.numdigits()] = t1._digits
 
     # Zero-out the digits higher than the ah*bh copy. */
     ## ignored, assuming that we initialize to zero
@@ -900,8 +932,8 @@
     # 3. t2 <- al*bl, and copy into the low digits.
     t2 = _k_mul(al, bl)
     assert t2.sign >= 0
-    assert t2._numdigits() <= 2*shift # no overlap with high digits
-    ret.digits[:t2._numdigits()] = t2.digits
+    assert t2.numdigits() <= 2*shift # no overlap with high digits
+    ret._digits[:t2.numdigits()] = t2._digits
 
     # Zero out remaining digits.
     ## ignored, assuming that we initialize to zero
@@ -911,9 +943,9 @@
 
     # 4 & 5. Subtract ah*bh (t1) and al*bl (t2).  We do al*bl first
     # because it's fresher in cache.
-    i = ret._numdigits() - shift  # # digits after shift
-    _v_isub(ret, shift, i, t2, t2._numdigits())
-    _v_isub(ret, shift, i, t1, t1._numdigits())
+    i = ret.numdigits() - shift  # # digits after shift
+    _v_isub(ret, shift, i, t2, t2.numdigits())
+    _v_isub(ret, shift, i, t1, t1.numdigits())
     del t1, t2
 
     # 6. t3 <- (ah+al)(bh+bl), and add into result.
@@ -932,7 +964,7 @@
 
     # Add t3.  It's not obvious why we can't run out of room here.
     # See the (*) comment after this function.
-    _v_iadd(ret, shift, i, t3, t3._numdigits())
+    _v_iadd(ret, shift, i, t3, t3.numdigits())
     del t3
 
     ret._normalize()
@@ -993,20 +1025,20 @@
     at a time, then move on, never bactracking except for the helpful
     single-width slice overlap between successive partial sums).
     """
-    asize = a._numdigits()
-    bsize = b._numdigits()
+    asize = a.numdigits()
+    bsize = b.numdigits()
     # nbdone is # of b digits already multiplied
 
     assert asize > KARATSUBA_CUTOFF
     assert 2 * asize <= bsize
 
     # Allocate result space, and zero it out.
-    ret = rbigint([0] * (asize + bsize), 1)
+    ret = rbigint([NULLDIGIT] * (asize + bsize), 1)
 
     # Successive slices of b are copied into bslice.
     #bslice = rbigint([0] * asize, 1)
     # XXX we cannot pre-allocate, see comments below!
-    bslice = rbigint([0], 1)
+    bslice = rbigint([NULLDIGIT], 1)
 
     nbdone = 0;
     while bsize > 0:
@@ -1018,12 +1050,12 @@
         # XXX: this would be more efficient if we adopted CPython's
         # way to store the size, instead of resizing the list!
         # XXX change the implementation, encoding length via the sign.
-        bslice.digits = b.digits[nbdone : nbdone + nbtouse]
+        bslice._digits = b._digits[nbdone : nbdone + nbtouse]
         product = _k_mul(a, bslice)
 
         # Add into result.
-        _v_iadd(ret, nbdone, ret._numdigits() - nbdone,
-                 product, product._numdigits())
+        _v_iadd(ret, nbdone, ret.numdigits() - nbdone,
+                 product, product.numdigits())
         del product
 
         bsize -= nbtouse
@@ -1038,18 +1070,18 @@
     Divide bigint pin by non-zero digit n, storing quotient
     in pout, and returning the remainder. It's OK for pin == pout on entry.
     """
-    rem = widen_digit(0)
+    rem = _widen_digit(0)
     assert n > 0 and n <= MASK
     if not size:
-        size = pin._numdigits()
+        size = pin.numdigits()
     size -= 1
     while size >= 0:
-        rem = (rem << SHIFT) + pin._digit(size)
+        rem = (rem << SHIFT) + pin.widedigit(size)
         hi = rem // n
-        pout._setdigit(size, hi)
+        pout.setdigit(size, hi)
         rem -= hi * n
         size -= 1
-    return mask_digit(rem)
+    return _mask_digit(rem)
 
 def _divrem1(a, n):
     """
@@ -1058,8 +1090,8 @@
     The sign of a is ignored; n should not be zero.
     """
     assert n > 0 and n <= MASK
-    size = a._numdigits()
-    z = rbigint([0] * size, 1)
+    size = a.numdigits()
+    z = rbigint([NULLDIGIT] * size, 1)
     rem = _inplace_divrem1(z, a, n)
     z._normalize()
     return z, rem
@@ -1071,21 +1103,21 @@
     x[m-1], and the remaining carry (0 or 1) is returned.
     Python adaptation: x is addressed relative to xofs!
     """
-    carry = 0;
+    carry = r_uint(0)
 
     assert m >= n
     i = xofs
     iend = xofs + n
     while i < iend:
-        carry += x._digit(i) + y._digit(i-xofs)
-        x._setdigit(i, carry)
+        carry += x.udigit(i) + y.udigit(i-xofs)
+        x.setdigit(i, carry)
         carry >>= SHIFT
         assert (carry & 1) == carry
         i += 1
     iend = xofs + m
     while carry and i < iend:
-        carry += x._digit(i)
-        x._setdigit(i, carry)
+        carry += x.udigit(i)
+        x.setdigit(i, carry)
         carry >>= SHIFT
         assert (carry & 1) == carry
         i += 1
@@ -1098,57 +1130,59 @@
     far as x[m-1], and the remaining borrow (0 or 1) is returned.
     Python adaptation: x is addressed relative to xofs!
     """
-    borrow = 0
+    borrow = r_uint(0)
 
     assert m >= n
     i = xofs
     iend = xofs + n
     while i < iend:
-        borrow = x._digit(i) - y._digit(i-xofs) - borrow
-        x._setdigit(i, borrow)
+        borrow = x.udigit(i) - y.udigit(i-xofs) - borrow
+        x.setdigit(i, borrow)
         borrow >>= SHIFT
         borrow &= 1    # keep only 1 sign bit
         i += 1
     iend = xofs + m
     while borrow and i < iend:
-        borrow = x._digit(i) - borrow
-        x._setdigit(i, borrow)
+        borrow = x.udigit(i) - borrow
+        x.setdigit(i, borrow)
         borrow >>= SHIFT
         borrow &= 1
         i += 1
     return borrow
 
 
-def _muladd1(a, n, extra=widen_digit(0)):
+def _muladd1(a, n, extra=0):
     """Multiply by a single digit and add a single digit, ignoring the sign.
     """
-    size_a = a._numdigits()
-    z = rbigint([0] * (size_a+1), 1)
-    carry = extra
-    assert carry & MASK == carry
+    size_a = a.numdigits()
+    z = rbigint([NULLDIGIT] * (size_a+1), 1)
+    assert extra & MASK == extra
+    carry = _widen_digit(extra)
     i = 0
     while i < size_a:
-        carry += a._digit(i) * n
-        z._setdigit(i, carry)
+        carry += a.widedigit(i) * n
+        z.setdigit(i, carry)
         carry >>= SHIFT
         i += 1
-    z._setdigit(i, carry)
+    z.setdigit(i, carry)
     z._normalize()
     return z
 
 
 def _x_divrem(v1, w1):
     """ Unsigned bigint division with remainder -- the algorithm """
-    size_w = w1._numdigits()
-    d = (widen_digit(MASK)+1) // (w1._digit(size_w-1) + 1)
+    size_w = w1.numdigits()
+    d = (r_uint(MASK)+1) // (w1.udigit(size_w-1) + 1)
+    assert d <= MASK    # because the first digit of w1 is not zero
+    d = intmask(d)
     v = _muladd1(v1, d)
     w = _muladd1(w1, d)
-    size_v = v._numdigits()
-    size_w = w._numdigits()
+    size_v = v.numdigits()
+    size_w = w.numdigits()
     assert size_v >= size_w and size_w > 1 # Assert checks by div()
 
     size_a = size_v - size_w + 1
-    a = rbigint([0] * size_a, 1)
+    a = rbigint([NULLDIGIT] * size_a, 1)
 
     j = size_v
     k = size_a - 1
@@ -1156,50 +1190,50 @@
         if j >= size_v:
             vj = 0
         else:
-            vj = v._digit(j)
+            vj = v.widedigit(j)
         carry = 0
 
-        if vj == w._digit(size_w-1):
+        if vj == w.widedigit(size_w-1):
             q = MASK
         else:
-            q = ((vj << SHIFT) + v._digit(j-1)) // w._digit(size_w-1)
+            q = ((vj << SHIFT) + v.widedigit(j-1)) // w.widedigit(size_w-1)
 
-        while (w._digit(size_w-2) * q >
+        while (w.widedigit(size_w-2) * q >
                 ((
                     (vj << SHIFT)
-                    + v._digit(j-1)
-                    - q * w._digit(size_w-1)
+                    + v.widedigit(j-1)
+                    - q * w.widedigit(size_w-1)
                                 ) << SHIFT)
-                + v._digit(j-2)):
+                + v.widedigit(j-2)):
             q -= 1
         i = 0
         while i < size_w and i+k < size_v:
-            z = w._digit(i) * q
+            z = w.widedigit(i) * q
             zz = z >> SHIFT
-            carry += v._digit(i+k) - z + (zz << SHIFT)
-            v._setdigit(i+k, carry)
+            carry += v.widedigit(i+k) - z + (zz << SHIFT)
+            v.setdigit(i+k, carry)
             carry >>= SHIFT
             carry -= zz
             i += 1
 
         if i+k < size_v:
-            carry += v._digit(i+k)
-            v._setdigit(i+k, 0)
+            carry += v.widedigit(i+k)
+            v.setdigit(i+k, 0)
 
         if carry == 0:
-            a._setdigit(k, q)
+            a.setdigit(k, q)
             assert not q >> SHIFT
         else:
             assert carry == -1
             q -= 1
-            a._setdigit(k, q)
+            a.setdigit(k, q)
             assert not q >> SHIFT
 
             carry = 0
             i = 0
             while i < size_w and i+k < size_v:
-                carry += v._digit(i+k) + w._digit(i)
-                v._setdigit(i+k, carry)
+                carry += v.udigit(i+k) + w.udigit(i)
+                v.setdigit(i+k, carry)
                 carry >>= SHIFT
                 i += 1
         j -= 1
@@ -1212,22 +1246,22 @@
 
 def _divrem(a, b):
     """ Long division with remainder, top-level routine """
-    size_a = a._numdigits()
-    size_b = b._numdigits()
+    size_a = a.numdigits()
+    size_b = b.numdigits()
 
     if b.sign == 0:
         raise ZeroDivisionError("long division or modulo by zero")
 
     if (size_a < size_b or
         (size_a == size_b and
-         a.digits[size_a-1] < b.digits[size_b-1])):
+         a.digit(size_a-1) < b.digit(size_b-1))):
         # |a| < |b|
-        z = rbigint([0], 0)
+        z = rbigint()   # result is 0
         rem = a
         return z, rem
     if size_b == 1:
-        z, urem = _divrem1(a, b._digit(0))
-        rem = rbigint([urem], int(urem != 0))
+        z, urem = _divrem1(a, b.digit(0))
+        rem = rbigint([_store_digit(urem)], int(urem != 0))
     else:
         z, rem = _x_divrem(a, b)
     # Set the signs.
@@ -1257,14 +1291,14 @@
     NBITS_WANTED = 57
     if v.sign == 0:
         return 0.0, 0
-    i = v._numdigits() - 1
+    i = v.numdigits() - 1
     sign = v.sign
-    x = float(v.digits[i])
+    x = float(v.digit(i))
     nbitsneeded = NBITS_WANTED - 1
     # Invariant:  i Python digits remain unaccounted for.
     while i > 0 and nbitsneeded > 0:
         i -= 1
-        x = x * FLOAT_MULTIPLIER + float(v.digits[i])
+        x = x * FLOAT_MULTIPLIER + float(v.digit(i))
         nbitsneeded -= SHIFT
     # There are i digits we didn't shift in.  Pretending they're all
     # zeroes, the true value is x * 2**(i*SHIFT).
@@ -1515,7 +1549,7 @@
     Convert a bigint object to a string, using a given conversion base.
     Return a string object.
     """
-    size_a = a._numdigits()
+    size_a = a.numdigits()
 
     base = len(digits)
     assert base >= 2 and base <= 36
@@ -1551,11 +1585,11 @@
             basebits += 1
 
         for i in range(size_a):
-            accum |= a._digit(i) << accumbits
+            accum |= a.widedigit(i) << accumbits
             accumbits += SHIFT
             assert accumbits >= basebits
             while 1:
-                cdigit = accum & (base - 1)
+                cdigit = intmask(accum & (base - 1))
                 p -= 1
                 assert p >= 0
                 s[p] = digits[cdigit]
@@ -1574,7 +1608,7 @@
         size = size_a
         pin = a # just for similarity to C source which uses the array
         # powbase <- largest power of base that fits in a digit.
-        powbase = widen_digit(base)  # powbase == base ** power
+        powbase = _widen_digit(base)  # powbase == base ** power
         power = 1
         while 1:
             newpow = powbase * base
@@ -1584,14 +1618,14 @@
             power += 1
 
         # Get a scratch area for repeated division.
-        scratch = rbigint([0] * size, 1)
+        scratch = rbigint([NULLDIGIT] * size, 1)
 
         # Repeatedly divide by powbase.
         while 1:
             ntostore = power
             rem = _inplace_divrem1(scratch, pin, powbase, size)
             pin = scratch  # no need to use a again
-            if pin.digits[size - 1] == 0:
+            if pin.digit(size - 1) == 0:
                 size -= 1
 
             # Break rem into digits.
@@ -1668,8 +1702,8 @@
     # iff one of these cases applies, and mask will be non-0 for operands
     # whose length should be ignored.
 
-    size_a = a._numdigits()
-    size_b = b._numdigits()
+    size_a = a.numdigits()
+    size_b = b.numdigits()
     if op == '&':
         if maska:
             size_z = size_b
@@ -1681,23 +1715,23 @@
     else:
         size_z = max(size_a, size_b)
 
-    z = rbigint([0] * size_z, 1)
+    z = rbigint([NULLDIGIT] * size_z, 1)
 
     for i in range(size_z):
         if i < size_a:
-            diga = a.digits[i] ^ maska
+            diga = a.digit(i) ^ maska
         else:
             diga = maska
         if i < size_b:
-            digb = b.digits[i] ^ maskb
+            digb = b.digit(i) ^ maskb
         else:
             digb = maskb
         if op == '&':
-            z.digits[i] = diga & digb
+            z.setdigit(i, diga & digb)
         elif op == '|':
-            z.digits[i] = diga | digb
+            z.setdigit(i, diga | digb)
         elif op == '^':
-            z.digits[i] = diga ^ digb
+            z.setdigit(i, diga ^ digb)
 
     z._normalize()
     if negz == 0:
@@ -1729,10 +1763,10 @@
 
 def _AsULonglong_ignore_sign(v):
     x = r_ulonglong(0)
-    i = v._numdigits() - 1
+    i = v.numdigits() - 1
     while i >= 0:
         prev = x
-        x = (x << SHIFT) + v._digit(i)
+        x = (x << SHIFT) + v.widedigit(i)
         if (x >> SHIFT) != prev:
                 raise OverflowError(
                     "long int too large to convert to unsigned long long int")
@@ -1742,10 +1776,9 @@
 def make_unsigned_mask_conversion(T):
     def _As_unsigned_mask(v):
         x = T(0)
-        i = v._numdigits() - 1
+        i = v.numdigits() - 1
         while i >= 0:
-            prev = x
-            x = (x << SHIFT) + T(v._digit(i))
+            x = (x << SHIFT) + T(v.digit(i))
             i -= 1
         if v.sign < 0:
             x = -x
@@ -1762,18 +1795,18 @@
     # to please decimal.py, we return a hash that satisfies
     # hash(x) == hash(x % ULONG_MAX).  In particular, this
     # implies that hash(x) == hash(x % (2**64-1)).
-    i = v._numdigits() - 1
+    i = v.numdigits() - 1
     sign = v.sign
     x = r_uint(0)
     LONG_BIT_SHIFT = LONG_BIT - SHIFT
     while i >= 0:
         # Force a native long #-bits (32 or 64) circular shift
         x = (x << SHIFT) | (x >> LONG_BIT_SHIFT)
-        x += r_uint(v.digits[i])
+        x += v.udigit(i)
         # If the addition above overflowed we compensate by
         # incrementing.  This preserves the value modulo
         # ULONG_MAX.
-        if x < r_uint(v.digits[i]):
+        if x < v.udigit(i):
             x += 1
         i -= 1
     x = intmask(x * sign)

From commits-noreply at bitbucket.org  Sun Feb  6 15:30:32 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sun,  6 Feb 2011 15:30:32 +0100 (CET)
Subject: [pypy-svn] pypy default: Test and fix: don't try to load the
	attribute name ''.
Message-ID: <20110206143032.2E94E36C223@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41655:f9c9d5bbdddb
Date: 2011-02-06 15:29 +0100
http://bitbucket.org/pypy/pypy/changeset/f9c9d5bbdddb/

Log:	Test and fix: don't try to load the attribute name ''.

diff --git a/pypy/objspace/std/newformat.py b/pypy/objspace/std/newformat.py
--- a/pypy/objspace/std/newformat.py
+++ b/pypy/objspace/std/newformat.py
@@ -203,6 +203,9 @@
                     if c == "[" or c == ".":
                         break
                     i += 1
+                if start == i:
+                    w_msg = space.wrap("Empty attribute in format string")
+                    raise OperationError(space.w_ValueError, w_msg)
                 w_obj = space.getattr(w_obj, space.wrap(name[start:i]))
             elif c == "[":
                 got_bracket = False

diff --git a/pypy/objspace/std/test/test_newformat.py b/pypy/objspace/std/test/test_newformat.py
--- a/pypy/objspace/std/test/test_newformat.py
+++ b/pypy/objspace/std/test/test_newformat.py
@@ -44,6 +44,8 @@
         class x:
             apple = 42
         assert self.s("{.apple}").format(x) == self.s("42")
+        #
+        raises(ValueError, self.s("{.}").format, x)
 
     def test_index(self):
         seq = (1, 42)

From commits-noreply at bitbucket.org  Sun Feb  6 15:57:42 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sun,  6 Feb 2011 15:57:42 +0100 (CET)
Subject: [pypy-svn] pypy default: This is an issue of dictionary ordering.
Message-ID: <20110206145742.74052282B9C@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41656:edba34cc68ff
Date: 2011-02-06 15:57 +0100
http://bitbucket.org/pypy/pypy/changeset/edba34cc68ff/

Log:	This is an issue of dictionary ordering.

diff --git a/lib-python/2.7.0/test/test_pprint.py b/lib-python/modified-2.7.0/test/test_pprint.py
copy from lib-python/2.7.0/test/test_pprint.py
copy to lib-python/modified-2.7.0/test/test_pprint.py
--- a/lib-python/2.7.0/test/test_pprint.py
+++ b/lib-python/modified-2.7.0/test/test_pprint.py
@@ -233,7 +233,16 @@
                                   frozenset([0, 2]),
                                   frozenset([0, 1])])}"""
         cube = test.test_set.cube(3)
-        self.assertEqual(pprint.pformat(cube), cube_repr_tgt)
+        # XXX issues of dictionary order, and for the case below,
+        # order of items in the frozenset([...]) representation.
+        # Whether we get precisely cube_repr_tgt or not is open
+        # to implementation-dependent choices (this test probably
+        # fails horribly in CPython if we tweak the dict order too).
+        got = pprint.pformat(cube)
+        if test.test_support.impl_detail(cpython=True):
+            self.assertEqual(got, cube_repr_tgt)
+        else:
+            self.assertEqual(eval(got), cube)
         cubo_repr_tgt = """\
 {frozenset([frozenset([0, 2]), frozenset([0])]): frozenset([frozenset([frozenset([0,
                                                                                   2]),
@@ -393,7 +402,11 @@
                                                                                         2])])])}"""
 
         cubo = test.test_set.linegraph(cube)
-        self.assertEqual(pprint.pformat(cubo), cubo_repr_tgt)
+        got = pprint.pformat(cubo)
+        if test.test_support.impl_detail(cpython=True):
+            self.assertEqual(got, cubo_repr_tgt)
+        else:
+            self.assertEqual(eval(got), cubo)
 
     def test_depth(self):
         nested_tuple = (1, (2, (3, (4, (5, 6)))))

From commits-noreply at bitbucket.org  Sun Feb  6 17:18:07 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Sun,  6 Feb 2011 17:18:07 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: better nonnull test
Message-ID: <20110206161807.C2154282B9C@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r41657:31774cca9d1f
Date: 2011-02-06 15:06 +0100
http://bitbucket.org/pypy/pypy/changeset/31774cca9d1f/

Log:	better nonnull test

diff --git a/pypy/jit/metainterp/optimizeopt/unroll.py b/pypy/jit/metainterp/optimizeopt/unroll.py
--- a/pypy/jit/metainterp/optimizeopt/unroll.py
+++ b/pypy/jit/metainterp/optimizeopt/unroll.py
@@ -212,7 +212,7 @@
         if not isinstance(other, NotVirtualInfo):
             raise InvalidLoop
         if self.level == LEVEL_KNOWNCLASS and \
-           box.value and \
+           box.nonnull() and \
            self.known_class.same_constant(cpu.ts.cls_of_box(box)):
             # Note: This is only a hint on what the class of box was
             # during the trace. There are actually no guarentees that this

diff --git a/pypy/jit/metainterp/test/test_optimizebasic.py b/pypy/jit/metainterp/test/test_optimizebasic.py
--- a/pypy/jit/metainterp/test/test_optimizebasic.py
+++ b/pypy/jit/metainterp/test/test_optimizebasic.py
@@ -3352,6 +3352,8 @@
         [i0]
         i1 = int_lt(i0, 4)
         guard_true(i1) []
+        i1p = int_gt(i0, -4)
+        guard_true(i1p) []        
         i2 = int_sub(i0, 10)
         i3 = int_lt(i2, -5)
         guard_true(i3) []
@@ -3361,6 +3363,8 @@
         [i0]
         i1 = int_lt(i0, 4)
         guard_true(i1) []
+        i1p = int_gt(i0, -4)
+        guard_true(i1p) []        
         i2 = int_sub(i0, 10)
         jump(i0)
         """

From commits-noreply at bitbucket.org  Sun Feb  6 17:18:08 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Sun,  6 Feb 2011 17:18:08 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: more senseble default
Message-ID: <20110206161808.5BC97282B9C@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r41658:ef472960ecc1
Date: 2011-02-06 15:14 +0100
http://bitbucket.org/pypy/pypy/changeset/ef472960ecc1/

Log:	more senseble default

diff --git a/pypy/jit/metainterp/resume.py b/pypy/jit/metainterp/resume.py
--- a/pypy/jit/metainterp/resume.py
+++ b/pypy/jit/metainterp/resume.py
@@ -444,7 +444,7 @@
         self._generate_guards(other, box, cpu, extra_guards)
 
     def _generate_guards(self, other, box, cpu, extra_guards):
-        raise NotImplementedError
+        raise InvalidLoop
         
 class AbstractVirtualStructInfo(AbstractVirtualInfo):
     def __init__(self, fielddescrs):
@@ -482,10 +482,6 @@
     def _generalization_of(self, other):
         raise NotImplementedError
 
-    def _generate_guards(self, other, box, cpu, extra_guards):
-        raise InvalidLoop
-            
-        
 
 class VirtualInfo(AbstractVirtualStructInfo):
     def __init__(self, known_class, fielddescrs):

From commits-noreply at bitbucket.org  Sun Feb  6 17:18:12 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Sun,  6 Feb 2011 17:18:12 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: only produce preamble for the
	first loop and	then reuse it for the rest
Message-ID: <20110206161812.1A6A6282BDB@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r41659:cd09aab2ef91
Date: 2011-02-06 16:42 +0100
http://bitbucket.org/pypy/pypy/changeset/cd09aab2ef91/

Log:	only produce preamble for the first loop and then reuse it for the
	rest

diff --git a/pypy/jit/metainterp/optimizeopt/unroll.py b/pypy/jit/metainterp/optimizeopt/unroll.py
--- a/pypy/jit/metainterp/optimizeopt/unroll.py
+++ b/pypy/jit/metainterp/optimizeopt/unroll.py
@@ -640,6 +640,7 @@
                 modifier = VirtualStateAdder(self.optimizer)
                 virtual_state = modifier.get_virtual_state(args)
                 for sh in short:
+                    #import pdb; pdb.set_trace()
                     ok = False
                     extra_guards = []
                     if sh.virtual_state.generalization_of(virtual_state):

diff --git a/pypy/jit/metainterp/compile.py b/pypy/jit/metainterp/compile.py
--- a/pypy/jit/metainterp/compile.py
+++ b/pypy/jit/metainterp/compile.py
@@ -80,11 +80,11 @@
 
 # ____________________________________________________________
 
-def compile_new_loop(metainterp, old_loop_tokens, greenkey, start, start_resumedescr):
+def compile_new_loop(metainterp, old_loop_tokens, greenkey, start,
+                     start_resumedescr, full_preamble_needed=True):
     """Try to compile a new loop by closing the current history back
     to the first operation.
     """
-    full_preamble_needed=True
     history = metainterp.history
     loop = create_empty_loop(metainterp)
     loop.inputargs = history.inputargs

diff --git a/pypy/jit/metainterp/pyjitpl.py b/pypy/jit/metainterp/pyjitpl.py
--- a/pypy/jit/metainterp/pyjitpl.py
+++ b/pypy/jit/metainterp/pyjitpl.py
@@ -1914,7 +1914,7 @@
         greenkey = original_boxes[:num_green_args]
         self.history.record(rop.JUMP, live_arg_boxes[num_green_args:], None)
         loop_token = compile.compile_new_loop(self, [], greenkey, start,
-                                              start_resumedescr)
+                                              start_resumedescr, False)
         self.history.operations.pop()     # remove the JUMP
         if loop_token is None:
             return
@@ -1928,7 +1928,8 @@
         self.history.record(rop.JUMP, bridge_arg_boxes[num_green_args:], None)
         try:
             target_loop_token = compile.compile_new_bridge(self,
-                                                           [loop_token],
+                                                           #[loop_token],
+                                                           old_loop_tokens,
                                                            self.resumekey,
                                                            True)
         except RetraceLoop:

diff --git a/pypy/jit/metainterp/resume.py b/pypy/jit/metainterp/resume.py
--- a/pypy/jit/metainterp/resume.py
+++ b/pypy/jit/metainterp/resume.py
@@ -466,6 +466,8 @@
                         str(untag(self.fieldnums[i])))
 
     def generalization_of(self, other):
+        if not self._generalization_of(other):
+            return False
         assert len(self.fielddescrs) == len(self.fieldstate)
         assert len(other.fielddescrs) == len(other.fieldstate)
         if len(self.fielddescrs) != len(other.fielddescrs):
@@ -477,7 +479,7 @@
             if not self.fieldstate[i].generalization_of(other.fieldstate[i]):
                 return False
 
-        return self._generalization_of(other)
+        return True
 
     def _generalization_of(self, other):
         raise NotImplementedError

diff --git a/pypy/jit/metainterp/test/test_basic.py b/pypy/jit/metainterp/test/test_basic.py
--- a/pypy/jit/metainterp/test/test_basic.py
+++ b/pypy/jit/metainterp/test/test_basic.py
@@ -1499,7 +1499,7 @@
         res = self.meta_interp(f, [299], listops=True)
         assert res == f(299)
         self.check_loops(guard_class=0, guard_value=3)        
-        self.check_loops(guard_class=0, guard_value=8, everywhere=True)
+        self.check_loops(guard_class=0, guard_value=6, everywhere=True)
 
     def test_merge_guardnonnull_guardclass(self):
         from pypy.rlib.objectmodel import instantiate
@@ -1530,7 +1530,7 @@
         self.check_loops(guard_class=0, guard_nonnull=0,
                          guard_nonnull_class=2, guard_isnull=1)
         self.check_loops(guard_class=0, guard_nonnull=0,
-                         guard_nonnull_class=5, guard_isnull=3,
+                         guard_nonnull_class=4, guard_isnull=2,
                          everywhere=True)
 
     def test_merge_guardnonnull_guardvalue(self):
@@ -1560,8 +1560,8 @@
         assert res == f(299)
         self.check_loops(guard_class=0, guard_nonnull=0, guard_value=2,
                          guard_nonnull_class=0, guard_isnull=1)
-        self.check_loops(guard_class=0, guard_nonnull=0, guard_value=5,
-                         guard_nonnull_class=0, guard_isnull=3,
+        self.check_loops(guard_class=0, guard_nonnull=0, guard_value=4,
+                         guard_nonnull_class=0, guard_isnull=2,
                          everywhere=True)
 
     def test_merge_guardnonnull_guardvalue_2(self):
@@ -1591,8 +1591,8 @@
         assert res == f(299)
         self.check_loops(guard_class=0, guard_nonnull=0, guard_value=2,
                          guard_nonnull_class=0, guard_isnull=1)
-        self.check_loops(guard_class=0, guard_nonnull=0, guard_value=5,
-                         guard_nonnull_class=0, guard_isnull=3,
+        self.check_loops(guard_class=0, guard_nonnull=0, guard_value=4,
+                         guard_nonnull_class=0, guard_isnull=2,
                          everywhere=True)
 
     def test_merge_guardnonnull_guardclass_guardvalue(self):
@@ -1625,8 +1625,8 @@
         assert res == f(399)
         self.check_loops(guard_class=0, guard_nonnull=0, guard_value=3,
                          guard_nonnull_class=0, guard_isnull=1)
-        self.check_loops(guard_class=0, guard_nonnull=0, guard_value=8,
-                         guard_nonnull_class=0, guard_isnull=3,
+        self.check_loops(guard_class=0, guard_nonnull=0, guard_value=6,
+                         guard_nonnull_class=0, guard_isnull=2,
                          everywhere=True)
 
     def test_residual_call_doesnt_lose_info(self):

diff --git a/pypy/jit/metainterp/test/test_virtual.py b/pypy/jit/metainterp/test/test_virtual.py
--- a/pypy/jit/metainterp/test/test_virtual.py
+++ b/pypy/jit/metainterp/test/test_virtual.py
@@ -541,6 +541,28 @@
             return sa
         assert self.meta_interp(f, [20]) == f(20)
 
+    def test_dual_counter(self):
+        myjitdriver = JitDriver(greens = [], reds = ['n', 's', 'node1', 'node2'])
+        def f(n, s):
+            node1 = self._new()
+            node1.value = 1
+            node2 = self._new()
+            node2.value = 2
+            while n > 0:
+                myjitdriver.can_enter_jit(n=n, s=s, node1=node1, node2=node2)
+                myjitdriver.jit_merge_point(n=n, s=s, node1=node1, node2=node2)
+                if (n>>s) & 1:
+                    next = self._new()
+                    next.value = node1.value + 1
+                    node1 = next
+                else:
+                    next = self._new()
+                    next.value = node2.value + 1
+                    node2 = next
+                n -= 1
+            return node1.value + node2.value
+        assert self.meta_interp(f, [300, 3]) == f(300, 3)
+        
 
 class VirtualMiscTests:
 

From commits-noreply at bitbucket.org  Sun Feb  6 17:18:13 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Sun,  6 Feb 2011 17:18:13 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: test
Message-ID: <20110206161813.3ECC0282BDB@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r41660:6f98022ad31c
Date: 2011-02-06 17:17 +0100
http://bitbucket.org/pypy/pypy/changeset/6f98022ad31c/

Log:	test

diff --git a/pypy/jit/metainterp/test/test_virtual.py b/pypy/jit/metainterp/test/test_virtual.py
--- a/pypy/jit/metainterp/test/test_virtual.py
+++ b/pypy/jit/metainterp/test/test_virtual.py
@@ -561,7 +561,11 @@
                     node2 = next
                 n -= 1
             return node1.value + node2.value
-        assert self.meta_interp(f, [300, 3]) == f(300, 3)
+        assert self.meta_interp(f, [40, 3]) == f(40, 3)
+        
+        def g(n1, n2, s):
+            return f(n1, s) + f(n2, s)
+        # FIXME: Try the case where we need to call the second version from the interpreter
         
 
 class VirtualMiscTests:

From commits-noreply at bitbucket.org  Sun Feb  6 19:04:11 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sun,  6 Feb 2011 19:04:11 +0100 (CET)
Subject: [pypy-svn] pypy default: issue631 resolved (at least on posix) (I
	think).
Message-ID: <20110206180411.4803B36C231@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41661:2ea70f35cc0e
Date: 2011-02-06 19:02 +0100
http://bitbucket.org/pypy/pypy/changeset/2ea70f35cc0e/

Log:	issue631 resolved (at least on posix) (I think).

diff --git a/pypy/translator/goal/test2/test_nanos.py b/pypy/translator/goal/test2/test_nanos.py
--- a/pypy/translator/goal/test2/test_nanos.py
+++ b/pypy/translator/goal/test2/test_nanos.py
@@ -48,6 +48,14 @@
         assert p2.abspath(base) == base
         assert p2.abspath('x') == os.path.join(os.getcwd(), 'x')
 
+    def test_abspath_uses_normpath(self):
+        p1 = os.path
+        p2 = self.getnanos().path
+        base = str(udir)
+        assert p2.abspath(p1.join(base, '.')) == base
+        assert p2.abspath(p1.join(base, '.', '.', '.')) == base
+        assert p2.abspath(p1.join(base, 'foo', '..')) == base
+
     def test_isfile(self):
         p2 = self.getnanos().path
         udir.join('test_isfile').write('\n')

diff --git a/pypy/translator/goal/nanos.py b/pypy/translator/goal/nanos.py
--- a/pypy/translator/goal/nanos.py
+++ b/pypy/translator/goal/nanos.py
@@ -47,12 +47,37 @@
                 path += '/' + b
             return path
 
+        def normpath(path):
+            if path == '':
+                return '.'
+            initial_slashes = path.startswith('/')
+            # POSIX allows one or two initial slashes, but treats three or more
+            # as single slash.
+            if (initial_slashes and
+                path.startswith('//') and not path.startswith('///')):
+                initial_slashes = 2
+            comps = path.split('/')
+            new_comps = []
+            for comp in comps:
+                if comp in ('', '.'):
+                    continue
+                if (comp != '..' or (not initial_slashes and not new_comps) or
+                     (new_comps and new_comps[-1] == '..')):
+                    new_comps.append(comp)
+                elif new_comps:
+                    new_comps.pop()
+            comps = new_comps
+            path = '/'.join(comps)
+            if initial_slashes:
+                path = '/'*initial_slashes + path
+            return path or '.'
+
         def abspath(path):
             if not path.startswith('/'):
                 import posix
                 cwd = posix.getcwd()
                 path = join(cwd, path)
-            return path       # this version does not call normpath()!
+            return normpath(path)
 
         def isfile(path):
             import posix
@@ -84,7 +109,7 @@
 
 elif os.name == 'nt':
     # code copied from ntpath.py
-    app_os_path = applevel("""
+    app_os_path = applevel(r"""
         def splitdrive(p):
             if p[1:2] == ':':
                 return p[0:2], p[2:]
@@ -158,6 +183,55 @@
 
             return path
 
+        def normpath(path):
+            if path.startswith(('\\\\.\\', '\\\\?\\')):
+                # in the case of paths with these prefixes:
+                # \\.\ -> device names
+                # \\?\ -> literal paths
+                # do not do any normalization, but return the path unchanged
+                return path
+            path = path.replace('/', '\\')
+            prefix, path = splitdrive(path)
+            # We need to be careful here. If the prefix is empty, and
+            # the path starts with a backslash, it could either be an
+            # absolute path on the current drive (\dir1\dir2\file) or a
+            # UNC filename (\\server\mount\dir1\file). It is therefore
+            # imperative NOT to collapse multiple backslashes blindly in
+            # that case.  The code below preserves multiple backslashes
+            # when there is no drive letter. This means that the invalid
+            # filename \\\a\b is preserved unchanged, where a\\\b is
+            # normalised to a\b. It's not clear that there is any better
+            # behaviour for such edge cases.
+            if prefix == '':
+                # No drive letter - preserve initial backslashes
+                while path[:1] == "\\":
+                    prefix = prefix + '\\'
+                    path = path[1:]
+            else:
+                # We have a drive letter - collapse initial backslashes
+                if path.startswith("\\"):
+                    prefix = prefix + '\\'
+                    path = path.lstrip("\\")
+            comps = path.split("\\")
+            i = 0
+            while i < len(comps):
+                if comps[i] in ('.', ''):
+                    del comps[i]
+                elif comps[i] == '..':
+                    if i > 0 and comps[i-1] != '..':
+                        del comps[i-1:i+1]
+                        i -= 1
+                    elif i == 0 and prefix.endswith("\\"):
+                        del comps[i]
+                    else:
+                        i += 1
+                else:
+                    i += 1
+            # If the path is now empty, substitute '.'
+            if not prefix and not comps:
+                comps.append('.')
+            return prefix + '\\'.join(comps)
+
         def abspath(path):
             import nt
             if path: # Empty path must return current working directory.
@@ -167,7 +241,7 @@
                     pass # Bad path - return unchanged.
             else:
                 path = nt.getcwd()
-            return path       # this version does not call normpath()!
+            return normpath(path)
 
         def isfile(path):
             import nt

From commits-noreply at bitbucket.org  Sun Feb  6 19:46:53 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sun,  6 Feb 2011 19:46:53 +0100 (CET)
Subject: [pypy-svn] pypy default: Change the test. This is explicitly a
	detail behavior, but it
Message-ID: <20110206184653.8150436C231@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41662:298b828120fe
Date: 2011-02-06 19:45 +0100
http://bitbucket.org/pypy/pypy/changeset/298b828120fe/

Log:	Change the test. This is explicitly a detail behavior, but it makes
	some sense to check that we get exactly CPython's or PyPy's behavior
	based on which one we are running on.

	Not fixed yet: on PyPy, _seen_generator_finally is not set to True.
	At least the test makes more sense this way.

diff --git a/lib-python/2.7.0/test/test_sys_settrace.py b/lib-python/modified-2.7.0/test/test_sys_settrace.py
copy from lib-python/2.7.0/test/test_sys_settrace.py
copy to lib-python/modified-2.7.0/test/test_sys_settrace.py
--- a/lib-python/2.7.0/test/test_sys_settrace.py
+++ b/lib-python/modified-2.7.0/test/test_sys_settrace.py
@@ -210,7 +210,7 @@
         yield True
         "continued"
     finally:
-        "finally"
+        global _seen_generator_finally; _seen_generator_finally = True
 def generator_example():
     # any() will leave the generator before its end
     x = any(generator_function())
@@ -219,16 +219,20 @@
     for x in range(10):
         y = x
 
+# On CPython, when the generator is decref'ed to zero, we see the trace
+# for the "finally:" portion.  On PyPy (and likely other implementations),
+# we don't see it.
+has_finally_in_trace = test_support.check_impl_detail(cpython=True)
 generator_example.events = ([(0, 'call'),
                              (2, 'line'),
                              (-6, 'call'),
                              (-5, 'line'),
                              (-4, 'line'),
-                             (-4, 'return'),
-                             (-4, 'call'),
+                             (-4, 'return')] +
+                            [(-4, 'call'),
                              (-4, 'exception'),
                              (-1, 'line'),
-                             (-1, 'return')] +
+                             (-1, 'return')] * has_finally_in_trace +
                             [(5, 'line'), (6, 'line')] * 10 +
                             [(5, 'line'), (5, 'return')])
 
@@ -323,15 +327,22 @@
         self.run_test(tighterloop_example)
 
     def test_13_genexp(self):
+        global _seen_generator_finally
+        _seen_generator_finally = False
         self.run_test(generator_example)
+        test_support.gc_collect()
+        assert _seen_generator_finally
         # issue1265: if the trace function contains a generator,
         # and if the traced function contains another generator
         # that is not completely exhausted, the trace stopped.
         # Worse: the 'finally' clause was not invoked.
         tracer = Tracer()
+        _seen_generator_finally = False
         sys.settrace(tracer.traceWithGenexp)
         generator_example()
         sys.settrace(None)
+        test_support.gc_collect()
+        assert _seen_generator_finally
         self.compare_events(generator_example.__code__.co_firstlineno,
                             tracer.events, generator_example.events)
 

From commits-noreply at bitbucket.org  Mon Feb  7 08:23:47 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Mon,  7 Feb 2011 08:23:47 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: array support
Message-ID: <20110207072347.9429B282B9C@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r41663:102f57821f4e
Date: 2011-02-07 07:57 +0100
http://bitbucket.org/pypy/pypy/changeset/102f57821f4e/

Log:	array support

diff --git a/pypy/jit/metainterp/resume.py b/pypy/jit/metainterp/resume.py
--- a/pypy/jit/metainterp/resume.py
+++ b/pypy/jit/metainterp/resume.py
@@ -562,6 +562,16 @@
         for i in self.fieldnums:
             debug_print("\t\t", str(untag(i)))
 
+    def generalization_of(self, other):
+        if self.arraydescr is not other.arraydescr:
+            return False
+        if len(self.fieldstate) != len(other.fieldstate):
+            return False
+        for i in range(len(self.fieldstate)):
+            if not self.fieldstate[i].generalization_of(other.fieldstate[i]):
+                return False
+        return True
+
 
 class VStrPlainInfo(AbstractVirtualInfo):
     """Stands for the string made out of the characters of all fieldnums."""

diff --git a/pypy/jit/metainterp/test/test_virtual.py b/pypy/jit/metainterp/test/test_virtual.py
--- a/pypy/jit/metainterp/test/test_virtual.py
+++ b/pypy/jit/metainterp/test/test_virtual.py
@@ -566,6 +566,38 @@
         def g(n1, n2, s):
             return f(n1, s) + f(n2, s)
         # FIXME: Try the case where we need to call the second version from the interpreter
+
+    def test_virtual_array_bridge(self):
+        myjitdriver = JitDriver(greens = [], reds = ['n', 'node'])
+        def f(n):
+            node = [42, 42]
+            while n > 0:
+                myjitdriver.can_enter_jit(n=n, node=node)
+                myjitdriver.jit_merge_point(n=n, node=node)
+                if (n>>3) & 1:
+                    node = [node[0], node[1] + n]
+                else:
+                    node = [node[0] + n, node[1]]
+                n -= 1
+            return node[0] + node[1]
+        assert self.meta_interp(f, [40]) == f(40)
+
+    def test_virtual_array_different_bridge(self):
+        myjitdriver = JitDriver(greens = [], reds = ['n', 'node'])
+        def f(n):
+            node = [42, 42]
+            while n > 0:
+                myjitdriver.can_enter_jit(n=n, node=node)
+                myjitdriver.jit_merge_point(n=n, node=node)
+                if (n>>3) & 1:
+                    node = [node[0], node[1] + n]
+                else:
+                    node = [node[0] + n, node[-1], node[0] + node[1]]
+                n -= 1
+            return node[0] + node[1]
+        assert self.meta_interp(f, [40]) == f(40)
+
+        
         
 
 class VirtualMiscTests:

From commits-noreply at bitbucket.org  Mon Feb  7 08:23:48 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Mon,  7 Feb 2011 08:23:48 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: fimxe
Message-ID: <20110207072348.1C08C282B9C@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r41664:262bfa7aed3f
Date: 2011-02-07 07:58 +0100
http://bitbucket.org/pypy/pypy/changeset/262bfa7aed3f/

Log:	fimxe

diff --git a/pypy/jit/metainterp/test/test_virtual.py b/pypy/jit/metainterp/test/test_virtual.py
--- a/pypy/jit/metainterp/test/test_virtual.py
+++ b/pypy/jit/metainterp/test/test_virtual.py
@@ -596,7 +596,7 @@
                 n -= 1
             return node[0] + node[1]
         assert self.meta_interp(f, [40]) == f(40)
-
+        # FIXME: Try the case where we need to call the second version from the interpreter
         
         
 

From commits-noreply at bitbucket.org  Mon Feb  7 08:23:48 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Mon,  7 Feb 2011 08:23:48 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: hg merge default
Message-ID: <20110207072348.95BF1282B9C@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r41665:90631a3d15ca
Date: 2011-02-07 07:58 +0100
http://bitbucket.org/pypy/pypy/changeset/90631a3d15ca/

Log:	hg merge default


From commits-noreply at bitbucket.org  Mon Feb  7 08:56:32 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Mon,  7 Feb 2011 08:56:32 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix SyntaxError on Windows
Message-ID: <20110207075632.3CCFF282B9D@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41666:4b9129d84a5f
Date: 2011-02-07 08:56 +0100
http://bitbucket.org/pypy/pypy/changeset/4b9129d84a5f/

Log:	Fix SyntaxError on Windows

diff --git a/pypy/translator/goal/nanos.py b/pypy/translator/goal/nanos.py
--- a/pypy/translator/goal/nanos.py
+++ b/pypy/translator/goal/nanos.py
@@ -256,7 +256,7 @@
 
     """, filename=__file__)
 
-    app_os = applevel("""
+    app_os = applevel(r"""
         sep = '\\'
         pathsep = ';'
         name = 'nt'

From commits-noreply at bitbucket.org  Mon Feb  7 14:38:40 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon,  7 Feb 2011 14:38:40 +0100 (CET)
Subject: [pypy-svn] pypy default: Reverted part of the changes now that I
	understand the
Message-ID: <20110207133840.52587282BE3@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41667:024ea3880c28
Date: 2011-02-07 14:38 +0100
http://bitbucket.org/pypy/pypy/changeset/024ea3880c28/

Log:	Reverted part of the changes now that I understand the problem
	better.

diff --git a/lib-python/modified-2.7.0/test/test_sys_settrace.py b/lib-python/modified-2.7.0/test/test_sys_settrace.py
--- a/lib-python/modified-2.7.0/test/test_sys_settrace.py
+++ b/lib-python/modified-2.7.0/test/test_sys_settrace.py
@@ -210,29 +210,29 @@
         yield True
         "continued"
     finally:
-        global _seen_generator_finally; _seen_generator_finally = True
+        "finally"
 def generator_example():
     # any() will leave the generator before its end
-    x = any(generator_function())
+    x = any(generator_function()); gc.collect()
 
     # the following lines were not traced
     for x in range(10):
         y = x
 
 # On CPython, when the generator is decref'ed to zero, we see the trace
-# for the "finally:" portion.  On PyPy (and likely other implementations),
-# we don't see it.
-has_finally_in_trace = test_support.check_impl_detail(cpython=True)
+# for the "finally:" portion.  On PyPy, we don't see it before the next
+# garbage collection.  That's why we put gc.collect() on the same line above.
+
 generator_example.events = ([(0, 'call'),
                              (2, 'line'),
                              (-6, 'call'),
                              (-5, 'line'),
                              (-4, 'line'),
-                             (-4, 'return')] +
-                            [(-4, 'call'),
+                             (-4, 'return'),
+                             (-4, 'call'),
                              (-4, 'exception'),
                              (-1, 'line'),
-                             (-1, 'return')] * has_finally_in_trace +
+                             (-1, 'return')] +
                             [(5, 'line'), (6, 'line')] * 10 +
                             [(5, 'line'), (5, 'return')])
 
@@ -327,24 +327,24 @@
         self.run_test(tighterloop_example)
 
     def test_13_genexp(self):
-        global _seen_generator_finally
-        _seen_generator_finally = False
-        self.run_test(generator_example)
-        test_support.gc_collect()
-        assert _seen_generator_finally
-        # issue1265: if the trace function contains a generator,
-        # and if the traced function contains another generator
-        # that is not completely exhausted, the trace stopped.
-        # Worse: the 'finally' clause was not invoked.
-        tracer = Tracer()
-        _seen_generator_finally = False
-        sys.settrace(tracer.traceWithGenexp)
-        generator_example()
-        sys.settrace(None)
-        test_support.gc_collect()
-        assert _seen_generator_finally
-        self.compare_events(generator_example.__code__.co_firstlineno,
-                            tracer.events, generator_example.events)
+        if self.using_gc:
+            test_support.gc_collect()
+            gc.enable()
+        try:
+            self.run_test(generator_example)
+            # issue1265: if the trace function contains a generator,
+            # and if the traced function contains another generator
+            # that is not completely exhausted, the trace stopped.
+            # Worse: the 'finally' clause was not invoked.
+            tracer = Tracer()
+            sys.settrace(tracer.traceWithGenexp)
+            generator_example()
+            sys.settrace(None)
+            self.compare_events(generator_example.__code__.co_firstlineno,
+                                tracer.events, generator_example.events)
+        finally:
+            if self.using_gc:
+                gc.disable()
 
     def test_14_onliner_if(self):
         def onliners():

From commits-noreply at bitbucket.org  Mon Feb  7 15:10:58 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon,  7 Feb 2011 15:10:58 +0100 (CET)
Subject: [pypy-svn] pypy default: Extend test_trace_generator_finalisation
	to check for	the correct
Message-ID: <20110207141058.3903C36C374@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41668:9dc525f2d8e4
Date: 2011-02-07 15:10 +0100
http://bitbucket.org/pypy/pypy/changeset/9dc525f2d8e4/

Log:	Extend test_trace_generator_finalisation to check for the correct
	order of the events: generator.throw() used to get the tracing
	events "exception" followed by "call", whereas on CPython it is the
	opposite order. Fix it.

diff --git a/pypy/interpreter/generator.py b/pypy/interpreter/generator.py
--- a/pypy/interpreter/generator.py
+++ b/pypy/interpreter/generator.py
@@ -55,7 +55,7 @@
         frame = self.frame
         if frame is None:
             # xxx a bit ad-hoc, but we don't want to go inside
-            # execute_generator_frame() if the frame is actually finished
+            # execute_frame() if the frame is actually finished
             if operr is None:
                 operr = OperationError(space.w_StopIteration, space.w_None)
             raise operr
@@ -72,7 +72,7 @@
         self.running = True
         try:
             try:
-                w_result = frame.execute_generator_frame(w_arg, operr)
+                w_result = frame.execute_frame(w_arg, operr)
             except OperationError:
                 # errors finish a frame
                 self.frame = None

diff --git a/pypy/interpreter/test/test_pyframe.py b/pypy/interpreter/test/test_pyframe.py
--- a/pypy/interpreter/test/test_pyframe.py
+++ b/pypy/interpreter/test/test_pyframe.py
@@ -279,12 +279,13 @@
         
 
     def test_trace_generator_finalisation(self):
-        # XXX expand to check more aspects
         import sys
         l = []
+        got_exc = []
         def trace(frame, event, arg):
+            l.append((frame.f_lineno, event))
             if event == 'exception':
-                l.append(arg)
+                got_exc.append(arg)
             return trace
 
         d = {}
@@ -308,8 +309,22 @@
         sys.settrace(trace)
         f()
         sys.settrace(None)
-        assert len(l) == 1
-        assert issubclass(l[0][0], GeneratorExit)
+        assert len(got_exc) == 1
+        assert issubclass(got_exc[0][0], GeneratorExit)
+        assert l == [(8, 'call'),
+                     (9, 'line'),
+                     (10, 'line'),
+                     (11, 'line'),
+                     (2, 'call'),
+                     (3, 'line'),
+                     (4, 'line'),
+                     (4, 'return'),
+                     (12, 'line'),
+                     (4, 'call'),
+                     (4, 'exception'),
+                     (6, 'line'),
+                     (6, 'return'),
+                     (12, 'return')]
 
     def test_dont_trace_on_reraise(self):
         import sys

diff --git a/pypy/interpreter/pyframe.py b/pypy/interpreter/pyframe.py
--- a/pypy/interpreter/pyframe.py
+++ b/pypy/interpreter/pyframe.py
@@ -125,17 +125,12 @@
         else:
             return self.execute_frame()
 
-    def execute_generator_frame(self, w_inputvalue, operr=None):
-        if operr is not None:
-            ec = self.space.getexecutioncontext()
-            next_instr = self.handle_operation_error(ec, operr)
-            self.last_instr = intmask(next_instr - 1)
-        elif self.last_instr != -1:
-            self.pushvalue(w_inputvalue)
-        return self.execute_frame()
-
-    def execute_frame(self):
-        """Execute this frame.  Main entry point to the interpreter."""
+    def execute_frame(self, w_inputvalue=None, operr=None):
+        """Execute this frame.  Main entry point to the interpreter.
+        The optional arguments are there to handle a generator's frame:
+        w_inputvalue is for generator.send()) and operr is for
+        generator.throw()).
+        """
         # the following 'assert' is an annotation hint: it hides from
         # the annotator all methods that are defined in PyFrame but
         # overridden in the {,Host}FrameClass subclasses of PyFrame.
@@ -145,10 +140,19 @@
         executioncontext.enter(self)
         try:
             executioncontext.call_trace(self)
-            # Execution starts just after the last_instr.  Initially,
-            # last_instr is -1.  After a generator suspends it points to
-            # the YIELD_VALUE instruction.
-            next_instr = self.last_instr + 1
+            #
+            if operr is not None:
+                ec = self.space.getexecutioncontext()
+                next_instr = self.handle_operation_error(ec, operr)
+                self.last_instr = intmask(next_instr - 1)
+            else:
+                # Execution starts just after the last_instr.  Initially,
+                # last_instr is -1.  After a generator suspends it points to
+                # the YIELD_VALUE instruction.
+                next_instr = self.last_instr + 1
+                if next_instr != 0:
+                    self.pushvalue(w_inputvalue)
+            #
             try:
                 w_exitvalue = self.dispatch(self.pycode, next_instr,
                                             executioncontext)

From commits-noreply at bitbucket.org  Mon Feb  7 15:30:06 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Mon,  7 Feb 2011 15:30:06 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix the math module when the mingw32
	compiler is used.
Message-ID: <20110207143006.D3C5736C374@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41669:398ee49204b8
Date: 2011-02-07 15:27 +0100
http://bitbucket.org/pypy/pypy/changeset/398ee49204b8/

Log:	Fix the math module when the mingw32 compiler is used.

diff --git a/pypy/rpython/lltypesystem/module/test/test_ll_math.py b/pypy/rpython/lltypesystem/module/test/test_ll_math.py
--- a/pypy/rpython/lltypesystem/module/test/test_ll_math.py
+++ b/pypy/rpython/lltypesystem/module/test/test_ll_math.py
@@ -34,7 +34,8 @@
                 repr,)
         else:
             if not get_tester(expected)(got):
-                raise AssertionError("%r: got %s" % (repr, got))
+                raise AssertionError("%r: got %r, expected %r" %
+                                     (repr, got, expected))
     #
     dict[fnname] = dict.get(fnname, 0) + 1
     testname = 'test_%s_%d' % (fnname, dict[fnname])

diff --git a/pypy/translator/c/src/ll_math.c b/pypy/translator/c/src/ll_math.c
--- a/pypy/translator/c/src/ll_math.c
+++ b/pypy/translator/c/src/ll_math.c
@@ -5,18 +5,22 @@
 
 /* The following macros are copied from CPython header files */
 
-#ifdef _MSC_VER
+#ifdef _WIN32
 #include <float.h>
 #include <math.h>
+#endif
+
+#ifdef _MSC_VER
 #define PyPy_IS_NAN _isnan
 #define PyPy_IS_INFINITY(X) (!_finite(X) && !_isnan(X))
 #define copysign _copysign
 #else
 #define PyPy_IS_NAN(X) ((X) != (X))
+#define Py_FORCE_DOUBLE(X) ((double)(X))
 #define PyPy_IS_INFINITY(X) ((X) &&                                   \
                              (Py_FORCE_DOUBLE(X)*0.5 == Py_FORCE_DOUBLE(X)))
 #endif
-#define PyPy_NAN (HUGE * 0.)
+#define PyPy_NAN (HUGE_VAL * 0.)
 
 int
 _pypy_math_isinf(double x)

From commits-noreply at bitbucket.org  Mon Feb  7 15:30:07 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Mon,  7 Feb 2011 15:30:07 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix the socket functions when compiled
	with mingw32
Message-ID: <20110207143007.93B6936C374@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41670:3494ee074c60
Date: 2011-02-07 15:28 +0100
http://bitbucket.org/pypy/pypy/changeset/3494ee074c60/

Log:	Fix the socket functions when compiled with mingw32

diff --git a/pypy/rlib/_rsocket_rffi.py b/pypy/rlib/_rsocket_rffi.py
--- a/pypy/rlib/_rsocket_rffi.py
+++ b/pypy/rlib/_rsocket_rffi.py
@@ -55,19 +55,32 @@
     header_lines = [
         '#include <WinSock2.h>',
         '#include <WS2tcpip.h>',
-        '#include <Mstcpip.h>',
         # winsock2 defines AF_UNIX, but not sockaddr_un
         '#undef AF_UNIX',
         ]
     if _MSVC:
         header_lines.extend([
+            '#include <Mstcpip.h>',
             # these types do not exist on microsoft compilers
             'typedef int ssize_t;',
             'typedef unsigned __int16 uint16_t;',
             'typedef unsigned __int32 uint32_t;',
             ])
-    else:
+    else: # MINGW
         includes = ('stdint.h',)
+        header_lines.extend([
+            '#define SIO_RCVALL             _WSAIOW(IOC_VENDOR,1)',
+            '#define SIO_KEEPALIVE_VALS     _WSAIOW(IOC_VENDOR,4)',
+            '#define RCVALL_OFF             0',
+            '#define RCVALL_ON              1',
+            '#define RCVALL_SOCKETLEVELONLY 2',
+            '''\
+            struct tcp_keepalive {
+                u_long  onoff;
+                u_long  keepalivetime;
+                u_long  keepaliveinterval;
+            };'''
+            ])
     HEADER = '\n'.join(header_lines)
     COND_HEADER = ''
 constants = {}
@@ -208,7 +221,7 @@
 
 if _WIN32:
     # some SDKs define these values with an enum, #ifdef won't work
-    for name in ('RCVALL_ON', 'RCVALL_OFF'):
+    for name in ('RCVALL_ON', 'RCVALL_OFF', 'RCVALL_SOCKETLEVELONLY'):
         setattr(CConfig, name, platform.ConstantInteger(name))
         constant_names.append(name)
 
@@ -418,7 +431,7 @@
 WIN32 = cConfig.WIN32
 assert WIN32 == _WIN32
 
-if WIN32:
+if _MSVC:
     def invalid_socket(fd):
         return fd == INVALID_SOCKET
     INVALID_SOCKET = cConfig.INVALID_SOCKET

From commits-noreply at bitbucket.org  Mon Feb  7 15:53:05 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon,  7 Feb 2011 15:53:05 +0100 (CET)
Subject: [pypy-svn] pypy default: Mostly a copy of the failing test from
	CPython.
Message-ID: <20110207145305.0F264282BE3@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41671:d8b5b9171185
Date: 2011-02-07 15:49 +0100
http://bitbucket.org/pypy/pypy/changeset/d8b5b9171185/

Log:	Mostly a copy of the failing test from CPython.

diff --git a/pypy/interpreter/test/test_pyframe.py b/pypy/interpreter/test/test_pyframe.py
--- a/pypy/interpreter/test/test_pyframe.py
+++ b/pypy/interpreter/test/test_pyframe.py
@@ -45,21 +45,44 @@
     def test_f_lineno_set(self):
         def tracer(f, *args):
             def x(f, *args):
-                if f.f_lineno == origin + 1:
-                    f.f_lineno = origin + 2
+                f.f_lineno += 1
             return x
 
         def function():
             xyz
             return 3
         
+        import sys
+        sys.settrace(tracer)
+        function()
+        sys.settrace(None)
+        # assert did not crash
+
+    def test_f_lineno_set_firstline(self):
+        seen = []
+        def tracer(f, event, *args):
+            seen.append((event, f.f_lineno))
+            if len(seen) == 5:
+                f.f_lineno = 1       # bug shown only when setting lineno to 1
+            return tracer
+
         def g():
             import sys
             sys.settrace(tracer)
-            function()
+            exec "x=1\ny=x+1\nz=y+1\nt=z+1\ns=t+1\n" in {}
             sys.settrace(None)
-        origin = function.func_code.co_firstlineno
-        g() # assert did not crash
+
+        g()
+        assert seen == [('call', 1),
+                        ('line', 1),
+                        ('line', 2),
+                        ('line', 3),
+                        ('line', 4),
+                        ('line', 2),
+                        ('line', 3),
+                        ('line', 4),
+                        ('line', 5),
+                        ('return', 5)]
 
     def test_f_back(self):
         import sys

From commits-noreply at bitbucket.org  Mon Feb  7 15:53:05 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon,  7 Feb 2011 15:53:05 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix.
Message-ID: <20110207145305.8B159282BE3@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41672:d368b02f7f09
Date: 2011-02-07 15:52 +0100
http://bitbucket.org/pypy/pypy/changeset/d368b02f7f09/

Log:	Fix.

diff --git a/pypy/interpreter/pyframe.py b/pypy/interpreter/pyframe.py
--- a/pypy/interpreter/pyframe.py
+++ b/pypy/interpreter/pyframe.py
@@ -463,28 +463,30 @@
             raise OperationError(space.w_ValueError,
                   space.wrap("f_lineno can only be set by a trace function."))
 
-        if new_lineno < self.pycode.co_firstlineno:
+        line = self.pycode.co_firstlineno
+        if new_lineno < line:
             raise operationerrfmt(space.w_ValueError,
                   "line %d comes before the current code.", new_lineno)
-        code = self.pycode.co_code
-        addr = 0
-        line = self.pycode.co_firstlineno
-        new_lasti = -1
-        offset = 0
-        lnotab = self.pycode.co_lnotab
-        for offset in xrange(0, len(lnotab), 2):
-            addr += ord(lnotab[offset])
-            line += ord(lnotab[offset + 1])
-            if line >= new_lineno:
-                new_lasti = addr
-                new_lineno = line
-                break
+        elif new_lineno == line:
+            new_lasti = 0
+        else:
+            new_lasti = -1
+            addr = 0
+            lnotab = self.pycode.co_lnotab
+            for offset in xrange(0, len(lnotab), 2):
+                addr += ord(lnotab[offset])
+                line += ord(lnotab[offset + 1])
+                if line >= new_lineno:
+                    new_lasti = addr
+                    new_lineno = line
+                    break
 
         if new_lasti == -1:
             raise operationerrfmt(space.w_ValueError,
                   "line %d comes after the current code.", new_lineno)
 
         # Don't jump to a line with an except in it.
+        code = self.pycode.co_code
         if ord(code[new_lasti]) in (DUP_TOP, POP_TOP):
             raise OperationError(space.w_ValueError,
                   space.wrap("can't jump to 'except' line as there's no exception"))

From cfbolz at codespeak.net  Mon Feb  7 17:19:02 2011
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Mon,  7 Feb 2011 17:19:02 +0100 (CET)
Subject: [pypy-svn] r80306 - pypy/extradoc/talk/ibm-feb-2011
Message-ID: <20110207161902.68B46282BE7@codespeak.net>

Author: cfbolz
Date: Mon Feb  7 17:18:59 2011
New Revision: 80306

Added:
   pypy/extradoc/talk/ibm-feb-2011/
      - copied from r80287, pypy/extradoc/talk/microsoft-jan-2011/
Log:
IBM talk



From commits-noreply at bitbucket.org  Mon Feb  7 17:49:20 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon,  7 Feb 2011 17:49:20 +0100 (CET)
Subject: [pypy-svn] pypy default: Implement str._formatter_parser().
Message-ID: <20110207164920.22D0D282BE7@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41673:bc55799462bc
Date: 2011-02-07 17:20 +0100
http://bitbucket.org/pypy/pypy/changeset/bc55799462bc/

Log:	Implement str._formatter_parser().

diff --git a/pypy/objspace/std/unicodetype.py b/pypy/objspace/std/unicodetype.py
--- a/pypy/objspace/std/unicodetype.py
+++ b/pypy/objspace/std/unicodetype.py
@@ -152,6 +152,13 @@
                              ' field\nof the specified width. The string x is'
                              ' never truncated.')
 
+unicode_formatter_parser = SMM('_formatter_parser', 1)
+
+def unicode_formatter_parser__ANY(space, w_unicode):
+    from pypy.objspace.std.newformat import unicode_template_formatter
+    tformat = unicode_template_formatter(space, space.unicode_w(w_unicode))
+    return tformat.formatter_parser()
+
 # stuff imported from stringtype for interoperability
 
 from pypy.objspace.std.stringtype import str_endswith as unicode_endswith

diff --git a/pypy/objspace/std/newformat.py b/pypy/objspace/std/newformat.py
--- a/pypy/objspace/std/newformat.py
+++ b/pypy/objspace/std/newformat.py
@@ -39,6 +39,8 @@
 
     _annspecialcase_ = "specialize:ctr_location"
 
+    parser_list_w = None
+
     def __init__(self, space, is_unicode, template):
         self.space = space
         self.is_unicode = is_unicode
@@ -129,12 +131,12 @@
                     i += 1
                     c = s[i]
                 else:
-                    conversion = self.empty
+                    conversion = None
                 if c == ':':
                     i += 1
                 return s[start:end_name], conversion, i
             i += 1
-        return s[start:end], self.empty, end
+        return s[start:end], None, end
 
     def _get_argument(self, name):
         # First, find the argument.
@@ -248,8 +250,24 @@
     def _render_field(self, start, end, recursive, level):
         name, conversion, spec_start = self._parse_field(start, end)
         spec = self.template[spec_start:end]
+        #
+        if self.parser_list_w is not None:
+            # used from formatter_parser()
+            if level == 1:    # ignore recursive calls
+                space = self.space
+                startm1 = start - 1
+                assert startm1 >= self.last_end
+                w_entry = space.newtuple([
+                    space.wrap(self.template[self.last_end:startm1]),
+                    space.wrap(name),
+                    space.wrap(spec),
+                    space.wrap(conversion)])
+                self.parser_list_w.append(w_entry)
+                self.last_end = end + 1
+            return self.empty
+        #
         w_obj = self._get_argument(name)
-        if conversion:
+        if conversion is not None:
             w_obj = self._convert(w_obj, conversion)
         if recursive:
             spec = self._build_string(spec_start, end, level)
@@ -258,13 +276,36 @@
         to_interp = getattr(self.space, unwrapper)
         return to_interp(w_rendered)
 
+    def formatter_parser(self):
+        self.parser_list_w = []
+        self.last_end = 0
+        self._build_string(0, len(self.template), 2)
+        #
+        space = self.space
+        if self.last_end < len(self.template):
+            w_lastentry = space.newtuple([
+                space.wrap(self.template[self.last_end:]),
+                space.w_None,
+                space.w_None,
+                space.w_None])
+            self.parser_list_w.append(w_lastentry)
+        return space.iter(space.newlist(self.parser_list_w))
+
+
+def str_template_formatter(space, template):
+    return TemplateFormatter(space, False, template)
+
+def unicode_template_formatter(space, template):
+    return TemplateFormatter(space, True, template)
+
 
 def format_method(space, w_string, args, is_unicode):
     if is_unicode:
-        template = TemplateFormatter(space, True, space.unicode_w(w_string))
+        template = unicode_template_formatter(space,
+                                              space.unicode_w(w_string))
         return space.wrap(template.build(args))
     else:
-        template = TemplateFormatter(space, False, space.str_w(w_string))
+        template = str_template_formatter(space, space.str_w(w_string))
         return space.wrap(template.build(args))
 
 

diff --git a/pypy/objspace/std/stringtype.py b/pypy/objspace/std/stringtype.py
--- a/pypy/objspace/std/stringtype.py
+++ b/pypy/objspace/std/stringtype.py
@@ -1,6 +1,8 @@
 from pypy.interpreter import gateway
 from pypy.objspace.std.stdtypedef import StdTypeDef, SMM
 from pypy.objspace.std.basestringtype import basestring_typedef
+from pypy.objspace.std.register_all import register_all
+
 
 from sys import maxint
 from pypy.rlib.objectmodel import specialize
@@ -271,6 +273,15 @@
                          ' with\ncodecs.register_error that is able to handle'
                          ' UnicodeEncodeErrors.')
 
+str_formatter_parser = SMM('_formatter_parser', 1)
+
+def str_formatter_parser__ANY(space, w_str):
+    from pypy.objspace.std.newformat import str_template_formatter
+    tformat = str_template_formatter(space, space.str_w(w_str))
+    return tformat.formatter_parser()
+
+register_all(vars(), globals())
+
 # ____________________________________________________________
 
 def descr__new__(space, w_stringtype, w_object=''):

diff --git a/pypy/objspace/std/test/test_newformat.py b/pypy/objspace/std/test/test_newformat.py
--- a/pypy/objspace/std/test/test_newformat.py
+++ b/pypy/objspace/std/test/test_newformat.py
@@ -324,3 +324,38 @@
         assert format(inf, "F") == "INF"
         assert format(nan, "f") == "nan"
         assert format(nan, "F") == "NAN"
+
+
+class AppTestInternalMethods:
+    # undocumented API on string and unicode object, but used by string.py
+
+    def test_formatter_parser(self):
+        l = list('abcd'._formatter_parser())
+        assert l == [('abcd', None, None, None)]
+        #
+        l = list('ab{0}cd'._formatter_parser())
+        assert l == [('ab', '0', '', None), ('cd', None, None, None)]
+        #
+        l = list('{0}cd'._formatter_parser())
+        assert l == [('', '0', '', None), ('cd', None, None, None)]
+        #
+        l = list('ab{0}'._formatter_parser())
+        assert l == [('ab', '0', '', None)]
+        #
+        l = list(''._formatter_parser())
+        assert l == []
+        #
+        l = list('{0:123}'._formatter_parser())
+        assert l == [('', '0', '123', None)]
+        #
+        l = list('{0!x:123}'._formatter_parser())
+        assert l == [('', '0', '123', 'x')]
+        #
+        l = list('{0!x:12{sdd}3}'._formatter_parser())
+        assert l == [('', '0', '12{sdd}3', 'x')]
+
+    def test_u_formatter_parser(self):
+        l = list(u'{0!x:12{sdd}3}'._formatter_parser())
+        assert l == [(u'', u'0', u'12{sdd}3', u'x')]
+        for x in l[0]:
+            assert isinstance(x, unicode)

From commits-noreply at bitbucket.org  Mon Feb  7 17:49:21 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon,  7 Feb 2011 17:49:21 +0100 (CET)
Subject: [pypy-svn] pypy default: Add the internal
	method	str._formatter_field_name_split().
Message-ID: <20110207164921.74BBA282BE7@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41674:85c84e92a83e
Date: 2011-02-07 17:48 +0100
http://bitbucket.org/pypy/pypy/changeset/85c84e92a83e/

Log:	Add the internal method str._formatter_field_name_split().

diff --git a/pypy/objspace/std/unicodetype.py b/pypy/objspace/std/unicodetype.py
--- a/pypy/objspace/std/unicodetype.py
+++ b/pypy/objspace/std/unicodetype.py
@@ -152,13 +152,19 @@
                              ' field\nof the specified width. The string x is'
                              ' never truncated.')
 
-unicode_formatter_parser = SMM('_formatter_parser', 1)
+unicode_formatter_parser           = SMM('_formatter_parser', 1)
+unicode_formatter_field_name_split = SMM('_formatter_field_name_split', 1)
 
 def unicode_formatter_parser__ANY(space, w_unicode):
     from pypy.objspace.std.newformat import unicode_template_formatter
     tformat = unicode_template_formatter(space, space.unicode_w(w_unicode))
     return tformat.formatter_parser()
 
+def unicode_formatter_field_name_split__ANY(space, w_unicode):
+    from pypy.objspace.std.newformat import unicode_template_formatter
+    tformat = unicode_template_formatter(space, space.unicode_w(w_unicode))
+    return tformat.formatter_field_name_split()
+
 # stuff imported from stringtype for interoperability
 
 from pypy.objspace.std.stringtype import str_endswith as unicode_endswith

diff --git a/pypy/objspace/std/newformat.py b/pypy/objspace/std/newformat.py
--- a/pypy/objspace/std/newformat.py
+++ b/pypy/objspace/std/newformat.py
@@ -192,9 +192,12 @@
             except IndexError:
                 w_msg = space.wrap("index out of range")
                 raise OperationError(space.w_IndexError, w_msg)
+        return self._resolve_lookups(w_arg, name, i, end)
+
+    def _resolve_lookups(self, w_obj, name, start, end):
         # Resolve attribute and item lookups.
-        w_obj = w_arg
-        is_attribute = False
+        space = self.space
+        i = start
         while i < end:
             c = name[i]
             if c == ".":
@@ -208,7 +211,12 @@
                 if start == i:
                     w_msg = space.wrap("Empty attribute in format string")
                     raise OperationError(space.w_ValueError, w_msg)
-                w_obj = space.getattr(w_obj, space.wrap(name[start:i]))
+                w_attr = space.wrap(name[start:i])
+                if w_obj is not None:
+                    w_obj = space.getattr(w_obj, w_attr)
+                else:
+                    self.parser_list_w.append(space.newtuple([
+                        space.w_True, w_attr]))
             elif c == "[":
                 got_bracket = False
                 i += 1
@@ -228,12 +236,32 @@
                 else:
                     w_item = space.wrap(name[start:i])
                 i += 1 # Skip "]"
-                w_obj = space.getitem(w_obj, w_item)
+                if w_obj is not None:
+                    w_obj = space.getitem(w_obj, w_item)
+                else:
+                    self.parser_list_w.append(space.newtuple([
+                        space.w_False, w_item]))
             else:
                 msg = "Only '[' and '.' may follow ']'"
                 raise OperationError(space.w_ValueError, space.wrap(msg))
         return w_obj
 
+    def formatter_field_name_split(self):
+        name = self.template
+        i = 0
+        end = len(name)
+        while i < end:
+            c = name[i]
+            if c == "[" or c == ".":
+                break
+            i += 1
+        self.parser_list_w = []
+        self._resolve_lookups(None, name, i, end)
+        #
+        space = self.space
+        return space.newtuple([space.wrap(name[:i]),
+                               space.iter(space.newlist(self.parser_list_w))])
+
     def _convert(self, w_obj, conversion):
         space = self.space
         conv = conversion[0]

diff --git a/pypy/objspace/std/stringtype.py b/pypy/objspace/std/stringtype.py
--- a/pypy/objspace/std/stringtype.py
+++ b/pypy/objspace/std/stringtype.py
@@ -273,13 +273,19 @@
                          ' with\ncodecs.register_error that is able to handle'
                          ' UnicodeEncodeErrors.')
 
-str_formatter_parser = SMM('_formatter_parser', 1)
+str_formatter_parser           = SMM('_formatter_parser', 1)
+str_formatter_field_name_split = SMM('_formatter_field_name_split', 1)
 
 def str_formatter_parser__ANY(space, w_str):
     from pypy.objspace.std.newformat import str_template_formatter
     tformat = str_template_formatter(space, space.str_w(w_str))
     return tformat.formatter_parser()
 
+def str_formatter_field_name_split__ANY(space, w_str):
+    from pypy.objspace.std.newformat import str_template_formatter
+    tformat = str_template_formatter(space, space.str_w(w_str))
+    return tformat.formatter_field_name_split()
+
 register_all(vars(), globals())
 
 # ____________________________________________________________

diff --git a/pypy/objspace/std/test/test_newformat.py b/pypy/objspace/std/test/test_newformat.py
--- a/pypy/objspace/std/test/test_newformat.py
+++ b/pypy/objspace/std/test/test_newformat.py
@@ -359,3 +359,33 @@
         assert l == [(u'', u'0', u'12{sdd}3', u'x')]
         for x in l[0]:
             assert isinstance(x, unicode)
+
+    def test_formatter_field_name_split(self):
+        first, rest = 'foo'._formatter_field_name_split()
+        assert first == 'foo'
+        assert list(rest) == []
+        #
+        first, rest = 'foo.bar'._formatter_field_name_split()
+        assert first == 'foo'
+        assert list(rest) == [(True, 'bar')]
+        #
+        first, rest = 'foo[123]'._formatter_field_name_split()
+        assert first == 'foo'
+        assert list(rest) == [(False, 123)]
+        #
+        first, rest = 'foo.baz[123].bok'._formatter_field_name_split()
+        assert first == 'foo'
+        assert list(rest) == [(True, 'baz'), (False, 123), (True, 'bok')]
+        #
+        first, rest = 'foo.baz[hi].bok'._formatter_field_name_split()
+        assert first == 'foo'
+        assert list(rest) == [(True, 'baz'), (False, 'hi'), (True, 'bok')]
+
+    def test_u_formatter_field_name_split(self):
+        first, rest = u'foo.baz[hi].bok'._formatter_field_name_split()
+        l = list(rest)
+        assert first == u'foo'
+        assert l == [(True, u'baz'), (False, u'hi'), (True, u'bok')]
+        assert isinstance(first, unicode)
+        for x, y in l:
+            assert isinstance(y, unicode)

From commits-noreply at bitbucket.org  Mon Feb  7 17:53:55 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Mon,  7 Feb 2011 17:53:55 +0100 (CET)
Subject: [pypy-svn] pypy default: libffi now works with mingw32
Message-ID: <20110207165355.3EB20282BE7@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41675:8c746acf48f9
Date: 2011-02-07 17:37 +0100
http://bitbucket.org/pypy/pypy/changeset/8c746acf48f9/

Log:	libffi now works with mingw32

diff --git a/pypy/module/_rawffi/test/test__rawffi.py b/pypy/module/_rawffi/test/test__rawffi.py
--- a/pypy/module/_rawffi/test/test__rawffi.py
+++ b/pypy/module/_rawffi/test/test__rawffi.py
@@ -5,7 +5,6 @@
 from pypy.translator.tool.cbuild import ExternalCompilationInfo
 from pypy.module._rawffi.interp_rawffi import TYPEMAP
 from pypy.module._rawffi.tracker import Tracker
-from pypy.translator.platform import platform
 
 import os, sys, py
 
@@ -211,6 +210,7 @@
             cls.w_libm_name = space.wrap('libm.so')
             if sys.platform == "darwin":
                 cls.w_libm_name = space.wrap('libm.dylib')
+        cls.w_platform = space.wrap(platform.name)
         cls.w_sizes_and_alignments = space.wrap(dict(
             [(k, (v.c_size, v.c_alignment)) for k,v in TYPEMAP.iteritems()]))
 
@@ -908,8 +908,8 @@
         raises(_rawffi.SegfaultException, a.__setitem__, 3, 3)
 
     def test_stackcheck(self):
-        if not self.iswin32:
-            skip("win32 specific")
+        if self.platform != "msvc":
+            skip("win32 msvc specific")
 
         # Even if the call corresponds to the specified signature,
         # the STDCALL calling convention may detect some errors

diff --git a/pypy/doc/windows.txt b/pypy/doc/windows.txt
--- a/pypy/doc/windows.txt
+++ b/pypy/doc/windows.txt
@@ -96,4 +96,18 @@
 Using the mingw compiler
 ------------------------
 
-XXX Write me
+You can compile pypy with the mingw compiler, using the --cc=mingw32 option;
+mingw.exe must be on the PATH.
+
+libffi for the mingw32 compiler
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+To enable the _rawffi (and ctypes) module, you need to compile a mingw32
+version of libffi.  I downloaded the `libffi source files`_, and extracted
+them in the base directory.  Then run::
+
+    sh ./configure
+    make
+    cp .libs/libffi-5.dll <somewhere on the PATH>
+
+.. _`libffi source files`: http://sourceware.org/libffi/

diff --git a/pypy/rlib/rwin32.py b/pypy/rlib/rwin32.py
--- a/pypy/rlib/rwin32.py
+++ b/pypy/rlib/rwin32.py
@@ -5,6 +5,7 @@
 from pypy.rpython.tool import rffi_platform
 from pypy.tool.udir import udir
 from pypy.translator.tool.cbuild import ExternalCompilationInfo
+from pypy.translator.platform import CompilationError
 from pypy.rpython.lltypesystem import lltype, rffi
 from pypy.rlib.rarithmetic import intmask
 from pypy.rlib import jit
@@ -145,7 +146,7 @@
                 [cfile], ExternalCompilationInfo(),
                 outputfilename = "dosmaperr",
                 standalone=True)
-        except WindowsError:
+        except (CompilationError, WindowsError):
             # Fallback for the mingw32 compiler
             errors = {
                 2: 2, 3: 2, 4: 24, 5: 13, 6: 9, 7: 12, 8: 12, 9: 12, 10: 7,

diff --git a/pypy/rlib/clibffi.py b/pypy/rlib/clibffi.py
--- a/pypy/rlib/clibffi.py
+++ b/pypy/rlib/clibffi.py
@@ -31,6 +31,7 @@
 if _WIN32:
     separate_module_sources = ['''
     #include <stdio.h>
+    #include <windows.h>
 
     /* Get the module where the "fopen" function resides in */
     HANDLE pypy_get_libc_handle() {
@@ -49,14 +50,11 @@
 else:
     separate_module_sources = []
 
-if not _MSVC:
+if not _WIN32:
     # On some platforms, we try to link statically libffi, which is small
     # anyway and avoids endless troubles for installing.  On other platforms
     # libffi.a is typically not there, so we link dynamically.
-    if _MINGW:
-        includes = ['windows.h', 'ffi.h']
-    else:
-        includes = ['ffi.h']
+    includes = ['ffi.h']
 
     if _MAC_OS:
         pre_include_bits = ['#define MACOSX']
@@ -90,6 +88,22 @@
         link_files = link_files,
         testonly_libraries = ['ffi'],
     )
+elif _MINGW:
+    includes = ['ffi.h']
+    libraries = ['libffi-5']
+
+    eci = ExternalCompilationInfo(
+        libraries = libraries,
+        includes = includes,
+        export_symbols = [],
+        separate_module_sources = separate_module_sources,
+        )
+
+    eci = rffi_platform.configure_external_library(
+        'libffi', eci,
+        [dict(prefix='libffi-',
+              include_dir='include', library_dir='.libs'),
+         ])
 else:
     libffidir = py.path.local(pypydir).join('translator', 'c', 'src', 'libffi_msvc')
     eci = ExternalCompilationInfo(
@@ -214,17 +228,10 @@
     return rffi.llexternal(name, args, result, compilation_info=eci, calling_conv='win')
 
 
-if not _WIN32:
+if not _MSVC:
     def check_fficall_result(result, flags):
         pass # No check
-    
-    libc_name = ctypes.util.find_library('c')
-    assert libc_name is not None, "Cannot find C library, ctypes.util.find_library('c') returned None"
-
-    def get_libc_name():
-        return libc_name
-
-if _WIN32:
+else:
     def check_fficall_result(result, flags):
         if result == 0:
             return
@@ -245,8 +252,13 @@
                 "Procedure called with too many "
                 "arguments (%d bytes in excess) " % (result,))
 
-    LoadLibrary = rwin32.LoadLibrary
+if not _WIN32:
+    libc_name = ctypes.util.find_library('c')
+    assert libc_name is not None, "Cannot find C library, ctypes.util.find_library('c') returned None"
 
+    def get_libc_name():
+        return libc_name
+elif _MSVC:
     get_libc_handle = external('pypy_get_libc_handle', [], DLLHANDLE)
 
     def get_libc_name():
@@ -254,7 +266,12 @@
 
     assert "msvcr" in get_libc_name().lower(), \
            "Suspect msvcrt library: %s" % (get_libc_name(),)
+elif _MINGW:
+    def get_libc_name():
+        return 'msvcrt.dll'
 
+if _WIN32:
+    LoadLibrary = rwin32.LoadLibrary
 
 FFI_OK = cConfig.FFI_OK
 FFI_BAD_TYPEDEF = cConfig.FFI_BAD_TYPEDEF
@@ -270,7 +287,7 @@
 
 c_ffi_prep_cif = external('ffi_prep_cif', [FFI_CIFP, ffi_abi, rffi.UINT,
                                            FFI_TYPE_P, FFI_TYPE_PP], rffi.INT)
-if _WIN32:
+if _MSVC:
     c_ffi_call_return_type = rffi.INT
 else:
     c_ffi_call_return_type = lltype.Void

From commits-noreply at bitbucket.org  Mon Feb  7 17:53:55 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Mon,  7 Feb 2011 17:53:55 +0100 (CET)
Subject: [pypy-svn] pypy default: Mingw compiler does not provide
	replacement functions	for getnameinfo() &co
Message-ID: <20110207165355.BD955282BE7@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41676:a1b1dc7fbbba
Date: 2011-02-07 17:51 +0100
http://bitbucket.org/pypy/pypy/changeset/a1b1dc7fbbba/

Log:	Mingw compiler does not provide replacement functions for
	getnameinfo() &co Enable these functions provided by the Windows
	kernel since Windows XP.

	pypy compiled with mingw won't run on Windows 2000, we probably
	don't care.

diff --git a/pypy/rlib/_rsocket_rffi.py b/pypy/rlib/_rsocket_rffi.py
--- a/pypy/rlib/_rsocket_rffi.py
+++ b/pypy/rlib/_rsocket_rffi.py
@@ -69,6 +69,10 @@
     else: # MINGW
         includes = ('stdint.h',)
         header_lines.extend([
+            '''\
+            #ifndef _WIN32_WINNT
+            #define _WIN32_WINNT 0x0501
+            #endif''',
             '#define SIO_RCVALL             _WSAIOW(IOC_VENDOR,1)',
             '#define SIO_KEEPALIVE_VALS     _WSAIOW(IOC_VENDOR,4)',
             '#define RCVALL_OFF             0',

From commits-noreply at bitbucket.org  Mon Feb  7 17:53:56 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Mon,  7 Feb 2011 17:53:56 +0100 (CET)
Subject: [pypy-svn] pypy default: Merge heads
Message-ID: <20110207165356.09929282BE8@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41677:730e3ce23e7a
Date: 2011-02-07 17:53 +0100
http://bitbucket.org/pypy/pypy/changeset/730e3ce23e7a/

Log:	Merge heads


From commits-noreply at bitbucket.org  Mon Feb  7 18:14:28 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon,  7 Feb 2011 18:14:28 +0100 (CET)
Subject: [pypy-svn] pypy default: Add another include for memcpy() here.
Message-ID: <20110207171428.CF38C36C231@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41678:b0e07371559d
Date: 2011-02-07 18:02 +0100
http://bitbucket.org/pypy/pypy/changeset/b0e07371559d/

Log:	Add another include for memcpy() here.

diff --git a/pypy/translator/c/src/obmalloc.c b/pypy/translator/c/src/obmalloc.c
--- a/pypy/translator/c/src/obmalloc.c
+++ b/pypy/translator/c/src/obmalloc.c
@@ -1,6 +1,8 @@
 
 #ifdef WITH_PYMALLOC
 
+#include <string.h>
+
 /* An object allocator for Python.
 
    Here is an introduction to the layers of the Python memory architecture,

From commits-noreply at bitbucket.org  Mon Feb  7 18:14:30 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon,  7 Feb 2011 18:14:30 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix a case where we need to get an integer
	instead of a	string.
Message-ID: <20110207171430.64CDE282BE9@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41679:0f4acf9a7c84
Date: 2011-02-07 18:13 +0100
http://bitbucket.org/pypy/pypy/changeset/0f4acf9a7c84/

Log:	Fix a case where we need to get an integer instead of a string.

diff --git a/pypy/objspace/std/newformat.py b/pypy/objspace/std/newformat.py
--- a/pypy/objspace/std/newformat.py
+++ b/pypy/objspace/std/newformat.py
@@ -247,6 +247,7 @@
         return w_obj
 
     def formatter_field_name_split(self):
+        space = self.space
         name = self.template
         i = 0
         end = len(name)
@@ -255,11 +256,19 @@
             if c == "[" or c == ".":
                 break
             i += 1
+        if i == 0:
+            index = -1
+        else:
+            index = _parse_int(self.space, name, 0, i)[0]
+        if index >= 0:
+            w_first = space.wrap(index)
+        else:
+            w_first = space.wrap(name[:i])
+        #
         self.parser_list_w = []
         self._resolve_lookups(None, name, i, end)
         #
-        space = self.space
-        return space.newtuple([space.wrap(name[:i]),
+        return space.newtuple([w_first,
                                space.iter(space.newlist(self.parser_list_w))])
 
     def _convert(self, w_obj, conversion):

diff --git a/pypy/objspace/std/test/test_newformat.py b/pypy/objspace/std/test/test_newformat.py
--- a/pypy/objspace/std/test/test_newformat.py
+++ b/pypy/objspace/std/test/test_newformat.py
@@ -361,6 +361,14 @@
             assert isinstance(x, unicode)
 
     def test_formatter_field_name_split(self):
+        first, rest = ''._formatter_field_name_split()
+        assert first == ''
+        assert list(rest) == []
+        #
+        first, rest = '31'._formatter_field_name_split()
+        assert first == 31
+        assert list(rest) == []
+        #
         first, rest = 'foo'._formatter_field_name_split()
         assert first == 'foo'
         assert list(rest) == []

From commits-noreply at bitbucket.org  Mon Feb  7 18:25:46 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon,  7 Feb 2011 18:25:46 +0100 (CET)
Subject: [pypy-svn] pypy default: use int(int(frac)) as a workaround for a
	CPython bug:
Message-ID: <20110207172546.2925E282BE7@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41680:9eda9264b36a
Date: 2011-02-07 18:25 +0100
http://bitbucket.org/pypy/pypy/changeset/9eda9264b36a/

Log:	use int(int(frac)) as a workaround for a CPython bug: with frac ==
	2147483647.0, int(frac) == 2147483647L

diff --git a/pypy/rlib/test/test_rbigint.py b/pypy/rlib/test/test_rbigint.py
--- a/pypy/rlib/test/test_rbigint.py
+++ b/pypy/rlib/test/test_rbigint.py
@@ -228,6 +228,9 @@
         x = 12345.6789e200
         x *= x
         assert raises(OverflowError, rbigint.fromfloat, x)
+        #
+        f1 = rbigint.fromfloat(9007199254740991.0)
+        assert f1.tolong() == 9007199254740991
 
     def test_eq(self):
         x = 5858393919192332223L

diff --git a/pypy/rlib/rbigint.py b/pypy/rlib/rbigint.py
--- a/pypy/rlib/rbigint.py
+++ b/pypy/rlib/rbigint.py
@@ -161,25 +161,25 @@
 
     def fromfloat(dval):
         """ Create a new bigint object from a float """
-        neg = 0
+        sign = 1
         if isinf(dval) or isnan(dval):
             raise OverflowError
         if dval < 0.0:
-            neg = 1
+            sign = -1
             dval = -dval
         frac, expo = math.frexp(dval) # dval = frac*2**expo; 0.0 <= frac < 1.0
         if expo <= 0:
             return rbigint()
         ndig = (expo-1) // SHIFT + 1 # Number of 'digits' in result
-        v = rbigint([NULLDIGIT] * ndig, 1)
+        v = rbigint([NULLDIGIT] * ndig, sign)
         frac = math.ldexp(frac, (expo-1) % SHIFT + 1)
         for i in range(ndig-1, -1, -1):
-            bits = int(frac)
+            # use int(int(frac)) as a workaround for a CPython bug:
+            # with frac == 2147483647.0, int(frac) == 2147483647L
+            bits = int(int(frac))
             v.setdigit(i, bits)
             frac -= float(bits)
             frac = math.ldexp(frac, SHIFT)
-        if neg:
-            v.sign = -1
         return v
     fromfloat = staticmethod(fromfloat)
 

From cfbolz at codespeak.net  Mon Feb  7 18:42:33 2011
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Mon,  7 Feb 2011 18:42:33 +0100 (CET)
Subject: [pypy-svn] r80307 - in pypy/extradoc/talk/ibm-feb-2011: . figures
Message-ID: <20110207174233.A61E7282BE7@codespeak.net>

Author: cfbolz
Date: Mon Feb  7 18:42:32 2011
New Revision: 80307

Added:
   pypy/extradoc/talk/ibm-feb-2011/figures/output1.pdf   (contents, props changed)
   pypy/extradoc/talk/ibm-feb-2011/figures/output2.pdf   (contents, props changed)
   pypy/extradoc/talk/ibm-feb-2011/figures/output3.pdf   (contents, props changed)
   pypy/extradoc/talk/ibm-feb-2011/figures/output4.pdf   (contents, props changed)
   pypy/extradoc/talk/ibm-feb-2011/figures/output5.pdf   (contents, props changed)
Modified:
   pypy/extradoc/talk/ibm-feb-2011/talk.pdf
   pypy/extradoc/talk/ibm-feb-2011/talk.tex
Log:
replace requirements stuff with a table


Added: pypy/extradoc/talk/ibm-feb-2011/figures/output1.pdf
==============================================================================
Binary file. No diff available.

Added: pypy/extradoc/talk/ibm-feb-2011/figures/output2.pdf
==============================================================================
Binary file. No diff available.

Added: pypy/extradoc/talk/ibm-feb-2011/figures/output3.pdf
==============================================================================
Binary file. No diff available.

Added: pypy/extradoc/talk/ibm-feb-2011/figures/output4.pdf
==============================================================================
Binary file. No diff available.

Added: pypy/extradoc/talk/ibm-feb-2011/figures/output5.pdf
==============================================================================
Binary file. No diff available.

Modified: pypy/extradoc/talk/ibm-feb-2011/talk.pdf
==============================================================================
Binary files. No diff available.

Modified: pypy/extradoc/talk/ibm-feb-2011/talk.tex
==============================================================================
--- pypy/extradoc/talk/ibm-feb-2011/talk.tex	(original)
+++ pypy/extradoc/talk/ibm-feb-2011/talk.tex	Mon Feb  7 18:42:32 2011
@@ -44,8 +44,7 @@
 
 \date{
 %\includegraphics[scale=0.25]{figures/hpi_logo_kl.jpg}\\
-%IBM Watson Research Center, February 9th, 2011
-Microsoft Research, January 31st, 2011
+IBM Watson Research Center, February 8th, 2011
 }
 
 %\pgfdeclareimage[height=0.5cm]{pypy-logo}{image/py-web.png}
@@ -384,15 +383,16 @@
 
 \begin{frame}
   \frametitle{How do Interpreters Meet the Requirements?}
-  \begin{enumerate}
-      \item \alert{control flow slowed by bytecode or AST dispatch overhead}
-      \item \alert{late binding works but slow}
-      \item \alert{dispatching works but slow}
-      \item \alert{no special dependencies support}
-      \item \alert{everything is boxed}
-      \item \alert{reified frames are easy but slow}
-%      \item \alert{access to native libraries easy but slow}
-  \end{enumerate}
+  \includegraphics[scale=0.6]{figures/output1.pdf}
+%  \begin{enumerate}
+%      \item \alert{control flow slowed by bytecode or AST dispatch overhead}
+%      \item \alert{late binding works but slow}
+%      \item \alert{dispatching works but slow}
+%      \item \alert{no special dependencies support}
+%      \item \alert{everything is boxed}
+%      \item \alert{reified frames are easy but slow}
+%%      \item \alert{access to native libraries easy but slow}
+%  \end{enumerate}
 \end{frame}
 
 
@@ -422,15 +422,16 @@
 
 \begin{frame}
   \frametitle{How do Static Compilers Meet the Requirements?}
-  \begin{enumerate}
-      \item control flow works well
-      \item \alert{late binding not improved}
-      \item \alert{dispatching not improved}
-      \item \alert{dependencies not improved}
-      \item \alert{everything is boxed}
-      \item \alert{reified frames often not supported}
-%      \item direct access to native libraries
-  \end{enumerate}
+  \includegraphics[scale=0.6]{figures/output2.pdf}
+%  \begin{enumerate}
+%      \item control flow works well
+%      \item \alert{late binding not improved}
+%      \item \alert{dispatching not improved}
+%      \item \alert{dependencies not improved}
+%      \item \alert{everything is boxed}
+%      \item \alert{reified frames often not supported}
+%%      \item direct access to native libraries
+%  \end{enumerate}
 \end{frame}
 
 
@@ -513,15 +514,16 @@
 
 \begin{frame}
   \frametitle{How do Method-Based JIT Compilers Meet the Requirements?}
-  \begin{enumerate}
-      \item control flow works well
-      \item late binding can be handled
-      \item dispatching can be handled
-      \item \alert{dependencies not necessarily improved}
-      \item boxing hard to support
-      \item \alert{reified frames hard to support}
-%      \item \alert{access to native libraries not improved}
-  \end{enumerate}
+  \includegraphics[scale=0.6]{figures/output3.pdf}
+%  \begin{enumerate}
+%      \item control flow works well
+%      \item late binding can be handled
+%      \item dispatching can be handled
+%      \item \alert{dependencies not necessarily improved}
+%      \item boxing hard to support
+%      \item \alert{reified frames hard to support}
+%%      \item \alert{access to native libraries not improved}
+%  \end{enumerate}
 \end{frame}
 
 
@@ -671,16 +673,17 @@
 
 \begin{frame}
   \frametitle{How do Tracing JITs Meet the Requirements?}
-  \begin{enumerate}
-      \item control flow works rather well
-      \item late binding can be handled
-      \item dispatching can be handled
-      \item dependencies improved by path splitting
-      \item unboxing optimization much simpler
-      \item reified frames can be implemented \\
-      by falling back to the interpreter
-%      \item \alert{access to native libraries not improved}
-  \end{enumerate}
+  \includegraphics[scale=0.6]{figures/output4.pdf}
+%  \begin{enumerate}
+%      \item control flow works rather well
+%      \item late binding can be handled
+%      \item dispatching can be handled
+%      \item dependencies improved by path splitting
+%      \item unboxing optimization much simpler
+%      \item reified frames can be implemented \\
+%      by falling back to the interpreter
+%%      \item \alert{access to native libraries not improved}
+%  \end{enumerate}
 \end{frame}
 
 
@@ -770,15 +773,16 @@
 
 \begin{frame}
   \frametitle{How do OO VMs Meet the Requirements?}
-  \begin{enumerate}
-      \item control flow works well
-      \item late binding can be handled with a lot of effort
-      \item dispatching can be handled with a lot of effort
-      \item \alert{dependencies not improved}
-      \item \alert{boxing not necessarily improved}
-      \item \alert{reified frames are inefficient}
-%      \item access to native libraries not improved, but VM libraries help
-  \end{enumerate}
+  \includegraphics[scale=0.6]{figures/output5.pdf}
+%  \begin{enumerate}
+%      \item control flow works well
+%      \item late binding can be handled with a lot of effort
+%      \item dispatching can be handled with a lot of effort
+%      \item \alert{dependencies not improved}
+%      \item \alert{boxing not necessarily improved}
+%      \item \alert{reified frames are inefficient}
+%%      \item access to native libraries not improved, but VM libraries help
+%  \end{enumerate}
 \end{frame}
 
 \section{PyPy's Approach to VM Construction}


From commits-noreply at bitbucket.org  Mon Feb  7 19:22:21 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon,  7 Feb 2011 19:22:21 +0100 (CET)
Subject: [pypy-svn] pypy default: Some more tests for round(), and a fix.
Message-ID: <20110207182221.8224B36C226@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41681:2668082371d2
Date: 2011-02-07 19:21 +0100
http://bitbucket.org/pypy/pypy/changeset/2668082371d2/

Log:	Some more tests for round(), and a fix.

diff --git a/pypy/module/__builtin__/test/test_builtin.py b/pypy/module/__builtin__/test/test_builtin.py
--- a/pypy/module/__builtin__/test/test_builtin.py
+++ b/pypy/module/__builtin__/test/test_builtin.py
@@ -646,10 +646,23 @@
         assert res == 18
 
     def test_round(self):
+        assert round(11.234) == 11.0
+        assert round(11.234, -1) == 10.0
+        assert round(11.234, 0) == 11.0
+        assert round(11.234, 1) == 11.2
+        #
         assert round(5e15-1) == 5e15-1
         assert round(5e15) == 5e15
         assert round(-(5e15-1)) == -(5e15-1)
         assert round(-5e15) == -5e15
+        #
+        inf = 1e200 * 1e200
+        assert round(inf) == inf
+        assert round(-inf) == -inf
+        nan = inf / inf
+        assert repr(round(nan)) == repr(nan)
+        #
+        raises(OverflowError, round, 1.6e308, -308)
 
     def test_vars_obscure_case(self):
         class C_get_vars(object):

diff --git a/pypy/module/__builtin__/operation.py b/pypy/module/__builtin__/operation.py
--- a/pypy/module/__builtin__/operation.py
+++ b/pypy/module/__builtin__/operation.py
@@ -182,6 +182,9 @@
         z = (z / pow2) / pow1
     else:
         z *= pow1
+    if isinf(z):
+        raise OperationError(space.w_OverflowError,
+                            space.wrap("rounded value too large to represent"))
     return space.wrap(z)
 #
 round.unwrap_spec = [ObjSpace, float, W_Root]

From commits-noreply at bitbucket.org  Mon Feb  7 19:50:45 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon,  7 Feb 2011 19:50:45 +0100 (CET)
Subject: [pypy-svn] pypy default: Update to not check for the exact error
	message on	non-CPython
Message-ID: <20110207185045.8221336C394@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41682:f9a321017a4d
Date: 2011-02-07 19:30 +0100
http://bitbucket.org/pypy/pypy/changeset/f9a321017a4d/

Log:	Update to not check for the exact error message on non-CPython
	implementations.

diff --git a/lib-python/2.7.0/test/test_format.py b/lib-python/modified-2.7.0/test/test_format.py
copy from lib-python/2.7.0/test/test_format.py
copy to lib-python/modified-2.7.0/test/test_format.py
--- a/lib-python/2.7.0/test/test_format.py
+++ b/lib-python/modified-2.7.0/test/test_format.py
@@ -242,7 +242,7 @@
             try:
                 testformat(formatstr, args)
             except exception, exc:
-                if str(exc) == excmsg:
+                if str(exc) == excmsg or not test_support.check_impl_detail():
                     if verbose:
                         print "yes"
                 else:

From commits-noreply at bitbucket.org  Mon Feb  7 19:50:46 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon,  7 Feb 2011 19:50:46 +0100 (CET)
Subject: [pypy-svn] pypy default: On CPython,
	__oct__() is called if Foobar inherits from 'long', 
Message-ID: <20110207185046.1AC2F36C394@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41683:30f40ccba8de
Date: 2011-02-07 19:50 +0100
http://bitbucket.org/pypy/pypy/changeset/30f40ccba8de/

Log:	On CPython, __oct__() is called if Foobar inherits from 'long', but
	not, say, 'object' or 'int' or 'str'. This seems strange enough to
	consider it a complete implementation detail (or really a bug, if
	you prefer).

diff --git a/lib-python/modified-2.7.0/test/test_format.py b/lib-python/modified-2.7.0/test/test_format.py
--- a/lib-python/modified-2.7.0/test/test_format.py
+++ b/lib-python/modified-2.7.0/test/test_format.py
@@ -272,13 +272,16 @@
         test_exc(u'no format', u'1', TypeError,
                  "not all arguments converted during string formatting")
 
-        class Foobar(long):
-            def __oct__(self):
-                # Returning a non-string should not blow up.
-                return self + 1
-
-        test_exc('%o', Foobar(), TypeError,
-                 "expected string or Unicode object, long found")
+        if test_support.check_impl_detail():
+            # __oct__() is called if Foobar inherits from 'long', but
+            # not, say, 'object' or 'int' or 'str'.  This seems strange
+            # enough to consider it a complete implementation detail.
+            class Foobar(long):
+                def __oct__(self):
+                    # Returning a non-string should not blow up.
+                    return self + 1
+            test_exc('%o', Foobar(), TypeError,
+                     "expected string or Unicode object, long found")
 
         if maxsize == 2**31-1:
             # crashes 2.2.1 and earlier:

From commits-noreply at bitbucket.org  Mon Feb  7 20:15:16 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon,  7 Feb 2011 20:15:16 +0100 (CET)
Subject: [pypy-svn] pypy default: On Win64 running a (32-bit) PyPy we get
	'AMD64'.
Message-ID: <20110207191516.46F75282BE7@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41684:42457f587e82
Date: 2011-02-07 20:14 +0100
http://bitbucket.org/pypy/pypy/changeset/42457f587e82/

Log:	On Win64 running a (32-bit) PyPy we get 'AMD64'.

diff --git a/pypy/jit/backend/detect_cpu.py b/pypy/jit/backend/detect_cpu.py
--- a/pypy/jit/backend/detect_cpu.py
+++ b/pypy/jit/backend/detect_cpu.py
@@ -32,7 +32,8 @@
                 'x86':   'x86',    # Apple
                 'Power Macintosh': 'ppc',
                 'x86_64': 'x86',
-                'amd64': 'x86'     # freebsd
+                'amd64': 'x86',    # freebsd
+                'AMD64': 'x86',    # win64
                 }[mach]
     except KeyError:
         return mach

From commits-noreply at bitbucket.org  Mon Feb  7 22:01:33 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Mon,  7 Feb 2011 22:01:33 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix test_fcntl,
	by casting all constants to signed longs.
Message-ID: <20110207210133.355D0282BE7@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41685:d6115a952624
Date: 2011-02-07 22:00 +0100
http://bitbucket.org/pypy/pypy/changeset/d6115a952624/

Log:	Fix test_fcntl, by casting all constants to signed longs.

diff --git a/pypy/module/fcntl/test/test_fcntl.py b/pypy/module/fcntl/test/test_fcntl.py
--- a/pypy/module/fcntl/test/test_fcntl.py
+++ b/pypy/module/fcntl/test/test_fcntl.py
@@ -196,3 +196,8 @@
         import fcntl
         f = open(self.tmp, "w")
         fcntl.lockf(f, fcntl.LOCK_EX | fcntl.LOCK_NB)
+
+    def test_large_flag(self):
+        import fcntl
+        assert fcntl.DN_MULTISHOT < 0
+        fcntl.fcntl(0, fcntl.F_NOTIFY, fcntl.DN_MULTISHOT)

diff --git a/pypy/module/fcntl/__init__.py b/pypy/module/fcntl/__init__.py
--- a/pypy/module/fcntl/__init__.py
+++ b/pypy/module/fcntl/__init__.py
@@ -1,4 +1,5 @@
 from pypy.interpreter.mixedmodule import MixedModule
+from pypy.rlib.rarithmetic import intmask
 
 class Module(MixedModule):
     interpleveldefs = {
@@ -16,6 +17,7 @@
     def buildloaders(cls):
         from pypy.module.fcntl import interp_fcntl
         for constant, value in interp_fcntl.constants.iteritems():
+            value = intmask(value)
             Module.interpleveldefs[constant] = "space.wrap(%r)" % value
         super(Module, cls).buildloaders()
     buildloaders = classmethod(buildloaders)

From commits-noreply at bitbucket.org  Mon Feb  7 22:33:51 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon,  7 Feb 2011 22:33:51 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix the failing test on 64bit by
	simplifying the	functions
Message-ID: <20110207213351.5B610282BE7@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41686:13af86f13f3c
Date: 2011-02-07 22:31 +0100
http://bitbucket.org/pypy/pypy/changeset/13af86f13f3c/

Log:	Fix the failing test on 64bit by simplifying the functions so that
	they no longer access the internals of rbigint. We no longer have a
	special fast case for SHIFT=15, but anyway nowadays we use SHIFT=31
	systematically.

diff --git a/pypy/objspace/std/marshal_impl.py b/pypy/objspace/std/marshal_impl.py
--- a/pypy/objspace/std/marshal_impl.py
+++ b/pypy/objspace/std/marshal_impl.py
@@ -10,9 +10,8 @@
 
 from pypy.interpreter.error import OperationError
 from pypy.objspace.std.register_all import register_all
-from pypy.rlib.rarithmetic import LONG_BIT, r_longlong, r_uint
+from pypy.rlib.rarithmetic import LONG_BIT, r_longlong, r_uint, intmask
 from pypy.objspace.std import model
-from pypy.objspace.std.longobject import SHIFT as long_bits
 from pypy.interpreter.special import Ellipsis
 from pypy.interpreter.pycode import PyCode
 from pypy.interpreter import gateway, unicodehelper
@@ -209,62 +208,37 @@
 def marshal_w__Long(space, w_long, m):
     from pypy.rlib.rbigint import rbigint
     m.start(TYPE_LONG)
-    # XXX access internals
-    if long_bits != 15:
-        SHIFT = 15
-        MASK = (1 << SHIFT) - 1
-        BIGMASK = rbigint.fromint(MASK)
-        num = w_long.num
-        sign = num.sign
-        num = num.abs()
-        ints = []
-        while num.tobool():
-            next = num.and_(BIGMASK).toint()
-            ints.append(next)
-            num = num.rshift(SHIFT)
-        m.put_int(len(ints) * sign)
-        for i in ints:
-            m.put_short(i)
-        return
-    lng = len(w_long.num.digits)
-    if w_long.num.sign < 0:
-        m.put_int(-lng)
-    else:
-        m.put_int(lng)
-    for digit in w_long.num.digits:
-        m.put_short(digit)
+    SHIFT = 15
+    MASK = (1 << SHIFT) - 1
+    num = w_long.num
+    sign = num.sign
+    num = num.abs()
+    ints = []
+    while num.tobool():
+        next = intmask(num.uintmask() & MASK)
+        ints.append(next)
+        num = num.rshift(SHIFT)
+    m.put_int(len(ints) * sign)
+    for i in ints:
+        m.put_short(i)
 
 def unmarshal_Long(space, u, tc):
-    # XXX access internals
     from pypy.rlib.rbigint import rbigint
     lng = u.get_int()
     if lng < 0:
-        sign = -1
+        negative = True
         lng = -lng
-    elif lng > 0:
-        sign = 1
     else:
-        sign = 0
-    if long_bits != 15:
-        SHIFT = 15
-        result = rbigint([0], 0)
-        for i in range(lng):
-            shift = i * SHIFT
-            result = result.add(rbigint.fromint(u.get_short()).lshift(shift))
-        if lng and not result.tobool():
-            raise_exception(space, 'bad marshal data')
-        if sign == -1:
-            result = result.neg()
-    else:
-        digits = [0] * lng
-        for i in range(lng):
-            digit = u.get_int()
-            if digit < 0:
-                raise_exception(space, 'bad marshal data')
-            digits[i] = digit
-        if digits[-1] == 0:
-            raise_exception(space, 'bad marshal data')
-        result = rbigint(digits, sign)
+        negative = False
+    SHIFT = 15
+    result = rbigint.fromint(0)
+    for i in range(lng):
+        shift = i * SHIFT
+        result = result.or_(rbigint.fromint(u.get_short()).lshift(shift))
+    if lng and not result.tobool():
+        raise_exception(space, 'bad marshal data')
+    if negative:
+        result = result.neg()
     w_long = newlong(space, result)
     return w_long
 register(TYPE_LONG, unmarshal_Long)

From commits-noreply at bitbucket.org  Mon Feb  7 22:33:51 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon,  7 Feb 2011 22:33:51 +0100 (CET)
Subject: [pypy-svn] pypy default: merge heads
Message-ID: <20110207213351.96DE6282BE8@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41687:dfda39dc7634
Date: 2011-02-07 22:33 +0100
http://bitbucket.org/pypy/pypy/changeset/dfda39dc7634/

Log:	merge heads


From commits-noreply at bitbucket.org  Mon Feb  7 22:37:56 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Mon,  7 Feb 2011 22:37:56 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix format('n') to correctly use the
	thousands grouping	indicated by the locale
Message-ID: <20110207213756.E6B6B282BE7@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41688:a32d795de509
Date: 2011-02-07 22:37 +0100
http://bitbucket.org/pypy/pypy/changeset/a32d795de509/

Log:	Fix format('n') to correctly use the thousands grouping indicated by
	the locale

diff --git a/pypy/objspace/std/newformat.py b/pypy/objspace/std/newformat.py
--- a/pypy/objspace/std/newformat.py
+++ b/pypy/objspace/std/newformat.py
@@ -863,6 +863,7 @@
             msg = "alternate form not allowed in float formats"
             raise OperationError(space.w_ValueError, space.wrap(msg))
         tp = self._type
+        self._get_locale(tp)
         if tp == "\0":
             tp = "g"
             default_precision = 12
@@ -892,7 +893,6 @@
             to_number = 0
         have_dec_point, to_remainder = self._parse_number(result, to_number)
         n_remainder = len(result) - to_remainder
-        self._get_locale(tp)
         if self.is_unicode:
             digits = result.decode("ascii")
         else:

diff --git a/pypy/objspace/std/test/test_newformat.py b/pypy/objspace/std/test/test_newformat.py
--- a/pypy/objspace/std/test/test_newformat.py
+++ b/pypy/objspace/std/test/test_newformat.py
@@ -1,5 +1,6 @@
 """Test unicode/str's format method"""
 from __future__ import with_statement
+from pypy.conftest import gettestobjspace
 
 
 class BaseStringFormatTests:
@@ -291,6 +292,8 @@
 
 
 class AppTestFloatFormatting:
+    def setup_class(cls):
+        cls.space = gettestobjspace(usemodules=('_locale',))
 
     def test_alternate(self):
         raises(ValueError, format, 1.0, "#")
@@ -313,6 +316,16 @@
     def test_digit_separator(self):
         assert format(-1234., "012,f") == "-1,234.000000"
 
+    def test_locale(self):
+        import locale
+        locale.setlocale(locale.LC_NUMERIC, 'en_US.UTF8')
+        x = 1234.567890
+        try:
+            assert locale.format('%g', x, grouping=True) == '1,234.57'
+            assert format(x, 'n') == '1,234.57'
+        finally:
+            locale.setlocale(locale.LC_NUMERIC, 'C')
+
     def test_dont_switch_to_g(self):
         skip("must fix when float formatting is figured out")
         assert len(format(1.1234e90, "f")) == 98

From commits-noreply at bitbucket.org  Mon Feb  7 22:54:06 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon,  7 Feb 2011 22:54:06 +0100 (CET)
Subject: [pypy-svn] pypy default: Test the error message for 'f(**42)',
	and improve and test
Message-ID: <20110207215406.26C2D36C231@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41689:2bbf38842bd1
Date: 2011-02-07 22:52 +0100
http://bitbucket.org/pypy/pypy/changeset/2bbf38842bd1/

Log:	Test the error message for 'f(**42)', and improve and test the error
	message for 'f(*42)'.

diff --git a/pypy/interpreter/argument.py b/pypy/interpreter/argument.py
--- a/pypy/interpreter/argument.py
+++ b/pypy/interpreter/argument.py
@@ -147,8 +147,19 @@
 
     def _combine_starargs_wrapped(self, w_stararg):
         # unpack the * arguments 
-        self.arguments_w = (self.arguments_w +
-                            self.space.fixedview(w_stararg))
+        space = self.space
+        try:
+            args_w = space.fixedview(w_stararg)
+        except OperationError, e:
+            if e.match(space, space.w_TypeError):
+                w_type = space.type(w_stararg)
+                typename = w_type.getname(space)
+                raise OperationError(
+                    space.w_TypeError,
+                    space.wrap("argument after * must be "
+                               "a sequence, not %s" % (typename,)))
+            raise
+        self.arguments_w = self.arguments_w + args_w
 
     def _combine_starstarargs_wrapped(self, w_starstararg):
         # unpack the ** arguments
@@ -163,7 +174,7 @@
             except OperationError, e:
                 if e.match(space, space.w_AttributeError):
                     w_type = space.type(w_starstararg)
-                    typename = w_type.getname(space, '?')
+                    typename = w_type.getname(space)
                     raise OperationError(
                         space.w_TypeError,
                         space.wrap("argument after ** must be "

diff --git a/pypy/interpreter/test/test_argument.py b/pypy/interpreter/test/test_argument.py
--- a/pypy/interpreter/test/test_argument.py
+++ b/pypy/interpreter/test/test_argument.py
@@ -117,7 +117,7 @@
 
     def type(self, obj):
         class Type:
-            def getname(self, space, default):
+            def getname(self, space, default='?'):
                 return type(obj).__name__
         return Type()
 
@@ -500,6 +500,22 @@
         s = err.getmsg('foo')
         assert s == "foo() takes exactly 1 argument (3 given)"
 
+    def test_bad_type_for_star(self):
+        space = self.space
+        try:
+            Arguments(space, [], w_stararg=space.wrap(42))
+        except OperationError, e:
+            msg = space.str_w(space.str(e.get_w_value(space)))
+            assert msg == "argument after * must be a sequence, not int"
+        else:
+            assert 0, "did not raise"
+        try:
+            Arguments(space, [], w_starstararg=space.wrap(42))
+        except OperationError, e:
+            msg = space.str_w(space.str(e.get_w_value(space)))
+            assert msg == "argument after ** must be a mapping, not int"
+        else:
+            assert 0, "did not raise"
 
     def test_unknown_keywords(self):
         err = ArgErrUnknownKwds(1, ['a', 'b'], [True, False])

From cfbolz at codespeak.net  Mon Feb  7 23:06:37 2011
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Mon,  7 Feb 2011 23:06:37 +0100 (CET)
Subject: [pypy-svn] r80308 - in pypy/extradoc/talk/ibm-feb-2011: . figures
Message-ID: <20110207220637.41283282BE7@codespeak.net>

Author: cfbolz
Date: Mon Feb  7 23:06:35 2011
New Revision: 80308

Added:
   pypy/extradoc/talk/ibm-feb-2011/figures/add1.dot
   pypy/extradoc/talk/ibm-feb-2011/figures/add1.pdf   (contents, props changed)
   pypy/extradoc/talk/ibm-feb-2011/figures/add2.dot
   pypy/extradoc/talk/ibm-feb-2011/figures/add2.pdf   (contents, props changed)
   pypy/extradoc/talk/ibm-feb-2011/figures/add_fast.dot
   pypy/extradoc/talk/ibm-feb-2011/figures/add_fast.pdf   (contents, props changed)
Modified:
   pypy/extradoc/talk/ibm-feb-2011/figures/add.dot
   pypy/extradoc/talk/ibm-feb-2011/figures/add.pdf
   pypy/extradoc/talk/ibm-feb-2011/talk.pdf
   pypy/extradoc/talk/ibm-feb-2011/talk.tex
Log:
replace pseudo-code by diagram


Modified: pypy/extradoc/talk/ibm-feb-2011/figures/add.dot
==============================================================================
--- pypy/extradoc/talk/ibm-feb-2011/figures/add.dot	(original)
+++ pypy/extradoc/talk/ibm-feb-2011/figures/add.dot	Mon Feb  7 23:06:35 2011
@@ -21,8 +21,8 @@
     b_int -> long [label="Long"];
     b_int -> user [label="User defined"];
 
-    b_float -> float [label="Integer"];
-    b_long -> long [label="Integer"];
+    b_float -> float [label="Float"];
+    b_long -> long [label="Long"];
 
     int [label="Integer"];
     float [label="Float"];

Modified: pypy/extradoc/talk/ibm-feb-2011/figures/add.pdf
==============================================================================
Binary files. No diff available.

Added: pypy/extradoc/talk/ibm-feb-2011/figures/add1.dot
==============================================================================
--- (empty file)
+++ pypy/extradoc/talk/ibm-feb-2011/figures/add1.dot	Mon Feb  7 23:06:35 2011
@@ -0,0 +1,21 @@
+digraph G {
+    a -> b_int [label="Integer"];
+    a -> b_float [label="Float"];
+
+    a [label="a", shape=box];
+    b_int [label="b", shape=box];
+    b_float [label="b", shape=box];
+
+    b_int -> int [label="Integer"];
+    b_int -> float [label="Float"];
+
+    b_float -> float [label="Float"];
+    b_float -> float [label="Integer"];
+
+    int [label="Integer\laddition"];
+    float [label="Float\laddition"];
+
+    int -> x;
+    float -> x;
+    x [label="x", shape=box];
+}

Added: pypy/extradoc/talk/ibm-feb-2011/figures/add1.pdf
==============================================================================
Binary file. No diff available.

Added: pypy/extradoc/talk/ibm-feb-2011/figures/add2.dot
==============================================================================
--- (empty file)
+++ pypy/extradoc/talk/ibm-feb-2011/figures/add2.dot	Mon Feb  7 23:06:35 2011
@@ -0,0 +1,40 @@
+digraph G {
+    a -> b_int [label="Integer", color=red];
+    a -> b_float [label="Float"];
+
+    a [label="a", shape=box];
+    b_int [label="b", shape=box];
+    b_float [label="b", shape=box];
+
+    b_int -> int [label="Integer", color=red];
+    b_int -> float [label="Float"];
+
+    b_float -> float [label="Float"];
+    b_float -> float [label="Integer"];
+
+    int [label="Integer\laddition"];
+    float [label="Float\laddition"];
+
+    int -> x [color=red];
+    float -> x;
+    x [label="x", shape=box];
+
+    x -> c_int [label="Integer", color=red];
+    x -> c_float [label="Float"];
+
+    c_int [label="c", shape=box];
+    c_float [label="c", shape=box];
+
+    c_int -> int2 [label="Integer", color=red];
+    c_int -> float2 [label="Float"];
+
+    c_float -> float2 [label="Float"];
+    c_float -> float2 [label="Integer"];
+
+    int2 [label="Integer\laddition"];
+    float2 [label="Float\laddition"];
+
+    int2 -> r [color=red];
+    float2 -> r;
+    r [label="r", shape=box];
+}

Added: pypy/extradoc/talk/ibm-feb-2011/figures/add2.pdf
==============================================================================
Binary file. No diff available.

Added: pypy/extradoc/talk/ibm-feb-2011/figures/add_fast.dot
==============================================================================
--- (empty file)
+++ pypy/extradoc/talk/ibm-feb-2011/figures/add_fast.dot	Mon Feb  7 23:06:35 2011
@@ -0,0 +1,35 @@
+digraph G {
+    a -> b_int [label="Integer", color=red];
+    a -> b_float [label="Float"];
+
+    a [label="a", shape=box];
+    b_int [label="b", shape=box];
+    b_float [label="b", shape=box];
+
+    b_int -> int [label="Integer", color=red];
+    b_int -> float [label="Float"];
+
+    b_float -> float [label="Float"];
+    b_float -> float [label="Integer"];
+
+    int [label="Integer\laddition"];
+    float [label="Float\laddition"];
+
+    int -> c_int [color=red];
+    float -> c_float;
+    c_int [label="c", shape=box];
+    c_float [label="c", shape=box];
+
+    c_int -> int2 [label="Integer", color=red];
+    c_int -> float2 [label="Float"];
+
+    c_float -> float2 [label="Float"];
+    c_float -> float2 [label="Integer"];
+
+    int2 [label="Integer\laddition"];
+    float2 [label="Float\laddition"];
+
+    int2 -> r [color=red];
+    float2 -> r;
+    r [label="r", shape=box];
+}

Added: pypy/extradoc/talk/ibm-feb-2011/figures/add_fast.pdf
==============================================================================
Binary file. No diff available.

Modified: pypy/extradoc/talk/ibm-feb-2011/talk.pdf
==============================================================================
Binary files. No diff available.

Modified: pypy/extradoc/talk/ibm-feb-2011/talk.tex
==============================================================================
--- pypy/extradoc/talk/ibm-feb-2011/talk.tex	(original)
+++ pypy/extradoc/talk/ibm-feb-2011/talk.tex	Mon Feb  7 23:06:35 2011
@@ -493,23 +493,32 @@
 
 \begin{frame}[containsverbatim, plain]
   \frametitle{Method-Based JITs and Dispatching Dependencies}
-\begin{verbatim}
-    if isinstance(a, Integer):
-        if isinstance(b, Integer):
-            x = Integer(<perform int addition>)
-        else:
-            x = Float(<perform float addition>)
-    else:
-        x = Float(<perform float addition>)
-    # x can be Float or Integer here
-    if isinstance(x, Integer):
-        if isinstance(c, Integer):
-            r = Integer(<perform int addition>)
-        else:
-            r = Float(<perform float addition>)
-    else:
-        r = Float(<perform float addition>)
-\end{verbatim}
+  \includegraphics[scale=0.5]{figures/add1.pdf}
+%\begin{verbatim}
+%    if (isinstance(a, Integer) and
+%            isinstance(b, Integer)):
+%        x = Integer(<perform int addition>)
+%    else:
+%        x = Float(<perform float addition>)
+%
+%    # x can be Float or Integer here
+%
+%    if (isinstance(x, Integer) and
+%            isinstance(c, Integer)):
+%        r = Integer(<perform int addition>)
+%    else:
+%        r = Float(<perform float addition>)
+%\end{verbatim}
+\end{frame}
+
+\begin{frame}[containsverbatim, plain]
+  \frametitle{Method-Based JITs and Dispatching Dependencies}
+  \includegraphics[scale=0.39]{figures/add2.pdf}
+\end{frame}
+
+\begin{frame}[containsverbatim, plain]
+  \frametitle{Method-Based JITs and Dispatching Dependencies}
+  \includegraphics[scale=0.41]{figures/add_fast.pdf}
 \end{frame}
 
 \begin{frame}


From commits-noreply at bitbucket.org  Mon Feb  7 23:10:35 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon,  7 Feb 2011 23:10:35 +0100 (CET)
Subject: [pypy-svn] pypy default: Add a modified version of this test with
	"..." in the	expected
Message-ID: <20110207221035.75D7F282BE7@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41690:6c67cd695980
Date: 2011-02-07 23:10 +0100
http://bitbucket.org/pypy/pypy/changeset/6c67cd695980/

Log:	Add a modified version of this test with "..." in the expected
	places where there is a difference between CPython and PyPy, which
	are, luckily, not too numerous.

diff --git a/lib-python/2.7.0/test/test_extcall.py b/lib-python/modified-2.7.0/test/test_extcall.py
copy from lib-python/2.7.0/test/test_extcall.py
copy to lib-python/modified-2.7.0/test/test_extcall.py
--- a/lib-python/2.7.0/test/test_extcall.py
+++ b/lib-python/modified-2.7.0/test/test_extcall.py
@@ -90,19 +90,19 @@
 
     >>> class Nothing: pass
     ...
-    >>> g(*Nothing())
+    >>> g(*Nothing())                     #doctest: +ELLIPSIS
     Traceback (most recent call last):
       ...
-    TypeError: g() argument after * must be a sequence, not instance
+    TypeError: ...argument after * must be a sequence, not instance
 
     >>> class Nothing:
     ...     def __len__(self): return 5
     ...
 
-    >>> g(*Nothing())
+    >>> g(*Nothing())                     #doctest: +ELLIPSIS
     Traceback (most recent call last):
       ...
-    TypeError: g() argument after * must be a sequence, not instance
+    TypeError: ...argument after * must be a sequence, not instance
 
     >>> class Nothing():
     ...     def __len__(self): return 5
@@ -154,52 +154,50 @@
       ...
     TypeError: g() got multiple values for keyword argument 'x'
 
-    >>> f(**{1:2})
+    >>> f(**{1:2})                             #doctest: +ELLIPSIS
     Traceback (most recent call last):
       ...
-    TypeError: f() keywords must be strings
+    TypeError: ...keywords must be strings
 
     >>> h(**{'e': 2})
     Traceback (most recent call last):
       ...
     TypeError: h() got an unexpected keyword argument 'e'
 
-    >>> h(*h)
+    >>> h(*h)                                  #doctest: +ELLIPSIS
     Traceback (most recent call last):
       ...
-    TypeError: h() argument after * must be a sequence, not function
+    TypeError: ...argument after * must be a sequence, not function
 
-    >>> dir(*h)
+    >>> dir(*h)                                #doctest: +ELLIPSIS
     Traceback (most recent call last):
       ...
-    TypeError: dir() argument after * must be a sequence, not function
+    TypeError: ...argument after * must be a sequence, not function
 
-    >>> None(*h)
+    >>> None(*h)                               #doctest: +ELLIPSIS
     Traceback (most recent call last):
       ...
-    TypeError: NoneType object argument after * must be a sequence, \
-not function
+    TypeError: ...argument after * must be a sequence, not function
 
-    >>> h(**h)
+    >>> h(**h)                                 #doctest: +ELLIPSIS
     Traceback (most recent call last):
       ...
-    TypeError: h() argument after ** must be a mapping, not function
+    TypeError: ...argument after ** must be a mapping, not function
 
-    >>> dir(**h)
+    >>> dir(**h)                               #doctest: +ELLIPSIS
     Traceback (most recent call last):
       ...
-    TypeError: dir() argument after ** must be a mapping, not function
+    TypeError: ...argument after ** must be a mapping, not function
 
-    >>> None(**h)
+    >>> None(**h)                              #doctest: +ELLIPSIS
     Traceback (most recent call last):
       ...
-    TypeError: NoneType object argument after ** must be a mapping, \
-not function
+    TypeError: ...argument after ** must be a mapping, not function
 
-    >>> dir(b=1, **{'b': 1})
+    >>> dir(b=1, **{'b': 1})                   #doctest: +ELLIPSIS
     Traceback (most recent call last):
       ...
-    TypeError: dir() got multiple values for keyword argument 'b'
+    TypeError: ...got multiple values for keyword argument 'b'
 
 Another helper function
 
@@ -247,10 +245,10 @@
     ...     False
     True
 
-    >>> id(1, **{'foo': 1})
+    >>> id(1, **{'foo': 1})                 #doctest: +ELLIPSIS
     Traceback (most recent call last):
       ...
-    TypeError: id() takes no keyword arguments
+    TypeError: id() ... keyword argument...
 
 A corner case of keyword dictionary items being deleted during
 the function call setup. See <http://bugs.python.org/issue2016>.

From commits-noreply at bitbucket.org  Mon Feb  7 23:26:54 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon,  7 Feb 2011 23:26:54 +0100 (CET)
Subject: [pypy-svn] pypy default: Use globals.setdefault() instead of this
	try:except:	path.
Message-ID: <20110207222654.1A8B2282BE7@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41691:aa2c9f31676f
Date: 2011-02-07 23:26 +0100
http://bitbucket.org/pypy/pypy/changeset/aa2c9f31676f/

Log:	Use globals.setdefault() instead of this try:except: path. 'globals'
	is expected to be some subclass of dict or some class that has all
	dict-like methods anyway --- we can ask this because CPython asks
	more, namely that it is precisely a dict.

diff --git a/pypy/interpreter/pyopcode.py b/pypy/interpreter/pyopcode.py
--- a/pypy/interpreter/pyopcode.py
+++ b/pypy/interpreter/pyopcode.py
@@ -1513,10 +1513,7 @@
         if not isinstance(globals, dict):
             if not hasattr(globals, '__getitem__'):
                 raise TypeError("exec: arg 2 must be a dictionary or None")
-        try:
-            globals['__builtins__']
-        except KeyError:
-            globals['__builtins__'] = builtin
+        globals.setdefault('__builtins__', builtin)
         if not isinstance(locals, dict):
             if not hasattr(locals, '__getitem__'):
                 raise TypeError("exec: arg 3 must be a dictionary or None")

diff --git a/pypy/interpreter/test/test_exec.py b/pypy/interpreter/test/test_exec.py
--- a/pypy/interpreter/test/test_exec.py
+++ b/pypy/interpreter/test/test_exec.py
@@ -185,6 +185,8 @@
                 return key
             def __setitem__(self, key, value):
                 self.result[key] = value
+            def setdefault(self, key, value):
+                assert key == '__builtins__'
         m = M()
         m.result = {}
         exec "x=m" in {}, m

From cfbolz at codespeak.net  Mon Feb  7 23:28:03 2011
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Mon,  7 Feb 2011 23:28:03 +0100 (CET)
Subject: [pypy-svn] r80309 - pypy/extradoc/talk/ibm-feb-2011
Message-ID: <20110207222803.0DB6C282BE7@codespeak.net>

Author: cfbolz
Date: Mon Feb  7 23:28:01 2011
New Revision: 80309

Modified:
   pypy/extradoc/talk/ibm-feb-2011/talk.pdf
   pypy/extradoc/talk/ibm-feb-2011/talk.tex
Log:
add some more concrete examples for traces


Modified: pypy/extradoc/talk/ibm-feb-2011/talk.pdf
==============================================================================
Binary files. No diff available.

Modified: pypy/extradoc/talk/ibm-feb-2011/talk.tex
==============================================================================
--- pypy/extradoc/talk/ibm-feb-2011/talk.tex	(original)
+++ pypy/extradoc/talk/ibm-feb-2011/talk.tex	Mon Feb  7 23:28:01 2011
@@ -24,8 +24,8 @@
 \usepackage{wrapfig}
 \usepackage{url}
 \usepackage{alltt}
-\usepackage{tikz}
-\usetikzlibrary{arrows,positioning,shadows,shapes,calc,backgrounds,fit}
+
+\newcommand\redsout[1]{{\color{red}\sout{\hbox{\color{black}{#1}}}}}
 
 %\input{pygmentheader.tex}
 
@@ -638,6 +638,26 @@
   \end{itemize}
 \end{frame}
 
+\begin{frame}[containsverbatim]
+  \frametitle{Example: Dispatching in a Tracing JIT}
+\begin{verbatim}
+x = ADD(a : Integer, b : Integer)
+\end{verbatim}
+\end{frame}
+
+\begin{frame}[containsverbatim]
+  \frametitle{Example: Dispatching in a Tracing JIT}
+\begin{verbatim}
+guard_class(a, Integer)
+guard_class(b, Integer)
+u_a = unbox(a)
+u_b = unbox(b)
+u_x = int_add(a, b)
+x = new(Integer, u_x)
+\end{verbatim}
+\end{frame}
+
+
 
 \begin{frame}
   \frametitle{Dispatching Dependencies in a Tracing JIT}
@@ -650,6 +670,44 @@
   \end{itemize}
 \end{frame}
 
+\begin{frame}[containsverbatim]
+  \frametitle{Example: Dependencies in a Tracing JIT}
+\begin{alltt}
+guard_class(a, Integer)
+guard_class(b, Integer)
+u_a = unbox(a)
+u_b = unbox(b)
+u_x = int_add(u_a, u_b)
+x = new(Integer, u_x)
+
+guard_class(x, Integer)
+guard_class(c, Integer)
+u_x2 = unbox(x)
+u_c = unbox(c)
+u_r = int_add(u_x2, u_c)
+r = new(Integer, u_r)
+\end{alltt}
+\end{frame}
+
+\begin{frame}[containsverbatim]
+  \frametitle{Example: Dependencies in a Tracing JIT}
+\begin{alltt}
+guard_class(a, Integer)
+guard_class(b, Integer)
+u_a = unbox(a)
+u_b = unbox(b)
+u_x = int_add(u_a, u_b)
+x = new(Integer, u_x)
+
+\redsout{guard_class(x, Integer)}
+guard_class(c, Integer)
+u_x2 = unbox(x)
+u_c = unbox(c)
+u_r = int_add(u_x2, u_c)
+r = new(Integer, u_r)
+\end{alltt}
+\end{frame}
+
 \begin{frame}
   \frametitle{Boxing Optimizations in a Tracing JIT}
   \begin{itemize}
@@ -659,6 +717,41 @@
   \end{itemize}
 \end{frame}
 
+\begin{frame}[containsverbatim]
+  \frametitle{Example: Boxing in a Tracing JIT}
+\begin{alltt}
+guard_class(a, Integer)
+guard_class(b, Integer)
+u_a = unbox(a)
+u_b = unbox(b)
+u_x = int_add(u_a, u_b)
+x = new(Integer, u_x)
+
+guard_class(c, Integer)
+u_x2 = unbox(x)
+u_c = unbox(c)
+u_r = int_add(u_x2, u_c)
+r = new(Integer, u_r)
+\end{alltt}
+\end{frame}
+
+\begin{frame}[containsverbatim]
+  \frametitle{Example: Boxing in a Tracing JIT}
+\begin{alltt}
+guard_class(a, Integer)
+guard_class(b, Integer)
+u_a = unbox(a)
+u_b = unbox(b)
+u_x = int_add(u_a, u_b)
+\redsout{x = new(Integer, u_x)}
+
+guard_class(c, Integer)
+\redsout{u_x2 = unbox(x)}
+u_c = unbox(c)
+u_r = int_add(u_x, u_c)
+r = new(Integer, u_r)
+\end{alltt}
+\end{frame}
 
 \begin{frame}
     \frametitle{Advantages of Tracing JITs}


From commits-noreply at bitbucket.org  Mon Feb  7 23:29:46 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon,  7 Feb 2011 23:29:46 +0100 (CET)
Subject: [pypy-svn] pypy default: Filter out the method names that start
	with '_', because the
Message-ID: <20110207222946.B92F0282BE7@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41692:84883dd58523
Date: 2011-02-07 23:29 +0100
http://bitbucket.org/pypy/pypy/changeset/84883dd58523/

Log:	Filter out the method names that start with '_', because the precise
	list we get really depends on the implementation.

diff --git a/lib-python/2.7.0/test/test_descrtut.py b/lib-python/modified-2.7.0/test/test_descrtut.py
copy from lib-python/2.7.0/test/test_descrtut.py
copy to lib-python/modified-2.7.0/test/test_descrtut.py
--- a/lib-python/2.7.0/test/test_descrtut.py
+++ b/lib-python/modified-2.7.0/test/test_descrtut.py
@@ -172,46 +172,12 @@
     AttributeError: 'list' object has no attribute '__methods__'
     >>>
 
-Instead, you can get the same information from the list type:
+Instead, you can get the same information from the list type
+(the following example filters out the numerous method names
+starting with '_'):
 
-    >>> pprint.pprint(dir(list))    # like list.__dict__.keys(), but sorted
-    ['__add__',
-     '__class__',
-     '__contains__',
-     '__delattr__',
-     '__delitem__',
-     '__delslice__',
-     '__doc__',
-     '__eq__',
-     '__format__',
-     '__ge__',
-     '__getattribute__',
-     '__getitem__',
-     '__getslice__',
-     '__gt__',
-     '__hash__',
-     '__iadd__',
-     '__imul__',
-     '__init__',
-     '__iter__',
-     '__le__',
-     '__len__',
-     '__lt__',
-     '__mul__',
-     '__ne__',
-     '__new__',
-     '__reduce__',
-     '__reduce_ex__',
-     '__repr__',
-     '__reversed__',
-     '__rmul__',
-     '__setattr__',
-     '__setitem__',
-     '__setslice__',
-     '__sizeof__',
-     '__str__',
-     '__subclasshook__',
-     'append',
+    >>> pprint.pprint([name for name in dir(list) if not name.startswith('_')])
+    ['append',
      'count',
      'extend',
      'index',

From commits-noreply at bitbucket.org  Mon Feb  7 23:38:25 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon,  7 Feb 2011 23:38:25 +0100 (CET)
Subject: [pypy-svn] pypy default: Attempt to fix the issue shown by
Message-ID: <20110207223825.174DF282BE7@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41693:3923b795b4f8
Date: 2011-02-07 23:37 +0100
http://bitbucket.org/pypy/pypy/changeset/3923b795b4f8/

Log:	Attempt to fix the issue shown by
	test_appinterp:test_geninterp_can_unfreeze.

diff --git a/pypy/interpreter/mixedmodule.py b/pypy/interpreter/mixedmodule.py
--- a/pypy/interpreter/mixedmodule.py
+++ b/pypy/interpreter/mixedmodule.py
@@ -108,6 +108,7 @@
         self.getdict()
         self.w_initialdict = None
         self.startup_called = False
+        self._frozen = True
         # hint for the annotator: Modules can hold state, so they are
         # not constant
         return False

diff --git a/pypy/interpreter/module.py b/pypy/interpreter/module.py
--- a/pypy/interpreter/module.py
+++ b/pypy/interpreter/module.py
@@ -4,10 +4,13 @@
 
 from pypy.interpreter.baseobjspace import Wrappable
 from pypy.interpreter.error import OperationError
+from pypy.rlib.objectmodel import we_are_translated
 
 class Module(Wrappable):
     """A module."""
 
+    _frozen = False
+
     def __init__(self, space, w_name, w_dict=None):
         self.space = space
         if w_dict is None: 
@@ -26,6 +29,12 @@
         """This is called each time the module is imported or reloaded
         """
         if not self.startup_called:
+            if not we_are_translated():
+                # this special case is to handle the case, during annotation,
+                # of module A that gets frozen, then module B (e.g. during
+                # a getdict()) runs some code that imports A
+                if self._frozen:
+                    return
             self.startup_called = True
             self.startup(space)
 

From commits-noreply at bitbucket.org  Mon Feb  7 23:48:04 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon,  7 Feb 2011 23:48:04 +0100 (CET)
Subject: [pypy-svn] pypy default: Trying harder to reinit the thread's
	bootstrapper.
Message-ID: <20110207224804.CE3A4282BE7@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41694:4498f2657fe8
Date: 2011-02-07 23:47 +0100
http://bitbucket.org/pypy/pypy/changeset/4498f2657fe8/

Log:	Trying harder to reinit the thread's bootstrapper. Probably not
	enough to make test_thread.py pass on the nightly run, but the issue
	is that it passes for me.

diff --git a/pypy/module/thread/os_thread.py b/pypy/module/thread/os_thread.py
--- a/pypy/module/thread/os_thread.py
+++ b/pypy/module/thread/os_thread.py
@@ -76,8 +76,15 @@
                 raise wrap_thread_error(space, "can't allocate bootstrap lock")
 
     @staticmethod
-    def reinit(space):
+    def reinit():
         bootstrapper.lock = None
+        bootstrapper.nbthreads = 0
+        bootstrapper.w_callable = None
+        bootstrapper.args = None
+
+    def _freeze_(self):
+        self.reinit()
+        return False
 
     def bootstrap():
         # Note that when this runs, we already hold the GIL.  This is ensured
@@ -143,7 +150,7 @@
 def reinit_threads(space):
     "Called in the child process after a fork()"
     space.threadlocals.reinit_threads(space)
-    bootstrapper.reinit(space)
+    bootstrapper.reinit()
 
     # Clean the threading module after a fork()
     w_modules = space.sys.get('modules')

From cfbolz at codespeak.net  Mon Feb  7 23:58:31 2011
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Mon,  7 Feb 2011 23:58:31 +0100 (CET)
Subject: [pypy-svn] r80310 - in pypy/extradoc/talk/ibm-feb-2011: . figures
Message-ID: <20110207225831.DE93E282BE7@codespeak.net>

Author: cfbolz
Date: Mon Feb  7 23:58:30 2011
New Revision: 80310

Added:
   pypy/extradoc/talk/ibm-feb-2011/figures/architecture.pdf   (contents, props changed)
   pypy/extradoc/talk/ibm-feb-2011/figures/architecture.svg
   pypy/extradoc/talk/ibm-feb-2011/figures/requirements.gnumeric   (contents, props changed)
   pypy/extradoc/talk/ibm-feb-2011/figures/timeline.png   (contents, props changed)
Modified:
   pypy/extradoc/talk/ibm-feb-2011/talk.pdf
   pypy/extradoc/talk/ibm-feb-2011/talk.tex
Log:
a few more figures


Added: pypy/extradoc/talk/ibm-feb-2011/figures/architecture.pdf
==============================================================================
Binary file. No diff available.

Added: pypy/extradoc/talk/ibm-feb-2011/figures/architecture.svg
==============================================================================
--- (empty file)
+++ pypy/extradoc/talk/ibm-feb-2011/figures/architecture.svg	Mon Feb  7 23:58:30 2011
@@ -0,0 +1,203 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+   xmlns:dc="http://purl.org/dc/elements/1.1/"
+   xmlns:cc="http://creativecommons.org/ns#"
+   xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
+   xmlns:svg="http://www.w3.org/2000/svg"
+   xmlns="http://www.w3.org/2000/svg"
+   xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+   xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+   width="744.09448819"
+   height="1052.3622047"
+   id="svg2"
+   version="1.1"
+   inkscape:version="0.48.0 r9654"
+   sodipodi:docname="architecture.svg">
+  <defs
+     id="defs4">
+    <marker
+       inkscape:stockid="Arrow2Lend"
+       orient="auto"
+       refY="0.0"
+       refX="0.0"
+       id="Arrow2Lend"
+       style="overflow:visible;">
+      <path
+         id="path3928"
+         style="font-size:12.0;fill-rule:evenodd;stroke-width:0.62500000;stroke-linejoin:round;"
+         d="M 8.7185878,4.0337352 L -2.2072895,0.016013256 L 8.7185884,-4.0017078 C 6.9730900,-1.6296469 6.9831476,1.6157441 8.7185878,4.0337352 z "
+         transform="scale(1.1) rotate(180) translate(1,0)" />
+    </marker>
+    <inkscape:perspective
+       sodipodi:type="inkscape:persp3d"
+       inkscape:vp_x="0 : 526.18109 : 1"
+       inkscape:vp_y="0 : 1000 : 0"
+       inkscape:vp_z="744.09448 : 526.18109 : 1"
+       inkscape:persp3d-origin="372.04724 : 350.78739 : 1"
+       id="perspective3861" />
+  </defs>
+  <sodipodi:namedview
+     id="base"
+     pagecolor="#ffffff"
+     bordercolor="#666666"
+     borderopacity="1.0"
+     inkscape:pageopacity="0.0"
+     inkscape:pageshadow="2"
+     inkscape:zoom="0.98994949"
+     inkscape:cx="440.47379"
+     inkscape:cy="785.39582"
+     inkscape:document-units="px"
+     inkscape:current-layer="g3012-0"
+     showgrid="false"
+     inkscape:window-width="1280"
+     inkscape:window-height="769"
+     inkscape:window-x="0"
+     inkscape:window-y="1"
+     inkscape:window-maximized="1">
+    <inkscape:grid
+       type="xygrid"
+       id="grid4352" />
+  </sodipodi:namedview>
+  <metadata
+     id="metadata7">
+    <rdf:RDF>
+      <cc:Work
+         rdf:about="">
+        <dc:format>image/svg+xml</dc:format>
+        <dc:type
+           rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
+        <dc:title></dc:title>
+      </cc:Work>
+    </rdf:RDF>
+  </metadata>
+  <g
+     inkscape:label="Layer 1"
+     inkscape:groupmode="layer"
+     id="layer1">
+    <g
+       id="g3012">
+      <rect
+         y="73.524368"
+         x="146.47212"
+         height="108.08633"
+         width="364.66507"
+         id="rect2985"
+         style="color:#000000;fill:#ff8080;fill-opacity:1;fill-rule:nonzero;stroke:#000000;stroke-width:2;marker:none;visibility:visible;display:inline;overflow:visible;enable-background:accumulate;stroke-opacity:1;stroke-miterlimit:4;stroke-dasharray:none" />
+      <text
+         sodipodi:linespacing="125%"
+         id="text2987"
+         y="108.87971"
+         x="213.6244"
+         style="font-size:28px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:125%;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;font-family:FreeSans;-inkscape-font-specification:FreeSans"
+         xml:space="preserve"><tspan
+           y="108.87971"
+           x="213.6244"
+           id="tspan2989"
+           sodipodi:role="line">Python Interpreter</tspan></text>
+      <text
+         sodipodi:linespacing="125%"
+         id="text2987-9"
+         y="154.41951"
+         x="301.40448"
+         style="font-size:28px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:125%;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;font-family:FreeSans;-inkscape-font-specification:FreeSans"
+         xml:space="preserve"><tspan
+           y="154.41951"
+           x="301.40448"
+           id="tspan2989-2"
+           sodipodi:role="line">RPython</tspan></text>
+    </g>
+    <g
+       transform="translate(-2.1923828e-6,266)"
+       id="g3012-0">
+      <rect
+         y="73.524368"
+         x="146.47212"
+         height="108.08633"
+         width="364.66507"
+         id="rect2985-8"
+         style="color:#000000;fill:#99ff55;fill-opacity:1;fill-rule:nonzero;stroke:#000000;stroke-width:2;marker:none;visibility:visible;display:inline;overflow:visible;enable-background:accumulate;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none;stroke-dashoffset:0" />
+      <text
+         sodipodi:linespacing="125%"
+         id="text2987-90"
+         y="108.87971"
+         x="213.6244"
+         style="font-size:28px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:125%;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;font-family:FreeSans;-inkscape-font-specification:FreeSans"
+         xml:space="preserve"><tspan
+           y="108.87971"
+           x="213.6244"
+           id="tspan2989-1"
+           sodipodi:role="line">Python Interpreter</tspan></text>
+      <text
+         sodipodi:linespacing="125%"
+         id="text2987-9-3"
+         y="154.41951"
+         x="341.40448"
+         style="font-size:28px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:125%;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;font-family:FreeSans;-inkscape-font-specification:FreeSans"
+         xml:space="preserve"><tspan
+           y="154.41951"
+           x="341.40448"
+           id="tspan2989-2-4"
+           sodipodi:role="line">C</tspan></text>
+      <rect
+         style="color:#000000;fill:#abc837;fill-opacity:1;fill-rule:nonzero;stroke:#000000;stroke-width:2;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none;stroke-dashoffset:0;marker:none;visibility:visible;display:inline;overflow:visible;enable-background:accumulate"
+         id="rect3877"
+         width="78.214287"
+         height="56.785713"
+         x="432.5"
+         y="390.93362"
+         transform="translate(2.1923828e-6,-266)" />
+      <text
+         sodipodi:linespacing="125%"
+         id="text2987-9-3-2"
+         y="162.41951"
+         x="451.40448"
+         style="font-size:28px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:125%;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;font-family:FreeSans;-inkscape-font-specification:FreeSans"
+         xml:space="preserve"><tspan
+           y="162.41951"
+           x="451.40448"
+           id="tspan2989-2-4-4"
+           sodipodi:role="line">GC</tspan></text>
+      <rect
+         style="color:#000000;fill:#abc837;fill-opacity:1;fill-rule:nonzero;stroke:#000000;stroke-width:2;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none;stroke-dashoffset:0;marker:none;visibility:visible;display:inline;overflow:visible;enable-background:accumulate"
+         id="rect3877-2"
+         width="78.214287"
+         height="56.785713"
+         x="146.5"
+         y="124.93362" />
+      <text
+         sodipodi:linespacing="125%"
+         id="text2987-9-3-2-9"
+         y="162.41951"
+         x="165.40448"
+         style="font-size:28px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:125%;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;font-family:FreeSans;-inkscape-font-specification:FreeSans"
+         xml:space="preserve"><tspan
+           y="162.41951"
+           x="165.40448"
+           id="tspan2989-2-4-4-0"
+           sodipodi:role="line">JIT</tspan></text>
+    </g>
+    <path
+       style="fill:none;stroke:#000000;stroke-width:2;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow2Lend);stroke-miterlimit:4;stroke-dasharray:none"
+       d="m 328.80466,181.61069 0,157.91368"
+       id="path3902"
+       inkscape:connector-type="polyline"
+       inkscape:connector-curvature="0"
+       inkscape:connection-start="#g3012"
+       inkscape:connection-start-point="d4"
+       inkscape:connection-end="#g3012-0"
+       inkscape:connection-end-point="d4" />
+    <text
+       xml:space="preserve"
+       style="font-size:40px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Bitstream Vera Sans"
+       x="338.57144"
+       y="271.64789"
+       id="text4348"
+       sodipodi:linespacing="125%"><tspan
+         sodipodi:role="line"
+         id="tspan4350"
+         x="338.57144"
+         y="271.64789">Translation</tspan></text>
+  </g>
+</svg>

Added: pypy/extradoc/talk/ibm-feb-2011/figures/requirements.gnumeric
==============================================================================
Binary file. No diff available.

Added: pypy/extradoc/talk/ibm-feb-2011/figures/timeline.png
==============================================================================
Binary file. No diff available.

Modified: pypy/extradoc/talk/ibm-feb-2011/talk.pdf
==============================================================================
Binary files. No diff available.

Modified: pypy/extradoc/talk/ibm-feb-2011/talk.tex
==============================================================================
--- pypy/extradoc/talk/ibm-feb-2011/talk.tex	(original)
+++ pypy/extradoc/talk/ibm-feb-2011/talk.tex	Mon Feb  7 23:58:30 2011
@@ -910,6 +910,22 @@
 \end{frame}
 
 \begin{frame}
+  \frametitle{Project Status}
+  \begin{itemize}
+      \item about two people work on it full-time
+      \item sizeable open source community
+      \item one-week development sprints several times per year
+      \item about 20-30 person-years so far
+      \item heavily dedicated to testing and quality
+  \end{itemize}
+\end{frame}
+
+
+\begin{frame}[plain]
+  \includegraphics[scale=0.25]{figures/timeline.png}
+\end{frame}
+
+\begin{frame}
   \frametitle{PyPy's Approach to VM Construction}
   \emph{Goal: achieve flexibility, simplicity and performance together}
 
@@ -958,6 +974,12 @@
   \end{block}
 \end{frame}
 
+\begin{frame}[plain]
+  \frametitle{Architecture}
+  \includegraphics[scale=0.4]{figures/architecture.pdf}
+\end{frame}
+
+
 
 \begin{frame}
   \frametitle{Good Points of the Approach}


From commits-noreply at bitbucket.org  Tue Feb  8 09:12:18 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Tue,  8 Feb 2011 09:12:18 +0100 (CET)
Subject: [pypy-svn] pypy default: Update the TODO list
Message-ID: <20110208081218.B4749282BF2@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41695:bdc7858f84e4
Date: 2011-02-08 09:16 +0100
http://bitbucket.org/pypy/pypy/changeset/bdc7858f84e4/

Log:	Update the TODO list

diff --git a/lib-python/TODO b/lib-python/TODO
--- a/lib-python/TODO
+++ b/lib-python/TODO
@@ -11,20 +11,11 @@
 - (unicode|bytearray).(index|find) should accept None as indices (see
   test_unicode.py)
 
-- Fix fcntl.fcntl(fd, fcntl.F_NOTIFY, fcntl.DN_MULTISHOT) on 32bit platform.
-
-- in test_os.py, fix posix.setregid(-1, -1), posix.setreuid(-1, -1).  This
-  proably requires to use the git_t typedef instead of rffi.INT.
-
-- missing posix.(confstr|pathconf|fpathconf)(_names)? (see 
+- missing posix.confstr and posix.confstr_names
 
 - remove code duplication: bit_length() and _count_bits() in rlib/rbigint.py,
   objspace/std/longobject.py and objspace/std/longtype.py.
 
-- Add missing methods to bytearray (see test_bytes.py).  Also ideally refactor
-  stringobject.py and unicodeobject.py to allow more code reuse between these
-  (now) three types.
-
 
 Medium tasks
 ------------
@@ -37,18 +28,10 @@
 
 - socket module has a couple of changes (including AF_TIPC packet range)
 
-- module/unicodedata/generate_unicodedb.py should parse LineBreaks.txt
-  see http://svn.python.org/view?view=rev&revision=79494
-
 
 Longer tasks
 ------------
 
-- many features are missing from the _ssl module
-
-- "Shorter float representation" (see pypy/translator/c/test/test_dtoa.py) to
-  format/parse floats.  Enable this with a translation option.
-
 - Fix usage of __cmp__ in subclasses::
 
     class badint(int):
@@ -90,4 +73,7 @@
 
   * In test_codecs, commented out various items in `all_unicode_encodings`.
 
+- Error messages about ill-formed calls (like "argument after ** must be a
+  mapping") don't show the function name.
 
+  * Some "..." were added to doctests in test_extcall.py

From commits-noreply at bitbucket.org  Tue Feb  8 10:50:50 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue,  8 Feb 2011 10:50:50 +0100 (CET)
Subject: [pypy-svn] pypy default: A test and fix for someone doing
	f(**{u'\u1234': 5}).
Message-ID: <20110208095050.ABBA336C225@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41696:62ccd916df24
Date: 2011-02-08 10:50 +0100
http://bitbucket.org/pypy/pypy/changeset/62ccd916df24/

Log:	A test and fix for someone doing f(**{u'\u1234': 5}). CPython's
	tests don't expect to get the UnicodeEncodeError here. Actually
	CPython has a different behavior, but see comment.

diff --git a/pypy/interpreter/argument.py b/pypy/interpreter/argument.py
--- a/pypy/interpreter/argument.py
+++ b/pypy/interpreter/argument.py
@@ -196,10 +196,15 @@
             try:
                 key = space.str_w(w_key)
             except OperationError, e:
-                if not e.match(space, space.w_TypeError):
-                    raise
-                raise OperationError(space.w_TypeError,
-                                     space.wrap("keywords must be strings"))
+                if e.match(space, space.w_TypeError):
+                    raise OperationError(
+                        space.w_TypeError,
+                        space.wrap("keywords must be strings"))
+                if e.match(space, space.w_UnicodeEncodeError):
+                    raise OperationError(
+                        space.w_TypeError,
+                        space.wrap("keyword cannot be encoded to ascii"))
+                raise
             if self.keywords and key in self.keywords:
                 raise operationerrfmt(self.space.w_TypeError,
                                       "got multiple values "

diff --git a/pypy/interpreter/test/test_argument.py b/pypy/interpreter/test/test_argument.py
--- a/pypy/interpreter/test/test_argument.py
+++ b/pypy/interpreter/test/test_argument.py
@@ -124,6 +124,7 @@
 
     w_TypeError = TypeError
     w_AttributeError = AttributeError
+    w_UnicodeEncodeError = UnicodeEncodeError
     w_dict = dict
 
 class TestArgumentsNormal(object):
@@ -474,6 +475,35 @@
         assert w_args == (1, )
         assert not w_kwds
 
+    def test_argument_unicode(self):
+        space = DummySpace()
+        w_starstar = space.wrap({u'abc': 5})
+        args = Arguments(space, [], w_starstararg=w_starstar)
+        l = [None]
+        args._match_signature(None, l, Signature(['abc']))
+        assert len(l) == 1
+        assert l[0] == space.wrap(5)
+        #
+        def str_w(w):
+            try:
+                return str(w)
+            except UnicodeEncodeError:
+                raise OperationError(space.w_UnicodeEncodeError,
+                                     space.wrap("oups"))
+        space.str_w = str_w
+        w_starstar = space.wrap({u'\u1234': 5})
+        err = py.test.raises(OperationError, Arguments,
+                             space, [], w_starstararg=w_starstar)
+        # Check that we get a TypeError.  On CPython it is because of
+        # "no argument called '?'".  On PyPy we get a TypeError too, but
+        # earlier: "keyword cannot be encoded to ascii".  The
+        # difference, besides the error message, is only apparent if the
+        # receiver also takes a **arg.  Then CPython passes the
+        # non-ascii unicode unmodified, whereas PyPy complains.  We will
+        # not care until someone has a use case for that.
+        assert not err.value.match(space, space.w_UnicodeEncodeError)
+        assert     err.value.match(space, space.w_TypeError)
+
 class TestErrorHandling(object):
     def test_missing_args(self):
         # got_nargs, nkwds, expected_nargs, has_vararg, has_kwarg,

From lac at codespeak.net  Tue Feb  8 12:38:41 2011
From: lac at codespeak.net (lac at codespeak.net)
Date: Tue,  8 Feb 2011 12:38:41 +0100 (CET)
Subject: [pypy-svn] r80311 - pypy/extradoc/talk/ustour2011
Message-ID: <20110208113841.90BC4282BF2@codespeak.net>

Author: lac
Date: Tue Feb  8 12:38:38 2011
New Revision: 80311

Added:
   pypy/extradoc/talk/ustour2011/blog1.txt   (contents, props changed)
Log:
The very beginning of what will be a blog post about our planned us road show.


Added: pypy/extradoc/talk/ustour2011/blog1.txt
==============================================================================
--- (empty file)
+++ pypy/extradoc/talk/ustour2011/blog1.txt	Tue Feb  8 12:38:38 2011
@@ -0,0 +1,25 @@
+Agenda:
+
+Wednesday March 2, 4:15 p.m.  Armin gives talk at Stanford.
+http://www.stanford.edu/class/ee380/winter-schedule-20102011.html
+open to the public.
+
+Thursday March 3, 6:00 p.m.  General talk at Yelp, 706 Mission St 9th Floor,
+San Francisco CA 94103 
+open to the public.  See yelp-abstract.txt
+
+Saturday and Sunday March 5 and 6.
+PyPy mini sprint at noisebridge. 2169 mission street between 17th and 18th in 
+san francisco. http://www.noisebridge.net   Open to the public.
+See noisebridge.txt
+
+Monday March 7th.
+11:30 a.m.  Google Tech talk.  Not open to the public.  See
+google-abstract2.txt 
+
+2:30 p.m. Talk at Mozilla.  Not open to the public.  See mozilla-abstract.txt
+
+--------
+We're open to meeting more people at other times.  We're especially interested
+in discussing PyPy consulting opportunities with interested companies.
+Insert plug here about what we would be willing to do.


From commits-noreply at bitbucket.org  Tue Feb  8 13:32:23 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue,  8 Feb 2011 13:32:23 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix this test: passing a Python long
	to	PyLong_FromLong() hits some assert now.
Message-ID: <20110208123223.7D39D282BF2@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41697:25f385082dd7
Date: 2011-02-08 11:44 +0100
http://bitbucket.org/pypy/pypy/changeset/25f385082dd7/

Log:	Fix this test: passing a Python long to PyLong_FromLong() hits some
	assert now.

diff --git a/pypy/module/cpyext/test/test_longobject.py b/pypy/module/cpyext/test/test_longobject.py
--- a/pypy/module/cpyext/test/test_longobject.py
+++ b/pypy/module/cpyext/test/test_longobject.py
@@ -12,9 +12,9 @@
         assert isinstance(value, W_LongObject)
         assert space.unwrap(value) == 3
 
-        value = api.PyLong_FromLong(sys.maxint + 1)
+        value = api.PyLong_FromLong(sys.maxint)
         assert isinstance(value, W_LongObject)
-        assert space.unwrap(value) == sys.maxint + 1 # should obviously fail but doesnt
+        assert space.unwrap(value) == sys.maxint
 
     def test_aslong(self, space, api):
         w_value = api.PyLong_FromLong((sys.maxint - 1) / 2)

From commits-noreply at bitbucket.org  Tue Feb  8 13:32:24 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue,  8 Feb 2011 13:32:24 +0100 (CET)
Subject: [pypy-svn] pypy default: Test and fix.
Message-ID: <20110208123224.573C1282BF2@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41698:361332648e20
Date: 2011-02-08 13:32 +0100
http://bitbucket.org/pypy/pypy/changeset/361332648e20/

Log:	Test and fix.

diff --git a/pypy/objspace/std/newformat.py b/pypy/objspace/std/newformat.py
--- a/pypy/objspace/std/newformat.py
+++ b/pypy/objspace/std/newformat.py
@@ -152,7 +152,9 @@
         if empty:
             index = -1
         else:
-            index = _parse_int(self.space, name, 0, i)[0]
+            index, stop = _parse_int(self.space, name, 0, i)
+            if stop != i:
+                index = -1
         use_numeric = empty or index != -1
         if self.auto_numbering_state == ANS_INIT and use_numeric:
             if empty:
@@ -259,7 +261,9 @@
         if i == 0:
             index = -1
         else:
-            index = _parse_int(self.space, name, 0, i)[0]
+            index, stop = _parse_int(self.space, name, 0, i)
+            if stop != i:
+                index = -1
         if index >= 0:
             w_first = space.wrap(index)
         else:
@@ -426,10 +430,10 @@
             i += 1
         start_i = i
         self._width, i = _parse_int(self.space, spec, i, length)
-        if length - i and spec[i] == ",":
+        if length != i and spec[i] == ",":
             self._thousands_sep = True
             i += 1
-        if length - i and spec[i] == ".":
+        if length != i and spec[i] == ".":
             i += 1
             self._precision, i = _parse_int(self.space, spec, i, length)
             if self._precision == -1:

diff --git a/pypy/objspace/std/test/test_newformat.py b/pypy/objspace/std/test/test_newformat.py
--- a/pypy/objspace/std/test/test_newformat.py
+++ b/pypy/objspace/std/test/test_newformat.py
@@ -164,6 +164,9 @@
             assert type(log[2].message) is PendingDeprecationWarning
         assert len(log) == 3
 
+    def test_bogus_cases(self):
+        raises(KeyError, '{0]}'.format, 5)
+
 
 class AppTestUnicodeFormat(BaseStringFormatTests):
 

From commits-noreply at bitbucket.org  Tue Feb  8 13:44:40 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue,  8 Feb 2011 13:44:40 +0100 (CET)
Subject: [pypy-svn] pypy default: Another test and fix.
Message-ID: <20110208124440.7CA4636C22F@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41699:18d9c14ffea0
Date: 2011-02-08 13:44 +0100
http://bitbucket.org/pypy/pypy/changeset/18d9c14ffea0/

Log:	Another test and fix.

diff --git a/pypy/objspace/std/newformat.py b/pypy/objspace/std/newformat.py
--- a/pypy/objspace/std/newformat.py
+++ b/pypy/objspace/std/newformat.py
@@ -129,10 +129,15 @@
                         raise OperationError(self.space.w_ValueError, w_msg)
                     conversion = s[i]
                     i += 1
-                    c = s[i]
+                    if i < end:
+                        if s[i] != ':':
+                            w_msg = self.space.wrap("expected ':' after"
+                                                    " format specifier")
+                            raise OperationError(self.space.w_ValueError,
+                                                 w_msg)
+                        i += 1
                 else:
                     conversion = None
-                if c == ':':
                     i += 1
                 return s[start:end_name], conversion, i
             i += 1

diff --git a/pypy/objspace/std/test/test_newformat.py b/pypy/objspace/std/test/test_newformat.py
--- a/pypy/objspace/std/test/test_newformat.py
+++ b/pypy/objspace/std/test/test_newformat.py
@@ -166,6 +166,8 @@
 
     def test_bogus_cases(self):
         raises(KeyError, '{0]}'.format, 5)
+        raises(ValueError, '{0!r'.format, 5)
+        raises(ValueError, '{0!rs}'.format, 5)
 
 
 class AppTestUnicodeFormat(BaseStringFormatTests):

From commits-noreply at bitbucket.org  Tue Feb  8 13:54:48 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue,  8 Feb 2011 13:54:48 +0100 (CET)
Subject: [pypy-svn] pypy default: "Justify" a bit more this topic.
Message-ID: <20110208125448.13C59282BF2@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41700:5f4d329b69a8
Date: 2011-02-08 13:54 +0100
http://bitbucket.org/pypy/pypy/changeset/5f4d329b69a8/

Log:	"Justify" a bit more this topic.

diff --git a/lib-python/TODO b/lib-python/TODO
--- a/lib-python/TODO
+++ b/lib-python/TODO
@@ -74,6 +74,8 @@
   * In test_codecs, commented out various items in `all_unicode_encodings`.
 
 - Error messages about ill-formed calls (like "argument after ** must be a
-  mapping") don't show the function name.
+  mapping") don't always show the function name.  That's hard to fix for
+  the case of errors raised when the Argument object is created (as opposed
+  to when parsing for a given target function, which occurs later).
 
   * Some "..." were added to doctests in test_extcall.py

From commits-noreply at bitbucket.org  Tue Feb  8 17:45:57 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue,  8 Feb 2011 17:45:57 +0100 (CET)
Subject: [pypy-svn] pypy default: Oups.
Message-ID: <20110208164557.71A7036C226@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41701:6fc2ba42902e
Date: 2011-02-08 17:45 +0100
http://bitbucket.org/pypy/pypy/changeset/6fc2ba42902e/

Log:	Oups.

diff --git a/lib-python/modified-2.7.0/test/test_pprint.py b/lib-python/modified-2.7.0/test/test_pprint.py
--- a/lib-python/modified-2.7.0/test/test_pprint.py
+++ b/lib-python/modified-2.7.0/test/test_pprint.py
@@ -239,7 +239,7 @@
         # to implementation-dependent choices (this test probably
         # fails horribly in CPython if we tweak the dict order too).
         got = pprint.pformat(cube)
-        if test.test_support.impl_detail(cpython=True):
+        if test.test_support.check_impl_detail(cpython=True):
             self.assertEqual(got, cube_repr_tgt)
         else:
             self.assertEqual(eval(got), cube)
@@ -403,7 +403,7 @@
 
         cubo = test.test_set.linegraph(cube)
         got = pprint.pformat(cubo)
-        if test.test_support.impl_detail(cpython=True):
+        if test.test_support.check_impl_detail(cpython=True):
             self.assertEqual(got, cubo_repr_tgt)
         else:
             self.assertEqual(eval(got), cubo)

From commits-noreply at bitbucket.org  Tue Feb  8 17:58:37 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue,  8 Feb 2011 17:58:37 +0100 (CET)
Subject: [pypy-svn] pypy default: - Mark the undocumented bytes.__alloc__()
	as an	implementation
Message-ID: <20110208165837.E45C4282BFC@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41702:1b4cca12e352
Date: 2011-02-08 17:58 +0100
http://bitbucket.org/pypy/pypy/changeset/1b4cca12e352/

Log:	- Mark the undocumented bytes.__alloc__() as an implementation
	detail for now.

	- Test the different behavior we have w.r.t. taking a buffer() on
	a bytearray and mutating the bytearray at the same time.

diff --git a/lib-python/2.7.0/test/test_bytes.py b/lib-python/modified-2.7.0/test/test_bytes.py
copy from lib-python/2.7.0/test/test_bytes.py
copy to lib-python/modified-2.7.0/test/test_bytes.py
--- a/lib-python/2.7.0/test/test_bytes.py
+++ b/lib-python/modified-2.7.0/test/test_bytes.py
@@ -632,6 +632,7 @@
         self.assertEqual(b, b1)
         self.assertTrue(b is b1)
 
+    @test.test_support.impl_detail("undocumented bytes.__alloc__()")
     def test_alloc(self):
         b = bytearray()
         alloc = b.__alloc__()
@@ -759,6 +760,8 @@
         self.assertEqual(b, b"")
         self.assertEqual(c, b"")
 
+    @test.test_support.impl_detail(
+        "resizing semantics of CPython rely on refcounting")
     def test_resize_forbidden(self):
         # #4509: can't resize a bytearray when there are buffer exports, even
         # if it wouldn't reallocate the underlying buffer.
@@ -791,6 +794,26 @@
         self.assertRaises(BufferError, delslice)
         self.assertEquals(b, orig)
 
+    @test.test_support.impl_detail("resizing semantics", cpython=False)
+    def test_resize_forbidden_non_cpython(self):
+        # on non-CPython implementations, we cannot prevent changes to
+        # bytearrays just because there are buffers around.  Instead,
+        # we get (on PyPy) a buffer that follows the changes and resizes.
+        b = bytearray(range(10))
+        for v in [memoryview(b), buffer(b)]:
+            b[5] = 99
+            self.assertIn(v[5], (99, chr(99)))
+            b[5] = 100
+            b += b
+            b += b
+            b += b
+            self.assertEquals(len(v), 80)
+            self.assertIn(v[5], (100, chr(100)))
+            self.assertIn(v[79], (9, chr(9)))
+            del b[10:]
+            self.assertRaises(IndexError, lambda: v[10])
+            self.assertEquals(len(v), 10)
+
     def test_empty_bytearray(self):
         # Issue #7561: operations on empty bytearrays could crash in many
         # situations, due to a fragile implementation of the

From commits-noreply at bitbucket.org  Tue Feb  8 18:31:07 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Tue,  8 Feb 2011 18:31:07 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: We never want to produce a
	second preamble as	it will replace the first as entrybridge.
	This will produce a	loop which would never get called. I
	guess the gc will kill it	eventually but an optimization
	would be to never send it to the	backend.
Message-ID: <20110208173107.A214A282BF7@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r41703:6937cbed65e9
Date: 2011-02-08 18:30 +0100
http://bitbucket.org/pypy/pypy/changeset/6937cbed65e9/

Log:	We never want to produce a second preamble as it will replace the
	first as entrybridge. This will produce a loop which would never get
	called. I guess the gc will kill it eventually but an optimization
	would be to never send it to the backend.

diff --git a/pypy/jit/metainterp/test/test_basic.py b/pypy/jit/metainterp/test/test_basic.py
--- a/pypy/jit/metainterp/test/test_basic.py
+++ b/pypy/jit/metainterp/test/test_basic.py
@@ -1918,8 +1918,8 @@
             return a1.val + b1.val
         res = self.meta_interp(g, [6, 14])
         assert res == g(6, 14)
-        self.check_loop_count(11)
-        self.check_loops(getarrayitem_gc=17, everywhere=True)
+        self.check_loop_count(9)
+        self.check_loops(getarrayitem_gc=6, everywhere=True)
 
     def test_multiple_specialied_versions_bridge(self):
         myjitdriver = JitDriver(greens = [], reds = ['y', 'x', 'z', 'res'])

diff --git a/pypy/jit/metainterp/compile.py b/pypy/jit/metainterp/compile.py
--- a/pypy/jit/metainterp/compile.py
+++ b/pypy/jit/metainterp/compile.py
@@ -117,7 +117,7 @@
         send_loop_to_backend(metainterp_sd, loop, "loop")
         record_loop_or_bridge(metainterp_sd, loop)
         token = loop.preamble.token
-        if full_preamble_needed or not loop.preamble.token.short_preamble:
+        if full_preamble_needed:
             send_loop_to_backend(metainterp_sd, loop.preamble, "entry bridge")
             insert_loop_token(old_loop_tokens, loop.preamble.token)
             jitdriver_sd.warmstate.attach_unoptimized_bridge_from_interp(

From commits-noreply at bitbucket.org  Tue Feb  8 18:32:59 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Tue,  8 Feb 2011 18:32:59 +0100 (CET)
Subject: [pypy-svn] pypy default: Following CPython,
	extensive rewrite of the utf-7 decoder, 
Message-ID: <20110208173259.A51FF282BF8@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41704:69d17c51e630
Date: 2011-02-08 14:55 +0100
http://bitbucket.org/pypy/pypy/changeset/69d17c51e630/

Log:	Following CPython, extensive rewrite of the utf-7 decoder, which did
	not accept some legal sequences. (issue4426)

	Add utf-7 to the sytematic tests, and ensure that it works with
	surrogate pairs.

diff --git a/pypy/rlib/test/test_runicode.py b/pypy/rlib/test/test_runicode.py
--- a/pypy/rlib/test/test_runicode.py
+++ b/pypy/rlib/test/test_runicode.py
@@ -99,13 +99,13 @@
 
     def test_all_first_256(self):
         for i in range(256):
-            for encoding in ("utf-8 latin-1 utf-16 utf-16-be utf-16-le "
+            for encoding in ("utf-7 utf-8 latin-1 utf-16 utf-16-be utf-16-le "
                              "utf-32 utf-32-be utf-32-le").split():
                 self.checkdecode(unichr(i), encoding)
 
     def test_first_10000(self):
         for i in range(10000):
-            for encoding in ("utf-8 utf-16 utf-16-be utf-16-le "
+            for encoding in ("utf-7 utf-8 utf-16 utf-16-be utf-16-le "
                              "utf-32 utf-32-be utf-32-le").split():
                 self.checkdecode(unichr(i), encoding)
 
@@ -115,13 +115,13 @@
             if 0xd800 <= v <= 0xdfff:
                 continue
             uni = unichr(v)
-            for encoding in ("utf-8 utf-16 utf-16-be utf-16-le "
+            for encoding in ("utf-7 utf-8 utf-16 utf-16-be utf-16-le "
                              "utf-32 utf-32-be utf-32-le").split():
                 self.checkdecode(uni, encoding)
 
     def test_maxunicode(self):
         uni = unichr(sys.maxunicode)
-        for encoding in ("utf-8 utf-16 utf-16-be utf-16-le "
+        for encoding in ("utf-7 utf-8 utf-16 utf-16-be utf-16-le "
                          "utf-32 utf-32-be utf-32-le").split():
             self.checkdecode(uni, encoding)
 
@@ -184,22 +184,27 @@
         assert decode(s, 4, None) == (u'a+-', 4)
         assert decode(s, 5, None) == (u'a+-b', 5)
 
+    def test_utf7_surrogates(self):
+        assert u'\U000abcde'.encode('utf-7') == '+2m/c3g-'
+        raises(UnicodeError, unicode, '+3ADYAA-', 'utf-7')
+        assert unicode('+3ADYAA-', 'utf-7', 'replace') == u'\ufffd\ufffd'
+
 
 class TestEncoding(UnicodeTests):
     def test_all_ascii(self):
         for i in range(128):
-            for encoding in "utf-8 latin-1 ascii".split():
+            for encoding in "utf-7 utf-8 latin-1 ascii".split():
                 self.checkencode(unichr(i), encoding)
 
     def test_all_first_256(self):
         for i in range(256):
-            for encoding in ("utf-8 utf-16 utf-16-be utf-16-le "
+            for encoding in ("utf-7 utf-8 utf-16 utf-16-be utf-16-le "
                              "utf-32 utf-32-be utf-32-le").split():
                 self.checkencode(unichr(i), encoding)
 
     def test_first_10000(self):
         for i in range(10000):
-            for encoding in ("utf-8 utf-16 utf-16-be utf-16-le "
+            for encoding in ("utf-7 utf-8 utf-16 utf-16-be utf-16-le "
                              "utf-32 utf-32-be utf-32-le").split():
                 self.checkencode(unichr(i), encoding)
 
@@ -209,13 +214,13 @@
             if 0xd800 <= v <= 0xdfff:
                 continue
             uni = unichr(v)
-            for encoding in ("utf-8 utf-16 utf-16-be utf-16-le "
+            for encoding in ("utf-7 utf-8 utf-16 utf-16-be utf-16-le "
                              "utf-32 utf-32-be utf-32-le").split():
                 self.checkencode(uni, encoding)                
 
     def test_maxunicode(self):
         uni = unichr(sys.maxunicode)
-        for encoding in ("utf-8 utf-16 utf-16-be utf-16-le "
+        for encoding in ("utf-7 utf-8 utf-16 utf-16-be utf-16-le "
                          "utf-32 utf-32-be utf-32-le").split():
             self.checkencode(uni, encoding)        
 

diff --git a/pypy/rlib/runicode.py b/pypy/rlib/runicode.py
--- a/pypy/rlib/runicode.py
+++ b/pypy/rlib/runicode.py
@@ -579,81 +579,91 @@
 # ____________________________________________________________
 # utf-7
 
-## indicate whether a UTF-7 character is special i.e. cannot be directly
-##       encoded:
-##         0 - not special
-##         1 - special
-##         2 - whitespace (optional)
-##         3 - RFC2152 Set O (optional)
+# Three simple macros defining base-64
 
-_utf7_special = [
-    1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 1, 1, 2, 1, 1,
-    1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
-    2, 3, 3, 3, 3, 3, 3, 0, 0, 0, 3, 1, 0, 0, 0, 1,
-    0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 3, 3, 3, 0,
-    3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-    0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 1, 3, 3, 3,
-    3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-    0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 3, 3, 1, 1,
-]
-
-def _utf7_SPECIAL(oc, encodeO=False, encodeWS=False):
-    return (oc > 127 or _utf7_special[oc] == 1 or
-            (encodeWS and _utf7_special[oc] == 2) or
-            (encodeO and _utf7_special[oc] == 3))
-
-def _utf7_B64CHAR(oc):
-    if oc > 127:
-        return False
+def _utf7_IS_BASE64(oc):
+    "Is c a base-64 character?"
     c = chr(oc)
     return c.isalnum() or c == '+' or c == '/'
 def _utf7_TO_BASE64(n):
     "Returns the base-64 character of the bottom 6 bits of n"
     return "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"[n & 0x3f]
 def _utf7_FROM_BASE64(c):
-    "Retuns the base-64 value of a base-64 character"
-    if c == '+':
-        return 62
-    elif c == '/':
-        return 63
-    elif c >= 'a':
+    "given that c is a base-64 character, what is its base-64 value?"
+    if c >= 'a':
         return ord(c) - 71
     elif c >= 'A':
         return ord(c) - 65
-    else:
+    elif c >= '0':
         return ord(c) + 4
+    elif c == '+':
+        return 62
+    else: # c == '/'
+        return 63
 
-def _utf7_ENCODE(result, ch, bits):
-    while bits >= 6:
-        result.append(_utf7_TO_BASE64(ch >> (bits - 6)))
-        bits -= 6
-    return bits
+def _utf7_DECODE_DIRECT(oc):
+    return oc <= 127 and oc != ord('+')
 
-def _utf7_DECODE(s, result, errorhandler, errors,
-                 pos, charsleft, bitsleft, surrogate):
-    while bitsleft >= 16:
-        outCh =  (charsleft >> (bitsleft-16)) & 0xffff
-        bitsleft -= 16
+# The UTF-7 encoder treats ASCII characters differently according to
+# whether they are Set D, Set O, Whitespace, or special (i.e. none of
+# the above).  See RFC2152.  This array identifies these different
+# sets:
+# 0 : "Set D"
+#      alphanumeric and '(),-./:?
+# 1 : "Set O"
+#     !"#$%&*;<=>@[]^_`{|}
+# 2 : "whitespace"
+#     ht nl cr sp
+# 3 : special (must be base64 encoded)
+#     everything else (i.e. +\~ and non-printing codes 0-8 11-12 14-31 127)
 
-        if surrogate:
-            ## We have already generated an error for the high
-            ## surrogate so let's not bother seeing if the low
-            ## surrogate is correct or not
-            surrogate = False
-        elif 0xDC00 <= outCh <= 0xDFFF:
-            ## This is a surrogate pair. Unfortunately we can't
-            ## represent it in a 16-bit character
-            surrogate = True
-            msg = "code pairs are not supported"
-            res, pos = errorhandler(errors, 'utf-7',
-                                    msg, s, pos-1, pos)
-            result.append(res)
-            bitsleft = 0
-            break
-        else:
-            result.append(unichr(outCh))
-    return pos, charsleft, bitsleft, surrogate
+utf7_category = [
+#  nul soh stx etx eot enq ack bel bs  ht  nl  vt  np  cr  so  si
+    3,  3,  3,  3,  3,  3,  3,  3,  3,  2,  2,  3,  3,  2,  3,  3,
+#  dle dc1 dc2 dc3 dc4 nak syn etb can em  sub esc fs  gs  rs  us
+    3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,
+#  sp   !   "   #   $   %   &   '   (   )   *   +   ,   -   .   /
+    2,  1,  1,  1,  1,  1,  1,  0,  0,  0,  1,  3,  0,  0,  0,  0,
+#   0   1   2   3   4   5   6   7   8   9   :   ;   <   =   >   ? 
+    0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  1,  1,  1,  1,  0,
+#   @   A   B   C   D   E   F   G   H   I   J   K   L   M   N   O
+    1,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
+#   P   Q   R   S   T   U   V   W   X   Y   Z   [   \   ]   ^   _
+    0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  1,  3,  1,  1,  1,
+#   `   a   b   c   d   e   f   g   h   i   j   k   l   m   n   o
+    1,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
+#   p   q   r   s   t   u   v   w   x   y   z   {   |   }   ~  del
+    0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  1,  1,  1,  3,  3,
+]
 
+# ENCODE_DIRECT: this character should be encoded as itself.  The
+# answer depends on whether we are encoding set O as itself, and also
+# on whether we are encoding whitespace as itself.  RFC2152 makes it
+# clear that the answers to these questions vary between
+# applications, so this code needs to be flexible.
+
+def _utf7_ENCODE_DIRECT(oc, directO, directWS):
+    return(oc < 128 and oc > 0 and
+           (utf7_category[oc] == 0 or
+            (directWS and utf7_category[oc] == 2) or
+            (directO and utf7_category[oc] == 1)))
+
+def _utf7_ENCODE_CHAR(result, oc, base64bits, base64buffer):
+    if MAXUNICODE > 65535 and oc >= 0x10000:
+        # code first surrogate
+        base64bits += 16
+        base64buffer = (base64buffer << 16) | 0xd800 | ((oc-0x10000) >> 10)
+        while base64bits >= 6:
+            result.append(_utf7_TO_BASE64(base64buffer >> (base64bits-6)))
+            base64bits -= 6
+        # prepare second surrogate
+        oc = 0xDC00 | ((oc-0x10000) & 0x3FF)
+    base64bits += 16
+    base64buffer = (base64buffer << 16) | oc
+    while base64bits >= 6:
+        result.append(_utf7_TO_BASE64(base64buffer >> (base64bits-6)))
+        base64bits -= 6
+    return base64bits, base64buffer
 
 def str_decode_utf_7(s, size, errors, final=False,
                      errorhandler=None):
@@ -663,83 +673,126 @@
         return u'', 0
 
     inShift = False
-    bitsleft = 0
-    startinpos = 0
-    charsleft = 0
-    surrogate = False
+    base64bits = 0
+    base64buffer = 0
+    surrogate = 0
 
     result = UnicodeBuilder(size)
     pos = 0
+    shiftOutStartPos = 0
     while pos < size:
         ch = s[pos]
         oc = ord(ch)
 
-        if inShift:
-            if ch == '-' or not _utf7_B64CHAR(oc):
+        if inShift: # in a base-64 section
+            if _utf7_IS_BASE64(oc): #consume a base-64 character
+                base64buffer = (base64buffer << 6) | _utf7_FROM_BASE64(ch)
+                base64bits += 6
+                pos += 1
+
+                if base64bits >= 16:
+                    # enough bits for a UTF-16 value
+                    outCh = base64buffer >> (base64bits - 16)
+                    base64bits -= 16
+                    base64buffer &= (1 << base64bits) - 1 # clear high bits
+                    if surrogate:
+                        # expecting a second surrogate
+                        if outCh >= 0xDC00 and outCh <= 0xDFFFF:
+                            if MAXUNICODE < 65536:
+                                result.append(unichr(surrogate))
+                                result.append(unichr(outCh))
+                            else:
+                                result.append(
+                                    UNICHR((((surrogate & 0x3FF)<<10) |
+                                            (outCh & 0x3FF)) + 0x10000))
+                        else:
+                            surrogate = 0
+                            msg = "second surrogate missing"
+                            res, pos = errorhandler(errors, 'utf-7',
+                                                    msg, s, pos-1, pos)
+                            result.append(res)
+                            continue
+                    elif outCh >= 0xD800 and outCh <= 0xDBFF:
+                        # first surrogate
+                        surrogate = outCh
+                    elif outCh >= 0xDC00 and outCh <= 0xDFFF:
+                        msg = "unexpected second surrogate"
+                        res, pos = errorhandler(errors, 'utf-7',
+                                                msg, s, pos-1, pos)
+                        result.append(res)
+                        continue
+                    else:
+                        result.append(unichr(outCh))
+
+            else:
+                # now leaving a base-64 section
                 inShift = 0
                 pos += 1
 
-                pos, charsleft, bitsleft, surrogate = _utf7_DECODE(
-                    s, result, errorhandler, errors,
-                    pos, charsleft, bitsleft, surrogate)
-                if bitsleft >= 6:
-                    ## The shift sequence has a partial character in it. If
-                    ## bitsleft < 6 then we could just classify it as padding
-                    ## but that is not the case here
-                    msg = "partial character in shift sequence"
+                if surrogate:
+                    msg = "second surrogate missing at end of shift sequence"
                     res, pos = errorhandler(errors, 'utf-7',
                                             msg, s, pos-1, pos)
                     result.append(res)
-                    ## According to RFC2152 the remaining bits should be
-                    ## zero. We choose to signal an error/insert a replacement
-                    ## character here so indicate the potential of a
-                    ## misencoded character.
+                    continue
+
+                if base64bits > 0: # left-over bits
+                    if base64bits >= 6:
+                        # We've seen at least one base-64 character
+                        msg = "partial character in shift sequence"
+                        res, pos = errorhandler(errors, 'utf-7',
+                                                msg, s, pos-1, pos)
+                        result.append(res)
+                        continue
+                    else:
+                        # Some bits remain; they should be zero
+                        if base64buffer != 0:
+                            msg = "non-zero padding bits in shift sequence"
+                            res, pos = errorhandler(errors, 'utf-7',
+                                                    msg, s, pos-1, pos)
+                            result.append(res)
+                            continue
+
                 if ch == '-':
-                    if pos < size and s[pos] == '-':
-                        result.append(u'-')
-                        inShift = True
+                    # '-' is absorbed; other terminating characters are
+                    # preserved
+                    pass
+                else:
+                    result.append(ch)
 
-                elif _utf7_SPECIAL(oc):
-                    msg = "unexpected special character"
-                    res, pos = errorhandler(errors, 'utf-7',
-                                            msg, s, pos-1, pos)
-                    result.append(res)
-                else:
-                    result.append(unichr(ord(ch)))
-            else:
-                charsleft = (charsleft << 6) | _utf7_FROM_BASE64(ch)
-                bitsleft += 6
-                pos += 1
-
-                pos, charsleft, bitsleft, surrogate = _utf7_DECODE(
-                    s, result, errorhandler, errors,
-                    pos, charsleft, bitsleft, surrogate)
         elif ch == '+':
             startinpos = pos
-            pos += 1
-            if pos < size and s[pos] == '-':
+            pos += 1 # consume '+'
+            if pos < size and s[pos] == '-': # '+-' encodes '+'
                 pos += 1
                 result.append(u'+')
-            else:
+            else: # begin base64-encoded section
                 inShift = 1
+                shiftOutStartPos = pos - 1
                 bitsleft = 0
 
-        elif _utf7_SPECIAL(oc):
+        elif _utf7_DECODE_DIRECT(oc): # character decodes at itself
+            result.append(unichr(oc))
+            pos += 1
+        else:
             pos += 1
             msg = "unexpected special character"
             res, pos = errorhandler(errors, 'utf-7', msg, s, pos-1, pos)
             result.append(res)
-        else:
-            result.append(unichr(oc))
-            pos += 1
 
-    if inShift and final:
-        endinpos = size
-        msg = "unterminated shift sequence"
-        res, pos = errorhandler(errors, 'utf-7', msg, s, startinpos, pos)
-        result.append(res)
+    # end of string
+
+    if inShift and final: # in shift sequence, no more to follow
+        # if we're in an inconsistent state, that's an error
+        if (surrogate or
+            base64bits >= 6 or
+            (base64bits > 0 and base64buffer != 0)):
+            endinpos = size
+            msg = "unterminated shift sequence"
+            res, pos = errorhandler(errors, 'utf-7', msg, s, startinpos, pos)
+            result.append(res)
     elif inShift:
-        pos = startinpos
+        pos = shiftOutStartPos # back off output
 
     return result.build(), pos
 
@@ -751,8 +804,8 @@
     encodeSetO = encodeWhiteSpace = False
 
     inShift = False
-    bitsleft = 0
-    charsleft = 0
+    base64bits = 0
+    base64buffer = 0
 
     pos = 0
     while pos < size:
@@ -761,53 +814,36 @@
         if not inShift:
             if ch == u'+':
                 result.append('+-')
-            elif _utf7_SPECIAL(oc, encodeSetO, encodeWhiteSpace):
-                charsleft = oc
-                bitsleft = 16
+            elif _utf7_ENCODE_DIRECT(oc, not encodeSetO, not encodeWhiteSpace):
+                result.append(chr(oc))
+            else:
                 result.append('+')
-                bitsleft = _utf7_ENCODE(result, charsleft, bitsleft)
-                inShift = bitsleft > 0
-            else:
-                result.append(chr(oc))
+                inShift = True
+                base64bits, base64buffer = _utf7_ENCODE_CHAR(
+                    result, oc, base64bits, base64buffer)
         else:
-            if not _utf7_SPECIAL(oc, encodeSetO, encodeWhiteSpace):
-                result.append(_utf7_TO_BASE64(charsleft << (6-bitsleft)))
-                charsleft = 0
-                bitsleft = 0
+            if _utf7_ENCODE_DIRECT(oc, not encodeSetO, not encodeWhiteSpace):
+                # shifting out
+                if base64bits: # output remaining bits
+                    result.append(_utf7_TO_BASE64(base64buffer >> (base64bits-6)))
+                    base64buffer = 0
+                    base64bits = 0
+
+                inShift = False
                 ## Characters not in the BASE64 set implicitly unshift the
                 ## sequence so no '-' is required, except if the character is
                 ## itself a '-'
-                if _utf7_B64CHAR(oc) or ch == u'-':
+                if _utf7_IS_BASE64(oc) or ch == u'-':
                     result.append('-')
-                inShift = False
                 result.append(chr(oc))
             else:
-                bitsleft += 16
-                charsleft = (charsleft << 16) | oc
-                bitsleft =  _utf7_ENCODE(result, charsleft, bitsleft)
-                ## If the next character is special then we dont' need to
-                ## terminate the shift sequence. If the next character is not
-                ## a BASE64 character or '-' then the shift sequence will be
-                ## terminated implicitly and we don't have to insert a '-'.
-                if bitsleft == 0:
-                    if pos + 1 < size:
-                        ch2 = s[pos + 1]
-                        oc2 = ord(ch2)
-
-                        if _utf7_SPECIAL(oc2, encodeSetO, encodeWhiteSpace):
-                            pass
-                        elif _utf7_B64CHAR(oc2) or ch2 == u'-':
-                            result.append('-')
-                            inShift = False
-                        else:
-                            inShift = False
-                    else:
-                        result.append('-')
-                        inShift = False
+                base64bits, base64buffer = _utf7_ENCODE_CHAR(
+                    result, oc, base64bits, base64buffer)
         pos += 1
 
-    if bitsleft:
-        result.append(_utf7_TO_BASE64(charsleft << (6 - bitsleft)))
+    if base64bits:
+        result.append(_utf7_TO_BASE64(base64buffer << (6 - base64bits)))
+    if inShift:
         result.append('-')
 
     return result.build()

From commits-noreply at bitbucket.org  Tue Feb  8 18:33:00 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Tue,  8 Feb 2011 18:33:00 +0100 (CET)
Subject: [pypy-svn] pypy default: *really* run the implementation in
	runicode.py
Message-ID: <20110208173300.46FC4282BF8@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41705:ab67f8cf666d
Date: 2011-02-08 15:05 +0100
http://bitbucket.org/pypy/pypy/changeset/ab67f8cf666d/

Log:	*really* run the implementation in runicode.py

diff --git a/pypy/rlib/test/test_runicode.py b/pypy/rlib/test/test_runicode.py
--- a/pypy/rlib/test/test_runicode.py
+++ b/pypy/rlib/test/test_runicode.py
@@ -185,9 +185,16 @@
         assert decode(s, 5, None) == (u'a+-b', 5)
 
     def test_utf7_surrogates(self):
-        assert u'\U000abcde'.encode('utf-7') == '+2m/c3g-'
-        raises(UnicodeError, unicode, '+3ADYAA-', 'utf-7')
-        assert unicode('+3ADYAA-', 'utf-7', 'replace') == u'\ufffd\ufffd'
+        encode = self.getencoder('utf-7')
+        u = u'\U000abcde'
+        assert encode(u, len(u), None) == '+2m/c3g-'
+        decode = self.getdecoder('utf-7')
+        s = '+3ADYAA-'
+        raises(UnicodeError, decode, s, len(s), None)
+        def replace_handler(errors, codec, message, input, start, end):
+            return u'?', end
+        assert decode(s, len(s), None, final=True,
+                      errorhandler = replace_handler) == (u'??', len(s))
 
 
 class TestEncoding(UnicodeTests):

From commits-noreply at bitbucket.org  Tue Feb  8 18:33:01 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Tue,  8 Feb 2011 18:33:01 +0100 (CET)
Subject: [pypy-svn] pypy default: CPython Issue #8271: during the decoding
	of an invalid	UTF-8 byte sequence, 
Message-ID: <20110208173301.218D1282BF8@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41706:fd91ba2df7bb
Date: 2011-02-08 17:13 +0100
http://bitbucket.org/pypy/pypy/changeset/fd91ba2df7bb/

Log:	CPython Issue #8271: during the decoding of an invalid UTF-8 byte
	sequence, only the start byte and the continuation byte(s) are now
	considered invalid, instead of the number of bytes specified by the
	start byte.

diff --git a/pypy/rlib/test/test_runicode.py b/pypy/rlib/test/test_runicode.py
--- a/pypy/rlib/test/test_runicode.py
+++ b/pypy/rlib/test/test_runicode.py
@@ -133,20 +133,126 @@
         for s in [# unexpected end of data
                   "\xd7", "\xd6", "\xeb\x96", "\xf0\x90\x91"]:
             self.checkdecodeerror(s, "utf-8", 0, len(s), addstuff=False)
-            
-        for s in [# unexpected code byte
-                  "\x81", "\xbf",
-                  # invalid data 2 byte
-                  "\xd7\x50", "\xd6\x06", "\xd6\xD6",
-                  # invalid data 3 byte
-                  "\xeb\x56\x95", "\xeb\x06\x95", "\xeb\xD6\x95",
-                  "\xeb\x96\x55", "\xeb\x96\x05", "\xeb\x96\xD5",
-                  # invalid data 4 byte
-                  "\xf0\x50\x91\x93", "\xf0\x00\x91\x93", "\xf0\xd0\x91\x93", 
-                  "\xf0\x90\x51\x93", "\xf0\x90\x01\x93", "\xf0\x90\xd1\x93", 
-                  "\xf0\x90\x91\x53", "\xf0\x90\x91\x03", "\xf0\x90\x91\xd3", 
-                  ]:
-            self.checkdecodeerror(s, "utf-8", 0, len(s), addstuff=True)
+
+        # unexpected code byte
+        for s in ["\x81", "\xbf"]:
+            self.checkdecodeerror(s, "utf-8", 0, 1, addstuff=True)
+
+        # invalid data 2 byte
+        for s in ["\xd7\x50", "\xd6\x06", "\xd6\xD6"]:
+            self.checkdecodeerror(s, "utf-8", 0, 1, addstuff=True)
+        # invalid data 3 byte
+        for s in ["\xeb\x56\x95", "\xeb\x06\x95", "\xeb\xD6\x95"]:
+            self.checkdecodeerror(s, "utf-8", 0, 1, addstuff=True)
+        for s in ["\xeb\x96\x55", "\xeb\x96\x05", "\xeb\x96\xD5"]:
+            self.checkdecodeerror(s, "utf-8", 0, 2, addstuff=True)
+        # invalid data 4 byte
+        for s in ["\xf0\x50\x91\x93", "\xf0\x00\x91\x93", "\xf0\xd0\x91\x93"]:
+            self.checkdecodeerror(s, "utf-8", 0, 1, addstuff=True)
+        for s in ["\xf0\x90\x51\x93", "\xf0\x90\x01\x93", "\xf0\x90\xd1\x93"]:
+            self.checkdecodeerror(s, "utf-8", 0, 2, addstuff=True)
+        for s in ["\xf0\x90\x91\x53", "\xf0\x90\x91\x03", "\xf0\x90\x91\xd3"]:
+            self.checkdecodeerror(s, "utf-8", 0, 3, addstuff=True)
+
+
+    def test_issue8271(self):
+        # From CPython
+        # Issue #8271: during the decoding of an invalid UTF-8 byte sequence,
+        # only the start byte and the continuation byte(s) are now considered
+        # invalid, instead of the number of bytes specified by the start byte.
+        # See http://www.unicode.org/versions/Unicode5.2.0/ch03.pdf (page 95,
+        # table 3-8, Row 2) for more information about the algorithm used.
+        FFFD = u'\ufffd'
+        sequences = [
+            # invalid start bytes
+            ('\x80', FFFD), # continuation byte
+            ('\x80\x80', FFFD*2), # 2 continuation bytes
+            ('\xc0', FFFD),
+            ('\xc0\xc0', FFFD*2),
+            ('\xc1', FFFD),
+            ('\xc1\xc0', FFFD*2),
+            ('\xc0\xc1', FFFD*2),
+            # with start byte of a 2-byte sequence
+            ('\xc2', FFFD), # only the start byte
+            ('\xc2\xc2', FFFD*2), # 2 start bytes
+            ('\xc2\xc2\xc2', FFFD*3), # 2 start bytes
+            ('\xc2\x41', FFFD+'A'), # invalid continuation byte
+            # with start byte of a 3-byte sequence
+            ('\xe1', FFFD), # only the start byte
+            ('\xe1\xe1', FFFD*2), # 2 start bytes
+            ('\xe1\xe1\xe1', FFFD*3), # 3 start bytes
+            ('\xe1\xe1\xe1\xe1', FFFD*4), # 4 start bytes
+            ('\xe1\x80', FFFD), # only 1 continuation byte
+            ('\xe1\x41', FFFD+'A'), # invalid continuation byte
+            ('\xe1\x41\x80', FFFD+'A'+FFFD), # invalid cb followed by valid cb
+            ('\xe1\x41\x41', FFFD+'AA'), # 2 invalid continuation bytes
+            ('\xe1\x80\x41', FFFD+'A'), # only 1 valid continuation byte
+            ('\xe1\x80\xe1\x41', FFFD*2+'A'), # 1 valid and the other invalid
+            ('\xe1\x41\xe1\x80', FFFD+'A'+FFFD), # 1 invalid and the other valid
+            # with start byte of a 4-byte sequence
+            ('\xf1', FFFD), # only the start byte
+            ('\xf1\xf1', FFFD*2), # 2 start bytes
+            ('\xf1\xf1\xf1', FFFD*3), # 3 start bytes
+            ('\xf1\xf1\xf1\xf1', FFFD*4), # 4 start bytes
+            ('\xf1\xf1\xf1\xf1\xf1', FFFD*5), # 5 start bytes
+            ('\xf1\x80', FFFD), # only 1 continuation bytes
+            ('\xf1\x80\x80', FFFD), # only 2 continuation bytes
+            ('\xf1\x80\x41', FFFD+'A'), # 1 valid cb and 1 invalid
+            ('\xf1\x80\x41\x41', FFFD+'AA'), # 1 valid cb and 1 invalid
+            ('\xf1\x80\x80\x41', FFFD+'A'), # 2 valid cb and 1 invalid
+            ('\xf1\x41\x80', FFFD+'A'+FFFD), # 1 invalid cv and 1 valid
+            ('\xf1\x41\x80\x80', FFFD+'A'+FFFD*2), # 1 invalid cb and 2 invalid
+            ('\xf1\x41\x80\x41', FFFD+'A'+FFFD+'A'), # 2 invalid cb and 1 invalid
+            ('\xf1\x41\x41\x80', FFFD+'AA'+FFFD), # 1 valid cb and 1 invalid
+            ('\xf1\x41\xf1\x80', FFFD+'A'+FFFD),
+            ('\xf1\x41\x80\xf1', FFFD+'A'+FFFD*2),
+            ('\xf1\xf1\x80\x41', FFFD*2+'A'),
+            ('\xf1\x41\xf1\xf1', FFFD+'A'+FFFD*2),
+            # with invalid start byte of a 4-byte sequence (rfc2279)
+            ('\xf5', FFFD), # only the start byte
+            ('\xf5\xf5', FFFD*2), # 2 start bytes
+            ('\xf5\x80', FFFD*2), # only 1 continuation byte
+            ('\xf5\x80\x80', FFFD*3), # only 2 continuation byte
+            ('\xf5\x80\x80\x80', FFFD*4), # 3 continuation bytes
+            ('\xf5\x80\x41', FFFD*2+'A'), #  1 valid cb and 1 invalid
+            ('\xf5\x80\x41\xf5', FFFD*2+'A'+FFFD),
+            ('\xf5\x41\x80\x80\x41', FFFD+'A'+FFFD*2+'A'),
+            # with invalid start byte of a 5-byte sequence (rfc2279)
+            ('\xf8', FFFD), # only the start byte
+            ('\xf8\xf8', FFFD*2), # 2 start bytes
+            ('\xf8\x80', FFFD*2), # only one continuation byte
+            ('\xf8\x80\x41', FFFD*2 + 'A'), # 1 valid cb and 1 invalid
+            ('\xf8\x80\x80\x80\x80', FFFD*5), # invalid 5 bytes seq with 5 bytes
+            # with invalid start byte of a 6-byte sequence (rfc2279)
+            ('\xfc', FFFD), # only the start byte
+            ('\xfc\xfc', FFFD*2), # 2 start bytes
+            ('\xfc\x80\x80', FFFD*3), # only 2 continuation bytes
+            ('\xfc\x80\x80\x80\x80\x80', FFFD*6), # 6 continuation bytes
+            # invalid start byte
+            ('\xfe', FFFD),
+            ('\xfe\x80\x80', FFFD*3),
+            # other sequences
+            ('\xf1\x80\x41\x42\x43', u'\ufffd\x41\x42\x43'),
+            ('\xf1\x80\xff\x42\x43', u'\ufffd\ufffd\x42\x43'),
+            ('\xf1\x80\xc2\x81\x43', u'\ufffd\x81\x43'),
+            ('\x61\xF1\x80\x80\xE1\x80\xC2\x62\x80\x63\x80\xBF\x64',
+             u'\x61\uFFFD\uFFFD\uFFFD\x62\uFFFD\x63\uFFFD\uFFFD\x64'),
+        ]
+        def replace_handler(errors, codec, message, input, start, end):
+            return FFFD, end
+        def ignore_handler(errors, codec, message, input, start, end):
+            return u'', end
+        for n, (seq, res) in enumerate(sequences):
+            decoder = self.getdecoder('utf-8')
+            raises(UnicodeDecodeError, decoder, seq, len(seq), None, final=True)
+            assert decoder(seq, len(seq), None, final=True,
+                           errorhandler=replace_handler) == (res, len(seq))
+            assert decoder(seq + 'b', len(seq) + 1, None, final=True,
+                           errorhandler=replace_handler) == (res + u'b',
+                                                             len(seq) + 1)
+            res = res.replace(FFFD, u'')
+            assert decoder(seq, len(seq), None, final=True,
+                           errorhandler=ignore_handler) == (res, len(seq))
 
     def test_ascii_error(self):
         self.checkdecodeerror("abc\xFF\xFF\xFFcde", "ascii", 3, 4)

diff --git a/pypy/rlib/runicode.py b/pypy/rlib/runicode.py
--- a/pypy/rlib/runicode.py
+++ b/pypy/rlib/runicode.py
@@ -59,22 +59,22 @@
 # utf-8
 
 utf8_code_length = [
+    1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, # 00-0F
     1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
     1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
     1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
     1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
     1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
     1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
-    1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
-    1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
+    1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, # 70-7F
+    0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, # 80-8F
     0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
     0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-    0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-    0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-    2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
-    2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
-    3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3,
-    4, 4, 4, 4, 4, 4, 4, 4, 5, 5, 5, 5, 6, 6, 0, 0
+    0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, # B0-BF
+    0, 0, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, # C0-C1 + C2-CF
+    2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, # D0-DF
+    3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, # E0-EF
+    4, 4, 4, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0  # F0-F4 - F5-FF
 ]
 
 def str_decode_utf_8(s, size, errors, final=False,
@@ -99,13 +99,18 @@
             if not final:
                 break
             else:
+                endpos = pos + 1
+                while endpos < size and ord(s[endpos]) & 0xC0 == 0x80:
+                    endpos += 1
                 r, pos = errorhandler(errors, "utf-8",
-                                      "unexpected end of data", s,  pos, size)
+                                      "unexpected end of data",
+                                      s,  pos, endpos)
                 result.append(r)
-                if pos + n > size:
-                    break
+                continue
+
         if n == 0:
-            r, pos = errorhandler(errors, "utf-8", "unexpected code byte",
+            r, pos = errorhandler(errors, "utf-8",
+                                  "invalid start byte",
                                   s,  pos, pos + 1)
             result.append(r)
         elif n == 1:
@@ -118,18 +123,14 @@
             y, six = splitter[5, 3](ordch1)
             assert six == 6
             if two != 2:
-                r, pos = errorhandler(errors, "utf-8", "invalid data",
-                                      s,  pos, pos + 2)
+                r, pos = errorhandler(errors, "utf-8",
+                                      "invalid continuation byte",
+                                      s,  pos, pos + 1)
                 result.append(r)
             else:
                 c = (y << 6) + z
-                if c < 0x80:
-                    r, pos = errorhandler(errors, "utf-8", "illegal encoding",
-                                          s,  pos, pos + 2)
-                    result.append(r)
-                else:
-                    result.append(unichr(c))
-                    pos += n
+                result.append(unichr(c))
+                pos += n
         elif n == 3:
             #  1110xxxx 10yyyyyy 10zzzzzz ====> 00000000 xxxxyyyy yyzzzzzz
             ordch2 = ord(s[pos+1])
@@ -138,24 +139,27 @@
             y, two2 = splitter[6, 2](ordch2)
             x, fourteen = splitter[4, 4](ordch1)
             assert fourteen == 14
-            if two1 != 2 or two2 != 2:
-                r, pos = errorhandler(errors, "utf-8", "invalid data",
-                                      s,  pos, pos + 3)
+            if (two1 != 2 or two2 != 2 or
+                (ordch1 == 0xe0 and ordch2 < 0xa0)
+                # surrogates shouldn't be valid UTF-8!
+                # Uncomment the line below to make them invalid.
+                # or (ordch1 == 0xed and ordch2 > 0x9f)
+                ):
+
+                # if ordch2 first two bits are 1 and 0, then the invalid
+                # continuation byte is ordch3; else ordch2 is invalid.
+                if two2 == 2:
+                    endpos = pos + 2
+                else:
+                    endpos = pos + 1
+                r, pos = errorhandler(errors, "utf-8",
+                                      "invalid continuation byte",
+                                      s,  pos, endpos)
                 result.append(r)
             else:
                 c = (x << 12) + (y << 6) + z
-                # Note: UTF-8 encodings of surrogates are considered
-                # legal UTF-8 sequences;
-                # XXX For wide builds (UCS-4) we should probably try
-                #     to recombine the surrogates into a single code
-                #     unit.
-                if c < 0x0800:
-                    r, pos = errorhandler(errors, "utf-8", "illegal encoding",
-                                          s,  pos, pos + 3)
-                    result.append(r)
-                else:
-                    result.append(unichr(c))
-                    pos += n
+                result.append(unichr(c))
+                pos += n
         elif n == 4:
             # 11110www 10xxxxxx 10yyyyyy 10zzzzzz ====>
             # 000wwwxx xxxxyyyy yyzzzzzz
@@ -167,31 +171,32 @@
             x, two3 = splitter[6, 2](ordch2)
             w, thirty = splitter[3, 5](ordch1)
             assert thirty == 30
-            if two1 != 2 or two2 != 2 or two3 != 2:
-                r, pos = errorhandler(errors, "utf-8", "invalid data",
-                                      s,  pos, pos + 4)
+            if (two1 != 2 or two2 != 2 or two3 != 2 or
+                (ordch1 == 0xf0 and ordch2 < 0x90) or
+                (ordch1 == 0xf4 and ordch2 > 0x8f)):
+                endpos = pos + 1
+                if ordch2 & 0xc0 == 0x80:
+                    endpos += 1
+                    if ordch3 & 0xc0 == 0x80:
+                        endpos += 1
+                r, pos = errorhandler(errors, "utf-8",
+                                      "invalid continuation byte",
+                                      s,  pos, endpos)
                 result.append(r)
             else:
                 c = (w << 18) + (x << 12) + (y << 6) + z
-                # minimum value allowed for 4 byte encoding
-                # maximum value allowed for UTF-16
-                if c < 0x10000 or c > 0x10ffff:
-                    r, pos = errorhandler(errors, "utf-8", "illegal encoding",
-                                          s,  pos, pos + 4)
-                    result.append(r)
+                # convert to UTF-16 if necessary
+                if c <= MAXUNICODE:
+                    result.append(UNICHR(c))
                 else:
-                    # convert to UTF-16 if necessary
-                    if c <= MAXUNICODE:
-                        result.append(UNICHR(c))
-                    else:
-                        # compute and append the two surrogates:
-                        # translate from 10000..10FFFF to 0..FFFF
-                        c -= 0x10000
-                        # high surrogate = top 10 bits added to D800
-                        result.append(unichr(0xD800 + (c >> 10)))
-                        # low surrogate = bottom 10 bits added to DC00
-                        result.append(unichr(0xDC00 + (c & 0x03FF)))
-                    pos += n
+                    # compute and append the two surrogates:
+                    # translate from 10000..10FFFF to 0..FFFF
+                    c -= 0x10000
+                    # high surrogate = top 10 bits added to D800
+                    result.append(unichr(0xD800 + (c >> 10)))
+                    # low surrogate = bottom 10 bits added to DC00
+                    result.append(unichr(0xDC00 + (c & 0x03FF)))
+                pos += n
         else:
             r, pos = errorhandler(errors, "utf-8",
                                   "unsupported Unicode code range",

From commits-noreply at bitbucket.org  Tue Feb  8 18:33:03 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Tue,  8 Feb 2011 18:33:03 +0100 (CET)
Subject: [pypy-svn] pypy default: Translation fixes
Message-ID: <20110208173303.5DDD9282BF8@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41707:6c63e187e82c
Date: 2011-02-08 18:26 +0100
http://bitbucket.org/pypy/pypy/changeset/6c63e187e82c/

Log:	Translation fixes

diff --git a/pypy/rlib/runicode.py b/pypy/rlib/runicode.py
--- a/pypy/rlib/runicode.py
+++ b/pypy/rlib/runicode.py
@@ -763,10 +763,9 @@
                     # preserved
                     pass
                 else:
-                    result.append(ch)
+                    result.append(unichr(ord(ch)))
 
         elif ch == '+':
-            startinpos = pos
             pos += 1 # consume '+'
             if pos < size and s[pos] == '-': # '+-' encodes '+'
                 pos += 1
@@ -794,7 +793,7 @@
             (base64bits > 0 and base64buffer != 0)):
             endinpos = size
             msg = "unterminated shift sequence"
-            res, pos = errorhandler(errors, 'utf-7', msg, s, startinpos, pos)
+            res, pos = errorhandler(errors, 'utf-7', msg, s, shiftOutStartPos, pos)
             result.append(res)
     elif inShift:
         pos = shiftOutStartPos # back off output
@@ -1249,7 +1248,6 @@
             pos += 1
             continue
 
-        startinpos = pos
         # \u-escapes are only interpreted iff the number of leading
         # backslashes is odd
         bs = pos

From commits-noreply at bitbucket.org  Tue Feb  8 18:33:04 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Tue,  8 Feb 2011 18:33:04 +0100 (CET)
Subject: [pypy-svn] pypy default: Loosen the regexp that matches the MSVC
	banner, to cope with translations of the version string.
Message-ID: <20110208173304.B2DB1282BF8@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41708:3ce31c149e51
Date: 2011-02-08 18:29 +0100
http://bitbucket.org/pypy/pypy/changeset/3ce31c149e51/

Log:	Loosen the regexp that matches the MSVC banner, to cope with
	translations of the version string.

diff --git a/pypy/translator/platform/windows.py b/pypy/translator/platform/windows.py
--- a/pypy/translator/platform/windows.py
+++ b/pypy/translator/platform/windows.py
@@ -89,7 +89,7 @@
         # detect version of current compiler
         returncode, stdout, stderr = _run_subprocess(self.cc, '',
                                                      env=self.c_environ)
-        r = re.search('[Vv]ersion\W([0-9]+)\.([0-9]+)', stderr)
+        r = re.match(r'Microsoft.+C/C\+\+.+\s([0-9]+)\.([0-9]+).*', stderr)
         if r is not None:
             self.version = int(''.join(r.groups())) / 10 - 60
         else:

From commits-noreply at bitbucket.org  Tue Feb  8 18:33:05 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Tue,  8 Feb 2011 18:33:05 +0100 (CET)
Subject: [pypy-svn] pypy default: Merge heads
Message-ID: <20110208173305.2476C282BFC@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41709:f433a552303f
Date: 2011-02-08 18:29 +0100
http://bitbucket.org/pypy/pypy/changeset/f433a552303f/

Log:	Merge heads


From commits-noreply at bitbucket.org  Tue Feb  8 19:30:29 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Tue,  8 Feb 2011 19:30:29 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: fixed test
Message-ID: <20110208183029.BD5E3282BF6@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r41710:8b2870b07feb
Date: 2011-02-08 18:40 +0100
http://bitbucket.org/pypy/pypy/changeset/8b2870b07feb/

Log:	fixed test

diff --git a/pypy/jit/metainterp/test/test_send.py b/pypy/jit/metainterp/test/test_send.py
--- a/pypy/jit/metainterp/test/test_send.py
+++ b/pypy/jit/metainterp/test/test_send.py
@@ -479,7 +479,7 @@
         if self.optimizer != OPTIMIZER_FULL:
             self.check_tree_loop_count(1)
         else:
-            self.check_tree_loop_count(6)
+            self.check_tree_loop_count(4)
 
     def test_three_classes(self):
         class Base:

From commits-noreply at bitbucket.org  Tue Feb  8 19:30:30 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Tue,  8 Feb 2011 19:30:30 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: we now get a second specialied
	version
Message-ID: <20110208183030.61692282BF6@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r41711:018f5b6f12c6
Date: 2011-02-08 18:44 +0100
http://bitbucket.org/pypy/pypy/changeset/018f5b6f12c6/

Log:	we now get a second specialied version

diff --git a/pypy/jit/metainterp/test/test_recursive.py b/pypy/jit/metainterp/test/test_recursive.py
--- a/pypy/jit/metainterp/test/test_recursive.py
+++ b/pypy/jit/metainterp/test/test_recursive.py
@@ -1070,7 +1070,7 @@
         assert portal(0, 1) == 2095
         res = self.meta_interp(portal, [0, 1], inline=True)
         assert res == 2095
-        self.check_loops(call_assembler=6, everywhere=True)
+        self.check_loops(call_assembler=12, everywhere=True)
 
     def test_inline_with_hitting_the_loop_sometimes_exc(self):
         driver = JitDriver(greens = ['codeno'], reds = ['i', 'k'],
@@ -1108,7 +1108,7 @@
         assert main(0, 1) == 2095
         res = self.meta_interp(main, [0, 1], inline=True)
         assert res == 2095
-        self.check_loops(call_assembler=6, everywhere=True)
+        self.check_loops(call_assembler=12, everywhere=True)
 
     def test_handle_jitexception_in_portal(self):
         # a test for _handle_jitexception_in_portal in blackhole.py

From commits-noreply at bitbucket.org  Tue Feb  8 19:30:30 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Tue,  8 Feb 2011 19:30:30 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: we already do
Message-ID: <20110208183030.F1743282BF6@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r41712:1f5bf0398d28
Date: 2011-02-08 19:01 +0100
http://bitbucket.org/pypy/pypy/changeset/1f5bf0398d28/

Log:	we already do

diff --git a/pypy/jit/metainterp/test/test_virtual.py b/pypy/jit/metainterp/test/test_virtual.py
--- a/pypy/jit/metainterp/test/test_virtual.py
+++ b/pypy/jit/metainterp/test/test_virtual.py
@@ -562,10 +562,6 @@
                 n -= 1
             return node1.value + node2.value
         assert self.meta_interp(f, [40, 3]) == f(40, 3)
-        
-        def g(n1, n2, s):
-            return f(n1, s) + f(n2, s)
-        # FIXME: Try the case where we need to call the second version from the interpreter
 
     def test_virtual_array_bridge(self):
         myjitdriver = JitDriver(greens = [], reds = ['n', 'node'])
@@ -596,8 +592,6 @@
                 n -= 1
             return node[0] + node[1]
         assert self.meta_interp(f, [40]) == f(40)
-        # FIXME: Try the case where we need to call the second version from the interpreter
-        
         
 
 class VirtualMiscTests:

From commits-noreply at bitbucket.org  Tue Feb  8 19:30:34 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Tue,  8 Feb 2011 19:30:34 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: dont make the same mistake twice
Message-ID: <20110208183034.96BD3282BF6@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r41713:3ee660b2f234
Date: 2011-02-08 19:29 +0100
http://bitbucket.org/pypy/pypy/changeset/3ee660b2f234/

Log:	dont make the same mistake twice

diff --git a/pypy/jit/metainterp/optimizeopt/unroll.py b/pypy/jit/metainterp/optimizeopt/unroll.py
--- a/pypy/jit/metainterp/optimizeopt/unroll.py
+++ b/pypy/jit/metainterp/optimizeopt/unroll.py
@@ -671,8 +671,20 @@
                                         "jumping to preamble instead")
                             self.emit_operation(op)
                         return
-                if not self.retraced:    
-                    raise RetraceLoop
+                if not self.retraced:
+                    if not descr.failed_states:
+                        raise RetraceLoop
+                    for failed in descr.failed_states:
+                        if failed.generalization_of(virtual_state):
+                            # Retracing once more will most likely fail again
+                            break
+                    else:
+                        raise RetraceLoop
+                else:
+                    if not descr.failed_states:
+                        descr.failed_states=[virtual_state]
+                    else:
+                        descr.failed_states.append(virtual_state)
         self.emit_operation(op)
                 
         

diff --git a/pypy/jit/metainterp/test/test_basic.py b/pypy/jit/metainterp/test/test_basic.py
--- a/pypy/jit/metainterp/test/test_basic.py
+++ b/pypy/jit/metainterp/test/test_basic.py
@@ -1498,8 +1498,8 @@
             return x
         res = self.meta_interp(f, [299], listops=True)
         assert res == f(299)
-        self.check_loops(guard_class=0, guard_value=3)        
-        self.check_loops(guard_class=0, guard_value=6, everywhere=True)
+        self.check_loops(guard_class=0, guard_value=2)        
+        self.check_loops(guard_class=0, guard_value=5, everywhere=True)
 
     def test_merge_guardnonnull_guardclass(self):
         from pypy.rlib.objectmodel import instantiate
@@ -1528,9 +1528,9 @@
         res = self.meta_interp(f, [299], listops=True)
         assert res == f(299)
         self.check_loops(guard_class=0, guard_nonnull=0,
-                         guard_nonnull_class=2, guard_isnull=1)
+                         guard_nonnull_class=2, guard_isnull=0)
         self.check_loops(guard_class=0, guard_nonnull=0,
-                         guard_nonnull_class=4, guard_isnull=2,
+                         guard_nonnull_class=4, guard_isnull=1,
                          everywhere=True)
 
     def test_merge_guardnonnull_guardvalue(self):
@@ -1558,9 +1558,9 @@
             return x
         res = self.meta_interp(f, [299], listops=True)
         assert res == f(299)
-        self.check_loops(guard_class=0, guard_nonnull=0, guard_value=2,
+        self.check_loops(guard_class=0, guard_nonnull=0, guard_value=1,
                          guard_nonnull_class=0, guard_isnull=1)
-        self.check_loops(guard_class=0, guard_nonnull=0, guard_value=4,
+        self.check_loops(guard_class=0, guard_nonnull=0, guard_value=3,
                          guard_nonnull_class=0, guard_isnull=2,
                          everywhere=True)
 
@@ -1590,9 +1590,9 @@
         res = self.meta_interp(f, [299], listops=True)
         assert res == f(299)
         self.check_loops(guard_class=0, guard_nonnull=0, guard_value=2,
-                         guard_nonnull_class=0, guard_isnull=1)
+                         guard_nonnull_class=0, guard_isnull=0)
         self.check_loops(guard_class=0, guard_nonnull=0, guard_value=4,
-                         guard_nonnull_class=0, guard_isnull=2,
+                         guard_nonnull_class=0, guard_isnull=1,
                          everywhere=True)
 
     def test_merge_guardnonnull_guardclass_guardvalue(self):
@@ -1623,10 +1623,10 @@
             return x
         res = self.meta_interp(f, [399], listops=True)
         assert res == f(399)
-        self.check_loops(guard_class=0, guard_nonnull=0, guard_value=3,
-                         guard_nonnull_class=0, guard_isnull=1)
-        self.check_loops(guard_class=0, guard_nonnull=0, guard_value=6,
-                         guard_nonnull_class=0, guard_isnull=2,
+        self.check_loops(guard_class=0, guard_nonnull=0, guard_value=2,
+                         guard_nonnull_class=0, guard_isnull=0)
+        self.check_loops(guard_class=0, guard_nonnull=0, guard_value=5,
+                         guard_nonnull_class=0, guard_isnull=1,
                          everywhere=True)
 
     def test_residual_call_doesnt_lose_info(self):

diff --git a/pypy/jit/metainterp/history.py b/pypy/jit/metainterp/history.py
--- a/pypy/jit/metainterp/history.py
+++ b/pypy/jit/metainterp/history.py
@@ -749,6 +749,7 @@
     generated assembler.
     """
     short_preamble = None
+    failed_states = None
     terminating = False # see TerminatingLoopToken in compile.py
     outermost_jitdriver_sd = None
     # and more data specified by the backend when the loop is compiled

diff --git a/pypy/jit/metainterp/test/test_virtual.py b/pypy/jit/metainterp/test/test_virtual.py
--- a/pypy/jit/metainterp/test/test_virtual.py
+++ b/pypy/jit/metainterp/test/test_virtual.py
@@ -562,6 +562,7 @@
                 n -= 1
             return node1.value + node2.value
         assert self.meta_interp(f, [40, 3]) == f(40, 3)
+        self.check_loop_count(6)
 
     def test_virtual_array_bridge(self):
         myjitdriver = JitDriver(greens = [], reds = ['n', 'node'])

From commits-noreply at bitbucket.org  Tue Feb  8 19:42:38 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Tue,  8 Feb 2011 19:42:38 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: never retrace a loop more than 5
	times
Message-ID: <20110208184238.EDF99282BF6@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r41714:91586ef4b9c5
Date: 2011-02-08 19:42 +0100
http://bitbucket.org/pypy/pypy/changeset/91586ef4b9c5/

Log:	never retrace a loop more than 5 times

diff --git a/pypy/jit/metainterp/optimizeopt/unroll.py b/pypy/jit/metainterp/optimizeopt/unroll.py
--- a/pypy/jit/metainterp/optimizeopt/unroll.py
+++ b/pypy/jit/metainterp/optimizeopt/unroll.py
@@ -671,7 +671,10 @@
                                         "jumping to preamble instead")
                             self.emit_operation(op)
                         return
-                if not self.retraced:
+                retraced_count = len(short)
+                if descr.failed_states:
+                    retraced_count += len(descr.failed_states)
+                if not self.retraced and retraced_count<5:
                     if not descr.failed_states:
                         raise RetraceLoop
                     for failed in descr.failed_states:

From commits-noreply at bitbucket.org  Wed Feb  9 01:58:13 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed,  9 Feb 2011 01:58:13 +0100 (CET)
Subject: [pypy-svn] pypy default: Includes the interp-level implementation
	of the _locale	module.
Message-ID: <20110209005813.E608E36C536@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41715:415e713bc52d
Date: 2011-02-09 01:28 +0100
http://bitbucket.org/pypy/pypy/changeset/415e713bc52d/

Log:	Includes the interp-level implementation of the _locale module. (it
	was already translated on Windows) this should unskip test_locale.py

diff --git a/pypy/config/pypyoption.py b/pypy/config/pypyoption.py
--- a/pypy/config/pypyoption.py
+++ b/pypy/config/pypyoption.py
@@ -27,7 +27,7 @@
 # --allworkingmodules
 working_modules = default_modules.copy()
 working_modules.update(dict.fromkeys(
-    ["_socket", "unicodedata", "mmap", "fcntl",
+    ["_socket", "unicodedata", "mmap", "fcntl", "_locale",
      "rctime" , "select", "zipimport", "_lsprof",
      "crypt", "signal", "_rawffi", "termios", "zlib", "bz2",
      "struct", "_hashlib", "_md5", "_sha", "_minimal_curses", "cStringIO",

From commits-noreply at bitbucket.org  Wed Feb  9 01:58:15 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed,  9 Feb 2011 01:58:15 +0100 (CET)
Subject: [pypy-svn] pypy default: sqlite3 is now part of Python,
	there is little interest
Message-ID: <20110209005815.BD785282BF6@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41716:edddd05bbacb
Date: 2011-02-09 01:32 +0100
http://bitbucket.org/pypy/pypy/changeset/edddd05bbacb/

Log:	sqlite3 is now part of Python, there is little interest to keep it
	as an external resource.

	(The new _sqlite3.py is mostly made of the previous dbapi2.py)

diff --git a/.hgsub b/.hgsub
--- a/.hgsub
+++ b/.hgsub
@@ -1,4 +1,3 @@
 greenlet = [svn]http://codespeak.net/svn/greenlet/trunk/c
 testrunner = [svn]http://codespeak.net/svn/pypy/build/testrunner
 lib_pypy/pyrepl = [svn]http://codespeak.net/svn/pyrepl/trunk/pyrepl/pyrepl
-lib_pypy/sqlite3 = [svn]http://codespeak.net/svn/pypy/pysqlite2

diff --git a/lib_pypy/_sqlite3.py b/lib_pypy/_sqlite3.py
new file mode 100644
--- /dev/null
+++ b/lib_pypy/_sqlite3.py
@@ -0,0 +1,968 @@
+#-*- coding: utf-8 -*-
+# pysqlite2/dbapi.py: pysqlite DB-API module
+#
+# Copyright (C) 2007-2008 Gerhard H&#228;ring <gh at ghaering.de>
+#
+# This file is part of pysqlite.
+#
+# This software is provided 'as-is', without any express or implied
+# warranty.  In no event will the authors be held liable for any damages
+# arising from the use of this software.
+#
+# Permission is granted to anyone to use this software for any purpose,
+# including commercial applications, and to alter it and redistribute it
+# freely, subject to the following restrictions:
+#
+# 1. The origin of this software must not be misrepresented; you must not
+#    claim that you wrote the original software. If you use this software
+#    in a product, an acknowledgment in the product documentation would be
+#    appreciated but is not required.
+# 2. Altered source versions must be plainly marked as such, and must not be
+#    misrepresented as being the original software.
+# 3. This notice may not be removed or altered from any source distribution.
+
+from ctypes import c_void_p, c_int, c_double, c_int64, c_char_p, cdll
+from ctypes import POINTER, byref, string_at, CFUNCTYPE, cast
+import datetime
+import sys
+import time
+import weakref
+from threading import _get_ident as thread_get_ident
+
+names = "sqlite3.dll libsqlite3.so.0 libsqlite3.so libsqlite3.dylib".split()
+for name in names: 
+    try:
+        sqlite = cdll.LoadLibrary(name) 
+        break
+    except OSError:
+        continue
+else:
+    raise ImportError("Could not load C-library, tried: %s" %(names,))
+
+# pysqlite version information
+version = "2.6.0"
+
+# pysqlite constants
+PARSE_COLNAMES = 1
+PARSE_DECLTYPES = 2
+
+
+##########################################
+# BEGIN Wrapped SQLite C API and constants
+##########################################
+
+SQLITE_OK = 0
+SQLITE_ERROR = 1
+SQLITE_INTERNAL = 2
+SQLITE_PERM = 3
+SQLITE_ABORT = 4
+SQLITE_BUSY = 5
+SQLITE_LOCKED = 6
+SQLITE_NOMEM = 7
+SQLITE_READONLY = 8
+SQLITE_INTERRUPT = 9
+SQLITE_IOERR = 10
+SQLITE_CORRUPT = 11
+SQLITE_NOTFOUND = 12
+SQLITE_FULL = 13
+SQLITE_CANTOPEN = 14
+SQLITE_PROTOCOL = 15
+SQLITE_EMPTY = 16
+SQLITE_SCHEMA = 17
+SQLITE_TOOBIG = 18
+SQLITE_CONSTRAINT = 19
+SQLITE_MISMATCH = 20
+SQLITE_MISUSE = 21
+SQLITE_NOLFS = 22
+SQLITE_AUTH = 23
+SQLITE_FORMAT = 24
+SQLITE_RANGE = 25
+SQLITE_NOTADB = 26
+SQLITE_ROW = 100
+SQLITE_DONE = 101
+SQLITE_INTEGER = 1
+SQLITE_FLOAT = 2
+SQLITE_BLOB = 4
+SQLITE_NULL = 5
+SQLITE_TEXT = 3
+SQLITE3_TEXT = 3
+
+SQLITE_TRANSIENT = cast(-1, c_void_p)
+SQLITE_UTF8 = 1
+
+SQLITE_DENY     = 1
+SQLITE_IGNORE   = 2
+
+SQLITE_CREATE_INDEX             = 1
+SQLITE_CREATE_TABLE             = 2
+SQLITE_CREATE_TEMP_INDEX        = 3
+SQLITE_CREATE_TEMP_TABLE        = 4
+SQLITE_CREATE_TEMP_TRIGGER      = 5
+SQLITE_CREATE_TEMP_VIEW         = 6
+SQLITE_CREATE_TRIGGER           = 7
+SQLITE_CREATE_VIEW              = 8
+SQLITE_DELETE                   = 9
+SQLITE_DROP_INDEX               = 10
+SQLITE_DROP_TABLE               = 11
+SQLITE_DROP_TEMP_INDEX          = 12
+SQLITE_DROP_TEMP_TABLE          = 13
+SQLITE_DROP_TEMP_TRIGGER        = 14
+SQLITE_DROP_TEMP_VIEW           = 15
+SQLITE_DROP_TRIGGER             = 16
+SQLITE_DROP_VIEW                = 17
+SQLITE_INSERT                   = 18
+SQLITE_PRAGMA                   = 19
+SQLITE_READ                     = 20
+SQLITE_SELECT                   = 21
+SQLITE_TRANSACTION              = 22
+SQLITE_UPDATE                   = 23
+SQLITE_ATTACH                   = 24
+SQLITE_DETACH                   = 25
+SQLITE_ALTER_TABLE              = 26
+SQLITE_REINDEX                  = 27
+SQLITE_ANALYZE                  = 28
+SQLITE_CREATE_VTABLE            = 29
+SQLITE_DROP_VTABLE              = 30
+SQLITE_FUNCTION                 = 31
+
+# SQLite C API
+sqlite.sqlite3_bind_double.argtypes = [c_void_p, c_int, c_double]
+sqlite.sqlite3_bind_int64.argtypes = [c_void_p, c_int, c_int64]
+
+sqlite.sqlite3_value_int.argtypes = [c_void_p]
+sqlite.sqlite3_value_int.restype = c_int
+
+sqlite.sqlite3_value_int64.argtypes = [c_void_p]
+sqlite.sqlite3_value_int64.restype = c_int64
+
+sqlite.sqlite3_value_blob.argtypes = [c_void_p]
+sqlite.sqlite3_value_blob.restype = c_void_p
+
+sqlite.sqlite3_value_bytes.argtypes = [c_void_p]
+sqlite.sqlite3_value_bytes.restype = c_int
+
+sqlite.sqlite3_value_double.argtypes = [c_void_p]
+sqlite.sqlite3_value_double.restype = c_double
+
+sqlite.sqlite3_value_text.argtypes = [c_void_p]
+sqlite.sqlite3_value_text.restype = c_char_p
+
+sqlite.sqlite3_bind_int.argtypes = [c_void_p, c_int, c_int]
+sqlite.sqlite3_bind_parameter_count.argtypes = [c_void_p]
+sqlite.sqlite3_bind_parameter_count.restype = c_int
+sqlite.sqlite3_bind_parameter_index.argtypes = [c_void_p, c_char_p]
+sqlite.sqlite3_bind_parameter_index.restype = c_int
+sqlite.sqlite3_bind_parameter_name.argtypes = [c_void_p, c_int]
+sqlite.sqlite3_bind_parameter_name.restype = c_char_p
+sqlite.sqlite3_bind_text.argtypes = [c_void_p, c_int, c_char_p, c_int,c_void_p]
+sqlite.sqlite3_bind_blob.argtypes = [c_void_p, c_int, c_void_p, c_int,c_void_p]
+sqlite.sqlite3_bind_blob.restype = c_int
+sqlite.sqlite3_changes.argtypes = [c_void_p]
+sqlite.sqlite3_changes.restype = c_int
+sqlite.sqlite3_close.argtypes = [c_void_p]
+sqlite.sqlite3_close.restype = c_int
+sqlite.sqlite3_column_blob.restype = c_void_p
+sqlite.sqlite3_column_bytes.restype = c_int
+sqlite.sqlite3_column_double.restype = c_double
+sqlite.sqlite3_column_int64.restype = c_int64
+sqlite.sqlite3_column_name.restype = c_char_p
+sqlite.sqlite3_column_text.restype = c_char_p
+sqlite.sqlite3_complete.argtypes = [c_char_p]
+sqlite.sqlite3_complete.restype = c_int
+sqlite.sqlite3_errcode.restype = c_int
+sqlite.sqlite3_errmsg.restype = c_char_p
+sqlite.sqlite3_get_autocommit.argtypes = [c_void_p]
+sqlite.sqlite3_get_autocommit.restype = c_int
+sqlite.sqlite3_libversion.restype = c_char_p
+sqlite.sqlite3_open.argtypes = [c_char_p, c_void_p]
+sqlite.sqlite3_prepare_v2.argtypes = [c_void_p, c_char_p, c_int, c_void_p, POINTER(c_char_p)]
+sqlite.sqlite3_column_decltype.argtypes = [c_void_p, c_int]
+sqlite.sqlite3_column_decltype.restype = c_char_p
+
+sqlite.sqlite3_result_blob.argtypes = [c_void_p, c_char_p, c_int, c_void_p]
+sqlite.sqlite3_result_int64.argtypes = [c_void_p, c_int64]
+sqlite.sqlite3_result_null.argtypes = [c_void_p]
+sqlite.sqlite3_result_double.argtypes = [c_void_p, c_double]
+sqlite.sqlite3_result_error.argtypes = [c_void_p, c_char_p, c_int]
+sqlite.sqlite3_result_text.argtypes = [c_void_p, c_char_p, c_int, c_void_p]
+
+##########################################
+# END Wrapped SQLite C API and constants
+##########################################
+
+# SQLite version information
+sqlite_version = sqlite.sqlite3_libversion()
+
+class Error(StandardError):
+    pass
+
+class Warning(StandardError):
+    pass
+
+class InterfaceError(Error):
+    pass
+
+class DatabaseError(Error):
+    pass
+
+class InternalError(DatabaseError):
+    pass
+
+class OperationalError(DatabaseError):
+    pass
+
+class ProgrammingError(DatabaseError):
+    pass
+
+class IntegrityError(DatabaseError):
+    pass
+
+class DataError(DatabaseError):
+    pass
+
+class NotSupportedError(DatabaseError):
+    pass
+
+def connect(database, **kwargs):
+    factory = kwargs.get("factory", Connection)
+    return factory(database, **kwargs)
+
+class Connection(object):
+    def __init__(self, database, isolation_level="", detect_types=0, timeout=None, *args, **kwargs):
+        self.db = c_void_p()
+        if sqlite.sqlite3_open(database, byref(self.db)) != SQLITE_OK:
+            raise OperationalError("Could not open database")
+        if timeout is not None:
+            timeout = int(timeout * 1000) # pysqlite2 uses timeout in seconds
+            sqlite.sqlite3_busy_timeout(self.db, timeout)
+
+        self.text_factory = lambda x: unicode(x, "utf-8")
+        self.closed = False
+        self.statements = []
+        self.statement_counter = 0
+        self.row_factory = None
+        self._isolation_level = isolation_level
+        self.detect_types = detect_types
+
+        self.Error = Error
+        self.Warning = Warning
+        self.InterfaceError = InterfaceError
+        self.DatabaseError = DatabaseError
+        self.InternalError = InternalError
+        self.OperationalError = OperationalError
+        self.ProgrammingError = ProgrammingError
+        self.IntegrityError = IntegrityError
+        self.DataError = DataError
+        self.NotSupportedError = NotSupportedError
+
+        self.func_cache = {}
+        self.thread_ident = thread_get_ident()
+
+    def _get_exception(self, error_code = None):
+        if error_code is None:
+            error_code = sqlite.sqlite3_errcode(self.db)
+        error_message = sqlite.sqlite3_errmsg(self.db)
+
+        if error_code == SQLITE_OK:
+            raise ValueError("error signalled but got SQLITE_OK")
+        elif error_code in (SQLITE_INTERNAL, SQLITE_NOTFOUND):
+            exc = InternalError
+        elif error_code == SQLITE_NOMEM:
+            exc = MemoryError
+        elif error_code in (SQLITE_ERROR, SQLITE_PERM, SQLITE_ABORT, SQLITE_BUSY, SQLITE_LOCKED,
+            SQLITE_READONLY, SQLITE_INTERRUPT, SQLITE_IOERR, SQLITE_FULL, SQLITE_CANTOPEN,
+            SQLITE_PROTOCOL, SQLITE_EMPTY, SQLITE_SCHEMA):
+            exc = OperationalError
+        elif error_code == SQLITE_CORRUPT:
+            exc = DatabaseError
+        elif error_code == SQLITE_TOOBIG:
+            exc = DataError
+        elif error_code in (SQLITE_CONSTRAINT, SQLITE_MISMATCH):
+            exc = IntegrityError
+        elif error_code == SQLITE_MISUSE:
+            exc = ProgrammingError
+        else:
+            exc = DatabaseError
+        exc = exc(error_message)
+        exc.error_code = error_code
+        return exc
+
+    def _remember_statement(self, statement):
+        self.statements.append(weakref.ref(statement))
+        self.statement_counter += 1
+
+        if self.statement_counter % 100 == 0:
+            self.statements = [ref for ref in self.statements if ref() is not None]
+
+    def _check_thread(self):
+        if self.thread_ident != thread_get_ident():
+            raise ProgrammingError(
+                "SQLite objects created in a thread can only be used in that same thread."
+                "The object was created in thread id %d and this is thread id %d",
+                self.thread_ident, thread_get_ident())
+
+    def cursor(self, factory=None):
+        self._check_thread()
+        self._check_closed()
+        if factory is None:
+            factory = Cursor
+        cur = factory(self)
+        if self.row_factory is not None:
+            cur.row_factory = self.row_factory
+        return cur
+
+    def executemany(self, *args):
+        self._check_closed()
+        cur = Cursor(self)
+        if self.row_factory is not None:
+            cur.row_factory = self.row_factory
+        return cur.executemany(*args)
+
+    def execute(self, *args):
+        self._check_closed()
+        cur = Cursor(self)
+        if self.row_factory is not None:
+            cur.row_factory = self.row_factory
+        return cur.execute(*args)
+
+    def executescript(self, *args):
+        self._check_closed()
+        cur = Cursor(self)
+        if self.row_factory is not None:
+            cur.row_factory = self.row_factory
+        return cur.executescript(*args)
+
+    def _get_isolation_level(self):
+        return self._isolation_level
+    def _set_isolation_level(self, val):
+        if val is None:
+            self.commit()
+        self._isolation_level = val
+    isolation_level = property(_get_isolation_level, _set_isolation_level)
+
+    def _begin(self):
+        self._check_closed()
+        if self._isolation_level is None:
+            return
+        if sqlite.sqlite3_get_autocommit(self.db):
+            try:
+                sql = "BEGIN " + self._isolation_level
+                statement = c_void_p()
+                next_char = c_char_p()
+                ret = sqlite.sqlite3_prepare_v2(self.db, sql, -1, byref(statement), next_char)
+                if ret != SQLITE_OK:
+                    raise self._get_exception(ret)
+                ret = sqlite.sqlite3_step(statement)
+                if ret != SQLITE_DONE:
+                    raise self._get_exception(ret)
+            finally:
+                sqlite.sqlite3_finalize(statement)
+
+    def commit(self):
+        self._check_thread()
+        self._check_closed()
+        if sqlite.sqlite3_get_autocommit(self.db):
+            return
+        try:
+            sql = "COMMIT"
+            statement = c_void_p()
+            next_char = c_char_p()
+            ret = sqlite.sqlite3_prepare_v2(self.db, sql, -1, byref(statement), next_char)
+            if ret != SQLITE_OK:
+                raise self._get_exception(ret)
+            ret = sqlite.sqlite3_step(statement)
+            if ret != SQLITE_DONE:
+                raise self._get_exception(ret)
+        finally:
+            sqlite.sqlite3_finalize(statement)
+
+    def rollback(self):
+        self._check_thread()
+        self._check_closed()
+        if sqlite.sqlite3_get_autocommit(self.db):
+            return
+        try:
+            sql = "ROLLBACK"
+            statement = c_void_p()
+            next_char = c_char_p()
+            ret = sqlite.sqlite3_prepare_v2(self.db, sql, -1, byref(statement), next_char)
+            if ret != SQLITE_OK:
+                raise self._get_exception(ret)
+            ret = sqlite.sqlite3_step(statement)
+            if ret != SQLITE_DONE:
+                raise self._get_exception(ret)
+        finally:
+            sqlite.sqlite3_finalize(statement)
+
+    def _check_closed(self):
+        if self.closed:
+            raise ProgrammingError("Cannot operate on a closed database.")
+
+    def __enter__(self):
+        return self
+
+    def __exit__(self, exc_type, exc_value, exc_tb):
+        if exc_type is None and exc_value is None and exc_tb is None:
+            self.commit()
+        else:
+            self.rollback()
+
+    def _get_total_changes(self):
+        return sqlite.sqlite3_total_changes(self.db)
+    total_changes = property(_get_total_changes)
+
+    def close(self):
+        self._check_thread()
+        if self.closed:
+            return
+        for statement in self.statements:
+            obj = statement()
+            if obj is not None:
+                obj.finalize()
+
+        self.closed = True
+        ret = sqlite.sqlite3_close(self.db)
+        if ret != SQLITE_OK:
+            raise self._get_exception(ret)
+
+    def create_collation(self, name, callback):
+        raise NotImplementedError
+
+    def set_progress_handler(self, callable, nsteps):
+        raise NotImplementedError
+
+    def set_authorizer(self, callback):
+        raise NotImplementedError
+
+    def create_function(self, name, num_args, callback):
+        self._check_thread()
+        self._check_closed()
+        try:
+            c_closure, _ = self.func_cache[callback]
+        except KeyError:            
+            def closure(context, nargs, c_params):
+                function_callback(callback, context, nargs, c_params)
+                return 0
+            c_closure = FUNC(closure)
+            self.func_cache[callback] = c_closure, closure
+        ret = sqlite.sqlite3_create_function(self.db, name, num_args,
+                                             SQLITE_UTF8, None,
+                                             c_closure,
+                                             cast(None, STEP),
+                                             cast(None, FINAL))
+        if ret != SQLITE_OK:
+            raise self._get_exception(ret)
+
+    def create_aggregate(self, name, num_args, cls):
+        raise NotImplementedError
+
+class Cursor(object):
+    def __init__(self, con):
+        if not isinstance(con, Connection):
+            raise ValueError
+        con._check_thread()
+        con._check_closed()
+        self.connection = con
+        self._description = None
+        self.arraysize = 1
+        self.text_factory = con.text_factory
+        self.row_factory = None
+        self.rowcount = -1
+        self.statement = None
+
+    def execute(self, sql, params=None):
+        self._description = None
+        if type(sql) is unicode:
+            sql = sql.encode("utf-8")
+        self.connection._check_thread()
+        self.connection._check_closed()
+        self.statement = Statement(self, sql, self.row_factory)
+
+        if self.connection._isolation_level is not None:
+            if self.statement.kind == "DDL":
+                self.connection.commit()
+            elif self.statement.kind == "DML":
+                self.connection._begin()
+
+        self.statement.set_params(params)
+        if self.statement.kind in ("DML", "DDL"):
+            ret = sqlite.sqlite3_step(self.statement.statement)
+            if ret != SQLITE_DONE:
+                raise self.connection._get_exception(ret)
+        if self.statement.kind == "DML":
+            self.rowcount = sqlite.sqlite3_changes(self.connection.db)
+
+        return self
+
+    def executemany(self, sql, many_params):
+        self._description = None
+        if type(sql) is unicode:
+            sql = sql.encode("utf-8")
+        self.connection._check_closed()
+        self.connection._check_thread()
+        self.statement = Statement(self, sql, self.row_factory)
+        if self.statement.kind == "DML":
+            self.connection._begin()
+        else:
+            raise ProgrammingError, "executemany is only for DML statements"
+
+        self.rowcount = 0
+        for params in many_params:
+            self.statement.set_params(params)
+            ret = sqlite.sqlite3_step(self.statement.statement)
+            if ret != SQLITE_DONE:
+                raise self.connection._get_exception(ret)
+            self.rowcount += sqlite.sqlite3_changes(self.connection.db)
+
+        return self
+
+    def executescript(self, sql):
+        self._description = None
+        if type(sql) is unicode:
+            sql = sql.encode("utf-8")
+        self.connection._check_closed()
+        self.connection._check_thread()
+        statement = c_void_p()
+        next_char = c_char_p(sql)
+
+        while True:
+            if not sqlite.sqlite3_complete(next_char):
+                raise ProgrammingError, "Incomplete statement '%s'" % next_char.value
+            ret = sqlite.sqlite3_prepare_v2(self.connection.db, next_char, -1, byref(statement), byref(next_char))
+            if ret != SQLITE_OK:
+                raise self.connection._get_exception(ret)
+            if statement.value is None:
+                break
+            ret = sqlite.sqlite3_step(statement)
+            sqlite.sqlite3_finalize(statement)
+            if ret not in (SQLITE_OK, SQLITE_DONE, SQLITE_ROW):
+                raise self.connection._get_exception(ret)
+            if next_char.value is None:
+                break
+
+        return self
+
+    def __iter__(self):
+        return self.statement
+
+    def fetchone(self):
+        if self.statement is None:
+            return None
+        try:
+            return self.statement.next()
+        except StopIteration:
+            return None
+
+    def fetchmany(self, size=None):
+        if self.statement is None:
+            return []
+        if size is None:
+            size = self.arraysize
+        lst = []
+        for row in self.statement:
+            lst.append(row)
+            if len(lst) == size:
+                break
+        return lst
+
+    def fetchall(self):
+        if self.statement is None:
+            return []
+        return list(self.statement)
+
+    def _getdescription(self):
+        if self._description is None:
+            self._description = self.statement._get_description()
+        return self._description
+
+    def _getlastrowid(self):
+        return sqlite.sqlite3_last_insert_rowid(self.connection.db)
+
+    def close(self):
+        self.connection._check_thread()
+        self.connection._check_closed()
+        # XXX this should do reset and set statement to None it seems
+
+    def setinputsize(self, *args):
+        pass
+    def setoutputsize(self, *args):
+        pass
+
+
+    description = property(_getdescription)
+    lastrowid = property(_getlastrowid)
+
+class Statement(object):
+    def __init__(self, cur, sql, row_factory):
+        self.statement = None
+        if not isinstance(sql, str):
+            raise ValueError, "sql must be a string"
+        self.con = cur.connection
+        self.cur = weakref.ref(cur)
+        self.sql = sql # DEBUG ONLY
+        self.row_factory = row_factory
+        first_word = self._statement_kind = sql.lstrip().split(" ")[0].upper()
+        if first_word in ("INSERT", "UPDATE", "DELETE", "REPLACE"):
+            self.kind = "DML"
+        elif first_word in ("SELECT", "PRAGMA"):
+            self.kind = "DQL"
+        else:
+            self.kind = "DDL"
+        self.exhausted = False
+
+        self.statement = c_void_p()
+        next_char = c_char_p()
+        ret = sqlite.sqlite3_prepare_v2(self.con.db, sql, -1, byref(self.statement), byref(next_char))
+        if ret == SQLITE_OK and self.statement.value is None:
+            # an empty statement, we work around that, as it's the least trouble
+            ret = sqlite.sqlite3_prepare_v2(self.con.db, "select 42", -1, byref(self.statement), byref(next_char))
+            self.kind = "DQL"
+
+        if ret != SQLITE_OK:
+            raise self.con._get_exception(ret)
+        self.con._remember_statement(self)
+        if _check_remaining_sql(next_char.value):
+            raise Warning, "One and only one statement required"
+
+        self._build_row_cast_map()
+
+        self.started = False
+
+    def _build_row_cast_map(self):
+        self.row_cast_map = []
+        for i in range(sqlite.sqlite3_column_count(self.statement)):
+            converter = None
+
+            if self.con.detect_types & PARSE_COLNAMES:
+                colname = sqlite.sqlite3_column_name(self.statement, i)
+                if colname is not None:
+                    type_start = -1
+                    key = None
+                    for pos in range(len(colname)):
+                        if colname[pos] == '[':
+                            type_start = pos + 1
+                        elif colname[pos] == ']' and type_start != -1:
+                            key = colname[type_start:pos]
+                            converter = converters[key.upper()]
+
+            if converter is None and self.con.detect_types & PARSE_DECLTYPES:
+                decltype = sqlite.sqlite3_column_decltype(self.statement, i)
+                if decltype is not None:
+                    decltype = decltype.split()[0]      # if multiple words, use first, eg. "INTEGER NOT NULL" => "INTEGER"
+                    converter = converters.get(decltype.upper(), None)
+
+            self.row_cast_map.append(converter)
+
+    def set_param(self, idx, param):
+        cvt = converters.get(type(param))
+        if cvt is not None:
+            cvt = param = cvt(param)
+
+        adapter = adapters.get(type(param), None)
+        if adapter is not None:
+            param = adapter(param)
+
+        if param is None:
+            sqlite.sqlite3_bind_null(self.statement, idx)
+        elif type(param) in (bool, int, long):
+            if -2147483648 <= param <= 2147483647:
+                sqlite.sqlite3_bind_int(self.statement, idx, param)
+            else:
+                sqlite.sqlite3_bind_int64(self.statement, idx, param)
+        elif type(param) is float:
+            sqlite.sqlite3_bind_double(self.statement, idx, param)
+        elif isinstance(param, str):
+            sqlite.sqlite3_bind_text(self.statement, idx, param, -1, SQLITE_TRANSIENT)
+        elif isinstance(param, unicode):
+            param = param.encode("utf-8")
+            sqlite.sqlite3_bind_text(self.statement, idx, param, -1, SQLITE_TRANSIENT)
+        elif type(param) is buffer:
+            sqlite.sqlite3_bind_blob(self.statement, idx, str(param), len(param), SQLITE_TRANSIENT)
+        else:
+            raise InterfaceError, "parameter type %s is not supported" % str(type(param))
+
+    def set_params(self, params):
+        ret = sqlite.sqlite3_reset(self.statement)
+        if ret != SQLITE_OK:
+            raise self.con._get_exception(ret)
+
+        if params is None:
+            if sqlite.sqlite3_bind_parameter_count(self.statement) != 0:
+                raise ProgrammingError("wrong number of arguments")
+            return
+
+        params_type = None
+        if isinstance(params, dict):
+            params_type = dict
+        else:
+            params_type = list
+
+        if params_type == list:
+            if len(params) != sqlite.sqlite3_bind_parameter_count(self.statement):
+                raise ProgrammingError("wrong number of arguments")
+
+            for idx, param in enumerate(params):
+                self.set_param(idx+1, param)
+        else:
+            for idx in range(1, sqlite.sqlite3_bind_parameter_count(self.statement) + 1):
+                param_name = sqlite.sqlite3_bind_parameter_name(self.statement, idx)
+                if param_name is None:
+                    raise ProgrammingError, "need named parameters"
+                param_name = param_name[1:]
+                try:
+                    param = params[param_name]
+                except KeyError, e:
+                    raise ProgrammingError("missing parameter '%s'" %param)
+                self.set_param(idx, param)
+
+    def __iter__(self):
+        return self
+
+    def next(self):
+        self.con._check_closed()
+        self.con._check_thread()
+        if not self.started:
+            self.item = self._readahead()
+            self.started = True
+        if self.exhausted:
+            raise StopIteration
+        item = self.item
+        self.item = self._readahead()
+        return item
+
+    def _readahead(self):
+        ret = sqlite.sqlite3_step(self.statement)
+        if ret == SQLITE_DONE:
+            self.exhausted = True
+            return
+        elif ret == SQLITE_ERROR:
+            sqlite.sqlite3_reset(self.statement)
+            exc = self.con._get_exception(ret)
+            raise exc
+        else:
+            assert ret == SQLITE_ROW
+
+        self.column_count = sqlite.sqlite3_column_count(self.statement)
+        row = []
+        for i in xrange(self.column_count):
+            typ =  sqlite.sqlite3_column_type(self.statement, i)
+            converter = self.row_cast_map[i]
+            if converter is None:
+                if typ == SQLITE_INTEGER:
+                    val = sqlite.sqlite3_column_int64(self.statement, i)
+                    if -sys.maxint-1 <= val <= sys.maxint:
+                        val = int(val)
+                elif typ == SQLITE_FLOAT:
+                    val = sqlite.sqlite3_column_double(self.statement, i)
+                elif typ == SQLITE_BLOB:
+                    blob_len = sqlite.sqlite3_column_bytes(self.statement, i)
+                    blob = sqlite.sqlite3_column_blob(self.statement, i)
+                    val = buffer(string_at(blob, blob_len))
+                elif typ == SQLITE_NULL:
+                    val = None
+                elif typ == SQLITE_TEXT:
+                    val = sqlite.sqlite3_column_text(self.statement, i)
+                    val = self.cur().text_factory(val)
+            else:
+                blob = sqlite.sqlite3_column_blob(self.statement, i)
+                if not blob:
+                    val = None
+                else:
+                    blob_len = sqlite.sqlite3_column_bytes(self.statement, i)
+                    val = string_at(blob, blob_len)
+                    val = converter(val)
+            row.append(val)
+
+        row = tuple(row)
+        if self.row_factory is not None:
+            row = self.row_factory(self.cur(), row)
+        return row
+
+    def reset(self):
+        self.row_cast_map = None
+        sqlite.sqlite3_reset(self.statement)
+
+    def finalize(self):
+        sqlite.sqlite3_finalize(self.statement)
+        self.statement = None
+
+    def __del__(self):
+        sqlite.sqlite3_finalize(self.statement)
+        self.statement = None
+
+    def _get_description(self):
+        desc = []
+        for i in xrange(sqlite.sqlite3_column_count(self.statement)):
+            name = sqlite.sqlite3_column_name(self.statement, i).split("[")[0].strip()
+            desc.append((name, None, None, None, None, None, None))
+        return desc
+
+class Row(object):
+    def __init__(self, cursor, values):
+        self.description = cursor.description
+        self.values = values
+
+    def __getitem__(self, item):
+        if type(item) is int:
+            return self.values[item]
+        else:
+            item = item.lower()
+            for idx, desc in enumerate(self.description):
+                if desc[0].lower() == item:
+                    return self.values[idx]
+            raise KeyError
+
+    def keys(self):
+        return [desc[0] for desc in self.description]
+
+def _check_remaining_sql(s):
+    state = "NORMAL"
+    for char in s:
+        if char == chr(0):
+            return 0
+        elif char == '-':
+            if state == "NORMAL":
+                state = "LINECOMMENT_1"
+            elif state == "LINECOMMENT_1":
+                state = "IN_LINECOMMENT"
+        elif char in (' ', '\t'):
+            pass
+        elif char == '\n':
+            if state == "IN_LINECOMMENT":
+                state = "NORMAL"
+        elif char == '/':
+            if state == "NORMAL":
+                state = "COMMENTSTART_1"
+            elif state == "COMMENTEND_1":
+                state = "NORMAL"
+            elif state == "COMMENTSTART_1":
+                return 1
+        elif char == '*':
+            if state == "NORMAL":
+                return 1
+            elif state == "LINECOMMENT_1":
+                return 1
+            elif state == "COMMENTSTART_1":
+                state = "IN_COMMENT"
+            elif state == "IN_COMMENT":
+                state = "COMMENTEND_1"
+        else:
+            if state == "COMMENTEND_1":
+                state = "IN_COMMENT"
+            elif state == "IN_LINECOMMENT":
+                pass
+            elif state == "IN_COMMENT":
+                pass
+            else:
+                return 1
+    return 0
+
+def register_adapter(typ, callable):
+    adapters[typ] = callable
+
+def register_converter(name, callable):
+    converters[name.upper()] = callable
+
+def _convert_params(con, nargs, params):
+    _params  = []
+    for i in range(nargs):
+        typ = sqlite.sqlite3_value_type(params[i])
+        if typ == SQLITE_INTEGER:
+            val = sqlite.sqlite3_value_int64(params[i])
+            if -sys.maxint-1 <= val <= sys.maxint:
+                val = int(val)
+        elif typ == SQLITE_FLOAT:
+            val = sqlite.sqlite3_value_double(params[i])
+        elif typ == SQLITE_BLOB:
+            blob_len = sqlite.sqlite3_value_bytes(params[i])
+            blob = sqlite.sqlite3_value_blob(params[i])
+            val = buffer(string_at(blob, blob_len))
+        elif typ == SQLITE_NULL:
+            val = None
+        elif typ == SQLITE_TEXT:
+            val = sqlite.sqlite3_value_text(params[i])
+            # XXX changed from con.text_factory
+            val = unicode(val, 'utf-8')
+        else:
+            raise NotImplementedError
+        _params.append(val)
+    return _params
+
+def _convert_result(con, val):
+    if val is None:
+        sqlite.sqlite3_result_null(con)        
+    elif isinstance(val, (bool, int, long)):
+        sqlite.sqlite3_result_int64(con, int(val))
+    elif isinstance(val, str):
+        # XXX ignoring unicode issue
+        sqlite.sqlite3_result_text(con, val, len(val), SQLITE_TRANSIENT)
+    elif isinstance(val, unicode):
+        val = val.encode('utf-8')
+        sqlite.sqlite3_result_text(con, val, len(val), SQLITE_TRANSIENT)
+    elif isinstance(val, float):
+        sqlite.sqlite3_result_double(con, val)
+    elif isinstance(val, buffer):
+        sqlite.sqlite3_result_blob(con, str(val), len(val), SQLITE_TRANSIENT)
+    else:
+        raise NotImplementedError
+
+def function_callback(real_cb, context, nargs, c_params):
+    params = _convert_params(context, nargs, c_params)
+    try:
+        val = real_cb(*params)
+    except Exception, e:
+        msg = "user-defined function raised exception"
+        sqlite.sqlite3_result_error(context, msg, len(msg))
+    else:
+        _convert_result(context, val)
+    return 0
+
+FUNC = CFUNCTYPE(c_int, c_void_p, c_int, POINTER(c_void_p))
+STEP = CFUNCTYPE(c_int, c_void_p, c_int, POINTER(c_void_p))
+FINAL = CFUNCTYPE(c_int, c_void_p)
+sqlite.sqlite3_create_function.argtypes = [c_void_p, c_char_p, c_int, c_int, c_void_p, FUNC, STEP, FINAL]
+sqlite.sqlite3_create_function.restype = c_int
+
+converters = {}
+adapters = {}
+
+class PrepareProtocol(object):
+    pass
+
+def register_adapters_and_converters():
+    def adapt_date(val):
+        return val.isoformat()
+
+    def adapt_datetime(val):
+        return val.isoformat(" ")
+
+    def convert_date(val):
+        return datetime.date(*map(int, val.split("-")))
+
+    def convert_timestamp(val):
+        datepart, timepart = val.split(" ")
+        year, month, day = map(int, datepart.split("-"))
+        timepart_full = timepart.split(".")
+        hours, minutes, seconds = map(int, timepart_full[0].split(":"))
+        if len(timepart_full) == 2:
+            microseconds = int(timepart_full[1])
+        else:
+            microseconds = 0
+
+        val = datetime.datetime(year, month, day, hours, minutes, seconds, microseconds)
+        return val
+
+
+    register_adapter(datetime.date, adapt_date)
+    register_adapter(datetime.datetime, adapt_datetime)
+    register_converter("date", convert_date)
+    register_converter("timestamp", convert_timestamp)
+
+def OptimizedUnicode(s):
+    try:
+        val = unicode(s, "ascii").encode("ascii")
+    except UnicodeDecodeError:
+        val = unicode(s, "utf-8")
+    return val
+
+register_adapters_and_converters()

diff --git a/.hgsubstate b/.hgsubstate
--- a/.hgsubstate
+++ b/.hgsubstate
@@ -1,4 +1,3 @@
 80037 greenlet
 80037 lib_pypy/pyrepl
-80037 lib_pypy/sqlite3
 80037 testrunner

From commits-noreply at bitbucket.org  Wed Feb  9 01:58:17 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed,  9 Feb 2011 01:58:17 +0100 (CET)
Subject: [pypy-svn] pypy default: ElementTree is supported nowadays...
Message-ID: <20110209005817.0CC01282BF6@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41717:7fcc075bef00
Date: 2011-02-09 01:37 +0100
http://bitbucket.org/pypy/pypy/changeset/7fcc075bef00/

Log:	ElementTree is supported nowadays... Remove all restrictions from
	our side, and let the tests skip the C extension.

diff --git a/lib-python/conftest.py b/lib-python/conftest.py
--- a/lib-python/conftest.py
+++ b/lib-python/conftest.py
@@ -519,8 +519,8 @@
     RegrTest('test_with.py'),
     RegrTest('test_wsgiref.py'),
     RegrTest('test_xdrlib.py'),
-    RegrTest('test_xml_etree.py', skip="unsupported ext module"),
-    RegrTest('test_xml_etree_c.py', skip="unsupported ext module"),
+    RegrTest('test_xml_etree.py'),
+    RegrTest('test_xml_etree_c.py'),
     RegrTest('test_zipfile64.py'),
 ]
 

From commits-noreply at bitbucket.org  Wed Feb  9 01:58:18 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed,  9 Feb 2011 01:58:18 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix some failures in test_sqlite
Message-ID: <20110209005818.279C5282BF6@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41718:3fb486695f20
Date: 2011-02-09 01:57 +0100
http://bitbucket.org/pypy/pypy/changeset/3fb486695f20/

Log:	Fix some failures in test_sqlite

diff --git a/lib_pypy/_sqlite3.py b/lib_pypy/_sqlite3.py
--- a/lib_pypy/_sqlite3.py
+++ b/lib_pypy/_sqlite3.py
@@ -295,6 +295,8 @@
             self.statements = [ref for ref in self.statements if ref() is not None]
 
     def _check_thread(self):
+        if not hasattr(self, 'thread_ident'):
+            return
         if self.thread_ident != thread_get_ident():
             raise ProgrammingError(
                 "SQLite objects created in a thread can only be used in that same thread."
@@ -395,7 +397,7 @@
             sqlite.sqlite3_finalize(statement)
 
     def _check_closed(self):
-        if self.closed:
+        if getattr(self, 'closed', True):
             raise ProgrammingError("Cannot operate on a closed database.")
 
     def __enter__(self):
@@ -470,12 +472,17 @@
         self.rowcount = -1
         self.statement = None
 
+    def _check_closed(self):
+        if not getattr(self, 'connection', None):
+            raise ProgrammingError("Cannot operate on a closed cursor.")
+        self.connection._check_thread()
+        self.connection._check_closed()
+
     def execute(self, sql, params=None):
         self._description = None
         if type(sql) is unicode:
             sql = sql.encode("utf-8")
-        self.connection._check_thread()
-        self.connection._check_closed()
+        self._check_closed()
         self.statement = Statement(self, sql, self.row_factory)
 
         if self.connection._isolation_level is not None:
@@ -498,8 +505,7 @@
         self._description = None
         if type(sql) is unicode:
             sql = sql.encode("utf-8")
-        self.connection._check_closed()
-        self.connection._check_thread()
+        self._check_closed()
         self.statement = Statement(self, sql, self.row_factory)
         if self.statement.kind == "DML":
             self.connection._begin()
@@ -520,8 +526,7 @@
         self._description = None
         if type(sql) is unicode:
             sql = sql.encode("utf-8")
-        self.connection._check_closed()
-        self.connection._check_thread()
+        self._check_closed()
         statement = c_void_p()
         next_char = c_char_p(sql)
 
@@ -579,8 +584,7 @@
         return sqlite.sqlite3_last_insert_rowid(self.connection.db)
 
     def close(self):
-        self.connection._check_thread()
-        self.connection._check_closed()
+        self._check_closed()
         # XXX this should do reset and set statement to None it seems
 
     def setinputsize(self, *args):

From commits-noreply at bitbucket.org  Wed Feb  9 08:15:36 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Wed,  9 Feb 2011 08:15:36 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: make sure we dont mix up target
	and source	loop tokens
Message-ID: <20110209071536.06423282BF6@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r41720:ee08b4b4c98f
Date: 2011-02-09 08:13 +0100
http://bitbucket.org/pypy/pypy/changeset/ee08b4b4c98f/

Log:	make sure we dont mix up target and source loop tokens

diff --git a/pypy/jit/metainterp/test/test_loop.py b/pypy/jit/metainterp/test/test_loop.py
--- a/pypy/jit/metainterp/test/test_loop.py
+++ b/pypy/jit/metainterp/test/test_loop.py
@@ -400,6 +400,51 @@
             res = self.meta_interp(f, [25, th])
             assert res == expected
 
+    def test_two_bridged_loops(self):
+        myjitdriver = JitDriver(greens = ['pos'], reds = ['i', 'n', 's', 'x'])
+        bytecode = "zI7izI8i"
+        def f(n, s):
+            i = x = 0
+            pos = 0
+            op = '-'
+            while pos < len(bytecode):
+                myjitdriver.jit_merge_point(pos=pos, i=i, n=n, s=s, x=x)
+                op = bytecode[pos]
+                if op == 'z':
+                    i = 0
+                if op == 'i':
+                    i += 1
+                    pos -= 2
+                    myjitdriver.can_enter_jit(pos=pos, i=i, n=n, s=s, x=x)
+                    continue
+                elif op == 'I':
+                    if not (i < n):
+                        pos += 2
+                elif op == '7':
+                    if s==1: 
+                        x = x + 7
+                    else:
+                        x = x + 2
+                elif op == '8':
+                    if s==1: 
+                        x = x + 8
+                    else:
+                        x = x + 3
+
+                pos += 1
+            return x
+        
+        def g(n, s):
+            sa = 0
+            for i in range(7):
+                sa += f(n, s)
+            return sa
+        assert self.meta_interp(g, [25, 1]) == 7 * 25 * (7 + 8) 
+
+        def h(n):
+            return g(n, 1) + g(n, 2)
+        assert self.meta_interp(h, [25]) == 7 * 25 * (7 + 8 + 2 + 3) 
+        
     def test_three_nested_loops(self):
         myjitdriver = JitDriver(greens = ['i'], reds = ['x'])
         bytecode = ".+357"

From commits-noreply at bitbucket.org  Wed Feb  9 08:15:36 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Wed,  9 Feb 2011 08:15:36 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: hg merge default
Message-ID: <20110209071536.506B8282BF8@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r41721:a39ae4a9aa43
Date: 2011-02-09 08:15 +0100
http://bitbucket.org/pypy/pypy/changeset/a39ae4a9aa43/

Log:	hg merge default


From commits-noreply at bitbucket.org  Wed Feb  9 08:53:06 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Wed,  9 Feb 2011 08:53:06 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: Added a retrace_limit parameter
	instaed of	hardocing it to 5
Message-ID: <20110209075306.53BF2282BF6@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r41722:f1717153e7e7
Date: 2011-02-09 08:52 +0100
http://bitbucket.org/pypy/pypy/changeset/f1717153e7e7/

Log:	Added a retrace_limit parameter instaed of hardocing it to 5

diff --git a/pypy/jit/metainterp/optimizeopt/unroll.py b/pypy/jit/metainterp/optimizeopt/unroll.py
--- a/pypy/jit/metainterp/optimizeopt/unroll.py
+++ b/pypy/jit/metainterp/optimizeopt/unroll.py
@@ -674,7 +674,8 @@
                 retraced_count = len(short)
                 if descr.failed_states:
                     retraced_count += len(descr.failed_states)
-                if not self.retraced and retraced_count<5:
+                limit = self.optimizer.metainterp_sd.warmrunnerdesc.retrace_limit
+                if not self.retraced and retraced_count<limit:
                     if not descr.failed_states:
                         raise RetraceLoop
                     for failed in descr.failed_states:

diff --git a/pypy/rlib/jit.py b/pypy/rlib/jit.py
--- a/pypy/rlib/jit.py
+++ b/pypy/rlib/jit.py
@@ -266,6 +266,7 @@
               'inlining': False,
               'optimizer': OPTIMIZER_FULL,
               'loop_longevity': 1000,
+              'retrace_limit': 5,
               }
 unroll_parameters = unrolling_iterable(PARAMETERS.keys())
 

diff --git a/pypy/jit/metainterp/warmstate.py b/pypy/jit/metainterp/warmstate.py
--- a/pypy/jit/metainterp/warmstate.py
+++ b/pypy/jit/metainterp/warmstate.py
@@ -241,6 +241,9 @@
             self.warmrunnerdesc.memory_manager is not None):   # all for tests
             self.warmrunnerdesc.memory_manager.set_max_age(value)
 
+    def set_param_retrace_limit(self, value):
+        self.warmrunnerdesc.retrace_limit = value
+
     def disable_noninlinable_function(self, greenkey):
         cell = self.jit_cell_at_key(greenkey)
         cell.dont_trace_here = True

diff --git a/pypy/jit/metainterp/warmspot.py b/pypy/jit/metainterp/warmspot.py
--- a/pypy/jit/metainterp/warmspot.py
+++ b/pypy/jit/metainterp/warmspot.py
@@ -62,7 +62,7 @@
 
 def jittify_and_run(interp, graph, args, repeat=1,
                     backendopt=False, trace_limit=sys.maxint,
-                    inline=False, loop_longevity=0, **kwds):
+                    inline=False, loop_longevity=0, retrace_limit=5, **kwds):
     from pypy.config.config import ConfigError
     translator = interp.typer.annotator.translator
     try:
@@ -80,6 +80,7 @@
         jd.warmstate.set_param_trace_limit(trace_limit)
         jd.warmstate.set_param_inlining(inline)
         jd.warmstate.set_param_loop_longevity(loop_longevity)
+        jd.warmstate.set_param_retrace_limit(retrace_limit)
     warmrunnerdesc.finish()
     res = interp.eval_graph(graph, args)
     if not kwds.get('translate_support_code', False):

From commits-noreply at bitbucket.org  Wed Feb  9 09:08:54 2011
From: commits-noreply at bitbucket.org (lac)
Date: Wed,  9 Feb 2011 09:08:54 +0100 (CET)
Subject: [pypy-svn] pypy default: Ok,
	so this wasn't the place to add this file.
Message-ID: <20110209080854.521D636C221@codespeak.net>

Author: lac
Branch: 
Changeset: r41723:4c5519d074b9
Date: 2011-02-09 09:07 +0100
http://bitbucket.org/pypy/pypy/changeset/4c5519d074b9/

Log:	Ok, so this wasn't the place to add this file.

diff --git a/pypy/doc/getting-started-python.txt b/pypy/doc/getting-started-python.txt
--- a/pypy/doc/getting-started-python.txt
+++ b/pypy/doc/getting-started-python.txt
@@ -42,7 +42,6 @@
    * ``python-dev``
    * ``python-ctypes`` if you are still using Python2.4
    * ``libffi-dev``
-   * ``libapr0-dev`  (to make apache happy, and run some tests)
    * ``pkg-config`` (to help us locate libffi files)
    * ``libz-dev`` (for the optional ``zlib`` module)
    * ``libbz2-dev`` (for the optional ``bz2`` module)

From commits-noreply at bitbucket.org  Wed Feb  9 09:46:21 2011
From: commits-noreply at bitbucket.org (lac)
Date: Wed,  9 Feb 2011 09:46:21 +0100 (CET)
Subject: [pypy-svn] pypy default: remove references to svn. Fix typo. Add
	the information	that you will need
Message-ID: <20110209084621.53063282B90@codespeak.net>

Author: Laura Creighton <lac at openend.se>
Branch: 
Changeset: r41724:f66eb633cf21
Date: 2011-02-09 09:45 +0100
http://bitbucket.org/pypy/pypy/changeset/f66eb633cf21/

Log:	remove references to svn. Fix typo. Add the information that you
	will need libaprutil-1

diff --git a/pypy/doc/getting-started.txt b/pypy/doc/getting-started.txt
--- a/pypy/doc/getting-started.txt
+++ b/pypy/doc/getting-started.txt
@@ -25,10 +25,6 @@
 Just the facts 
 ============== 
 
-.. _gettingpypy: 
-.. _`latest stable version via subversion`:   
-.. _`get via Subversion`:
-
 Clone the repository
 --------------------
 
@@ -69,11 +65,14 @@
 
 .. _`our nightly tests:`: http://buildbot.pypy.org/summary?branch=<trunk>
 
+If your mercurial installation did not automatically give you a libaprutil-1
+you will have to install this manually if you want to commit changes to
+bitbucket.  
 
 Where to go from here
 ----------------------
 
-After you succesfully managed to get PyPy's source you can read more about:
+After you succesfully manage to get PyPy's source you can read more about:
 
  - `Building and using PyPy's Python interpreter`_
  - `Learning more about the translation toolchain and how to develop (with) PyPy`_

From commits-noreply at bitbucket.org  Wed Feb  9 10:28:23 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Wed,  9 Feb 2011 10:28:23 +0100 (CET)
Subject: [pypy-svn] pypy out-of-line-guards: replace ropaque calls with
	simple flag on	the looptoken. Backends have to
Message-ID: <20110209092823.7B891282B90@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: out-of-line-guards
Changeset: r41725:4a84157ae216
Date: 2011-02-07 20:47 +0200
http://bitbucket.org/pypy/pypy/changeset/4a84157ae216/

Log:	replace ropaque calls with simple flag on the looptoken. Backends
	have to be adjusted.

diff --git a/pypy/jit/metainterp/compile.py b/pypy/jit/metainterp/compile.py
--- a/pypy/jit/metainterp/compile.py
+++ b/pypy/jit/metainterp/compile.py
@@ -81,6 +81,7 @@
     """Try to compile a new loop by closing the current history back
     to the first operation.
     """
+    assert not metainterp.invalidated_array[0]
     history = metainterp.history
     loop = create_empty_loop(metainterp)
     loop.inputargs = history.inputargs
@@ -92,6 +93,7 @@
     metainterp_sd = metainterp.staticdata
     jitdriver_sd = metainterp.jitdriver_sd
     loop_token = make_loop_token(len(loop.inputargs), jitdriver_sd)
+    loop_token.invalidated_array = metainterp.invalidated_array
     loop.token = loop_token
     loop.operations[-1].setdescr(loop_token)     # patch the target of the JUMP
     try:
@@ -102,9 +104,6 @@
     if old_loop_token is not None:
         metainterp.staticdata.log("reusing old loop")
         return old_loop_token
-    if we_are_translated() or hasattr(metainterp, 'remember_jit_invariants'):
-        # for tests
-        metainterp.remember_jit_invariants(loop)
     send_loop_to_backend(metainterp_sd, loop, "loop")
     insert_loop_token(old_loop_tokens, loop_token)
     record_loop_or_bridge(loop)
@@ -547,6 +546,7 @@
     #
     # Attempt to use optimize_bridge().  This may return None in case
     # it does not work -- i.e. none of the existing old_loop_tokens match.
+    assert not metainterp.invalidated_array[0]
     new_loop = create_empty_loop(metainterp)
     new_loop.inputargs = metainterp.history.inputargs
     # clone ops, as optimize_bridge can mutate the ops
@@ -567,9 +567,6 @@
         # know exactly what we must do (ResumeGuardDescr/ResumeFromInterpDescr)
         prepare_last_operation(new_loop, target_loop_token)
         resumekey.compile_and_attach(metainterp, new_loop)
-        if we_are_translated() or hasattr(metainterp, 'remember_jit_invariants'):
-            # for tests
-            metainterp.remember_jit_invariants(new_loop)
         record_loop_or_bridge(new_loop)
     return target_loop_token
 

diff --git a/pypy/jit/backend/llgraph/runner.py b/pypy/jit/backend/llgraph/runner.py
--- a/pypy/jit/backend/llgraph/runner.py
+++ b/pypy/jit/backend/llgraph/runner.py
@@ -5,7 +5,7 @@
 import sys
 from pypy.rlib.unroll import unrolling_iterable
 from pypy.rlib.objectmodel import we_are_translated
-from pypy.rpython.lltypesystem import lltype, llmemory, rclass
+from pypy.rpython.lltypesystem import lltype, llmemory, rclass, rffi
 from pypy.rpython.ootypesystem import ootype
 from pypy.rpython.llinterp import LLInterpreter
 from pypy.jit.metainterp import history
@@ -21,7 +21,6 @@
 class MiniStats:
     pass
 
-
 class Descr(history.AbstractDescr):
 
     def __init__(self, ofs, typeinfo, extrainfo=None, name=None,
@@ -285,6 +284,9 @@
         return self.getdescr(symbolic.get_size(S))
 
 
+# can't be on LLtypeCPU thanks to closures not rendered correctly
+INVALIDATED_ARRAY_TP = rffi.CArray(lltype.Signed)
+
 class LLtypeCPU(BaseCPU):
     is_oo = False
     ts = llhelper
@@ -488,32 +490,21 @@
         self.latest_frame = frame
         return self.get_fail_descr_from_number(fail_index)
 
+    def create_invalidate_array(self):
+        v = lltype.malloc(INVALIDATED_ARRAY_TP, 1, flavor='raw',
+                          track_allocation=False)
+        v[0] = 0
+        return v
+
     def get_invalidate_asm(self, TP, fieldname):
         def invalidate_asm(arg):
             next = getattr(arg, fieldname)
             all = []
             while next:
-                llx = llmemory.weakref_deref(ropaque.ROPAQUE, next.address)
+                llx = rffi.cast(lltype.Ptr(INVALIDATED_ARRAY_TP), next.address)
                 if llx:
-                    all.append(ropaque.cast_ropaque_to_obj(history.LoopToken,
-                                                           llx))
+                    llx[0] = 1
                 next = next.next
-
-            while all:
-                next = all.pop()
-                next.invalidated = True
-                compiled = next.compiled_loop_token.compiled_version
-                llimpl.mark_as_invalid(compiled)
-                for elem in next._back_looptokens_call_asm:
-                    token = elem()
-                    if token:
-                        tk = token.compiled_loop_token.compiled_version
-                        llimpl.invalidate_call_asm(tk,
-                                                   next.compiled_loop_token)
-                for elem in next._back_looptokens:
-                    elem = elem()
-                    if elem:
-                        all.append(elem)
         return invalidate_asm
 
 class OOtypeCPU_xxx_disabled(BaseCPU):

diff --git a/pypy/jit/metainterp/pyjitpl.py b/pypy/jit/metainterp/pyjitpl.py
--- a/pypy/jit/metainterp/pyjitpl.py
+++ b/pypy/jit/metainterp/pyjitpl.py
@@ -1,5 +1,5 @@
 import py, os, sys
-from pypy.rpython.lltypesystem import lltype, llmemory, rclass
+from pypy.rpython.lltypesystem import lltype, llmemory, rclass, rffi
 from pypy.rpython.annlowlevel import cast_instance_to_base_ptr
 from pypy.rlib.objectmodel import we_are_translated
 from pypy.rlib.unroll import unrolling_iterable
@@ -512,7 +512,7 @@
 
     @arguments("box", "descr", "box")
     def _opimpl_getfield_gc_invariant_any(self, box, fielddescr, c_func):
-        self.metainterp.invariant_structs.append((box, c_func))
+        self.metainterp.mark_invariant_struct(box, c_func)
         return self.execute_with_descr(rop.GETFIELD_GC_PURE, fielddescr, box)
     opimpl_getfield_gc_i_invariant = _opimpl_getfield_gc_invariant_any
     opimpl_getfield_gc_r_invariant = _opimpl_getfield_gc_invariant_any
@@ -1402,7 +1402,7 @@
         self.portal_trace_positions = []
         self.free_frames_list = []
         self.last_exc_value_box = None
-        self.invariant_structs = []
+        self.invalidated_array = self.cpu.create_invalidate_array()
 
     def perform_call(self, jitcode, boxes, greenkey=None):
         # causes the metainterp to enter the given subfunction
@@ -1842,13 +1842,16 @@
         greenkey = original_boxes[:num_green_args]
         old_loop_tokens = self.get_compiled_merge_points(greenkey)
         self.history.record(rop.JUMP, live_arg_boxes[num_green_args:], None)
-        loop_token = compile.compile_new_loop(self, old_loop_tokens, start)
-        if loop_token is not None: # raise if it *worked* correctly
-            self.set_compiled_merge_points(greenkey, old_loop_tokens)
-            raise GenerateMergePoint(live_arg_boxes, loop_token)
+        if not self.invalidated_array[0]:
+            loop_token = compile.compile_new_loop(self, old_loop_tokens, start)
+            if loop_token is not None: # raise if it *worked* correctly
+                self.set_compiled_merge_points(greenkey, old_loop_tokens)
+                raise GenerateMergePoint(live_arg_boxes, loop_token)
         self.history.operations.pop()     # remove the JUMP
 
     def compile_bridge(self, live_arg_boxes):
+        if self.invalidated_array[0]:
+            return
         num_green_args = self.jitdriver_sd.num_green_args
         greenkey = live_arg_boxes[:num_green_args]
         old_loop_tokens = self.get_compiled_merge_points(greenkey)
@@ -1863,6 +1866,9 @@
 
     def compile_done_with_this_frame(self, exitbox):
         self.gen_store_back_in_virtualizable()
+        if self.invalidated_array[0]:
+            compile.giveup()
+            return
         # temporarily put a JUMP to a pseudo-loop
         sd = self.staticdata
         result_type = self.jitdriver_sd.result_type
@@ -1889,7 +1895,10 @@
 
     def compile_exit_frame_with_exception(self, valuebox):
         self.gen_store_back_in_virtualizable()
-        # temporarily put a JUMP to a pseudo-loop
+        if self.invalidated_array[0]:
+            compile.giveup()
+            return 
+       # temporarily put a JUMP to a pseudo-loop
         self.history.record(rop.JUMP, [valuebox], None)
         sd = self.staticdata
         loop_tokens = sd.loop_tokens_exit_frame_with_exception_ref
@@ -2237,17 +2246,12 @@
         op = op.copy_and_change(rop.CALL_ASSEMBLER, args=args, descr=token)
         self.history.operations.append(op)
 
-    def remember_jit_invariants(self, loop):
-        lllooptoken = ropaque.cast_obj_to_ropaque(loop.token)
-        lltoken_weakref = llmemory.weakref_create(lllooptoken)
-        seen = {}
-        for b_struct, c_appender in self.invariant_structs:
-            if (b_struct, c_appender) not in seen:
-                adr = heaptracker.int2adr(c_appender.getint())
-                ptr = llmemory.cast_adr_to_ptr(adr,
-                                               rclass.ASMCODE_APPENDER_PTR)
-                ptr(b_struct.getref_base(), lltoken_weakref)
-                seen[(b_struct, c_appender)] = None
+    def mark_invariant_struct(self, b_struct, c_appender):
+        addr = rffi.cast(lltype.Signed, self.invalidated_array)
+        func_addr = heaptracker.int2adr(c_appender.getint())
+        funcptr = llmemory.cast_adr_to_ptr(func_addr,
+                                           rclass.ASMCODE_APPENDER_PTR)
+        funcptr(b_struct.getref_base(), addr)
 
 # ____________________________________________________________
 

diff --git a/pypy/jit/codewriter/jtransform.py b/pypy/jit/codewriter/jtransform.py
--- a/pypy/jit/codewriter/jtransform.py
+++ b/pypy/jit/codewriter/jtransform.py
@@ -606,7 +606,7 @@
                     prev.next = new_asmcode
 
             args_s = [lltype_to_annotation(llmemory.GCREF),
-                      lltype_to_annotation(llmemory.WeakRefPtr)]
+                      lltype_to_annotation(lltype.Signed)]
             s_result = lltype_to_annotation(lltype.Void)
             mixlevelann = MixLevelHelperAnnotator(self.cpu.rtyper)
             c_appender = mixlevelann.constfunc(appender, args_s, s_result)

diff --git a/pypy/jit/metainterp/memmgr.py b/pypy/jit/metainterp/memmgr.py
--- a/pypy/jit/metainterp/memmgr.py
+++ b/pypy/jit/metainterp/memmgr.py
@@ -67,6 +67,7 @@
         for looptoken in self.alive_loops.keys():
             if (looptoken.invalidated or
                 0 <= looptoken.generation < max_generation):
+                # XXX free looptoken invalidated ptr
                 del self.alive_loops[looptoken]
         newtotal = len(self.alive_loops)
         debug_print("Loop tokens freed: ", oldtotal - newtotal)

diff --git a/pypy/rpython/lltypesystem/rclass.py b/pypy/rpython/lltypesystem/rclass.py
--- a/pypy/rpython/lltypesystem/rclass.py
+++ b/pypy/rpython/lltypesystem/rclass.py
@@ -86,11 +86,13 @@
 
 # a linked-list of assembler codes to invalidate in case jit_invariant_fields
 # are modified
+# address is a pointer to a loop-level one-element raw array that's
+# either 0 or 1
 ASMCODE = lltype.GcForwardReference()
-ASMCODE.become(GcStruct('asmcode', ('address', llmemory.WeakRefPtr),
+ASMCODE.become(GcStruct('asmcode', ('address', lltype.Signed),
                         ('next', lltype.Ptr(ASMCODE))))
 
-ASMCODE_APPENDER = lltype.FuncType([llmemory.GCREF, llmemory.WeakRefPtr],
+ASMCODE_APPENDER = lltype.FuncType([llmemory.GCREF, lltype.Signed],
                                    lltype.Void)
 ASMCODE_APPENDER_PTR = lltype.Ptr(ASMCODE_APPENDER)
 

From commits-noreply at bitbucket.org  Wed Feb  9 13:57:21 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed,  9 Feb 2011 13:57:21 +0100 (CET)
Subject: [pypy-svn] pypy default: Tests and fixes.  Thanks victorg5.
Message-ID: <20110209125721.9FF7A2A2003@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41727:43121832f78d
Date: 2011-02-09 13:56 +0100
http://bitbucket.org/pypy/pypy/changeset/43121832f78d/

Log:	Tests and fixes. Thanks victorg5.

diff --git a/pypy/interpreter/generator.py b/pypy/interpreter/generator.py
--- a/pypy/interpreter/generator.py
+++ b/pypy/interpreter/generator.py
@@ -15,7 +15,7 @@
         self.running = False
 
     def descr__repr__(self, space):
-        code_name = self.frame.pycode.co_name
+        code_name = self.pycode.co_name
         addrstring = self.getaddrstring(space)
         return space.wrap("<generator object %s at 0x%s>" %
                           (code_name, addrstring))
@@ -137,7 +137,7 @@
         return self.pycode
 
     def descr__name__(space, self):
-        code_name = self.frame.pycode.co_name
+        code_name = self.pycode.co_name
         return space.wrap(code_name)
 
     def descr__del__(self):        

diff --git a/pypy/interpreter/test/test_generator.py b/pypy/interpreter/test/test_generator.py
--- a/pypy/interpreter/test/test_generator.py
+++ b/pypy/interpreter/test/test_generator.py
@@ -27,6 +27,7 @@
         assert not g.gi_running
         assert g.gi_frame is None
         assert g.gi_code is f.func_code
+        assert g.__name__ == 'f'
 
     def test_generator3(self):
         def f():
@@ -261,4 +262,8 @@
     def test_repr(self):
         def myFunc():
             yield 1
-        assert repr(myFunc()).startswith("<generator object myFunc at 0x")
+        g = myFunc()
+        r = repr(g)
+        assert r.startswith("<generator object myFunc at 0x")
+        assert list(g) == [1]
+        assert repr(g) == r

From commits-noreply at bitbucket.org  Wed Feb  9 14:44:12 2011
From: commits-noreply at bitbucket.org (lac)
Date: Wed,  9 Feb 2011 14:44:12 +0100 (CET)
Subject: [pypy-svn] pypy default: Ok, this isn't needed,
	the whole problem was due to an improper subversion install
Message-ID: <20110209134412.A768C282C18@codespeak.net>

Author: Laura Creighton <lac at openend.se>
Branch: 
Changeset: r41728:30ff9f7c74fd
Date: 2011-02-09 14:31 +0100
http://bitbucket.org/pypy/pypy/changeset/30ff9f7c74fd/

Log:	Ok, this isn't needed, the whole problem was due to an improper
	subversion install

diff --git a/pypy/doc/getting-started.txt b/pypy/doc/getting-started.txt
--- a/pypy/doc/getting-started.txt
+++ b/pypy/doc/getting-started.txt
@@ -65,10 +65,6 @@
 
 .. _`our nightly tests:`: http://buildbot.pypy.org/summary?branch=<trunk>
 
-If your mercurial installation did not automatically give you a libaprutil-1
-you will have to install this manually if you want to commit changes to
-bitbucket.  
-
 Where to go from here
 ----------------------
 

From commits-noreply at bitbucket.org  Wed Feb  9 14:44:13 2011
From: commits-noreply at bitbucket.org (lac)
Date: Wed,  9 Feb 2011 14:44:13 +0100 (CET)
Subject: [pypy-svn] pypy default: The latest attempt to have the correct
	text in the file, so we don't omit something next time.
Message-ID: <20110209134413.45D72282C18@codespeak.net>

Author: Laura Creighton <lac at openend.se>
Branch: 
Changeset: r41729:189533754876
Date: 2011-02-09 14:43 +0100
http://bitbucket.org/pypy/pypy/changeset/189533754876/

Log:	The latest attempt to have the correct text in the file, so we don't
	omit something next time.

diff --git a/pypy/doc/getting-started.txt b/pypy/doc/getting-started.txt
--- a/pypy/doc/getting-started.txt
+++ b/pypy/doc/getting-started.txt
@@ -65,6 +65,9 @@
 
 .. _`our nightly tests:`: http://buildbot.pypy.org/summary?branch=<trunk>
 
+If you want to commit to our repository on bitbucket, you will have to
+install subversion in addition to mercurial.
+
 Where to go from here
 ----------------------
 

From commits-noreply at bitbucket.org  Wed Feb  9 15:51:44 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed,  9 Feb 2011 15:51:44 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix: on PyPy,
	**{u''} gets you immediately TypeError
Message-ID: <20110209145144.610D836C374@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41730:74ee8dda471c
Date: 2011-02-09 15:49 +0100
http://bitbucket.org/pypy/pypy/changeset/74ee8dda471c/

Log:	Fix: on PyPy, **{u''} gets you immediately TypeError

diff --git a/lib-python/modified-2.7.0/test/test_extcall.py b/lib-python/modified-2.7.0/test/test_extcall.py
--- a/lib-python/modified-2.7.0/test/test_extcall.py
+++ b/lib-python/modified-2.7.0/test/test_extcall.py
@@ -299,7 +299,7 @@
         def f(a):
             return a
         self.assertEqual(f(**{u'a': 4}), 4)
-        self.assertRaises(TypeError, f, **{u'st&#246;ren': 4})
+        self.assertRaises(TypeError, lambda: f(**{u'st&#246;ren': 4}))
         self.assertRaises(TypeError, f, **{u'someLongString':2})
         try:
             f(a=4, **{u'a': 4})

From commits-noreply at bitbucket.org  Wed Feb  9 15:51:57 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed,  9 Feb 2011 15:51:57 +0100 (CET)
Subject: [pypy-svn] pypy default: Big patch by victorg5: fix typos in
	pypy/doc/, found by aspell.
Message-ID: <20110209145157.0A93B36C374@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41731:d0dd8792776a
Date: 2011-02-09 15:51 +0100
http://bitbucket.org/pypy/pypy/changeset/d0dd8792776a/

Log:	Big patch by victorg5: fix typos in pypy/doc/, found by aspell.

diff --git a/pypy/doc/config/translation.backendopt.clever_malloc_removal.txt b/pypy/doc/config/translation.backendopt.clever_malloc_removal.txt
--- a/pypy/doc/config/translation.backendopt.clever_malloc_removal.txt
+++ b/pypy/doc/config/translation.backendopt.clever_malloc_removal.txt
@@ -1,6 +1,6 @@
 Try to inline flowgraphs based on whether doing so would enable malloc
 removal (:config:`translation.backendopt.mallocs`.) by eliminating
-calls that result in escaping. This is an experimental optimisation,
+calls that result in escaping. This is an experimental optimization,
 also right now some eager inlining is necessary for helpers doing
 malloc itself to be inlined first for this to be effective.
 This option enable also an extra subsequent malloc removal phase.

diff --git a/pypy/doc/coding-guide.txt b/pypy/doc/coding-guide.txt
--- a/pypy/doc/coding-guide.txt
+++ b/pypy/doc/coding-guide.txt
@@ -219,7 +219,7 @@
 
   ``range`` and ``xrange`` are identical. ``range`` does not necessarily create an array,
   only if the result is modified. It is allowed everywhere and completely
-  implemented. The only visible difference to CPython is the inaccessability
+  implemented. The only visible difference to CPython is the inaccessibility
   of the ``xrange`` fields start, stop and step.
 
 **definitions**
@@ -306,7 +306,7 @@
   concerns mainly method calls, when the method is overridden or in any
   way given different definitions in different classes.  It also concerns
   the less common case of explicitly manipulated function objects.
-  Describing the exact compability rules is rather involved (but if you
+  Describing the exact compatibility rules is rather involved (but if you
   break them, you should get explicit errors from the rtyper and not
   obscure crashes.)
 
@@ -466,9 +466,9 @@
 (>=0.13.0) can be run with the ``--rpython-mode`` command line option. This option
 enables the RPython checker which will checks for some of the
 restrictions RPython adds on standard Python code (and uses a 
-more agressive type inference than the one used by default by
+more aggressive type inference than the one used by default by
 pylint). The full list of checks is available in the documentation of
-Pylin. 
+Pylint. 
 
 RPylint can be a nice tool to get some information about how much work
 will be needed to convert a piece of Python code to RPython, or to get
@@ -636,7 +636,7 @@
 
 When we need access to interpreter-level objects we put the module into
 `pypy/module`_.  Such modules use a `mixed module mechanism`_
-which makes it convenient to use both interpreter- and applicationlevel parts
+which makes it convenient to use both interpreter- and application-level parts
 for the implementation.  Note that there is no extra facility for
 pure-interpreter level modules, you just write a mixed module and leave the
 application-level part empty.

diff --git a/pypy/doc/config/translation.type_system.txt b/pypy/doc/config/translation.type_system.txt
--- a/pypy/doc/config/translation.type_system.txt
+++ b/pypy/doc/config/translation.type_system.txt
@@ -1,4 +1,4 @@
 Which type system to use when rtyping_. This option should not be set
-explicitely.
+explicitly.
 
 .. _rtyping: ../rtyper.html

diff --git a/pypy/doc/discussion/distribution-newattempt.txt b/pypy/doc/discussion/distribution-newattempt.txt
--- a/pypy/doc/discussion/distribution-newattempt.txt
+++ b/pypy/doc/discussion/distribution-newattempt.txt
@@ -54,7 +54,7 @@
 
 We need:
 
-* app-level primitives for having 'remote proxy' accessable
+* app-level primitives for having 'remote proxy' accessible
 
 * some "serialiser" which is not truly serialising stuff, but making
   sure communication will go.

diff --git a/pypy/doc/glossary.txt b/pypy/doc/glossary.txt
--- a/pypy/doc/glossary.txt
+++ b/pypy/doc/glossary.txt
@@ -195,7 +195,7 @@
 
 **translation-time**
    In the context of the JIT_, translation time is when the PyPy
-   source is being analysed and the JIT itself is being created.
+   source is being analyzed and the JIT itself is being created.
 
 .. _`translator`:
 

diff --git a/pypy/doc/discussion/somepbc-refactoring-plan.txt b/pypy/doc/discussion/somepbc-refactoring-plan.txt
--- a/pypy/doc/discussion/somepbc-refactoring-plan.txt
+++ b/pypy/doc/discussion/somepbc-refactoring-plan.txt
@@ -27,7 +27,7 @@
 
 To be declared in module pypy.annotator.desc, with a mapping
 annotator.bookkeeper.descs = {<python object>: <XxxDesc instance>}
-accessed with bookkeepeer.getdesc(<python object>).
+accessed with bookkeeper.getdesc(<python object>).
 
 Maybe later the module should be moved out of pypy.annotation but for now I
 suppose that it's the best place.

diff --git a/pypy/doc/discussion/security-ideas.txt b/pypy/doc/discussion/security-ideas.txt
--- a/pypy/doc/discussion/security-ideas.txt
+++ b/pypy/doc/discussion/security-ideas.txt
@@ -191,7 +191,7 @@
     def serve_admin(token):
         print "Highest big is:", declassify(highest_bid, token=token)
 
-The declassify() function reads the value if the "token" is priviledged
+The declassify() function reads the value if the "token" is privileged
 enough, and raises an exception otherwise.
 
 What are we protecting here?  The fact that we need the administrator

diff --git a/pypy/doc/config/translation.gcrootfinder.txt b/pypy/doc/config/translation.gcrootfinder.txt
--- a/pypy/doc/config/translation.gcrootfinder.txt
+++ b/pypy/doc/config/translation.gcrootfinder.txt
@@ -2,7 +2,7 @@
 methods, this is mostly only interesting for framework GCs. For those you have
 a choice of various alternatives:
 
- - use a shadow stack (XXX link to paper), e.g. explicitely maintaining a stack
+ - use a shadow stack (XXX link to paper), e.g. explicitly maintaining a stack
    of roots
 
  - use stackless to find roots by unwinding the stack.  Requires

diff --git a/pypy/doc/dev_method.txt b/pypy/doc/dev_method.txt
--- a/pypy/doc/dev_method.txt
+++ b/pypy/doc/dev_method.txt
@@ -77,11 +77,11 @@
     pairs.
 
 It is a method that fits distributed teams well because it gets the team
-focused around clear (and challenging) goals while working collarobative
+focused around clear (and challenging) goals while working collaborative
 (pairprogramming, status meeting, discussions etc) as well as accelerated
 (short increments and tasks, "doing" and testing instead of long start ups of
 planning and requirement gathering). This means that most of the time a sprint
-is a great way of getting results, but also to get new people aquinted with
+is a great way of getting results, but also to get new people acquainted with
 the codebase. It is also a great method for dissemination and learning within
 the team because of the pairprogramming.
 
@@ -112,7 +112,7 @@
    month before the event. Beforehand we have some rough plans called "between
    sprints" and the sprintplan is based on the status of those issues but also
    with a focus on upcoming releases and deliverables. Usually its the core
-   developers who does this but the transparancy and participation have
+   developers who does this but the transparency and participation have
    increased since we started with our weekly "pypy-sync meetings" on IRC. The
    sync meetings in combination with a rough in between planning makes it
    easier for other developer to follow the progress and thus participating in
@@ -121,7 +121,7 @@
    The goal needs to be challenging or it won't rally the full effort of the
    team, but it must not be unrealistic as that tends to be very frustrating
    and dissatisfying. It is also very important to take into account the
-   particpants when you set the goal for the sprint. If the sprint takes place
+   participants when you set the goal for the sprint. If the sprint takes place
    connected to a conference (or similar open events) the goals for the actual
    coding progress should be set lower (or handled in another way) and focus
    should shift to dissemination and getting new/interested people to a
@@ -140,8 +140,8 @@
    evacuated by 23:00? These are important questions that can gravely affect
    the "feel and atmosphere" of the sprint as well as the desired results!
 
-   Also, somewhat close to low cost places to eat and accomodate
-   participants. Facilities for making tea/coffe as well as some kind of
+   Also, somewhat close to low cost places to eat and accommodate
+   participants. Facilities for making tea/coffee as well as some kind of
    refrigerator for storing food. A permanent Internet connection is a must -
    has the venue were the sprint is planned to be weird rules for access to
    their network etc etc?
@@ -151,7 +151,7 @@
    project also owns one beamer - specifically for sprint purposes.
 
    The person making sure that the requirements for a good sprint venue is
-   being met should therefore have very good local connections or, preferrably
+   being met should therefore have very good local connections or, preferably
    live there.
 
 3. Information - discussions about content and goals (pre announcements) are
@@ -164,7 +164,7 @@
    the sprint announcements points to information about local transportation
    (to the country and to the city and to the venue), currency issues, food
    and restaurants etc. There are also webpages in which people announce when
-   they will arrive and where they are accomodated.
+   they will arrive and where they are accommodated.
 
    The planning text for the sprint is updated up till the sprint and is then
    used during the status meetings and between to track work. After the sprint
@@ -174,11 +174,11 @@
    people involved. 
 
    One very important strategy when planning the venue is cost
-   efficiency. Keeping accomodation and food/travel costs as low as possible
+   efficiency. Keeping accommodation and food/travel costs as low as possible
    makes sure that more people can afford to visit or join the sprint
    fully. The partially EU funded parts of the project do have a so called sprint budget
    which we use to try to help developers to participate in our sprints
-   (travel expenses and accomodation) and because most of the funding is so
+   (travel expenses and accommodation) and because most of the funding is so
    called matched funding we pay for most of our expenses in our own
    organizations and companies anyway.
  
@@ -205,8 +205,8 @@
    formally responsible for the sprints.
 
    Suggestions for off hours activities and social events for the break day is
-   a good way of emphazising how important it is to take breaks - some
-   pointers in that direction from the local organiziser is good.
+   a good way of emphasizing how important it is to take breaks - some
+   pointers in that direction from the local organizer is good.
 
    At the end of the sprint we do a technical summary (did we achieve the
    goals/content), what should be a rough focus for the work until the next
@@ -231,7 +231,7 @@
 
 Of course. Just follow the work on pypy-dev and if you specifically are
 interested in information about our sprints - subscribe to
-pypy-sprint at codespeak.net and read the news on codespeak for anouncements etc.
+pypy-sprint at codespeak.net and read the news on codespeak for announcements etc.
 
 If you think we should sprint in your town - send us an email - we are very
 interested in using sprints as away of making contact with active developers

diff --git a/pypy/doc/config/objspace.usemodules.zipimport.txt b/pypy/doc/config/objspace.usemodules.zipimport.txt
--- a/pypy/doc/config/objspace.usemodules.zipimport.txt
+++ b/pypy/doc/config/objspace.usemodules.zipimport.txt
@@ -1,3 +1,3 @@
-This module implements zipimport mechanise described
+This module implements zipimport mechanism described
 in PEP 302. It's supposed to work and translate, so it's included
 by default
\ No newline at end of file

diff --git a/pypy/doc/architecture.txt b/pypy/doc/architecture.txt
--- a/pypy/doc/architecture.txt
+++ b/pypy/doc/architecture.txt
@@ -14,7 +14,7 @@
 We aim to provide:
 
  * a common translation and support framework for producing
-   implementations of dynamic languages, emphasising a clean
+   implementations of dynamic languages, emphasizing a clean
    separation between language specification and implementation
    aspects.
 

diff --git a/pypy/doc/discussion/distribution-implementation.txt b/pypy/doc/discussion/distribution-implementation.txt
--- a/pypy/doc/discussion/distribution-implementation.txt
+++ b/pypy/doc/discussion/distribution-implementation.txt
@@ -14,7 +14,7 @@
 
 First we do split objects into value-only primitives (like int) and other.
 Basically immutable builtin types which cannot contain user-level objects
-(int, float, long, str, None, etc.) will be always transfered as value-only
+(int, float, long, str, None, etc.) will be always transferred as value-only
 objects (having no states etc.). The every other object (user created classes,
 instances, modules, lists, tuples, etc. etc.) are always executed by reference.
 (Of course if somebody wants to ie. copy the instance, he can marshal/pickle
@@ -65,7 +65,7 @@
   but not needed yet.
 
 * From the remote point of view, every exported object which needs such
-  has got a local apropriate storage W_LocalXXX where XXX is a type 
+  has got a local appropriate storage W_LocalXXX where XXX is a type 
   by which it could be accessed from a wire.
 
 The real pain:
@@ -76,7 +76,7 @@
 in annotator, which sucks a lot). The (some) idea is to wrap all the methods
 with additional checks, but that's both unclear and probably not necessary.
 
-XXX If we can easily change underlaying implementation of an object, than
+XXX If we can easily change underlying implementation of an object, than
 this might become way easier. Right now I'll try to have it working and
 thing about RPython later.
 

diff --git a/pypy/doc/discussion/VM-integration.txt b/pypy/doc/discussion/VM-integration.txt
--- a/pypy/doc/discussion/VM-integration.txt
+++ b/pypy/doc/discussion/VM-integration.txt
@@ -24,7 +24,7 @@
   - handle native .NET objects as transparently as possible
 
   - automatically apply obvious Python <--> .NET conversions when
-    crossing the borders (e.g. intgers, string, etc.)
+    crossing the borders (e.g. integers, string, etc.)
 
 One possible solution is the "proxy" approach, in which we manually
 (un)wrap/(un)box all the objects when they cross the border.
@@ -64,13 +64,13 @@
 stateful part and a behavioral part.
 
 We need to introduce the new ootypesystem type ``Pair``: it acts like
-a Record but it hasn't its own identiy: the id of the Pair is the id
+a Record but it hasn't its own identity: the id of the Pair is the id
 of its first member.
 
   XXX about ``Pair``: I'm not sure this is totally right. It means
   that an object can change identity simply by changing the value of a
   field???  Maybe we could add the constraint that the "id" field
-  can't be modifiend after initialization (but it's not easy to
+  can't be modified after initialization (but it's not easy to
   enforce).
 
   XXX-2 about ``Pair``: how to implement it in the backends? One

diff --git a/pypy/doc/eventhistory.txt b/pypy/doc/eventhistory.txt
--- a/pypy/doc/eventhistory.txt
+++ b/pypy/doc/eventhistory.txt
@@ -1,7 +1,7 @@
 
 
     The PyPy project is a worldwide collaborative effort and its
-    members are organising sprints and presenting results at conferences
+    members are organizing sprints and presenting results at conferences
     all year round.  **This page is no longer maintained!**  See `our blog`_
     for upcoming events. 
 
@@ -120,7 +120,7 @@
 
 The Mallorca sprint that took place in Palma de Mallorca is over.
 Topics included progressing with the JIT work started in G&#246;teborg
-and Paris, GC and optimisation work, stackless, and
+and Paris, GC and optimization work, stackless, and
 improving our way to write glue code for C libraries.
 
 Read more in `the announcement`_, there is a `sprint report`_

diff --git a/pypy/doc/index-report.txt b/pypy/doc/index-report.txt
--- a/pypy/doc/index-report.txt
+++ b/pypy/doc/index-report.txt
@@ -30,7 +30,7 @@
 `D08.2 JIT Compiler Architecture`_ is a report about the Architecture and
 working of our JIT compiler generator. *(2007-05-01)*
 
-`D08.1 JIT Compiler Release`_ reports on our sucessfully including a
+`D08.1 JIT Compiler Release`_ reports on our successfully including a
 JIT compiler for Python and the novel framework we used to
 automatically generate it in PyPy 1.0. *(2007-04-30)*
 
@@ -76,7 +76,7 @@
 
 
 `D02.1 Development Tools and Website`_ is a report
-about the codespeak_ development environment and aditional tool support for the
+about the codespeak_ development environment and additional tool support for the
 PyPy development process. *(2007-03-21)*
 
 `D03.1 Extension Compiler`_ is a report about

diff --git a/pypy/doc/interpreter.txt b/pypy/doc/interpreter.txt
--- a/pypy/doc/interpreter.txt
+++ b/pypy/doc/interpreter.txt
@@ -35,7 +35,7 @@
 module body source codes.  Interpreting such code objects means
 instantiating and initializing a `Frame class`_ and then
 calling its ``frame.eval()`` method.  This main entry point 
-initialise appropriate namespaces and then interprets each 
+initialize appropriate namespaces and then interprets each 
 bytecode instruction.  Python's standard library contains
 the `lib-python/2.5.2/dis.py`_ module which allows to view
 the Virtual's machine bytecode instructions:: 
@@ -278,11 +278,11 @@
 ---------------------- 
 
 A unique PyPy property is the ability to easily cross the barrier
-between interpreted and machine-level code (often refered to as
+between interpreted and machine-level code (often referred to as
 the difference between `interpreter-level and application-level`_). 
 Be aware that the according code (in `pypy/interpreter/gateway.py`_) 
 for crossing the barrier in both directions is somewhat
-involved, mostly due to the fact that the type-infering
+involved, mostly due to the fact that the type-inferring
 annotator needs to keep track of the types of objects flowing
 across those barriers. 
 
@@ -323,7 +323,7 @@
 Summarizing, the ``interp2app`` mechanism takes care to route 
 an application level access or call to an internal interpreter-level 
 object appropriately to the descriptor, providing enough precision
-and hints to keep the type-infering annotator happy. 
+and hints to keep the type-inferring annotator happy. 
 
 
 Calling into application level code from interpreter-level 

diff --git a/pypy/doc/config/translation.list_comprehension_operations.txt b/pypy/doc/config/translation.list_comprehension_operations.txt
--- a/pypy/doc/config/translation.list_comprehension_operations.txt
+++ b/pypy/doc/config/translation.list_comprehension_operations.txt
@@ -1,2 +1,2 @@
-Experimental optimisation for list comprehensions in RPython.
+Experimental optimization for list comprehensions in RPython.
 

diff --git a/pypy/doc/cleanup-todo.txt b/pypy/doc/cleanup-todo.txt
--- a/pypy/doc/cleanup-todo.txt
+++ b/pypy/doc/cleanup-todo.txt
@@ -15,7 +15,7 @@
    simplify translator/c/gc.py
  - clean up the tangle of including headers in the C backend
  - make approach for loading modules more sane, mixedmodule capture
-   too many platform dependecies especially for pypy-cli
+   too many platform dependencies especially for pypy-cli
  - review pdbplus, especially the graph commands, also in the light of
    https://codespeak.net/issue/pypy-dev/issue303 and the fact that
    we can have more than one translator/annotator around (with the

diff --git a/pypy/doc/discussion/use_case_of_logic.txt b/pypy/doc/discussion/use_case_of_logic.txt
--- a/pypy/doc/discussion/use_case_of_logic.txt
+++ b/pypy/doc/discussion/use_case_of_logic.txt
@@ -13,10 +13,10 @@
 We define Business Logic as expressing consistency (as an example) on
 a set of objects in a business application.
 
-For exeample checking the consistency of a calculation before
+For example checking the consistency of a calculation before
 committing the changes.
 
-The domain is quite rich in example of uses of Busines Logic.
+The domain is quite rich in example of uses of Business Logic.
 
 Datamining
 ===========
@@ -26,7 +26,7 @@
 Databases
 =========
 
-Validity contraints for the data can be expressed as constraints.
+Validity constraints for the data can be expressed as constraints.
 
 Constraints can be used to perform type inference when querying the
 database.
@@ -34,7 +34,7 @@
 Semantic web
 =============
 
-The use case is like the database case, except the ontology langauge
+The use case is like the database case, except the ontology language
 it self is born out of Descriptive Logic
 
 
@@ -50,8 +50,8 @@
 Configuration
 ==============
 
-User configuration can use information infered from : the current
-user, current plantform , version requirements, ...
+User configuration can use information inferred from : the current
+user, current platforms , version requirements, ...
 
 The validity of the configuration can be checked with the constraints.
 
@@ -61,15 +61,15 @@
 
 Timetables, process scheduling, task scheduling.
 
-Use rules to determin when to execute tasks (only start batch, if load
+Use rules to determine when to execute tasks (only start batch, if load
 is low, and previous batch is finished.
 
 Load sharing.
 
-Route optimisation. Planning the routes of a technitian based on tools
+Route optimization. Planning the routes of a technician based on tools
 needed and such
 
-An example is scheduling a confenre like Europython see:
+An example is scheduling a conference like Europython see:
 
 http://lists.logilab.org/pipermail/python-logic/2005-May/000107.html
 

diff --git a/pypy/doc/config/translation.backendopt.print_statistics.txt b/pypy/doc/config/translation.backendopt.print_statistics.txt
--- a/pypy/doc/config/translation.backendopt.print_statistics.txt
+++ b/pypy/doc/config/translation.backendopt.print_statistics.txt
@@ -1,2 +1,2 @@
 Debugging option. Print statics about the forest of flowgraphs as they
-go through the various backend optimisations.
\ No newline at end of file
+go through the various backend optimizations.
\ No newline at end of file

diff --git a/pypy/doc/config/translation.vanilla.txt b/pypy/doc/config/translation.vanilla.txt
--- a/pypy/doc/config/translation.vanilla.txt
+++ b/pypy/doc/config/translation.vanilla.txt
@@ -1,2 +1,2 @@
-Try to make the resulting compiled program as portable (=moveable to another
+Try to make the resulting compiled program as portable (=movable to another
 machine) as possible. Which is not much.

diff --git a/pypy/doc/discussion/outline-external-ootype.txt b/pypy/doc/discussion/outline-external-ootype.txt
--- a/pypy/doc/discussion/outline-external-ootype.txt
+++ b/pypy/doc/discussion/outline-external-ootype.txt
@@ -108,7 +108,7 @@
 
     Foo = ootype.ExternalInstance({'bar': ([Signed, Signed], Float)})
 
-Then, the annotation for Foo's intances is SomeExternalInstance(Foo).
+Then, the annotation for Foo's instances is SomeExternalInstance(Foo).
 This way, the transformation from high-level types to platform-level
 types is straightforward and correct.
 
@@ -156,7 +156,7 @@
 Exceptions
 ~~~~~~~~~~
 
-.NET and JVM users want to catch external exceptions in a natuarl way;
+.NET and JVM users want to catch external exceptions in a natural way;
 e.g.::
 
     try:
@@ -164,7 +164,7 @@
     except System.OverflowException:
         ...
 
-This is not straighforward because to make the flow objspace happy the
+This is not straightforward because to make the flow objspace happy the
 object which represent System.OverflowException must be a real Python
 class that inherits from Exception.
 
@@ -200,7 +200,7 @@
 necessary to rewrite a part of it.
 
 To represent low-level types, it uses NativeInstance, a subclass of
-ootype.Instance that contains all the informations needed by the
+ootype.Instance that contains all the information needed by the
 backend to reference the class (e.g., the namespace). It also supports
 overloading.
 
@@ -209,5 +209,5 @@
 helpers. It might be saner to use another annotation not to mix apples
 and oranges, maybe factoring out common code.
 
-I don't know whether and how much code can be reused from the exising
+I don't know whether and how much code can be reused from the existing
 bltregistry.

diff --git a/pypy/doc/configuration.txt b/pypy/doc/configuration.txt
--- a/pypy/doc/configuration.txt
+++ b/pypy/doc/configuration.txt
@@ -89,7 +89,7 @@
         ``default`` specifies the default value of the option. ``requires`` is
         a list of two-element tuples describing the requirements when the
         option is set to true, ``suggests`` is a list of the same structure but
-        the options in there are only suggested, not absolutely neccessary. The
+        the options in there are only suggested, not absolutely necessary. The
         difference is small: if the current option is set to True, both the
         required and the suggested options are set. The required options cannot
         be changed later, though. ``negation`` specifies whether the negative
@@ -127,7 +127,7 @@
 Configuration Objects
 ---------------------
 
-``Config`` objects hold the choosen values for the options (of the default,
+``Config`` objects hold the chosen values for the options (of the default,
 if no choice was made). A ``Config`` object is described by an
 ``OptionDescription`` instance. The attributes of the ``Config`` objects are the
 names of the children of the ``OptionDescription``. Example::

diff --git a/pypy/doc/config/translation.backendopt.inline.txt b/pypy/doc/config/translation.backendopt.inline.txt
--- a/pypy/doc/config/translation.backendopt.inline.txt
+++ b/pypy/doc/config/translation.backendopt.inline.txt
@@ -7,4 +7,4 @@
 inlined is needed for malloc removal
 (:config:`translation.backendopt.mallocs`) to be effective.
 
-This optimisation is used by default.
+This optimization is used by default.

diff --git a/pypy/doc/faq.txt b/pypy/doc/faq.txt
--- a/pypy/doc/faq.txt
+++ b/pypy/doc/faq.txt
@@ -147,7 +147,7 @@
 .. _`prolog and javascript`:
 
 ---------------------------------------------------------------
-Can PyPy support intepreters for other languages beyond Python?
+Can PyPy support interpreters for other languages beyond Python?
 ---------------------------------------------------------------
 
 The toolsuite that translates the PyPy interpreter is quite

diff --git a/pypy/doc/discussion/gc.txt b/pypy/doc/discussion/gc.txt
--- a/pypy/doc/discussion/gc.txt
+++ b/pypy/doc/discussion/gc.txt
@@ -9,7 +9,7 @@
 =============
 
 This document tries to gather gc-related issues which are very recent
-or in-developement. Also, it tries to document needed gc refactorings
+or in-development. Also, it tries to document needed gc refactorings
 and expected performance of certain gc-related operations.
 
 Problem area

diff --git a/pypy/doc/carbonpython.txt b/pypy/doc/carbonpython.txt
--- a/pypy/doc/carbonpython.txt
+++ b/pypy/doc/carbonpython.txt
@@ -167,7 +167,7 @@
             return self.x
 
 
-Note that the type of ``self`` must not be speficied: it will
+Note that the type of ``self`` must not be specified: it will
 automatically assumed to be ``MyClass``.
 
 The ``__init__`` method is not automatically mapped to the .NET
@@ -182,7 +182,7 @@
 outside; the RPython compiler automatically calls ``__init__``
 whenever an RPython class is instantiated.
 
-In the future this discrepacy will be fixed and the ``__init__``
+In the future this discrepancy will be fixed and the ``__init__``
 method will be automatically mapped to the constructor.
 
 
@@ -208,7 +208,7 @@
 
 At the moment there is no special syntax support for indexers and
 properties: for example, you can't access ArrayList's elements using
-the square bracked notation, but you have to call the call the
+the square bracket notation, but you have to call the call the
 ``get_Item`` and ``set_Item`` methods; similarly, to access a property
 ``XXX`` you need to call ``get_XXX`` and ``set_XXX``::
 

diff --git a/pypy/doc/config/objspace.usemodules._rawffi.txt b/pypy/doc/config/objspace.usemodules._rawffi.txt
--- a/pypy/doc/config/objspace.usemodules._rawffi.txt
+++ b/pypy/doc/config/objspace.usemodules._rawffi.txt
@@ -1,3 +1,3 @@
 An experimental module providing very low-level interface to
 C-level libraries, for use when implementing ctypes, not
-indended for a direct use at all.
\ No newline at end of file
+intended for a direct use at all.
\ No newline at end of file

diff --git a/pypy/doc/discussion/ctypes_todo.txt b/pypy/doc/discussion/ctypes_todo.txt
--- a/pypy/doc/discussion/ctypes_todo.txt
+++ b/pypy/doc/discussion/ctypes_todo.txt
@@ -3,7 +3,7 @@
 * Write down missing parts and port all tests, eventually adding
   additional tests.
 
-  - for unions and structs, late assignement of _fields_ is somewhat buggy.
+  - for unions and structs, late assignment of _fields_ is somewhat buggy.
     Tests about behavior of getattr working properly on instances
     are missing or not comprehensive. Some tests are skipped because I didn't
     understand the details.
@@ -15,7 +15,7 @@
   - there are features, which we don't support like buffer() and
     array() protocols.
 
-  - are the _CData_value return lifetime/gc sematics correct?
+  - are the _CData_value return lifetime/gc semantics correct?
 
   - for some ABIs we will need completely filled ffitypes to do the
     right thing for passing structures by value, we are now passing enough

diff --git a/pypy/doc/getting-started.txt b/pypy/doc/getting-started.txt
--- a/pypy/doc/getting-started.txt
+++ b/pypy/doc/getting-started.txt
@@ -71,7 +71,7 @@
 Where to go from here
 ----------------------
 
-After you succesfully manage to get PyPy's source you can read more about:
+After you successfully manage to get PyPy's source you can read more about:
 
  - `Building and using PyPy's Python interpreter`_
  - `Learning more about the translation toolchain and how to develop (with) PyPy`_

diff --git a/pypy/doc/config/index.txt b/pypy/doc/config/index.txt
--- a/pypy/doc/config/index.txt
+++ b/pypy/doc/config/index.txt
@@ -33,7 +33,7 @@
 There is an `overview`_ of all command line arguments that can be
 passed in either position.
 
-Many of the more interesting object space options enable optimzations,
+Many of the more interesting object space options enable optimizations,
 which are described in `Standard Interpreter Optimizations`_, or allow
 the creation of objects that can barely be imagined in CPython, which
 are documented in `What PyPy can do for your objects`_.

diff --git a/pypy/doc/clr-module.txt b/pypy/doc/clr-module.txt
--- a/pypy/doc/clr-module.txt
+++ b/pypy/doc/clr-module.txt
@@ -139,5 +139,5 @@
     >>>> Application.Run(frm)
 
 Unfortunately at the moment you can't do much more than this with Windows
-Forms, because we still miss support for delegates and so it's not possibile
+Forms, because we still miss support for delegates and so it's not possible
 to handle events.

diff --git a/pypy/doc/cli-backend.txt b/pypy/doc/cli-backend.txt
--- a/pypy/doc/cli-backend.txt
+++ b/pypy/doc/cli-backend.txt
@@ -14,7 +14,7 @@
 
 While in an ideal world we might suppose GenCLI to run fine with
 every implementation conforming to that standard, we know the world we
-live in is far from ideal, so extra efforts can be needed to mantain
+live in is far from ideal, so extra efforts can be needed to maintain
 compatibility with more than one implementation.
 
 At the moment of writing the two most popular implementations of the
@@ -84,7 +84,7 @@
 returning the name of the helpers and one subclass for each of the two
 supported platforms.
 
-Since Microfost ``ilasm`` is not capable of compiling the PyPy
+Since Microsoft ``ilasm`` is not capable of compiling the PyPy
 standard interpreter due to its size, on Windows machines we also look
 for an existing Mono installation: if present, we use CLR for
 everything except the assembling phase, for which we use Mono's
@@ -132,7 +132,7 @@
 while .NET only supports Unicode with the char type. There are at
 least two ways to map plain Char to CTS:
 
-  - map UniChar to char, thus mantaining the original distinction
+  - map UniChar to char, thus maintaining the original distinction
     between the two types: this has the advantage of being a
     one-to-one translation, but has the disadvantage that RPython
     strings will not be recognized as .NET strings, since they only
@@ -234,7 +234,7 @@
 subtle bugs in more complex ones, because the two exception
 hierarchies don't completely overlap.
 
-At the moment we've choosen to build an RPython exception hierarchy
+At the moment we've chosen to build an RPython exception hierarchy
 completely independent from the CLI one, but this means that we can't
 rely on exceptions raised by built-in operations.  The currently
 implemented solution is to do an exception translation on-the-fly.
@@ -242,7 +242,7 @@
 As an example consider the RPython int_add_ovf operation, that sums
 two integers and raises an OverflowError exception in case of
 overflow. For implementing it we can use the built-in add.ovf CLI
-instruction that raises System.OverflowExcepion when the result
+instruction that raises System.OverflowException when the result
 overflows, catch that exception and throw a new one::
 
     .try 
@@ -267,7 +267,7 @@
 represented by flow graphs that we need to translate CLI IL code. Flow
 graphs are expressed in a format that is very suitable for being
 translated to low level code, so that phase is quite straightforward,
-though the code is a bit involed because we need to take care of three
+though the code is a bit involved because we need to take care of three
 different types of blocks.
 
 The code doing this work is located in the Function.render
@@ -278,7 +278,7 @@
 statement used for indicating the virtual machine the number and type
 of local variables used.
 
-Then it sequentally renders all blocks in the graph, starting from the
+Then it sequentially renders all blocks in the graph, starting from the
 start block; special care is taken for the return block which is
 always rendered at last to meet CLI requirements.
 

diff --git a/pypy/doc/index.txt b/pypy/doc/index.txt
--- a/pypy/doc/index.txt
+++ b/pypy/doc/index.txt
@@ -27,7 +27,7 @@
 
 * `Development bug/feature tracker`_: filing bugs and feature requests. 
 
-* `Sprint mailing list`_: mailing list for organising upcoming sprints. 
+* `Sprint mailing list`_: mailing list for organizing upcoming sprints. 
 
 * **IRC channel #pypy on freenode**: Many of the core developers are hanging out 
   at #pypy on irc.freenode.net.  You are welcome to join and ask questions
@@ -39,7 +39,7 @@
 Meeting PyPy developers
 =======================
 
-The PyPy developers are organising sprints and presenting results at
+The PyPy developers are organizing sprints and presenting results at
 conferences all year round. They will be happy to meet in person with
 anyone interested in the project.  Watch out for sprint announcements
 on the `development mailing list`_.

diff --git a/pypy/doc/discussion/chained_getattr.txt b/pypy/doc/discussion/chained_getattr.txt
--- a/pypy/doc/discussion/chained_getattr.txt
+++ b/pypy/doc/discussion/chained_getattr.txt
@@ -25,7 +25,7 @@
        
     now for the LOAD_CHAINED_GLOBAL bytecode implementation:
 
-        Module dicts have a special implemnetation, providing: 
+        Module dicts have a special implementation, providing: 
 
         - an extra "fastlookup" rpython-dict serving as a cache for
           LOAD_CHAINED_GLOBAL places within the modules: 

diff --git a/pypy/doc/jit/overview.txt b/pypy/doc/jit/overview.txt
--- a/pypy/doc/jit/overview.txt
+++ b/pypy/doc/jit/overview.txt
@@ -136,7 +136,7 @@
 to the complete language.  This would be a major effort and be platform- and
 language-specific.  Moreover maintenance would be a headache: we believe that
 many changes that are trivial to implement in CPython, are likely to invalidate
-previous carefully-tuned optimisations.
+previous carefully-tuned optimizations.
 
 For major improvements in speed, JIT techniques are necessary.  For Python,
 Psyco gives typical speedups of 2 to 4 times - up to 100 times in algorithmic

diff --git a/pypy/doc/extradoc.txt b/pypy/doc/extradoc.txt
--- a/pypy/doc/extradoc.txt
+++ b/pypy/doc/extradoc.txt
@@ -323,7 +323,7 @@
 * Jython_ is a Python implementation in Java.
 
 * IronPython_ a new Python implementation compiling Python into 
-  Microsofts Common Language Runtime (CLR) Intermediate Language (IL).
+  Microsoft's Common Language Runtime (CLR) Intermediate Language (IL).
 
 * Tunes_ is not entirely unrelated.  The web site changed a lot, but a
   snapshot of the `old Tunes Wiki`_ is available on codespeak; browsing

diff --git a/pypy/doc/discussion/cli-optimizations.txt b/pypy/doc/discussion/cli-optimizations.txt
--- a/pypy/doc/discussion/cli-optimizations.txt
+++ b/pypy/doc/discussion/cli-optimizations.txt
@@ -1,7 +1,7 @@
 Possible optimizations for the CLI backend
 ==========================================
 
-Stack push/pop optimitazion
+Stack push/pop optimization
 ---------------------------
 
 The CLI's VM is a stack based machine: this fact doesn't play nicely
@@ -61,7 +61,7 @@
 generated by Microsoft CLR, yet.
 
 Thus, we might consider to optimize it manually; it should not be so
-difficult, but it is not trivial becasue we have to make sure that the
+difficult, but it is not trivial because we have to make sure that the
 dropped locals are used only once.
 
 
@@ -79,8 +79,8 @@
 subtle bugs in more complex ones, because the two exception
 hierarchies don't completely overlap.
 
-For now I've choosen to build an RPython exception hierarchy
-completely indipendent from the CLI one, but this means that we can't
+For now I've chosen to build an RPython exception hierarchy
+completely independent from the CLI one, but this means that we can't
 rely on exceptions raised by standard operations. The currently
 implemented solution is to do an exception translation on-the-fly; for
 example, the 'ind_add_ovf' is translated into the following IL code::
@@ -105,7 +105,7 @@
 I.e., it catches the builtin OverflowException and raises a RPython
 OverflowError.
 
-I haven't misured timings yet, but I guess that this machinery brings
+I haven't measured timings yet, but I guess that this machinery brings
 to some performance penalties even in the non-overflow case; a
 possible optimization is to do the on-the-fly translation only when it
 is strictly necessary, i.e. only when the except clause catches an
@@ -188,7 +188,7 @@
 The current implementations of ll_dict_getitem and ll_dict_get in
 ootypesystem.rdict do two consecutive lookups (calling ll_contains and
 ll_get) on the same key. We might cache the result of
-pypylib.Dict.ll_contains so that the succesive ll_get don't need a
+pypylib.Dict.ll_contains so that the successive ll_get don't need a
 lookup. Btw, we need some profiling before choosing the best way. Or
 we could directly refactor ootypesystem.rdict for doing a single
 lookup.

diff --git a/pypy/doc/discussion/howtoimplementpickling.txt b/pypy/doc/discussion/howtoimplementpickling.txt
--- a/pypy/doc/discussion/howtoimplementpickling.txt
+++ b/pypy/doc/discussion/howtoimplementpickling.txt
@@ -39,7 +39,7 @@
 of the program and its state (example: a Forth backend),
 I would see it as a valid solution, since it is
 relocatable. It is of course a possible fall-back to write
-sucn a backend of we fail otherwise.
+such a backend of we fail otherwise.
 
 There are some simple steps and some more difficult ones.
 Let's start with the simple.
@@ -94,7 +94,7 @@
 the prickelpit.c file in the Stackless distribution.
 
 As a conclusion, pickling of tasklets is an addition to Stackless,
-but not meand to be an extension to Python. The need to support
+but not meant to be an extension to Python. The need to support
 pickling of certain objects should not change the interface.
 It is better to decouple this and to use surrogate types for
 pickling which cannot collide with future additions to Python.
@@ -145,7 +145,7 @@
 SLP switching strategies
 ........................
 
-SLP has undergone several rewrites. The first implemenation was aiming
+SLP has undergone several rewrites. The first implementation was aiming
 at complete collaboration. A new frame's execution was deferred until
 all the preparational C function calls had left the C stack. There
 was no extra state to be saved.
@@ -195,13 +195,13 @@
 
 Right now, PyPy saves C state of functions in tiny activation records:
 the alive variables of a block, together with the entry point of
-the function tnat was left.
+the function that was left.
 This is an improvement over storing raw stack slices, but the pattern
 is similar: The C stack state gets restored when we switch.
 
 In this sense, it was the astonishing resume when Richard and I discussed
 this last week: PyPy essentially does a variant of Hard switching! At least it
-does a compromize that does not really help with pickling.
+does a compromise that does not really help with pickling.
 
 On the other hand, this approach is half the way. It turns out to
 be an improvement over SLP not to have to avoid recursions in the
@@ -240,7 +240,7 @@
 we really need to restore before we can do the function call?
 
 - the argument decoding is done, already, and the fact that we could have done
-  the function call shows, that no exception occured. We can ignore the rest
+  the function call shows, that no exception occurred. We can ignore the rest
   of this activation record and do the housekeeping.
   
 - the frame is prepared, and arguments are stored in it. The operation
@@ -268,7 +268,7 @@
 A rough sketch of the necessary analysis:
 
 for every block in an RPython function that can reach unwind:
-Analyse control flow. It should be immediately leading to
+Analyze control flow. It should be immediately leading to
 the return block with only one output variable. All other alive variables
 should have ended their liveness in this block.
 

diff --git a/pypy/doc/extending.txt b/pypy/doc/extending.txt
--- a/pypy/doc/extending.txt
+++ b/pypy/doc/extending.txt
@@ -12,8 +12,8 @@
 Possibilities
 =============
 
-Right now, there are three posibilities of providing third-party modules
-for the PyPy python interpreter (in order of usefulnes):
+Right now, there are three possibilities of providing third-party modules
+for the PyPy python interpreter (in order of usefulness):
 
 * Write them in pure python and use ctypes, see ctypes_
   section
@@ -39,7 +39,7 @@
 
 We also provide a `ctypes-configure`_ for overcoming the platform dependencies,
 not relying on the ctypes codegen. This tool works by querying gcc about
-platform-depenent details (compiling small snippets of C code and running
+platform-dependent details (compiling small snippets of C code and running
 them), so it'll benefit not pypy-related ctypes-based modules as well.
 
 .. _`ctypes-configure`: http://codespeak.net/~fijal/configure.html
@@ -91,9 +91,9 @@
   compilation-check requires to recompile whole PyPy python interpreter,
   which takes 0.5-1h. We plan to solve this at some point in near future.
 
-* although rpython is a garabage-collected language, the border between
+* although rpython is a garbage-collected language, the border between
   C and RPython needs to be managed by hand (each object that goes into the
-  C level must be explicitely freed) XXX we try to solve this
+  C level must be explicitly freed) XXX we try to solve this
 
 Some document is available `here`_
 

diff --git a/pypy/doc/distribution.txt b/pypy/doc/distribution.txt
--- a/pypy/doc/distribution.txt
+++ b/pypy/doc/distribution.txt
@@ -10,7 +10,7 @@
 
 The implementation uses an RPC-like protocol, which accesses
 only members of objects, rather than whole objects. This means it
-does not rely on objects being picklable, nor on having the same
+does not rely on objects being pickleable, nor on having the same
 source code available on both sides. On each call, only the members
 that are used on the client side are retrieved, objects which
 are not used are merely references to their remote counterparts.
@@ -27,7 +27,7 @@
 remote side replies by providing a bound method. On the client this bound
 method appears as a remote reference: this reference is called with a remote
 reference to x as self, the integer 1 which is copied as a primitive type, a
-reference to a list and a refence to y. The remote side receives this call,
+reference to a list and a reference to y. The remote side receives this call,
 processes it as a call to the bound method x.foo, where 'x' is resolved as a
 local object, 1 as an immutable primitive, [1,2,3] as a reference to a mutable
 primitive and y as a reference to a remote object. If the type of y is not
@@ -74,7 +74,7 @@
 
   - No attribute access
 
-  - Arguments of calls must be picklable on one side and unpicklable on
+  - Arguments of calls must be pickleable on one side and unpickleable on
     remote side, which means they must share source code, they do not
     become remote references
 
@@ -105,7 +105,7 @@
 
   - two way RPC (unlike Pyro)
 
-  - also heavy restrictions on objects - they must sublcass certain class
+  - also heavy restrictions on objects - they must subclass certain class
 
 .. _`Pyro`: http://pyro.sourceforge.net/
 .. _`transparent proxies`: objspace-proxies.html#tproxy

diff --git a/pypy/doc/cpython_differences.txt b/pypy/doc/cpython_differences.txt
--- a/pypy/doc/cpython_differences.txt
+++ b/pypy/doc/cpython_differences.txt
@@ -160,7 +160,7 @@
 ----------------------------
 
 Officially, CPython has no rule at all for when exactly
-overriden method of subclasses of built-in types get
+overridden method of subclasses of built-in types get
 implicitly called or not.  As an approximation, these methods
 are never called by other built-in methods of the same object.
 For example, an overridden ``__getitem__()`` in a subclass of
@@ -211,7 +211,7 @@
   PyPy.  On CPython it would set the maximum number of nested
   calls that can occur before a RuntimeError is raised; on PyPy
   overflowing the stack also causes RuntimeErrors, but the limit
-  is checked at a lower level.  (The limit is currenty hard-coded
+  is checked at a lower level.  (The limit is currently hard-coded
   at 768 KB, corresponding to roughly 1480 Python calls on
   Linux.)
 

diff --git a/pypy/doc/config/objspace.usemodules.posix.txt b/pypy/doc/config/objspace.usemodules.posix.txt
--- a/pypy/doc/config/objspace.usemodules.posix.txt
+++ b/pypy/doc/config/objspace.usemodules.posix.txt
@@ -1,3 +1,3 @@
 Use the essential 'posix' module.
 This module is essential, included by default and cannot be removed (even when
-specified explicitly, the option gets overriden later).
+specified explicitly, the option gets overridden later).

diff --git a/pypy/doc/buildtool.txt b/pypy/doc/buildtool.txt
--- a/pypy/doc/buildtool.txt
+++ b/pypy/doc/buildtool.txt
@@ -165,7 +165,7 @@
 Note that there is a test project in 'tool/build/testproject' that can serve
 as an example.
 
-Prerequisities
+Prerequisites
 --------------
 
 Your project can use the build tool if:

diff --git a/pypy/doc/config/translation.backendopt.profile_based_inline.txt b/pypy/doc/config/translation.backendopt.profile_based_inline.txt
--- a/pypy/doc/config/translation.backendopt.profile_based_inline.txt
+++ b/pypy/doc/config/translation.backendopt.profile_based_inline.txt
@@ -7,4 +7,4 @@
 The option takes as value a string which is the arguments to pass to
 the program for the instrumented run.
 
-This optimisation is not used by default.
\ No newline at end of file
+This optimization is not used by default.
\ No newline at end of file

diff --git a/pypy/doc/discussion/thoughts_string_interning.txt b/pypy/doc/discussion/thoughts_string_interning.txt
--- a/pypy/doc/discussion/thoughts_string_interning.txt
+++ b/pypy/doc/discussion/thoughts_string_interning.txt
@@ -8,7 +8,7 @@
 the dict lookup method will find the string always by identity,
 saving the need to do a string comparison.
 
-Interned Srings in CPython
+Interned Strings in CPython
 --------------------------
 
 CPython keeps an internal dictionary named ``interned`` for all of these
@@ -137,7 +137,7 @@
   D:\pypy\dist\pypy\translator\goal>
 
 
-This was just an exercize to get an idea. For sure this is not to be checked in.
+This was just an exercise to get an idea. For sure this is not to be checked in.
 Instead, I'm attaching the simple patch here for reference.
 ::
 

diff --git a/pypy/doc/discussion/summer-of-pypy-pytest.txt b/pypy/doc/discussion/summer-of-pypy-pytest.txt
--- a/pypy/doc/discussion/summer-of-pypy-pytest.txt
+++ b/pypy/doc/discussion/summer-of-pypy-pytest.txt
@@ -16,7 +16,7 @@
 Remote imports:
 ---------------
 
-On the beggining of communication, master server sends to client
+On the beginning of communication, master server sends to client
 import hook code, which then can import all needed libraries.
 
 Libraries are uploaded server -> client if they're needed (when
@@ -47,7 +47,7 @@
 Then we transfer output data to server as string, possibly tweaking
 file names (which is quite easy).
 
-Delivarables:
+Deliverables:
 =============
 
 - better use of testing machines

diff --git a/pypy/doc/discussion/improve-rpython.txt b/pypy/doc/discussion/improve-rpython.txt
--- a/pypy/doc/discussion/improve-rpython.txt
+++ b/pypy/doc/discussion/improve-rpython.txt
@@ -4,7 +4,7 @@
 Improve the interpreter API
 ---------------------------
 
-- Rationalize the modules, and the names, of the differents functions needed to
+- Rationalize the modules, and the names, of the different functions needed to
   implement a pypy module. A typical rpython file is likely to contain many
   `import` statements::
 
@@ -34,7 +34,7 @@
 ----------------
 
 - Arithmetic with unsigned integer, and between integer of different signedness,
-  when this is not ambiguous.  At least, comparison and assignement with
+  when this is not ambiguous.  At least, comparison and assignment with
   constants should be allowed.
 
 - Allocate variables on the stack, and pass their address ("by reference") to

diff --git a/pypy/doc/discussion/compiled-swamp.txt b/pypy/doc/discussion/compiled-swamp.txt
--- a/pypy/doc/discussion/compiled-swamp.txt
+++ b/pypy/doc/discussion/compiled-swamp.txt
@@ -3,7 +3,7 @@
 
 * benchmarks
 * tests
-* compliancy tests
+* compliance tests
 * play1
 * downloads
 * ...

diff --git a/pypy/doc/discussion/distribution-roadmap.txt b/pypy/doc/discussion/distribution-roadmap.txt
--- a/pypy/doc/discussion/distribution-roadmap.txt
+++ b/pypy/doc/discussion/distribution-roadmap.txt
@@ -13,7 +13,7 @@
 some kind of remote control over program execution. For start I would
 suggest using RMI (Remote Method Invocation) and remote object access
 (in case of low level it would be struct access). For the simplicity
-it will make some sense to target high-level platform at the beggining
+it will make some sense to target high-level platform at the beginning
 (CLI platform seems like obvious choice), which provides more primitives
 for performing such operations. To make attempt easier, I'll provide
 some subset of type system to be serializable which can go as parameters
@@ -52,7 +52,7 @@
 ---------------------------------------
 
 The biggest step here is to provide JIT integration into distribution
-system. This should allow to make it really usefull (probably compile-time
+system. This should allow to make it really useful (probably compile-time
 distribution will not work for example for whole Python interpreter, because
 of too huge granularity). This is quite unclear for me how to do that
 (JIT is not complete and I don't know too much about it). Probably we

From commits-noreply at bitbucket.org  Wed Feb  9 15:54:00 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed,  9 Feb 2011 15:54:00 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix a few failures in test_sqlite
Message-ID: <20110209145400.59CAE36C374@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41732:745e4490331c
Date: 2011-02-09 10:40 +0100
http://bitbucket.org/pypy/pypy/changeset/745e4490331c/

Log:	Fix a few failures in test_sqlite

diff --git a/lib_pypy/_sqlite3.py b/lib_pypy/_sqlite3.py
--- a/lib_pypy/_sqlite3.py
+++ b/lib_pypy/_sqlite3.py
@@ -339,6 +339,8 @@
     def _set_isolation_level(self, val):
         if val is None:
             self.commit()
+        if isinstance(val, unicode):
+            val = str(val)
         self._isolation_level = val
     isolation_level = property(_get_isolation_level, _set_isolation_level)
 
@@ -461,7 +463,7 @@
 class Cursor(object):
     def __init__(self, con):
         if not isinstance(con, Connection):
-            raise ValueError
+            raise TypeError
         con._check_thread()
         con._check_closed()
         self.connection = con
@@ -532,7 +534,7 @@
 
         while True:
             if not sqlite.sqlite3_complete(next_char):
-                raise ProgrammingError, "Incomplete statement '%s'" % next_char.value
+                raise OperationalError, "Incomplete statement '%s'" % next_char.value
             ret = sqlite.sqlite3_prepare_v2(self.connection.db, next_char, -1, byref(statement), byref(next_char))
             if ret != SQLITE_OK:
                 raise self.connection._get_exception(ret)
@@ -587,7 +589,7 @@
         self._check_closed()
         # XXX this should do reset and set statement to None it seems
 
-    def setinputsize(self, *args):
+    def setinputsizes(self, *args):
         pass
     def setoutputsize(self, *args):
         pass
@@ -819,6 +821,21 @@
     def keys(self):
         return [desc[0] for desc in self.description]
 
+    def __eq__(self, other):
+        if not isinstance(other, Row):
+            return NotImplemented
+        if self.description != other.description:
+            return False
+        if self.values != other.values:
+            return False
+        return True
+
+    def __ne__(self, other):
+        return not self == other
+
+    def __hash__(self):
+        return hash(tuple(self.description)) ^ hash(tuple(self.values))
+
 def _check_remaining_sql(s):
     state = "NORMAL"
     for char in s:

From commits-noreply at bitbucket.org  Wed Feb  9 15:54:05 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed,  9 Feb 2011 15:54:05 +0100 (CET)
Subject: [pypy-svn] pypy default: Add a modifiable copy of the sqlite3
	package
Message-ID: <20110209145405.4EC9E282C1A@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41733:e7682def25a3
Date: 2011-02-09 10:42 +0100
http://bitbucket.org/pypy/pypy/changeset/e7682def25a3/

Log:	Add a modifiable copy of the sqlite3 package Only tests should be
	modified...

diff --git a/lib-python/2.7.0/sqlite3/test/dbapi.py b/lib-python/modified-2.7.0/sqlite3/test/dbapi.py
copy from lib-python/2.7.0/sqlite3/test/dbapi.py
copy to lib-python/modified-2.7.0/sqlite3/test/dbapi.py

diff --git a/lib-python/2.7.0/sqlite3/test/py25tests.py b/lib-python/modified-2.7.0/sqlite3/test/py25tests.py
copy from lib-python/2.7.0/sqlite3/test/py25tests.py
copy to lib-python/modified-2.7.0/sqlite3/test/py25tests.py

diff --git a/lib-python/2.7.0/sqlite3/test/factory.py b/lib-python/modified-2.7.0/sqlite3/test/factory.py
copy from lib-python/2.7.0/sqlite3/test/factory.py
copy to lib-python/modified-2.7.0/sqlite3/test/factory.py

diff --git a/lib-python/2.7.0/sqlite3/dump.py b/lib-python/modified-2.7.0/sqlite3/dump.py
copy from lib-python/2.7.0/sqlite3/dump.py
copy to lib-python/modified-2.7.0/sqlite3/dump.py

diff --git a/lib-python/2.7.0/sqlite3/test/dump.py b/lib-python/modified-2.7.0/sqlite3/test/dump.py
copy from lib-python/2.7.0/sqlite3/test/dump.py
copy to lib-python/modified-2.7.0/sqlite3/test/dump.py

diff --git a/lib-python/2.7.0/sqlite3/test/userfunctions.py b/lib-python/modified-2.7.0/sqlite3/test/userfunctions.py
copy from lib-python/2.7.0/sqlite3/test/userfunctions.py
copy to lib-python/modified-2.7.0/sqlite3/test/userfunctions.py

diff --git a/lib-python/2.7.0/sqlite3/test/types.py b/lib-python/modified-2.7.0/sqlite3/test/types.py
copy from lib-python/2.7.0/sqlite3/test/types.py
copy to lib-python/modified-2.7.0/sqlite3/test/types.py

diff --git a/lib-python/2.7.0/sqlite3/dbapi2.py b/lib-python/modified-2.7.0/sqlite3/dbapi2.py
copy from lib-python/2.7.0/sqlite3/dbapi2.py
copy to lib-python/modified-2.7.0/sqlite3/dbapi2.py

diff --git a/lib-python/2.7.0/sqlite3/test/__init__.py b/lib-python/modified-2.7.0/sqlite3/test/__init__.py
copy from lib-python/2.7.0/sqlite3/test/__init__.py
copy to lib-python/modified-2.7.0/sqlite3/test/__init__.py

diff --git a/lib-python/2.7.0/sqlite3/__init__.py b/lib-python/modified-2.7.0/sqlite3/__init__.py
copy from lib-python/2.7.0/sqlite3/__init__.py
copy to lib-python/modified-2.7.0/sqlite3/__init__.py

diff --git a/lib-python/2.7.0/sqlite3/test/transactions.py b/lib-python/modified-2.7.0/sqlite3/test/transactions.py
copy from lib-python/2.7.0/sqlite3/test/transactions.py
copy to lib-python/modified-2.7.0/sqlite3/test/transactions.py

diff --git a/lib-python/2.7.0/sqlite3/test/hooks.py b/lib-python/modified-2.7.0/sqlite3/test/hooks.py
copy from lib-python/2.7.0/sqlite3/test/hooks.py
copy to lib-python/modified-2.7.0/sqlite3/test/hooks.py

diff --git a/lib-python/2.7.0/sqlite3/test/regression.py b/lib-python/modified-2.7.0/sqlite3/test/regression.py
copy from lib-python/2.7.0/sqlite3/test/regression.py
copy to lib-python/modified-2.7.0/sqlite3/test/regression.py

From commits-noreply at bitbucket.org  Wed Feb  9 15:54:06 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed,  9 Feb 2011 15:54:06 +0100 (CET)
Subject: [pypy-svn] pypy default: Every iterator should implement an
	__iter__ method
Message-ID: <20110209145406.1CF07282C18@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41734:e8463de728da
Date: 2011-02-09 10:45 +0100
http://bitbucket.org/pypy/pypy/changeset/e8463de728da/

Log:	Every iterator should implement an __iter__ method "This is required
	to allow both containers and iterators to be used with the for and
	in statements"

diff --git a/lib-python/modified-2.7.0/sqlite3/test/dbapi.py b/lib-python/modified-2.7.0/sqlite3/test/dbapi.py
--- a/lib-python/modified-2.7.0/sqlite3/test/dbapi.py
+++ b/lib-python/modified-2.7.0/sqlite3/test/dbapi.py
@@ -1,4 +1,4 @@
-#-*- coding: ISO-8859-1 -*-
+#-*- coding: iso-8859-1 -*-
 # pysqlite2/test/dbapi.py: tests for DB-API compliance
 #
 # Copyright (C) 2004-2010 Gerhard H&#65533;ring <gh at ghaering.de>
@@ -332,6 +332,9 @@
             def __init__(self):
                 self.value = 5
 
+            def __iter__(self):
+                return self
+
             def next(self):
                 if self.value == 10:
                     raise StopIteration

From commits-noreply at bitbucket.org  Wed Feb  9 15:54:06 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed,  9 Feb 2011 15:54:06 +0100 (CET)
Subject: [pypy-svn] pypy default: Implement aggregate functions in sqlite
Message-ID: <20110209145406.A5AED282C18@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41735:253b4ff37ba6
Date: 2011-02-09 14:08 +0100
http://bitbucket.org/pypy/pypy/changeset/253b4ff37ba6/

Log:	Implement aggregate functions in sqlite

diff --git a/lib_pypy/_sqlite3.py b/lib_pypy/_sqlite3.py
--- a/lib_pypy/_sqlite3.py
+++ b/lib_pypy/_sqlite3.py
@@ -23,6 +23,7 @@
 
 from ctypes import c_void_p, c_int, c_double, c_int64, c_char_p, cdll
 from ctypes import POINTER, byref, string_at, CFUNCTYPE, cast
+from ctypes import sizeof, c_ssize_t
 import datetime
 import sys
 import time
@@ -256,6 +257,7 @@
         self.NotSupportedError = NotSupportedError
 
         self.func_cache = {}
+        self.aggregate_instances = {}
         self.thread_ident = thread_get_ident()
 
     def _get_exception(self, error_code = None):
@@ -334,6 +336,14 @@
             cur.row_factory = self.row_factory
         return cur.executescript(*args)
 
+    def __call__(self, sql):
+        self._check_closed()
+        cur = Cursor(self)
+        if not isinstance(sql, (str, unicode)):
+            raise Warning("SQL is of wrong type. Must be string or unicode.")
+        statement = Statement(cur, sql, self.row_factory)
+        return statement
+
     def _get_isolation_level(self):
         return self._isolation_level
     def _set_isolation_level(self, val):
@@ -458,7 +468,75 @@
             raise self._get_exception(ret)
 
     def create_aggregate(self, name, num_args, cls):
-        raise NotImplementedError
+        self._check_thread()
+        self._check_closed()
+
+        try:
+            c_step_callback, c_final_callback = self.func_cache[cls]
+        except KeyError:
+            def step_callback(context, argc, c_params):
+
+                aggregate_ptr = cast(
+                    sqlite.sqlite3_aggregate_context(
+                    context, sizeof(c_ssize_t)),
+                    POINTER(c_ssize_t))
+
+                if not aggregate_ptr[0]:
+                    try:
+                        aggregate = cls()
+                    except Exception, e:
+                        msg = ("user-defined aggregate's '__init__' "
+                               "method raised error")
+                        sqlite.sqlite3_result_error(context, msg, len(msg))
+                        return
+                    aggregate_id = id(aggregate)
+                    self.aggregate_instances[aggregate_id] = aggregate
+                    aggregate_ptr[0] = aggregate_id
+                else:
+                    aggregate = self.aggregate_instances[aggregate_ptr[0]]
+
+                params = _convert_params(context, argc, c_params)
+                try:
+                    aggregate.step(*params)
+                except Exception, e:
+                    msg = ("user-defined aggregate's 'step' "
+                           "method raised error")
+                    sqlite.sqlite3_result_error(context, msg, len(msg))
+                return 0
+
+            def final_callback(context):
+
+                aggregate_ptr = cast(
+                    sqlite.sqlite3_aggregate_context(
+                    context, sizeof(c_ssize_t)),
+                    POINTER(c_ssize_t))
+
+                if aggregate_ptr[0]:
+                    aggregate = self.aggregate_instances[aggregate_ptr[0]]
+                    try:
+                        val = aggregate.finalize()
+                    except Exception, e:
+                        msg = ("user-defined aggregate's 'finalize' "
+                               "method raised error")
+                        sqlite.sqlite3_result_error(context, msg, len(msg))
+                    else:
+                        _convert_result(context, val)
+                    finally:
+                        del self.aggregate_instances[aggregate_ptr[0]]
+                return 0
+
+            c_step_callback = STEP(step_callback)
+            c_final_callback = FINAL(final_callback)
+
+            self.func_cache[cls] = c_step_callback, c_final_callback
+
+        ret = sqlite.sqlite3_create_function(self.db, name, num_args,
+                                             SQLITE_UTF8, None,
+                                             cast(None, FUNC),
+                                             c_step_callback,
+                                             c_final_callback)
+        if ret != SQLITE_OK:
+            raise self._get_exception(ret)
 
 class Cursor(object):
     def __init__(self, con):
@@ -936,14 +1014,16 @@
         sqlite.sqlite3_result_error(context, msg, len(msg))
     else:
         _convert_result(context, val)
-    return 0
 
-FUNC = CFUNCTYPE(c_int, c_void_p, c_int, POINTER(c_void_p))
-STEP = CFUNCTYPE(c_int, c_void_p, c_int, POINTER(c_void_p))
-FINAL = CFUNCTYPE(c_int, c_void_p)
+FUNC = CFUNCTYPE(None, c_void_p, c_int, POINTER(c_void_p))
+STEP = CFUNCTYPE(None, c_void_p, c_int, POINTER(c_void_p))
+FINAL = CFUNCTYPE(None, c_void_p)
 sqlite.sqlite3_create_function.argtypes = [c_void_p, c_char_p, c_int, c_int, c_void_p, FUNC, STEP, FINAL]
 sqlite.sqlite3_create_function.restype = c_int
 
+sqlite.sqlite3_aggregate_context.argtypes = [c_void_p, c_int]
+sqlite.sqlite3_aggregate_context.restype = c_void_p
+
 converters = {}
 adapters = {}
 

From commits-noreply at bitbucket.org  Wed Feb  9 15:54:09 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed,  9 Feb 2011 15:54:09 +0100 (CET)
Subject: [pypy-svn] pypy default: Don't print a TypeError when a "void f()"
	callback	function returns None.
Message-ID: <20110209145409.A4BF5282C18@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41736:b6997048efb8
Date: 2011-02-09 15:50 +0100
http://bitbucket.org/pypy/pypy/changeset/b6997048efb8/

Log:	Don't print a TypeError when a "void f()" callback function returns
	None.

diff --git a/pypy/module/test_lib_pypy/ctypes_tests/test_callbacks.py b/pypy/module/test_lib_pypy/ctypes_tests/test_callbacks.py
--- a/pypy/module/test_lib_pypy/ctypes_tests/test_callbacks.py
+++ b/pypy/module/test_lib_pypy/ctypes_tests/test_callbacks.py
@@ -205,4 +205,19 @@
         cfunc.restype = c_int
         res = cfunc(CTP(callback), lambda : 3)
         assert res == 3
-        
+
+    def test_callback_void(self, capsys):
+        import conftest
+        _ctypes_test = str(conftest.sofile)
+        dll = CDLL(_ctypes_test)
+
+        def callback():
+            pass
+
+        CTP = CFUNCTYPE(None)
+        cfunc = dll._testfunc_callback_void
+        cfunc.argtypes = [CTP]
+        cfunc(CTP(callback))
+        out, err = capsys.readouterr()
+        assert (out, err) == ("", "")
+

diff --git a/lib_pypy/_ctypes/function.py b/lib_pypy/_ctypes/function.py
--- a/lib_pypy/_ctypes/function.py
+++ b/lib_pypy/_ctypes/function.py
@@ -124,6 +124,8 @@
             # A callback into python
             self.callable = argument
             ffiargs, ffires = self._ffishapes(self._argtypes_, self._restype_)
+            if self._restype_ is None:
+                ffires = None
             self._ptr = _rawffi.CallbackPtr(self._wrap_callable(argument,
                                                                 self.argtypes),
                                             ffiargs, ffires, self._flags_)

diff --git a/pypy/module/test_lib_pypy/ctypes_tests/_ctypes_test.c b/pypy/module/test_lib_pypy/ctypes_tests/_ctypes_test.c
--- a/pypy/module/test_lib_pypy/ctypes_tests/_ctypes_test.c
+++ b/pypy/module/test_lib_pypy/ctypes_tests/_ctypes_test.c
@@ -166,6 +166,12 @@
   return (*func)(arg);
 }
 
+EXPORT(int) _testfunc_callback_void(void (*func)(void))
+{
+    func();
+    return 0;
+}
+
 #ifdef HAVE_LONG_LONG
 EXPORT(LONG_LONG) _testfunc_q_bhilfdq(signed char b, short h, int i, long l, float f,
 				     double d, LONG_LONG q)

From commits-noreply at bitbucket.org  Wed Feb  9 15:54:10 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed,  9 Feb 2011 15:54:10 +0100 (CET)
Subject: [pypy-svn] pypy default: These callbacks return void,
	no need to return 0
Message-ID: <20110209145410.7AD1E282C18@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41737:ef02e56e1e32
Date: 2011-02-09 15:52 +0100
http://bitbucket.org/pypy/pypy/changeset/ef02e56e1e32/

Log:	These callbacks return void, no need to return 0

diff --git a/lib_pypy/_sqlite3.py b/lib_pypy/_sqlite3.py
--- a/lib_pypy/_sqlite3.py
+++ b/lib_pypy/_sqlite3.py
@@ -456,7 +456,6 @@
         except KeyError:            
             def closure(context, nargs, c_params):
                 function_callback(callback, context, nargs, c_params)
-                return 0
             c_closure = FUNC(closure)
             self.func_cache[callback] = c_closure, closure
         ret = sqlite.sqlite3_create_function(self.db, name, num_args,
@@ -502,7 +501,6 @@
                     msg = ("user-defined aggregate's 'step' "
                            "method raised error")
                     sqlite.sqlite3_result_error(context, msg, len(msg))
-                return 0
 
             def final_callback(context):
 
@@ -523,7 +521,6 @@
                         _convert_result(context, val)
                     finally:
                         del self.aggregate_instances[aggregate_ptr[0]]
-                return 0
 
             c_step_callback = STEP(step_callback)
             c_final_callback = FINAL(final_callback)

From commits-noreply at bitbucket.org  Wed Feb  9 15:54:11 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed,  9 Feb 2011 15:54:11 +0100 (CET)
Subject: [pypy-svn] pypy default: /bin/bash: q: command not found
Message-ID: <20110209145411.1699F282C21@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41738:b588bca53453
Date: 2011-02-09 15:52 +0100
http://bitbucket.org/pypy/pypy/changeset/b588bca53453/

Log:	/bin/bash: q: command not found


From commits-noreply at bitbucket.org  Wed Feb  9 16:22:31 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed,  9 Feb 2011 16:22:31 +0100 (CET)
Subject: [pypy-svn] pypy default: Mark this test as an implementation detail.
Message-ID: <20110209152231.E3E84282C18@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41739:fdc5cd4f5273
Date: 2011-02-09 16:22 +0100
http://bitbucket.org/pypy/pypy/changeset/fdc5cd4f5273/

Log:	Mark this test as an implementation detail.

diff --git a/lib-python/2.7.0/test/test_list.py b/lib-python/modified-2.7.0/test/test_list.py
copy from lib-python/2.7.0/test/test_list.py
copy to lib-python/modified-2.7.0/test/test_list.py
--- a/lib-python/2.7.0/test/test_list.py
+++ b/lib-python/modified-2.7.0/test/test_list.py
@@ -15,6 +15,10 @@
         self.assertEqual(list(''), [])
         self.assertEqual(list('spam'), ['s', 'p', 'a', 'm'])
 
+    # the following test also works with pypy, but eats all your address
+    # space's RAM before raising and takes too long.
+    @test_support.impl_detail("eats all your RAM before working", pypy=False)
+    def test_segfault_1(self):
         if sys.maxsize == 0x7fffffff:
             # This test can currently only work on 32-bit machines.
             # XXX If/when PySequence_Length() returns a ssize_t, it should be
@@ -32,6 +36,7 @@
             #     http://sources.redhat.com/ml/newlib/2002/msg00369.html
             self.assertRaises(MemoryError, list, xrange(sys.maxint // 2))
 
+    def test_segfault_2(self):
         # This code used to segfault in Py2.4a3
         x = []
         x.extend(-y for y in x)

From commits-noreply at bitbucket.org  Wed Feb  9 16:50:43 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed,  9 Feb 2011 16:50:43 +0100 (CET)
Subject: [pypy-svn] pypy default: Reproduce the failing test in
	test_long_future.
Message-ID: <20110209155043.73F93282C18@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41740:d2af8c419459
Date: 2011-02-09 16:38 +0100
http://bitbucket.org/pypy/pypy/changeset/d2af8c419459/

Log:	Reproduce the failing test in test_long_future. For some reason it
	only fails when translated.

diff --git a/pypy/rlib/test/test_rbigint.py b/pypy/rlib/test/test_rbigint.py
--- a/pypy/rlib/test/test_rbigint.py
+++ b/pypy/rlib/test/test_rbigint.py
@@ -51,6 +51,15 @@
         f = op1.truediv(op2)
         assert f == 4.7298422347492634e-61      # exactly
 
+    def test_truediv_overflow(self):
+        overflowing = 2**1024 - 2**(1024-53-1)
+        op1 = rbigint.fromlong(overflowing-1)
+        op2 = rbigint.fromlong(1)
+        f = op1.truediv(op2)
+        assert f == 1.7976931348623157e+308     # exactly
+        op1 = rbigint.fromlong(overflowing)
+        py.test.raises(OverflowError, op1.truediv, op2)
+
     def test_mod(self):
         for op1 in [-50, -12, -2, -1, 1, 2, 50, 52]:
             for op2 in [-4, -2, -1, 1, 2, 8]:
@@ -637,3 +646,16 @@
                 assert res == str(long(values[i]))
                 res = interpret(fn, [i])
                 assert ''.join(res.chars) == str(long(values[i]))
+
+    def test_truediv_overflow(self):
+        overflowing = 2**1024 - 2**(1024-53-1)
+        op1 = rbigint.fromlong(overflowing)
+
+        def fn():
+            try:
+                return op1.truediv(rbigint.fromint(1))
+            except OverflowError:
+                return -42.0
+
+        res = interpret(fn, [])
+        assert res == -42.0

From commits-noreply at bitbucket.org  Wed Feb  9 16:50:44 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed,  9 Feb 2011 16:50:44 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix the test.  Phew,
	a lot of work just to find where it was.
Message-ID: <20110209155044.0185D282C18@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41741:d8158876f71f
Date: 2011-02-09 16:50 +0100
http://bitbucket.org/pypy/pypy/changeset/d8158876f71f/

Log:	Fix the test. Phew, a lot of work just to find where it was.

diff --git a/pypy/rlib/rbigint.py b/pypy/rlib/rbigint.py
--- a/pypy/rlib/rbigint.py
+++ b/pypy/rlib/rbigint.py
@@ -1517,7 +1517,7 @@
 
     # Round by remembering a modified copy of the low digit of x
     mask = 1 << (extra_bits - 1)
-    low = x.digit(0) | inexact
+    low = x.udigit(0) | inexact
     if (low & mask) != 0 and (low & (3*mask-1)) != 0:
         low += mask
     x_digit_0 = low & ~(mask-1)

From arigo at codespeak.net  Wed Feb  9 17:06:07 2011
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed,  9 Feb 2011 17:06:07 +0100 (CET)
Subject: [pypy-svn] r80315 - pypy/extradoc/pypy.org
Message-ID: <20110209160607.3EFEF282C18@codespeak.net>

Author: arigo
Date: Wed Feb  9 17:06:03 2011
New Revision: 80315

Modified:
   pypy/extradoc/pypy.org/compat.html
Log:
Regenerate.


Modified: pypy/extradoc/pypy.org/compat.html
==============================================================================
--- pypy/extradoc/pypy.org/compat.html	(original)
+++ pypy/extradoc/pypy.org/compat.html	Wed Feb  9 17:06:03 2011
@@ -82,7 +82,7 @@
 <li>divmod's nevow</li>
 <li>pyglet</li>
 </ul>
-<p>Known differencies that are not going to be fixed:</p>
+<p>Known differences that are not going to be fixed:</p>
 <ul>
 <li><p class="first">PyPy does not support refcounting semantics. The following code
 won't fill the file immediately, but only after a certain period


From arigo at codespeak.net  Wed Feb  9 17:06:33 2011
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed,  9 Feb 2011 17:06:33 +0100 (CET)
Subject: [pypy-svn] r80316 - in pypy/extradoc/pypy.org: . source
Message-ID: <20110209160633.EC8AD282C18@codespeak.net>

Author: arigo
Date: Wed Feb  9 17:06:31 2011
New Revision: 80316

Modified:
   pypy/extradoc/pypy.org/download.html
   pypy/extradoc/pypy.org/source/download.txt
Log:
Mention that the "Other versions" section contains the most up-to-date versions.


Modified: pypy/extradoc/pypy.org/download.html
==============================================================================
--- pypy/extradoc/pypy.org/download.html	(original)
+++ pypy/extradoc/pypy.org/download.html	Wed Feb  9 17:06:31 2011
@@ -53,7 +53,7 @@
 <ul class="download-menu simple">
 <li>Download<ul>
 <li><a class="reference internal" href="#default-with-a-jit-compiler">Default (with a JIT Compiler)</a></li>
-<li><a class="reference internal" href="#other-versions-without-a-jit">Other versions (without a JIT)</a></li>
+<li><a class="reference internal" href="#other-versions">Other versions</a></li>
 </ul>
 </li>
 <li><a class="reference internal" href="#installing">Installing</a> (optional)</li>
@@ -64,7 +64,9 @@
 <span id="default-with-a-jit-compiler"></span><h1>&ldquo;JIT Compiler&rdquo; version</h1>
 <p>These binaries include a Just-in-Time compiler.  They only work on
 x86 CPUs that have the <a class="reference external" href="http://en.wikipedia.org/wiki/SSE2">SSE2</a> instruction set (most of
-them do, nowadays), or on x86-64 CPUs.</p>
+them do, nowadays), or on x86-64 CPUs.
+(This is the official release 1.4.1;
+for the most up-to-date version see below.)</p>
 <ul class="simple">
 <li><a class="reference external" href="http://pypy.org/download/pypy-1.4.1-linux.tar.bz2">Linux binary (32bit)</a></li>
 <li><a class="reference external" href="http://pypy.org/download/pypy-1.4.1-linux64.tar.bz2">Linux binary (64bit)</a></li>

Modified: pypy/extradoc/pypy.org/source/download.txt
==============================================================================
--- pypy/extradoc/pypy.org/source/download.txt	(original)
+++ pypy/extradoc/pypy.org/source/download.txt	Wed Feb  9 17:06:31 2011
@@ -15,8 +15,8 @@
  * Download                         
                                                  
    * `Default (with a JIT Compiler)`_
-   * `Other versions (without a JIT)`_
-                                                 
+   * `Other versions`_
+
  * `Installing`_ (optional)                      
  * `Building from source`_                       
  * `Checksums`_
@@ -29,6 +29,8 @@
 These binaries include a Just-in-Time compiler.  They only work on
 x86 CPUs that have the SSE2_ instruction set (most of
 them do, nowadays), or on x86-64 CPUs.
+(This is the official release 1.4.1;
+for the most up-to-date version see below.)
 
 * `Linux binary (32bit)`__
 * `Linux binary (64bit)`__


From arigo at codespeak.net  Wed Feb  9 17:08:59 2011
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed,  9 Feb 2011 17:08:59 +0100 (CET)
Subject: [pypy-svn] r80317 - in pypy/extradoc/pypy.org: . source
Message-ID: <20110209160859.9CDB8282BEC@codespeak.net>

Author: arigo
Date: Wed Feb  9 17:08:53 2011
New Revision: 80317

Modified:
   pypy/extradoc/pypy.org/download.html
   pypy/extradoc/pypy.org/source/download.txt
Log:
Add a paragraph.

Modified: pypy/extradoc/pypy.org/download.html
==============================================================================
--- pypy/extradoc/pypy.org/download.html	(original)
+++ pypy/extradoc/pypy.org/download.html	Wed Feb  9 17:08:53 2011
@@ -81,6 +81,8 @@
 <span id="other-versions-without-a-jit"></span><h1>Other versions</h1>
 <p>The other versions of PyPy are:</p>
 <ul class="simple">
+<li>The most up-to-date nightly build with a JIT, if the official
+release is too old for what you want to do.</li>
 <li>No JIT: A version without the JIT.  Consumes a bit less memory
 and may be faster on short-running scripts.</li>
 <li>Stackless: Provides <a class="reference external" href="http://www.stackless.com/">Stackless</a> extensions, as well as <a class="reference external" href="http://codespeak.net/svn/greenlet/trunk/doc/greenlet.txt">greenlets</a>.

Modified: pypy/extradoc/pypy.org/source/download.txt
==============================================================================
--- pypy/extradoc/pypy.org/source/download.txt	(original)
+++ pypy/extradoc/pypy.org/source/download.txt	Wed Feb  9 17:08:53 2011
@@ -55,6 +55,9 @@
 
 The other versions of PyPy are:
 
+* The most up-to-date nightly build with a JIT, if the official
+  release is too old for what you want to do.
+
 * No JIT: A version without the JIT.  Consumes a bit less memory
   and may be faster on short-running scripts.
 


From arigo at codespeak.net  Wed Feb  9 17:09:45 2011
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed,  9 Feb 2011 17:09:45 +0100 (CET)
Subject: [pypy-svn] r80318 - in pypy/extradoc/pypy.org: . source
Message-ID: <20110209160945.215E6282BEC@codespeak.net>

Author: arigo
Date: Wed Feb  9 17:09:42 2011
New Revision: 80318

Modified:
   pypy/extradoc/pypy.org/download.html
   pypy/extradoc/pypy.org/source/download.txt
Log:
Add a link.

Modified: pypy/extradoc/pypy.org/download.html
==============================================================================
--- pypy/extradoc/pypy.org/download.html	(original)
+++ pypy/extradoc/pypy.org/download.html	Wed Feb  9 17:09:42 2011
@@ -81,7 +81,7 @@
 <span id="other-versions-without-a-jit"></span><h1>Other versions</h1>
 <p>The other versions of PyPy are:</p>
 <ul class="simple">
-<li>The most up-to-date nightly build with a JIT, if the official
+<li>The most up-to-date <a class="reference external" href="http://buildbot.pypy.org/nightly/trunk/">nightly build</a> with a JIT, if the official
 release is too old for what you want to do.</li>
 <li>No JIT: A version without the JIT.  Consumes a bit less memory
 and may be faster on short-running scripts.</li>

Modified: pypy/extradoc/pypy.org/source/download.txt
==============================================================================
--- pypy/extradoc/pypy.org/source/download.txt	(original)
+++ pypy/extradoc/pypy.org/source/download.txt	Wed Feb  9 17:09:42 2011
@@ -55,7 +55,7 @@
 
 The other versions of PyPy are:
 
-* The most up-to-date nightly build with a JIT, if the official
+* The most up-to-date `nightly build`_ with a JIT, if the official
   release is too old for what you want to do.
 
 * No JIT: A version without the JIT.  Consumes a bit less memory


From commits-noreply at bitbucket.org  Wed Feb  9 17:19:35 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Wed,  9 Feb 2011 17:19:35 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: for tests
Message-ID: <20110209161935.D2C38282BEC@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r41742:df6eaad56cde
Date: 2011-02-09 16:56 +0100
http://bitbucket.org/pypy/pypy/changeset/df6eaad56cde/

Log:	for tests

diff --git a/pypy/jit/metainterp/warmstate.py b/pypy/jit/metainterp/warmstate.py
--- a/pypy/jit/metainterp/warmstate.py
+++ b/pypy/jit/metainterp/warmstate.py
@@ -242,7 +242,8 @@
             self.warmrunnerdesc.memory_manager.set_max_age(value)
 
     def set_param_retrace_limit(self, value):
-        self.warmrunnerdesc.retrace_limit = value
+        if self.warmrunnerdesc:
+            self.warmrunnerdesc.retrace_limit = value
 
     def disable_noninlinable_function(self, greenkey):
         cell = self.jit_cell_at_key(greenkey)

From commits-noreply at bitbucket.org  Wed Feb  9 17:19:36 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Wed,  9 Feb 2011 17:19:36 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: warmrunnerdesc seems immutable
	when	translated? Is this a suitable place to store this param?
Message-ID: <20110209161936.E8A13282C18@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r41743:d3d58ea58c80
Date: 2011-02-09 17:18 +0100
http://bitbucket.org/pypy/pypy/changeset/d3d58ea58c80/

Log:	warmrunnerdesc seems immutable when translated? Is this a suitable
	place to store this param?

diff --git a/pypy/jit/metainterp/optimizeopt/unroll.py b/pypy/jit/metainterp/optimizeopt/unroll.py
--- a/pypy/jit/metainterp/optimizeopt/unroll.py
+++ b/pypy/jit/metainterp/optimizeopt/unroll.py
@@ -674,7 +674,7 @@
                 retraced_count = len(short)
                 if descr.failed_states:
                     retraced_count += len(descr.failed_states)
-                limit = self.optimizer.metainterp_sd.warmrunnerdesc.retrace_limit
+                limit = self.optimizer.metainterp_sd.warmrunnerdesc.memory_manager.retrace_limit
                 if not self.retraced and retraced_count<limit:
                     if not descr.failed_states:
                         raise RetraceLoop

diff --git a/pypy/jit/metainterp/warmstate.py b/pypy/jit/metainterp/warmstate.py
--- a/pypy/jit/metainterp/warmstate.py
+++ b/pypy/jit/metainterp/warmstate.py
@@ -243,7 +243,7 @@
 
     def set_param_retrace_limit(self, value):
         if self.warmrunnerdesc:
-            self.warmrunnerdesc.retrace_limit = value
+            self.warmrunnerdesc.memory_manager.retrace_limit = value
 
     def disable_noninlinable_function(self, greenkey):
         cell = self.jit_cell_at_key(greenkey)

From commits-noreply at bitbucket.org  Wed Feb  9 18:03:32 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed,  9 Feb 2011 18:03:32 +0100 (CET)
Subject: [pypy-svn] pypy default: Remove this unused import. It can actually
	come in the	way and
Message-ID: <20110209170332.B7259282BEC@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41744:0b4e504eedfc
Date: 2011-02-09 18:03 +0100
http://bitbucket.org/pypy/pypy/changeset/0b4e504eedfc/

Log:	Remove this unused import. It can actually come in the way and
	prevent some setup.py from running, if there is a local directory
	called "test/", for example.

diff --git a/lib-python/modified-2.7.0/distutils/command/install.py b/lib-python/modified-2.7.0/distutils/command/install.py
--- a/lib-python/modified-2.7.0/distutils/command/install.py
+++ b/lib-python/modified-2.7.0/distutils/command/install.py
@@ -20,7 +20,6 @@
 from distutils.errors import DistutilsOptionError
 from site import USER_BASE
 from site import USER_SITE
-from test.test_support import check_impl_detail
 
 
 if sys.version < "2.2":

From commits-noreply at bitbucket.org  Wed Feb  9 18:31:32 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Wed,  9 Feb 2011 18:31:32 +0100 (CET)
Subject: [pypy-svn] pypy default: Implemented PyDict_DelItemString in cpyext.
Message-ID: <20110209173132.7A94436C374@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41745:14abe2b13d33
Date: 2011-02-09 12:31 -0500
http://bitbucket.org/pypy/pypy/changeset/14abe2b13d33/

Log:	Implemented PyDict_DelItemString in cpyext.

diff --git a/pypy/module/cpyext/dictobject.py b/pypy/module/cpyext/dictobject.py
--- a/pypy/module/cpyext/dictobject.py
+++ b/pypy/module/cpyext/dictobject.py
@@ -60,6 +60,19 @@
         return None
     return borrow_from(w_dict, w_res)
 
+ at cpython_api([PyObject, rffi.CCHARP], rffi.INT_real, error=-1)
+def PyDict_DelItemString(space, w_dict, key_ptr):
+    """Remove the entry in dictionary p which has a key specified by the string
+    key.  Return 0 on success or -1 on failure."""
+    if PyDict_Check(space, w_dict):
+        key = rffi.charp2str(key_ptr)
+        # our dicts dont have a standardized interface, so we need
+        # to go through the space
+        space.delitem(w_dict, space.wrap(key))
+        return 0
+    else:
+        PyErr_BadInternalCall(space)
+
 @cpython_api([PyObject], Py_ssize_t, error=-1)
 def PyDict_Size(space, w_obj):
     """

diff --git a/pypy/module/cpyext/stubs.py b/pypy/module/cpyext/stubs.py
--- a/pypy/module/cpyext/stubs.py
+++ b/pypy/module/cpyext/stubs.py
@@ -564,12 +564,6 @@
     """
     raise NotImplementedError
 
- at cpython_api([PyObject, rffi.CCHARP], rffi.INT_real, error=-1)
-def PyDict_DelItemString(space, p, key):
-    """Remove the entry in dictionary p which has a key specified by the string
-    key.  Return 0 on success or -1 on failure."""
-    raise NotImplementedError
-
 @cpython_api([PyObject, PyObject, rffi.INT_real], rffi.INT_real, error=-1)
 def PyDict_Merge(space, a, b, override):
     """Iterate over mapping object b adding key-value pairs to dictionary a.

diff --git a/pypy/module/cpyext/test/test_dictobject.py b/pypy/module/cpyext/test/test_dictobject.py
--- a/pypy/module/cpyext/test/test_dictobject.py
+++ b/pypy/module/cpyext/test/test_dictobject.py
@@ -38,6 +38,15 @@
         api.PyErr_Clear()
         assert api.PyDict_Size(d) == 0
 
+        space.setitem(d, space.wrap("some_key"), space.wrap(3))
+        buf = rffi.str2charp("some_key")
+        assert api.PyDict_DelItemString(d, buf) == 0
+        assert api.PyDict_Size(d) == 0
+        assert api.PyDict_DelItemString(d, buf) < 0
+        assert api.PyErr_Occurred() is space.w_KeyError
+        api.PyErr_Clear()
+        rffi.free_charp(buf)
+
         d = space.wrap({'a': 'b'})
         api.PyDict_Clear(d)
         assert api.PyDict_Size(d) == 0

From commits-noreply at bitbucket.org  Wed Feb  9 19:13:25 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed,  9 Feb 2011 19:13:25 +0100 (CET)
Subject: [pypy-svn] pypy default: Implement collation functions in sqlite
Message-ID: <20110209181325.9F327282BEC@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41746:a1a0d7037b85
Date: 2011-02-09 16:19 +0100
http://bitbucket.org/pypy/pypy/changeset/a1a0d7037b85/

Log:	Implement collation functions in sqlite

diff --git a/lib_pypy/_sqlite3.py b/lib_pypy/_sqlite3.py
--- a/lib_pypy/_sqlite3.py
+++ b/lib_pypy/_sqlite3.py
@@ -258,6 +258,7 @@
 
         self.func_cache = {}
         self.aggregate_instances = {}
+        self._collations = {}
         self.thread_ident = thread_get_ident()
 
     def _get_exception(self, error_code = None):
@@ -440,7 +441,33 @@
             raise self._get_exception(ret)
 
     def create_collation(self, name, callback):
-        raise NotImplementedError
+        self._check_thread()
+        self._check_closed()
+        name = name.upper()
+        if not name.replace('_', '').isalnum():
+            raise ProgrammingError("invalid character in collation name")
+
+        if callback is None:
+            del self._collations[name]
+            c_collation_callback = cast(None, COLLATION)
+        else:
+            if not callable(callback):
+                raise TypeError("parameter must be callable")
+
+            def collation_callback(context, len1, str1, len2, str2):
+                text1 = string_at(str1, len1)
+                text2 = string_at(str2, len2)
+
+                return callback(text1, text2)
+
+            c_collation_callback = COLLATION(collation_callback)
+            self._collations[name] = collation_callback
+
+
+        ret = sqlite.sqlite3_create_collation(self.db, name,
+                                              SQLITE_UTF8,
+                                              None,
+                                              c_collation_callback)
 
     def set_progress_handler(self, callable, nsteps):
         raise NotImplementedError
@@ -1021,6 +1048,10 @@
 sqlite.sqlite3_aggregate_context.argtypes = [c_void_p, c_int]
 sqlite.sqlite3_aggregate_context.restype = c_void_p
 
+COLLATION = CFUNCTYPE(c_int, c_void_p, c_int, c_void_p, c_int, c_void_p)
+sqlite.sqlite3_create_collation.argtypes = [c_void_p, c_char_p, c_int, c_void_p, COLLATION]
+sqlite.sqlite3_create_collation.restype = c_int
+
 converters = {}
 adapters = {}
 

From commits-noreply at bitbucket.org  Wed Feb  9 19:13:26 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed,  9 Feb 2011 19:13:26 +0100 (CET)
Subject: [pypy-svn] pypy default: Implement sqlite progress handlers
Message-ID: <20110209181326.2B43E282BEC@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41747:1efaa4da25c9
Date: 2011-02-09 16:41 +0100
http://bitbucket.org/pypy/pypy/changeset/1efaa4da25c9/

Log:	Implement sqlite progress handlers

diff --git a/lib_pypy/_sqlite3.py b/lib_pypy/_sqlite3.py
--- a/lib_pypy/_sqlite3.py
+++ b/lib_pypy/_sqlite3.py
@@ -468,9 +468,33 @@
                                               SQLITE_UTF8,
                                               None,
                                               c_collation_callback)
+        if ret != SQLITE_OK:
+            raise self._get_exception(ret)
 
     def set_progress_handler(self, callable, nsteps):
-        raise NotImplementedError
+        self._check_thread()
+        self._check_closed()
+        if callable is None:
+            c_progress_handler = cast(None, PROGRESS)
+        else:
+            try:
+                c_progress_handler, _ = self.func_cache[callable]
+            except KeyError:
+                def progress_handler(userdata):
+                    try:
+                        ret = callable()
+                        return bool(ret)
+                    except Exception:
+                        # abort query if error occurred
+                        return 1
+                c_progress_handler = PROGRESS(progress_handler)
+
+                self.func_cache[callable] = c_progress_handler, progress_handler
+        ret = sqlite.sqlite3_progress_handler(self.db, nsteps,
+                                              c_progress_handler,
+                                              None)
+        if ret != SQLITE_OK:
+            raise self._get_exception(ret)
 
     def set_authorizer(self, callback):
         raise NotImplementedError
@@ -1052,6 +1076,10 @@
 sqlite.sqlite3_create_collation.argtypes = [c_void_p, c_char_p, c_int, c_void_p, COLLATION]
 sqlite.sqlite3_create_collation.restype = c_int
 
+PROGRESS = CFUNCTYPE(c_int, c_void_p)
+sqlite.sqlite3_progress_handler.argtypes = [c_void_p, c_int, PROGRESS, c_void_p]
+sqlite.sqlite3_progress_handler.restype = c_int
+
 converters = {}
 adapters = {}
 

From commits-noreply at bitbucket.org  Wed Feb  9 19:13:27 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed,  9 Feb 2011 19:13:27 +0100 (CET)
Subject: [pypy-svn] pypy default: sqlite: Fix the "check_closed()" tests
Message-ID: <20110209181327.040EA282BEC@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41748:9df946a3f31c
Date: 2011-02-09 16:54 +0100
http://bitbucket.org/pypy/pypy/changeset/9df946a3f31c/

Log:	sqlite: Fix the "check_closed()" tests

diff --git a/lib_pypy/_sqlite3.py b/lib_pypy/_sqlite3.py
--- a/lib_pypy/_sqlite3.py
+++ b/lib_pypy/_sqlite3.py
@@ -497,6 +497,8 @@
             raise self._get_exception(ret)
 
     def set_authorizer(self, callback):
+        self._check_thread()
+        self._check_closed()
         raise NotImplementedError
 
     def create_function(self, name, num_args, callback):
@@ -679,6 +681,7 @@
         return self.statement
 
     def fetchone(self):
+        self._check_closed()
         if self.statement is None:
             return None
         try:
@@ -687,6 +690,7 @@
             return None
 
     def fetchmany(self, size=None):
+        self._check_closed()
         if self.statement is None:
             return []
         if size is None:
@@ -699,6 +703,7 @@
         return lst
 
     def fetchall(self):
+        self._check_closed()
         if self.statement is None:
             return []
         return list(self.statement)
@@ -712,8 +717,13 @@
         return sqlite.sqlite3_last_insert_rowid(self.connection.db)
 
     def close(self):
+        if not self.connection:
+            return
         self._check_closed()
-        # XXX this should do reset and set statement to None it seems
+        if self.statement:
+            self.statement.reset()
+            self.statement = None
+        self.connection = None
 
     def setinputsizes(self, *args):
         pass

diff --git a/lib-python/modified-2.7.0/sqlite3/test/dbapi.py b/lib-python/modified-2.7.0/sqlite3/test/dbapi.py
--- a/lib-python/modified-2.7.0/sqlite3/test/dbapi.py
+++ b/lib-python/modified-2.7.0/sqlite3/test/dbapi.py
@@ -829,7 +829,7 @@
         con = sqlite.connect(":memory:")
         con.close()
         try:
-            con()
+            con("select 1")
             self.fail("Should have raised a ProgrammingError")
         except sqlite.ProgrammingError:
             pass

From commits-noreply at bitbucket.org  Wed Feb  9 19:13:28 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed,  9 Feb 2011 19:13:28 +0100 (CET)
Subject: [pypy-svn] pypy default: sqlite: Fix the "adapters" protocol.
Message-ID: <20110209181328.D218A282BEC@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41749:db3e8c4c7432
Date: 2011-02-09 17:05 +0100
http://bitbucket.org/pypy/pypy/changeset/db3e8c4c7432/

Log:	sqlite: Fix the "adapters" protocol.

diff --git a/lib_pypy/_sqlite3.py b/lib_pypy/_sqlite3.py
--- a/lib_pypy/_sqlite3.py
+++ b/lib_pypy/_sqlite3.py
@@ -800,7 +800,7 @@
         if cvt is not None:
             cvt = param = cvt(param)
 
-        adapter = adapters.get(type(param), None)
+        adapter = adapters.get((type(param), PrepareProtocol), None)
         if adapter is not None:
             param = adapter(param)
 
@@ -1015,7 +1015,7 @@
     return 0
 
 def register_adapter(typ, callable):
-    adapters[typ] = callable
+    adapters[typ, PrepareProtocol] = callable
 
 def register_converter(name, callable):
     converters[name.upper()] = callable

From commits-noreply at bitbucket.org  Wed Feb  9 19:13:30 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed,  9 Feb 2011 19:13:30 +0100 (CET)
Subject: [pypy-svn] pypy default: sqlite: implement set_authorizer
Message-ID: <20110209181330.2C657282BEC@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41750:a1c06bf59204
Date: 2011-02-09 17:19 +0100
http://bitbucket.org/pypy/pypy/changeset/a1c06bf59204/

Log:	sqlite: implement set_authorizer

diff --git a/lib_pypy/_sqlite3.py b/lib_pypy/_sqlite3.py
--- a/lib_pypy/_sqlite3.py
+++ b/lib_pypy/_sqlite3.py
@@ -499,7 +499,24 @@
     def set_authorizer(self, callback):
         self._check_thread()
         self._check_closed()
-        raise NotImplementedError
+
+        try:
+            c_authorizer, _ = self.func_cache[callable]
+        except KeyError:
+            def authorizer(userdata, action, arg1, arg2, dbname, source):
+                try:
+                    return int(callback(action, arg1, arg2, dbname, source))
+                except Exception, e:
+                    return SQLITE_DENY
+            c_authorizer = AUTHORIZER(authorizer)
+
+            self.func_cache[callable] = c_authorizer, authorizer
+
+        ret = sqlite.sqlite3_set_authorizer(self.db,
+                                            c_authorizer,
+                                            None)
+        if ret != SQLITE_OK:
+            raise self._get_exception(ret)
 
     def create_function(self, name, num_args, callback):
         self._check_thread()
@@ -1090,6 +1107,10 @@
 sqlite.sqlite3_progress_handler.argtypes = [c_void_p, c_int, PROGRESS, c_void_p]
 sqlite.sqlite3_progress_handler.restype = c_int
 
+AUTHORIZER = CFUNCTYPE(c_int, c_void_p, c_int, c_char_p, c_char_p, c_char_p, c_char_p)
+sqlite.sqlite3_set_authorizer.argtypes = [c_void_p, AUTHORIZER, c_void_p]
+sqlite.sqlite3_set_authorizer.restype = c_int
+
 converters = {}
 adapters = {}
 

From commits-noreply at bitbucket.org  Wed Feb  9 19:13:31 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed,  9 Feb 2011 19:13:31 +0100 (CET)
Subject: [pypy-svn] pypy default: SELECT statements should execute the query
	in execute(), and not wait for the user to call fetchone()
Message-ID: <20110209181331.032C7282BEC@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41751:a45980624e0e
Date: 2011-02-09 19:12 +0100
http://bitbucket.org/pypy/pypy/changeset/a45980624e0e/

Log:	SELECT statements should execute the query in execute(), and not
	wait for the user to call fetchone()

diff --git a/lib_pypy/_sqlite3.py b/lib_pypy/_sqlite3.py
--- a/lib_pypy/_sqlite3.py
+++ b/lib_pypy/_sqlite3.py
@@ -257,6 +257,7 @@
         self.NotSupportedError = NotSupportedError
 
         self.func_cache = {}
+        self._aggregates = {}
         self.aggregate_instances = {}
         self._collations = {}
         self.thread_ident = thread_get_ident()
@@ -378,6 +379,12 @@
         self._check_closed()
         if sqlite.sqlite3_get_autocommit(self.db):
             return
+
+        for statement in self.statements:
+            obj = statement()
+            if obj is not None:
+                obj.reset()
+
         try:
             sql = "COMMIT"
             statement = c_void_p()
@@ -396,6 +403,12 @@
         self._check_closed()
         if sqlite.sqlite3_get_autocommit(self.db):
             return
+
+        for statement in self.statements:
+            obj = statement()
+            if obj is not None:
+                obj.reset()
+
         try:
             sql = "ROLLBACK"
             statement = c_void_p()
@@ -501,7 +514,7 @@
         self._check_closed()
 
         try:
-            c_authorizer, _ = self.func_cache[callable]
+            c_authorizer, _ = self.func_cache[callback]
         except KeyError:
             def authorizer(userdata, action, arg1, arg2, dbname, source):
                 try:
@@ -510,7 +523,7 @@
                     return SQLITE_DENY
             c_authorizer = AUTHORIZER(authorizer)
 
-            self.func_cache[callable] = c_authorizer, authorizer
+            self.func_cache[callback] = c_authorizer, authorizer
 
         ret = sqlite.sqlite3_set_authorizer(self.db,
                                             c_authorizer,
@@ -541,7 +554,7 @@
         self._check_closed()
 
         try:
-            c_step_callback, c_final_callback = self.func_cache[cls]
+            c_step_callback, c_final_callback, _, _ = self._aggregates[cls]
         except KeyError:
             def step_callback(context, argc, c_params):
 
@@ -565,8 +578,9 @@
                     aggregate = self.aggregate_instances[aggregate_ptr[0]]
 
                 params = _convert_params(context, argc, c_params)
+                step = aggregate.step
                 try:
-                    aggregate.step(*params)
+                    step(*params)
                 except Exception, e:
                     msg = ("user-defined aggregate's 'step' "
                            "method raised error")
@@ -595,7 +609,8 @@
             c_step_callback = STEP(step_callback)
             c_final_callback = FINAL(final_callback)
 
-            self.func_cache[cls] = c_step_callback, c_final_callback
+            self._aggregates[cls] = (c_step_callback, c_final_callback,
+                                     step_callback, final_callback)
 
         ret = sqlite.sqlite3_create_function(self.db, name, num_args,
                                              SQLITE_UTF8, None,
@@ -639,10 +654,21 @@
                 self.connection._begin()
 
         self.statement.set_params(params)
+
+        # Actually execute the SQL statement
+        ret = sqlite.sqlite3_step(self.statement.statement)
+        if ret not in (SQLITE_DONE, SQLITE_ROW):
+            self.statement.reset()
+            raise self.connection._get_exception(ret)
+
+        if self.statement.kind == "DQL":
+            self.statement._readahead()
+            self.statement._build_row_cast_map()
+
         if self.statement.kind in ("DML", "DDL"):
-            ret = sqlite.sqlite3_step(self.statement.statement)
-            if ret != SQLITE_DONE:
-                raise self.connection._get_exception(ret)
+            self.statement.reset()
+
+        self.rowcount = -1
         if self.statement.kind == "DML":
             self.rowcount = sqlite.sqlite3_changes(self.connection.db)
 
@@ -701,11 +727,14 @@
         self._check_closed()
         if self.statement is None:
             return None
+
         try:
             return self.statement.next()
         except StopIteration:
             return None
 
+        return nextrow
+
     def fetchmany(self, size=None):
         self._check_closed()
         if self.statement is None:
@@ -785,8 +814,6 @@
 
         self._build_row_cast_map()
 
-        self.started = False
-
     def _build_row_cast_map(self):
         self.row_cast_map = []
         for i in range(sqlite.sqlite3_column_count(self.statement)):
@@ -880,27 +907,23 @@
     def next(self):
         self.con._check_closed()
         self.con._check_thread()
-        if not self.started:
-            self.item = self._readahead()
-            self.started = True
         if self.exhausted:
             raise StopIteration
         item = self.item
-        self.item = self._readahead()
+
+        ret = sqlite.sqlite3_step(self.statement)
+        if ret == SQLITE_DONE:
+            self.exhausted = True
+            self.item = None
+        elif ret != SQLITE_ROW:
+            exc = self.con._get_exception(ret)
+            sqlite.sqlite3_reset(self.statement)
+            raise exc
+
+        self._readahead()
         return item
 
     def _readahead(self):
-        ret = sqlite.sqlite3_step(self.statement)
-        if ret == SQLITE_DONE:
-            self.exhausted = True
-            return
-        elif ret == SQLITE_ERROR:
-            sqlite.sqlite3_reset(self.statement)
-            exc = self.con._get_exception(ret)
-            raise exc
-        else:
-            assert ret == SQLITE_ROW
-
         self.column_count = sqlite.sqlite3_column_count(self.statement)
         row = []
         for i in xrange(self.column_count):
@@ -935,11 +958,11 @@
         row = tuple(row)
         if self.row_factory is not None:
             row = self.row_factory(self.cur(), row)
-        return row
+        self.item = row
 
     def reset(self):
         self.row_cast_map = None
-        sqlite.sqlite3_reset(self.statement)
+        return sqlite.sqlite3_reset(self.statement)
 
     def finalize(self):
         sqlite.sqlite3_finalize(self.statement)

From commits-noreply at bitbucket.org  Wed Feb  9 19:13:31 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed,  9 Feb 2011 19:13:31 +0100 (CET)
Subject: [pypy-svn] pypy default: Merge heads
Message-ID: <20110209181331.79FEC282C20@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41752:f38d47adc761
Date: 2011-02-09 19:12 +0100
http://bitbucket.org/pypy/pypy/changeset/f38d47adc761/

Log:	Merge heads


From commits-noreply at bitbucket.org  Wed Feb  9 19:44:36 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed,  9 Feb 2011 19:44:36 +0100 (CET)
Subject: [pypy-svn] pypy default: A failing test,
	but skipped because it looks not important.
Message-ID: <20110209184436.3E084282BEC@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41753:0807e9d2e666
Date: 2011-02-09 18:33 +0100
http://bitbucket.org/pypy/pypy/changeset/0807e9d2e666/

Log:	A failing test, but skipped because it looks not important. Will
	come back to it later.

diff --git a/pypy/interpreter/test/test_pyframe.py b/pypy/interpreter/test/test_pyframe.py
--- a/pypy/interpreter/test/test_pyframe.py
+++ b/pypy/interpreter/test/test_pyframe.py
@@ -407,3 +407,30 @@
         res = f(1)
         sys.settrace(None)
         assert res == 42
+
+    def test_set_unset_f_trace(self):
+        skip("in-progress, but not too important")
+        import sys
+        seen = []
+        def trace1(frame, what, arg):
+            seen.append((1, frame, frame.f_lineno, what, arg))
+            return trace1
+        def trace2(frame, what, arg):
+            seen.append((2, frame, frame.f_lineno, what, arg))
+            return trace2
+        def set_the_trace(f):
+            f.f_trace = trace1
+            sys.settrace(trace2)
+            len(seen)     # take one line: should not be traced
+        f = sys._getframe()
+        set_the_trace(f)
+        len(seen)     # take one line: should not be traced
+        len(seen)     # take one line: should not be traced
+        sys.settrace(None)   # and this line should be the last line traced
+        len(seen)     # take one line
+        del f.f_trace
+        len(seen)     # take one line
+        firstline = set_the_trace.func_code.co_firstlineno
+        assert seen == [(1, f, firstline + 6, 'line', None),
+                        (1, f, firstline + 7, 'line', None),
+                        (1, f, firstline + 8, 'line', None)]

From commits-noreply at bitbucket.org  Wed Feb  9 19:44:36 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed,  9 Feb 2011 19:44:36 +0100 (CET)
Subject: [pypy-svn] pypy default: Managed to write a failing test.
Message-ID: <20110209184436.D0F73282BEC@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41754:58c7ba7fb150
Date: 2011-02-09 19:28 +0100
http://bitbucket.org/pypy/pypy/changeset/58c7ba7fb150/

Log:	Managed to write a failing test.

diff --git a/pypy/interpreter/test/test_pyframe.py b/pypy/interpreter/test/test_pyframe.py
--- a/pypy/interpreter/test/test_pyframe.py
+++ b/pypy/interpreter/test/test_pyframe.py
@@ -1,6 +1,12 @@
+from pypy.tool import udir
+
 
 class AppTestPyFrame:
 
+    def setup_class(cls):
+        cls.w_udir = cls.space.wrap(str(udir.udir))
+        cls.w_tempfile1 = cls.space.wrap(str(udir.udir.join('tempfile1')))
+
     # test for the presence of the attributes, not functionality
 
     def test_f_locals(self):
@@ -202,6 +208,32 @@
         assert l[0][1] == 'call'
         assert res == 'hidden' # sanity
 
+    def test_trace_hidden_prints(self):
+        import sys
+
+        l = []
+        def trace(a,b,c):
+            l.append((a,b,c))
+            return trace
+
+        outputf = open(self.tempfile1, 'w')
+        def f():
+            print >> outputf, 1
+            print >> outputf, 2
+            print >> outputf, 3
+            return "that's the return value"
+
+        sys.settrace(trace)
+        f()
+        sys.settrace(None)
+        outputf.close()
+        # should get 1 "call", 3 "line" and 1 "return" events, and no call
+        # or return for the internal app-level implementation of 'print'
+        assert len(l) == 6
+        assert [what for (frame, what, arg) in l] == [
+            'call', 'line', 'line', 'line', 'line', 'return']
+        assert l[-1][2] == "that's the return value"
+
     def test_trace_return_exc(self):
         import sys
         l = []

From commits-noreply at bitbucket.org  Wed Feb  9 19:44:37 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed,  9 Feb 2011 19:44:37 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix the test.
Message-ID: <20110209184437.7EC57282BEC@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41755:05a43810eb35
Date: 2011-02-09 19:43 +0100
http://bitbucket.org/pypy/pypy/changeset/05a43810eb35/

Log:	Fix the test.

diff --git a/pypy/interpreter/executioncontext.py b/pypy/interpreter/executioncontext.py
--- a/pypy/interpreter/executioncontext.py
+++ b/pypy/interpreter/executioncontext.py
@@ -165,7 +165,12 @@
     def return_trace(self, frame, w_retval):
         "Trace the return from a function"
         if self.w_tracefunc is not None:
-            self._trace(frame, 'return', w_retval)
+            return_from_hidden = self._trace(frame, 'return', w_retval)
+            # special case: if we are returning from a hidden function,
+            # then maybe we have to fire() the action again; otherwise
+            # it will not be called.  See test_trace_hidden_prints.
+            if return_from_hidden:
+                self.space.frame_trace_action.fire()
 
     def bytecode_trace(self, frame, decr_by=TICK_COUNTER_STEP):
         "Trace function called before each bytecode."
@@ -254,7 +259,7 @@
 
     def _trace(self, frame, event, w_arg, operr=None):
         if self.is_tracing or frame.hide():
-            return
+            return True
 
         space = self.space
         
@@ -292,7 +297,7 @@
         if self.profilefunc is not None:
             if event not in ['leaveframe', 'call', 'c_call',
                              'c_return', 'c_exception']:
-                return
+                return False
 
             last_exception = frame.last_exception
             if event == 'leaveframe':
@@ -312,6 +317,7 @@
             finally:
                 frame.last_exception = last_exception
                 self.is_tracing -= 1
+        return False
 
     def checksignals(self):
         """Similar to PyErr_CheckSignals().  If called in the main thread,

From commits-noreply at bitbucket.org  Wed Feb  9 19:44:39 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed,  9 Feb 2011 19:44:39 +0100 (CET)
Subject: [pypy-svn] pypy default: merge heads
Message-ID: <20110209184439.08777282BEC@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41756:7c1044ee2b80
Date: 2011-02-09 19:44 +0100
http://bitbucket.org/pypy/pypy/changeset/7c1044ee2b80/

Log:	merge heads

diff --git a/pypy/interpreter/test/test_pyframe.py b/pypy/interpreter/test/test_pyframe.py
--- a/pypy/interpreter/test/test_pyframe.py
+++ b/pypy/interpreter/test/test_pyframe.py
@@ -439,3 +439,30 @@
         res = f(1)
         sys.settrace(None)
         assert res == 42
+
+    def test_set_unset_f_trace(self):
+        skip("in-progress, but not too important")
+        import sys
+        seen = []
+        def trace1(frame, what, arg):
+            seen.append((1, frame, frame.f_lineno, what, arg))
+            return trace1
+        def trace2(frame, what, arg):
+            seen.append((2, frame, frame.f_lineno, what, arg))
+            return trace2
+        def set_the_trace(f):
+            f.f_trace = trace1
+            sys.settrace(trace2)
+            len(seen)     # take one line: should not be traced
+        f = sys._getframe()
+        set_the_trace(f)
+        len(seen)     # take one line: should not be traced
+        len(seen)     # take one line: should not be traced
+        sys.settrace(None)   # and this line should be the last line traced
+        len(seen)     # take one line
+        del f.f_trace
+        len(seen)     # take one line
+        firstline = set_the_trace.func_code.co_firstlineno
+        assert seen == [(1, f, firstline + 6, 'line', None),
+                        (1, f, firstline + 7, 'line', None),
+                        (1, f, firstline + 8, 'line', None)]

From commits-noreply at bitbucket.org  Wed Feb  9 19:55:35 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed,  9 Feb 2011 19:55:35 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix the test. Maybe that's marginally too
	much code for	such a
Message-ID: <20110209185535.B2A1D282BEC@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41757:1bf35a5504c8
Date: 2011-02-09 19:55 +0100
http://bitbucket.org/pypy/pypy/changeset/1bf35a5504c8/

Log:	Fix the test. Maybe that's marginally too much code for such a
	corner case, but well, you never know who is going to rely on it.

diff --git a/pypy/interpreter/test/test_pyframe.py b/pypy/interpreter/test/test_pyframe.py
--- a/pypy/interpreter/test/test_pyframe.py
+++ b/pypy/interpreter/test/test_pyframe.py
@@ -441,7 +441,6 @@
         assert res == 42
 
     def test_set_unset_f_trace(self):
-        skip("in-progress, but not too important")
         import sys
         seen = []
         def trace1(frame, what, arg):

diff --git a/pypy/interpreter/executioncontext.py b/pypy/interpreter/executioncontext.py
--- a/pypy/interpreter/executioncontext.py
+++ b/pypy/interpreter/executioncontext.py
@@ -209,6 +209,7 @@
         """Set the global trace function."""
         if self.space.is_w(w_func, self.space.w_None):
             self.w_tracefunc = None
+            self.space.frame_trace_action.unfire()
         else:
             self.force_all_frames()
             self.w_tracefunc = w_func
@@ -357,6 +358,15 @@
             # to run at the next possible bytecode
             self.reset_ticker(-1)
 
+    def unfire(self, action):
+        """Cancel the action, if scheduled."""
+        alist = self.fired_actions
+        if alist is not None:
+            for i in range(len(alist)-1, -1, -1):
+                if alist[i] is action:
+                    del alist[i]
+        action._fired = False
+
     def register_periodic_action(self, action, use_bytecode_counter):
         """NOT_RPYTHON:
         Register the PeriodicAsyncAction action to be called whenever the
@@ -439,6 +449,10 @@
         The action must have been registered at space initalization time."""
         self.space.actionflag.fire(self)
 
+    def unfire(self):
+        """Cancel the action, if scheduled."""
+        self.space.actionflag.unfire(self)
+
     def fire_after_thread_switch(self):
         """Bit of a hack: fire() the action but only the next time the GIL
         is released and re-acquired (i.e. after a potential thread switch).

From commits-noreply at bitbucket.org  Wed Feb  9 19:59:22 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed,  9 Feb 2011 19:59:22 +0100 (CET)
Subject: [pypy-svn] pypy default: Revert 1bf35a5504c8, and do it differently:
Message-ID: <20110209185922.A7DD1282BEC@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41758:e677603812b9
Date: 2011-02-09 19:58 +0100
http://bitbucket.org/pypy/pypy/changeset/e677603812b9/

Log:	Revert 1bf35a5504c8, and do it differently: found a simpler one-
	liner solution.

diff --git a/pypy/interpreter/executioncontext.py b/pypy/interpreter/executioncontext.py
--- a/pypy/interpreter/executioncontext.py
+++ b/pypy/interpreter/executioncontext.py
@@ -209,7 +209,6 @@
         """Set the global trace function."""
         if self.space.is_w(w_func, self.space.w_None):
             self.w_tracefunc = None
-            self.space.frame_trace_action.unfire()
         else:
             self.force_all_frames()
             self.w_tracefunc = w_func
@@ -358,15 +357,6 @@
             # to run at the next possible bytecode
             self.reset_ticker(-1)
 
-    def unfire(self, action):
-        """Cancel the action, if scheduled."""
-        alist = self.fired_actions
-        if alist is not None:
-            for i in range(len(alist)-1, -1, -1):
-                if alist[i] is action:
-                    del alist[i]
-        action._fired = False
-
     def register_periodic_action(self, action, use_bytecode_counter):
         """NOT_RPYTHON:
         Register the PeriodicAsyncAction action to be called whenever the
@@ -449,10 +439,6 @@
         The action must have been registered at space initalization time."""
         self.space.actionflag.fire(self)
 
-    def unfire(self):
-        """Cancel the action, if scheduled."""
-        self.space.actionflag.unfire(self)
-
     def fire_after_thread_switch(self):
         """Bit of a hack: fire() the action but only the next time the GIL
         is released and re-acquired (i.e. after a potential thread switch).
@@ -525,7 +511,8 @@
     """An action that calls the local trace functions (w_f_trace)."""
 
     def perform(self, executioncontext, frame):
-        if frame.w_f_trace is None or executioncontext.is_tracing:
+        if (frame.w_f_trace is None or executioncontext.is_tracing or
+            executioncontext.w_tracefunc is None):
             return
         code = frame.pycode
         if frame.instr_lb <= frame.last_instr < frame.instr_ub:

From commits-noreply at bitbucket.org  Wed Feb  9 20:07:13 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Wed,  9 Feb 2011 20:07:13 +0100 (CET)
Subject: [pypy-svn] pypy out-of-line-guards: Ooops, fix broken merge
Message-ID: <20110209190713.1FFD9282BEC@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: out-of-line-guards
Changeset: r41759:907f1fb18d59
Date: 2011-02-09 11:38 +0200
http://bitbucket.org/pypy/pypy/changeset/907f1fb18d59/

Log:	Ooops, fix broken merge

diff --git a/pypy/jit/metainterp/test/test_optimizeopt.py b/pypy/jit/metainterp/test/test_optimizeopt.py
--- a/pypy/jit/metainterp/test/test_optimizeopt.py
+++ b/pypy/jit/metainterp/test/test_optimizeopt.py
@@ -606,6 +606,7 @@
         self.optimize_loop(ops, expected)
 
     def test_int_is_true_is_zero2(self):
+        py.test.skip("Not there yet")
         ops = """
         [i0]
         i2 = int_is_zero(i0)

diff --git a/pypy/jit/metainterp/optimizeopt/rewrite.py b/pypy/jit/metainterp/optimizeopt/rewrite.py
--- a/pypy/jit/metainterp/optimizeopt/rewrite.py
+++ b/pypy/jit/metainterp/optimizeopt/rewrite.py
@@ -228,15 +228,8 @@
     def optimize_GUARD_TRUE(self, op, dryrun=False):
         self.optimize_guard(op, CONST_1, dryrun=dryrun)
 
-    def optimize_GUARD_FALSE(self, op):
-        value = self.getvalue(op.getarg(0))
-        if value in self.optimizer.int_to_bool_nullity:
-            v, int_is_true = self.optimizer.int_to_bool_nullity[value]
-            if int_is_true:
-                v.make_constant(ConstInt(0))
-            elif not v.is_nonnull():
-                v.make_nonnull(len(self.optimizer.newoperations) - 1)
-        self.optimize_guard(op, CONST_0)
+    def optimize_GUARD_FALSE(self, op, dryrun=False):
+        self.optimize_guard(op, CONST_0, dryrun=dryrun)
 
     def optimize_GUARD_CLASS(self, op, dryrun=False):
         value = self.getvalue(op.getarg(0))
@@ -300,7 +293,7 @@
         self.emit_operation(op)
         resvalue = self.getvalue(op.result)
         self.optimizer.loop_invariant_results[key] = resvalue
-    
+
     def _optimize_nullness(self, op, box, expect_nonnull):
         value = self.getvalue(box)
         if value.is_nonnull():
@@ -316,13 +309,9 @@
             self.make_equal_to(op.result, v)
             return
         self._optimize_nullness(op, op.getarg(0), True)
-        self.optimizer.int_to_bool_nullity[self.getvalue(op.result)] = (v, True)
 
     def optimize_INT_IS_ZERO(self, op):
-        v = self.getvalue(op.getarg(0))
         self._optimize_nullness(op, op.getarg(0), False)
-        self.optimizer.int_to_bool_nullity[self.getvalue(op.result)] = (v,
-                                                                        False)
 
     def _optimize_oois_ooisnot(self, op, expect_isnot):
         value0 = self.getvalue(op.getarg(0))

diff --git a/pypy/jit/metainterp/optimizeopt/optimizer.py b/pypy/jit/metainterp/optimizeopt/optimizer.py
--- a/pypy/jit/metainterp/optimizeopt/optimizer.py
+++ b/pypy/jit/metainterp/optimizeopt/optimizer.py
@@ -238,8 +238,6 @@
         self.interned_refs = self.cpu.ts.new_ref_dict()
         self.resumedata_memo = resume.ResumeDataLoopMemo(metainterp_sd)
         self.bool_boxes = {}
-        self.int_to_bool_nullity = {} # a mapping from bool boxes to
-        # respective ints, to know their value
         self.loop_invariant_results = {}
         self.pure_operations = args_dict()
         self.producer = {}

From commits-noreply at bitbucket.org  Wed Feb  9 20:07:14 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Wed,  9 Feb 2011 20:07:14 +0100 (CET)
Subject: [pypy-svn] pypy numpy-exp: Initial fun
Message-ID: <20110209190714.98CD0282C1A@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: numpy-exp
Changeset: r41760:f715a1e1078a
Date: 2011-02-09 20:04 +0200
http://bitbucket.org/pypy/pypy/changeset/f715a1e1078a/

Log:	Initial fun

diff --git a/pypy/module/micronumpy/__init__.py b/pypy/module/micronumpy/__init__.py
--- a/pypy/module/micronumpy/__init__.py
+++ b/pypy/module/micronumpy/__init__.py
@@ -6,8 +6,7 @@
     applevel_name = 'numpy'
     
     interpleveldefs = {
-        'zeros'    : 'numarray.zeros',
-        'minimum'  : 'ufunc.minimum',
-        }
+        'array': 'numarray.SingleDimArray',
+    }
 
     appleveldefs = {}

diff --git a/pypy/module/micronumpy/numarray.py b/pypy/module/micronumpy/numarray.py
--- a/pypy/module/micronumpy/numarray.py
+++ b/pypy/module/micronumpy/numarray.py
@@ -1,127 +1,67 @@
 
 from pypy.interpreter.baseobjspace import ObjSpace, W_Root, Wrappable
-from pypy.interpreter.error import OperationError
+from pypy.interpreter.error import OperationError, operationerrfmt
 from pypy.interpreter.typedef import TypeDef
 from pypy.interpreter.gateway import interp2app, NoneNotWrapped
-from pypy.rlib.debug import make_sure_not_resized
+from pypy.rpython.lltypesystem import lltype
 
-class BaseNumArray(Wrappable):
+TP = lltype.GcArray(lltype.Float)
+
+class BaseBytecode(Wrappable):
     pass
 
-class NumArray(BaseNumArray):
-    def __init__(self, space, dim, dtype):
-        self.dim = dim
-        self.space = space
-        # ignore dtype for now
-        self.storage = [0] * dim
-        make_sure_not_resized(self.storage)
+class Add(BaseBytecode):
+    def __init__(self, left, right):
+        self.left = left
+        self.right = right
 
-    def descr_getitem(self, index):
-        space = self.space
-        try:
-            return space.wrap(self.storage[index])
-        except IndexError:
-            raise OperationError(space.w_IndexError,
-                                 space.wrap("list index out of range"))
-    descr_getitem.unwrap_spec = ['self', int]
+class SingleDimArray(Wrappable):
+    def __init__(self, size):
+        self.size = size
+        self.storage = lltype.malloc(TP, size, zero=True)
 
-    def descr_setitem(self, index, value):
-        space = self.space
-        try:
-            self.storage[index] = value
-        except IndexError:
-            raise OperationError(space.w_IndexError,
-                                 space.wrap("list index out of range"))
-        return space.w_None
-    descr_setitem.unwrap_spec = ['self', int, int]
+    def descr_getitem(self, space, item):
+        if item < 0:
+            raise operationerrfmt(space.w_TypeError,
+              '%d below zero', item)
+        if item > self.size:
+            raise operationerrfmt(space.w_TypeError,
+              '%d above array size', item)
+        return space.wrap(self.storage[item])
+    descr_getitem.unwrap_spec = ['self', ObjSpace, int]
 
-    def descr_len(self):
-        return self.space.wrap(len(self.storage))
-    descr_len.unwrap_spec = ['self']
+    def descr_setitem(self, space, item, value):
+        if item < 0:
+            raise operationerrfmt(space.w_TypeError,
+              '%d below zero', item)
+        if item > self.size:
+            raise operationerrfmt(space.w_TypeError,
+              '%d above array size', item)
+        self.storage[item] = value
+    descr_setitem.unwrap_spec = ['self', ObjSpace, int, float]
 
-NumArray.typedef = TypeDef(
-    'NumArray',
-    __getitem__ = interp2app(NumArray.descr_getitem),
-    __setitem__ = interp2app(NumArray.descr_setitem),
-    __len__     = interp2app(NumArray.descr_len),
+    def descr_add(self, space, w_other):
+        return space.wrap(Add(self, w_other))
+    descr_add.unwrap_spec = ['self', ObjSpace, W_Root]
+
+def descr_new_numarray(space, w_type, w_size_or_iterable):
+    if space.isinstance_w(w_size_or_iterable, space.w_int):
+        arr = SingleDimArray(space.int_w(w_size_or_iterable))
+    else:
+        l = space.listview(w_size_or_iterable)
+        arr = SingleDimArray(len(l))
+        i = 0
+        for w_elem in l:
+            arr.storage[i] = space.float_w(space.float(w_elem))
+            i += 1
+    return space.wrap(arr)
+descr_new_numarray.unwrap_spec = [ObjSpace, W_Root, W_Root]
+
+SingleDimArray.typedef = TypeDef(
+    'numarray',
+    __new__ = interp2app(descr_new_numarray),
+    __getitem__ = interp2app(SingleDimArray.descr_getitem),
+    __setitem__ = interp2app(SingleDimArray.descr_setitem),
+    __add__ = interp2app(SingleDimArray.descr_add),
 )
 
-def compute_pos(space, indexes, dim):
-    current = 1
-    pos = 0
-    for i in range(len(indexes)):
-        index = indexes[i]
-        d = dim[i]
-        if index >= d or index <= -d - 1:
-            raise OperationError(space.w_IndexError,
-                                 space.wrap("invalid index"))
-        if index < 0:
-            index = d + index
-        pos += index * current
-        current *= d
-    return pos
-
-class MultiDimArray(BaseNumArray):
-    def __init__(self, space, dim, dtype):
-        self.dim = dim
-        self.space = space
-        # ignore dtype for now
-        size = 1
-        for el in dim:
-            size *= el
-        self.storage = [0] * size
-        make_sure_not_resized(self.storage)
-
-    def _unpack_indexes(self, space, w_index):
-        indexes = [space.int_w(w_i) for w_i in space.fixedview(w_index)]
-        if len(indexes) != len(self.dim):
-            raise OperationError(space.w_IndexError, space.wrap(
-                'Wrong index'))
-        return indexes
-
-    def descr_getitem(self, w_index):
-        space = self.space
-        indexes = self._unpack_indexes(space, w_index)
-        pos = compute_pos(space, indexes, self.dim)
-        return space.wrap(self.storage[pos])
-    descr_getitem.unwrap_spec = ['self', W_Root]
-
-    def descr_setitem(self, w_index, value):
-        space = self.space
-        indexes = self._unpack_indexes(space, w_index)
-        pos = compute_pos(space, indexes, self.dim)
-        self.storage[pos] = value
-        return space.w_None
-    descr_setitem.unwrap_spec = ['self', W_Root, int]
-
-    def descr_len(self):
-        return self.space.wrap(self.dim[0])
-    descr_len.unwrap_spec = ['self']
-
-MultiDimArray.typedef = TypeDef(
-    'NumArray',
-    __getitem__ = interp2app(MultiDimArray.descr_getitem),
-    __setitem__ = interp2app(MultiDimArray.descr_setitem),
-    __len__     = interp2app(MultiDimArray.descr_len),
-)
-
-def unpack_dim(space, w_dim):
-    if space.is_true(space.isinstance(w_dim, space.w_int)):
-        return [space.int_w(w_dim)]
-    dim_w = space.fixedview(w_dim)
-    return [space.int_w(w_i) for w_i in dim_w]
-
-def unpack_dtype(space, w_dtype):
-    if space.is_w(w_dtype, space.w_int):
-        return 'i'
-    else:
-        raise NotImplementedError
-
-def zeros(space, w_dim, w_dtype):
-    dim = unpack_dim(space, w_dim)
-    dtype = unpack_dtype(space, w_dtype)
-    if len(dim) == 1:
-        return space.wrap(NumArray(space, dim[0], dtype))
-    else:
-        return space.wrap(MultiDimArray(space, dim, dtype))
-zeros.unwrap_spec = [ObjSpace, W_Root, W_Root]

diff --git a/pypy/module/micronumpy/test/test_numpy.py b/pypy/module/micronumpy/test/test_numpy.py
--- a/pypy/module/micronumpy/test/test_numpy.py
+++ b/pypy/module/micronumpy/test/test_numpy.py
@@ -1,8 +1,26 @@
 
+import py
 from pypy.conftest import gettestobjspace
 
+class AppTestNumpyLike(object):
+    def setup_class(cls):
+        cls.space = gettestobjspace(usemodules=('micronumpy',))
+
+    def test_init(self):
+        from numpy import array
+        a = array(15)
+        assert a[10] == 0.0
+        a[13] = 5.3
+        assert a[13] == 5.3
+
+    def test_iterator_init(self):
+        from numpy import array
+        a = array(range(5))
+        assert a[3] == 3
+
 class AppTestNumpy(object):
     def setup_class(cls):
+        py.test.skip("skip")
         cls.space = gettestobjspace(usemodules=('micronumpy',))
     
     def test_zeroes(self):
@@ -42,6 +60,7 @@
 
 class AppTestMultiDim(object):
     def setup_class(cls):
+        py.test.skip("skip")
         cls.space = gettestobjspace(usemodules=('micronumpy',))
 
     def test_multidim(self):

From commits-noreply at bitbucket.org  Wed Feb  9 20:07:16 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Wed,  9 Feb 2011 20:07:16 +0100 (CET)
Subject: [pypy-svn] pypy numpy-exp: Progress on interpretation
Message-ID: <20110209190716.8282A282C1A@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: numpy-exp
Changeset: r41761:69e26bb0970e
Date: 2011-02-09 20:45 +0200
http://bitbucket.org/pypy/pypy/changeset/69e26bb0970e/

Log:	Progress on interpretation

diff --git a/pypy/module/micronumpy/numarray.py b/pypy/module/micronumpy/numarray.py
--- a/pypy/module/micronumpy/numarray.py
+++ b/pypy/module/micronumpy/numarray.py
@@ -7,19 +7,72 @@
 
 TP = lltype.GcArray(lltype.Float)
 
-class BaseBytecode(Wrappable):
-    pass
+def compute(bytecode, input):
+    result_size = input[0].size
+    result = SingleDimArray(result_size)
+    bytecode_pos = len(bytecode) - 1
+    input_pos = len(input) - 1
+    valuestack = [0.0] * len(input)
+    valuestackdepth = 0
+    i = 0
+    while i < result_size:
+        # merge point
+        if bytecode_pos == -1:
+            bytecode_pos = len(bytecode) - 1
+            input_pos = len(input) - 1
+            result.storage[i] = valuestack[0]
+            valuestack = [0.0] * len(input)
+            valuestackdepth = 0
+            i += 1
+            # can_enter_jit
+        else:
+            opcode = bytecode[bytecode_pos]
+            if opcode == 'l':
+                valuestack[valuestackdepth] = input[input_pos].storage[i]
+                valuestackdepth += 1
+                input_pos -= 1
+            elif opcode == 'a':
+                a = valuestack[valuestackdepth - 1]
+                b = valuestack[valuestackdepth - 2]
+                valuestack[valuestackdepth - 2] = a + b
+                valuestackdepth -= 1
+            else:
+                raise NotImplementedError
+            bytecode_pos -= 1
+    return result
+    
+class BaseArray(Wrappable):
+    def force(self):
+        bytecode, stack = self.compile()
+        return compute(bytecode, stack)
+    force.unwrap_spec = ['self']
 
-class Add(BaseBytecode):
+    def compile(self):
+        raise NotImplementedError("abstract base class")
+
+class Add(BaseArray):
     def __init__(self, left, right):
         self.left = left
         self.right = right
 
-class SingleDimArray(Wrappable):
+    def compile(self):
+        left_bc, left_stack = self.left.compile()
+        right_bc, right_stack = self.right.compile()
+        return 'a' + left_bc + right_bc, left_stack + right_stack
+
+BaseArray.typedef = TypeDef(
+    'Operation',
+    force=interp2app(BaseArray.force),
+)
+
+class SingleDimArray(BaseArray):
     def __init__(self, size):
         self.size = size
         self.storage = lltype.malloc(TP, size, zero=True)
 
+    def compile(self):
+        return "l", [self]
+
     def descr_getitem(self, space, item):
         if item < 0:
             raise operationerrfmt(space.w_TypeError,
@@ -44,6 +97,9 @@
         return space.wrap(Add(self, w_other))
     descr_add.unwrap_spec = ['self', ObjSpace, W_Root]
 
+    def force(self):
+        return self
+
 def descr_new_numarray(space, w_type, w_size_or_iterable):
     if space.isinstance_w(w_size_or_iterable, space.w_int):
         arr = SingleDimArray(space.int_w(w_size_or_iterable))
@@ -63,5 +119,6 @@
     __getitem__ = interp2app(SingleDimArray.descr_getitem),
     __setitem__ = interp2app(SingleDimArray.descr_setitem),
     __add__ = interp2app(SingleDimArray.descr_add),
+    force = interp2app(SingleDimArray.force),
 )
 

diff --git a/pypy/module/micronumpy/test/test_numpy.py b/pypy/module/micronumpy/test/test_numpy.py
--- a/pypy/module/micronumpy/test/test_numpy.py
+++ b/pypy/module/micronumpy/test/test_numpy.py
@@ -18,6 +18,13 @@
         a = array(range(5))
         assert a[3] == 3
 
+    def test_add(self):
+        from numpy import array
+        a = array(range(5))
+        b = a + a
+        b = b.force()
+        assert b[2] == 2 + 2
+
 class AppTestNumpy(object):
     def setup_class(cls):
         py.test.skip("skip")

From commits-noreply at bitbucket.org  Wed Feb  9 20:07:18 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Wed,  9 Feb 2011 20:07:18 +0100 (CET)
Subject: [pypy-svn] pypy numpy-exp: Add jit driver
Message-ID: <20110209190718.DA383282C1A@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: numpy-exp
Changeset: r41762:0a7324b724b5
Date: 2011-02-09 21:06 +0200
http://bitbucket.org/pypy/pypy/changeset/0a7324b724b5/

Log:	Add jit driver

diff --git a/pypy/jit/tl/pypyjit.py b/pypy/jit/tl/pypyjit.py
--- a/pypy/jit/tl/pypyjit.py
+++ b/pypy/jit/tl/pypyjit.py
@@ -41,6 +41,7 @@
 config.objspace.usemodules._sre = False
 #
 config.objspace.usemodules._ffi = True
+config.objspace.usemodules.micronumpy = True
 #
 set_pypy_opt_level(config, level='jit')
 

diff --git a/pypy/module/micronumpy/numarray.py b/pypy/module/micronumpy/numarray.py
--- a/pypy/module/micronumpy/numarray.py
+++ b/pypy/module/micronumpy/numarray.py
@@ -4,9 +4,15 @@
 from pypy.interpreter.typedef import TypeDef
 from pypy.interpreter.gateway import interp2app, NoneNotWrapped
 from pypy.rpython.lltypesystem import lltype
+from pypy.rlib import jit
 
 TP = lltype.GcArray(lltype.Float)
 
+numpy_driver = jit.JitDriver(greens = ['bytecode'],
+                             reds = ['result', 'result_size', 'i',
+                                     'valuestack', 'valuestackdepth',
+                                     'input', 'input_pos'])
+
 def compute(bytecode, input):
     result_size = input[0].size
     result = SingleDimArray(result_size)
@@ -16,7 +22,11 @@
     valuestackdepth = 0
     i = 0
     while i < result_size:
-        # merge point
+        numpy_driver.jit_merge_point(bytecode=bytecode, result=result,
+                                     result_size=result_size,
+                                     valuestackdepth=valuestackdepth,
+                                     valuestack=valuestack,
+                                     input=input, input_pos=input_pos, i=i)
         if bytecode_pos == -1:
             bytecode_pos = len(bytecode) - 1
             input_pos = len(input) - 1
@@ -24,7 +34,11 @@
             valuestack = [0.0] * len(input)
             valuestackdepth = 0
             i += 1
-            # can_enter_jit
+            numpy_driver.can_enter_jit(bytecode=bytecode, result=result,
+                                       result_size=result_size,
+                                       valuestackdepth=valuestackdepth,
+                                       valuestack=valuestack,
+                                       input=input, input_pos=input_pos, i=i)
         else:
             opcode = bytecode[bytecode_pos]
             if opcode == 'l':

diff --git a/pypy/rlib/jit.py b/pypy/rlib/jit.py
--- a/pypy/rlib/jit.py
+++ b/pypy/rlib/jit.py
@@ -271,7 +271,7 @@
 
 # ____________________________________________________________
 
-class JitDriver:    
+class JitDriver:
     """Base class to declare fine-grained user control on the JIT.  So
     far, there must be a singleton instance of JitDriver.  This style
     will allow us (later) to support a single RPython program with

From commits-noreply at bitbucket.org  Thu Feb 10 00:26:45 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Thu, 10 Feb 2011 00:26:45 +0100 (CET)
Subject: [pypy-svn] jitviewer default: Added an hgignore.
Message-ID: <20110209232645.9BA3236C226@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r84:cdae1d85c968
Date: 2011-02-09 18:26 -0500
http://bitbucket.org/pypy/jitviewer/changeset/cdae1d85c968/

Log:	Added an hgignore.

diff --git a/.hgignore b/.hgignore
new file mode 100644
--- /dev/null
+++ b/.hgignore
@@ -0,0 +1,3 @@
+syntax: glob
+
+*.pyc

From hpk at codespeak.net  Thu Feb 10 10:31:27 2011
From: hpk at codespeak.net (hpk at codespeak.net)
Date: Thu, 10 Feb 2011 10:31:27 +0100 (CET)
Subject: [pypy-svn] r80320 - pypy/extradoc/talk/ustour2011
Message-ID: <20110210093127.90ADD282C18@codespeak.net>

Author: hpk
Date: Thu Feb 10 10:31:24 2011
New Revision: 80320

Modified:
   pypy/extradoc/talk/ustour2011/blog1.txt
Log:
a draft posting for the San Franciso tour 
(or is there another city involved?)



Modified: pypy/extradoc/talk/ustour2011/blog1.txt
==============================================================================
--- pypy/extradoc/talk/ustour2011/blog1.txt	(original)
+++ pypy/extradoc/talk/ustour2011/blog1.txt	Thu Feb 10 10:31:24 2011
@@ -1,25 +1,43 @@
-Agenda:
+The PyPy San Franciso tour 2011
 
-Wednesday March 2, 4:15 p.m.  Armin gives talk at Stanford.
-http://www.stanford.edu/class/ee380/winter-schedule-20102011.html
-open to the public.
-
-Thursday March 3, 6:00 p.m.  General talk at Yelp, 706 Mission St 9th Floor,
-San Francisco CA 94103 
-open to the public.  See yelp-abstract.txt
-
-Saturday and Sunday March 5 and 6.
-PyPy mini sprint at noisebridge. 2169 mission street between 17th and 18th in 
-san francisco. http://www.noisebridge.net   Open to the public.
-See noisebridge.txt
-
-Monday March 7th.
-11:30 a.m.  Google Tech talk.  Not open to the public.  See
-google-abstract2.txt 
-
-2:30 p.m. Talk at Mozilla.  Not open to the public.  See mozilla-abstract.txt
-
---------
-We're open to meeting more people at other times.  We're especially interested
-in discussing PyPy consulting opportunities with interested companies.
-Insert plug here about what we would be willing to do.
+PyPy is coming to San Francisco beginning of March with a series of talks
+and sprints:
+
+* Wednesday March 2, 4:15 p.m.  Armin gives talk at Stanford.
+  http://www.stanford.edu/class/ee380/winter-schedule-20102011.html
+  open to the public.
+
+* Thursday March 3, 6:00 p.m.  General talk at Yelp, 706 Mission St 9th Floor,
+  San Francisco CA 94103 open to the public.
+
+* Saturday and Sunday March 5 and 6.
+  PyPy mini sprint at noisebridge. 2169 mission street between 17th and 18th in
+  san francisco. http://www.noisebridge.net   Open to the public.
+
+* Monday March 7th.  11:30 a.m.  Google Tech talk.  Not open to the public
+  (but probably to Googlers).
+
+* Monday March 7th.  2:30 p.m. Talk at Mozilla.  Not open to the public
+  (but probably to Mozilla devs).
+
+From the PyPy project team we will have Armin Rigo, Maciej Fijalkowski
+(from 6th March), Laura Creighton and Jacob Hallen at these events.
+Most of the talks will focus on (some of) the highlights and the
+status of pypy:
+
+* most Python benchmarks run much faster than with CPython or Psyco
+* the real-world PyPy compiler toolchain itself (200 KLocs) runs twice as fast
+* supports 64bit and is in the process of supporting ARM
+* full compatibility with CPython (more than Jython/IronPython)
+* full (and JIT-ed) ctypes support to call C libraries from Python
+* supports Stackless Python (in-progress)
+* new "cpyext" layer which integrates existing CPython C extensions
+* an experimental super-fast JIT-compilation of calls to C++ libraries
+
+Apart from getting more technical and project insight, our travel is
+a also a good possibility for companies in the SF area to talk to us
+regarding contracting.  In September 2011 our current "Eurostars" research
+project ends and some of us are looking for consulting/subcontracts or hiring
+related to PyPy.  We have two companies, Open End and merlinux, who
+have successfully done a number of contracts and projects in the past.
+If you want to talk business or get together for dinner, let us know!


From lac at codespeak.net  Thu Feb 10 11:03:56 2011
From: lac at codespeak.net (lac at codespeak.net)
Date: Thu, 10 Feb 2011 11:03:56 +0100 (CET)
Subject: [pypy-svn] r80321 - pypy/extradoc/talk/ustour2011
Message-ID: <20110210100356.DD100282C18@codespeak.net>

Author: lac
Date: Thu Feb 10 11:03:55 2011
New Revision: 80321

Modified:
   pypy/extradoc/talk/ustour2011/blog1.txt
Log:
no 'perhaps' about who can attend Google and Mozilla talks


Modified: pypy/extradoc/talk/ustour2011/blog1.txt
==============================================================================
--- pypy/extradoc/talk/ustour2011/blog1.txt	(original)
+++ pypy/extradoc/talk/ustour2011/blog1.txt	Thu Feb 10 11:03:55 2011
@@ -14,11 +14,12 @@
   PyPy mini sprint at noisebridge. 2169 mission street between 17th and 18th in
   san francisco. http://www.noisebridge.net   Open to the public.
 
-* Monday March 7th.  11:30 a.m.  Google Tech talk.  Not open to the public
-  (but probably to Googlers).
+* Monday March 7th.  11:30 a.m.  Google Tech talk in Mountain View at the
+  Googleplex.  Not open to the public (but the video should be available 
+  later).
 
-* Monday March 7th.  2:30 p.m. Talk at Mozilla.  Not open to the public
-  (but probably to Mozilla devs).
+* Monday March 7th.  2:30 p.m. Talk at Mozilla in Mountain View.  Not 
+  open to the public (but Mozilla develoeprs can videoconference).
 
 From the PyPy project team we will have Armin Rigo, Maciej Fijalkowski
 (from 6th March), Laura Creighton and Jacob Hallen at these events.


From lac at codespeak.net  Thu Feb 10 11:39:44 2011
From: lac at codespeak.net (lac at codespeak.net)
Date: Thu, 10 Feb 2011 11:39:44 +0100 (CET)
Subject: [pypy-svn] r80322 - pypy/extradoc/talk/ustour2011
Message-ID: <20110210103944.89A90282C18@codespeak.net>

Author: lac
Date: Thu Feb 10 11:39:42 2011
New Revision: 80322

Modified:
   pypy/extradoc/talk/ustour2011/blog1.txt
Log:
Add note that Christian might be attending.  Change the wording that could
be interpreted that we are looking to hire new PyPy developers.  Mention
that we are then heading for PyCon, and will have a sprint there as well.



Modified: pypy/extradoc/talk/ustour2011/blog1.txt
==============================================================================
--- pypy/extradoc/talk/ustour2011/blog1.txt	(original)
+++ pypy/extradoc/talk/ustour2011/blog1.txt	Thu Feb 10 11:39:42 2011
@@ -22,7 +22,9 @@
   open to the public (but Mozilla develoeprs can videoconference).
 
 From the PyPy project team we will have Armin Rigo, Maciej Fijalkowski
-(from 6th March), Laura Creighton and Jacob Hallen at these events.
+(from 6th March), Laura Creighton and Jacob Hall?n and possibly 
+Christian Tismer attending.
+
 Most of the talks will focus on (some of) the highlights and the
 status of pypy:
 
@@ -35,10 +37,34 @@
 * new "cpyext" layer which integrates existing CPython C extensions
 * an experimental super-fast JIT-compilation of calls to C++ libraries
 
+As is usual for us, there is vastly more material that is available for
+us to cover than time, especially when it comes to possible future 
+directions for PyPy.  We want to reserve a certain amount of time at
+each talk purely to discuss things that are of interest to audience 
+members.  However, if you already know what you wish we would discuss,
+and are attending a talk (or even if you aren't), please let us know.
+You can either reply to this blog post, 
+
 Apart from getting more technical and project insight, our travel is
 a also a good possibility for companies in the SF area to talk to us
 regarding contracting.  In September 2011 our current "Eurostars" research
-project ends and some of us are looking for consulting/subcontracts or hiring
-related to PyPy.  We have two companies, Open End and merlinux, who
-have successfully done a number of contracts and projects in the past.
-If you want to talk business or get together for dinner, let us know!
+project ends and some of us are looking for ways to continue working on
+PyPy through consulting, subcontracting or hiring.  The two companies, 
+Open End and merlinux, have successfully done a number of such contracts 
+and projects in the past.  If you want to talk business or get together for
+lunch or dinner, let us know! If you would like us to come to your company
+and make a presentation, let us know!  If you have any ideas about what
+we should discuss in a presentation so that you could use it to convince
+the powers-that-be at your place of employment that investing time and 
+money in PyPy would be a good idea, let us know! 
+
+On Tuesday March 8th we will be heading for Atlanta for the Python VM
+and Language Summits before attending PyCon.  Maciej Fijalkowski and
+Alex Gaynor will be giving a talk entitled _Why is Python slow and how
+can PyPy help?_ http://us.pycon.org/2011/schedule/presentations/160/
+Maciej will also be giving the talk _Running ultra large telescopes
+in Python_ http://us.pycon.org/2011/schedule/presentations/159/ which is
+partially about his experiences using PyPy in the  Square Kilometer Array
+project in South Africa.  There will be a PyPy Sprint March 14-17.  
+All are welcome.
+


From commits-noreply at bitbucket.org  Thu Feb 10 11:41:25 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Thu, 10 Feb 2011 11:41:25 +0100 (CET)
Subject: [pypy-svn] pypy default: sqlite: It seems better to exit the query
	earlier when	a callback cannot be run.
Message-ID: <20110210104125.934CD36C231@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41763:48d194e3ac07
Date: 2011-02-09 19:40 +0100
http://bitbucket.org/pypy/pypy/changeset/48d194e3ac07/

Log:	sqlite: It seems better to exit the query earlier when a callback
	cannot be run.

diff --git a/lib-python/modified-2.7.0/sqlite3/test/userfunctions.py b/lib-python/modified-2.7.0/sqlite3/test/userfunctions.py
--- a/lib-python/modified-2.7.0/sqlite3/test/userfunctions.py
+++ b/lib-python/modified-2.7.0/sqlite3/test/userfunctions.py
@@ -275,12 +275,14 @@
             pass
 
     def CheckAggrNoStep(self):
+        # XXX it's better to raise OperationalError in order to stop
+        # the query earlier.
         cur = self.con.cursor()
         try:
             cur.execute("select nostep(t) from test")
-            self.fail("should have raised an AttributeError")
-        except AttributeError, e:
-            self.assertEqual(e.args[0], "AggrNoStep instance has no attribute 'step'")
+            self.fail("should have raised an OperationalError")
+        except sqlite.OperationalError, e:
+            self.assertEqual(e.args[0], "user-defined aggregate's 'step' method raised error")
 
     def CheckAggrNoFinalize(self):
         cur = self.con.cursor()

diff --git a/lib_pypy/_sqlite3.py b/lib_pypy/_sqlite3.py
--- a/lib_pypy/_sqlite3.py
+++ b/lib_pypy/_sqlite3.py
@@ -578,9 +578,8 @@
                     aggregate = self.aggregate_instances[aggregate_ptr[0]]
 
                 params = _convert_params(context, argc, c_params)
-                step = aggregate.step
                 try:
-                    step(*params)
+                    aggregate.step(*params)
                 except Exception, e:
                     msg = ("user-defined aggregate's 'step' "
                            "method raised error")

From commits-noreply at bitbucket.org  Thu Feb 10 11:41:26 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Thu, 10 Feb 2011 11:41:26 +0100 (CET)
Subject: [pypy-svn] pypy default: sqlite: cut converter names at '('
	character
Message-ID: <20110210104126.2FDE436C231@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41764:4823c4f0dfcb
Date: 2011-02-09 19:48 +0100
http://bitbucket.org/pypy/pypy/changeset/4823c4f0dfcb/

Log:	sqlite: cut converter names at '(' character

diff --git a/lib_pypy/_sqlite3.py b/lib_pypy/_sqlite3.py
--- a/lib_pypy/_sqlite3.py
+++ b/lib_pypy/_sqlite3.py
@@ -834,6 +834,8 @@
                 decltype = sqlite.sqlite3_column_decltype(self.statement, i)
                 if decltype is not None:
                     decltype = decltype.split()[0]      # if multiple words, use first, eg. "INTEGER NOT NULL" => "INTEGER"
+                    if '(' in decltype:
+                        decltype = decltype[:decltype.index('(')]
                     converter = converters.get(decltype.upper(), None)
 
             self.row_cast_map.append(converter)

From commits-noreply at bitbucket.org  Thu Feb 10 11:41:26 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Thu, 10 Feb 2011 11:41:26 +0100 (CET)
Subject: [pypy-svn] pypy default: sqlite: Add iterdump()
Message-ID: <20110210104126.A288E36C231@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41765:46ef7685a777
Date: 2011-02-10 09:16 +0100
http://bitbucket.org/pypy/pypy/changeset/46ef7685a777/

Log:	sqlite: Add iterdump()

diff --git a/lib_pypy/_sqlite3.py b/lib_pypy/_sqlite3.py
--- a/lib_pypy/_sqlite3.py
+++ b/lib_pypy/_sqlite3.py
@@ -620,6 +620,10 @@
         if ret != SQLITE_OK:
             raise self._get_exception(ret)
 
+    def iterdump(self):
+        from sqlite3.dump import _iterdump
+        return _iterdump(self)
+
 class Cursor(object):
     def __init__(self, con):
         if not isinstance(con, Connection):

From commits-noreply at bitbucket.org  Thu Feb 10 11:41:30 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Thu, 10 Feb 2011 11:41:30 +0100 (CET)
Subject: [pypy-svn] pypy default: pyexpat.error constructor should accept
	only one	message; 
Message-ID: <20110210104130.1A2B82A2033@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41766:9a6df5a28474
Date: 2011-02-10 10:22 +0100
http://bitbucket.org/pypy/pypy/changeset/9a6df5a28474/

Log:	pyexpat.error constructor should accept only one message; other
	attributes are set after the object is built. This fixes one test in
	test_xml_etree

diff --git a/lib-python/TODO b/lib-python/TODO
--- a/lib-python/TODO
+++ b/lib-python/TODO
@@ -16,6 +16,8 @@
 - remove code duplication: bit_length() and _count_bits() in rlib/rbigint.py,
   objspace/std/longobject.py and objspace/std/longtype.py.
 
+- missing module pyexpat.errors
+
 
 Medium tasks
 ------------

diff --git a/pypy/module/pyexpat/app_pyexpat.py b/pypy/module/pyexpat/app_pyexpat.py
--- a/pypy/module/pyexpat/app_pyexpat.py
+++ b/pypy/module/pyexpat/app_pyexpat.py
@@ -1,6 +1,2 @@
 class ExpatError(Exception):
-    def __init__(self, msg, code, lineno, colno):
-        Exception.__init__(self, msg)
-        self.code = code
-        self.lineno = lineno
-        self.colno = colno
+    pass

diff --git a/pypy/module/pyexpat/interp_pyexpat.py b/pypy/module/pyexpat/interp_pyexpat.py
--- a/pypy/module/pyexpat/interp_pyexpat.py
+++ b/pypy/module/pyexpat/interp_pyexpat.py
@@ -583,10 +583,11 @@
         msg = "%s: line %d, column %d" % (err, lineno, colno)
         w_module = space.getbuiltinmodule('pyexpat')
         w_errorcls = space.getattr(w_module, space.wrap('error'))
-        w_error = space.call_function(
-            w_errorcls,
-            space.wrap(msg), space.wrap(code),
-            space.wrap(colno), space.wrap(lineno))
+        w_error = space.call_function(w_errorcls, space.wrap(msg))
+        space.setattr(w_error, space.wrap("code"), space.wrap(code))
+        space.setattr(w_error, space.wrap("offset"), space.wrap(colno))
+        space.setattr(w_error, space.wrap("lineno"), space.wrap(lineno))
+
         self.w_error = w_error
         return OperationError(w_errorcls, w_error)
 

diff --git a/pypy/module/pyexpat/test/test_parser.py b/pypy/module/pyexpat/test/test_parser.py
--- a/pypy/module/pyexpat/test/test_parser.py
+++ b/pypy/module/pyexpat/test/test_parser.py
@@ -14,7 +14,12 @@
         res = p.Parse("<xml></xml>")
         assert res == 1
 
-        raises(pyexpat.ExpatError, p.Parse, "3")
+        exc = raises(pyexpat.ExpatError, p.Parse, "3")
+        assert exc.value.lineno == 1
+        assert exc.value.offset == 11
+        assert exc.value.code == 9 # XML_ERROR_JUNK_AFTER_DOC_ELEMENT
+
+        pyexpat.ExpatError("error")
 
     def test_encoding(self):
         import pyexpat

From commits-noreply at bitbucket.org  Thu Feb 10 11:41:31 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Thu, 10 Feb 2011 11:41:31 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix tests on 64bit,
	where rffi.INT != lltype.Signed
Message-ID: <20110210104131.8B00E2A2033@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41767:371dc6f01951
Date: 2011-02-10 10:30 +0100
http://bitbucket.org/pypy/pypy/changeset/371dc6f01951/

Log:	Fix tests on 64bit, where rffi.INT != lltype.Signed

diff --git a/pypy/module/pyexpat/interp_pyexpat.py b/pypy/module/pyexpat/interp_pyexpat.py
--- a/pypy/module/pyexpat/interp_pyexpat.py
+++ b/pypy/module/pyexpat/interp_pyexpat.py
@@ -181,8 +181,8 @@
     if name in ['ExternalEntityRefHandler',
                 'NotStandaloneHandler']:
         result_type = rffi.INT
-        result_converter = "space.int_w(w_result)"
-        result_error = "0"
+        result_converter = "rffi.cast(rffi.INT, space.int_w(w_result))"
+        result_error = "rffi.cast(rffi.INT, 0)"
     else:
         result_type = lltype.Void
         result_converter = "None"

From commits-noreply at bitbucket.org  Thu Feb 10 11:41:33 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Thu, 10 Feb 2011 11:41:33 +0100 (CET)
Subject: [pypy-svn] pypy default: Add various version attributes to pyexpat
Message-ID: <20110210104133.3FD6136C231@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41768:6fd1dcfe3760
Date: 2011-02-10 10:50 +0100
http://bitbucket.org/pypy/pypy/changeset/6fd1dcfe3760/

Log:	Add various version attributes to pyexpat

diff --git a/pypy/module/pyexpat/__init__.py b/pypy/module/pyexpat/__init__.py
--- a/pypy/module/pyexpat/__init__.py
+++ b/pypy/module/pyexpat/__init__.py
@@ -14,6 +14,10 @@
         'ParserCreate':  'interp_pyexpat.ParserCreate',
         'XMLParserType': 'interp_pyexpat.W_XMLParserType',
         'ErrorString':   'interp_pyexpat.ErrorString',
+
+        '__version__':   'space.wrap("85819")',
+        'EXPAT_VERSION': 'interp_pyexpat.get_expat_version(space)',
+        'version_info':  'interp_pyexpat.get_expat_version_info(space)',
         }
 
     for name in ['XML_PARAM_ENTITY_PARSING_NEVER',

diff --git a/pypy/module/pyexpat/interp_pyexpat.py b/pypy/module/pyexpat/interp_pyexpat.py
--- a/pypy/module/pyexpat/interp_pyexpat.py
+++ b/pypy/module/pyexpat/interp_pyexpat.py
@@ -20,10 +20,6 @@
 eci = ExternalCompilationInfo(
     libraries=[libname],
     includes=['expat.h'],
-    pre_include_bits=[
-    '#define XML_COMBINED_VERSION' +
-    ' (10000*XML_MAJOR_VERSION+100*XML_MINOR_VERSION+XML_MICRO_VERSION)',
-    ],
     )
 
 eci = rffi_platform.configure_external_library(
@@ -54,13 +50,17 @@
                  'XML_PARAM_ENTITY_PARSING_UNLESS_STANDALONE',
                  'XML_PARAM_ENTITY_PARSING_ALWAYS']:
         locals()[name] = rffi_platform.ConstantInteger(name)
-    XML_COMBINED_VERSION = rffi_platform.ConstantInteger('XML_COMBINED_VERSION')
+    XML_MAJOR_VERSION = rffi_platform.ConstantInteger('XML_MAJOR_VERSION')
+    XML_MINOR_VERSION = rffi_platform.ConstantInteger('XML_MINOR_VERSION')
+    XML_MICRO_VERSION = rffi_platform.ConstantInteger('XML_MICRO_VERSION')
     XML_FALSE = rffi_platform.ConstantInteger('XML_FALSE')
     XML_TRUE = rffi_platform.ConstantInteger('XML_TRUE')
 
 for k, v in rffi_platform.configure(CConfigure).items():
     globals()[k] = v
 
+XML_COMBINED_VERSION = 10000*XML_MAJOR_VERSION+100*XML_MINOR_VERSION+XML_MICRO_VERSION
+
 XML_Content_Ptr.TO.become(rffi.CArray(XML_Content))
 XML_Encoding_Ptr = lltype.Ptr(XML_Encoding)
 
@@ -313,6 +313,18 @@
     'XML_ExternalEntityParserCreate', [XML_Parser, rffi.CCHARP, rffi.CCHARP],
     XML_Parser)
 
+XML_ExpatVersion = expat_external(
+    'XML_ExpatVersion', [], rffi.CCHARP)
+
+def get_expat_version(space):
+    return space.wrap(rffi.charp2str(XML_ExpatVersion()))
+
+def get_expat_version_info(space):
+    return space.newtuple([
+        space.wrap(XML_MAJOR_VERSION),
+        space.wrap(XML_MINOR_VERSION),
+        space.wrap(XML_MICRO_VERSION)])
+
 class W_XMLParserType(Wrappable):
 
     def __init__(self, space, parser, w_intern):

diff --git a/pypy/module/pyexpat/test/test_parser.py b/pypy/module/pyexpat/test/test_parser.py
--- a/pypy/module/pyexpat/test/test_parser.py
+++ b/pypy/module/pyexpat/test/test_parser.py
@@ -21,6 +21,13 @@
 
         pyexpat.ExpatError("error")
 
+    def test_version(self):
+        import pyexpat
+        assert isinstance(pyexpat.__version__, str)
+        assert pyexpat.EXPAT_VERSION.startswith('expat_')
+        assert isinstance(pyexpat.version_info, tuple)
+        assert isinstance(pyexpat.version_info[0], int)
+
     def test_encoding(self):
         import pyexpat
         for encoding_arg in (None, 'utf-8', 'iso-8859-1'):

From commits-noreply at bitbucket.org  Thu Feb 10 11:41:34 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Thu, 10 Feb 2011 11:41:34 +0100 (CET)
Subject: [pypy-svn] pypy default: Add a modifiable copy of test_xml_etree.py
Message-ID: <20110210104134.1D11636C231@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41769:7cb5acce08b1
Date: 2011-02-10 11:27 +0100
http://bitbucket.org/pypy/pypy/changeset/7cb5acce08b1/

Log:	Add a modifiable copy of test_xml_etree.py

diff --git a/lib-python/2.7.0/test/test_xml_etree.py b/lib-python/modified-2.7.0/test/test_xml_etree.py
copy from lib-python/2.7.0/test/test_xml_etree.py
copy to lib-python/modified-2.7.0/test/test_xml_etree.py

From commits-noreply at bitbucket.org  Thu Feb 10 11:41:34 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Thu, 10 Feb 2011 11:41:34 +0100 (CET)
Subject: [pypy-svn] pypy default: Skip two calls to sys.getrefcount
Message-ID: <20110210104134.B50D036C231@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41770:e934bba15959
Date: 2011-02-10 11:28 +0100
http://bitbucket.org/pypy/pypy/changeset/e934bba15959/

Log:	Skip two calls to sys.getrefcount

diff --git a/lib-python/modified-2.7.0/test/test_xml_etree.py b/lib-python/modified-2.7.0/test/test_xml_etree.py
--- a/lib-python/modified-2.7.0/test/test_xml_etree.py
+++ b/lib-python/modified-2.7.0/test/test_xml_etree.py
@@ -1628,10 +1628,10 @@
 
     Check reference leak.
     >>> xmltoolkit63()
-    >>> count = sys.getrefcount(None)
+    >>> count = sys.getrefcount(None)  #doctest: +SKIP
     >>> for i in range(1000):
     ...     xmltoolkit63()
-    >>> sys.getrefcount(None) - count
+    >>> sys.getrefcount(None) - count  #doctest: +SKIP
     0
 
     """

From commits-noreply at bitbucket.org  Thu Feb 10 11:41:35 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Thu, 10 Feb 2011 11:41:35 +0100 (CET)
Subject: [pypy-svn] pypy default: Merge heads
Message-ID: <20110210104135.594BA282C1E@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41771:4f27c419b8b1
Date: 2011-02-10 11:32 +0100
http://bitbucket.org/pypy/pypy/changeset/4f27c419b8b1/

Log:	Merge heads

diff --git a/lib_pypy/_sqlite3.py b/lib_pypy/_sqlite3.py
--- a/lib_pypy/_sqlite3.py
+++ b/lib_pypy/_sqlite3.py
@@ -578,9 +578,8 @@
                     aggregate = self.aggregate_instances[aggregate_ptr[0]]
 
                 params = _convert_params(context, argc, c_params)
-                step = aggregate.step
                 try:
-                    step(*params)
+                    aggregate.step(*params)
                 except Exception, e:
                     msg = ("user-defined aggregate's 'step' "
                            "method raised error")
@@ -839,6 +838,8 @@
                 decltype = sqlite.sqlite3_column_decltype(self.statement, i)
                 if decltype is not None:
                     decltype = decltype.split()[0]      # if multiple words, use first, eg. "INTEGER NOT NULL" => "INTEGER"
+                    if '(' in decltype:
+                        decltype = decltype[:decltype.index('(')]
                     converter = converters.get(decltype.upper(), None)
 
             self.row_cast_map.append(converter)

From lac at codespeak.net  Thu Feb 10 11:59:46 2011
From: lac at codespeak.net (lac at codespeak.net)
Date: Thu, 10 Feb 2011 11:59:46 +0100 (CET)
Subject: [pypy-svn] r80323 - pypy/extradoc/talk/ustour2011
Message-ID: <20110210105946.AE146282C18@codespeak.net>

Author: lac
Date: Thu Feb 10 11:59:45 2011
New Revision: 80323

Modified:
   pypy/extradoc/talk/ustour2011/blog1.txt
Log:
Ooops, I forgot to finish that line.


Modified: pypy/extradoc/talk/ustour2011/blog1.txt
==============================================================================
--- pypy/extradoc/talk/ustour2011/blog1.txt	(original)
+++ pypy/extradoc/talk/ustour2011/blog1.txt	Thu Feb 10 11:59:45 2011
@@ -43,7 +43,8 @@
 each talk purely to discuss things that are of interest to audience 
 members.  However, if you already know what you wish we would discuss,
 and are attending a talk (or even if you aren't), please let us know.
-You can either reply to this blog post, 
+You can either reply to this blog post, or mail Laura directly at
+lac at openend.se .
 
 Apart from getting more technical and project insight, our travel is
 a also a good possibility for companies in the SF area to talk to us


From arigo at codespeak.net  Thu Feb 10 12:04:41 2011
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Thu, 10 Feb 2011 12:04:41 +0100 (CET)
Subject: [pypy-svn] r80324 - pypy/extradoc/talk/ustour2011
Message-ID: <20110210110441.21EE4282C18@codespeak.net>

Author: arigo
Date: Thu Feb 10 12:04:39 2011
New Revision: 80324

Modified:
   pypy/extradoc/talk/ustour2011/blog1.txt
Log:
Typos.

Modified: pypy/extradoc/talk/ustour2011/blog1.txt
==============================================================================
--- pypy/extradoc/talk/ustour2011/blog1.txt	(original)
+++ pypy/extradoc/talk/ustour2011/blog1.txt	Thu Feb 10 12:04:39 2011
@@ -19,7 +19,7 @@
   later).
 
 * Monday March 7th.  2:30 p.m. Talk at Mozilla in Mountain View.  Not 
-  open to the public (but Mozilla develoeprs can videoconference).
+  open to the public (but Mozilla developers can videoconference).
 
 From the PyPy project team we will have Armin Rigo, Maciej Fijalkowski
 (from 6th March), Laura Creighton and Jacob Hall?n and possibly 
@@ -47,7 +47,7 @@
 lac at openend.se .
 
 Apart from getting more technical and project insight, our travel is
-a also a good possibility for companies in the SF area to talk to us
+also a good possibility for companies in the SF area to talk to us
 regarding contracting.  In September 2011 our current "Eurostars" research
 project ends and some of us are looking for ways to continue working on
 PyPy through consulting, subcontracting or hiring.  The two companies, 


From fijal at codespeak.net  Thu Feb 10 14:07:11 2011
From: fijal at codespeak.net (fijal at codespeak.net)
Date: Thu, 10 Feb 2011 14:07:11 +0100 (CET)
Subject: [pypy-svn] r80330 - pypy/extradoc/talk/ustour2011
Message-ID: <20110210130711.8F9E42A202E@codespeak.net>

Author: fijal
Date: Thu Feb 10 14:07:08 2011
New Revision: 80330

Modified:
   pypy/extradoc/talk/ustour2011/blog1.txt
Log:
Add "Rigo" to Armin and "??" in my name


Modified: pypy/extradoc/talk/ustour2011/blog1.txt
==============================================================================
--- pypy/extradoc/talk/ustour2011/blog1.txt	(original)
+++ pypy/extradoc/talk/ustour2011/blog1.txt	Thu Feb 10 14:07:08 2011
@@ -3,7 +3,7 @@
 PyPy is coming to San Francisco beginning of March with a series of talks
 and sprints:
 
-* Wednesday March 2, 4:15 p.m.  Armin gives talk at Stanford.
+* Wednesday March 2, 4:15 p.m.  Armin Rigo gives talk at Stanford.
   http://www.stanford.edu/class/ee380/winter-schedule-20102011.html
   open to the public.
 
@@ -21,8 +21,8 @@
 * Monday March 7th.  2:30 p.m. Talk at Mozilla in Mountain View.  Not 
   open to the public (but Mozilla developers can videoconference).
 
-From the PyPy project team we will have Armin Rigo, Maciej Fijalkowski
-(from 6th March), Laura Creighton and Jacob Hall?n and possibly 
+From the PyPy project team we will have Armin Rigo, Maciej Fija??kowski
+(from 6th March), Laura Creighton and Jacob Hall??n and possibly 
 Christian Tismer attending.
 
 Most of the talks will focus on (some of) the highlights and the


From fijal at codespeak.net  Thu Feb 10 14:09:04 2011
From: fijal at codespeak.net (fijal at codespeak.net)
Date: Thu, 10 Feb 2011 14:09:04 +0100 (CET)
Subject: [pypy-svn] r80331 - pypy/extradoc/talk/ustour2011
Message-ID: <20110210130904.717B6282C18@codespeak.net>

Author: fijal
Date: Thu Feb 10 14:09:02 2011
New Revision: 80331

Modified:
   pypy/extradoc/talk/ustour2011/blog1.txt
Log:
elaborate a bit on architectures


Modified: pypy/extradoc/talk/ustour2011/blog1.txt
==============================================================================
--- pypy/extradoc/talk/ustour2011/blog1.txt	(original)
+++ pypy/extradoc/talk/ustour2011/blog1.txt	Thu Feb 10 14:09:02 2011
@@ -30,7 +30,7 @@
 
 * most Python benchmarks run much faster than with CPython or Psyco
 * the real-world PyPy compiler toolchain itself (200 KLocs) runs twice as fast
-* supports 64bit and is in the process of supporting ARM
+* supports x86 32 and 64bit and is in the process of supporting ARM
 * full compatibility with CPython (more than Jython/IronPython)
 * full (and JIT-ed) ctypes support to call C libraries from Python
 * supports Stackless Python (in-progress)


From fijal at codespeak.net  Thu Feb 10 14:10:33 2011
From: fijal at codespeak.net (fijal at codespeak.net)
Date: Thu, 10 Feb 2011 14:10:33 +0100 (CET)
Subject: [pypy-svn] r80332 - pypy/extradoc/talk/ustour2011
Message-ID: <20110210131033.45220282C18@codespeak.net>

Author: fijal
Date: Thu Feb 10 14:10:31 2011
New Revision: 80332

Modified:
   pypy/extradoc/talk/ustour2011/blog1.txt
Log:
add an extra letter as well

Modified: pypy/extradoc/talk/ustour2011/blog1.txt
==============================================================================
--- pypy/extradoc/talk/ustour2011/blog1.txt	(original)
+++ pypy/extradoc/talk/ustour2011/blog1.txt	Thu Feb 10 14:10:31 2011
@@ -60,7 +60,7 @@
 money in PyPy would be a good idea, let us know! 
 
 On Tuesday March 8th we will be heading for Atlanta for the Python VM
-and Language Summits before attending PyCon.  Maciej Fijalkowski and
+and Language Summits before attending PyCon.  Maciej Fija?kowski and
 Alex Gaynor will be giving a talk entitled _Why is Python slow and how
 can PyPy help?_ http://us.pycon.org/2011/schedule/presentations/160/
 Maciej will also be giving the talk _Running ultra large telescopes


From commits-noreply at bitbucket.org  Thu Feb 10 14:43:14 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Thu, 10 Feb 2011 14:43:14 +0100 (CET)
Subject: [pypy-svn] pypy default: Add the module name to the _lsprof.Profile
	class
Message-ID: <20110210134314.EADE1282C18@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41772:40e1fca7c494
Date: 2011-02-10 13:12 +0100
http://bitbucket.org/pypy/pypy/changeset/40e1fca7c494/

Log:	Add the module name to the _lsprof.Profile class This may fix
	CPython's test_cprofile.py

diff --git a/pypy/module/_lsprof/interp_lsprof.py b/pypy/module/_lsprof/interp_lsprof.py
--- a/pypy/module/_lsprof/interp_lsprof.py
+++ b/pypy/module/_lsprof/interp_lsprof.py
@@ -309,6 +309,7 @@
 
 W_Profiler.typedef = TypeDef(
     'Profiler',
+    __module__ = '_lsprof',
     __new__ = interp2app(descr_new_profile),
     enable = interp2app(W_Profiler.enable),
     disable = interp2app(W_Profiler.disable),

diff --git a/pypy/module/_lsprof/test/test_cprofile.py b/pypy/module/_lsprof/test/test_cprofile.py
--- a/pypy/module/_lsprof/test/test_cprofile.py
+++ b/pypy/module/_lsprof/test/test_cprofile.py
@@ -10,6 +10,10 @@
         cls.space = space
         cls.w_file = space.wrap(__file__)
 
+    def test_repr(self):
+        import _lsprof
+        assert repr(_lsprof.Profiler) == "<type '_lsprof.Profiler'>"
+
     def test_direct(self):
         import _lsprof
         def getticks():

From commits-noreply at bitbucket.org  Thu Feb 10 14:43:15 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Thu, 10 Feb 2011 14:43:15 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix a long.__truediv__ corner case by
	using a version	of divmod() that does not depend
Message-ID: <20110210134315.B1D8D282C18@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41773:e2b10fbcbaeb
Date: 2011-02-10 13:45 +0100
http://bitbucket.org/pypy/pypy/changeset/e2b10fbcbaeb/

Log:	Fix a long.__truediv__ corner case by using a version of divmod()
	that does not depend on the sign of the arguments.

diff --git a/pypy/rlib/test/test_rbigint.py b/pypy/rlib/test/test_rbigint.py
--- a/pypy/rlib/test/test_rbigint.py
+++ b/pypy/rlib/test/test_rbigint.py
@@ -57,9 +57,25 @@
         op2 = rbigint.fromlong(1)
         f = op1.truediv(op2)
         assert f == 1.7976931348623157e+308     # exactly
+
+        op2 = rbigint.fromlong(-1)
+        f = op1.truediv(op2)
+        assert f == -1.7976931348623157e+308    # exactly
+
         op1 = rbigint.fromlong(overflowing)
+        op2 = rbigint.fromlong(1)
         py.test.raises(OverflowError, op1.truediv, op2)
 
+    def test_truediv_overflow2(self):
+        overflowing = 2**1024 - 2**(1024-53-1)
+        op1 = rbigint.fromlong(2*overflowing - 10)
+        op2 = rbigint.fromlong(2)
+        f = op1.truediv(op2)
+        assert f == 1.7976931348623157e+308    # exactly
+        op2 = rbigint.fromlong(-2)
+        f = op1.truediv(op2)
+        assert f == -1.7976931348623157e+308   # exactly
+
     def test_mod(self):
         for op1 in [-50, -12, -2, -1, 1, 2, 50, 52]:
             for op2 in [-4, -2, -1, 1, 2, 8]:

diff --git a/pypy/rlib/rbigint.py b/pypy/rlib/rbigint.py
--- a/pypy/rlib/rbigint.py
+++ b/pypy/rlib/rbigint.py
@@ -1503,7 +1503,7 @@
             inexact = True
 
     # x //= b. If the remainder is nonzero, set inexact.
-    x, rem = x.divmod(b)
+    x, rem = _divrem(x, b)
     if rem.tobool():
         inexact = True
 

From commits-noreply at bitbucket.org  Thu Feb 10 14:43:16 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Thu, 10 Feb 2011 14:43:16 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix test to run on top of CPython 2.7 with
	the -A option
Message-ID: <20110210134316.4F41B282C18@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41774:1afdac31f6f2
Date: 2011-02-10 13:56 +0100
http://bitbucket.org/pypy/pypy/changeset/1afdac31f6f2/

Log:	Fix test to run on top of CPython 2.7 with the -A option

diff --git a/pypy/objspace/std/test/test_unicodeobject.py b/pypy/objspace/std/test/test_unicodeobject.py
--- a/pypy/objspace/std/test/test_unicodeobject.py
+++ b/pypy/objspace/std/test/test_unicodeobject.py
@@ -471,7 +471,7 @@
         # surrogates not supported
         raises(UnicodeError, unicode, '+3ADYAA-', 'utf-7')
 
-        assert unicode('+3ADYAA-', 'utf-7', 'replace') == u'\ufffd'
+        assert unicode('+3ADYAA-', 'utf-7', 'replace') == u'\ufffd\ufffd'
 
     def test_codecs_utf8(self):
         assert u''.encode('utf-8') == ''

From commits-noreply at bitbucket.org  Thu Feb 10 14:43:19 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Thu, 10 Feb 2011 14:43:19 +0100 (CET)
Subject: [pypy-svn] pypy default: Test and fix in unicode_encode_utf_7
Message-ID: <20110210134319.05E4F282C26@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41775:8b5da4af26cf
Date: 2011-02-10 13:57 +0100
http://bitbucket.org/pypy/pypy/changeset/8b5da4af26cf/

Log:	Test and fix in unicode_encode_utf_7

diff --git a/pypy/rlib/test/test_runicode.py b/pypy/rlib/test/test_runicode.py
--- a/pypy/rlib/test/test_runicode.py
+++ b/pypy/rlib/test/test_runicode.py
@@ -280,6 +280,10 @@
         py.test.raises(UnicodeDecodeError, runicode.str_decode_utf_16_le,
                        s, len(s), True)
 
+    def test_utf7_bugs(self):
+        u = u'A\u2262\u0391.'
+        assert runicode.unicode_encode_utf_7(u, len(u), None) == 'A+ImIDkQ.'
+
     def test_utf7_partial(self):
         s = u"a+-b".encode('utf-7')
         assert s == "a+--b"

diff --git a/pypy/rlib/runicode.py b/pypy/rlib/runicode.py
--- a/pypy/rlib/runicode.py
+++ b/pypy/rlib/runicode.py
@@ -829,7 +829,7 @@
             if _utf7_ENCODE_DIRECT(oc, not encodeSetO, not encodeWhiteSpace):
                 # shifting out
                 if base64bits: # output remaining bits
-                    result.append(_utf7_TO_BASE64(base64buffer >> (base64bits-6)))
+                    result.append(_utf7_TO_BASE64(base64buffer << (6-base64bits)))
                     base64buffer = 0
                     base64bits = 0
 

From commits-noreply at bitbucket.org  Thu Feb 10 14:43:19 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Thu, 10 Feb 2011 14:43:19 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix remaining failures in test_runicode.py
Message-ID: <20110210134319.DE90F2A202E@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41776:022efa18b2a2
Date: 2011-02-10 14:28 +0100
http://bitbucket.org/pypy/pypy/changeset/022efa18b2a2/

Log:	Fix remaining failures in test_runicode.py

diff --git a/pypy/rlib/runicode.py b/pypy/rlib/runicode.py
--- a/pypy/rlib/runicode.py
+++ b/pypy/rlib/runicode.py
@@ -74,7 +74,7 @@
     0, 0, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, # C0-C1 + C2-CF
     2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, # D0-DF
     3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, # E0-EF
-    4, 4, 4, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0  # F0-F4 - F5-FF
+    4, 4, 4, 4, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0  # F0-F4 - F5-FF
 ]
 
 def str_decode_utf_8(s, size, errors, final=False,
@@ -710,6 +710,7 @@
                                 result.append(
                                     UNICHR((((surrogate & 0x3FF)<<10) |
                                             (outCh & 0x3FF)) + 0x10000))
+                            surrogate = 0
                         else:
                             surrogate = 0
                             msg = "second surrogate missing"

From commits-noreply at bitbucket.org  Thu Feb 10 14:43:20 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Thu, 10 Feb 2011 14:43:20 +0100 (CET)
Subject: [pypy-svn] pypy default: Add an unmodified copy of text_xpickle.py
	to	modified-2.7.0, 
Message-ID: <20110210134320.E4E52282C20@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41777:6bc38ff7c3de
Date: 2011-02-10 14:37 +0100
http://bitbucket.org/pypy/pypy/changeset/6bc38ff7c3de/

Log:	Add an unmodified copy of text_xpickle.py to modified-2.7.0, so that
	it picks our modified version or pickletester.py

diff --git a/lib-python/2.7.0/test/test_xpickle.py b/lib-python/modified-2.7.0/test/test_xpickle.py
copy from lib-python/2.7.0/test/test_xpickle.py
copy to lib-python/modified-2.7.0/test/test_xpickle.py

From commits-noreply at bitbucket.org  Thu Feb 10 14:48:47 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Thu, 10 Feb 2011 14:48:47 +0100 (CET)
Subject: [pypy-svn] pypy default: Skip this whole test file on top of
	non-CPython	implementations.
Message-ID: <20110210134847.D2618282C18@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41778:388cf91da597
Date: 2011-02-10 14:48 +0100
http://bitbucket.org/pypy/pypy/changeset/388cf91da597/

Log:	Skip this whole test file on top of non-CPython implementations.

diff --git a/lib-python/2.7.0/test/test_ascii_formatd.py b/lib-python/modified-2.7.0/test/test_ascii_formatd.py
copy from lib-python/2.7.0/test/test_ascii_formatd.py
copy to lib-python/modified-2.7.0/test/test_ascii_formatd.py
--- a/lib-python/2.7.0/test/test_ascii_formatd.py
+++ b/lib-python/modified-2.7.0/test/test_ascii_formatd.py
@@ -4,6 +4,10 @@
 
 import unittest
 from test.test_support import check_warnings, run_unittest, import_module
+from test.test_support import check_impl_detail
+
+if not check_impl_detail(cpython=True):
+    raise unittest.SkipTest("this test is only for CPython")
 
 # Skip tests if _ctypes module does not exist
 import_module('_ctypes')

From commits-noreply at bitbucket.org  Thu Feb 10 14:56:44 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Thu, 10 Feb 2011 14:56:44 +0100 (CET)
Subject: [pypy-svn] pypy default: Cannot test that sys.stdin.seek(-1) raises
	an exception	when we
Message-ID: <20110210135644.8527A282C18@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41779:dc2ff616e4e8
Date: 2011-02-10 14:56 +0100
http://bitbucket.org/pypy/pypy/changeset/dc2ff616e4e8/

Log:	Cannot test that sys.stdin.seek(-1) raises an exception when we are
	running with sys.stdin redirected (e.g. from /dev/null). This is not
	PyPy-specific: the same issue exists on CPython.

diff --git a/lib-python/modified-2.7.0/test/test_file.py b/lib-python/modified-2.7.0/test/test_file.py
--- a/lib-python/modified-2.7.0/test/test_file.py
+++ b/lib-python/modified-2.7.0/test/test_file.py
@@ -158,7 +158,12 @@
     def testStdin(self):
         # This causes the interpreter to exit on OSF1 v5.1.
         if sys.platform != 'osf1V5':
-            self.assertRaises((IOError, ValueError), sys.stdin.seek, -1)
+            if sys.stdin.isatty():
+                self.assertRaises((IOError, ValueError), sys.stdin.seek, -1)
+            else:
+                print((
+                    '  Skipping sys.stdin.seek(-1): stdin is not a tty.'
+                    ' Test manually.'), file=sys.__stdout__)
         else:
             print((
                 '  Skipping sys.stdin.seek(-1), it may crash the interpreter.'

From commits-noreply at bitbucket.org  Thu Feb 10 15:14:02 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Thu, 10 Feb 2011 15:14:02 +0100 (CET)
Subject: [pypy-svn] pypy default: One missing f.close().
Message-ID: <20110210141402.69F1F282C18@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41780:559784617931
Date: 2011-02-10 15:13 +0100
http://bitbucket.org/pypy/pypy/changeset/559784617931/

Log:	One missing f.close().

	Only call _create_file() once, not at the start of every thread.

diff --git a/lib-python/modified-2.7.0/test/test_file2k.py b/lib-python/modified-2.7.0/test/test_file2k.py
--- a/lib-python/modified-2.7.0/test/test_file2k.py
+++ b/lib-python/modified-2.7.0/test/test_file2k.py
@@ -389,6 +389,7 @@
             if lines != testlines:
                 self.fail("readlines() after next() with empty buffer "
                           "failed. Got %r, expected %r" % (line, testline))
+            f.close()
             # Reading after iteration hit EOF shouldn't hurt either
             f = open(TESTFN)
             try:
@@ -497,7 +498,6 @@
 
     def _test_close_open_io(self, io_func, nb_workers=5):
         def worker():
-            self._create_file()
             funcs = itertools.cycle((
                 lambda: io_func(),
                 lambda: self._close_and_reopen_file(),
@@ -509,6 +509,7 @@
                     f()
                 except (IOError, ValueError):
                     pass
+        self._create_file()
         self._run_workers(worker, nb_workers)
         if test_support.verbose:
             # Useful verbose statistics when tuning this test to take

From lac at codespeak.net  Thu Feb 10 15:41:13 2011
From: lac at codespeak.net (lac at codespeak.net)
Date: Thu, 10 Feb 2011 15:41:13 +0100 (CET)
Subject: [pypy-svn] r80334 - pypy/extradoc/talk/ustour2011
Message-ID: <20110210144113.63983282C18@codespeak.net>

Author: lac
Date: Thu Feb 10 15:41:11 2011
New Revision: 80334

Modified:
   pypy/extradoc/talk/ustour2011/blog1.txt
Log:
Fix capitalisation, and change San Francisco to the SF Bay Area, because that
matters to those in the Bay Area.


Modified: pypy/extradoc/talk/ustour2011/blog1.txt
==============================================================================
--- pypy/extradoc/talk/ustour2011/blog1.txt	(original)
+++ pypy/extradoc/talk/ustour2011/blog1.txt	Thu Feb 10 15:41:11 2011
@@ -1,7 +1,7 @@
-The PyPy San Franciso tour 2011
+The PyPy San Franciso Bay Area Tour 2011
 
-PyPy is coming to San Francisco beginning of March with a series of talks
-and sprints:
+PyPy is coming to the San Francisco Bay Area in the beginning of March with 
+a series of talks and a mini sprint.
 
 * Wednesday March 2, 4:15 p.m.  Armin Rigo gives talk at Stanford.
   http://www.stanford.edu/class/ee380/winter-schedule-20102011.html
@@ -11,8 +11,8 @@
   San Francisco CA 94103 open to the public.
 
 * Saturday and Sunday March 5 and 6.
-  PyPy mini sprint at noisebridge. 2169 mission street between 17th and 18th in
-  san francisco. http://www.noisebridge.net   Open to the public.
+  PyPy mini sprint at noisebridge. 2169 Mission street between 17th and 18th in
+  San Francisco. http://www.noisebridge.net   Open to the public.
 
 * Monday March 7th.  11:30 a.m.  Google Tech talk in Mountain View at the
   Googleplex.  Not open to the public (but the video should be available 


From commits-noreply at bitbucket.org  Thu Feb 10 16:15:50 2011
From: commits-noreply at bitbucket.org (lac)
Date: Thu, 10 Feb 2011 16:15:50 +0100 (CET)
Subject: [pypy-svn] pypy default: merge heads
Message-ID: <20110210151550.AD88D282C25@codespeak.net>

Author: Laura Creighton <lac at openend.se>
Branch: 
Changeset: r41782:eb4ec044dd18
Date: 2011-02-10 16:14 +0100
http://bitbucket.org/pypy/pypy/changeset/eb4ec044dd18/

Log:	merge heads


From commits-noreply at bitbucket.org  Thu Feb 10 16:15:49 2011
From: commits-noreply at bitbucket.org (lac)
Date: Thu, 10 Feb 2011 16:15:49 +0100 (CET)
Subject: [pypy-svn] pypy default: Add section on using virtualenv.
Message-ID: <20110210151549.57D8A282C18@codespeak.net>

Author: Laura Creighton <lac at openend.se>
Branch: 
Changeset: r41781:5c4aa3c1d45f
Date: 2011-02-10 16:14 +0100
http://bitbucket.org/pypy/pypy/changeset/5c4aa3c1d45f/

Log:	Add section on using virtualenv.

diff --git a/pypy/doc/getting-started.txt b/pypy/doc/getting-started.txt
--- a/pypy/doc/getting-started.txt
+++ b/pypy/doc/getting-started.txt
@@ -68,6 +68,23 @@
 If you want to commit to our repository on bitbucket, you will have to
 install subversion in addition to mercurial.
 
+Installing using virtualenv
+---------------------------
+
+It is often convenient to run pypy inside a virtualenv.  To do this
+you need a recent version of virtualenv -- 1.5 or greater.  You can
+then install PyPy both from a precompiled tarball or from a mercurial
+checkout::
+
+	# from a tarball
+	$ virtualenv -p /opt/pypy-c-jit-41718-3fb486695f20-linux/bin/pypy my-pypy-env
+
+	# from the mercurial checkout
+	$ virtualenv -p /path/to/pypy/pypy/translator/goal/pypy-c my-pypy-env
+
+Note that bin/python is now a symlink to bin/pypy.
+
+
 Where to go from here
 ----------------------
 

From commits-noreply at bitbucket.org  Thu Feb 10 16:48:05 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Thu, 10 Feb 2011 16:48:05 +0100 (CET)
Subject: [pypy-svn] pypy default: Minor test and fix to be more
	2.7-compliant.
Message-ID: <20110210154805.896B5282C18@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41783:0401c84c6fd8
Date: 2011-02-10 15:42 +0100
http://bitbucket.org/pypy/pypy/changeset/0401c84c6fd8/

Log:	Minor test and fix to be more 2.7-compliant.

diff --git a/pypy/module/mmap/test/test_mmap.py b/pypy/module/mmap/test/test_mmap.py
--- a/pypy/module/mmap/test/test_mmap.py
+++ b/pypy/module/mmap/test/test_mmap.py
@@ -44,7 +44,7 @@
         raises(TypeError, mmap, 0, "foo")
              
         if os.name == "posix":
-            raises(TypeError, mmap, 0, 1, 2, 3, 4, 5)
+            raises(ValueError, mmap, 0, 1, 2, 3, 4)
             raises(TypeError, mmap, 0, 1, 2, 3, "foo", 5)
             raises(TypeError, mmap, 0, 1, foo="foo")
             raises((TypeError, OverflowError), mmap, 0, -1)

diff --git a/pypy/rlib/rmmap.py b/pypy/rlib/rmmap.py
--- a/pypy/rlib/rmmap.py
+++ b/pypy/rlib/rmmap.py
@@ -592,15 +592,15 @@
 
         fd = fileno
 
-        # check size boundaries
-        _check_map_size(length)
-        map_size = length
-
         # check access is not there when flags and prot are there
         if access != _ACCESS_DEFAULT and ((flags != MAP_SHARED) or\
                                           (prot != (PROT_WRITE | PROT_READ))):
             raise RValueError("mmap can't specify both access and flags, prot.")
 
+        # check size boundaries
+        _check_map_size(length)
+        map_size = length
+
         if access == ACCESS_READ:
             flags = MAP_SHARED
             prot = PROT_READ

From commits-noreply at bitbucket.org  Thu Feb 10 16:48:06 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Thu, 10 Feb 2011 16:48:06 +0100 (CET)
Subject: [pypy-svn] pypy default: Quick fix: the value returned in case of
	error is	actually 0...
Message-ID: <20110210154806.7285D282C18@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41784:cfcac542508c
Date: 2011-02-10 15:52 +0100
http://bitbucket.org/pypy/pypy/changeset/cfcac542508c/

Log:	Quick fix: the value returned in case of error is actually 0...

diff --git a/pypy/module/thread/ll_thread.py b/pypy/module/thread/ll_thread.py
--- a/pypy/module/thread/ll_thread.py
+++ b/pypy/module/thread/ll_thread.py
@@ -133,7 +133,7 @@
     # lock objects, as well as from the GIL, which exists at shutdown.
     ll_lock = lltype.malloc(TLOCKP.TO, flavor='raw', track_allocation=False)
     res = c_thread_lock_init(ll_lock)
-    if res == -1:
+    if res <= 0:
         lltype.free(ll_lock, flavor='raw', track_allocation=False)
         raise error("out of resources")
     return ll_lock

From commits-noreply at bitbucket.org  Thu Feb 10 16:48:09 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Thu, 10 Feb 2011 16:48:09 +0100 (CET)
Subject: [pypy-svn] pypy default: Multiple tests and fixes for mmap.find().
Message-ID: <20110210154809.88E93282C26@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41785:be673fcbd2e4
Date: 2011-02-10 16:09 +0100
http://bitbucket.org/pypy/pypy/changeset/be673fcbd2e4/

Log:	Multiple tests and fixes for mmap.find().

diff --git a/pypy/module/mmap/interp_mmap.py b/pypy/module/mmap/interp_mmap.py
--- a/pypy/module/mmap/interp_mmap.py
+++ b/pypy/module/mmap/interp_mmap.py
@@ -3,7 +3,7 @@
 from pypy.interpreter.error import OperationError, wrap_oserror
 from pypy.interpreter.baseobjspace import W_Root, ObjSpace, Wrappable
 from pypy.interpreter.typedef import TypeDef
-from pypy.interpreter.gateway import interp2app
+from pypy.interpreter.gateway import interp2app, NoneNotWrapped
 from pypy.rlib import rmmap
 from pypy.rlib.rmmap import RValueError, RTypeError
 import sys
@@ -37,9 +37,18 @@
         return self.space.wrap(self.mmap.read(num))
     read.unwrap_spec = ['self', int]
 
-    def find(self, tofind, start=0):
-        return self.space.wrap(self.mmap.find(tofind, start))
-    find.unwrap_spec = ['self', 'bufferstr', int]
+    def find(self, tofind, w_start=NoneNotWrapped, w_end=NoneNotWrapped):
+        space = self.space
+        if w_start is None:
+            start = self.mmap.pos
+        else:
+            start = space.getindex_w(w_start, None)
+        if w_end is None:
+            end = self.mmap.size
+        else:
+            end = space.getindex_w(w_end, None)
+        return space.wrap(self.mmap.find(tofind, start, end))
+    find.unwrap_spec = ['self', 'bufferstr', W_Root, W_Root]
 
     def seek(self, pos, whence=0):
         try:

diff --git a/pypy/rlib/test/test_rmmap.py b/pypy/rlib/test/test_rmmap.py
--- a/pypy/rlib/test/test_rmmap.py
+++ b/pypy/rlib/test/test_rmmap.py
@@ -140,18 +140,37 @@
     def test_find(self):
         f = open(self.tmpname + "g", "w+")
 
-        f.write("foobar\0")
+        f.write("foobarfoobar\0")
         f.flush()
 
+        # prebuilt a table of expected search results
+        expected = {}
+        for s1 in range(-20, 20):
+            for e1 in range(-20, 20):
+                res = "foobarfoobar\0".find("ob", s1, e1)
+                expected.setdefault(s1, {})[e1] = res
+
         def func(no):
-            m = mmap.mmap(no, 7)
-            assert m.find("b") == 3
-            assert m.find("z") == -1
-            assert m.find("o", 5) == -1
-            assert m.find("ob") == 2
-            assert m.find("\0") == 6
+            m = mmap.mmap(no, 12)
+            assert m.find("\0", 0, 13) == -1    # no searching past the stop
+            m.close()
+            #
+            m = mmap.mmap(no, 13)
+            assert m.find("b", 0, 7) == 3
+            assert m.find("z", 0, 7) == -1
+            assert m.find("o", 11, 13) == -1
+            assert m.find("ob", 0, 7) == 2
+            assert m.find("\0", 0, 13) == 12
+            assert m.find("o", 1, 4) == 1
+            assert m.find("o", 2, 4) == 2
+            assert m.find("o", 2, -4) == 2
+            assert m.find("o", 8, -5) == -1
+            for s1 in range(-20, 20):
+                for e1 in range(-20, 20):
+                    assert m.find("ob", s1, e1) == expected[s1][e1]
             m.close()
 
+        func(f.fileno())
         interpret(func, [f.fileno()])
         f.close()
 

diff --git a/pypy/module/mmap/test/test_mmap.py b/pypy/module/mmap/test/test_mmap.py
--- a/pypy/module/mmap/test/test_mmap.py
+++ b/pypy/module/mmap/test/test_mmap.py
@@ -155,6 +155,25 @@
         assert m.find("o", 5) == -1
         assert m.find("ob") == 2
         assert m.find("\0") == 6
+        assert m.find("ob", 1) == 2
+        assert m.find("ob", 2) == 2
+        assert m.find("ob", 3) == -1
+        assert m.find("ob", -4) == -1
+        assert m.find("ob", -5) == 2
+        assert m.find("ob", -999999999) == 2
+        assert m.find("ob", 1, 3) == -1
+        assert m.find("ob", 1, 4) == 2
+        assert m.find("ob", 1, 999999999) == 2
+        assert m.find("ob", 1, 0) == -1
+        assert m.find("ob", 1, -1) == 2
+        assert m.find("ob", 1, -3) == 2
+        assert m.find("ob", 1, -4) == -1
+        #
+        data = m.read(2)
+        assert data == "fo"
+        assert m.find("o") == 2
+        assert m.find("oo") == -1
+        assert m.find("o", 0) == 1
         m.close()
         f.close()
 

diff --git a/pypy/rlib/rmmap.py b/pypy/rlib/rmmap.py
--- a/pypy/rlib/rmmap.py
+++ b/pypy/rlib/rmmap.py
@@ -361,7 +361,7 @@
         self.pos += len(res)
         return res
 
-    def find(self, tofind, start=0):
+    def find(self, tofind, start, end):
         self.check_valid()
 
         # XXX naive! how can we reuse the rstr algorithm?
@@ -369,8 +369,14 @@
             start += self.size
             if start < 0:
                 start = 0
+        if end < 0:
+            end += self.size
+            if end < 0:
+                end = 0
+        elif end > self.size:
+            end = self.size
         data = self.data
-        for p in xrange(start, self.size - len(tofind) + 1):
+        for p in xrange(start, end - len(tofind) + 1):
             for q in range(len(tofind)):
                 if data[p+q] != tofind[q]:
                     break     # position 'p' is not a match

From commits-noreply at bitbucket.org  Thu Feb 10 16:48:11 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Thu, 10 Feb 2011 16:48:11 +0100 (CET)
Subject: [pypy-svn] pypy default: Support in rmmap find(reverse=True).
Message-ID: <20110210154811.3480C282C27@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41786:8f6d23bc36ce
Date: 2011-02-10 16:43 +0100
http://bitbucket.org/pypy/pypy/changeset/8f6d23bc36ce/

Log:	Support in rmmap find(reverse=True).

diff --git a/pypy/rlib/test/test_rmmap.py b/pypy/rlib/test/test_rmmap.py
--- a/pypy/rlib/test/test_rmmap.py
+++ b/pypy/rlib/test/test_rmmap.py
@@ -137,22 +137,31 @@
         interpret(func, [f.fileno()])
         f.close()
 
+    def test_find_rfind(self):
+        f = open(self.tmpname + "g", "w+")
+        f.write("foobarfoobar\0")
+        f.flush()
+        m = mmap.mmap(f.fileno(), 13)
+
+        for s1 in range(-20, 20):
+            for e1 in range(-20, 20):
+                expected = "foobarfoobar\0".find("ob", s1, e1)
+                assert m.find("ob", s1, e1, False) == expected
+                expected = "foobarfoobar\0".rfind("ob", s1, e1)
+                assert m.find("ob", s1, e1, True) == expected
+
+        m.close()
+        f.close()
+
     def test_find(self):
         f = open(self.tmpname + "g", "w+")
-
         f.write("foobarfoobar\0")
         f.flush()
 
-        # prebuilt a table of expected search results
-        expected = {}
-        for s1 in range(-20, 20):
-            for e1 in range(-20, 20):
-                res = "foobarfoobar\0".find("ob", s1, e1)
-                expected.setdefault(s1, {})[e1] = res
-
         def func(no):
             m = mmap.mmap(no, 12)
             assert m.find("\0", 0, 13) == -1    # no searching past the stop
+            assert m.find("\0", 0, 13, True) == -1
             m.close()
             #
             m = mmap.mmap(no, 13)
@@ -165,9 +174,6 @@
             assert m.find("o", 2, 4) == 2
             assert m.find("o", 2, -4) == 2
             assert m.find("o", 8, -5) == -1
-            for s1 in range(-20, 20):
-                for e1 in range(-20, 20):
-                    assert m.find("ob", s1, e1) == expected[s1][e1]
             m.close()
 
         func(f.fileno())

diff --git a/pypy/rlib/rmmap.py b/pypy/rlib/rmmap.py
--- a/pypy/rlib/rmmap.py
+++ b/pypy/rlib/rmmap.py
@@ -361,7 +361,7 @@
         self.pos += len(res)
         return res
 
-    def find(self, tofind, start, end):
+    def find(self, tofind, start, end, reverse=False):
         self.check_valid()
 
         # XXX naive! how can we reuse the rstr algorithm?
@@ -375,16 +375,33 @@
                 end = 0
         elif end > self.size:
             end = self.size
+        #
+        upto = end - len(tofind)
+        if not reverse:
+            step = 1
+            p = start
+            if p > upto:
+                return -1      # failure (empty range to search)
+        else:
+            step = -1
+            p = upto
+            upto = start
+            if p < upto:
+                return -1      # failure (empty range to search)
+        #
         data = self.data
-        for p in xrange(start, end - len(tofind) + 1):
+        while True:
+            assert p >= 0
             for q in range(len(tofind)):
                 if data[p+q] != tofind[q]:
                     break     # position 'p' is not a match
             else:
                 # full match
                 return p
-        # failure
-        return -1
+            #
+            if p == upto:
+                return -1   # failure
+            p += step
 
     def seek(self, pos, whence=0):
         self.check_valid()

From commits-noreply at bitbucket.org  Thu Feb 10 16:48:13 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Thu, 10 Feb 2011 16:48:13 +0100 (CET)
Subject: [pypy-svn] pypy default: mmap.rfind().
Message-ID: <20110210154813.70E9F282C28@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41787:e1961ad34d06
Date: 2011-02-10 16:47 +0100
http://bitbucket.org/pypy/pypy/changeset/e1961ad34d06/

Log:	mmap.rfind().

diff --git a/pypy/module/mmap/interp_mmap.py b/pypy/module/mmap/interp_mmap.py
--- a/pypy/module/mmap/interp_mmap.py
+++ b/pypy/module/mmap/interp_mmap.py
@@ -50,6 +50,19 @@
         return space.wrap(self.mmap.find(tofind, start, end))
     find.unwrap_spec = ['self', 'bufferstr', W_Root, W_Root]
 
+    def rfind(self, tofind, w_start=NoneNotWrapped, w_end=NoneNotWrapped):
+        space = self.space
+        if w_start is None:
+            start = self.mmap.pos
+        else:
+            start = space.getindex_w(w_start, None)
+        if w_end is None:
+            end = self.mmap.size
+        else:
+            end = space.getindex_w(w_end, None)
+        return space.wrap(self.mmap.find(tofind, start, end, True))
+    rfind.unwrap_spec = ['self', 'bufferstr', W_Root, W_Root]
+
     def seek(self, pos, whence=0):
         try:
             self.mmap.seek(pos, whence)
@@ -232,6 +245,7 @@
     readline = interp2app(W_MMap.readline),
     read = interp2app(W_MMap.read),
     find = interp2app(W_MMap.find),
+    rfind = interp2app(W_MMap.rfind),
     seek = interp2app(W_MMap.seek),
     tell = interp2app(W_MMap.tell),
     size = interp2app(W_MMap.descr_size),

diff --git a/pypy/module/mmap/test/test_mmap.py b/pypy/module/mmap/test/test_mmap.py
--- a/pypy/module/mmap/test/test_mmap.py
+++ b/pypy/module/mmap/test/test_mmap.py
@@ -177,6 +177,42 @@
         m.close()
         f.close()
 
+    def test_rfind(self):
+        from mmap import mmap
+        f = open(self.tmpname + "g", "w+")
+
+        f.write("foobarfoobar\0")
+        f.flush()
+        m = mmap(f.fileno(), 13)
+        raises(TypeError, m.rfind, 123)
+        raises(TypeError, m.rfind, "foo", "baz")
+        assert m.rfind("b") == 9
+        assert m.rfind("z") == -1
+        assert m.rfind("o", 11) == -1
+        assert m.rfind("ob") == 8
+        assert m.rfind("\0") == 12
+        assert m.rfind("ob", 7) == 8
+        assert m.rfind("ob", 8) == 8
+        assert m.rfind("ob", 9) == -1
+        assert m.rfind("ob", -4) == -1
+        assert m.rfind("ob", -5) == 8
+        assert m.rfind("ob", -999999999) == 8
+        assert m.rfind("ob", 1, 3) == -1
+        assert m.rfind("ob", 1, 4) == 2
+        assert m.rfind("ob", 1, 999999999) == 8
+        assert m.rfind("ob", 1, 0) == -1
+        assert m.rfind("ob", 1, -1) == 8
+        assert m.rfind("ob", 1, -3) == 8
+        assert m.rfind("ob", 1, -4) == 2
+        #
+        data = m.read(8)
+        assert data == "foobarfo"
+        assert m.rfind("o") == 8
+        assert m.rfind("oo") == -1
+        assert m.rfind("o", 0) == 8
+        m.close()
+        f.close()
+
     def test_is_modifiable(self):
         import mmap
         f = open(self.tmpname + "h", "w+")

From commits-noreply at bitbucket.org  Thu Feb 10 17:03:10 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Thu, 10 Feb 2011 17:03:10 +0100 (CET)
Subject: [pypy-svn] pypy default: Give the offset keyword argument to mmap().
Message-ID: <20110210160310.5C88B282C18@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41788:943e58cee14c
Date: 2011-02-10 17:02 +0100
http://bitbucket.org/pypy/pypy/changeset/943e58cee14c/

Log:	Give the offset keyword argument to mmap().

diff --git a/pypy/module/mmap/interp_mmap.py b/pypy/module/mmap/interp_mmap.py
--- a/pypy/module/mmap/interp_mmap.py
+++ b/pypy/module/mmap/interp_mmap.py
@@ -5,7 +5,7 @@
 from pypy.interpreter.typedef import TypeDef
 from pypy.interpreter.gateway import interp2app, NoneNotWrapped
 from pypy.rlib import rmmap
-from pypy.rlib.rmmap import RValueError, RTypeError
+from pypy.rlib.rmmap import RValueError, RTypeError, ROverflowError
 import sys
 import os
 import platform
@@ -207,36 +207,42 @@
 
     def mmap(space, w_subtype, fileno, length, flags=rmmap.MAP_SHARED,
              prot=rmmap.PROT_WRITE | rmmap.PROT_READ,
-             access=rmmap._ACCESS_DEFAULT):
+             access=rmmap._ACCESS_DEFAULT, offset=0):
         self = space.allocate_instance(W_MMap, w_subtype)
         try:
             W_MMap.__init__(self, space,
-                            rmmap.mmap(fileno, length, flags, prot, access))
+                            rmmap.mmap(fileno, length, flags, prot, access,
+                                       offset))
         except OSError, e:
             raise mmap_error(space, e)
         except RValueError, e:
             raise OperationError(space.w_ValueError, space.wrap(e.message))
         except RTypeError, e:
             raise OperationError(space.w_TypeError, space.wrap(e.message))
+        except ROverflowError, e:
+            raise OperationError(space.w_OverflowError, space.wrap(e.message))
         return space.wrap(self)
-    mmap.unwrap_spec = [ObjSpace, W_Root, int, 'index', int, int, int]
+    mmap.unwrap_spec = [ObjSpace, W_Root, int, 'index', int, int, int, 'index']
 
 elif rmmap._MS_WINDOWS:
 
     def mmap(space, w_subtype, fileno, length, tagname="",
-             access=rmmap._ACCESS_DEFAULT):
+             access=rmmap._ACCESS_DEFAULT, offset=0):
         self = space.allocate_instance(W_MMap, w_subtype)
         try:
             W_MMap.__init__(self, space,
-                            rmmap.mmap(fileno, length, tagname, access))
+                            rmmap.mmap(fileno, length, tagname, access,
+                                       offset))
         except OSError, e:
             raise mmap_error(space, e)
         except RValueError, e:
             raise OperationError(space.w_ValueError, space.wrap(e.message))
         except RTypeError, e:
             raise OperationError(space.w_TypeError, space.wrap(e.message))
+        except ROverflowError, e:
+            raise OperationError(space.w_OverflowError, space.wrap(e.message))
         return space.wrap(self)
-    mmap.unwrap_spec = [ObjSpace, W_Root, int, 'index', str, int]
+    mmap.unwrap_spec = [ObjSpace, W_Root, int, 'index', str, int, 'index']
 
 W_MMap.typedef = TypeDef("mmap",
     __new__ = interp2app(mmap),

diff --git a/pypy/module/mmap/test/test_mmap.py b/pypy/module/mmap/test/test_mmap.py
--- a/pypy/module/mmap/test/test_mmap.py
+++ b/pypy/module/mmap/test/test_mmap.py
@@ -518,6 +518,17 @@
         assert b[3] == "b"
         assert b[:] == "foobar"
 
+    def test_offset(self):
+        from mmap import mmap
+        f = open(self.tmpname + "y", "w+")
+        f.write("foobar" * 3000)
+        f.flush()
+        m = mmap(f.fileno(), 4, offset=8192)
+        assert m[:] == "obar"
+        assert len(m) == 4
+        m.close()
+        f.close()
+
     def test_all(self):
         # this is a global test, ported from test_mmap.py
         import mmap

diff --git a/pypy/rlib/rmmap.py b/pypy/rlib/rmmap.py
--- a/pypy/rlib/rmmap.py
+++ b/pypy/rlib/rmmap.py
@@ -21,7 +21,11 @@
 
 class RTypeError(Exception):
     def __init__(self, message):
-        self.message = message    
+        self.message = message
+
+class ROverflowError(Exception):
+    def __init__(self, message):
+        self.message = message
 
 includes = ["sys/types.h"]
 if _POSIX:
@@ -607,11 +611,11 @@
     if size < 0:
         raise RTypeError("memory mapped size must be positive")
     if rffi.cast(size_t, size) != size:
-        raise OverflowError("memory mapped size is too large (limited by C int)")
+        raise ROverflowError("memory mapped size is too large (limited by C int)")
 
 if _POSIX:
     def mmap(fileno, length, flags=MAP_SHARED,
-        prot=PROT_WRITE | PROT_READ, access=_ACCESS_DEFAULT):
+        prot=PROT_WRITE | PROT_READ, access=_ACCESS_DEFAULT, offset=0):
 
         fd = fileno
 
@@ -623,6 +627,8 @@
         # check size boundaries
         _check_map_size(length)
         map_size = length
+        if offset < 0:
+            raise RValueError("negative offset")
 
         if access == ACCESS_READ:
             flags = MAP_SHARED
@@ -649,6 +655,7 @@
         else:
             mode = st[stat.ST_MODE]
             size = st[stat.ST_SIZE]
+            size -= offset
             if size > sys.maxint:
                 size = sys.maxint
             else:
@@ -674,7 +681,7 @@
         # XXX if we use hintp below in alloc, the NonConstant
         #     is necessary since we want a general version of c_mmap
         #     to be annotated with a non-constant pointer.
-        res = c_mmap(NonConstant(NULL), map_size, prot, flags, fd, 0)
+        res = c_mmap(NonConstant(NULL), map_size, prot, flags, fd, offset)
         if res == rffi.cast(PTR, -1):
             errno = _get_error_no()
             raise OSError(errno, os.strerror(errno))
@@ -711,10 +718,12 @@
     free = c_munmap_safe
     
 elif _MS_WINDOWS:
-    def mmap(fileno, length, tagname="", access=_ACCESS_DEFAULT):
+    def mmap(fileno, length, tagname="", access=_ACCESS_DEFAULT, offset=0):
         # check size boundaries
         _check_map_size(length)
         map_size = length
+        if offset < 0:
+            raise RValueError("negative offset")
         
         flProtect = 0
         dwDesiredAccess = 0
@@ -785,16 +794,20 @@
         if _64BIT:
             size_hi = map_size >> 32
             size_lo = map_size & 0xFFFFFFFF
+            offset_hi = offset >> 32
+            offset_lo = offset & 0xFFFFFFFF
         else:
             size_hi = 0
             size_lo = map_size
+            offset_hi = 0
+            offset_lo = offset
 
         m.map_handle = CreateFileMapping(m.file_handle, NULL, flProtect,
                                          size_hi, size_lo, m.tagname)
 
         if m.map_handle:
             res = MapViewOfFile(m.map_handle, dwDesiredAccess,
-                                0, 0, 0)
+                                offset_hi, offset_lo, 0)
             if res:
                 # XXX we should have a real LPVOID which must always be casted
                 charp = rffi.cast(LPCSTR, res)

From commits-noreply at bitbucket.org  Thu Feb 10 17:10:59 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Thu, 10 Feb 2011 17:10:59 +0100 (CET)
Subject: [pypy-svn] pypy default: file.writelines() should release the
	stream lock	between two lines
Message-ID: <20110210161059.3E640282C18@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41789:f95db44eb35a
Date: 2011-02-10 15:45 +0100
http://bitbucket.org/pypy/pypy/changeset/f95db44eb35a/

Log:	file.writelines() should release the stream lock between two lines

diff --git a/pypy/module/_file/interp_file.py b/pypy/module/_file/interp_file.py
--- a/pypy/module/_file/interp_file.py
+++ b/pypy/module/_file/interp_file.py
@@ -216,20 +216,6 @@
         self.softspace = 0
         self.getstream().write(data)
 
-    def direct_writelines(self, w_lines):    # note: a wrapped list!
-        stream = self.getstream()
-        space = self.space
-        w_iterator = space.iter(w_lines)
-        self.softspace = 0
-        while True:
-            try:
-                w_line = space.next(w_iterator)
-            except OperationError, e:
-                if not e.match(space, space.w_StopIteration):
-                    raise
-                break  # done
-            stream.write(space.str_w(w_line))
-
     def direct___iter__(self):
         self.getstream()
         return self
@@ -374,12 +360,6 @@
 Note that due to buffering, flush() or close() may be needed before
 the file on disk reflects the data written.""")
 
-    _decl(locals(), "writelines", ['self', W_Root],
-        """writelines(sequence_of_strings) -> None.  Write the strings to the file.
-
-Note that newlines are not added.  The sequence can be any iterable object
-producing strings. This is equivalent to calling write() for each string.""")
-
     _decl(locals(), "__iter__", ['self'],
         """Iterating over files, as in 'for line in f:', returns each line of
 the file one by one.""")
@@ -412,6 +392,24 @@
         else:
             return self.space.str_w(self.space.repr(w_name))
 
+    def file_writelines(self, w_lines):
+        """writelines(sequence_of_strings) -> None.  Write the strings to the file.
+
+Note that newlines are not added.  The sequence can be any iterable object
+producing strings. This is equivalent to calling write() for each string."""
+
+        space = self.space
+        w_iterator = space.iter(w_lines)
+        while True:
+            try:
+                w_line = space.next(w_iterator)
+            except OperationError, e:
+                if not e.match(space, space.w_StopIteration):
+                    raise
+                break  # done
+            self.file_write(space.str_w(w_line))
+    file_writelines.unwrap_spec = ['self', W_Root]
+
     def file_readinto(self, w_rwbuffer):
         """readinto() -> Undocumented.  Don't use this; it may go away."""
         # XXX not the most efficient solution as it doesn't avoid the copying
@@ -509,6 +507,7 @@
                               doc="Support for 'print'."),
     __repr__ = interp2app(W_File.file__repr__),
     readinto = interp2app(W_File.file_readinto),
+    writelines = interp2app(W_File.file_writelines),
     __weakref__ = make_weakref_descr(W_File),
     **dict([(name, interp2app(getattr(W_File, 'file_' + name)))
                 for name in W_File._exposed_method_names])

diff --git a/pypy/module/_file/test/test_file_extra.py b/pypy/module/_file/test/test_file_extra.py
--- a/pypy/module/_file/test/test_file_extra.py
+++ b/pypy/module/_file/test/test_file_extra.py
@@ -382,6 +382,20 @@
         assert len(somelines) > 200
         assert somelines == lines[:len(somelines)]
 
+    def test_nasty_writelines(self):
+        # The stream lock should be released between writes
+        fn = self.temptestfile
+        f = file(fn, 'w')
+        def nasty():
+            for i in range(5):
+                if i == 3:
+                    # should not raise because of acquired lock
+                    f.close()
+                yield str(i)
+        exc = raises(ValueError, f.writelines, nasty())
+        assert exc.value.message == "I/O operation on closed file"
+        f.close()
+
     def test_rw_bin(self):
         import random
         flags = 'w+b'

From commits-noreply at bitbucket.org  Thu Feb 10 17:10:59 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Thu, 10 Feb 2011 17:10:59 +0100 (CET)
Subject: [pypy-svn] pypy default: Merge heads
Message-ID: <20110210161059.84301282C24@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41790:75d55fe59418
Date: 2011-02-10 15:45 +0100
http://bitbucket.org/pypy/pypy/changeset/75d55fe59418/

Log:	Merge heads


From commits-noreply at bitbucket.org  Thu Feb 10 17:11:00 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Thu, 10 Feb 2011 17:11:00 +0100 (CET)
Subject: [pypy-svn] pypy default: Simplify code,
	now that pypy natively understands the with statement
Message-ID: <20110210161100.172A4282C18@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41791:5eb55f87d111
Date: 2011-02-10 15:49 +0100
http://bitbucket.org/pypy/pypy/changeset/5eb55f87d111/

Log:	Simplify code, now that pypy natively understands the with statement

diff --git a/pypy/module/thread/test/test_lock.py b/pypy/module/thread/test/test_lock.py
--- a/pypy/module/thread/test/test_lock.py
+++ b/pypy/module/thread/test/test_lock.py
@@ -1,3 +1,4 @@
+from __future__ import with_statement
 from pypy.module.thread.test.support import GenericTestThread
 from pypy.translator.c.test.test_genc import compile
 
@@ -40,13 +41,9 @@
             lock.release()
         assert lock.locked() is True
         thread.start_new_thread(f, ())
-        exec """
-from __future__ import with_statement
-if 1:
         with lock:
             assert lock.locked() is True
             assert feedback == [42]
-""" in {"lock": lock, "feedback": feedback}
         assert lock.locked() is False
 
 def test_compile_lock():

From commits-noreply at bitbucket.org  Thu Feb 10 17:11:00 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Thu, 10 Feb 2011 17:11:00 +0100 (CET)
Subject: [pypy-svn] pypy default: Merge heads
Message-ID: <20110210161100.59566282C24@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41792:2e87262a36b1
Date: 2011-02-10 17:10 +0100
http://bitbucket.org/pypy/pypy/changeset/2e87262a36b1/

Log:	Merge heads


From commits-noreply at bitbucket.org  Thu Feb 10 17:28:06 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Thu, 10 Feb 2011 17:28:06 +0100 (CET)
Subject: [pypy-svn] pypy default: Bah,
	c_thread_lock_init was declared to return Void anyway :-(
Message-ID: <20110210162806.21B19282C18@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41793:402bab283b1d
Date: 2011-02-10 17:25 +0100
http://bitbucket.org/pypy/pypy/changeset/402bab283b1d/

Log:	Bah, c_thread_lock_init was declared to return Void anyway :-(

diff --git a/pypy/module/thread/ll_thread.py b/pypy/module/thread/ll_thread.py
--- a/pypy/module/thread/ll_thread.py
+++ b/pypy/module/thread/ll_thread.py
@@ -49,7 +49,7 @@
 TLOCKP = rffi.COpaquePtr('struct RPyOpaque_ThreadLock',
                           compilation_info=eci)
 
-c_thread_lock_init = llexternal('RPyThreadLockInit', [TLOCKP], lltype.Void)
+c_thread_lock_init = llexternal('RPyThreadLockInit', [TLOCKP], rffi.INT)
 c_thread_acquirelock = llexternal('RPyThreadAcquireLock', [TLOCKP, rffi.INT],
                                   rffi.INT,
                                   threadsafe=True)    # release the GIL
@@ -133,7 +133,7 @@
     # lock objects, as well as from the GIL, which exists at shutdown.
     ll_lock = lltype.malloc(TLOCKP.TO, flavor='raw', track_allocation=False)
     res = c_thread_lock_init(ll_lock)
-    if res <= 0:
+    if rffi.cast(lltype.Signed, res) <= 0:
         lltype.free(ll_lock, flavor='raw', track_allocation=False)
         raise error("out of resources")
     return ll_lock

From commits-noreply at bitbucket.org  Thu Feb 10 17:28:06 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Thu, 10 Feb 2011 17:28:06 +0100 (CET)
Subject: [pypy-svn] pypy default: merge heads
Message-ID: <20110210162806.5928D282C24@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41794:13bc02ca6a1b
Date: 2011-02-10 17:27 +0100
http://bitbucket.org/pypy/pypy/changeset/13bc02ca6a1b/

Log:	merge heads


From commits-noreply at bitbucket.org  Thu Feb 10 17:56:37 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Thu, 10 Feb 2011 17:56:37 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: hg merge default
Message-ID: <20110210165637.120BC2A2002@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r41795:7bea42e36e7e
Date: 2011-02-10 17:27 +0100
http://bitbucket.org/pypy/pypy/changeset/7bea42e36e7e/

Log:	hg merge default


From commits-noreply at bitbucket.org  Thu Feb 10 19:18:27 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Thu, 10 Feb 2011 19:18:27 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: Make sure we dont get cahced
	values
Message-ID: <20110210181827.8B490282C18@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r41796:ef79eba9e3af
Date: 2011-02-10 19:17 +0100
http://bitbucket.org/pypy/pypy/changeset/ef79eba9e3af/

Log:	Make sure we dont get cahced values

diff --git a/pypy/jit/metainterp/optimizeopt/unroll.py b/pypy/jit/metainterp/optimizeopt/unroll.py
--- a/pypy/jit/metainterp/optimizeopt/unroll.py
+++ b/pypy/jit/metainterp/optimizeopt/unroll.py
@@ -165,7 +165,7 @@
             info = self.info[box]
         except KeyError:
             value = self.getvalue(box)
-            self.info[box] = info = value.make_virtual_info(self, [])
+            self.info[box] = info = value.make_virtual_info(self, None)
             if box in self.fieldboxes:
                 info.fieldstate = [self.state(b) for b in self.fieldboxes[box]]
                 # FIXME: Do we realy want to add fields to the VirtualInfo's?

diff --git a/pypy/jit/metainterp/optimizeopt/optimizer.py b/pypy/jit/metainterp/optimizeopt/optimizer.py
--- a/pypy/jit/metainterp/optimizeopt/optimizer.py
+++ b/pypy/jit/metainterp/optimizeopt/optimizer.py
@@ -70,6 +70,7 @@
 
     def make_virtual_info(self, modifier, fieldnums):
         #raise NotImplementedError # should not be called on this level
+        assert fieldnums is None
         return modifier.make_not_virtual(self)
 
     def is_constant(self):

diff --git a/pypy/jit/metainterp/optimizeopt/virtualize.py b/pypy/jit/metainterp/optimizeopt/virtualize.py
--- a/pypy/jit/metainterp/optimizeopt/virtualize.py
+++ b/pypy/jit/metainterp/optimizeopt/virtualize.py
@@ -30,6 +30,8 @@
         return self.box
 
     def make_virtual_info(self, modifier, fieldnums):
+        if fieldnums is None:
+            return self._make_virtual(modifier)
         vinfo = self._cached_vinfo
         if vinfo is not None and vinfo.equals(fieldnums):
             return vinfo

From commits-noreply at bitbucket.org  Thu Feb 10 19:21:01 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Thu, 10 Feb 2011 19:21:01 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: hg merge default
Message-ID: <20110210182101.B407F282C18@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r41797:4e6f0f9ef69f
Date: 2011-02-10 19:20 +0100
http://bitbucket.org/pypy/pypy/changeset/4e6f0f9ef69f/

Log:	hg merge default


From commits-noreply at bitbucket.org  Thu Feb 10 19:28:30 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Thu, 10 Feb 2011 19:28:30 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: for tests
Message-ID: <20110210182830.EA3E4282C18@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r41798:ebe6d0cd0e30
Date: 2011-02-10 19:28 +0100
http://bitbucket.org/pypy/pypy/changeset/ebe6d0cd0e30/

Log:	for tests

diff --git a/pypy/jit/metainterp/warmstate.py b/pypy/jit/metainterp/warmstate.py
--- a/pypy/jit/metainterp/warmstate.py
+++ b/pypy/jit/metainterp/warmstate.py
@@ -243,7 +243,8 @@
 
     def set_param_retrace_limit(self, value):
         if self.warmrunnerdesc:
-            self.warmrunnerdesc.memory_manager.retrace_limit = value
+            if self.warmrunnerdesc.memory_manager:
+                self.warmrunnerdesc.memory_manager.retrace_limit = value
 
     def disable_noninlinable_function(self, greenkey):
         cell = self.jit_cell_at_key(greenkey)

From commits-noreply at bitbucket.org  Thu Feb 10 19:34:07 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Thu, 10 Feb 2011 19:34:07 +0100 (CET)
Subject: [pypy-svn] pypy default: Don't be lazy and import sys in each
	function instead	of stuffing it into __builtins__.
Message-ID: <20110210183407.E096B282C18@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41799:093925e5bcd1
Date: 2011-02-10 19:07 +0100
http://bitbucket.org/pypy/pypy/changeset/093925e5bcd1/

Log:	Don't be lazy and import sys in each function instead of stuffing it
	into __builtins__. This revealed a missing import in the
	excepthook...

diff --git a/pypy/module/sys/app.py b/pypy/module/sys/app.py
--- a/pypy/module/sys/app.py
+++ b/pypy/module/sys/app.py
@@ -4,6 +4,7 @@
 """
 
 from _structseq import structseqtype, structseqfield
+import sys
 
 def excepthook(exctype, value, traceback):
     """Handle an exception by displaying it with a traceback on sys.stderr."""

diff --git a/pypy/module/sys/test/test_sysmodule.py b/pypy/module/sys/test/test_sysmodule.py
--- a/pypy/module/sys/test/test_sysmodule.py
+++ b/pypy/module/sys/test/test_sysmodule.py
@@ -5,14 +5,6 @@
 from pypy.interpreter.gateway import app2interp_temp
 import sys
 
-def init_globals_via_builtins_hack(space):
-    space.appexec([], """():
-    import __builtin__ as b
-    import cStringIO, sys
-    b.cStringIO = cStringIO
-    b.sys = sys
-    """)
-
 def test_stdin_exists(space):
     space.sys.get('stdin') 
     space.sys.get('__stdin__')
@@ -138,11 +130,10 @@
 class AppTestSysModulePortedFromCPython:
 
     def setup_class(cls):
-        init_globals_via_builtins_hack(cls.space)
         cls.w_appdirect = cls.space.wrap(option.runappdirect)
 
     def test_original_displayhook(self):
-        import __builtin__
+        import sys, cStringIO, __builtin__
         savestdout = sys.stdout
         out = cStringIO.StringIO()
         sys.stdout = out
@@ -166,6 +157,7 @@
         sys.stdout = savestdout
 
     def test_lost_displayhook(self):
+        import sys
         olddisplayhook = sys.displayhook
         del sys.displayhook
         code = compile("42", "<string>", "single")
@@ -173,6 +165,7 @@
         sys.displayhook = olddisplayhook
 
     def test_custom_displayhook(self):
+        import sys
         olddisplayhook = sys.displayhook
         def baddisplayhook(obj):
             raise ValueError
@@ -182,7 +175,7 @@
         sys.displayhook = olddisplayhook
 
     def test_original_excepthook(self):
-        import cStringIO
+        import sys, cStringIO
         savestderr = sys.stderr
         err = cStringIO.StringIO()
         sys.stderr = err
@@ -201,7 +194,7 @@
     def test_excepthook_failsafe_path(self):
         import traceback
         original_print_exception = traceback.print_exception
-        import cStringIO
+        import sys, cStringIO
         savestderr = sys.stderr
         err = cStringIO.StringIO()
         sys.stderr = err
@@ -222,6 +215,7 @@
     # Python/pythonrun.c::PyErr_PrintEx() is tricky.
 
     def test_exc_clear(self):
+        import sys
         raises(TypeError, sys.exc_clear, 42)
 
         # Verify that exc_info is present and matches exc, then clear it, and
@@ -266,6 +260,7 @@
         clear_check(exc)
 
     def test_exit(self):
+        import sys
         raises(TypeError, sys.exit, 42, 42)
 
         # call without argument
@@ -320,11 +315,13 @@
             raise AssertionError, "no exception"
 
     def test_getdefaultencoding(self):
+        import sys
         raises(TypeError, sys.getdefaultencoding, 42)
         # can't check more than the type, as the user might have changed it
         assert isinstance(sys.getdefaultencoding(), str)
 
     def test_setdefaultencoding(self):
+        import sys
         if self.appdirect:
             skip("not worth running appdirect")
             
@@ -346,6 +343,7 @@
     # testing sys.setprofile() is done in test_profile.py
 
     def test_setcheckinterval(self):
+        import sys
         raises(TypeError, sys.setcheckinterval)
         orig = sys.getcheckinterval()
         for n in 0, 100, 120, orig: # orig last to restore starting state
@@ -353,6 +351,7 @@
             assert sys.getcheckinterval() == n
 
     def test_recursionlimit(self):
+        import sys
         raises(TypeError, sys.getrecursionlimit, 42)
         oldlimit = sys.getrecursionlimit()
         raises(TypeError, sys.setrecursionlimit)
@@ -362,6 +361,7 @@
         sys.setrecursionlimit(oldlimit)
 
     def test_getwindowsversion(self):
+        import sys
         if hasattr(sys, "getwindowsversion"):
             v = sys.getwindowsversion()
             assert isinstance(v, tuple)
@@ -373,10 +373,12 @@
             assert isinstance(v[4], str)
 
     def test_winver(self):
+        import sys
         if hasattr(sys, "winver"):
             assert sys.winver == sys.version[:3]
 
     def test_dlopenflags(self):
+        import sys
         if hasattr(sys, "setdlopenflags"):
             assert hasattr(sys, "getdlopenflags")
             raises(TypeError, sys.getdlopenflags, 42)
@@ -387,6 +389,7 @@
             sys.setdlopenflags(oldflags)
 
     def test_refcount(self):
+        import sys
         if not hasattr(sys, "getrefcount"):
             skip('Reference counting is not implemented.')
 
@@ -400,6 +403,7 @@
             assert isinstance(sys.gettotalrefcount(), int)
 
     def test_getframe(self):
+        import sys
         raises(TypeError, sys._getframe, 42, 42)
         raises(ValueError, sys._getframe, 2000000000)
         assert sys._getframe().f_code.co_name == 'test_getframe'
@@ -409,6 +413,7 @@
         #)
 
     def test_getframe_in_returned_func(self):
+        import sys
         def f():
             return g()
         def g():
@@ -419,6 +424,7 @@
         assert frame.f_back.f_back.f_code.co_name == 'test_getframe_in_returned_func'
 
     def test_attributes(self):
+        import sys
         assert sys.__name__ == 'sys'
         assert isinstance(sys.modules, dict)
         assert isinstance(sys.path, list)
@@ -447,6 +453,7 @@
         assert isinstance(vi[4], int)
 
     def test_settrace(self):
+        import sys
         counts = []
         def trace(x, y, z):
             counts.append(None)
@@ -461,6 +468,7 @@
         assert len(counts) == 1
 
     def test_pypy_attributes(self):
+        import sys
         assert isinstance(sys.pypy_objspaceclass, str)
         vi = sys.pypy_version_info
         assert isinstance(vi, tuple)
@@ -472,13 +480,15 @@
         assert isinstance(vi[4], int)
 
     def test_allattributes(self):
+        import sys
         sys.__dict__   # check that we don't crash initializing any attribute
 
     def test_subversion(self):
+        import sys
         assert sys.subversion == ('PyPy', '', '')
 
     def test__mercurial(self):
-        import re
+        import sys, re
         project, hgtag, hgid = sys._mercurial
         assert project == 'PyPy'
         # the tag or branch may be anything, including the empty string
@@ -491,6 +501,7 @@
             assert hgid in sys.version
 
     def test_trace_exec_execfile(self):
+        import sys
         found = []
         def do_tracing(f, *args):
             print f.f_code.co_filename, f.f_lineno, args

From commits-noreply at bitbucket.org  Thu Feb 10 19:34:08 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Thu, 10 Feb 2011 19:34:08 +0100 (CET)
Subject: [pypy-svn] pypy default: Try to fix issue with mixed output.
Message-ID: <20110210183408.79335282C18@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41800:91563780c6ec
Date: 2011-02-10 19:33 +0100
http://bitbucket.org/pypy/pypy/changeset/91563780c6ec/

Log:	Try to fix issue with mixed output. Don't know how to test this
	directly, though

diff --git a/pypy/module/sys/app.py b/pypy/module/sys/app.py
--- a/pypy/module/sys/app.py
+++ b/pypy/module/sys/app.py
@@ -8,6 +8,13 @@
 
 def excepthook(exctype, value, traceback):
     """Handle an exception by displaying it with a traceback on sys.stderr."""
+
+    # Flush stdout as well, both files may refer to the same file
+    try:
+        sys.stdout.flush()
+    except:
+        pass
+
     try:
         from traceback import print_exception
         print_exception(exctype, value, traceback)

From commits-noreply at bitbucket.org  Thu Feb 10 20:52:34 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Thu, 10 Feb 2011 20:52:34 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix mmap.resize() when the mmap has an
	offset.
Message-ID: <20110210195234.ADD29282BE9@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41801:62b77a7d8f53
Date: 2011-02-10 14:51 -0500
http://bitbucket.org/pypy/pypy/changeset/62b77a7d8f53/

Log:	Fix mmap.resize() when the mmap has an offset.

diff --git a/pypy/module/mmap/test/test_mmap.py b/pypy/module/mmap/test/test_mmap.py
--- a/pypy/module/mmap/test/test_mmap.py
+++ b/pypy/module/mmap/test/test_mmap.py
@@ -529,6 +529,35 @@
         m.close()
         f.close()
 
+    def test_offset_more(self):
+        from mmap import mmap, ALLOCATIONGRANULARITY
+
+        with open(self.tmpname, "w+b") as f:
+            halfsize = ALLOCATIONGRANULARITY
+            f.write("\0" * halfsize)
+            f.write("foo")
+            f.write("\0" * (halfsize - 3))
+            m = mmap(f.fileno(), 0)
+            m.close()
+
+        with open(self.tmpname, "r+b") as f:
+            m = mmap(f.fileno(), halfsize, offset=halfsize)
+            assert m[0:3] == "foo"
+
+        try:
+            m.resize(512)
+        except SystemError:
+            pass
+        else:
+            assert len(m) == 512
+            raises(ValueError, m.seek, 513, 0)
+            assert m[0:3] == "foo"
+            with open(self.tmpname) as f:
+                f.seek(0, 2)
+                assert f.tell() == halfsize + 512
+            assert m.size() == halfsize + 512
+        m.close()
+
     def test_all(self):
         # this is a global test, ported from test_mmap.py
         import mmap
@@ -710,4 +739,3 @@
         assert m.read(10) == "ABCDEABCDE"
         m.close()
         f.close()
-

diff --git a/pypy/rlib/rmmap.py b/pypy/rlib/rmmap.py
--- a/pypy/rlib/rmmap.py
+++ b/pypy/rlib/rmmap.py
@@ -261,10 +261,11 @@
 NODATA = lltype.nullptr(PTR.TO)
 
 class MMap(object):
-    def __init__(self, access):
+    def __init__(self, access, offset):
         self.size = 0
         self.pos = 0
         self.access = access
+        self.offset = offset
 
         if _MS_WINDOWS:
             self.map_handle = NULL_HANDLE
@@ -534,7 +535,7 @@
                 raise OSError(-11111, "No mremap available")
             
             # resize the underlying file first
-            os.ftruncate(self.fd, newsize)
+            os.ftruncate(self.fd, self.offset + newsize)
                 
             # now resize the mmap
             newdata = c_mremap(self.getptr(0), self.size, newsize,
@@ -547,11 +548,15 @@
 
             # move to the desired EOF position
             if _64BIT:
-                newsize_high = newsize >> 32
-                newsize_low = newsize & 0xFFFFFFFF
+                newsize_high = (self.offset + newsize) >> 32
+                newsize_low = (self.offset + newsize) & 0xFFFFFFFF
+                offset_high = self.offset >> 32
+                offset_low = self.offset & 0xFFFFFFFF
             else:
                 newsize_high = 0
-                newsize_low = newsize
+                newsize_low = self.offset + newsize
+                offset_high = 0
+                offset_low = self.offset
 
             FILE_BEGIN = 0
             high_ref = lltype.malloc(PLONG.TO, 1, flavor='raw')
@@ -666,7 +671,7 @@
                 elif map_size > size:
                     raise RValueError("mmap length is greater than file size")
 
-        m = MMap(access)
+        m = MMap(access, offset)
         if fd == -1:
             # Assume the caller wants to map anonymous memory.
             # This is the same behaviour as Windows.  mmap.mmap(-1, size)
@@ -753,7 +758,7 @@
             # SEEK_SET = 0
             # libc._lseek(fileno, 0, SEEK_SET)
         
-        m = MMap(access)
+        m = MMap(access, offset)
         m.file_handle = INVALID_HANDLE
         m.map_handle = INVALID_HANDLE
         if fh:

From commits-noreply at bitbucket.org  Thu Feb 10 20:52:34 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Thu, 10 Feb 2011 20:52:34 +0100 (CET)
Subject: [pypy-svn] pypy default: Merged upstream.
Message-ID: <20110210195234.EBD0D282BF2@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41802:5473d70410a3
Date: 2011-02-10 14:52 -0500
http://bitbucket.org/pypy/pypy/changeset/5473d70410a3/

Log:	Merged upstream.


From commits-noreply at bitbucket.org  Thu Feb 10 20:56:10 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Thu, 10 Feb 2011 20:56:10 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix mmap.resize() on windows.
Message-ID: <20110210195610.B8608282BE9@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41803:6c999e5e543f
Date: 2011-02-10 14:55 -0500
http://bitbucket.org/pypy/pypy/changeset/6c999e5e543f/

Log:	Fix mmap.resize() on windows.

diff --git a/pypy/rlib/rmmap.py b/pypy/rlib/rmmap.py
--- a/pypy/rlib/rmmap.py
+++ b/pypy/rlib/rmmap.py
@@ -576,7 +576,7 @@
             dwErrCode = 0
             if self.map_handle:
                 data = MapViewOfFile(self.map_handle, FILE_MAP_WRITE,
-                                     0, 0, 0)
+                                     offset_high, offset_low, newsize)
                 if data:
                     # XXX we should have a real LPVOID which must always be casted
                     charp = rffi.cast(LPCSTR, data)

From commits-noreply at bitbucket.org  Thu Feb 10 23:37:39 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Thu, 10 Feb 2011 23:37:39 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix usage of mmap offset on Windows
Message-ID: <20110210223739.93AC6282BE9@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41805:22b4129af523
Date: 2011-02-10 23:37 +0100
http://bitbucket.org/pypy/pypy/changeset/22b4129af523/

Log:	Fix usage of mmap offset on Windows

diff --git a/pypy/rlib/rmmap.py b/pypy/rlib/rmmap.py
--- a/pypy/rlib/rmmap.py
+++ b/pypy/rlib/rmmap.py
@@ -769,13 +769,13 @@
         
         # DWORD is a 4-byte int. If int > 4-byte it must be divided
         if _64BIT:
-            size_hi = map_size >> 32
-            size_lo = map_size & 0xFFFFFFFF
+            size_hi = (map_size + offset) >> 32
+            size_lo = (map_size + offset) & 0xFFFFFFFF
             offset_hi = offset >> 32
             offset_lo = offset & 0xFFFFFFFF
         else:
             size_hi = 0
-            size_lo = map_size
+            size_lo = map_size + offset
             offset_hi = 0
             offset_lo = offset
 
@@ -784,7 +784,7 @@
 
         if m.map_handle:
             data = MapViewOfFile(m.map_handle, dwDesiredAccess,
-                                offset_hi, offset_lo, 0)
+                                 offset_hi, offset_lo, 0)
             if data:
                 # XXX we should have a real LPVOID which must always be casted
                 charp = rffi.cast(LPCSTR, data)

From commits-noreply at bitbucket.org  Fri Feb 11 04:43:25 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Fri, 11 Feb 2011 04:43:25 +0100 (CET)
Subject: [pypy-svn] pypy default: Make sure BZ2File has a writelines method.
Message-ID: <20110211034325.7CBC4282BE9@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41806:0a8c4bf2785c
Date: 2011-02-10 22:43 -0500
http://bitbucket.org/pypy/pypy/changeset/0a8c4bf2785c/

Log:	Make sure BZ2File has a writelines method.

diff --git a/pypy/module/bz2/interp_bz2.py b/pypy/module/bz2/interp_bz2.py
--- a/pypy/module/bz2/interp_bz2.py
+++ b/pypy/module/bz2/interp_bz2.py
@@ -290,7 +290,7 @@
 same_attributes_as_in_file.remove('__init__')
 same_attributes_as_in_file.extend([
     'name', 'mode', 'encoding', 'closed', 'newlines', 'softspace',
-    '__weakref__'])
+    'writelines', '__weakref__'])
 
 W_BZ2File.typedef = TypeDef(
     "BZ2File",

From commits-noreply at bitbucket.org  Fri Feb 11 06:18:49 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Fri, 11 Feb 2011 06:18:49 +0100 (CET)
Subject: [pypy-svn] pypy default: file.writelines should raise when its
	closed, even if the iterable has no items.
Message-ID: <20110211051849.390C5282BF2@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41807:24c33ac7e91a
Date: 2011-02-11 00:18 -0500
http://bitbucket.org/pypy/pypy/changeset/24c33ac7e91a/

Log:	file.writelines should raise when its closed, even if the iterable
	has no items.

diff --git a/pypy/module/_file/interp_file.py b/pypy/module/_file/interp_file.py
--- a/pypy/module/_file/interp_file.py
+++ b/pypy/module/_file/interp_file.py
@@ -58,15 +58,17 @@
             raise operationerrfmt(space.w_ValueError,
                                   "invalid mode: '%s'", mode)
 
+    def check_closed(self):
+        if self.stream is None:
+            raise OperationError(self.space.w_ValueError,
+                self.space.wrap("I/O operation on closed file")
+            )
+
     def getstream(self):
         """Return self.stream or raise an app-level ValueError if missing
         (i.e. if the file is closed)."""
-        stream = self.stream
-        if stream is None:
-            space = self.space
-            raise OperationError(space.w_ValueError,
-                                 space.wrap('I/O operation on closed file'))
-        return stream
+        self.check_closed()
+        return self.stream
 
     def _when_reading_first_flush(self, otherfile):
         """Flush otherfile before reading from self."""
@@ -399,6 +401,7 @@
 producing strings. This is equivalent to calling write() for each string."""
 
         space = self.space
+        self.check_closed()
         w_iterator = space.iter(w_lines)
         while True:
             try:

From commits-noreply at bitbucket.org  Fri Feb 11 09:34:48 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Fri, 11 Feb 2011 09:34:48 +0100 (CET)
Subject: [pypy-svn] pypy default: Make JitDriver a newstyle class
Message-ID: <20110211083448.19702282BE9@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: 
Changeset: r41808:908501484fdd
Date: 2011-02-09 21:07 +0200
http://bitbucket.org/pypy/pypy/changeset/908501484fdd/

Log:	Make JitDriver a newstyle class

diff --git a/pypy/rlib/jit.py b/pypy/rlib/jit.py
--- a/pypy/rlib/jit.py
+++ b/pypy/rlib/jit.py
@@ -271,7 +271,7 @@
 
 # ____________________________________________________________
 
-class JitDriver:    
+class JitDriver(object):    
     """Base class to declare fine-grained user control on the JIT.  So
     far, there must be a singleton instance of JitDriver.  This style
     will allow us (later) to support a single RPython program with

From commits-noreply at bitbucket.org  Fri Feb 11 09:34:48 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Fri, 11 Feb 2011 09:34:48 +0100 (CET)
Subject: [pypy-svn] pypy numpy-exp: A bit more rpythonic
Message-ID: <20110211083448.BC26C282BE9@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: numpy-exp
Changeset: r41809:6b18ac4a698f
Date: 2011-02-09 21:47 +0200
http://bitbucket.org/pypy/pypy/changeset/6b18ac4a698f/

Log:	A bit more rpythonic

diff --git a/pypy/module/micronumpy/numarray.py b/pypy/module/micronumpy/numarray.py
--- a/pypy/module/micronumpy/numarray.py
+++ b/pypy/module/micronumpy/numarray.py
@@ -54,6 +54,7 @@
                 raise NotImplementedError
             bytecode_pos -= 1
     return result
+
     
 class BaseArray(Wrappable):
     def force(self):
@@ -61,11 +62,17 @@
         return compute(bytecode, stack)
     force.unwrap_spec = ['self']
 
+    def descr_add(self, space, w_other):
+        return space.wrap(Add(self, w_other))
+    descr_add.unwrap_spec = ['self', ObjSpace, W_Root]
+
     def compile(self):
         raise NotImplementedError("abstract base class")
 
 class Add(BaseArray):
     def __init__(self, left, right):
+        assert isinstance(left, BaseArray)
+        assert isinstance(right, BaseArray)
         self.left = left
         self.right = right
 
@@ -77,6 +84,7 @@
 BaseArray.typedef = TypeDef(
     'Operation',
     force=interp2app(BaseArray.force),
+    __add__ = interp2app(BaseArray.descr_add),
 )
 
 class SingleDimArray(BaseArray):
@@ -107,10 +115,6 @@
         self.storage[item] = value
     descr_setitem.unwrap_spec = ['self', ObjSpace, int, float]
 
-    def descr_add(self, space, w_other):
-        return space.wrap(Add(self, w_other))
-    descr_add.unwrap_spec = ['self', ObjSpace, W_Root]
-
     def force(self):
         return self
 
@@ -132,7 +136,7 @@
     __new__ = interp2app(descr_new_numarray),
     __getitem__ = interp2app(SingleDimArray.descr_getitem),
     __setitem__ = interp2app(SingleDimArray.descr_setitem),
-    __add__ = interp2app(SingleDimArray.descr_add),
+    __add__ = interp2app(BaseArray.descr_add),
     force = interp2app(SingleDimArray.force),
 )
 

From commits-noreply at bitbucket.org  Fri Feb 11 09:34:49 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Fri, 11 Feb 2011 09:34:49 +0100 (CET)
Subject: [pypy-svn] pypy numpy-exp: Write a jit test, easy so far
Message-ID: <20110211083449.7FEEB282BE9@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: numpy-exp
Changeset: r41810:88a25221eb86
Date: 2011-02-11 10:32 +0200
http://bitbucket.org/pypy/pypy/changeset/88a25221eb86/

Log:	Write a jit test, easy so far

diff --git a/pypy/module/micronumpy/numarray.py b/pypy/module/micronumpy/numarray.py
--- a/pypy/module/micronumpy/numarray.py
+++ b/pypy/module/micronumpy/numarray.py
@@ -8,10 +8,10 @@
 
 TP = lltype.GcArray(lltype.Float)
 
-numpy_driver = jit.JitDriver(greens = ['bytecode'],
-                             reds = ['result', 'result_size', 'i',
-                                     'valuestack', 'valuestackdepth',
-                                     'input', 'input_pos'])
+numpy_driver = jit.JitDriver(greens = ['bytecode_pos', 'bytecode'],
+                             reds = ['result_size', 'i', 'input_pos',
+                                     'valuestackdepth', 'valuestack',
+                                     'input', 'result'])
 
 def compute(bytecode, input):
     result_size = input[0].size
@@ -26,7 +26,8 @@
                                      result_size=result_size,
                                      valuestackdepth=valuestackdepth,
                                      valuestack=valuestack,
-                                     input=input, input_pos=input_pos, i=i)
+                                     input=input, input_pos=input_pos, i=i,
+                                     bytecode_pos=bytecode_pos)
         if bytecode_pos == -1:
             bytecode_pos = len(bytecode) - 1
             input_pos = len(input) - 1
@@ -38,7 +39,8 @@
                                        result_size=result_size,
                                        valuestackdepth=valuestackdepth,
                                        valuestack=valuestack,
-                                       input=input, input_pos=input_pos, i=i)
+                                       input=input, input_pos=input_pos, i=i,
+                                       bytecode_pos=bytecode_pos)
         else:
             opcode = bytecode[bytecode_pos]
             if opcode == 'l':

diff --git a/pypy/module/micronumpy/test/test_jit.py b/pypy/module/micronumpy/test/test_jit.py
new file mode 100644
--- /dev/null
+++ b/pypy/module/micronumpy/test/test_jit.py
@@ -0,0 +1,24 @@
+
+from pypy.module.micronumpy.numarray import SingleDimArray, Add
+from pypy.conftest import gettestobjspace
+from pypy.jit.metainterp.test.test_basic import LLJitMixin
+
+class FakeSpace(object):
+    pass
+
+class TestNumpyJIt(LLJitMixin):
+    def setup_class(cls):
+        cls.space = FakeSpace()
+    
+    def test_add(self):
+        space = self.space
+        
+        def f(i):
+            ar = SingleDimArray(i)
+            if i:
+                v = Add(ar, ar)
+            else:
+                v = ar
+            return v.force().storage[3]
+
+        self.meta_interp(f, [5])

From commits-noreply at bitbucket.org  Fri Feb 11 09:34:50 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Fri, 11 Feb 2011 09:34:50 +0100 (CET)
Subject: [pypy-svn] pypy numpy-exp: update the demo
Message-ID: <20110211083450.6591E282BE9@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: numpy-exp
Changeset: r41811:6f8db8706654
Date: 2011-02-11 10:34 +0200
http://bitbucket.org/pypy/pypy/changeset/6f8db8706654/

Log:	update the demo

diff --git a/pypy/jit/tl/pypyjit_demo.py b/pypy/jit/tl/pypyjit_demo.py
--- a/pypy/jit/tl/pypyjit_demo.py
+++ b/pypy/jit/tl/pypyjit_demo.py
@@ -1,18 +1,9 @@
 
 try:
-    import pypyjit
-    pypyjit.set_param(threshold=3, inlining=True)
-
-    def sqrt(y, n=10000):
-        x = y / 2
-        while n > 0:
-            #assert y > 0 and x > 0
-            if y > 0 and x > 0: pass
-            n -= 1
-            x = (x + y/x) / 2
-        return x
-
-    print sqrt(1234, 4)
+    import numpy
+    a = numpy.array(range(10))
+    b = a + a + a
+    print b[3]
     
 except Exception, e:
     print "Exception: ", type(e)

From commits-noreply at bitbucket.org  Fri Feb 11 09:34:51 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Fri, 11 Feb 2011 09:34:51 +0100 (CET)
Subject: [pypy-svn] pypy default: merge heads
Message-ID: <20110211083451.C57E22A2032@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: 
Changeset: r41812:02e3c7c252a5
Date: 2011-02-11 10:34 +0200
http://bitbucket.org/pypy/pypy/changeset/02e3c7c252a5/

Log:	merge heads


From commits-noreply at bitbucket.org  Fri Feb 11 10:10:19 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri, 11 Feb 2011 10:10:19 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix test when run with a CPython before 2.7
Message-ID: <20110211091019.44B5F282BE9@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41813:3b23b78bf46c
Date: 2011-02-11 09:25 +0100
http://bitbucket.org/pypy/pypy/changeset/3b23b78bf46c/

Log:	Fix test when run with a CPython before 2.7

diff --git a/pypy/rlib/test/test_runicode.py b/pypy/rlib/test/test_runicode.py
--- a/pypy/rlib/test/test_runicode.py
+++ b/pypy/rlib/test/test_runicode.py
@@ -310,18 +310,24 @@
 class TestEncoding(UnicodeTests):
     def test_all_ascii(self):
         for i in range(128):
-            for encoding in "utf-7 utf-8 latin-1 ascii".split():
+            if sys.version >= "2.7":
+                self.checkencode(unichr(i), "utf-7")
+            for encoding in "utf-8 latin-1 ascii".split():
                 self.checkencode(unichr(i), encoding)
 
     def test_all_first_256(self):
         for i in range(256):
-            for encoding in ("utf-7 utf-8 utf-16 utf-16-be utf-16-le "
+            if sys.version >= "2.7":
+                self.checkencode(unichr(i), "utf-7")
+            for encoding in ("utf-8 utf-16 utf-16-be utf-16-le "
                              "utf-32 utf-32-be utf-32-le").split():
                 self.checkencode(unichr(i), encoding)
 
     def test_first_10000(self):
         for i in range(10000):
-            for encoding in ("utf-7 utf-8 utf-16 utf-16-be utf-16-le "
+            if sys.version >= "2.7":
+                self.checkencode(unichr(i), "utf-7")
+            for encoding in ("utf-8 utf-16 utf-16-be utf-16-le "
                              "utf-32 utf-32-be utf-32-le").split():
                 self.checkencode(unichr(i), encoding)
 
@@ -331,7 +337,9 @@
             if 0xd800 <= v <= 0xdfff:
                 continue
             uni = unichr(v)
-            for encoding in ("utf-7 utf-8 utf-16 utf-16-be utf-16-le "
+            if sys.version >= "2.7":
+                self.checkencode(uni, "utf-7")
+            for encoding in ("utf-8 utf-16 utf-16-be utf-16-le "
                              "utf-32 utf-32-be utf-32-le").split():
                 self.checkencode(uni, encoding)                
 

From commits-noreply at bitbucket.org  Fri Feb 11 10:10:20 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri, 11 Feb 2011 10:10:20 +0100 (CET)
Subject: [pypy-svn] pypy default: long.truediv should work identically
	regardless of the	sign
Message-ID: <20110211091020.2ABE4282BE9@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41814:a73b5d006110
Date: 2011-02-11 10:04 +0100
http://bitbucket.org/pypy/pypy/changeset/a73b5d006110/

Log:	long.truediv should work identically regardless of the sign (rshift
	does not: -7 >> 1 == -4)

diff --git a/pypy/rlib/test/test_rbigint.py b/pypy/rlib/test/test_rbigint.py
--- a/pypy/rlib/test/test_rbigint.py
+++ b/pypy/rlib/test/test_rbigint.py
@@ -58,10 +58,16 @@
         f = op1.truediv(op2)
         assert f == 1.7976931348623157e+308     # exactly
 
+        op1 = rbigint.fromlong(overflowing-1)
         op2 = rbigint.fromlong(-1)
         f = op1.truediv(op2)
         assert f == -1.7976931348623157e+308    # exactly
 
+        op1 = rbigint.fromlong(-overflowing+1)
+        op2 = rbigint.fromlong(-1)
+        f = op1.truediv(op2)
+        assert f == +1.7976931348623157e+308    # exactly
+
         op1 = rbigint.fromlong(overflowing)
         op2 = rbigint.fromlong(1)
         py.test.raises(OverflowError, op1.truediv, op2)

diff --git a/pypy/rlib/rbigint.py b/pypy/rlib/rbigint.py
--- a/pypy/rlib/rbigint.py
+++ b/pypy/rlib/rbigint.py
@@ -511,12 +511,12 @@
         z._normalize()
         return z
 
-    def rshift(self, int_other):
+    def rshift(self, int_other, dont_invert=False):
         if int_other < 0:
             raise ValueError("negative shift count")
         elif int_other == 0:
             return self
-        if self.sign == -1:
+        if self.sign == -1 and not dont_invert:
             a1 = self.invert()
             a2 = a1.rshift(int_other)
             return a2.invert()
@@ -1497,7 +1497,7 @@
     if shift <= 0:
         x = a.lshift(-shift)
     else:
-        x = a.rshift(shift)
+        x = a.rshift(shift, dont_invert=True)
         # set inexact if any of the bits shifted out is nonzero
         if not a.eq(x.lshift(shift)):
             inexact = True

From commits-noreply at bitbucket.org  Fri Feb 11 10:10:20 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri, 11 Feb 2011 10:10:20 +0100 (CET)
Subject: [pypy-svn] pypy default: Merge heads
Message-ID: <20110211091020.6779A282C1B@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41815:5844cf2390e9
Date: 2011-02-11 10:05 +0100
http://bitbucket.org/pypy/pypy/changeset/5844cf2390e9/

Log:	Merge heads


From commits-noreply at bitbucket.org  Fri Feb 11 10:23:52 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Fri, 11 Feb 2011 10:23:52 +0100 (CET)
Subject: [pypy-svn] pypy numpy-exp: Move viewloops one level up
Message-ID: <20110211092352.9BE6A282BE9@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: numpy-exp
Changeset: r41816:5de5a9da40d1
Date: 2011-02-11 11:11 +0200
http://bitbucket.org/pypy/pypy/changeset/5de5a9da40d1/

Log:	Move viewloops one level up

diff --git a/pypy/jit/conftest.py b/pypy/jit/conftest.py
--- a/pypy/jit/conftest.py
+++ b/pypy/jit/conftest.py
@@ -5,7 +5,4 @@
     group.addoption('--slow', action="store_true",
            default=False, dest="run_slow_tests",
            help="run all the compiled tests (instead of just a few)")
-    group.addoption('--viewloops', action="store_true",
-           default=False, dest="viewloops",
-           help="show only the compiled loops")
 

diff --git a/pypy/conftest.py b/pypy/conftest.py
--- a/pypy/conftest.py
+++ b/pypy/conftest.py
@@ -41,6 +41,10 @@
     group.addoption('-P', '--platform', action="callback", type="string",
            default="host", callback=_set_platform,
            help="set up tests to use specified platform as compile/run target")
+    group = parser.getgroup("JIT options")
+    group.addoption('--viewloops', action="store_true",
+           default=False, dest="viewloops",
+           help="show only the compiled loops")
 
 def pytest_sessionstart():
     # have python subprocesses avoid startup customizations by default

From commits-noreply at bitbucket.org  Fri Feb 11 10:23:54 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Fri, 11 Feb 2011 10:23:54 +0100 (CET)
Subject: [pypy-svn] pypy numpy-exp: Add virtualizables
Message-ID: <20110211092354.47E74282BE9@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: numpy-exp
Changeset: r41817:f889b5f9ad62
Date: 2011-02-11 11:12 +0200
http://bitbucket.org/pypy/pypy/changeset/f889b5f9ad62/

Log:	Add virtualizables

diff --git a/pypy/module/micronumpy/numarray.py b/pypy/module/micronumpy/numarray.py
--- a/pypy/module/micronumpy/numarray.py
+++ b/pypy/module/micronumpy/numarray.py
@@ -9,49 +9,69 @@
 TP = lltype.GcArray(lltype.Float)
 
 numpy_driver = jit.JitDriver(greens = ['bytecode_pos', 'bytecode'],
-                             reds = ['result_size', 'i', 'input_pos',
-                                     'valuestackdepth', 'valuestack',
-                                     'input', 'result'])
+                             reds = ['result_size', 'i', 'frame',
+                                     'result'],
+                             virtualizables = ['frame'])
+
+class ComputationFrame(object):
+    _virtualizable2_ = ['valuestackdepth', 'valuestack[*]', 'local_pos',
+                        'locals[*]']
+    
+    def __init__(self, input):
+        self.valuestackdepth = 0
+        self.valuestack = [0.0] * len(input)
+        self.locals = input[:]
+        self.local_pos = len(input)
+
+    def getlocal(self):
+        p = self.local_pos - 1
+        assert p >= 0
+        res = self.locals[p]
+        self.local_pos = p
+        return res
+
+    def popvalue(self):
+        v = self.valuestackdepth - 1
+        assert v >= 0
+        res = self.valuestack[v]
+        self.valuestackdepth = v
+        return res
+
+    def pushvalue(self, v):
+        self.valuestack[self.valuestackdepth] = v
+        self.valuestackdepth += 1
 
 def compute(bytecode, input):
     result_size = input[0].size
     result = SingleDimArray(result_size)
     bytecode_pos = len(bytecode) - 1
-    input_pos = len(input) - 1
-    valuestack = [0.0] * len(input)
-    valuestackdepth = 0
     i = 0
+    frame = ComputationFrame(input)
     while i < result_size:
         numpy_driver.jit_merge_point(bytecode=bytecode, result=result,
                                      result_size=result_size,
-                                     valuestackdepth=valuestackdepth,
-                                     valuestack=valuestack,
-                                     input=input, input_pos=input_pos, i=i,
+                                     i=i, frame=frame,
                                      bytecode_pos=bytecode_pos)
         if bytecode_pos == -1:
             bytecode_pos = len(bytecode) - 1
-            input_pos = len(input) - 1
-            result.storage[i] = valuestack[0]
-            valuestack = [0.0] * len(input)
-            valuestackdepth = 0
+            frame.local_pos = len(frame.locals)
+            result.storage[i] = frame.valuestack[0]
+            frame.valuestackdepth = 0
             i += 1
             numpy_driver.can_enter_jit(bytecode=bytecode, result=result,
                                        result_size=result_size,
-                                       valuestackdepth=valuestackdepth,
-                                       valuestack=valuestack,
-                                       input=input, input_pos=input_pos, i=i,
+                                       i=i, frame=frame,
                                        bytecode_pos=bytecode_pos)
         else:
             opcode = bytecode[bytecode_pos]
             if opcode == 'l':
-                valuestack[valuestackdepth] = input[input_pos].storage[i]
-                valuestackdepth += 1
-                input_pos -= 1
+                val = frame.getlocal().storage[i]
+                frame.valuestack[frame.valuestackdepth] = val
+                frame.valuestackdepth += 1
             elif opcode == 'a':
-                a = valuestack[valuestackdepth - 1]
-                b = valuestack[valuestackdepth - 2]
-                valuestack[valuestackdepth - 2] = a + b
-                valuestackdepth -= 1
+                b = frame.popvalue()
+                a = frame.popvalue()
+                frame.pushvalue(a + b)
             else:
                 raise NotImplementedError
             bytecode_pos -= 1

diff --git a/pypy/module/micronumpy/test/test_jit.py b/pypy/module/micronumpy/test/test_jit.py
--- a/pypy/module/micronumpy/test/test_jit.py
+++ b/pypy/module/micronumpy/test/test_jit.py
@@ -21,4 +21,4 @@
                 v = ar
             return v.force().storage[3]
 
-        self.meta_interp(f, [5])
+        self.meta_interp(f, [5], listops=True, backendopt=True)

From commits-noreply at bitbucket.org  Fri Feb 11 10:23:56 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Fri, 11 Feb 2011 10:23:56 +0100 (CET)
Subject: [pypy-svn] pypy numpy-exp: finish the test
Message-ID: <20110211092356.09420282BE9@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: numpy-exp
Changeset: r41818:9c76b99f319d
Date: 2011-02-11 11:23 +0200
http://bitbucket.org/pypy/pypy/changeset/9c76b99f319d/

Log:	finish the test

diff --git a/pypy/module/micronumpy/test/test_jit.py b/pypy/module/micronumpy/test/test_jit.py
--- a/pypy/module/micronumpy/test/test_jit.py
+++ b/pypy/module/micronumpy/test/test_jit.py
@@ -22,3 +22,6 @@
             return v.force().storage[3]
 
         self.meta_interp(f, [5], listops=True, backendopt=True)
+        self.check_loops({'getarrayitem_gc': 2, 'float_add': 1,
+                          'setarrayitem_gc': 1, 'int_add': 1,
+                          'int_lt': 1, 'guard_true': 1, 'jump': 1})

From commits-noreply at bitbucket.org  Fri Feb 11 11:42:19 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Fri, 11 Feb 2011 11:42:19 +0100 (CET)
Subject: [pypy-svn] pypy numpy-exp: Add an essential hint
Message-ID: <20110211104219.D4F17282BE9@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: numpy-exp
Changeset: r41819:3910329595cc
Date: 2011-02-11 12:41 +0200
http://bitbucket.org/pypy/pypy/changeset/3910329595cc/

Log:	Add an essential hint

diff --git a/pypy/module/micronumpy/numarray.py b/pypy/module/micronumpy/numarray.py
--- a/pypy/module/micronumpy/numarray.py
+++ b/pypy/module/micronumpy/numarray.py
@@ -18,6 +18,7 @@
                         'locals[*]']
     
     def __init__(self, input):
+        self = jit.hint(self, access_directly=True, fresh_virtualizable=True) 
         self.valuestackdepth = 0
         self.valuestack = [0.0] * len(input)
         self.locals = input[:]

From antocuni at codespeak.net  Fri Feb 11 11:49:03 2011
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Fri, 11 Feb 2011 11:49:03 +0100 (CET)
Subject: [pypy-svn] r80336 - pypy/extradoc/talk/dyncon2011
Message-ID: <20110211104903.950B12A2002@codespeak.net>

Author: antocuni
Date: Fri Feb 11 11:48:59 2011
New Revision: 80336

Added:
   pypy/extradoc/talk/dyncon2011/
   pypy/extradoc/talk/dyncon2011/abstract.txt   (contents, props changed)
Log:
abstract for my talk at dyncon


Added: pypy/extradoc/talk/dyncon2011/abstract.txt
==============================================================================
--- (empty file)
+++ pypy/extradoc/talk/dyncon2011/abstract.txt	Fri Feb 11 11:48:59 2011
@@ -0,0 +1,19 @@
+PyPy: JIT (and more) for free
+=============================
+
+The PyPy project has recently gathered a lot of attention for its progress in
+speeding up the Python language -- it is the fastest, most compatible and most
+stable 'alternative? Python interpreter.  No longer merely a research
+curiosity, PyPy is now suitable for production use.  
+
+PyPy is not only a Python implementation, but also a general framework to
+implement dynamic languages and get many features for free, like a JIT
+compiler and a fast GC.  Examples of other languages implemented in PyPy are
+Smalltalk, Prolog and Javascript.
+
+This talk will be divided in two parts, with different target audience:
+
+  * for Python developers: why (and how) to use PyPy instead of CPython?
+
+  * for language implementors: why should you use PyPy to implement your
+    revolutionary language?


From antocuni at codespeak.net  Fri Feb 11 12:02:12 2011
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Fri, 11 Feb 2011 12:02:12 +0100 (CET)
Subject: [pypy-svn] r80337 - pypy/extradoc/talk/dyncon2011
Message-ID: <20110211110212.2E974282BE9@codespeak.net>

Author: antocuni
Date: Fri Feb 11 12:02:09 2011
New Revision: 80337

Modified:
   pypy/extradoc/talk/dyncon2011/abstract.txt
Log:
add "dynamic"

Modified: pypy/extradoc/talk/dyncon2011/abstract.txt
==============================================================================
--- pypy/extradoc/talk/dyncon2011/abstract.txt	(original)
+++ pypy/extradoc/talk/dyncon2011/abstract.txt	Fri Feb 11 12:02:09 2011
@@ -16,4 +16,4 @@
   * for Python developers: why (and how) to use PyPy instead of CPython?
 
   * for language implementors: why should you use PyPy to implement your
-    revolutionary language?
+    revolutionary dynamic language?


From antocuni at codespeak.net  Fri Feb 11 12:15:24 2011
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Fri, 11 Feb 2011 12:15:24 +0100 (CET)
Subject: [pypy-svn] r80338 - pypy/extradoc/talk/dyncon2011
Message-ID: <20110211111524.401782A2002@codespeak.net>

Author: antocuni
Date: Fri Feb 11 12:15:22 2011
New Revision: 80338

Modified:
   pypy/extradoc/talk/dyncon2011/abstract.txt
Log:
underline that the other languages are experimental, else we might give too much hope on them

Modified: pypy/extradoc/talk/dyncon2011/abstract.txt
==============================================================================
--- pypy/extradoc/talk/dyncon2011/abstract.txt	(original)
+++ pypy/extradoc/talk/dyncon2011/abstract.txt	Fri Feb 11 12:15:22 2011
@@ -8,8 +8,10 @@
 
 PyPy is not only a Python implementation, but also a general framework to
 implement dynamic languages and get many features for free, like a JIT
-compiler and a fast GC.  Examples of other languages implemented in PyPy are
-Smalltalk, Prolog and Javascript.
+compiler and a fast GC.  In addition to the Python interpreter, which is the
+main focus of the core development team, there are experimental Smalltalk,
+Prolog and Javascript implementations (and even a Gameboy emulator) for PyPy,
+with different degrees of completeness.
 
 This talk will be divided in two parts, with different target audience:
 


From antocuni at codespeak.net  Fri Feb 11 12:25:04 2011
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Fri, 11 Feb 2011 12:25:04 +0100 (CET)
Subject: [pypy-svn] r80339 - pypy/extradoc/talk/dyncon2011
Message-ID: <20110211112504.9E0B0282BE9@codespeak.net>

Author: antocuni
Date: Fri Feb 11 12:25:02 2011
New Revision: 80339

Modified:
   pypy/extradoc/talk/dyncon2011/abstract.txt
Log:
expand the abstract by stealing bits from the ustour ones


Modified: pypy/extradoc/talk/dyncon2011/abstract.txt
==============================================================================
--- pypy/extradoc/talk/dyncon2011/abstract.txt	(original)
+++ pypy/extradoc/talk/dyncon2011/abstract.txt	Fri Feb 11 12:25:02 2011
@@ -4,18 +4,35 @@
 The PyPy project has recently gathered a lot of attention for its progress in
 speeding up the Python language -- it is the fastest, most compatible and most
 stable 'alternative? Python interpreter.  No longer merely a research
-curiosity, PyPy is now suitable for production use.  
+curiosity, PyPy is now suitable for production use.  Since 2009, the PyPy
+project has received funding from the Eurostars Eureka program, which targets
+small European firms which produce research.  (See:
+http://www.eurostars-eureka.eu/)
 
-PyPy is not only a Python implementation, but also a general framework to
-implement dynamic languages and get many features for free, like a JIT
-compiler and a fast GC.  In addition to the Python interpreter, which is the
-main focus of the core development team, there are experimental Smalltalk,
-Prolog and Javascript implementations (and even a Gameboy emulator) for PyPy,
-with different degrees of completeness.
+We will discuss what the PyPy project has achieved, with a particular
+focus on the past two years' work  in the area of dynamic (Just-in-Time) 
+interpreters:
+
+* most Python benchmarks run much faster than with CPython or Psyco
+* the real-world PyPy compiler toolchain itself (200 KLocs) runs twice as fast
+* already supports 64bit and is in the process of supporting ARM
+* full compatibility with  CPython (more than Jython/IronPython)
+* full (and JIT-ed) ctypes support to call C libraries from Python
+* supports Stackless Python (in-progress)
+* new "cpyext" layer which integrates existing CPython C extensions
+* an experimental super-fast JIT-compilation of calls to C++ libraries
+
+Moreover, PyPy is not only a Python implementation, but also a general
+framework to implement dynamic languages and get many features for free, such
+as a JIT compiler and a fast GC.  In addition to the Python interpreter, which
+is the main focus of the core development team, there are experimental
+Smalltalk, Prolog and Javascript implementations (and even a Gameboy emulator)
+for PyPy, with different degrees of completeness.
 
 This talk will be divided in two parts, with different target audience:
 
   * for Python developers: why (and how) to use PyPy instead of CPython?
 
-  * for language implementors: why should you use PyPy to implement your
+  * for language implementors: why should you use PyPy for implementing your
     revolutionary dynamic language?
+


From antocuni at codespeak.net  Fri Feb 11 12:29:25 2011
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Fri, 11 Feb 2011 12:29:25 +0100 (CET)
Subject: [pypy-svn] r80340 - pypy/extradoc/talk/dyncon2011
Message-ID: <20110211112925.0D8322A2002@codespeak.net>

Author: antocuni
Date: Fri Feb 11 12:29:24 2011
New Revision: 80340

Modified:
   pypy/extradoc/talk/dyncon2011/abstract.txt
Log:
mention JVM and .NET backends

Modified: pypy/extradoc/talk/dyncon2011/abstract.txt
==============================================================================
--- pypy/extradoc/talk/dyncon2011/abstract.txt	(original)
+++ pypy/extradoc/talk/dyncon2011/abstract.txt	Fri Feb 11 12:29:24 2011
@@ -19,6 +19,7 @@
 * full compatibility with  CPython (more than Jython/IronPython)
 * full (and JIT-ed) ctypes support to call C libraries from Python
 * supports Stackless Python (in-progress)
+* experimental JVM and .NET backend (to get alternatives to Jython and IronPython)
 * new "cpyext" layer which integrates existing CPython C extensions
 * an experimental super-fast JIT-compilation of calls to C++ libraries
 


From commits-noreply at bitbucket.org  Fri Feb 11 13:38:24 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Fri, 11 Feb 2011 13:38:24 +0100 (CET)
Subject: [pypy-svn] pypy default: Kill the outdated,
	never-finished 'readline' module.
Message-ID: <20110211123824.BD8B3282BE9@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41820:224cb70cb88e
Date: 2011-02-11 13:37 +0100
http://bitbucket.org/pypy/pypy/changeset/224cb70cb88e/

Log:	Kill the outdated, never-finished 'readline' module. We are using
	lib_pypy/readline.py nowadays, which is a wrapper around
	lib_pypy/pyrepl/.

diff --git a/pypy/module/readline/c_readline.py b/pypy/module/readline/c_readline.py
deleted file mode 100644
--- a/pypy/module/readline/c_readline.py
+++ /dev/null
@@ -1,77 +0,0 @@
-from pypy.rpython.tool import rffi_platform as platform
-from pypy.rpython.lltypesystem import lltype, rffi
-from pypy.interpreter.error import OperationError
-from pypy.interpreter.gateway import ObjSpace, interp2app
-from pypy.translator.tool.cbuild import ExternalCompilationInfo
-
-# On various platforms, linking only with libreadline is not enough;
-# we also need to link with some variant of curses or libtermcap.
-# We follow the logic of CPython below.
-def try_with_lib(extralibs, **kwds):
-    global most_recent_error
-    # at least on Gentoo Linux, readline.h doesn't compile if stdio.h is not
-    # included before
-    eci = ExternalCompilationInfo(
-        includes = ["stdio.h", "readline/readline.h", "readline/history.h"],
-        libraries = extralibs + ['readline'],
-        )
-    try:
-        platform.verify_eci(eci)
-        return eci
-    except platform.CompilationError, e:
-        most_recent_error = e
-        return None
-
-eci = (try_with_lib([]) or
-       try_with_lib(['ncursesw']) or
-       try_with_lib(['ncurses']) or
-       try_with_lib(['curses']) or
-       try_with_lib(['termcap'], library_dirs=['/usr/lib/termcap']))
-if eci is None:
-    raise most_recent_error
-
-# ____________________________________________________________
-
-def external(name, args, result):
-    return rffi.llexternal(name, args, result, compilation_info=eci)
-
-# get a binding to  c library functions and define their args and return types
-# char *readline(char *)
-c_readline = external('readline', [rffi.CCHARP], rffi.CCHARP)
-
-# void rl_initiliaze(void)
-c_rl_initialize = external('rl_initialize', [], lltype.Void)
-
-# void using_history(void)
-c_using_history = external('using_history', [], lltype.Void)
-
-# void add_history(const char *)
-c_add_history = external('add_history', [rffi.CCHARP], lltype.Void)
-
-#------------------------------------------------------------
-# special initialization of readline 
-
-class ReadlineState(object):
-    lastline = ""        # XXX possibly temporary hack
-readlinestate = ReadlineState()
-
-def setup_readline(space, w_module):
-    c_using_history()
-    # XXX CPython initializes more stuff here
-    c_rl_initialize()
-    # install sys.__raw_input__, a hook that will be used by raw_input()
-    space.setitem(space.sys.w_dict, space.wrap('__raw_input__'),
-                  space.wrap(app_readline_func))
-
-def readline_func(space, prompt):
-    ll_res = c_readline(prompt)
-    if not ll_res:
-        raise OperationError(space.w_EOFError, space.w_None)
-    res = rffi.charp2str(ll_res)
-    if res and res != readlinestate.lastline:
-        readlinestate.lastline = res
-        c_add_history(res)
-    return space.wrap(res)
-
-readline_func.unwrap_spec = [ObjSpace, str]
-app_readline_func = interp2app(readline_func)

diff --git a/pypy/module/readline/test/test_c_readline.py b/pypy/module/readline/test/test_c_readline.py
deleted file mode 100644
--- a/pypy/module/readline/test/test_c_readline.py
+++ /dev/null
@@ -1,16 +0,0 @@
-"""
-Directly test the basic ctypes wrappers.
-"""
-
-import py
-from pypy import conftest; conftest.translation_test_so_skip_if_appdirect()
-from pypy.rpython.tool import rffi_platform as platform
-
-try:
-    from pypy.module.readline import c_readline
-except platform.CompilationError, e:
-    py.test.skip(e)
-
-
-def test_basic_import():
-    c_readline.c_rl_initialize()

diff --git a/pypy/module/readline/app_stub.py b/pypy/module/readline/app_stub.py
deleted file mode 100644
--- a/pypy/module/readline/app_stub.py
+++ /dev/null
@@ -1,13 +0,0 @@
-# NOT_RPYTHON
-
-def stub(*args, **kwds):
-    import warnings
-    warnings.warn("the 'readline' module is only a stub so far")
-
-def stub_str(*args, **kwds):
-    stub()
-    return ''
-
-def stub_int(*args, **kwds):
-    stub()
-    return 0

diff --git a/pypy/module/readline/test/__init__.py b/pypy/module/readline/test/__init__.py
deleted file mode 100644
--- a/pypy/module/readline/test/__init__.py
+++ /dev/null
@@ -1,1 +0,0 @@
-#

diff --git a/pypy/module/readline/test/test_with_pypy.py b/pypy/module/readline/test/test_with_pypy.py
deleted file mode 100644
--- a/pypy/module/readline/test/test_with_pypy.py
+++ /dev/null
@@ -1,27 +0,0 @@
-"""
-Test the readline library on top of PyPy.  The following tests run
-in the PyPy interpreter, itself running on top of CPython
-"""
-
-import py
-from pypy.conftest import gettestobjspace
-from pypy.rpython.tool import rffi_platform as platform
-
-try:
-    from pypy.module.readline import c_readline
-except platform.CompilationError, e:
-    py.test.skip(e)
-
-
-class AppTestReadline:
-
-    def setup_class(cls):
-        # enable usage of the readline mixedmodule
-        space = gettestobjspace(usemodules=('readline',))
-        cls.space = space
-
-    def test_basic_import(self):
-        # this is interpreted by PyPy
-        import readline 
-        readline.readline
-        # XXX test more

diff --git a/pypy/module/readline/interp_readline.py b/pypy/module/readline/interp_readline.py
deleted file mode 100644
--- a/pypy/module/readline/interp_readline.py
+++ /dev/null
@@ -1,23 +0,0 @@
-# this is a sketch of how one might one day be able to define a pretty simple
-# ctypes-using module, suitable for feeding to the ext-compiler
-
-from pypy.interpreter.baseobjspace import ObjSpace
-
-from pypy.module.readline import c_readline
-from pypy.rpython.lltypesystem import rffi
-
-#------------------------------------------------------------
-# exported API  (see interpleveldefs in __init__.py) 
-#
-def readline(space, prompt):
-    return space.wrap(rffi.charp2str(c_readline.c_readline(prompt)))
-readline.unwrap_spec = [ObjSpace, str]
-
-def setcompleter(space, w_callback):
-    """Set or remove the completer function.
-    The function is called as function(text, state),
-    for state in 0, 1, 2, ..., until it returns a non-string.
-    It should return the next possible completion starting with 'text'.
-    """ 
-    # XXX set internal completion function 
-    

diff --git a/pypy/module/readline/__init__.py b/pypy/module/readline/__init__.py
deleted file mode 100644
--- a/pypy/module/readline/__init__.py
+++ /dev/null
@@ -1,45 +0,0 @@
-# this is a sketch of how one might one day be able to define a pretty simple
-# ctypes-using module, suitable for feeding to the ext-compiler
-
-from pypy.interpreter.mixedmodule import MixedModule
-
-# XXX raw_input needs to check for space.readline_func and use
-# it if its there 
-
-class Module(MixedModule):
-    """Importing this module enables command line editing using GNU readline."""
-    # the above line is the doc string of the translated module  
-
-    def setup_after_space_initialization(self):
-        from pypy.module.readline import c_readline 
-        c_readline.setup_readline(self.space, self)
-
-    interpleveldefs = {
-        'readline'    : 'interp_readline.readline',
-    }
-
-    appleveldefs = {
-        'parse_and_bind':     'app_stub.stub',
-        'get_line_buffer':    'app_stub.stub_str',
-        'insert_text':        'app_stub.stub',
-        'read_init_file':     'app_stub.stub',
-        'read_history_file':  'app_stub.stub',
-        'write_history_file': 'app_stub.stub',
-        'clear_history':      'app_stub.stub',
-        'get_history_length': 'app_stub.stub_int',
-        'set_history_length': 'app_stub.stub',
-        'get_current_history_length': 'app_stub.stub_int',
-        'get_history_item':           'app_stub.stub_str',
-        'remove_history_item':        'app_stub.stub',
-        'replace_history_item':       'app_stub.stub',
-        'redisplay':                  'app_stub.stub',
-        'set_startup_hook':           'app_stub.stub',
-        'set_pre_input_hook':         'app_stub.stub',
-        'set_completer':      'app_stub.stub',
-        'get_completer':      'app_stub.stub',
-        'get_begidx':         'app_stub.stub_int',
-        'get_endidx':         'app_stub.stub_int',
-        'set_completer_delims':       'app_stub.stub',
-        'get_completer_delims':       'app_stub.stub_str',
-        'add_history':        'app_stub.stub',
-    }

From commits-noreply at bitbucket.org  Fri Feb 11 15:23:18 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Fri, 11 Feb 2011 15:23:18 +0100 (CET)
Subject: [pypy-svn] pypy default: file.__exit__ should call self.close,
	not file.close.  Also brought some tests into the 21st century.
Message-ID: <20110211142318.291152A2002@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41821:f5278f05c3d1
Date: 2011-02-11 09:23 -0500
http://bitbucket.org/pypy/pypy/changeset/f5278f05c3d1/

Log:	file.__exit__ should call self.close, not file.close. Also brought
	some tests into the 21st century.

diff --git a/pypy/module/_file/interp_file.py b/pypy/module/_file/interp_file.py
--- a/pypy/module/_file/interp_file.py
+++ b/pypy/module/_file/interp_file.py
@@ -6,7 +6,7 @@
 from pypy.module._file.interp_stream import StreamErrors, wrap_streamerror
 from pypy.module.posix.interp_posix import dispatch_filename
 from pypy.interpreter.error import OperationError, operationerrfmt
-from pypy.interpreter.gateway import ObjSpace, W_Root, Arguments
+from pypy.interpreter.gateway import ObjSpace, W_Root, Arguments, unwrap_spec
 from pypy.interpreter.typedef import TypeDef, GetSetProperty
 from pypy.interpreter.typedef import interp_attrproperty, make_weakref_descr
 from pypy.interpreter.typedef import interp_attrproperty_w
@@ -95,14 +95,13 @@
         self.fdopenstream(stream, fd, mode)
 
     def direct___enter__(self):
-        if self.stream is None:
-            space = self.space
-            raise OperationError(space.w_ValueError,
-                                 space.wrap('I/O operation on closed file'))
+        self.check_closed()
         return self
 
-    def direct___exit__(self, __args__):
-        self.direct_close()
+    @unwrap_spec("self", Arguments)
+    def file__exit__(self, __args__):
+        """__exit__(*excinfo) -> None. Closes the file."""
+        self.space.call_method(self, "close")
         # can't return close() value
         return None
 
@@ -286,9 +285,6 @@
 
     _decl(locals(), "__enter__", ['self'], """__enter__() -> self.""")
 
-    _decl(locals(), "__exit__", ['self', Arguments], 
-        """__exit__(*excinfo) -> None. Closes the file.""")
-
     _decl(locals(), "close", ['self'],
         """close() -> None or (perhaps) an integer.  Close the file.
 
@@ -511,6 +507,7 @@
     __repr__ = interp2app(W_File.file__repr__),
     readinto = interp2app(W_File.file_readinto),
     writelines = interp2app(W_File.file_writelines),
+    __exit__ = interp2app(W_File.file__exit__),
     __weakref__ = make_weakref_descr(W_File),
     **dict([(name, interp2app(getattr(W_File, 'file_' + name)))
                 for name in W_File._exposed_method_names])

diff --git a/pypy/module/_file/test/test_file.py b/pypy/module/_file/test/test_file.py
--- a/pypy/module/_file/test/test_file.py
+++ b/pypy/module/_file/test/test_file.py
@@ -312,21 +312,30 @@
         assert f.closed
 
     def test_file_and_with_statement(self):
-        s1 = """from __future__ import with_statement
-with self.file(self.temppath, 'w') as f:
-    f.write('foo')
-"""
-        exec s1
+        with self.file(self.temppath, 'w') as f:
+            f.write('foo')
         assert f.closed
         
-        s2 = """from __future__ import with_statement
-with self.file(self.temppath, 'r') as f:
-    s = f.readline()
-"""
-    
-        exec s2
+        with self.file(self.temppath, 'r') as f:
+            s = f.readline()
+
         assert s == "foo"
         assert f.closed
+    
+    def test_subclass_with(self):
+        file = self.file
+        class C(file):
+            def __init__(self, *args, **kwargs):
+                self.subclass_closed = False
+                file.__init__(self, *args, **kwargs)
+            
+            def close(self):
+                self.subclass_closed = True
+                file.close(self)
+        
+        with C(self.temppath, 'w') as f:
+            pass
+        assert f.subclass_closed
 
 def test_flush_at_exit():
     from pypy import conftest

From commits-noreply at bitbucket.org  Fri Feb 11 18:17:53 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Fri, 11 Feb 2011 18:17:53 +0100 (CET)
Subject: [pypy-svn] pypy default: Added sys.float_repr_style.
Message-ID: <20110211171753.61579282BAD@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41822:7d23136597fd
Date: 2011-02-11 12:17 -0500
http://bitbucket.org/pypy/pypy/changeset/7d23136597fd/

Log:	Added sys.float_repr_style.

diff --git a/pypy/module/sys/__init__.py b/pypy/module/sys/__init__.py
--- a/pypy/module/sys/__init__.py
+++ b/pypy/module/sys/__init__.py
@@ -79,6 +79,7 @@
 
         'float_info'            : 'system.get_float_info(space)',
         'long_info'             : 'system.get_long_info(space)',
+        'float_repr_style'      : 'system.get_float_repr_style(space)'
         }
 
     if sys.platform == 'win32':

diff --git a/pypy/module/sys/system.py b/pypy/module/sys/system.py
--- a/pypy/module/sys/system.py
+++ b/pypy/module/sys/system.py
@@ -1,6 +1,7 @@
 """Information about the current system."""
 from pypy.interpreter import gateway
 from pypy.rlib import rfloat, rbigint
+from pypy.rlib.rarithmetic import USE_SHORT_FLOAT_REPR
 from pypy.rpython.lltypesystem import rffi
 
 
@@ -56,3 +57,9 @@
     ]
     w_long_info = app.wget(space, "long_info")
     return space.call_function(w_long_info, space.newtuple(info_w))
+
+def get_float_repr_style(space):
+    if USE_SHORT_FLOAT_REPR:
+        return space.wrap("short")
+    else:
+        return space.wrap("legacy")
\ No newline at end of file

diff --git a/pypy/module/sys/test/test_sysmodule.py b/pypy/module/sys/test/test_sysmodule.py
--- a/pypy/module/sys/test/test_sysmodule.py
+++ b/pypy/module/sys/test/test_sysmodule.py
@@ -517,3 +517,10 @@
             pass
         sys.settrace(None)
         assert found == ['call', 'line', 'exception', 'return']
+
+    def test_float_repr_style(self):
+        import sys
+
+        # If this ever actually becomes a compilation option this test should
+        # be changed.
+        assert sys.float_repr_style == "short"
\ No newline at end of file

From commits-noreply at bitbucket.org  Fri Feb 11 18:29:33 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri, 11 Feb 2011 18:29:33 +0100 (CET)
Subject: [pypy-svn] pypy default: Pypy's implementation of file() is similar
	to the io	module and allows mixing next() and readline().
Message-ID: <20110211172933.C86352A2002@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41823:957f4c379c91
Date: 2011-02-11 11:08 +0100
http://bitbucket.org/pypy/pypy/changeset/957f4c379c91/

Log:	Pypy's implementation of file() is similar to the io module and
	allows mixing next() and readline(). Consistency of the results is
	checked in subsequent tests.

diff --git a/lib-python/modified-2.7.0/test/test_file2k.py b/lib-python/modified-2.7.0/test/test_file2k.py
--- a/lib-python/modified-2.7.0/test/test_file2k.py
+++ b/lib-python/modified-2.7.0/test/test_file2k.py
@@ -337,8 +337,9 @@
                 except ValueError:
                     pass
                 else:
-                    self.fail("%s%r after next() didn't raise ValueError" %
-                                     (methodname, args))
+                    if test_support.check_impl_detail():
+                        self.fail("%s%r after next() didn't raise ValueError" %
+                                  (methodname, args))
                 f.close()
 
             # Test to see if harmless (by accident) mixing of read* and

From commits-noreply at bitbucket.org  Fri Feb 11 18:29:34 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri, 11 Feb 2011 18:29:34 +0100 (CET)
Subject: [pypy-svn] pypy default: Provide arguments to methods that need
	them.
Message-ID: <20110211172934.5BB7F2A2002@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41824:bec074681c44
Date: 2011-02-11 11:15 +0100
http://bitbucket.org/pypy/pypy/changeset/bec074681c44/

Log:	Provide arguments to methods that need them. Unlike CPython, PyPy
	checks the arguments before calling the function...

diff --git a/lib-python/modified-2.7.0/test/test_file2k.py b/lib-python/modified-2.7.0/test/test_file2k.py
--- a/lib-python/modified-2.7.0/test/test_file2k.py
+++ b/lib-python/modified-2.7.0/test/test_file2k.py
@@ -117,8 +117,12 @@
 
         for methodname in methods:
             method = getattr(self.f, methodname)
+            args = {'readinto': (bytearray(''),),
+                    'seek':     (0,),
+                    'write':    ('',),
+                    }.get(methodname, ())
             # should raise on closed file
-            self.assertRaises(ValueError, method)
+            self.assertRaises(ValueError, method, *args)
         with test_support.check_py3k_warnings():
             for methodname in deprecated_methods:
                 method = getattr(self.f, methodname)

From commits-noreply at bitbucket.org  Fri Feb 11 18:29:35 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri, 11 Feb 2011 18:29:35 +0100 (CET)
Subject: [pypy-svn] pypy default: Implement CPython issue1764407. The test
	was already	written, uncomment it!
Message-ID: <20110211172935.2CC802A2002@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41825:9376e51b04d2
Date: 2011-02-11 13:36 +0100
http://bitbucket.org/pypy/pypy/changeset/9376e51b04d2/

Log:	Implement CPython issue1764407. The test was already written,
	uncomment it!

diff --git a/pypy/translator/goal/app_main.py b/pypy/translator/goal/app_main.py
--- a/pypy/translator/goal/app_main.py
+++ b/pypy/translator/goal/app_main.py
@@ -506,7 +506,7 @@
             # handle the "-m" command
             def run_it():
                 import runpy
-                runpy.run_module(sys.argv[0], None, '__main__', True)
+                runpy._run_module_as_main(sys.argv[0])
             success = run_toplevel(run_it)
         elif run_stdin:
             # handle the case where no command/filename/module is specified

diff --git a/pypy/translator/goal/test2/test_app_main.py b/pypy/translator/goal/test2/test_app_main.py
--- a/pypy/translator/goal/test2/test_app_main.py
+++ b/pypy/translator/goal/test2/test_app_main.py
@@ -391,10 +391,9 @@
         child.expect(re.escape('File: ' + p))
         child.expect(re.escape('Argv: ' + repr([p, 'extra'])))
         child.expect('>>> ')
-        #XXX the following doesn't work on CPython 2.5 either
-        #child.sendline('somevalue')
-        #child.expect(re.escape(repr("foobar")))
-        #child.expect('>>> ')
+        child.sendline('somevalue')
+        child.expect(re.escape(repr("foobar")))
+        child.expect('>>> ')
         child.sendline('import sys')
         child.sendline('"pypy.translator.goal.test2" in sys.modules')
         child.expect('True')

From commits-noreply at bitbucket.org  Fri Feb 11 18:29:40 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri, 11 Feb 2011 18:29:40 +0100 (CET)
Subject: [pypy-svn] pypy default: merge heads
Message-ID: <20110211172940.10957282BAD@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41826:1f3a9f60c822
Date: 2011-02-11 13:48 +0100
http://bitbucket.org/pypy/pypy/changeset/1f3a9f60c822/

Log:	merge heads

diff --git a/pypy/module/readline/c_readline.py b/pypy/module/readline/c_readline.py
deleted file mode 100644
--- a/pypy/module/readline/c_readline.py
+++ /dev/null
@@ -1,77 +0,0 @@
-from pypy.rpython.tool import rffi_platform as platform
-from pypy.rpython.lltypesystem import lltype, rffi
-from pypy.interpreter.error import OperationError
-from pypy.interpreter.gateway import ObjSpace, interp2app
-from pypy.translator.tool.cbuild import ExternalCompilationInfo
-
-# On various platforms, linking only with libreadline is not enough;
-# we also need to link with some variant of curses or libtermcap.
-# We follow the logic of CPython below.
-def try_with_lib(extralibs, **kwds):
-    global most_recent_error
-    # at least on Gentoo Linux, readline.h doesn't compile if stdio.h is not
-    # included before
-    eci = ExternalCompilationInfo(
-        includes = ["stdio.h", "readline/readline.h", "readline/history.h"],
-        libraries = extralibs + ['readline'],
-        )
-    try:
-        platform.verify_eci(eci)
-        return eci
-    except platform.CompilationError, e:
-        most_recent_error = e
-        return None
-
-eci = (try_with_lib([]) or
-       try_with_lib(['ncursesw']) or
-       try_with_lib(['ncurses']) or
-       try_with_lib(['curses']) or
-       try_with_lib(['termcap'], library_dirs=['/usr/lib/termcap']))
-if eci is None:
-    raise most_recent_error
-
-# ____________________________________________________________
-
-def external(name, args, result):
-    return rffi.llexternal(name, args, result, compilation_info=eci)
-
-# get a binding to  c library functions and define their args and return types
-# char *readline(char *)
-c_readline = external('readline', [rffi.CCHARP], rffi.CCHARP)
-
-# void rl_initiliaze(void)
-c_rl_initialize = external('rl_initialize', [], lltype.Void)
-
-# void using_history(void)
-c_using_history = external('using_history', [], lltype.Void)
-
-# void add_history(const char *)
-c_add_history = external('add_history', [rffi.CCHARP], lltype.Void)
-
-#------------------------------------------------------------
-# special initialization of readline 
-
-class ReadlineState(object):
-    lastline = ""        # XXX possibly temporary hack
-readlinestate = ReadlineState()
-
-def setup_readline(space, w_module):
-    c_using_history()
-    # XXX CPython initializes more stuff here
-    c_rl_initialize()
-    # install sys.__raw_input__, a hook that will be used by raw_input()
-    space.setitem(space.sys.w_dict, space.wrap('__raw_input__'),
-                  space.wrap(app_readline_func))
-
-def readline_func(space, prompt):
-    ll_res = c_readline(prompt)
-    if not ll_res:
-        raise OperationError(space.w_EOFError, space.w_None)
-    res = rffi.charp2str(ll_res)
-    if res and res != readlinestate.lastline:
-        readlinestate.lastline = res
-        c_add_history(res)
-    return space.wrap(res)
-
-readline_func.unwrap_spec = [ObjSpace, str]
-app_readline_func = interp2app(readline_func)

diff --git a/pypy/module/readline/test/test_c_readline.py b/pypy/module/readline/test/test_c_readline.py
deleted file mode 100644
--- a/pypy/module/readline/test/test_c_readline.py
+++ /dev/null
@@ -1,16 +0,0 @@
-"""
-Directly test the basic ctypes wrappers.
-"""
-
-import py
-from pypy import conftest; conftest.translation_test_so_skip_if_appdirect()
-from pypy.rpython.tool import rffi_platform as platform
-
-try:
-    from pypy.module.readline import c_readline
-except platform.CompilationError, e:
-    py.test.skip(e)
-
-
-def test_basic_import():
-    c_readline.c_rl_initialize()

diff --git a/pypy/module/readline/app_stub.py b/pypy/module/readline/app_stub.py
deleted file mode 100644
--- a/pypy/module/readline/app_stub.py
+++ /dev/null
@@ -1,13 +0,0 @@
-# NOT_RPYTHON
-
-def stub(*args, **kwds):
-    import warnings
-    warnings.warn("the 'readline' module is only a stub so far")
-
-def stub_str(*args, **kwds):
-    stub()
-    return ''
-
-def stub_int(*args, **kwds):
-    stub()
-    return 0

diff --git a/pypy/module/readline/test/__init__.py b/pypy/module/readline/test/__init__.py
deleted file mode 100644
--- a/pypy/module/readline/test/__init__.py
+++ /dev/null
@@ -1,1 +0,0 @@
-#

diff --git a/pypy/module/readline/test/test_with_pypy.py b/pypy/module/readline/test/test_with_pypy.py
deleted file mode 100644
--- a/pypy/module/readline/test/test_with_pypy.py
+++ /dev/null
@@ -1,27 +0,0 @@
-"""
-Test the readline library on top of PyPy.  The following tests run
-in the PyPy interpreter, itself running on top of CPython
-"""
-
-import py
-from pypy.conftest import gettestobjspace
-from pypy.rpython.tool import rffi_platform as platform
-
-try:
-    from pypy.module.readline import c_readline
-except platform.CompilationError, e:
-    py.test.skip(e)
-
-
-class AppTestReadline:
-
-    def setup_class(cls):
-        # enable usage of the readline mixedmodule
-        space = gettestobjspace(usemodules=('readline',))
-        cls.space = space
-
-    def test_basic_import(self):
-        # this is interpreted by PyPy
-        import readline 
-        readline.readline
-        # XXX test more

diff --git a/pypy/module/readline/interp_readline.py b/pypy/module/readline/interp_readline.py
deleted file mode 100644
--- a/pypy/module/readline/interp_readline.py
+++ /dev/null
@@ -1,23 +0,0 @@
-# this is a sketch of how one might one day be able to define a pretty simple
-# ctypes-using module, suitable for feeding to the ext-compiler
-
-from pypy.interpreter.baseobjspace import ObjSpace
-
-from pypy.module.readline import c_readline
-from pypy.rpython.lltypesystem import rffi
-
-#------------------------------------------------------------
-# exported API  (see interpleveldefs in __init__.py) 
-#
-def readline(space, prompt):
-    return space.wrap(rffi.charp2str(c_readline.c_readline(prompt)))
-readline.unwrap_spec = [ObjSpace, str]
-
-def setcompleter(space, w_callback):
-    """Set or remove the completer function.
-    The function is called as function(text, state),
-    for state in 0, 1, 2, ..., until it returns a non-string.
-    It should return the next possible completion starting with 'text'.
-    """ 
-    # XXX set internal completion function 
-    

diff --git a/pypy/module/readline/__init__.py b/pypy/module/readline/__init__.py
deleted file mode 100644
--- a/pypy/module/readline/__init__.py
+++ /dev/null
@@ -1,45 +0,0 @@
-# this is a sketch of how one might one day be able to define a pretty simple
-# ctypes-using module, suitable for feeding to the ext-compiler
-
-from pypy.interpreter.mixedmodule import MixedModule
-
-# XXX raw_input needs to check for space.readline_func and use
-# it if its there 
-
-class Module(MixedModule):
-    """Importing this module enables command line editing using GNU readline."""
-    # the above line is the doc string of the translated module  
-
-    def setup_after_space_initialization(self):
-        from pypy.module.readline import c_readline 
-        c_readline.setup_readline(self.space, self)
-
-    interpleveldefs = {
-        'readline'    : 'interp_readline.readline',
-    }
-
-    appleveldefs = {
-        'parse_and_bind':     'app_stub.stub',
-        'get_line_buffer':    'app_stub.stub_str',
-        'insert_text':        'app_stub.stub',
-        'read_init_file':     'app_stub.stub',
-        'read_history_file':  'app_stub.stub',
-        'write_history_file': 'app_stub.stub',
-        'clear_history':      'app_stub.stub',
-        'get_history_length': 'app_stub.stub_int',
-        'set_history_length': 'app_stub.stub',
-        'get_current_history_length': 'app_stub.stub_int',
-        'get_history_item':           'app_stub.stub_str',
-        'remove_history_item':        'app_stub.stub',
-        'replace_history_item':       'app_stub.stub',
-        'redisplay':                  'app_stub.stub',
-        'set_startup_hook':           'app_stub.stub',
-        'set_pre_input_hook':         'app_stub.stub',
-        'set_completer':      'app_stub.stub',
-        'get_completer':      'app_stub.stub',
-        'get_begidx':         'app_stub.stub_int',
-        'get_endidx':         'app_stub.stub_int',
-        'set_completer_delims':       'app_stub.stub',
-        'get_completer_delims':       'app_stub.stub_str',
-        'add_history':        'app_stub.stub',
-    }

From commits-noreply at bitbucket.org  Fri Feb 11 18:29:40 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri, 11 Feb 2011 18:29:40 +0100 (CET)
Subject: [pypy-svn] pypy default: Remove "doc" for deleted module
Message-ID: <20110211172940.E217D282BAD@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41827:59b059f33bb8
Date: 2011-02-11 13:49 +0100
http://bitbucket.org/pypy/pypy/changeset/59b059f33bb8/

Log:	Remove "doc" for deleted module

diff --git a/pypy/doc/config/objspace.usemodules.readline.txt b/pypy/doc/config/objspace.usemodules.readline.txt
deleted file mode 100644
--- a/pypy/doc/config/objspace.usemodules.readline.txt
+++ /dev/null
@@ -1,1 +0,0 @@
-Use the 'readline' module. 

From commits-noreply at bitbucket.org  Fri Feb 11 18:29:43 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri, 11 Feb 2011 18:29:43 +0100 (CET)
Subject: [pypy-svn] pypy default: str.decode('utf-8') must call the decoder
	with	final=True
Message-ID: <20110211172943.33311282C29@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41828:61dd02aae88f
Date: 2011-02-11 17:46 +0100
http://bitbucket.org/pypy/pypy/changeset/61dd02aae88f/

Log:	str.decode('utf-8') must call the decoder with final=True

diff --git a/pypy/objspace/std/test/test_unicodeobject.py b/pypy/objspace/std/test/test_unicodeobject.py
--- a/pypy/objspace/std/test/test_unicodeobject.py
+++ b/pypy/objspace/std/test/test_unicodeobject.py
@@ -524,6 +524,8 @@
         # Error handling (truncated escape sequence)
         raises(UnicodeError, "\\".decode, "unicode-escape")
 
+        raises(UnicodeError, "\xc2".decode, "utf-8")
+
     def test_repr_bug(self):
         assert (repr(u'\U00090418\u027d\U000582b9\u54c3\U000fcb6e') == 
                 "u'\\U00090418\\u027d\\U000582b9\\u54c3\\U000fcb6e'")

diff --git a/pypy/objspace/std/unicodetype.py b/pypy/objspace/std/unicodetype.py
--- a/pypy/objspace/std/unicodetype.py
+++ b/pypy/objspace/std/unicodetype.py
@@ -266,11 +266,13 @@
             s = space.bufferstr_w(w_obj)
             eh = decode_error_handler(space)
             return space.wrap(str_decode_ascii(s, len(s), None,
+                                               final=True,
                                                errorhandler=eh)[0])
         if encoding == 'utf-8':
             s = space.bufferstr_w(w_obj)
             eh = decode_error_handler(space)
             return space.wrap(str_decode_utf_8(s, len(s), None,
+                                               final=True,
                                                errorhandler=eh)[0])
     w_codecs = space.getbuiltinmodule("_codecs")
     w_decode = space.getattr(w_codecs, space.wrap("decode"))

From commits-noreply at bitbucket.org  Fri Feb 11 18:29:43 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri, 11 Feb 2011 18:29:43 +0100 (CET)
Subject: [pypy-svn] pypy default: Merge heads
Message-ID: <20110211172943.8F675282C2A@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41829:d3a0c245da18
Date: 2011-02-11 18:28 +0100
http://bitbucket.org/pypy/pypy/changeset/d3a0c245da18/

Log:	Merge heads


From commits-noreply at bitbucket.org  Fri Feb 11 19:10:15 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Fri, 11 Feb 2011 19:10:15 +0100 (CET)
Subject: [pypy-svn] pypy default: Do constant folding on subscripting.
Message-ID: <20110211181015.57074282BAD@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41830:b644f003dbe8
Date: 2011-02-11 13:09 -0500
http://bitbucket.org/pypy/pypy/changeset/b644f003dbe8/

Log:	Do constant folding on subscripting.

diff --git a/pypy/interpreter/astcompiler/test/test_compiler.py b/pypy/interpreter/astcompiler/test/test_compiler.py
--- a/pypy/interpreter/astcompiler/test/test_compiler.py
+++ b/pypy/interpreter/astcompiler/test/test_compiler.py
@@ -1,5 +1,5 @@
 import py
-from pypy.interpreter.astcompiler import codegen, astbuilder, symtable
+from pypy.interpreter.astcompiler import codegen, astbuilder, symtable, optimize
 from pypy.interpreter.pyparser import pyparse
 from pypy.interpreter.pyparser.test import expressions
 from pypy.interpreter.pycode import PyCode
@@ -18,6 +18,7 @@
     info = pyparse.CompileInfo("<test>", 'exec')
     cst = p.parse_source(expr, info)
     ast = astbuilder.ast_from_node(space, cst, info)
+    function_ast = optimize.optimize_ast(space, ast.body[0], info)
     function_ast = ast.body[0]
     symbols = symtable.SymtableBuilder(space, ast, info)
     generator = codegen.FunctionCodeGenerator(
@@ -786,11 +787,7 @@
          """ in {}
 
 class TestOptimizations:
-
-    def test_elim_jump_to_return(self):
-        source = """def f():
-        return true_value if cond else false_value
-        """
+    def count_instructions(self, source):
         code, blocks = generate_function_code(source, self.space)
         instrs = []
         for block in blocks:
@@ -799,6 +796,26 @@
         counts = {}
         for instr in instrs:
             counts[instr.opcode] = counts.get(instr.opcode, 0) + 1
+        return counts
+
+    def test_elim_jump_to_return(self):
+        source = """def f():
+        return true_value if cond else false_value
+        """
+        counts = self.count_instructions(source)
         assert ops.JUMP_FORWARD not in counts
         assert ops.JUMP_ABSOLUTE not in counts
         assert counts[ops.RETURN_VALUE] == 2
+
+    def test_const_fold_subscr(self):
+        source = """def f():
+        return (0, 1)[0]
+        """
+        counts = self.count_instructions(source)
+        assert counts == {ops.LOAD_CONST: 1, ops.RETURN_VALUE: 1}
+
+        source = """def f():
+        return (0, 1)[:2]
+        """
+        # Just checking this doesn't crash out
+        self.count_instructions(source)
\ No newline at end of file

diff --git a/pypy/interpreter/astcompiler/optimize.py b/pypy/interpreter/astcompiler/optimize.py
--- a/pypy/interpreter/astcompiler/optimize.py
+++ b/pypy/interpreter/astcompiler/optimize.py
@@ -67,6 +67,15 @@
     def as_constant(self):
         return self.value
 
+class __extend__(ast.Index):
+    def as_constant(self):
+        return self.value.as_constant()
+
+class __extend__(ast.Slice):
+    def as_constant(self):
+        # XXX: this ought to return a slice object if all the indices are
+        # constants, but we don't have a space here.
+        return None
 
 class __extend__(ast.UnaryOp):
 
@@ -125,7 +134,7 @@
     ast.RShift : _binary_fold("rshift"),
     ast.BitOr : _binary_fold("or_"),
     ast.BitXor : _binary_fold("xor"),
-    ast.BitAnd : _binary_fold("and_")
+    ast.BitAnd : _binary_fold("and_"),
 }
 unrolling_binary_folders = unrolling_iterable(binary_folders.items())
 
@@ -274,3 +283,16 @@
             consts_w = []
         w_consts = self.space.newtuple(consts_w)
         return ast.Const(w_consts, tup.lineno, tup.col_offset)
+
+    def visit_Subscript(self, subs):
+        if subs.ctx == ast.Load:
+            w_obj = subs.value.as_constant()
+            if w_obj is not None:
+                w_idx = subs.slice.as_constant()
+                if w_idx is not None:
+                    try:
+                        return ast.Const(self.space.getitem(w_obj, w_idx), subs.lineno, subs.col_offset)
+                    except OperationError:
+                        # Let exceptions propgate at runtime.
+                        pass
+        return subs

From commits-noreply at bitbucket.org  Fri Feb 11 19:10:16 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Fri, 11 Feb 2011 19:10:16 +0100 (CET)
Subject: [pypy-svn] pypy default: Merged upstream.
Message-ID: <20110211181016.2FC99282BAD@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41831:69fd3500847d
Date: 2011-02-11 13:09 -0500
http://bitbucket.org/pypy/pypy/changeset/69fd3500847d/

Log:	Merged upstream.

diff --git a/pypy/doc/config/objspace.usemodules.readline.txt b/pypy/doc/config/objspace.usemodules.readline.txt
deleted file mode 100644
--- a/pypy/doc/config/objspace.usemodules.readline.txt
+++ /dev/null
@@ -1,1 +0,0 @@
-Use the 'readline' module. 

From commits-noreply at bitbucket.org  Sat Feb 12 05:56:24 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Sat, 12 Feb 2011 05:56:24 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix using BZ2File as a contextmanager.
Message-ID: <20110212045624.AA00D282BAD@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41832:eeec2ae3a0f3
Date: 2011-02-11 23:56 -0500
http://bitbucket.org/pypy/pypy/changeset/eeec2ae3a0f3/

Log:	Fix using BZ2File as a contextmanager.

diff --git a/pypy/module/bz2/test/test_bz2_file.py b/pypy/module/bz2/test/test_bz2_file.py
--- a/pypy/module/bz2/test/test_bz2_file.py
+++ b/pypy/module/bz2/test/test_bz2_file.py
@@ -1,3 +1,5 @@
+from __future__ import with_statement
+
 import py
 from pypy.conftest import gettestobjspace
 from pypy.module.bz2.test.support import CheckAllocation
@@ -414,6 +416,19 @@
         bz2f.close()
         bz2f = BZ2File(self.temppath, 'r')
         assert bz2f.read() == self.random_data
+
+    def test_context_manager(self):
+        from bz2 import BZ2File
+
+        with BZ2File(self.temppath, 'w') as f:
+            assert not f.closed
+            f.write("abc")
+        assert f.closed
+        with BZ2File(self.temppath, 'r') as f:
+            data = f.read()
+            assert data == "abc"
+        assert f.closed
+
         
         
 # has_cmdline_bunzip2 = sys.platform not in ("win32", "os2emx", "riscos")

diff --git a/pypy/module/bz2/interp_bz2.py b/pypy/module/bz2/interp_bz2.py
--- a/pypy/module/bz2/interp_bz2.py
+++ b/pypy/module/bz2/interp_bz2.py
@@ -290,7 +290,7 @@
 same_attributes_as_in_file.remove('__init__')
 same_attributes_as_in_file.extend([
     'name', 'mode', 'encoding', 'closed', 'newlines', 'softspace',
-    'writelines', '__weakref__'])
+    'writelines', '__exit__', '__weakref__'])
 
 W_BZ2File.typedef = TypeDef(
     "BZ2File",

From commits-noreply at bitbucket.org  Sat Feb 12 05:58:24 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Sat, 12 Feb 2011 05:58:24 +0100 (CET)
Subject: [pypy-svn] pypy default: Skip this test is stdin isn't a tty.
Message-ID: <20110212045824.E0D16282BAD@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41833:8f0b690981b4
Date: 2011-02-11 23:58 -0500
http://bitbucket.org/pypy/pypy/changeset/8f0b690981b4/

Log:	Skip this test is stdin isn't a tty.

diff --git a/lib-python/modified-2.7.0/test/test_file2k.py b/lib-python/modified-2.7.0/test/test_file2k.py
--- a/lib-python/modified-2.7.0/test/test_file2k.py
+++ b/lib-python/modified-2.7.0/test/test_file2k.py
@@ -221,7 +221,12 @@
     def testStdin(self):
         # This causes the interpreter to exit on OSF1 v5.1.
         if sys.platform != 'osf1V5':
-            self.assertRaises(IOError, sys.stdin.seek, -1)
+            if sys.stdin.isatty():
+                self.assertRaises(IOError, sys.stdin.seek, -1)
+            else:
+                print >>sys.__stdout__, (
+                    '  Skipping sys.stdin.seek(-1): stdin is not a tty.'
+                    ' Test manualy.')
         else:
             print >>sys.__stdout__, (
                 '  Skipping sys.stdin.seek(-1), it may crash the interpreter.'

From commits-noreply at bitbucket.org  Sat Feb 12 06:18:50 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Sat, 12 Feb 2011 06:18:50 +0100 (CET)
Subject: [pypy-svn] pypy default: Added a test for opening a directory,
	currently failing.
Message-ID: <20110212051850.091B1282BAD@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41834:51aa0f0e4cf2
Date: 2011-02-12 00:18 -0500
http://bitbucket.org/pypy/pypy/changeset/51aa0f0e4cf2/

Log:	Added a test for opening a directory, currently failing.

diff --git a/pypy/module/_file/test/test_file.py b/pypy/module/_file/test/test_file.py
--- a/pypy/module/_file/test/test_file.py
+++ b/pypy/module/_file/test/test_file.py
@@ -198,6 +198,15 @@
         assert res == ".,."
         f.close()
 
+    def test_open_dir(self):
+        import os
+
+        exc = raises(IOError, self.file, os.curdir)
+        assert exc.value.filename == os.curdir
+        exc = raises(IOError, self.file, os.curdir, 'w')
+        assert exc.value.filename == os.curdir
+
+
 class AppTestConcurrency(object):
     # these tests only really make sense on top of a translated pypy-c,
     # because on top of py.py the inner calls to os.write() don't

From commits-noreply at bitbucket.org  Sat Feb 12 11:57:39 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Sat, 12 Feb 2011 11:57:39 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: Better handling of forced
	virtuals
Message-ID: <20110212105739.D60FE282B8B@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r41835:d475e67d90a5
Date: 2011-02-12 10:48 +0100
http://bitbucket.org/pypy/pypy/changeset/d475e67d90a5/

Log:	Better handling of forced virtuals

diff --git a/pypy/jit/metainterp/optimizeopt/unroll.py b/pypy/jit/metainterp/optimizeopt/unroll.py
--- a/pypy/jit/metainterp/optimizeopt/unroll.py
+++ b/pypy/jit/metainterp/optimizeopt/unroll.py
@@ -165,12 +165,13 @@
             info = self.info[box]
         except KeyError:
             value = self.getvalue(box)
-            self.info[box] = info = value.make_virtual_info(self, None)
-            if box in self.fieldboxes:
-                info.fieldstate = [self.state(b) for b in self.fieldboxes[box]]
-                # FIXME: Do we realy want to add fields to the VirtualInfo's?
-            elif isinstance(value, AbstractVirtualValue):
-                info.fieldstate = []
+            if value.is_virtual():
+                info = value.make_virtual_info(self, None)
+                flds = self.fieldboxes[box]
+                info.fieldstate = [self.state(b) for b in flds]
+            else:
+                info = self.make_not_virtual(value)
+            self.info[box] = info 
         return info
 
     def get_virtual_state(self, jump_args):
@@ -187,7 +188,10 @@
     def __init__(self, value):
         self.known_class = value.known_class
         self.level = value.level
-        self.intbound = value.intbound.clone()
+        if value.intbound is None:
+            self.intbound = IntBound(MININT, MAXINT)
+        else:
+            self.intbound = value.intbound.clone()
         if value.is_constant():
             self.constbox = value.box
         else:
@@ -661,11 +665,14 @@
                         try:
                             self.inline(sh.operations, sh.inputargs,
                                         op.getarglist())
+                            jumpop = self.optimizer.newoperations.pop()
+                            assert jumpop.getopnum() == rop.JUMP
                             for guard in extra_guards:
                                 descr = sh.start_resumedescr.clone_if_mutable()
                                 self.inliner.inline_descr_inplace(descr)
                                 guard.setdescr(descr)
                                 self.emit_operation(guard)
+                            self.optimizer.newoperations.append(jumpop)
                         except InvalidLoop:
                             debug_print("Inlining failed unexpectedly",
                                         "jumping to preamble instead")

diff --git a/pypy/jit/metainterp/test/test_virtual.py b/pypy/jit/metainterp/test/test_virtual.py
--- a/pypy/jit/metainterp/test/test_virtual.py
+++ b/pypy/jit/metainterp/test/test_virtual.py
@@ -564,6 +564,77 @@
         assert self.meta_interp(f, [40, 3]) == f(40, 3)
         self.check_loop_count(6)
 
+    def test_single_virtual_forced_in_bridge(self):
+        myjitdriver = JitDriver(greens = [], reds = ['n', 's', 'node'])
+        def externfn(node):
+            node.value *= 2
+        def f(n, s):
+            node = self._new()
+            node.value = 1
+            while n > 0:
+                myjitdriver.can_enter_jit(n=n, s=s, node=node)
+                myjitdriver.jit_merge_point(n=n, s=s, node=node)
+                next = self._new()
+                next.value = node.value + 1
+                node = next
+                if (n>>s) & 1:
+                    externfn(node)
+                n -= 1
+            return node.value
+        res = self.meta_interp(f, [48, 3], policy=StopAtXPolicy(externfn))
+        assert res == f(48, 3)
+        res = self.meta_interp(f, [40, 3], policy=StopAtXPolicy(externfn))
+        assert res == f(40, 3)
+
+    def test_forced_virtual_assigned_in_bridge(self):
+        myjitdriver = JitDriver(greens = [], reds = ['n', 's', 'node', 'node2'])
+        def externfn(node):
+            node.value += 1
+        def f(n, s):
+            node = self._new()
+            node.value = 1
+            node2 = self._new()
+            node2.value = 2
+            while n > 0:
+                myjitdriver.can_enter_jit(n=n, s=s, node=node, node2=node2)
+                myjitdriver.jit_merge_point(n=n, s=s, node=node, node2=node2)
+                next = self._new()
+                next.value = node.value + 1
+                node = next
+                if (n>>s) & 1:
+                    node2.value += node.value
+                    node = node2
+                externfn(node)
+                n -= 1
+            return node.value
+        res = self.meta_interp(f, [48, 3], policy=StopAtXPolicy(externfn))
+        assert res == f(48, 3)
+        self.check_loop_count(3)
+        res = self.meta_interp(f, [40, 3], policy=StopAtXPolicy(externfn))
+        assert res == f(40, 3)
+        self.check_loop_count(3)
+
+    def test_empty_virtual_with_bridge(self):
+        myjitdriver = JitDriver(greens = [], reds = ['n', 's', 'sa', 'node'])
+        def f(n, s):
+            node = self._new()
+            sa = 0
+            while n > 0:
+                myjitdriver.can_enter_jit(n=n, s=s, sa=sa, node=node)
+                myjitdriver.jit_merge_point(n=n, s=s, sa=sa, node=node)
+                next = self._new()
+                node = next
+                if (n>>s) & 1:
+                    sa += 1
+                else:
+                    sa += 2
+                n -= 1
+            return sa
+        res = self.meta_interp(f, [48, 3])
+        assert res == f(48, 3)
+        res = self.meta_interp(f, [40, 3])
+        assert res == f(40, 3)
+
     def test_virtual_array_bridge(self):
         myjitdriver = JitDriver(greens = [], reds = ['n', 'node'])
         def f(n):

From commits-noreply at bitbucket.org  Sat Feb 12 11:57:40 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Sat, 12 Feb 2011 11:57:40 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: Testcase where the guards
	produced by	VirtualState.generate_guards are needed
Message-ID: <20110212105740.A6A5A282B8B@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r41836:fa157d2a410f
Date: 2011-02-12 11:20 +0100
http://bitbucket.org/pypy/pypy/changeset/fa157d2a410f/

Log:	Testcase where the guards produced by VirtualState.generate_guards
	are needed

diff --git a/pypy/jit/metainterp/optimizeopt/unroll.py b/pypy/jit/metainterp/optimizeopt/unroll.py
--- a/pypy/jit/metainterp/optimizeopt/unroll.py
+++ b/pypy/jit/metainterp/optimizeopt/unroll.py
@@ -644,7 +644,6 @@
                 modifier = VirtualStateAdder(self.optimizer)
                 virtual_state = modifier.get_virtual_state(args)
                 for sh in short:
-                    #import pdb; pdb.set_trace()
                     ok = False
                     extra_guards = []
                     if sh.virtual_state.generalization_of(virtual_state):
@@ -655,6 +654,7 @@
                             sh.virtual_state.generate_guards(virtual_state,
                                                              args, cpu,
                                                              extra_guards)
+                            
                             ok = True
                         except InvalidLoop:
                             pass

diff --git a/pypy/jit/metainterp/test/test_virtual.py b/pypy/jit/metainterp/test/test_virtual.py
--- a/pypy/jit/metainterp/test/test_virtual.py
+++ b/pypy/jit/metainterp/test/test_virtual.py
@@ -614,6 +614,45 @@
         assert res == f(40, 3)
         self.check_loop_count(3)
 
+    def test_forced_virtual_assigned_different_class_in_bridge(self):
+        myjitdriver = JitDriver(greens = [], reds = ['n', 's', 'node', 'node2'])
+        def externfn(node):
+            node.value += 1
+        class A(object):
+            def __init__(self, value):
+                self.value = value
+            def op(self, val):
+                return self.value + val
+        class B(A):
+            def op(self, val):
+                return self.value - val
+        def f(n, s, node2):
+            node = A(1)
+            while n > 0:
+                myjitdriver.can_enter_jit(n=n, s=s, node=node, node2=node2)
+                myjitdriver.jit_merge_point(n=n, s=s, node=node, node2=node2)
+                if (n>>s) & 1:
+                    node2.value += node.value
+                    node = node2
+                else:
+                    node.value = node.op(1)
+                    node = A(node.value + 7)
+                    externfn(node)
+                n -= 1
+            return node.value
+        def g1(n, s):
+            return f(n, s, A(2)) + f(n, s, B(2))
+        def g2(n, s):
+            return f(n, s, B(2)) + f(n, s, A(2))
+        res = self.meta_interp(g1, [40, 3], policy=StopAtXPolicy(externfn))
+        assert res == g1(40, 3)
+        res = self.meta_interp(g1, [48, 3], policy=StopAtXPolicy(externfn))
+        assert res == g1(48, 3)
+        res = self.meta_interp(g2, [40, 3], policy=StopAtXPolicy(externfn))
+        assert res == g2(40, 3)
+        res = self.meta_interp(g2, [48, 3], policy=StopAtXPolicy(externfn))
+        assert res == g2(48, 3)
+
     def test_empty_virtual_with_bridge(self):
         myjitdriver = JitDriver(greens = [], reds = ['n', 's', 'sa', 'node'])
         def f(n, s):

From commits-noreply at bitbucket.org  Sat Feb 12 11:57:41 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Sat, 12 Feb 2011 11:57:41 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: box must be placed in info
	before recursing
Message-ID: <20110212105741.749DB2A2033@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r41837:c9cedf3b5f68
Date: 2011-02-12 11:23 +0100
http://bitbucket.org/pypy/pypy/changeset/c9cedf3b5f68/

Log:	box must be placed in info before recursing

diff --git a/pypy/jit/metainterp/optimizeopt/unroll.py b/pypy/jit/metainterp/optimizeopt/unroll.py
--- a/pypy/jit/metainterp/optimizeopt/unroll.py
+++ b/pypy/jit/metainterp/optimizeopt/unroll.py
@@ -166,12 +166,11 @@
         except KeyError:
             value = self.getvalue(box)
             if value.is_virtual():
-                info = value.make_virtual_info(self, None)
+                self.info[box] = info = value.make_virtual_info(self, None)
                 flds = self.fieldboxes[box]
                 info.fieldstate = [self.state(b) for b in flds]
             else:
-                info = self.make_not_virtual(value)
-            self.info[box] = info 
+                self.info[box] = info = self.make_not_virtual(value)
         return info
 
     def get_virtual_state(self, jump_args):

From commits-noreply at bitbucket.org  Sat Feb 12 17:08:54 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Sat, 12 Feb 2011 17:08:54 +0100 (CET)
Subject: [pypy-svn] pypy default: More error checking in signal.setitimer.
Message-ID: <20110212160854.AFAF1282B8B@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41838:89bbc0ca0a4b
Date: 2011-02-12 11:08 -0500
http://bitbucket.org/pypy/pypy/changeset/89bbc0ca0a4b/

Log:	More error checking in signal.setitimer.

diff --git a/pypy/module/select/interp_epoll.py b/pypy/module/select/interp_epoll.py
--- a/pypy/module/select/interp_epoll.py
+++ b/pypy/module/select/interp_epoll.py
@@ -1,11 +1,10 @@
 from __future__ import with_statement
 
 import errno
-import os
 
 from pypy.interpreter.baseobjspace import Wrappable
 from pypy.interpreter.gateway import interp2app, unwrap_spec, ObjSpace, W_Root
-from pypy.interpreter.error import OperationError, wrap_oserror, operationerrfmt
+from pypy.interpreter.error import OperationError, operationerrfmt, exception_from_errno
 from pypy.interpreter.typedef import TypeDef, GetSetProperty
 from pypy.rpython.lltypesystem import lltype, rffi
 from pypy.rpython.tool import rffi_platform
@@ -56,12 +55,6 @@
     compilation_info=eci,
 )
 
-def exception_from_errno(space, w_type):
-    errno = get_errno()
-    msg = os.strerror(errno)
-    w_error = space.call_function(w_type, space.wrap(errno), space.wrap(msg))
-    return OperationError(w_type, w_error)
-
 class W_Epoll(Wrappable):
     def __init__(self, space, epfd):
         self.space = space

diff --git a/pypy/module/signal/test/test_signal.py b/pypy/module/signal/test/test_signal.py
--- a/pypy/module/signal/test/test_signal.py
+++ b/pypy/module/signal/test/test_signal.py
@@ -255,3 +255,7 @@
         signal.pause()
         assert self.called
 
+    def test_itimer_exc(self):
+        import signal
+
+        raises(signal.ItimerError, signal.setitimer, -1, 0)

diff --git a/pypy/module/signal/interp_signal.py b/pypy/module/signal/interp_signal.py
--- a/pypy/module/signal/interp_signal.py
+++ b/pypy/module/signal/interp_signal.py
@@ -1,5 +1,5 @@
 from __future__ import with_statement
-from pypy.interpreter.error import OperationError
+from pypy.interpreter.error import OperationError, exception_from_errno
 from pypy.interpreter.baseobjspace import W_Root, ObjSpace
 from pypy.interpreter.executioncontext import AsyncAction, AbstractActionFlag
 from pypy.interpreter.executioncontext import PeriodicAsyncAction
@@ -303,6 +303,10 @@
     w_interval = space.wrap(double_from_timeval(val.c_it_interval))
     return space.newtuple([w_value, w_interval])
 
+def get_itimer_error(space):
+    mod = space.getbuiltinmodule("signal")
+    return space.getattr(mod, space.wrap("ItimerError"))
+
 @jit.dont_look_inside
 @unwrap_spec(ObjSpace, int, float, float)
 def setitimer(space, which, first, interval=0):
@@ -313,7 +317,10 @@
 
         with lltype.scoped_alloc(itimervalP.TO, 1) as old:
 
-            c_setitimer(which, new, old)
+            ret = c_setitimer(which, new, old)
+            if ret != 0:
+                raise exception_from_errno(space, get_itimer_error(space))
+
 
             return itimer_retval(space, old[0])
 

diff --git a/pypy/module/signal/app_signal.py b/pypy/module/signal/app_signal.py
--- a/pypy/module/signal/app_signal.py
+++ b/pypy/module/signal/app_signal.py
@@ -8,3 +8,7 @@
     It raises KeyboardInterrupt.
     """
     raise KeyboardInterrupt()
+
+
+class ItimerError(IOError):
+    pass
\ No newline at end of file

diff --git a/pypy/module/signal/__init__.py b/pypy/module/signal/__init__.py
--- a/pypy/module/signal/__init__.py
+++ b/pypy/module/signal/__init__.py
@@ -26,6 +26,7 @@
 
     appleveldefs = {
         'default_int_handler': 'app_signal.default_int_handler',
+        'ItimerError': 'app_signal.ItimerError',
     }
 
     def buildloaders(cls):

diff --git a/pypy/interpreter/error.py b/pypy/interpreter/error.py
--- a/pypy/interpreter/error.py
+++ b/pypy/interpreter/error.py
@@ -1,6 +1,6 @@
 import os, sys
+from pypy.rlib import jit
 from pypy.rlib.objectmodel import we_are_translated
-from pypy.rlib import jit
 from errno import EINTR
 
 AUTO_DEBUG = os.getenv('PYPY_DEBUG')
@@ -401,3 +401,10 @@
                              w_exception_class=w_exception_class)
 wrap_oserror._annspecialcase_ = 'specialize:arg(3)'
 
+def exception_from_errno(space, w_type):
+    from pypy.rlib.rposix import get_errno
+
+    errno = get_errno()
+    msg = os.strerror(errno)
+    w_error = space.call_function(w_type, space.wrap(errno), space.wrap(msg))
+    return OperationError(w_type, w_error)

From commits-noreply at bitbucket.org  Sat Feb 12 17:39:58 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sat, 12 Feb 2011 17:39:58 +0100 (CET)
Subject: [pypy-svn] pypy default: More direct tests of the longlong facility.
Message-ID: <20110212163958.1033F282B8B@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41839:4a3e44b38afd
Date: 2011-02-11 14:17 +0100
http://bitbucket.org/pypy/pypy/changeset/4a3e44b38afd/

Log:	More direct tests of the longlong facility.

diff --git a/pypy/jit/backend/x86/runner.py b/pypy/jit/backend/x86/runner.py
--- a/pypy/jit/backend/x86/runner.py
+++ b/pypy/jit/backend/x86/runner.py
@@ -13,7 +13,6 @@
 class AbstractX86CPU(AbstractLLCPU):
     debug = True
     supports_floats = True
-    supports_longlong = True
 
     BOOTSTRAP_TP = lltype.FuncType([], lltype.Signed)
     dont_keepalive_stuff = False # for tests
@@ -150,12 +149,15 @@
     CALLEE_SAVE_REGISTERS = [regloc.ebx, regloc.esi, regloc.edi]
     FRAME_FIXED_SIZE = len(CALLEE_SAVE_REGISTERS) + 2
 
+    supports_longlong = True
+
     def __init__(self, *args, **kwargs):
         assert sys.maxint == (2**31 - 1)
         super(CPU386, self).__init__(*args, **kwargs)
 
 class CPU386_NO_SSE2(CPU386):
     supports_floats = False
+    supports_longlong = False
 
 class CPU_X86_64(AbstractX86CPU):
     WORD = 8

diff --git a/pypy/jit/backend/test/runner_test.py b/pypy/jit/backend/test/runner_test.py
--- a/pypy/jit/backend/test/runner_test.py
+++ b/pypy/jit/backend/test/runner_test.py
@@ -2351,6 +2351,72 @@
             assert res.value == expected, (
                 "%r: got %r, expected %r" % (RESTYPE, res.value, expected))
 
+    def test_supports_longlong(self):
+        if sys.maxint > 2147483647:
+            assert not self.cpu.supports_longlong, (
+                "supports_longlong should be False on 64-bit platforms")
+
+    def test_longlong_result_of_call_direct(self):
+        if not self.cpu.supports_longlong:
+            py.test.skip("longlong test")
+        from pypy.translator.tool.cbuild import ExternalCompilationInfo
+        from pypy.rlib.rarithmetic import r_longlong
+        from pypy.rlib.longlong2float import longlong2float, float2longlong
+        eci = ExternalCompilationInfo(
+            separate_module_sources=["""
+            long long fn_test_result_of_call(long long x)
+            {
+                return x * 17;
+            }
+            """],
+            export_symbols=['fn_test_result_of_call'])
+        f = rffi.llexternal('fn_test_result_of_call', [lltype.SignedLongLong],
+                            lltype.SignedLongLong,
+                            compilation_info=eci, _nowrapper=True)
+        value = r_longlong(0x06778899aabbccdd)
+        expected = r_longlong(0x6df0123456789aad)
+        assert f(value) == expected
+        #
+        FUNC = self.FuncType([lltype.SignedLongLong], lltype.SignedLongLong)
+        FPTR = self.Ptr(FUNC)
+        calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
+        valuef = longlong2float(value)
+        xf = self.cpu.bh_call_f(self.get_funcbox(self.cpu, f).value,
+                                calldescr, None, None, [valuef])
+        x = float2longlong(xf)
+        assert x == expected
+
+    def test_longlong_result_of_call_compiled(self):
+        if not self.cpu.supports_longlong:
+            py.test.skip("test of longlong result")
+        from pypy.translator.tool.cbuild import ExternalCompilationInfo
+        from pypy.rlib.rarithmetic import r_longlong
+        from pypy.rlib.longlong2float import longlong2float, float2longlong
+        eci = ExternalCompilationInfo(
+            separate_module_sources=["""
+            long long fn_test_result_of_call(long long x)
+            {
+                return x * 17;
+            }
+            """],
+            export_symbols=['fn_test_result_of_call'])
+        f = rffi.llexternal('fn_test_result_of_call', [lltype.SignedLongLong],
+                            lltype.SignedLongLong,
+                            compilation_info=eci, _nowrapper=True)
+        value = r_longlong(0x06778899aabbccdd)
+        expected = r_longlong(0x6df0123456789aad)
+        assert f(value) == expected
+        #
+        FUNC = self.FuncType([lltype.SignedLongLong], lltype.SignedLongLong)
+        FPTR = self.Ptr(FUNC)
+        calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
+        funcbox = self.get_funcbox(self.cpu, f)
+        valuef = longlong2float(value)
+        resf = self.execute_operation(rop.CALL, [funcbox, BoxFloat(valuef)],
+                                     'float', descr=calldescr)
+        x = float2longlong(resf.value)
+        assert x == expected
+
     def test_free_loop_and_bridges(self):
         from pypy.jit.backend.llsupport.llmodel import AbstractLLCPU
         if not isinstance(self.cpu, AbstractLLCPU):

From commits-noreply at bitbucket.org  Sat Feb 12 17:39:59 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sat, 12 Feb 2011 17:39:59 +0100 (CET)
Subject: [pypy-svn] pypy default: Uh?  Ok,
	I got a failing test by reproducing exactly the result
Message-ID: <20110212163959.A6137282B8B@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41840:36fbedc0c19f
Date: 2011-02-11 14:24 +0100
http://bitbucket.org/pypy/pypy/changeset/36fbedc0c19f/

Log:	Uh? Ok, I got a failing test by reproducing exactly the result
	observed on running pypy -- it fails in exactly the same way. Why it
	fails when returning *this* value and not the previous one, I have
	no clue yet.

diff --git a/pypy/jit/backend/test/runner_test.py b/pypy/jit/backend/test/runner_test.py
--- a/pypy/jit/backend/test/runner_test.py
+++ b/pypy/jit/backend/test/runner_test.py
@@ -2366,15 +2366,15 @@
             separate_module_sources=["""
             long long fn_test_result_of_call(long long x)
             {
-                return x * 17;
+                return x - 100000000000000;
             }
             """],
             export_symbols=['fn_test_result_of_call'])
         f = rffi.llexternal('fn_test_result_of_call', [lltype.SignedLongLong],
                             lltype.SignedLongLong,
                             compilation_info=eci, _nowrapper=True)
-        value = r_longlong(0x06778899aabbccdd)
-        expected = r_longlong(0x6df0123456789aad)
+        value = r_longlong(0x7ff05af3307a3fff)
+        expected = r_longlong(0x7ff000001fffffff)
         assert f(value) == expected
         #
         FUNC = self.FuncType([lltype.SignedLongLong], lltype.SignedLongLong)
@@ -2396,15 +2396,15 @@
             separate_module_sources=["""
             long long fn_test_result_of_call(long long x)
             {
-                return x * 17;
+                return x - 100000000000000;
             }
             """],
             export_symbols=['fn_test_result_of_call'])
         f = rffi.llexternal('fn_test_result_of_call', [lltype.SignedLongLong],
                             lltype.SignedLongLong,
                             compilation_info=eci, _nowrapper=True)
-        value = r_longlong(0x06778899aabbccdd)
-        expected = r_longlong(0x6df0123456789aad)
+        value = r_longlong(0x7ff05af3307a3fff)
+        expected = r_longlong(0x7ff000001fffffff)
         assert f(value) == expected
         #
         FUNC = self.FuncType([lltype.SignedLongLong], lltype.SignedLongLong)

From commits-noreply at bitbucket.org  Sat Feb 12 17:40:00 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sat, 12 Feb 2011 17:40:00 +0100 (CET)
Subject: [pypy-svn] pypy default: This test fails.  Bad :-(
Message-ID: <20110212164000.3FD8C282B8B@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41841:c0ed680b39ff
Date: 2011-02-11 14:43 +0100
http://bitbucket.org/pypy/pypy/changeset/c0ed680b39ff/

Log:	This test fails. Bad :-(

diff --git a/pypy/rlib/test/test_longlong2float.py b/pypy/rlib/test/test_longlong2float.py
--- a/pypy/rlib/test/test_longlong2float.py
+++ b/pypy/rlib/test/test_longlong2float.py
@@ -17,3 +17,7 @@
     fn2 = compile(fn, [r_longlong])
     res = fn2(maxint64)
     assert res == maxint64
+
+def test_roundtrip():
+    value = 0x7ff06af3307a3fef
+    assert float2longlong(longlong2float(value)) == value

From commits-noreply at bitbucket.org  Sat Feb 12 17:40:01 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sat, 12 Feb 2011 17:40:01 +0100 (CET)
Subject: [pypy-svn] pypy default: For some reason,
	this example works fine when translated, 
Message-ID: <20110212164001.88849282C29@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41842:f8b115f22bcf
Date: 2011-02-11 15:06 +0100
http://bitbucket.org/pypy/pypy/changeset/f8b115f22bcf/

Log:	For some reason, this example works fine when translated, so the
	real reason of the pypy-c-jit error is more subtle. But the general
	idea is most probably the same.

diff --git a/pypy/rlib/test/test_longlong2float.py b/pypy/rlib/test/test_longlong2float.py
--- a/pypy/rlib/test/test_longlong2float.py
+++ b/pypy/rlib/test/test_longlong2float.py
@@ -13,11 +13,15 @@
 def test_longlong_as_float():
     assert fn(maxint64) == maxint64
 
+def test_roundtrip():
+    value = 0x7ff06af3307a3fef
+    assert fn(value) == value
+
 def test_compiled():
     fn2 = compile(fn, [r_longlong])
     res = fn2(maxint64)
     assert res == maxint64
-
-def test_roundtrip():
-    value = 0x7ff06af3307a3fef
-    assert float2longlong(longlong2float(value)) == value
+    #
+    special_value = 0x7ff06af3307a3fef
+    res = fn2(special_value)
+    assert res == special_value

From commits-noreply at bitbucket.org  Sat Feb 12 17:40:12 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sat, 12 Feb 2011 17:40:12 +0100 (CET)
Subject: [pypy-svn] pypy jit-longlong-2: In-progress: turns the BoxFloat and
	ConstFloat	attribute '.value'
Message-ID: <20110212164012.675412A2031@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: jit-longlong-2
Changeset: r41843:1bea1d8976c2
Date: 2011-02-11 16:59 +0100
http://bitbucket.org/pypy/pypy/changeset/1bea1d8976c2/

Log:	In-progress: turns the BoxFloat and ConstFloat attribute '.value' to
	be of type SignedLongLong on 32-bit machines. This is needed
	because it seems not reliable to store longlongs in a float field;
	see the test.

diff --git a/pypy/jit/metainterp/logger.py b/pypy/jit/metainterp/logger.py
--- a/pypy/jit/metainterp/logger.py
+++ b/pypy/jit/metainterp/logger.py
@@ -67,7 +67,7 @@
         elif isinstance(arg, self.ts.BoxRef):
             return 'p' + str(mv)
         elif isinstance(arg, ConstFloat):
-            return str(arg.value)
+            return str(arg.getfloat())
         elif isinstance(arg, BoxFloat):
             return 'f' + str(mv)
         elif arg is None:

diff --git a/pypy/jit/backend/llgraph/runner.py b/pypy/jit/backend/llgraph/runner.py
--- a/pypy/jit/backend/llgraph/runner.py
+++ b/pypy/jit/backend/llgraph/runner.py
@@ -181,7 +181,7 @@
                 elif isinstance(x, self.ts.ConstRef):
                     llimpl.compile_add_ref_const(c, x.value, self.ts.BASETYPE)
                 elif isinstance(x, history.ConstFloat):
-                    llimpl.compile_add_float_const(c, x.value)
+                    llimpl.compile_add_float_const(c, x.valuestorage)
                 else:
                     raise Exception("'%s' args contain: %r" % (op.getopname(),
                                                                x))

diff --git a/pypy/jit/backend/llgraph/llimpl.py b/pypy/jit/backend/llgraph/llimpl.py
--- a/pypy/jit/backend/llgraph/llimpl.py
+++ b/pypy/jit/backend/llgraph/llimpl.py
@@ -20,6 +20,7 @@
 from pypy.jit.metainterp import resoperation, executor
 from pypy.jit.metainterp.resoperation import rop
 from pypy.jit.backend.llgraph import symbolic
+from pypy.jit.codewriter import longlong
 
 from pypy.rlib.objectmodel import ComputedIntSymbolic, we_are_translated
 from pypy.rlib.rarithmetic import ovfcheck
@@ -301,7 +302,7 @@
     return compile_start_ref_var(loop, lltype.Signed)
 
 def compile_start_float_var(loop):
-    return compile_start_ref_var(loop, lltype.Float)
+    return compile_start_ref_var(loop, longlong.FLOATSTORAGE)
 
 def compile_start_ref_var(loop, TYPE):
     loop = _from_opaque(loop)
@@ -340,7 +341,7 @@
     compile_add_ref_const(loop, value, lltype.Signed)
 
 def compile_add_float_const(loop, value):
-    compile_add_ref_const(loop, value, lltype.Float)
+    compile_add_ref_const(loop, value, longlong.FLOATSTORAGE)
 
 def compile_add_ref_const(loop, value, TYPE):
     loop = _from_opaque(loop)
@@ -353,7 +354,7 @@
     return compile_add_ref_result(loop, lltype.Signed)
 
 def compile_add_float_result(loop):
-    return compile_add_ref_result(loop, lltype.Float)
+    return compile_add_ref_result(loop, longlong.FLOATSTORAGE)
 
 def compile_add_ref_result(loop, TYPE):
     loop = _from_opaque(loop)
@@ -486,8 +487,8 @@
                         x = self.as_ptr(result)
                     elif RESTYPE is ootype.Object:
                         x = self.as_object(result)
-                    elif RESTYPE is lltype.Float:
-                        x = self.as_float(result)
+                    elif RESTYPE is longlong.FLOATSTORAGE:
+                        x = self.as_floatstorage(result)
                     else:
                         raise Exception("op.result.concretetype is %r"
                                         % (RESTYPE,))
@@ -550,8 +551,8 @@
     def as_object(self, x):
         return ootype.cast_to_object(x)
 
-    def as_float(self, x):
-        return cast_to_float(x)
+    def as_floatstorage(self, x):
+        return cast_to_floatstorage(x)
 
     def log_progress(self):
         count = sum(_stats.exec_counters.values())
@@ -823,7 +824,7 @@
             elif T == llmemory.GCREF:
                 args_in_order.append('r')
                 _call_args_r.append(x)
-            elif T is lltype.Float:
+            elif T is longlong.FLOATSTORAGE:
                 args_in_order.append('f')
                 _call_args_f.append(x)
             else:
@@ -886,7 +887,7 @@
                     set_future_value_int(i, args[i])
                 elif isinstance(TYPE, lltype.Ptr):
                     set_future_value_ref(i, args[i])
-                elif TYPE is lltype.Float:
+                elif TYPE is longlong.FLOATSTORAGE:
                     set_future_value_float(i, args[i])
                 else:
                     raise Exception("Nonsense type %s" % TYPE)
@@ -1072,25 +1073,27 @@
 def cast_from_ptr(TYPE, x):
     return lltype.cast_opaque_ptr(TYPE, x)
 
-def cast_to_float(x):
+def cast_to_floatstorage(x):
     if isinstance(x, float):
-        return x      # common case
+        return longlong.getfloatstorage(x)      # common case
     if IS_32_BIT:
+        assert longlong.supports_longlong
         if isinstance(x, r_longlong):
-            return longlong2float(x)
+            return x
         if isinstance(x, r_ulonglong):
-            return longlong2float(rffi.cast(lltype.SignedLongLong, x))
+            return rffi.cast(lltype.SignedLongLong, x)
     raise TypeError(type(x))
 
-def cast_from_float(TYPE, x):
-    assert isinstance(x, float)
+def cast_from_floatstorage(TYPE, x):
+    assert isinstance(x, longlong.FloatStorage)
     if TYPE is lltype.Float:
-        return x
+        return longlong.getrealfloat(x)
     if IS_32_BIT:
+        assert longlong.supports_longlong
         if TYPE is lltype.SignedLongLong:
-            return float2longlong(x)
+            return x
         if TYPE is lltype.UnsignedLongLong:
-            return r_ulonglong(float2longlong(x))
+            return r_ulonglong(x)
     raise TypeError(TYPE)
 
 
@@ -1119,6 +1122,7 @@
     set_future_value_ref(index, value)
 
 def set_future_value_float(index, value):
+    assert isinstance(value, longlong.FloatStorage)
     set_future_value_ref(index, value)
 
 def set_future_value_ref(index, value):
@@ -1157,7 +1161,7 @@
     frame = _from_opaque(frame)
     assert num >= 0
     x = frame.fail_args[num]
-    assert lltype.typeOf(x) is lltype.Float
+    assert lltype.typeOf(x) is longlong.FLOATSTORAGE
     return x
 
 def frame_ptr_getvalue(frame, num):
@@ -1295,11 +1299,11 @@
 
 def do_getarrayitem_gc_float(array, index):
     array = array._obj.container
-    return cast_to_float(array.getitem(index))
+    return cast_to_floatstorage(array.getitem(index))
 
 def do_getarrayitem_raw_float(array, index):
     array = array.adr.ptr._obj
-    return cast_to_float(array.getitem(index))
+    return cast_to_floatstorage(array.getitem(index))
 
 def do_getarrayitem_gc_ptr(array, index):
     array = array._obj.container
@@ -1314,7 +1318,7 @@
     return cast_to_int(_getfield_gc(struct, fieldnum))
 
 def do_getfield_gc_float(struct, fieldnum):
-    return cast_to_float(_getfield_gc(struct, fieldnum))
+    return cast_to_floatstorage(_getfield_gc(struct, fieldnum))
 
 def do_getfield_gc_ptr(struct, fieldnum):
     return cast_to_ptr(_getfield_gc(struct, fieldnum))
@@ -1328,7 +1332,7 @@
     return cast_to_int(_getfield_raw(struct, fieldnum))
 
 def do_getfield_raw_float(struct, fieldnum):
-    return cast_to_float(_getfield_raw(struct, fieldnum))
+    return cast_to_floatstorage(_getfield_raw(struct, fieldnum))
 
 def do_getfield_raw_ptr(struct, fieldnum):
     return cast_to_ptr(_getfield_raw(struct, fieldnum))
@@ -1358,7 +1362,7 @@
 def do_setarrayitem_gc_float(array, index, newvalue):
     array = array._obj.container
     ITEMTYPE = lltype.typeOf(array).OF
-    newvalue = cast_from_float(ITEMTYPE, newvalue)
+    newvalue = cast_from_floatstorage(ITEMTYPE, newvalue)
     array.setitem(index, newvalue)
 
 def do_setarrayitem_raw_float(array, index, newvalue):
@@ -1384,7 +1388,7 @@
     STRUCT, fieldname = symbolic.TokenToField[fieldnum]
     ptr = lltype.cast_opaque_ptr(lltype.Ptr(STRUCT), struct)
     FIELDTYPE = getattr(STRUCT, fieldname)
-    newvalue = cast_from_float(FIELDTYPE, newvalue)
+    newvalue = cast_from_floatstorage(FIELDTYPE, newvalue)
     setattr(ptr, fieldname, newvalue)
 
 def do_setfield_gc_ptr(struct, fieldnum, newvalue):
@@ -1405,7 +1409,7 @@
     STRUCT, fieldname = symbolic.TokenToField[fieldnum]
     ptr = cast_from_int(lltype.Ptr(STRUCT), struct)
     FIELDTYPE = getattr(STRUCT, fieldname)
-    newvalue = cast_from_float(FIELDTYPE, newvalue)
+    newvalue = cast_from_floatstorage(FIELDTYPE, newvalue)
     setattr(ptr, fieldname, newvalue)
 
 def do_setfield_raw_ptr(struct, fieldnum, newvalue):
@@ -1461,7 +1465,7 @@
 
 kind2TYPE = {
     'i': lltype.Signed,
-    'f': lltype.Float,
+    'f': longlong.FLOATSTORAGE,
     'v': lltype.Void,
     }
 
@@ -1502,7 +1506,7 @@
 
 def do_call_float(f):
     x = _do_call_common(f)
-    return cast_to_float(x)
+    return cast_to_floatstorage(x)
 
 def do_call_ptr(f):
     x = _do_call_common(f)
@@ -1538,7 +1542,7 @@
                     n = orderiter.next()
                     assert n == 'f'
                 x = argsiter_f.next()
-                x = cast_from_float(TYPE, x)
+                x = cast_from_floatstorage(TYPE, x)
             else:
                 if args_in_order is not None:
                     n = orderiter.next()

diff --git a/pypy/jit/metainterp/history.py b/pypy/jit/metainterp/history.py
--- a/pypy/jit/metainterp/history.py
+++ b/pypy/jit/metainterp/history.py
@@ -3,12 +3,12 @@
 from pypy.rpython.lltypesystem import lltype, llmemory, rffi
 from pypy.rpython.ootypesystem import ootype
 from pypy.rlib.objectmodel import we_are_translated, r_dict, Symbolic
-from pypy.rlib.objectmodel import compute_hash, compute_unique_id
+from pypy.rlib.objectmodel import compute_unique_id
 from pypy.rlib.rarithmetic import intmask, r_int64
 from pypy.conftest import option
 
 from pypy.jit.metainterp.resoperation import ResOperation, rop
-from pypy.jit.codewriter import heaptracker
+from pypy.jit.codewriter import heaptracker, longlong
 
 # ____________________________________________________________
 
@@ -82,8 +82,15 @@
     def getint(self):
         raise NotImplementedError
 
+    def getfloatstorage(self):
+        raise NotImplementedError
+
     def getfloat(self):
-        raise NotImplementedError
+        return longlong.getrealfloat(self.getfloatstorage())
+
+    def getlonglong(self):
+        assert longlong.supports_longlong
+        return self.getfloatstorage()
 
     def getref_base(self):
         raise NotImplementedError
@@ -218,7 +225,7 @@
         elif kind == "ref":
             return cpu.ts.new_ConstRef(x)
         elif kind == "float":
-            return ConstFloat(x)
+            return ConstFloat(longlong.getfloatstorage(x))
         else:
             raise NotImplementedError(kind)
 
@@ -301,42 +308,46 @@
 
 class ConstFloat(Const):
     type = FLOAT
-    value = 0.0
-    _attrs_ = ('value',)
+    valuestorage = longlong.getfloatstorage(0.0)
+    _attrs_ = ('valuestorage',)
 
-    def __init__(self, floatval):
-        assert isinstance(floatval, float)
-        self.value = floatval
+    @property
+    def value(self):
+        XXX      # temporary
+
+    def __init__(self, valuestorage):
+        assert isinstance(valuestorage, longlong.FloatStorage)
+        self.valuestorage = valuestorage
 
     def clonebox(self):
-        return BoxFloat(self.value)
+        return BoxFloat(self.valuestorage)
 
     nonconstbox = clonebox
 
-    def getfloat(self):
-        return self.value
+    def getfloatstorage(self):
+        return self.valuestorage
 
     def _get_hash_(self):
-        return compute_hash(self.value)
+        return longlong.gethash(self.valuestorage)
 
     def set_future_value(self, cpu, j):
-        cpu.set_future_value_float(j, self.getfloat())
+        cpu.set_future_value_float(j, self.valuestorage)
 
     def same_constant(self, other):
         if isinstance(other, ConstFloat):
-            return self.value == other.value
+            return self.valuestorage == other.valuestorage
         return False
 
     def nonnull(self):
-        return self.value != 0.0
+        return self.valuestorage != 0
 
     def _getrepr_(self):
-        return self.value
+        return self.getfloat()
 
     def repr_rpython(self):
         return repr_rpython(self, 'cf')
 
-CONST_FZERO = ConstFloat(0.0)
+CONST_FZERO = ConstFloat(longlong.getfloatstorage(0.0))
 
 class ConstPtr(Const):
     type = REF
@@ -466,7 +477,7 @@
             ptrval = lltype.cast_opaque_ptr(llmemory.GCREF, x)
             return BoxPtr(ptrval)
         elif kind == "float":
-            return BoxFloat(x)
+            return BoxFloat(longlong.getfloatstorage(x))
         else:
             raise NotImplementedError(kind)
 
@@ -544,35 +555,39 @@
 
 class BoxFloat(Box):
     type = FLOAT
-    _attrs_ = ('value',)
+    _attrs_ = ('valuestorage',)
 
-    def __init__(self, floatval=0.0):
-        assert isinstance(floatval, float)
-        self.value = floatval
+    @property
+    def value(self):
+        XXX      # temporary
+
+    def __init__(self, valuestorage=longlong.getfloatstorage(0.0)):
+        assert isinstance(valuestorage, longlong.FloatStorage)
+        self.valuestorage = valuestorage
 
     def forget_value(self):
-        self.value = 0.0
+        self.valuestorage = longlong.getfloatstorage(0.0)
 
     def clonebox(self):
-        return BoxFloat(self.value)
+        return BoxFloat(self.valuestorage)
 
     def constbox(self):
-        return ConstFloat(self.value)
+        return ConstFloat(self.valuestorage)
 
-    def getfloat(self):
-        return self.value
+    def getfloatstorage(self):
+        return self.valuestorage
 
     def _get_hash_(self):
-        return compute_hash(self.value)
+        return longlong.gethash(self.valuestorage)
 
     def set_future_value(self, cpu, j):
-        cpu.set_future_value_float(j, self.value)
+        cpu.set_future_value_float(j, self.valuestorage)
 
     def nonnull(self):
-        return self.value != 0.0
+        return self.valuestorage != 0
 
     def _getrepr_(self):
-        return self.value
+        return self.getfloat()
 
     def repr_rpython(self):
         return repr_rpython(self, 'bf')

diff --git a/pypy/jit/metainterp/warmspot.py b/pypy/jit/metainterp/warmspot.py
--- a/pypy/jit/metainterp/warmspot.py
+++ b/pypy/jit/metainterp/warmspot.py
@@ -23,7 +23,7 @@
 from pypy.jit.metainterp.jitprof import Profiler, EmptyProfiler
 from pypy.jit.metainterp.jitexc import JitException
 from pypy.jit.metainterp.jitdriver import JitDriverStaticData
-from pypy.jit.codewriter import support, codewriter
+from pypy.jit.codewriter import support, codewriter, longlong
 from pypy.jit.codewriter.policy import JitPolicy
 
 # ____________________________________________________________
@@ -344,7 +344,7 @@
 
         class DoneWithThisFrameFloat(JitException):
             def __init__(self, result):
-                assert lltype.typeOf(result) is lltype.Float
+                assert lltype.typeOf(result) is longlong.FLOATSTORAGE
                 self.result = result
             def __str__(self):
                 return 'DoneWithThisFrameFloat(%s)' % (self.result,)

diff --git a/pypy/jit/metainterp/pyjitpl.py b/pypy/jit/metainterp/pyjitpl.py
--- a/pypy/jit/metainterp/pyjitpl.py
+++ b/pypy/jit/metainterp/pyjitpl.py
@@ -20,7 +20,7 @@
 from pypy.rlib.rarithmetic import intmask
 from pypy.rlib.objectmodel import specialize
 from pypy.jit.codewriter.jitcode import JitCode, SwitchDictDescr, MissingLiveness
-from pypy.jit.codewriter import heaptracker
+from pypy.jit.codewriter import heaptracker, longlong
 from pypy.jit.metainterp.optimizeutil import RetraceLoop
 
 # ____________________________________________________________

diff --git a/pypy/rlib/test/test_longlong2float.py b/pypy/rlib/test/test_longlong2float.py
--- a/pypy/rlib/test/test_longlong2float.py
+++ b/pypy/rlib/test/test_longlong2float.py
@@ -1,27 +1,30 @@
 from pypy.translator.c.test.test_genc import compile
 from pypy.rlib.longlong2float import longlong2float, float2longlong
-from pypy.rlib.rarithmetic import r_longlong
 
 
-maxint64 = r_longlong(9223372036854775807)
+def fn(f1):
+    ll = float2longlong(f1)
+    f2 = longlong2float(ll)
+    return f2
 
-def fn(x):
-    d = longlong2float(x)
-    ll = float2longlong(d)
-    return ll
+def enum_floats():
+    inf = 1e200 * 1e200
+    yield 0.0
+    yield -0.0
+    yield 1.0
+    yield -2.34567
+    yield 2.134891117e22
+    yield inf
+    yield -inf
+    yield inf / inf     # nan
 
 def test_longlong_as_float():
-    assert fn(maxint64) == maxint64
-
-def test_roundtrip():
-    value = 0x7ff06af3307a3fef
-    assert fn(value) == value
+    for x in enum_floats():
+        res = fn(x)
+        assert repr(res) == repr(x)
 
 def test_compiled():
-    fn2 = compile(fn, [r_longlong])
-    res = fn2(maxint64)
-    assert res == maxint64
-    #
-    special_value = 0x7ff06af3307a3fef
-    res = fn2(special_value)
-    assert res == special_value
+    fn2 = compile(fn, [float])
+    for x in enum_floats():
+        res = fn2(x)
+        assert repr(res) == repr(x)

diff --git a/pypy/jit/backend/model.py b/pypy/jit/backend/model.py
--- a/pypy/jit/backend/model.py
+++ b/pypy/jit/backend/model.py
@@ -1,4 +1,5 @@
 from pypy.rlib.debug import debug_start, debug_print, debug_stop
+from pypy.rlib.longlong2float import longlong2float, float2longlong
 from pypy.jit.metainterp import history, compile
 
 
@@ -184,11 +185,33 @@
     def typedescrof(TYPE):
         raise NotImplementedError
 
-    #def cast_adr_to_int(self, adr):
-    #    raise NotImplementedError
+    # ---------- float-like storage ----------
+    # On 32-bit, we want to absure occasionally the storage in BoxFloat
+    # and ConstFloat to store a 64-bit integer instead of a float.
+    # This is done by using the following helpers.  A "floatish" is either
+    # a float or a longlong, suitable for storage.
 
-    #def cast_int_to_adr(self, int):
-    #    raise NotImplementedError
+    def check_floatish(self, xish):
+        if self.supports_longlong:
+            assert isinstance(xish, r_longlong)
+        else:
+            assert isinstance(xish, float)
+
+    def getfloatish(self, x):
+        """Return the float 'x' as a floatish."""
+        assert isinstance(x, float)
+        if self.supports_longlong:
+            return float2longlong(x)
+        else:
+            return x
+
+    def getfloat(self, xish):
+        """Return the floatish 'x' as a float."""
+        self.check_floatish(xish)
+        if self.supports_longlong:
+            return longlong2float(xish)
+        else:
+            return xish
 
     # ---------- the backend-dependent operations ----------
 

diff --git a/pypy/jit/metainterp/warmstate.py b/pypy/jit/metainterp/warmstate.py
--- a/pypy/jit/metainterp/warmstate.py
+++ b/pypy/jit/metainterp/warmstate.py
@@ -12,13 +12,13 @@
 from pypy.rlib.jit import BaseJitCell
 from pypy.rlib.debug import debug_start, debug_stop, debug_print
 from pypy.jit.metainterp import history
-from pypy.jit.codewriter import support, heaptracker
+from pypy.jit.codewriter import support, heaptracker, longlong
 
 # ____________________________________________________________
 
 @specialize.arg(0)
 def specialize_value(TYPE, x):
-    """'x' must be a Signed, a GCREF or a Float.
+    """'x' must be a Signed, a GCREF or a FLOATSTORAGE.
     This function casts it to a more specialized type, like Char or Ptr(..).
     """
     INPUT = lltype.typeOf(x)
@@ -28,15 +28,15 @@
             return rffi.cast(TYPE, x)
         else:
             return lltype.cast_primitive(TYPE, x)
-    elif INPUT is lltype.Float:
+    elif INPUT is longlong.FLOATSTORAGE:
         assert TYPE is lltype.Float
-        return x
+        return longlong.getrealfloat(x)
     else:
         return lltype.cast_opaque_ptr(TYPE, x)
 
 @specialize.ll()
 def unspecialize_value(value):
-    """Casts 'value' to a Signed, a GCREF or a Float."""
+    """Casts 'value' to a Signed, a GCREF or a FLOATSTORAGE."""
     if isinstance(lltype.typeOf(value), lltype.Ptr):
         if lltype.typeOf(value).TO._gckind == 'gc':
             return lltype.cast_opaque_ptr(llmemory.GCREF, value)
@@ -46,7 +46,7 @@
     elif isinstance(lltype.typeOf(value), ootype.OOType):
         return ootype.cast_to_object(value)
     elif isinstance(value, float):
-        return value
+        return longlong.getfloatstorage(value)
     else:
         return intmask(value)
 
@@ -83,6 +83,7 @@
         else:
             return history.BoxObj(value)
     elif isinstance(value, float):
+        value = longlong.getfloatstorage(value)
         if in_const_box:
             return history.ConstFloat(value)
         else:

diff --git a/pypy/jit/metainterp/executor.py b/pypy/jit/metainterp/executor.py
--- a/pypy/jit/metainterp/executor.py
+++ b/pypy/jit/metainterp/executor.py
@@ -12,6 +12,7 @@
 from pypy.jit.metainterp import resoperation
 from pypy.jit.metainterp.resoperation import rop
 from pypy.jit.metainterp.blackhole import BlackholeInterpreter, NULL
+from pypy.jit.codewriter import longlong
 
 # ____________________________________________________________
 
@@ -42,7 +43,7 @@
             args_r[count_r] = box.getref_base()
             count_r += 1
         elif box.type == FLOAT:
-            args_f[count_f] = box.getfloat()
+            args_f[count_f] = box.getfloatstorage()
             count_f += 1
     # get the function address as an integer
     func = argboxes[0].getint()
@@ -109,7 +110,8 @@
         cpu.bh_setarrayitem_gc_r(arraydescr, array, index,
                                  itembox.getref_base())
     elif arraydescr.is_array_of_floats():
-        cpu.bh_setarrayitem_gc_f(arraydescr, array, index, itembox.getfloat())
+        cpu.bh_setarrayitem_gc_f(arraydescr, array, index,
+                                 itembox.getfloatstorage())
     else:
         cpu.bh_setarrayitem_gc_i(arraydescr, array, index, itembox.getint())
 
@@ -118,7 +120,8 @@
     index = indexbox.getint()
     assert not arraydescr.is_array_of_pointers()
     if arraydescr.is_array_of_floats():
-        cpu.bh_setarrayitem_raw_f(arraydescr, array, index, itembox.getfloat())
+        cpu.bh_setarrayitem_raw_f(arraydescr, array, index,
+                                  itembox.getfloatstorage())
     else:
         cpu.bh_setarrayitem_raw_i(arraydescr, array, index, itembox.getint())
 
@@ -146,7 +149,7 @@
     if fielddescr.is_pointer_field():
         cpu.bh_setfield_gc_r(struct, fielddescr, itembox.getref_base())
     elif fielddescr.is_float_field():
-        cpu.bh_setfield_gc_f(struct, fielddescr, itembox.getfloat())
+        cpu.bh_setfield_gc_f(struct, fielddescr, itembox.getfloatstorage())
     else:
         cpu.bh_setfield_gc_i(struct, fielddescr, itembox.getint())
 
@@ -155,7 +158,7 @@
     if fielddescr.is_pointer_field():
         cpu.bh_setfield_raw_r(struct, fielddescr, itembox.getref_base())
     elif fielddescr.is_float_field():
-        cpu.bh_setfield_raw_f(struct, fielddescr, itembox.getfloat())
+        cpu.bh_setfield_raw_f(struct, fielddescr, itembox.getfloatstorage())
     else:
         cpu.bh_setfield_raw_i(struct, fielddescr, itembox.getint())
 
@@ -342,7 +345,7 @@
                 argboxes = argboxes[1:]
                 if argtype == 'i':   value = argbox.getint()
                 elif argtype == 'r': value = argbox.getref_base()
-                elif argtype == 'f': value = argbox.getfloat()
+                elif argtype == 'f': value = argbox.getfloatstorage()
             newargs = newargs + (value,)
         assert not argboxes
         #

diff --git a/pypy/jit/metainterp/blackhole.py b/pypy/jit/metainterp/blackhole.py
--- a/pypy/jit/metainterp/blackhole.py
+++ b/pypy/jit/metainterp/blackhole.py
@@ -7,7 +7,7 @@
 from pypy.rpython.lltypesystem.lloperation import llop
 from pypy.rpython.llinterp import LLException
 from pypy.jit.codewriter.jitcode import JitCode, SwitchDictDescr
-from pypy.jit.codewriter import heaptracker
+from pypy.jit.codewriter import heaptracker, longlong
 from pypy.jit.metainterp.jitexc import JitException, get_llexception, reraise
 from pypy.jit.metainterp.compile import ResumeAtPositionDescr
 
@@ -204,7 +204,7 @@
                 assert argcodes[next_argcode] == '>'
                 assert argcodes[next_argcode + 1] == 'f'
                 next_argcode = next_argcode + 2
-                assert lltype.typeOf(result) is lltype.Float
+                assert lltype.typeOf(result) is longlong.FLOATSTORAGE
                 self.registers_f[ord(code[position])] = result
                 position += 1
             elif resulttype == 'L':
@@ -535,45 +535,74 @@
 
     @arguments("f", returns="f")
     def bhimpl_float_neg(a):
-        return -a
+        a = longlong.getrealfloat(a)
+        x = -a
+        return longlong.getfloatstorage(x)
     @arguments("f", returns="f")
     def bhimpl_float_abs(a):
-        return abs(a)
+        a = longlong.getrealfloat(a)
+        x = abs(a)
+        return longlong.getfloatstorage(x)
 
     @arguments("f", "f", returns="f")
     def bhimpl_float_add(a, b):
-        return a + b
+        a = longlong.getrealfloat(a)
+        b = longlong.getrealfloat(b)
+        x = a + b
+        return longlong.getfloatstorage(x)
     @arguments("f", "f", returns="f")
     def bhimpl_float_sub(a, b):
-        return a - b
+        a = longlong.getrealfloat(a)
+        b = longlong.getrealfloat(b)
+        x = a - b
+        return longlong.getfloatstorage(x)
     @arguments("f", "f", returns="f")
     def bhimpl_float_mul(a, b):
-        return a * b
+        a = longlong.getrealfloat(a)
+        b = longlong.getrealfloat(b)
+        x = a * b
+        return longlong.getfloatstorage(x)
     @arguments("f", "f", returns="f")
     def bhimpl_float_truediv(a, b):
-        return a / b
+        a = longlong.getrealfloat(a)
+        b = longlong.getrealfloat(b)
+        x = a / b
+        return longlong.getfloatstorage(x)
 
     @arguments("f", "f", returns="i")
     def bhimpl_float_lt(a, b):
+        a = longlong.getrealfloat(a)
+        b = longlong.getrealfloat(b)
         return a < b
     @arguments("f", "f", returns="i")
     def bhimpl_float_le(a, b):
+        a = longlong.getrealfloat(a)
+        b = longlong.getrealfloat(b)
         return a <= b
     @arguments("f", "f", returns="i")
     def bhimpl_float_eq(a, b):
+        a = longlong.getrealfloat(a)
+        b = longlong.getrealfloat(b)
         return a == b
     @arguments("f", "f", returns="i")
     def bhimpl_float_ne(a, b):
+        a = longlong.getrealfloat(a)
+        b = longlong.getrealfloat(b)
         return a != b
     @arguments("f", "f", returns="i")
     def bhimpl_float_gt(a, b):
+        a = longlong.getrealfloat(a)
+        b = longlong.getrealfloat(b)
         return a > b
     @arguments("f", "f", returns="i")
     def bhimpl_float_ge(a, b):
+        a = longlong.getrealfloat(a)
+        b = longlong.getrealfloat(b)
         return a >= b
 
     @arguments("f", returns="i")
     def bhimpl_cast_float_to_int(a):
+        a = longlong.getrealfloat(a)
         # note: we need to call int() twice to care for the fact that
         # int(-2147483648.0) returns a long :-(
         return int(int(a))
@@ -581,6 +610,7 @@
     @arguments("i", returns="f")
     def bhimpl_cast_int_to_float(a):
         return float(a)
+        return longlong.getfloatstorage(x)
 
     # ----------
     # control flow operations

diff --git a/pypy/jit/codewriter/test/test_longlong.py b/pypy/jit/codewriter/test/test_longlong.py
--- a/pypy/jit/codewriter/test/test_longlong.py
+++ b/pypy/jit/codewriter/test/test_longlong.py
@@ -4,7 +4,7 @@
 from pypy.objspace.flow.model import SpaceOperation, Variable, Constant
 from pypy.objspace.flow.model import Block, Link
 from pypy.translator.unsimplify import varoftype
-from pypy.rpython.lltypesystem import lltype
+from pypy.rpython.lltypesystem import lltype, rffi
 from pypy.jit.codewriter.jtransform import Transformer, NotSupported
 from pypy.jit.codewriter.effectinfo import EffectInfo
 from pypy.jit.codewriter.test.test_jtransform import const
@@ -58,42 +58,6 @@
                                      if is_llf(v.concretetype)]
         assert op1.result == v_result
 
-    def test_remove_longlong_constant(self):
-        c1 = Constant(r_longlong(-123), lltype.SignedLongLong)
-        c2 = Constant(r_longlong(-124), lltype.SignedLongLong)
-        c3 = Constant(r_longlong(0x987654321), lltype.SignedLongLong)
-        v1 = varoftype(lltype.SignedLongLong)
-        v2 = varoftype(lltype.Bool)
-        block = Block([v1])
-        block.operations = [SpaceOperation('foo', [c1, v1, c3], v2)]
-        block.exitswitch = v2
-        block.closeblock(Link([c2], block, exitcase=False),
-                         Link([c1], block, exitcase=True))
-        tr = Transformer()
-        tr.remove_longlong_constants(block)
-        assert len(block.operations) == 4
-        assert block.operations[0].opname == 'cast_int_to_longlong'
-        assert block.operations[0].args[0].value == -123
-        assert block.operations[0].args[0].concretetype == lltype.Signed
-        v3 = block.operations[0].result
-        assert block.operations[1].opname == 'two_ints_to_longlong'
-        assert block.operations[1].args[0].value == intmask(0x87654321)
-        assert block.operations[1].args[0].concretetype == lltype.Signed
-        assert block.operations[1].args[1].value == 0x9
-        assert block.operations[1].args[1].concretetype == lltype.Signed
-        v4 = block.operations[1].result
-        assert block.operations[2].opname == 'foo'
-        assert block.operations[2].args[0] is v3
-        assert block.operations[2].args[1] is v1
-        assert block.operations[2].args[2] is v4
-        assert block.operations[2].result is v2
-        assert block.operations[3].opname == 'cast_int_to_longlong'
-        assert block.operations[3].args[0].value == -124
-        assert block.operations[3].args[0].concretetype == lltype.Signed
-        v5 = block.operations[3].result
-        assert block.exits[0].args[0] is v5
-        assert block.exits[1].args[0] is v3
-
     def test_is_true(self):
         for opname, T in [('llong_is_true', lltype.SignedLongLong),
                           ('ullong_is_true', lltype.UnsignedLongLong)]:
@@ -218,3 +182,18 @@
                       [lltype.Float], lltype.SignedLongLong)
         self.do_check('cast_longlong_to_float', EffectInfo.OS_LLONG_TO_FLOAT,
                       [lltype.SignedLongLong], lltype.Float)
+
+    def test_constants(self):
+        for TYPE in [lltype.SignedLongLong, lltype.UnsignedLongLong]:
+            v_x = varoftype(TYPE)
+            vlist = [v_x, const(rffi.cast(TYPE, 7))]
+            v_result = varoftype(TYPE)
+            op = SpaceOperation('llong_add', vlist, v_result)
+            tr = Transformer(FakeCPU(), FakeBuiltinCallControl())
+            op1 = tr.rewrite_operation(op)
+            #
+            assert op1.opname == 'residual_call_irf_f'
+            assert list(op1.args[2]) == []
+            assert list(op1.args[3]) == []
+            assert list(op1.args[4]) == vlist
+            assert op1.result == v_result

diff --git a/pypy/jit/codewriter/jtransform.py b/pypy/jit/codewriter/jtransform.py
--- a/pypy/jit/codewriter/jtransform.py
+++ b/pypy/jit/codewriter/jtransform.py
@@ -39,7 +39,6 @@
     def optimize_block(self, block):
         if block.operations == ():
             return
-        self.remove_longlong_constants(block)
         self.vable_array_vars = {}
         self.vable_flags = {}
         renamings = {}
@@ -135,55 +134,6 @@
         block.exits = block.exits[:1]
         block.exitswitch = None
 
-    def remove_longlong_constants(self, block):
-        # remove all Constant({Un}signedLongLong), and replace them with
-        # cast_int_to_longlong(Constant(Signed)) or
-        # two_ints_to_longlong(Constant(Signed), Constant(Signed)).
-        operations = []
-        all_constants = {}
-        #
-        def _get_const_as_var(c):
-            v = all_constants.get(c)
-            if v is None:
-                from pypy.rlib.rarithmetic import intmask
-                v = varoftype(c.concretetype)
-                value = int(c.value)
-                c_hi = Constant(intmask(value >> 32), lltype.Signed)
-                c_lo = Constant(intmask(value), lltype.Signed)
-                if c_lo.value == value:
-                    # a long long constant, but it fits in 32 bits
-                    op1 = SpaceOperation('cast_int_to_longlong', [c_lo], v)
-                else:
-                    # a 64-bit long long constant, requires two ints
-                    op1 = SpaceOperation('two_ints_to_longlong', [c_lo, c_hi],
-                                         v)
-                operations.append(op1)
-                all_constants[c] = v
-            return v
-        #
-        for op in block.operations:
-            for i, v in enumerate(op.args):
-                if (isinstance(v, Constant) and
-                        self._is_longlong(v.concretetype)):
-                    args = op.args[:]
-                    args[i] = _get_const_as_var(v)
-                    op = SpaceOperation(op.opname, args, op.result)
-            operations.append(op)
-        #
-        last_op = None
-        if block.exitswitch == c_last_exception:
-            last_op = operations.pop()
-        for link in block.exits:
-            for i, v in enumerate(link.args):
-                if (isinstance(v, Constant) and
-                        self._is_longlong(v.concretetype)):
-                    args = link.args[:]
-                    args[i] = _get_const_as_var(v)
-                    link.args = args
-        if last_op is not None:
-            operations.append(last_op)
-        block.operations = operations
-
     # ----------
 
     def follow_constant_exit(self, block):

diff --git a/pypy/jit/codewriter/assembler.py b/pypy/jit/codewriter/assembler.py
--- a/pypy/jit/codewriter/assembler.py
+++ b/pypy/jit/codewriter/assembler.py
@@ -3,10 +3,10 @@
 from pypy.jit.codewriter.flatten import ListOfKind, IndirectCallTargets
 from pypy.jit.codewriter.format import format_assembler
 from pypy.jit.codewriter.jitcode import SwitchDictDescr, JitCode
-from pypy.jit.codewriter import heaptracker
+from pypy.jit.codewriter import heaptracker, longlong
 from pypy.rlib.objectmodel import ComputedIntSymbolic
 from pypy.objspace.flow.model import Constant
-from pypy.rpython.lltypesystem import lltype, llmemory, rclass
+from pypy.rpython.lltypesystem import lltype, llmemory, rclass, rffi
 
 
 class AssemblerError(Exception):
@@ -87,7 +87,15 @@
             value = lltype.cast_opaque_ptr(llmemory.GCREF, value)
             constants = self.constants_r
         elif kind == 'float':
-            assert const.concretetype == lltype.Float
+            if const.concretetype == lltype.Float:
+                value = longlong.getfloatstorage(value)
+            elif const.concretetype == lltype.SignedLongLong:
+                assert longlong.supports_longlong
+            elif const.concretetype == lltype.UnsignedLongLong:
+                assert longlong.supports_longlong
+                value = rffi.cast(lltype.SignedLongLong, value)
+            else:
+                raise AssertionError(const.concretetype)
             constants = self.constants_f
         else:
             raise AssemblerError('unimplemented %r in %r' %

diff --git a/pypy/jit/metainterp/optimizeopt/optimizer.py b/pypy/jit/metainterp/optimizeopt/optimizer.py
--- a/pypy/jit/metainterp/optimizeopt/optimizer.py
+++ b/pypy/jit/metainterp/optimizeopt/optimizer.py
@@ -162,7 +162,7 @@
 CONST_0      = ConstInt(0)
 CONST_1      = ConstInt(1)
 CVAL_ZERO    = ConstantValue(CONST_0)
-CVAL_ZERO_FLOAT = ConstantValue(ConstFloat(0.0))
+CVAL_ZERO_FLOAT = ConstantValue(Const._new(0.0))
 CVAL_UNINITIALIZED_ZERO = ConstantValue(CONST_0)
 llhelper.CVAL_NULLREF = ConstantValue(llhelper.CONST_NULL)
 oohelper.CVAL_NULLREF = ConstantValue(oohelper.CONST_NULL)

diff --git a/pypy/jit/codewriter/test/test_assembler.py b/pypy/jit/codewriter/test/test_assembler.py
--- a/pypy/jit/codewriter/test/test_assembler.py
+++ b/pypy/jit/codewriter/test/test_assembler.py
@@ -1,9 +1,9 @@
-import py, struct
+import py, struct, sys
 from pypy.jit.codewriter.assembler import Assembler, AssemblerError
 from pypy.jit.codewriter.flatten import SSARepr, Label, TLabel, Register
 from pypy.jit.codewriter.flatten import ListOfKind, IndirectCallTargets
 from pypy.jit.codewriter.jitcode import MissingLiveness
-from pypy.jit.codewriter import heaptracker
+from pypy.jit.codewriter import heaptracker, longlong
 from pypy.jit.metainterp.history import AbstractDescr
 from pypy.objspace.flow.model import Constant
 from pypy.rpython.lltypesystem import lltype, llmemory
@@ -61,7 +61,28 @@
                             "\x00\xFE"
                             "\x00\xFD")
     assert assembler.insns == {'float_return/f': 0}
-    assert jitcode.constants_f == [18.0, -4.0, 128.1]
+    assert jitcode.constants_f == [longlong.getfloatstorage(18.0),
+                                   longlong.getfloatstorage(-4.0),
+                                   longlong.getfloatstorage(128.1)]
+
+def test_assemble_llong_consts():
+    if sys.maxint > 2147483647:
+        py.test.skip("only for 32-bit platforms")
+    from pypy.rlib.rarithmetic import r_longlong, r_ulonglong
+    ssarepr = SSARepr("test")
+    ssarepr.insns = [
+        ('float_return', Constant(r_longlong(-18000000000000000),
+                                  lltype.SignedLongLong)),
+        ('float_return', Constant(r_ulonglong(9900000000000000000),
+                                  lltype.UnsignedLongLong)),
+        ]
+    assembler = Assembler()
+    jitcode = assembler.assemble(ssarepr)
+    assert jitcode.code == ("\x00\xFF"
+                            "\x00\xFE")
+    assert assembler.insns == {'float_return/f': 0}
+    assert jitcode.constants_f == [r_longlong(-18000000000000000),
+                                   r_longlong(-8546744073709551616)]
 
 def test_assemble_cast_consts():
     ssarepr = SSARepr("test")

diff --git a/pypy/rlib/longlong2float.py b/pypy/rlib/longlong2float.py
--- a/pypy/rlib/longlong2float.py
+++ b/pypy/rlib/longlong2float.py
@@ -1,6 +1,9 @@
 """
 This module exposes the functions longlong2float() and float2longlong(),
-which cast the bit pattern of a long long into a float and back.
+which cast the bit pattern of a float into a long long and back.
+Warning: don't use in the other direction, i.e. don't cast a random
+long long to a float and back to a long long.  There are corner cases
+in which it does not work.
 """
 from pypy.rpython.lltypesystem import lltype, rffi
 

From commits-noreply at bitbucket.org  Sat Feb 12 17:40:14 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sat, 12 Feb 2011 17:40:14 +0100 (CET)
Subject: [pypy-svn] pypy jit-longlong-2: More progress...
Message-ID: <20110212164014.EF8982A2031@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: jit-longlong-2
Changeset: r41844:cd5d0d55ef89
Date: 2011-02-11 17:14 +0100
http://bitbucket.org/pypy/pypy/changeset/cd5d0d55ef89/

Log:	More progress...

diff --git a/pypy/jit/metainterp/history.py b/pypy/jit/metainterp/history.py
--- a/pypy/jit/metainterp/history.py
+++ b/pypy/jit/metainterp/history.py
@@ -316,7 +316,7 @@
         XXX      # temporary
 
     def __init__(self, valuestorage):
-        assert isinstance(valuestorage, longlong.FloatStorage)
+        assert isinstance(valuestorage, longlong.r_float_storage)
         self.valuestorage = valuestorage
 
     def clonebox(self):
@@ -562,7 +562,7 @@
         XXX      # temporary
 
     def __init__(self, valuestorage=longlong.getfloatstorage(0.0)):
-        assert isinstance(valuestorage, longlong.FloatStorage)
+        assert isinstance(valuestorage, longlong.r_float_storage)
         self.valuestorage = valuestorage
 
     def forget_value(self):

diff --git a/pypy/jit/metainterp/pyjitpl.py b/pypy/jit/metainterp/pyjitpl.py
--- a/pypy/jit/metainterp/pyjitpl.py
+++ b/pypy/jit/metainterp/pyjitpl.py
@@ -1473,7 +1473,7 @@
             elif result_type == history.REF:
                 raise sd.DoneWithThisFrameRef(self.cpu, resultbox.getref_base())
             elif result_type == history.FLOAT:
-                raise sd.DoneWithThisFrameFloat(resultbox.getfloat())
+                raise sd.DoneWithThisFrameFloat(resultbox.getfloatstorage())
             else:
                 assert False
 

diff --git a/pypy/jit/metainterp/resume.py b/pypy/jit/metainterp/resume.py
--- a/pypy/jit/metainterp/resume.py
+++ b/pypy/jit/metainterp/resume.py
@@ -1158,7 +1158,7 @@
     def decode_float(self, tagged):
         num, tag = untag(tagged)
         if tag == TAGCONST:
-            return self.consts[num].getfloat()
+            return self.consts[num].getfloatstorage()
         else:
             assert tag == TAGBOX
             if num < 0:

diff --git a/pypy/jit/metainterp/warmstate.py b/pypy/jit/metainterp/warmstate.py
--- a/pypy/jit/metainterp/warmstate.py
+++ b/pypy/jit/metainterp/warmstate.py
@@ -139,7 +139,12 @@
         intvalue = lltype.cast_primitive(lltype.Signed, value)
         cpu.set_future_value_int(j, intvalue)
     elif typecode == 'float':
-        assert isinstance(value, float)
+        if lltype.typeOf(value) is lltype.Float:
+            value = longlong.getfloatstorage(value)
+        else:
+            assert lltype.typeOf(value) in (lltype.SignedLongLong,
+                                            lltype.UnsignedLongLong)
+            value = rffi.cast(lltype.SignedLongLong, value)
         cpu.set_future_value_float(j, value)
     else:
         assert False

diff --git a/pypy/jit/metainterp/blackhole.py b/pypy/jit/metainterp/blackhole.py
--- a/pypy/jit/metainterp/blackhole.py
+++ b/pypy/jit/metainterp/blackhole.py
@@ -252,7 +252,7 @@
         if we_are_translated():
             default_i = 0
             default_r = NULL
-            default_f = 0.0
+            default_f = longlong.getfloatstorage(0.0)
         else:
             default_i = MissingValue()
             default_r = MissingValue()
@@ -281,12 +281,15 @@
         self.position = position
 
     def setarg_i(self, index, value):
+        assert lltype.typeOf(value) is lltype.Signed
         self.registers_i[index] = value
 
     def setarg_r(self, index, value):
+        assert lltype.typeOf(value) == llmemory.GCREF
         self.registers_r[index] = value
 
     def setarg_f(self, index, value):
+        assert lltype.typeOf(value) is longlong.FLOATSTORAGE
         self.registers_f[index] = value
 
     def run(self):
@@ -1292,10 +1295,13 @@
     # connect the return of values from the called frame to the
     # 'xxx_call_yyy' instructions from the caller frame
     def _setup_return_value_i(self, result):
+        assert lltype.typeOf(result) is lltype.Signed
         self.registers_i[ord(self.jitcode.code[self.position-1])] = result
     def _setup_return_value_r(self, result):
+        assert lltype.typeOf(result) == llmemory.GCREF
         self.registers_r[ord(self.jitcode.code[self.position-1])] = result
     def _setup_return_value_f(self, result):
+        assert lltype.typeOf(result) is longlong.FLOATSTORAGE
         self.registers_f[ord(self.jitcode.code[self.position-1])] = result
 
     def _done_with_this_frame(self):
@@ -1359,7 +1365,7 @@
         for i in range(self.jitcode.num_regs_f()):
             box = miframe.registers_f[i]
             if box is not None:
-                self.setarg_f(i, box.getfloat())
+                self.setarg_f(i, box.getfloatstorage())
 
 # ____________________________________________________________
 

diff --git a/pypy/jit/backend/llgraph/llimpl.py b/pypy/jit/backend/llgraph/llimpl.py
--- a/pypy/jit/backend/llgraph/llimpl.py
+++ b/pypy/jit/backend/llgraph/llimpl.py
@@ -1085,7 +1085,7 @@
     raise TypeError(type(x))
 
 def cast_from_floatstorage(TYPE, x):
-    assert isinstance(x, longlong.FloatStorage)
+    assert isinstance(x, longlong.r_float_storage)
     if TYPE is lltype.Float:
         return longlong.getrealfloat(x)
     if IS_32_BIT:
@@ -1122,7 +1122,7 @@
     set_future_value_ref(index, value)
 
 def set_future_value_float(index, value):
-    assert isinstance(value, longlong.FloatStorage)
+    assert isinstance(value, longlong.r_float_storage)
     set_future_value_ref(index, value)
 
 def set_future_value_ref(index, value):
@@ -1368,7 +1368,7 @@
 def do_setarrayitem_raw_float(array, index, newvalue):
     array = array.adr.ptr
     ITEMTYPE = lltype.typeOf(array).TO.OF
-    newvalue = cast_from_int(ITEMTYPE, newvalue)
+    newvalue = cast_from_floatstorage(ITEMTYPE, newvalue)
     array._obj.setitem(index, newvalue)
 
 def do_setarrayitem_gc_ptr(array, index, newvalue):

diff --git a/pypy/jit/metainterp/test/test_basic.py b/pypy/jit/metainterp/test/test_basic.py
--- a/pypy/jit/metainterp/test/test_basic.py
+++ b/pypy/jit/metainterp/test/test_basic.py
@@ -9,6 +9,7 @@
 from pypy.jit.metainterp import pyjitpl, history
 from pypy.jit.metainterp.warmstate import set_future_value
 from pypy.jit.codewriter.policy import JitPolicy, StopAtXPolicy
+from pypy.jit.codewriter import longlong
 from pypy import conftest
 from pypy.rlib.rarithmetic import ovfcheck
 from pypy.jit.metainterp.typesystem import LLTypeHelper, OOTypeHelper
@@ -86,6 +87,7 @@
             blackholeinterp.setarg_r(count_r, value)
             count_r += 1
         elif T == lltype.Float:
+            value = longlong.getfloatstorage(value)
             blackholeinterp.setarg_f(count_f, value)
             count_f += 1
         else:
@@ -192,6 +194,10 @@
         # try to run it by running the code compiled just before
         result3 = _run_with_machine_code(self, args)
         assert result1 == result3 or result3 == NotImplemented
+        #
+        if (longlong.supports_longlong and
+            isinstance(result1, longlong.r_float_storage)):
+            result1 = longlong.getrealfloat(result1)
         return result1
 
     def check_history(self, expected=None, **isns):

From commits-noreply at bitbucket.org  Sat Feb 12 17:40:16 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sat, 12 Feb 2011 17:40:16 +0100 (CET)
Subject: [pypy-svn] pypy jit-longlong-2: Kill this.  Not used.
Message-ID: <20110212164016.E61892A2031@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: jit-longlong-2
Changeset: r41845:afda3fbfb547
Date: 2011-02-11 17:15 +0100
http://bitbucket.org/pypy/pypy/changeset/afda3fbfb547/

Log:	Kill this. Not used.

diff --git a/pypy/jit/backend/model.py b/pypy/jit/backend/model.py
--- a/pypy/jit/backend/model.py
+++ b/pypy/jit/backend/model.py
@@ -185,34 +185,6 @@
     def typedescrof(TYPE):
         raise NotImplementedError
 
-    # ---------- float-like storage ----------
-    # On 32-bit, we want to absure occasionally the storage in BoxFloat
-    # and ConstFloat to store a 64-bit integer instead of a float.
-    # This is done by using the following helpers.  A "floatish" is either
-    # a float or a longlong, suitable for storage.
-
-    def check_floatish(self, xish):
-        if self.supports_longlong:
-            assert isinstance(xish, r_longlong)
-        else:
-            assert isinstance(xish, float)
-
-    def getfloatish(self, x):
-        """Return the float 'x' as a floatish."""
-        assert isinstance(x, float)
-        if self.supports_longlong:
-            return float2longlong(x)
-        else:
-            return x
-
-    def getfloat(self, xish):
-        """Return the floatish 'x' as a float."""
-        self.check_floatish(xish)
-        if self.supports_longlong:
-            return longlong2float(xish)
-        else:
-            return xish
-
     # ---------- the backend-dependent operations ----------
 
     # lltype specific operations

From commits-noreply at bitbucket.org  Sat Feb 12 17:40:18 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sat, 12 Feb 2011 17:40:18 +0100 (CET)
Subject: [pypy-svn] pypy jit-longlong-2: In-progress.
Message-ID: <20110212164018.20CBA282B8B@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: jit-longlong-2
Changeset: r41846:22a8b3065cd7
Date: 2011-02-11 17:20 +0100
http://bitbucket.org/pypy/pypy/changeset/22a8b3065cd7/

Log:	In-progress.

diff --git a/pypy/jit/metainterp/test/test_executor.py b/pypy/jit/metainterp/test/test_executor.py
--- a/pypy/jit/metainterp/test/test_executor.py
+++ b/pypy/jit/metainterp/test/test_executor.py
@@ -10,6 +10,7 @@
 from pypy.jit.metainterp.history import BoxFloat, ConstFloat
 from pypy.jit.metainterp.history import AbstractDescr, Box
 from pypy.jit.metainterp import history
+from pypy.jit.codewriter import longlong
 from pypy.jit.backend.model import AbstractCPU
 from pypy.rpython.lltypesystem import  llmemory, rffi
 
@@ -59,11 +60,17 @@
 
     def bh_call_f(self, func, calldescr, args_i, args_r, args_f):
         self.fakecalled = (func, calldescr, args_i, args_r, args_f)
-        return 42.5
+        return longlong.getfloatstorage(42.5)
 
     def bh_strsetitem(self, string, index, newvalue):
         self.fakestrsetitem = (string, index, newvalue)
 
+def boxfloat(x):
+    return BoxFloat(longlong.getfloatstorage(x))
+
+def constfloat(x):
+    return ConstFloat(longlong.getfloatstorage(x))
+
 
 def test_execute():
     cpu = FakeCPU()
@@ -76,12 +83,14 @@
 def test_execute_varargs():
     cpu = FakeCPU()
     descr = FakeCallDescr()
-    argboxes = [BoxInt(99999), BoxInt(321), ConstFloat(2.25), ConstInt(123),
-                BoxPtr(), BoxFloat(5.5)]
+    argboxes = [BoxInt(99999), BoxInt(321), constfloat(2.25), ConstInt(123),
+                BoxPtr(), boxfloat(5.5)]
     box = execute_varargs(cpu, FakeMetaInterp(), rop.CALL, argboxes, descr)
-    assert box.value == 42.5
+    assert box.getfloat() == 42.5
     assert cpu.fakecalled == (99999, descr, [321, 123],
-                              [ConstPtr.value], [2.25, 5.5])
+                              [ConstPtr.value],
+                              [longlong.getfloatstorage(2.25),
+                               longlong.getfloatstorage(5.5)])
 
 def test_execute_nonspec():
     cpu = FakeCPU()
@@ -91,7 +100,7 @@
     argboxes = [BoxInt(321), ConstInt(123)]
     box = execute_nonspec(cpu, FakeMetaInterp(), rop.CALL,
                           argboxes, FakeCallDescr())
-    assert box.value == 42.5
+    assert box.getfloat() == 42.5
     # arity == 0
     box = execute_nonspec(cpu, None, rop.NEW, [], descr)
     assert box.value.fakeargs == ('new', descr)
@@ -100,16 +109,16 @@
     box = execute_nonspec(cpu, None, rop.ARRAYLEN_GC, [box1], descr)
     assert box.value == 55
     # arity == 2
-    box2 = BoxFloat(222.2)
+    box2 = boxfloat(222.2)
     fielddescr = FakeFieldDescr()
     execute_nonspec(cpu, None, rop.SETFIELD_GC, [box1, box2], fielddescr)
-    assert cpu.fakesetfield == (box1.value, box2.value, fielddescr)
+    assert cpu.fakesetfield == (box1.value, box2.valuestorage, fielddescr)
     # arity == 3
     box3 = BoxInt(33)
     arraydescr = FakeArrayDescr()
     execute_nonspec(cpu, None, rop.SETARRAYITEM_GC, [box1, box3, box2],
                     arraydescr)
-    assert cpu.fakesetarrayitem == (box1.value, box3.value, box2.value,
+    assert cpu.fakesetarrayitem == (box1.value, box3.value, box2.valuestorage,
                                     arraydescr)
     # cases without descr
     # arity == 1
@@ -288,7 +297,7 @@
         boxargs = []
         for x in args:
             if isinstance(x, float):
-                boxargs.append(BoxFloat(x))
+                boxargs.append(boxfloat(x))
             else:
                 boxargs.append(BoxInt(x))
         yield opnum, boxargs, rettype, retvalue
@@ -299,7 +308,7 @@
             if (isinstance(args[0], float) and
                 isinstance(args[1], float) and
                 args[0] == args[1]):
-                commonbox = BoxFloat(args[0])
+                commonbox = boxfloat(args[0])
                 yield opnum, [commonbox, commonbox], rettype, retvalue
 
 def test_float_ops():
@@ -319,8 +328,8 @@
         arg1 = ConstInt(a)
         arg2 = ConstInt(b)
     elif n[0:5] == 'FLOAT':
-        arg1 = ConstFloat(float(a))
-        arg2 = ConstFloat(float(b))
+        arg1 = constfloat(float(a))
+        arg2 = constfloat(float(b))
     elif n[0:3] == 'PTR':
         arg1 = ConstPtr(rffi.cast(llmemory.GCREF, a))
         arg2 = ConstPtr(rffi.cast(llmemory.GCREF, b))

diff --git a/pypy/jit/metainterp/blackhole.py b/pypy/jit/metainterp/blackhole.py
--- a/pypy/jit/metainterp/blackhole.py
+++ b/pypy/jit/metainterp/blackhole.py
@@ -612,7 +612,7 @@
 
     @arguments("i", returns="f")
     def bhimpl_cast_int_to_float(a):
-        return float(a)
+        x = float(a)
         return longlong.getfloatstorage(x)
 
     # ----------

From commits-noreply at bitbucket.org  Sat Feb 12 17:40:18 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sat, 12 Feb 2011 17:40:18 +0100 (CET)
Subject: [pypy-svn] pypy jit-longlong-2: Fix the test.
Message-ID: <20110212164018.E37E8282B8B@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: jit-longlong-2
Changeset: r41847:37d0f3e71950
Date: 2011-02-11 17:38 +0100
http://bitbucket.org/pypy/pypy/changeset/37d0f3e71950/

Log:	Fix the test.

diff --git a/pypy/jit/backend/test/runner_test.py b/pypy/jit/backend/test/runner_test.py
--- a/pypy/jit/backend/test/runner_test.py
+++ b/pypy/jit/backend/test/runner_test.py
@@ -14,9 +14,15 @@
 from pypy.rpython.ootypesystem import ootype
 from pypy.rpython.annlowlevel import llhelper
 from pypy.rpython.llinterp import LLException
-from pypy.jit.codewriter import heaptracker
+from pypy.jit.codewriter import heaptracker, longlong
 from pypy.rlib.rarithmetic import intmask
 
+def boxfloat(x):
+    return BoxFloat(longlong.getfloatstorage(x))
+
+def constfloat(x):
+    return ConstFloat(longlong.getfloatstorage(x))
+
 
 class Runner(object):
 
@@ -36,7 +42,7 @@
                 self.cpu.set_future_value_ref(j, box.getref_base())
                 j += 1
             elif isinstance(box, BoxFloat):
-                self.cpu.set_future_value_float(j, box.getfloat())
+                self.cpu.set_future_value_float(j, box.getfloatstorage())
                 j += 1
             else:
                 raise NotImplementedError(box)
@@ -344,7 +350,10 @@
         from pypy.jit.metainterp.test.test_executor import get_float_tests
         for opnum, boxargs, rettype, retvalue in get_float_tests(self.cpu):
             res = self.execute_operation(opnum, boxargs, rettype)
-            assert res.value == retvalue
+            if isinstance(res, BoxFloat):
+                assert res.getfloat() == retvalue
+            else:
+                assert res.value == retvalue
 
     def test_ovf_operations(self, reversed=False):
         minint = -sys.maxint-1
@@ -424,8 +433,8 @@
             calldescr = cpu.calldescrof(FTP, FTP.ARGS, FTP.RESULT)
             x = cpu.bh_call_f(self.get_funcbox(cpu, func_ptr).value,
                               calldescr,
-                              [42], None, [3.5])
-            assert x == 3.5 - 42
+                              [42], None, [longlong.getfloatstorage(3.5)])
+            assert longlong.getrealfloat(x) == 3.5 - 42
 
     def test_call(self):
         from pypy.rlib.libffi import types
@@ -472,13 +481,13 @@
             func_ptr = llhelper(FPTR, func)
             calldescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
             funcbox = self.get_funcbox(cpu, func_ptr)
-            args = ([BoxFloat(.1) for i in range(7)] +
-                    [BoxInt(1), BoxInt(2), BoxFloat(.2), BoxFloat(.3),
-                     BoxFloat(.4)])
+            args = ([boxfloat(.1) for i in range(7)] +
+                    [BoxInt(1), BoxInt(2), boxfloat(.2), boxfloat(.3),
+                     boxfloat(.4)])
             res = self.execute_operation(rop.CALL,
                                          [funcbox] + args,
                                          'float', descr=calldescr)
-            assert abs(res.value - 4.6) < 0.0001
+            assert abs(res.getfloat() - 4.6) < 0.0001
 
     def test_call_many_arguments(self):
         # Test calling a function with a large number of arguments (more than
@@ -592,17 +601,17 @@
         assert res.value == null_const.value
         if self.cpu.supports_floats:
             floatdescr = self.cpu.fielddescrof(self.S, 'float')
-            self.execute_operation(rop.SETFIELD_GC, [t_box, BoxFloat(3.4)],
+            self.execute_operation(rop.SETFIELD_GC, [t_box, boxfloat(3.4)],
                                    'void', descr=floatdescr)
             res = self.execute_operation(rop.GETFIELD_GC, [t_box],
                                          'float', descr=floatdescr)
-            assert res.value == 3.4
+            assert res.getfloat() == 3.4
             #
-            self.execute_operation(rop.SETFIELD_GC, [t_box, ConstFloat(-3.6)],
+            self.execute_operation(rop.SETFIELD_GC, [t_box, constfloat(-3.6)],
                                    'void', descr=floatdescr)
             res = self.execute_operation(rop.GETFIELD_GC, [t_box],
                                          'float', descr=floatdescr)
-            assert res.value == -3.6
+            assert res.getfloat() == -3.6
 
 
     def test_passing_guards(self):
@@ -618,7 +627,7 @@
                (rop.GUARD_ISNULL, [nullbox])
                ])
         if self.cpu.supports_floats:
-            all.append((rop.GUARD_VALUE, [BoxFloat(3.5), ConstFloat(3.5)]))
+            all.append((rop.GUARD_VALUE, [boxfloat(3.5), constfloat(3.5)]))
         for (opname, args) in all:
             assert self.execute_operation(opname, args, 'void') == None
             assert not self.guard_failed
@@ -644,7 +653,7 @@
                (rop.GUARD_NONNULL, [nullbox]),
                (rop.GUARD_ISNULL, [t_box])])
         if self.cpu.supports_floats:
-            all.append((rop.GUARD_VALUE, [BoxFloat(-1.0), ConstFloat(1.0)]))
+            all.append((rop.GUARD_VALUE, [boxfloat(-1.0), constfloat(1.0)]))
         for opname, args in all:
             assert self.execute_operation(opname, args, 'void') == None
             assert self.guard_failed
@@ -809,17 +818,17 @@
             a_box, A = self.alloc_array_of(lltype.Float, 31)
             arraydescr = self.cpu.arraydescrof(A)
             self.execute_operation(rop.SETARRAYITEM_GC, [a_box, BoxInt(1),
-                                                         BoxFloat(3.5)],
+                                                         boxfloat(3.5)],
                                    'void', descr=arraydescr)
             self.execute_operation(rop.SETARRAYITEM_GC, [a_box, BoxInt(2),
-                                                         ConstFloat(4.5)],
+                                                         constfloat(4.5)],
                                    'void', descr=arraydescr)
             r = self.execute_operation(rop.GETARRAYITEM_GC, [a_box, BoxInt(1)],
                                        'float', descr=arraydescr)
-            assert r.value == 3.5
+            assert r.getfloat() == 3.5
             r = self.execute_operation(rop.GETARRAYITEM_GC, [a_box, BoxInt(2)],
                                        'float', descr=arraydescr)
-            assert r.value == 4.5
+            assert r.getfloat() == 4.5
 
         # For platforms where sizeof(INT) != sizeof(Signed) (ie, x86-64)
         a_box, A = self.alloc_array_of(rffi.INT, 342)
@@ -919,10 +928,10 @@
         assert r.value == u_box.value
 
         if self.cpu.supports_floats:
-            r = self.execute_operation(rop.SAME_AS, [ConstFloat(5.5)], 'float')
-            assert r.value == 5.5
-            r = self.execute_operation(rop.SAME_AS, [BoxFloat(5.5)], 'float')
-            assert r.value == 5.5
+            r = self.execute_operation(rop.SAME_AS, [constfloat(5.5)], 'float')
+            assert r.getfloat() == 5.5
+            r = self.execute_operation(rop.SAME_AS, [boxfloat(5.5)], 'float')
+            assert r.getfloat() == 5.5
 
     def test_virtual_ref(self):
         pass   # VIRTUAL_REF must not reach the backend nowadays
@@ -993,7 +1002,7 @@
                     p = lltype.malloc(S)
                     values.append(lltype.cast_opaque_ptr(llmemory.GCREF, p))
                 elif isinstance(box, BoxFloat):
-                    values.append(r.random())
+                    values.append(longlong.getfloatstorage(r.random()))
                 else:
                     assert 0
             values[index_counter] = 11
@@ -1041,7 +1050,7 @@
         faildescr1 = BasicFailDescr(1)
         faildescr2 = BasicFailDescr(2)
         operations = [
-            ResOperation(rop.FLOAT_LE, [fboxes[0], ConstFloat(9.2)], i2),
+            ResOperation(rop.FLOAT_LE, [fboxes[0], constfloat(9.2)], i2),
             ResOperation(rop.GUARD_TRUE, [i2], None, descr=faildescr1),
             ResOperation(rop.FINISH, fboxes, None, descr=faildescr2),
             ]
@@ -1052,20 +1061,22 @@
         fboxes2 = [BoxFloat() for i in range(12)]
         f3 = BoxFloat()
         bridge = [
-            ResOperation(rop.FLOAT_SUB, [fboxes2[0], ConstFloat(1.0)], f3),
+            ResOperation(rop.FLOAT_SUB, [fboxes2[0], constfloat(1.0)], f3),
             ResOperation(rop.JUMP, [f3] + fboxes2[1:], None, descr=looptoken),
         ]
 
         self.cpu.compile_bridge(faildescr1, fboxes2, bridge, looptoken)
 
         for i in range(len(fboxes)):
-            self.cpu.set_future_value_float(i, 13.5 + 6.73 * i)
+            x = 13.5 + 6.73 * i
+            self.cpu.set_future_value_float(i, longlong.getfloatstorage(x))
         fail = self.cpu.execute_token(looptoken)
         assert fail.identifier == 2
         res = self.cpu.get_latest_value_float(0)
-        assert res == 8.5
+        assert longlong.getrealfloat(res) == 8.5
         for i in range(1, len(fboxes)):
-            assert self.cpu.get_latest_value_float(i) == 13.5 + 6.73 * i
+            got = longlong.getrealfloat(self.cpu.get_latest_value_float(i))
+            assert got == 13.5 + 6.73 * i
 
     def test_integers_and_guards(self):
         for opname, compare in [
@@ -1142,11 +1153,11 @@
                     if combinaison[0] == 'b':
                         fbox1 = BoxFloat()
                     else:
-                        fbox1 = ConstFloat(-4.5)
+                        fbox1 = constfloat(-4.5)
                     if combinaison[1] == 'b':
                         fbox2 = BoxFloat()
                     else:
-                        fbox2 = ConstFloat(-4.5)
+                        fbox2 = constfloat(-4.5)
                     b1 = BoxInt()
                     faildescr1 = BasicFailDescr(1)
                     faildescr2 = BasicFailDescr(2)
@@ -1170,10 +1181,12 @@
                                 if test2 == -4.5 or combinaison[1] == 'b':
                                     n = 0
                                     if combinaison[0] == 'b':
-                                        cpu.set_future_value_float(n, test1)
+                                        cpu.set_future_value_float(
+                                            n, longlong.getfloatstorage(test1))
                                         n += 1
                                     if combinaison[1] == 'b':
-                                        cpu.set_future_value_float(n, test2)
+                                        cpu.set_future_value_float(
+                                            n, longlong.getfloatstorage(test2))
                                         n += 1
                                     fail = cpu.execute_token(looptoken)
                                     #
@@ -1223,7 +1236,7 @@
             if isinstance(box, BoxInt):
                 self.cpu.set_future_value_int(i, box.getint())
             elif isinstance(box, BoxFloat):
-                self.cpu.set_future_value_float(i, box.getfloat())
+                self.cpu.set_future_value_float(i, box.getfloatstorage())
             else:
                 assert 0
         #
@@ -1237,12 +1250,12 @@
         from pypy.rlib.rarithmetic import INFINITY, NAN, isinf, isnan
         from pypy.jit.metainterp.resoperation import opname
 
-        fzer = BoxFloat(0.0)
-        fone = BoxFloat(1.0)
-        fmqr = BoxFloat(-0.25)
-        finf = BoxFloat(INFINITY)
-        fmnf = BoxFloat(-INFINITY)
-        fnan = BoxFloat(NAN)
+        fzer = boxfloat(0.0)
+        fone = boxfloat(1.0)
+        fmqr = boxfloat(-0.25)
+        finf = boxfloat(INFINITY)
+        fmnf = boxfloat(-INFINITY)
+        fnan = boxfloat(NAN)
 
         all_cases_unary =  [(a,)   for a in [fzer,fone,fmqr,finf,fmnf,fnan]]
         all_cases_binary = [(a, b) for a in [fzer,fone,fmqr,finf,fmnf,fnan]
@@ -1252,7 +1265,7 @@
 
         def nan_and_infinity(opnum, realoperation, testcases):
             for testcase in testcases:
-                realvalues = [b.value for b in testcase]
+                realvalues = [b.getfloat() for b in testcase]
                 expected = realoperation(*realvalues)
                 if isinstance(expected, float):
                     expectedtype = 'float'
@@ -1261,15 +1274,17 @@
                 got = self.execute_operation(opnum, list(testcase),
                                              expectedtype)
                 if isnan(expected):
-                    ok = isnan(got.value)
+                    ok = isnan(got.getfloat())
                 elif isinf(expected):
-                    ok = isinf(got.value)
+                    ok = isinf(got.getfloat())
+                elif isinstance(got, BoxFloat):
+                    ok = (got.getfloat() == expected)
                 else:
-                    ok = (got.value == expected)
+                    ok = got.value == expected
                 if not ok:
                     raise AssertionError("%s(%s): got %r, expected %r" % (
                         opname[opnum], ', '.join(map(repr, realvalues)),
-                        got.value, expected))
+                        got.getfloat(), expected))
                 # if we expect a boolean, also check the combination with
                 # a GUARD_TRUE or GUARD_FALSE
                 if isinstance(expected, bool):
@@ -1289,7 +1304,8 @@
                         self.cpu.compile_loop(unique_testcase_list, operations,
                                               looptoken)
                         for i, box in enumerate(unique_testcase_list):
-                            self.cpu.set_future_value_float(i, box.value)
+                            self.cpu.set_future_value_float(
+                                i, box.getfloatstorage())
                         fail = self.cpu.execute_token(looptoken)
                         if fail.identifier != 5 - (expected_id^expected):
                             if fail.identifier == 4:
@@ -1758,7 +1774,8 @@
         self.cpu.set_future_value_int(1, 0)
         fail = self.cpu.execute_token(looptoken)
         assert fail.identifier == 0
-        assert self.cpu.get_latest_value_float(0) == 42.5
+        x = self.cpu.get_latest_value_float(0)
+        assert longlong.getrealfloat(x) == 42.5
         assert values == []
 
         self.cpu.set_future_value_int(0, 10)
@@ -1766,7 +1783,8 @@
         fail = self.cpu.execute_token(looptoken)
         assert fail.identifier == 1
         assert self.cpu.get_latest_value_int(0) == 1
-        assert self.cpu.get_latest_value_float(1) == 42.5
+        x = self.cpu.get_latest_value_float(1)
+        assert longlong.getrealfloat(x) == 42.5
         assert self.cpu.get_latest_value_int(2) == 10
         assert values == [1, 10]
 
@@ -1801,9 +1819,10 @@
             descr_C = cpu.arraydescrof(C)
             x = cpu.bh_getarrayitem_gc_f(
                 descr_C, lltype.cast_opaque_ptr(llmemory.GCREF, c), 3)
-            assert x == 3.5
+            assert longlong.getrealfloat(x) == 3.5
             cpu.bh_setarrayitem_gc_f(
-                descr_C, lltype.cast_opaque_ptr(llmemory.GCREF, c), 4, 4.5)
+                descr_C, lltype.cast_opaque_ptr(llmemory.GCREF, c), 4,
+                longlong.getfloatstorage(4.5))
             assert c[4] == 4.5
         s = rstr.mallocstr(6)
         x = cpu.bh_strlen(lltype.cast_opaque_ptr(llmemory.GCREF, s))
@@ -1857,13 +1876,13 @@
             descrfld_z = cpu.fielddescrof(S, 'z')
             cpu.bh_setfield_gc_f(
                 lltype.cast_opaque_ptr(llmemory.GCREF, s),
-                descrfld_z, 3.5)
+                descrfld_z, longlong.getfloatstorage(3.5))
             assert s.z == 3.5
             s.z = 3.2
             x = cpu.bh_getfield_gc_f(
                 lltype.cast_opaque_ptr(llmemory.GCREF, s),
                 descrfld_z)
-            assert x == 3.2
+            assert longlong.getrealfloat(x) == 3.2
         ### we don't support in the JIT for now GC pointers
         ### stored inside non-GC structs.
         #descrfld_ry = cpu.fielddescrof(RS, 'y')
@@ -2002,12 +2021,13 @@
     def test_assembler_call_float(self):
         called = []
         def assembler_helper(failindex, virtualizable):
-            assert self.cpu.get_latest_value_float(0) == 1.2 + 3.2
+            x = self.cpu.get_latest_value_float(0)
+            assert longlong.getrealfloat(x) == 1.2 + 3.2
             called.append(failindex)
-            return 13.5
+            return longlong.getfloatstorage(13.5)
 
         FUNCPTR = lltype.Ptr(lltype.FuncType([lltype.Signed, llmemory.GCREF],
-                                             lltype.Float))
+                                             longlong.FLOATSTORAGE))
         class FakeJitDriverSD:
             index_of_virtualizable = -1
             _assembler_helper_ptr = llhelper(FUNCPTR, assembler_helper)
@@ -2028,10 +2048,11 @@
         looptoken = LoopToken()
         looptoken.outermost_jitdriver_sd = FakeJitDriverSD()
         self.cpu.compile_loop(loop.inputargs, loop.operations, looptoken)
-        self.cpu.set_future_value_float(0, 1.2)
-        self.cpu.set_future_value_float(1, 2.3)
+        self.cpu.set_future_value_float(0, longlong.getfloatstorage(1.2))
+        self.cpu.set_future_value_float(1, longlong.getfloatstorage(2.3))
         res = self.cpu.execute_token(looptoken)
-        assert self.cpu.get_latest_value_float(0) == 1.2 + 2.3
+        x = self.cpu.get_latest_value_float(0)
+        assert longlong.getrealfloat(x) == 1.2 + 2.3
         ops = '''
         [f4, f5]
         f3 = call_assembler(f4, f5, descr=looptoken)
@@ -2041,10 +2062,11 @@
         loop = parse(ops, namespace=locals())
         othertoken = LoopToken()
         self.cpu.compile_loop(loop.inputargs, loop.operations, othertoken)
-        self.cpu.set_future_value_float(0, 1.2)
-        self.cpu.set_future_value_float(1, 3.2)
+        self.cpu.set_future_value_float(0, longlong.getfloatstorage(1.2))
+        self.cpu.set_future_value_float(1, longlong.getfloatstorage(3.2))
         res = self.cpu.execute_token(othertoken)
-        assert self.cpu.get_latest_value_float(0) == 13.5
+        x = self.cpu.get_latest_value_float(0)
+        assert longlong.getrealfloat(x) == 13.5
         assert called
 
         # test the fast path, which should not call assembler_helper()
@@ -2053,10 +2075,11 @@
         try:
             othertoken = LoopToken()
             self.cpu.compile_loop(loop.inputargs, loop.operations, othertoken)
-            self.cpu.set_future_value_float(0, 1.2)
-            self.cpu.set_future_value_float(1, 3.2)
+            self.cpu.set_future_value_float(0, longlong.getfloatstorage(1.2))
+            self.cpu.set_future_value_float(1, longlong.getfloatstorage(3.2))
             res = self.cpu.execute_token(othertoken)
-            assert self.cpu.get_latest_value_float(0) == 1.2 + 3.2
+            x = self.cpu.get_latest_value_float(0)
+            assert longlong.getrealfloat(x) == 1.2 + 3.2
             assert not called
         finally:
             del self.cpu.done_with_this_frame_float_v
@@ -2088,9 +2111,10 @@
     def test_redirect_call_assembler(self):
         called = []
         def assembler_helper(failindex, virtualizable):
-            assert self.cpu.get_latest_value_float(0) == 1.25 + 3.25
+            x = self.cpu.get_latest_value_float(0)
+            assert longlong.getrealfloat(x) == 1.25 + 3.25
             called.append(failindex)
-            return 13.5
+            return longlong.getfloatstorage(13.5)
 
         FUNCPTR = lltype.Ptr(lltype.FuncType([lltype.Signed, llmemory.GCREF],
                                              lltype.Float))
@@ -2113,10 +2137,11 @@
         looptoken = LoopToken()
         looptoken.outermost_jitdriver_sd = FakeJitDriverSD()
         self.cpu.compile_loop(loop.inputargs, loop.operations, looptoken)
-        self.cpu.set_future_value_float(0, 1.25)
-        self.cpu.set_future_value_float(1, 2.35)
+        self.cpu.set_future_value_float(0, longlong.getfloatstorage(1.25))
+        self.cpu.set_future_value_float(1, longlong.getfloatstorage(2.35))
         res = self.cpu.execute_token(looptoken)
-        assert self.cpu.get_latest_value_float(0) == 1.25 + 2.35
+        x = self.cpu.get_latest_value_float(0)
+        assert longlong.getrealfloat(x) == 1.25 + 2.35
         assert not called
 
         ops = '''
@@ -2130,10 +2155,11 @@
         self.cpu.compile_loop(loop.inputargs, loop.operations, othertoken)
 
         # normal call_assembler: goes to looptoken
-        self.cpu.set_future_value_float(0, 1.25)
-        self.cpu.set_future_value_float(1, 3.25)
+        self.cpu.set_future_value_float(0, longlong.getfloatstorage(1.25))
+        self.cpu.set_future_value_float(1, longlong.getfloatstorage(3.25))
         res = self.cpu.execute_token(othertoken)
-        assert self.cpu.get_latest_value_float(0) == 13.5
+        x = self.cpu.get_latest_value_float(0)
+        assert longlong.getrealfloat(x) == 13.5
         assert called
         del called[:]
 
@@ -2151,10 +2177,12 @@
         self.cpu.redirect_call_assembler(looptoken, looptoken2)
 
         # now, our call_assembler should go to looptoken2
-        self.cpu.set_future_value_float(0, 6.0)
-        self.cpu.set_future_value_float(1, 1.5)    # 6.0-1.5 == 1.25+3.25
+        self.cpu.set_future_value_float(0, longlong.getfloatstorage(6.0))
+        self.cpu.set_future_value_float(1, longlong.getfloatstorage(1.5))
+                                                       # 6.0-1.5 == 1.25+3.25
         res = self.cpu.execute_token(othertoken)
-        assert self.cpu.get_latest_value_float(0) == 13.5
+        x = self.cpu.get_latest_value_float(0)
+        assert longlong.getrealfloat(x) == 13.5
         assert called
 
     def test_short_result_of_getfield_direct(self):
@@ -2380,10 +2408,8 @@
         FUNC = self.FuncType([lltype.SignedLongLong], lltype.SignedLongLong)
         FPTR = self.Ptr(FUNC)
         calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
-        valuef = longlong2float(value)
-        xf = self.cpu.bh_call_f(self.get_funcbox(self.cpu, f).value,
-                                calldescr, None, None, [valuef])
-        x = float2longlong(xf)
+        x = self.cpu.bh_call_f(self.get_funcbox(self.cpu, f).value,
+                               calldescr, None, None, [value])
         assert x == expected
 
     def test_longlong_result_of_call_compiled(self):
@@ -2411,11 +2437,9 @@
         FPTR = self.Ptr(FUNC)
         calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
         funcbox = self.get_funcbox(self.cpu, f)
-        valuef = longlong2float(value)
-        resf = self.execute_operation(rop.CALL, [funcbox, BoxFloat(valuef)],
+        res = self.execute_operation(rop.CALL, [funcbox, BoxFloat(value)],
                                      'float', descr=calldescr)
-        x = float2longlong(resf.value)
-        assert x == expected
+        assert res.getfloatstorage() == expected
 
     def test_free_loop_and_bridges(self):
         from pypy.jit.backend.llsupport.llmodel import AbstractLLCPU

From commits-noreply at bitbucket.org  Sat Feb 12 17:40:19 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sat, 12 Feb 2011 17:40:19 +0100 (CET)
Subject: [pypy-svn] pypy jit-longlong-2: Add a missing module.
Message-ID: <20110212164019.AC7FE282B8B@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: jit-longlong-2
Changeset: r41848:93e5b69cfa7b
Date: 2011-02-11 17:39 +0100
http://bitbucket.org/pypy/pypy/changeset/93e5b69cfa7b/

Log:	Add a missing module.

diff --git a/pypy/jit/codewriter/longlong.py b/pypy/jit/codewriter/longlong.py
new file mode 100644
--- /dev/null
+++ b/pypy/jit/codewriter/longlong.py
@@ -0,0 +1,42 @@
+"""
+Support for 'long long' on 32-bits: this is done by casting all floats
+to long longs, and using long longs systematically.
+
+On 64-bit platforms, we use float directly to avoid the cost of
+converting them back and forth.
+"""
+
+import sys
+from pypy.rpython.lltypesystem import lltype
+
+
+if sys.maxint > 2147483647:
+    # ---------- 64-bit platform ----------
+    # the type FloatStorage is just a float
+
+    from pypy.rlib.objectmodel import compute_hash
+
+    supports_longlong = False
+    r_float_storage = float
+    FLOATSTORAGE = lltype.Float
+
+    getfloatstorage = lambda x: x
+    getrealfloat    = lambda x: x
+    gethash         = compute_hash
+
+    # -------------------------------------
+else:
+    # ---------- 32-bit platform ----------
+    # the type FloatStorage is r_longlong, and conversion is needed
+
+    from pypy.rlib import rarithmetic, longlong2float
+
+    supports_longlong = True
+    r_float_storage = rarithmetic.r_longlong
+    FLOATSTORAGE = lltype.SignedLongLong
+
+    getfloatstorage = longlong2float.float2longlong
+    getrealfloat    = longlong2float.longlong2float
+    gethash         = lambda xll: xll - (xll >> 32)
+
+    # -------------------------------------

From commits-noreply at bitbucket.org  Sat Feb 12 17:40:24 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sat, 12 Feb 2011 17:40:24 +0100 (CET)
Subject: [pypy-svn] pypy jit-longlong-2: Simplify 'llong_from_two_ints' into
	just	'llong_from_uint'.
Message-ID: <20110212164024.693EF2A2031@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: jit-longlong-2
Changeset: r41849:5fddae1b98e2
Date: 2011-02-12 13:54 +0100
http://bitbucket.org/pypy/pypy/changeset/5fddae1b98e2/

Log:	Simplify 'llong_from_two_ints' into just 'llong_from_uint'.

diff --git a/pypy/jit/backend/x86/assembler.py b/pypy/jit/backend/x86/assembler.py
--- a/pypy/jit/backend/x86/assembler.py
+++ b/pypy/jit/backend/x86/assembler.py
@@ -1151,24 +1151,11 @@
             self.mc.MOVD_xr(loc2.value, edx.value)
             self.mc.PUNPCKLDQ_xx(resloc.value, loc2.value)
 
-    def genop_llong_from_two_ints(self, op, arglocs, resloc):
+    def genop_llong_from_uint(self, op, arglocs, resloc):
+        loc1, = arglocs
         assert isinstance(resloc, RegLoc)
-        loc1, loc2, loc3 = arglocs
-        #
-        if isinstance(loc1, ConstFloatLoc):
-            self.mc.MOVSD(resloc, loc1)
-        else:
-            assert isinstance(loc1, RegLoc)
-            self.mc.MOVD_xr(resloc.value, loc1.value)
-        #
-        if loc2 is not None:
-            assert isinstance(loc3, RegLoc)
-            if isinstance(loc2, ConstFloatLoc):
-                self.mc.MOVSD(loc3, loc2)
-            else:
-                assert isinstance(loc2, RegLoc)
-                self.mc.MOVD_xr(loc3.value, loc2.value)
-            self.mc.PUNPCKLDQ_xx(resloc.value, loc3.value)
+        assert isinstance(loc1, RegLoc)
+        self.mc.MOVD_xr(resloc.value, loc1.value)
 
     def genop_llong_eq(self, op, arglocs, resloc):
         loc1, loc2, locxtmp = arglocs

diff --git a/pypy/jit/codewriter/support.py b/pypy/jit/codewriter/support.py
--- a/pypy/jit/codewriter/support.py
+++ b/pypy/jit/codewriter/support.py
@@ -300,9 +300,8 @@
 def _ll_1_llong_from_int(x):
     return r_longlong(intmask(x))
 
-def _ll_2_llong_from_two_ints(x_lo, x_hi):
-    z = (r_ulonglong(r_uint(x_hi)) << 32) | r_ulonglong(r_uint(x_lo))
-    return u_to_longlong(z)
+def _ll_2_llong_from_uint(x):
+    return r_longlong(r_uint(x))
 
 def _ll_1_llong_to_int(xll):
     return intmask(xll)

diff --git a/pypy/jit/codewriter/effectinfo.py b/pypy/jit/codewriter/effectinfo.py
--- a/pypy/jit/codewriter/effectinfo.py
+++ b/pypy/jit/codewriter/effectinfo.py
@@ -71,7 +71,7 @@
     OS_LLONG_UGT                = 90
     OS_LLONG_UGE                = 91
     OS_LLONG_URSHIFT            = 92
-    OS_LLONG_FROM_TWO_INTS      = 93
+    OS_LLONG_FROM_UINT          = 93
 
     def __new__(cls, readonly_descrs_fields,
                 write_descrs_fields, write_descrs_arrays,

diff --git a/pypy/jit/codewriter/jtransform.py b/pypy/jit/codewriter/jtransform.py
--- a/pypy/jit/codewriter/jtransform.py
+++ b/pypy/jit/codewriter/jtransform.py
@@ -836,8 +836,7 @@
                           ('truncate_longlong_to_int', 'TO_INT'),
                           ('cast_float_to_longlong',   'FROM_FLOAT'),
                           ('cast_longlong_to_float',   'TO_FLOAT'),
-                          # internal pseuso-operation:
-                          ('two_ints_to_longlong',     'FROM_TWO_INTS'),
+                          ('cast_uint_to_longlong',    'FROM_UINT'),
                           ]:
         exec py.code.Source('''
             def rewrite_op_%s(self, op):
@@ -890,9 +889,7 @@
                 if rffi.cast(op.args[0].concretetype, -1) < 0:
                     opname = 'cast_int_to_longlong'
                 else:
-                    opname = 'two_ints_to_longlong'
-                    c_hi = Constant(0, lltype.Signed)
-                    args = [args[0], c_hi]
+                    opname = 'cast_uint_to_longlong'
             op1 = SpaceOperation(opname, args, op.result)
             return self.rewrite_operation(op1)
 

diff --git a/pypy/jit/backend/x86/regalloc.py b/pypy/jit/backend/x86/regalloc.py
--- a/pypy/jit/backend/x86/regalloc.py
+++ b/pypy/jit/backend/x86/regalloc.py
@@ -726,36 +726,11 @@
         self.PerformLLong(op, [loc1, loc2], loc0)
         self.rm.possibly_free_var(box)
 
-    def _consider_llong_from_two_ints(self, op):
+    def _consider_llong_from_uint(self, op):
         assert IS_X86_32
-        box1 = op.getarg(1)
-        box2 = op.getarg(2)
         loc0 = self.xrm.force_allocate_reg(op.result)
-        #
-        if isinstance(box1, ConstInt) and isinstance(box2, ConstInt):
-            # all-constant arguments: load the result value in a single step
-            value64 = r_longlong(box2.value) << 32
-            value64 |= r_longlong(r_uint(box1.value))
-            loc1 = self._loc_of_const_longlong(value64)
-            loc2 = None    # unused
-            loc3 = None    # unused
-        #
-        else:
-            tmpxvar = TempBox()
-            loc3 = self.xrm.force_allocate_reg(tmpxvar, [op.result])
-            self.xrm.possibly_free_var(tmpxvar)
-            #
-            if isinstance(box1, ConstInt):
-                loc1 = self._loc_of_const_longlong(r_longlong(box1.value))
-            else:
-                loc1 = self.rm.make_sure_var_in_reg(box1)
-            #
-            if isinstance(box2, ConstInt):
-                loc2 = self._loc_of_const_longlong(r_longlong(box2.value))
-            else:
-                loc2 = self.rm.make_sure_var_in_reg(box2, [box1])
-        #
-        self.PerformLLong(op, [loc1, loc2, loc3], loc0)
+        loc1 = self.rm.make_sure_var_in_reg(op.getarg(1))
+        self.PerformLLong(op, [loc1], loc0)
         self.rm.possibly_free_vars_for_op(op)
 
     def _call(self, op, arglocs, force_store=[], guard_not_forced_op=None):
@@ -805,8 +780,8 @@
                     return self._consider_llong_to_int(op)
                 if oopspecindex == EffectInfo.OS_LLONG_FROM_INT:
                     return self._consider_llong_from_int(op)
-                if oopspecindex == EffectInfo.OS_LLONG_FROM_TWO_INTS:
-                    return self._consider_llong_from_two_ints(op)
+                if oopspecindex == EffectInfo.OS_LLONG_FROM_UINT:
+                    return self._consider_llong_from_uint(op)
                 if (oopspecindex == EffectInfo.OS_LLONG_EQ or
                     oopspecindex == EffectInfo.OS_LLONG_NE):
                     return self._consider_llong_eq_ne_xx(op)

From commits-noreply at bitbucket.org  Sat Feb 12 17:40:32 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sat, 12 Feb 2011 17:40:32 +0100 (CET)
Subject: [pypy-svn] pypy jit-longlong-2: In-progress.
Message-ID: <20110212164032.1E88E2A2031@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: jit-longlong-2
Changeset: r41850:39cdf1fcafcb
Date: 2011-02-12 14:33 +0100
http://bitbucket.org/pypy/pypy/changeset/39cdf1fcafcb/

Log:	In-progress.

diff --git a/pypy/jit/backend/llgraph/runner.py b/pypy/jit/backend/llgraph/runner.py
--- a/pypy/jit/backend/llgraph/runner.py
+++ b/pypy/jit/backend/llgraph/runner.py
@@ -15,7 +15,7 @@
 from pypy.jit.backend import model
 from pypy.jit.backend.llgraph import llimpl, symbolic
 from pypy.jit.metainterp.typesystem import llhelper, oohelper
-from pypy.jit.codewriter import heaptracker
+from pypy.jit.codewriter import heaptracker, longlong
 from pypy.rlib import rgc
 
 class MiniStats:
@@ -181,7 +181,7 @@
                 elif isinstance(x, self.ts.ConstRef):
                     llimpl.compile_add_ref_const(c, x.value, self.ts.BASETYPE)
                 elif isinstance(x, history.ConstFloat):
-                    llimpl.compile_add_float_const(c, x.valuestorage)
+                    llimpl.compile_add_float_const(c, x.value)
                 else:
                     raise Exception("'%s' args contain: %r" % (op.getopname(),
                                                                x))
@@ -305,8 +305,12 @@
         for ARG in ARGS:
             token = history.getkind(ARG)
             if token != 'void':
+                if token == 'float' and longlong.is_longlong(ARG):
+                    token = 'L'
                 arg_types.append(token[0])
         token = history.getkind(RESULT)
+        if token == 'float' and longlong.is_longlong(RESULT):
+            token = 'L'
         return self.getdescr(0, token[0], extrainfo=extrainfo,
                              arg_types=''.join(arg_types))
 
@@ -464,7 +468,7 @@
         self._prepare_call(REF, calldescr, args_i, args_r, args_f)
         return llimpl.do_call_ptr(func)
     def bh_call_f(self, func, calldescr, args_i, args_r, args_f):
-        self._prepare_call(FLOAT, calldescr, args_i, args_r, args_f)
+        self._prepare_call(FLOAT + 'L', calldescr, args_i, args_r, args_f)
         return llimpl.do_call_float(func)
     def bh_call_v(self, func, calldescr, args_i, args_r, args_f):
         self._prepare_call('v', calldescr, args_i, args_r, args_f)
@@ -472,7 +476,7 @@
 
     def _prepare_call(self, resulttypeinfo, calldescr, args_i, args_r, args_f):
         assert isinstance(calldescr, Descr)
-        assert calldescr.typeinfo == resulttypeinfo
+        assert calldescr.typeinfo in resulttypeinfo
         if args_i is not None:
             for x in args_i:
                 llimpl.do_call_pushint(x)

diff --git a/pypy/jit/metainterp/history.py b/pypy/jit/metainterp/history.py
--- a/pypy/jit/metainterp/history.py
+++ b/pypy/jit/metainterp/history.py
@@ -144,6 +144,7 @@
     def get_return_type(self):
         """ Implement in call descr.
         Must return INT, REF, FLOAT, or 'v' for void.
+        On 32-bit (hack) it can also be 'L' for longlongs.
         """
         raise NotImplementedError
 
@@ -308,38 +309,34 @@
 
 class ConstFloat(Const):
     type = FLOAT
-    valuestorage = longlong.getfloatstorage(0.0)
-    _attrs_ = ('valuestorage',)
-
-    @property
-    def value(self):
-        XXX      # temporary
+    value = longlong.getfloatstorage(0.0)
+    _attrs_ = ('value',)
 
     def __init__(self, valuestorage):
         assert isinstance(valuestorage, longlong.r_float_storage)
-        self.valuestorage = valuestorage
+        self.value = valuestorage
 
     def clonebox(self):
-        return BoxFloat(self.valuestorage)
+        return BoxFloat(self.value)
 
     nonconstbox = clonebox
 
     def getfloatstorage(self):
-        return self.valuestorage
+        return self.value
 
     def _get_hash_(self):
-        return longlong.gethash(self.valuestorage)
+        return longlong.gethash(self.value)
 
     def set_future_value(self, cpu, j):
-        cpu.set_future_value_float(j, self.valuestorage)
+        cpu.set_future_value_float(j, self.value)
 
     def same_constant(self, other):
         if isinstance(other, ConstFloat):
-            return self.valuestorage == other.valuestorage
+            return self.value == other.value
         return False
 
     def nonnull(self):
-        return self.valuestorage != 0
+        return self.value != 0
 
     def _getrepr_(self):
         return self.getfloat()
@@ -555,36 +552,32 @@
 
 class BoxFloat(Box):
     type = FLOAT
-    _attrs_ = ('valuestorage',)
-
-    @property
-    def value(self):
-        XXX      # temporary
+    _attrs_ = ('value',)
 
     def __init__(self, valuestorage=longlong.getfloatstorage(0.0)):
         assert isinstance(valuestorage, longlong.r_float_storage)
-        self.valuestorage = valuestorage
+        self.value = valuestorage
 
     def forget_value(self):
-        self.valuestorage = longlong.getfloatstorage(0.0)
+        self.value = longlong.getfloatstorage(0.0)
 
     def clonebox(self):
-        return BoxFloat(self.valuestorage)
+        return BoxFloat(self.value)
 
     def constbox(self):
-        return ConstFloat(self.valuestorage)
+        return ConstFloat(self.value)
 
     def getfloatstorage(self):
-        return self.valuestorage
+        return self.value
 
     def _get_hash_(self):
-        return longlong.gethash(self.valuestorage)
+        return longlong.gethash(self.value)
 
     def set_future_value(self, cpu, j):
-        cpu.set_future_value_float(j, self.valuestorage)
+        cpu.set_future_value_float(j, self.value)
 
     def nonnull(self):
-        return self.valuestorage != 0
+        return self.value != 0
 
     def _getrepr_(self):
         return self.getfloat()

diff --git a/pypy/jit/metainterp/test/test_resume.py b/pypy/jit/metainterp/test/test_resume.py
--- a/pypy/jit/metainterp/test/test_resume.py
+++ b/pypy/jit/metainterp/test/test_resume.py
@@ -8,7 +8,7 @@
 from pypy.jit.metainterp.history import ConstPtr, ConstFloat
 from pypy.jit.metainterp.test.test_optimizeutil import LLtypeMixin
 from pypy.jit.metainterp import executor
-from pypy.jit.codewriter import heaptracker
+from pypy.jit.codewriter import heaptracker, longlong
 
 class Storage:
     rd_frame_info_list = None
@@ -114,7 +114,7 @@
                         callback_i(index, count_i); count_i += 1
                     elif ARG == llmemory.GCREF:
                         callback_r(index, count_r); count_r += 1
-                    elif ARG == lltype.Float:
+                    elif ARG == longlong.FLOATSTORAGE:
                         callback_f(index, count_f); count_f += 1
                     else:
                         assert 0
@@ -137,7 +137,8 @@
     reader.consume_one_section(bh)
     expected_i = [x for x in expected if lltype.typeOf(x) == lltype.Signed]
     expected_r = [x for x in expected if lltype.typeOf(x) == llmemory.GCREF]
-    expected_f = [x for x in expected if lltype.typeOf(x) == lltype.Float]
+    expected_f = [x for x in expected if lltype.typeOf(x) ==
+                                                      longlong.FLOATSTORAGE]
     assert bh.written_i == expected_i
     assert bh.written_r == expected_r
     assert bh.written_f == expected_f
@@ -749,7 +750,7 @@
 
 def test_ResumeDataLoopMemo_other():
     memo = ResumeDataLoopMemo(FakeMetaInterpStaticData())
-    const = ConstFloat(-1.0)
+    const = ConstFloat(longlong.getfloatstorage(-1.0))
     tagged = memo.getconst(const)
     index, tagbits = untag(tagged)
     assert tagbits == TAGCONST

diff --git a/pypy/jit/backend/x86/regalloc.py b/pypy/jit/backend/x86/regalloc.py
--- a/pypy/jit/backend/x86/regalloc.py
+++ b/pypy/jit/backend/x86/regalloc.py
@@ -71,11 +71,13 @@
 
     def convert_to_imm(self, c):
         adr = self.assembler.datablockwrapper.malloc_aligned(8, 8)
+        xxxxxxxxx
         rffi.cast(rffi.CArrayPtr(rffi.DOUBLE), adr)[0] = c.getfloat()
         return ConstFloatLoc(adr)
 
     def convert_to_imm_16bytes_align(self, c):
         adr = self.assembler.datablockwrapper.malloc_aligned(16, 16)
+        xxxxxxxxx
         rffi.cast(rffi.CArrayPtr(rffi.DOUBLE), adr)[0] = c.getfloat()
         rffi.cast(rffi.CArrayPtr(rffi.DOUBLE), adr)[1] = 0.0
         return ConstFloatLoc(adr)
@@ -221,6 +223,7 @@
                              selected_reg=None, need_lower_byte=False):
         if var.type == FLOAT:
             if isinstance(var, ConstFloat):
+                xxxxxxxxxxxxxx
                 return FloatImmedLoc(var.getfloat())
             return self.xrm.make_sure_var_in_reg(var, forbidden_vars,
                                                  selected_reg, need_lower_byte)
@@ -703,8 +706,7 @@
         self.xrm.possibly_free_var(op.getarg(1))
 
     def _loc_of_const_longlong(self, value64):
-        from pypy.rlib.longlong2float import longlong2float
-        c = ConstFloat(longlong2float(value64))
+        c = ConstFloat(value64)
         return self.xrm.convert_to_imm(c)
 
     def _consider_llong_from_int(self, op):

diff --git a/pypy/jit/metainterp/warmstate.py b/pypy/jit/metainterp/warmstate.py
--- a/pypy/jit/metainterp/warmstate.py
+++ b/pypy/jit/metainterp/warmstate.py
@@ -142,8 +142,7 @@
         if lltype.typeOf(value) is lltype.Float:
             value = longlong.getfloatstorage(value)
         else:
-            assert lltype.typeOf(value) in (lltype.SignedLongLong,
-                                            lltype.UnsignedLongLong)
+            assert longlong.is_longlong(lltype.typeOf(value))
             value = rffi.cast(lltype.SignedLongLong, value)
         cpu.set_future_value_float(j, value)
     else:

diff --git a/pypy/jit/metainterp/test/test_executor.py b/pypy/jit/metainterp/test/test_executor.py
--- a/pypy/jit/metainterp/test/test_executor.py
+++ b/pypy/jit/metainterp/test/test_executor.py
@@ -112,13 +112,13 @@
     box2 = boxfloat(222.2)
     fielddescr = FakeFieldDescr()
     execute_nonspec(cpu, None, rop.SETFIELD_GC, [box1, box2], fielddescr)
-    assert cpu.fakesetfield == (box1.value, box2.valuestorage, fielddescr)
+    assert cpu.fakesetfield == (box1.value, box2.value, fielddescr)
     # arity == 3
     box3 = BoxInt(33)
     arraydescr = FakeArrayDescr()
     execute_nonspec(cpu, None, rop.SETARRAYITEM_GC, [box1, box3, box2],
                     arraydescr)
-    assert cpu.fakesetarrayitem == (box1.value, box3.value, box2.valuestorage,
+    assert cpu.fakesetarrayitem == (box1.value, box3.value, box2.value,
                                     arraydescr)
     # cases without descr
     # arity == 1

diff --git a/pypy/jit/backend/test/runner_test.py b/pypy/jit/backend/test/runner_test.py
--- a/pypy/jit/backend/test/runner_test.py
+++ b/pypy/jit/backend/test/runner_test.py
@@ -425,6 +425,8 @@
         assert x == ord('B')
         if cpu.supports_floats:
             def func(f, i):
+                assert isinstance(f, float)
+                assert isinstance(i, int)
                 return f - float(i)
             FPTR = self.Ptr(self.FuncType([lltype.Float, lltype.Signed],
                                           lltype.Float))

diff --git a/pypy/jit/backend/llsupport/descr.py b/pypy/jit/backend/llsupport/descr.py
--- a/pypy/jit/backend/llsupport/descr.py
+++ b/pypy/jit/backend/llsupport/descr.py
@@ -5,9 +5,8 @@
 from pypy.jit.metainterp.history import BasicFailDescr, LoopToken, BoxFloat
 from pypy.jit.metainterp import history
 from pypy.jit.metainterp.resoperation import ResOperation, rop
-from pypy.jit.codewriter import heaptracker
+from pypy.jit.codewriter import heaptracker, longlong
 from pypy.rlib.rarithmetic import r_longlong, r_ulonglong
-from pypy.rlib.longlong2float import longlong2float, float2longlong
 
 # The point of the class organization in this file is to make instances
 # as compact as possible.  This is done by not storing the field size or
@@ -275,7 +274,10 @@
     def create_call_stub(self, rtyper, RESULT):
         def process(c):
             if c == 'L':
-                return 'float2longlong(%s)' % (process('f'),)
+                assert longlong.supports_longlong
+                c = 'f'
+            elif c == 'f' and longlong.supports_longlong:
+                return 'longlong.getrealfloat(%s)' % (process('L'),)
             arg = 'args_%s[%d]' % (c, seen[c])
             seen[c] += 1
             return arg
@@ -302,7 +304,9 @@
         elif self.get_return_type() == history.REF:
             result = 'lltype.cast_opaque_ptr(llmemory.GCREF, res)'
         elif self.get_return_type() == history.FLOAT:
-            result = 'cast_to_float(res)'
+            result = 'longlong.getfloatstorage(res)'
+        elif self.get_return_type() == 'L':
+            result = 'rffi.cast(lltype.SignedLongLong, res)'
         elif self.get_return_type() == history.VOID:
             result = 'None'
         else:
@@ -330,15 +334,6 @@
     def repr_of_descr(self):
         return '<%s>' % self._clsname
 
-def cast_to_float(x):
-    if isinstance(x, r_longlong):
-        return longlong2float(x)
-    if isinstance(x, r_ulonglong):
-        return longlong2float(rffi.cast(lltype.SignedLongLong, x))
-    assert isinstance(x, float)
-    return x
-cast_to_float._annspecialcase_ = 'specialize:argtype(0)'
-
 
 class BaseIntCallDescr(BaseCallDescr):
     # Base class of the various subclasses of descrs corresponding to
@@ -399,6 +394,7 @@
 
 class LongLongCallDescr(FloatCallDescr):
     _clsname = 'LongLongCallDescr'
+    _return_type = 'L'
 
 class VoidCallDescr(BaseCallDescr):
     _clsname = 'VoidCallDescr'

diff --git a/pypy/jit/codewriter/longlong.py b/pypy/jit/codewriter/longlong.py
--- a/pypy/jit/codewriter/longlong.py
+++ b/pypy/jit/codewriter/longlong.py
@@ -23,6 +23,7 @@
     getfloatstorage = lambda x: x
     getrealfloat    = lambda x: x
     gethash         = compute_hash
+    is_longlong     = lambda TYPE: False
 
     # -------------------------------------
 else:
@@ -38,5 +39,7 @@
     getfloatstorage = longlong2float.float2longlong
     getrealfloat    = longlong2float.longlong2float
     gethash         = lambda xll: xll - (xll >> 32)
+    is_longlong     = lambda TYPE: (TYPE == lltype.SignedLongLong or
+                                    TYPE == lltype.UnsignedLongLong)
 
     # -------------------------------------

diff --git a/pypy/jit/tool/oparser.py b/pypy/jit/tool/oparser.py
--- a/pypy/jit/tool/oparser.py
+++ b/pypy/jit/tool/oparser.py
@@ -9,6 +9,7 @@
 from pypy.jit.metainterp.resoperation import rop, ResOperation, ResOpWithDescr, N_aryOp
 from pypy.jit.metainterp.typesystem import llhelper
 from pypy.jit.codewriter.heaptracker import adr2int
+from pypy.jit.codewriter import longlong
 from pypy.rpython.lltypesystem import lltype, llmemory
 from pypy.rpython.ootypesystem import ootype
 
@@ -160,7 +161,7 @@
             return ConstInt(int(arg))
         except ValueError:
             if self.is_float(arg):
-                return ConstFloat(float(arg))
+                return ConstFloat(longlong.getfloatstorage(float(arg)))
             if (arg.startswith('"') or arg.startswith("'") or
                 arg.startswith('s"')):
                 # XXX ootype

diff --git a/pypy/jit/backend/llgraph/llimpl.py b/pypy/jit/backend/llgraph/llimpl.py
--- a/pypy/jit/backend/llgraph/llimpl.py
+++ b/pypy/jit/backend/llgraph/llimpl.py
@@ -1088,12 +1088,8 @@
     assert isinstance(x, longlong.r_float_storage)
     if TYPE is lltype.Float:
         return longlong.getrealfloat(x)
-    if IS_32_BIT:
-        assert longlong.supports_longlong
-        if TYPE is lltype.SignedLongLong:
-            return x
-        if TYPE is lltype.UnsignedLongLong:
-            return r_ulonglong(x)
+    if longlong.is_longlong(TYPE):
+        return rffi.cast(TYPE, x)
     raise TypeError(TYPE)
 
 
@@ -1465,7 +1461,8 @@
 
 kind2TYPE = {
     'i': lltype.Signed,
-    'f': longlong.FLOATSTORAGE,
+    'f': lltype.Float,
+    'L': lltype.SignedLongLong,
     'v': lltype.Void,
     }
 
@@ -1535,9 +1532,7 @@
                     assert n == 'r'
                 x = argsiter_r.next()
                 x = cast_from_ptr(TYPE, x)
-            elif TYPE is lltype.Float or (
-                    IS_32_BIT and TYPE in (lltype.SignedLongLong,
-                                           lltype.UnsignedLongLong)):
+            elif TYPE is lltype.Float or longlong.is_longlong(TYPE):
                 if args_in_order is not None:
                     n = orderiter.next()
                     assert n == 'f'
@@ -1646,6 +1641,13 @@
 s_CompiledLoop = annmodel.SomePtr(COMPILEDLOOP)
 s_Frame = annmodel.SomePtr(FRAME)
 
+if longlong.FLOATSTORAGE is lltype.Float:
+    s_FloatStorage = annmodel.SomeFloat()
+elif longlong.FLOATSTORAGE is lltype.SignedLongLong:
+    s_FloatStorage = annmodel.SomeInteger(knowntype=longlong.r_float_storage)
+else:
+    assert 0
+
 setannotation(compile_start, s_CompiledLoop)
 setannotation(compile_start_int_var, annmodel.SomeInteger())
 setannotation(compile_start_ref_var, annmodel.SomeInteger())
@@ -1674,7 +1676,7 @@
 setannotation(frame_execute, annmodel.SomeInteger())
 setannotation(frame_int_getvalue, annmodel.SomeInteger())
 setannotation(frame_ptr_getvalue, annmodel.SomePtr(llmemory.GCREF))
-setannotation(frame_float_getvalue, annmodel.SomeFloat())
+setannotation(frame_float_getvalue, s_FloatStorage)
 setannotation(frame_get_value_count, annmodel.SomeInteger())
 setannotation(frame_clear_latest_values, annmodel.s_None)
 
@@ -1690,15 +1692,15 @@
 setannotation(do_unicodegetitem, annmodel.SomeInteger())
 setannotation(do_getarrayitem_gc_int, annmodel.SomeInteger())
 setannotation(do_getarrayitem_gc_ptr, annmodel.SomePtr(llmemory.GCREF))
-setannotation(do_getarrayitem_gc_float, annmodel.SomeFloat())
+setannotation(do_getarrayitem_gc_float, s_FloatStorage)
 setannotation(do_getarrayitem_raw_int, annmodel.SomeInteger())
-setannotation(do_getarrayitem_raw_float, annmodel.SomeFloat())
+setannotation(do_getarrayitem_raw_float, s_FloatStorage)
 setannotation(do_getfield_gc_int, annmodel.SomeInteger())
 setannotation(do_getfield_gc_ptr, annmodel.SomePtr(llmemory.GCREF))
-setannotation(do_getfield_gc_float, annmodel.SomeFloat())
+setannotation(do_getfield_gc_float, s_FloatStorage)
 setannotation(do_getfield_raw_int, annmodel.SomeInteger())
 setannotation(do_getfield_raw_ptr, annmodel.SomePtr(llmemory.GCREF))
-setannotation(do_getfield_raw_float, annmodel.SomeFloat())
+setannotation(do_getfield_raw_float, s_FloatStorage)
 setannotation(do_new, annmodel.SomePtr(llmemory.GCREF))
 setannotation(do_new_array, annmodel.SomePtr(llmemory.GCREF))
 setannotation(do_setarrayitem_gc_int, annmodel.s_None)
@@ -1720,5 +1722,5 @@
 setannotation(do_call_pushptr, annmodel.s_None)
 setannotation(do_call_int, annmodel.SomeInteger())
 setannotation(do_call_ptr, annmodel.SomePtr(llmemory.GCREF))
-setannotation(do_call_float, annmodel.SomeFloat())
+setannotation(do_call_float, s_FloatStorage)
 setannotation(do_call_void, annmodel.s_None)

diff --git a/pypy/jit/backend/x86/assembler.py b/pypy/jit/backend/x86/assembler.py
--- a/pypy/jit/backend/x86/assembler.py
+++ b/pypy/jit/backend/x86/assembler.py
@@ -38,6 +38,7 @@
 from pypy.jit.backend.x86.jump import remap_frame_layout
 from pypy.jit.metainterp.history import ConstInt, BoxInt
 from pypy.jit.codewriter.effectinfo import EffectInfo
+from pypy.jit.codewriter import longlong
 
 # darwin requires the stack to be 16 bytes aligned on calls. Same for gcc 4.5.0,
 # better safe than sorry
@@ -71,7 +72,8 @@
         self.malloc_unicode_func_addr = 0
         self.fail_boxes_int = values_array(lltype.Signed, failargs_limit)
         self.fail_boxes_ptr = values_array(llmemory.GCREF, failargs_limit)
-        self.fail_boxes_float = values_array(lltype.Float, failargs_limit)
+        self.fail_boxes_float = values_array(longlong.FLOATSTORAGE,
+                                             failargs_limit)
         self.fail_ebp = 0
         self.loop_run_counters = []
         self.float_const_neg_addr = 0
@@ -1801,11 +1803,13 @@
 
         if IS_X86_32 and isinstance(resloc, StackLoc) and resloc.width == 8:
             # a float or a long long return
-            from pypy.jit.backend.llsupport.descr import LongLongCallDescr
-            if isinstance(op.getdescr(), LongLongCallDescr):
+            if op.getdescr().get_return_type() == 'L':
                 self.mc.MOV_br(resloc.value, eax.value)      # long long
                 self.mc.MOV_br(resloc.value + 4, edx.value)
-                # XXX should ideally not move the result on the stack
+                # XXX should ideally not move the result on the stack,
+                #     but it's a mess to load eax/edx into a xmm register
+                #     and this way is simpler also because the result loc
+                #     can just be always a stack location
             else:
                 self.mc.FSTP_b(resloc.value)   # float return
         elif size == WORD:

diff --git a/pypy/jit/codewriter/jtransform.py b/pypy/jit/codewriter/jtransform.py
--- a/pypy/jit/codewriter/jtransform.py
+++ b/pypy/jit/codewriter/jtransform.py
@@ -5,7 +5,7 @@
 from pypy.objspace.flow.model import SpaceOperation, Variable, Constant
 from pypy.objspace.flow.model import Block, Link, c_last_exception
 from pypy.jit.codewriter.flatten import ListOfKind, IndirectCallTargets
-from pypy.jit.codewriter import support, heaptracker
+from pypy.jit.codewriter import support, heaptracker, longlong
 from pypy.jit.codewriter.effectinfo import EffectInfo
 from pypy.jit.codewriter.policy import log
 from pypy.jit.metainterp.typesystem import deref, arrayItem
@@ -791,17 +791,6 @@
     # and unsupported ones are turned into a call to a function from
     # jit.codewriter.support.
 
-    if lltype.SignedLongLong != lltype.Signed:
-        @staticmethod
-        def _is_longlong(TYPE):
-            return (TYPE == lltype.SignedLongLong or
-                    TYPE == lltype.UnsignedLongLong)
-    else:
-        # on 64-bit, _is_longlong() returns always False
-        @staticmethod
-        def _is_longlong(TYPE):
-            return False
-
     for _op, _oopspec in [('llong_invert',  'INVERT'),
                           ('ullong_invert', 'INVERT'),
                           ('llong_lt',      'LT'),
@@ -878,8 +867,8 @@
     rewrite_op_ullong_is_true = rewrite_op_llong_is_true
 
     def rewrite_op_cast_primitive(self, op):
-        fromll = self._is_longlong(op.args[0].concretetype)
-        toll   = self._is_longlong(op.result.concretetype)
+        fromll = longlong.is_longlong(op.args[0].concretetype)
+        toll   = longlong.is_longlong(op.result.concretetype)
         if fromll != toll:
             args = op.args
             if fromll:

diff --git a/pypy/jit/codewriter/assembler.py b/pypy/jit/codewriter/assembler.py
--- a/pypy/jit/codewriter/assembler.py
+++ b/pypy/jit/codewriter/assembler.py
@@ -89,13 +89,9 @@
         elif kind == 'float':
             if const.concretetype == lltype.Float:
                 value = longlong.getfloatstorage(value)
-            elif const.concretetype == lltype.SignedLongLong:
-                assert longlong.supports_longlong
-            elif const.concretetype == lltype.UnsignedLongLong:
-                assert longlong.supports_longlong
+            else:
+                assert longlong.is_longlong(const.concretetype)
                 value = rffi.cast(lltype.SignedLongLong, value)
-            else:
-                raise AssertionError(const.concretetype)
             constants = self.constants_f
         else:
             raise AssemblerError('unimplemented %r in %r' %

diff --git a/pypy/jit/backend/llsupport/llmodel.py b/pypy/jit/backend/llsupport/llmodel.py
--- a/pypy/jit/backend/llsupport/llmodel.py
+++ b/pypy/jit/backend/llsupport/llmodel.py
@@ -7,7 +7,7 @@
 from pypy.jit.metainterp.history import BoxInt, BoxPtr, set_future_values,\
      BoxFloat
 from pypy.jit.metainterp import history
-from pypy.jit.codewriter import heaptracker
+from pypy.jit.codewriter import heaptracker, longlong
 from pypy.jit.backend.model import AbstractCPU
 from pypy.jit.backend.llsupport import symbolic
 from pypy.jit.backend.llsupport.symbolic import WORD, unroll_basic_sizes
@@ -315,7 +315,7 @@
         ofs = self.unpack_arraydescr(arraydescr)
         # --- start of GC unsafe code (no GC operation!) ---
         items = rffi.ptradd(rffi.cast(rffi.CCHARP, gcref), ofs)
-        items = rffi.cast(rffi.CArrayPtr(lltype.Float), items)
+        items = rffi.cast(rffi.CArrayPtr(longlong.FLOATSTORAGE), items)
         fval = items[itemindex]
         # --- end of GC unsafe code ---
         return fval
@@ -348,7 +348,7 @@
         ofs = self.unpack_arraydescr(arraydescr)
         # --- start of GC unsafe code (no GC operation!) ---
         items = rffi.ptradd(rffi.cast(rffi.CCHARP, gcref), ofs)
-        items = rffi.cast(rffi.CArrayPtr(lltype.Float), items)
+        items = rffi.cast(rffi.CArrayPtr(longlong.FLOATSTORAGE), items)
         items[itemindex] = newvalue
         # --- end of GC unsafe code ---
 
@@ -410,7 +410,7 @@
         ofs = self.unpack_fielddescr(fielddescr)
         # --- start of GC unsafe code (no GC operation!) ---
         fieldptr = rffi.ptradd(rffi.cast(rffi.CCHARP, struct), ofs)
-        fval = rffi.cast(rffi.CArrayPtr(lltype.Float), fieldptr)[0]
+        fval = rffi.cast(rffi.CArrayPtr(longlong.FLOATSTORAGE), fieldptr)[0]
         # --- end of GC unsafe code ---
         return fval
 
@@ -452,7 +452,7 @@
         ofs = self.unpack_fielddescr(fielddescr)
         # --- start of GC unsafe code (no GC operation!) ---
         fieldptr = rffi.ptradd(rffi.cast(rffi.CCHARP, struct), ofs)
-        fieldptr = rffi.cast(rffi.CArrayPtr(lltype.Float), fieldptr)
+        fieldptr = rffi.cast(rffi.CArrayPtr(longlong.FLOATSTORAGE), fieldptr)
         fieldptr[0] = newvalue
         # --- end of GC unsafe code ---
 
@@ -509,7 +509,7 @@
         return calldescr.call_stub(func, args_i, args_r, args_f)
 
     def bh_call_f(self, func, calldescr, args_i, args_r, args_f):
-        assert isinstance(calldescr, FloatCallDescr)
+        assert isinstance(calldescr, FloatCallDescr)  # or LongLongCallDescr
         if not we_are_translated():
             calldescr.verify_types(args_i, args_r, args_f, history.FLOAT)
         return calldescr.call_stub(func, args_i, args_r, args_f)

diff --git a/pypy/jit/codewriter/support.py b/pypy/jit/codewriter/support.py
--- a/pypy/jit/codewriter/support.py
+++ b/pypy/jit/codewriter/support.py
@@ -300,7 +300,7 @@
 def _ll_1_llong_from_int(x):
     return r_longlong(intmask(x))
 
-def _ll_2_llong_from_uint(x):
+def _ll_1_llong_from_uint(x):
     return r_longlong(r_uint(x))
 
 def _ll_1_llong_to_int(xll):

diff --git a/pypy/jit/metainterp/executor.py b/pypy/jit/metainterp/executor.py
--- a/pypy/jit/metainterp/executor.py
+++ b/pypy/jit/metainterp/executor.py
@@ -63,7 +63,7 @@
             metainterp.execute_raised(e)
             result = NULL
         return BoxPtr(result)
-    if rettype == FLOAT:
+    if rettype == FLOAT or rettype == 'L':
         try:
             result = cpu.bh_call_f(func, descr, args_i, args_r, args_f)
         except Exception, e:

From commits-noreply at bitbucket.org  Sat Feb 12 17:40:37 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sat, 12 Feb 2011 17:40:37 +0100 (CET)
Subject: [pypy-svn] pypy jit-longlong-2: More fixes. metainterp/test passes
	again.
Message-ID: <20110212164037.58BEE2A2031@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: jit-longlong-2
Changeset: r41851:5be8666b0f60
Date: 2011-02-12 14:59 +0100
http://bitbucket.org/pypy/pypy/changeset/5be8666b0f60/

Log:	More fixes. metainterp/test passes again.

diff --git a/pypy/jit/metainterp/compile.py b/pypy/jit/metainterp/compile.py
--- a/pypy/jit/metainterp/compile.py
+++ b/pypy/jit/metainterp/compile.py
@@ -15,7 +15,7 @@
 from pypy.jit.metainterp.typesystem import llhelper, oohelper
 from pypy.jit.metainterp.optimizeutil import InvalidLoop
 from pypy.jit.metainterp.resume import NUMBERING
-from pypy.jit.codewriter import heaptracker
+from pypy.jit.codewriter import heaptracker, longlong
 
 def giveup():
     from pypy.jit.metainterp.pyjitpl import SwitchToBlackhole
@@ -528,7 +528,7 @@
 class ResumeGuardCountersFloat(AbstractResumeGuardCounters):
     def __init__(self):
         self.counters = [0] * 5
-        self.values = [0.0] * 5
+        self.values = [longlong.ZEROF] * 5
     see_float = func_with_new_name(_see, 'see_float')
 
 

diff --git a/pypy/jit/backend/x86/test/test_runner.py b/pypy/jit/backend/x86/test/test_runner.py
--- a/pypy/jit/backend/x86/test/test_runner.py
+++ b/pypy/jit/backend/x86/test/test_runner.py
@@ -390,18 +390,6 @@
         res = self.cpu.get_latest_value_int(0)
         assert res == 20
 
-    def test_call_with_const_floats(self):
-        def func(f1, f2):
-            return f1 + f2
-
-        FUNC = self.FuncType([lltype.Float, lltype.Float], lltype.Float)
-        FPTR = self.Ptr(FUNC)
-        calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
-        func_ptr = llhelper(FPTR, func)
-        funcbox = self.get_funcbox(self.cpu, func_ptr)
-        res = self.execute_operation(rop.CALL, [funcbox, ConstFloat(1.5), ConstFloat(2.5)], 'float', descr=calldescr)
-        assert res.value == 4.0
-
 
 class TestDebuggingAssembler(object):
     def setup_method(self, meth):

diff --git a/pypy/jit/backend/llgraph/llimpl.py b/pypy/jit/backend/llgraph/llimpl.py
--- a/pypy/jit/backend/llgraph/llimpl.py
+++ b/pypy/jit/backend/llgraph/llimpl.py
@@ -25,7 +25,6 @@
 from pypy.rlib.objectmodel import ComputedIntSymbolic, we_are_translated
 from pypy.rlib.rarithmetic import ovfcheck
 from pypy.rlib.rarithmetic import r_longlong, r_ulonglong, r_uint
-from pypy.rlib.longlong2float import longlong2float, float2longlong
 
 import py
 from pypy.tool.ansi_print import ansi_log

diff --git a/pypy/jit/backend/x86/regalloc.py b/pypy/jit/backend/x86/regalloc.py
--- a/pypy/jit/backend/x86/regalloc.py
+++ b/pypy/jit/backend/x86/regalloc.py
@@ -12,7 +12,7 @@
 from pypy.rlib import rgc
 from pypy.jit.backend.llsupport import symbolic
 from pypy.jit.backend.x86.jump import remap_frame_layout
-from pypy.jit.codewriter import heaptracker
+from pypy.jit.codewriter import heaptracker, longlong
 from pypy.jit.codewriter.effectinfo import EffectInfo
 from pypy.jit.metainterp.resoperation import rop
 from pypy.jit.backend.llsupport.descr import BaseFieldDescr, BaseArrayDescr
@@ -71,15 +71,16 @@
 
     def convert_to_imm(self, c):
         adr = self.assembler.datablockwrapper.malloc_aligned(8, 8)
-        xxxxxxxxx
-        rffi.cast(rffi.CArrayPtr(rffi.DOUBLE), adr)[0] = c.getfloat()
+        x = c.getfloatstorage()
+        rffi.cast(rffi.CArrayPtr(longlong.FLOATSTORAGE), adr)[0] = x
         return ConstFloatLoc(adr)
 
     def convert_to_imm_16bytes_align(self, c):
         adr = self.assembler.datablockwrapper.malloc_aligned(16, 16)
-        xxxxxxxxx
-        rffi.cast(rffi.CArrayPtr(rffi.DOUBLE), adr)[0] = c.getfloat()
-        rffi.cast(rffi.CArrayPtr(rffi.DOUBLE), adr)[1] = 0.0
+        x = c.getfloatstorage()
+        y = longlong.ZEROF
+        rffi.cast(rffi.CArrayPtr(longlong.FLOATSTORAGE), adr)[0] = x
+        rffi.cast(rffi.CArrayPtr(longlong.FLOATSTORAGE), adr)[1] = y
         return ConstFloatLoc(adr)
 
     def after_call(self, v):
@@ -223,8 +224,7 @@
                              selected_reg=None, need_lower_byte=False):
         if var.type == FLOAT:
             if isinstance(var, ConstFloat):
-                xxxxxxxxxxxxxx
-                return FloatImmedLoc(var.getfloat())
+                return FloatImmedLoc(var.getfloatstorage())
             return self.xrm.make_sure_var_in_reg(var, forbidden_vars,
                                                  selected_reg, need_lower_byte)
         else:
@@ -683,11 +683,10 @@
 
     def _maybe_consider_llong_lt(self, op):
         # XXX just a special case for now
-        from pypy.rlib.longlong2float import longlong2float
         box = op.getarg(2)
         if not isinstance(box, ConstFloat):
             return False
-        if not (box.value == longlong2float(r_longlong(0))):
+        if not box.aslonglong():
             return False
         # "x < 0"
         box = op.getarg(1)

diff --git a/pypy/jit/backend/model.py b/pypy/jit/backend/model.py
--- a/pypy/jit/backend/model.py
+++ b/pypy/jit/backend/model.py
@@ -1,5 +1,4 @@
 from pypy.rlib.debug import debug_start, debug_print, debug_stop
-from pypy.rlib.longlong2float import longlong2float, float2longlong
 from pypy.jit.metainterp import history, compile
 
 

diff --git a/pypy/jit/metainterp/test/test_warmstate.py b/pypy/jit/metainterp/test/test_warmstate.py
--- a/pypy/jit/metainterp/test/test_warmstate.py
+++ b/pypy/jit/metainterp/test/test_warmstate.py
@@ -7,6 +7,13 @@
 from pypy.jit.metainterp.warmstate import WarmEnterState, JitCell
 from pypy.jit.metainterp.history import BoxInt, BoxFloat, BoxPtr
 from pypy.jit.metainterp.history import ConstInt, ConstFloat, ConstPtr
+from pypy.jit.codewriter import longlong
+
+def boxfloat(x):
+    return BoxFloat(longlong.getfloatstorage(x))
+
+def constfloat(x):
+    return ConstFloat(longlong.getfloatstorage(x))
 
 
 def test_unwrap():
@@ -16,7 +23,7 @@
     assert unwrap(lltype.Void, BoxInt(42)) is None
     assert unwrap(lltype.Signed, BoxInt(42)) == 42
     assert unwrap(lltype.Char, BoxInt(42)) == chr(42)
-    assert unwrap(lltype.Float, BoxFloat(42.5)) == 42.5
+    assert unwrap(lltype.Float, boxfloat(42.5)) == 42.5
     assert unwrap(lltype.Ptr(S), BoxPtr(po)) == p
 
 def test_wrap():
@@ -26,10 +33,10 @@
     p = lltype.malloc(lltype.GcStruct('S'))
     po = lltype.cast_opaque_ptr(llmemory.GCREF, p)
     assert _is(wrap(None, 42), BoxInt(42))
-    assert _is(wrap(None, 42.5), BoxFloat(42.5))
+    assert _is(wrap(None, 42.5), boxfloat(42.5))
     assert _is(wrap(None, p), BoxPtr(po))
     assert _is(wrap(None, 42, in_const_box=True), ConstInt(42))
-    assert _is(wrap(None, 42.5, in_const_box=True), ConstFloat(42.5))
+    assert _is(wrap(None, 42.5, in_const_box=True), constfloat(42.5))
     assert _is(wrap(None, p, in_const_box=True), ConstPtr(po))
 
 def test_hash_equal_whatever_lltype():
@@ -138,7 +145,7 @@
     set_future_values(5, 42.5)
     assert future_values == {
         0: ("int", 5),
-        1: ("float", 42.5),
+        1: ("float", longlong.getfloatstorage(42.5)),
     }
     assert set_future_values is state.make_set_future_values()
 
@@ -147,7 +154,7 @@
         _green_args_spec = [lltype.Signed, lltype.Float]
     state = WarmEnterState(None, FakeJitDriverSD())
     unwrap_greenkey = state.make_unwrap_greenkey()
-    greenargs = unwrap_greenkey([ConstInt(42), ConstFloat(42.5)])
+    greenargs = unwrap_greenkey([ConstInt(42), constfloat(42.5)])
     assert greenargs == (42, 42.5)
     assert type(greenargs[0]) is int
 
@@ -161,7 +168,7 @@
         pass
     looptoken = FakeLoopToken()
     state.attach_unoptimized_bridge_from_interp([ConstInt(5),
-                                                 ConstFloat(2.25)],
+                                                 constfloat(2.25)],
                                                 looptoken)
     cell1 = get_jitcell(True, 5, 2.25)
     assert cell1.counter < 0
@@ -183,7 +190,7 @@
         return FakeCell()
     state.jit_getter = jit_getter
     state.make_jitdriver_callbacks()
-    res = state.get_location_str([ConstInt(5), ConstFloat(42.5)])
+    res = state.get_location_str([ConstInt(5), constfloat(42.5)])
     assert res == '(no jitdriver.get_printable_location!)'
 
 def test_make_jitdriver_callbacks_3():
@@ -205,7 +212,7 @@
         _get_jitcell_at_ptr = None
     state = WarmEnterState(FakeWarmRunnerDesc(), FakeJitDriverSD())
     state.make_jitdriver_callbacks()
-    res = state.get_location_str([ConstInt(5), ConstFloat(42.5)])
+    res = state.get_location_str([ConstInt(5), constfloat(42.5)])
     assert res == "hi there"
 
 def test_make_jitdriver_callbacks_4():

diff --git a/pypy/jit/metainterp/executor.py b/pypy/jit/metainterp/executor.py
--- a/pypy/jit/metainterp/executor.py
+++ b/pypy/jit/metainterp/executor.py
@@ -30,7 +30,7 @@
     else:       args_i = None
     if count_r: args_r = [NULL] * count_r
     else:       args_r = None
-    if count_f: args_f = [0.0] * count_f
+    if count_f: args_f = [longlong.ZEROF] * count_f
     else:       args_f = None
     # fill in the lists
     count_i = count_r = count_f = 0
@@ -68,7 +68,7 @@
             result = cpu.bh_call_f(func, descr, args_i, args_r, args_f)
         except Exception, e:
             metainterp.execute_raised(e)
-            result = 0.0
+            result = longlong.ZEROF
         return BoxFloat(result)
     if rettype == VOID:
         try:

diff --git a/pypy/jit/backend/test/runner_test.py b/pypy/jit/backend/test/runner_test.py
--- a/pypy/jit/backend/test/runner_test.py
+++ b/pypy/jit/backend/test/runner_test.py
@@ -551,6 +551,20 @@
                                      descr=calldescr)
         assert res.value == ord('a')
 
+    def test_call_with_const_floats(self):
+        def func(f1, f2):
+            return f1 + f2
+
+        FUNC = self.FuncType([lltype.Float, lltype.Float], lltype.Float)
+        FPTR = self.Ptr(FUNC)
+        calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
+        func_ptr = llhelper(FPTR, func)
+        funcbox = self.get_funcbox(self.cpu, func_ptr)
+        res = self.execute_operation(rop.CALL, [funcbox, constfloat(1.5),
+                                                constfloat(2.5)], 'float',
+                                     descr=calldescr)
+        assert res.getfloat() == 4.0
+
 
     def test_field_basic(self):
         t_box, T_box = self.alloc_instance(self.T)
@@ -2026,10 +2040,10 @@
             x = self.cpu.get_latest_value_float(0)
             assert longlong.getrealfloat(x) == 1.2 + 3.2
             called.append(failindex)
-            return longlong.getfloatstorage(13.5)
+            return 13.5
 
         FUNCPTR = lltype.Ptr(lltype.FuncType([lltype.Signed, llmemory.GCREF],
-                                             longlong.FLOATSTORAGE))
+                                             lltype.Float))
         class FakeJitDriverSD:
             index_of_virtualizable = -1
             _assembler_helper_ptr = llhelper(FUNCPTR, assembler_helper)
@@ -2116,7 +2130,7 @@
             x = self.cpu.get_latest_value_float(0)
             assert longlong.getrealfloat(x) == 1.25 + 3.25
             called.append(failindex)
-            return longlong.getfloatstorage(13.5)
+            return 13.5
 
         FUNCPTR = lltype.Ptr(lltype.FuncType([lltype.Signed, llmemory.GCREF],
                                              lltype.Float))
@@ -2391,7 +2405,6 @@
             py.test.skip("longlong test")
         from pypy.translator.tool.cbuild import ExternalCompilationInfo
         from pypy.rlib.rarithmetic import r_longlong
-        from pypy.rlib.longlong2float import longlong2float, float2longlong
         eci = ExternalCompilationInfo(
             separate_module_sources=["""
             long long fn_test_result_of_call(long long x)
@@ -2419,7 +2432,6 @@
             py.test.skip("test of longlong result")
         from pypy.translator.tool.cbuild import ExternalCompilationInfo
         from pypy.rlib.rarithmetic import r_longlong
-        from pypy.rlib.longlong2float import longlong2float, float2longlong
         eci = ExternalCompilationInfo(
             separate_module_sources=["""
             long long fn_test_result_of_call(long long x)

diff --git a/pypy/jit/codewriter/longlong.py b/pypy/jit/codewriter/longlong.py
--- a/pypy/jit/codewriter/longlong.py
+++ b/pypy/jit/codewriter/longlong.py
@@ -43,3 +43,5 @@
                                     TYPE == lltype.UnsignedLongLong)
 
     # -------------------------------------
+
+ZEROF = getfloatstorage(0.0)

diff --git a/pypy/jit/backend/llsupport/descr.py b/pypy/jit/backend/llsupport/descr.py
--- a/pypy/jit/backend/llsupport/descr.py
+++ b/pypy/jit/backend/llsupport/descr.py
@@ -325,7 +325,7 @@
         self.call_stub = d['call_stub']
 
     def verify_types(self, args_i, args_r, args_f, return_type):
-        assert self._return_type == return_type
+        assert self._return_type in return_type
         assert self.arg_classes.count('i') == len(args_i or ())
         assert self.arg_classes.count('r') == len(args_r or ())
         assert (self.arg_classes.count('f') +

diff --git a/pypy/jit/backend/x86/regloc.py b/pypy/jit/backend/x86/regloc.py
--- a/pypy/jit/backend/x86/regloc.py
+++ b/pypy/jit/backend/x86/regloc.py
@@ -6,6 +6,7 @@
 from pypy.rlib.objectmodel import specialize, instantiate
 from pypy.rlib.rarithmetic import intmask
 from pypy.jit.metainterp.history import FLOAT
+from pypy.jit.codewriter import longlong
 
 #
 # This module adds support for "locations", which can be either in a Const,
@@ -212,9 +213,8 @@
         _immutable_ = True
         width = 8
 
-        def __init__(self, floatvalue):
-            from pypy.rlib.longlong2float import float2longlong
-            self.aslonglong = float2longlong(floatvalue)
+        def __init__(self, floatstorage):
+            self.aslonglong = floatstorage
 
         def low_part(self):
             return intmask(self.aslonglong)
@@ -229,15 +229,14 @@
             return ImmedLoc(self.high_part())
 
         def __repr__(self):
-            from pypy.rlib.longlong2float import longlong2float
-            floatvalue = longlong2float(self.aslonglong)
+            floatvalue = longlong.getrealfloat(self.aslonglong)
             return '<FloatImmedLoc(%s)>' % (floatvalue,)
 
         def location_code(self):
             raise NotImplementedError
 
 if IS_X86_64:
-    def FloatImmedLoc(floatvalue):
+    def FloatImmedLoc(floatstorage):
         from pypy.rlib.longlong2float import float2longlong
         value = intmask(float2longlong(floatvalue))
         return ImmedLoc(value)

diff --git a/pypy/jit/metainterp/history.py b/pypy/jit/metainterp/history.py
--- a/pypy/jit/metainterp/history.py
+++ b/pypy/jit/metainterp/history.py
@@ -309,11 +309,11 @@
 
 class ConstFloat(Const):
     type = FLOAT
-    value = longlong.getfloatstorage(0.0)
+    value = longlong.ZEROF
     _attrs_ = ('value',)
 
     def __init__(self, valuestorage):
-        assert isinstance(valuestorage, longlong.r_float_storage)
+        assert lltype.typeOf(valuestorage) is longlong.FLOATSTORAGE
         self.value = valuestorage
 
     def clonebox(self):
@@ -336,7 +336,7 @@
         return False
 
     def nonnull(self):
-        return self.value != 0
+        return self.value != longlong.ZEROF
 
     def _getrepr_(self):
         return self.getfloat()
@@ -344,7 +344,7 @@
     def repr_rpython(self):
         return repr_rpython(self, 'cf')
 
-CONST_FZERO = ConstFloat(longlong.getfloatstorage(0.0))
+CONST_FZERO = ConstFloat(longlong.ZEROF)
 
 class ConstPtr(Const):
     type = REF
@@ -554,12 +554,12 @@
     type = FLOAT
     _attrs_ = ('value',)
 
-    def __init__(self, valuestorage=longlong.getfloatstorage(0.0)):
-        assert isinstance(valuestorage, longlong.r_float_storage)
+    def __init__(self, valuestorage=longlong.ZEROF):
+        assert lltype.typeOf(valuestorage) is longlong.FLOATSTORAGE
         self.value = valuestorage
 
     def forget_value(self):
-        self.value = longlong.getfloatstorage(0.0)
+        self.value = longlong.ZEROF
 
     def clonebox(self):
         return BoxFloat(self.value)
@@ -577,7 +577,7 @@
         cpu.set_future_value_float(j, self.value)
 
     def nonnull(self):
-        return self.value != 0
+        return self.value != longlong.ZEROF
 
     def _getrepr_(self):
         return self.getfloat()

diff --git a/pypy/jit/metainterp/blackhole.py b/pypy/jit/metainterp/blackhole.py
--- a/pypy/jit/metainterp/blackhole.py
+++ b/pypy/jit/metainterp/blackhole.py
@@ -252,7 +252,7 @@
         if we_are_translated():
             default_i = 0
             default_r = NULL
-            default_f = longlong.getfloatstorage(0.0)
+            default_f = longlong.ZEROF
         else:
             default_i = MissingValue()
             default_r = MissingValue()

diff --git a/pypy/jit/backend/llsupport/llmodel.py b/pypy/jit/backend/llsupport/llmodel.py
--- a/pypy/jit/backend/llsupport/llmodel.py
+++ b/pypy/jit/backend/llsupport/llmodel.py
@@ -511,7 +511,7 @@
     def bh_call_f(self, func, calldescr, args_i, args_r, args_f):
         assert isinstance(calldescr, FloatCallDescr)  # or LongLongCallDescr
         if not we_are_translated():
-            calldescr.verify_types(args_i, args_r, args_f, history.FLOAT)
+            calldescr.verify_types(args_i, args_r, args_f, history.FLOAT + 'L')
         return calldescr.call_stub(func, args_i, args_r, args_f)
 
     def bh_call_v(self, func, calldescr, args_i, args_r, args_f):

From commits-noreply at bitbucket.org  Sat Feb 12 17:40:38 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sat, 12 Feb 2011 17:40:38 +0100 (CET)
Subject: [pypy-svn] pypy jit-longlong-2: Fix tests.
Message-ID: <20110212164038.D65472A2035@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: jit-longlong-2
Changeset: r41852:d6df4d38a57e
Date: 2011-02-12 17:34 +0100
http://bitbucket.org/pypy/pypy/changeset/d6df4d38a57e/

Log:	Fix tests.

diff --git a/pypy/jit/backend/x86/rx86.py b/pypy/jit/backend/x86/rx86.py
--- a/pypy/jit/backend/x86/rx86.py
+++ b/pypy/jit/backend/x86/rx86.py
@@ -551,7 +551,7 @@
 
     MOVD_rx = xmminsn('\x66', rex_w, '\x0F\x7E', register(2, 8), register(1), '\xC0')
     MOVD_xr = xmminsn('\x66', rex_w, '\x0F\x6E', register(1, 8), register(2), '\xC0')
-    PMOVMSKB_rx = xmminsn('\x66', rex_nw, '\x0F\xD7', register(1, 8), register(2), '\xC0')
+    PMOVMSKB_rx = xmminsn('\x66', rex_w, '\x0F\xD7', register(1, 8), register(2), '\xC0')
 
     # ------------------------------------------------------------
 

diff --git a/pypy/jit/backend/x86/test/test_regalloc.py b/pypy/jit/backend/x86/test/test_regalloc.py
--- a/pypy/jit/backend/x86/test/test_regalloc.py
+++ b/pypy/jit/backend/x86/test/test_regalloc.py
@@ -15,6 +15,7 @@
 from pypy.rpython.lltypesystem import lltype, llmemory, rffi
 from pypy.rpython.annlowlevel import llhelper
 from pypy.rpython.lltypesystem import rclass, rstr
+from pypy.jit.codewriter import longlong
 from pypy.jit.backend.x86.rx86 import *
 
 def test_is_comparison_or_ovf_op():
@@ -133,6 +134,7 @@
             if isinstance(arg, int):
                 self.cpu.set_future_value_int(i, arg)
             elif isinstance(arg, float):
+                arg = longlong.getfloatstorage(arg)
                 self.cpu.set_future_value_float(i, arg)
             else:
                 assert isinstance(lltype.typeOf(arg), lltype.Ptr)
@@ -153,8 +155,8 @@
                 index in range(0, end)]
 
     def getfloats(self, end):
-        return [self.cpu.get_latest_value_float(index) for
-                index in range(0, end)]
+        return [longlong.getrealfloat(self.cpu.get_latest_value_float(index))
+                for index in range(0, end)]
 
     def getptr(self, index, T):
         gcref = self.cpu.get_latest_value_ref(index)

diff --git a/pypy/jit/backend/x86/test/test_assembler.py b/pypy/jit/backend/x86/test/test_assembler.py
--- a/pypy/jit/backend/x86/test/test_assembler.py
+++ b/pypy/jit/backend/x86/test/test_assembler.py
@@ -7,6 +7,7 @@
 from pypy.jit.backend.x86.arch import WORD, IS_X86_32, IS_X86_64
 from pypy.jit.backend.detect_cpu import getcpuclass 
 from pypy.jit.backend.x86.regalloc import X86RegisterManager, X86_64_RegisterManager, X86XMMRegisterManager, X86_64_XMMRegisterManager
+from pypy.jit.codewriter import longlong
 
 ACTUAL_CPU = getcpuclass()
 
@@ -144,7 +145,7 @@
                           immortal=True)
     expected_ints = [0] * len(content)
     expected_ptrs = [lltype.nullptr(llmemory.GCREF.TO)] * len(content)
-    expected_floats = [0.0] * len(content)
+    expected_floats = [longlong.ZEROF] * len(content)
 
     def write_in_stack(loc, value):
         assert loc >= 0
@@ -160,7 +161,7 @@
         else:
             if kind == 'float':
                 value, lo, hi = get_random_float()
-                expected_floats[i] = value
+                expected_floats[i] = longlong.getfloatstorage(value)
                 kind = Assembler386.DESCR_FLOAT
                 if isinstance(loc, RegLoc):
                     if WORD == 4:

diff --git a/pypy/jit/backend/x86/regalloc.py b/pypy/jit/backend/x86/regalloc.py
--- a/pypy/jit/backend/x86/regalloc.py
+++ b/pypy/jit/backend/x86/regalloc.py
@@ -686,7 +686,7 @@
         box = op.getarg(2)
         if not isinstance(box, ConstFloat):
             return False
-        if not box.aslonglong():
+        if box.getlonglong() != 0:
             return False
         # "x < 0"
         box = op.getarg(1)

From commits-noreply at bitbucket.org  Sat Feb 12 17:40:39 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sat, 12 Feb 2011 17:40:39 +0100 (CET)
Subject: [pypy-svn] pypy default: merge heads
Message-ID: <20110212164039.896082A2032@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41853:4bc1a9b281f4
Date: 2011-02-12 17:37 +0100
http://bitbucket.org/pypy/pypy/changeset/4bc1a9b281f4/

Log:	merge heads


From commits-noreply at bitbucket.org  Sat Feb 12 18:34:46 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sat, 12 Feb 2011 18:34:46 +0100 (CET)
Subject: [pypy-svn] pypy jit-longlong-2: Translation fixes.
Message-ID: <20110212173446.A7D68282B8B@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: jit-longlong-2
Changeset: r41854:a0dd6430dad6
Date: 2011-02-12 18:28 +0100
http://bitbucket.org/pypy/pypy/changeset/a0dd6430dad6/

Log:	Translation fixes.

diff --git a/pypy/jit/backend/x86/rx86.py b/pypy/jit/backend/x86/rx86.py
--- a/pypy/jit/backend/x86/rx86.py
+++ b/pypy/jit/backend/x86/rx86.py
@@ -551,7 +551,6 @@
 
     MOVD_rx = xmminsn('\x66', rex_w, '\x0F\x7E', register(2, 8), register(1), '\xC0')
     MOVD_xr = xmminsn('\x66', rex_w, '\x0F\x6E', register(1, 8), register(2), '\xC0')
-    PMOVMSKB_rx = xmminsn('\x66', rex_w, '\x0F\xD7', register(1, 8), register(2), '\xC0')
 
     # ------------------------------------------------------------
 
@@ -580,6 +579,8 @@
 class X86_32_CodeBuilder(AbstractX86CodeBuilder):
     WORD = 4
 
+    PMOVMSKB_rx = xmminsn('\x66', rex_nw, '\x0F\xD7', register(1, 8), register(2), '\xC0')
+
 class X86_64_CodeBuilder(AbstractX86CodeBuilder):
     WORD = 8
 

diff --git a/pypy/jit/backend/llsupport/descr.py b/pypy/jit/backend/llsupport/descr.py
--- a/pypy/jit/backend/llsupport/descr.py
+++ b/pypy/jit/backend/llsupport/descr.py
@@ -388,7 +388,7 @@
 class FloatCallDescr(BaseCallDescr):
     _clsname = 'FloatCallDescr'
     _return_type = history.FLOAT
-    call_stub = staticmethod(lambda func, args_i, args_r, args_f: 0.0)
+    call_stub = staticmethod(lambda func,args_i,args_r,args_f: longlong.ZEROF)
     def get_result_size(self, translate_support_code):
         return symbolic.get_size(lltype.Float, translate_support_code)
 

diff --git a/pypy/jit/backend/x86/regloc.py b/pypy/jit/backend/x86/regloc.py
--- a/pypy/jit/backend/x86/regloc.py
+++ b/pypy/jit/backend/x86/regloc.py
@@ -238,7 +238,7 @@
 if IS_X86_64:
     def FloatImmedLoc(floatstorage):
         from pypy.rlib.longlong2float import float2longlong
-        value = intmask(float2longlong(floatvalue))
+        value = intmask(float2longlong(floatstorage))
         return ImmedLoc(value)
 
 

From commits-noreply at bitbucket.org  Sat Feb 12 19:24:57 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sat, 12 Feb 2011 19:24:57 +0100 (CET)
Subject: [pypy-svn] pypy default: Merge jit-longlong-2: on 32bit,
	use a longlong instead of a float
Message-ID: <20110212182457.89EFE36C37A@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41855:9398f031b3f8
Date: 2011-02-12 19:23 +0100
http://bitbucket.org/pypy/pypy/changeset/9398f031b3f8/

Log:	Merge jit-longlong-2: on 32bit, use a longlong instead of a float in
	the 'value' field of BoxFloat. This fixes some issues that appeared
	to be caused by the cast from longlongs to floats and back. Indeed,
	when cast to float and back, we may (if unlucky) get a different
	value. More precisely, the longlong values 0x7ff0xxxxxxxxxxxxx
	encodes a signalling NaN, which get converted somewhere to a quiet
	NaN --- and we end up with 0x7ff8xxxxxxxxxxxxx.


From commits-noreply at bitbucket.org  Sat Feb 12 19:24:57 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sat, 12 Feb 2011 19:24:57 +0100 (CET)
Subject: [pypy-svn] pypy jit-longlong-2: close merged branch
Message-ID: <20110212182457.DE60636C536@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: jit-longlong-2
Changeset: r41856:3b9f8965a601
Date: 2011-02-12 19:24 +0100
http://bitbucket.org/pypy/pypy/changeset/3b9f8965a601/

Log:	close merged branch


From commits-noreply at bitbucket.org  Sat Feb 12 21:05:20 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sat, 12 Feb 2011 21:05:20 +0100 (CET)
Subject: [pypy-svn] pypy default: For test_cprofile: get the exact same repr
	as CPython, which is a
Message-ID: <20110212200520.04666282BAD@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41857:905a25d69a1e
Date: 2011-02-12 21:03 +0100
http://bitbucket.org/pypy/pypy/changeset/905a25d69a1e/

Log:	For test_cprofile: get the exact same repr as CPython, which is a
	bit more precise. Unfortunately it requires a bit of hacking for
	built-in methods.

diff --git a/pypy/module/_lsprof/interp_lsprof.py b/pypy/module/_lsprof/interp_lsprof.py
--- a/pypy/module/_lsprof/interp_lsprof.py
+++ b/pypy/module/_lsprof/interp_lsprof.py
@@ -163,11 +163,19 @@
 def create_spec(space, w_arg):
     if isinstance(w_arg, Method):
         w_function = w_arg.w_function
-        class_name = w_arg.w_class.getname(space, '?')
         if isinstance(w_function, Function):
             name = w_function.name
         else:
             name = '?'
+        # try to get the real class that defines the method,
+        # which is a superclass of the class of the instance
+        from pypy.objspace.std.typeobject import W_TypeObject   # xxx
+        w_type = w_arg.w_class
+        class_name = w_type.getname(space)    # if the rest doesn't work
+        if isinstance(w_type, W_TypeObject) and name != '?':
+            w_realclass, _ = space.lookup_in_type_where(w_type, name)
+            if isinstance(w_realclass, W_TypeObject):
+                class_name = w_realclass.get_module_type_name()
         return "{method '%s' of '%s' objects}" % (name, class_name)
     elif isinstance(w_arg, Function):
         if w_arg.w_module is None:

diff --git a/pypy/objspace/std/typeobject.py b/pypy/objspace/std/typeobject.py
--- a/pypy/objspace/std/typeobject.py
+++ b/pypy/objspace/std/typeobject.py
@@ -408,6 +408,18 @@
                 return w_self.dict_w['__module__']
             return space.wrap('__builtin__')
 
+    def get_module_type_name(w_self):
+        space = w_self.space
+        w_mod = w_self.get_module()
+        if not space.is_true(space.isinstance(w_mod, space.w_str)):
+            mod = '__builtin__'
+        else:
+            mod = space.str_w(w_mod)
+        if mod !='__builtin__':
+            return '%s.%s' % (mod, w_self.name)
+        else:
+            return w_self.name
+
     def add_subclass(w_self, w_subclass):
         space = w_self.space
         if not space.config.translation.rweakref:

diff --git a/pypy/module/_lsprof/test/test_cprofile.py b/pypy/module/_lsprof/test/test_cprofile.py
--- a/pypy/module/_lsprof/test/test_cprofile.py
+++ b/pypy/module/_lsprof/test/test_cprofile.py
@@ -177,7 +177,7 @@
         8    0.312    0.039    0.400    0.050 profilee.py:88(helper2)
         8    0.064    0.008    0.080    0.010 profilee.py:98(subhelper)
         4    0.000    0.000    0.000    0.000 {method 'append' of 'list' objects}
-        1    0.000    0.000    0.000    0.000 {.*disable.*}
+        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
        12    0.000    0.000    0.012    0.001 {hasattr}
         8    0.000    0.000    0.000    0.000 {range}
         4    0.000    0.000    0.000    0.000 {sys.exc_info}

From commits-noreply at bitbucket.org  Sat Feb 12 22:26:39 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sat, 12 Feb 2011 22:26:39 +0100 (CET)
Subject: [pypy-svn] pypy default: Modify one test to allow for the better
	peephole	optimization
Message-ID: <20110212212639.D69F636C231@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41858:5c5ac2449970
Date: 2011-02-12 22:00 +0100
http://bitbucket.org/pypy/pypy/changeset/5c5ac2449970/

Log:	Modify one test to allow for the better peephole optimization
	performed by pypy.

diff --git a/lib-python/2.7.0/test/test_peepholer.py b/lib-python/modified-2.7.0/test/test_peepholer.py
copy from lib-python/2.7.0/test/test_peepholer.py
copy to lib-python/modified-2.7.0/test/test_peepholer.py
--- a/lib-python/2.7.0/test/test_peepholer.py
+++ b/lib-python/modified-2.7.0/test/test_peepholer.py
@@ -67,10 +67,13 @@
             self.assertIn(elem, asm)
 
     def test_pack_unpack(self):
+        # On PyPy, "a, b = ..." is even more optimized, by removing
+        # the ROT_TWO.  But the ROT_TWO is not removed if assigning
+        # to more complex expressions, so check that.
         for line, elem in (
             ('a, = a,', 'LOAD_CONST',),
-            ('a, b = a, b', 'ROT_TWO',),
-            ('a, b, c = a, b, c', 'ROT_THREE',),
+            ('a[1], b = a, b', 'ROT_TWO',),
+            ('a, b[2], c = a, b, c', 'ROT_THREE',),
             ):
             asm = dis_single(line)
             self.assertIn(elem, asm)

From commits-noreply at bitbucket.org  Sat Feb 12 22:26:40 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sat, 12 Feb 2011 22:26:40 +0100 (CET)
Subject: [pypy-svn] pypy default: Change this,
	otherwise just None gets turned into a LOAD_CONST
Message-ID: <20110212212640.6F9B336C231@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41859:8ce6dadd8e8d
Date: 2011-02-12 22:02 +0100
http://bitbucket.org/pypy/pypy/changeset/8ce6dadd8e8d/

Log:	Change this, otherwise just None gets turned into a LOAD_CONST and
	then dropped completely in pypy.

diff --git a/lib-python/modified-2.7.0/test/test_peepholer.py b/lib-python/modified-2.7.0/test/test_peepholer.py
--- a/lib-python/modified-2.7.0/test/test_peepholer.py
+++ b/lib-python/modified-2.7.0/test/test_peepholer.py
@@ -41,7 +41,7 @@
     def test_none_as_constant(self):
         # LOAD_GLOBAL None  -->  LOAD_CONST None
         def f(x):
-            None
+            y = None
             return x
         asm = disassemble(f)
         for elem in ('LOAD_GLOBAL',):

From commits-noreply at bitbucket.org  Sat Feb 12 22:26:41 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sat, 12 Feb 2011 22:26:41 +0100 (CET)
Subject: [pypy-svn] pypy default: Another valid difference.
Message-ID: <20110212212641.679242A2030@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41860:5023ef8ded65
Date: 2011-02-12 22:25 +0100
http://bitbucket.org/pypy/pypy/changeset/5023ef8ded65/

Log:	Another valid difference.

diff --git a/lib-python/modified-2.7.0/test/test_peepholer.py b/lib-python/modified-2.7.0/test/test_peepholer.py
--- a/lib-python/modified-2.7.0/test/test_peepholer.py
+++ b/lib-python/modified-2.7.0/test/test_peepholer.py
@@ -81,6 +81,8 @@
             self.assertNotIn('UNPACK_TUPLE', asm)
 
     def test_folding_of_tuples_of_constants(self):
+        # On CPython, "a,b,c=1,2,3" turns into "a,b,c=<constant (1,2,3)>"
+        # but on PyPy, it turns into "a=1;b=2;c=3".
         for line, elem in (
             ('a = 1,2,3', '((1, 2, 3))'),
             ('("a","b","c")', "(('a', 'b', 'c'))"),
@@ -89,7 +91,8 @@
             ('((1, 2), 3, 4)', '(((1, 2), 3, 4))'),
             ):
             asm = dis_single(line)
-            self.assertIn(elem, asm)
+            self.assert_(elem in asm or (
+                line == 'a,b,c = 1,2,3' and 'UNPACK_TUPLE' not in asm))
             self.assertNotIn('BUILD_TUPLE', asm)
 
         # Bug 1053819:  Tuple of constants misidentified when presented with:

From commits-noreply at bitbucket.org  Sat Feb 12 22:49:45 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Sat, 12 Feb 2011 22:49:45 +0100 (CET)
Subject: [pypy-svn] pypy default: Module objects have an __package__
	attribute, eventually this needs to be set/used by the imp module,
	but for now just expose it.
Message-ID: <20110212214945.CD2F1282BAD@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41861:0baf937585ba
Date: 2011-02-12 16:48 -0500
http://bitbucket.org/pypy/pypy/changeset/0baf937585ba/

Log:	Module objects have an __package__ attribute, eventually this needs
	to be set/used by the imp module, but for now just expose it.

diff --git a/pypy/interpreter/test/test_module.py b/pypy/interpreter/test/test_module.py
--- a/pypy/interpreter/test/test_module.py
+++ b/pypy/interpreter/test/test_module.py
@@ -65,3 +65,10 @@
 
         m = type(_pypy_interact).__new__(type(_pypy_interact))
         assert repr(m).startswith("<module '?'")
+
+    def test_package(self):
+        import sys
+        import os
+
+        assert type(sys)('foo').__package__ is None
+        assert os.__package__ is None
\ No newline at end of file

diff --git a/pypy/interpreter/module.py b/pypy/interpreter/module.py
--- a/pypy/interpreter/module.py
+++ b/pypy/interpreter/module.py
@@ -19,6 +19,7 @@
         self.w_name = w_name 
         if w_name is not None:
             space.setitem(w_dict, space.new_interned_str('__name__'), w_name)
+        space.setitem(w_dict, space.new_interned_str('__package__'), space.w_None)
         self.startup_called = False
 
     def setup_after_space_initialization(self):
@@ -63,6 +64,7 @@
             w_doc = space.w_None
         space.setitem(self.w_dict, space.new_interned_str('__name__'), w_name)
         space.setitem(self.w_dict, space.new_interned_str('__doc__'), w_doc)
+        space.setitem(self.w_dict, space.new_interned_str('__package__'), space.w_None)
 
     def descr__reduce__(self, space):
         w_name = space.finditem(self.w_dict, space.wrap('__name__'))

From commits-noreply at bitbucket.org  Sat Feb 12 22:49:46 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Sat, 12 Feb 2011 22:49:46 +0100 (CET)
Subject: [pypy-svn] pypy default: Merged upstream.
Message-ID: <20110212214946.3450A282BDE@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41862:74bb7c529efc
Date: 2011-02-12 16:49 -0500
http://bitbucket.org/pypy/pypy/changeset/74bb7c529efc/

Log:	Merged upstream.


From commits-noreply at bitbucket.org  Sat Feb 12 23:00:33 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sat, 12 Feb 2011 23:00:33 +0100 (CET)
Subject: [pypy-svn] pypy default: Remove code that the compiler attempt to
	put in a basic	block
Message-ID: <20110212220033.5781C282BAD@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41863:09525c62ac04
Date: 2011-02-12 22:59 +0100
http://bitbucket.org/pypy/pypy/changeset/09525c62ac04/

Log:	Remove code that the compiler attempt to put in a basic block after
	a RETURN_VALUE statement. Such code would be dead. It should be
	enough to pass test_peepholer, hopefully.

diff --git a/pypy/interpreter/astcompiler/test/test_compiler.py b/pypy/interpreter/astcompiler/test/test_compiler.py
--- a/pypy/interpreter/astcompiler/test/test_compiler.py
+++ b/pypy/interpreter/astcompiler/test/test_compiler.py
@@ -818,4 +818,43 @@
         return (0, 1)[:2]
         """
         # Just checking this doesn't crash out
-        self.count_instructions(source)
\ No newline at end of file
+        self.count_instructions(source)
+
+    def test_remove_dead_code(self):
+        source = """def f(x):
+            return 5
+            x += 1
+        """
+        counts = self.count_instructions(source)
+        assert counts == {ops.LOAD_CONST:1, ops.RETURN_VALUE: 1}
+
+    def test_remove_dead_jump_after_return(self):
+        source = """def f(x, y, z):
+            if x:
+                return y
+            else:
+                return z
+        """
+        counts = self.count_instructions(source)
+        assert counts == {ops.LOAD_FAST: 3,
+                          ops.POP_JUMP_IF_FALSE: 1,
+                          ops.RETURN_VALUE: 2}
+
+    def test_remove_dead_yield(self):
+        source = """def f(x):
+            return
+            yield 6
+        """
+        counts = self.count_instructions(source)
+        assert counts == {ops.LOAD_CONST:1, ops.RETURN_VALUE: 1}
+        #
+        space = self.space
+        w_generator = space.appexec([], """():
+            d = {}
+            exec '''def f(x):
+                return
+                yield 6
+            ''' in d
+            return d['f'](5)
+        """)
+        assert 'generator' in space.str_w(space.repr(w_generator))

diff --git a/pypy/interpreter/astcompiler/assemble.py b/pypy/interpreter/astcompiler/assemble.py
--- a/pypy/interpreter/astcompiler/assemble.py
+++ b/pypy/interpreter/astcompiler/assemble.py
@@ -167,15 +167,22 @@
         self.use_block(block)
         return block
 
+    def is_dead_code(self):
+        """Return False if any code can be meaningfully added to the
+        current block, or True if it would be dead code."""
+        # currently only True after a RETURN_VALUE.
+        return self.current_block.have_return
+
     def emit_op(self, op):
         """Emit an opcode without an argument."""
         instr = Instruction(op)
         if not self.lineno_set:
             instr.lineno = self.lineno
             self.lineno_set = True
-        self.instrs.append(instr)
-        if op == ops.RETURN_VALUE:
-            self.current_block.have_return = True
+        if not self.is_dead_code():
+            self.instrs.append(instr)
+            if op == ops.RETURN_VALUE:
+                self.current_block.have_return = True
         return instr
 
     def emit_op_arg(self, op, arg):
@@ -184,7 +191,8 @@
         if not self.lineno_set:
             instr.lineno = self.lineno
             self.lineno_set = True
-        self.instrs.append(instr)
+        if not self.is_dead_code():
+            self.instrs.append(instr)
 
     def emit_op_name(self, op, container, name):
         """Emit an opcode referencing a name."""

From commits-noreply at bitbucket.org  Sat Feb 12 23:00:33 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sat, 12 Feb 2011 23:00:33 +0100 (CET)
Subject: [pypy-svn] pypy default: merge heads
Message-ID: <20110212220033.9465E282BDE@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41864:51a18835dd32
Date: 2011-02-12 23:00 +0100
http://bitbucket.org/pypy/pypy/changeset/51a18835dd32/

Log:	merge heads


From commits-noreply at bitbucket.org  Sat Feb 12 23:41:55 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sat, 12 Feb 2011 23:41:55 +0100 (CET)
Subject: [pypy-svn] pypy default: Add missing no-op support for
	gc_thread_start
Message-ID: <20110212224155.A3738282BAD@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41865:44598bdc11c8
Date: 2011-02-12 23:37 +0100
http://bitbucket.org/pypy/pypy/changeset/44598bdc11c8/

Log:	Add missing no-op support for gc_thread_start

diff --git a/pypy/translator/c/gc.py b/pypy/translator/c/gc.py
--- a/pypy/translator/c/gc.py
+++ b/pypy/translator/c/gc.py
@@ -89,6 +89,9 @@
     def OP_GC_THREAD_RUN(self, funcgen, op):
         return ''
 
+    def OP_GC_THREAD_START(self, funcgen, op):
+        return ''
+
     def OP_GC_THREAD_DIE(self, funcgen, op):
         return ''
 

From commits-noreply at bitbucket.org  Sun Feb 13 06:04:46 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Sun, 13 Feb 2011 06:04:46 +0100 (CET)
Subject: [pypy-svn] pypy default: Remove a good bit of nonsense from the
	complex tests.
Message-ID: <20110213050446.CA547282BDE@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41866:ada2e37a9123
Date: 2011-02-13 00:04 -0500
http://bitbucket.org/pypy/pypy/changeset/ada2e37a9123/

Log:	Remove a good bit of nonsense from the complex tests.

diff --git a/pypy/objspace/std/test/test_complexobject.py b/pypy/objspace/std/test/test_complexobject.py
--- a/pypy/objspace/std/test/test_complexobject.py
+++ b/pypy/objspace/std/test/test_complexobject.py
@@ -84,19 +84,53 @@
 
 
 class AppTestAppComplexTest:
-    def setup_class(cls):
-        # XXX these tests probably copied directly from CPython
-        # please port them to pypy style :-/
-        cls.w_helper = cls.space.appexec([], """
-            ():
-                import sys
-                sys.path.append(%r)
-                import helper
-                return helper
-        """ % (str(py.path.local(__file__).dirpath())))
+    def w_check_div(self, x, y):
+        """Compute complex z=x*y, and check that z/x==y and z/y==x."""
+        z = x * y
+        if x != 0:
+            q = z / x
+            assert self.close(q, y)
+            q = z.__div__(x)
+            assert self.close(q, y)
+            q = z.__truediv__(x)
+            assert self.close(q, y)
+        if y != 0:
+            q = z / y
+            assert self.close(q, x)
+            q = z.__div__(y)
+            assert self.close(q, x)
+            q = z.__truediv__(y)
+            assert self.close(q, x)
+
+    def w_close(self, x, y):
+        """Return true iff complexes x and y "are close\""""
+        return self.close_abs(x.real, y.real) and self.close_abs(x.imag, y.imag)
+
+    def w_close_abs(self, x, y, eps=1e-9):
+        """Return true iff floats x and y "are close\""""
+        # put the one with larger magnitude second
+        if abs(x) > abs(y):
+            x, y = y, x
+        if y == 0:
+            return abs(x) < eps
+        if x == 0:
+            return abs(y) < eps
+        # check that relative difference < eps
+        return abs((x - y) / y) < eps
+
+    def w_almost_equal(self, a, b, eps=1e-9):
+        if isinstance(a, complex):
+            if isinstance(b, complex):
+                return a.real - b.real < eps and a.imag - b.imag < eps
+            else:
+                return a.real - b < eps and a.imag < eps
+        else:
+            if isinstance(b, complex):
+                return a - b.real < eps and b.imag < eps
+            else:
+                return a - b < eps
 
     def test_div(self):
-        h = self.helper
         from random import random
         # XXX this test passed but took waaaaay to long
         # look at dist/lib-python/modified-2.5.2/test/test_complex.py
@@ -105,47 +139,43 @@
         simple_complex = [complex(x, y) for x in simple_real for y in simple_real]
         for x in simple_complex:
             for y in simple_complex:
-                h.check_div(x, y)
+                self.check_div(x, y)
 
         # A naive complex division algorithm (such as in 2.0) is very prone to
         # nonsense errors for these (overflows and underflows).
-        h.check_div(complex(1e200, 1e200), 1+0j)
-        h.check_div(complex(1e-200, 1e-200), 1+0j)
+        self.check_div(complex(1e200, 1e200), 1+0j)
+        self.check_div(complex(1e-200, 1e-200), 1+0j)
 
         # Just for fun.
         for i in xrange(100):
-            h.check_div(complex(random(), random()),
+            self.check_div(complex(random(), random()),
                            complex(random(), random()))
 
-        h.raises(ZeroDivisionError, complex.__div__, 1+1j, 0+0j)
+        raises(ZeroDivisionError, complex.__div__, 1+1j, 0+0j)
         # FIXME: The following currently crashes on Alpha
         # raises(OverflowError, pow, 1e200+1j, 1e200+1j)
 
     def test_truediv(self):
-        h = self.helper
-        h.assertAlmostEqual(complex.__truediv__(2+0j, 1+1j), 1-1j)
+        assert self.almost_equal(complex.__truediv__(2+0j, 1+1j), 1-1j)
         raises(ZeroDivisionError, complex.__truediv__, 1+1j, 0+0j)
 
     def test_floordiv(self):
-        h = self.helper
-        h.assertAlmostEqual(complex.__floordiv__(3+0j, 1.5+0j), 2)
+        assert self.almost_equal(complex.__floordiv__(3+0j, 1.5+0j), 2)
         raises(ZeroDivisionError, complex.__floordiv__, 3+0j, 0+0j)
 
     def test_coerce(self):
-        h = self.helper
-        h.raises(OverflowError, complex.__coerce__, 1+1j, 1L<<10000)
+        raises(OverflowError, complex.__coerce__, 1+1j, 1L<<10000)
 
     def test_richcompare(self):
-        h = self.helper
-        h.assertEqual(complex.__lt__(1+1j, None), NotImplemented)
-        h.assertIs(complex.__eq__(1+1j, 1+1j), True)
-        h.assertIs(complex.__eq__(1+1j, 2+2j), False)
-        h.assertIs(complex.__ne__(1+1j, 1+1j), False)
-        h.assertIs(complex.__ne__(1+1j, 2+2j), True)
-        h.raises(TypeError, complex.__lt__, 1+1j, 2+2j)
-        h.raises(TypeError, complex.__le__, 1+1j, 2+2j)
-        h.raises(TypeError, complex.__gt__, 1+1j, 2+2j)
-        h.raises(TypeError, complex.__ge__, 1+1j, 2+2j)
+        assert complex.__lt__(1+1j, None) is NotImplemented
+        assert complex.__eq__(1+1j, 2+2j) is False
+        assert complex.__eq__(1+1j, 1+1j) is True
+        assert complex.__ne__(1+1j, 1+1j) is False
+        assert complex.__ne__(1+1j, 2+2j) is True
+        raises(TypeError, complex.__lt__, 1+1j, 2+2j)
+        raises(TypeError, complex.__le__, 1+1j, 2+2j)
+        raises(TypeError, complex.__gt__, 1+1j, 2+2j)
+        raises(TypeError, complex.__ge__, 1+1j, 2+2j)
         large = 1 << 10000
         assert not (5+0j) == large
         assert not large == (5+0j)
@@ -153,173 +183,156 @@
         assert large != (5+0j)
 
     def test_mod(self):
-        h = self.helper
         raises(ZeroDivisionError, (1+1j).__mod__, 0+0j)
 
         a = 3.33+4.43j
-        try:
-            a % 0
-        except ZeroDivisionError:
-            pass
-        else:
-            self.fail("modulo parama can't be 0")
+        raises(ZeroDivisionError, "a % 0")
 
     def test_divmod(self):
-        h = self.helper
         raises(ZeroDivisionError, divmod, 1+1j, 0+0j)
 
     def test_pow(self):
-        h = self.helper
-        h.assertAlmostEqual(pow(1+1j, 0+0j), 1.0)
-        h.assertAlmostEqual(pow(0+0j, 2+0j), 0.0)
+        assert self.almost_equal(pow(1+1j, 0+0j), 1.0)
+        assert self.almost_equal(pow(0+0j, 2+0j), 0.0)
         raises(ZeroDivisionError, pow, 0+0j, 1j)
-        h.assertAlmostEqual(pow(1j, -1), 1/1j)
-        h.assertAlmostEqual(pow(1j, 200), 1)
+        assert self.almost_equal(pow(1j, -1), 1/1j)
+        assert self.almost_equal(pow(1j, 200), 1)
         raises(ValueError, pow, 1+1j, 1+1j, 1+1j)
 
         a = 3.33+4.43j
-        h.assertEqual(a ** 0j, 1)
-        h.assertEqual(a ** 0.+0.j, 1)
+        assert a ** 0j == 1
+        assert a ** 0.+0.j == 1
 
-        h.assertEqual(3j ** 0j, 1)
-        h.assertEqual(3j ** 0, 1)
+        assert 3j ** 0j == 1
+        assert 3j ** 0 == 1
 
-        try:
-            0j ** a
-        except ZeroDivisionError:
-            pass
-        else:
-            self.fail("should fail 0.0 to negative or complex power")
-
-        try:
-            0j ** (3-2j)
-        except ZeroDivisionError:
-            pass
-        else:
-            self.fail("should fail 0.0 to negative or complex power")
+        raises(ZeroDivisionError, "0j ** a")
+        raises(ZeroDivisionError, "0j ** (3-2j)")
 
         # The following is used to exercise certain code paths
-        h.assertEqual(a ** 105, a ** 105)
-        h.assertEqual(a ** -105, a ** -105)
-        h.assertEqual(a ** -30, a ** -30)
+        assert a ** 105 == a ** 105
+        assert a ** -105 == a ** -105
+        assert a ** -30 == a ** -30
 
-        h.assertEqual(0.0j ** 0, 1)
+        assert 0.0j ** 0 == 1
 
         b = 5.1+2.3j
-        h.raises(ValueError, pow, a, b, 0)
+        raises(ValueError, pow, a, b, 0)
 
     def test_boolcontext(self):
         from random import random
-        h = self.helper
         for i in xrange(100):
             assert complex(random() + 1e-6, random() + 1e-6)
         assert not complex(0.0, 0.0)
 
     def test_conjugate(self):
-        h = self.helper
-        h.assertClose(complex(5.3, 9.8).conjugate(), 5.3-9.8j)
+        assert self.close(complex(5.3, 9.8).conjugate(), 5.3-9.8j)
 
     def test_constructor(self):
-        h = self.helper
         class OS:
-            def __init__(self, value): self.value = value
-            def __complex__(self): return self.value
+            def __init__(self, value):
+                self.value = value
+            def __complex__(self):
+                return self.value
         class NS(object):
-            def __init__(self, value): self.value = value
-            def __complex__(self): return self.value
-        h.assertEqual(complex(OS(1+10j)), 1+10j)
-        h.assertEqual(complex(NS(1+10j)), 1+10j)
-        h.assertEqual(complex(OS(1+10j), 5), 1+15j)
-        h.assertEqual(complex(NS(1+10j), 5), 1+15j)
-        h.assertEqual(complex(OS(1+10j), 5j), -4+10j)
-        h.assertEqual(complex(NS(1+10j), 5j), -4+10j)
-        h.raises(TypeError, complex, OS(None))
-        h.raises(TypeError, complex, NS(None))
-        h.raises(TypeError, complex, OS(2.0))   # __complex__ must really
-        h.raises(TypeError, complex, NS(2.0))   # return a complex, not a float
+            def __init__(self, value):
+                self.value = value
+            def __complex__(self):
+                return self.value
+        assert complex(OS(1+10j)) == 1+10j
+        assert complex(NS(1+10j)) == 1+10j
+        assert complex(OS(1+10j), 5) == 1+15j
+        assert complex(NS(1+10j), 5) == 1+15j
+        assert complex(OS(1+10j), 5j) == -4+10j
+        assert complex(NS(1+10j), 5j) == -4+10j
+        raises(TypeError, complex, OS(None))
+        raises(TypeError, complex, NS(None))
+        raises(TypeError, complex, OS(2.0))   # __complex__ must really
+        raises(TypeError, complex, NS(2.0))   # return a complex, not a float
 
         # -- The following cases are not supported by CPython, but they
         # -- are supported by PyPy, which is most probably ok
-        #h.raises((TypeError, AttributeError), complex, OS(1+10j), OS(1+10j))
-        #h.raises((TypeError, AttributeError), complex, NS(1+10j), OS(1+10j))
-        #h.raises((TypeError, AttributeError), complex, OS(1+10j), NS(1+10j))
-        #h.raises((TypeError, AttributeError), complex, NS(1+10j), NS(1+10j))
+        #raises((TypeError, AttributeError), complex, OS(1+10j), OS(1+10j))
+        #raises((TypeError, AttributeError), complex, NS(1+10j), OS(1+10j))
+        #raises((TypeError, AttributeError), complex, OS(1+10j), NS(1+10j))
+        #raises((TypeError, AttributeError), complex, NS(1+10j), NS(1+10j))
 
         class F(object):
             def __float__(self):
                 return 2.0
-        h.assertEqual(complex(OS(1+10j), F()), 1+12j)
-        h.assertEqual(complex(NS(1+10j), F()), 1+12j)
+        assert complex(OS(1+10j), F()) == 1+12j
+        assert complex(NS(1+10j), F()) == 1+12j
 
-        h.assertAlmostEqual(complex("1+10j"), 1+10j)
-        h.assertAlmostEqual(complex(10), 10+0j)
-        h.assertAlmostEqual(complex(10.0), 10+0j)
-        h.assertAlmostEqual(complex(10L), 10+0j)
-        h.assertAlmostEqual(complex(10+0j), 10+0j)
-        h.assertAlmostEqual(complex(1,10), 1+10j)
-        h.assertAlmostEqual(complex(1,10L), 1+10j)
-        h.assertAlmostEqual(complex(1,10.0), 1+10j)
-        h.assertAlmostEqual(complex(1L,10), 1+10j)
-        h.assertAlmostEqual(complex(1L,10L), 1+10j)
-        h.assertAlmostEqual(complex(1L,10.0), 1+10j)
-        h.assertAlmostEqual(complex(1.0,10), 1+10j)
-        h.assertAlmostEqual(complex(1.0,10L), 1+10j)
-        h.assertAlmostEqual(complex(1.0,10.0), 1+10j)
-        h.assertAlmostEqual(complex(3.14+0j), 3.14+0j)
-        h.assertAlmostEqual(complex(3.14), 3.14+0j)
-        h.assertAlmostEqual(complex(314), 314.0+0j)
-        h.assertAlmostEqual(complex(314L), 314.0+0j)
-        h.assertAlmostEqual(complex(3.14+0j, 0j), 3.14+0j)
-        h.assertAlmostEqual(complex(3.14, 0.0), 3.14+0j)
-        h.assertAlmostEqual(complex(314, 0), 314.0+0j)
-        h.assertAlmostEqual(complex(314L, 0L), 314.0+0j)
-        h.assertAlmostEqual(complex(0j, 3.14j), -3.14+0j)
-        h.assertAlmostEqual(complex(0.0, 3.14j), -3.14+0j)
-        h.assertAlmostEqual(complex(0j, 3.14), 3.14j)
-        h.assertAlmostEqual(complex(0.0, 3.14), 3.14j)
-        h.assertAlmostEqual(complex("1"), 1+0j)
-        h.assertAlmostEqual(complex("1j"), 1j)
-        h.assertAlmostEqual(complex(),  0)
-        h.assertAlmostEqual(complex("-1"), -1)
-        h.assertAlmostEqual(complex("+1"), +1)
-        h.assertAlmostEqual(complex(" ( +3.14-6J )"), 3.14-6j)
+        assert self.almost_equal(complex("1+10j"), 1+10j)
+        assert self.almost_equal(complex(10), 10+0j)
+        assert self.almost_equal(complex(10.0), 10+0j)
+        assert self.almost_equal(complex(10L), 10+0j)
+        assert self.almost_equal(complex(10+0j), 10+0j)
+        assert self.almost_equal(complex(1,10), 1+10j)
+        assert self.almost_equal(complex(1,10L), 1+10j)
+        assert self.almost_equal(complex(1,10.0), 1+10j)
+        assert self.almost_equal(complex(1L,10), 1+10j)
+        assert self.almost_equal(complex(1L,10L), 1+10j)
+        assert self.almost_equal(complex(1L,10.0), 1+10j)
+        assert self.almost_equal(complex(1.0,10), 1+10j)
+        assert self.almost_equal(complex(1.0,10L), 1+10j)
+        assert self.almost_equal(complex(1.0,10.0), 1+10j)
+        assert self.almost_equal(complex(3.14+0j), 3.14+0j)
+        assert self.almost_equal(complex(3.14), 3.14+0j)
+        assert self.almost_equal(complex(314), 314.0+0j)
+        assert self.almost_equal(complex(314L), 314.0+0j)
+        assert self.almost_equal(complex(3.14+0j, 0j), 3.14+0j)
+        assert self.almost_equal(complex(3.14, 0.0), 3.14+0j)
+        assert self.almost_equal(complex(314, 0), 314.0+0j)
+        assert self.almost_equal(complex(314L, 0L), 314.0+0j)
+        assert self.almost_equal(complex(0j, 3.14j), -3.14+0j)
+        assert self.almost_equal(complex(0.0, 3.14j), -3.14+0j)
+        assert self.almost_equal(complex(0j, 3.14), 3.14j)
+        assert self.almost_equal(complex(0.0, 3.14), 3.14j)
+        assert self.almost_equal(complex("1"), 1+0j)
+        assert self.almost_equal(complex("1j"), 1j)
+        assert self.almost_equal(complex(),  0)
+        assert self.almost_equal(complex("-1"), -1)
+        assert self.almost_equal(complex("+1"), +1)
+        assert self.almost_equal(complex(" ( +3.14-6J )"), 3.14-6j)
 
-        class complex2(complex): pass
-        h.assertAlmostEqual(complex(complex2(1+1j)), 1+1j)
-        h.assertAlmostEqual(complex(real=17, imag=23), 17+23j)
-        h.assertAlmostEqual(complex(real=17+23j), 17+23j)
-        h.assertAlmostEqual(complex(real=17+23j, imag=23), 17+46j)
-        h.assertAlmostEqual(complex(real=1+2j, imag=3+4j), -3+5j)
+        class complex2(complex):
+            pass
+        assert self.almost_equal(complex(complex2(1+1j)), 1+1j)
+        assert self.almost_equal(complex(real=17, imag=23), 17+23j)
+        assert self.almost_equal(complex(real=17+23j), 17+23j)
+        assert self.almost_equal(complex(real=17+23j, imag=23), 17+46j)
+        assert self.almost_equal(complex(real=1+2j, imag=3+4j), -3+5j)
 
         c = 3.14 + 1j
         assert complex(c) is c
         del c
 
-        h.raises(TypeError, complex, "1", "1")
-        h.raises(TypeError, complex, 1, "1")
+        raises(TypeError, complex, "1", "1")
+        raises(TypeError, complex, 1, "1")
 
-        h.assertEqual(complex("  3.14+J  "), 3.14+1j)
+        assert complex("  3.14+J  ") == 3.14+1j
         #h.assertEqual(complex(unicode("  3.14+J  ")), 3.14+1j)
 
         # SF bug 543840:  complex(string) accepts strings with \0
         # Fixed in 2.3.
-        h.raises(ValueError, complex, '1+1j\0j')
+        raises(ValueError, complex, '1+1j\0j')
 
-        h.raises(TypeError, int, 5+3j)
-        h.raises(TypeError, long, 5+3j)
-        h.raises(TypeError, float, 5+3j)
-        h.raises(ValueError, complex, "")
-        h.raises(TypeError, complex, None)
-        h.raises(ValueError, complex, "\0")
-        h.raises(TypeError, complex, "1", "2")
-        h.raises(TypeError, complex, "1", 42)
-        h.raises(TypeError, complex, 1, "2")
-        h.raises(ValueError, complex, "1+")
-        h.raises(ValueError, complex, "1+1j+1j")
-        h.raises(ValueError, complex, "--")
+        raises(TypeError, int, 5+3j)
+        raises(TypeError, long, 5+3j)
+        raises(TypeError, float, 5+3j)
+        raises(ValueError, complex, "")
+        raises(TypeError, complex, None)
+        raises(ValueError, complex, "\0")
+        raises(TypeError, complex, "1", "2")
+        raises(TypeError, complex, "1", 42)
+        raises(TypeError, complex, 1, "2")
+        raises(ValueError, complex, "1+")
+        raises(ValueError, complex, "1+1j+1j")
+        raises(ValueError, complex, "--")
 #        if x_test_support.have_unicode:
-#            h.raises(ValueError, complex, unicode("1"*500))
-#            h.raises(ValueError, complex, unicode("x"))
+#            raises(ValueError, complex, unicode("1"*500))
+#            raises(ValueError, complex, unicode("x"))
 #
         class EvilExc(Exception):
             pass
@@ -328,7 +341,7 @@
             def __complex__(self):
                 raise EvilExc
 
-        h.raises(EvilExc, complex, evilcomplex())
+        raises(EvilExc, complex, evilcomplex())
 
         class float2:
             def __init__(self, value):
@@ -336,29 +349,27 @@
             def __float__(self):
                 return self.value
 
-        h.assertAlmostEqual(complex(float2(42.)), 42)
-        h.assertAlmostEqual(complex(real=float2(17.), imag=float2(23.)), 17+23j)
-        h.raises(TypeError, complex, float2(None))
+        assert self.almost_equal(complex(float2(42.)), 42)
+        assert self.almost_equal(complex(real=float2(17.), imag=float2(23.)), 17+23j)
+        raises(TypeError, complex, float2(None))
 
     def test_hash(self):
-        h = self.helper
         for x in xrange(-30, 30):
-            h.assertEqual(hash(x), hash(complex(x, 0)))
+            assert hash(x) == hash(complex(x, 0))
             x /= 3.0    # now check against floating point
-            h.assertEqual(hash(x), hash(complex(x, 0.)))
+            assert hash(x) == hash(complex(x, 0.))
 
     def test_abs(self):
-        h = self.helper
         nums = [complex(x/3., y/7.) for x in xrange(-9,9) for y in xrange(-9,9)]
         for num in nums:
-            h.assertAlmostEqual((num.real**2 + num.imag**2)  ** 0.5, abs(num))
+            assert self.almost_equal((num.real**2 + num.imag**2)  ** 0.5, abs(num))
 
     def test_complex_subclass_ctr(self):
         import sys
         class j(complex):
             pass
         assert j(100 + 0j) == 100 + 0j
-        assert isinstance(j(100),j)
+        assert isinstance(j(100), j)
         assert j(100L + 0j) == 100 + 0j
         assert j("100 + 0j") == 100 + 0j
         x = j(1+0j)
@@ -385,13 +396,12 @@
         assert repr(complex(1,-float("nan"))) == '(1+nanj)'
 
     def test_neg(self):
-        h = self.helper
-        h.assertEqual(-(1+6j), -1-6j)
+        assert -(1+6j) == -1-6j
 
     def test_file(self):
-        h = self.helper
         import os
         import tempfile
+
         a = 3.33+4.43j
         b = 5.1+2.3j
 
@@ -402,7 +412,8 @@
             print >>fo, a, b
             fo.close()
             fo = open(pth, "rb")
-            h.assertEqual(fo.read(), "%s %s\n" % (a, b))
+            res = fo.read()
+            assert res == "%s %s\n" % (a, b)
         finally:
             if (fo is not None) and (not fo.closed):
                 fo.close()

From commits-noreply at bitbucket.org  Sun Feb 13 10:29:56 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sun, 13 Feb 2011 10:29:56 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix test_descr and test_random.
Message-ID: <20110213092956.A734A282BDE@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41867:628ebe44e1f1
Date: 2011-02-13 10:29 +0100
http://bitbucket.org/pypy/pypy/changeset/628ebe44e1f1/

Log:	Fix test_descr and test_random.

diff --git a/pypy/jit/backend/llsupport/test/test_descr.py b/pypy/jit/backend/llsupport/test/test_descr.py
--- a/pypy/jit/backend/llsupport/test/test_descr.py
+++ b/pypy/jit/backend/llsupport/test/test_descr.py
@@ -5,6 +5,7 @@
 from pypy.rpython.annlowlevel import llhelper
 from pypy.jit.metainterp.history import BoxInt, BoxFloat, BoxPtr
 from pypy.jit.metainterp import history
+from pypy.jit.codewriter import longlong
 import sys, struct, py
 
 def test_get_size_descr():
@@ -248,7 +249,7 @@
     #
     descr6 = get_call_descr(c0, [lltype.Signed], lltype.SignedLongLong)
     assert descr6.get_result_size(False) == 8
-    assert descr6.get_return_type() == history.FLOAT
+    assert descr6.get_return_type() == "L"
     assert descr6.arg_classes == "i"
 
 def test_get_call_descr_translated():
@@ -351,5 +352,5 @@
     opaquea = lltype.cast_opaque_ptr(llmemory.GCREF, a)
     a[0] = 1
     res = descr2.call_stub(rffi.cast(lltype.Signed, fnptr),
-                           [], [opaquea], [3.5])
-    assert res == 4.5
+                           [], [opaquea], [longlong.getfloatstorage(3.5)])
+    assert longlong.getrealfloat(res) == 4.5

diff --git a/pypy/jit/backend/test/test_random.py b/pypy/jit/backend/test/test_random.py
--- a/pypy/jit/backend/test/test_random.py
+++ b/pypy/jit/backend/test/test_random.py
@@ -9,6 +9,7 @@
 from pypy.jit.metainterp.resoperation import ResOperation, rop
 from pypy.jit.metainterp.executor import execute_nonspec
 from pypy.jit.metainterp.resoperation import opname
+from pypy.jit.codewriter import longlong
 
 class PleaseRewriteMe(Exception):
     pass
@@ -100,7 +101,8 @@
 ##                        'ConstAddr(llmemory.cast_ptr_to_adr(%s_vtable), cpu)'
 ##                        % name)
             elif isinstance(v, ConstFloat):
-                args.append('ConstFloat(%r)' % v.value)
+                args.append('ConstFloat(longlong.getfloatstorage(%r))'
+                            % v.getfloat())
             elif isinstance(v, ConstInt):
                 args.append('ConstInt(%s)' % v.value)
             else:
@@ -182,8 +184,8 @@
         if hasattr(self.loop, 'inputargs'):
             for i, v in enumerate(self.loop.inputargs):
                 if isinstance(v, (BoxFloat, ConstFloat)):
-                    print >>s, '    cpu.set_future_value_float(%d, %r)' % (i,
-                                                                       v.value)
+                    print >>s, ('    cpu.set_future_value_float(%d,'
+                        'longlong.getfloatstorage(%r))' % (i, v.getfloat()))
                 else:
                     print >>s, '    cpu.set_future_value_int(%d, %d)' % (i,
                                                                        v.value)
@@ -194,8 +196,8 @@
             fail_args = self.should_fail_by.getfailargs()
         for i, v in enumerate(fail_args):
             if isinstance(v, (BoxFloat, ConstFloat)):
-                print >>s, ('    assert cpu.get_latest_value_float(%d) == %r'
-                            % (i, v.value))
+                print >>s, ('    assert longlong.getrealfloat('
+                    'cpu.get_latest_value_float(%d)) == %r' % (i, v.value))
             else:
                 print >>s, ('    assert cpu.get_latest_value_int(%d) == %d'
                             % (i, v.value))
@@ -244,7 +246,7 @@
         elif r.random() < 0.75 or not builder.cpu.supports_floats:
             self.put(builder, [ConstInt(r.random_integer())])
         else:
-            self.put(builder, [ConstFloat(r.random_float())])
+            self.put(builder, [ConstFloat(r.random_float_storage())])
 
 class BinaryOperation(AbstractOperation):
     def __init__(self, opnum, and_mask=-1, or_mask=0, boolres=False):
@@ -302,16 +304,16 @@
             raise CannotProduceOperation
         k = r.random()
         if k < 0.18:
-            v_first = ConstFloat(r.random_float())
+            v_first = ConstFloat(r.random_float_storage())
         else:
             v_first = r.choice(builder.floatvars)
         if k > 0.82:
-            v_second = ConstFloat(r.random_float())
+            v_second = ConstFloat(r.random_float_storage())
         else:
             v_second = r.choice(builder.floatvars)
-        if abs(v_first.value) > 1E100 or abs(v_second.value) > 1E100:
+        if abs(v_first.getfloat()) > 1E100 or abs(v_second.getfloat()) > 1E100:
             raise CannotProduceOperation     # avoid infinities
-        if abs(v_second.value) < 1E-100:
+        if abs(v_second.getfloat()) < 1E-100:
             raise CannotProduceOperation     # e.g. division by zero error
         self.put(builder, [v_first, v_second])
 
@@ -330,7 +332,7 @@
         if not builder.floatvars:
             raise CannotProduceOperation
         box = r.choice(builder.floatvars)
-        if not (-sys.maxint-1 <= box.value <= sys.maxint):
+        if not (-sys.maxint-1 <= box.getfloat() <= sys.maxint):
             raise CannotProduceOperation      # would give an overflow
         self.put(builder, [box])
 
@@ -480,9 +482,13 @@
         if k < 1.0:
             x += k
         return x
+    def get_random_float_storage():
+        x = get_random_float()
+        return longlong.getfloatstorage(x)
     r.random_integer = get_random_integer
     r.random_char = get_random_char
     r.random_float = get_random_float
+    r.random_float_storage = get_random_float_storage
     return r
 
 def get_cpu():
@@ -516,7 +522,7 @@
                 at_least_once = 0
             for i in range(demo_conftest.option.n_vars):
                 if r.random() < k and i != at_least_once:
-                    startvars.append(BoxFloat(r.random_float()))
+                    startvars.append(BoxFloat(r.random_float_storage()))
                 else:
                     startvars.append(BoxInt(r.random_integer()))
         assert len(dict.fromkeys(startvars)) == len(startvars)

From commits-noreply at bitbucket.org  Sun Feb 13 10:58:20 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sun, 13 Feb 2011 10:58:20 +0100 (CET)
Subject: [pypy-svn] pypy default: Only add '__package__' to the names in
	modules created
Message-ID: <20110213095820.B9D7836C226@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41868:41dd0bdbc028
Date: 2011-02-13 10:57 +0100
http://bitbucket.org/pypy/pypy/changeset/41dd0bdbc028/

Log:	Only add '__package__' to the names in modules created internally,
	but not in modules created from Python code, as in CPython.

diff --git a/pypy/interpreter/test/test_module.py b/pypy/interpreter/test/test_module.py
--- a/pypy/interpreter/test/test_module.py
+++ b/pypy/interpreter/test/test_module.py
@@ -70,5 +70,6 @@
         import sys
         import os
 
-        assert type(sys)('foo').__package__ is None
-        assert os.__package__ is None
\ No newline at end of file
+        assert sys.__package__ is None
+        assert os.__package__ is None
+        assert not hasattr(type(sys)('foo'), '__package__')

diff --git a/pypy/interpreter/module.py b/pypy/interpreter/module.py
--- a/pypy/interpreter/module.py
+++ b/pypy/interpreter/module.py
@@ -11,7 +11,7 @@
 
     _frozen = False
 
-    def __init__(self, space, w_name, w_dict=None):
+    def __init__(self, space, w_name, w_dict=None, add_package=True):
         self.space = space
         if w_dict is None: 
             w_dict = space.newdict(module=True)
@@ -19,7 +19,11 @@
         self.w_name = w_name 
         if w_name is not None:
             space.setitem(w_dict, space.new_interned_str('__name__'), w_name)
-        space.setitem(w_dict, space.new_interned_str('__package__'), space.w_None)
+        if add_package:
+            # add the __package__ attribute only when created from internal
+            # code, but not when created from Python code (as in CPython)
+            space.setitem(w_dict, space.new_interned_str('__package__'),
+                          space.w_None)
         self.startup_called = False
 
     def setup_after_space_initialization(self):
@@ -54,7 +58,7 @@
 
     def descr_module__new__(space, w_subtype, __args__):
         module = space.allocate_instance(Module, w_subtype)
-        Module.__init__(module, space, None)
+        Module.__init__(module, space, None, add_package=False)
         return space.wrap(module)
 
     def descr_module__init__(self, w_name, w_doc=None):
@@ -64,7 +68,6 @@
             w_doc = space.w_None
         space.setitem(self.w_dict, space.new_interned_str('__name__'), w_name)
         space.setitem(self.w_dict, space.new_interned_str('__doc__'), w_doc)
-        space.setitem(self.w_dict, space.new_interned_str('__package__'), space.w_None)
 
     def descr__reduce__(self, space):
         w_name = space.finditem(self.w_dict, space.wrap('__name__'))

From commits-noreply at bitbucket.org  Sun Feb 13 11:04:31 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sun, 13 Feb 2011 11:04:31 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix for unpickling modules (which is a
	PyPy extension)
Message-ID: <20110213100431.7D2ED36C226@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41869:bfe3e55250b6
Date: 2011-02-13 11:04 +0100
http://bitbucket.org/pypy/pypy/changeset/bfe3e55250b6/

Log:	Fix for unpickling modules (which is a PyPy extension)

diff --git a/pypy/module/_pickle_support/maker.py b/pypy/module/_pickle_support/maker.py
--- a/pypy/module/_pickle_support/maker.py
+++ b/pypy/module/_pickle_support/maker.py
@@ -32,7 +32,7 @@
 func_new.unwrap_spec = [ObjSpace]
 
 def module_new(space, w_name, w_dict):
-    new_mod = Module(space, w_name, w_dict)
+    new_mod = Module(space, w_name, w_dict, add_package=False)
     return space.wrap(new_mod)
 
 def method_new(space, __args__):

From commits-noreply at bitbucket.org  Sun Feb 13 11:25:38 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sun, 13 Feb 2011 11:25:38 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix siginterrupt().  Previously,
	the state would be reset
Message-ID: <20110213102538.DEB2E282BDE@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41870:a944602ffc05
Date: 2011-02-13 11:25 +0100
http://bitbucket.org/pypy/pypy/changeset/a944602ffc05/

Log:	Fix siginterrupt(). Previously, the state would be reset to default
	after the first time the signal was received.

diff --git a/pypy/module/signal/test/test_signal.py b/pypy/module/signal/test/test_signal.py
--- a/pypy/module/signal/test/test_signal.py
+++ b/pypy/module/signal/test/test_signal.py
@@ -197,11 +197,35 @@
         signal.signal(signal.SIGUSR1, signal.SIG_DFL)
 
     def test_siginterrupt(self):
-        import signal
+        import signal, os, time
         signum = signal.SIGUSR1
+        def readpipe_is_not_interrupted():
+            # from CPython's test_signal.readpipe_interrupted()
+            r, w = os.pipe()
+            ppid = os.getpid()
+            pid = os.fork()
+            if pid == 0:
+                try:
+                    time.sleep(1)
+                    os.kill(ppid, signum)
+                    time.sleep(1)
+                finally:
+                    os._exit(0)
+            else:
+                try:
+                    os.close(w)
+                    # we expect not to be interrupted.  If we are, the
+                    # following line raises OSError(EINTR).
+                    os.read(r, 1)
+                finally:
+                    os.waitpid(pid, 0)
+                    os.close(r)
+        #
         oldhandler = signal.signal(signum, lambda x,y: None)
         try:
             signal.siginterrupt(signum, 0)
+            readpipe_is_not_interrupted()
+            readpipe_is_not_interrupted()
         finally:
             signal.signal(signum, oldhandler)
 

diff --git a/pypy/module/signal/interp_signal.py b/pypy/module/signal/interp_signal.py
--- a/pypy/module/signal/interp_signal.py
+++ b/pypy/module/signal/interp_signal.py
@@ -35,6 +35,7 @@
     include_dirs = [str(py.path.local(autopath.pypydir).join('translator', 'c'))],
     export_symbols = ['pypysig_poll', 'pypysig_default',
                       'pypysig_ignore', 'pypysig_setflag',
+                      'pypysig_reinstall',
                       'pypysig_set_wakeup_fd',
                       'pypysig_getaddr_occurred'],
 )
@@ -65,6 +66,7 @@
 pypysig_ignore = external('pypysig_ignore', [rffi.INT], lltype.Void)
 pypysig_default = external('pypysig_default', [rffi.INT], lltype.Void)
 pypysig_setflag = external('pypysig_setflag', [rffi.INT], lltype.Void)
+pypysig_reinstall = external('pypysig_reinstall', [rffi.INT], lltype.Void)
 pypysig_set_wakeup_fd = external('pypysig_set_wakeup_fd', [rffi.INT], rffi.INT)
 pypysig_poll = external('pypysig_poll', [], rffi.INT, threadsafe=False)
 # don't bother releasing the GIL around a call to pypysig_poll: it's
@@ -150,7 +152,7 @@
         except KeyError:
             return    # no handler, ignore signal
         # re-install signal handler, for OSes that clear it
-        pypysig_setflag(n)
+        pypysig_reinstall(n)
         # invoke the app-level handler
         space = self.space
         ec = space.getexecutioncontext()

diff --git a/pypy/translator/c/src/signals.h b/pypy/translator/c/src/signals.h
--- a/pypy/translator/c/src/signals.h
+++ b/pypy/translator/c/src/signals.h
@@ -143,6 +143,21 @@
 #endif
 }
 
+void pypysig_reinstall(int signum)
+{
+#ifdef SA_RESTART
+    /* Assume sigaction was used.  We did not pass SA_RESETHAND to
+       sa_flags, so there is nothing to do here. */
+#else
+# ifdef SIGCHLD
+    /* To avoid infinite recursion, this signal remains
+       reset until explicitly re-instated.  (Copied from CPython) */
+    if (signum != SIGCHLD)
+# endif
+    pypysig_setflag(signum);
+#endif
+}
+
 int pypysig_poll(void)
 {
   if (pypysig_occurred)

From commits-noreply at bitbucket.org  Sun Feb 13 11:34:03 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sun, 13 Feb 2011 11:34:03 +0100 (CET)
Subject: [pypy-svn] pypy default: Implementation details.
Message-ID: <20110213103403.18E1B282BDE@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41871:5764d30e2086
Date: 2011-02-13 11:33 +0100
http://bitbucket.org/pypy/pypy/changeset/5764d30e2086/

Log:	Implementation details.

diff --git a/lib-python/2.7.0/test/test_types.py b/lib-python/modified-2.7.0/test/test_types.py
copy from lib-python/2.7.0/test/test_types.py
copy to lib-python/modified-2.7.0/test/test_types.py
--- a/lib-python/2.7.0/test/test_types.py
+++ b/lib-python/modified-2.7.0/test/test_types.py
@@ -1,7 +1,8 @@
 # Python test set -- part 6, built-in types
 
 from test.test_support import run_unittest, have_unicode, run_with_locale, \
-                              check_py3k_warnings
+                              check_py3k_warnings, \
+                              impl_detail, check_impl_detail
 import unittest
 import sys
 import locale
@@ -289,9 +290,14 @@
         # array.array() returns an object that does not implement a char buffer,
         # something which int() uses for conversion.
         import array
-        try: int(buffer(array.array('c')))
+        try: int(buffer(array.array('c', '5')))
         except TypeError: pass
-        else: self.fail("char buffer (at C level) not working")
+        else:
+            if check_impl_detail():
+                self.fail("char buffer (at C level) not working")
+            #else:
+            #   it works on PyPy, which does not have the distinction
+            #   between char buffer and binary buffer.  XXX fine enough?
 
     def test_int__format__(self):
         def test(i, format_spec, result):
@@ -741,6 +747,7 @@
         for code in 'xXobns':
             self.assertRaises(ValueError, format, 0, ',' + code)
 
+    @impl_detail("the types' internal size attributes are CPython-only")
     def test_internal_sizes(self):
         self.assertGreater(object.__basicsize__, 0)
         self.assertGreater(tuple.__itemsize__, 0)

From commits-noreply at bitbucket.org  Sun Feb 13 13:46:34 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sun, 13 Feb 2011 13:46:34 +0100 (CET)
Subject: [pypy-svn] pypy default: Copy the logic of CPython to fix the
	precision around
Message-ID: <20110213124634.58DA7282BDE@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41872:1ea103d126e9
Date: 2011-02-13 13:03 +0100
http://bitbucket.org/pypy/pypy/changeset/1ea103d126e9/

Log:	Copy the logic of CPython to fix the precision around calls to
	dtoa.c, on x87 fpus.

diff --git a/pypy/translator/c/src/asm.h b/pypy/translator/c/src/asm.h
new file mode 100644
--- /dev/null
+++ b/pypy/translator/c/src/asm.h
@@ -0,0 +1,13 @@
+
+/* optional assembler bits */
+#if defined(__GNUC__) && defined(__i386__)
+#  include "src/asm_gcc_x86.h"
+#endif
+
+#if defined(__GNUC__) && defined(__amd64__)
+#  include "src/asm_gcc_x86_64.h"
+#endif
+
+#if defined(__GNUC__) && defined(__ppc__)
+#  include "src/asm_ppc.h"
+#endif

diff --git a/pypy/jit/backend/x86/support.py b/pypy/jit/backend/x86/support.py
--- a/pypy/jit/backend/x86/support.py
+++ b/pypy/jit/backend/x86/support.py
@@ -39,7 +39,8 @@
     ensure_sse2_floats = lambda : None
 else:
     _sse2_eci = ExternalCompilationInfo(
-        compile_extra = ['-msse2', '-mfpmath=sse'],
+        compile_extra = ['-msse2', '-mfpmath=sse',
+                         '-DPYPY_CPU_HAS_STANDARD_PRECISION'],
         separate_module_sources = ['void PYPY_NO_OP(void) {}'],
         )
     ensure_sse2_floats = rffi.llexternal('PYPY_NO_OP', [], lltype.Void,

diff --git a/pypy/translator/c/src/g_include.h b/pypy/translator/c/src/g_include.h
--- a/pypy/translator/c/src/g_include.h
+++ b/pypy/translator/c/src/g_include.h
@@ -37,19 +37,7 @@
 #include "src/llgroup.h"
 
 #include "src/instrument.h"
-
-/* optional assembler bits */
-#if defined(__GNUC__) && defined(__i386__)
-#  include "src/asm_gcc_x86.h"
-#endif
-
-#if defined(__GNUC__) && defined(__amd64__)
-#  include "src/asm_gcc_x86_64.h"
-#endif
-
-#if defined(__GNUC__) && defined(__ppc__)
-#  include "src/asm_ppc.h"
-#endif
+#include "src/asm.h"
 
 
 /*** modules ***/

diff --git a/pypy/translator/c/src/asm_gcc_x86.h b/pypy/translator/c/src/asm_gcc_x86.h
--- a/pypy/translator/c/src/asm_gcc_x86.h
+++ b/pypy/translator/c/src/asm_gcc_x86.h
@@ -71,6 +71,37 @@
 // I don't know how important it is, comment talks about time warps
 
 
+#ifndef PYPY_CPU_HAS_STANDARD_PRECISION
+/* On x86-32, we have to use the following hacks to set and restore
+ * the CPU's precision to 53 bits around calls to dtoa.c.  The macro
+ * PYPY_CPU_HAS_STANDARD_PRECISION is defined if we are compiling
+ * with -mss2 -mfpmath=sse anyway, in which case the precision is
+ * already ok.
+ */
+#define _PyPy_SET_53BIT_PRECISION_HEADER                          \
+    unsigned short old_387controlword, new_387controlword
+#define _PyPy_SET_53BIT_PRECISION_START                                 \
+    do {                                                                \
+        old_387controlword = _PyPy_get_387controlword();                \
+        new_387controlword = (old_387controlword & ~0x0f00) | 0x0200;   \
+        if (new_387controlword != old_387controlword)                   \
+            _PyPy_set_387controlword(new_387controlword);               \
+    } while (0)
+#define _PyPy_SET_53BIT_PRECISION_END                           \
+    if (new_387controlword != old_387controlword)               \
+        _PyPy_set_387controlword(old_387controlword)
+
+static unsigned short _PyPy_get_387controlword(void) {
+    unsigned short cw;
+    __asm__ __volatile__ ("fnstcw %0" : "=m" (cw));
+    return cw;
+}
+static void _PyPy_set_387controlword(unsigned short cw) {
+    __asm__ __volatile__ ("fldcw %0" : : "m" (cw));
+}
+#endif  /* !PYPY_CPU_HAS_STANDARD_PRECISION */
+
+
 /* implementations */
 
 #ifndef PYPY_NOT_MAIN_FILE

diff --git a/pypy/translator/c/src/dtoa.c b/pypy/translator/c/src/dtoa.c
--- a/pypy/translator/c/src/dtoa.c
+++ b/pypy/translator/c/src/dtoa.c
@@ -130,11 +130,9 @@
 #include <string.h>
 #define PYPY_NOT_MAIN_FILE
 #include "src/allocator.h"
+#include "src/asm.h"
 #define PyMem_Malloc PyObject_Malloc
 #define PyMem_Free PyObject_Free
-#define _Py_dg_strtod _PyPy_dg_strtod
-#define _Py_dg_dtoa _PyPy_dg_dtoa
-#define _Py_dg_freedtoa _PyPy_dg_freedtoa
 /* End PYPY hacks */
 
 
@@ -1508,7 +1506,7 @@
     return 0;
 }
 
-double
+static double
 _Py_dg_strtod(const char *s00, char **se)
 {
     int bb2, bb5, bbe, bd2, bd5, bs2, c, dsign, e, e1, error;
@@ -2286,7 +2284,7 @@
  * when MULTIPLE_THREADS is not defined.
  */
 
-void
+static void
 _Py_dg_freedtoa(char *s)
 {
     Bigint *b = (Bigint *)((int *)s - 1);
@@ -2332,7 +2330,7 @@
    leakage, a successful call to _Py_dg_dtoa should always be matched by a
    call to _Py_dg_freedtoa. */
 
-char *
+static char *
 _Py_dg_dtoa(double dd, int mode, int ndigits,
             int *decpt, int *sign, char **rve)
 {
@@ -2933,6 +2931,47 @@
         _Py_dg_freedtoa(s0);
     return NULL;
 }
+
+
+/* Begin PYPY hacks */
+#ifndef _PyPy_SET_53BIT_PRECISION_HEADER
+# define _PyPy_SET_53BIT_PRECISION_HEADER  /* nothing */
+#endif
+#ifndef _PyPy_SET_53BIT_PRECISION_START
+# define _PyPy_SET_53BIT_PRECISION_START   /* nothing */
+#endif
+#ifndef _PyPy_SET_53BIT_PRECISION_END
+# define _PyPy_SET_53BIT_PRECISION_END     /* nothing */
+#endif
+double _PyPy_dg_strtod(const char *s00, char **se)
+{
+    double result;
+    _PyPy_SET_53BIT_PRECISION_HEADER;
+
+    _PyPy_SET_53BIT_PRECISION_START;
+    result = _Py_dg_strtod(s00, se);
+    _PyPy_SET_53BIT_PRECISION_END;
+    return result;
+}
+
+char * _PyPy_dg_dtoa(double dd, int mode, int ndigits,
+                   int *decpt, int *sign, char **rve)
+{
+    char* result;
+    _PyPy_SET_53BIT_PRECISION_HEADER;
+
+    _PyPy_SET_53BIT_PRECISION_START;
+    result = _Py_dg_dtoa(dd, mode, ndigits, decpt, sign, rve);
+    _PyPy_SET_53BIT_PRECISION_END;
+    return result;
+}
+
+void _PyPy_dg_freedtoa(char *s)
+{
+    _Py_dg_freedtoa(s);
+}
+/* End PYPY hacks */
+
 #ifdef __cplusplus
 }
 #endif

From commits-noreply at bitbucket.org  Sun Feb 13 15:23:47 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sun, 13 Feb 2011 15:23:47 +0100 (CET)
Subject: [pypy-svn] buildbot default: (arigo, pedronis around)
Message-ID: <20110213142347.F2A5E282BE9@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r445:5ec272322af7
Date: 2011-02-13 15:23 +0100
http://bitbucket.org/pypy/buildbot/changeset/5ec272322af7/

Log:	(arigo, pedronis around)

	Cancel the nonsense of the previous checkin. Instead, outcome_set
	objects have a .revision attribute, so just use that directly for
	sorting. We end up with the same sorting algorithm than the one
	used to select the 5 most recent columns to display, which is good.

diff --git a/bot2/pypybuildbot/summary.py b/bot2/pypybuildbot/summary.py
--- a/bot2/pypybuildbot/summary.py
+++ b/bot2/pypybuildbot/summary.py
@@ -408,6 +408,16 @@
             # rev
             return outcome_set.revision
 
+    def _label_for_sorting(self, outcome_set):
+        encodedrev = encode_rev_for_ordering(outcome_set.revision)
+        if self.fixed_builder:
+            # (rev, buildNumber)
+            buildNumber = self._builder_num(outcome_set)[1]
+            return (encodedrev, buildNumber)
+        else:
+            # rev
+            return encodedrev
+
     def _label_anchor(self, outcome_set, revsize):
         rev = outcome_set.revision
         if self.fixed_builder:
@@ -421,53 +431,10 @@
         rightalign = ' '*(revsize-len(revtxt))
         return [rev_anchor, rightalign]
 
-    def is_older_than(self, outcome_set_1, outcome_set_2):
-        # entries1 is a list [(category, buildbum)]
-        entries1 = [x.key for x in outcome_set_1.map.values()]
-        # entries2 is a dict {category: buildnum}
-        entries2 = dict([x.key for x in outcome_set_2.map.values()])
-        for cat1, buildnum1 in entries1:
-            if cat1 in entries2:
-                return buildnum1 < entries2[cat1]
-        else:
-            return False     # unknown
-
-    def sort_outcome_sets(self, outcome_sets):
-        # Sorting is a bit messy to get right: for example,
-        # outcome_sets[0] might contain two outcomes whose keys are
-        # ('pypy-c-jit-linux-x86-32', 541) and
-        # ('pypy-c-app-level-linux-x86-32', 956),
-        # while outcome_sets[1] might contain three outcomes including
-        # one whose key is ('pypy-c-app-level-linux-x86-32', 957);
-        # then it goes afterwards.  The mess starts if one outcome set
-        # doesn't have any common category with another.
-        #
-        remaining = list(outcome_sets)
-        del outcome_sets[:]
-        # repeatedly pick one of the remaining outcome sets as "the
-        # oldest one", for some definition of "oldest".
-        while remaining:
-            # we start with a random outcome set, and for each outcome set
-            # that seems to be older, we choose that one instead.  We
-            # ignore cycles by explicitly not comparing again.
-            s = remaining.pop()
-            maybe_older = remaining[:]
-            while maybe_older:
-                for outcome_set in maybe_older:
-                    if self.is_older_than(outcome_set, s):
-		        remaining.remove(outcome_set)
-			remaining.append(s)
-                        s = outcome_set
-                        maybe_older.remove(s)
-                        break   # try again, if maybe_older is not empty
-                else:
-                    break    # no older found, keep 's'
-            outcome_sets.append(s)
-
     def add_section(self, cat_branch, outcome_sets):
         if not outcome_sets:
             return
-        self.sort_outcome_sets(outcome_sets)
+        outcome_sets.sort(key=self._label_for_sorting)
         labels = [self._label(outcome_set) for outcome_set in outcome_sets]
         by_label = [(self._label(outcome_set), outcome_set)
                     for outcome_set in outcome_sets]
@@ -659,6 +626,17 @@
     except ValueError:
         return None
 
+def encode_rev_for_ordering(rev):
+    # subversion: just an integer
+    if isinstance(rev, int) or rev.isdigit():
+        return (1, int(rev))
+    # mercurial: "integer:globalid"
+    if ':' in rev and rev[:rev.index(':')].isdigit():
+        i = rev.index(':')
+        return (2, int(rev[:i]), rev)
+    # unknown
+    return (3, rev)
+
 HEAD_ELEMENTS = [
     '<title>%(title)s</title>',
     '<link href="%(root)ssummary.css" rel="stylesheet" type="text/css" />',
@@ -688,23 +666,15 @@
 
     @staticmethod
     def _prune_runs(runs, cutnum):
+        keys = runs.keys()
         #
         def revkey(rev):
             if isinstance(rev, tuple):
-                extra, rev = rev
+                buildNumber, rev = rev
             else:
-                extra = None
-            # subversion: just an integer
-            if isinstance(rev, int) or rev.isdigit():
-                return (extra, 1, int(rev))
-            # mercurial: "integer:globalid"
-            if ':' in rev and rev[:rev.index(':')].isdigit():
-                i = rev.index(':')
-                return (extra, 2, int(rev[:i]), rev)
-            # unknown
-            return (extra, 3, rev)
+                buildNumber = None
+            return (buildNumber, encode_rev_for_ordering(rev))
         #
-        keys = runs.keys()
         keys.sort(key=revkey)
         if len(runs) > cutnum:
             for rev in keys[:-cutnum]:

From commits-noreply at bitbucket.org  Sun Feb 13 15:35:23 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sun, 13 Feb 2011 15:35:23 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix on 64-bits.
Message-ID: <20110213143523.909FD282BE9@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41873:c2afb329a2d8
Date: 2011-02-13 15:30 +0100
http://bitbucket.org/pypy/pypy/changeset/c2afb329a2d8/

Log:	Fix on 64-bits.

diff --git a/pypy/jit/backend/llgraph/runner.py b/pypy/jit/backend/llgraph/runner.py
--- a/pypy/jit/backend/llgraph/runner.py
+++ b/pypy/jit/backend/llgraph/runner.py
@@ -86,7 +86,7 @@
 
 class BaseCPU(model.AbstractCPU):
     supports_floats = True
-    supports_longlong = True
+    supports_longlong = llimpl.IS_32_BIT
 
     def __init__(self, rtyper, stats=None, opts=None,
                  translate_support_code=False,

From commits-noreply at bitbucket.org  Sun Feb 13 15:35:24 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sun, 13 Feb 2011 15:35:24 +0100 (CET)
Subject: [pypy-svn] pypy default: Test and fix on 64-bit.
Message-ID: <20110213143524.654DD282BE9@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41874:9f20df673030
Date: 2011-02-13 15:34 +0100
http://bitbucket.org/pypy/pypy/changeset/9f20df673030/

Log:	Test and fix on 64-bit.

diff --git a/pypy/rlib/test/test_rbigint.py b/pypy/rlib/test/test_rbigint.py
--- a/pypy/rlib/test/test_rbigint.py
+++ b/pypy/rlib/test/test_rbigint.py
@@ -19,6 +19,10 @@
                     r2 = getattr(operator, op)(op1, op2)
                     assert r1.tolong() == r2
 
+    def test_frombool(self):
+        assert rbigint.frombool(False).tolong() == 0
+        assert rbigint.frombool(True).tolong() == 1
+
     def test_str(self):
         for i in range(100):
             n = 3 ** i

diff --git a/pypy/rlib/rbigint.py b/pypy/rlib/rbigint.py
--- a/pypy/rlib/rbigint.py
+++ b/pypy/rlib/rbigint.py
@@ -71,6 +71,7 @@
     return rffi.cast(lltype.Unsigned, x)
 
 NULLDIGIT = _store_digit(0)
+ONEDIGIT  = _store_digit(1)
 
 def _check_digits(l):
     for x in l:
@@ -151,7 +152,7 @@
 
     def frombool(b):
         if b:
-            return rbigint([1], 1)
+            return rbigint([ONEDIGIT], 1)
         return rbigint()
     frombool = staticmethod(frombool)
 

From commits-noreply at bitbucket.org  Sun Feb 13 16:06:58 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sun, 13 Feb 2011 16:06:58 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix the test on 64-bit.
Message-ID: <20110213150658.912EA282BE9@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41875:9058f6b139d0
Date: 2011-02-13 15:45 +0100
http://bitbucket.org/pypy/pypy/changeset/9058f6b139d0/

Log:	Fix the test on 64-bit.

diff --git a/pypy/module/fcntl/test/test_fcntl.py b/pypy/module/fcntl/test/test_fcntl.py
--- a/pypy/module/fcntl/test/test_fcntl.py
+++ b/pypy/module/fcntl/test/test_fcntl.py
@@ -198,6 +198,9 @@
         fcntl.lockf(f, fcntl.LOCK_EX | fcntl.LOCK_NB)
 
     def test_large_flag(self):
-        import fcntl
-        assert fcntl.DN_MULTISHOT < 0
+        import fcntl, sys
+        if sys.maxint == 2147483647:
+            assert fcntl.DN_MULTISHOT == -2147483648
+        else:
+            assert fcntl.DN_MULTISHOT == 2147483648
         fcntl.fcntl(0, fcntl.F_NOTIFY, fcntl.DN_MULTISHOT)

From commits-noreply at bitbucket.org  Sun Feb 13 16:06:59 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sun, 13 Feb 2011 16:06:59 +0100 (CET)
Subject: [pypy-svn] pypy default: Add a custom pydoc.py: check that we don't
	report the	__builtin__
Message-ID: <20110213150659.30793282BE9@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41876:83531d4e78dc
Date: 2011-02-13 16:06 +0100
http://bitbucket.org/pypy/pypy/changeset/83531d4e78dc/

Log:	Add a custom pydoc.py: check that we don't report the __builtin__
	module. This issue arises because '__builtins__' is sometimes the
	module and sometimes the dict, and when it is which are not exactly
	the same in PyPy and CPython.

diff --git a/lib-python/2.7.0/pydoc.py b/lib-python/modified-2.7.0/pydoc.py
copy from lib-python/2.7.0/pydoc.py
copy to lib-python/modified-2.7.0/pydoc.py
--- a/lib-python/2.7.0/pydoc.py
+++ b/lib-python/modified-2.7.0/pydoc.py
@@ -620,7 +620,9 @@
             head, '#ffffff', '#7799ee',
             '<a href=".">index</a><br>' + filelink + docloc)
 
-        modules = inspect.getmembers(object, inspect.ismodule)
+        def isnonbuiltinmodule(obj):
+            return inspect.ismodule(obj) and obj is not __builtin__
+        modules = inspect.getmembers(object, isnonbuiltinmodule)
 
         classes, cdict = [], {}
         for key, value in inspect.getmembers(object, inspect.isclass):

From commits-noreply at bitbucket.org  Sun Feb 13 16:16:01 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Sun, 13 Feb 2011 16:16:01 +0100 (CET)
Subject: [pypy-svn] pypy numpy-exp: Refactor virtualizable.py to actually
	allow multiple	jits with virtualizables
Message-ID: <20110213151601.6F15F282BE9@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: numpy-exp
Changeset: r41877:e211d75bf89f
Date: 2011-02-11 22:02 +0200
http://bitbucket.org/pypy/pypy/changeset/e211d75bf89f/

Log:	Refactor virtualizable.py to actually allow multiple jits with
	virtualizables in one compiled executable

diff --git a/pypy/jit/metainterp/resume.py b/pypy/jit/metainterp/resume.py
--- a/pypy/jit/metainterp/resume.py
+++ b/pypy/jit/metainterp/resume.py
@@ -1007,17 +1007,16 @@
             return len(numb.nums)
         index = len(numb.nums) - 1
         virtualizable = self.decode_ref(numb.nums[index])
-        virtualizable = vinfo.cast_gcref_to_vtype(virtualizable)
         if self.resume_after_guard_not_forced == 1:
             # in the middle of handle_async_forcing()
-            assert virtualizable.vable_token
-            virtualizable.vable_token = vinfo.TOKEN_NONE
+            assert vinfo.gettoken(virtualizable)
+            vinfo.settoken(virtualizable, vinfo.TOKEN_NONE)
         else:
             # just jumped away from assembler (case 4 in the comment in
             # virtualizable.py) into tracing (case 2); check that vable_token
             # is and stays 0.  Note the call to reset_vable_token() in
             # warmstate.py.
-            assert not virtualizable.vable_token
+            assert not vinfo.gettoken(virtualizable)
         return vinfo.write_from_resume_data_partial(virtualizable, self, numb)
 
     def load_value_of_type(self, TYPE, tagged):

diff --git a/pypy/jit/metainterp/pyjitpl.py b/pypy/jit/metainterp/pyjitpl.py
--- a/pypy/jit/metainterp/pyjitpl.py
+++ b/pypy/jit/metainterp/pyjitpl.py
@@ -2151,7 +2151,7 @@
             # warmstate.py.
             virtualizable_box = self.virtualizable_boxes[-1]
             virtualizable = vinfo.unwrap_virtualizable_box(virtualizable_box)
-            assert not virtualizable.vable_token
+            assert not vinfo.gettoken(virtualizable)
             # fill the virtualizable with the local boxes
             self.synchronize_virtualizable()
         #

diff --git a/pypy/jit/metainterp/test/test_ztranslation.py b/pypy/jit/metainterp/test/test_ztranslation.py
--- a/pypy/jit/metainterp/test/test_ztranslation.py
+++ b/pypy/jit/metainterp/test/test_ztranslation.py
@@ -29,6 +29,12 @@
             def __init__(self, i):
                 self.i = i
 
+        class OtherFrame(object):
+            _virtualizable2_ = ['i']
+
+            def __init__(self, i):
+                self.i = i
+
         class JitCellCache:
             entry = None
         jitcellcache = JitCellCache()
@@ -61,18 +67,20 @@
                 frame.i -= 1
             return total * 10
         #
-        myjitdriver2 = JitDriver(greens = ['g'], reds = ['m', 'x', 's'])
+        myjitdriver2 = JitDriver(greens = ['g'], reds = ['m', 's', 'f'],
+                                 virtualizables = ['f'])
         def f2(g, m, x):
             s = ""
+            f = OtherFrame(x)
             while m > 0:
-                myjitdriver2.can_enter_jit(g=g, m=m, x=x, s=s)
-                myjitdriver2.jit_merge_point(g=g, m=m, x=x, s=s)
+                myjitdriver2.can_enter_jit(g=g, m=m, f=f, s=s)
+                myjitdriver2.jit_merge_point(g=g, m=m, f=f, s=s)
                 s += 'xy'
                 if s[:2] == 'yz':
                     return -666
                 m -= 1
-                x += 3
-            return x
+                f.i += 3
+            return f.i
         #
         def main(i, j):
             return f(i) - f2(i+j, i, j)
@@ -124,9 +132,7 @@
                                      type_system=self.type_system,
                                      optimizer=OPTIMIZER_FULL,
                                      ProfilerClass=Profiler)
-        assert res == main(40)
-
-
+        assert res == main(40)        
 
 class TestTranslationLLtype(TranslationTest):
 

diff --git a/pypy/jit/metainterp/virtualizable.py b/pypy/jit/metainterp/virtualizable.py
--- a/pypy/jit/metainterp/virtualizable.py
+++ b/pypy/jit/metainterp/virtualizable.py
@@ -1,4 +1,4 @@
-from pypy.rpython.lltypesystem import lltype
+from pypy.rpython.lltypesystem import lltype, llmemory
 from pypy.rpython.ootypesystem import ootype
 from pypy.rpython.annlowlevel import cast_base_ptr_to_instance
 from pypy.rpython import rvirtualizable2
@@ -8,7 +8,7 @@
 from pypy.jit.metainterp.typesystem import deref, fieldType, arrayItem
 from pypy.jit.metainterp import history
 from pypy.jit.metainterp.warmstate import wrap, unwrap
-
+from pypy.rlib.objectmodel import specialize
 
 class VirtualizableInfo:
     TOKEN_NONE            = 0      # must be 0 -- see also x86.call_assembler
@@ -76,6 +76,8 @@
         setarrayitem = cpu.ts.setarrayitem
         #
         def read_boxes(cpu, virtualizable):
+            assert lltype.typeOf(virtualizable) == llmemory.GCREF
+            virtualizable = cast_gcref_to_vtype(virtualizable)
             boxes = []
             for _, fieldname in unroll_static_fields:
                 x = getattr(virtualizable, fieldname)
@@ -87,6 +89,7 @@
             return boxes
         #
         def write_boxes(virtualizable, boxes):
+            virtualizable = cast_gcref_to_vtype(virtualizable)
             i = 0
             for FIELDTYPE, fieldname in unroll_static_fields:
                 x = unwrap(FIELDTYPE, boxes[i])
@@ -101,6 +104,7 @@
             assert len(boxes) == i + 1
         #
         def write_from_resume_data_partial(virtualizable, reader, numb):
+            virtualizable = cast_gcref_to_vtype(virtualizable)
             # Load values from the reader (see resume.py) described by
             # the list of numbers 'nums', and write them in their proper
             # place in the 'virtualizable'.  This works from the end of
@@ -124,6 +128,7 @@
             return i
         #
         def load_list_of_boxes(virtualizable, reader, numb):
+            virtualizable = cast_gcref_to_vtype(virtualizable)
             # Uses 'virtualizable' only to know the length of the arrays;
             # does not write anything into it.  The returned list is in
             # the format expected of virtualizable_boxes, so it ends in
@@ -147,6 +152,7 @@
             return boxes
         #
         def check_boxes(virtualizable, boxes):
+            virtualizable = cast_gcref_to_vtype(virtualizable)
             # for debugging
             i = 0
             for FIELDTYPE, fieldname in unroll_static_fields:
@@ -162,6 +168,7 @@
             assert len(boxes) == i + 1
         #
         def get_index_in_array(virtualizable, arrayindex, index):
+            virtualizable = cast_gcref_to_vtype(virtualizable)
             index += self.num_static_extra_boxes
             j = 0
             for _, fieldname in unroll_array_fields:
@@ -173,6 +180,7 @@
             assert False, "invalid arrayindex"
         #
         def get_array_length(virtualizable, arrayindex):
+            virtualizable = cast_gcref_to_vtype(virtualizable)
             j = 0
             for _, fieldname in unroll_array_fields:
                 if arrayindex == j:
@@ -197,6 +205,70 @@
         self.get_index_in_array = get_index_in_array
         self.get_array_length = get_array_length
 
+        def cast_to_vtype(virtualizable):
+            return self.cpu.ts.cast_to_instance_maybe(VTYPEPTR, virtualizable)
+        self.cast_to_vtype = cast_to_vtype
+
+        def cast_gcref_to_vtype(virtualizable):
+            assert lltype.typeOf(virtualizable) == llmemory.GCREF
+            return lltype.cast_opaque_ptr(VTYPEPTR, virtualizable)
+        self.cast_gcref_to_vtype = cast_gcref_to_vtype
+
+        def reset_vable_token(virtualizable):
+            virtualizable.vable_token = VirtualizableInfo.TOKEN_NONE
+        self.reset_vable_token = reset_vable_token
+
+        def clear_vable_token(virtualizable):
+            virtualizable = cast_gcref_to_vtype(virtualizable)
+            if virtualizable.vable_token:
+                force_now(virtualizable)
+                assert not virtualizable.vable_token
+        self.clear_vable_token = clear_vable_token
+
+        def tracing_before_residual_call(virtualizable):
+            virtualizable = cast_gcref_to_vtype(virtualizable)
+            assert not virtualizable.vable_token
+            virtualizable.vable_token = VirtualizableInfo.TOKEN_TRACING_RESCALL
+        self.tracing_before_residual_call = tracing_before_residual_call
+
+        def tracing_after_residual_call(virtualizable):
+            virtualizable = cast_gcref_to_vtype(virtualizable)
+            if virtualizable.vable_token:
+                # not modified by the residual call; assert that it is still
+                # set to TOKEN_TRACING_RESCALL and clear it.
+                assert virtualizable.vable_token == VirtualizableInfo.TOKEN_TRACING_RESCALL
+                virtualizable.vable_token = VirtualizableInfo.TOKEN_NONE
+                return False
+            else:
+                # marker "modified during residual call" set.
+                return True
+        self.tracing_after_residual_call = tracing_after_residual_call
+
+        def force_now(virtualizable):
+            token = virtualizable.vable_token
+            if token == VirtualizableInfo.TOKEN_TRACING_RESCALL:
+                # The values in the virtualizable are always correct during
+                # tracing.  We only need to reset vable_token to TOKEN_NONE
+                # as a marker for the tracing, to tell it that this
+                # virtualizable escapes.
+                virtualizable.vable_token = VirtualizableInfo.TOKEN_NONE
+            else:
+                from pypy.jit.metainterp.compile import ResumeGuardForcedDescr
+                ResumeGuardForcedDescr.force_now(cpu, token)
+                assert virtualizable.vable_token == VirtualizableInfo.TOKEN_NONE
+        force_now._dont_inline_ = True
+        self.force_now = force_now
+
+        def gettoken(virtualizable):
+            virtualizable = cast_gcref_to_vtype(virtualizable)
+            return virtualizable.vable_token
+        self.gettoken = gettoken
+
+        def settoken(virtualizable, token):
+            virtualizable = cast_gcref_to_vtype(virtualizable)
+            virtualizable.vable_token = token
+        self.settoken = settoken
+
     def _freeze_(self):
         return True
 
@@ -216,55 +288,11 @@
             all_graphs, self.VTYPEPTR, funcptr)
 
     def unwrap_virtualizable_box(self, virtualizable_box):
-        return virtualizable_box.getref(self.VTYPEPTR)
-
-    def cast_to_vtype(self, virtualizable):
-        return self.cpu.ts.cast_to_instance_maybe(self.VTYPEPTR, virtualizable)
-    cast_to_vtype._annspecialcase_ = 'specialize:ll'
-
-    def cast_gcref_to_vtype(self, virtualizable):
-        return lltype.cast_opaque_ptr(self.VTYPEPTR, virtualizable)
-
+        return virtualizable_box.getref(llmemory.GCREF)
+     
     def is_vtypeptr(self, TYPE):
         return rvirtualizable2.match_virtualizable_type(TYPE, self.VTYPEPTR)
 
-    def reset_vable_token(self, virtualizable):
-        virtualizable.vable_token = self.TOKEN_NONE
-
-    def clear_vable_token(self, virtualizable):
-        if virtualizable.vable_token:
-            self.force_now(virtualizable)
-            assert not virtualizable.vable_token
-
-    def tracing_before_residual_call(self, virtualizable):
-        assert not virtualizable.vable_token
-        virtualizable.vable_token = self.TOKEN_TRACING_RESCALL
-
-    def tracing_after_residual_call(self, virtualizable):
-        if virtualizable.vable_token:
-            # not modified by the residual call; assert that it is still
-            # set to TOKEN_TRACING_RESCALL and clear it.
-            assert virtualizable.vable_token == self.TOKEN_TRACING_RESCALL
-            virtualizable.vable_token = self.TOKEN_NONE
-            return False
-        else:
-            # marker "modified during residual call" set.
-            return True
-
-    def force_now(self, virtualizable):
-        token = virtualizable.vable_token
-        if token == self.TOKEN_TRACING_RESCALL:
-            # The values in the virtualizable are always correct during
-            # tracing.  We only need to reset vable_token to TOKEN_NONE
-            # as a marker for the tracing, to tell it that this
-            # virtualizable escapes.
-            virtualizable.vable_token = self.TOKEN_NONE
-        else:
-            from pypy.jit.metainterp.compile import ResumeGuardForcedDescr
-            ResumeGuardForcedDescr.force_now(self.cpu, token)
-            assert virtualizable.vable_token == self.TOKEN_NONE
-    force_now._dont_inline_ = True
-
 # ____________________________________________________________
 #
 # The 'vable_token' field of a virtualizable is either 0, -1, or points

From commits-noreply at bitbucket.org  Sun Feb 13 16:16:02 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Sun, 13 Feb 2011 16:16:02 +0100 (CET)
Subject: [pypy-svn] pypy numpy-exp: enable micronympy by default on this
	branch
Message-ID: <20110213151602.0A7EB282BE9@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: numpy-exp
Changeset: r41878:2f61693f910a
Date: 2011-02-11 22:03 +0200
http://bitbucket.org/pypy/pypy/changeset/2f61693f910a/

Log:	enable micronympy by default on this branch

diff --git a/pypy/config/pypyoption.py b/pypy/config/pypyoption.py
--- a/pypy/config/pypyoption.py
+++ b/pypy/config/pypyoption.py
@@ -32,7 +32,7 @@
      "crypt", "signal", "_rawffi", "termios", "zlib", "bz2",
      "struct", "_hashlib", "_md5", "_sha", "_minimal_curses", "cStringIO",
      "thread", "itertools", "pyexpat", "_ssl", "cpyext", "array",
-     "_bisect", "binascii", "_multiprocessing", '_warnings']
+     "_bisect", "binascii", "_multiprocessing", '_warnings', 'micronumpy']
 ))
 
 translation_modules = default_modules.copy()

From commits-noreply at bitbucket.org  Sun Feb 13 16:16:03 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Sun, 13 Feb 2011 16:16:03 +0100 (CET)
Subject: [pypy-svn] pypy numpy-exp: Improve the test and fix it by adding
	specialization	left and right
Message-ID: <20110213151603.1D44D2A2031@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: numpy-exp
Changeset: r41879:bbad36eb3203
Date: 2011-02-11 23:46 +0200
http://bitbucket.org/pypy/pypy/changeset/bbad36eb3203/

Log:	Improve the test and fix it by adding specialization left and right

diff --git a/pypy/jit/metainterp/test/test_ztranslation.py b/pypy/jit/metainterp/test/test_ztranslation.py
--- a/pypy/jit/metainterp/test/test_ztranslation.py
+++ b/pypy/jit/metainterp/test/test_ztranslation.py
@@ -2,7 +2,7 @@
 from pypy.jit.metainterp.warmspot import rpython_ll_meta_interp, ll_meta_interp
 from pypy.jit.backend.llgraph import runner
 from pypy.rlib.jit import JitDriver, OPTIMIZER_FULL, unroll_parameters
-from pypy.rlib.jit import PARAMETERS, dont_look_inside
+from pypy.rlib.jit import PARAMETERS, dont_look_inside, hint
 from pypy.jit.metainterp.jitprof import Profiler
 from pypy.rpython.lltypesystem import lltype, llmemory
 
@@ -24,16 +24,21 @@
         # - string concatenation, slicing and comparison
 
         class Frame(object):
-            _virtualizable2_ = ['i']
+            _virtualizable2_ = ['l[*]']
 
             def __init__(self, i):
-                self.i = i
+                self = hint(self, fresh_virtualizable=True,
+                            access_directly=True)
+                self.l = [i]
 
         class OtherFrame(object):
-            _virtualizable2_ = ['i']
+            _virtualizable2_ = ['i', 'l[*]']
 
             def __init__(self, i):
+                self = hint(self, fresh_virtualizable=True,
+                            access_directly=True)
                 self.i = i
+                self.l = [float(i)]
 
         class JitCellCache:
             entry = None
@@ -58,40 +63,46 @@
             jitdriver.set_param("trace_eagerness", 2)
             total = 0
             frame = Frame(i)
-            while frame.i > 3:
+            while frame.l[0] > 3:
                 jitdriver.can_enter_jit(frame=frame, total=total)
                 jitdriver.jit_merge_point(frame=frame, total=total)
-                total += frame.i
-                if frame.i >= 20:
-                    frame.i -= 2
-                frame.i -= 1
+                total += frame.l[0]
+                if frame.l[0] >= 20:
+                    frame.l[0] -= 2
+                frame.l[0] -= 1
             return total * 10
         #
-        myjitdriver2 = JitDriver(greens = ['g'], reds = ['m', 's', 'f'],
+        myjitdriver2 = JitDriver(greens = ['g'],
+                                 reds = ['m', 's', 'f', 'float_s'],
                                  virtualizables = ['f'])
         def f2(g, m, x):
             s = ""
             f = OtherFrame(x)
+            float_s = 0.0
             while m > 0:
-                myjitdriver2.can_enter_jit(g=g, m=m, f=f, s=s)
-                myjitdriver2.jit_merge_point(g=g, m=m, f=f, s=s)
+                myjitdriver2.can_enter_jit(g=g, m=m, f=f, s=s, float_s=float_s)
+                myjitdriver2.jit_merge_point(g=g, m=m, f=f, s=s,
+                                             float_s=float_s)
                 s += 'xy'
                 if s[:2] == 'yz':
                     return -666
                 m -= 1
                 f.i += 3
+                float_s += f.l[0]
             return f.i
         #
         def main(i, j):
             return f(i) - f2(i+j, i, j)
         res = ll_meta_interp(main, [40, 5], CPUClass=self.CPUClass,
-                             type_system=self.type_system)
+                             type_system=self.type_system,
+                             listops=True)
         assert res == main(40, 5)
         res = rpython_ll_meta_interp(main, [40, 5],
                                      CPUClass=self.CPUClass,
                                      type_system=self.type_system,
                                      optimizer=OPTIMIZER_FULL,
-                                     ProfilerClass=Profiler)
+                                     ProfilerClass=Profiler,
+                                     listops=True)
         assert res == main(40, 5)
 
     def test_external_exception_handling_translates(self):
@@ -132,7 +143,7 @@
                                      type_system=self.type_system,
                                      optimizer=OPTIMIZER_FULL,
                                      ProfilerClass=Profiler)
-        assert res == main(40)        
+        assert res == main(40)
 
 class TestTranslationLLtype(TranslationTest):
 

diff --git a/pypy/jit/metainterp/typesystem.py b/pypy/jit/metainterp/typesystem.py
--- a/pypy/jit/metainterp/typesystem.py
+++ b/pypy/jit/metainterp/typesystem.py
@@ -5,7 +5,7 @@
 from pypy.rpython.annlowlevel import cast_instance_to_base_obj
 from pypy.jit.metainterp import history
 from pypy.jit.codewriter import heaptracker
-from pypy.rlib.objectmodel import r_dict
+from pypy.rlib.objectmodel import r_dict, specialize
 
 def deref(T):
     if isinstance(T, lltype.Ptr):
@@ -97,12 +97,15 @@
     def cast_to_baseclass(self, value):
         return lltype.cast_opaque_ptr(lltype.Ptr(rclass.OBJECT), value)
 
+    @specialize.ll()
     def getlength(self, array):
         return len(array)
 
+    @specialize.ll()
     def getarrayitem(self, array, i):
         return array[i]
 
+    @specialize.ll()
     def setarrayitem(self, array, i, newvalue):
         array[i] = newvalue
 
@@ -201,12 +204,15 @@
     def cast_to_baseclass(self, value):
         return ootype.cast_from_object(ootype.ROOT, value)
 
+    @specialize.ll()
     def getlength(self, array):
         return array.ll_length()
 
+    @specialize.ll()
     def getarrayitem(self, array, i):
         return array.ll_getitem_fast(i)
 
+    @specialize.ll()
     def setarrayitem(self, array, i, newvalue):
         array.ll_setitem_fast(i, newvalue)
 

From commits-noreply at bitbucket.org  Sun Feb 13 16:16:05 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Sun, 13 Feb 2011 16:16:05 +0100 (CET)
Subject: [pypy-svn] pypy numpy-exp: Start adding sse_float_add ops
Message-ID: <20110213151605.9F6772A2031@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: numpy-exp
Changeset: r41880:084ea2d068ea
Date: 2011-02-12 15:06 +0200
http://bitbucket.org/pypy/pypy/changeset/084ea2d068ea/

Log:	Start adding sse_float_add ops

diff --git a/pypy/rpython/lltypesystem/lloperation.py b/pypy/rpython/lltypesystem/lloperation.py
--- a/pypy/rpython/lltypesystem/lloperation.py
+++ b/pypy/rpython/lltypesystem/lloperation.py
@@ -441,6 +441,10 @@
     'get_write_barrier_from_array_failing_case': LLOp(sideeffects=False),
     'gc_get_type_info_group': LLOp(sideeffects=False),
 
+    # __________ vectorization ops _______
+
+    'sse_float_add': LLOp(sideeffects=False, canrun=True),
+
     # __________ GC operations __________
 
     'gc__collect':          LLOp(canunwindgc=True),

diff --git a/pypy/rpython/lltypesystem/opimpl.py b/pypy/rpython/lltypesystem/opimpl.py
--- a/pypy/rpython/lltypesystem/opimpl.py
+++ b/pypy/rpython/lltypesystem/opimpl.py
@@ -570,6 +570,10 @@
 def op_shrink_array(array, smallersize):
     return False
 
+def op_sse_float_add(arr1, arr2, arr_res, index):
+    arr_res[index] = arr1[index] + arr2[index]
+    arr_res[index + 1] = arr1[index + 1] + arr2[index + 1]
+
 # ____________________________________________________________
 
 def get_op_impl(opname):

From commits-noreply at bitbucket.org  Sun Feb 13 16:16:11 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Sun, 13 Feb 2011 16:16:11 +0100 (CET)
Subject: [pypy-svn] pypy numpy-exp: Make a basic SSE operation on array
	work. It doesn't	guarantee as of now
Message-ID: <20110213151611.B78C5282BE9@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: numpy-exp
Changeset: r41881:19755a39b71e
Date: 2011-02-12 20:28 +0200
http://bitbucket.org/pypy/pypy/changeset/19755a39b71e/

Log:	Make a basic SSE operation on array work. It doesn't guarantee as of
	now that arrays are aligned. Have to think for a second about how to
	do it, probably by operating on raw arrays with no length and
	calling special functions for POSIX. *VERY* experimental

diff --git a/pypy/jit/backend/x86/rx86.py b/pypy/jit/backend/x86/rx86.py
--- a/pypy/jit/backend/x86/rx86.py
+++ b/pypy/jit/backend/x86/rx86.py
@@ -656,10 +656,15 @@
 
 define_modrm_modes('MOVSD_x*', ['\xF2', rex_nw, '\x0F\x10', register(1,8)], regtype='XMM')
 define_modrm_modes('MOVSD_*x', ['\xF2', rex_nw, '\x0F\x11', register(2,8)], regtype='XMM')
+define_modrm_modes('MOVAPD_x*', ['\x66', rex_nw, '\x0F\x28', register(1,8)],
+                   regtype='XMM')
+define_modrm_modes('MOVAPD_*x', ['\x66', rex_nw, '\x0F\x29', register(2,8)],
+                   regtype='XMM')
 
 define_modrm_modes('XCHG_r*', [rex_w, '\x87', register(1, 8)])
 
 define_modrm_modes('ADDSD_x*', ['\xF2', rex_nw, '\x0F\x58', register(1, 8)], regtype='XMM')
+define_modrm_modes('ADDPD_x*', ['\x66', rex_nw, '\x0F\x58', register(1, 8)], regtype='XMM')
 define_modrm_modes('SUBSD_x*', ['\xF2', rex_nw, '\x0F\x5C', register(1, 8)], regtype='XMM')
 define_modrm_modes('MULSD_x*', ['\xF2', rex_nw, '\x0F\x59', register(1, 8)], regtype='XMM')
 define_modrm_modes('DIVSD_x*', ['\xF2', rex_nw, '\x0F\x5E', register(1, 8)], regtype='XMM')

diff --git a/pypy/jit/backend/x86/regloc.py b/pypy/jit/backend/x86/regloc.py
--- a/pypy/jit/backend/x86/regloc.py
+++ b/pypy/jit/backend/x86/regloc.py
@@ -500,7 +500,9 @@
     LEA = _binaryop('LEA')
 
     MOVSD = _binaryop('MOVSD')
+    MOVAPD = _binaryop('MOVAPD')
     ADDSD = _binaryop('ADDSD')
+    ADDPD = _binaryop('ADDPD')
     SUBSD = _binaryop('SUBSD')
     MULSD = _binaryop('MULSD')
     DIVSD = _binaryop('DIVSD')

diff --git a/pypy/jit/metainterp/pyjitpl.py b/pypy/jit/metainterp/pyjitpl.py
--- a/pypy/jit/metainterp/pyjitpl.py
+++ b/pypy/jit/metainterp/pyjitpl.py
@@ -394,6 +394,11 @@
     opimpl_getarrayitem_gc_r = _opimpl_getarrayitem_gc_any
     opimpl_getarrayitem_gc_f = _opimpl_getarrayitem_gc_any
 
+    @arguments("descr", "box", "box", "box", "box")
+    def opimpl_sse_float_add(self, arraydescr, array1, array2, arrayres, index):
+        return self.execute_with_descr(rop.SSE_FLOAT_ADD, arraydescr, array1,
+                                       array2, arrayres, index)
+
     @arguments("box", "descr", "box")
     def _opimpl_getarrayitem_raw_any(self, arraybox, arraydescr, indexbox):
         return self.execute_with_descr(rop.GETARRAYITEM_RAW,

diff --git a/pypy/jit/backend/x86/regalloc.py b/pypy/jit/backend/x86/regalloc.py
--- a/pypy/jit/backend/x86/regalloc.py
+++ b/pypy/jit/backend/x86/regalloc.py
@@ -1053,6 +1053,33 @@
         self.Perform(op, [base_loc, ofs_loc, imm(itemsize), imm(ofs),
                           sign_loc], result_loc)
 
+    def _read_elem_into_xmmreg(self, xmmreg, array, index, ofs_loc, arraydescr):
+        itemsize, ofs, _, _, _ = self._unpack_arraydescr(arraydescr)
+        base_loc = self.rm.make_sure_var_in_reg(array, [index])
+        self.assembler.move_from_array_to_xmm(base_loc, ofs_loc,
+                                              imm(itemsize), imm(ofs),
+                                              xmmreg)
+
+    def consider_sse_float_add(self, op):
+        box1 = TempBox()
+        box2 = TempBox()
+        loc1 = self.xrm.force_allocate_reg(box1)
+        loc2 = self.xrm.force_allocate_reg(box2, [box1])
+        arraydescr = op.getdescr()
+        ofs_loc = self.rm.make_sure_var_in_reg(op.getarg(3))
+        self._read_elem_into_xmmreg(loc1, op.getarg(0), op.getarg(3), ofs_loc,
+                                    arraydescr)
+        if op.getarg(1) != op.getarg(0):
+            self._read_elem_into_xmmreg(loc2, op.getarg(1), op.getarg(3),
+                                        ofs_loc, arraydescr)
+        base_loc = self.rm.make_sure_var_in_reg(op.getarg(2), [op.getarg(3)])
+        itemsize, ofs, _, _, _ = self._unpack_arraydescr(arraydescr)
+        self.possibly_free_vars(op.getarglist())
+        self.xrm.possibly_free_var(box1)
+        self.xrm.possibly_free_var(box2)
+        self.PerformDiscard(op, [base_loc, ofs_loc, imm(itemsize), imm(ofs),
+                                 loc1, loc2])
+
     consider_getarrayitem_raw = consider_getarrayitem_gc
     consider_getarrayitem_gc_pure = consider_getarrayitem_gc
 

diff --git a/pypy/translator/c/src/float.h b/pypy/translator/c/src/float.h
--- a/pypy/translator/c/src/float.h
+++ b/pypy/translator/c/src/float.h
@@ -43,3 +43,7 @@
 #define OP_CAST_FLOAT_TO_LONGLONG(x,r) r = (long long)(x)
 #define OP_CAST_FLOAT_TO_ULONGLONG(x,r) r = (unsigned long long)(x)
 #endif
+
+/*** those operations don't do anything because they're in
+     if we_are_jitted() path ***/
+#define OP_SSE_FLOAT_ADD(a, b, c, d, e)

diff --git a/pypy/jit/metainterp/executor.py b/pypy/jit/metainterp/executor.py
--- a/pypy/jit/metainterp/executor.py
+++ b/pypy/jit/metainterp/executor.py
@@ -122,6 +122,17 @@
     else:
         cpu.bh_setarrayitem_raw_i(arraydescr, array, index, itembox.getint())
 
+def do_sse_float_add(cpu, _, array1, array2, arrayres, indexbox, arraydescr):
+    onebox = do_getarrayitem_raw(cpu, _, array1, indexbox, arraydescr)
+    twobox = do_getarrayitem_raw(cpu, _, array2, indexbox, arraydescr)
+    res = onebox.getfloat() + twobox.getfloat()
+    do_setarrayitem_raw(cpu, _, arrayres, indexbox, BoxFloat(res), arraydescr)
+    indexbox = BoxInt(indexbox.getint() + 1)
+    onebox = do_getarrayitem_raw(cpu, _, array1, indexbox, arraydescr)
+    twobox = do_getarrayitem_raw(cpu, _, array2, indexbox, arraydescr)
+    res = onebox.getfloat() + twobox.getfloat()
+    do_setarrayitem_raw(cpu, _, arrayres, indexbox, BoxFloat(res), arraydescr)
+
 def do_getfield_gc(cpu, _, structbox, fielddescr):
     struct = structbox.getref_base()
     if fielddescr.is_pointer_field():

diff --git a/pypy/jit/metainterp/blackhole.py b/pypy/jit/metainterp/blackhole.py
--- a/pypy/jit/metainterp/blackhole.py
+++ b/pypy/jit/metainterp/blackhole.py
@@ -1036,6 +1036,15 @@
     def bhimpl_setarrayitem_raw_f(cpu, array, arraydescr, index, newvalue):
         cpu.bh_setarrayitem_raw_f(arraydescr, array, index, newvalue)
 
+    @arguments("cpu", "d", "i", "i", "i", "i")
+    def bhimpl_sse_float_add(cpu, arraydescr, array1, array2, array_res, index):
+        one = cpu.bh_getarrayitem_raw_f(arraydescr, array1, index)
+        two = cpu.bh_getarrayitem_raw_f(arraydescr, array2, index)
+        cpu.bh_setarrayitem_raw_f(arraydescr, array_res, index, one + two)
+        one = cpu.bh_getarrayitem_raw_f(arraydescr, array1, index + 1)
+        two = cpu.bh_getarrayitem_raw_f(arraydescr, array2, index + 1)
+        cpu.bh_setarrayitem_raw_f(arraydescr, array_res, index + 1, one + two)
+
     # note, there is no 'r' here, since it can't happen
 
     @arguments("cpu", "r", "d", returns="i")

diff --git a/pypy/rpython/lltypesystem/lloperation.py b/pypy/rpython/lltypesystem/lloperation.py
--- a/pypy/rpython/lltypesystem/lloperation.py
+++ b/pypy/rpython/lltypesystem/lloperation.py
@@ -443,7 +443,7 @@
 
     # __________ vectorization ops _______
 
-    'sse_float_add': LLOp(sideeffects=False, canrun=True),
+    'sse_float_add': LLOp(canrun=True),
 
     # __________ GC operations __________
 

diff --git a/pypy/jit/codewriter/jtransform.py b/pypy/jit/codewriter/jtransform.py
--- a/pypy/jit/codewriter/jtransform.py
+++ b/pypy/jit/codewriter/jtransform.py
@@ -526,6 +526,14 @@
         return self._do_builtin_call(op, 'raw_free', [op.args[0]],
                                      extra = (ARRAY,), extrakey = ARRAY)
 
+    def rewrite_op_sse_float_add(self, op):
+        ARRAY = op.args[0].concretetype.TO
+        arraydescr = self.cpu.arraydescrof(ARRAY)
+        kind = getkind(op.result.concretetype)
+        assert kind == 'void'
+        return SpaceOperation('sse_float_add',
+                              [arraydescr] + op.args, op.result)
+
     def rewrite_op_getarrayitem(self, op):
         ARRAY = op.args[0].concretetype.TO
         if self._array_of_voids(ARRAY):

diff --git a/pypy/jit/backend/llgraph/llimpl.py b/pypy/jit/backend/llgraph/llimpl.py
--- a/pypy/jit/backend/llgraph/llimpl.py
+++ b/pypy/jit/backend/llgraph/llimpl.py
@@ -161,6 +161,7 @@
     'force_token'     : ((), 'int'),
     'call_may_force'  : (('int', 'varargs'), 'intorptr'),
     'guard_not_forced': ((), None),
+    'sse_float_add'   : (('int', 'int', 'int', 'int'), None),
 }
 
 # ____________________________________________________________
@@ -735,6 +736,12 @@
 
     op_getarrayitem_raw_pure = op_getarrayitem_raw
 
+    def op_sse_float_add(self, arraydescr, array1, array2, arrayres,
+                         index):
+        from pypy.jit.metainterp.blackhole import BlackholeInterpreter
+        return BlackholeInterpreter.bhimpl_sse_float_add.im_func(self.cpu,
+               arraydescr, array1, array2, arrayres, index)
+
     def op_getfield_gc(self, fielddescr, struct):
         if fielddescr.typeinfo == REF:
             return do_getfield_gc_ptr(struct, fielddescr.ofs)

diff --git a/pypy/jit/backend/x86/assembler.py b/pypy/jit/backend/x86/assembler.py
--- a/pypy/jit/backend/x86/assembler.py
+++ b/pypy/jit/backend/x86/assembler.py
@@ -740,6 +740,14 @@
         else:
             self.mc.MOV(to_loc, from_loc)
 
+    def move_from_array_to_xmm(self, base_loc, ofs_loc, itemsize, ofs, xmmreg):
+        assert isinstance(xmmreg, RegLoc)
+        assert isinstance(ofs, ImmedLoc)
+        assert isinstance(itemsize, ImmedLoc)
+        scale = _get_scale(itemsize.value)
+        src_addr = addr_add(base_loc, ofs_loc, ofs.value, scale)
+        self.mc.MOVAPD(xmmreg, src_addr)
+
     regalloc_mov = mov # legacy interface
 
     def regalloc_push(self, loc):
@@ -1121,6 +1129,17 @@
         self.mc.XOR_rr(edx.value, edx.value)
         self.mc.DIV_r(ecx.value)
 
+    def genop_discard_sse_float_add(self, op, arglocs):
+        base_loc, ofs_loc, itemsize, ofs, loc1, loc2 = arglocs
+        assert isinstance(loc1, RegLoc)
+        assert isinstance(loc2, RegLoc)
+        assert isinstance(ofs, ImmedLoc)
+        assert isinstance(itemsize, ImmedLoc)
+        scale = _get_scale(itemsize.value)
+        dest_addr = addr_add(base_loc, ofs_loc, ofs.value, scale)
+        self.mc.ADDPD(loc1, loc2)
+        self.mc.MOVAPD(dest_addr, loc1)
+
     genop_llong_add = _binaryop("PADDQ", True)
     genop_llong_sub = _binaryop("PSUBQ")
     genop_llong_and = _binaryop("PAND",  True)

diff --git a/pypy/jit/metainterp/resoperation.py b/pypy/jit/metainterp/resoperation.py
--- a/pypy/jit/metainterp/resoperation.py
+++ b/pypy/jit/metainterp/resoperation.py
@@ -466,6 +466,7 @@
     'SETARRAYITEM_RAW/3d',
     'SETFIELD_GC/2d',
     'SETFIELD_RAW/2d',
+    'SSE_FLOAT_ADD/4d',
     'STRSETITEM/3',
     'UNICODESETITEM/3',
     #'RUNTIMENEW/1',     # ootype operation    

From commits-noreply at bitbucket.org  Sun Feb 13 16:16:12 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Sun, 13 Feb 2011 16:16:12 +0100 (CET)
Subject: [pypy-svn] pypy numpy-exp: Add a standalone numpy target
Message-ID: <20110213151612.DD03D282BE9@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: numpy-exp
Changeset: r41882:3f25b6bccdd4
Date: 2011-02-12 20:46 +0200
http://bitbucket.org/pypy/pypy/changeset/3f25b6bccdd4/

Log:	Add a standalone numpy target

diff --git a/pypy/translator/goal/targetnumpystandalone.py b/pypy/translator/goal/targetnumpystandalone.py
new file mode 100644
--- /dev/null
+++ b/pypy/translator/goal/targetnumpystandalone.py
@@ -0,0 +1,55 @@
+
+""" Usage:
+
+./targetnumpystandalone-c <bytecode> array_size
+
+Will execute a give numpy bytecode. Arrays will be ranges (in float) modulo 10,
+constants would be consecutive starting from one.
+
+Bytecode should contain letters 'a' 'l' and 'f' so far and be correct
+"""
+
+import time
+from pypy.module.micronumpy.numarray import SingleDimArray, compute
+from pypy.jit.codewriter.policy import JitPolicy
+
+def create_array(size):
+    a = SingleDimArray(size)
+    for i in range(size):
+        a.storage[i] = float(i % 10)
+    return a
+
+def entry_point(argv):
+    if len(argv) == 2:
+        print __doc__
+        return 1
+    bytecode = argv[1]
+    for b in bytecode:
+        if b not in 'alf':
+            print "WRONG BYTECODE"
+            print __doc__
+            return 2
+    try:
+        size = int(argv[2])
+    except ValueError:
+        print "INVALID LITERAL FOR INT:", argv[2]
+        print __doc__
+        return 3
+    no_arrays = bytecode.count('l')
+    no_floats = bytecode.count('f')
+    arrays = []
+    floats = []
+    for i in range(no_arrays):
+        arrays.append(create_array(size))
+    for i in range(no_floats):
+        floats.append(float(i + 1))
+    t0 = time.time()
+    compute(bytecode, arrays)
+    print "TOOK:", time.time() - t0
+    return 0
+
+def target(*args):
+    return entry_point, None
+
+def jitpolicy(driver):
+    return JitPolicy()

From commits-noreply at bitbucket.org  Sun Feb 13 16:16:13 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Sun, 13 Feb 2011 16:16:13 +0100 (CET)
Subject: [pypy-svn] pypy numpy-exp: fix
Message-ID: <20110213151613.754D9282BE9@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: numpy-exp
Changeset: r41883:f5508711fb3f
Date: 2011-02-12 20:47 +0200
http://bitbucket.org/pypy/pypy/changeset/f5508711fb3f/

Log:	fix

diff --git a/pypy/translator/goal/targetnumpystandalone.py b/pypy/translator/goal/targetnumpystandalone.py
--- a/pypy/translator/goal/targetnumpystandalone.py
+++ b/pypy/translator/goal/targetnumpystandalone.py
@@ -20,7 +20,7 @@
     return a
 
 def entry_point(argv):
-    if len(argv) == 2:
+    if len(argv) != 3:
         print __doc__
         return 1
     bytecode = argv[1]

From commits-noreply at bitbucket.org  Sun Feb 13 16:16:14 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Sun, 13 Feb 2011 16:16:14 +0100 (CET)
Subject: [pypy-svn] pypy numpy-exp: print more info
Message-ID: <20110213151614.11B4D282BE9@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: numpy-exp
Changeset: r41884:0a8bc1fd4660
Date: 2011-02-12 20:48 +0200
http://bitbucket.org/pypy/pypy/changeset/0a8bc1fd4660/

Log:	print more info

diff --git a/pypy/translator/goal/targetnumpystandalone.py b/pypy/translator/goal/targetnumpystandalone.py
--- a/pypy/translator/goal/targetnumpystandalone.py
+++ b/pypy/translator/goal/targetnumpystandalone.py
@@ -45,7 +45,8 @@
         floats.append(float(i + 1))
     t0 = time.time()
     compute(bytecode, arrays)
-    print "TOOK:", time.time() - t0
+    print "bytecode:", bytecode, "size:", size
+    print "took:", time.time() - t0
     return 0
 
 def target(*args):

From commits-noreply at bitbucket.org  Sun Feb 13 16:25:00 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Sun, 13 Feb 2011 16:25:00 +0100 (CET)
Subject: [pypy-svn] pypy default: Refactor virtualizable.py to actually
	allow multiple	jits with virtualizables
Message-ID: <20110213152500.CD377282BE9@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: 
Changeset: r41885:af2afa7b822b
Date: 2011-02-11 22:02 +0200
http://bitbucket.org/pypy/pypy/changeset/af2afa7b822b/

Log:	Refactor virtualizable.py to actually allow multiple jits with
	virtualizables in one compiled executable

diff --git a/pypy/jit/metainterp/resume.py b/pypy/jit/metainterp/resume.py
--- a/pypy/jit/metainterp/resume.py
+++ b/pypy/jit/metainterp/resume.py
@@ -1007,17 +1007,16 @@
             return len(numb.nums)
         index = len(numb.nums) - 1
         virtualizable = self.decode_ref(numb.nums[index])
-        virtualizable = vinfo.cast_gcref_to_vtype(virtualizable)
         if self.resume_after_guard_not_forced == 1:
             # in the middle of handle_async_forcing()
-            assert virtualizable.vable_token
-            virtualizable.vable_token = vinfo.TOKEN_NONE
+            assert vinfo.gettoken(virtualizable)
+            vinfo.settoken(virtualizable, vinfo.TOKEN_NONE)
         else:
             # just jumped away from assembler (case 4 in the comment in
             # virtualizable.py) into tracing (case 2); check that vable_token
             # is and stays 0.  Note the call to reset_vable_token() in
             # warmstate.py.
-            assert not virtualizable.vable_token
+            assert not vinfo.gettoken(virtualizable)
         return vinfo.write_from_resume_data_partial(virtualizable, self, numb)
 
     def load_value_of_type(self, TYPE, tagged):

diff --git a/pypy/jit/metainterp/pyjitpl.py b/pypy/jit/metainterp/pyjitpl.py
--- a/pypy/jit/metainterp/pyjitpl.py
+++ b/pypy/jit/metainterp/pyjitpl.py
@@ -2151,7 +2151,7 @@
             # warmstate.py.
             virtualizable_box = self.virtualizable_boxes[-1]
             virtualizable = vinfo.unwrap_virtualizable_box(virtualizable_box)
-            assert not virtualizable.vable_token
+            assert not vinfo.gettoken(virtualizable)
             # fill the virtualizable with the local boxes
             self.synchronize_virtualizable()
         #

diff --git a/pypy/jit/metainterp/test/test_ztranslation.py b/pypy/jit/metainterp/test/test_ztranslation.py
--- a/pypy/jit/metainterp/test/test_ztranslation.py
+++ b/pypy/jit/metainterp/test/test_ztranslation.py
@@ -29,6 +29,12 @@
             def __init__(self, i):
                 self.i = i
 
+        class OtherFrame(object):
+            _virtualizable2_ = ['i']
+
+            def __init__(self, i):
+                self.i = i
+
         class JitCellCache:
             entry = None
         jitcellcache = JitCellCache()
@@ -61,18 +67,20 @@
                 frame.i -= 1
             return total * 10
         #
-        myjitdriver2 = JitDriver(greens = ['g'], reds = ['m', 'x', 's'])
+        myjitdriver2 = JitDriver(greens = ['g'], reds = ['m', 's', 'f'],
+                                 virtualizables = ['f'])
         def f2(g, m, x):
             s = ""
+            f = OtherFrame(x)
             while m > 0:
-                myjitdriver2.can_enter_jit(g=g, m=m, x=x, s=s)
-                myjitdriver2.jit_merge_point(g=g, m=m, x=x, s=s)
+                myjitdriver2.can_enter_jit(g=g, m=m, f=f, s=s)
+                myjitdriver2.jit_merge_point(g=g, m=m, f=f, s=s)
                 s += 'xy'
                 if s[:2] == 'yz':
                     return -666
                 m -= 1
-                x += 3
-            return x
+                f.i += 3
+            return f.i
         #
         def main(i, j):
             return f(i) - f2(i+j, i, j)
@@ -124,9 +132,7 @@
                                      type_system=self.type_system,
                                      optimizer=OPTIMIZER_FULL,
                                      ProfilerClass=Profiler)
-        assert res == main(40)
-
-
+        assert res == main(40)        
 
 class TestTranslationLLtype(TranslationTest):
 

diff --git a/pypy/jit/metainterp/virtualizable.py b/pypy/jit/metainterp/virtualizable.py
--- a/pypy/jit/metainterp/virtualizable.py
+++ b/pypy/jit/metainterp/virtualizable.py
@@ -1,4 +1,4 @@
-from pypy.rpython.lltypesystem import lltype
+from pypy.rpython.lltypesystem import lltype, llmemory
 from pypy.rpython.ootypesystem import ootype
 from pypy.rpython.annlowlevel import cast_base_ptr_to_instance
 from pypy.rpython import rvirtualizable2
@@ -8,7 +8,7 @@
 from pypy.jit.metainterp.typesystem import deref, fieldType, arrayItem
 from pypy.jit.metainterp import history
 from pypy.jit.metainterp.warmstate import wrap, unwrap
-
+from pypy.rlib.objectmodel import specialize
 
 class VirtualizableInfo:
     TOKEN_NONE            = 0      # must be 0 -- see also x86.call_assembler
@@ -76,6 +76,8 @@
         setarrayitem = cpu.ts.setarrayitem
         #
         def read_boxes(cpu, virtualizable):
+            assert lltype.typeOf(virtualizable) == llmemory.GCREF
+            virtualizable = cast_gcref_to_vtype(virtualizable)
             boxes = []
             for _, fieldname in unroll_static_fields:
                 x = getattr(virtualizable, fieldname)
@@ -87,6 +89,7 @@
             return boxes
         #
         def write_boxes(virtualizable, boxes):
+            virtualizable = cast_gcref_to_vtype(virtualizable)
             i = 0
             for FIELDTYPE, fieldname in unroll_static_fields:
                 x = unwrap(FIELDTYPE, boxes[i])
@@ -101,6 +104,7 @@
             assert len(boxes) == i + 1
         #
         def write_from_resume_data_partial(virtualizable, reader, numb):
+            virtualizable = cast_gcref_to_vtype(virtualizable)
             # Load values from the reader (see resume.py) described by
             # the list of numbers 'nums', and write them in their proper
             # place in the 'virtualizable'.  This works from the end of
@@ -124,6 +128,7 @@
             return i
         #
         def load_list_of_boxes(virtualizable, reader, numb):
+            virtualizable = cast_gcref_to_vtype(virtualizable)
             # Uses 'virtualizable' only to know the length of the arrays;
             # does not write anything into it.  The returned list is in
             # the format expected of virtualizable_boxes, so it ends in
@@ -147,6 +152,7 @@
             return boxes
         #
         def check_boxes(virtualizable, boxes):
+            virtualizable = cast_gcref_to_vtype(virtualizable)
             # for debugging
             i = 0
             for FIELDTYPE, fieldname in unroll_static_fields:
@@ -162,6 +168,7 @@
             assert len(boxes) == i + 1
         #
         def get_index_in_array(virtualizable, arrayindex, index):
+            virtualizable = cast_gcref_to_vtype(virtualizable)
             index += self.num_static_extra_boxes
             j = 0
             for _, fieldname in unroll_array_fields:
@@ -173,6 +180,7 @@
             assert False, "invalid arrayindex"
         #
         def get_array_length(virtualizable, arrayindex):
+            virtualizable = cast_gcref_to_vtype(virtualizable)
             j = 0
             for _, fieldname in unroll_array_fields:
                 if arrayindex == j:
@@ -197,6 +205,70 @@
         self.get_index_in_array = get_index_in_array
         self.get_array_length = get_array_length
 
+        def cast_to_vtype(virtualizable):
+            return self.cpu.ts.cast_to_instance_maybe(VTYPEPTR, virtualizable)
+        self.cast_to_vtype = cast_to_vtype
+
+        def cast_gcref_to_vtype(virtualizable):
+            assert lltype.typeOf(virtualizable) == llmemory.GCREF
+            return lltype.cast_opaque_ptr(VTYPEPTR, virtualizable)
+        self.cast_gcref_to_vtype = cast_gcref_to_vtype
+
+        def reset_vable_token(virtualizable):
+            virtualizable.vable_token = VirtualizableInfo.TOKEN_NONE
+        self.reset_vable_token = reset_vable_token
+
+        def clear_vable_token(virtualizable):
+            virtualizable = cast_gcref_to_vtype(virtualizable)
+            if virtualizable.vable_token:
+                force_now(virtualizable)
+                assert not virtualizable.vable_token
+        self.clear_vable_token = clear_vable_token
+
+        def tracing_before_residual_call(virtualizable):
+            virtualizable = cast_gcref_to_vtype(virtualizable)
+            assert not virtualizable.vable_token
+            virtualizable.vable_token = VirtualizableInfo.TOKEN_TRACING_RESCALL
+        self.tracing_before_residual_call = tracing_before_residual_call
+
+        def tracing_after_residual_call(virtualizable):
+            virtualizable = cast_gcref_to_vtype(virtualizable)
+            if virtualizable.vable_token:
+                # not modified by the residual call; assert that it is still
+                # set to TOKEN_TRACING_RESCALL and clear it.
+                assert virtualizable.vable_token == VirtualizableInfo.TOKEN_TRACING_RESCALL
+                virtualizable.vable_token = VirtualizableInfo.TOKEN_NONE
+                return False
+            else:
+                # marker "modified during residual call" set.
+                return True
+        self.tracing_after_residual_call = tracing_after_residual_call
+
+        def force_now(virtualizable):
+            token = virtualizable.vable_token
+            if token == VirtualizableInfo.TOKEN_TRACING_RESCALL:
+                # The values in the virtualizable are always correct during
+                # tracing.  We only need to reset vable_token to TOKEN_NONE
+                # as a marker for the tracing, to tell it that this
+                # virtualizable escapes.
+                virtualizable.vable_token = VirtualizableInfo.TOKEN_NONE
+            else:
+                from pypy.jit.metainterp.compile import ResumeGuardForcedDescr
+                ResumeGuardForcedDescr.force_now(cpu, token)
+                assert virtualizable.vable_token == VirtualizableInfo.TOKEN_NONE
+        force_now._dont_inline_ = True
+        self.force_now = force_now
+
+        def gettoken(virtualizable):
+            virtualizable = cast_gcref_to_vtype(virtualizable)
+            return virtualizable.vable_token
+        self.gettoken = gettoken
+
+        def settoken(virtualizable, token):
+            virtualizable = cast_gcref_to_vtype(virtualizable)
+            virtualizable.vable_token = token
+        self.settoken = settoken
+
     def _freeze_(self):
         return True
 
@@ -216,55 +288,11 @@
             all_graphs, self.VTYPEPTR, funcptr)
 
     def unwrap_virtualizable_box(self, virtualizable_box):
-        return virtualizable_box.getref(self.VTYPEPTR)
-
-    def cast_to_vtype(self, virtualizable):
-        return self.cpu.ts.cast_to_instance_maybe(self.VTYPEPTR, virtualizable)
-    cast_to_vtype._annspecialcase_ = 'specialize:ll'
-
-    def cast_gcref_to_vtype(self, virtualizable):
-        return lltype.cast_opaque_ptr(self.VTYPEPTR, virtualizable)
-
+        return virtualizable_box.getref(llmemory.GCREF)
+     
     def is_vtypeptr(self, TYPE):
         return rvirtualizable2.match_virtualizable_type(TYPE, self.VTYPEPTR)
 
-    def reset_vable_token(self, virtualizable):
-        virtualizable.vable_token = self.TOKEN_NONE
-
-    def clear_vable_token(self, virtualizable):
-        if virtualizable.vable_token:
-            self.force_now(virtualizable)
-            assert not virtualizable.vable_token
-
-    def tracing_before_residual_call(self, virtualizable):
-        assert not virtualizable.vable_token
-        virtualizable.vable_token = self.TOKEN_TRACING_RESCALL
-
-    def tracing_after_residual_call(self, virtualizable):
-        if virtualizable.vable_token:
-            # not modified by the residual call; assert that it is still
-            # set to TOKEN_TRACING_RESCALL and clear it.
-            assert virtualizable.vable_token == self.TOKEN_TRACING_RESCALL
-            virtualizable.vable_token = self.TOKEN_NONE
-            return False
-        else:
-            # marker "modified during residual call" set.
-            return True
-
-    def force_now(self, virtualizable):
-        token = virtualizable.vable_token
-        if token == self.TOKEN_TRACING_RESCALL:
-            # The values in the virtualizable are always correct during
-            # tracing.  We only need to reset vable_token to TOKEN_NONE
-            # as a marker for the tracing, to tell it that this
-            # virtualizable escapes.
-            virtualizable.vable_token = self.TOKEN_NONE
-        else:
-            from pypy.jit.metainterp.compile import ResumeGuardForcedDescr
-            ResumeGuardForcedDescr.force_now(self.cpu, token)
-            assert virtualizable.vable_token == self.TOKEN_NONE
-    force_now._dont_inline_ = True
-
 # ____________________________________________________________
 #
 # The 'vable_token' field of a virtualizable is either 0, -1, or points

From commits-noreply at bitbucket.org  Sun Feb 13 16:25:05 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Sun, 13 Feb 2011 16:25:05 +0100 (CET)
Subject: [pypy-svn] pypy default: merge heads
Message-ID: <20110213152505.D25FF2A202D@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: 
Changeset: r41886:1ade308774ba
Date: 2011-02-13 17:24 +0200
http://bitbucket.org/pypy/pypy/changeset/1ade308774ba/

Log:	merge heads

diff --git a/pypy/module/readline/c_readline.py b/pypy/module/readline/c_readline.py
deleted file mode 100644
--- a/pypy/module/readline/c_readline.py
+++ /dev/null
@@ -1,77 +0,0 @@
-from pypy.rpython.tool import rffi_platform as platform
-from pypy.rpython.lltypesystem import lltype, rffi
-from pypy.interpreter.error import OperationError
-from pypy.interpreter.gateway import ObjSpace, interp2app
-from pypy.translator.tool.cbuild import ExternalCompilationInfo
-
-# On various platforms, linking only with libreadline is not enough;
-# we also need to link with some variant of curses or libtermcap.
-# We follow the logic of CPython below.
-def try_with_lib(extralibs, **kwds):
-    global most_recent_error
-    # at least on Gentoo Linux, readline.h doesn't compile if stdio.h is not
-    # included before
-    eci = ExternalCompilationInfo(
-        includes = ["stdio.h", "readline/readline.h", "readline/history.h"],
-        libraries = extralibs + ['readline'],
-        )
-    try:
-        platform.verify_eci(eci)
-        return eci
-    except platform.CompilationError, e:
-        most_recent_error = e
-        return None
-
-eci = (try_with_lib([]) or
-       try_with_lib(['ncursesw']) or
-       try_with_lib(['ncurses']) or
-       try_with_lib(['curses']) or
-       try_with_lib(['termcap'], library_dirs=['/usr/lib/termcap']))
-if eci is None:
-    raise most_recent_error
-
-# ____________________________________________________________
-
-def external(name, args, result):
-    return rffi.llexternal(name, args, result, compilation_info=eci)
-
-# get a binding to  c library functions and define their args and return types
-# char *readline(char *)
-c_readline = external('readline', [rffi.CCHARP], rffi.CCHARP)
-
-# void rl_initiliaze(void)
-c_rl_initialize = external('rl_initialize', [], lltype.Void)
-
-# void using_history(void)
-c_using_history = external('using_history', [], lltype.Void)
-
-# void add_history(const char *)
-c_add_history = external('add_history', [rffi.CCHARP], lltype.Void)
-
-#------------------------------------------------------------
-# special initialization of readline 
-
-class ReadlineState(object):
-    lastline = ""        # XXX possibly temporary hack
-readlinestate = ReadlineState()
-
-def setup_readline(space, w_module):
-    c_using_history()
-    # XXX CPython initializes more stuff here
-    c_rl_initialize()
-    # install sys.__raw_input__, a hook that will be used by raw_input()
-    space.setitem(space.sys.w_dict, space.wrap('__raw_input__'),
-                  space.wrap(app_readline_func))
-
-def readline_func(space, prompt):
-    ll_res = c_readline(prompt)
-    if not ll_res:
-        raise OperationError(space.w_EOFError, space.w_None)
-    res = rffi.charp2str(ll_res)
-    if res and res != readlinestate.lastline:
-        readlinestate.lastline = res
-        c_add_history(res)
-    return space.wrap(res)
-
-readline_func.unwrap_spec = [ObjSpace, str]
-app_readline_func = interp2app(readline_func)

diff --git a/pypy/module/readline/test/test_c_readline.py b/pypy/module/readline/test/test_c_readline.py
deleted file mode 100644
--- a/pypy/module/readline/test/test_c_readline.py
+++ /dev/null
@@ -1,16 +0,0 @@
-"""
-Directly test the basic ctypes wrappers.
-"""
-
-import py
-from pypy import conftest; conftest.translation_test_so_skip_if_appdirect()
-from pypy.rpython.tool import rffi_platform as platform
-
-try:
-    from pypy.module.readline import c_readline
-except platform.CompilationError, e:
-    py.test.skip(e)
-
-
-def test_basic_import():
-    c_readline.c_rl_initialize()

diff --git a/pypy/module/readline/app_stub.py b/pypy/module/readline/app_stub.py
deleted file mode 100644
--- a/pypy/module/readline/app_stub.py
+++ /dev/null
@@ -1,13 +0,0 @@
-# NOT_RPYTHON
-
-def stub(*args, **kwds):
-    import warnings
-    warnings.warn("the 'readline' module is only a stub so far")
-
-def stub_str(*args, **kwds):
-    stub()
-    return ''
-
-def stub_int(*args, **kwds):
-    stub()
-    return 0

diff --git a/pypy/jit/metainterp/pyjitpl.py b/pypy/jit/metainterp/pyjitpl.py
--- a/pypy/jit/metainterp/pyjitpl.py
+++ b/pypy/jit/metainterp/pyjitpl.py
@@ -20,7 +20,7 @@
 from pypy.rlib.rarithmetic import intmask
 from pypy.rlib.objectmodel import specialize
 from pypy.jit.codewriter.jitcode import JitCode, SwitchDictDescr, MissingLiveness
-from pypy.jit.codewriter import heaptracker
+from pypy.jit.codewriter import heaptracker, longlong
 from pypy.jit.metainterp.optimizeutil import RetraceLoop
 
 # ____________________________________________________________
@@ -1473,7 +1473,7 @@
             elif result_type == history.REF:
                 raise sd.DoneWithThisFrameRef(self.cpu, resultbox.getref_base())
             elif result_type == history.FLOAT:
-                raise sd.DoneWithThisFrameFloat(resultbox.getfloat())
+                raise sd.DoneWithThisFrameFloat(resultbox.getfloatstorage())
             else:
                 assert False
 

diff --git a/pypy/doc/config/objspace.usemodules.readline.txt b/pypy/doc/config/objspace.usemodules.readline.txt
deleted file mode 100644
--- a/pypy/doc/config/objspace.usemodules.readline.txt
+++ /dev/null
@@ -1,1 +0,0 @@
-Use the 'readline' module. 

diff --git a/pypy/jit/metainterp/resume.py b/pypy/jit/metainterp/resume.py
--- a/pypy/jit/metainterp/resume.py
+++ b/pypy/jit/metainterp/resume.py
@@ -1157,7 +1157,7 @@
     def decode_float(self, tagged):
         num, tag = untag(tagged)
         if tag == TAGCONST:
-            return self.consts[num].getfloat()
+            return self.consts[num].getfloatstorage()
         else:
             assert tag == TAGBOX
             if num < 0:

diff --git a/pypy/module/readline/test/__init__.py b/pypy/module/readline/test/__init__.py
deleted file mode 100644
--- a/pypy/module/readline/test/__init__.py
+++ /dev/null
@@ -1,1 +0,0 @@
-#

diff --git a/pypy/module/readline/test/test_with_pypy.py b/pypy/module/readline/test/test_with_pypy.py
deleted file mode 100644
--- a/pypy/module/readline/test/test_with_pypy.py
+++ /dev/null
@@ -1,27 +0,0 @@
-"""
-Test the readline library on top of PyPy.  The following tests run
-in the PyPy interpreter, itself running on top of CPython
-"""
-
-import py
-from pypy.conftest import gettestobjspace
-from pypy.rpython.tool import rffi_platform as platform
-
-try:
-    from pypy.module.readline import c_readline
-except platform.CompilationError, e:
-    py.test.skip(e)
-
-
-class AppTestReadline:
-
-    def setup_class(cls):
-        # enable usage of the readline mixedmodule
-        space = gettestobjspace(usemodules=('readline',))
-        cls.space = space
-
-    def test_basic_import(self):
-        # this is interpreted by PyPy
-        import readline 
-        readline.readline
-        # XXX test more

diff --git a/pypy/module/readline/interp_readline.py b/pypy/module/readline/interp_readline.py
deleted file mode 100644
--- a/pypy/module/readline/interp_readline.py
+++ /dev/null
@@ -1,23 +0,0 @@
-# this is a sketch of how one might one day be able to define a pretty simple
-# ctypes-using module, suitable for feeding to the ext-compiler
-
-from pypy.interpreter.baseobjspace import ObjSpace
-
-from pypy.module.readline import c_readline
-from pypy.rpython.lltypesystem import rffi
-
-#------------------------------------------------------------
-# exported API  (see interpleveldefs in __init__.py) 
-#
-def readline(space, prompt):
-    return space.wrap(rffi.charp2str(c_readline.c_readline(prompt)))
-readline.unwrap_spec = [ObjSpace, str]
-
-def setcompleter(space, w_callback):
-    """Set or remove the completer function.
-    The function is called as function(text, state),
-    for state in 0, 1, 2, ..., until it returns a non-string.
-    It should return the next possible completion starting with 'text'.
-    """ 
-    # XXX set internal completion function 
-    

diff --git a/pypy/module/readline/__init__.py b/pypy/module/readline/__init__.py
deleted file mode 100644
--- a/pypy/module/readline/__init__.py
+++ /dev/null
@@ -1,45 +0,0 @@
-# this is a sketch of how one might one day be able to define a pretty simple
-# ctypes-using module, suitable for feeding to the ext-compiler
-
-from pypy.interpreter.mixedmodule import MixedModule
-
-# XXX raw_input needs to check for space.readline_func and use
-# it if its there 
-
-class Module(MixedModule):
-    """Importing this module enables command line editing using GNU readline."""
-    # the above line is the doc string of the translated module  
-
-    def setup_after_space_initialization(self):
-        from pypy.module.readline import c_readline 
-        c_readline.setup_readline(self.space, self)
-
-    interpleveldefs = {
-        'readline'    : 'interp_readline.readline',
-    }
-
-    appleveldefs = {
-        'parse_and_bind':     'app_stub.stub',
-        'get_line_buffer':    'app_stub.stub_str',
-        'insert_text':        'app_stub.stub',
-        'read_init_file':     'app_stub.stub',
-        'read_history_file':  'app_stub.stub',
-        'write_history_file': 'app_stub.stub',
-        'clear_history':      'app_stub.stub',
-        'get_history_length': 'app_stub.stub_int',
-        'set_history_length': 'app_stub.stub',
-        'get_current_history_length': 'app_stub.stub_int',
-        'get_history_item':           'app_stub.stub_str',
-        'remove_history_item':        'app_stub.stub',
-        'replace_history_item':       'app_stub.stub',
-        'redisplay':                  'app_stub.stub',
-        'set_startup_hook':           'app_stub.stub',
-        'set_pre_input_hook':         'app_stub.stub',
-        'set_completer':      'app_stub.stub',
-        'get_completer':      'app_stub.stub',
-        'get_begidx':         'app_stub.stub_int',
-        'get_endidx':         'app_stub.stub_int',
-        'set_completer_delims':       'app_stub.stub',
-        'get_completer_delims':       'app_stub.stub_str',
-        'add_history':        'app_stub.stub',
-    }

From commits-noreply at bitbucket.org  Sun Feb 13 19:51:23 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Sun, 13 Feb 2011 19:51:23 +0100 (CET)
Subject: [pypy-svn] pypy default: Fixup sqlite3.Connection.executescript
Message-ID: <20110213185123.C8325282B8B@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41887:2a98310a8a24
Date: 2011-02-13 13:51 -0500
http://bitbucket.org/pypy/pypy/changeset/2a98310a8a24/

Log:	Fixup sqlite3.Connection.executescript

diff --git a/lib_pypy/_sqlite3.py b/lib_pypy/_sqlite3.py
--- a/lib_pypy/_sqlite3.py
+++ b/lib_pypy/_sqlite3.py
@@ -704,23 +704,33 @@
             sql = sql.encode("utf-8")
         self._check_closed()
         statement = c_void_p()
-        next_char = c_char_p(sql)
+        c_sql = c_char_p(sql)
 
+        self.connection.commit()
         while True:
-            if not sqlite.sqlite3_complete(next_char):
-                raise OperationalError, "Incomplete statement '%s'" % next_char.value
-            ret = sqlite.sqlite3_prepare_v2(self.connection.db, next_char, -1, byref(statement), byref(next_char))
-            if ret != SQLITE_OK:
-                raise self.connection._get_exception(ret)
-            if statement.value is None:
+            rc = sqlite.sqlite3_prepare(self.connection.db, c_sql, -1, byref(statement), byref(c_sql))
+            if rc != SQLITE_OK:
+                raise self.connection._get_exception(rc)
+
+            rc = SQLITE_ROW
+            while rc == SQLITE_ROW:
+                if not statement:
+                    rc = SQLITE_OK
+                else:
+                    rc = sqlite.sqlite3_step(statement)
+
+            if rc != SQLITE_DONE:
+                sqlite.sqlite3_finalize(statement)
+                if rc == SQLITE_OK:
+                    return self
+                else:
+                    raise self.connection._get_exception(rc)
+            rc = sqlite.sqlite3_finalize(statement)
+            if rc != SQLITE_OK:
+                raise self.connection._get_exception(rc)
+
+            if not c_sql.value:
                 break
-            ret = sqlite.sqlite3_step(statement)
-            sqlite.sqlite3_finalize(statement)
-            if ret not in (SQLITE_OK, SQLITE_DONE, SQLITE_ROW):
-                raise self.connection._get_exception(ret)
-            if next_char.value is None:
-                break
-
         return self
 
     def __iter__(self):

From commits-noreply at bitbucket.org  Mon Feb 14 08:20:12 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Mon, 14 Feb 2011 08:20:12 +0100 (CET)
Subject: [pypy-svn] pypy default: Remove this file.
Message-ID: <20110214072012.1A9A2282B8B@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41888:0cce3622d1af
Date: 2011-02-14 02:19 -0500
http://bitbucket.org/pypy/pypy/changeset/0cce3622d1af/

Log:	Remove this file.

diff --git a/pypy/objspace/std/test/helper.py b/pypy/objspace/std/test/helper.py
deleted file mode 100644
--- a/pypy/objspace/std/test/helper.py
+++ /dev/null
@@ -1,69 +0,0 @@
-def raises(excp, func, *args):
-    try:
-        func(*args)
-        assert 1 == 0
-    except excp:pass
-
-def assertEqual(a, b):
-    assert a == b
-
-def assertNotEqual(a, b):
-    assert a != b
-
-def assertIs(a, b):
-    assert a is b
-
-# complex specific tests
-
-EPS = 1e-9
-
-def assertAlmostEqual(a, b):
-    if isinstance(a, complex):
-        if isinstance(b, complex):
-            assert a.real - b.real < EPS
-            assert a.imag - b.imag < EPS
-        else:
-            assert a.real - b < EPS
-            assert a.imag < EPS
-    else:
-        if isinstance(b, complex):
-            assert a - b.real < EPS
-            assert b.imag < EPS
-        else:
-            assert a - b < EPS
-
-def assertCloseAbs(x, y, eps=1e-9):
-    """Return true iff floats x and y "are close\""""
-    # put the one with larger magnitude second
-    if abs(x) > abs(y):
-        x, y = y, x
-    if y == 0:
-        return abs(x) < eps
-    if x == 0:
-        return abs(y) < eps
-    # check that relative difference < eps
-    assert abs((x-y)/y) < eps
-
-def assertClose(x, y, eps=1e-9):
-    """Return true iff complexes x and y "are close\""""
-    assertCloseAbs(x.real, y.real, eps)
-    assertCloseAbs(x.imag, y.imag, eps)
-
-
-def check_div(x, y):
-    """Compute complex z=x*y, and check that z/x==y and z/y==x."""
-    z = x * y
-    if x != 0:
-        q = z / x
-        assertClose(q, y)
-        q = z.__div__(x)
-        assertClose(q, y)
-        q = z.__truediv__(x)
-        assertClose(q, y)
-    if y != 0:
-        q = z / y
-        assertClose(q, x)
-        q = z.__div__(y)
-        assertClose(q, x)
-        q = z.__truediv__(y)
-        assertClose(q, x)

From commits-noreply at bitbucket.org  Mon Feb 14 08:45:24 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Mon, 14 Feb 2011 08:45:24 +0100 (CET)
Subject: [pypy-svn] pypy default: Skip more utf-7 tests when run with
	python2.6
Message-ID: <20110214074524.761A5282B8B@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41889:993a9960ce24
Date: 2011-02-14 08:44 +0100
http://bitbucket.org/pypy/pypy/changeset/993a9960ce24/

Log:	Skip more utf-7 tests when run with python2.6

diff --git a/pypy/rlib/test/test_runicode.py b/pypy/rlib/test/test_runicode.py
--- a/pypy/rlib/test/test_runicode.py
+++ b/pypy/rlib/test/test_runicode.py
@@ -115,13 +115,17 @@
             if 0xd800 <= v <= 0xdfff:
                 continue
             uni = unichr(v)
-            for encoding in ("utf-7 utf-8 utf-16 utf-16-be utf-16-le "
+            if sys.version >= "2.7":
+                self.checkdecode(uni, "utf-7")
+            for encoding in ("utf-8 utf-16 utf-16-be utf-16-le "
                              "utf-32 utf-32-be utf-32-le").split():
                 self.checkdecode(uni, encoding)
 
     def test_maxunicode(self):
         uni = unichr(sys.maxunicode)
-        for encoding in ("utf-7 utf-8 utf-16 utf-16-be utf-16-le "
+        if sys.version >= "2.7":
+            self.checkdecode(uni, "utf-7")
+        for encoding in ("utf-8 utf-16 utf-16-be utf-16-le "
                          "utf-32 utf-32-be utf-32-le").split():
             self.checkdecode(uni, encoding)
 
@@ -345,7 +349,9 @@
 
     def test_maxunicode(self):
         uni = unichr(sys.maxunicode)
-        for encoding in ("utf-7 utf-8 utf-16 utf-16-be utf-16-le "
+        if sys.version >= "2.7":
+            self.checkencode(uni, "utf-7")
+        for encoding in ("utf-8 utf-16 utf-16-be utf-16-le "
                          "utf-32 utf-32-be utf-32-le").split():
             self.checkencode(uni, encoding)        
 

From commits-noreply at bitbucket.org  Mon Feb 14 08:45:25 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Mon, 14 Feb 2011 08:45:25 +0100 (CET)
Subject: [pypy-svn] pypy default: merge heads
Message-ID: <20110214074525.08C35282B8B@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41890:f881d31c4c50
Date: 2011-02-14 08:45 +0100
http://bitbucket.org/pypy/pypy/changeset/f881d31c4c50/

Log:	merge heads

diff --git a/pypy/objspace/std/test/helper.py b/pypy/objspace/std/test/helper.py
deleted file mode 100644
--- a/pypy/objspace/std/test/helper.py
+++ /dev/null
@@ -1,69 +0,0 @@
-def raises(excp, func, *args):
-    try:
-        func(*args)
-        assert 1 == 0
-    except excp:pass
-
-def assertEqual(a, b):
-    assert a == b
-
-def assertNotEqual(a, b):
-    assert a != b
-
-def assertIs(a, b):
-    assert a is b
-
-# complex specific tests
-
-EPS = 1e-9
-
-def assertAlmostEqual(a, b):
-    if isinstance(a, complex):
-        if isinstance(b, complex):
-            assert a.real - b.real < EPS
-            assert a.imag - b.imag < EPS
-        else:
-            assert a.real - b < EPS
-            assert a.imag < EPS
-    else:
-        if isinstance(b, complex):
-            assert a - b.real < EPS
-            assert b.imag < EPS
-        else:
-            assert a - b < EPS
-
-def assertCloseAbs(x, y, eps=1e-9):
-    """Return true iff floats x and y "are close\""""
-    # put the one with larger magnitude second
-    if abs(x) > abs(y):
-        x, y = y, x
-    if y == 0:
-        return abs(x) < eps
-    if x == 0:
-        return abs(y) < eps
-    # check that relative difference < eps
-    assert abs((x-y)/y) < eps
-
-def assertClose(x, y, eps=1e-9):
-    """Return true iff complexes x and y "are close\""""
-    assertCloseAbs(x.real, y.real, eps)
-    assertCloseAbs(x.imag, y.imag, eps)
-
-
-def check_div(x, y):
-    """Compute complex z=x*y, and check that z/x==y and z/y==x."""
-    z = x * y
-    if x != 0:
-        q = z / x
-        assertClose(q, y)
-        q = z.__div__(x)
-        assertClose(q, y)
-        q = z.__truediv__(x)
-        assertClose(q, y)
-    if y != 0:
-        q = z / y
-        assertClose(q, x)
-        q = z.__div__(y)
-        assertClose(q, x)
-        q = z.__truediv__(y)
-        assertClose(q, x)

From commits-noreply at bitbucket.org  Mon Feb 14 10:51:22 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Mon, 14 Feb 2011 10:51:22 +0100 (CET)
Subject: [pypy-svn] pypy default: Reindent test code,
	it will also give better messages on failure
Message-ID: <20110214095122.4BBAA282BAD@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41891:77c809396311
Date: 2011-02-14 10:45 +0100
http://bitbucket.org/pypy/pypy/changeset/77c809396311/

Log:	Reindent test code, it will also give better messages on failure

diff --git a/pypy/module/_codecs/test/test_codecs.py b/pypy/module/_codecs/test/test_codecs.py
--- a/pypy/module/_codecs/test/test_codecs.py
+++ b/pypy/module/_codecs/test/test_codecs.py
@@ -408,18 +408,13 @@
 
     def test_errors(self):
         import codecs
-        assert (
-            codecs.replace_errors(UnicodeEncodeError("ascii", u"\u3042", 0, 1, "ouch"))) == (
-            (u"?", 1)
-        )
-        assert (
-            codecs.replace_errors(UnicodeDecodeError("ascii", "\xff", 0, 1, "ouch"))) == (
-            (u"\ufffd", 1)
-        )
-        assert (
-            codecs.replace_errors(UnicodeTranslateError(u"\u3042", 0, 1, "ouch"))) == (
-            (u"\ufffd", 1)
-        )
+        assert codecs.replace_errors(UnicodeEncodeError(
+            "ascii", u"\u3042", 0, 1, "ouch")) == (u"?", 1)
+        assert codecs.replace_errors(UnicodeDecodeError(
+            "ascii", "\xff", 0, 1, "ouch")) == (u"\ufffd", 1)
+        assert codecs.replace_errors(UnicodeTranslateError(
+            u"\u3042", 0, 1, "ouch")) == (u"\ufffd", 1)
+
         class BadStartUnicodeEncodeError(UnicodeEncodeError):
             def __init__(self):
                 UnicodeEncodeError.__init__(self, "ascii", u"", 0, 1, "bad")

From commits-noreply at bitbucket.org  Mon Feb 14 10:51:23 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Mon, 14 Feb 2011 10:51:23 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix the "replace" error handler: on
	UnicodeDecodeError, only one U+FFFD is returned, 
Message-ID: <20110214095123.5B32D282BAD@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41892:ed6b9cdcc22f
Date: 2011-02-14 10:50 +0100
http://bitbucket.org/pypy/pypy/changeset/ed6b9cdcc22f/

Log:	Fix the "replace" error handler: on UnicodeDecodeError, only one
	U+FFFD is returned, even when the error spans multiple bytes.

diff --git a/pypy/module/_codecs/test/test_codecs.py b/pypy/module/_codecs/test/test_codecs.py
--- a/pypy/module/_codecs/test/test_codecs.py
+++ b/pypy/module/_codecs/test/test_codecs.py
@@ -415,6 +415,13 @@
         assert codecs.replace_errors(UnicodeTranslateError(
             u"\u3042", 0, 1, "ouch")) == (u"\ufffd", 1)
 
+        assert codecs.replace_errors(UnicodeEncodeError(
+            "ascii", u"\u3042\u3042", 0, 2, "ouch")) == (u"??", 2)
+        assert codecs.replace_errors(UnicodeDecodeError(
+            "ascii", "\xff\xff", 0, 2, "ouch")) == (u"\ufffd", 2)
+        assert codecs.replace_errors(UnicodeTranslateError(
+            u"\u3042\u3042", 0, 2, "ouch")) == (u"\ufffd\ufffd", 2)
+
         class BadStartUnicodeEncodeError(UnicodeEncodeError):
             def __init__(self):
                 UnicodeEncodeError.__init__(self, "ascii", u"", 0, 1, "bad")

diff --git a/pypy/objspace/std/test/test_unicodeobject.py b/pypy/objspace/std/test/test_unicodeobject.py
--- a/pypy/objspace/std/test/test_unicodeobject.py
+++ b/pypy/objspace/std/test/test_unicodeobject.py
@@ -525,6 +525,7 @@
         raises(UnicodeError, "\\".decode, "unicode-escape")
 
         raises(UnicodeError, "\xc2".decode, "utf-8")
+        assert '\xe1\x80'.decode('utf-8', 'replace') == u"\ufffd"
 
     def test_repr_bug(self):
         assert (repr(u'\U00090418\u027d\U000582b9\u54c3\U000fcb6e') == 

diff --git a/pypy/module/_codecs/interp_codecs.py b/pypy/module/_codecs/interp_codecs.py
--- a/pypy/module/_codecs/interp_codecs.py
+++ b/pypy/module/_codecs/interp_codecs.py
@@ -181,8 +181,10 @@
     if space.isinstance_w(w_exc, space.w_UnicodeEncodeError):
         text = '?' * size
         return space.newtuple([space.wrap(text), w_end])
-    elif (space.isinstance_w(w_exc, space.w_UnicodeDecodeError) or
-          space.isinstance_w(w_exc, space.w_UnicodeTranslateError)):
+    elif space.isinstance_w(w_exc, space.w_UnicodeDecodeError):
+        text = u'\ufffd'
+        return space.newtuple([space.wrap(text), w_end])
+    elif space.isinstance_w(w_exc, space.w_UnicodeTranslateError):
         text = u'\ufffd' * size
         return space.newtuple([space.wrap(text), w_end])
     else:

From commits-noreply at bitbucket.org  Mon Feb 14 11:48:10 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon, 14 Feb 2011 11:48:10 +0100 (CET)
Subject: [pypy-svn] pypy default: Support "pypy filename.pyc".
Message-ID: <20110214104810.25AF3282BEB@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41893:723cc5563f33
Date: 2011-02-13 18:30 +0100
http://bitbucket.org/pypy/pypy/changeset/723cc5563f33/

Log:	Support "pypy filename.pyc".

diff --git a/pypy/module/imp/interp_imp.py b/pypy/module/imp/interp_imp.py
--- a/pypy/module/imp/interp_imp.py
+++ b/pypy/module/imp/interp_imp.py
@@ -3,7 +3,7 @@
 from pypy.rlib import streamio
 from pypy.interpreter.error import OperationError, operationerrfmt
 from pypy.interpreter.module import Module
-from pypy.interpreter.gateway import NoneNotWrapped
+from pypy.interpreter.gateway import NoneNotWrapped, W_Root, ObjSpace
 import struct
 
 def get_suffixes(space):
@@ -98,23 +98,26 @@
         stream.close()
     return w_mod
 
-def load_compiled(space, w_modulename, w_filename, w_file=None):
-    filename = space.str_w(w_filename)
-
+def _run_compiled_module(space, w_modulename, filename, w_file, w_module):
+    # the function 'imp._run_compiled_module' is a pypy-only extension
     stream = get_file(space, w_file, filename, 'rb')
 
-    w_mod = space.wrap(Module(space, w_modulename))
-    importing._prepare_module(space, w_mod, filename, None)
-
     magic = importing._r_long(stream)
     timestamp = importing._r_long(stream)
 
     importing.load_compiled_module(
-        space, w_modulename, w_mod, filename, magic, timestamp,
+        space, w_modulename, w_module, filename, magic, timestamp,
         stream.readall())
     if space.is_w(w_file, space.w_None):
         stream.close()
+_run_compiled_module.unwrap_spec = [ObjSpace, W_Root, str, W_Root, W_Root]
+
+def load_compiled(space, w_modulename, filename, w_file=None):
+    w_mod = space.wrap(Module(space, w_modulename))
+    importing._prepare_module(space, w_mod, filename, None)
+    _run_compiled_module(space, w_modulename, filename, w_file, w_mod)
     return w_mod
+load_compiled.unwrap_spec = [ObjSpace, W_Root, str, W_Root]
 
 def new_module(space, w_name):
     return space.wrap(Module(space, w_name))

diff --git a/pypy/translator/goal/app_main.py b/pypy/translator/goal/app_main.py
--- a/pypy/translator/goal/app_main.py
+++ b/pypy/translator/goal/app_main.py
@@ -550,10 +550,21 @@
         else:
             # handle the common case where a filename is specified
             # on the command-line.
-            mainmodule.__file__ = sys.argv[0]
-            scriptdir = resolvedirof(sys.argv[0])
-            sys.path.insert(0, scriptdir)
-            success = run_toplevel(execfile, sys.argv[0], mainmodule.__dict__)
+            filename = sys.argv[0]
+            mainmodule.__file__ = filename
+            sys.path.insert(0, resolvedirof(filename))
+            # assume it's a pyc file only if its name says so.
+            # CPython goes to great lengths to detect other cases
+            # of pyc file format, but I think it's ok not to care.
+            if IS_WINDOWS:
+                filename = filename.lower()
+            if filename.endswith('.pyc') or filename.endswith('.pyo'):
+                import imp
+                success = run_toplevel(imp._run_compiled_module, '__main__',
+                                       sys.argv[0], None, mainmodule)
+            else:
+                success = run_toplevel(execfile, sys.argv[0],
+                                       mainmodule.__dict__)
 
     except SystemExit, e:
         status = e.code
@@ -623,6 +634,19 @@
         except OSError:
             return None
 
+    # add an emulator for this pypy-only function
+    # (for test_pyc_commandline_argument)
+    import imp
+    def _run_compiled_module(modulename, filename, file, module):
+        import os
+        assert modulename == '__main__'
+        assert os.path.isfile(filename)
+        assert filename.endswith('.pyc')
+        assert file is None
+        assert module.__name__ == '__main__'
+        print 'in _run_compiled_module'
+    imp._run_compiled_module = _run_compiled_module
+
     # stick the current sys.path into $PYTHONPATH, so that CPython still
     # finds its own extension modules :-/
     import os

diff --git a/pypy/translator/goal/test2/test_app_main.py b/pypy/translator/goal/test2/test_app_main.py
--- a/pypy/translator/goal/test2/test_app_main.py
+++ b/pypy/translator/goal/test2/test_app_main.py
@@ -14,14 +14,36 @@
 app_main = os.path.abspath(app_main)
 
 _counter = 0
-def getscript(source):
+def _get_next_path():
     global _counter
     p = udir.join('demo_test_app_main_%d.py' % (_counter,))
     _counter += 1
+    return p
+
+def getscript(source):
+    p = _get_next_path()
     p.write(str(py.code.Source(source)))
     # return relative path for testing purposes 
     return py.path.local().bestrelpath(p) 
 
+def getscriptpyc(space, source):
+    p = _get_next_path()
+    p.write(str(py.code.Source(source)))
+    w_dir = space.wrap(str(p.dirpath()))
+    w_modname = space.wrap(p.purebasename)
+    space.appexec([w_dir, w_modname], """(dir, modname):
+        import sys
+        d = sys.modules.copy()
+        sys.path.insert(0, dir)
+        __import__(modname)
+        sys.path.pop(0)
+        for key in sys.modules.keys():
+            if key not in d:
+                del sys.modules[key]
+    """)
+    p = str(p) + 'c'
+    assert os.path.isfile(p)   # the .pyc file should have been created above
+    return p
 
 demo_script = getscript("""
     print 'hello'
@@ -671,6 +693,12 @@
         data = self.run('-c "import sys; print sys.path"')
         assert data.startswith("[''")
 
+    def test_pyc_commandline_argument(self):
+        p = getscriptpyc(self.space, "print 6*7\n")
+        assert p.endswith('.pyc')
+        data = self.run(p)
+        assert data == 'in _run_compiled_module\n'
+
 
 class AppTestAppMain:
 

diff --git a/pypy/module/imp/__init__.py b/pypy/module/imp/__init__.py
--- a/pypy/module/imp/__init__.py
+++ b/pypy/module/imp/__init__.py
@@ -19,6 +19,7 @@
         'load_module':     'interp_imp.load_module',
         'load_source':     'interp_imp.load_source',
         'load_compiled':   'interp_imp.load_compiled',
+        '_run_compiled_module': 'interp_imp._run_compiled_module',
         #'run_module':      'interp_imp.run_module',
         'new_module':      'interp_imp.new_module',
         'init_builtin':    'interp_imp.init_builtin',

From commits-noreply at bitbucket.org  Mon Feb 14 11:58:10 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon, 14 Feb 2011 11:58:10 +0100 (CET)
Subject: [pypy-svn] pypy default: Don't use always port 50007 (risk of
	collision when	running twice).
Message-ID: <20110214105810.0B3ED2A2002@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41894:e159fe203839
Date: 2011-02-14 11:57 +0100
http://bitbucket.org/pypy/pypy/changeset/e159fe203839/

Log:	Don't use always port 50007 (risk of collision when running twice).
	Instead, use a random port number.

diff --git a/pypy/module/_socket/test/test_sock_app.py b/pypy/module/_socket/test/test_sock_app.py
--- a/pypy/module/_socket/test/test_sock_app.py
+++ b/pypy/module/_socket/test/test_sock_app.py
@@ -1,10 +1,12 @@
 from pypy.conftest import gettestobjspace
-import sys
+import sys, random
 import py
 from pypy.tool.udir import udir
 from pypy.rlib import rsocket
 from pypy.rpython.lltypesystem import lltype, rffi
 
+PORT_NUMBER = random.randrange(40000, 60000)
+
 def setup_module(mod):
     mod.space = gettestobjspace(usemodules=['_socket', 'array'])
     global socket
@@ -297,6 +299,7 @@
     def setup_class(cls):
         cls.space = space
         cls.w_udir = space.wrap(str(udir))
+        cls.w_PORT = space.wrap(PORT_NUMBER)
 
     def test_ntoa_exception(self):
         import _socket
@@ -498,7 +501,7 @@
             skip('No dup() on this platform')
         s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
         s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
-        s.bind(('localhost', 50007))
+        s.bind(('localhost', self.PORT))
         s2 = s.dup()
         assert s.fileno() != s2.fileno()
         assert s.getsockname() == s2.getsockname()
@@ -554,7 +557,7 @@
     def setup_class(cls):
         cls.space = space
 
-    PORT = 50007
+    PORT = PORT_NUMBER
     HOST = 'localhost'
         
     def setup_method(self, method):

From commits-noreply at bitbucket.org  Mon Feb 14 13:31:45 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Mon, 14 Feb 2011 13:31:45 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix stupid typo introduced in b8f612ad32e1
Message-ID: <20110214123145.E031B282B90@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41895:43da6eb6a4e0
Date: 2011-02-14 13:30 +0100
http://bitbucket.org/pypy/pypy/changeset/43da6eb6a4e0/

Log:	Fix stupid typo introduced in b8f612ad32e1

diff --git a/pypy/rlib/_rsocket_rffi.py b/pypy/rlib/_rsocket_rffi.py
--- a/pypy/rlib/_rsocket_rffi.py
+++ b/pypy/rlib/_rsocket_rffi.py
@@ -425,7 +425,7 @@
 FIONBIO = cConfig.FIONBIO
 INET_ADDRSTRLEN = cConfig.INET_ADDRSTRLEN
 INET6_ADDRSTRLEN = cConfig.INET6_ADDRSTRLEN
-EINTR = cConfig.EINPROGRESS or cConfig.WSAEINTR
+EINTR = cConfig.EINTR or cConfig.WSAEINTR
 EINPROGRESS = cConfig.EINPROGRESS or cConfig.WSAEINPROGRESS
 EWOULDBLOCK = cConfig.EWOULDBLOCK or cConfig.WSAEWOULDBLOCK
 EAFNOSUPPORT = cConfig.EAFNOSUPPORT or cConfig.WSAEAFNOSUPPORT

From commits-noreply at bitbucket.org  Mon Feb 14 13:31:46 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Mon, 14 Feb 2011 13:31:46 +0100 (CET)
Subject: [pypy-svn] pypy default: Merge heads
Message-ID: <20110214123146.25111282BAD@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41896:074e39de8588
Date: 2011-02-14 13:30 +0100
http://bitbucket.org/pypy/pypy/changeset/074e39de8588/

Log:	Merge heads


From commits-noreply at bitbucket.org  Mon Feb 14 13:55:54 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon, 14 Feb 2011 13:55:54 +0100 (CET)
Subject: [pypy-svn] pypy default: Remove 'separate_module_sources',
	which seems to be useless because
Message-ID: <20110214125554.60193282B90@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41897:5951571530f3
Date: 2011-02-14 13:54 +0100
http://bitbucket.org/pypy/pypy/changeset/5951571530f3/

Log:	Remove 'separate_module_sources', which seems to be useless because
	compiling a .c file with only these #include has no effect.

diff --git a/pypy/rlib/rdtoa.py b/pypy/rlib/rdtoa.py
--- a/pypy/rlib/rdtoa.py
+++ b/pypy/rlib/rdtoa.py
@@ -15,11 +15,6 @@
     includes = ['src/dtoa.h'],
     libraries = [],
     separate_module_files = [cdir / 'src' / 'dtoa.c'],
-    separate_module_sources = ['''
-       #include <stdlib.h>
-       #include <assert.h>
-       #include "src/allocator.h"
-    '''],
     export_symbols = ['_PyPy_dg_strtod',
                       '_PyPy_dg_dtoa',
                       '_PyPy_dg_freedtoa',

From commits-noreply at bitbucket.org  Mon Feb 14 13:55:54 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon, 14 Feb 2011 13:55:54 +0100 (CET)
Subject: [pypy-svn] pypy default: Put the temporary 'externmod_%d.so' in
	udir.
Message-ID: <20110214125554.ECAE1282B90@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41898:d34da489e896
Date: 2011-02-14 13:55 +0100
http://bitbucket.org/pypy/pypy/changeset/d34da489e896/

Log:	Put the temporary 'externmod_%d.so' in udir.

diff --git a/pypy/translator/tool/cbuild.py b/pypy/translator/tool/cbuild.py
--- a/pypy/translator/tool/cbuild.py
+++ b/pypy/translator/tool/cbuild.py
@@ -276,7 +276,8 @@
                 return self
             basepath = udir.join('module_cache')
         else:
-            basepath = py.path.local(self.separate_module_files[0]).dirpath()
+            #basepath = py.path.local(self.separate_module_files[0]).dirpath()
+            basepath = udir.join('shared_cache')
         if outputfilename is None:
             # find more or less unique name there
             pth = basepath.join('externmod').new(ext=host.so_ext)
@@ -285,7 +286,8 @@
                 pth = basepath.join(
                     'externmod_%d' % (num,)).new(ext=host.so_ext)
                 num += 1
-            outputfilename=pth.purebasename
+            basepath.ensure(dir=1)
+            outputfilename = str(pth.dirpath().join(pth.purebasename))
         lib = str(host.compile([], self, outputfilename=outputfilename,
                                standalone=False))
         d = self._copy_attributes()

From commits-noreply at bitbucket.org  Mon Feb 14 13:55:55 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon, 14 Feb 2011 13:55:55 +0100 (CET)
Subject: [pypy-svn] pypy default: merge heads
Message-ID: <20110214125555.3F436282BAD@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41899:15edfec066c5
Date: 2011-02-14 13:55 +0100
http://bitbucket.org/pypy/pypy/changeset/15edfec066c5/

Log:	merge heads


From commits-noreply at bitbucket.org  Mon Feb 14 13:57:34 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon, 14 Feb 2011 13:57:34 +0100 (CET)
Subject: [pypy-svn] pypy default: Ignore dtoa.o.
Message-ID: <20110214125734.93D4B282B90@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41900:40368928a1c5
Date: 2011-02-14 13:57 +0100
http://bitbucket.org/pypy/pypy/changeset/40368928a1c5/

Log:	Ignore dtoa.o.

diff --git a/.hgignore b/.hgignore
--- a/.hgignore
+++ b/.hgignore
@@ -16,6 +16,7 @@
 ^pypy/doc/.+\.html$
 ^pypy/doc/basicblock\.asc$
 ^pypy/doc/.+\.svninfo$
+^pypy/translator/c/src/dtoa.o$
 ^pypy/translator/c/src/libffi_msvc/.+\.obj$
 ^pypy/translator/c/src/libffi_msvc/.+\.dll$
 ^pypy/translator/c/src/libffi_msvc/.+\.lib$
@@ -59,4 +60,4 @@
 ^pypy/doc/image/lattice3\.png$
 ^pypy/doc/image/stackless_informal\.png$
 ^pypy/doc/image/parsing_example.+\.png$
-^compiled
\ No newline at end of file
+^compiled

From commits-noreply at bitbucket.org  Mon Feb 14 14:06:14 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon, 14 Feb 2011 14:06:14 +0100 (CET)
Subject: [pypy-svn] pypy default: Modify 'test_all.py' to refuse to run if
	started	naively, i.e. in
Message-ID: <20110214130614.C7E492A2002@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41901:32f5cd9226de
Date: 2011-02-14 14:06 +0100
http://bitbucket.org/pypy/pypy/changeset/32f5cd9226de/

Log:	Modify 'test_all.py' to refuse to run if started naively, i.e. in
	the top directory and with no option. Instead it prints some
	minimal help.

diff --git a/pypy/test_all.py b/pypy/test_all.py
--- a/pypy/test_all.py
+++ b/pypy/test_all.py
@@ -1,4 +1,18 @@
 #! /usr/bin/env python
+"""
+PyPy Test runner interface
+--------------------------
+
+Running test_all.py is equivalent to running py.test
+(either installed from the py lib package, or from ../py/bin/).
+
+For more information, use test_all.py -h.
+"""
+import sys, os
+
+if len(sys.argv) == 1 and os.path.dirname(sys.argv[0]) in '.':
+    print >> sys.stderr, __doc__
+    sys.exit(2)
 
 if __name__ == '__main__':
     import tool.autopath

From commits-noreply at bitbucket.org  Mon Feb 14 14:11:05 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon, 14 Feb 2011 14:11:05 +0100 (CET)
Subject: [pypy-svn] pypy default: Syntax for 2.5.
Message-ID: <20110214131105.C9E4E282B90@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41902:5d8a2eb2b157
Date: 2011-02-14 14:10 +0100
http://bitbucket.org/pypy/pypy/changeset/5d8a2eb2b157/

Log:	Syntax for 2.5.

diff --git a/pypy/objspace/std/test/test_bytes.py b/pypy/objspace/std/test/test_bytes.py
--- a/pypy/objspace/std/test/test_bytes.py
+++ b/pypy/objspace/std/test/test_bytes.py
@@ -222,8 +222,8 @@
         b = bytearray('mississippi')
         check(b.split('i'), ['m', 'ss', 'ss', 'pp', ''])
         check(b.split(memoryview('i')), ['m', 'ss', 'ss', 'pp', ''])
-        check(b.rsplit('i'), ['m', b'ss', b'ss', b'pp', b''])
-        check(b.rsplit(memoryview('i')), ['m', b'ss', b'ss', b'pp', b''])
+        check(b.rsplit('i'), ['m', 'ss', 'ss', 'pp', ''])
+        check(b.rsplit(memoryview('i')), ['m', 'ss', 'ss', 'pp', ''])
         check(b.rsplit('i', 2), ['mississ', 'pp', ''])
 
         check(bytearray('foo bar').split(), ['foo', 'bar'])
@@ -267,14 +267,14 @@
         assert b.pop(-2) == ord('r')
         raises(IndexError, b.pop, 10)
         raises(OverflowError, bytearray().pop)
-        assert bytearray(b'\xff').pop() == 0xff
+        assert bytearray('\xff').pop() == 0xff
 
     def test_remove(self):
         class Indexable:
             def __index__(self):
                 return ord('e')
 
-        b = bytearray(b'hello')
+        b = bytearray('hello')
         b.remove(ord('l'))
         assert b == 'helo'
         b.remove(ord('l'))

From commits-noreply at bitbucket.org  Mon Feb 14 15:21:59 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Mon, 14 Feb 2011 15:21:59 +0100 (CET)
Subject: [pypy-svn] pypy default: Implement CPython 2.7 "correct" rounding
	of floats.
Message-ID: <20110214142159.22A1A2A2002@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41903:85cf8a4621be
Date: 2011-02-14 15:21 +0100
http://bitbucket.org/pypy/pypy/changeset/85cf8a4621be/

Log:	Implement CPython 2.7 "correct" rounding of floats. Code and
	comments are copied from floatobject.c.

diff --git a/pypy/module/__builtin__/test/test_builtin.py b/pypy/module/__builtin__/test/test_builtin.py
--- a/pypy/module/__builtin__/test/test_builtin.py
+++ b/pypy/module/__builtin__/test/test_builtin.py
@@ -663,6 +663,9 @@
         assert repr(round(nan)) == repr(nan)
         #
         raises(OverflowError, round, 1.6e308, -308)
+        #
+        assert round(562949953421312.5, 1) == 562949953421312.5
+        assert round(56294995342131.5, 3) == 56294995342131.5
 
     def test_vars_obscure_case(self):
         class C_get_vars(object):

diff --git a/pypy/rlib/rarithmetic.py b/pypy/rlib/rarithmetic.py
--- a/pypy/rlib/rarithmetic.py
+++ b/pypy/rlib/rarithmetic.py
@@ -681,6 +681,130 @@
     result = formatd(value, tp, precision, flags)
     return result, special
 
+if USE_SHORT_FLOAT_REPR:
+    def round_double(value, ndigits):
+        # The basic idea is very simple: convert and round the double to
+        # a decimal string using _Py_dg_dtoa, then convert that decimal
+        # string back to a double with _Py_dg_strtod.  There's one minor
+        # difficulty: Python 2.x expects round to do
+        # round-half-away-from-zero, while _Py_dg_dtoa does
+        # round-half-to-even.  So we need some way to detect and correct
+        # the halfway cases.
+
+        # a halfway value has the form k * 0.5 * 10**-ndigits for some
+        # odd integer k.  Or in other words, a rational number x is
+        # exactly halfway between two multiples of 10**-ndigits if its
+        # 2-valuation is exactly -ndigits-1 and its 5-valuation is at
+        # least -ndigits.  For ndigits >= 0 the latter condition is
+        # automatically satisfied for a binary float x, since any such
+        # float has nonnegative 5-valuation.  For 0 > ndigits >= -22, x
+        # needs to be an integral multiple of 5**-ndigits; we can check
+        # this using fmod.  For -22 > ndigits, there are no halfway
+        # cases: 5**23 takes 54 bits to represent exactly, so any odd
+        # multiple of 0.5 * 10**n for n >= 23 takes at least 54 bits of
+        # precision to represent exactly.
+
+        sign = copysign(1.0, value)
+        value = abs(value)
+
+        # find 2-valuation value
+        m, expo = math.frexp(value)
+        while m != math.floor(m):
+            m *= 2.0
+            expo -= 1
+
+        # determine whether this is a halfway case.
+        halfway_case = 0
+        if expo == -ndigits - 1:
+            if ndigits >= 0:
+                halfway_case = 1
+            elif ndigits >= -22:
+                # 22 is the largest k such that 5**k is exactly
+                # representable as a double
+                five_pow = 1.0
+                for i in range(-ndigits):
+                    five_pow *= 5.0
+                if math.fmod(value, five_pow) == 0.0:
+                    halfway_case = 1
+
+        # round to a decimal string; use an extra place for halfway case
+        strvalue = formatd(value, 'f', ndigits + halfway_case)
+
+        buf = [c for c in strvalue]
+
+        if halfway_case:
+            if ndigits >= 0:
+                endpos = len(buf) - 1
+            else:
+                endpos = len(buf) + ndigits
+            # Sanity checks: there should be exactly ndigits+1 places
+            # following the decimal point, and the last digit in the
+            # buffer should be a '5'
+            if not objectmodel.we_are_translated():
+                assert buf[endpos] == '5'
+                if '.' in buf:
+                    assert endpos == len(buf) - 1
+                    assert buf.index('.') == len(buf) - ndigits - 2
+
+            # increment and shift right at the same time
+            i = endpos - 1
+            carry = 1
+            while i >= 0:
+                digit = ord(buf[i])
+                if digit == ord('.'):
+                    buf[i+1] = chr(digit)
+                    i -= 1
+                    digit = ord(buf[i])
+
+                carry += digit - ord('0')
+                buf[i+1] = chr(carry % 10 + ord('0'))
+                carry /= 10
+                i -= 1
+            buf[0] = chr(carry + ord('0'))
+            if ndigits < 0:
+                buf.append('0')
+
+            strvalue = ''.join(buf)
+
+        return sign * rstring_to_float(strvalue)
+
+else:
+    # fallback version, to be used when correctly rounded
+    # binary<->decimal conversions aren't available
+    def round_double(value, ndigits):
+        if ndigits >= 0:
+            if ndigits > 22:
+                # pow1 and pow2 are each safe from overflow, but
+                # pow1*pow2 ~= pow(10.0, ndigits) might overflow
+                pow1 = math.pow(10.0, ndigits - 22)
+                pow2 = 1e22
+            else:
+                pow1 = math.pow(10.0, ndigits)
+                pow2 = 1.0
+
+            y = (value * pow1) * pow2
+            # if y overflows, then rounded value is exactly x
+            if isinf(y):
+                return value
+
+        else:
+            pow1 = math.pow(10.0, -ndigits);
+            pow2 = 1.0 # unused; for translation
+            y = value / pow1
+
+        if y >= 0.0:
+            z = math.floor(y + 0.5)
+        else:
+            z = math.ceil(y - 0.5)
+        if math.fabs(y-z) == 1.0:   # obscure case, see the test
+            z = y
+
+        if ndigits >= 0:
+            z = (z / pow2) / pow1
+        else:
+            z *= pow1
+        return z
+
 # the 'float' C type
 
 class r_singlefloat(object):

diff --git a/pypy/module/__builtin__/operation.py b/pypy/module/__builtin__/operation.py
--- a/pypy/module/__builtin__/operation.py
+++ b/pypy/module/__builtin__/operation.py
@@ -8,7 +8,7 @@
 from pypy.interpreter.gateway import interp2app
 from pypy.interpreter.typedef import TypeDef
 from pypy.rlib.runicode import UNICHR
-from pypy.rlib.rarithmetic import isnan, isinf
+from pypy.rlib.rarithmetic import isnan, isinf, round_double
 from pypy.rlib import rfloat
 import math
 import __builtin__
@@ -151,40 +151,11 @@
         # return 0.0, but with sign of x
         return space.wrap(0.0 * number)
 
-    if ndigits >= 0:
-        if ndigits > 22:
-            # pow1 and pow2 are each safe from overflow, but
-            # pow1*pow2 ~= pow(10.0, ndigits) might overflow
-            pow1 = math.pow(10.0, ndigits - 22)
-            pow2 = 1e22
-        else:
-            pow1 = math.pow(10.0, ndigits)
-            pow2 = 1.0
-
-        y = (number * pow1) * pow2
-        # if y overflows, then rounded value is exactly x
-        if isinf(y):
-            return space.wrap(number)
-
-    else:
-        pow1 = math.pow(10.0, -ndigits);
-        pow2 = 1.0 # unused; for translation
-        y = number / pow1
-
-    if y >= 0.0:
-        z = math.floor(y + 0.5)
-    else:
-        z = math.ceil(y - 0.5)
-    if math.fabs(y-z) == 1.0:   # obscure case, see the test
-        z = y
-
-    if ndigits >= 0:
-        z = (z / pow2) / pow1
-    else:
-        z *= pow1
+    # finite x, and ndigits is not unreasonably large
+    z = round_double(number, ndigits)
     if isinf(z):
         raise OperationError(space.w_OverflowError,
-                            space.wrap("rounded value too large to represent"))
+                             space.wrap("rounded value too large to represent"))
     return space.wrap(z)
 #
 round.unwrap_spec = [ObjSpace, float, W_Root]

diff --git a/pypy/rlib/test/test_rarithmetic.py b/pypy/rlib/test/test_rarithmetic.py
--- a/pypy/rlib/test/test_rarithmetic.py
+++ b/pypy/rlib/test/test_rarithmetic.py
@@ -464,3 +464,62 @@
     assert round_away(-.1) == 0.
     assert round_away(-.7) == -1.
     assert round_away(0.) == 0.
+
+def test_round_double():
+    def almost_equal(x, y):
+        assert round(abs(x-y), 7) == 0
+
+    almost_equal(round_double(0.125, 2), 0.13)
+    almost_equal(round_double(0.375, 2), 0.38)
+    almost_equal(round_double(0.625, 2), 0.63)
+    almost_equal(round_double(0.875, 2), 0.88)
+    almost_equal(round_double(-0.125, 2), -0.13)
+    almost_equal(round_double(-0.375, 2), -0.38)
+    almost_equal(round_double(-0.625, 2), -0.63)
+    almost_equal(round_double(-0.875, 2), -0.88)
+
+    almost_equal(round_double(0.25, 1), 0.3)
+    almost_equal(round_double(0.75, 1), 0.8)
+    almost_equal(round_double(-0.25, 1), -0.3)
+    almost_equal(round_double(-0.75, 1), -0.8)
+
+    round_double(-6.5, 0) == -7.0
+    round_double(-5.5, 0) == -6.0
+    round_double(-1.5, 0) == -2.0
+    round_double(-0.5, 0) == -1.0
+    round_double(0.5, 0) == 1.0
+    round_double(1.5, 0) == 2.0
+    round_double(2.5, 0) == 3.0
+    round_double(3.5, 0) == 4.0
+    round_double(4.5, 0) == 5.0
+    round_double(5.5, 0) == 6.0
+    round_double(6.5, 0) == 7.0
+
+    round_double(-25.0, -1) == -30.0
+    round_double(-15.0, -1) == -20.0
+    round_double(-5.0, -1) == -10.0
+    round_double(5.0, -1) == 10.0
+    round_double(15.0, -1) == 20.0
+    round_double(25.0, -1) == 30.0
+    round_double(35.0, -1) == 40.0
+    round_double(45.0, -1) == 50.0
+    round_double(55.0, -1) == 60.0
+    round_double(65.0, -1) == 70.0
+    round_double(75.0, -1) == 80.0
+    round_double(85.0, -1) == 90.0
+    round_double(95.0, -1) == 100.0
+    round_double(12325.0, -1) == 12330.0
+
+    round_double(350.0, -2) == 400.0
+    round_double(450.0, -2) == 500.0
+
+    almost_equal(round_double(0.5e21, -21), 1e21)
+    almost_equal(round_double(1.5e21, -21), 2e21)
+    almost_equal(round_double(2.5e21, -21), 3e21)
+    almost_equal(round_double(5.5e21, -21), 6e21)
+    almost_equal(round_double(8.5e21, -21), 9e21)
+
+    almost_equal(round_double(-1.5e22, -22), -2e22)
+    almost_equal(round_double(-0.5e22, -22), -1e22)
+    almost_equal(round_double(0.5e22, -22), 1e22)
+    almost_equal(round_double(1.5e22, -22), 2e22)

From commits-noreply at bitbucket.org  Mon Feb 14 15:21:59 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Mon, 14 Feb 2011 15:21:59 +0100 (CET)
Subject: [pypy-svn] pypy default: Merge heads
Message-ID: <20110214142159.72D082A202D@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41904:a4fbde429033
Date: 2011-02-14 15:21 +0100
http://bitbucket.org/pypy/pypy/changeset/a4fbde429033/

Log:	Merge heads


From commits-noreply at bitbucket.org  Mon Feb 14 15:23:56 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Mon, 14 Feb 2011 15:23:56 +0100 (CET)
Subject: [pypy-svn] pypy default: add an editable copy of test_site.py
Message-ID: <20110214142356.34FE9282B90@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r41905:90de1db7f06a
Date: 2011-02-14 14:37 +0100
http://bitbucket.org/pypy/pypy/changeset/90de1db7f06a/

Log:	add an editable copy of test_site.py

diff --git a/lib-python/2.7.0/test/test_site.py b/lib-python/modified-2.7.0/test/test_site.py
copy from lib-python/2.7.0/test/test_site.py
copy to lib-python/modified-2.7.0/test/test_site.py

From commits-noreply at bitbucket.org  Mon Feb 14 15:23:56 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Mon, 14 Feb 2011 15:23:56 +0100 (CET)
Subject: [pypy-svn] pypy default: typo
Message-ID: <20110214142356.E4725282B90@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r41906:52ce92664c16
Date: 2011-02-14 14:46 +0100
http://bitbucket.org/pypy/pypy/changeset/52ce92664c16/

Log:	typo


diff --git a/lib-python/modified-2.7.0/distutils/sysconfig.py b/lib-python/modified-2.7.0/distutils/sysconfig.py
--- a/lib-python/modified-2.7.0/distutils/sysconfig.py
+++ b/lib-python/modified-2.7.0/distutils/sysconfig.py
@@ -22,6 +22,6 @@
     from distutils.sysconfig_pypy import _config_vars # needed by setuptools
 else:
     from distutils.sysconfig_cpython import *
-    from distutils.sysconfig_pypy import _config_vars # needed by setuptools
+    from distutils.sysconfig_cpython import _config_vars # needed by setuptools
 
 

From commits-noreply at bitbucket.org  Mon Feb 14 15:23:58 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Mon, 14 Feb 2011 15:23:58 +0100 (CET)
Subject: [pypy-svn] pypy default: fix test_getsitepackages by deciding that
	pypy does not	support site-python, only site-packages
Message-ID: <20110214142358.73F3E282C1B@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r41907:3849dfadc6b2
Date: 2011-02-14 15:12 +0100
http://bitbucket.org/pypy/pypy/changeset/3849dfadc6b2/

Log:	fix test_getsitepackages by deciding that pypy does not support
	site-python, only site-packages

diff --git a/lib-python/modified-2.7.0/test/test_site.py b/lib-python/modified-2.7.0/test/test_site.py
--- a/lib-python/modified-2.7.0/test/test_site.py
+++ b/lib-python/modified-2.7.0/test/test_site.py
@@ -168,6 +168,10 @@
             self.assertEqual(len(dirs), 1)
             wanted = os.path.join('xoxo', 'Lib', 'site-packages')
             self.assertEquals(dirs[0], wanted)
+        elif '__pypy__' in sys.builtin_module_names:
+            self.assertEquals(len(dirs), 1)
+            wanted = os.path.join('xoxo', 'site-packages')
+            self.assertEquals(dirs[0], wanted)
         elif os.sep == '/':
             self.assertEqual(len(dirs), 2)
             wanted = os.path.join('xoxo', 'lib', 'python' + sys.version[:3],

diff --git a/lib-python/modified-2.7.0/site.py b/lib-python/modified-2.7.0/site.py
--- a/lib-python/modified-2.7.0/site.py
+++ b/lib-python/modified-2.7.0/site.py
@@ -281,13 +281,7 @@
     will find its `site-packages` subdirectory depending on the system
     environment, and will return a list of full paths.
     """
-    # pypy specific logic
-    if hasattr(sys, 'pypy_version_info') and hasattr(sys, 'prefix'):
-        from distutils.sysconfig import get_python_lib
-        sitedir = get_python_lib(standard_lib=False)
-        return [sitedir]
-
-    # cpython logic
+    is_pypy = '__pypy__' in sys.builtin_module_names
     sitepackages = []
     seen = set()
 
@@ -298,6 +292,10 @@
 
         if sys.platform in ('os2emx', 'riscos'):
             sitepackages.append(os.path.join(prefix, "Lib", "site-packages"))
+        elif is_pypy:
+            from distutils.sysconfig import get_python_lib
+            sitedir = get_python_lib(standard_lib=False, prefix=prefix)
+            sitepackages.append(sitedir)
         elif os.sep == '/':
             sitepackages.append(os.path.join(prefix, "lib",
                                         "python" + sys.version[:3],

From commits-noreply at bitbucket.org  Mon Feb 14 15:40:18 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Mon, 14 Feb 2011 15:40:18 +0100 (CET)
Subject: [pypy-svn] pypy default: in dtoa.c,
	use malloc() directly. The code has its own freelists anyway.
Message-ID: <20110214144018.96C4F282B90@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41908:5da6e2d7ccde
Date: 2011-02-14 15:38 +0100
http://bitbucket.org/pypy/pypy/changeset/5da6e2d7ccde/

Log:	in dtoa.c, use malloc() directly. The code has its own freelists
	anyway.

diff --git a/pypy/translator/c/src/dtoa.c b/pypy/translator/c/src/dtoa.c
--- a/pypy/translator/c/src/dtoa.c
+++ b/pypy/translator/c/src/dtoa.c
@@ -129,10 +129,9 @@
 #include <stdio.h>
 #include <string.h>
 #define PYPY_NOT_MAIN_FILE
-#include "src/allocator.h"
 #include "src/asm.h"
-#define PyMem_Malloc PyObject_Malloc
-#define PyMem_Free PyObject_Free
+#define PyMem_Malloc malloc
+#define PyMem_Free free
 /* End PYPY hacks */
 
 

From commits-noreply at bitbucket.org  Mon Feb 14 16:55:22 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Mon, 14 Feb 2011 16:55:22 +0100 (CET)
Subject: [pypy-svn] pypy default: Subclasses of unicode should call their
	eventual	__unicode__ method when
Message-ID: <20110214155522.BFC14282B90@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41909:f63f339a7811
Date: 2011-02-14 16:54 +0100
http://bitbucket.org/pypy/pypy/changeset/f63f339a7811/

Log:	Subclasses of unicode should call their eventual __unicode__ method
	when they are rendered with %s or "{}".format

diff --git a/pypy/objspace/std/formatting.py b/pypy/objspace/std/formatting.py
--- a/pypy/objspace/std/formatting.py
+++ b/pypy/objspace/std/formatting.py
@@ -7,6 +7,7 @@
 from pypy.interpreter.error import OperationError
 from pypy.tool.sourcetools import func_with_new_name
 from pypy.rlib.rstring import StringBuilder, UnicodeBuilder
+from pypy.objspace.std.unicodetype import unicode_from_object
 
 class BaseStringFormatter(object):
     def __init__(self, space, values_w, w_valuedict):
@@ -430,6 +431,8 @@
             else:
                 if not got_unicode:
                     w_value = space.call_function(space.w_unicode, w_value)
+                else:
+                    w_value = unicode_from_object(space, w_value)
                 s = space.unicode_w(w_value)
             self.std_wp(s)
 

diff --git a/pypy/objspace/std/test/test_unicodeobject.py b/pypy/objspace/std/test/test_unicodeobject.py
--- a/pypy/objspace/std/test/test_unicodeobject.py
+++ b/pypy/objspace/std/test/test_unicodeobject.py
@@ -812,6 +812,14 @@
 
         assert unicode(A()) == u'bar'
 
+    def test_format_unicode_subclass(self):
+        class U(unicode):
+            def __unicode__(self):
+                return u'__unicode__ overridden'
+        u = U(u'xxx')
+        assert repr("%s" % u) == "u'__unicode__ overridden'"
+        assert repr("{}".format(u)) == "'__unicode__ overridden'"
+
     def test_replace_with_buffer(self):
         assert u'abc'.replace(buffer('b'), buffer('e')) == u'aec'
         assert u'abc'.replace(buffer('b'), u'e') == u'aec'

diff --git a/pypy/objspace/std/unicodeobject.py b/pypy/objspace/std/unicodeobject.py
--- a/pypy/objspace/std/unicodeobject.py
+++ b/pypy/objspace/std/unicodeobject.py
@@ -927,6 +927,8 @@
 def format__Unicode_ANY(space, w_unicode, w_format_spec):
     if not space.isinstance_w(w_format_spec, space.w_unicode):
         w_format_spec = space.call_function(space.w_unicode, w_format_spec)
+    from pypy.objspace.std.unicodetype import unicode_from_object
+    w_unicode = unicode_from_object(space, w_unicode)
     spec = space.unicode_w(w_format_spec)
     formatter = newformat.unicode_formatter(space, spec)
     return formatter.format_string(w_unicode._value)

diff --git a/pypy/objspace/std/unicodetype.py b/pypy/objspace/std/unicodetype.py
--- a/pypy/objspace/std/unicodetype.py
+++ b/pypy/objspace/std/unicodetype.py
@@ -293,7 +293,9 @@
     return w_retval
 
 def unicode_from_object(space, w_obj):
-    if space.is_w(space.type(w_obj), space.w_str):
+    if space.is_w(space.type(w_obj), space.w_unicode):
+        return w_obj
+    elif space.is_w(space.type(w_obj), space.w_str):
         w_res = w_obj
     else:
         w_unicode_method = space.lookup(w_obj, "__unicode__")

From commits-noreply at bitbucket.org  Mon Feb 14 17:19:23 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Mon, 14 Feb 2011 17:19:23 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix translation
Message-ID: <20110214161923.2925F282B90@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41910:c14ca767e02b
Date: 2011-02-14 17:17 +0100
http://bitbucket.org/pypy/pypy/changeset/c14ca767e02b/

Log:	Fix translation

diff --git a/pypy/objspace/std/unicodeobject.py b/pypy/objspace/std/unicodeobject.py
--- a/pypy/objspace/std/unicodeobject.py
+++ b/pypy/objspace/std/unicodeobject.py
@@ -931,7 +931,7 @@
     w_unicode = unicode_from_object(space, w_unicode)
     spec = space.unicode_w(w_format_spec)
     formatter = newformat.unicode_formatter(space, spec)
-    return formatter.format_string(w_unicode._value)
+    return formatter.format_string(space.unicode_w(w_unicode))
 
 
 import unicodetype

From commits-noreply at bitbucket.org  Mon Feb 14 17:29:04 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon, 14 Feb 2011 17:29:04 +0100 (CET)
Subject: [pypy-svn] pypy default: (Kevin Burke, a bit of arigo)
Message-ID: <20110214162904.851AD282B90@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41911:665baf750859
Date: 2011-02-14 17:27 +0100
http://bitbucket.org/pypy/pypy/changeset/665baf750859/

Log:	(Kevin Burke, a bit of arigo)

	Implement complex.__format__().

diff --git a/pypy/objspace/std/complexobject.py b/pypy/objspace/std/complexobject.py
--- a/pypy/objspace/std/complexobject.py
+++ b/pypy/objspace/std/complexobject.py
@@ -1,5 +1,6 @@
 from pypy.interpreter import gateway
 from pypy.interpreter.error import OperationError
+from pypy.objspace.std import newformat
 from pypy.objspace.std.model import registerimplementation, W_Object
 from pypy.objspace.std.register_all import register_all
 from pypy.objspace.std.floatobject import W_FloatObject, _hash_float
@@ -207,11 +208,11 @@
         raise OperationError(space.w_OverflowError, space.wrap(str(e)))
 
 def eq__Complex_Complex(space, w_complex1, w_complex2):
-    return space.newbool((w_complex1.realval == w_complex2.realval) and 
+    return space.newbool((w_complex1.realval == w_complex2.realval) and
             (w_complex1.imagval == w_complex2.imagval))
 
 def ne__Complex_Complex(space, w_complex1, w_complex2):
-    return space.newbool((w_complex1.realval != w_complex2.realval) or 
+    return space.newbool((w_complex1.realval != w_complex2.realval) or
             (w_complex1.imagval != w_complex2.imagval))
 
 def eq__Complex_Long(space, w_complex1, w_long2):
@@ -288,5 +289,8 @@
     return space.wrap('(' + str_format(w_complex.realval)
                       + sign + str_format(w_complex.imagval) + 'j)')
 
+def format__Complex_ANY(space, w_complex, w_format_spec):
+    return newformat.run_formatter(space, w_format_spec, "format_complex", w_complex)
+
 from pypy.objspace.std import complextype
 register_all(vars(), complextype)

diff --git a/pypy/objspace/std/test/test_complexobject.py b/pypy/objspace/std/test/test_complexobject.py
--- a/pypy/objspace/std/test/test_complexobject.py
+++ b/pypy/objspace/std/test/test_complexobject.py
@@ -153,7 +153,7 @@
 
         raises(ZeroDivisionError, complex.__div__, 1+1j, 0+0j)
         # FIXME: The following currently crashes on Alpha
-        # raises(OverflowError, pow, 1e200+1j, 1e200+1j)
+        raises(OverflowError, pow, 1e200+1j, 1e200+1j)
 
     def test_truediv(self):
         assert self.almost_equal(complex.__truediv__(2+0j, 1+1j), 1-1j)
@@ -460,7 +460,6 @@
         assert complex(a) == 42j
 
     def test_format(self):
-        skip("FIXME")
         # empty format string is same as str()
         assert format(1+3j, '') == str(1+3j)
         assert format(1.5+3.5j, '') == str(1.5+3.5j)
@@ -471,9 +470,11 @@
 
         # empty presentation type should still be analogous to str,
         # even when format string is nonempty (issue #5920).
+
+        assert format(3.2, '-') == str(3.2)
         assert format(3.2+0j, '-') == str(3.2+0j)
         assert format(3.2+0j, '<') == str(3.2+0j)
-        z = 4/7. - 100j/7.
+        z = 10/7. - 100j/7.
         assert format(z, '') == str(z)
         assert format(z, '-') == str(z)
         assert format(z, '<') == str(z)
@@ -537,12 +538,16 @@
         raises(ValueError, (1.5+3j).__format__, '=20')
 
         # integer presentation types are an error
-        for t in 'bcdoxX':
+        for t in 'bcdoxX%':
             raises(ValueError, (1.5+0.5j).__format__, t)
 
         # make sure everything works in ''.format()
         assert '*{0:.3f}*'.format(3.14159+2.71828j) == '*3.142+2.718j*'
+        assert u'*{0:.3f}*'.format(3.14159+2.71828j) == u'*3.142+2.718j*'
+        assert u'{:-}'.format(1.5+3.5j) == u'(1.5+3.5j)'
 
+        INF = float("inf")
+        NAN = float("nan")
         # issue 3382: 'f' and 'F' with inf's and nan's
         assert '{0:f}'.format(INF+0j) == 'inf+0.000000j'
         assert '{0:F}'.format(INF+0j) == 'INF+0.000000j'

diff --git a/pypy/objspace/std/newformat.py b/pypy/objspace/std/newformat.py
--- a/pypy/objspace/std/newformat.py
+++ b/pypy/objspace/std/newformat.py
@@ -5,6 +5,7 @@
 from pypy.interpreter.error import OperationError
 from pypy.rlib import rstring, runicode, rlocale, rarithmetic
 from pypy.rlib.objectmodel import specialize
+from pypy.rlib.rarithmetic import copysign, formatd
 
 
 @specialize.argtype(1)
@@ -366,6 +367,9 @@
     def format_float(self, w_num):
         raise NotImplementedError
 
+    def format_complex(self, w_num):
+        raise NotImplementedError
+
 
 INT_KIND = 1
 LONG_KIND = 2
@@ -380,6 +384,7 @@
     """__format__ implementation for builtin types."""
 
     _annspecialcase_ = "specialize:ctr_location"
+    _grouped_digits = None
 
     def __init__(self, space, is_unicode, spec):
         self.space = space
@@ -478,6 +483,7 @@
         return False
 
     def _calc_padding(self, string, length):
+        """compute left and right padding, return total width of string"""
         if self._width != -1 and length < self._width:
             total = self._width
         else:
@@ -568,6 +574,17 @@
 
     def _calc_num_width(self, n_prefix, sign_char, to_number, n_number,
                         n_remainder, has_dec, digits):
+        """Calculate widths of all parts of formatted number.
+
+        Output will look like:
+
+            <lpadding> <sign> <prefix> <spadding> <grouped_digits> <decimal>
+            <remainder> <rpadding>
+
+        sign is computed from self._sign, and the sign of the number
+        prefix is given
+        digits is known
+        """
         spec = NumberSpec()
         spec.n_digits = n_number - n_remainder - has_dec
         spec.n_prefix = n_prefix
@@ -577,6 +594,7 @@
         spec.n_spadding = 0
         spec.n_rpadding = 0
         spec.n_min_width = 0
+        spec.n_total = 0
         spec.sign = "\0"
         spec.n_sign = 0
         sign = self._sign
@@ -612,6 +630,9 @@
                 spec.n_spadding = n_padding
             else:
                 raise AssertionError("shouldn't reach")
+        spec.n_total = spec.n_lpadding + spec.n_sign + spec.n_prefix + \
+                       spec.n_spadding + n_grouped_digits + \
+                       spec.n_decimal + spec.n_remainder + spec.n_rpadding
         return spec
 
     def _fill_digits(self, buf, digits, d_state, n_chars, n_zeros,
@@ -677,7 +698,7 @@
 
 
     def _fill_number(self, spec, num, to_digits, to_prefix, fill_char,
-                     to_remainder, upper):
+                     to_remainder, upper, grouped_digits=None):
         out = self._builder()
         if spec.n_lpadding:
             out.append_multiple_char(fill_char[0], spec.n_lpadding)
@@ -696,7 +717,8 @@
             out.append_multiple_char(fill_char[0], spec.n_spadding)
         if spec.n_digits != 0:
             if self._loc_thousands:
-                digits = self._grouped_digits
+                digits = grouped_digits if grouped_digits is not None \
+                         else self._grouped_digits
             else:
                 stop = to_digits + spec.n_digits
                 assert stop >= 0
@@ -710,7 +732,8 @@
             out.append(num[to_remainder:])
         if spec.n_rpadding:
             out.append_multiple_char(fill_char[0], spec.n_rpadding)
-        return self.space.wrap(out.build())
+        #if complex, need to call twice - just retun the buffer
+        return out.build()
 
     def _format_int_or_long(self, w_num, kind):
         space = self.space
@@ -768,8 +791,8 @@
                                     n_remainder, False, result)
         fill = self._lit(" ") if self._fill_char == "\0" else self._fill_char
         upper = self._type == "X"
-        return self._fill_number(spec, result, to_numeric, to_prefix,
-                                 fill, to_remainder, upper)
+        return self.space.wrap(self._fill_number(spec, result, to_numeric,
+                                 to_prefix, fill, to_remainder, upper))
 
     def _long_to_base(self, base, value):
         prefix = ""
@@ -855,6 +878,8 @@
             self._unknown_presentation("int" if kind == INT_KIND else "long")
 
     def _parse_number(self, s, i):
+        """Determine if s has a decimal point, and the index of the first #
+        after the decimal, or the end of the number."""
         length = len(s)
         while i < length and "0" <= s[i] <= "9":
             i += 1
@@ -862,9 +887,11 @@
         dec_point = i < length and s[i] == "."
         if dec_point:
             rest += 1
+        #differs from CPython method - CPython sets n_remainder
         return dec_point, rest
 
     def _format_float(self, w_float):
+        """helper for format_float"""
         space = self.space
         flags = 0
         default_precision = 6
@@ -909,8 +936,8 @@
         spec = self._calc_num_width(0, sign, to_number, n_digits,
                                     n_remainder, have_dec_point, digits)
         fill = self._lit(" ") if self._fill_char == "\0" else self._fill_char
-        return self._fill_number(spec, digits, to_number, 0, fill,
-                                 to_remainder, False)
+        return self.space.wrap(self._fill_number(spec, digits, to_number, 0,
+                                  fill, to_remainder, False))
 
     def format_float(self, w_float):
         space = self.space
@@ -931,6 +958,168 @@
             return self._format_float(w_float)
         self._unknown_presentation("float")
 
+    def _format_complex(self, w_complex):
+        space = self.space
+        tp = self._type
+        self._get_locale(tp)
+        default_precision = 6
+        if self._align == "=":
+            # '=' alignment is invalid
+            msg = ("'=' alignment flag is not allowed in"
+                   " complex format specifier")
+            raise OperationError(space.w_ValueError, space.wrap(msg))
+        if self._fill_char == "0":
+            #zero padding is invalid
+            msg = "Zero padding is not allowed in complex format specifier"
+            raise OperationError(space.w_ValueError, space.wrap(msg))
+        if self._alternate:
+            #alternate is invalid
+            msg = "Alternate form %s not allowed in complex format specifier"
+            raise OperationError(space.w_ValueError,
+                                 space.wrap(msg % (self._alternate)))
+        skip_re = 0
+        add_parens = 0
+        if tp == "\0":
+            #should mirror str() output
+            tp = "g"
+            default_precision = 12
+            #test if real part is non-zero
+            if (w_complex.realval == 0 and
+                copysign(1., w_complex.realval) == 1.):
+                skip_re = 1
+            else:
+                add_parens = 1
+
+        if tp == "n":
+            #same as 'g' except for locale, taken care of later
+            tp = "g"
+
+        #check if precision not set
+        if self._precision == -1:
+            self._precision = default_precision
+
+        #might want to switch to double_to_string from formatd
+        #in CPython it's named 're' - clashes with re module
+        re_num = formatd(w_complex.realval, tp, self._precision)
+        im_num = formatd(w_complex.imagval, tp, self._precision)
+        n_re_digits = len(re_num)
+        n_im_digits = len(im_num)
+
+        to_real_number = 0
+        to_imag_number = 0
+        re_sign = im_sign = ''
+        #if a sign character is in the output, remember it and skip
+        if re_num[0] == "-":
+            re_sign = "-"
+            to_real_number = 1
+            n_re_digits -= 1
+        if im_num[0] == "-":
+            im_sign = "-"
+            to_imag_number = 1
+            n_im_digits -= 1
+
+        #turn off padding - do it after number composition
+        #calc_num_width uses self._width, so assign to temporary variable,
+        #calculate width of real and imag parts, then reassign padding, align
+        tmp_fill_char = self._fill_char
+        tmp_align = self._align
+        tmp_width = self._width
+        self._fill_char = "\0"
+        self._align = "<"
+        self._width = -1
+
+        #determine if we have remainder, might include dec or exponent or both
+        re_have_dec, re_remainder_ptr = self._parse_number(re_num,
+                                                           to_real_number)
+        im_have_dec, im_remainder_ptr = self._parse_number(im_num,
+                                                           to_imag_number)
+
+        if self.is_unicode:
+            re_num = re_num.decode("ascii")
+            im_num = im_num.decode("ascii")
+
+        #set remainder, in CPython _parse_number sets this
+        #using n_re_digits causes tests to fail
+        re_n_remainder = len(re_num) - re_remainder_ptr
+        im_n_remainder = len(im_num) - im_remainder_ptr
+        re_spec = self._calc_num_width(0, re_sign, to_real_number, n_re_digits,
+                                       re_n_remainder, re_have_dec,
+                                       re_num)
+
+        #capture grouped digits b/c _fill_number reads from self._grouped_digits
+        #self._grouped_digits will get overwritten in imaginary calc_num_width
+        re_grouped_digits = self._grouped_digits
+        if not skip_re:
+            self._sign = "+"
+        im_spec = self._calc_num_width(0, im_sign, to_imag_number, n_im_digits,
+                                       im_n_remainder, im_have_dec,
+                                       im_num)
+
+        im_grouped_digits = self._grouped_digits
+        if skip_re:
+            re_spec.n_total = 0
+
+        #reassign width, alignment, fill character
+        self._align = tmp_align
+        self._width = tmp_width
+        self._fill_char = tmp_fill_char
+
+        #compute L and R padding - stored in self._left_pad and self._right_pad
+        self._calc_padding(self.empty, re_spec.n_total + im_spec.n_total + 1 +
+                                       add_parens * 2)
+
+        out = self._builder()
+        fill = self._fill_char
+        if fill == "\0":
+            fill = self._lit(" ")[0]
+
+        #compose the string
+        #add left padding
+        out.append_multiple_char(fill, self._left_pad)
+        if add_parens:
+            out.append(self._lit('(')[0])
+
+        #if the no. has a real component, add it
+        if not skip_re:
+            out.append(self._fill_number(re_spec, re_num, to_real_number, 0,
+                                         fill, re_remainder_ptr, False,
+                                         re_grouped_digits))
+
+        #add imaginary component
+        out.append(self._fill_number(im_spec, im_num, to_imag_number, 0,
+                                     fill, im_remainder_ptr, False,
+                                     im_grouped_digits))
+
+        #add 'j' character
+        out.append(self._lit('j')[0])
+
+        if add_parens:
+            out.append(self._lit(')')[0])
+
+        #add right padding
+        out.append_multiple_char(fill, self._right_pad)
+
+        return self.space.wrap(out.build())
+
+
+    def format_complex(self, w_complex):
+        """return the string representation of a complex number"""
+        space = self.space
+        #parse format specification, set associated variables
+        if self._parse_spec("\0", ">"):
+            return space.str(w_complex)
+        tp = self._type
+        if (tp == "\0" or
+            tp == "e" or
+            tp == "E" or
+            tp == "f" or
+            tp == "F" or
+            tp == "g" or
+            tp == "G" or
+            tp == "n"):
+            return self._format_complex(w_complex)
+        self._unknown_presentation("complex")
+
 
 def unicode_formatter(space, spec):
     return Formatter(space, True, spec)

From commits-noreply at bitbucket.org  Mon Feb 14 18:28:58 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Mon, 14 Feb 2011 18:28:58 +0100 (CET)
Subject: [pypy-svn] pypy default: add support for the PYTHONNOUSERSITE env
	variable
Message-ID: <20110214172858.D9EBB282B90@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r41912:b07350bf637b
Date: 2011-02-14 17:21 +0100
http://bitbucket.org/pypy/pypy/changeset/b07350bf637b/

Log:	add support for the PYTHONNOUSERSITE env variable

diff --git a/pypy/translator/goal/app_main.py b/pypy/translator/goal/app_main.py
--- a/pypy/translator/goal/app_main.py
+++ b/pypy/translator/goal/app_main.py
@@ -404,6 +404,9 @@
     # (relevant in case of "reload(sys)")
     sys.argv[:] = argv
 
+    if (PYTHON26 and not options["ignore_environment"] and os.getenv('PYTHONNOUSERSITE')):
+        options["no_user_site"] = True
+
     if (options["interactive"] or
         (not options["ignore_environment"] and os.getenv('PYTHONINSPECT'))):
         options["inspect"] = True

diff --git a/pypy/translator/goal/test2/test_app_main.py b/pypy/translator/goal/test2/test_app_main.py
--- a/pypy/translator/goal/test2/test_app_main.py
+++ b/pypy/translator/goal/test2/test_app_main.py
@@ -181,6 +181,11 @@
             self.check([opt, '-c', 'pass'], sys_argv=['-c'],
                        run_command='pass', **expected)
 
+    def test_sysflags_envvar(self, monkeypatch):
+        monkeypatch.setenv('PYTHONNOUSERSITE', '1')
+        expected = {"no_user_site": True}
+        self.check(['-c', 'pass'], sys_argv=['-c'], run_command='pass', **expected)
+        
 
 class TestInteraction:
     """

From commits-noreply at bitbucket.org  Mon Feb 14 18:30:44 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon, 14 Feb 2011 18:30:44 +0100 (CET)
Subject: [pypy-svn] pypy default: Try to work at least a bit even if the
	module fcntl is	not translated.
Message-ID: <20110214173044.F04D5282B90@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41913:f1c33fb37c9d
Date: 2011-02-14 17:25 +0100
http://bitbucket.org/pypy/pypy/changeset/f1c33fb37c9d/

Log:	Try to work at least a bit even if the module fcntl is not
	translated.

diff --git a/pypy/module/posix/app_posix.py b/pypy/module/posix/app_posix.py
--- a/pypy/module/posix/app_posix.py
+++ b/pypy/module/posix/app_posix.py
@@ -66,7 +66,10 @@
 
 if osname == 'posix':
     def _validate_fd(fd):
-        import fcntl
+        try:
+            import fcntl
+        except ImportError:
+            return
         try:
             fcntl.fcntl(fd, fcntl.F_GETFD)
         except IOError, e:

From commits-noreply at bitbucket.org  Mon Feb 14 18:30:45 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon, 14 Feb 2011 18:30:45 +0100 (CET)
Subject: [pypy-svn] pypy default: Patch by vincele: complain when open()ing
	a directory.
Message-ID: <20110214173045.9754D282B90@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41914:ed2843c85746
Date: 2011-02-14 18:28 +0100
http://bitbucket.org/pypy/pypy/changeset/ed2843c85746/

Log:	Patch by vincele: complain when open()ing a directory.

diff --git a/pypy/module/_file/interp_file.py b/pypy/module/_file/interp_file.py
--- a/pypy/module/_file/interp_file.py
+++ b/pypy/module/_file/interp_file.py
@@ -1,9 +1,11 @@
 import py
 import os
+import stat
+import errno
 from pypy.rlib import streamio
 from pypy.rlib.rarithmetic import r_longlong
 from pypy.module._file.interp_stream import W_AbstractStream
-from pypy.module._file.interp_stream import StreamErrors, wrap_streamerror
+from pypy.module._file.interp_stream import StreamErrors, wrap_streamerror, wrap_oserror_as_ioerror
 from pypy.module.posix.interp_posix import dispatch_filename
 from pypy.interpreter.error import OperationError, operationerrfmt
 from pypy.interpreter.gateway import ObjSpace, W_Root, Arguments, unwrap_spec
@@ -51,6 +53,16 @@
         if stream.flushable():
             getopenstreams(self.space)[stream] = None
 
+    def check_not_dir(self, fd):
+        try:
+            st = os.fstat(fd)
+        except OSError:
+            pass
+        else:
+            if (stat.S_ISDIR(st[0])):
+                ose = OSError(errno.EISDIR, '')
+                raise wrap_oserror_as_ioerror(self.space, ose, self.w_name)
+
     def check_mode_ok(self, mode):
         if (not mode or mode[0] not in ['r', 'w', 'a', 'U'] or
             ('U' in mode and ('w' in mode or 'a' in mode))):
@@ -92,6 +104,7 @@
         stream = dispatch_filename(streamio.open_file_as_stream)(
             self.space, w_name, mode, buffering)
         fd = stream.try_to_find_file_descriptor()
+        self.check_not_dir(fd)
         self.fdopenstream(stream, fd, mode)
 
     def direct___enter__(self):

From commits-noreply at bitbucket.org  Mon Feb 14 18:30:46 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon, 14 Feb 2011 18:30:46 +0100 (CET)
Subject: [pypy-svn] pypy default: 2.5 compat
Message-ID: <20110214173046.20C77282B90@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41915:6f806be8838c
Date: 2011-02-14 18:29 +0100
http://bitbucket.org/pypy/pypy/changeset/6f806be8838c/

Log:	2.5 compat

diff --git a/pypy/module/_file/test/test_file.py b/pypy/module/_file/test/test_file.py
--- a/pypy/module/_file/test/test_file.py
+++ b/pypy/module/_file/test/test_file.py
@@ -1,3 +1,4 @@
+from __future__ import with_statement
 import py
 
 from pypy.conftest import gettestobjspace, option

From commits-noreply at bitbucket.org  Mon Feb 14 18:30:46 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon, 14 Feb 2011 18:30:46 +0100 (CET)
Subject: [pypy-svn] pypy default: merge heads
Message-ID: <20110214173046.7697C282BAD@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41916:b21c05bdaca0
Date: 2011-02-14 18:30 +0100
http://bitbucket.org/pypy/pypy/changeset/b21c05bdaca0/

Log:	merge heads


From commits-noreply at bitbucket.org  Mon Feb 14 18:31:20 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Mon, 14 Feb 2011 18:31:20 +0100 (CET)
Subject: [pypy-svn] pypy default: Small optimization in round() for the
	usual case
Message-ID: <20110214173120.B9029282B90@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41917:5b6bd0574016
Date: 2011-02-14 18:22 +0100
http://bitbucket.org/pypy/pypy/changeset/5b6bd0574016/

Log:	Small optimization in round() for the usual case

diff --git a/pypy/rlib/rarithmetic.py b/pypy/rlib/rarithmetic.py
--- a/pypy/rlib/rarithmetic.py
+++ b/pypy/rlib/rarithmetic.py
@@ -730,9 +730,8 @@
         # round to a decimal string; use an extra place for halfway case
         strvalue = formatd(value, 'f', ndigits + halfway_case)
 
-        buf = [c for c in strvalue]
-
         if halfway_case:
+            buf = [c for c in strvalue]
             if ndigits >= 0:
                 endpos = len(buf) - 1
             else:

From commits-noreply at bitbucket.org  Mon Feb 14 18:31:21 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Mon, 14 Feb 2011 18:31:21 +0100 (CET)
Subject: [pypy-svn] pypy default: Copy two files (unchanged),
	to let pyclbr import files in the correct location
Message-ID: <20110214173121.C0E86282B90@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41918:00d28638e29d
Date: 2011-02-14 18:30 +0100
http://bitbucket.org/pypy/pypy/changeset/00d28638e29d/

Log:	Copy two files (unchanged), to let pyclbr import files in the
	correct location

diff --git a/lib-python/2.7.0/test/pyclbr_input.py b/lib-python/modified-2.7.0/test/pyclbr_input.py
copy from lib-python/2.7.0/test/pyclbr_input.py
copy to lib-python/modified-2.7.0/test/pyclbr_input.py

diff --git a/lib-python/2.7.0/test/test_pyclbr.py b/lib-python/modified-2.7.0/test/test_pyclbr.py
copy from lib-python/2.7.0/test/test_pyclbr.py
copy to lib-python/modified-2.7.0/test/test_pyclbr.py

From commits-noreply at bitbucket.org  Mon Feb 14 18:31:22 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Mon, 14 Feb 2011 18:31:22 +0100 (CET)
Subject: [pypy-svn] pypy default: Merge heads
Message-ID: <20110214173122.76E03282B90@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41919:8b748f8f8f6a
Date: 2011-02-14 18:30 +0100
http://bitbucket.org/pypy/pypy/changeset/8b748f8f8f6a/

Log:	Merge heads


From commits-noreply at bitbucket.org  Mon Feb 14 18:31:23 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Mon, 14 Feb 2011 18:31:23 +0100 (CET)
Subject: [pypy-svn] pypy default: Merge heads again
Message-ID: <20110214173123.29D2E282C1B@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41920:a015edbf94c1
Date: 2011-02-14 18:31 +0100
http://bitbucket.org/pypy/pypy/changeset/a015edbf94c1/

Log:	Merge heads again


From commits-noreply at bitbucket.org  Mon Feb 14 19:14:31 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Mon, 14 Feb 2011 19:14:31 +0100 (CET)
Subject: [pypy-svn] pypy default: Quit the "timeout killer" thread as soon
	as the process	exits.
Message-ID: <20110214181431.DF094282B90@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41921:657d5ff5412b
Date: 2011-02-14 19:14 +0100
http://bitbucket.org/pypy/pypy/changeset/657d5ff5412b/

Log:	Quit the "timeout killer" thread as soon as the process exits.
	Otherwise it may stay too long and other tests will count it among
	the active threads...

diff --git a/pypy/module/thread/test/support.py b/pypy/module/thread/test/support.py
--- a/pypy/module/thread/test/support.py
+++ b/pypy/module/thread/test/support.py
@@ -22,7 +22,9 @@
 
 def timeout_killer(pid, delay):
     def kill():
-        time.sleep(delay)
+        for x in range(delay * 10):
+            time.sleep(0.1)
+            os.kill(pid, 0)
         os.kill(pid, 9)
         print "process %s killed!" % (pid,)
     thread.start_new_thread(kill, ())

From commits-noreply at bitbucket.org  Mon Feb 14 19:26:32 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon, 14 Feb 2011 19:26:32 +0100 (CET)
Subject: [pypy-svn] pypy default: Catch StreamErrors that can be raised when
	opening a	file in
Message-ID: <20110214182632.09E5A282B90@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41922:f44f38661a6a
Date: 2011-02-14 19:26 +0100
http://bitbucket.org/pypy/pypy/changeset/f44f38661a6a/

Log:	Catch StreamErrors that can be raised when opening a file in
	interp_imp.py. Fix an issue in wrap_oserror_as_ioerror().

	This is not the full fix for issue XXX (<=insert number here) but
	it's a start.

diff --git a/pypy/module/imp/interp_imp.py b/pypy/module/imp/interp_imp.py
--- a/pypy/module/imp/interp_imp.py
+++ b/pypy/module/imp/interp_imp.py
@@ -4,6 +4,7 @@
 from pypy.interpreter.error import OperationError, operationerrfmt
 from pypy.interpreter.module import Module
 from pypy.interpreter.gateway import NoneNotWrapped, W_Root, ObjSpace
+from pypy.module._file.interp_stream import StreamErrors, wrap_streamerror
 import struct
 
 def get_suffixes(space):
@@ -32,7 +33,13 @@
 
 def get_file(space, w_file, filename, filemode):
     if w_file is None or space.is_w(w_file, space.w_None):
-        return streamio.open_file_as_stream(filename, filemode)
+        try:
+            return streamio.open_file_as_stream(filename, filemode)
+        except StreamErrors, e:
+            # XXX this is not quite the correct place, but it will do for now.
+            # XXX see the issue which I'm sure exists already but whose number
+            # XXX I cannot find any more...
+            raise wrap_streamerror(space, e)
     else:
         return space.interp_w(W_File, w_file).stream
 

diff --git a/pypy/module/_file/interp_stream.py b/pypy/module/_file/interp_stream.py
--- a/pypy/module/_file/interp_stream.py
+++ b/pypy/module/_file/interp_stream.py
@@ -26,6 +26,8 @@
         msg = os.strerror(errno)
     except ValueError:
         msg = 'error %d' % errno
+    if w_filename is None:
+        w_filename = space.w_None
     w_error = space.call_function(space.w_IOError,
                                   space.wrap(errno),
                                   space.wrap(msg),

diff --git a/pypy/module/imp/test/test_import.py b/pypy/module/imp/test/test_import.py
--- a/pypy/module/imp/test/test_import.py
+++ b/pypy/module/imp/test/test_import.py
@@ -975,6 +975,13 @@
             sys.meta_path.pop()
             sys.path_hooks.pop()
 
+    def test_run_compiled_module(self):
+        # XXX minimal test only
+        import imp, new
+        module = new.module('foobar')
+        raises(IOError, imp._run_compiled_module,
+               'foobar', 'this_file_does_not_exist', None, module)
+
 class AppTestNoPycFile(object):
     spaceconfig = {
         "objspace.usepycfiles": False,

From commits-noreply at bitbucket.org  Mon Feb 14 20:30:21 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Mon, 14 Feb 2011 20:30:21 +0100 (CET)
Subject: [pypy-svn] pypy default: add a modifiable copy of test_sysconfig.py
Message-ID: <20110214193021.5E765282B90@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r41923:bf773b1d2ac9
Date: 2011-02-14 20:23 +0100
http://bitbucket.org/pypy/pypy/changeset/bf773b1d2ac9/

Log:	add a modifiable copy of test_sysconfig.py

diff --git a/lib-python/2.7.0/test/test_sysconfig.py b/lib-python/modified-2.7.0/test/test_sysconfig.py
copy from lib-python/2.7.0/test/test_sysconfig.py
copy to lib-python/modified-2.7.0/test/test_sysconfig.py

From commits-noreply at bitbucket.org  Mon Feb 14 20:30:22 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Mon, 14 Feb 2011 20:30:22 +0100 (CET)
Subject: [pypy-svn] pypy default: add a new 'pypy' installation scheme for
	sysconfig.py:	most notably, it fixes
Message-ID: <20110214193022.28062282B90@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r41924:a65b19d5c46f
Date: 2011-02-14 20:25 +0100
http://bitbucket.org/pypy/pypy/changeset/a65b19d5c46f/

Log:	add a new 'pypy' installation scheme for sysconfig.py: most notably,
	it fixes the place where to find the *.h files and fixes
	test_get_config_h_filename()

diff --git a/lib-python/modified-2.7.0/sysconfig.py b/lib-python/modified-2.7.0/sysconfig.py
--- a/lib-python/modified-2.7.0/sysconfig.py
+++ b/lib-python/modified-2.7.0/sysconfig.py
@@ -26,6 +26,16 @@
         'scripts': '{base}/bin',
         'data'   : '{base}',
         },
+    'pypy': {
+        'stdlib': '{base}/lib-python',
+        'platstdlib': '{base}/lib-python',
+        'purelib': '{base}/lib-python',
+        'platlib': '{base}/lib-python',
+        'include': '{base}/include',
+        'platinclude': '{base}/include',
+        'scripts': '{base}/bin',
+        'data'   : '{base}',
+        },
     'nt': {
         'stdlib': '{base}/Lib',
         'platstdlib': '{base}/Lib',
@@ -158,7 +168,9 @@
     return res
 
 def _get_default_scheme():
-    if os.name == 'posix':
+    if '__pypy__' in sys.builtin_module_names:
+        return 'pypy'
+    elif os.name == 'posix':
         # the default scheme for posix is posix_prefix
         return 'posix_prefix'
     return os.name

diff --git a/lib-python/modified-2.7.0/test/test_sysconfig.py b/lib-python/modified-2.7.0/test/test_sysconfig.py
--- a/lib-python/modified-2.7.0/test/test_sysconfig.py
+++ b/lib-python/modified-2.7.0/test/test_sysconfig.py
@@ -236,7 +236,7 @@
 
     def test_get_scheme_names(self):
         wanted = ('nt', 'nt_user', 'os2', 'os2_home', 'osx_framework_user',
-                  'posix_home', 'posix_prefix', 'posix_user')
+                  'posix_home', 'posix_prefix', 'posix_user', 'pypy')
         self.assertEquals(get_scheme_names(), wanted)
 
     def test_symlink(self):

From commits-noreply at bitbucket.org  Mon Feb 14 20:49:01 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Mon, 14 Feb 2011 20:49:01 +0100 (CET)
Subject: [pypy-svn] pypy default: a failing test
Message-ID: <20110214194901.E2480282B90@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r41925:d091e48c4355
Date: 2011-02-14 20:48 +0100
http://bitbucket.org/pypy/pypy/changeset/d091e48c4355/

Log:	a failing test

diff --git a/pypy/rpython/test/test_exception.py b/pypy/rpython/test/test_exception.py
--- a/pypy/rpython/test/test_exception.py
+++ b/pypy/rpython/test/test_exception.py
@@ -124,6 +124,20 @@
         res = self.interpret(f, [11])
         assert res == 11
 
+    def test_pop_inside_try(self):
+        py.test.skip('fixme!')
+        class MyException(Exception):
+            pass
+        def f(n):
+            lst = []
+            lst.append(n)
+            try:
+                return lst.pop()
+            except MyException:
+                return -1
+        res = self.interpret(f, [42])
+        assert res == 42
+
 
 class TestLLtype(BaseTestException, LLRtypeMixin):
     def test_cannot_raise_ll_exception(self):

From commits-noreply at bitbucket.org  Mon Feb 14 20:57:36 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Mon, 14 Feb 2011 20:57:36 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: dont catch more than intended
Message-ID: <20110214195736.35C6A282B90@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r41926:7882fb12b3d9
Date: 2011-02-14 20:57 +0100
http://bitbucket.org/pypy/pypy/changeset/7882fb12b3d9/

Log:	dont catch more than intended

diff --git a/pypy/jit/metainterp/optimizeopt/unroll.py b/pypy/jit/metainterp/optimizeopt/unroll.py
--- a/pypy/jit/metainterp/optimizeopt/unroll.py
+++ b/pypy/jit/metainterp/optimizeopt/unroll.py
@@ -664,6 +664,11 @@
                         try:
                             self.inline(sh.operations, sh.inputargs,
                                         op.getarglist())
+                        except InvalidLoop:
+                            debug_print("Inlining failed unexpectedly",
+                                        "jumping to preamble instead")
+                            self.emit_operation(op)
+                        else:
                             jumpop = self.optimizer.newoperations.pop()
                             assert jumpop.getopnum() == rop.JUMP
                             for guard in extra_guards:
@@ -672,10 +677,6 @@
                                 guard.setdescr(descr)
                                 self.emit_operation(guard)
                             self.optimizer.newoperations.append(jumpop)
-                        except InvalidLoop:
-                            debug_print("Inlining failed unexpectedly",
-                                        "jumping to preamble instead")
-                            self.emit_operation(op)
                         return
                 retraced_count = len(short)
                 if descr.failed_states:

From commits-noreply at bitbucket.org  Mon Feb 14 21:59:01 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon, 14 Feb 2011 21:59:01 +0100 (CET)
Subject: [pypy-svn] pypy default: Extract and test the functionality of
	getting an	importer.
Message-ID: <20110214205901.AA0DB282B90@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41927:ed3ce9c8044b
Date: 2011-02-14 19:53 +0100
http://bitbucket.org/pypy/pypy/changeset/ed3ce9c8044b/

Log:	Extract and test the functionality of getting an importer. Expose it
	to app-level under a pypy-specific name.

diff --git a/pypy/module/imp/__init__.py b/pypy/module/imp/__init__.py
--- a/pypy/module/imp/__init__.py
+++ b/pypy/module/imp/__init__.py
@@ -19,7 +19,8 @@
         'load_module':     'interp_imp.load_module',
         'load_source':     'interp_imp.load_source',
         'load_compiled':   'interp_imp.load_compiled',
-        '_run_compiled_module': 'interp_imp._run_compiled_module',
+        '_run_compiled_module': 'interp_imp._run_compiled_module',   # pypy
+        '_getimporter':    'importing._getimporter',                 # pypy
         #'run_module':      'interp_imp.run_module',
         'new_module':      'interp_imp.new_module',
         'init_builtin':    'interp_imp.init_builtin',

diff --git a/pypy/module/imp/test/test_import.py b/pypy/module/imp/test/test_import.py
--- a/pypy/module/imp/test/test_import.py
+++ b/pypy/module/imp/test/test_import.py
@@ -975,6 +975,12 @@
             sys.meta_path.pop()
             sys.path_hooks.pop()
 
+
+class AppTestPyPyExtension(object):
+    def setup_class(cls):
+        cls.space = gettestobjspace(usemodules=['imp', 'zipimport'])
+        cls.w_udir = cls.space.wrap(str(udir))
+
     def test_run_compiled_module(self):
         # XXX minimal test only
         import imp, new
@@ -982,6 +988,37 @@
         raises(IOError, imp._run_compiled_module,
                'foobar', 'this_file_does_not_exist', None, module)
 
+    def test_getimporter(self):
+        import imp, os
+        # an existing directory
+        importer = imp._getimporter(self.udir)
+        assert importer is None
+        # an existing file
+        path = os.path.join(self.udir, 'test_getimporter')
+        open(path, 'w').close()
+        importer = imp._getimporter(path)
+        assert isinstance(importer, imp.NullImporter)
+        # a non-existing path
+        path = os.path.join(self.udir, 'does_not_exist_at_all')
+        importer = imp._getimporter(path)
+        assert isinstance(importer, imp.NullImporter)
+        # a mostly-empty zip file
+        path = os.path.join(self.udir, 'test_getimporter.zip')
+        f = open(path, 'wb')
+        f.write('PK\x03\x04\n\x00\x00\x00\x00\x00P\x9eN>\x00\x00\x00\x00\x00'
+                '\x00\x00\x00\x00\x00\x00\x00\x05\x00\x15\x00emptyUT\t\x00'
+                '\x03wyYMwyYMUx\x04\x00\xf4\x01d\x00PK\x01\x02\x17\x03\n\x00'
+                '\x00\x00\x00\x00P\x9eN>\x00\x00\x00\x00\x00\x00\x00\x00\x00'
+                '\x00\x00\x00\x05\x00\r\x00\x00\x00\x00\x00\x00\x00\x00\x00'
+                '\xa4\x81\x00\x00\x00\x00emptyUT\x05\x00\x03wyYMUx\x00\x00PK'
+                '\x05\x06\x00\x00\x00\x00\x01\x00\x01\x00@\x00\x00\x008\x00'
+                '\x00\x00\x00\x00')
+        f.close()
+        importer = imp._getimporter(path)
+        import zipimport
+        assert isinstance(importer, zipimport.zipimporter)
+
+
 class AppTestNoPycFile(object):
     spaceconfig = {
         "objspace.usepycfiles": False,

diff --git a/pypy/module/imp/importing.py b/pypy/module/imp/importing.py
--- a/pypy/module/imp/importing.py
+++ b/pypy/module/imp/importing.py
@@ -291,7 +291,8 @@
         if space.is_true(w_loader):
             return w_loader
 
-def find_in_path_hooks(space, w_modulename, w_pathitem):
+def _getimporter(space, w_pathitem):
+    # the function 'imp._getimporter' is a pypy-only extension
     w_path_importer_cache = space.sys.get("path_importer_cache")
     w_importer = space.finditem(w_path_importer_cache, w_pathitem)
     if w_importer is None:
@@ -311,11 +312,15 @@
                 )
             except OperationError, e:
                 if e.match(space, space.w_ImportError):
-                    return
+                    return None
                 raise
         if space.is_true(w_importer):
             space.setitem(w_path_importer_cache, w_pathitem, w_importer)
-    if space.is_true(w_importer):
+    return w_importer
+
+def find_in_path_hooks(space, w_modulename, w_pathitem):
+    w_importer = _getimporter(space, w_pathitem)
+    if w_importer is not None and space.is_true(w_importer):
         w_loader = space.call_method(w_importer, "find_module", w_modulename)
         if space.is_true(w_loader):
             return w_loader

From commits-noreply at bitbucket.org  Mon Feb 14 21:59:02 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon, 14 Feb 2011 21:59:02 +0100 (CET)
Subject: [pypy-svn] pypy default: Support for "pypy somedir" or "pypy
	file.zip", using the hook
Message-ID: <20110214205902.BFDCD282B90@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41928:641b29ed1501
Date: 2011-02-14 20:02 +0100
http://bitbucket.org/pypy/pypy/changeset/641b29ed1501/

Log:	Support for "pypy somedir" or "pypy file.zip", using the hook
	imp._getimporter().

diff --git a/pypy/translator/goal/app_main.py b/pypy/translator/goal/app_main.py
--- a/pypy/translator/goal/app_main.py
+++ b/pypy/translator/goal/app_main.py
@@ -507,10 +507,8 @@
             success = run_toplevel(run_it)
         elif run_module:
             # handle the "-m" command
-            def run_it():
-                import runpy
-                runpy._run_module_as_main(sys.argv[0])
-            success = run_toplevel(run_it)
+            import runpy
+            success = run_toplevel(runpy._run_module_as_main, sys.argv[0])
         elif run_stdin:
             # handle the case where no command/filename/module is specified
             # on the command-line.
@@ -559,15 +557,26 @@
             # assume it's a pyc file only if its name says so.
             # CPython goes to great lengths to detect other cases
             # of pyc file format, but I think it's ok not to care.
+            import imp
             if IS_WINDOWS:
                 filename = filename.lower()
             if filename.endswith('.pyc') or filename.endswith('.pyo'):
-                import imp
-                success = run_toplevel(imp._run_compiled_module, '__main__',
-                                       sys.argv[0], None, mainmodule)
+                args = (imp._run_compiled_module, '__main__',
+                        sys.argv[0], None, mainmodule)
             else:
-                success = run_toplevel(execfile, sys.argv[0],
-                                       mainmodule.__dict__)
+                # maybe it's the name of a directory or a zip file
+                filename = sys.argv[0]
+                importer = imp._getimporter(filename)
+                if not isinstance(importer, imp.NullImporter):
+                    # yes.  put the filename in sys.path[0] and import
+                    # the module __main__
+                    import runpy
+                    sys.path.insert(0, filename)
+                    args = (runpy._run_module_as_main, '__main__')
+                else:
+                    # no.  That's the normal path, "pypy stuff.py".
+                    args = (execfile, filename, mainmodule.__dict__)
+            success = run_toplevel(*args)
 
     except SystemExit, e:
         status = e.code
@@ -637,9 +646,9 @@
         except OSError:
             return None
 
-    # add an emulator for this pypy-only function
+    # add an emulator for these pypy-only or 2.7-only functions
     # (for test_pyc_commandline_argument)
-    import imp
+    import imp, runpy
     def _run_compiled_module(modulename, filename, file, module):
         import os
         assert modulename == '__main__'
@@ -648,7 +657,20 @@
         assert file is None
         assert module.__name__ == '__main__'
         print 'in _run_compiled_module'
+    def _getimporter(path):
+        import os, imp
+        if os.path.isdir(path):
+            return None
+        else:
+            return imp.NullImporter()
+    def _run_module_as_main(module):
+        import sys, os
+        path = os.path.join(sys.path[0], module + '.py')
+        execfile(path, {'__name__': '__main__'})
+
     imp._run_compiled_module = _run_compiled_module
+    imp._getimporter = _getimporter
+    runpy._run_module_as_main = _run_module_as_main
 
     # stick the current sys.path into $PYTHONPATH, so that CPython still
     # finds its own extension modules :-/

diff --git a/pypy/translator/goal/test2/test_app_main.py b/pypy/translator/goal/test2/test_app_main.py
--- a/pypy/translator/goal/test2/test_app_main.py
+++ b/pypy/translator/goal/test2/test_app_main.py
@@ -14,9 +14,9 @@
 app_main = os.path.abspath(app_main)
 
 _counter = 0
-def _get_next_path():
+def _get_next_path(ext='.py'):
     global _counter
-    p = udir.join('demo_test_app_main_%d.py' % (_counter,))
+    p = udir.join('demo_test_app_main_%d%s' % (_counter, ext))
     _counter += 1
     return p
 
@@ -26,7 +26,7 @@
     # return relative path for testing purposes 
     return py.path.local().bestrelpath(p) 
 
-def getscriptpyc(space, source):
+def getscript_pyc(space, source):
     p = _get_next_path()
     p.write(str(py.code.Source(source)))
     w_dir = space.wrap(str(p.dirpath()))
@@ -45,6 +45,13 @@
     assert os.path.isfile(p)   # the .pyc file should have been created above
     return p
 
+def getscript_in_dir(source):
+    pdir = _get_next_path(ext='')
+    p = pdir.ensure(dir=1).join('__main__.py')
+    p.write(str(py.code.Source(source)))
+    # return relative path for testing purposes 
+    return py.path.local().bestrelpath(pdir)
+
 demo_script = getscript("""
     print 'hello'
     print 'Name:', __name__
@@ -699,11 +706,18 @@
         assert data.startswith("[''")
 
     def test_pyc_commandline_argument(self):
-        p = getscriptpyc(self.space, "print 6*7\n")
-        assert p.endswith('.pyc')
+        p = getscript_pyc(self.space, "print 6*7\n")
+        assert os.path.isfile(p) and p.endswith('.pyc')
         data = self.run(p)
         assert data == 'in _run_compiled_module\n'
 
+    def test_main_in_dir_commandline_argument(self):
+        p = getscript_in_dir('print 6*7\n')
+        data = self.run(p)
+        assert data == '42\n'
+        data = self.run(p + os.sep)
+        assert data == '42\n'
+
 
 class AppTestAppMain:
 

From commits-noreply at bitbucket.org  Mon Feb 14 21:59:03 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon, 14 Feb 2011 21:59:03 +0100 (CET)
Subject: [pypy-svn] pypy default: merge heads
Message-ID: <20110214205903.32D45282BAD@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41929:184ab1e60759
Date: 2011-02-14 21:58 +0100
http://bitbucket.org/pypy/pypy/changeset/184ab1e60759/

Log:	merge heads


From commits-noreply at bitbucket.org  Mon Feb 14 22:12:49 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Mon, 14 Feb 2011 22:12:49 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix ropeobject tests
Message-ID: <20110214211249.298BE282B90@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41930:0eac337ce516
Date: 2011-02-14 22:09 +0100
http://bitbucket.org/pypy/pypy/changeset/0eac337ce516/

Log:	Fix ropeobject tests

diff --git a/pypy/objspace/std/ropeobject.py b/pypy/objspace/std/ropeobject.py
--- a/pypy/objspace/std/ropeobject.py
+++ b/pypy/objspace/std/ropeobject.py
@@ -14,8 +14,10 @@
 from pypy.objspace.std.stringtype import wrapchar
 
 from pypy.rlib import rope
-from pypy.objspace.std.stringobject import mod__String_ANY as mod__Rope_ANY,\
-    _upper, _lower, DEFAULT_NOOP_TABLE
+from pypy.objspace.std.stringobject import (
+    mod__String_ANY as mod__Rope_ANY,
+    str_format__String as str_format__Rope,
+    _upper, _lower, DEFAULT_NOOP_TABLE)
 
 class W_RopeObject(W_Object):
     from pypy.objspace.std.stringtype import str_typedef as typedef

From commits-noreply at bitbucket.org  Mon Feb 14 22:12:49 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Mon, 14 Feb 2011 22:12:49 +0100 (CET)
Subject: [pypy-svn] pypy default: update TODO
Message-ID: <20110214211249.F3675282B90@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41931:299183fd9740
Date: 2011-02-14 22:12 +0100
http://bitbucket.org/pypy/pypy/changeset/299183fd9740/

Log:	update TODO

diff --git a/lib-python/TODO b/lib-python/TODO
--- a/lib-python/TODO
+++ b/lib-python/TODO
@@ -18,6 +18,10 @@
 
 - missing module pyexpat.errors
 
+- support for PYTHONIOENCODING, this needs a way to update file.encoding
+
+- implement format__Complex_ANY() in pypy/objspace/std/complexobject.py
+
 
 Medium tasks
 ------------
@@ -30,6 +34,12 @@
 
 - socket module has a couple of changes (including AF_TIPC packet range)
 
+- (test_lib2to3) When a "for" loop runs a generator function, if the loop is
+  exited before the end, the "finally" clause of the generator is not called
+  until the next gc collection.  In our case, in lib2to3/pytree.py,
+  WildcardPattern.match_seq() does not exhaust the generate_matches() generator,
+  and stderr is not restored.
+
 
 Longer tasks
 ------------

From commits-noreply at bitbucket.org  Mon Feb 14 22:12:50 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Mon, 14 Feb 2011 22:12:50 +0100 (CET)
Subject: [pypy-svn] pypy default: Merge heads
Message-ID: <20110214211250.C77F5282B90@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41932:45d6db31158e
Date: 2011-02-14 22:12 +0100
http://bitbucket.org/pypy/pypy/changeset/45d6db31158e/

Log:	Merge heads


From commits-noreply at bitbucket.org  Mon Feb 14 22:18:01 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Mon, 14 Feb 2011 22:18:01 +0100 (CET)
Subject: [pypy-svn] pypy default: Added a modifiable _threading_local.py
Message-ID: <20110214211801.B8C36282B90@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41933:edc00805777f
Date: 2011-02-14 16:16 -0500
http://bitbucket.org/pypy/pypy/changeset/edc00805777f/

Log:	Added a modifiable _threading_local.py

diff --git a/lib-python/2.7.0/_threading_local.py b/lib-python/modified-2.7.0/_threading_local.py
copy from lib-python/2.7.0/_threading_local.py
copy to lib-python/modified-2.7.0/_threading_local.py

From commits-noreply at bitbucket.org  Mon Feb 14 22:18:02 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Mon, 14 Feb 2011 22:18:02 +0100 (CET)
Subject: [pypy-svn] pypy default: Don't rely on unbound methods having the
	same identity	for builtin methods in _threading_local.
Message-ID: <20110214211802.4E7D6282B90@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r41934:34a6ced3ce53
Date: 2011-02-14 16:17 -0500
http://bitbucket.org/pypy/pypy/changeset/34a6ced3ce53/

Log:	Don't rely on unbound methods having the same identity for builtin
	methods in _threading_local.

diff --git a/lib-python/modified-2.7.0/_threading_local.py b/lib-python/modified-2.7.0/_threading_local.py
--- a/lib-python/modified-2.7.0/_threading_local.py
+++ b/lib-python/modified-2.7.0/_threading_local.py
@@ -155,7 +155,7 @@
         object.__setattr__(self, '_local__args', (args, kw))
         object.__setattr__(self, '_local__lock', RLock())
 
-        if (args or kw) and (cls.__init__ is object.__init__):
+        if (args or kw) and (cls.__init__ == object.__init__):
             raise TypeError("Initialization arguments are not supported")
 
         # We need to create the thread dict in anticipation of

From commits-noreply at bitbucket.org  Mon Feb 14 22:21:56 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon, 14 Feb 2011 22:21:56 +0100 (CET)
Subject: [pypy-svn] pypy default: Rewrite this with an assert so that the
	annotator knows
Message-ID: <20110214212156.AF1582A2002@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41935:df5c99fc3f20
Date: 2011-02-14 22:20 +0100
http://bitbucket.org/pypy/pypy/changeset/df5c99fc3f20/

Log:	Rewrite this with an assert so that the annotator knows it cannot be
	None. Thanks amaury for the general idea.

diff --git a/pypy/objspace/std/newformat.py b/pypy/objspace/std/newformat.py
--- a/pypy/objspace/std/newformat.py
+++ b/pypy/objspace/std/newformat.py
@@ -717,8 +717,11 @@
             out.append_multiple_char(fill_char[0], spec.n_spadding)
         if spec.n_digits != 0:
             if self._loc_thousands:
-                digits = grouped_digits if grouped_digits is not None \
-                         else self._grouped_digits
+                if grouped_digits is not None:
+                    digits = grouped_digits
+                else:
+                    digits = self._grouped_digits
+                    assert digits is not None
             else:
                 stop = to_digits + spec.n_digits
                 assert stop >= 0

From commits-noreply at bitbucket.org  Mon Feb 14 22:30:56 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Mon, 14 Feb 2011 22:30:56 +0100 (CET)
Subject: [pypy-svn] pypy default: Add a paragraph about identity of bound
	builtin methods.
Message-ID: <20110214213056.7576C282B90@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41936:8aaa7efc505b
Date: 2011-02-14 22:26 +0100
http://bitbucket.org/pypy/pypy/changeset/8aaa7efc505b/

Log:	Add a paragraph about identity of bound builtin methods.

diff --git a/lib-python/TODO b/lib-python/TODO
--- a/lib-python/TODO
+++ b/lib-python/TODO
@@ -91,3 +91,10 @@
   to when parsing for a given target function, which occurs later).
 
   * Some "..." were added to doctests in test_extcall.py
+
+- CPython's builtin methods are both functions and unbound methods (for
+  example, `str.upper is dict(str.__dict__)['upper']`). This is not the case
+  in pypy, and assertions like `object.__str__ is object.__str__` are False
+  with pypy.  Use the `==` operator instead.
+
+  * pprint.py, _threading_local.py

From commits-noreply at bitbucket.org  Mon Feb 14 23:02:12 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon, 14 Feb 2011 23:02:12 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix a few random occasional
	failures:	StringBuilder.append(s)
Message-ID: <20110214220212.674462A202D@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41937:e6ea0c95851b
Date: 2011-02-14 23:01 +0100
http://bitbucket.org/pypy/pypy/changeset/e6ea0c95851b/

Log:	Fix a few random occasional failures: StringBuilder.append(s) used
	to crash occasionally during annotation if s is a str-or-None.

	Lenient fix; as opposed to a strict fix, which would be to check
	that the annotation says SomeString-and-not-None.

diff --git a/pypy/rlib/rstring.py b/pypy/rlib/rstring.py
--- a/pypy/rlib/rstring.py
+++ b/pypy/rlib/rstring.py
@@ -111,11 +111,13 @@
 
 class SomeStringBuilder(SomeObject):
     def method_append(self, s_str):
-        assert isinstance(s_str, (SomeString, SomeChar))
+        if s_str != s_None:
+            assert isinstance(s_str, (SomeString, SomeChar))
         return s_None
 
     def method_append_slice(self, s_str, s_start, s_end):
-        assert isinstance(s_str, SomeString)
+        if s_str != s_None:
+            assert isinstance(s_str, SomeString)
         assert isinstance(s_start, SomeInteger)
         assert isinstance(s_end, SomeInteger)
         return s_None
@@ -136,11 +138,13 @@
 
 class SomeUnicodeBuilder(SomeObject):
     def method_append(self, s_str):
-        assert isinstance(s_str, (SomeUnicodeCodePoint, SomeUnicodeString))
+        if s_str != s_None:
+            assert isinstance(s_str, (SomeUnicodeCodePoint, SomeUnicodeString))
         return s_None
 
     def method_append_slice(self, s_str, s_start, s_end):
-        assert isinstance(s_str, SomeUnicodeString)
+        if s_str != s_None:
+            assert isinstance(s_str, SomeUnicodeString)
         assert isinstance(s_start, SomeInteger)
         assert isinstance(s_end, SomeInteger)
         return s_None

From commits-noreply at bitbucket.org  Tue Feb 15 00:18:17 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Tue, 15 Feb 2011 00:18:17 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix the termios module: now RPython termios
Message-ID: <20110214231817.DBA4A282B90@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41938:7e5a11ab5022
Date: 2011-02-15 00:16 +0100
http://bitbucket.org/pypy/pypy/changeset/7e5a11ab5022/

Log:	Fix the termios module: now RPython termios raises OSError to better
	convey the errno attribute. Also fix a failure with readline when
	stdin is not a tty.

diff --git a/pypy/rpython/module/test/test_ll_termios.py b/pypy/rpython/module/test/test_ll_termios.py
--- a/pypy/rpython/module/test/test_ll_termios.py
+++ b/pypy/rpython/module/test/test_ll_termios.py
@@ -26,17 +26,22 @@
         from pypy.translator.c.test.test_genc import compile
         from pypy.rpython.module import ll_termios
         from pypy.rlib import rtermios
+        import os, errno
         import termios
-        def runs_tcgetattr():
+        def runs_tcgetattr(fd):
             try:
-                rtermios.tcgetattr(338)
-            except termios.error, e:
-                return 2
-            return 3
+                rtermios.tcgetattr(fd)
+            except OSError, e:
+                return e.errno
+            return 0
 
-        fn = compile(runs_tcgetattr, [], backendopt=False)
-        res = fn()
-        assert res == 2
+        fn = compile(runs_tcgetattr, [int], backendopt=False)
+        fd = os.open('.', 0)
+        try:
+            res = fn(fd)
+            assert res == errno.ENOTTY
+        finally:
+            os.close(fd)
 
     def test_tcsetattr(self):
         # a test, which doesn't even check anything.

diff --git a/pypy/rlib/rtermios.py b/pypy/rlib/rtermios.py
--- a/pypy/rlib/rtermios.py
+++ b/pypy/rlib/rtermios.py
@@ -8,7 +8,10 @@
 
 def tcgetattr(fd):
     # NOT_RPYTHON
-    lst = list(termios.tcgetattr(fd))
+    try:
+        lst = list(termios.tcgetattr(fd))
+    except termios.error, e:
+        raise OSError(*e.args)
     cc = lst[-1]
     next_cc = []
     for c in cc:
@@ -29,4 +32,7 @@
         cc[termios.VMIN] = ord(cc[termios.VMIN])
         cc[termios.VTIME] = ord(cc[termios.VTIME])
         mode_l[-1] = cc
-    return termios.tcsetattr(fd, when, mode_l)
+    try:
+        return termios.tcsetattr(fd, when, mode_l)
+    except termios.error, e:
+        raise OSError(*e.args)

diff --git a/pypy/module/termios/interp_termios.py b/pypy/module/termios/interp_termios.py
--- a/pypy/module/termios/interp_termios.py
+++ b/pypy/module/termios/interp_termios.py
@@ -4,7 +4,7 @@
 """
 
 from pypy.interpreter.baseobjspace import ObjSpace, W_Root
-from pypy.interpreter.error import OperationError
+from pypy.interpreter.error import OperationError, wrap_oserror
 from pypy.rpython.module import ll_termios
 from pypy.rlib.objectmodel import we_are_translated
 import os
@@ -16,16 +16,9 @@
 # some details what is missing in RPython modules though
 
 def convert_error(space, error):
-    errno = error.errno
     w_module = space.getbuiltinmodule('termios')
     w_exception_class = space.getattr(w_module, space.wrap('error'))
-    try:
-        msg = os.strerror(errno)
-    except ValueError:
-        msg = 'error %d' % errno
-    w_exception = space.call_function(w_exception_class, space.wrap(errno),
-                                      space.wrap(msg))
-    return OperationError(w_exception_class, w_exception)
+    return wrap_oserror(space, error, w_exception_class=w_exception_class)
 
 def tcsetattr(space, fd, when, w_attributes):
     w_iflag, w_oflag, w_cflag, w_lflag, w_ispeed, w_ospeed, w_cc = \
@@ -44,16 +37,14 @@
            space.int_w(w_ispeed), space.int_w(w_ospeed), cc)
     try:
         rtermios.tcsetattr(fd, when, tup)
-    except termios.error, e:
-        e.errno = e.args[0]
+    except OSError, e:
         raise convert_error(space, e)
 tcsetattr.unwrap_spec = [ObjSpace, int, int, W_Root]
 
 def tcgetattr(space, fd):
     try:
         tup = rtermios.tcgetattr(fd)
-    except termios.error, e:
-        e.errno = e.args[0]
+    except OSError, e:
         raise convert_error(space, e)
     iflag, oflag, cflag, lflag, ispeed, ospeed, cc = tup
     l_w = [space.wrap(i) for i in [iflag, oflag, cflag, lflag, ispeed, ospeed]]
@@ -70,31 +61,27 @@
 def tcsendbreak(space, fd, duration):
     try:
         termios.tcsendbreak(fd, duration)
-    except termios.error, e:
-        e.errno = e.args[0]
+    except OSError, e:
         raise convert_error(space, e)
 tcsendbreak.unwrap_spec = [ObjSpace, int, int]
 
 def tcdrain(space, fd):
     try:
         termios.tcdrain(fd)
-    except termios.error, e:
-        e.errno = e.args[0]
+    except OSError, e:
         raise convert_error(space, e)
 tcdrain.unwrap_spec = [ObjSpace, int]
 
 def tcflush(space, fd, queue):
     try:
         termios.tcflush(fd, queue)
-    except termios.error, e:
-        e.errno = e.args[0]
+    except OSError, e:
         raise convert_error(space, e)
 tcflush.unwrap_spec = [ObjSpace, int, int]
 
 def tcflow(space, fd, action):
     try:
         termios.tcflow(fd, action)
-    except termios.error, e:
-        e.errno = e.args[0]
+    except OSError, e:
         raise convert_error(space, e)
 tcflow.unwrap_spec = [ObjSpace, int, int]

diff --git a/pypy/rpython/module/ll_termios.py b/pypy/rpython/module/ll_termios.py
--- a/pypy/rpython/module/ll_termios.py
+++ b/pypy/rpython/module/ll_termios.py
@@ -13,7 +13,7 @@
 from pypy.rpython.extregistry import ExtRegistryEntry
 from pypy.annotation import model as annmodel
 from pypy.rpython import rclass
-from pypy.rlib import rtermios
+from pypy.rlib import rtermios, rposix
 from pypy.rpython.tool import rffi_platform
 from pypy.translator.tool.cbuild import ExternalCompilationInfo
 
@@ -32,11 +32,6 @@
 SPEED_T = rffi.UINT
 INT = rffi.INT
 
-def termios_error_init(self, num, msg):
-    self.args = (num, msg)
-
-termios.error.__init__ = termios_error_init
-
 TERMIOSP = rffi.CStructPtr('termios', ('c_iflag', TCFLAG_T), ('c_oflag', TCFLAG_T),
                            ('c_cflag', TCFLAG_T), ('c_lflag', TCFLAG_T),
                            ('c_cc', lltype.FixedSizeArray(CC_T, NCCS)))
@@ -54,18 +49,14 @@
 c_tcflush = c_external('tcflush', [INT, INT], INT)
 c_tcflow = c_external('tcflow', [INT, INT], INT)
 
-#class termios_error(termios.error):
-#    def __init__(self, num, msg):
-#        self.args = (num, msg)
-
 c_tcgetattr = c_external('tcgetattr', [INT, TERMIOSP], INT)
 
 def tcgetattr_llimpl(fd):
     c_struct = lltype.malloc(TERMIOSP.TO, flavor='raw')
-    error = c_tcgetattr(fd, c_struct)
+
     try:
-        if error == -1:
-            raise termios.error(error, 'tcgetattr failed')
+        if c_tcgetattr(fd, c_struct) < 0:
+            raise OSError(rposix.get_errno(), 'tcgetattr failed')
         cc = [chr(c_struct.c_c_cc[i]) for i in range(NCCS)]
         ispeed = c_cfgetispeed(c_struct)
         ospeed = c_cfgetospeed(c_struct)
@@ -86,15 +77,12 @@
     try:
         for i in range(NCCS):
             c_struct.c_c_cc[i] = rffi.r_uchar(ord(cc[i][0]))
-        error = c_cfsetispeed(c_struct, ispeed)
-        if error == -1:
-            raise termios.error(error, 'tcsetattr failed')
-        error = c_cfsetospeed(c_struct, ospeed)
-        if error == -1:
-            raise termios.error(error, 'tcsetattr failed')
-        error = c_tcsetattr(fd, when, c_struct)
-        if error == -1:
-            raise termios.error(error, 'tcsetattr failed')
+        if c_cfsetispeed(c_struct, ispeed) < 0:
+            raise OSError(rposix.get_errno(), 'tcsetattr failed')
+        if c_cfsetospeed(c_struct, ospeed) < 0:
+            raise OSError(rposix.get_errno(), 'tcsetattr failed')
+        if c_tcsetattr(fd, when, c_struct) < 0:
+            raise OSError(rposix.get_errno(), 'tcsetattr failed')
     finally:
         lltype.free(c_struct, flavor='raw')
 
@@ -106,30 +94,26 @@
 # a bit C-c C-v code follows...
 
 def tcsendbreak_llimpl(fd, duration):
-    error = c_tcsendbreak(fd, duration)
-    if error == -1:
-        raise termios.error(error, 'tcsendbreak failed')
+    if c_tcsendbreak(fd, duration):
+        raise OSError(rposix.get_errno(), 'tcsendbreak failed')
 register_external(termios.tcsendbreak, [int, int],
                   llimpl=tcsendbreak_llimpl,
                   export_name='termios.tcsendbreak')
 
 def tcdrain_llimpl(fd):
-    error = c_tcdrain(fd)
-    if error == -1:
-        raise termios.error(error, 'tcdrain failed')
+    if c_tcdrain(fd) < 0:
+        raise OSError(rposix.get_errno(), 'tcdrain failed')
 register_external(termios.tcdrain, [int], llimpl=tcdrain_llimpl,
                   export_name='termios.tcdrain')
 
 def tcflush_llimpl(fd, queue_selector):
-    error = c_tcflush(fd, queue_selector)
-    if error == -1:
-        raise termios.error(error, 'tcflush failed')
+    if c_tcflush(fd, queue_selector) < 0:
+        raise OSError(rposix.get_errno(), 'tcflush failed')
 register_external(termios.tcflush, [int, int], llimpl=tcflush_llimpl,
                   export_name='termios.tcflush')
 
 def tcflow_llimpl(fd, action):
-    error = c_tcflow(fd, action)
-    if error == -1:
-        raise termios.error(error, 'tcflow failed')
+    if c_tcflow(fd, action) < 0:
+        raise OSError(rposix.get_errno(), 'tcflow failed')
 register_external(termios.tcflow, [int, int], llimpl=tcflow_llimpl,
                   export_name='termios.tcflow')

diff --git a/.hgsubstate b/.hgsubstate
--- a/.hgsubstate
+++ b/.hgsubstate
@@ -1,3 +1,3 @@
 80037 greenlet
-80037 lib_pypy/pyrepl
+80345 lib_pypy/pyrepl
 80037 testrunner

diff --git a/pypy/module/termios/test/test_termios.py b/pypy/module/termios/test/test_termios.py
--- a/pypy/module/termios/test/test_termios.py
+++ b/pypy/module/termios/test/test_termios.py
@@ -124,10 +124,14 @@
         assert d == self.orig_module_dict
 
     def test_error(self):
-        # XXX not always true, but good assumption
-        import termios
-        raises(termios.error, "termios.tcgetattr(334)")
-        
+        import termios, errno, os
+        fd = os.open('.', 0)
+        try:
+            exc = raises(termios.error, termios.tcgetattr, fd)
+            assert exc.value.args[0] == errno.ENOTTY
+        finally:
+            os.close(fd)
+
     def test_error_tcsetattr(self):
         import termios
         raises(ValueError, termios.tcsetattr, 0, 1, (1, 2))

From commits-noreply at bitbucket.org  Tue Feb 15 09:58:50 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 15 Feb 2011 09:58:50 +0100 (CET)
Subject: [pypy-svn] pypy default: Hum. Sorry.
Message-ID: <20110215085850.2212A282BEC@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41939:1de07618270a
Date: 2011-02-15 09:39 +0100
http://bitbucket.org/pypy/pypy/changeset/1de07618270a/

Log:	Hum. Sorry.

diff --git a/pypy/translator/goal/app_main.py b/pypy/translator/goal/app_main.py
--- a/pypy/translator/goal/app_main.py
+++ b/pypy/translator/goal/app_main.py
@@ -662,7 +662,7 @@
         if os.path.isdir(path):
             return None
         else:
-            return imp.NullImporter()
+            return imp.NullImporter(path)
     def _run_module_as_main(module):
         import sys, os
         path = os.path.join(sys.path[0], module + '.py')

From commits-noreply at bitbucket.org  Tue Feb 15 09:58:50 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 15 Feb 2011 09:58:50 +0100 (CET)
Subject: [pypy-svn] pypy default: Just skip the test if no
	_run_module_as_main is	available in cpython's runpy.
Message-ID: <20110215085850.DF7B9282BEC@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41940:de9f6aee5d6f
Date: 2011-02-15 09:52 +0100
http://bitbucket.org/pypy/pypy/changeset/de9f6aee5d6f/

Log:	Just skip the test if no _run_module_as_main is available in
	cpython's runpy.

diff --git a/pypy/translator/goal/app_main.py b/pypy/translator/goal/app_main.py
--- a/pypy/translator/goal/app_main.py
+++ b/pypy/translator/goal/app_main.py
@@ -663,14 +663,9 @@
             return None
         else:
             return imp.NullImporter(path)
-    def _run_module_as_main(module):
-        import sys, os
-        path = os.path.join(sys.path[0], module + '.py')
-        execfile(path, {'__name__': '__main__'})
 
     imp._run_compiled_module = _run_compiled_module
     imp._getimporter = _getimporter
-    runpy._run_module_as_main = _run_module_as_main
 
     # stick the current sys.path into $PYTHONPATH, so that CPython still
     # finds its own extension modules :-/

diff --git a/pypy/translator/goal/test2/test_app_main.py b/pypy/translator/goal/test2/test_app_main.py
--- a/pypy/translator/goal/test2/test_app_main.py
+++ b/pypy/translator/goal/test2/test_app_main.py
@@ -3,7 +3,7 @@
 """
 from __future__ import with_statement
 import py
-import sys, os, re
+import sys, os, re, runpy
 import autopath
 from pypy.tool.udir import udir
 from contextlib import contextmanager
@@ -415,6 +415,8 @@
     def test_options_i_m(self):
         if sys.platform == "win32":
             skip("close_fds is not supported on Windows platforms")
+        if not hasattr(runpy, '_run_module_as_main'):
+            skip("requires CPython >= 2.6")
         p = os.path.join(autopath.this_dir, 'mymodule.py')
         p = os.path.abspath(p)
         child = self.spawn(['-i',

From commits-noreply at bitbucket.org  Tue Feb 15 09:58:51 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 15 Feb 2011 09:58:51 +0100 (CET)
Subject: [pypy-svn] pypy default: Skip a few more tests with CPython < 2.6.
Message-ID: <20110215085851.61854282BEC@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41941:bb59b8211a7e
Date: 2011-02-15 09:57 +0100
http://bitbucket.org/pypy/pypy/changeset/bb59b8211a7e/

Log:	Skip a few more tests with CPython < 2.6.

diff --git a/pypy/translator/goal/test2/test_app_main.py b/pypy/translator/goal/test2/test_app_main.py
--- a/pypy/translator/goal/test2/test_app_main.py
+++ b/pypy/translator/goal/test2/test_app_main.py
@@ -517,6 +517,8 @@
         child.expect('A five ounce bird could not carry a one pound coconut.')
 
     def test_no_space_before_argument(self):
+        if not hasattr(runpy, '_run_module_as_main'):
+            skip("requires CPython >= 2.6")
         child = self.spawn(['-cprint "hel" + "lo"'])
         child.expect('hello')
 
@@ -614,6 +616,8 @@
             os.environ['PYTHONWARNINGS'] = old
 
     def test_option_m(self):
+        if not hasattr(runpy, '_run_module_as_main'):
+            skip("requires CPython >= 2.6")
         p = os.path.join(autopath.this_dir, 'mymodule.py')
         p = os.path.abspath(p)
         data = self.run('-m pypy.translator.goal.test2.mymodule extra')
@@ -714,6 +718,8 @@
         assert data == 'in _run_compiled_module\n'
 
     def test_main_in_dir_commandline_argument(self):
+        if not hasattr(runpy, '_run_module_as_main'):
+            skip("requires CPython >= 2.6")
         p = getscript_in_dir('print 6*7\n')
         data = self.run(p)
         assert data == '42\n'

From commits-noreply at bitbucket.org  Tue Feb 15 10:27:05 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Tue, 15 Feb 2011 10:27:05 +0100 (CET)
Subject: [pypy-svn] pypy default: always create the stdout and stderr files
	when running	py.test -s,
	else we get an file not found error few lines later
Message-ID: <20110215092705.D921D282BEC@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r41942:3845990a1a7f
Date: 2011-02-15 10:25 +0100
http://bitbucket.org/pypy/pypy/changeset/3845990a1a7f/

Log:	always create the stdout and stderr files when running py.test -s,
	else we get an file not found error few lines later

diff --git a/lib-python/conftest.py b/lib-python/conftest.py
--- a/lib-python/conftest.py
+++ b/lib-python/conftest.py
@@ -696,6 +696,8 @@
                 cmd += ' --pdb'
             if self.config.option.capture == 'no':
                 status = os.system(cmd)
+                stdout.write('')
+                stderr.write('')
             else:
                 status = os.system("%s >>%s 2>>%s" %(cmd, stdout, stderr))
             if os.WIFEXITED(status):

From commits-noreply at bitbucket.org  Tue Feb 15 10:31:33 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Tue, 15 Feb 2011 10:31:33 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix failure in test_ropeunicodeobject.py
Message-ID: <20110215093133.79A1C282BEC@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41943:a121b17d6fea
Date: 2011-02-15 10:19 +0100
http://bitbucket.org/pypy/pypy/changeset/a121b17d6fea/

Log:	Fix failure in test_ropeunicodeobject.py

diff --git a/pypy/objspace/std/ropeunicodeobject.py b/pypy/objspace/std/ropeunicodeobject.py
--- a/pypy/objspace/std/ropeunicodeobject.py
+++ b/pypy/objspace/std/ropeunicodeobject.py
@@ -17,6 +17,9 @@
 
 from pypy.objspace.std.formatting import mod_format
 
+from pypy.objspace.std.unicodeobject import (
+    format__Unicode_ANY as format__RopeUnicode_ANY)
+
 
 def wrapunicode(space, uni):
     return W_RopeUnicodeObject(rope.rope_from_unicode(uni))

From commits-noreply at bitbucket.org  Tue Feb 15 10:59:24 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 15 Feb 2011 10:59:24 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix the test in test_exception.py by
	adding the missing	'can_only_throw'
Message-ID: <20110215095924.A8A3D282BEC@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41944:523583dfb09e
Date: 2011-02-15 10:59 +0100
http://bitbucket.org/pypy/pypy/changeset/523583dfb09e/

Log:	Fix the test in test_exception.py by adding the missing
	'can_only_throw' attribute to method_pop(). Similar issues can be
	fixed by adding more 'can_only_throw' to the other method_xxxs().

diff --git a/pypy/annotation/unaryop.py b/pypy/annotation/unaryop.py
--- a/pypy/annotation/unaryop.py
+++ b/pypy/annotation/unaryop.py
@@ -303,6 +303,7 @@
     def method_pop(lst, s_index=None):
         lst.listdef.resize()
         return lst.listdef.read_item()
+    method_pop.can_only_throw = [IndexError]
 
     def method_index(lst, s_value):
         getbookkeeper().count("list_index")
@@ -642,6 +643,11 @@
 
 
 class __extend__(SomeBuiltin):
+    def _can_only_throw(bltn, *args):
+        analyser_func = getattr(bltn.analyser, 'im_func', None)
+        can_only_throw = getattr(analyser_func, 'can_only_throw', None)
+        return can_only_throw   # or None to mean "anything"
+
     def simple_call(bltn, *args):
         if bltn.s_self is not None:
             return bltn.analyser(bltn.s_self, *args)
@@ -649,6 +655,7 @@
             if bltn.methodname:
                 getbookkeeper().count(bltn.methodname.replace('.', '_'), *args)
             return bltn.analyser(*args)
+    simple_call.can_only_throw = _can_only_throw
 
     def call(bltn, args, implicit_init=False):
         args_s, kwds = args.unpack()

diff --git a/.hgsubstate b/.hgsubstate
--- a/.hgsubstate
+++ b/.hgsubstate
@@ -1,3 +1,3 @@
 80037 greenlet
-80345 lib_pypy/pyrepl
+80037 lib_pypy/pyrepl
 80037 testrunner

diff --git a/pypy/rpython/test/test_exception.py b/pypy/rpython/test/test_exception.py
--- a/pypy/rpython/test/test_exception.py
+++ b/pypy/rpython/test/test_exception.py
@@ -125,7 +125,6 @@
         assert res == 11
 
     def test_pop_inside_try(self):
-        py.test.skip('fixme!')
         class MyException(Exception):
             pass
         def f(n):

From commits-noreply at bitbucket.org  Tue Feb 15 11:03:51 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 15 Feb 2011 11:03:51 +0100 (CET)
Subject: [pypy-svn] pypy default: Uh? Revert this part of the previous
	checkin. It was	not intended.
Message-ID: <20110215100351.C6546282BEC@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41945:8f85ffcdcff2
Date: 2011-02-15 11:03 +0100
http://bitbucket.org/pypy/pypy/changeset/8f85ffcdcff2/

Log:	Uh? Revert this part of the previous checkin. It was not intended.


From commits-noreply at bitbucket.org  Tue Feb 15 11:09:55 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 15 Feb 2011 11:09:55 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix: it seems that test_rlist.py shows a
	case in which	can_only_throw
Message-ID: <20110215100955.C96F4282BEC@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41946:ec76a8c2303d
Date: 2011-02-15 11:09 +0100
http://bitbucket.org/pypy/pypy/changeset/ec76a8c2303d/

Log:	Fix: it seems that test_rlist.py shows a case in which
	can_only_throw here can again be a callable.

diff --git a/pypy/annotation/unaryop.py b/pypy/annotation/unaryop.py
--- a/pypy/annotation/unaryop.py
+++ b/pypy/annotation/unaryop.py
@@ -646,7 +646,12 @@
     def _can_only_throw(bltn, *args):
         analyser_func = getattr(bltn.analyser, 'im_func', None)
         can_only_throw = getattr(analyser_func, 'can_only_throw', None)
-        return can_only_throw   # or None to mean "anything"
+        if can_only_throw is None or isinstance(can_only_throw, list):
+            return can_only_throw
+        if bltn.s_self is not None:
+            return can_only_throw(bltn.s_self, *args)
+        else:
+            return can_only_throw(*args)
 
     def simple_call(bltn, *args):
         if bltn.s_self is not None:

From commits-noreply at bitbucket.org  Tue Feb 15 11:58:40 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Tue, 15 Feb 2011 11:58:40 +0100 (CET)
Subject: [pypy-svn] pypy default: Add modifiable copy of test_pydoc.py
Message-ID: <20110215105840.D79CE282BEC@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41947:b2f4de1d9ac7
Date: 2011-02-15 11:14 +0100
http://bitbucket.org/pypy/pypy/changeset/b2f4de1d9ac7/

Log:	Add modifiable copy of test_pydoc.py

diff --git a/lib-python/2.7.0/test/test_pydoc.py b/lib-python/modified-2.7.0/test/test_pydoc.py
copy from lib-python/2.7.0/test/test_pydoc.py
copy to lib-python/modified-2.7.0/test/test_pydoc.py

From commits-noreply at bitbucket.org  Tue Feb 15 11:58:41 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Tue, 15 Feb 2011 11:58:41 +0100 (CET)
Subject: [pypy-svn] pypy default: loosen the test a little,
	and only check that the missing component is mentioned in
	the	ImportError message
Message-ID: <20110215105841.62571282BEC@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41948:3501a4c13d85
Date: 2011-02-15 11:48 +0100
http://bitbucket.org/pypy/pypy/changeset/3501a4c13d85/

Log:	loosen the test a little, and only check that the missing component
	is mentioned in the ImportError message

diff --git a/lib-python/modified-2.7.0/test/test_pydoc.py b/lib-python/modified-2.7.0/test/test_pydoc.py
--- a/lib-python/modified-2.7.0/test/test_pydoc.py
+++ b/lib-python/modified-2.7.0/test/test_pydoc.py
@@ -265,8 +265,8 @@
         testpairs = (
             ('i_am_not_here', 'i_am_not_here'),
             ('test.i_am_not_here_either', 'i_am_not_here_either'),
-            ('test.i_am_not_here.neither_am_i', 'i_am_not_here.neither_am_i'),
-            ('i_am_not_here.{}'.format(modname), 'i_am_not_here.{}'.format(modname)),
+            ('test.i_am_not_here.neither_am_i', 'i_am_not_here'),
+            ('i_am_not_here.{}'.format(modname), 'i_am_not_here'),
             ('test.{}'.format(modname), modname),
             )
 
@@ -290,8 +290,8 @@
                     result = run_pydoc(modname)
                 finally:
                     forget(modname)
-                expected = badimport_pattern % (modname, expectedinmsg)
-                self.assertEqual(expected, result)
+                expected = badimport_pattern % (modname, '(.+\\.)?' + expectedinmsg + '(\\..+)?$')
+                self.assertTrue(re.match(expected, result))
 
     def test_input_strip(self):
         missing_module = " test.i_am_not_here "

From commits-noreply at bitbucket.org  Tue Feb 15 12:52:42 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 15 Feb 2011 12:52:42 +0100 (CET)
Subject: [pypy-svn] pypy default: Another try...
Message-ID: <20110215115242.658E736C053@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41949:b294bc61eec6
Date: 2011-02-15 12:50 +0100
http://bitbucket.org/pypy/pypy/changeset/b294bc61eec6/

Log:	Another try...

diff --git a/.hgsubstate b/.hgsubstate
--- a/.hgsubstate
+++ b/.hgsubstate
@@ -1,3 +1,3 @@
 80037 greenlet
-80037 lib_pypy/pyrepl
+80345 lib_pypy/pyrepl
 80037 testrunner

From commits-noreply at bitbucket.org  Tue Feb 15 12:52:42 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 15 Feb 2011 12:52:42 +0100 (CET)
Subject: [pypy-svn] pypy default: merge heads
Message-ID: <20110215115242.A0B4736C21D@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41950:c31fe423979e
Date: 2011-02-15 12:52 +0100
http://bitbucket.org/pypy/pypy/changeset/c31fe423979e/

Log:	merge heads


From commits-noreply at bitbucket.org  Tue Feb 15 13:41:58 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Tue, 15 Feb 2011 13:41:58 +0100 (CET)
Subject: [pypy-svn] pypy default: Update TODO list:
Message-ID: <20110215124158.B7743282BEC@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r41951:5abd0cfa317f
Date: 2011-02-15 13:41 +0100
http://bitbucket.org/pypy/pypy/changeset/5abd0cfa317f/

Log:	Update TODO list:

diff --git a/lib-python/TODO b/lib-python/TODO
--- a/lib-python/TODO
+++ b/lib-python/TODO
@@ -22,6 +22,14 @@
 
 - implement format__Complex_ANY() in pypy/objspace/std/complexobject.py
 
+- Code like this does not work, for two reasons::
+
+  \
+  from __future__ import (with_statement,
+                          unicode_literals)
+  assert type("") is unicode
+
+
 
 Medium tasks
 ------------
@@ -29,9 +37,6 @@
 - Ast objects should be picklable, see in pypy/module/_ast/test/test_ast.py:
   test_pickle()
 
-- add 'unicode' in ObjSpace.MethodTable + probably a default implementation that
-  falls back to space.str(). (WHY?)
-
 - socket module has a couple of changes (including AF_TIPC packet range)
 
 - (test_lib2to3) When a "for" loop runs a generator function, if the loop is
@@ -59,10 +64,6 @@
       a = A()
       a.__class__ = B
 
-- Remove "unbound builtin methods": some code in pprint.py, _threading_local.py
-  relies on comparisons like "str.__init__ is object.__init__", or
-  "type(x).__repr__ is dict.__repr__"
-
 - Show a ResourceWarning when a file/socket is not explicitely closed, like
   CPython did for 3.2: http://svn.python.org/view?view=rev&revision=85920
   in PyPy this should be enabled by default
@@ -98,3 +99,7 @@
   with pypy.  Use the `==` operator instead.
 
   * pprint.py, _threading_local.py
+
+- When importing a nested module fails, the ImportError message mentions the
+  name of the package up to the component that could not be imported (CPython
+  prefers to display the names starting with the failing part).

From commits-noreply at bitbucket.org  Tue Feb 15 14:51:33 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Tue, 15 Feb 2011 14:51:33 +0100 (CET)
Subject: [pypy-svn] pypy default: use a weakref to store the class,
	else we might keep it alive forever
Message-ID: <20110215135133.3EFA02A202D@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r41952:260697f1c3e6
Date: 2011-02-15 14:00 +0100
http://bitbucket.org/pypy/pypy/changeset/260697f1c3e6/

Log:	use a weakref to store the class, else we might keep it alive
	forever

diff --git a/pypy/objspace/std/mapdict.py b/pypy/objspace/std/mapdict.py
--- a/pypy/objspace/std/mapdict.py
+++ b/pypy/objspace/std/mapdict.py
@@ -1,3 +1,4 @@
+import weakref
 from pypy.rlib import jit, objectmodel, debug
 from pypy.rlib.rarithmetic import intmask, r_uint
 
@@ -162,7 +163,15 @@
 
     def __init__(self, space, w_cls):
         AbstractAttribute.__init__(self, space, self)
-        self.w_cls = w_cls
+        if w_cls is None:
+            self.w_cls_wref = None
+        else:
+            self.w_cls_wref = weakref.ref(w_cls)
+
+    def get_w_cls(self):
+        if self.w_cls_wref is None:
+            return None
+        return self.w_cls_wref()
 
     def _read_terminator(self, obj, selector):
         return None
@@ -396,7 +405,7 @@
         assert flag
 
     def getclass(self, space):
-        return self._get_mapdict_map().terminator.w_cls
+        return self._get_mapdict_map().terminator.get_w_cls()
 
     def setclass(self, space, w_cls):
         new_obj = self._get_mapdict_map().set_terminator(self, w_cls.terminator)
@@ -685,7 +694,10 @@
 
     def is_valid_for_map(self, map):
         if map is self.map:
-            version_tag = map.terminator.w_cls.version_tag()
+            w_cls = map.terminator.get_w_cls()
+            if w_cls is None: # probably a dead weakref
+                return False
+            version_tag = w_cls.version_tag()
             if version_tag is self.version_tag:
                 # everything matches, it's incredibly fast
                 if map.space.config.objspace.std.withmethodcachecounter:
@@ -730,7 +742,7 @@
     space = pycode.space
     w_name = pycode.co_names_w[nameindex]
     if map is not None:
-        w_type = map.terminator.w_cls
+        w_type = map.terminator.get_w_cls()
         w_descr = w_type.getattribute_if_not_from_object()
         if w_descr is not None:
             return space._handle_getattribute(w_descr, w_obj, w_name)

diff --git a/pypy/objspace/std/test/test_mapdict.py b/pypy/objspace/std/test/test_mapdict.py
--- a/pypy/objspace/std/test/test_mapdict.py
+++ b/pypy/objspace/std/test/test_mapdict.py
@@ -24,12 +24,14 @@
         hasdict = False
 
 def test_plain_attribute():
-    w_cls = "class"
+    class FakeClass(object):
+        pass
+    w_cls = FakeClass
     aa = PlainAttribute(("b", DICT),
                         PlainAttribute(("a", DICT),
                                        Terminator(space, w_cls)))
     assert aa.space is space
-    assert aa.terminator.w_cls is w_cls
+    assert aa.terminator.get_w_cls() is w_cls
     assert aa.get_terminator() is aa.terminator
 
     obj = Object()
@@ -866,6 +868,23 @@
         res = self.check(f, 'm')
         assert res == (0, 2, 1)
 
+    def test_dont_keep_class_alive(self):
+        import weakref
+        import gc
+        def f():
+            class C(object):
+                def m(self):
+                    pass
+            r = weakref.ref(C)
+            # Trigger cache.
+            C().m()
+            del C
+            gc.collect(); gc.collect(); gc.collect()
+            assert r() is None
+            return 42
+        f()
+
+
 class AppTestGlobalCaching(AppTestWithMapDict):
     def setup_class(cls):
         cls.space = gettestobjspace(

From commits-noreply at bitbucket.org  Tue Feb 15 15:40:19 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 15 Feb 2011 15:40:19 +0100 (CET)
Subject: [pypy-svn] pypy default: Change the test to keep track of all files
	opened, and close
Message-ID: <20110215144019.5FB5C282BEA@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41953:468752b873fc
Date: 2011-02-15 15:33 +0100
http://bitbucket.org/pypy/pypy/changeset/468752b873fc/

Log:	Change the test to keep track of all files opened, and close the
	older ones, with a buffer of 50. This avoids getting an infinite
	number of open files, hopefully.

diff --git a/lib-python/modified-2.7.0/test/test_file2k.py b/lib-python/modified-2.7.0/test/test_file2k.py
--- a/lib-python/modified-2.7.0/test/test_file2k.py
+++ b/lib-python/modified-2.7.0/test/test_file2k.py
@@ -450,6 +450,9 @@
         self.close_count = 0
         self.close_success_count = 0
         self.use_buffering = False
+        # to prevent running out of file descriptors on PyPy,
+        # we only keep the 50 most recent files open
+        self.all_files = [None] * 50
 
     def tearDown(self):
         if self.f:
@@ -465,9 +468,14 @@
 
     def _create_file(self):
         if self.use_buffering:
-            self.f = open(self.filename, "w+", buffering=1024*16)
+            f = open(self.filename, "w+", buffering=1024*16)
         else:
-            self.f = open(self.filename, "w+")
+            f = open(self.filename, "w+")
+        self.f = f
+        self.all_files.append(f)
+        oldf = self.all_files.pop(0)
+        if oldf is not None:
+            oldf.close()
 
     def _close_file(self):
         with self._count_lock:

From commits-noreply at bitbucket.org  Tue Feb 15 15:40:19 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 15 Feb 2011 15:40:19 +0100 (CET)
Subject: [pypy-svn] pypy default: merge heads
Message-ID: <20110215144019.EAE88282BEA@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41954:7794bdc70efb
Date: 2011-02-15 15:39 +0100
http://bitbucket.org/pypy/pypy/changeset/7794bdc70efb/

Log:	merge heads

diff --git a/.hgsubstate b/.hgsubstate
--- a/.hgsubstate
+++ b/.hgsubstate
@@ -1,3 +1,3 @@
 80037 greenlet
-80345 lib_pypy/pyrepl
+80037 lib_pypy/pyrepl
 80037 testrunner

From commits-noreply at bitbucket.org  Tue Feb 15 15:52:27 2011
From: commits-noreply at bitbucket.org (cfbolz)
Date: Tue, 15 Feb 2011 15:52:27 +0100 (CET)
Subject: [pypy-svn] pypy default: a test to check that a mapdict instance
	keeps its class	alive. fails currently.
Message-ID: <20110215145227.01DEF36C21E@codespeak.net>

Author: Carl Friedrich Bolz <cfbolz at gmx.de>
Branch: 
Changeset: r41955:db9bc5957db6
Date: 2011-02-15 15:48 +0100
http://bitbucket.org/pypy/pypy/changeset/db9bc5957db6/

Log:	a test to check that a mapdict instance keeps its class alive. fails
	currently.

diff --git a/pypy/objspace/std/test/test_mapdict.py b/pypy/objspace/std/test/test_mapdict.py
--- a/pypy/objspace/std/test/test_mapdict.py
+++ b/pypy/objspace/std/test/test_mapdict.py
@@ -884,6 +884,20 @@
             return 42
         f()
 
+    def test_instance_keeps_class_alive(self):
+        import weakref
+        import gc
+        def f():
+            class C(object):
+                def m(self):
+                    42
+            r = weakref.ref(C)
+            c = C()
+            del C
+            gc.collect(); gc.collect(); gc.collect()
+            return c.m()
+        val = f()
+        assert val == 42
 
 class AppTestGlobalCaching(AppTestWithMapDict):
     def setup_class(cls):

From commits-noreply at bitbucket.org  Tue Feb 15 15:52:27 2011
From: commits-noreply at bitbucket.org (cfbolz)
Date: Tue, 15 Feb 2011 15:52:27 +0100 (CET)
Subject: [pypy-svn] pypy default: merge heads
Message-ID: <20110215145227.5583636C21F@codespeak.net>

Author: Carl Friedrich Bolz <cfbolz at gmx.de>
Branch: 
Changeset: r41956:94abade1a137
Date: 2011-02-15 15:52 +0100
http://bitbucket.org/pypy/pypy/changeset/94abade1a137/

Log:	merge heads


From commits-noreply at bitbucket.org  Tue Feb 15 16:00:49 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Tue, 15 Feb 2011 16:00:49 +0100 (CET)
Subject: [pypy-svn] pypy default: revert 260697f1c3e6,
	I'll try another approach for it
Message-ID: <20110215150049.B3CCE282BEA@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r41957:b837e02736be
Date: 2011-02-15 15:16 +0100
http://bitbucket.org/pypy/pypy/changeset/b837e02736be/

Log:	revert 260697f1c3e6, I'll try another approach for it

diff --git a/pypy/objspace/std/mapdict.py b/pypy/objspace/std/mapdict.py
--- a/pypy/objspace/std/mapdict.py
+++ b/pypy/objspace/std/mapdict.py
@@ -1,4 +1,3 @@
-import weakref
 from pypy.rlib import jit, objectmodel, debug
 from pypy.rlib.rarithmetic import intmask, r_uint
 
@@ -163,15 +162,7 @@
 
     def __init__(self, space, w_cls):
         AbstractAttribute.__init__(self, space, self)
-        if w_cls is None:
-            self.w_cls_wref = None
-        else:
-            self.w_cls_wref = weakref.ref(w_cls)
-
-    def get_w_cls(self):
-        if self.w_cls_wref is None:
-            return None
-        return self.w_cls_wref()
+        self.w_cls = w_cls
 
     def _read_terminator(self, obj, selector):
         return None
@@ -405,7 +396,7 @@
         assert flag
 
     def getclass(self, space):
-        return self._get_mapdict_map().terminator.get_w_cls()
+        return self._get_mapdict_map().terminator.w_cls
 
     def setclass(self, space, w_cls):
         new_obj = self._get_mapdict_map().set_terminator(self, w_cls.terminator)
@@ -694,10 +685,7 @@
 
     def is_valid_for_map(self, map):
         if map is self.map:
-            w_cls = map.terminator.get_w_cls()
-            if w_cls is None: # probably a dead weakref
-                return False
-            version_tag = w_cls.version_tag()
+            version_tag = map.terminator.w_cls.version_tag()
             if version_tag is self.version_tag:
                 # everything matches, it's incredibly fast
                 if map.space.config.objspace.std.withmethodcachecounter:
@@ -742,7 +730,7 @@
     space = pycode.space
     w_name = pycode.co_names_w[nameindex]
     if map is not None:
-        w_type = map.terminator.get_w_cls()
+        w_type = map.terminator.w_cls
         w_descr = w_type.getattribute_if_not_from_object()
         if w_descr is not None:
             return space._handle_getattribute(w_descr, w_obj, w_name)

diff --git a/pypy/objspace/std/test/test_mapdict.py b/pypy/objspace/std/test/test_mapdict.py
--- a/pypy/objspace/std/test/test_mapdict.py
+++ b/pypy/objspace/std/test/test_mapdict.py
@@ -24,14 +24,12 @@
         hasdict = False
 
 def test_plain_attribute():
-    class FakeClass(object):
-        pass
-    w_cls = FakeClass
+    w_cls = "class"
     aa = PlainAttribute(("b", DICT),
                         PlainAttribute(("a", DICT),
                                        Terminator(space, w_cls)))
     assert aa.space is space
-    assert aa.terminator.get_w_cls() is w_cls
+    assert aa.terminator.w_cls is w_cls
     assert aa.get_terminator() is aa.terminator
 
     obj = Object()
@@ -868,23 +866,6 @@
         res = self.check(f, 'm')
         assert res == (0, 2, 1)
 
-    def test_dont_keep_class_alive(self):
-        import weakref
-        import gc
-        def f():
-            class C(object):
-                def m(self):
-                    pass
-            r = weakref.ref(C)
-            # Trigger cache.
-            C().m()
-            del C
-            gc.collect(); gc.collect(); gc.collect()
-            assert r() is None
-            return 42
-        f()
-
-
 class AppTestGlobalCaching(AppTestWithMapDict):
     def setup_class(cls):
         cls.space = gettestobjspace(

From commits-noreply at bitbucket.org  Tue Feb 15 16:00:50 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Tue, 15 Feb 2011 16:00:50 +0100 (CET)
Subject: [pypy-svn] pypy default: (arigo,
	antocuni): use a weakref to the map in the cache entry,
	else we indirectly store a strong reference to the class forever
Message-ID: <20110215150050.80758282BEA@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r41958:d17fde44288e
Date: 2011-02-15 15:25 +0100
http://bitbucket.org/pypy/pypy/changeset/d17fde44288e/

Log:	(arigo, antocuni): use a weakref to the map in the cache entry, else
	we indirectly store a strong reference to the class forever

diff --git a/pypy/objspace/std/mapdict.py b/pypy/objspace/std/mapdict.py
--- a/pypy/objspace/std/mapdict.py
+++ b/pypy/objspace/std/mapdict.py
@@ -1,3 +1,4 @@
+import weakref
 from pypy.rlib import jit, objectmodel, debug
 from pypy.rlib.rarithmetic import intmask, r_uint
 
@@ -672,7 +673,7 @@
 # Magic caching
 
 class CacheEntry(object):
-    map = None
+    map_wref = None
     version_tag = None
     index = 0
     w_method = None # for callmethod
@@ -684,7 +685,8 @@
         return self.is_valid_for_map(map)
 
     def is_valid_for_map(self, map):
-        if map is self.map:
+        mymap = self.map_wref()
+        if mymap is not None and mymap is map:
             version_tag = map.terminator.w_cls.version_tag()
             if version_tag is self.version_tag:
                 # everything matches, it's incredibly fast
@@ -693,11 +695,11 @@
                 return True
         return False
 
+_invalid_cache_entry_map = objectmodel.instantiate(AbstractAttribute)
+_invalid_cache_entry_map.terminator = None
 INVALID_CACHE_ENTRY = CacheEntry()
-INVALID_CACHE_ENTRY.map = objectmodel.instantiate(AbstractAttribute)
-                             # different from any real map ^^^
-INVALID_CACHE_ENTRY.map.terminator = None
-
+INVALID_CACHE_ENTRY.map_wref = weakref.ref(_invalid_cache_entry_map)
+                                 # different from any real map ^^^
 
 def init_mapdict_cache(pycode):
     num_entries = len(pycode.co_names_w)
@@ -708,7 +710,7 @@
     if entry is INVALID_CACHE_ENTRY:
         entry = CacheEntry()
         pycode._mapdict_caches[nameindex] = entry
-    entry.map = map
+    entry.map_wref = weakref.ref(map)
     entry.version_tag = version_tag
     entry.index = index
     entry.w_method = w_method

diff --git a/pypy/objspace/std/test/test_mapdict.py b/pypy/objspace/std/test/test_mapdict.py
--- a/pypy/objspace/std/test/test_mapdict.py
+++ b/pypy/objspace/std/test/test_mapdict.py
@@ -866,6 +866,22 @@
         res = self.check(f, 'm')
         assert res == (0, 2, 1)
 
+    def test_dont_keep_class_alive(self):
+        import weakref
+        import gc
+        def f():
+            class C(object):
+                def m(self):
+                    pass
+            r = weakref.ref(C)
+            # Trigger cache.
+            C().m()
+            del C
+            gc.collect(); gc.collect(); gc.collect()
+            assert r() is None
+            return 42
+        f() 
+
 class AppTestGlobalCaching(AppTestWithMapDict):
     def setup_class(cls):
         cls.space = gettestobjspace(

From commits-noreply at bitbucket.org  Tue Feb 15 16:00:50 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Tue, 15 Feb 2011 16:00:50 +0100 (CET)
Subject: [pypy-svn] pypy default: merge heads and fix the test,
	which now passes
Message-ID: <20110215150050.F3A2B282BEA@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r41959:1cb79451c7dd
Date: 2011-02-15 16:00 +0100
http://bitbucket.org/pypy/pypy/changeset/1cb79451c7dd/

Log:	merge heads and fix the test, which now passes

diff --git a/pypy/objspace/std/test/test_mapdict.py b/pypy/objspace/std/test/test_mapdict.py
--- a/pypy/objspace/std/test/test_mapdict.py
+++ b/pypy/objspace/std/test/test_mapdict.py
@@ -24,14 +24,12 @@
         hasdict = False
 
 def test_plain_attribute():
-    class FakeClass(object):
-        pass
-    w_cls = FakeClass
+    w_cls = "class"
     aa = PlainAttribute(("b", DICT),
                         PlainAttribute(("a", DICT),
                                        Terminator(space, w_cls)))
     assert aa.space is space
-    assert aa.terminator.get_w_cls() is w_cls
+    assert aa.terminator.w_cls is w_cls
     assert aa.get_terminator() is aa.terminator
 
     obj = Object()
@@ -890,7 +888,7 @@
         def f():
             class C(object):
                 def m(self):
-                    42
+                    return 42
             r = weakref.ref(C)
             c = C()
             del C
@@ -898,6 +896,7 @@
             return c.m()
         val = f()
         assert val == 42
+        f() 
 
 class AppTestGlobalCaching(AppTestWithMapDict):
     def setup_class(cls):

From commits-noreply at bitbucket.org  Tue Feb 15 16:02:40 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 15 Feb 2011 16:02:40 +0100 (CET)
Subject: [pypy-svn] pypy collections-module: A branch in which to implement
	the	_collections module in RPython.
Message-ID: <20110215150240.2CFEC282BEA@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: collections-module
Changeset: r41960:06eeb36afb26
Date: 2011-02-15 15:52 +0100
http://bitbucket.org/pypy/pypy/changeset/06eeb36afb26/

Log:	A branch in which to implement the _collections module in RPython.


From commits-noreply at bitbucket.org  Tue Feb 15 16:02:41 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 15 Feb 2011 16:02:41 +0100 (CET)
Subject: [pypy-svn] pypy collections-module: Start to rewrite the
	_collections module in	RPython.
Message-ID: <20110215150241.B6027282BEA@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: collections-module
Changeset: r41961:83f2f12acb90
Date: 2011-02-15 13:03 +0100
http://bitbucket.org/pypy/pypy/changeset/83f2f12acb90/

Log:	Start to rewrite the _collections module in RPython.

diff --git a/pypy/module/_collections/__init__.py b/pypy/module/_collections/__init__.py
new file mode 100644
--- /dev/null
+++ b/pypy/module/_collections/__init__.py
@@ -0,0 +1,10 @@
+import py
+
+from pypy.interpreter.mixedmodule import MixedModule
+
+class Module(MixedModule):
+    appleveldefs = {}
+
+    interpleveldefs = {
+        'deque' : 'interp_deque.W_Deque',
+        }

diff --git a/pypy/module/_collections/test/test_deque.py b/pypy/module/_collections/test/test_deque.py
new file mode 100644
--- /dev/null
+++ b/pypy/module/_collections/test/test_deque.py
@@ -0,0 +1,623 @@
+import py
+from pypy.conftest import gettestobjspace
+
+class AppTestBasic:
+    def setup_class(cls):
+        cls.space = gettestobjspace(usemodules=['_collections'])
+
+    def test_basics(self):
+        from _collections import deque
+        d = deque(xrange(-5125, -5000))
+        d.__init__(xrange(200))
+        for i in xrange(200, 400):
+            d.append(i)
+        for i in reversed(xrange(-200, 0)):
+            d.appendleft(i)
+        assert list(d) == range(-200, 400)
+        assert len(d) == 600
+
+        left = [d.popleft() for i in xrange(250)]
+        assert left == range(-200, 50)
+        assert list(d) == range(50, 400)
+
+        right = [d.pop() for i in xrange(250)]
+        right.reverse()
+        assert right == range(150, 400)
+        assert list(d) == range(50, 150)
+
+    def test_maxlen(self):
+        self.assertRaises(ValueError, deque, 'abc', -1)
+        self.assertRaises(ValueError, deque, 'abc', -2)
+        it = iter(range(10))
+        d = deque(it, maxlen=3)
+        self.assertEqual(list(it), [])
+        self.assertEqual(repr(d), 'deque([7, 8, 9], maxlen=3)')
+        self.assertEqual(list(d), range(7, 10))
+        self.assertEqual(d, deque(range(10), 3))
+        d.append(10)
+        self.assertEqual(list(d), range(8, 11))
+        d.appendleft(7)
+        self.assertEqual(list(d), range(7, 10))
+        d.extend([10, 11])
+        self.assertEqual(list(d), range(9, 12))
+        d.extendleft([8, 7])
+        self.assertEqual(list(d), range(7, 10))
+        d = deque(xrange(200), maxlen=10)
+        d.append(d)
+        test_support.unlink(test_support.TESTFN)
+        fo = open(test_support.TESTFN, "wb")
+        try:
+            print >> fo, d,
+            fo.close()
+            fo = open(test_support.TESTFN, "rb")
+            self.assertEqual(fo.read(), repr(d))
+        finally:
+            fo.close()
+            test_support.unlink(test_support.TESTFN)
+
+        d = deque(range(10), maxlen=None)
+        self.assertEqual(repr(d), 'deque([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])')
+        fo = open(test_support.TESTFN, "wb")
+        try:
+            print >> fo, d,
+            fo.close()
+            fo = open(test_support.TESTFN, "rb")
+            self.assertEqual(fo.read(), repr(d))
+        finally:
+            fo.close()
+            test_support.unlink(test_support.TESTFN)
+
+    def test_maxlen_zero(self):
+        it = iter(range(100))
+        deque(it, maxlen=0)
+        self.assertEqual(list(it), [])
+
+        it = iter(range(100))
+        d = deque(maxlen=0)
+        d.extend(it)
+        self.assertEqual(list(it), [])
+
+        it = iter(range(100))
+        d = deque(maxlen=0)
+        d.extendleft(it)
+        self.assertEqual(list(it), [])
+
+    def test_maxlen_attribute(self):
+        self.assertEqual(deque().maxlen, None)
+        self.assertEqual(deque('abc').maxlen, None)
+        self.assertEqual(deque('abc', maxlen=4).maxlen, 4)
+        self.assertEqual(deque('abc', maxlen=2).maxlen, 2)
+        self.assertEqual(deque('abc', maxlen=0).maxlen, 0)
+        ##with self.assertRaises(AttributeError):
+        ##    d = deque('abc')
+        ##    d.maxlen = 10
+
+    def test_count(self):
+        for s in ('', 'abracadabra', 'simsalabim'*500+'abc'):
+            s = list(s)
+            d = deque(s)
+            for letter in 'abcdefghijklmnopqrstuvwxyz':
+                self.assertEqual(s.count(letter), d.count(letter), (s, d, letter))
+        self.assertRaises(TypeError, d.count)       # too few args
+        self.assertRaises(TypeError, d.count, 1, 2) # too many args
+        class BadCompare:
+            def __eq__(self, other):
+                raise ArithmeticError
+        d = deque([1, 2, BadCompare(), 3])
+        self.assertRaises(ArithmeticError, d.count, 2)
+        d = deque([1, 2, 3])
+        self.assertRaises(ArithmeticError, d.count, BadCompare())
+        class MutatingCompare:
+            def __eq__(self, other):
+                self.d.pop()
+                return True
+        m = MutatingCompare()
+        d = deque([1, 2, 3, m, 4, 5])
+        m.d = d
+        self.assertRaises(RuntimeError, d.count, 3)
+
+    def test_comparisons(self):
+        d = deque('xabc'); d.popleft()
+        for e in [d, deque('abc'), deque('ab'), deque(), list(d)]:
+            self.assertEqual(d==e, type(d)==type(e) and list(d)==list(e))
+            self.assertEqual(d!=e, not(type(d)==type(e) and list(d)==list(e)))
+
+        args = map(deque, ('', 'a', 'b', 'ab', 'ba', 'abc', 'xba', 'xabc', 'cba'))
+        for x in args:
+            for y in args:
+                self.assertEqual(x == y, list(x) == list(y), (x,y))
+                self.assertEqual(x != y, list(x) != list(y), (x,y))
+                self.assertEqual(x <  y, list(x) <  list(y), (x,y))
+                self.assertEqual(x <= y, list(x) <= list(y), (x,y))
+                self.assertEqual(x >  y, list(x) >  list(y), (x,y))
+                self.assertEqual(x >= y, list(x) >= list(y), (x,y))
+                self.assertEqual(cmp(x,y), cmp(list(x),list(y)), (x,y))
+
+    def test_extend(self):
+        d = deque('a')
+        self.assertRaises(TypeError, d.extend, 1)
+        d.extend('bcd')
+        self.assertEqual(list(d), list('abcd'))
+        d.extend(d)
+        self.assertEqual(list(d), list('abcdabcd'))
+
+    def test_iadd(self):
+        d = deque('a')
+        d += 'bcd'
+        self.assertEqual(list(d), list('abcd'))
+        d += d
+        self.assertEqual(list(d), list('abcdabcd'))
+
+    def test_extendleft(self):
+        d = deque('a')
+        self.assertRaises(TypeError, d.extendleft, 1)
+        d.extendleft('bcd')
+        self.assertEqual(list(d), list(reversed('abcd')))
+        d.extendleft(d)
+        self.assertEqual(list(d), list('abcddcba'))
+        d = deque()
+        d.extendleft(range(1000))
+        self.assertEqual(list(d), list(reversed(range(1000))))
+        self.assertRaises(SyntaxError, d.extendleft, fail())
+
+    def test_getitem(self):
+        n = 200
+        d = deque(xrange(n))
+        l = range(n)
+        for i in xrange(n):
+            d.popleft()
+            l.pop(0)
+            if random.random() < 0.5:
+                d.append(i)
+                l.append(i)
+            for j in xrange(1-len(l), len(l)):
+                assert d[j] == l[j]
+
+        d = deque('superman')
+        self.assertEqual(d[0], 's')
+        self.assertEqual(d[-1], 'n')
+        d = deque()
+        self.assertRaises(IndexError, d.__getitem__, 0)
+        self.assertRaises(IndexError, d.__getitem__, -1)
+
+    def test_setitem(self):
+        n = 200
+        d = deque(xrange(n))
+        for i in xrange(n):
+            d[i] = 10 * i
+        self.assertEqual(list(d), [10*i for i in xrange(n)])
+        l = list(d)
+        for i in xrange(1-n, 0, -1):
+            d[i] = 7*i
+            l[i] = 7*i
+        self.assertEqual(list(d), l)
+
+    def test_delitem(self):
+        n = 500         # O(n**2) test, don't make this too big
+        d = deque(xrange(n))
+        self.assertRaises(IndexError, d.__delitem__, -n-1)
+        self.assertRaises(IndexError, d.__delitem__, n)
+        for i in xrange(n):
+            self.assertEqual(len(d), n-i)
+            j = random.randrange(-len(d), len(d))
+            val = d[j]
+            self.assertIn(val, d)
+            del d[j]
+            self.assertNotIn(val, d)
+        self.assertEqual(len(d), 0)
+
+    def test_reverse(self):
+        n = 500         # O(n**2) test, don't make this too big
+        data = [random.random() for i in range(n)]
+        for i in range(n):
+            d = deque(data[:i])
+            r = d.reverse()
+            self.assertEqual(list(d), list(reversed(data[:i])))
+            self.assert_(r is None)
+            d.reverse()
+            self.assertEqual(list(d), data[:i])
+        self.assertRaises(TypeError, d.reverse, 1)          # Arity is zero
+
+    def test_rotate(self):
+        s = tuple('abcde')
+        n = len(s)
+
+        d = deque(s)
+        d.rotate(1)             # verify rot(1)
+        self.assertEqual(''.join(d), 'eabcd')
+
+        d = deque(s)
+        d.rotate(-1)            # verify rot(-1)
+        self.assertEqual(''.join(d), 'bcdea')
+        d.rotate()              # check default to 1
+        self.assertEqual(tuple(d), s)
+
+        for i in xrange(n*3):
+            d = deque(s)
+            e = deque(d)
+            d.rotate(i)         # check vs. rot(1) n times
+            for j in xrange(i):
+                e.rotate(1)
+            self.assertEqual(tuple(d), tuple(e))
+            d.rotate(-i)        # check that it works in reverse
+            self.assertEqual(tuple(d), s)
+            e.rotate(n-i)       # check that it wraps forward
+            self.assertEqual(tuple(e), s)
+
+        for i in xrange(n*3):
+            d = deque(s)
+            e = deque(d)
+            d.rotate(-i)
+            for j in xrange(i):
+                e.rotate(-1)    # check vs. rot(-1) n times
+            self.assertEqual(tuple(d), tuple(e))
+            d.rotate(i)         # check that it works in reverse
+            self.assertEqual(tuple(d), s)
+            e.rotate(i-n)       # check that it wraps backaround
+            self.assertEqual(tuple(e), s)
+
+        d = deque(s)
+        e = deque(s)
+        e.rotate(BIG+17)        # verify on long series of rotates
+        dr = d.rotate
+        for i in xrange(BIG+17):
+            dr()
+        self.assertEqual(tuple(d), tuple(e))
+
+        self.assertRaises(TypeError, d.rotate, 'x')   # Wrong arg type
+        self.assertRaises(TypeError, d.rotate, 1, 10) # Too many args
+
+        d = deque()
+        d.rotate()              # rotate an empty deque
+        self.assertEqual(d, deque())
+
+    def test_len(self):
+        d = deque('ab')
+        self.assertEqual(len(d), 2)
+        d.popleft()
+        self.assertEqual(len(d), 1)
+        d.pop()
+        self.assertEqual(len(d), 0)
+        self.assertRaises(IndexError, d.pop)
+        self.assertEqual(len(d), 0)
+        d.append('c')
+        self.assertEqual(len(d), 1)
+        d.appendleft('d')
+        self.assertEqual(len(d), 2)
+        d.clear()
+        self.assertEqual(len(d), 0)
+
+    def test_underflow(self):
+        d = deque()
+        self.assertRaises(IndexError, d.pop)
+        self.assertRaises(IndexError, d.popleft)
+
+    def test_clear(self):
+        d = deque(xrange(100))
+        self.assertEqual(len(d), 100)
+        d.clear()
+        self.assertEqual(len(d), 0)
+        self.assertEqual(list(d), [])
+        d.clear()               # clear an emtpy deque
+        self.assertEqual(list(d), [])
+
+    def test_remove(self):
+        d = deque('abcdefghcij')
+        d.remove('c')
+        self.assertEqual(d, deque('abdefghcij'))
+        d.remove('c')
+        self.assertEqual(d, deque('abdefghij'))
+        self.assertRaises(ValueError, d.remove, 'c')
+        self.assertEqual(d, deque('abdefghij'))
+
+        # Handle comparison errors
+        d = deque(['a', 'b', BadCmp(), 'c'])
+        e = deque(d)
+        self.assertRaises(RuntimeError, d.remove, 'c')
+        for x, y in zip(d, e):
+            # verify that original order and values are retained.
+            self.assertTrue(x is y)
+
+        # Handle evil mutator
+        for match in (True, False):
+            d = deque(['ab'])
+            d.extend([MutateCmp(d, match), 'c'])
+            self.assertRaises(IndexError, d.remove, 'c')
+            self.assertEqual(d, deque())
+
+    def test_repr(self):
+        d = deque(xrange(200))
+        e = eval(repr(d))
+        self.assertEqual(list(d), list(e))
+        d.append(d)
+        self.assertIn('...', repr(d))
+
+    def test_print(self):
+        d = deque(xrange(200))
+        d.append(d)
+        test_support.unlink(test_support.TESTFN)
+        fo = open(test_support.TESTFN, "wb")
+        try:
+            print >> fo, d,
+            fo.close()
+            fo = open(test_support.TESTFN, "rb")
+            self.assertEqual(fo.read(), repr(d))
+        finally:
+            fo.close()
+            test_support.unlink(test_support.TESTFN)
+
+    def test_init(self):
+        self.assertRaises(TypeError, deque, 'abc', 2, 3);
+        self.assertRaises(TypeError, deque, 1);
+
+    def test_hash(self):
+        self.assertRaises(TypeError, hash, deque('abc'))
+
+    def test_long_steadystate_queue_popleft(self):
+        for size in (0, 1, 2, 100, 1000):
+            d = deque(xrange(size))
+            append, pop = d.append, d.popleft
+            for i in xrange(size, BIG):
+                append(i)
+                x = pop()
+                if x != i - size:
+                    self.assertEqual(x, i-size)
+            self.assertEqual(list(d), range(BIG-size, BIG))
+
+    def test_long_steadystate_queue_popright(self):
+        for size in (0, 1, 2, 100, 1000):
+            d = deque(reversed(xrange(size)))
+            append, pop = d.appendleft, d.pop
+            for i in xrange(size, BIG):
+                append(i)
+                x = pop()
+                if x != i - size:
+                    self.assertEqual(x, i-size)
+            self.assertEqual(list(reversed(list(d))), range(BIG-size, BIG))
+
+    def test_big_queue_popleft(self):
+        pass
+        d = deque()
+        append, pop = d.append, d.popleft
+        for i in xrange(BIG):
+            append(i)
+        for i in xrange(BIG):
+            x = pop()
+            if x != i:
+                self.assertEqual(x, i)
+
+    def test_big_queue_popright(self):
+        d = deque()
+        append, pop = d.appendleft, d.pop
+        for i in xrange(BIG):
+            append(i)
+        for i in xrange(BIG):
+            x = pop()
+            if x != i:
+                self.assertEqual(x, i)
+
+    def test_big_stack_right(self):
+        d = deque()
+        append, pop = d.append, d.pop
+        for i in xrange(BIG):
+            append(i)
+        for i in reversed(xrange(BIG)):
+            x = pop()
+            if x != i:
+                self.assertEqual(x, i)
+        self.assertEqual(len(d), 0)
+
+    def test_big_stack_left(self):
+        d = deque()
+        append, pop = d.appendleft, d.popleft
+        for i in xrange(BIG):
+            append(i)
+        for i in reversed(xrange(BIG)):
+            x = pop()
+            if x != i:
+                self.assertEqual(x, i)
+        self.assertEqual(len(d), 0)
+
+    def test_roundtrip_iter_init(self):
+        d = deque(xrange(200))
+        e = deque(d)
+        self.assertNotEqual(id(d), id(e))
+        self.assertEqual(list(d), list(e))
+
+    def test_pickle(self):
+        d = deque(xrange(200))
+        for i in range(pickle.HIGHEST_PROTOCOL + 1):
+            s = pickle.dumps(d, i)
+            e = pickle.loads(s)
+            self.assertNotEqual(id(d), id(e))
+            self.assertEqual(list(d), list(e))
+
+##    def test_pickle_recursive(self):
+##        d = deque('abc')
+##        d.append(d)
+##        for i in range(pickle.HIGHEST_PROTOCOL + 1):
+##            e = pickle.loads(pickle.dumps(d, i))
+##            self.assertNotEqual(id(d), id(e))
+##            self.assertEqual(id(e), id(e[-1]))
+
+    def test_deepcopy(self):
+        mut = [10]
+        d = deque([mut])
+        e = copy.deepcopy(d)
+        self.assertEqual(list(d), list(e))
+        mut[0] = 11
+        self.assertNotEqual(id(d), id(e))
+        self.assertNotEqual(list(d), list(e))
+
+    def test_copy(self):
+        mut = [10]
+        d = deque([mut])
+        e = copy.copy(d)
+        self.assertEqual(list(d), list(e))
+        mut[0] = 11
+        self.assertNotEqual(id(d), id(e))
+        self.assertEqual(list(d), list(e))
+
+    def test_reversed(self):
+        for s in ('abcd', xrange(2000)):
+            self.assertEqual(list(reversed(deque(s))), list(reversed(s)))
+
+    def test_gc_doesnt_blowup(self):
+        import gc
+        # This used to assert-fail in deque_traverse() under a debug
+        # build, or run wild with a NULL pointer in a release build.
+        d = deque()
+        for i in xrange(100):
+            d.append(1)
+            gc.collect()
+
+    def test_container_iterator(self):
+        # Bug #3680: tp_traverse was not implemented for deque iterator objects
+        class C(object):
+            pass
+        for i in range(2):
+            obj = C()
+            ref = weakref.ref(obj)
+            if i == 0:
+                container = deque([obj, 1])
+            else:
+                container = reversed(deque([obj, 1]))
+            obj.x = iter(container)
+            del obj, container
+            test_support.gc_collect()
+            self.assertTrue(ref() is None, "Cycle was not collected")
+
+class XXXXXXXXXTestVariousIteratorArgs:
+
+    def test_constructor(self):
+        for s in ("123", "", range(1000), ('do', 1.2), xrange(2000,2200,5)):
+            for g in (seq_tests.Sequence, seq_tests.IterFunc,
+                      seq_tests.IterGen, seq_tests.IterFuncStop,
+                      seq_tests.itermulti, seq_tests.iterfunc):
+                self.assertEqual(list(deque(g(s))), list(g(s)))
+            self.assertRaises(TypeError, deque, seq_tests.IterNextOnly(s))
+            self.assertRaises(TypeError, deque, seq_tests.IterNoNext(s))
+            self.assertRaises(ZeroDivisionError, deque, seq_tests.IterGenExc(s))
+
+    def test_iter_with_altered_data(self):
+        d = deque('abcdefg')
+        it = iter(d)
+        d.pop()
+        self.assertRaises(RuntimeError, it.next)
+
+    def test_runtime_error_on_empty_deque(self):
+        d = deque()
+        it = iter(d)
+        d.append(10)
+        self.assertRaises(RuntimeError, it.next)
+
+##class Deque(deque):
+##    pass
+
+##class DequeWithBadIter(deque):
+##    def __iter__(self):
+##        raise TypeError
+
+class XXXXXXXXXTestSubclass:
+
+    def test_basics(self):
+        d = Deque(xrange(25))
+        d.__init__(xrange(200))
+        for i in xrange(200, 400):
+            d.append(i)
+        for i in reversed(xrange(-200, 0)):
+            d.appendleft(i)
+        self.assertEqual(list(d), range(-200, 400))
+        self.assertEqual(len(d), 600)
+
+        left = [d.popleft() for i in xrange(250)]
+        self.assertEqual(left, range(-200, 50))
+        self.assertEqual(list(d), range(50, 400))
+
+        right = [d.pop() for i in xrange(250)]
+        right.reverse()
+        self.assertEqual(right, range(150, 400))
+        self.assertEqual(list(d), range(50, 150))
+
+        d.clear()
+        self.assertEqual(len(d), 0)
+
+    def test_copy_pickle(self):
+
+        d = Deque('abc')
+
+        e = d.__copy__()
+        self.assertEqual(type(d), type(e))
+        self.assertEqual(list(d), list(e))
+
+        e = Deque(d)
+        self.assertEqual(type(d), type(e))
+        self.assertEqual(list(d), list(e))
+
+        s = pickle.dumps(d)
+        e = pickle.loads(s)
+        self.assertNotEqual(id(d), id(e))
+        self.assertEqual(type(d), type(e))
+        self.assertEqual(list(d), list(e))
+
+        d = Deque('abcde', maxlen=4)
+
+        e = d.__copy__()
+        self.assertEqual(type(d), type(e))
+        self.assertEqual(list(d), list(e))
+
+        e = Deque(d)
+        self.assertEqual(type(d), type(e))
+        self.assertEqual(list(d), list(e))
+
+        s = pickle.dumps(d)
+        e = pickle.loads(s)
+        self.assertNotEqual(id(d), id(e))
+        self.assertEqual(type(d), type(e))
+        self.assertEqual(list(d), list(e))
+
+##    def test_pickle(self):
+##        d = Deque('abc')
+##        d.append(d)
+##
+##        e = pickle.loads(pickle.dumps(d))
+##        self.assertNotEqual(id(d), id(e))
+##        self.assertEqual(type(d), type(e))
+##        dd = d.pop()
+##        ee = e.pop()
+##        self.assertEqual(id(e), id(ee))
+##        self.assertEqual(d, e)
+##
+##        d.x = d
+##        e = pickle.loads(pickle.dumps(d))
+##        self.assertEqual(id(e), id(e.x))
+##
+##        d = DequeWithBadIter('abc')
+##        self.assertRaises(TypeError, pickle.dumps, d)
+
+    def test_weakref(self):
+        d = deque('gallahad')
+        p = weakref.proxy(d)
+        self.assertEqual(str(p), str(d))
+        d = None
+        test_support.gc_collect()
+        self.assertRaises(ReferenceError, str, p)
+
+    def test_strange_subclass(self):
+        class X(deque):
+            def __iter__(self):
+                return iter([])
+        d1 = X([1,2,3])
+        d2 = X([4,5,6])
+        d1 == d2   # not clear if this is supposed to be True or False,
+                   # but it used to give a SystemError
+
+
+##class SubclassWithKwargs(deque):
+##    def __init__(self, newarg=1):
+##        deque.__init__(self)
+
+class XXXXXXXXXTestSubclassWithKwargs:
+    def test_subclass_with_kwargs(self):
+        # SF bug #1486663 -- this used to erroneously raise a TypeError
+        SubclassWithKwargs(newarg=1)

diff --git a/pypy/module/_collections/test/__init__.py b/pypy/module/_collections/test/__init__.py
new file mode 100644

diff --git a/pypy/module/_collections/interp_deque.py b/pypy/module/_collections/interp_deque.py
new file mode 100644
--- /dev/null
+++ b/pypy/module/_collections/interp_deque.py
@@ -0,0 +1,88 @@
+from pypy.interpreter.baseobjspace import Wrappable
+from pypy.interpreter.typedef import TypeDef, interp2app, make_weakref_descr
+from pypy.interpreter.gateway import ObjSpace, W_Root, Arguments
+
+
+# A `dequeobject` is composed of a doubly-linked list of `block` nodes.
+# This list is not circular (the leftmost block has leftlink==NULL,
+# and the rightmost block has rightlink==NULL).  A deque d's first
+# element is at d.leftblock[leftindex] and its last element is at
+# d.rightblock[rightindex]; note that, unlike as for Python slice
+# indices, these indices are inclusive on both ends.  By being inclusive
+# on both ends, algorithms for left and right operations become
+# symmetrical which simplifies the design.
+#
+# The list of blocks is never empty, so d.leftblock and d.rightblock
+# are never equal to NULL.
+#
+# The indices, d.leftindex and d.rightindex are always in the range
+#     0 <= index < BLOCKLEN.
+# Their exact relationship is:
+#     (d.leftindex + d.len - 1) % BLOCKLEN == d.rightindex.
+#
+# Empty deques have d.len == 0; d.leftblock==d.rightblock;
+# d.leftindex == CENTER+1; and d.rightindex == CENTER.
+# Checking for d.len == 0 is the intended way to see whether d is empty.
+#
+# Whenever d.leftblock == d.rightblock,
+#     d.leftindex + d.len - 1 == d.rightindex.
+#
+# However, when d.leftblock != d.rightblock, d.leftindex and d.rightindex
+# become indices into distinct blocks and either may be larger than the
+# other.
+
+BLOCKLEN = 62
+CENTER   = ((BLOCKLEN - 1) / 2)
+
+class Block(object):
+    def __init__(self, leftlink, rightlink):
+        self.leftlink = leftlink
+        self.rightlink = rightlink
+        self.data = [None] * BLOCKLEN
+
+
+class W_Deque(Wrappable):
+    def __init__(self, space):
+        self.space = space
+        self.len = 0
+        self.leftblock = self.rightblock = Block(None, None)
+        self.leftindex = CENTER + 1
+        self.rightindex = CENTER
+
+    def append(self, w_x):
+        ri = self.rightindex + 1
+        if ri >= BLOCKLEN:
+            b = Block(self.rightblock, None)
+            self.rightblock.rightlink = b
+            self.rightblock = b
+            ri = 0
+        self.rightindex = ri
+        self.rightblock.data[ri] = w_x
+        self.len += 1
+    append.unwrap_spec = ['self', W_Root]
+
+    def appendleft(self, w_x):
+        li = self.leftindex - 1
+        if li < 0:
+            b = Block(None, self.leftblock)
+            self.leftblock.leftlink = b
+            self.leftblock = b
+            li = BLOCKLEN - 1
+        self.leftindex = li
+        self.leftblock.data[li] = w_x
+        self.len += 1
+    appendleft.unwrap_spec = ['self', W_Root]
+
+
+def descr__new__(space, w_subtype, args):
+    w_self = space.allocate_instance(W_Deque, w_subtype)
+    W_Deque.__init__(space.interp_w(W_Deque, w_self), space)
+    return w_self
+descr__new__.unwrap_spec = [ObjSpace, W_Root, Arguments]
+
+W_Deque.typedef = TypeDef("deque",
+    __new__ = interp2app(descr__new__),
+    append = interp2app(W_Deque.append),
+    appendleft = interp2app(W_Deque.appendleft),
+    __weakref__ = make_weakref_descr(W_Deque),
+)

From commits-noreply at bitbucket.org  Tue Feb 15 16:02:43 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 15 Feb 2011 16:02:43 +0100 (CET)
Subject: [pypy-svn] pypy collections-module: In-progress.
Message-ID: <20110215150243.543B0282BEA@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: collections-module
Changeset: r41962:3643b8394947
Date: 2011-02-15 13:26 +0100
http://bitbucket.org/pypy/pypy/changeset/3643b8394947/

Log:	In-progress.

diff --git a/pypy/module/_collections/interp_deque.py b/pypy/module/_collections/interp_deque.py
--- a/pypy/module/_collections/interp_deque.py
+++ b/pypy/module/_collections/interp_deque.py
@@ -1,6 +1,9 @@
 from pypy.interpreter.baseobjspace import Wrappable
 from pypy.interpreter.typedef import TypeDef, interp2app, make_weakref_descr
-from pypy.interpreter.gateway import ObjSpace, W_Root, Arguments
+from pypy.interpreter.gateway import ObjSpace, W_Root, unwrap_spec, Arguments
+from pypy.interpreter.gateway import NoneNotWrapped
+from pypy.interpreter.error import OperationError
+from pypy.rlib.debug import check_nonneg
 
 
 # A `dequeobject` is composed of a doubly-linked list of `block` nodes.
@@ -35,20 +38,29 @@
 CENTER   = ((BLOCKLEN - 1) / 2)
 
 class Block(object):
+    __slots__ = ('leftlink', 'rightlink', 'data')
     def __init__(self, leftlink, rightlink):
         self.leftlink = leftlink
         self.rightlink = rightlink
         self.data = [None] * BLOCKLEN
 
+# ------------------------------------------------------------
 
 class W_Deque(Wrappable):
     def __init__(self, space):
         self.space = space
-        self.len = 0
-        self.leftblock = self.rightblock = Block(None, None)
-        self.leftindex = CENTER + 1
-        self.rightindex = CENTER
+        self.clear()
+        check_nonneg(self.leftindex)
+        check_nonneg(self.rightindex)
 
+    @unwrap_spec('self', W_Root)  #, W_Root)
+    def init(self, w_iterable=NoneNotWrapped):  #, w_maxlen=None):
+        if self.len > 0:
+            self.clear()
+        if w_iterable is not None:
+            self.extend(w_iterable)
+
+    @unwrap_spec('self', W_Root)
     def append(self, w_x):
         ri = self.rightindex + 1
         if ri >= BLOCKLEN:
@@ -59,8 +71,8 @@
         self.rightindex = ri
         self.rightblock.data[ri] = w_x
         self.len += 1
-    append.unwrap_spec = ['self', W_Root]
 
+    @unwrap_spec('self', W_Root)
     def appendleft(self, w_x):
         li = self.leftindex - 1
         if li < 0:
@@ -71,18 +83,104 @@
         self.leftindex = li
         self.leftblock.data[li] = w_x
         self.len += 1
-    appendleft.unwrap_spec = ['self', W_Root]
 
+    @unwrap_spec('self')
+    def clear(self):
+        self.leftblock = Block(None, None)
+        self.rightblock = self.leftblock
+        self.leftindex = CENTER + 1
+        self.rightindex = CENTER
+        self.len = 0
 
+    @unwrap_spec('self', W_Root)
+    def extend(self, w_iterable):
+        # XXX Handle case where id(deque) == id(iterable)
+        space = self.space
+        w_iter = space.iter(w_iterable)
+        while True:
+            try:
+                w_obj = space.next(w_iter)
+            except OperationError, e:
+                if e.match(space, space.w_StopIteration):
+                    break
+                raise
+            self.append(w_obj)
+
+    @unwrap_spec('self', W_Root)
+    def extendleft(self, w_iterable):
+        # XXX Handle case where id(deque) == id(iterable)
+        space = self.space
+        w_iter = space.iter(w_iterable)
+        while True:
+            try:
+                w_obj = space.next(w_iter)
+            except OperationError, e:
+                if e.match(space, space.w_StopIteration):
+                    break
+                raise
+            self.appendleft(w_obj)
+
+    @unwrap_spec('self')
+    def iter(self):
+        return W_DequeIter(self)
+
+    @unwrap_spec('self')
+    def length(self):
+        return self.space.wrap(self.len)
+
+
+ at unwrap_spec(ObjSpace, W_Root, Arguments)
 def descr__new__(space, w_subtype, args):
     w_self = space.allocate_instance(W_Deque, w_subtype)
     W_Deque.__init__(space.interp_w(W_Deque, w_self), space)
     return w_self
-descr__new__.unwrap_spec = [ObjSpace, W_Root, Arguments]
 
 W_Deque.typedef = TypeDef("deque",
     __new__ = interp2app(descr__new__),
-    append = interp2app(W_Deque.append),
+    __init__ = interp2app(W_Deque.init),
+    append     = interp2app(W_Deque.append),
     appendleft = interp2app(W_Deque.appendleft),
+    clear      = interp2app(W_Deque.clear),
+    extend     = interp2app(W_Deque.extend),
+    extendleft = interp2app(W_Deque.extendleft),
     __weakref__ = make_weakref_descr(W_Deque),
+    __iter__ = interp2app(W_Deque.iter),
+    __len__ = interp2app(W_Deque.length),
 )
+
+# ------------------------------------------------------------
+
+class W_DequeIter(Wrappable):
+    def __init__(self, deque):
+        self.space = deque.space
+        self.deque = deque
+        self.block = deque.leftblock
+        self.index = deque.leftindex
+        self.counter = deque.len
+        check_nonneg(self.index)
+
+    @unwrap_spec('self')
+    def iter(self):
+        return self.space.wrap(self)
+
+    @unwrap_spec('self')
+    def next(self):
+        if self.counter == 0:
+            raise OperationError(self.space.w_StopIteration, self.space.w_None)
+        self.counter -= 1
+        ri = self.index
+        w_x = self.block.data[ri]
+        ri += 1
+        if ri == BLOCKLEN:
+            self.block = self.block.rightlink
+            ri = 0
+        self.index = ri
+        return w_x
+
+W_DequeIter.typedef = TypeDef("deque_iterator",
+    __iter__ = interp2app(W_DequeIter.iter),
+    next = interp2app(W_DequeIter.next),
+)
+W_DequeIter.typedef.acceptable_as_base_class = False
+
+# ------------------------------------------------------------

From commits-noreply at bitbucket.org  Tue Feb 15 16:02:44 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 15 Feb 2011 16:02:44 +0100 (CET)
Subject: [pypy-svn] pypy collections-module: pop(), popleft()
Message-ID: <20110215150244.2CCB7282BEA@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: collections-module
Changeset: r41963:28a49a7dc3bb
Date: 2011-02-15 13:31 +0100
http://bitbucket.org/pypy/pypy/changeset/28a49a7dc3bb/

Log:	pop(), popleft()

diff --git a/pypy/module/_collections/interp_deque.py b/pypy/module/_collections/interp_deque.py
--- a/pypy/module/_collections/interp_deque.py
+++ b/pypy/module/_collections/interp_deque.py
@@ -121,6 +121,50 @@
             self.appendleft(w_obj)
 
     @unwrap_spec('self')
+    def pop(self):
+        if self.len == 0:
+            msg = "pop from an empty deque"
+            raise OperationError(self.space.w_IndexError, self.space.wrap(msg))
+        self.len -= 1
+        ri = self.rightindex
+        w_obj = self.rightblock.data[ri]
+        ri -= 1
+        if ri < 0:
+            if self.len == 0:
+                # re-center instead of freeing the last block
+                self.leftindex = CENTER + 1
+                ri = CENTER
+            else:
+                b = self.rightblock.leftlink
+                self.rightblock = b
+                b.rightlink = None
+                ri = BLOCKLEN - 1
+        self.rightindex = ri
+        return w_obj
+
+    @unwrap_spec('self')
+    def popleft(self):
+        if self.len == 0:
+            msg = "pop from an empty deque"
+            raise OperationError(self.space.w_IndexError, self.space.wrap(msg))
+        self.len -= 1
+        li = self.leftindex
+        w_obj = self.leftblock.data[li]
+        li += 1
+        if li >= BLOCKLEN:
+            if self.len == 0:
+                # re-center instead of freeing the last block
+                li = CENTER + 1
+                self.rightindex = CENTER
+            else:
+                b = self.leftblock.rightlink
+                self.leftblock = b
+                b.leftlink = None
+                li = 0
+        self.leftindex = li
+        return w_obj
+
+    @unwrap_spec('self')
     def iter(self):
         return W_DequeIter(self)
 
@@ -143,6 +187,8 @@
     clear      = interp2app(W_Deque.clear),
     extend     = interp2app(W_Deque.extend),
     extendleft = interp2app(W_Deque.extendleft),
+    pop        = interp2app(W_Deque.pop),
+    popleft    = interp2app(W_Deque.popleft),
     __weakref__ = make_weakref_descr(W_Deque),
     __iter__ = interp2app(W_Deque.iter),
     __len__ = interp2app(W_Deque.length),

From commits-noreply at bitbucket.org  Tue Feb 15 16:02:45 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 15 Feb 2011 16:02:45 +0100 (CET)
Subject: [pypy-svn] pypy collections-module: maxlen, __repr__, locking.
Message-ID: <20110215150245.CAC6D282C23@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: collections-module
Changeset: r41964:4a46278c7d08
Date: 2011-02-15 14:11 +0100
http://bitbucket.org/pypy/pypy/changeset/4a46278c7d08/

Log:	maxlen, __repr__, locking.

diff --git a/pypy/module/_collections/test/test_deque.py b/pypy/module/_collections/test/test_deque.py
--- a/pypy/module/_collections/test/test_deque.py
+++ b/pypy/module/_collections/test/test_deque.py
@@ -26,95 +26,71 @@
         assert list(d) == range(50, 150)
 
     def test_maxlen(self):
-        self.assertRaises(ValueError, deque, 'abc', -1)
-        self.assertRaises(ValueError, deque, 'abc', -2)
+        from _collections import deque
+        raises(ValueError, deque, 'abc', -1)
+        raises(ValueError, deque, 'abc', -2)
         it = iter(range(10))
         d = deque(it, maxlen=3)
-        self.assertEqual(list(it), [])
-        self.assertEqual(repr(d), 'deque([7, 8, 9], maxlen=3)')
-        self.assertEqual(list(d), range(7, 10))
-        self.assertEqual(d, deque(range(10), 3))
-        d.append(10)
-        self.assertEqual(list(d), range(8, 11))
-        d.appendleft(7)
-        self.assertEqual(list(d), range(7, 10))
-        d.extend([10, 11])
-        self.assertEqual(list(d), range(9, 12))
-        d.extendleft([8, 7])
-        self.assertEqual(list(d), range(7, 10))
-        d = deque(xrange(200), maxlen=10)
-        d.append(d)
-        test_support.unlink(test_support.TESTFN)
-        fo = open(test_support.TESTFN, "wb")
-        try:
-            print >> fo, d,
-            fo.close()
-            fo = open(test_support.TESTFN, "rb")
-            self.assertEqual(fo.read(), repr(d))
-        finally:
-            fo.close()
-            test_support.unlink(test_support.TESTFN)
-
-        d = deque(range(10), maxlen=None)
-        self.assertEqual(repr(d), 'deque([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])')
-        fo = open(test_support.TESTFN, "wb")
-        try:
-            print >> fo, d,
-            fo.close()
-            fo = open(test_support.TESTFN, "rb")
-            self.assertEqual(fo.read(), repr(d))
-        finally:
-            fo.close()
-            test_support.unlink(test_support.TESTFN)
+        assert list(it) == []
+        assert repr(d) == 'deque([7, 8, 9], maxlen=3)'
+        assert list(d) == range(7, 10)
+        d.appendleft(3)
+        assert list(d) == [3, 7, 8]
+        d.extend([20, 21])
+        assert list(d) == [8, 20, 21]
+        d.extendleft([-7, -6])
+        assert list(d) == [-6, -7, 8]
 
     def test_maxlen_zero(self):
+        from _collections import deque
         it = iter(range(100))
-        deque(it, maxlen=0)
-        self.assertEqual(list(it), [])
-
-        it = iter(range(100))
-        d = deque(maxlen=0)
-        d.extend(it)
-        self.assertEqual(list(it), [])
-
-        it = iter(range(100))
-        d = deque(maxlen=0)
-        d.extendleft(it)
-        self.assertEqual(list(it), [])
+        d = deque(it, maxlen=0)
+        assert list(d) == []
+        assert list(it) == []
+        d.extend(range(100))
+        assert list(d) == []
+        d.extendleft(range(100))
+        assert list(d) == []
 
     def test_maxlen_attribute(self):
-        self.assertEqual(deque().maxlen, None)
-        self.assertEqual(deque('abc').maxlen, None)
-        self.assertEqual(deque('abc', maxlen=4).maxlen, 4)
-        self.assertEqual(deque('abc', maxlen=2).maxlen, 2)
-        self.assertEqual(deque('abc', maxlen=0).maxlen, 0)
-        ##with self.assertRaises(AttributeError):
-        ##    d = deque('abc')
-        ##    d.maxlen = 10
+        from _collections import deque
+        assert deque().maxlen is None
+        assert deque('abc').maxlen is None
+        assert deque('abc', maxlen=4).maxlen == 4
+        assert deque('abc', maxlen=0).maxlen == 0
+        raises((AttributeError, TypeError), "deque('abc').maxlen = 10")
+
+    def test_runtimeerror(self):
+        from _collections import deque
+        d = deque('abcdefg')
+        it = iter(d)
+        d.pop()
+        raises(RuntimeError, it.next)
+        #
+        d = deque('abcdefg')
+        it = iter(d)
+        d.append(d.pop())
+        raises(RuntimeError, it.next)
+        #
+        d = deque()
+        it = iter(d)
+        d.append(10)
+        raises(RuntimeError, it.next)
 
     def test_count(self):
+        from _collections import deque
         for s in ('', 'abracadabra', 'simsalabim'*500+'abc'):
             s = list(s)
             d = deque(s)
             for letter in 'abcdefghijklmnopqrstuvwxyz':
-                self.assertEqual(s.count(letter), d.count(letter), (s, d, letter))
-        self.assertRaises(TypeError, d.count)       # too few args
-        self.assertRaises(TypeError, d.count, 1, 2) # too many args
-        class BadCompare:
-            def __eq__(self, other):
-                raise ArithmeticError
-        d = deque([1, 2, BadCompare(), 3])
-        self.assertRaises(ArithmeticError, d.count, 2)
-        d = deque([1, 2, 3])
-        self.assertRaises(ArithmeticError, d.count, BadCompare())
+                assert s.count(letter) == d.count(letter)
         class MutatingCompare:
             def __eq__(self, other):
-                self.d.pop()
+                d.pop()
                 return True
         m = MutatingCompare()
         d = deque([1, 2, 3, m, 4, 5])
-        m.d = d
-        self.assertRaises(RuntimeError, d.count, 3)
+        raises(RuntimeError, d.count, 3)
 
     def test_comparisons(self):
         d = deque('xabc'); d.popleft()

diff --git a/pypy/module/_collections/interp_deque.py b/pypy/module/_collections/interp_deque.py
--- a/pypy/module/_collections/interp_deque.py
+++ b/pypy/module/_collections/interp_deque.py
@@ -1,5 +1,8 @@
+import sys
+from pypy.interpreter import gateway
 from pypy.interpreter.baseobjspace import Wrappable
 from pypy.interpreter.typedef import TypeDef, interp2app, make_weakref_descr
+from pypy.interpreter.typedef import GetSetProperty
 from pypy.interpreter.gateway import ObjSpace, W_Root, unwrap_spec, Arguments
 from pypy.interpreter.gateway import NoneNotWrapped
 from pypy.interpreter.error import OperationError
@@ -49,17 +52,50 @@
 class W_Deque(Wrappable):
     def __init__(self, space):
         self.space = space
+        self.maxlen = sys.maxint
         self.clear()
         check_nonneg(self.leftindex)
         check_nonneg(self.rightindex)
+        #
+        # lightweight locking: any modification to the content of the deque
+        # sets the lock to None.  Taking an iterator sets it to a non-None
+        # value.  The iterator can check if further modifications occurred
+        # by checking if the lock still has the same non-None value.
+        # (In CPython, this is implemented using d->state.)
+        self.lock = None
 
-    @unwrap_spec('self', W_Root)  #, W_Root)
-    def init(self, w_iterable=NoneNotWrapped):  #, w_maxlen=None):
+    def modified(self):
+        self.lock = None
+
+    def getlock(self, iterator):
+        if self.lock is None:
+            self.lock = iterator     # actually use the iterator itself,
+                                     # instead of some new empty object
+        return self.lock
+
+    @unwrap_spec('self', W_Root, W_Root)
+    def init(self, w_iterable=NoneNotWrapped, w_maxlen=None):
+        space = self.space
+        if space.is_w(w_maxlen, space.w_None):
+            maxlen = sys.maxint
+        else:
+            maxlen = space.gateway_nonnegint_w(w_maxlen)
+        self.maxlen = maxlen
         if self.len > 0:
             self.clear()
         if w_iterable is not None:
             self.extend(w_iterable)
 
+    def trimleft(self):
+        if self.len > self.maxlen:
+            self.popleft()
+            assert self.len == self.maxlen
+
+    def trimright(self):
+        if self.len > self.maxlen:
+            self.pop()
+            assert self.len == self.maxlen
+
     @unwrap_spec('self', W_Root)
     def append(self, w_x):
         ri = self.rightindex + 1
@@ -71,6 +107,8 @@
         self.rightindex = ri
         self.rightblock.data[ri] = w_x
         self.len += 1
+        self.trimleft()
+        self.modified()
 
     @unwrap_spec('self', W_Root)
     def appendleft(self, w_x):
@@ -83,6 +121,8 @@
         self.leftindex = li
         self.leftblock.data[li] = w_x
         self.len += 1
+        self.trimright()
+        self.modified()
 
     @unwrap_spec('self')
     def clear(self):
@@ -91,6 +131,7 @@
         self.leftindex = CENTER + 1
         self.rightindex = CENTER
         self.len = 0
+        self.modified()
 
     @unwrap_spec('self', W_Root)
     def extend(self, w_iterable):
@@ -140,6 +181,7 @@
                 b.rightlink = None
                 ri = BLOCKLEN - 1
         self.rightindex = ri
+        self.modified()
         return w_obj
 
     @unwrap_spec('self')
@@ -162,6 +204,7 @@
                 b.leftlink = None
                 li = 0
         self.leftindex = li
+        self.modified()
         return w_obj
 
     @unwrap_spec('self')
@@ -172,6 +215,46 @@
     def length(self):
         return self.space.wrap(self.len)
 
+    @unwrap_spec('self')
+    def repr(self):
+        space = self.space
+        ec = space.getexecutioncontext()
+        w_currently_in_repr = ec._py_repr
+        if w_currently_in_repr is None:
+            w_currently_in_repr = ec._py_repr = space.newdict()
+        return dequerepr(space, w_currently_in_repr, space.wrap(self))
+
+    def get_maxlen(space, self):
+        if self.maxlen == sys.maxint:
+            return self.space.w_None
+        else:
+            return self.space.wrap(self.maxlen)
+
+
+app = gateway.applevel("""
+    def dequerepr(currently_in_repr, d):
+        'The app-level part of repr().'
+        deque_id = id(d)
+        if deque_id in currently_in_repr:
+            listrepr = '[...]'
+        else:
+            currently_in_repr[deque_id] = 1
+            try:
+                listrepr = "[" + ", ".join([repr(x) for x in d]) + ']'
+            finally:
+                try:
+                    del currently_in_repr[deque_id]
+                except:
+                    pass
+        if d.maxlen is None:
+            maxlenrepr = ''
+        else:
+            maxlenrepr = ', maxlen=%d' % (d.maxlen,)
+        return 'deque(%s%s)' % (listrepr, maxlenrepr)
+""", filename=__file__)
+
+dequerepr = app.interphook("dequerepr")
+
 
 @unwrap_spec(ObjSpace, W_Root, Arguments)
 def descr__new__(space, w_subtype, args):
@@ -192,6 +275,8 @@
     __weakref__ = make_weakref_descr(W_Deque),
     __iter__ = interp2app(W_Deque.iter),
     __len__ = interp2app(W_Deque.length),
+    __repr__ = interp2app(W_Deque.repr),
+    maxlen = GetSetProperty(W_Deque.get_maxlen),
 )
 
 # ------------------------------------------------------------
@@ -203,6 +288,7 @@
         self.block = deque.leftblock
         self.index = deque.leftindex
         self.counter = deque.len
+        self.lock = deque.getlock(self)
         check_nonneg(self.index)
 
     @unwrap_spec('self')
@@ -211,6 +297,10 @@
 
     @unwrap_spec('self')
     def next(self):
+        if self.lock is not self.deque.lock:
+            raise OperationError(
+                self.space.w_RuntimeError,
+                self.space.wrap("deque mutated during iteration"))
         if self.counter == 0:
             raise OperationError(self.space.w_StopIteration, self.space.w_None)
         self.counter -= 1

From commits-noreply at bitbucket.org  Tue Feb 15 16:02:48 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 15 Feb 2011 16:02:48 +0100 (CET)
Subject: [pypy-svn] pypy collections-module: count().
Message-ID: <20110215150248.92DEA282BF7@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: collections-module
Changeset: r41965:70352f1caae2
Date: 2011-02-15 14:18 +0100
http://bitbucket.org/pypy/pypy/changeset/70352f1caae2/

Log:	count().

diff --git a/pypy/module/_collections/test/test_deque.py b/pypy/module/_collections/test/test_deque.py
--- a/pypy/module/_collections/test/test_deque.py
+++ b/pypy/module/_collections/test/test_deque.py
@@ -79,10 +79,10 @@
 
     def test_count(self):
         from _collections import deque
-        for s in ('', 'abracadabra', 'simsalabim'*500+'abc'):
+        for s in ('', 'abracadabra', 'simsalabim'*50+'abc'):
             s = list(s)
             d = deque(s)
-            for letter in 'abcdefghijklmnopqrstuvwxyz':
+            for letter in 'abcdeilmrs':
                 assert s.count(letter) == d.count(letter)
         class MutatingCompare:
             def __eq__(self, other):

diff --git a/pypy/module/_collections/interp_deque.py b/pypy/module/_collections/interp_deque.py
--- a/pypy/module/_collections/interp_deque.py
+++ b/pypy/module/_collections/interp_deque.py
@@ -47,6 +47,9 @@
         self.rightlink = rightlink
         self.data = [None] * BLOCKLEN
 
+class Lock(object):
+    pass
+
 # ------------------------------------------------------------
 
 class W_Deque(Wrappable):
@@ -67,12 +70,17 @@
     def modified(self):
         self.lock = None
 
-    def getlock(self, iterator):
+    def getlock(self):
         if self.lock is None:
-            self.lock = iterator     # actually use the iterator itself,
-                                     # instead of some new empty object
+            self.lock = Lock()
         return self.lock
 
+    def checklock(self, lock):
+        if lock is not self.lock:
+            raise OperationError(
+                self.space.w_RuntimeError,
+                self.space.wrap("deque mutated during iteration"))
+
     @unwrap_spec('self', W_Root, W_Root)
     def init(self, w_iterable=NoneNotWrapped, w_maxlen=None):
         space = self.space
@@ -134,6 +142,25 @@
         self.modified()
 
     @unwrap_spec('self', W_Root)
+    def count(self, w_x):
+        space = self.space
+        result = 0
+        block = self.leftblock
+        index = self.leftindex
+        lock = self.getlock()
+        for i in range(self.len):
+            w_item = block.data[index]
+            if space.eq_w(w_item, w_x):
+                result += 1
+            self.checklock(lock)
+            # Advance the block/index pair
+            index += 1
+            if index >= BLOCKLEN:
+                block = block.rightlink
+                index = 0
+        return space.wrap(result)
+
+    @unwrap_spec('self', W_Root)
     def extend(self, w_iterable):
         # XXX Handle case where id(deque) == id(iterable)
         space = self.space
@@ -268,6 +295,7 @@
     append     = interp2app(W_Deque.append),
     appendleft = interp2app(W_Deque.appendleft),
     clear      = interp2app(W_Deque.clear),
+    count      = interp2app(W_Deque.count),
     extend     = interp2app(W_Deque.extend),
     extendleft = interp2app(W_Deque.extendleft),
     pop        = interp2app(W_Deque.pop),
@@ -288,7 +316,7 @@
         self.block = deque.leftblock
         self.index = deque.leftindex
         self.counter = deque.len
-        self.lock = deque.getlock(self)
+        self.lock = deque.getlock()
         check_nonneg(self.index)
 
     @unwrap_spec('self')
@@ -297,10 +325,7 @@
 
     @unwrap_spec('self')
     def next(self):
-        if self.lock is not self.deque.lock:
-            raise OperationError(
-                self.space.w_RuntimeError,
-                self.space.wrap("deque mutated during iteration"))
+        self.deque.checklock(self.lock)
         if self.counter == 0:
             raise OperationError(self.space.w_StopIteration, self.space.w_None)
         self.counter -= 1

From commits-noreply at bitbucket.org  Tue Feb 15 16:02:50 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 15 Feb 2011 16:02:50 +0100 (CET)
Subject: [pypy-svn] pypy collections-module: A helper:
	space.compare_by_iteration().
Message-ID: <20110215150250.791C0282BF7@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: collections-module
Changeset: r41966:5fbfabfab60a
Date: 2011-02-15 14:49 +0100
http://bitbucket.org/pypy/pypy/changeset/5fbfabfab60a/

Log:	A helper: space.compare_by_iteration().

diff --git a/pypy/interpreter/baseobjspace.py b/pypy/interpreter/baseobjspace.py
--- a/pypy/interpreter/baseobjspace.py
+++ b/pypy/interpreter/baseobjspace.py
@@ -1009,6 +1009,38 @@
         return self.call_args(w_func, args)
     appexec._annspecialcase_ = 'specialize:arg(2)'
 
+    def _next_or_none(self, w_it):
+        try:
+            return self.next(w_it)
+        except OperationError, e:
+            if not e.match(self, self.w_StopIteration):
+                raise
+            return None
+
+    def compare_by_iteration(self, w_iterable1, w_iterable2, op):
+        w_it1 = self.iter(w_iterable1)
+        w_it2 = self.iter(w_iterable2)
+        while True:
+            w_x1 = self._next_or_none(w_it1)
+            w_x2 = self._next_or_none(w_it2)
+            if w_x1 is None or w_x2 is None:
+                if op == 'eq': return self.newbool(w_x1 is w_x2)  # both None
+                if op == 'ne': return self.newbool(w_x1 is not w_x2)
+                if op == 'lt': return self.newbool(w_x2 is not None)
+                if op == 'le': return self.newbool(w_x1 is None)
+                if op == 'gt': return self.newbool(w_x1 is not None)
+                if op == 'ge': return self.newbool(w_x2 is None)
+                assert False, "bad value for op"
+            if not self.eq_w(w_x1, w_x2):
+                if op == 'eq': return self.w_False
+                if op == 'ne': return self.w_True
+                if op == 'lt': return self.lt(w_x1, w_x2)
+                if op == 'le': return self.le(w_x1, w_x2)
+                if op == 'gt': return self.gt(w_x1, w_x2)
+                if op == 'ge': return self.ge(w_x1, w_x2)
+                assert False, "bad value for op"
+    compare_by_iteration._annspecialcase_ = 'specialize:arg(3)'
+
     def decode_index(self, w_index_or_slice, seqlength):
         """Helper for custom sequence implementations
              -> (index, 0, 0) or

diff --git a/pypy/interpreter/test/test_objspace.py b/pypy/interpreter/test/test_objspace.py
--- a/pypy/interpreter/test/test_objspace.py
+++ b/pypy/interpreter/test/test_objspace.py
@@ -243,6 +243,21 @@
         w_res = space.call_obj_args(w_a, w_9, Arguments(space, []))        
         assert w_res is w_9
 
+    def test_compare_by_iteration(self):
+        import operator
+        space = self.space
+        for op in ['eq', 'ne', 'lt', 'le', 'gt', 'ge']:
+            comparer = getattr(operator, op)
+            for x in [[], [0], [0, 1], [0, 1, 2]]:
+                for y in [[], [-1], [0], [1], [-1, 0],
+                          [0, 0], [0, 1], [0, 2],
+                          [0, 1, 1], [0, 1, 2], [0, 1, 3]]:
+                        w_res = space.compare_by_iteration(space.wrap(x),
+                                                           space.wrap(y), op)
+                        if comparer(x, y):
+                            assert w_res is space.w_True
+                        else:
+                            assert w_res is space.w_False
 
 class TestModuleMinimal: 
     def test_sys_exists(self):

From commits-noreply at bitbucket.org  Tue Feb 15 16:02:51 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 15 Feb 2011 16:02:51 +0100 (CET)
Subject: [pypy-svn] pypy collections-module: Comparison.
Message-ID: <20110215150251.C911B2A202C@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: collections-module
Changeset: r41967:68683618f8b6
Date: 2011-02-15 14:53 +0100
http://bitbucket.org/pypy/pypy/changeset/68683618f8b6/

Log:	Comparison.

diff --git a/pypy/module/_collections/test/test_deque.py b/pypy/module/_collections/test/test_deque.py
--- a/pypy/module/_collections/test/test_deque.py
+++ b/pypy/module/_collections/test/test_deque.py
@@ -93,21 +93,22 @@
         raises(RuntimeError, d.count, 3)
 
     def test_comparisons(self):
+        from _collections import deque
         d = deque('xabc'); d.popleft()
         for e in [d, deque('abc'), deque('ab'), deque(), list(d)]:
-            self.assertEqual(d==e, type(d)==type(e) and list(d)==list(e))
-            self.assertEqual(d!=e, not(type(d)==type(e) and list(d)==list(e)))
+            assert (d==e) == (type(d)==type(e) and list(d)==list(e))
+            assert (d!=e) == (not(type(d)==type(e) and list(d)==list(e)))
 
         args = map(deque, ('', 'a', 'b', 'ab', 'ba', 'abc', 'xba', 'xabc', 'cba'))
         for x in args:
             for y in args:
-                self.assertEqual(x == y, list(x) == list(y), (x,y))
-                self.assertEqual(x != y, list(x) != list(y), (x,y))
-                self.assertEqual(x <  y, list(x) <  list(y), (x,y))
-                self.assertEqual(x <= y, list(x) <= list(y), (x,y))
-                self.assertEqual(x >  y, list(x) >  list(y), (x,y))
-                self.assertEqual(x >= y, list(x) >= list(y), (x,y))
-                self.assertEqual(cmp(x,y), cmp(list(x),list(y)), (x,y))
+                assert (x == y) == (list(x) == list(y))
+                assert (x != y) == (list(x) != list(y))
+                assert (x <  y) == (list(x) <  list(y))
+                assert (x <= y) == (list(x) <= list(y))
+                assert (x >  y) == (list(x) >  list(y))
+                assert (x >= y) == (list(x) >= list(y))
+                assert cmp(x,y) == cmp(list(x),list(y))
 
     def test_extend(self):
         d = deque('a')

diff --git a/pypy/module/_collections/interp_deque.py b/pypy/module/_collections/interp_deque.py
--- a/pypy/module/_collections/interp_deque.py
+++ b/pypy/module/_collections/interp_deque.py
@@ -251,6 +251,32 @@
             w_currently_in_repr = ec._py_repr = space.newdict()
         return dequerepr(space, w_currently_in_repr, space.wrap(self))
 
+    def compare(self, w_other, op):
+        space = self.space
+        if not isinstance(space.interpclass_w(w_other), W_Deque):
+            return space.w_NotImplemented
+        return space.compare_by_iteration(space.wrap(self), w_other, op)
+    compare._annspecialcase_ = 'specialize:arg(2)'
+
+    @unwrap_spec('self', W_Root)
+    def lt(self, w_other):
+        return self.compare(w_other, 'lt')
+    @unwrap_spec('self', W_Root)
+    def le(self, w_other):
+        return self.compare(w_other, 'le')
+    @unwrap_spec('self', W_Root)
+    def eq(self, w_other):
+        return self.compare(w_other, 'eq')
+    @unwrap_spec('self', W_Root)
+    def ne(self, w_other):
+        return self.compare(w_other, 'ne')
+    @unwrap_spec('self', W_Root)
+    def gt(self, w_other):
+        return self.compare(w_other, 'gt')
+    @unwrap_spec('self', W_Root)
+    def ge(self, w_other):
+        return self.compare(w_other, 'ge')
+
     def get_maxlen(space, self):
         if self.maxlen == sys.maxint:
             return self.space.w_None
@@ -304,6 +330,12 @@
     __iter__ = interp2app(W_Deque.iter),
     __len__ = interp2app(W_Deque.length),
     __repr__ = interp2app(W_Deque.repr),
+    __lt__ = interp2app(W_Deque.lt),
+    __le__ = interp2app(W_Deque.le),
+    __eq__ = interp2app(W_Deque.eq),
+    __ne__ = interp2app(W_Deque.ne),
+    __gt__ = interp2app(W_Deque.gt),
+    __ge__ = interp2app(W_Deque.ge),
     maxlen = GetSetProperty(W_Deque.get_maxlen),
 )
 

From commits-noreply at bitbucket.org  Tue Feb 15 16:02:54 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 15 Feb 2011 16:02:54 +0100 (CET)
Subject: [pypy-svn] pypy collections-module: d.extend(d), d += e.
Message-ID: <20110215150254.10DA82A2036@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: collections-module
Changeset: r41968:5b92b891b703
Date: 2011-02-15 15:21 +0100
http://bitbucket.org/pypy/pypy/changeset/5b92b891b703/

Log:	d.extend(d), d += e.

diff --git a/pypy/module/_collections/test/test_deque.py b/pypy/module/_collections/test/test_deque.py
--- a/pypy/module/_collections/test/test_deque.py
+++ b/pypy/module/_collections/test/test_deque.py
@@ -111,19 +111,22 @@
                 assert cmp(x,y) == cmp(list(x),list(y))
 
     def test_extend(self):
+        from _collections import deque
         d = deque('a')
-        self.assertRaises(TypeError, d.extend, 1)
         d.extend('bcd')
-        self.assertEqual(list(d), list('abcd'))
+        assert list(d) == list('abcd')
         d.extend(d)
-        self.assertEqual(list(d), list('abcdabcd'))
+        assert list(d) == list('abcdabcd')
 
     def test_iadd(self):
+        from _collections import deque
         d = deque('a')
+        original_d = d
         d += 'bcd'
-        self.assertEqual(list(d), list('abcd'))
+        assert list(d) == list('abcd')
         d += d
-        self.assertEqual(list(d), list('abcdabcd'))
+        assert list(d) == list('abcdabcd')
+        assert original_d is d
 
     def test_extendleft(self):
         d = deque('a')

diff --git a/pypy/module/_collections/interp_deque.py b/pypy/module/_collections/interp_deque.py
--- a/pypy/module/_collections/interp_deque.py
+++ b/pypy/module/_collections/interp_deque.py
@@ -162,8 +162,11 @@
 
     @unwrap_spec('self', W_Root)
     def extend(self, w_iterable):
-        # XXX Handle case where id(deque) == id(iterable)
+        # Handle case where id(deque) == id(iterable)
         space = self.space
+        if space.is_w(space.wrap(self), w_iterable):
+            w_iterable = space.call_function(space.w_list, w_iterable)
+        #
         w_iter = space.iter(w_iterable)
         while True:
             try:
@@ -175,8 +178,17 @@
             self.append(w_obj)
 
     @unwrap_spec('self', W_Root)
+    def iadd(self, w_iterable):
+        self.extend(w_iterable)
+        return self.space.wrap(self)
+
+    @unwrap_spec('self', W_Root)
     def extendleft(self, w_iterable):
-        # XXX Handle case where id(deque) == id(iterable)
+        # Handle case where id(deque) == id(iterable)
+        space = self.space
+        if space.is_w(space.wrap(self), w_iterable):
+            w_iterable = space.call_function(space.w_list, w_iterable)
+        #
         space = self.space
         w_iter = space.iter(w_iterable)
         while True:
@@ -336,6 +348,7 @@
     __ne__ = interp2app(W_Deque.ne),
     __gt__ = interp2app(W_Deque.gt),
     __ge__ = interp2app(W_Deque.ge),
+    __iadd__ = interp2app(W_Deque.iadd),
     maxlen = GetSetProperty(W_Deque.get_maxlen),
 )
 

From commits-noreply at bitbucket.org  Tue Feb 15 16:02:56 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 15 Feb 2011 16:02:56 +0100 (CET)
Subject: [pypy-svn] pypy collections-module: In-progress.
Message-ID: <20110215150256.729C8282BEC@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: collections-module
Changeset: r41969:f59e583ba0bc
Date: 2011-02-15 15:50 +0100
http://bitbucket.org/pypy/pypy/changeset/f59e583ba0bc/

Log:	In-progress.

diff --git a/pypy/module/_collections/test/test_deque.py b/pypy/module/_collections/test/test_deque.py
--- a/pypy/module/_collections/test/test_deque.py
+++ b/pypy/module/_collections/test/test_deque.py
@@ -129,28 +129,25 @@
         assert original_d is d
 
     def test_extendleft(self):
+        from _collections import deque
         d = deque('a')
-        self.assertRaises(TypeError, d.extendleft, 1)
         d.extendleft('bcd')
-        self.assertEqual(list(d), list(reversed('abcd')))
+        assert list(d) == list(reversed('abcd'))
         d.extendleft(d)
-        self.assertEqual(list(d), list('abcddcba'))
-        d = deque()
-        d.extendleft(range(1000))
-        self.assertEqual(list(d), list(reversed(range(1000))))
-        self.assertRaises(SyntaxError, d.extendleft, fail())
+        assert list(d) == list('abcddcba')
 
     def test_getitem(self):
+        from _collections import deque
         n = 200
         d = deque(xrange(n))
         l = range(n)
         for i in xrange(n):
             d.popleft()
             l.pop(0)
-            if random.random() < 0.5:
+            if i & 1:
                 d.append(i)
                 l.append(i)
-            for j in xrange(1-len(l), len(l)):
+            for j in xrange(-len(l), len(l)):
                 assert d[j] == l[j]
 
         d = deque('superman')

diff --git a/pypy/module/_collections/interp_deque.py b/pypy/module/_collections/interp_deque.py
--- a/pypy/module/_collections/interp_deque.py
+++ b/pypy/module/_collections/interp_deque.py
@@ -289,6 +289,15 @@
     def ge(self, w_other):
         return self.compare(w_other, 'ge')
 
+    @unwrap_spec('self', W_Root)
+    def getitem(self, w_index):
+        space = self.space
+        start, stop, step = space.decode_index(w_index, self.len)
+        if step == 0:  # index only
+            return space.wrap(self.mmap.getitem(start))
+        else:
+            xxx
+
     def get_maxlen(space, self):
         if self.maxlen == sys.maxint:
             return self.space.w_None
@@ -349,6 +358,7 @@
     __gt__ = interp2app(W_Deque.gt),
     __ge__ = interp2app(W_Deque.ge),
     __iadd__ = interp2app(W_Deque.iadd),
+    __getitem__ = interp2app(W_Deque.getitem),
     maxlen = GetSetProperty(W_Deque.get_maxlen),
 )
 

From commits-noreply at bitbucket.org  Tue Feb 15 16:02:56 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 15 Feb 2011 16:02:56 +0100 (CET)
Subject: [pypy-svn] pypy default: *I* *don't* *get* *it*
Message-ID: <20110215150256.EA85A282BF7@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41970:38b7407780ec
Date: 2011-02-15 15:57 +0100
http://bitbucket.org/pypy/pypy/changeset/38b7407780ec/

Log:	*I* *don't* *get* *it*


From commits-noreply at bitbucket.org  Tue Feb 15 16:02:57 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 15 Feb 2011 16:02:57 +0100 (CET)
Subject: [pypy-svn] pypy default: merge heads
Message-ID: <20110215150257.498E8282BEC@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41971:6a46309e9e13
Date: 2011-02-15 16:01 +0100
http://bitbucket.org/pypy/pypy/changeset/6a46309e9e13/

Log:	merge heads


From commits-noreply at bitbucket.org  Tue Feb 15 16:42:58 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 15 Feb 2011 16:42:58 +0100 (CET)
Subject: [pypy-svn] pypy collections-module: getitem(), setitem(),
	and maybe delitem() if we already had rotate()
Message-ID: <20110215154258.D0295282BEA@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: collections-module
Changeset: r41972:95ee17576483
Date: 2011-02-15 16:30 +0100
http://bitbucket.org/pypy/pypy/changeset/95ee17576483/

Log:	getitem(), setitem(), and maybe delitem() if we already had rotate()

diff --git a/pypy/module/_collections/test/test_deque.py b/pypy/module/_collections/test/test_deque.py
--- a/pypy/module/_collections/test/test_deque.py
+++ b/pypy/module/_collections/test/test_deque.py
@@ -139,49 +139,31 @@
     def test_getitem(self):
         from _collections import deque
         n = 200
-        d = deque(xrange(n))
-        l = range(n)
-        for i in xrange(n):
-            d.popleft()
-            l.pop(0)
-            if i & 1:
-                d.append(i)
-                l.append(i)
-            for j in xrange(-len(l), len(l)):
-                assert d[j] == l[j]
-
-        d = deque('superman')
-        self.assertEqual(d[0], 's')
-        self.assertEqual(d[-1], 'n')
-        d = deque()
-        self.assertRaises(IndexError, d.__getitem__, 0)
-        self.assertRaises(IndexError, d.__getitem__, -1)
+        l = xrange(1000, 1000 + n)
+        d = deque(l)
+        for j in xrange(-n, n):
+            assert d[j] == l[j]
+        raises(IndexError, "d[-n-1]")
+        raises(IndexError, "d[n]")
 
     def test_setitem(self):
+        from _collections import deque
         n = 200
         d = deque(xrange(n))
         for i in xrange(n):
             d[i] = 10 * i
-        self.assertEqual(list(d), [10*i for i in xrange(n)])
+        assert list(d) == [10*i for i in xrange(n)]
         l = list(d)
-        for i in xrange(1-n, 0, -1):
+        for i in xrange(1-n, 0, -3):
             d[i] = 7*i
             l[i] = 7*i
-        self.assertEqual(list(d), l)
+        assert list(d) == l
 
     def test_delitem(self):
-        n = 500         # O(n**2) test, don't make this too big
-        d = deque(xrange(n))
-        self.assertRaises(IndexError, d.__delitem__, -n-1)
-        self.assertRaises(IndexError, d.__delitem__, n)
-        for i in xrange(n):
-            self.assertEqual(len(d), n-i)
-            j = random.randrange(-len(d), len(d))
-            val = d[j]
-            self.assertIn(val, d)
-            del d[j]
-            self.assertNotIn(val, d)
-        self.assertEqual(len(d), 0)
+        from _collections import deque
+        d = deque("abcdef")
+        del d[-2]
+        assert list(d) == list("abcdf")
 
     def test_reverse(self):
         n = 500         # O(n**2) test, don't make this too big

diff --git a/pypy/module/_collections/interp_deque.py b/pypy/module/_collections/interp_deque.py
--- a/pypy/module/_collections/interp_deque.py
+++ b/pypy/module/_collections/interp_deque.py
@@ -289,14 +289,58 @@
     def ge(self, w_other):
         return self.compare(w_other, 'ge')
 
+    def locate(self, i):
+        if i < (self.len >> 1):
+            i += self.leftindex
+            b = self.leftblock
+            while i >= BLOCKLEN:
+                b = b.rightlink
+                i -= BLOCKLEN
+        else:
+            i = i - self.len + 1     # then i <= 0
+            i += self.rightindex
+            b = self.rightblock
+            while i < 0:
+                b = b.leftlink
+                i += BLOCKLEN
+        assert i >= 0
+        return b, i
+
     @unwrap_spec('self', W_Root)
     def getitem(self, w_index):
         space = self.space
         start, stop, step = space.decode_index(w_index, self.len)
         if step == 0:  # index only
-            return space.wrap(self.mmap.getitem(start))
+            b, i = self.locate(start)
+            return b.data[i]
         else:
-            xxx
+            raise OperationError(self.w_TypeError,
+                                 self.wrap("deque[:] is not supported"))
+
+    @unwrap_spec('self', W_Root, W_Root)
+    def setitem(self, w_index, w_newobj):
+        space = self.space
+        start, stop, step = space.decode_index(w_index, self.len)
+        if step == 0:  # index only
+            b, i = self.locate(start)
+            b.data[i] = w_newobj
+        else:
+            raise OperationError(self.w_TypeError,
+                                 self.wrap("deque[:] is not supported"))
+
+    @unwrap_spec('self', W_Root)
+    def delitem(self, w_index):
+        space = self.space
+        start, stop, step = space.decode_index(w_index, self.len)
+        if step == 0:  # index only
+            # delitem() implemented in terms of rotate for simplicity and
+            # reasonable performance near the end points.
+            self.rotate(-i)
+            self.popleft()
+            self.rotate(i)
+        else:
+            raise OperationError(self.w_TypeError,
+                                 self.wrap("deque[:] is not supported"))
 
     def get_maxlen(space, self):
         if self.maxlen == sys.maxint:
@@ -359,6 +403,8 @@
     __ge__ = interp2app(W_Deque.ge),
     __iadd__ = interp2app(W_Deque.iadd),
     __getitem__ = interp2app(W_Deque.getitem),
+    __setitem__ = interp2app(W_Deque.setitem),
+    __delitem__ = interp2app(W_Deque.delitem),
     maxlen = GetSetProperty(W_Deque.get_maxlen),
 )
 

From commits-noreply at bitbucket.org  Tue Feb 15 16:42:59 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 15 Feb 2011 16:42:59 +0100 (CET)
Subject: [pypy-svn] pypy default: Remove an unnecessary check.
Message-ID: <20110215154259.6E84F282BEA@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41973:8b2a646e506b
Date: 2011-02-15 16:42 +0100
http://bitbucket.org/pypy/pypy/changeset/8b2a646e506b/

Log:	Remove an unnecessary check.

diff --git a/pypy/objspace/std/mapdict.py b/pypy/objspace/std/mapdict.py
--- a/pypy/objspace/std/mapdict.py
+++ b/pypy/objspace/std/mapdict.py
@@ -673,7 +673,6 @@
 # Magic caching
 
 class CacheEntry(object):
-    map_wref = None
     version_tag = None
     index = 0
     w_method = None # for callmethod
@@ -686,7 +685,7 @@
 
     def is_valid_for_map(self, map):
         mymap = self.map_wref()
-        if mymap is not None and mymap is map:
+        if mymap is map:     # also handles the case self.map_wref()->None
             version_tag = map.terminator.w_cls.version_tag()
             if version_tag is self.version_tag:
                 # everything matches, it's incredibly fast

From commits-noreply at bitbucket.org  Tue Feb 15 17:26:05 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 15 Feb 2011 17:26:05 +0100 (CET)
Subject: [pypy-svn] pypy collections-module: rotate().
Message-ID: <20110215162605.323F9282BEA@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: collections-module
Changeset: r41974:28ab4b7a2149
Date: 2011-02-15 16:45 +0100
http://bitbucket.org/pypy/pypy/changeset/28ab4b7a2149/

Log:	rotate().

diff --git a/pypy/module/_collections/interp_deque.py b/pypy/module/_collections/interp_deque.py
--- a/pypy/module/_collections/interp_deque.py
+++ b/pypy/module/_collections/interp_deque.py
@@ -246,6 +246,24 @@
         self.modified()
         return w_obj
 
+    @unwrap_spec('self', int)
+    def rotate(self, n):
+        len = self.len
+        if len == 0:
+            return
+        halflen = (len+1) >> 1
+        if n > halflen or n < -halflen:
+            n %= len
+            if n > halflen:
+                n -= len
+        i = 0
+        while i < n:
+            self.appendleft(self.pop())
+            i += 1
+        while i > n:
+            self.append(self.popleft())
+            i -= 1
+
     @unwrap_spec('self')
     def iter(self):
         return W_DequeIter(self)
@@ -335,6 +353,7 @@
         if step == 0:  # index only
             # delitem() implemented in terms of rotate for simplicity and
             # reasonable performance near the end points.
+            i = start
             self.rotate(-i)
             self.popleft()
             self.rotate(i)
@@ -391,6 +410,7 @@
     extendleft = interp2app(W_Deque.extendleft),
     pop        = interp2app(W_Deque.pop),
     popleft    = interp2app(W_Deque.popleft),
+    rotate     = interp2app(W_Deque.rotate),
     __weakref__ = make_weakref_descr(W_Deque),
     __iter__ = interp2app(W_Deque.iter),
     __len__ = interp2app(W_Deque.length),

From commits-noreply at bitbucket.org  Tue Feb 15 17:26:05 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 15 Feb 2011 17:26:05 +0100 (CET)
Subject: [pypy-svn] pypy collections-module: reverse().
Message-ID: <20110215162605.DE9AD282BEA@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: collections-module
Changeset: r41975:cc16fdf18b64
Date: 2011-02-15 17:17 +0100
http://bitbucket.org/pypy/pypy/changeset/cc16fdf18b64/

Log:	reverse().

diff --git a/pypy/module/_collections/test/test_deque.py b/pypy/module/_collections/test/test_deque.py
--- a/pypy/module/_collections/test/test_deque.py
+++ b/pypy/module/_collections/test/test_deque.py
@@ -166,16 +166,10 @@
         assert list(d) == list("abcdf")
 
     def test_reverse(self):
-        n = 500         # O(n**2) test, don't make this too big
-        data = [random.random() for i in range(n)]
-        for i in range(n):
-            d = deque(data[:i])
-            r = d.reverse()
-            self.assertEqual(list(d), list(reversed(data[:i])))
-            self.assert_(r is None)
-            d.reverse()
-            self.assertEqual(list(d), data[:i])
-        self.assertRaises(TypeError, d.reverse, 1)          # Arity is zero
+        from _collections import deque
+        d = deque(xrange(1000, 1200))
+        d.reverse()
+        assert list(d) == list(reversed(range(1000, 1200)))
 
     def test_rotate(self):
         s = tuple('abcde')

diff --git a/pypy/module/_collections/interp_deque.py b/pypy/module/_collections/interp_deque.py
--- a/pypy/module/_collections/interp_deque.py
+++ b/pypy/module/_collections/interp_deque.py
@@ -246,6 +246,23 @@
         self.modified()
         return w_obj
 
+    @unwrap_spec('self')
+    def reverse(self):
+        li = self.leftindex
+        lb = self.leftblock
+        ri = self.rightindex
+        rb = self.rightblock
+        while lb is not rb or li < ri:
+            lb.data[li], rb.data[ri] = rb.data[ri], lb.data[li]
+            li += 1
+            if li >= BLOCKLEN:
+                lb = lb.rightlink
+                li = 0
+            ri -= 1
+            if ri < 0:
+                rb = rb.leftlink
+                ri = BLOCKLEN - 1
+
     @unwrap_spec('self', int)
     def rotate(self, n):
         len = self.len
@@ -410,6 +427,7 @@
     extendleft = interp2app(W_Deque.extendleft),
     pop        = interp2app(W_Deque.pop),
     popleft    = interp2app(W_Deque.popleft),
+    reverse    = interp2app(W_Deque.reverse),
     rotate     = interp2app(W_Deque.rotate),
     __weakref__ = make_weakref_descr(W_Deque),
     __iter__ = interp2app(W_Deque.iter),

From commits-noreply at bitbucket.org  Tue Feb 15 17:26:06 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 15 Feb 2011 17:26:06 +0100 (CET)
Subject: [pypy-svn] pypy collections-module: rotate().
Message-ID: <20110215162606.B06B5282C19@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: collections-module
Changeset: r41976:8b2c6f893aee
Date: 2011-02-15 17:20 +0100
http://bitbucket.org/pypy/pypy/changeset/8b2c6f893aee/

Log:	rotate().

diff --git a/pypy/module/_collections/test/test_deque.py b/pypy/module/_collections/test/test_deque.py
--- a/pypy/module/_collections/test/test_deque.py
+++ b/pypy/module/_collections/test/test_deque.py
@@ -172,57 +172,24 @@
         assert list(d) == list(reversed(range(1000, 1200)))
 
     def test_rotate(self):
+        from _collections import deque
         s = tuple('abcde')
         n = len(s)
 
         d = deque(s)
         d.rotate(1)             # verify rot(1)
-        self.assertEqual(''.join(d), 'eabcd')
+        assert ''.join(d) == 'eabcd'
 
         d = deque(s)
         d.rotate(-1)            # verify rot(-1)
-        self.assertEqual(''.join(d), 'bcdea')
+        assert ''.join(d) == 'bcdea'
         d.rotate()              # check default to 1
-        self.assertEqual(tuple(d), s)
+        assert tuple(d) == s
 
-        for i in xrange(n*3):
-            d = deque(s)
-            e = deque(d)
-            d.rotate(i)         # check vs. rot(1) n times
-            for j in xrange(i):
-                e.rotate(1)
-            self.assertEqual(tuple(d), tuple(e))
-            d.rotate(-i)        # check that it works in reverse
-            self.assertEqual(tuple(d), s)
-            e.rotate(n-i)       # check that it wraps forward
-            self.assertEqual(tuple(e), s)
-
-        for i in xrange(n*3):
-            d = deque(s)
-            e = deque(d)
-            d.rotate(-i)
-            for j in xrange(i):
-                e.rotate(-1)    # check vs. rot(-1) n times
-            self.assertEqual(tuple(d), tuple(e))
-            d.rotate(i)         # check that it works in reverse
-            self.assertEqual(tuple(d), s)
-            e.rotate(i-n)       # check that it wraps backaround
-            self.assertEqual(tuple(e), s)
-
-        d = deque(s)
-        e = deque(s)
-        e.rotate(BIG+17)        # verify on long series of rotates
-        dr = d.rotate
-        for i in xrange(BIG+17):
-            dr()
-        self.assertEqual(tuple(d), tuple(e))
-
-        self.assertRaises(TypeError, d.rotate, 'x')   # Wrong arg type
-        self.assertRaises(TypeError, d.rotate, 1, 10) # Too many args
-
-        d = deque()
-        d.rotate()              # rotate an empty deque
-        self.assertEqual(d, deque())
+        d.rotate(500000002)
+        assert tuple(d) == tuple('deabc')
+        d.rotate(-5000002)
+        assert tuple(d) == tuple(s)
 
     def test_len(self):
         d = deque('ab')

diff --git a/pypy/module/_collections/interp_deque.py b/pypy/module/_collections/interp_deque.py
--- a/pypy/module/_collections/interp_deque.py
+++ b/pypy/module/_collections/interp_deque.py
@@ -264,7 +264,7 @@
                 ri = BLOCKLEN - 1
 
     @unwrap_spec('self', int)
-    def rotate(self, n):
+    def rotate(self, n=1):
         len = self.len
         if len == 0:
             return

From commits-noreply at bitbucket.org  Tue Feb 15 17:26:08 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 15 Feb 2011 17:26:08 +0100 (CET)
Subject: [pypy-svn] pypy collections-module: len().
Message-ID: <20110215162608.1D4DA282C19@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: collections-module
Changeset: r41977:a85cb55fcf53
Date: 2011-02-15 17:21 +0100
http://bitbucket.org/pypy/pypy/changeset/a85cb55fcf53/

Log:	len().

diff --git a/pypy/module/_collections/test/test_deque.py b/pypy/module/_collections/test/test_deque.py
--- a/pypy/module/_collections/test/test_deque.py
+++ b/pypy/module/_collections/test/test_deque.py
@@ -192,20 +192,21 @@
         assert tuple(d) == tuple(s)
 
     def test_len(self):
+        from _collections import deque
         d = deque('ab')
-        self.assertEqual(len(d), 2)
+        assert len(d) == 2
         d.popleft()
-        self.assertEqual(len(d), 1)
+        assert len(d) == 1
         d.pop()
-        self.assertEqual(len(d), 0)
-        self.assertRaises(IndexError, d.pop)
-        self.assertEqual(len(d), 0)
+        assert len(d) == 0
+        raises(IndexError, d.pop)
+        assert len(d) == 0
         d.append('c')
-        self.assertEqual(len(d), 1)
+        assert len(d) == 1
         d.appendleft('d')
-        self.assertEqual(len(d), 2)
+        assert len(d) == 2
         d.clear()
-        self.assertEqual(len(d), 0)
+        assert len(d) == 0
 
     def test_underflow(self):
         d = deque()

From commits-noreply at bitbucket.org  Tue Feb 15 17:26:09 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 15 Feb 2011 17:26:09 +0100 (CET)
Subject: [pypy-svn] pypy collections-module: remove().
Message-ID: <20110215162609.61D7A282C19@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: collections-module
Changeset: r41978:7ae2fd1e1675
Date: 2011-02-15 17:25 +0100
http://bitbucket.org/pypy/pypy/changeset/7ae2fd1e1675/

Log:	remove().

diff --git a/pypy/module/_collections/test/test_deque.py b/pypy/module/_collections/test/test_deque.py
--- a/pypy/module/_collections/test/test_deque.py
+++ b/pypy/module/_collections/test/test_deque.py
@@ -200,6 +200,7 @@
         d.pop()
         assert len(d) == 0
         raises(IndexError, d.pop)
+        raises(IndexError, d.popleft)
         assert len(d) == 0
         d.append('c')
         assert len(d) == 1
@@ -207,44 +208,17 @@
         assert len(d) == 2
         d.clear()
         assert len(d) == 0
-
-    def test_underflow(self):
-        d = deque()
-        self.assertRaises(IndexError, d.pop)
-        self.assertRaises(IndexError, d.popleft)
-
-    def test_clear(self):
-        d = deque(xrange(100))
-        self.assertEqual(len(d), 100)
-        d.clear()
-        self.assertEqual(len(d), 0)
-        self.assertEqual(list(d), [])
-        d.clear()               # clear an emtpy deque
-        self.assertEqual(list(d), [])
+        assert list(d) == []
 
     def test_remove(self):
+        from _collections import deque
         d = deque('abcdefghcij')
         d.remove('c')
-        self.assertEqual(d, deque('abdefghcij'))
+        assert d == deque('abdefghcij')
         d.remove('c')
-        self.assertEqual(d, deque('abdefghij'))
-        self.assertRaises(ValueError, d.remove, 'c')
-        self.assertEqual(d, deque('abdefghij'))
-
-        # Handle comparison errors
-        d = deque(['a', 'b', BadCmp(), 'c'])
-        e = deque(d)
-        self.assertRaises(RuntimeError, d.remove, 'c')
-        for x, y in zip(d, e):
-            # verify that original order and values are retained.
-            self.assertTrue(x is y)
-
-        # Handle evil mutator
-        for match in (True, False):
-            d = deque(['ab'])
-            d.extend([MutateCmp(d, match), 'c'])
-            self.assertRaises(IndexError, d.remove, 'c')
-            self.assertEqual(d, deque())
+        assert d == deque('abdefghij')
+        raises(ValueError, d.remove, 'c')
+        assert d == deque('abdefghij')
 
     def test_repr(self):
         d = deque(xrange(200))

diff --git a/pypy/module/_collections/interp_deque.py b/pypy/module/_collections/interp_deque.py
--- a/pypy/module/_collections/interp_deque.py
+++ b/pypy/module/_collections/interp_deque.py
@@ -246,6 +246,27 @@
         self.modified()
         return w_obj
 
+    @unwrap_spec('self', W_Root)
+    def remove(self, w_x):
+        space = self.space
+        block = self.leftblock
+        index = self.leftindex
+        lock = self.getlock()
+        for i in range(self.len):
+            w_item = block.data[index]
+            equal = space.eq_w(w_item, w_x)
+            self.checklock(lock)
+            if equal:
+                self.del_item(i)
+                return
+            # Advance the block/index pair
+            index += 1
+            if index >= BLOCKLEN:
+                block = block.rightlink
+                index = 0
+        raise OperationError(space.w_ValueError,
+                             space.wrap("deque.remove(x): x not in deque"))
+
     @unwrap_spec('self')
     def reverse(self):
         li = self.leftindex
@@ -341,6 +362,13 @@
         assert i >= 0
         return b, i
 
+    def del_item(self, i):
+        # delitem() implemented in terms of rotate for simplicity and
+        # reasonable performance near the end points.
+        self.rotate(-i)
+        self.popleft()
+        self.rotate(i)
+
     @unwrap_spec('self', W_Root)
     def getitem(self, w_index):
         space = self.space
@@ -368,12 +396,7 @@
         space = self.space
         start, stop, step = space.decode_index(w_index, self.len)
         if step == 0:  # index only
-            # delitem() implemented in terms of rotate for simplicity and
-            # reasonable performance near the end points.
-            i = start
-            self.rotate(-i)
-            self.popleft()
-            self.rotate(i)
+            self.del_item(start)
         else:
             raise OperationError(self.w_TypeError,
                                  self.wrap("deque[:] is not supported"))
@@ -427,6 +450,7 @@
     extendleft = interp2app(W_Deque.extendleft),
     pop        = interp2app(W_Deque.pop),
     popleft    = interp2app(W_Deque.popleft),
+    remove     = interp2app(W_Deque.remove),
     reverse    = interp2app(W_Deque.reverse),
     rotate     = interp2app(W_Deque.rotate),
     __weakref__ = make_weakref_descr(W_Deque),

From commits-noreply at bitbucket.org  Tue Feb 15 17:58:42 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 15 Feb 2011 17:58:42 +0100 (CET)
Subject: [pypy-svn] pypy collections-module: repr() works.
Message-ID: <20110215165842.5E46C282BEA@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: collections-module
Changeset: r41979:20675ebb30d5
Date: 2011-02-15 17:30 +0100
http://bitbucket.org/pypy/pypy/changeset/20675ebb30d5/

Log:	repr() works.

diff --git a/pypy/module/_collections/test/test_deque.py b/pypy/module/_collections/test/test_deque.py
--- a/pypy/module/_collections/test/test_deque.py
+++ b/pypy/module/_collections/test/test_deque.py
@@ -221,11 +221,12 @@
         assert d == deque('abdefghij')
 
     def test_repr(self):
-        d = deque(xrange(200))
+        from _collections import deque
+        d = deque(xrange(20))
         e = eval(repr(d))
-        self.assertEqual(list(d), list(e))
+        assert d == e
         d.append(d)
-        self.assertIn('...', repr(d))
+        assert '...' in repr(d)
 
     def test_print(self):
         d = deque(xrange(200))

From commits-noreply at bitbucket.org  Tue Feb 15 17:58:43 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 15 Feb 2011 17:58:43 +0100 (CET)
Subject: [pypy-svn] pypy collections-module: Hash.
Message-ID: <20110215165843.13335282BEA@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: collections-module
Changeset: r41980:ddd748e32e70
Date: 2011-02-15 17:39 +0100
http://bitbucket.org/pypy/pypy/changeset/ddd748e32e70/

Log:	Hash.

diff --git a/pypy/module/_collections/test/test_deque.py b/pypy/module/_collections/test/test_deque.py
--- a/pypy/module/_collections/test/test_deque.py
+++ b/pypy/module/_collections/test/test_deque.py
@@ -228,26 +228,9 @@
         d.append(d)
         assert '...' in repr(d)
 
-    def test_print(self):
-        d = deque(xrange(200))
-        d.append(d)
-        test_support.unlink(test_support.TESTFN)
-        fo = open(test_support.TESTFN, "wb")
-        try:
-            print >> fo, d,
-            fo.close()
-            fo = open(test_support.TESTFN, "rb")
-            self.assertEqual(fo.read(), repr(d))
-        finally:
-            fo.close()
-            test_support.unlink(test_support.TESTFN)
-
-    def test_init(self):
-        self.assertRaises(TypeError, deque, 'abc', 2, 3);
-        self.assertRaises(TypeError, deque, 1);
-
     def test_hash(self):
-        self.assertRaises(TypeError, hash, deque('abc'))
+        from _collections import deque
+        raises(TypeError, hash, deque('abc'))
 
     def test_long_steadystate_queue_popleft(self):
         for size in (0, 1, 2, 100, 1000):

diff --git a/pypy/module/_collections/interp_deque.py b/pypy/module/_collections/interp_deque.py
--- a/pypy/module/_collections/interp_deque.py
+++ b/pypy/module/_collections/interp_deque.py
@@ -463,6 +463,7 @@
     __ne__ = interp2app(W_Deque.ne),
     __gt__ = interp2app(W_Deque.gt),
     __ge__ = interp2app(W_Deque.ge),
+    __hash__ = None,
     __iadd__ = interp2app(W_Deque.iadd),
     __getitem__ = interp2app(W_Deque.getitem),
     __setitem__ = interp2app(W_Deque.setitem),

From commits-noreply at bitbucket.org  Tue Feb 15 17:58:43 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 15 Feb 2011 17:58:43 +0100 (CET)
Subject: [pypy-svn] pypy collections-module: __copy__().
Message-ID: <20110215165843.C9AEB282BEA@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: collections-module
Changeset: r41981:af7947ac8cd4
Date: 2011-02-15 17:46 +0100
http://bitbucket.org/pypy/pypy/changeset/af7947ac8cd4/

Log:	__copy__().

diff --git a/pypy/module/_collections/test/test_deque.py b/pypy/module/_collections/test/test_deque.py
--- a/pypy/module/_collections/test/test_deque.py
+++ b/pypy/module/_collections/test/test_deque.py
@@ -232,110 +232,24 @@
         from _collections import deque
         raises(TypeError, hash, deque('abc'))
 
-    def test_long_steadystate_queue_popleft(self):
-        for size in (0, 1, 2, 100, 1000):
-            d = deque(xrange(size))
-            append, pop = d.append, d.popleft
-            for i in xrange(size, BIG):
-                append(i)
-                x = pop()
-                if x != i - size:
-                    self.assertEqual(x, i-size)
-            self.assertEqual(list(d), range(BIG-size, BIG))
-
-    def test_long_steadystate_queue_popright(self):
-        for size in (0, 1, 2, 100, 1000):
-            d = deque(reversed(xrange(size)))
-            append, pop = d.appendleft, d.pop
-            for i in xrange(size, BIG):
-                append(i)
-                x = pop()
-                if x != i - size:
-                    self.assertEqual(x, i-size)
-            self.assertEqual(list(reversed(list(d))), range(BIG-size, BIG))
-
-    def test_big_queue_popleft(self):
-        pass
-        d = deque()
-        append, pop = d.append, d.popleft
-        for i in xrange(BIG):
-            append(i)
-        for i in xrange(BIG):
-            x = pop()
-            if x != i:
-                self.assertEqual(x, i)
-
-    def test_big_queue_popright(self):
-        d = deque()
-        append, pop = d.appendleft, d.pop
-        for i in xrange(BIG):
-            append(i)
-        for i in xrange(BIG):
-            x = pop()
-            if x != i:
-                self.assertEqual(x, i)
-
-    def test_big_stack_right(self):
-        d = deque()
-        append, pop = d.append, d.pop
-        for i in xrange(BIG):
-            append(i)
-        for i in reversed(xrange(BIG)):
-            x = pop()
-            if x != i:
-                self.assertEqual(x, i)
-        self.assertEqual(len(d), 0)
-
-    def test_big_stack_left(self):
-        d = deque()
-        append, pop = d.appendleft, d.popleft
-        for i in xrange(BIG):
-            append(i)
-        for i in reversed(xrange(BIG)):
-            x = pop()
-            if x != i:
-                self.assertEqual(x, i)
-        self.assertEqual(len(d), 0)
-
     def test_roundtrip_iter_init(self):
+        from _collections import deque
         d = deque(xrange(200))
         e = deque(d)
-        self.assertNotEqual(id(d), id(e))
-        self.assertEqual(list(d), list(e))
-
-    def test_pickle(self):
-        d = deque(xrange(200))
-        for i in range(pickle.HIGHEST_PROTOCOL + 1):
-            s = pickle.dumps(d, i)
-            e = pickle.loads(s)
-            self.assertNotEqual(id(d), id(e))
-            self.assertEqual(list(d), list(e))
-
-##    def test_pickle_recursive(self):
-##        d = deque('abc')
-##        d.append(d)
-##        for i in range(pickle.HIGHEST_PROTOCOL + 1):
-##            e = pickle.loads(pickle.dumps(d, i))
-##            self.assertNotEqual(id(d), id(e))
-##            self.assertEqual(id(e), id(e[-1]))
-
-    def test_deepcopy(self):
-        mut = [10]
-        d = deque([mut])
-        e = copy.deepcopy(d)
-        self.assertEqual(list(d), list(e))
-        mut[0] = 11
-        self.assertNotEqual(id(d), id(e))
-        self.assertNotEqual(list(d), list(e))
+        assert d is not e
+        assert d == e
+        assert list(d) == list(e)
 
     def test_copy(self):
+        from _collections import deque
+        import copy
         mut = [10]
         d = deque([mut])
         e = copy.copy(d)
-        self.assertEqual(list(d), list(e))
+        assert d is not e
+        assert d == e
         mut[0] = 11
-        self.assertNotEqual(id(d), id(e))
-        self.assertEqual(list(d), list(e))
+        assert d == e
 
     def test_reversed(self):
         for s in ('abcd', xrange(2000)):

diff --git a/pypy/module/_collections/interp_deque.py b/pypy/module/_collections/interp_deque.py
--- a/pypy/module/_collections/interp_deque.py
+++ b/pypy/module/_collections/interp_deque.py
@@ -401,6 +401,16 @@
             raise OperationError(self.w_TypeError,
                                  self.wrap("deque[:] is not supported"))
 
+    @unwrap_spec('self')
+    def copy(self):
+        space = self.space
+        w_self = space.wrap(self)
+        if self.maxlen == sys.maxint:
+            return space.call_function(space.type(w_self), w_self)
+        else:
+            return space.call_function(space.type(w_self), w_self,
+                                       space.wrap(self.maxint))
+
     def get_maxlen(space, self):
         if self.maxlen == sys.maxint:
             return self.space.w_None
@@ -468,6 +478,7 @@
     __getitem__ = interp2app(W_Deque.getitem),
     __setitem__ = interp2app(W_Deque.setitem),
     __delitem__ = interp2app(W_Deque.delitem),
+    __copy__ = interp2app(W_Deque.copy),
     maxlen = GetSetProperty(W_Deque.get_maxlen),
 )
 

From commits-noreply at bitbucket.org  Tue Feb 15 17:58:45 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 15 Feb 2011 17:58:45 +0100 (CET)
Subject: [pypy-svn] pypy collections-module: __reversed__().
Message-ID: <20110215165845.AD64C282C19@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: collections-module
Changeset: r41982:5ae43fa4657b
Date: 2011-02-15 17:49 +0100
http://bitbucket.org/pypy/pypy/changeset/5ae43fa4657b/

Log:	__reversed__().

diff --git a/pypy/module/_collections/test/test_deque.py b/pypy/module/_collections/test/test_deque.py
--- a/pypy/module/_collections/test/test_deque.py
+++ b/pypy/module/_collections/test/test_deque.py
@@ -252,17 +252,9 @@
         assert d == e
 
     def test_reversed(self):
-        for s in ('abcd', xrange(2000)):
-            self.assertEqual(list(reversed(deque(s))), list(reversed(s)))
-
-    def test_gc_doesnt_blowup(self):
-        import gc
-        # This used to assert-fail in deque_traverse() under a debug
-        # build, or run wild with a NULL pointer in a release build.
-        d = deque()
-        for i in xrange(100):
-            d.append(1)
-            gc.collect()
+        from _collections import deque
+        for s in ('abcd', xrange(200)):
+            assert list(reversed(deque(s))) == list(reversed(s))
 
     def test_container_iterator(self):
         # Bug #3680: tp_traverse was not implemented for deque iterator objects

diff --git a/pypy/module/_collections/interp_deque.py b/pypy/module/_collections/interp_deque.py
--- a/pypy/module/_collections/interp_deque.py
+++ b/pypy/module/_collections/interp_deque.py
@@ -307,6 +307,10 @@
         return W_DequeIter(self)
 
     @unwrap_spec('self')
+    def reviter(self):
+        return W_DequeRevIter(self)
+
+    @unwrap_spec('self')
     def length(self):
         return self.space.wrap(self.len)
 
@@ -465,6 +469,7 @@
     rotate     = interp2app(W_Deque.rotate),
     __weakref__ = make_weakref_descr(W_Deque),
     __iter__ = interp2app(W_Deque.iter),
+    __reversed__ = interp2app(W_Deque.reviter),
     __len__ = interp2app(W_Deque.length),
     __repr__ = interp2app(W_Deque.repr),
     __lt__ = interp2app(W_Deque.lt),
@@ -520,3 +525,40 @@
 W_DequeIter.typedef.acceptable_as_base_class = False
 
 # ------------------------------------------------------------
+
+class W_DequeRevIter(Wrappable):
+    def __init__(self, deque):
+        self.space = deque.space
+        self.deque = deque
+        self.block = deque.rightblock
+        self.index = deque.rightindex
+        self.counter = deque.len
+        self.lock = deque.getlock()
+        check_nonneg(self.index)
+
+    @unwrap_spec('self')
+    def iter(self):
+        return self.space.wrap(self)
+
+    @unwrap_spec('self')
+    def next(self):
+        self.deque.checklock(self.lock)
+        if self.counter == 0:
+            raise OperationError(self.space.w_StopIteration, self.space.w_None)
+        self.counter -= 1
+        ri = self.index
+        w_x = self.block.data[ri]
+        ri -= 1
+        if ri < 0:
+            self.block = self.block.leftlink
+            ri = BLOCKLEN - 1
+        self.index = ri
+        return w_x
+
+W_DequeRevIter.typedef = TypeDef("deque_reverse_iterator",
+    __iter__ = interp2app(W_DequeRevIter.iter),
+    next = interp2app(W_DequeRevIter.next),
+)
+W_DequeRevIter.typedef.acceptable_as_base_class = False
+
+# ------------------------------------------------------------

From commits-noreply at bitbucket.org  Tue Feb 15 17:58:46 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 15 Feb 2011 17:58:46 +0100 (CET)
Subject: [pypy-svn] pypy collections-module: __reduce__().
Message-ID: <20110215165846.CF7CC282C19@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: collections-module
Changeset: r41983:eddc2340f69e
Date: 2011-02-15 17:57 +0100
http://bitbucket.org/pypy/pypy/changeset/eddc2340f69e/

Log:	__reduce__().

diff --git a/pypy/module/_collections/test/test_deque.py b/pypy/module/_collections/test/test_deque.py
--- a/pypy/module/_collections/test/test_deque.py
+++ b/pypy/module/_collections/test/test_deque.py
@@ -240,6 +240,31 @@
         assert d == e
         assert list(d) == list(e)
 
+    def test_reduce(self):
+        from _collections import deque
+        #
+        d = deque('hello world')
+        r = d.__reduce__()
+        assert r == (deque, (list('hello world'),))
+        #
+        d = deque('hello world', 42)
+        r = d.__reduce__()
+        assert r == (deque, (list('hello world'), 42))
+        #
+        class D(deque):
+            pass
+        d = D('hello world')
+        d.a = 5
+        r = d.__reduce__()
+        assert r == (D, (list('hello world'), None), {'a': 5})
+        #
+        class D(deque):
+            pass
+        d = D('hello world', 42)
+        d.a = 5
+        r = d.__reduce__()
+        assert r == (D, (list('hello world'), 42), {'a': 5})
+
     def test_copy(self):
         from _collections import deque
         import copy

diff --git a/pypy/module/_collections/interp_deque.py b/pypy/module/_collections/interp_deque.py
--- a/pypy/module/_collections/interp_deque.py
+++ b/pypy/module/_collections/interp_deque.py
@@ -415,6 +415,29 @@
             return space.call_function(space.type(w_self), w_self,
                                        space.wrap(self.maxint))
 
+    @unwrap_spec('self')
+    def reduce(self):
+        space = self.space
+        w_self = space.wrap(self)
+        w_type = space.type(w_self)
+        w_dict = space.findattr(w_self, space.wrap('__dict__'))
+        w_list = space.call_function(space.w_list, w_self)
+        if w_dict is None:
+            if self.maxlen == sys.maxint:
+                result = [
+                    w_type, space.newtuple([w_list])]
+            else:
+                result = [
+                    w_type, space.newtuple([w_list, space.wrap(self.maxlen)])]
+        else:
+            if self.maxlen == sys.maxint:
+                w_len = space.w_None
+            else:
+                w_len = space.wrap(self.maxlen)
+            result = [
+                w_type, space.newtuple([w_list, w_len]), w_dict]
+        return space.newtuple(result)
+
     def get_maxlen(space, self):
         if self.maxlen == sys.maxint:
             return self.space.w_None
@@ -484,6 +507,7 @@
     __setitem__ = interp2app(W_Deque.setitem),
     __delitem__ = interp2app(W_Deque.delitem),
     __copy__ = interp2app(W_Deque.copy),
+    __reduce__ = interp2app(W_Deque.reduce),
     maxlen = GetSetProperty(W_Deque.get_maxlen),
 )
 

From commits-noreply at bitbucket.org  Tue Feb 15 18:19:18 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Tue, 15 Feb 2011 18:19:18 +0100 (CET)
Subject: [pypy-svn] pypy default: make sure that the code objects have a
	co_filename in	the form 'zipfile.zip/pyfile.py'
Message-ID: <20110215171918.10BD9282BEA@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r41984:088b8b251525
Date: 2011-02-15 16:48 +0100
http://bitbucket.org/pypy/pypy/changeset/088b8b251525/

Log:	make sure that the code objects have a co_filename in the form
	'zipfile.zip/pyfile.py'

diff --git a/pypy/module/zipimport/interp_zipimport.py b/pypy/module/zipimport/interp_zipimport.py
--- a/pypy/module/zipimport/interp_zipimport.py
+++ b/pypy/module/zipimport/interp_zipimport.py
@@ -237,6 +237,7 @@
         last_exc = None
         for compiled, is_package, ext in ENUMERATE_EXTS:
             fname = filename + ext
+            co_filename = self.filename + os.path.sep + fname
             try:
                 zip_file = RZipFile(self.filename, 'r')
                 try:
@@ -253,10 +254,10 @@
                     pkgpath = None
                 try:
                     if compiled:
-                        return self.import_pyc_file(space, fullname, fname,
+                        return self.import_pyc_file(space, fullname, co_filename,
                                                     buf, pkgpath)
                     else:
-                        return self.import_py_file(space, fullname, fname,
+                        return self.import_py_file(space, fullname, co_filename,
                                                    buf, pkgpath)
                 except OperationError, e:
                     last_exc = e

diff --git a/pypy/module/zipimport/test/test_zipimport.py b/pypy/module/zipimport/test/test_zipimport.py
--- a/pypy/module/zipimport/test/test_zipimport.py
+++ b/pypy/module/zipimport/test/test_zipimport.py
@@ -338,6 +338,18 @@
         import zipimport
         assert sys.path_hooks.count(zipimport.zipimporter) == 1
 
+    def test_co_filename(self):
+        self.writefile('mymodule.py', """
+def fn():
+    return fn.func_code.co_filename
+""")
+        import os
+        import mymodule
+        co_filename = mymodule.fn()
+        expected = self.zipfile + os.sep + 'mymodule.py'
+        assert co_filename == expected
+
+
 class AppTestZipimportDeflated(AppTestZipimport):
     compression = ZIP_DEFLATED
 

From commits-noreply at bitbucket.org  Tue Feb 15 18:21:27 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Tue, 15 Feb 2011 18:21:27 +0100 (CET)
Subject: [pypy-svn] pypy default: (hopefully) fix translation
Message-ID: <20110215172127.C895E282BEA@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r41985:bb95747a5bd8
Date: 2011-02-15 18:21 +0100
http://bitbucket.org/pypy/pypy/changeset/bb95747a5bd8/

Log:	(hopefully) fix translation

diff --git a/pypy/objspace/std/mapdict.py b/pypy/objspace/std/mapdict.py
--- a/pypy/objspace/std/mapdict.py
+++ b/pypy/objspace/std/mapdict.py
@@ -683,6 +683,7 @@
         map = w_obj._get_mapdict_map()
         return self.is_valid_for_map(map)
 
+    @jit.dont_look_inside
     def is_valid_for_map(self, map):
         mymap = self.map_wref()
         if mymap is map:     # also handles the case self.map_wref()->None
@@ -704,6 +705,7 @@
     num_entries = len(pycode.co_names_w)
     pycode._mapdict_caches = [INVALID_CACHE_ENTRY] * num_entries
 
+ at jit.dont_look_inside
 def _fill_cache(pycode, nameindex, map, version_tag, index, w_method=None):
     entry = pycode._mapdict_caches[nameindex]
     if entry is INVALID_CACHE_ENTRY:

From commits-noreply at bitbucket.org  Tue Feb 15 18:31:42 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 15 Feb 2011 18:31:42 +0100 (CET)
Subject: [pypy-svn] pypy collections-module: Write defaultdict... cheating.
Message-ID: <20110215173142.1ACB52A202C@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: collections-module
Changeset: r41986:2a2352811697
Date: 2011-02-15 18:31 +0100
http://bitbucket.org/pypy/pypy/changeset/2a2352811697/

Log:	Write defaultdict... cheating.

diff --git a/pypy/module/_collections/__init__.py b/pypy/module/_collections/__init__.py
--- a/pypy/module/_collections/__init__.py
+++ b/pypy/module/_collections/__init__.py
@@ -3,7 +3,9 @@
 from pypy.interpreter.mixedmodule import MixedModule
 
 class Module(MixedModule):
-    appleveldefs = {}
+    appleveldefs = {
+        'defaultdict': 'app_defaultdict.defaultdict',
+        }
 
     interpleveldefs = {
         'deque' : 'interp_deque.W_Deque',

diff --git a/pypy/module/_collections/app_defaultdict.py b/pypy/module/_collections/app_defaultdict.py
new file mode 100644
--- /dev/null
+++ b/pypy/module/_collections/app_defaultdict.py
@@ -0,0 +1,56 @@
+# NOT_RPYTHON
+
+# For now this is here, living at app-level.
+#
+# The issue is that for now we don't support writing interp-level
+# subclasses of Wrappable that inherit at app-level from a type like
+# 'dict'.
+
+
+class defaultdict(dict):
+    
+    def __init__(self, *args, **kwds):
+        self.default_factory = None
+        if 'default_factory' in kwds:
+            self.default_factory = kwds.pop('default_factory')
+        elif len(args) > 0 and (callable(args[0]) or args[0] is None):
+            self.default_factory = args[0]
+            args = args[1:]
+        super(defaultdict, self).__init__(*args, **kwds)
+ 
+    def __missing__(self, key):
+        # from defaultdict docs
+        if self.default_factory is None: 
+            raise KeyError(key)
+        self[key] = value = self.default_factory()
+        return value
+
+    def __repr__(self, recurse=set()):
+        # XXX not thread-safe, but good enough
+        if id(self) in recurse:
+            return "defaultdict(...)"
+        try:
+            recurse.add(id(self))
+            return "defaultdict(%s, %s)" % (repr(self.default_factory), super(defaultdict, self).__repr__())
+        finally:
+            recurse.remove(id(self))
+
+    def copy(self):
+        return type(self)(self, default_factory=self.default_factory)
+    
+    def __copy__(self):
+        return self.copy()
+
+    def __reduce__(self):
+        """
+        __reduce__ must return a 5-tuple as follows:
+
+           - factory function
+           - tuple of args for the factory function
+           - additional state (here None)
+           - sequence iterator (here None)
+           - dictionary iterator (yielding successive (key, value) pairs
+
+           This API is used by pickle.py and copy.py.
+        """
+        return (type(self), (self.default_factory,), None, None, self.iteritems())

diff --git a/pypy/module/_collections/test/test_defaultdict.py b/pypy/module/_collections/test/test_defaultdict.py
new file mode 100644
--- /dev/null
+++ b/pypy/module/_collections/test/test_defaultdict.py
@@ -0,0 +1,18 @@
+import py
+from pypy.conftest import gettestobjspace
+
+class AppTestBasic:
+    def setup_class(cls):
+        cls.space = gettestobjspace(usemodules=['_collections'])
+
+    def test_basics(self):
+        from _collections import defaultdict
+        d = defaultdict(list)
+        l = d[5]
+        d[5].append(42)
+        d[5].append(43)
+        assert l == [42, 43]
+        l2 = []
+        d[5] = l2
+        d[5].append(44)
+        assert l == [42, 43] and l2 == [44]

From commits-noreply at bitbucket.org  Tue Feb 15 18:45:27 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 15 Feb 2011 18:45:27 +0100 (CET)
Subject: [pypy-svn] pypy default: Pom pom pom
Message-ID: <20110215174527.331BE282BEA@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r41987:1be437cf8a3d
Date: 2011-02-15 18:45 +0100
http://bitbucket.org/pypy/pypy/changeset/1be437cf8a3d/

Log:	Pom pom pom

diff --git a/.hgsubstate b/.hgsubstate
--- a/.hgsubstate
+++ b/.hgsubstate
@@ -1,3 +1,3 @@
 80037 greenlet
-80037 lib_pypy/pyrepl
+80345 lib_pypy/pyrepl
 80037 testrunner

From commits-noreply at bitbucket.org  Tue Feb 15 18:48:59 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Tue, 15 Feb 2011 18:48:59 +0100 (CET)
Subject: [pypy-svn] pypy default: revert checkin 088b8b251525
Message-ID: <20110215174859.0BD3F2A202C@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r41988:9fa99de64ba1
Date: 2011-02-15 18:48 +0100
http://bitbucket.org/pypy/pypy/changeset/9fa99de64ba1/

Log:	revert checkin 088b8b251525 , it breaks tests :-(

diff --git a/pypy/module/zipimport/interp_zipimport.py b/pypy/module/zipimport/interp_zipimport.py
--- a/pypy/module/zipimport/interp_zipimport.py
+++ b/pypy/module/zipimport/interp_zipimport.py
@@ -237,7 +237,6 @@
         last_exc = None
         for compiled, is_package, ext in ENUMERATE_EXTS:
             fname = filename + ext
-            co_filename = self.filename + os.path.sep + fname
             try:
                 zip_file = RZipFile(self.filename, 'r')
                 try:
@@ -254,10 +253,10 @@
                     pkgpath = None
                 try:
                     if compiled:
-                        return self.import_pyc_file(space, fullname, co_filename,
+                        return self.import_pyc_file(space, fullname, fname,
                                                     buf, pkgpath)
                     else:
-                        return self.import_py_file(space, fullname, co_filename,
+                        return self.import_py_file(space, fullname, fname,
                                                    buf, pkgpath)
                 except OperationError, e:
                     last_exc = e

diff --git a/pypy/module/zipimport/test/test_zipimport.py b/pypy/module/zipimport/test/test_zipimport.py
--- a/pypy/module/zipimport/test/test_zipimport.py
+++ b/pypy/module/zipimport/test/test_zipimport.py
@@ -338,18 +338,6 @@
         import zipimport
         assert sys.path_hooks.count(zipimport.zipimporter) == 1
 
-    def test_co_filename(self):
-        self.writefile('mymodule.py', """
-def fn():
-    return fn.func_code.co_filename
-""")
-        import os
-        import mymodule
-        co_filename = mymodule.fn()
-        expected = self.zipfile + os.sep + 'mymodule.py'
-        assert co_filename == expected
-
-
 class AppTestZipimportDeflated(AppTestZipimport):
     compression = ZIP_DEFLATED
 

From commits-noreply at bitbucket.org  Tue Feb 15 18:50:12 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 15 Feb 2011 18:50:12 +0100 (CET)
Subject: [pypy-svn] pypy collections-module: merge default
Message-ID: <20110215175012.EEBAE282BEA@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: collections-module
Changeset: r41989:678a480f508a
Date: 2011-02-15 18:50 +0100
http://bitbucket.org/pypy/pypy/changeset/678a480f508a/

Log:	merge default


From commits-noreply at bitbucket.org  Tue Feb 15 19:34:46 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 15 Feb 2011 19:34:46 +0100 (CET)
Subject: [pypy-svn] pypy collections-module: Docs and typos.
Message-ID: <20110215183446.49DBD2A202C@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: collections-module
Changeset: r41990:7456ac84360d
Date: 2011-02-15 19:03 +0100
http://bitbucket.org/pypy/pypy/changeset/7456ac84360d/

Log:	Docs and typos.

diff --git a/pypy/module/_collections/__init__.py b/pypy/module/_collections/__init__.py
--- a/pypy/module/_collections/__init__.py
+++ b/pypy/module/_collections/__init__.py
@@ -3,6 +3,11 @@
 from pypy.interpreter.mixedmodule import MixedModule
 
 class Module(MixedModule):
+    """High performance data structures.
+- deque:        ordered collection accessible from endpoints only
+- defaultdict:  dict subclass with a default value factory
+"""
+
     appleveldefs = {
         'defaultdict': 'app_defaultdict.defaultdict',
         }

diff --git a/pypy/module/_collections/interp_deque.py b/pypy/module/_collections/interp_deque.py
--- a/pypy/module/_collections/interp_deque.py
+++ b/pypy/module/_collections/interp_deque.py
@@ -106,6 +106,7 @@
 
     @unwrap_spec('self', W_Root)
     def append(self, w_x):
+        "Add an element to the right side of the deque."
         ri = self.rightindex + 1
         if ri >= BLOCKLEN:
             b = Block(self.rightblock, None)
@@ -120,6 +121,7 @@
 
     @unwrap_spec('self', W_Root)
     def appendleft(self, w_x):
+        "Add an element to the left side of the deque."
         li = self.leftindex - 1
         if li < 0:
             b = Block(None, self.leftblock)
@@ -134,6 +136,7 @@
 
     @unwrap_spec('self')
     def clear(self):
+        "Remove all elements from the deque."
         self.leftblock = Block(None, None)
         self.rightblock = self.leftblock
         self.leftindex = CENTER + 1
@@ -143,6 +146,7 @@
 
     @unwrap_spec('self', W_Root)
     def count(self, w_x):
+        "Return number of occurrences of value."
         space = self.space
         result = 0
         block = self.leftblock
@@ -162,6 +166,7 @@
 
     @unwrap_spec('self', W_Root)
     def extend(self, w_iterable):
+        "Extend the right side of the deque with elements from the iterable"
         # Handle case where id(deque) == id(iterable)
         space = self.space
         if space.is_w(space.wrap(self), w_iterable):
@@ -184,6 +189,7 @@
 
     @unwrap_spec('self', W_Root)
     def extendleft(self, w_iterable):
+        "Extend the left side of the deque with elements from the iterable"
         # Handle case where id(deque) == id(iterable)
         space = self.space
         if space.is_w(space.wrap(self), w_iterable):
@@ -202,6 +208,7 @@
 
     @unwrap_spec('self')
     def pop(self):
+        "Remove and return the rightmost element."
         if self.len == 0:
             msg = "pop from an empty deque"
             raise OperationError(self.space.w_IndexError, self.space.wrap(msg))
@@ -225,6 +232,7 @@
 
     @unwrap_spec('self')
     def popleft(self):
+        "Remove and return the leftmost element."
         if self.len == 0:
             msg = "pop from an empty deque"
             raise OperationError(self.space.w_IndexError, self.space.wrap(msg))
@@ -248,6 +256,7 @@
 
     @unwrap_spec('self', W_Root)
     def remove(self, w_x):
+        "Remove first occurrence of value."
         space = self.space
         block = self.leftblock
         index = self.leftindex
@@ -269,6 +278,7 @@
 
     @unwrap_spec('self')
     def reverse(self):
+        "Reverse *IN PLACE*."
         li = self.leftindex
         lb = self.leftblock
         ri = self.rightindex
@@ -286,6 +296,7 @@
 
     @unwrap_spec('self', int)
     def rotate(self, n=1):
+        "Rotate the deque n steps to the right (default n=1).  If n is negative, rotates left."
         len = self.len
         if len == 0:
             return
@@ -308,6 +319,7 @@
 
     @unwrap_spec('self')
     def reviter(self):
+        "Return a reverse iterator over the deque."
         return W_DequeRevIter(self)
 
     @unwrap_spec('self')
@@ -407,16 +419,18 @@
 
     @unwrap_spec('self')
     def copy(self):
+        "Return a shallow copy of a deque."
         space = self.space
         w_self = space.wrap(self)
         if self.maxlen == sys.maxint:
             return space.call_function(space.type(w_self), w_self)
         else:
             return space.call_function(space.type(w_self), w_self,
-                                       space.wrap(self.maxint))
+                                       space.wrap(self.maxlen))
 
     @unwrap_spec('self')
     def reduce(self):
+        "Return state information for pickling."
         space = self.space
         w_self = space.wrap(self)
         w_type = space.type(w_self)
@@ -477,6 +491,9 @@
     return w_self
 
 W_Deque.typedef = TypeDef("deque",
+    __doc__ = """deque(iterable[, maxlen]) --> deque object
+
+Build an ordered collection accessible from endpoints only.""",
     __new__ = interp2app(descr__new__),
     __init__ = interp2app(W_Deque.init),
     append     = interp2app(W_Deque.append),

From commits-noreply at bitbucket.org  Tue Feb 15 19:34:46 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 15 Feb 2011 19:34:46 +0100 (CET)
Subject: [pypy-svn] pypy collections-module: merge heads
Message-ID: <20110215183446.88B0F2A202D@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: collections-module
Changeset: r41991:ace85d7334e4
Date: 2011-02-15 19:04 +0100
http://bitbucket.org/pypy/pypy/changeset/ace85d7334e4/

Log:	merge heads


From commits-noreply at bitbucket.org  Tue Feb 15 19:34:47 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 15 Feb 2011 19:34:47 +0100 (CET)
Subject: [pypy-svn] pypy collections-module: Add a __module__.
Message-ID: <20110215183447.167872A202C@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: collections-module
Changeset: r41992:935f567813be
Date: 2011-02-15 19:06 +0100
http://bitbucket.org/pypy/pypy/changeset/935f567813be/

Log:	Add a __module__.

diff --git a/pypy/module/_collections/interp_deque.py b/pypy/module/_collections/interp_deque.py
--- a/pypy/module/_collections/interp_deque.py
+++ b/pypy/module/_collections/interp_deque.py
@@ -494,6 +494,7 @@
     __doc__ = """deque(iterable[, maxlen]) --> deque object
 
 Build an ordered collection accessible from endpoints only.""",
+    __module__ = '_collections',
     __new__ = interp2app(descr__new__),
     __init__ = interp2app(W_Deque.init),
     append     = interp2app(W_Deque.append),

From commits-noreply at bitbucket.org  Tue Feb 15 19:34:47 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 15 Feb 2011 19:34:47 +0100 (CET)
Subject: [pypy-svn] pypy collections-module: merge heads
Message-ID: <20110215183447.4E6C52A202D@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: collections-module
Changeset: r41993:9d3b26aa0df2
Date: 2011-02-15 19:07 +0100
http://bitbucket.org/pypy/pypy/changeset/9d3b26aa0df2/

Log:	merge heads


From commits-noreply at bitbucket.org  Tue Feb 15 19:34:47 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 15 Feb 2011 19:34:47 +0100 (CET)
Subject: [pypy-svn] pypy collections-module: Typos.
Message-ID: <20110215183447.D88F12A202C@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: collections-module
Changeset: r41994:0c0ffacbf5ab
Date: 2011-02-15 19:33 +0100
http://bitbucket.org/pypy/pypy/changeset/0c0ffacbf5ab/

Log:	Typos.

diff --git a/pypy/module/_collections/interp_deque.py b/pypy/module/_collections/interp_deque.py
--- a/pypy/module/_collections/interp_deque.py
+++ b/pypy/module/_collections/interp_deque.py
@@ -393,8 +393,8 @@
             b, i = self.locate(start)
             return b.data[i]
         else:
-            raise OperationError(self.w_TypeError,
-                                 self.wrap("deque[:] is not supported"))
+            raise OperationError(space.w_TypeError,
+                                 space.wrap("deque[:] is not supported"))
 
     @unwrap_spec('self', W_Root, W_Root)
     def setitem(self, w_index, w_newobj):
@@ -404,8 +404,8 @@
             b, i = self.locate(start)
             b.data[i] = w_newobj
         else:
-            raise OperationError(self.w_TypeError,
-                                 self.wrap("deque[:] is not supported"))
+            raise OperationError(space.w_TypeError,
+                                 space.wrap("deque[:] is not supported"))
 
     @unwrap_spec('self', W_Root)
     def delitem(self, w_index):
@@ -414,8 +414,8 @@
         if step == 0:  # index only
             self.del_item(start)
         else:
-            raise OperationError(self.w_TypeError,
-                                 self.wrap("deque[:] is not supported"))
+            raise OperationError(space.w_TypeError,
+                                 space.wrap("deque[:] is not supported"))
 
     @unwrap_spec('self')
     def copy(self):

From commits-noreply at bitbucket.org  Tue Feb 15 20:37:45 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Tue, 15 Feb 2011 20:37:45 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: hg merge default
Message-ID: <20110215193745.8A711282BEA@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r41995:d262c528bcd2
Date: 2011-02-14 21:02 +0100
http://bitbucket.org/pypy/pypy/changeset/d262c528bcd2/

Log:	hg merge default

diff --git a/pypy/module/readline/c_readline.py b/pypy/module/readline/c_readline.py
deleted file mode 100644
--- a/pypy/module/readline/c_readline.py
+++ /dev/null
@@ -1,77 +0,0 @@
-from pypy.rpython.tool import rffi_platform as platform
-from pypy.rpython.lltypesystem import lltype, rffi
-from pypy.interpreter.error import OperationError
-from pypy.interpreter.gateway import ObjSpace, interp2app
-from pypy.translator.tool.cbuild import ExternalCompilationInfo
-
-# On various platforms, linking only with libreadline is not enough;
-# we also need to link with some variant of curses or libtermcap.
-# We follow the logic of CPython below.
-def try_with_lib(extralibs, **kwds):
-    global most_recent_error
-    # at least on Gentoo Linux, readline.h doesn't compile if stdio.h is not
-    # included before
-    eci = ExternalCompilationInfo(
-        includes = ["stdio.h", "readline/readline.h", "readline/history.h"],
-        libraries = extralibs + ['readline'],
-        )
-    try:
-        platform.verify_eci(eci)
-        return eci
-    except platform.CompilationError, e:
-        most_recent_error = e
-        return None
-
-eci = (try_with_lib([]) or
-       try_with_lib(['ncursesw']) or
-       try_with_lib(['ncurses']) or
-       try_with_lib(['curses']) or
-       try_with_lib(['termcap'], library_dirs=['/usr/lib/termcap']))
-if eci is None:
-    raise most_recent_error
-
-# ____________________________________________________________
-
-def external(name, args, result):
-    return rffi.llexternal(name, args, result, compilation_info=eci)
-
-# get a binding to  c library functions and define their args and return types
-# char *readline(char *)
-c_readline = external('readline', [rffi.CCHARP], rffi.CCHARP)
-
-# void rl_initiliaze(void)
-c_rl_initialize = external('rl_initialize', [], lltype.Void)
-
-# void using_history(void)
-c_using_history = external('using_history', [], lltype.Void)
-
-# void add_history(const char *)
-c_add_history = external('add_history', [rffi.CCHARP], lltype.Void)
-
-#------------------------------------------------------------
-# special initialization of readline 
-
-class ReadlineState(object):
-    lastline = ""        # XXX possibly temporary hack
-readlinestate = ReadlineState()
-
-def setup_readline(space, w_module):
-    c_using_history()
-    # XXX CPython initializes more stuff here
-    c_rl_initialize()
-    # install sys.__raw_input__, a hook that will be used by raw_input()
-    space.setitem(space.sys.w_dict, space.wrap('__raw_input__'),
-                  space.wrap(app_readline_func))
-
-def readline_func(space, prompt):
-    ll_res = c_readline(prompt)
-    if not ll_res:
-        raise OperationError(space.w_EOFError, space.w_None)
-    res = rffi.charp2str(ll_res)
-    if res and res != readlinestate.lastline:
-        readlinestate.lastline = res
-        c_add_history(res)
-    return space.wrap(res)
-
-readline_func.unwrap_spec = [ObjSpace, str]
-app_readline_func = interp2app(readline_func)

diff --git a/pypy/jit/metainterp/compile.py b/pypy/jit/metainterp/compile.py
--- a/pypy/jit/metainterp/compile.py
+++ b/pypy/jit/metainterp/compile.py
@@ -15,7 +15,7 @@
 from pypy.jit.metainterp.typesystem import llhelper, oohelper
 from pypy.jit.metainterp.optimizeutil import InvalidLoop
 from pypy.jit.metainterp.resume import NUMBERING
-from pypy.jit.codewriter import heaptracker
+from pypy.jit.codewriter import heaptracker, longlong
 
 def giveup():
     from pypy.jit.metainterp.pyjitpl import SwitchToBlackhole
@@ -528,7 +528,7 @@
 class ResumeGuardCountersFloat(AbstractResumeGuardCounters):
     def __init__(self):
         self.counters = [0] * 5
-        self.values = [0.0] * 5
+        self.values = [longlong.ZEROF] * 5
     see_float = func_with_new_name(_see, 'see_float')
 
 

diff --git a/pypy/module/readline/app_stub.py b/pypy/module/readline/app_stub.py
deleted file mode 100644
--- a/pypy/module/readline/app_stub.py
+++ /dev/null
@@ -1,13 +0,0 @@
-# NOT_RPYTHON
-
-def stub(*args, **kwds):
-    import warnings
-    warnings.warn("the 'readline' module is only a stub so far")
-
-def stub_str(*args, **kwds):
-    stub()
-    return ''
-
-def stub_int(*args, **kwds):
-    stub()
-    return 0

diff --git a/pypy/jit/metainterp/history.py b/pypy/jit/metainterp/history.py
--- a/pypy/jit/metainterp/history.py
+++ b/pypy/jit/metainterp/history.py
@@ -3,12 +3,12 @@
 from pypy.rpython.lltypesystem import lltype, llmemory, rffi
 from pypy.rpython.ootypesystem import ootype
 from pypy.rlib.objectmodel import we_are_translated, r_dict, Symbolic
-from pypy.rlib.objectmodel import compute_hash, compute_unique_id
+from pypy.rlib.objectmodel import compute_unique_id
 from pypy.rlib.rarithmetic import intmask, r_int64
 from pypy.conftest import option
 
 from pypy.jit.metainterp.resoperation import ResOperation, rop
-from pypy.jit.codewriter import heaptracker
+from pypy.jit.codewriter import heaptracker, longlong
 
 # ____________________________________________________________
 
@@ -82,8 +82,15 @@
     def getint(self):
         raise NotImplementedError
 
+    def getfloatstorage(self):
+        raise NotImplementedError
+
     def getfloat(self):
-        raise NotImplementedError
+        return longlong.getrealfloat(self.getfloatstorage())
+
+    def getlonglong(self):
+        assert longlong.supports_longlong
+        return self.getfloatstorage()
 
     def getref_base(self):
         raise NotImplementedError
@@ -137,6 +144,7 @@
     def get_return_type(self):
         """ Implement in call descr.
         Must return INT, REF, FLOAT, or 'v' for void.
+        On 32-bit (hack) it can also be 'L' for longlongs.
         """
         raise NotImplementedError
 
@@ -218,7 +226,7 @@
         elif kind == "ref":
             return cpu.ts.new_ConstRef(x)
         elif kind == "float":
-            return ConstFloat(x)
+            return ConstFloat(longlong.getfloatstorage(x))
         else:
             raise NotImplementedError(kind)
 
@@ -301,26 +309,26 @@
 
 class ConstFloat(Const):
     type = FLOAT
-    value = 0.0
+    value = longlong.ZEROF
     _attrs_ = ('value',)
 
-    def __init__(self, floatval):
-        assert isinstance(floatval, float)
-        self.value = floatval
+    def __init__(self, valuestorage):
+        assert lltype.typeOf(valuestorage) is longlong.FLOATSTORAGE
+        self.value = valuestorage
 
     def clonebox(self):
         return BoxFloat(self.value)
 
     nonconstbox = clonebox
 
-    def getfloat(self):
+    def getfloatstorage(self):
         return self.value
 
     def _get_hash_(self):
-        return compute_hash(self.value)
+        return longlong.gethash(self.value)
 
     def set_future_value(self, cpu, j):
-        cpu.set_future_value_float(j, self.getfloat())
+        cpu.set_future_value_float(j, self.value)
 
     def same_constant(self, other):
         if isinstance(other, ConstFloat):
@@ -328,15 +336,15 @@
         return False
 
     def nonnull(self):
-        return self.value != 0.0
+        return self.value != longlong.ZEROF
 
     def _getrepr_(self):
-        return self.value
+        return self.getfloat()
 
     def repr_rpython(self):
         return repr_rpython(self, 'cf')
 
-CONST_FZERO = ConstFloat(0.0)
+CONST_FZERO = ConstFloat(longlong.ZEROF)
 
 class ConstPtr(Const):
     type = REF
@@ -466,7 +474,7 @@
             ptrval = lltype.cast_opaque_ptr(llmemory.GCREF, x)
             return BoxPtr(ptrval)
         elif kind == "float":
-            return BoxFloat(x)
+            return BoxFloat(longlong.getfloatstorage(x))
         else:
             raise NotImplementedError(kind)
 
@@ -546,12 +554,12 @@
     type = FLOAT
     _attrs_ = ('value',)
 
-    def __init__(self, floatval=0.0):
-        assert isinstance(floatval, float)
-        self.value = floatval
+    def __init__(self, valuestorage=longlong.ZEROF):
+        assert lltype.typeOf(valuestorage) is longlong.FLOATSTORAGE
+        self.value = valuestorage
 
     def forget_value(self):
-        self.value = 0.0
+        self.value = longlong.ZEROF
 
     def clonebox(self):
         return BoxFloat(self.value)
@@ -559,20 +567,20 @@
     def constbox(self):
         return ConstFloat(self.value)
 
-    def getfloat(self):
+    def getfloatstorage(self):
         return self.value
 
     def _get_hash_(self):
-        return compute_hash(self.value)
+        return longlong.gethash(self.value)
 
     def set_future_value(self, cpu, j):
         cpu.set_future_value_float(j, self.value)
 
     def nonnull(self):
-        return self.value != 0.0
+        return self.value != longlong.ZEROF
 
     def _getrepr_(self):
-        return self.value
+        return self.getfloat()
 
     def repr_rpython(self):
         return repr_rpython(self, 'bf')

diff --git a/pypy/jit/metainterp/warmspot.py b/pypy/jit/metainterp/warmspot.py
--- a/pypy/jit/metainterp/warmspot.py
+++ b/pypy/jit/metainterp/warmspot.py
@@ -23,7 +23,7 @@
 from pypy.jit.metainterp.jitprof import Profiler, EmptyProfiler
 from pypy.jit.metainterp.jitexc import JitException
 from pypy.jit.metainterp.jitdriver import JitDriverStaticData
-from pypy.jit.codewriter import support, codewriter
+from pypy.jit.codewriter import support, codewriter, longlong
 from pypy.jit.codewriter.policy import JitPolicy
 
 # ____________________________________________________________
@@ -345,7 +345,7 @@
 
         class DoneWithThisFrameFloat(JitException):
             def __init__(self, result):
-                assert lltype.typeOf(result) is lltype.Float
+                assert lltype.typeOf(result) is longlong.FLOATSTORAGE
                 self.result = result
             def __str__(self):
                 return 'DoneWithThisFrameFloat(%s)' % (self.result,)

diff --git a/pypy/jit/metainterp/pyjitpl.py b/pypy/jit/metainterp/pyjitpl.py
--- a/pypy/jit/metainterp/pyjitpl.py
+++ b/pypy/jit/metainterp/pyjitpl.py
@@ -20,7 +20,7 @@
 from pypy.rlib.rarithmetic import intmask
 from pypy.rlib.objectmodel import specialize
 from pypy.jit.codewriter.jitcode import JitCode, SwitchDictDescr, MissingLiveness
-from pypy.jit.codewriter import heaptracker
+from pypy.jit.codewriter import heaptracker, longlong
 from pypy.jit.metainterp.optimizeutil import RetraceLoop
 
 # ____________________________________________________________
@@ -1473,7 +1473,7 @@
             elif result_type == history.REF:
                 raise sd.DoneWithThisFrameRef(self.cpu, resultbox.getref_base())
             elif result_type == history.FLOAT:
-                raise sd.DoneWithThisFrameFloat(resultbox.getfloat())
+                raise sd.DoneWithThisFrameFloat(resultbox.getfloatstorage())
             else:
                 assert False
 
@@ -2152,7 +2152,7 @@
             # warmstate.py.
             virtualizable_box = self.virtualizable_boxes[-1]
             virtualizable = vinfo.unwrap_virtualizable_box(virtualizable_box)
-            assert not virtualizable.vable_token
+            assert not vinfo.gettoken(virtualizable)
             # fill the virtualizable with the local boxes
             self.synchronize_virtualizable()
         #

diff --git a/pypy/doc/config/objspace.usemodules.readline.txt b/pypy/doc/config/objspace.usemodules.readline.txt
deleted file mode 100644
--- a/pypy/doc/config/objspace.usemodules.readline.txt
+++ /dev/null
@@ -1,1 +0,0 @@
-Use the 'readline' module. 

diff --git a/pypy/jit/metainterp/resume.py b/pypy/jit/metainterp/resume.py
--- a/pypy/jit/metainterp/resume.py
+++ b/pypy/jit/metainterp/resume.py
@@ -1066,17 +1066,16 @@
             return len(numb.nums)
         index = len(numb.nums) - 1
         virtualizable = self.decode_ref(numb.nums[index])
-        virtualizable = vinfo.cast_gcref_to_vtype(virtualizable)
         if self.resume_after_guard_not_forced == 1:
             # in the middle of handle_async_forcing()
-            assert virtualizable.vable_token
-            virtualizable.vable_token = vinfo.TOKEN_NONE
+            assert vinfo.gettoken(virtualizable)
+            vinfo.settoken(virtualizable, vinfo.TOKEN_NONE)
         else:
             # just jumped away from assembler (case 4 in the comment in
             # virtualizable.py) into tracing (case 2); check that vable_token
             # is and stays 0.  Note the call to reset_vable_token() in
             # warmstate.py.
-            assert not virtualizable.vable_token
+            assert not vinfo.gettoken(virtualizable)
         return vinfo.write_from_resume_data_partial(virtualizable, self, numb)
 
     def load_value_of_type(self, TYPE, tagged):
@@ -1217,7 +1216,7 @@
     def decode_float(self, tagged):
         num, tag = untag(tagged)
         if tag == TAGCONST:
-            return self.consts[num].getfloat()
+            return self.consts[num].getfloatstorage()
         else:
             assert tag == TAGBOX
             if num < 0:

diff --git a/pypy/jit/metainterp/warmstate.py b/pypy/jit/metainterp/warmstate.py
--- a/pypy/jit/metainterp/warmstate.py
+++ b/pypy/jit/metainterp/warmstate.py
@@ -12,13 +12,13 @@
 from pypy.rlib.jit import BaseJitCell
 from pypy.rlib.debug import debug_start, debug_stop, debug_print
 from pypy.jit.metainterp import history
-from pypy.jit.codewriter import support, heaptracker
+from pypy.jit.codewriter import support, heaptracker, longlong
 
 # ____________________________________________________________
 
 @specialize.arg(0)
 def specialize_value(TYPE, x):
-    """'x' must be a Signed, a GCREF or a Float.
+    """'x' must be a Signed, a GCREF or a FLOATSTORAGE.
     This function casts it to a more specialized type, like Char or Ptr(..).
     """
     INPUT = lltype.typeOf(x)
@@ -28,15 +28,15 @@
             return rffi.cast(TYPE, x)
         else:
             return lltype.cast_primitive(TYPE, x)
-    elif INPUT is lltype.Float:
+    elif INPUT is longlong.FLOATSTORAGE:
         assert TYPE is lltype.Float
-        return x
+        return longlong.getrealfloat(x)
     else:
         return lltype.cast_opaque_ptr(TYPE, x)
 
 @specialize.ll()
 def unspecialize_value(value):
-    """Casts 'value' to a Signed, a GCREF or a Float."""
+    """Casts 'value' to a Signed, a GCREF or a FLOATSTORAGE."""
     if isinstance(lltype.typeOf(value), lltype.Ptr):
         if lltype.typeOf(value).TO._gckind == 'gc':
             return lltype.cast_opaque_ptr(llmemory.GCREF, value)
@@ -46,7 +46,7 @@
     elif isinstance(lltype.typeOf(value), ootype.OOType):
         return ootype.cast_to_object(value)
     elif isinstance(value, float):
-        return value
+        return longlong.getfloatstorage(value)
     else:
         return intmask(value)
 
@@ -83,6 +83,7 @@
         else:
             return history.BoxObj(value)
     elif isinstance(value, float):
+        value = longlong.getfloatstorage(value)
         if in_const_box:
             return history.ConstFloat(value)
         else:
@@ -138,7 +139,11 @@
         intvalue = lltype.cast_primitive(lltype.Signed, value)
         cpu.set_future_value_int(j, intvalue)
     elif typecode == 'float':
-        assert isinstance(value, float)
+        if lltype.typeOf(value) is lltype.Float:
+            value = longlong.getfloatstorage(value)
+        else:
+            assert longlong.is_longlong(lltype.typeOf(value))
+            value = rffi.cast(lltype.SignedLongLong, value)
         cpu.set_future_value_float(j, value)
     else:
         assert False

diff --git a/pypy/jit/metainterp/optimizeopt/optimizer.py b/pypy/jit/metainterp/optimizeopt/optimizer.py
--- a/pypy/jit/metainterp/optimizeopt/optimizer.py
+++ b/pypy/jit/metainterp/optimizeopt/optimizer.py
@@ -164,7 +164,7 @@
 CONST_0      = ConstInt(0)
 CONST_1      = ConstInt(1)
 CVAL_ZERO    = ConstantValue(CONST_0)
-CVAL_ZERO_FLOAT = ConstantValue(ConstFloat(0.0))
+CVAL_ZERO_FLOAT = ConstantValue(Const._new(0.0))
 CVAL_UNINITIALIZED_ZERO = ConstantValue(CONST_0)
 llhelper.CVAL_NULLREF = ConstantValue(llhelper.CONST_NULL)
 oohelper.CVAL_NULLREF = ConstantValue(oohelper.CONST_NULL)

diff --git a/pypy/jit/metainterp/test/test_basic.py b/pypy/jit/metainterp/test/test_basic.py
--- a/pypy/jit/metainterp/test/test_basic.py
+++ b/pypy/jit/metainterp/test/test_basic.py
@@ -9,6 +9,7 @@
 from pypy.jit.metainterp import pyjitpl, history
 from pypy.jit.metainterp.warmstate import set_future_value
 from pypy.jit.codewriter.policy import JitPolicy, StopAtXPolicy
+from pypy.jit.codewriter import longlong
 from pypy import conftest
 from pypy.rlib.rarithmetic import ovfcheck
 from pypy.jit.metainterp.typesystem import LLTypeHelper, OOTypeHelper
@@ -86,6 +87,7 @@
             blackholeinterp.setarg_r(count_r, value)
             count_r += 1
         elif T == lltype.Float:
+            value = longlong.getfloatstorage(value)
             blackholeinterp.setarg_f(count_f, value)
             count_f += 1
         else:
@@ -192,6 +194,10 @@
         # try to run it by running the code compiled just before
         result3 = _run_with_machine_code(self, args)
         assert result1 == result3 or result3 == NotImplemented
+        #
+        if (longlong.supports_longlong and
+            isinstance(result1, longlong.r_float_storage)):
+            result1 = longlong.getrealfloat(result1)
         return result1
 
     def check_history(self, expected=None, **isns):

diff --git a/pypy/module/readline/test/__init__.py b/pypy/module/readline/test/__init__.py
deleted file mode 100644
--- a/pypy/module/readline/test/__init__.py
+++ /dev/null
@@ -1,1 +0,0 @@
-#

diff --git a/pypy/module/readline/test/test_with_pypy.py b/pypy/module/readline/test/test_with_pypy.py
deleted file mode 100644
--- a/pypy/module/readline/test/test_with_pypy.py
+++ /dev/null
@@ -1,27 +0,0 @@
-"""
-Test the readline library on top of PyPy.  The following tests run
-in the PyPy interpreter, itself running on top of CPython
-"""
-
-import py
-from pypy.conftest import gettestobjspace
-from pypy.rpython.tool import rffi_platform as platform
-
-try:
-    from pypy.module.readline import c_readline
-except platform.CompilationError, e:
-    py.test.skip(e)
-
-
-class AppTestReadline:
-
-    def setup_class(cls):
-        # enable usage of the readline mixedmodule
-        space = gettestobjspace(usemodules=('readline',))
-        cls.space = space
-
-    def test_basic_import(self):
-        # this is interpreted by PyPy
-        import readline 
-        readline.readline
-        # XXX test more

diff --git a/pypy/module/readline/interp_readline.py b/pypy/module/readline/interp_readline.py
deleted file mode 100644
--- a/pypy/module/readline/interp_readline.py
+++ /dev/null
@@ -1,23 +0,0 @@
-# this is a sketch of how one might one day be able to define a pretty simple
-# ctypes-using module, suitable for feeding to the ext-compiler
-
-from pypy.interpreter.baseobjspace import ObjSpace
-
-from pypy.module.readline import c_readline
-from pypy.rpython.lltypesystem import rffi
-
-#------------------------------------------------------------
-# exported API  (see interpleveldefs in __init__.py) 
-#
-def readline(space, prompt):
-    return space.wrap(rffi.charp2str(c_readline.c_readline(prompt)))
-readline.unwrap_spec = [ObjSpace, str]
-
-def setcompleter(space, w_callback):
-    """Set or remove the completer function.
-    The function is called as function(text, state),
-    for state in 0, 1, 2, ..., until it returns a non-string.
-    It should return the next possible completion starting with 'text'.
-    """ 
-    # XXX set internal completion function 
-    

diff --git a/pypy/objspace/std/test/helper.py b/pypy/objspace/std/test/helper.py
deleted file mode 100644
--- a/pypy/objspace/std/test/helper.py
+++ /dev/null
@@ -1,69 +0,0 @@
-def raises(excp, func, *args):
-    try:
-        func(*args)
-        assert 1 == 0
-    except excp:pass
-
-def assertEqual(a, b):
-    assert a == b
-
-def assertNotEqual(a, b):
-    assert a != b
-
-def assertIs(a, b):
-    assert a is b
-
-# complex specific tests
-
-EPS = 1e-9
-
-def assertAlmostEqual(a, b):
-    if isinstance(a, complex):
-        if isinstance(b, complex):
-            assert a.real - b.real < EPS
-            assert a.imag - b.imag < EPS
-        else:
-            assert a.real - b < EPS
-            assert a.imag < EPS
-    else:
-        if isinstance(b, complex):
-            assert a - b.real < EPS
-            assert b.imag < EPS
-        else:
-            assert a - b < EPS
-
-def assertCloseAbs(x, y, eps=1e-9):
-    """Return true iff floats x and y "are close\""""
-    # put the one with larger magnitude second
-    if abs(x) > abs(y):
-        x, y = y, x
-    if y == 0:
-        return abs(x) < eps
-    if x == 0:
-        return abs(y) < eps
-    # check that relative difference < eps
-    assert abs((x-y)/y) < eps
-
-def assertClose(x, y, eps=1e-9):
-    """Return true iff complexes x and y "are close\""""
-    assertCloseAbs(x.real, y.real, eps)
-    assertCloseAbs(x.imag, y.imag, eps)
-
-
-def check_div(x, y):
-    """Compute complex z=x*y, and check that z/x==y and z/y==x."""
-    z = x * y
-    if x != 0:
-        q = z / x
-        assertClose(q, y)
-        q = z.__div__(x)
-        assertClose(q, y)
-        q = z.__truediv__(x)
-        assertClose(q, y)
-    if y != 0:
-        q = z / y
-        assertClose(q, x)
-        q = z.__div__(y)
-        assertClose(q, x)
-        q = z.__truediv__(y)
-        assertClose(q, x)

diff --git a/pypy/module/readline/__init__.py b/pypy/module/readline/__init__.py
deleted file mode 100644
--- a/pypy/module/readline/__init__.py
+++ /dev/null
@@ -1,45 +0,0 @@
-# this is a sketch of how one might one day be able to define a pretty simple
-# ctypes-using module, suitable for feeding to the ext-compiler
-
-from pypy.interpreter.mixedmodule import MixedModule
-
-# XXX raw_input needs to check for space.readline_func and use
-# it if its there 
-
-class Module(MixedModule):
-    """Importing this module enables command line editing using GNU readline."""
-    # the above line is the doc string of the translated module  
-
-    def setup_after_space_initialization(self):
-        from pypy.module.readline import c_readline 
-        c_readline.setup_readline(self.space, self)
-
-    interpleveldefs = {
-        'readline'    : 'interp_readline.readline',
-    }
-
-    appleveldefs = {
-        'parse_and_bind':     'app_stub.stub',
-        'get_line_buffer':    'app_stub.stub_str',
-        'insert_text':        'app_stub.stub',
-        'read_init_file':     'app_stub.stub',
-        'read_history_file':  'app_stub.stub',
-        'write_history_file': 'app_stub.stub',
-        'clear_history':      'app_stub.stub',
-        'get_history_length': 'app_stub.stub_int',
-        'set_history_length': 'app_stub.stub',
-        'get_current_history_length': 'app_stub.stub_int',
-        'get_history_item':           'app_stub.stub_str',
-        'remove_history_item':        'app_stub.stub',
-        'replace_history_item':       'app_stub.stub',
-        'redisplay':                  'app_stub.stub',
-        'set_startup_hook':           'app_stub.stub',
-        'set_pre_input_hook':         'app_stub.stub',
-        'set_completer':      'app_stub.stub',
-        'get_completer':      'app_stub.stub',
-        'get_begidx':         'app_stub.stub_int',
-        'get_endidx':         'app_stub.stub_int',
-        'set_completer_delims':       'app_stub.stub',
-        'get_completer_delims':       'app_stub.stub_str',
-        'add_history':        'app_stub.stub',
-    }

diff --git a/pypy/module/readline/test/test_c_readline.py b/pypy/module/readline/test/test_c_readline.py
deleted file mode 100644
--- a/pypy/module/readline/test/test_c_readline.py
+++ /dev/null
@@ -1,16 +0,0 @@
-"""
-Directly test the basic ctypes wrappers.
-"""
-
-import py
-from pypy import conftest; conftest.translation_test_so_skip_if_appdirect()
-from pypy.rpython.tool import rffi_platform as platform
-
-try:
-    from pypy.module.readline import c_readline
-except platform.CompilationError, e:
-    py.test.skip(e)
-
-
-def test_basic_import():
-    c_readline.c_rl_initialize()

diff --git a/pypy/rlib/jit.py b/pypy/rlib/jit.py
--- a/pypy/rlib/jit.py
+++ b/pypy/rlib/jit.py
@@ -272,7 +272,7 @@
 
 # ____________________________________________________________
 
-class JitDriver:    
+class JitDriver(object):    
     """Base class to declare fine-grained user control on the JIT.  So
     far, there must be a singleton instance of JitDriver.  This style
     will allow us (later) to support a single RPython program with

From commits-noreply at bitbucket.org  Tue Feb 15 20:37:46 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Tue, 15 Feb 2011 20:37:46 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: this should be based on key_boxes
Message-ID: <20110215193746.4D4C8282BEA@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r41996:3f71bc1c4427
Date: 2011-02-15 20:34 +0100
http://bitbucket.org/pypy/pypy/changeset/3f71bc1c4427/

Log:	this should be based on key_boxes

diff --git a/pypy/jit/metainterp/optimizeopt/unroll.py b/pypy/jit/metainterp/optimizeopt/unroll.py
--- a/pypy/jit/metainterp/optimizeopt/unroll.py
+++ b/pypy/jit/metainterp/optimizeopt/unroll.py
@@ -161,10 +161,11 @@
         return self.optimizer.getvalue(box)
 
     def state(self, box):
+        value = self.getvalue(box)
+        box = value.get_key_box()
         try:
             info = self.info[box]
         except KeyError:
-            value = self.getvalue(box)
             if value.is_virtual():
                 self.info[box] = info = value.make_virtual_info(self, None)
                 flds = self.fieldboxes[box]

diff --git a/pypy/jit/metainterp/test/test_virtual.py b/pypy/jit/metainterp/test/test_virtual.py
--- a/pypy/jit/metainterp/test/test_virtual.py
+++ b/pypy/jit/metainterp/test/test_virtual.py
@@ -703,10 +703,64 @@
                 n -= 1
             return node[0] + node[1]
         assert self.meta_interp(f, [40]) == f(40)
-        
+
+    def FIXME_why_does_this_force(self):
+        mydriver = JitDriver(reds = ['i', 'j'], greens = []) 
+        def f():
+            i = self._new()
+            i.value = 0
+            j = self._new()
+            while i.value < 10:
+                mydriver.can_enter_jit(i=i, j=j)
+                mydriver.jit_merge_point(i=i, j=j)
+                nxt = self._new()
+                nxt.value = i.value + 1
+                i = nxt
+                j = nxt
+            return i.value + j.value
+        assert self.meta_interp(f, []) == 20
+
+    def FIXME_why_does_this_force2(self):
+        mydriver = JitDriver(reds = ['i', 'j'], greens = []) 
+        def f():
+            i = self._new()
+            i.value = 0
+            j = self._new()
+            j.value = 0
+            while i.value < 10:
+                mydriver.can_enter_jit(i=i, j=j)
+                mydriver.jit_merge_point(i=i, j=j)
+                nxt = self._new()
+                nxt.value = i.value + 1
+                i = nxt
+                nxt = self._new()
+                nxt.value = i.value + 1
+                j = nxt
+                i = j
+            return i.value + j.value
+        assert self.meta_interp(f, []) == 20
+                
 
 class VirtualMiscTests:
 
+    def test_multiple_equal_virtuals(self):
+        mydriver = JitDriver(reds = ['i'], greens = [])
+        class A:
+            pass
+        def f():
+            i = A()
+            i.value = 0
+            while i.value < 10:
+                mydriver.can_enter_jit(i=i)
+                mydriver.jit_merge_point(i=i)
+                nxt = A()
+                nxt.value = i.value + 1
+                tmp = A()
+                tmp.ref = nxt
+                i = tmp.ref
+            return i.value
+        assert self.meta_interp(f, []) == 10
+
     def test_guards_around_forcing(self):
         class A(object):
             def __init__(self, x):
@@ -756,6 +810,23 @@
         expected = f(70)
         assert r == expected
 
+    def test_arraycopy_disappears(self):
+        mydriver = JitDriver(reds = ['i'], greens = []) 
+        def f():
+            i = 0
+            while i < 10:
+                mydriver.can_enter_jit(i=i)
+                mydriver.jit_merge_point(i=i)                
+                t = (1, 2, 3, i + 1)
+                t2 = t[:]
+                del t
+                i = t2[3]
+                del t2
+            return i
+        assert self.meta_interp(f, []) == 10
+        self.check_loops(new_array=0)
+
+
 # ____________________________________________________________
 # Run 1: all the tests instantiate a real RPython class
 

From commits-noreply at bitbucket.org  Tue Feb 15 20:37:46 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Tue, 15 Feb 2011 20:37:46 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: hg merge default
Message-ID: <20110215193746.85615282BEC@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r41997:5dd1ccc27de7
Date: 2011-02-15 20:35 +0100
http://bitbucket.org/pypy/pypy/changeset/5dd1ccc27de7/

Log:	hg merge default


From commits-noreply at bitbucket.org  Tue Feb 15 20:55:24 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 15 Feb 2011 20:55:24 +0100 (CET)
Subject: [pypy-svn] pypy collections-module: Test and fix.
Message-ID: <20110215195524.988F7282BEA@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: collections-module
Changeset: r41998:e5b990219303
Date: 2011-02-15 20:39 +0100
http://bitbucket.org/pypy/pypy/changeset/e5b990219303/

Log:	Test and fix.

diff --git a/pypy/module/_collections/test/test_deque.py b/pypy/module/_collections/test/test_deque.py
--- a/pypy/module/_collections/test/test_deque.py
+++ b/pypy/module/_collections/test/test_deque.py
@@ -170,6 +170,16 @@
         d = deque(xrange(1000, 1200))
         d.reverse()
         assert list(d) == list(reversed(range(1000, 1200)))
+        #
+        n = 100
+        data = map(str, range(n))
+        for i in range(n):
+            d = deque(data[:i])
+            r = d.reverse()
+            assert list(d) == list(reversed(data[:i]))
+            assert r is None
+            d.reverse()
+            assert list(d) == data[:i]
 
     def test_rotate(self):
         from _collections import deque

diff --git a/pypy/module/_collections/interp_deque.py b/pypy/module/_collections/interp_deque.py
--- a/pypy/module/_collections/interp_deque.py
+++ b/pypy/module/_collections/interp_deque.py
@@ -283,7 +283,7 @@
         lb = self.leftblock
         ri = self.rightindex
         rb = self.rightblock
-        while lb is not rb or li < ri:
+        for i in range(self.len >> 1):
             lb.data[li], rb.data[ri] = rb.data[ri], lb.data[li]
             li += 1
             if li >= BLOCKLEN:

From commits-noreply at bitbucket.org  Tue Feb 15 20:55:25 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 15 Feb 2011 20:55:25 +0100 (CET)
Subject: [pypy-svn] pypy collections-module: Missing a reset-item-to-None,
	otherwise the reference to the
Message-ID: <20110215195525.6219A282BEA@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: collections-module
Changeset: r41999:39f1b1be3f19
Date: 2011-02-15 20:43 +0100
http://bitbucket.org/pypy/pypy/changeset/39f1b1be3f19/

Log:	Missing a reset-item-to-None, otherwise the reference to the item
	stay around.

diff --git a/pypy/module/_collections/test/test_deque.py b/pypy/module/_collections/test/test_deque.py
--- a/pypy/module/_collections/test/test_deque.py
+++ b/pypy/module/_collections/test/test_deque.py
@@ -291,21 +291,18 @@
         for s in ('abcd', xrange(200)):
             assert list(reversed(deque(s))) == list(reversed(s))
 
-    def test_container_iterator(self):
-        # Bug #3680: tp_traverse was not implemented for deque iterator objects
-        class C(object):
-            pass
-        for i in range(2):
-            obj = C()
-            ref = weakref.ref(obj)
-            if i == 0:
-                container = deque([obj, 1])
-            else:
-                container = reversed(deque([obj, 1]))
-            obj.x = iter(container)
-            del obj, container
-            test_support.gc_collect()
-            self.assertTrue(ref() is None, "Cycle was not collected")
+    def test_free(self):
+        import gc
+        from _collections import deque
+        class X(object):
+            freed = False
+            def __del__(self):
+                X.freed = True
+        d = deque()
+        d.append(X())
+        d.pop()
+        gc.collect(); gc.collect(); gc.collect()
+        assert X.freed
 
 class XXXXXXXXXTestVariousIteratorArgs:
 

diff --git a/pypy/module/_collections/interp_deque.py b/pypy/module/_collections/interp_deque.py
--- a/pypy/module/_collections/interp_deque.py
+++ b/pypy/module/_collections/interp_deque.py
@@ -215,6 +215,7 @@
         self.len -= 1
         ri = self.rightindex
         w_obj = self.rightblock.data[ri]
+        self.rightblock.data[ri] = None
         ri -= 1
         if ri < 0:
             if self.len == 0:
@@ -239,6 +240,7 @@
         self.len -= 1
         li = self.leftindex
         w_obj = self.leftblock.data[li]
+        self.leftblock.data[li] = None
         li += 1
         if li >= BLOCKLEN:
             if self.len == 0:

From commits-noreply at bitbucket.org  Tue Feb 15 20:55:25 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 15 Feb 2011 20:55:25 +0100 (CET)
Subject: [pypy-svn] pypy collections-module: Kill the remainder of
	test_deque. It's good	enough if we run
Message-ID: <20110215195525.D5B02282BEA@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: collections-module
Changeset: r42000:dd969966196b
Date: 2011-02-15 20:44 +0100
http://bitbucket.org/pypy/pypy/changeset/dd969966196b/

Log:	Kill the remainder of test_deque. It's good enough if we run lib-
	python/modified-2.7.0/test/test_deque.

diff --git a/pypy/module/_collections/test/test_deque.py b/pypy/module/_collections/test/test_deque.py
--- a/pypy/module/_collections/test/test_deque.py
+++ b/pypy/module/_collections/test/test_deque.py
@@ -303,138 +303,3 @@
         d.pop()
         gc.collect(); gc.collect(); gc.collect()
         assert X.freed
-
-class XXXXXXXXXTestVariousIteratorArgs:
-
-    def test_constructor(self):
-        for s in ("123", "", range(1000), ('do', 1.2), xrange(2000,2200,5)):
-            for g in (seq_tests.Sequence, seq_tests.IterFunc,
-                      seq_tests.IterGen, seq_tests.IterFuncStop,
-                      seq_tests.itermulti, seq_tests.iterfunc):
-                self.assertEqual(list(deque(g(s))), list(g(s)))
-            self.assertRaises(TypeError, deque, seq_tests.IterNextOnly(s))
-            self.assertRaises(TypeError, deque, seq_tests.IterNoNext(s))
-            self.assertRaises(ZeroDivisionError, deque, seq_tests.IterGenExc(s))
-
-    def test_iter_with_altered_data(self):
-        d = deque('abcdefg')
-        it = iter(d)
-        d.pop()
-        self.assertRaises(RuntimeError, it.next)
-
-    def test_runtime_error_on_empty_deque(self):
-        d = deque()
-        it = iter(d)
-        d.append(10)
-        self.assertRaises(RuntimeError, it.next)
-
-##class Deque(deque):
-##    pass
-
-##class DequeWithBadIter(deque):
-##    def __iter__(self):
-##        raise TypeError
-
-class XXXXXXXXXTestSubclass:
-
-    def test_basics(self):
-        d = Deque(xrange(25))
-        d.__init__(xrange(200))
-        for i in xrange(200, 400):
-            d.append(i)
-        for i in reversed(xrange(-200, 0)):
-            d.appendleft(i)
-        self.assertEqual(list(d), range(-200, 400))
-        self.assertEqual(len(d), 600)
-
-        left = [d.popleft() for i in xrange(250)]
-        self.assertEqual(left, range(-200, 50))
-        self.assertEqual(list(d), range(50, 400))
-
-        right = [d.pop() for i in xrange(250)]
-        right.reverse()
-        self.assertEqual(right, range(150, 400))
-        self.assertEqual(list(d), range(50, 150))
-
-        d.clear()
-        self.assertEqual(len(d), 0)
-
-    def test_copy_pickle(self):
-
-        d = Deque('abc')
-
-        e = d.__copy__()
-        self.assertEqual(type(d), type(e))
-        self.assertEqual(list(d), list(e))
-
-        e = Deque(d)
-        self.assertEqual(type(d), type(e))
-        self.assertEqual(list(d), list(e))
-
-        s = pickle.dumps(d)
-        e = pickle.loads(s)
-        self.assertNotEqual(id(d), id(e))
-        self.assertEqual(type(d), type(e))
-        self.assertEqual(list(d), list(e))
-
-        d = Deque('abcde', maxlen=4)
-
-        e = d.__copy__()
-        self.assertEqual(type(d), type(e))
-        self.assertEqual(list(d), list(e))
-
-        e = Deque(d)
-        self.assertEqual(type(d), type(e))
-        self.assertEqual(list(d), list(e))
-
-        s = pickle.dumps(d)
-        e = pickle.loads(s)
-        self.assertNotEqual(id(d), id(e))
-        self.assertEqual(type(d), type(e))
-        self.assertEqual(list(d), list(e))
-
-##    def test_pickle(self):
-##        d = Deque('abc')
-##        d.append(d)
-##
-##        e = pickle.loads(pickle.dumps(d))
-##        self.assertNotEqual(id(d), id(e))
-##        self.assertEqual(type(d), type(e))
-##        dd = d.pop()
-##        ee = e.pop()
-##        self.assertEqual(id(e), id(ee))
-##        self.assertEqual(d, e)
-##
-##        d.x = d
-##        e = pickle.loads(pickle.dumps(d))
-##        self.assertEqual(id(e), id(e.x))
-##
-##        d = DequeWithBadIter('abc')
-##        self.assertRaises(TypeError, pickle.dumps, d)
-
-    def test_weakref(self):
-        d = deque('gallahad')
-        p = weakref.proxy(d)
-        self.assertEqual(str(p), str(d))
-        d = None
-        test_support.gc_collect()
-        self.assertRaises(ReferenceError, str, p)
-
-    def test_strange_subclass(self):
-        class X(deque):
-            def __iter__(self):
-                return iter([])
-        d1 = X([1,2,3])
-        d2 = X([4,5,6])
-        d1 == d2   # not clear if this is supposed to be True or False,
-                   # but it used to give a SystemError
-
-
-##class SubclassWithKwargs(deque):
-##    def __init__(self, newarg=1):
-##        deque.__init__(self)
-
-class XXXXXXXXXTestSubclassWithKwargs:
-    def test_subclass_with_kwargs(self):
-        # SF bug #1486663 -- this used to erroneously raise a TypeError
-        SubclassWithKwargs(newarg=1)

From commits-noreply at bitbucket.org  Tue Feb 15 20:55:26 2011
From: commits-noreply at bitbucket.org (vincentlegoll)
Date: Tue, 15 Feb 2011 20:55:26 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix for FAIL:
	test_run_module_bug1764407	(test.test_cmd_line.CmdLineTest)
Message-ID: <20110215195526.AEF24282BEA@codespeak.net>

Author: Vincent Legoll <vincent.legoll at gmail.com>
Branch: 
Changeset: r42001:fd3b18620932
Date: 2011-02-15 19:47 +0100
http://bitbucket.org/pypy/pypy/changeset/fd3b18620932/

Log:	Fix for FAIL: test_run_module_bug1764407
	(test.test_cmd_line.CmdLineTest)

	The test tries to run "pypy -i" and send input to its stdin via a
	pipe, which upsets pyrepl because it is not a tty-like file.

	Fix by only using pyrepl in interactive_console() if stdin is a tty.

	Signed-off-by: Vincent Legoll <vincent.legoll at gmail.com>

diff --git a/lib_pypy/_pypy_interact.py b/lib_pypy/_pypy_interact.py
--- a/lib_pypy/_pypy_interact.py
+++ b/lib_pypy/_pypy_interact.py
@@ -24,6 +24,11 @@
         pass
     #
     try:
+        if not os.isatty(sys.stdin.fileno()):
+            # Bail out if stdin is not tty-like, as pyrepl wouldn't be happy
+            # For example, with:
+            # subprocess.Popen(['pypy', '-i'], stdin=subprocess.PIPE)
+            raise ImportError
         from pyrepl.simple_interact import check
         if not check():
             raise ImportError

From commits-noreply at bitbucket.org  Tue Feb 15 22:26:05 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Tue, 15 Feb 2011 22:26:05 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix an ImportError.
Message-ID: <20110215212605.76E57282BEA@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r42002:2f79c2a9ff46
Date: 2011-02-15 16:25 -0500
http://bitbucket.org/pypy/pypy/changeset/2f79c2a9ff46/

Log:	Fix an ImportError.

diff --git a/.hgsubstate b/.hgsubstate
--- a/.hgsubstate
+++ b/.hgsubstate
@@ -1,3 +1,3 @@
 80037 greenlet
-80345 lib_pypy/pyrepl
+80348 lib_pypy/pyrepl
 80037 testrunner

diff --git a/lib_pypy/_pypy_interact.py b/lib_pypy/_pypy_interact.py
--- a/lib_pypy/_pypy_interact.py
+++ b/lib_pypy/_pypy_interact.py
@@ -1,6 +1,7 @@
 """Imported by app_main.py when PyPy needs to fire up the interactive console.
 """
 import sys
+import os
 
 
 def interactive_console(mainmodule=None):

From commits-noreply at bitbucket.org  Wed Feb 16 01:16:43 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 01:16:43 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: Need to regenerate ast.py
Message-ID: <20110216001643.DD30E36C231@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42011:8a49636ebdc4
Date: 2011-02-16 01:02 +0100
http://bitbucket.org/pypy/pypy/changeset/8a49636ebdc4/

Log:	Need to regenerate ast.py

diff --git a/pypy/interpreter/astcompiler/ast.py b/pypy/interpreter/astcompiler/ast.py
--- a/pypy/interpreter/astcompiler/ast.py
+++ b/pypy/interpreter/astcompiler/ast.py
@@ -3069,10 +3069,10 @@
     w_self.initialization_state |= 1
 
 _Module_field_unroller = unrolling_iterable(['body'])
-def Module_init(space, w_self, args):
+def Module_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(Module, w_self)
     w_self.w_body = None
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 1:
             w_err = space.wrap("Module constructor takes 0 or 1 positional arguments")
@@ -3111,10 +3111,10 @@
     w_self.initialization_state |= 1
 
 _Interactive_field_unroller = unrolling_iterable(['body'])
-def Interactive_init(space, w_self, args):
+def Interactive_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(Interactive, w_self)
     w_self.w_body = None
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 1:
             w_err = space.wrap("Interactive constructor takes 0 or 1 positional arguments")
@@ -3146,9 +3146,9 @@
     w_self.initialization_state |= 1
 
 _Expression_field_unroller = unrolling_iterable(['body'])
-def Expression_init(space, w_self, args):
+def Expression_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(Expression, w_self)
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 1:
             w_err = space.wrap("Expression constructor takes 0 or 1 positional arguments")
@@ -3187,10 +3187,10 @@
     w_self.initialization_state |= 1
 
 _Suite_field_unroller = unrolling_iterable(['body'])
-def Suite_init(space, w_self, args):
+def Suite_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(Suite, w_self)
     w_self.w_body = None
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 1:
             w_err = space.wrap("Suite constructor takes 0 or 1 positional arguments")
@@ -3294,11 +3294,11 @@
     w_self.initialization_state |= 8
 
 _FunctionDef_field_unroller = unrolling_iterable(['name', 'args', 'body', 'decorator_list', 'lineno', 'col_offset'])
-def FunctionDef_init(space, w_self, args):
+def FunctionDef_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(FunctionDef, w_self)
     w_self.w_body = None
     w_self.w_decorator_list = None
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 6:
             w_err = space.wrap("FunctionDef constructor takes 0 or 6 positional arguments")
@@ -3384,12 +3384,12 @@
     w_self.initialization_state |= 8
 
 _ClassDef_field_unroller = unrolling_iterable(['name', 'bases', 'body', 'decorator_list', 'lineno', 'col_offset'])
-def ClassDef_init(space, w_self, args):
+def ClassDef_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(ClassDef, w_self)
     w_self.w_bases = None
     w_self.w_body = None
     w_self.w_decorator_list = None
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 6:
             w_err = space.wrap("ClassDef constructor takes 0 or 6 positional arguments")
@@ -3424,9 +3424,9 @@
     w_self.initialization_state |= 1
 
 _Return_field_unroller = unrolling_iterable(['value', 'lineno', 'col_offset'])
-def Return_init(space, w_self, args):
+def Return_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(Return, w_self)
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 3:
             w_err = space.wrap("Return constructor takes 0 or 3 positional arguments")
@@ -3465,10 +3465,10 @@
     w_self.initialization_state |= 1
 
 _Delete_field_unroller = unrolling_iterable(['targets', 'lineno', 'col_offset'])
-def Delete_init(space, w_self, args):
+def Delete_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(Delete, w_self)
     w_self.w_targets = None
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 3:
             w_err = space.wrap("Delete constructor takes 0 or 3 positional arguments")
@@ -3517,10 +3517,10 @@
     w_self.initialization_state |= 2
 
 _Assign_field_unroller = unrolling_iterable(['targets', 'value', 'lineno', 'col_offset'])
-def Assign_init(space, w_self, args):
+def Assign_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(Assign, w_self)
     w_self.w_targets = None
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 4:
             w_err = space.wrap("Assign constructor takes 0 or 4 positional arguments")
@@ -3574,9 +3574,9 @@
     w_self.initialization_state |= 4
 
 _AugAssign_field_unroller = unrolling_iterable(['target', 'op', 'value', 'lineno', 'col_offset'])
-def AugAssign_init(space, w_self, args):
+def AugAssign_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(AugAssign, w_self)
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 5:
             w_err = space.wrap("AugAssign constructor takes 0 or 5 positional arguments")
@@ -3637,10 +3637,10 @@
     w_self.initialization_state |= 4
 
 _Print_field_unroller = unrolling_iterable(['dest', 'values', 'nl', 'lineno', 'col_offset'])
-def Print_init(space, w_self, args):
+def Print_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(Print, w_self)
     w_self.w_values = None
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 5:
             w_err = space.wrap("Print constructor takes 0 or 5 positional arguments")
@@ -3718,11 +3718,11 @@
     w_self.initialization_state |= 8
 
 _For_field_unroller = unrolling_iterable(['target', 'iter', 'body', 'orelse', 'lineno', 'col_offset'])
-def For_init(space, w_self, args):
+def For_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(For, w_self)
     w_self.w_body = None
     w_self.w_orelse = None
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 6:
             w_err = space.wrap("For constructor takes 0 or 6 positional arguments")
@@ -3791,11 +3791,11 @@
     w_self.initialization_state |= 4
 
 _While_field_unroller = unrolling_iterable(['test', 'body', 'orelse', 'lineno', 'col_offset'])
-def While_init(space, w_self, args):
+def While_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(While, w_self)
     w_self.w_body = None
     w_self.w_orelse = None
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 5:
             w_err = space.wrap("While constructor takes 0 or 5 positional arguments")
@@ -3863,11 +3863,11 @@
     w_self.initialization_state |= 4
 
 _If_field_unroller = unrolling_iterable(['test', 'body', 'orelse', 'lineno', 'col_offset'])
-def If_init(space, w_self, args):
+def If_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(If, w_self)
     w_self.w_body = None
     w_self.w_orelse = None
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 5:
             w_err = space.wrap("If constructor takes 0 or 5 positional arguments")
@@ -3928,10 +3928,10 @@
     w_self.initialization_state |= 4
 
 _With_field_unroller = unrolling_iterable(['context_expr', 'optional_vars', 'body', 'lineno', 'col_offset'])
-def With_init(space, w_self, args):
+def With_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(With, w_self)
     w_self.w_body = None
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 5:
             w_err = space.wrap("With constructor takes 0 or 5 positional arguments")
@@ -3985,9 +3985,9 @@
     w_self.initialization_state |= 4
 
 _Raise_field_unroller = unrolling_iterable(['type', 'inst', 'tback', 'lineno', 'col_offset'])
-def Raise_init(space, w_self, args):
+def Raise_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(Raise, w_self)
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 5:
             w_err = space.wrap("Raise constructor takes 0 or 5 positional arguments")
@@ -4062,12 +4062,12 @@
     w_self.initialization_state |= 4
 
 _TryExcept_field_unroller = unrolling_iterable(['body', 'handlers', 'orelse', 'lineno', 'col_offset'])
-def TryExcept_init(space, w_self, args):
+def TryExcept_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(TryExcept, w_self)
     w_self.w_body = None
     w_self.w_handlers = None
     w_self.w_orelse = None
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 5:
             w_err = space.wrap("TryExcept constructor takes 0 or 5 positional arguments")
@@ -4125,11 +4125,11 @@
     w_self.initialization_state |= 2
 
 _TryFinally_field_unroller = unrolling_iterable(['body', 'finalbody', 'lineno', 'col_offset'])
-def TryFinally_init(space, w_self, args):
+def TryFinally_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(TryFinally, w_self)
     w_self.w_body = None
     w_self.w_finalbody = None
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 4:
             w_err = space.wrap("TryFinally constructor takes 0 or 4 positional arguments")
@@ -4172,9 +4172,9 @@
     w_self.initialization_state |= 2
 
 _Assert_field_unroller = unrolling_iterable(['test', 'msg', 'lineno', 'col_offset'])
-def Assert_init(space, w_self, args):
+def Assert_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(Assert, w_self)
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 4:
             w_err = space.wrap("Assert constructor takes 0 or 4 positional arguments")
@@ -4214,10 +4214,10 @@
     w_self.initialization_state |= 1
 
 _Import_field_unroller = unrolling_iterable(['names', 'lineno', 'col_offset'])
-def Import_init(space, w_self, args):
+def Import_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(Import, w_self)
     w_self.w_names = None
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 3:
             w_err = space.wrap("Import constructor takes 0 or 3 positional arguments")
@@ -4279,10 +4279,10 @@
     w_self.initialization_state |= 4
 
 _ImportFrom_field_unroller = unrolling_iterable(['module', 'names', 'level', 'lineno', 'col_offset'])
-def ImportFrom_init(space, w_self, args):
+def ImportFrom_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(ImportFrom, w_self)
     w_self.w_names = None
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 5:
             w_err = space.wrap("ImportFrom constructor takes 0 or 5 positional arguments")
@@ -4336,9 +4336,9 @@
     w_self.initialization_state |= 4
 
 _Exec_field_unroller = unrolling_iterable(['body', 'globals', 'locals', 'lineno', 'col_offset'])
-def Exec_init(space, w_self, args):
+def Exec_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(Exec, w_self)
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 5:
             w_err = space.wrap("Exec constructor takes 0 or 5 positional arguments")
@@ -4379,10 +4379,10 @@
     w_self.initialization_state |= 1
 
 _Global_field_unroller = unrolling_iterable(['names', 'lineno', 'col_offset'])
-def Global_init(space, w_self, args):
+def Global_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(Global, w_self)
     w_self.w_names = None
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 3:
             w_err = space.wrap("Global constructor takes 0 or 3 positional arguments")
@@ -4414,9 +4414,9 @@
     w_self.initialization_state |= 1
 
 _Expr_field_unroller = unrolling_iterable(['value', 'lineno', 'col_offset'])
-def Expr_init(space, w_self, args):
+def Expr_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(Expr, w_self)
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 3:
             w_err = space.wrap("Expr constructor takes 0 or 3 positional arguments")
@@ -4438,9 +4438,9 @@
 Expr.typedef.acceptable_as_base_class = False
 
 _Pass_field_unroller = unrolling_iterable(['lineno', 'col_offset'])
-def Pass_init(space, w_self, args):
+def Pass_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(Pass, w_self)
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 2:
             w_err = space.wrap("Pass constructor takes 0 or 2 positional arguments")
@@ -4461,9 +4461,9 @@
 Pass.typedef.acceptable_as_base_class = False
 
 _Break_field_unroller = unrolling_iterable(['lineno', 'col_offset'])
-def Break_init(space, w_self, args):
+def Break_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(Break, w_self)
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 2:
             w_err = space.wrap("Break constructor takes 0 or 2 positional arguments")
@@ -4484,9 +4484,9 @@
 Break.typedef.acceptable_as_base_class = False
 
 _Continue_field_unroller = unrolling_iterable(['lineno', 'col_offset'])
-def Continue_init(space, w_self, args):
+def Continue_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(Continue, w_self)
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 2:
             w_err = space.wrap("Continue constructor takes 0 or 2 positional arguments")
@@ -4563,10 +4563,10 @@
     w_self.initialization_state |= 2
 
 _BoolOp_field_unroller = unrolling_iterable(['op', 'values', 'lineno', 'col_offset'])
-def BoolOp_init(space, w_self, args):
+def BoolOp_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(BoolOp, w_self)
     w_self.w_values = None
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 4:
             w_err = space.wrap("BoolOp constructor takes 0 or 4 positional arguments")
@@ -4620,9 +4620,9 @@
     w_self.initialization_state |= 4
 
 _BinOp_field_unroller = unrolling_iterable(['left', 'op', 'right', 'lineno', 'col_offset'])
-def BinOp_init(space, w_self, args):
+def BinOp_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(BinOp, w_self)
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 5:
             w_err = space.wrap("BinOp constructor takes 0 or 5 positional arguments")
@@ -4667,9 +4667,9 @@
     w_self.initialization_state |= 2
 
 _UnaryOp_field_unroller = unrolling_iterable(['op', 'operand', 'lineno', 'col_offset'])
-def UnaryOp_init(space, w_self, args):
+def UnaryOp_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(UnaryOp, w_self)
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 4:
             w_err = space.wrap("UnaryOp constructor takes 0 or 4 positional arguments")
@@ -4712,9 +4712,9 @@
     w_self.initialization_state |= 2
 
 _Lambda_field_unroller = unrolling_iterable(['args', 'body', 'lineno', 'col_offset'])
-def Lambda_init(space, w_self, args):
+def Lambda_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(Lambda, w_self)
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 4:
             w_err = space.wrap("Lambda constructor takes 0 or 4 positional arguments")
@@ -4767,9 +4767,9 @@
     w_self.initialization_state |= 4
 
 _IfExp_field_unroller = unrolling_iterable(['test', 'body', 'orelse', 'lineno', 'col_offset'])
-def IfExp_init(space, w_self, args):
+def IfExp_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(IfExp, w_self)
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 5:
             w_err = space.wrap("IfExp constructor takes 0 or 5 positional arguments")
@@ -4827,11 +4827,11 @@
     w_self.initialization_state |= 2
 
 _Dict_field_unroller = unrolling_iterable(['keys', 'values', 'lineno', 'col_offset'])
-def Dict_init(space, w_self, args):
+def Dict_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(Dict, w_self)
     w_self.w_keys = None
     w_self.w_values = None
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 4:
             w_err = space.wrap("Dict constructor takes 0 or 4 positional arguments")
@@ -4871,10 +4871,10 @@
     w_self.initialization_state |= 1
 
 _Set_field_unroller = unrolling_iterable(['elts', 'lineno', 'col_offset'])
-def Set_init(space, w_self, args):
+def Set_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(Set, w_self)
     w_self.w_elts = None
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 3:
             w_err = space.wrap("Set constructor takes 0 or 3 positional arguments")
@@ -4923,10 +4923,10 @@
     w_self.initialization_state |= 2
 
 _ListComp_field_unroller = unrolling_iterable(['elt', 'generators', 'lineno', 'col_offset'])
-def ListComp_init(space, w_self, args):
+def ListComp_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(ListComp, w_self)
     w_self.w_generators = None
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 4:
             w_err = space.wrap("ListComp constructor takes 0 or 4 positional arguments")
@@ -4976,10 +4976,10 @@
     w_self.initialization_state |= 2
 
 _SetComp_field_unroller = unrolling_iterable(['elt', 'generators', 'lineno', 'col_offset'])
-def SetComp_init(space, w_self, args):
+def SetComp_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(SetComp, w_self)
     w_self.w_generators = None
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 4:
             w_err = space.wrap("SetComp constructor takes 0 or 4 positional arguments")
@@ -5039,10 +5039,10 @@
     w_self.initialization_state |= 4
 
 _DictComp_field_unroller = unrolling_iterable(['key', 'value', 'generators', 'lineno', 'col_offset'])
-def DictComp_init(space, w_self, args):
+def DictComp_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(DictComp, w_self)
     w_self.w_generators = None
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 5:
             w_err = space.wrap("DictComp constructor takes 0 or 5 positional arguments")
@@ -5093,10 +5093,10 @@
     w_self.initialization_state |= 2
 
 _GeneratorExp_field_unroller = unrolling_iterable(['elt', 'generators', 'lineno', 'col_offset'])
-def GeneratorExp_init(space, w_self, args):
+def GeneratorExp_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(GeneratorExp, w_self)
     w_self.w_generators = None
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 4:
             w_err = space.wrap("GeneratorExp constructor takes 0 or 4 positional arguments")
@@ -5129,9 +5129,9 @@
     w_self.initialization_state |= 1
 
 _Yield_field_unroller = unrolling_iterable(['value', 'lineno', 'col_offset'])
-def Yield_init(space, w_self, args):
+def Yield_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(Yield, w_self)
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 3:
             w_err = space.wrap("Yield constructor takes 0 or 3 positional arguments")
@@ -5197,11 +5197,11 @@
     w_self.initialization_state |= 4
 
 _Compare_field_unroller = unrolling_iterable(['left', 'ops', 'comparators', 'lineno', 'col_offset'])
-def Compare_init(space, w_self, args):
+def Compare_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(Compare, w_self)
     w_self.w_ops = None
     w_self.w_comparators = None
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 5:
             w_err = space.wrap("Compare constructor takes 0 or 5 positional arguments")
@@ -5289,11 +5289,11 @@
     w_self.initialization_state |= 16
 
 _Call_field_unroller = unrolling_iterable(['func', 'args', 'keywords', 'starargs', 'kwargs', 'lineno', 'col_offset'])
-def Call_init(space, w_self, args):
+def Call_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(Call, w_self)
     w_self.w_args = None
     w_self.w_keywords = None
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 7:
             w_err = space.wrap("Call constructor takes 0 or 7 positional arguments")
@@ -5329,9 +5329,9 @@
     w_self.initialization_state |= 1
 
 _Repr_field_unroller = unrolling_iterable(['value', 'lineno', 'col_offset'])
-def Repr_init(space, w_self, args):
+def Repr_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(Repr, w_self)
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 3:
             w_err = space.wrap("Repr constructor takes 0 or 3 positional arguments")
@@ -5363,9 +5363,9 @@
     w_self.initialization_state |= 1
 
 _Num_field_unroller = unrolling_iterable(['n', 'lineno', 'col_offset'])
-def Num_init(space, w_self, args):
+def Num_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(Num, w_self)
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 3:
             w_err = space.wrap("Num constructor takes 0 or 3 positional arguments")
@@ -5400,9 +5400,9 @@
     w_self.initialization_state |= 1
 
 _Str_field_unroller = unrolling_iterable(['s', 'lineno', 'col_offset'])
-def Str_init(space, w_self, args):
+def Str_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(Str, w_self)
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 3:
             w_err = space.wrap("Str constructor takes 0 or 3 positional arguments")
@@ -5455,9 +5455,9 @@
     w_self.initialization_state |= 4
 
 _Attribute_field_unroller = unrolling_iterable(['value', 'attr', 'ctx', 'lineno', 'col_offset'])
-def Attribute_init(space, w_self, args):
+def Attribute_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(Attribute, w_self)
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 5:
             w_err = space.wrap("Attribute constructor takes 0 or 5 positional arguments")
@@ -5512,9 +5512,9 @@
     w_self.initialization_state |= 4
 
 _Subscript_field_unroller = unrolling_iterable(['value', 'slice', 'ctx', 'lineno', 'col_offset'])
-def Subscript_init(space, w_self, args):
+def Subscript_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(Subscript, w_self)
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 5:
             w_err = space.wrap("Subscript constructor takes 0 or 5 positional arguments")
@@ -5559,9 +5559,9 @@
     w_self.initialization_state |= 2
 
 _Name_field_unroller = unrolling_iterable(['id', 'ctx', 'lineno', 'col_offset'])
-def Name_init(space, w_self, args):
+def Name_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(Name, w_self)
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 4:
             w_err = space.wrap("Name constructor takes 0 or 4 positional arguments")
@@ -5612,10 +5612,10 @@
     w_self.initialization_state |= 2
 
 _List_field_unroller = unrolling_iterable(['elts', 'ctx', 'lineno', 'col_offset'])
-def List_init(space, w_self, args):
+def List_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(List, w_self)
     w_self.w_elts = None
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 4:
             w_err = space.wrap("List constructor takes 0 or 4 positional arguments")
@@ -5666,10 +5666,10 @@
     w_self.initialization_state |= 2
 
 _Tuple_field_unroller = unrolling_iterable(['elts', 'ctx', 'lineno', 'col_offset'])
-def Tuple_init(space, w_self, args):
+def Tuple_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(Tuple, w_self)
     w_self.w_elts = None
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 4:
             w_err = space.wrap("Tuple constructor takes 0 or 4 positional arguments")
@@ -5702,9 +5702,9 @@
     w_self.initialization_state |= 1
 
 _Const_field_unroller = unrolling_iterable(['value', 'lineno', 'col_offset'])
-def Const_init(space, w_self, args):
+def Const_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(Const, w_self)
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 3:
             w_err = space.wrap("Const constructor takes 0 or 3 positional arguments")
@@ -5780,9 +5780,9 @@
 slice.typedef.acceptable_as_base_class = False
 
 _Ellipsis_field_unroller = unrolling_iterable([])
-def Ellipsis_init(space, w_self, args):
+def Ellipsis_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(Ellipsis, w_self)
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         w_err = space.wrap("Ellipsis constructor takes no  arguments")
         raise OperationError(space.w_TypeError, w_err)
@@ -5828,9 +5828,9 @@
     w_self.initialization_state |= 4
 
 _Slice_field_unroller = unrolling_iterable(['lower', 'upper', 'step'])
-def Slice_init(space, w_self, args):
+def Slice_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(Slice, w_self)
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 3:
             w_err = space.wrap("Slice constructor takes 0 or 3 positional arguments")
@@ -5871,10 +5871,10 @@
     w_self.initialization_state |= 1
 
 _ExtSlice_field_unroller = unrolling_iterable(['dims'])
-def ExtSlice_init(space, w_self, args):
+def ExtSlice_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(ExtSlice, w_self)
     w_self.w_dims = None
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 1:
             w_err = space.wrap("ExtSlice constructor takes 0 or 1 positional arguments")
@@ -5906,9 +5906,9 @@
     w_self.initialization_state |= 1
 
 _Index_field_unroller = unrolling_iterable(['value'])
-def Index_init(space, w_self, args):
+def Index_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(Index, w_self)
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 1:
             w_err = space.wrap("Index constructor takes 0 or 1 positional arguments")
@@ -6187,10 +6187,10 @@
     w_self.initialization_state |= 4
 
 _comprehension_field_unroller = unrolling_iterable(['target', 'iter', 'ifs'])
-def comprehension_init(space, w_self, args):
+def comprehension_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(comprehension, w_self)
     w_self.w_ifs = None
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 3:
             w_err = space.wrap("comprehension constructor takes 0 or 3 positional arguments")
@@ -6279,10 +6279,10 @@
     w_self.initialization_state |= 4
 
 _ExceptHandler_field_unroller = unrolling_iterable(['type', 'name', 'body', 'lineno', 'col_offset'])
-def ExceptHandler_init(space, w_self, args):
+def ExceptHandler_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(ExceptHandler, w_self)
     w_self.w_body = None
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 5:
             w_err = space.wrap("ExceptHandler constructor takes 0 or 5 positional arguments")
@@ -6366,11 +6366,11 @@
     w_self.initialization_state |= 8
 
 _arguments_field_unroller = unrolling_iterable(['args', 'vararg', 'kwarg', 'defaults'])
-def arguments_init(space, w_self, args):
+def arguments_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(arguments, w_self)
     w_self.w_args = None
     w_self.w_defaults = None
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 4:
             w_err = space.wrap("arguments constructor takes 0 or 4 positional arguments")
@@ -6415,9 +6415,9 @@
     w_self.initialization_state |= 2
 
 _keyword_field_unroller = unrolling_iterable(['arg', 'value'])
-def keyword_init(space, w_self, args):
+def keyword_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(keyword, w_self)
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 2:
             w_err = space.wrap("keyword constructor takes 0 or 2 positional arguments")
@@ -6463,9 +6463,9 @@
     w_self.initialization_state |= 2
 
 _alias_field_unroller = unrolling_iterable(['name', 'asname'])
-def alias_init(space, w_self, args):
+def alias_init(space, w_self, __args__):
     w_self = space.descr_self_interp_w(alias, w_self)
-    args_w, kwargs_w = args.unpack()
+    args_w, kwargs_w = __args__.unpack()
     if args_w:
         if len(args_w) != 2:
             w_err = space.wrap("alias constructor takes 0 or 2 positional arguments")

From commits-noreply at bitbucket.org  Wed Feb 16 01:16:44 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 01:16:44 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: Remove dead code
Message-ID: <20110216001644.7A8EA36C231@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42012:e2a516b8be5e
Date: 2011-02-16 01:06 +0100
http://bitbucket.org/pypy/pypy/changeset/e2a516b8be5e/

Log:	Remove dead code

diff --git a/pypy/interpreter/typedef.py b/pypy/interpreter/typedef.py
--- a/pypy/interpreter/typedef.py
+++ b/pypy/interpreter/typedef.py
@@ -317,21 +317,7 @@
          func.__name__: func,
         'OperationError': OperationError
         }
-    if isinstance(cls, str):
-        assert 0, "unused?"
-        #print "<CHECK", func.__module__ or '?', func.__name__
-        assert cls.startswith('<'),"pythontype typecheck should begin with <"
-        source = """
-        def descr_typecheck_%(name)s(closure, space, w_obj, %(extra)s):
-            if not space.is_true(space.isinstance(w_obj, space.w_%(cls_name)s)):
-                # xxx improve msg
-                msg =  "descriptor is for '%(expected)s'"
-                raise OperationError(space.w_TypeError, space.wrap(msg))
-            return %(name)s(%(closure)s space, w_obj, %(extra)s)
-        """
-        cls_name = cls[1:]
-        expected = repr(cls_name)
-    elif cls is None:
+    if cls is None:
         source = """
         def descr_typecheck_%(name)s(closure, space, w_obj, %(extra)s):
             return %(name)s(%(closure)s space, w_obj, %(extra)s)

From commits-noreply at bitbucket.org  Wed Feb 16 01:16:45 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 01:16:45 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: A little magic to support
	GetSetProperties	getters with both signatures:
Message-ID: <20110216001645.E263D2A202C@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42013:a0b0f13a9e9d
Date: 2011-02-16 01:11 +0100
http://bitbucket.org/pypy/pypy/changeset/a0b0f13a9e9d/

Log:	A little magic to support GetSetProperties getters with both
	signatures: (space, self) or (self, space)

diff --git a/pypy/interpreter/typedef.py b/pypy/interpreter/typedef.py
--- a/pypy/interpreter/typedef.py
+++ b/pypy/interpreter/typedef.py
@@ -319,8 +319,8 @@
         }
     if cls is None:
         source = """
-        def descr_typecheck_%(name)s(closure, space, w_obj, %(extra)s):
-            return %(name)s(%(closure)s space, w_obj, %(extra)s)
+        def descr_typecheck_%(name)s(closure, space, obj, %(extra)s):
+            return %(name)s(%(args)s, %(extra)s)
         """
     else:
         cls_name = cls.__name__
@@ -328,16 +328,24 @@
         source = """
         def descr_typecheck_%(name)s(closure, space, w_obj, %(extra)s):
             obj = space.descr_self_interp_w(%(cls_name)s, w_obj)
-            return %(name)s(%(closure)s space, obj, %(extra)s)
+            return %(name)s(%(args)s, %(extra)s)
         """
         miniglobals[cls_name] = cls
     
     name = func.__name__
     extra = ', '.join(extraargs)
+    from pypy.interpreter import pycode
+    argnames, _, _ = pycode.cpython_code_signature(func.func_code)
     if use_closure:
-        closure = "closure,"
+        if argnames[1] == 'space':
+            args = "closure, space, obj"
+        else:
+            args = "closure, obj, space"
     else:
-        closure = ""
+        if argnames[0] == 'space':
+            args = "space, obj"
+        else:
+            args = "obj, space"
     source = py.code.Source(source % locals())
     exec source.compile() in miniglobals
     return miniglobals['descr_typecheck_%s' % func.__name__]

diff --git a/pypy/interpreter/test/test_typedef.py b/pypy/interpreter/test/test_typedef.py
--- a/pypy/interpreter/test/test_typedef.py
+++ b/pypy/interpreter/test/test_typedef.py
@@ -1,6 +1,7 @@
 from pypy.interpreter import typedef
 from pypy.tool.udir import udir
 from pypy.interpreter.baseobjspace import Wrappable
+from pypy.interpreter.gateway import ObjSpace
 
 # this test isn't so much to test that the objspace interface *works*
 # -- it's more to test that it's *there*
@@ -145,6 +146,24 @@
         w_obj = self.space.wrap(W_SomeType())
         assert self.space.getattr(w_obj, self.space.wrap('x')) is self.space.w_None
 
+    def test_getsetproperty_arguments(self):
+        class W_SomeType(Wrappable):
+            def fget1(space, w_self):
+                assert isinstance(space, ObjSpace)
+                assert isinstance(w_self, W_SomeType)
+            def fget2(self, space):
+                assert isinstance(space, ObjSpace)
+                assert isinstance(self, W_SomeType)
+        W_SomeType.typedef = typedef.TypeDef(
+            'some_type',
+            x1=typedef.GetSetProperty(W_SomeType.fget1),
+            x2=typedef.GetSetProperty(W_SomeType.fget2),
+            )
+        space = self.space
+        w_obj = space.wrap(W_SomeType())
+        assert space.getattr(w_obj, space.wrap('x1')) == space.w_None
+        assert space.getattr(w_obj, space.wrap('x2')) == space.w_None
+
     def test_unhashable(self):
         class W_SomeType(Wrappable):
             pass

From commits-noreply at bitbucket.org  Wed Feb 16 01:16:49 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 01:16:49 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: The order of arguments makes
	more sense	like this
Message-ID: <20110216001649.F038C282BEA@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42014:6737cf37ab55
Date: 2011-02-16 01:17 +0100
http://bitbucket.org/pypy/pypy/changeset/6737cf37ab55/

Log:	The order of arguments makes more sense like this

diff --git a/pypy/module/_io/interp_fileio.py b/pypy/module/_io/interp_fileio.py
--- a/pypy/module/_io/interp_fileio.py
+++ b/pypy/module/_io/interp_fileio.py
@@ -199,7 +199,7 @@
         else:
             return 'wb'
 
-    def descr_get_mode(space, self):
+    def descr_get_mode(self, space):
         return space.wrap(self._mode())
 
     def _closed(self, space):

diff --git a/pypy/module/_io/interp_bufferedio.py b/pypy/module/_io/interp_bufferedio.py
--- a/pypy/module/_io/interp_bufferedio.py
+++ b/pypy/module/_io/interp_bufferedio.py
@@ -150,14 +150,14 @@
         self.abs_pos = pos
         return pos
 
-    def closed_get_w(space, self):
+    def closed_get_w(self, space):
         self._check_init(space)
         return space.getattr(self.w_raw, space.wrap("closed"))
 
-    def name_get_w(space, self):
+    def name_get_w(self, space):
         return space.getattr(self.w_raw, space.wrap("name"))
 
-    def mode_get_w(space, self):
+    def mode_get_w(self, space):
         return space.getattr(self.w_raw, space.wrap("mode"))
 
     def readable_w(self, space):
@@ -863,7 +863,7 @@
             return space.w_True
         return space.call_method(self.w_reader, "isatty")
 
-    def closed_get_w(space, self):
+    def closed_get_w(self, space):
         return space.getattr(self.w_writer, space.wrap("closed"))
 
 methods = dict((method, interp2app(getattr(W_BufferedRWPair, method + '_w')))

diff --git a/pypy/module/_io/interp_iobase.py b/pypy/module/_io/interp_iobase.py
--- a/pypy/module/_io/interp_iobase.py
+++ b/pypy/module/_io/interp_iobase.py
@@ -83,7 +83,7 @@
             raise OperationError(
                 space.w_ValueError, space.wrap(message))
 
-    def closed_get_w(space, self):
+    def closed_get_w(self, space):
         return space.newbool(self.__IOBase_closed)
 
     def close_w(self, space):

diff --git a/pypy/module/_io/interp_textio.py b/pypy/module/_io/interp_textio.py
--- a/pypy/module/_io/interp_textio.py
+++ b/pypy/module/_io/interp_textio.py
@@ -54,7 +54,7 @@
         self.seennl = 0
         pendingcr = False
 
-    def newlines_get_w(space, self):
+    def newlines_get_w(self, space):
         return self.w_newlines_dict.get(self.seennl, space.w_None)
 
     @unwrap_spec(final=int)
@@ -214,7 +214,7 @@
     def detach_w(self, space):
         self._unsupportedoperation(space, "detach")
 
-    def errors_get_w(space, self):
+    def errors_get_w(self, space):
         return space.w_None
 
 
@@ -464,17 +464,17 @@
         self._check_init(space)
         return space.call_method(self.w_buffer, "fileno")
 
-    def closed_get_w(space, self):
+    def closed_get_w(self, space):
         self._check_init(space)
         return space.getattr(self.w_buffer, space.wrap("closed"))
 
-    def newlines_get_w(space, self):
+    def newlines_get_w(self, space):
         self._check_init(space)
         if self.w_decoder is None:
             return space.w_None
         return space.findattr(self.w_decoder, space.wrap("newlines"))
 
-    def name_get_w(space, self):
+    def name_get_w(self, space):
         self._check_init(space)
         return space.getattr(self.w_buffer, space.wrap("name"))
 
@@ -960,7 +960,7 @@
         cookie.chars_to_skip = chars_to_skip
         return space.newlong_from_rbigint(cookie.pack())
 
-    def chunk_size_get_w(space, self):
+    def chunk_size_get_w(self, space):
         self._check_init(space)
         return space.wrap(self.chunk_size)
 

diff --git a/pypy/module/_io/interp_bytesio.py b/pypy/module/_io/interp_bytesio.py
--- a/pypy/module/_io/interp_bytesio.py
+++ b/pypy/module/_io/interp_bytesio.py
@@ -171,7 +171,7 @@
     def close_w(self, space):
         self.buf = None
 
-    def closed_get_w(space, self):
+    def closed_get_w(self, space):
         return space.wrap(self.buf is None)
 
     def getstate_w(self, space):

diff --git a/pypy/module/_io/interp_stringio.py b/pypy/module/_io/interp_stringio.py
--- a/pypy/module/_io/interp_stringio.py
+++ b/pypy/module/_io/interp_stringio.py
@@ -256,13 +256,13 @@
     def close_w(self, space):
         self.buf = None
 
-    def closed_get_w(space, self):
+    def closed_get_w(self, space):
         return space.wrap(self.buf is None)
 
-    def line_buffering_get_w(space, self):
+    def line_buffering_get_w(self, space):
         return space.w_False
 
-    def newlines_get_w(space, self):
+    def newlines_get_w(self, space):
         if self.w_decoder is None:
             return space.w_None
         return space.getattr(self.w_decoder, space.wrap("newlines"))

From hakanardo at codespeak.net  Wed Feb 16 08:10:51 2011
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Wed, 16 Feb 2011 08:10:51 +0100 (CET)
Subject: [pypy-svn] r80349 - pypy/extradoc/planning
Message-ID: <20110216071051.37BA9282BE8@codespeak.net>

Author: hakanardo
Date: Wed Feb 16 08:10:49 2011
New Revision: 80349

Modified:
   pypy/extradoc/planning/jit.txt
Log:
Removed done items and added some new ideas and current plans

Modified: pypy/extradoc/planning/jit.txt
==============================================================================
--- pypy/extradoc/planning/jit.txt	(original)
+++ pypy/extradoc/planning/jit.txt	Wed Feb 16 08:10:49 2011
@@ -128,30 +128,19 @@
 ---------
  -  Replace full preamble with short preamble
 
- -  Move forcings of the failargs of inlined guards into the mini
-    bridges
-
  -  Reenable string optimizations in the preamble. This could be done
     currently, but would not make much sense as all string virtuals would
     be forced at the end of the preamble. Only the virtuals that
     contains new boxes inserted by the optimization that can possible be
     reused in the loops needs to be forced.
 
- -  Loop invariant operations that might raise OverflowError should
-    be moved out of the loop. Currently only the guard checking if there
-    was an OverflowError raise are moved out.
-
- -  The preamble of the second generated version of a loop 
-    is probably not needed if a short preamble was generated. It's
-    probably a better idea to have the mini bridges jump to the preamble
-    of the first generated version.
-
  -  Replace the list of short preambles with a tree, similar to the
     tree formed by the full preamble and it's bridges. This should
     enable specialisaton of loops in more complicated situations, e.g.
     test_dont_trace_every_iteration in test_basic.py. Currently the
     second case there become a badly optimized bridge from the
-    preamble to the preamble.
+    preamble to the preamble. This is solved differently with
+    jit-virtual_state, make sure the case mentioned is optimized.
 
  -  To remove more of the short preamble a lot more of the optimizer
     state would have to be saved and inherited by the bridges. However
@@ -162,3 +151,27 @@
     information is to introduce some VFromStartValue virtuals that
     would be some pseudo virtuals containing a single input argument
     box and it's index.
+
+  - When retracing a loop, make the optimizer optimizing the retraced
+    loop inherit the state of the optimizer optimizing the bridge
+    causing the loop to be retraced.
+
+  - After the jit-virtual_state is merge it should be possible to
+    generate the short preamble from the internal state of the
+    optimizer. This should be a lot easier and cleaner than trying to
+    decide when it is safe to reorder operations.
+
+  - Could the retracing be generalized to the point where the current
+    result after unrolling could be achieved by retracing a second
+    iteration of the loop instead of inlining the same trace? That
+    would remove the restricting assumptions made in unroll.py and
+    e.g. allow virtual string's to be kept alive across boundaries. It
+    should also better handle loops that don't take the exact same
+    path through the loop twice in a row.
+
+  - After the jit-virtual_state is merged, the curent policy of always
+    retracing (or jumping to the preamble) instead of forcing virtuals
+    when jumping to a loop should render the force_all_lazy_setfields()
+    at the end of the preamble unnessesary. If that policy wont hold
+    in the long run it should be straight forward to augument the
+    VirtualState objects with information about storesinking.


From commits-noreply at bitbucket.org  Wed Feb 16 10:26:01 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed, 16 Feb 2011 10:26:01 +0100 (CET)
Subject: [pypy-svn] pypy default: Oops.  Fix segfaults...
Message-ID: <20110216092601.CECC736C231@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r42015:f7234a720b93
Date: 2011-02-16 10:25 +0100
http://bitbucket.org/pypy/pypy/changeset/f7234a720b93/

Log:	Oops. Fix segfaults...

diff --git a/pypy/objspace/std/mapdict.py b/pypy/objspace/std/mapdict.py
--- a/pypy/objspace/std/mapdict.py
+++ b/pypy/objspace/std/mapdict.py
@@ -685,8 +685,9 @@
 
     @jit.dont_look_inside
     def is_valid_for_map(self, map):
+        # note that 'map' can be None here
         mymap = self.map_wref()
-        if mymap is map:     # also handles the case self.map_wref()->None
+        if mymap is not None and mymap is map:
             version_tag = map.terminator.w_cls.version_tag()
             if version_tag is self.version_tag:
                 # everything matches, it's incredibly fast

From commits-noreply at bitbucket.org  Wed Feb 16 10:40:15 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 10:40:15 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix warning messages "KeyError ignored"
	while running	tests, or during translation.
Message-ID: <20110216094015.29B8836C231@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r42016:fe67291f9e3e
Date: 2011-02-15 16:52 +0100
http://bitbucket.org/pypy/pypy/changeset/fe67291f9e3e/

Log:	Fix warning messages "KeyError ignored" while running tests, or
	during translation.

diff --git a/pypy/rpython/lltypesystem/lltype.py b/pypy/rpython/lltypesystem/lltype.py
--- a/pypy/rpython/lltypesystem/lltype.py
+++ b/pypy/rpython/lltypesystem/lltype.py
@@ -13,6 +13,27 @@
 
 TLS = tlsobject()
 
+class WeakValueDictionary(weakref.WeakValueDictionary):
+    """A subclass of weakref.WeakValueDictionary
+    which resets the 'nested_hash_level' when keys are being deleted.
+    """
+    def __init__(self, *args, **kwargs):
+        weakref.WeakValueDictionary.__init__(self, *args, **kwargs)
+        remove_base = self._remove
+        def remove(*args):
+            nested_hash_level = TLS.nested_hash_level
+            try:
+                # The 'remove' function is called when an object dies.  This
+                # can happen anywhere when they are reference cycles,
+                # especially when we are already computing another __hash__
+                # value.  It's not really a recursion in this case, so we
+                # reset the counter; otherwise the hash value may be be
+                # incorrect and the key won't be deleted.
+                TLS.nested_hash_level = 0
+                remove_base(*args)
+            finally:
+                TLS.nested_hash_level = nested_hash_level
+        self._remove = remove
 
 class _uninitialized(object):
     def __init__(self, TYPE):
@@ -397,7 +418,7 @@
     # behaves more or less like a Struct with fields item0, item1, ...
     # but also supports __getitem__(), __setitem__(), __len__().
 
-    _cache = weakref.WeakValueDictionary() # cache the length-1 FixedSizeArrays
+    _cache = WeakValueDictionary() # cache the length-1 FixedSizeArrays
     def __new__(cls, OF, length, **kwds):
         if length == 1 and not kwds:
             try:
@@ -620,7 +641,7 @@
 class Ptr(LowLevelType):
     __name__ = property(lambda self: '%sPtr' % self.TO.__name__)
 
-    _cache = weakref.WeakValueDictionary()  # cache the Ptrs
+    _cache = WeakValueDictionary()  # cache the Ptrs
     def __new__(cls, TO, use_cache=True):
         if not isinstance(TO, ContainerType):
             raise TypeError, ("can only point to a Container type, "

From commits-noreply at bitbucket.org  Wed Feb 16 10:40:15 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 10:40:15 +0100 (CET)
Subject: [pypy-svn] pypy default: Another TODO item
Message-ID: <20110216094015.A851736C231@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r42017:76129afaa04d
Date: 2011-02-16 10:13 +0100
http://bitbucket.org/pypy/pypy/changeset/76129afaa04d/

Log:	Another TODO item

diff --git a/lib-python/TODO b/lib-python/TODO
--- a/lib-python/TODO
+++ b/lib-python/TODO
@@ -29,6 +29,11 @@
                           unicode_literals)
   assert type("") is unicode
 
+- Code like::
+
+  assert(x is not None, "error message")
+
+  should emit a SyntaxWarning when compiled (the tuple is always true)
 
 
 Medium tasks

From commits-noreply at bitbucket.org  Wed Feb 16 11:03:57 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed, 16 Feb 2011 11:03:57 +0100 (CET)
Subject: [pypy-svn] pypy collections-module: Generalize the test.
Message-ID: <20110216100357.96B1236C21D@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: collections-module
Changeset: r42018:b375e0d99e91
Date: 2011-02-16 10:58 +0100
http://bitbucket.org/pypy/pypy/changeset/b375e0d99e91/

Log:	Generalize the test.

diff --git a/lib-python/modified-2.7.0/test/test_deque.py b/lib-python/modified-2.7.0/test/test_deque.py
--- a/lib-python/modified-2.7.0/test/test_deque.py
+++ b/lib-python/modified-2.7.0/test/test_deque.py
@@ -343,7 +343,10 @@
         for match in (True, False):
             d = deque(['ab'])
             d.extend([MutateCmp(d, match), 'c'])
-            self.assertRaises(IndexError, d.remove, 'c')
+            # On CPython we get IndexError: deque mutated during remove().
+            # Why is it an IndexError during remove() only???
+            # On PyPy it is a RuntimeError, as in the other operations.
+            self.assertRaises((IndexError, RuntimeError), d.remove, 'c')
             self.assertEqual(d, deque())
 
     def test_repr(self):

From commits-noreply at bitbucket.org  Wed Feb 16 11:03:58 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed, 16 Feb 2011 11:03:58 +0100 (CET)
Subject: [pypy-svn] pypy collections-module: Add usemodules=_collections.
Message-ID: <20110216100358.3320736C21D@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: collections-module
Changeset: r42019:37b257ad6cea
Date: 2011-02-16 10:58 +0100
http://bitbucket.org/pypy/pypy/changeset/37b257ad6cea/

Log:	Add usemodules=_collections.

diff --git a/lib-python/conftest.py b/lib-python/conftest.py
--- a/lib-python/conftest.py
+++ b/lib-python/conftest.py
@@ -195,7 +195,7 @@
     RegrTest('test_dbm.py'),
     RegrTest('test_decimal.py'),
     RegrTest('test_decorators.py', core=True),
-    RegrTest('test_deque.py', core=True),
+    RegrTest('test_deque.py', core=True, usemodules='_collections'),
     RegrTest('test_descr.py', core=True, usemodules='_weakref'),
     RegrTest('test_descrtut.py', core=True),
     RegrTest('test_dict.py', core=True),
@@ -499,7 +499,7 @@
     RegrTest('test_complex_args.py'),
     RegrTest('test_contextlib.py', usemodules="thread"),
     RegrTest('test_ctypes.py', usemodules="_rawffi"),
-    RegrTest('test_defaultdict.py'),
+    RegrTest('test_defaultdict.py', usemodules='_collections'),
     RegrTest('test_email_renamed.py'),
     RegrTest('test_exception_variations.py'),
     RegrTest('test_float.py'),

From commits-noreply at bitbucket.org  Wed Feb 16 11:03:58 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed, 16 Feb 2011 11:03:58 +0100 (CET)
Subject: [pypy-svn] pypy collections-module: Generalize this test. Now
	test_deque passes.
Message-ID: <20110216100358.AC18536C21D@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: collections-module
Changeset: r42020:9ac38af09338
Date: 2011-02-16 10:59 +0100
http://bitbucket.org/pypy/pypy/changeset/9ac38af09338/

Log:	Generalize this test. Now test_deque passes.

diff --git a/lib-python/modified-2.7.0/test/test_deque.py b/lib-python/modified-2.7.0/test/test_deque.py
--- a/lib-python/modified-2.7.0/test/test_deque.py
+++ b/lib-python/modified-2.7.0/test/test_deque.py
@@ -109,7 +109,7 @@
         self.assertEqual(deque('abc', maxlen=4).maxlen, 4)
         self.assertEqual(deque('abc', maxlen=2).maxlen, 2)
         self.assertEqual(deque('abc', maxlen=0).maxlen, 0)
-        with self.assertRaises(AttributeError):
+        with self.assertRaises((AttributeError, TypeError)):
             d = deque('abc')
             d.maxlen = 10
 

From commits-noreply at bitbucket.org  Wed Feb 16 11:07:29 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed, 16 Feb 2011 11:07:29 +0100 (CET)
Subject: [pypy-svn] pypy collections-module: Add _collections in
	all_working_modules.
Message-ID: <20110216100729.485A736C21D@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: collections-module
Changeset: r42021:ff8d589945e5
Date: 2011-02-16 11:06 +0100
http://bitbucket.org/pypy/pypy/changeset/ff8d589945e5/

Log:	Add _collections in all_working_modules.

diff --git a/pypy/config/pypyoption.py b/pypy/config/pypyoption.py
--- a/pypy/config/pypyoption.py
+++ b/pypy/config/pypyoption.py
@@ -32,7 +32,8 @@
      "crypt", "signal", "_rawffi", "termios", "zlib", "bz2",
      "struct", "_hashlib", "_md5", "_sha", "_minimal_curses", "cStringIO",
      "thread", "itertools", "pyexpat", "_ssl", "cpyext", "array",
-     "_bisect", "binascii", "_multiprocessing", '_warnings']
+     "_bisect", "binascii", "_multiprocessing", '_warnings',
+     "_collections"]
 ))
 
 translation_modules = default_modules.copy()

From commits-noreply at bitbucket.org  Wed Feb 16 11:07:29 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed, 16 Feb 2011 11:07:29 +0100 (CET)
Subject: [pypy-svn] pypy default: Merge collections-module,
	adding the RPython module "_collections".
Message-ID: <20110216100729.943C336C231@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r42022:834156df915e
Date: 2011-02-16 11:06 +0100
http://bitbucket.org/pypy/pypy/changeset/834156df915e/

Log:	Merge collections-module, adding the RPython module "_collections".


From commits-noreply at bitbucket.org  Wed Feb 16 11:07:29 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed, 16 Feb 2011 11:07:29 +0100 (CET)
Subject: [pypy-svn] pypy collections-module: Done.
Message-ID: <20110216100729.E91FB36C21D@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: collections-module
Changeset: r42023:72609f529790
Date: 2011-02-16 11:07 +0100
http://bitbucket.org/pypy/pypy/changeset/72609f529790/

Log:	Done.


From commits-noreply at bitbucket.org  Wed Feb 16 11:07:30 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed, 16 Feb 2011 11:07:30 +0100 (CET)
Subject: [pypy-svn] pypy default: merge heads
Message-ID: <20110216100730.4EAD336C231@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r42024:4df39c33ef9c
Date: 2011-02-16 11:07 +0100
http://bitbucket.org/pypy/pypy/changeset/4df39c33ef9c/

Log:	merge heads


From commits-noreply at bitbucket.org  Wed Feb 16 11:26:37 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Wed, 16 Feb 2011 11:26:37 +0100 (CET)
Subject: [pypy-svn] pypy arm-backend-2: Really free the blocks
Message-ID: <20110216102637.354572A202C@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backend-2
Changeset: r42025:02c1bf8e5637
Date: 2011-02-11 20:41 +0100
http://bitbucket.org/pypy/pypy/changeset/02c1bf8e5637/

Log:	Really free the blocks

diff --git a/pypy/jit/backend/arm/assembler.py b/pypy/jit/backend/arm/assembler.py
--- a/pypy/jit/backend/arm/assembler.py
+++ b/pypy/jit/backend/arm/assembler.py
@@ -68,6 +68,7 @@
         assert self.memcpy_addr != 0, 'setup_once() not called?'
         self.mc = ARMv7Builder()
         self.guard_descrs = []
+        self.blocks = []
 
     def setup_once(self):
         # Addresses of functions called by new_xxx operations
@@ -228,7 +229,8 @@
         # 1 separator byte
         # 4 bytes for the faildescr
         memsize = (len(arglocs)-1)*6+5
-        datablockwrapper = MachineDataBlockWrapper(self.cpu.asmmemmgr, [])
+        datablockwrapper = MachineDataBlockWrapper(self.cpu.asmmemmgr, 
+                                                    self.blocks)
         memaddr = datablockwrapper.malloc_aligned(memsize, alignment=WORD)
         datablockwrapper.done()
         mem = rffi.cast(rffi.CArrayPtr(lltype.Char), memaddr)
@@ -403,6 +405,8 @@
 
     def materialize_loop(self, looptoken):
         allblocks = self.get_asmmemmgr_blocks(looptoken)
+        for block in self.blocks:
+            allblocks.append(block)
         return self.mc.materialize(self.cpu.asmmemmgr, allblocks,
                                    self.cpu.gc_ll_descr.gcrootmap)
 

From commits-noreply at bitbucket.org  Wed Feb 16 11:26:37 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Wed, 16 Feb 2011 11:26:37 +0100 (CET)
Subject: [pypy-svn] pypy arm-backend-2: Split the constant loading into two
	different	methods. In total this saves 1 instruction per
	constant, in the worst case, and makes the exection a bit faster.
Message-ID: <20110216102637.ADFE72A202C@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backend-2
Changeset: r42026:03e49b433021
Date: 2011-02-11 22:57 +0100
http://bitbucket.org/pypy/pypy/changeset/03e49b433021/

Log:	Split the constant loading into two different methods. In total this
	saves 1 instruction per constant, in the worst case, and makes the
	exection a bit faster.

diff --git a/pypy/jit/backend/arm/codebuilder.py b/pypy/jit/backend/arm/codebuilder.py
--- a/pypy/jit/backend/arm/codebuilder.py
+++ b/pypy/jit/backend/arm/codebuilder.py
@@ -128,19 +128,29 @@
     def currpos(self):
         raise NotImplementedError
 
-    size_of_gen_load_int = 4 * WORD
-    ofs_shift = zip(range(8, 25, 8), range(12, 0, -4))
-    def gen_load_int(self, r, value, cond=cond.AL):
+    size_of_gen_load_int = 3 * WORD
+    def gen_load_int(self, r, value, c=cond.AL):
         """r is the register number, value is the value to be loaded to the
         register"""
-        self.MOV_ri(r, (value & 0xFF), cond=cond)
+        if c != cond.AL or 0 <= value <= 0xFFFF:
+            self._load_by_shifting(r, value, c)
+        else:
+            self.LDR_ri(r, reg.pc.value)
+            self.MOV_rr(reg.pc.value, reg.pc.value)
+            self.write32(value)
+
+    #size_of_gen_load_int = 4 * WORD
+    ofs_shift = zip(range(8, 25, 8), range(12, 0, -4))
+    def _load_by_shifting(self, r, value, c=cond.AL):
+        # to be sure it is only called for the correct cases
+        assert c != cond.AL or 0 <= value <= 0xFFFF
+        self.MOV_ri(r, (value & 0xFF), cond=c)
         for offset, shift in self.ofs_shift:
             b = (value >> offset) & 0xFF
             if b == 0:
                 continue
             t = b | (shift << 8)
-            self.ORR_ri(r, r, imm=t, cond=cond)
-
+            self.ORR_ri(r, r, imm=t, cond=c)
 
 class OverwritingBuilder(AbstractARMv7Builder):
     def __init__(self, cb, start, size):

From commits-noreply at bitbucket.org  Wed Feb 16 11:26:52 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Wed, 16 Feb 2011 11:26:52 +0100 (CET)
Subject: [pypy-svn] pypy arm-backend-2: merge default
Message-ID: <20110216102652.753D1282BE8@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backend-2
Changeset: r42027:f2d488757ef0
Date: 2011-02-12 10:51 +0100
http://bitbucket.org/pypy/pypy/changeset/f2d488757ef0/

Log:	merge default

diff --git a/pypy/module/readline/test/test_c_readline.py b/pypy/module/readline/test/test_c_readline.py
deleted file mode 100644
--- a/pypy/module/readline/test/test_c_readline.py
+++ /dev/null
@@ -1,16 +0,0 @@
-"""
-Directly test the basic ctypes wrappers.
-"""
-
-import py
-from pypy import conftest; conftest.translation_test_so_skip_if_appdirect()
-from pypy.rpython.tool import rffi_platform as platform
-
-try:
-    from pypy.module.readline import c_readline
-except platform.CompilationError, e:
-    py.test.skip(e)
-
-
-def test_basic_import():
-    c_readline.c_rl_initialize()

diff --git a/pypy/rpython/llinterp.py b/pypy/rpython/llinterp.py
--- a/pypy/rpython/llinterp.py
+++ b/pypy/rpython/llinterp.py
@@ -824,9 +824,18 @@
     def op_gc_thread_run(self):
         self.heap.thread_run()
 
+    def op_gc_thread_start(self):
+        self.heap.thread_start()
+
     def op_gc_thread_die(self):
         self.heap.thread_die()
 
+    def op_gc_thread_before_fork(self):
+        raise NotImplementedError
+
+    def op_gc_thread_after_fork(self):
+        raise NotImplementedError
+
     def op_gc_free(self, addr):
         # what can you do?
         pass

diff --git a/pypy/interpreter/argument.py b/pypy/interpreter/argument.py
--- a/pypy/interpreter/argument.py
+++ b/pypy/interpreter/argument.py
@@ -147,8 +147,19 @@
 
     def _combine_starargs_wrapped(self, w_stararg):
         # unpack the * arguments 
-        self.arguments_w = (self.arguments_w +
-                            self.space.fixedview(w_stararg))
+        space = self.space
+        try:
+            args_w = space.fixedview(w_stararg)
+        except OperationError, e:
+            if e.match(space, space.w_TypeError):
+                w_type = space.type(w_stararg)
+                typename = w_type.getname(space)
+                raise OperationError(
+                    space.w_TypeError,
+                    space.wrap("argument after * must be "
+                               "a sequence, not %s" % (typename,)))
+            raise
+        self.arguments_w = self.arguments_w + args_w
 
     def _combine_starstarargs_wrapped(self, w_starstararg):
         # unpack the ** arguments
@@ -163,7 +174,7 @@
             except OperationError, e:
                 if e.match(space, space.w_AttributeError):
                     w_type = space.type(w_starstararg)
-                    typename = w_type.getname(space, '?')
+                    typename = w_type.getname(space)
                     raise OperationError(
                         space.w_TypeError,
                         space.wrap("argument after ** must be "
@@ -185,10 +196,15 @@
             try:
                 key = space.str_w(w_key)
             except OperationError, e:
-                if not e.match(space, space.w_TypeError):
-                    raise
-                raise OperationError(space.w_TypeError,
-                                     space.wrap("keywords must be strings"))
+                if e.match(space, space.w_TypeError):
+                    raise OperationError(
+                        space.w_TypeError,
+                        space.wrap("keywords must be strings"))
+                if e.match(space, space.w_UnicodeEncodeError):
+                    raise OperationError(
+                        space.w_TypeError,
+                        space.wrap("keyword cannot be encoded to ascii"))
+                raise
             if self.keywords and key in self.keywords:
                 raise operationerrfmt(self.space.w_TypeError,
                                       "got multiple values "

diff --git a/pypy/jit/backend/detect_cpu.py b/pypy/jit/backend/detect_cpu.py
--- a/pypy/jit/backend/detect_cpu.py
+++ b/pypy/jit/backend/detect_cpu.py
@@ -32,7 +32,8 @@
                 'x86':   'x86',    # Apple
                 'Power Macintosh': 'ppc',
                 'x86_64': 'x86',
-                'amd64': 'x86',     # freebsd
+                'amd64': 'x86',    # freebsd
+                'AMD64': 'x86',    # win64
                 'armv7l': 'arm',
                 }[mach]
     except KeyError:

diff --git a/pypy/translator/goal/app_main.py b/pypy/translator/goal/app_main.py
--- a/pypy/translator/goal/app_main.py
+++ b/pypy/translator/goal/app_main.py
@@ -164,13 +164,20 @@
     print >> sys.stderr, 'usage: %s [options]' % (sys.executable,)
     print >> sys.stderr, 'Try `%s -h` for more information.' % (sys.executable,)
 
+def fdopen(fd, mode, bufsize=-1):
+    try:
+        fdopen = file.fdopen
+    except AttributeError:     # only on top of CPython, running tests
+        from os import fdopen
+    return fdopen(fd, mode, bufsize)
+
 def set_unbuffered_io():
-    sys.stdin  = sys.__stdin__  = os.fdopen(0, 'rb', 0)
-    sys.stdout = sys.__stdout__ = os.fdopen(1, 'wb', 0)
-    sys.stderr = sys.__stderr__ = os.fdopen(2, 'wb', 0)
+    sys.stdin  = sys.__stdin__  = fdopen(0, 'rb', 0)
+    sys.stdout = sys.__stdout__ = fdopen(1, 'wb', 0)
+    sys.stderr = sys.__stderr__ = fdopen(2, 'wb', 0)
 
 def set_fully_buffered_io():
-    sys.stdout = sys.__stdout__ = os.fdopen(1, 'w')
+    sys.stdout = sys.__stdout__ = fdopen(1, 'w')
 
 # ____________________________________________________________
 # Main entry point
@@ -499,7 +506,7 @@
             # handle the "-m" command
             def run_it():
                 import runpy
-                runpy.run_module(sys.argv[0], None, '__main__', True)
+                runpy._run_module_as_main(sys.argv[0])
             success = run_toplevel(run_it)
         elif run_stdin:
             # handle the case where no command/filename/module is specified

diff --git a/pypy/translator/goal/test2/test_app_main.py b/pypy/translator/goal/test2/test_app_main.py
--- a/pypy/translator/goal/test2/test_app_main.py
+++ b/pypy/translator/goal/test2/test_app_main.py
@@ -391,10 +391,9 @@
         child.expect(re.escape('File: ' + p))
         child.expect(re.escape('Argv: ' + repr([p, 'extra'])))
         child.expect('>>> ')
-        #XXX the following doesn't work on CPython 2.5 either
-        #child.sendline('somevalue')
-        #child.expect(re.escape(repr("foobar")))
-        #child.expect('>>> ')
+        child.sendline('somevalue')
+        child.expect(re.escape(repr("foobar")))
+        child.expect('>>> ')
         child.sendline('import sys')
         child.sendline('"pypy.translator.goal.test2" in sys.modules')
         child.expect('True')

diff --git a/pypy/module/_rawffi/test/test__rawffi.py b/pypy/module/_rawffi/test/test__rawffi.py
--- a/pypy/module/_rawffi/test/test__rawffi.py
+++ b/pypy/module/_rawffi/test/test__rawffi.py
@@ -5,7 +5,6 @@
 from pypy.translator.tool.cbuild import ExternalCompilationInfo
 from pypy.module._rawffi.interp_rawffi import TYPEMAP
 from pypy.module._rawffi.tracker import Tracker
-from pypy.translator.platform import platform
 
 import os, sys, py
 
@@ -211,6 +210,7 @@
             cls.w_libm_name = space.wrap('libm.so')
             if sys.platform == "darwin":
                 cls.w_libm_name = space.wrap('libm.dylib')
+        cls.w_platform = space.wrap(platform.name)
         cls.w_sizes_and_alignments = space.wrap(dict(
             [(k, (v.c_size, v.c_alignment)) for k,v in TYPEMAP.iteritems()]))
 
@@ -908,8 +908,8 @@
         raises(_rawffi.SegfaultException, a.__setitem__, 3, 3)
 
     def test_stackcheck(self):
-        if not self.iswin32:
-            skip("win32 specific")
+        if self.platform != "msvc":
+            skip("win32 msvc specific")
 
         # Even if the call corresponds to the specified signature,
         # the STDCALL calling convention may detect some errors

diff --git a/pypy/jit/metainterp/optimizeopt/__init__.py b/pypy/jit/metainterp/optimizeopt/__init__.py
--- a/pypy/jit/metainterp/optimizeopt/__init__.py
+++ b/pypy/jit/metainterp/optimizeopt/__init__.py
@@ -6,7 +6,8 @@
 from pypy.jit.metainterp.optimizeopt.string import OptString
 from pypy.jit.metainterp.optimizeopt.unroll import optimize_unroll, OptInlineShortPreamble
 
-def optimize_loop_1(metainterp_sd, loop, unroll=True, inline_short_preamble=True):
+def optimize_loop_1(metainterp_sd, loop, unroll=True,
+                    inline_short_preamble=True, retraced=False):
     """Optimize loop.operations to remove internal overheadish operations. 
     """
     opt_str = OptString()
@@ -17,7 +18,7 @@
                      OptHeap(),
                     ]
     if inline_short_preamble:
-        optimizations = [OptInlineShortPreamble()] +  optimizations
+        optimizations = [OptInlineShortPreamble(retraced)] +  optimizations
         
     if metainterp_sd.jit_ffi:
         from pypy.jit.metainterp.optimizeopt.fficall import OptFfiCall
@@ -33,6 +34,8 @@
         optimizer = Optimizer(metainterp_sd, loop, optimizations)
         optimizer.propagate_all_forward()
 
-def optimize_bridge_1(metainterp_sd, bridge, inline_short_preamble=True):
+def optimize_bridge_1(metainterp_sd, bridge, inline_short_preamble=True,
+                      retraced=False):
     """The same, but for a bridge. """
-    optimize_loop_1(metainterp_sd, bridge, False, inline_short_preamble)
+    optimize_loop_1(metainterp_sd, bridge, False, inline_short_preamble,
+                    retraced)

diff --git a/pypy/rlib/jit.py b/pypy/rlib/jit.py
--- a/pypy/rlib/jit.py
+++ b/pypy/rlib/jit.py
@@ -271,7 +271,7 @@
 
 # ____________________________________________________________
 
-class JitDriver:    
+class JitDriver(object):    
     """Base class to declare fine-grained user control on the JIT.  So
     far, there must be a singleton instance of JitDriver.  This style
     will allow us (later) to support a single RPython program with

diff --git a/pypy/module/readline/c_readline.py b/pypy/module/readline/c_readline.py
deleted file mode 100644
--- a/pypy/module/readline/c_readline.py
+++ /dev/null
@@ -1,77 +0,0 @@
-from pypy.rpython.tool import rffi_platform as platform
-from pypy.rpython.lltypesystem import lltype, rffi
-from pypy.interpreter.error import OperationError
-from pypy.interpreter.gateway import ObjSpace, interp2app
-from pypy.translator.tool.cbuild import ExternalCompilationInfo
-
-# On various platforms, linking only with libreadline is not enough;
-# we also need to link with some variant of curses or libtermcap.
-# We follow the logic of CPython below.
-def try_with_lib(extralibs, **kwds):
-    global most_recent_error
-    # at least on Gentoo Linux, readline.h doesn't compile if stdio.h is not
-    # included before
-    eci = ExternalCompilationInfo(
-        includes = ["stdio.h", "readline/readline.h", "readline/history.h"],
-        libraries = extralibs + ['readline'],
-        )
-    try:
-        platform.verify_eci(eci)
-        return eci
-    except platform.CompilationError, e:
-        most_recent_error = e
-        return None
-
-eci = (try_with_lib([]) or
-       try_with_lib(['ncursesw']) or
-       try_with_lib(['ncurses']) or
-       try_with_lib(['curses']) or
-       try_with_lib(['termcap'], library_dirs=['/usr/lib/termcap']))
-if eci is None:
-    raise most_recent_error
-
-# ____________________________________________________________
-
-def external(name, args, result):
-    return rffi.llexternal(name, args, result, compilation_info=eci)
-
-# get a binding to  c library functions and define their args and return types
-# char *readline(char *)
-c_readline = external('readline', [rffi.CCHARP], rffi.CCHARP)
-
-# void rl_initiliaze(void)
-c_rl_initialize = external('rl_initialize', [], lltype.Void)
-
-# void using_history(void)
-c_using_history = external('using_history', [], lltype.Void)
-
-# void add_history(const char *)
-c_add_history = external('add_history', [rffi.CCHARP], lltype.Void)
-
-#------------------------------------------------------------
-# special initialization of readline 
-
-class ReadlineState(object):
-    lastline = ""        # XXX possibly temporary hack
-readlinestate = ReadlineState()
-
-def setup_readline(space, w_module):
-    c_using_history()
-    # XXX CPython initializes more stuff here
-    c_rl_initialize()
-    # install sys.__raw_input__, a hook that will be used by raw_input()
-    space.setitem(space.sys.w_dict, space.wrap('__raw_input__'),
-                  space.wrap(app_readline_func))
-
-def readline_func(space, prompt):
-    ll_res = c_readline(prompt)
-    if not ll_res:
-        raise OperationError(space.w_EOFError, space.w_None)
-    res = rffi.charp2str(ll_res)
-    if res and res != readlinestate.lastline:
-        readlinestate.lastline = res
-        c_add_history(res)
-    return space.wrap(res)
-
-readline_func.unwrap_spec = [ObjSpace, str]
-app_readline_func = interp2app(readline_func)

diff --git a/pypy/module/readline/app_stub.py b/pypy/module/readline/app_stub.py
deleted file mode 100644
--- a/pypy/module/readline/app_stub.py
+++ /dev/null
@@ -1,13 +0,0 @@
-# NOT_RPYTHON
-
-def stub(*args, **kwds):
-    import warnings
-    warnings.warn("the 'readline' module is only a stub so far")
-
-def stub_str(*args, **kwds):
-    stub()
-    return ''
-
-def stub_int(*args, **kwds):
-    stub()
-    return 0

diff --git a/pypy/rpython/memory/gctransform/framework.py b/pypy/rpython/memory/gctransform/framework.py
--- a/pypy/rpython/memory/gctransform/framework.py
+++ b/pypy/rpython/memory/gctransform/framework.py
@@ -955,11 +955,32 @@
         if hasattr(self.root_walker, 'thread_run_ptr'):
             hop.genop("direct_call", [self.root_walker.thread_run_ptr])
 
+    def gct_gc_thread_start(self, hop):
+        assert self.translator.config.translation.thread
+        if hasattr(self.root_walker, 'thread_start_ptr'):
+            hop.genop("direct_call", [self.root_walker.thread_start_ptr])
+
     def gct_gc_thread_die(self, hop):
         assert self.translator.config.translation.thread
         if hasattr(self.root_walker, 'thread_die_ptr'):
             hop.genop("direct_call", [self.root_walker.thread_die_ptr])
 
+    def gct_gc_thread_before_fork(self, hop):
+        if (self.translator.config.translation.thread
+            and hasattr(self.root_walker, 'thread_before_fork_ptr')):
+            hop.genop("direct_call", [self.root_walker.thread_before_fork_ptr],
+                      resultvar=hop.spaceop.result)
+        else:
+            c_null = rmodel.inputconst(llmemory.Address, llmemory.NULL)
+            hop.genop("same_as", [c_null],
+                      resultvar=hop.spaceop.result)
+
+    def gct_gc_thread_after_fork(self, hop):
+        if (self.translator.config.translation.thread
+            and hasattr(self.root_walker, 'thread_after_fork_ptr')):
+            hop.genop("direct_call", [self.root_walker.thread_after_fork_ptr]
+                                     + hop.spaceop.args)
+
     def gct_gc_get_type_info_group(self, hop):
         return hop.cast_result(self.c_type_info_group)
 
@@ -1242,6 +1263,7 @@
 
 class BaseRootWalker(object):
     need_root_stack = False
+    thread_setup = None
 
     def __init__(self, gctransformer):
         self.gcdata = gctransformer.gcdata
@@ -1251,7 +1273,8 @@
         return True
 
     def setup_root_walker(self):
-        pass
+        if self.thread_setup is not None:
+            self.thread_setup()
 
     def walk_roots(self, collect_stack_root,
                    collect_static_in_prebuilt_nongc,
@@ -1284,7 +1307,6 @@
 
 class ShadowStackRootWalker(BaseRootWalker):
     need_root_stack = True
-    thread_setup = None
     collect_stacks_from_other_threads = None
 
     def __init__(self, gctransformer):
@@ -1324,8 +1346,7 @@
         ll_assert(bool(stackbase), "could not allocate root stack")
         self.gcdata.root_stack_top  = stackbase
         self.gcdata.root_stack_base = stackbase
-        if self.thread_setup is not None:
-            self.thread_setup()
+        BaseRootWalker.setup_root_walker(self)
 
     def walk_stack_roots(self, collect_stack_root):
         gcdata = self.gcdata
@@ -1390,6 +1411,9 @@
             occur in this thread.
             """
             aid = get_aid()
+            if aid == gcdata.main_thread:
+                return   # ignore calls to thread_die() in the main thread
+                         # (which can occur after a fork()).
             gcdata.thread_stacks.setitem(aid, llmemory.NULL)
             old = gcdata.root_stack_base
             if gcdata._fresh_rootstack == llmemory.NULL:
@@ -1435,7 +1459,7 @@
             gcdata.active_thread = new_aid
 
         def collect_stack(aid, stacktop, callback):
-            if stacktop != llmemory.NULL and aid != get_aid():
+            if stacktop != llmemory.NULL and aid != gcdata.active_thread:
                 # collect all valid stacks from the dict (the entry
                 # corresponding to the current thread is not valid)
                 gc = self.gc
@@ -1447,11 +1471,45 @@
                     addr += sizeofaddr
 
         def collect_more_stacks(callback):
+            ll_assert(get_aid() == gcdata.active_thread,
+                      "collect_more_stacks(): invalid active_thread")
             gcdata.thread_stacks.foreach(collect_stack, callback)
 
+        def _free_if_not_current(aid, stacktop, _):
+            if stacktop != llmemory.NULL and aid != gcdata.active_thread:
+                end = stacktop - sizeofaddr
+                base = end.address[0]
+                llmemory.raw_free(base)
+
+        def thread_after_fork(result_of_fork, opaqueaddr):
+            # we don't need a thread_before_fork in this case, so
+            # opaqueaddr == NULL.  This is called after fork().
+            if result_of_fork == 0:
+                # We are in the child process.  Assumes that only the
+                # current thread survived, so frees the shadow stacks
+                # of all the other ones.
+                gcdata.thread_stacks.foreach(_free_if_not_current, None)
+                # Clears the dict (including the current thread, which
+                # was an invalid entry anyway and will be recreated by
+                # the next call to save_away_current_stack()).
+                gcdata.thread_stacks.clear()
+                # Finally, reset the stored thread IDs, in case it
+                # changed because of fork().  Also change the main
+                # thread to the current one (because there is not any
+                # other left).
+                aid = get_aid()
+                gcdata.main_thread = aid
+                gcdata.active_thread = aid
+
         self.thread_setup = thread_setup
         self.thread_prepare_ptr = getfn(thread_prepare, [], annmodel.s_None)
         self.thread_run_ptr = getfn(thread_run, [], annmodel.s_None,
                                     inline=True)
+        # no thread_start_ptr here
         self.thread_die_ptr = getfn(thread_die, [], annmodel.s_None)
+        # no thread_before_fork_ptr here
+        self.thread_after_fork_ptr = getfn(thread_after_fork,
+                                           [annmodel.SomeInteger(),
+                                            annmodel.SomeAddress()],
+                                           annmodel.s_None)
         self.collect_stacks_from_other_threads = collect_more_stacks

diff --git a/pypy/rlib/clibffi.py b/pypy/rlib/clibffi.py
--- a/pypy/rlib/clibffi.py
+++ b/pypy/rlib/clibffi.py
@@ -31,6 +31,7 @@
 if _WIN32:
     separate_module_sources = ['''
     #include <stdio.h>
+    #include <windows.h>
 
     /* Get the module where the "fopen" function resides in */
     HANDLE pypy_get_libc_handle() {
@@ -49,14 +50,11 @@
 else:
     separate_module_sources = []
 
-if not _MSVC:
+if not _WIN32:
     # On some platforms, we try to link statically libffi, which is small
     # anyway and avoids endless troubles for installing.  On other platforms
     # libffi.a is typically not there, so we link dynamically.
-    if _MINGW:
-        includes = ['windows.h', 'ffi.h']
-    else:
-        includes = ['ffi.h']
+    includes = ['ffi.h']
 
     if _MAC_OS:
         pre_include_bits = ['#define MACOSX']
@@ -90,6 +88,22 @@
         link_files = link_files,
         testonly_libraries = ['ffi'],
     )
+elif _MINGW:
+    includes = ['ffi.h']
+    libraries = ['libffi-5']
+
+    eci = ExternalCompilationInfo(
+        libraries = libraries,
+        includes = includes,
+        export_symbols = [],
+        separate_module_sources = separate_module_sources,
+        )
+
+    eci = rffi_platform.configure_external_library(
+        'libffi', eci,
+        [dict(prefix='libffi-',
+              include_dir='include', library_dir='.libs'),
+         ])
 else:
     libffidir = py.path.local(pypydir).join('translator', 'c', 'src', 'libffi_msvc')
     eci = ExternalCompilationInfo(
@@ -214,17 +228,10 @@
     return rffi.llexternal(name, args, result, compilation_info=eci, calling_conv='win')
 
 
-if not _WIN32:
+if not _MSVC:
     def check_fficall_result(result, flags):
         pass # No check
-    
-    libc_name = ctypes.util.find_library('c')
-    assert libc_name is not None, "Cannot find C library, ctypes.util.find_library('c') returned None"
-
-    def get_libc_name():
-        return libc_name
-
-if _WIN32:
+else:
     def check_fficall_result(result, flags):
         if result == 0:
             return
@@ -245,8 +252,13 @@
                 "Procedure called with too many "
                 "arguments (%d bytes in excess) " % (result,))
 
-    LoadLibrary = rwin32.LoadLibrary
+if not _WIN32:
+    libc_name = ctypes.util.find_library('c')
+    assert libc_name is not None, "Cannot find C library, ctypes.util.find_library('c') returned None"
 
+    def get_libc_name():
+        return libc_name
+elif _MSVC:
     get_libc_handle = external('pypy_get_libc_handle', [], DLLHANDLE)
 
     def get_libc_name():
@@ -254,7 +266,12 @@
 
     assert "msvcr" in get_libc_name().lower(), \
            "Suspect msvcrt library: %s" % (get_libc_name(),)
+elif _MINGW:
+    def get_libc_name():
+        return 'msvcrt.dll'
 
+if _WIN32:
+    LoadLibrary = rwin32.LoadLibrary
 
 FFI_OK = cConfig.FFI_OK
 FFI_BAD_TYPEDEF = cConfig.FFI_BAD_TYPEDEF
@@ -270,7 +287,7 @@
 
 c_ffi_prep_cif = external('ffi_prep_cif', [FFI_CIFP, ffi_abi, rffi.UINT,
                                            FFI_TYPE_P, FFI_TYPE_PP], rffi.INT)
-if _WIN32:
+if _MSVC:
     c_ffi_call_return_type = rffi.INT
 else:
     c_ffi_call_return_type = lltype.Void

diff --git a/pypy/jit/metainterp/simple_optimize.py b/pypy/jit/metainterp/simple_optimize.py
--- a/pypy/jit/metainterp/simple_optimize.py
+++ b/pypy/jit/metainterp/simple_optimize.py
@@ -47,7 +47,8 @@
             jumpop.setdescr(loop.token)
         return None
 
-def optimize_bridge(metainterp_sd, old_loops, loop, inline_short_preamble):
+def optimize_bridge(metainterp_sd, old_loops, loop, inline_short_preamble,
+                    retraced):
     optimize_loop(metainterp_sd, [], loop)
     jumpop = loop.operations[-1]
     if jumpop.getopnum() == rop.JUMP:

diff --git a/pypy/interpreter/test/test_argument.py b/pypy/interpreter/test/test_argument.py
--- a/pypy/interpreter/test/test_argument.py
+++ b/pypy/interpreter/test/test_argument.py
@@ -117,13 +117,14 @@
 
     def type(self, obj):
         class Type:
-            def getname(self, space, default):
+            def getname(self, space, default='?'):
                 return type(obj).__name__
         return Type()
 
 
     w_TypeError = TypeError
     w_AttributeError = AttributeError
+    w_UnicodeEncodeError = UnicodeEncodeError
     w_dict = dict
 
 class TestArgumentsNormal(object):
@@ -474,6 +475,35 @@
         assert w_args == (1, )
         assert not w_kwds
 
+    def test_argument_unicode(self):
+        space = DummySpace()
+        w_starstar = space.wrap({u'abc': 5})
+        args = Arguments(space, [], w_starstararg=w_starstar)
+        l = [None]
+        args._match_signature(None, l, Signature(['abc']))
+        assert len(l) == 1
+        assert l[0] == space.wrap(5)
+        #
+        def str_w(w):
+            try:
+                return str(w)
+            except UnicodeEncodeError:
+                raise OperationError(space.w_UnicodeEncodeError,
+                                     space.wrap("oups"))
+        space.str_w = str_w
+        w_starstar = space.wrap({u'\u1234': 5})
+        err = py.test.raises(OperationError, Arguments,
+                             space, [], w_starstararg=w_starstar)
+        # Check that we get a TypeError.  On CPython it is because of
+        # "no argument called '?'".  On PyPy we get a TypeError too, but
+        # earlier: "keyword cannot be encoded to ascii".  The
+        # difference, besides the error message, is only apparent if the
+        # receiver also takes a **arg.  Then CPython passes the
+        # non-ascii unicode unmodified, whereas PyPy complains.  We will
+        # not care until someone has a use case for that.
+        assert not err.value.match(space, space.w_UnicodeEncodeError)
+        assert     err.value.match(space, space.w_TypeError)
+
 class TestErrorHandling(object):
     def test_missing_args(self):
         # got_nargs, nkwds, expected_nargs, has_vararg, has_kwarg,
@@ -500,6 +530,22 @@
         s = err.getmsg('foo')
         assert s == "foo() takes exactly 1 argument (3 given)"
 
+    def test_bad_type_for_star(self):
+        space = self.space
+        try:
+            Arguments(space, [], w_stararg=space.wrap(42))
+        except OperationError, e:
+            msg = space.str_w(space.str(e.get_w_value(space)))
+            assert msg == "argument after * must be a sequence, not int"
+        else:
+            assert 0, "did not raise"
+        try:
+            Arguments(space, [], w_starstararg=space.wrap(42))
+        except OperationError, e:
+            msg = space.str_w(space.str(e.get_w_value(space)))
+            assert msg == "argument after ** must be a mapping, not int"
+        else:
+            assert 0, "did not raise"
 
     def test_unknown_keywords(self):
         err = ArgErrUnknownKwds(1, ['a', 'b'], [True, False])

diff --git a/pypy/translator/c/src/signals.h b/pypy/translator/c/src/signals.h
--- a/pypy/translator/c/src/signals.h
+++ b/pypy/translator/c/src/signals.h
@@ -119,7 +119,12 @@
 
     if (wakeup_fd != -1) 
       {
-        ssize_t res = write(wakeup_fd, "\0", 1);
+#ifndef _WIN32
+        ssize_t res;
+#else
+        int res;
+#endif
+        res = write(wakeup_fd, "\0", 1);
         /* the return value is ignored here */
       }
 }

diff --git a/pypy/interpreter/pyopcode.py b/pypy/interpreter/pyopcode.py
--- a/pypy/interpreter/pyopcode.py
+++ b/pypy/interpreter/pyopcode.py
@@ -1513,10 +1513,7 @@
         if not isinstance(globals, dict):
             if not hasattr(globals, '__getitem__'):
                 raise TypeError("exec: arg 2 must be a dictionary or None")
-        try:
-            globals['__builtins__']
-        except KeyError:
-            globals['__builtins__'] = builtin
+        globals.setdefault('__builtins__', builtin)
         if not isinstance(locals, dict):
             if not hasattr(locals, '__getitem__'):
                 raise TypeError("exec: arg 3 must be a dictionary or None")

diff --git a/pypy/module/readline/test/test_with_pypy.py b/pypy/module/readline/test/test_with_pypy.py
deleted file mode 100644
--- a/pypy/module/readline/test/test_with_pypy.py
+++ /dev/null
@@ -1,27 +0,0 @@
-"""
-Test the readline library on top of PyPy.  The following tests run
-in the PyPy interpreter, itself running on top of CPython
-"""
-
-import py
-from pypy.conftest import gettestobjspace
-from pypy.rpython.tool import rffi_platform as platform
-
-try:
-    from pypy.module.readline import c_readline
-except platform.CompilationError, e:
-    py.test.skip(e)
-
-
-class AppTestReadline:
-
-    def setup_class(cls):
-        # enable usage of the readline mixedmodule
-        space = gettestobjspace(usemodules=('readline',))
-        cls.space = space
-
-    def test_basic_import(self):
-        # this is interpreted by PyPy
-        import readline 
-        readline.readline
-        # XXX test more

diff --git a/pypy/jit/backend/llgraph/llimpl.py b/pypy/jit/backend/llgraph/llimpl.py
--- a/pypy/jit/backend/llgraph/llimpl.py
+++ b/pypy/jit/backend/llgraph/llimpl.py
@@ -852,9 +852,6 @@
     def op_cast_ptr_to_int(self, descr, ptr):
         return cast_to_int(ptr)
 
-    def op_uint_xor(self, descr, arg1, arg2):
-        return arg1 ^ arg2
-
     def op_force_token(self, descr):
         opaque_frame = _to_opaque(self)
         return llmemory.cast_ptr_to_adr(opaque_frame)

diff --git a/pypy/translator/c/test/test_standalone.py b/pypy/translator/c/test/test_standalone.py
--- a/pypy/translator/c/test/test_standalone.py
+++ b/pypy/translator/c/test/test_standalone.py
@@ -1,6 +1,7 @@
 import py
 import sys, os, re
 
+from pypy.rlib.objectmodel import keepalive_until_here
 from pypy.rlib.rarithmetic import r_longlong
 from pypy.rlib.debug import ll_assert, have_debug_prints
 from pypy.rlib.debug import debug_print, debug_start, debug_stop
@@ -832,6 +833,7 @@
             ll_thread.release_NOAUTO(state.ll_lock)
         def after():
             ll_thread.acquire_NOAUTO(state.ll_lock, True)
+            ll_thread.gc_thread_run()
 
         class Cons:
             def __init__(self, head, tail):
@@ -839,8 +841,16 @@
                 self.tail = tail
 
         def bootstrap():
+            ll_thread.gc_thread_start()
             state.xlist.append(Cons(123, Cons(456, None)))
             gc.collect()
+            ll_thread.gc_thread_die()
+
+        def new_thread():
+            ll_thread.gc_thread_prepare()
+            ident = ll_thread.start_new_thread(bootstrap, ())
+            time.sleep(0.5)    # enough time to start, hopefully
+            return ident
 
         def entry_point(argv):
             os.write(1, "hello world\n")
@@ -850,14 +860,14 @@
             state.ll_lock = ll_thread.allocate_ll_lock()
             after()
             invoke_around_extcall(before, after)
-            ident1 = ll_thread.start_new_thread(bootstrap, ())
-            ident2 = ll_thread.start_new_thread(bootstrap, ())
+            ident1 = new_thread()
+            ident2 = new_thread()
             #
             gc.collect()
             #
-            ident3 = ll_thread.start_new_thread(bootstrap, ())
-            ident4 = ll_thread.start_new_thread(bootstrap, ())
-            ident5 = ll_thread.start_new_thread(bootstrap, ())
+            ident3 = new_thread()
+            ident4 = new_thread()
+            ident5 = new_thread()
             # wait for the 5 threads to finish
             while True:
                 gc.collect()
@@ -885,3 +895,115 @@
                                      '3 ok',
                                      '4 ok',
                                      '5 ok']
+
+
+    def test_thread_and_gc_with_fork(self):
+        # This checks that memory allocated for the shadow stacks of the
+        # other threads is really released when doing a fork() -- or at
+        # least that the object referenced from stacks that are no longer
+        # alive are really freed.
+        import time, gc, os
+        from pypy.module.thread import ll_thread
+        from pypy.rlib.objectmodel import invoke_around_extcall
+        if not hasattr(os, 'fork'):
+            py.test.skip("requires fork()")
+
+        class State:
+            pass
+        state = State()
+
+        def before():
+            ll_assert(not ll_thread.acquire_NOAUTO(state.ll_lock, False),
+                      "lock not held!")
+            ll_thread.release_NOAUTO(state.ll_lock)
+        def after():
+            ll_thread.acquire_NOAUTO(state.ll_lock, True)
+            ll_thread.gc_thread_run()
+
+        class Cons:
+            def __init__(self, head, tail):
+                self.head = head
+                self.tail = tail
+
+        class Stuff:
+            def __del__(self):
+                os.write(state.write_end, 'd')
+
+        def allocate_stuff():
+            s = Stuff()
+            os.write(state.write_end, 'a')
+            return s
+
+        def run_in_thread():
+            for i in range(10):
+                state.xlist.append(Cons(123, Cons(456, None)))
+                time.sleep(0.01)
+            childpid = os.fork()
+            return childpid
+
+        def bootstrap():
+            ll_thread.gc_thread_start()
+            childpid = run_in_thread()
+            gc.collect()        # collect both in the child and in the parent
+            gc.collect()
+            gc.collect()
+            if childpid == 0:
+                os.write(state.write_end, 'c')   # "I did not die!" from child
+            else:
+                os.write(state.write_end, 'p')   # "I did not die!" from parent
+            ll_thread.gc_thread_die()
+
+        def new_thread():
+            ll_thread.gc_thread_prepare()
+            ident = ll_thread.start_new_thread(bootstrap, ())
+            time.sleep(0.5)    # enough time to start, hopefully
+            return ident
+
+        def start_all_threads():
+            s = allocate_stuff()
+            ident1 = new_thread()
+            ident2 = new_thread()
+            ident3 = new_thread()
+            ident4 = new_thread()
+            ident5 = new_thread()
+            # wait for 4 more seconds, which should be plenty of time
+            time.sleep(4)
+            keepalive_until_here(s)
+
+        def entry_point(argv):
+            os.write(1, "hello world\n")
+            state.xlist = []
+            state.deleted = 0
+            state.read_end, state.write_end = os.pipe()
+            x2 = Cons(51, Cons(62, Cons(74, None)))
+            # start 5 new threads
+            state.ll_lock = ll_thread.allocate_ll_lock()
+            after()
+            invoke_around_extcall(before, after)
+            start_all_threads()
+            # force freeing
+            gc.collect()
+            gc.collect()
+            gc.collect()
+            # return everything that was written to the pipe so far,
+            # followed by the final dot.
+            os.write(state.write_end, '.')
+            result = os.read(state.read_end, 256)
+            os.write(1, "got: %s\n" % result)
+            return 0
+
+        t, cbuilder = self.compile(entry_point)
+        data = cbuilder.cmdexec('')
+        print repr(data)
+        header, footer = data.splitlines()
+        assert header == 'hello world'
+        assert footer.startswith('got: ')
+        result = footer[5:]
+        # check that all 5 threads and 5 forked processes
+        # finished successfully, that we did 1 allocation,
+        # and that it was freed 6 times -- once in the parent
+        # process and once in every child process.
+        assert (result[-1] == '.'
+                and result.count('c') == result.count('p') == 5
+                and result.count('a') == 1
+                and result.count('d') == 6)

diff --git a/pypy/jit/metainterp/pyjitpl.py b/pypy/jit/metainterp/pyjitpl.py
--- a/pypy/jit/metainterp/pyjitpl.py
+++ b/pypy/jit/metainterp/pyjitpl.py
@@ -1929,7 +1929,8 @@
         try:
             target_loop_token = compile.compile_new_bridge(self,
                                                            [loop_token],
-                                                           self.resumekey)
+                                                           self.resumekey,
+                                                           True)
         except RetraceLoop:
             assert False
         assert target_loop_token is not None

diff --git a/pypy/module/sys/__init__.py b/pypy/module/sys/__init__.py
--- a/pypy/module/sys/__init__.py
+++ b/pypy/module/sys/__init__.py
@@ -79,6 +79,7 @@
 
         'float_info'            : 'system.get_float_info(space)',
         'long_info'             : 'system.get_long_info(space)',
+        'float_repr_style'      : 'system.get_float_repr_style(space)'
         }
 
     if sys.platform == 'win32':

diff --git a/pypy/module/array/interp_array.py b/pypy/module/array/interp_array.py
--- a/pypy/module/array/interp_array.py
+++ b/pypy/module/array/interp_array.py
@@ -33,7 +33,7 @@
         if w_args.keywords: # XXX this might be forbidden fishing
             msg = 'array.array() does not take keyword arguments'
             raise OperationError(space.w_TypeError, space.wrap(msg))
-        
+
     for tc in unroll_typecodes:
         if typecode == tc:
             a = space.allocate_instance(types[tc].w_class, w_cls)
@@ -253,7 +253,7 @@
             space = self.space
             oldlen = self.len
             try:
-                new = space.int_w(space.len(w_seq))
+                new = space.len_w(w_seq)
                 self.setlen(self.len + new)
             except OperationError:
                 pass
@@ -534,7 +534,7 @@
         cbuf = self.charbuf()
         s = ''.join([cbuf[i] for i in xrange(self.len * mytype.bytes)])
         return self.space.wrap(s)
-##         
+##
 ##         s = ''
 ##         i = 0
 ##         while i < self.len * mytype.bytes:

diff --git a/pypy/rpython/lltypesystem/lloperation.py b/pypy/rpython/lltypesystem/lloperation.py
--- a/pypy/rpython/lltypesystem/lloperation.py
+++ b/pypy/rpython/lltypesystem/lloperation.py
@@ -465,7 +465,10 @@
                                  # ^^^ but canunwindgc=False, as it is
                                  # allocating non-GC structures only
     'gc_thread_run'       : LLOp(),
+    'gc_thread_start'     : LLOp(),
     'gc_thread_die'       : LLOp(),
+    'gc_thread_before_fork':LLOp(),   # returns an opaque address
+    'gc_thread_after_fork': LLOp(),   # arguments: (result_of_fork, opaqueaddr)
     'gc_assume_young_pointers': LLOp(canrun=True),
     'gc_writebarrier_before_copy': LLOp(canrun=True),
     'gc_heap_stats'       : LLOp(canunwindgc=True),

diff --git a/pypy/interpreter/baseobjspace.py b/pypy/interpreter/baseobjspace.py
--- a/pypy/interpreter/baseobjspace.py
+++ b/pypy/interpreter/baseobjspace.py
@@ -655,6 +655,10 @@
         """shortcut for space.int_w(space.hash(w_obj))"""
         return self.int_w(self.hash(w_obj))
 
+    def len_w(self, w_obj):
+        """shotcut for space.int_w(space.len(w_obj))"""
+        return self.int_w(self.len(w_obj))
+
     def setitem_str(self, w_obj, key, w_value):
         return self.setitem(w_obj, self.wrap(key), w_value)
 

diff --git a/pypy/module/readline/test/__init__.py b/pypy/module/readline/test/__init__.py
deleted file mode 100644
--- a/pypy/module/readline/test/__init__.py
+++ /dev/null
@@ -1,1 +0,0 @@
-#

diff --git a/pypy/interpreter/executioncontext.py b/pypy/interpreter/executioncontext.py
--- a/pypy/interpreter/executioncontext.py
+++ b/pypy/interpreter/executioncontext.py
@@ -165,7 +165,12 @@
     def return_trace(self, frame, w_retval):
         "Trace the return from a function"
         if self.w_tracefunc is not None:
-            self._trace(frame, 'return', w_retval)
+            return_from_hidden = self._trace(frame, 'return', w_retval)
+            # special case: if we are returning from a hidden function,
+            # then maybe we have to fire() the action again; otherwise
+            # it will not be called.  See test_trace_hidden_prints.
+            if return_from_hidden:
+                self.space.frame_trace_action.fire()
 
     def bytecode_trace(self, frame, decr_by=TICK_COUNTER_STEP):
         "Trace function called before each bytecode."
@@ -254,7 +259,7 @@
 
     def _trace(self, frame, event, w_arg, operr=None):
         if self.is_tracing or frame.hide():
-            return
+            return True
 
         space = self.space
         
@@ -292,7 +297,7 @@
         if self.profilefunc is not None:
             if event not in ['leaveframe', 'call', 'c_call',
                              'c_return', 'c_exception']:
-                return
+                return False
 
             last_exception = frame.last_exception
             if event == 'leaveframe':
@@ -312,6 +317,7 @@
             finally:
                 frame.last_exception = last_exception
                 self.is_tracing -= 1
+        return False
 
     def checksignals(self):
         """Similar to PyErr_CheckSignals().  If called in the main thread,
@@ -505,7 +511,8 @@
     """An action that calls the local trace functions (w_f_trace)."""
 
     def perform(self, executioncontext, frame):
-        if frame.w_f_trace is None or executioncontext.is_tracing:
+        if (frame.w_f_trace is None or executioncontext.is_tracing or
+            executioncontext.w_tracefunc is None):
             return
         code = frame.pycode
         if frame.instr_lb <= frame.last_instr < frame.instr_ub:

diff --git a/pypy/module/readline/__init__.py b/pypy/module/readline/__init__.py
deleted file mode 100644
--- a/pypy/module/readline/__init__.py
+++ /dev/null
@@ -1,45 +0,0 @@
-# this is a sketch of how one might one day be able to define a pretty simple
-# ctypes-using module, suitable for feeding to the ext-compiler
-
-from pypy.interpreter.mixedmodule import MixedModule
-
-# XXX raw_input needs to check for space.readline_func and use
-# it if its there 
-
-class Module(MixedModule):
-    """Importing this module enables command line editing using GNU readline."""
-    # the above line is the doc string of the translated module  
-
-    def setup_after_space_initialization(self):
-        from pypy.module.readline import c_readline 
-        c_readline.setup_readline(self.space, self)
-
-    interpleveldefs = {
-        'readline'    : 'interp_readline.readline',
-    }
-
-    appleveldefs = {
-        'parse_and_bind':     'app_stub.stub',
-        'get_line_buffer':    'app_stub.stub_str',
-        'insert_text':        'app_stub.stub',
-        'read_init_file':     'app_stub.stub',
-        'read_history_file':  'app_stub.stub',
-        'write_history_file': 'app_stub.stub',
-        'clear_history':      'app_stub.stub',
-        'get_history_length': 'app_stub.stub_int',
-        'set_history_length': 'app_stub.stub',
-        'get_current_history_length': 'app_stub.stub_int',
-        'get_history_item':           'app_stub.stub_str',
-        'remove_history_item':        'app_stub.stub',
-        'replace_history_item':       'app_stub.stub',
-        'redisplay':                  'app_stub.stub',
-        'set_startup_hook':           'app_stub.stub',
-        'set_pre_input_hook':         'app_stub.stub',
-        'set_completer':      'app_stub.stub',
-        'get_completer':      'app_stub.stub',
-        'get_begidx':         'app_stub.stub_int',
-        'get_endidx':         'app_stub.stub_int',
-        'set_completer_delims':       'app_stub.stub',
-        'get_completer_delims':       'app_stub.stub_str',
-        'add_history':        'app_stub.stub',
-    }

diff --git a/pypy/objspace/std/objspace.py b/pypy/objspace/std/objspace.py
--- a/pypy/objspace/std/objspace.py
+++ b/pypy/objspace/std/objspace.py
@@ -22,7 +22,7 @@
 from pypy.objspace.std.floatobject import W_FloatObject
 from pypy.objspace.std.intobject import W_IntObject
 from pypy.objspace.std.listobject import W_ListObject
-from pypy.objspace.std.longobject import W_LongObject
+from pypy.objspace.std.longobject import W_LongObject, newlong
 from pypy.objspace.std.noneobject import W_NoneObject
 from pypy.objspace.std.objectobject import W_ObjectObject
 from pypy.objspace.std.ropeobject import W_RopeObject
@@ -292,7 +292,7 @@
         return W_LongObject.fromint(self, val)
 
     def newlong_from_rbigint(self, val):
-        return W_LongObject(val)
+        return newlong(self, val)
 
     def newtuple(self, list_w):
         assert isinstance(list_w, list)

diff --git a/pypy/jit/metainterp/compile.py b/pypy/jit/metainterp/compile.py
--- a/pypy/jit/metainterp/compile.py
+++ b/pypy/jit/metainterp/compile.py
@@ -566,7 +566,7 @@
         pass
 
 
-def compile_new_bridge(metainterp, old_loop_tokens, resumekey):
+def compile_new_bridge(metainterp, old_loop_tokens, resumekey, retraced=False):
     """Try to compile a new bridge leading from the beginning of the history
     to some existing place.
     """
@@ -587,8 +587,9 @@
         inline_short_preamble = True
     try:
         target_loop_token = state.optimize_bridge(metainterp_sd,
-                                                  old_loop_tokens,
-                                                  new_loop, inline_short_preamble)
+                                                  old_loop_tokens, new_loop,
+                                                  inline_short_preamble,
+                                                  retraced)
     except InvalidLoop:
         # XXX I am fairly convinced that optimize_bridge cannot actually raise
         # InvalidLoop

diff --git a/pypy/doc/config/objspace.usemodules.readline.txt b/pypy/doc/config/objspace.usemodules.readline.txt
deleted file mode 100644
--- a/pypy/doc/config/objspace.usemodules.readline.txt
+++ /dev/null
@@ -1,1 +0,0 @@
-Use the 'readline' module. 

diff --git a/lib-python/conftest.py b/lib-python/conftest.py
--- a/lib-python/conftest.py
+++ b/lib-python/conftest.py
@@ -519,8 +519,8 @@
     RegrTest('test_with.py'),
     RegrTest('test_wsgiref.py'),
     RegrTest('test_xdrlib.py'),
-    RegrTest('test_xml_etree.py', skip="unsupported ext module"),
-    RegrTest('test_xml_etree_c.py', skip="unsupported ext module"),
+    RegrTest('test_xml_etree.py'),
+    RegrTest('test_xml_etree_c.py'),
     RegrTest('test_zipfile64.py'),
 ]
 

diff --git a/pypy/module/readline/interp_readline.py b/pypy/module/readline/interp_readline.py
deleted file mode 100644
--- a/pypy/module/readline/interp_readline.py
+++ /dev/null
@@ -1,23 +0,0 @@
-# this is a sketch of how one might one day be able to define a pretty simple
-# ctypes-using module, suitable for feeding to the ext-compiler
-
-from pypy.interpreter.baseobjspace import ObjSpace
-
-from pypy.module.readline import c_readline
-from pypy.rpython.lltypesystem import rffi
-
-#------------------------------------------------------------
-# exported API  (see interpleveldefs in __init__.py) 
-#
-def readline(space, prompt):
-    return space.wrap(rffi.charp2str(c_readline.c_readline(prompt)))
-readline.unwrap_spec = [ObjSpace, str]
-
-def setcompleter(space, w_callback):
-    """Set or remove the completer function.
-    The function is called as function(text, state),
-    for state in 0, 1, 2, ..., until it returns a non-string.
-    It should return the next possible completion starting with 'text'.
-    """ 
-    # XXX set internal completion function 
-    

diff --git a/pypy/config/pypyoption.py b/pypy/config/pypyoption.py
--- a/pypy/config/pypyoption.py
+++ b/pypy/config/pypyoption.py
@@ -27,7 +27,7 @@
 # --allworkingmodules
 working_modules = default_modules.copy()
 working_modules.update(dict.fromkeys(
-    ["_socket", "unicodedata", "mmap", "fcntl",
+    ["_socket", "unicodedata", "mmap", "fcntl", "_locale",
      "rctime" , "select", "zipimport", "_lsprof",
      "crypt", "signal", "_rawffi", "termios", "zlib", "bz2",
      "struct", "_hashlib", "_md5", "_sha", "_minimal_curses", "cStringIO",

From commits-noreply at bitbucket.org  Wed Feb 16 11:26:53 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Wed, 16 Feb 2011 11:26:53 +0100 (CET)
Subject: [pypy-svn] pypy arm-backend-2: fix interface for gen_load_int
Message-ID: <20110216102653.600FA282BE8@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backend-2
Changeset: r42028:47b47250e553
Date: 2011-02-13 14:54 +0100
http://bitbucket.org/pypy/pypy/changeset/47b47250e553/

Log:	fix interface for gen_load_int

diff --git a/pypy/jit/backend/arm/assembler.py b/pypy/jit/backend/arm/assembler.py
--- a/pypy/jit/backend/arm/assembler.py
+++ b/pypy/jit/backend/arm/assembler.py
@@ -17,6 +17,8 @@
 from pypy.rpython.annlowlevel import llhelper
 from pypy.rpython.lltypesystem import lltype, rffi, llmemory
 from pypy.jit.backend.arm.opassembler import ResOpAssembler
+from pypy.rlib.debug import (debug_print, debug_start, debug_stop,
+                             have_debug_prints)
 
 # XXX Move to llsupport
 from pypy.jit.backend.x86.support import values_array
@@ -229,7 +231,7 @@
         # 1 separator byte
         # 4 bytes for the faildescr
         memsize = (len(arglocs)-1)*6+5
-        datablockwrapper = MachineDataBlockWrapper(self.cpu.asmmemmgr, 
+        datablockwrapper = MachineDataBlockWrapper(self.cpu.asmmemmgr,
                                                     self.blocks)
         memaddr = datablockwrapper.malloc_aligned(memsize, alignment=WORD)
         datablockwrapper.done()
@@ -336,8 +338,15 @@
         self.mc.B_offs(loop_head)
         self._patch_sp_offset(sp_patch_location, regalloc)
 
+    def _dump(self, ops, type='loop'):
+        debug_start('jit-backend-ops')
+        debug_print(type)
+        for op in ops:
+            debug_print(op.repr())
+        debug_stop('jit-backend-ops')
     # cpu interface
     def assemble_loop(self, inputargs, operations, looptoken, log):
+        self._dump(operations)
         self.setup()
         longevity = compute_vars_longevity(inputargs, operations)
         regalloc = ARMRegisterManager(longevity, assembler=self, frame_manager=ARMFrameManager())
@@ -376,6 +385,7 @@
 
     def assemble_bridge(self, faildescr, inputargs, operations,
                                                     original_loop_token, log):
+        self._dump(operations, 'bridge')
         self.setup()
         assert isinstance(faildescr, AbstractFailDescr)
         code = faildescr._failure_recovery_code

diff --git a/pypy/jit/backend/arm/codebuilder.py b/pypy/jit/backend/arm/codebuilder.py
--- a/pypy/jit/backend/arm/codebuilder.py
+++ b/pypy/jit/backend/arm/codebuilder.py
@@ -129,11 +129,12 @@
         raise NotImplementedError
 
     size_of_gen_load_int = 3 * WORD
-    def gen_load_int(self, r, value, c=cond.AL):
+    def gen_load_int(self, r, value, cond=cond.AL):
         """r is the register number, value is the value to be loaded to the
         register"""
-        if c != cond.AL or 0 <= value <= 0xFFFF:
-            self._load_by_shifting(r, value, c)
+        from pypy.jit.backend.arm.conditions import AL
+        if cond != AL or 0 <= value <= 0xFFFF:
+            self._load_by_shifting(r, value, cond)
         else:
             self.LDR_ri(r, reg.pc.value)
             self.MOV_rr(reg.pc.value, reg.pc.value)

From commits-noreply at bitbucket.org  Wed Feb 16 11:26:53 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Wed, 16 Feb 2011 11:26:53 +0100 (CET)
Subject: [pypy-svn] pypy arm-backend-2: Pass cond flag to regalloc_mov and
	use it instead	of directly emiting operations.
Message-ID: <20110216102653.D6719282BE8@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backend-2
Changeset: r42029:e785d2f25336
Date: 2011-02-15 17:27 +0100
http://bitbucket.org/pypy/pypy/changeset/e785d2f25336/

Log:	Pass cond flag to regalloc_mov and use it instead of directly
	emiting operations.

diff --git a/pypy/jit/backend/arm/assembler.py b/pypy/jit/backend/arm/assembler.py
--- a/pypy/jit/backend/arm/assembler.py
+++ b/pypy/jit/backend/arm/assembler.py
@@ -542,23 +542,23 @@
         assert value.is_imm()
         self.mc.gen_load_int(loc.value, value.getint())
 
-    def regalloc_mov(self, prev_loc, loc):
+    def regalloc_mov(self, prev_loc, loc, cond=c.AL):
         if prev_loc.is_imm():
             if _check_imm_arg(ConstInt(prev_loc.getint())):
-                self.mc.MOV_ri(loc.value, prev_loc.getint())
+                self.mc.MOV_ri(loc.value, prev_loc.getint(), cond=cond)
             else:
-                self.mc.gen_load_int(loc.value, prev_loc.getint())
+                self.mc.gen_load_int(loc.value, prev_loc.getint(), cond=cond)
         elif loc.is_stack():
-            self.mc.STR_ri(prev_loc.value, r.fp.value, loc.position*-WORD)
+            self.mc.STR_ri(prev_loc.value, r.fp.value, loc.position*-WORD, cond=cond)
         elif prev_loc.is_stack():
-            self.mc.LDR_ri(loc.value, r.fp.value, prev_loc.position*-WORD)
+            self.mc.LDR_ri(loc.value, r.fp.value, prev_loc.position*-WORD, cond=cond)
         else:
-            self.mc.MOV_rr(loc.value, prev_loc.value)
+            self.mc.MOV_rr(loc.value, prev_loc.value, cond=cond)
     mov_loc_loc = regalloc_mov
 
     def regalloc_push(self, loc):
         if loc.is_stack():
-            self.mc.LDR_ri(r.ip.value, r.fp.value, loc.position*-WORD)
+            self.regalloc_mov(loc, r.ip)
             self.mc.PUSH([r.ip.value])
         elif loc.is_reg():
             self.mc.PUSH([loc.value])
@@ -568,7 +568,7 @@
     def regalloc_pop(self, loc):
         if loc.is_stack():
             self.mc.POP([r.ip.value])
-            self.mc.STR_ri(r.ip.value, r.fp.value, loc.position*-WORD)
+            self.regalloc_mov(r.ip, loc)
         elif loc.is_reg():
             self.mc.POP([loc.value])
         else:

From commits-noreply at bitbucket.org  Wed Feb 16 11:26:56 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Wed, 16 Feb 2011 11:26:56 +0100 (CET)
Subject: [pypy-svn] pypy arm-backend-2: Reset vable_token in call_assembler
	and some	other fixes to it
Message-ID: <20110216102656.257732A202C@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backend-2
Changeset: r42030:a273d7c8af2b
Date: 2011-02-15 17:29 +0100
http://bitbucket.org/pypy/pypy/changeset/a273d7c8af2b/

Log:	Reset vable_token in call_assembler and some other fixes to it

diff --git a/pypy/jit/backend/arm/opassembler.py b/pypy/jit/backend/arm/opassembler.py
--- a/pypy/jit/backend/arm/opassembler.py
+++ b/pypy/jit/backend/arm/opassembler.py
@@ -647,11 +647,16 @@
     # from: ../x86/assembler.py:1668
     # XXX Split into some helper methods
     def emit_guard_call_assembler(self, op, guard_op, arglocs, regalloc, fcond):
+        faildescr = guard_op.getdescr()
+        fail_index = self.cpu.get_fail_descr_number(faildescr)
+        self._write_fail_index(fail_index)
+
         descr = op.getdescr()
         assert isinstance(descr, LoopToken)
+        assert op.numargs() == len(descr._arm_arglocs)
         resbox = TempBox()
         self._emit_call(descr._arm_direct_bootstrap_code, op.getarglist(),
-                                regalloc, fcond, result=resbox, spill_all_regs=True)
+                                regalloc, fcond, result=resbox)
         if op.result is None:
             value = self.cpu.done_with_this_frame_void_v
         else:
@@ -664,30 +669,35 @@
                 value = self.cpu.done_with_this_frame_float_v
             else:
                 raise AssertionError(kind)
-        assert value <= 0xff
-
         # check value
         resloc = regalloc.force_allocate_reg(resbox)
+        assert resloc is r.r0
         self.mc.gen_load_int(r.ip.value, value)
         self.mc.CMP_rr(resloc.value, r.ip.value)
 
+
         fast_jmp_pos = self.mc.currpos()
-        #fast_jmp_location = self.mc.curraddr()
         self.mc.NOP()
 
-        #if values are equal we take the fast pat
+        # Path A: use assembler helper
+        #if values are equal we take the fast path
         # Slow path, calling helper
         # jump to merge point
         jd = descr.outermost_jitdriver_sd
         assert jd is not None
         asm_helper_adr = self.cpu.cast_adr_to_int(jd.assembler_helper_adr)
-        self._emit_call(asm_helper_adr, [resbox, op.getarg(0)], regalloc, fcond, op.result)
-        regalloc.possibly_free_var(resbox)
+        self.mc.PUSH([reg.value for reg in r.caller_resp][1:])
+        # resbox is allready in r0
+        self.mov_loc_loc(arglocs[1], r.r1)
+        self.mc.BL(asm_helper_adr)
+        self.mc.POP([reg.value for reg in r.caller_resp][1:])
+        regalloc.after_call(op.result)
         # jump to merge point
         jmp_pos = self.mc.currpos()
         #jmp_location = self.mc.curraddr()
         self.mc.NOP()
 
+        # Path B: load return value and reset token
         # Fast Path using result boxes
         # patch the jump to the fast path
         offset = self.mc.currpos() - fast_jmp_pos
@@ -696,20 +706,15 @@
         pmc.ADD_ri(r.pc.value, r.pc.value, offset - PC_OFFSET, cond=c.EQ)
 
         # Reset the vable token --- XXX really too much special logic here:-(
-        # XXX Enable and fix this once the stange errors procuded by its
-        # presence are fixed
-        #if jd.index_of_virtualizable >= 0:
-        #    from pypy.jit.backend.llsupport.descr import BaseFieldDescr
-        #    size = jd.portal_calldescr.get_result_size(self.cpu.translate_support_code)
-        #    vable_index = jd.index_of_virtualizable
-        #    regalloc._sync_var(op.getarg(vable_index))
-        #    vable = regalloc.frame_manager.loc(op.getarg(vable_index))
-        #    fielddescr = jd.vable_token_descr
-        #    assert isinstance(fielddescr, BaseFieldDescr)
-        #    ofs = fielddescr.offset
-        #    self.mc.MOV(eax, arglocs[1])
-        #    self.mc.MOV_mi((eax.value, ofs), 0)
-        #    # in the line above, TOKEN_NONE = 0
+        if jd.index_of_virtualizable >= 0:
+            from pypy.jit.backend.llsupport.descr import BaseFieldDescr
+            fielddescr = jd.vable_token_descr
+            assert isinstance(fielddescr, BaseFieldDescr)
+            ofs = fielddescr.offset
+            self.mov_loc_loc(arglocs[1], r.ip, cond=c.MI)
+            self.mc.MOV_ri(resloc.value, 0, cond=c.MI)
+            self.mc.STR_ri(resloc.value, r.ip.value, ofs*WORD, cond=c.MI)
+        regalloc.possibly_free_var(resbox)
 
         if op.result is not None:
             # load the return value from fail_boxes_xxx[0]
@@ -737,6 +742,7 @@
             regalloc.possibly_free_var(op.result)
         return fcond
 
+
     # ../x86/assembler.py:668
     def redirect_call_assembler(self, oldlooptoken, newlooptoken):
         # we overwrite the instructions at the old _x86_direct_bootstrap_code

diff --git a/pypy/jit/backend/arm/test/test_runner.py b/pypy/jit/backend/arm/test/test_runner.py
--- a/pypy/jit/backend/arm/test/test_runner.py
+++ b/pypy/jit/backend/arm/test/test_runner.py
@@ -71,7 +71,8 @@
             _assembler_helper_ptr = llhelper(FUNCPTR, assembler_helper)
             assembler_helper_adr = llmemory.cast_ptr_to_adr(
                 _assembler_helper_ptr)
-
+        FakeJitDriverSD.portal_calldescr = self.cpu.calldescrof(
+            lltype.Ptr(lltype.FuncType([lltype.Signed], lltype.Signed)), [lltype.Signed], lltype.Signed)
         lt1, lt2, lt3 = [LoopToken() for x in range(3)]
         lt2.outermost_jitdriver_sd = FakeJitDriverSD()
         loop1 = parse('''

diff --git a/pypy/jit/backend/arm/regalloc.py b/pypy/jit/backend/arm/regalloc.py
--- a/pypy/jit/backend/arm/regalloc.py
+++ b/pypy/jit/backend/arm/regalloc.py
@@ -675,10 +675,18 @@
         return self._prepare_guard(guard_op)
 
     def prepare_guard_call_assembler(self, op, guard_op, fcond):
-        faildescr = guard_op.getdescr()
-        fail_index = self.cpu.get_fail_descr_number(faildescr)
-        self.assembler._write_fail_index(fail_index)
-        return []
+        descr = op.getdescr()
+        assert isinstance(descr, LoopToken)
+        jd = descr.outermost_jitdriver_sd
+        assert jd is not None
+        size = jd.portal_calldescr.get_result_size(self.cpu.translate_support_code)
+        vable_index = jd.index_of_virtualizable
+        if vable_index >= 0:
+            self._sync_var(op.getarg(vable_index))
+            vable = self.frame_manager.loc(op.getarg(vable_index))
+        else:
+            vable = imm(0)
+        return [imm(size), vable]
 
     def _prepare_args_for_new_op(self, new_args):
         gc_ll_descr = self.cpu.gc_ll_descr

From commits-noreply at bitbucket.org  Wed Feb 16 11:27:11 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Wed, 16 Feb 2011 11:27:11 +0100 (CET)
Subject: [pypy-svn] pypy arm-backend-2: merge default
Message-ID: <20110216102711.9DD8B282BE8@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backend-2
Changeset: r42031:0aaf135fc111
Date: 2011-02-15 19:00 +0100
http://bitbucket.org/pypy/pypy/changeset/0aaf135fc111/

Log:	merge default

diff --git a/pypy/jit/backend/x86/test/test_runner.py b/pypy/jit/backend/x86/test/test_runner.py
--- a/pypy/jit/backend/x86/test/test_runner.py
+++ b/pypy/jit/backend/x86/test/test_runner.py
@@ -390,18 +390,6 @@
         res = self.cpu.get_latest_value_int(0)
         assert res == 20
 
-    def test_call_with_const_floats(self):
-        def func(f1, f2):
-            return f1 + f2
-
-        FUNC = self.FuncType([lltype.Float, lltype.Float], lltype.Float)
-        FPTR = self.Ptr(FUNC)
-        calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
-        func_ptr = llhelper(FPTR, func)
-        funcbox = self.get_funcbox(self.cpu, func_ptr)
-        res = self.execute_operation(rop.CALL, [funcbox, ConstFloat(1.5), ConstFloat(2.5)], 'float', descr=calldescr)
-        assert res.value == 4.0
-
 
 class TestDebuggingAssembler(object):
     def setup_method(self, meth):

diff --git a/pypy/objspace/std/mapdict.py b/pypy/objspace/std/mapdict.py
--- a/pypy/objspace/std/mapdict.py
+++ b/pypy/objspace/std/mapdict.py
@@ -1,3 +1,4 @@
+import weakref
 from pypy.rlib import jit, objectmodel, debug
 from pypy.rlib.rarithmetic import intmask, r_uint
 
@@ -672,7 +673,6 @@
 # Magic caching
 
 class CacheEntry(object):
-    map = None
     version_tag = None
     index = 0
     w_method = None # for callmethod
@@ -683,8 +683,10 @@
         map = w_obj._get_mapdict_map()
         return self.is_valid_for_map(map)
 
+    @jit.dont_look_inside
     def is_valid_for_map(self, map):
-        if map is self.map:
+        mymap = self.map_wref()
+        if mymap is map:     # also handles the case self.map_wref()->None
             version_tag = map.terminator.w_cls.version_tag()
             if version_tag is self.version_tag:
                 # everything matches, it's incredibly fast
@@ -693,22 +695,23 @@
                 return True
         return False
 
+_invalid_cache_entry_map = objectmodel.instantiate(AbstractAttribute)
+_invalid_cache_entry_map.terminator = None
 INVALID_CACHE_ENTRY = CacheEntry()
-INVALID_CACHE_ENTRY.map = objectmodel.instantiate(AbstractAttribute)
-                             # different from any real map ^^^
-INVALID_CACHE_ENTRY.map.terminator = None
-
+INVALID_CACHE_ENTRY.map_wref = weakref.ref(_invalid_cache_entry_map)
+                                 # different from any real map ^^^
 
 def init_mapdict_cache(pycode):
     num_entries = len(pycode.co_names_w)
     pycode._mapdict_caches = [INVALID_CACHE_ENTRY] * num_entries
 
+ at jit.dont_look_inside
 def _fill_cache(pycode, nameindex, map, version_tag, index, w_method=None):
     entry = pycode._mapdict_caches[nameindex]
     if entry is INVALID_CACHE_ENTRY:
         entry = CacheEntry()
         pycode._mapdict_caches[nameindex] = entry
-    entry.map = map
+    entry.map_wref = weakref.ref(map)
     entry.version_tag = version_tag
     entry.index = index
     entry.w_method = w_method

diff --git a/pypy/translator/goal/app_main.py b/pypy/translator/goal/app_main.py
--- a/pypy/translator/goal/app_main.py
+++ b/pypy/translator/goal/app_main.py
@@ -404,6 +404,9 @@
     # (relevant in case of "reload(sys)")
     sys.argv[:] = argv
 
+    if (PYTHON26 and not options["ignore_environment"] and os.getenv('PYTHONNOUSERSITE')):
+        options["no_user_site"] = True
+
     if (options["interactive"] or
         (not options["ignore_environment"] and os.getenv('PYTHONINSPECT'))):
         options["inspect"] = True
@@ -504,10 +507,8 @@
             success = run_toplevel(run_it)
         elif run_module:
             # handle the "-m" command
-            def run_it():
-                import runpy
-                runpy._run_module_as_main(sys.argv[0])
-            success = run_toplevel(run_it)
+            import runpy
+            success = run_toplevel(runpy._run_module_as_main, sys.argv[0])
         elif run_stdin:
             # handle the case where no command/filename/module is specified
             # on the command-line.
@@ -550,10 +551,32 @@
         else:
             # handle the common case where a filename is specified
             # on the command-line.
-            mainmodule.__file__ = sys.argv[0]
-            scriptdir = resolvedirof(sys.argv[0])
-            sys.path.insert(0, scriptdir)
-            success = run_toplevel(execfile, sys.argv[0], mainmodule.__dict__)
+            filename = sys.argv[0]
+            mainmodule.__file__ = filename
+            sys.path.insert(0, resolvedirof(filename))
+            # assume it's a pyc file only if its name says so.
+            # CPython goes to great lengths to detect other cases
+            # of pyc file format, but I think it's ok not to care.
+            import imp
+            if IS_WINDOWS:
+                filename = filename.lower()
+            if filename.endswith('.pyc') or filename.endswith('.pyo'):
+                args = (imp._run_compiled_module, '__main__',
+                        sys.argv[0], None, mainmodule)
+            else:
+                # maybe it's the name of a directory or a zip file
+                filename = sys.argv[0]
+                importer = imp._getimporter(filename)
+                if not isinstance(importer, imp.NullImporter):
+                    # yes.  put the filename in sys.path[0] and import
+                    # the module __main__
+                    import runpy
+                    sys.path.insert(0, filename)
+                    args = (runpy._run_module_as_main, '__main__')
+                else:
+                    # no.  That's the normal path, "pypy stuff.py".
+                    args = (execfile, filename, mainmodule.__dict__)
+            success = run_toplevel(*args)
 
     except SystemExit, e:
         status = e.code
@@ -623,6 +646,27 @@
         except OSError:
             return None
 
+    # add an emulator for these pypy-only or 2.7-only functions
+    # (for test_pyc_commandline_argument)
+    import imp, runpy
+    def _run_compiled_module(modulename, filename, file, module):
+        import os
+        assert modulename == '__main__'
+        assert os.path.isfile(filename)
+        assert filename.endswith('.pyc')
+        assert file is None
+        assert module.__name__ == '__main__'
+        print 'in _run_compiled_module'
+    def _getimporter(path):
+        import os, imp
+        if os.path.isdir(path):
+            return None
+        else:
+            return imp.NullImporter(path)
+
+    imp._run_compiled_module = _run_compiled_module
+    imp._getimporter = _getimporter
+
     # stick the current sys.path into $PYTHONPATH, so that CPython still
     # finds its own extension modules :-/
     import os

diff --git a/pypy/translator/goal/test2/test_app_main.py b/pypy/translator/goal/test2/test_app_main.py
--- a/pypy/translator/goal/test2/test_app_main.py
+++ b/pypy/translator/goal/test2/test_app_main.py
@@ -3,7 +3,7 @@
 """
 from __future__ import with_statement
 import py
-import sys, os, re
+import sys, os, re, runpy
 import autopath
 from pypy.tool.udir import udir
 from contextlib import contextmanager
@@ -14,14 +14,43 @@
 app_main = os.path.abspath(app_main)
 
 _counter = 0
+def _get_next_path(ext='.py'):
+    global _counter
+    p = udir.join('demo_test_app_main_%d%s' % (_counter, ext))
+    _counter += 1
+    return p
+
 def getscript(source):
-    global _counter
-    p = udir.join('demo_test_app_main_%d.py' % (_counter,))
-    _counter += 1
+    p = _get_next_path()
     p.write(str(py.code.Source(source)))
     # return relative path for testing purposes 
     return py.path.local().bestrelpath(p) 
 
+def getscript_pyc(space, source):
+    p = _get_next_path()
+    p.write(str(py.code.Source(source)))
+    w_dir = space.wrap(str(p.dirpath()))
+    w_modname = space.wrap(p.purebasename)
+    space.appexec([w_dir, w_modname], """(dir, modname):
+        import sys
+        d = sys.modules.copy()
+        sys.path.insert(0, dir)
+        __import__(modname)
+        sys.path.pop(0)
+        for key in sys.modules.keys():
+            if key not in d:
+                del sys.modules[key]
+    """)
+    p = str(p) + 'c'
+    assert os.path.isfile(p)   # the .pyc file should have been created above
+    return p
+
+def getscript_in_dir(source):
+    pdir = _get_next_path(ext='')
+    p = pdir.ensure(dir=1).join('__main__.py')
+    p.write(str(py.code.Source(source)))
+    # return relative path for testing purposes 
+    return py.path.local().bestrelpath(pdir)
 
 demo_script = getscript("""
     print 'hello'
@@ -159,6 +188,11 @@
             self.check([opt, '-c', 'pass'], sys_argv=['-c'],
                        run_command='pass', **expected)
 
+    def test_sysflags_envvar(self, monkeypatch):
+        monkeypatch.setenv('PYTHONNOUSERSITE', '1')
+        expected = {"no_user_site": True}
+        self.check(['-c', 'pass'], sys_argv=['-c'], run_command='pass', **expected)
+        
 
 class TestInteraction:
     """
@@ -381,6 +415,8 @@
     def test_options_i_m(self):
         if sys.platform == "win32":
             skip("close_fds is not supported on Windows platforms")
+        if not hasattr(runpy, '_run_module_as_main'):
+            skip("requires CPython >= 2.6")
         p = os.path.join(autopath.this_dir, 'mymodule.py')
         p = os.path.abspath(p)
         child = self.spawn(['-i',
@@ -481,6 +517,8 @@
         child.expect('A five ounce bird could not carry a one pound coconut.')
 
     def test_no_space_before_argument(self):
+        if not hasattr(runpy, '_run_module_as_main'):
+            skip("requires CPython >= 2.6")
         child = self.spawn(['-cprint "hel" + "lo"'])
         child.expect('hello')
 
@@ -578,6 +616,8 @@
             os.environ['PYTHONWARNINGS'] = old
 
     def test_option_m(self):
+        if not hasattr(runpy, '_run_module_as_main'):
+            skip("requires CPython >= 2.6")
         p = os.path.join(autopath.this_dir, 'mymodule.py')
         p = os.path.abspath(p)
         data = self.run('-m pypy.translator.goal.test2.mymodule extra')
@@ -671,6 +711,21 @@
         data = self.run('-c "import sys; print sys.path"')
         assert data.startswith("[''")
 
+    def test_pyc_commandline_argument(self):
+        p = getscript_pyc(self.space, "print 6*7\n")
+        assert os.path.isfile(p) and p.endswith('.pyc')
+        data = self.run(p)
+        assert data == 'in _run_compiled_module\n'
+
+    def test_main_in_dir_commandline_argument(self):
+        if not hasattr(runpy, '_run_module_as_main'):
+            skip("requires CPython >= 2.6")
+        p = getscript_in_dir('print 6*7\n')
+        data = self.run(p)
+        assert data == '42\n'
+        data = self.run(p + os.sep)
+        assert data == '42\n'
+
 
 class AppTestAppMain:
 

diff --git a/pypy/objspace/std/test/test_mapdict.py b/pypy/objspace/std/test/test_mapdict.py
--- a/pypy/objspace/std/test/test_mapdict.py
+++ b/pypy/objspace/std/test/test_mapdict.py
@@ -866,6 +866,38 @@
         res = self.check(f, 'm')
         assert res == (0, 2, 1)
 
+    def test_dont_keep_class_alive(self):
+        import weakref
+        import gc
+        def f():
+            class C(object):
+                def m(self):
+                    pass
+            r = weakref.ref(C)
+            # Trigger cache.
+            C().m()
+            del C
+            gc.collect(); gc.collect(); gc.collect()
+            assert r() is None
+            return 42
+        f()
+
+    def test_instance_keeps_class_alive(self):
+        import weakref
+        import gc
+        def f():
+            class C(object):
+                def m(self):
+                    return 42
+            r = weakref.ref(C)
+            c = C()
+            del C
+            gc.collect(); gc.collect(); gc.collect()
+            return c.m()
+        val = f()
+        assert val == 42
+        f() 
+
 class AppTestGlobalCaching(AppTestWithMapDict):
     def setup_class(cls):
         cls.space = gettestobjspace(

diff --git a/pypy/jit/codewriter/jtransform.py b/pypy/jit/codewriter/jtransform.py
--- a/pypy/jit/codewriter/jtransform.py
+++ b/pypy/jit/codewriter/jtransform.py
@@ -5,7 +5,7 @@
 from pypy.objspace.flow.model import SpaceOperation, Variable, Constant
 from pypy.objspace.flow.model import Block, Link, c_last_exception
 from pypy.jit.codewriter.flatten import ListOfKind, IndirectCallTargets
-from pypy.jit.codewriter import support, heaptracker
+from pypy.jit.codewriter import support, heaptracker, longlong
 from pypy.jit.codewriter.effectinfo import EffectInfo
 from pypy.jit.codewriter.policy import log
 from pypy.jit.metainterp.typesystem import deref, arrayItem
@@ -39,7 +39,6 @@
     def optimize_block(self, block):
         if block.operations == ():
             return
-        self.remove_longlong_constants(block)
         self.vable_array_vars = {}
         self.vable_flags = {}
         renamings = {}
@@ -135,55 +134,6 @@
         block.exits = block.exits[:1]
         block.exitswitch = None
 
-    def remove_longlong_constants(self, block):
-        # remove all Constant({Un}signedLongLong), and replace them with
-        # cast_int_to_longlong(Constant(Signed)) or
-        # two_ints_to_longlong(Constant(Signed), Constant(Signed)).
-        operations = []
-        all_constants = {}
-        #
-        def _get_const_as_var(c):
-            v = all_constants.get(c)
-            if v is None:
-                from pypy.rlib.rarithmetic import intmask
-                v = varoftype(c.concretetype)
-                value = int(c.value)
-                c_hi = Constant(intmask(value >> 32), lltype.Signed)
-                c_lo = Constant(intmask(value), lltype.Signed)
-                if c_lo.value == value:
-                    # a long long constant, but it fits in 32 bits
-                    op1 = SpaceOperation('cast_int_to_longlong', [c_lo], v)
-                else:
-                    # a 64-bit long long constant, requires two ints
-                    op1 = SpaceOperation('two_ints_to_longlong', [c_lo, c_hi],
-                                         v)
-                operations.append(op1)
-                all_constants[c] = v
-            return v
-        #
-        for op in block.operations:
-            for i, v in enumerate(op.args):
-                if (isinstance(v, Constant) and
-                        self._is_longlong(v.concretetype)):
-                    args = op.args[:]
-                    args[i] = _get_const_as_var(v)
-                    op = SpaceOperation(op.opname, args, op.result)
-            operations.append(op)
-        #
-        last_op = None
-        if block.exitswitch == c_last_exception:
-            last_op = operations.pop()
-        for link in block.exits:
-            for i, v in enumerate(link.args):
-                if (isinstance(v, Constant) and
-                        self._is_longlong(v.concretetype)):
-                    args = link.args[:]
-                    args[i] = _get_const_as_var(v)
-                    link.args = args
-        if last_op is not None:
-            operations.append(last_op)
-        block.operations = operations
-
     # ----------
 
     def follow_constant_exit(self, block):
@@ -841,17 +791,6 @@
     # and unsupported ones are turned into a call to a function from
     # jit.codewriter.support.
 
-    if lltype.SignedLongLong != lltype.Signed:
-        @staticmethod
-        def _is_longlong(TYPE):
-            return (TYPE == lltype.SignedLongLong or
-                    TYPE == lltype.UnsignedLongLong)
-    else:
-        # on 64-bit, _is_longlong() returns always False
-        @staticmethod
-        def _is_longlong(TYPE):
-            return False
-
     for _op, _oopspec in [('llong_invert',  'INVERT'),
                           ('ullong_invert', 'INVERT'),
                           ('llong_lt',      'LT'),
@@ -886,8 +825,7 @@
                           ('truncate_longlong_to_int', 'TO_INT'),
                           ('cast_float_to_longlong',   'FROM_FLOAT'),
                           ('cast_longlong_to_float',   'TO_FLOAT'),
-                          # internal pseuso-operation:
-                          ('two_ints_to_longlong',     'FROM_TWO_INTS'),
+                          ('cast_uint_to_longlong',    'FROM_UINT'),
                           ]:
         exec py.code.Source('''
             def rewrite_op_%s(self, op):
@@ -929,8 +867,8 @@
     rewrite_op_ullong_is_true = rewrite_op_llong_is_true
 
     def rewrite_op_cast_primitive(self, op):
-        fromll = self._is_longlong(op.args[0].concretetype)
-        toll   = self._is_longlong(op.result.concretetype)
+        fromll = longlong.is_longlong(op.args[0].concretetype)
+        toll   = longlong.is_longlong(op.result.concretetype)
         if fromll != toll:
             args = op.args
             if fromll:
@@ -940,9 +878,7 @@
                 if rffi.cast(op.args[0].concretetype, -1) < 0:
                     opname = 'cast_int_to_longlong'
                 else:
-                    opname = 'two_ints_to_longlong'
-                    c_hi = Constant(0, lltype.Signed)
-                    args = [args[0], c_hi]
+                    opname = 'cast_uint_to_longlong'
             op1 = SpaceOperation(opname, args, op.result)
             return self.rewrite_operation(op1)
 

diff --git a/pypy/jit/metainterp/optimizeopt/optimizer.py b/pypy/jit/metainterp/optimizeopt/optimizer.py
--- a/pypy/jit/metainterp/optimizeopt/optimizer.py
+++ b/pypy/jit/metainterp/optimizeopt/optimizer.py
@@ -162,7 +162,7 @@
 CONST_0      = ConstInt(0)
 CONST_1      = ConstInt(1)
 CVAL_ZERO    = ConstantValue(CONST_0)
-CVAL_ZERO_FLOAT = ConstantValue(ConstFloat(0.0))
+CVAL_ZERO_FLOAT = ConstantValue(Const._new(0.0))
 CVAL_UNINITIALIZED_ZERO = ConstantValue(CONST_0)
 llhelper.CVAL_NULLREF = ConstantValue(llhelper.CONST_NULL)
 oohelper.CVAL_NULLREF = ConstantValue(oohelper.CONST_NULL)

diff --git a/pypy/jit/metainterp/test/test_basic.py b/pypy/jit/metainterp/test/test_basic.py
--- a/pypy/jit/metainterp/test/test_basic.py
+++ b/pypy/jit/metainterp/test/test_basic.py
@@ -9,6 +9,7 @@
 from pypy.jit.metainterp import pyjitpl, history
 from pypy.jit.metainterp.warmstate import set_future_value
 from pypy.jit.codewriter.policy import JitPolicy, StopAtXPolicy
+from pypy.jit.codewriter import longlong
 from pypy import conftest
 from pypy.rlib.rarithmetic import ovfcheck
 from pypy.jit.metainterp.typesystem import LLTypeHelper, OOTypeHelper
@@ -86,6 +87,7 @@
             blackholeinterp.setarg_r(count_r, value)
             count_r += 1
         elif T == lltype.Float:
+            value = longlong.getfloatstorage(value)
             blackholeinterp.setarg_f(count_f, value)
             count_f += 1
         else:
@@ -192,6 +194,10 @@
         # try to run it by running the code compiled just before
         result3 = _run_with_machine_code(self, args)
         assert result1 == result3 or result3 == NotImplemented
+        #
+        if (longlong.supports_longlong and
+            isinstance(result1, longlong.r_float_storage)):
+            result1 = longlong.getrealfloat(result1)
         return result1
 
     def check_history(self, expected=None, **isns):

diff --git a/pypy/jit/backend/llgraph/runner.py b/pypy/jit/backend/llgraph/runner.py
--- a/pypy/jit/backend/llgraph/runner.py
+++ b/pypy/jit/backend/llgraph/runner.py
@@ -15,7 +15,7 @@
 from pypy.jit.backend import model
 from pypy.jit.backend.llgraph import llimpl, symbolic
 from pypy.jit.metainterp.typesystem import llhelper, oohelper
-from pypy.jit.codewriter import heaptracker
+from pypy.jit.codewriter import heaptracker, longlong
 from pypy.rlib import rgc
 
 class MiniStats:
@@ -86,7 +86,7 @@
 
 class BaseCPU(model.AbstractCPU):
     supports_floats = True
-    supports_longlong = True
+    supports_longlong = llimpl.IS_32_BIT
 
     def __init__(self, rtyper, stats=None, opts=None,
                  translate_support_code=False,
@@ -305,8 +305,12 @@
         for ARG in ARGS:
             token = history.getkind(ARG)
             if token != 'void':
+                if token == 'float' and longlong.is_longlong(ARG):
+                    token = 'L'
                 arg_types.append(token[0])
         token = history.getkind(RESULT)
+        if token == 'float' and longlong.is_longlong(RESULT):
+            token = 'L'
         return self.getdescr(0, token[0], extrainfo=extrainfo,
                              arg_types=''.join(arg_types))
 
@@ -464,7 +468,7 @@
         self._prepare_call(REF, calldescr, args_i, args_r, args_f)
         return llimpl.do_call_ptr(func)
     def bh_call_f(self, func, calldescr, args_i, args_r, args_f):
-        self._prepare_call(FLOAT, calldescr, args_i, args_r, args_f)
+        self._prepare_call(FLOAT + 'L', calldescr, args_i, args_r, args_f)
         return llimpl.do_call_float(func)
     def bh_call_v(self, func, calldescr, args_i, args_r, args_f):
         self._prepare_call('v', calldescr, args_i, args_r, args_f)
@@ -472,7 +476,7 @@
 
     def _prepare_call(self, resulttypeinfo, calldescr, args_i, args_r, args_f):
         assert isinstance(calldescr, Descr)
-        assert calldescr.typeinfo == resulttypeinfo
+        assert calldescr.typeinfo in resulttypeinfo
         if args_i is not None:
             for x in args_i:
                 llimpl.do_call_pushint(x)

diff --git a/pypy/jit/backend/x86/regalloc.py b/pypy/jit/backend/x86/regalloc.py
--- a/pypy/jit/backend/x86/regalloc.py
+++ b/pypy/jit/backend/x86/regalloc.py
@@ -12,7 +12,7 @@
 from pypy.rlib import rgc
 from pypy.jit.backend.llsupport import symbolic
 from pypy.jit.backend.x86.jump import remap_frame_layout
-from pypy.jit.codewriter import heaptracker
+from pypy.jit.codewriter import heaptracker, longlong
 from pypy.jit.codewriter.effectinfo import EffectInfo
 from pypy.jit.metainterp.resoperation import rop
 from pypy.jit.backend.llsupport.descr import BaseFieldDescr, BaseArrayDescr
@@ -71,13 +71,16 @@
 
     def convert_to_imm(self, c):
         adr = self.assembler.datablockwrapper.malloc_aligned(8, 8)
-        rffi.cast(rffi.CArrayPtr(rffi.DOUBLE), adr)[0] = c.getfloat()
+        x = c.getfloatstorage()
+        rffi.cast(rffi.CArrayPtr(longlong.FLOATSTORAGE), adr)[0] = x
         return ConstFloatLoc(adr)
 
     def convert_to_imm_16bytes_align(self, c):
         adr = self.assembler.datablockwrapper.malloc_aligned(16, 16)
-        rffi.cast(rffi.CArrayPtr(rffi.DOUBLE), adr)[0] = c.getfloat()
-        rffi.cast(rffi.CArrayPtr(rffi.DOUBLE), adr)[1] = 0.0
+        x = c.getfloatstorage()
+        y = longlong.ZEROF
+        rffi.cast(rffi.CArrayPtr(longlong.FLOATSTORAGE), adr)[0] = x
+        rffi.cast(rffi.CArrayPtr(longlong.FLOATSTORAGE), adr)[1] = y
         return ConstFloatLoc(adr)
 
     def after_call(self, v):
@@ -221,7 +224,7 @@
                              selected_reg=None, need_lower_byte=False):
         if var.type == FLOAT:
             if isinstance(var, ConstFloat):
-                return FloatImmedLoc(var.getfloat())
+                return FloatImmedLoc(var.getfloatstorage())
             return self.xrm.make_sure_var_in_reg(var, forbidden_vars,
                                                  selected_reg, need_lower_byte)
         else:
@@ -630,11 +633,10 @@
 
     def _maybe_consider_llong_lt(self, op):
         # XXX just a special case for now
-        from pypy.rlib.longlong2float import longlong2float
         box = op.getarg(2)
         if not isinstance(box, ConstFloat):
             return False
-        if not (box.value == longlong2float(r_longlong(0))):
+        if box.getlonglong() != 0:
             return False
         # "x < 0"
         box = op.getarg(1)
@@ -653,8 +655,7 @@
         self.xrm.possibly_free_var(op.getarg(1))
 
     def _loc_of_const_longlong(self, value64):
-        from pypy.rlib.longlong2float import longlong2float
-        c = ConstFloat(longlong2float(value64))
+        c = ConstFloat(value64)
         return self.xrm.convert_to_imm(c)
 
     def _consider_llong_from_int(self, op):
@@ -676,36 +677,11 @@
         self.PerformLLong(op, [loc1, loc2], loc0)
         self.rm.possibly_free_var(box)
 
-    def _consider_llong_from_two_ints(self, op):
+    def _consider_llong_from_uint(self, op):
         assert IS_X86_32
-        box1 = op.getarg(1)
-        box2 = op.getarg(2)
         loc0 = self.xrm.force_allocate_reg(op.result)
-        #
-        if isinstance(box1, ConstInt) and isinstance(box2, ConstInt):
-            # all-constant arguments: load the result value in a single step
-            value64 = r_longlong(box2.value) << 32
-            value64 |= r_longlong(r_uint(box1.value))
-            loc1 = self._loc_of_const_longlong(value64)
-            loc2 = None    # unused
-            loc3 = None    # unused
-        #
-        else:
-            tmpxvar = TempBox()
-            loc3 = self.xrm.force_allocate_reg(tmpxvar, [op.result])
-            self.xrm.possibly_free_var(tmpxvar)
-            #
-            if isinstance(box1, ConstInt):
-                loc1 = self._loc_of_const_longlong(r_longlong(box1.value))
-            else:
-                loc1 = self.rm.make_sure_var_in_reg(box1)
-            #
-            if isinstance(box2, ConstInt):
-                loc2 = self._loc_of_const_longlong(r_longlong(box2.value))
-            else:
-                loc2 = self.rm.make_sure_var_in_reg(box2, [box1])
-        #
-        self.PerformLLong(op, [loc1, loc2, loc3], loc0)
+        loc1 = self.rm.make_sure_var_in_reg(op.getarg(1))
+        self.PerformLLong(op, [loc1], loc0)
         self.rm.possibly_free_vars_for_op(op)
 
     def _call(self, op, arglocs, force_store=[], guard_not_forced_op=None):
@@ -755,8 +731,8 @@
                     return self._consider_llong_to_int(op)
                 if oopspecindex == EffectInfo.OS_LLONG_FROM_INT:
                     return self._consider_llong_from_int(op)
-                if oopspecindex == EffectInfo.OS_LLONG_FROM_TWO_INTS:
-                    return self._consider_llong_from_two_ints(op)
+                if oopspecindex == EffectInfo.OS_LLONG_FROM_UINT:
+                    return self._consider_llong_from_uint(op)
                 if (oopspecindex == EffectInfo.OS_LLONG_EQ or
                     oopspecindex == EffectInfo.OS_LLONG_NE):
                     return self._consider_llong_eq_ne_xx(op)

diff --git a/pypy/module/imp/test/test_import.py b/pypy/module/imp/test/test_import.py
--- a/pypy/module/imp/test/test_import.py
+++ b/pypy/module/imp/test/test_import.py
@@ -975,6 +975,50 @@
             sys.meta_path.pop()
             sys.path_hooks.pop()
 
+
+class AppTestPyPyExtension(object):
+    def setup_class(cls):
+        cls.space = gettestobjspace(usemodules=['imp', 'zipimport'])
+        cls.w_udir = cls.space.wrap(str(udir))
+
+    def test_run_compiled_module(self):
+        # XXX minimal test only
+        import imp, new
+        module = new.module('foobar')
+        raises(IOError, imp._run_compiled_module,
+               'foobar', 'this_file_does_not_exist', None, module)
+
+    def test_getimporter(self):
+        import imp, os
+        # an existing directory
+        importer = imp._getimporter(self.udir)
+        assert importer is None
+        # an existing file
+        path = os.path.join(self.udir, 'test_getimporter')
+        open(path, 'w').close()
+        importer = imp._getimporter(path)
+        assert isinstance(importer, imp.NullImporter)
+        # a non-existing path
+        path = os.path.join(self.udir, 'does_not_exist_at_all')
+        importer = imp._getimporter(path)
+        assert isinstance(importer, imp.NullImporter)
+        # a mostly-empty zip file
+        path = os.path.join(self.udir, 'test_getimporter.zip')
+        f = open(path, 'wb')
+        f.write('PK\x03\x04\n\x00\x00\x00\x00\x00P\x9eN>\x00\x00\x00\x00\x00'
+                '\x00\x00\x00\x00\x00\x00\x00\x05\x00\x15\x00emptyUT\t\x00'
+                '\x03wyYMwyYMUx\x04\x00\xf4\x01d\x00PK\x01\x02\x17\x03\n\x00'
+                '\x00\x00\x00\x00P\x9eN>\x00\x00\x00\x00\x00\x00\x00\x00\x00'
+                '\x00\x00\x00\x05\x00\r\x00\x00\x00\x00\x00\x00\x00\x00\x00'
+                '\xa4\x81\x00\x00\x00\x00emptyUT\x05\x00\x03wyYMUx\x00\x00PK'
+                '\x05\x06\x00\x00\x00\x00\x01\x00\x01\x00@\x00\x00\x008\x00'
+                '\x00\x00\x00\x00')
+        f.close()
+        importer = imp._getimporter(path)
+        import zipimport
+        assert isinstance(importer, zipimport.zipimporter)
+
+
 class AppTestNoPycFile(object):
     spaceconfig = {
         "objspace.usepycfiles": False,

diff --git a/pypy/translator/c/src/signals.h b/pypy/translator/c/src/signals.h
--- a/pypy/translator/c/src/signals.h
+++ b/pypy/translator/c/src/signals.h
@@ -143,6 +143,21 @@
 #endif
 }
 
+void pypysig_reinstall(int signum)
+{
+#ifdef SA_RESTART
+    /* Assume sigaction was used.  We did not pass SA_RESETHAND to
+       sa_flags, so there is nothing to do here. */
+#else
+# ifdef SIGCHLD
+    /* To avoid infinite recursion, this signal remains
+       reset until explicitly re-instated.  (Copied from CPython) */
+    if (signum != SIGCHLD)
+# endif
+    pypysig_setflag(signum);
+#endif
+}
+
 int pypysig_poll(void)
 {
   if (pypysig_occurred)

diff --git a/pypy/jit/backend/llgraph/llimpl.py b/pypy/jit/backend/llgraph/llimpl.py
--- a/pypy/jit/backend/llgraph/llimpl.py
+++ b/pypy/jit/backend/llgraph/llimpl.py
@@ -20,11 +20,11 @@
 from pypy.jit.metainterp import resoperation, executor
 from pypy.jit.metainterp.resoperation import rop
 from pypy.jit.backend.llgraph import symbolic
+from pypy.jit.codewriter import longlong
 
 from pypy.rlib.objectmodel import ComputedIntSymbolic, we_are_translated
 from pypy.rlib.rarithmetic import ovfcheck
 from pypy.rlib.rarithmetic import r_longlong, r_ulonglong, r_uint
-from pypy.rlib.longlong2float import longlong2float, float2longlong
 
 import py
 from pypy.tool.ansi_print import ansi_log
@@ -301,7 +301,7 @@
     return compile_start_ref_var(loop, lltype.Signed)
 
 def compile_start_float_var(loop):
-    return compile_start_ref_var(loop, lltype.Float)
+    return compile_start_ref_var(loop, longlong.FLOATSTORAGE)
 
 def compile_start_ref_var(loop, TYPE):
     loop = _from_opaque(loop)
@@ -340,7 +340,7 @@
     compile_add_ref_const(loop, value, lltype.Signed)
 
 def compile_add_float_const(loop, value):
-    compile_add_ref_const(loop, value, lltype.Float)
+    compile_add_ref_const(loop, value, longlong.FLOATSTORAGE)
 
 def compile_add_ref_const(loop, value, TYPE):
     loop = _from_opaque(loop)
@@ -353,7 +353,7 @@
     return compile_add_ref_result(loop, lltype.Signed)
 
 def compile_add_float_result(loop):
-    return compile_add_ref_result(loop, lltype.Float)
+    return compile_add_ref_result(loop, longlong.FLOATSTORAGE)
 
 def compile_add_ref_result(loop, TYPE):
     loop = _from_opaque(loop)
@@ -486,8 +486,8 @@
                         x = self.as_ptr(result)
                     elif RESTYPE is ootype.Object:
                         x = self.as_object(result)
-                    elif RESTYPE is lltype.Float:
-                        x = self.as_float(result)
+                    elif RESTYPE is longlong.FLOATSTORAGE:
+                        x = self.as_floatstorage(result)
                     else:
                         raise Exception("op.result.concretetype is %r"
                                         % (RESTYPE,))
@@ -550,8 +550,8 @@
     def as_object(self, x):
         return ootype.cast_to_object(x)
 
-    def as_float(self, x):
-        return cast_to_float(x)
+    def as_floatstorage(self, x):
+        return cast_to_floatstorage(x)
 
     def log_progress(self):
         count = sum(_stats.exec_counters.values())
@@ -823,7 +823,7 @@
             elif T == llmemory.GCREF:
                 args_in_order.append('r')
                 _call_args_r.append(x)
-            elif T is lltype.Float:
+            elif T is longlong.FLOATSTORAGE:
                 args_in_order.append('f')
                 _call_args_f.append(x)
             else:
@@ -886,7 +886,7 @@
                     set_future_value_int(i, args[i])
                 elif isinstance(TYPE, lltype.Ptr):
                     set_future_value_ref(i, args[i])
-                elif TYPE is lltype.Float:
+                elif TYPE is longlong.FLOATSTORAGE:
                     set_future_value_float(i, args[i])
                 else:
                     raise Exception("Nonsense type %s" % TYPE)
@@ -1072,25 +1072,23 @@
 def cast_from_ptr(TYPE, x):
     return lltype.cast_opaque_ptr(TYPE, x)
 
-def cast_to_float(x):
+def cast_to_floatstorage(x):
     if isinstance(x, float):
-        return x      # common case
+        return longlong.getfloatstorage(x)      # common case
     if IS_32_BIT:
+        assert longlong.supports_longlong
         if isinstance(x, r_longlong):
-            return longlong2float(x)
+            return x
         if isinstance(x, r_ulonglong):
-            return longlong2float(rffi.cast(lltype.SignedLongLong, x))
+            return rffi.cast(lltype.SignedLongLong, x)
     raise TypeError(type(x))
 
-def cast_from_float(TYPE, x):
-    assert isinstance(x, float)
+def cast_from_floatstorage(TYPE, x):
+    assert isinstance(x, longlong.r_float_storage)
     if TYPE is lltype.Float:
-        return x
-    if IS_32_BIT:
-        if TYPE is lltype.SignedLongLong:
-            return float2longlong(x)
-        if TYPE is lltype.UnsignedLongLong:
-            return r_ulonglong(float2longlong(x))
+        return longlong.getrealfloat(x)
+    if longlong.is_longlong(TYPE):
+        return rffi.cast(TYPE, x)
     raise TypeError(TYPE)
 
 
@@ -1119,6 +1117,7 @@
     set_future_value_ref(index, value)
 
 def set_future_value_float(index, value):
+    assert isinstance(value, longlong.r_float_storage)
     set_future_value_ref(index, value)
 
 def set_future_value_ref(index, value):
@@ -1157,7 +1156,7 @@
     frame = _from_opaque(frame)
     assert num >= 0
     x = frame.fail_args[num]
-    assert lltype.typeOf(x) is lltype.Float
+    assert lltype.typeOf(x) is longlong.FLOATSTORAGE
     return x
 
 def frame_ptr_getvalue(frame, num):
@@ -1295,11 +1294,11 @@
 
 def do_getarrayitem_gc_float(array, index):
     array = array._obj.container
-    return cast_to_float(array.getitem(index))
+    return cast_to_floatstorage(array.getitem(index))
 
 def do_getarrayitem_raw_float(array, index):
     array = array.adr.ptr._obj
-    return cast_to_float(array.getitem(index))
+    return cast_to_floatstorage(array.getitem(index))
 
 def do_getarrayitem_gc_ptr(array, index):
     array = array._obj.container
@@ -1314,7 +1313,7 @@
     return cast_to_int(_getfield_gc(struct, fieldnum))
 
 def do_getfield_gc_float(struct, fieldnum):
-    return cast_to_float(_getfield_gc(struct, fieldnum))
+    return cast_to_floatstorage(_getfield_gc(struct, fieldnum))
 
 def do_getfield_gc_ptr(struct, fieldnum):
     return cast_to_ptr(_getfield_gc(struct, fieldnum))
@@ -1328,7 +1327,7 @@
     return cast_to_int(_getfield_raw(struct, fieldnum))
 
 def do_getfield_raw_float(struct, fieldnum):
-    return cast_to_float(_getfield_raw(struct, fieldnum))
+    return cast_to_floatstorage(_getfield_raw(struct, fieldnum))
 
 def do_getfield_raw_ptr(struct, fieldnum):
     return cast_to_ptr(_getfield_raw(struct, fieldnum))
@@ -1358,13 +1357,13 @@
 def do_setarrayitem_gc_float(array, index, newvalue):
     array = array._obj.container
     ITEMTYPE = lltype.typeOf(array).OF
-    newvalue = cast_from_float(ITEMTYPE, newvalue)
+    newvalue = cast_from_floatstorage(ITEMTYPE, newvalue)
     array.setitem(index, newvalue)
 
 def do_setarrayitem_raw_float(array, index, newvalue):
     array = array.adr.ptr
     ITEMTYPE = lltype.typeOf(array).TO.OF
-    newvalue = cast_from_int(ITEMTYPE, newvalue)
+    newvalue = cast_from_floatstorage(ITEMTYPE, newvalue)
     array._obj.setitem(index, newvalue)
 
 def do_setarrayitem_gc_ptr(array, index, newvalue):
@@ -1384,7 +1383,7 @@
     STRUCT, fieldname = symbolic.TokenToField[fieldnum]
     ptr = lltype.cast_opaque_ptr(lltype.Ptr(STRUCT), struct)
     FIELDTYPE = getattr(STRUCT, fieldname)
-    newvalue = cast_from_float(FIELDTYPE, newvalue)
+    newvalue = cast_from_floatstorage(FIELDTYPE, newvalue)
     setattr(ptr, fieldname, newvalue)
 
 def do_setfield_gc_ptr(struct, fieldnum, newvalue):
@@ -1405,7 +1404,7 @@
     STRUCT, fieldname = symbolic.TokenToField[fieldnum]
     ptr = cast_from_int(lltype.Ptr(STRUCT), struct)
     FIELDTYPE = getattr(STRUCT, fieldname)
-    newvalue = cast_from_float(FIELDTYPE, newvalue)
+    newvalue = cast_from_floatstorage(FIELDTYPE, newvalue)
     setattr(ptr, fieldname, newvalue)
 
 def do_setfield_raw_ptr(struct, fieldnum, newvalue):
@@ -1462,6 +1461,7 @@
 kind2TYPE = {
     'i': lltype.Signed,
     'f': lltype.Float,
+    'L': lltype.SignedLongLong,
     'v': lltype.Void,
     }
 
@@ -1502,7 +1502,7 @@
 
 def do_call_float(f):
     x = _do_call_common(f)
-    return cast_to_float(x)
+    return cast_to_floatstorage(x)
 
 def do_call_ptr(f):
     x = _do_call_common(f)
@@ -1531,14 +1531,12 @@
                     assert n == 'r'
                 x = argsiter_r.next()
                 x = cast_from_ptr(TYPE, x)
-            elif TYPE is lltype.Float or (
-                    IS_32_BIT and TYPE in (lltype.SignedLongLong,
-                                           lltype.UnsignedLongLong)):
+            elif TYPE is lltype.Float or longlong.is_longlong(TYPE):
                 if args_in_order is not None:
                     n = orderiter.next()
                     assert n == 'f'
                 x = argsiter_f.next()
-                x = cast_from_float(TYPE, x)
+                x = cast_from_floatstorage(TYPE, x)
             else:
                 if args_in_order is not None:
                     n = orderiter.next()
@@ -1642,6 +1640,13 @@
 s_CompiledLoop = annmodel.SomePtr(COMPILEDLOOP)
 s_Frame = annmodel.SomePtr(FRAME)
 
+if longlong.FLOATSTORAGE is lltype.Float:
+    s_FloatStorage = annmodel.SomeFloat()
+elif longlong.FLOATSTORAGE is lltype.SignedLongLong:
+    s_FloatStorage = annmodel.SomeInteger(knowntype=longlong.r_float_storage)
+else:
+    assert 0
+
 setannotation(compile_start, s_CompiledLoop)
 setannotation(compile_start_int_var, annmodel.SomeInteger())
 setannotation(compile_start_ref_var, annmodel.SomeInteger())
@@ -1670,7 +1675,7 @@
 setannotation(frame_execute, annmodel.SomeInteger())
 setannotation(frame_int_getvalue, annmodel.SomeInteger())
 setannotation(frame_ptr_getvalue, annmodel.SomePtr(llmemory.GCREF))
-setannotation(frame_float_getvalue, annmodel.SomeFloat())
+setannotation(frame_float_getvalue, s_FloatStorage)
 setannotation(frame_get_value_count, annmodel.SomeInteger())
 setannotation(frame_clear_latest_values, annmodel.s_None)
 
@@ -1686,15 +1691,15 @@
 setannotation(do_unicodegetitem, annmodel.SomeInteger())
 setannotation(do_getarrayitem_gc_int, annmodel.SomeInteger())
 setannotation(do_getarrayitem_gc_ptr, annmodel.SomePtr(llmemory.GCREF))
-setannotation(do_getarrayitem_gc_float, annmodel.SomeFloat())
+setannotation(do_getarrayitem_gc_float, s_FloatStorage)
 setannotation(do_getarrayitem_raw_int, annmodel.SomeInteger())
-setannotation(do_getarrayitem_raw_float, annmodel.SomeFloat())
+setannotation(do_getarrayitem_raw_float, s_FloatStorage)
 setannotation(do_getfield_gc_int, annmodel.SomeInteger())
 setannotation(do_getfield_gc_ptr, annmodel.SomePtr(llmemory.GCREF))
-setannotation(do_getfield_gc_float, annmodel.SomeFloat())
+setannotation(do_getfield_gc_float, s_FloatStorage)
 setannotation(do_getfield_raw_int, annmodel.SomeInteger())
 setannotation(do_getfield_raw_ptr, annmodel.SomePtr(llmemory.GCREF))
-setannotation(do_getfield_raw_float, annmodel.SomeFloat())
+setannotation(do_getfield_raw_float, s_FloatStorage)
 setannotation(do_new, annmodel.SomePtr(llmemory.GCREF))
 setannotation(do_new_array, annmodel.SomePtr(llmemory.GCREF))
 setannotation(do_setarrayitem_gc_int, annmodel.s_None)
@@ -1716,5 +1721,5 @@
 setannotation(do_call_pushptr, annmodel.s_None)
 setannotation(do_call_int, annmodel.SomeInteger())
 setannotation(do_call_ptr, annmodel.SomePtr(llmemory.GCREF))
-setannotation(do_call_float, annmodel.SomeFloat())
+setannotation(do_call_float, s_FloatStorage)
 setannotation(do_call_void, annmodel.s_None)

diff --git a/pypy/jit/codewriter/effectinfo.py b/pypy/jit/codewriter/effectinfo.py
--- a/pypy/jit/codewriter/effectinfo.py
+++ b/pypy/jit/codewriter/effectinfo.py
@@ -71,7 +71,7 @@
     OS_LLONG_UGT                = 90
     OS_LLONG_UGE                = 91
     OS_LLONG_URSHIFT            = 92
-    OS_LLONG_FROM_TWO_INTS      = 93
+    OS_LLONG_FROM_UINT          = 93
 
     def __new__(cls, readonly_descrs_fields,
                 write_descrs_fields, write_descrs_arrays,

diff --git a/pypy/translator/c/src/g_include.h b/pypy/translator/c/src/g_include.h
--- a/pypy/translator/c/src/g_include.h
+++ b/pypy/translator/c/src/g_include.h
@@ -37,19 +37,7 @@
 #include "src/llgroup.h"
 
 #include "src/instrument.h"
-
-/* optional assembler bits */
-#if defined(__GNUC__) && defined(__i386__)
-#  include "src/asm_gcc_x86.h"
-#endif
-
-#if defined(__GNUC__) && defined(__amd64__)
-#  include "src/asm_gcc_x86_64.h"
-#endif
-
-#if defined(__GNUC__) && defined(__ppc__)
-#  include "src/asm_ppc.h"
-#endif
+#include "src/asm.h"
 
 
 /*** modules ***/

diff --git a/pypy/jit/metainterp/pyjitpl.py b/pypy/jit/metainterp/pyjitpl.py
--- a/pypy/jit/metainterp/pyjitpl.py
+++ b/pypy/jit/metainterp/pyjitpl.py
@@ -20,7 +20,7 @@
 from pypy.rlib.rarithmetic import intmask
 from pypy.rlib.objectmodel import specialize
 from pypy.jit.codewriter.jitcode import JitCode, SwitchDictDescr, MissingLiveness
-from pypy.jit.codewriter import heaptracker
+from pypy.jit.codewriter import heaptracker, longlong
 from pypy.jit.metainterp.optimizeutil import RetraceLoop
 
 # ____________________________________________________________
@@ -1473,7 +1473,7 @@
             elif result_type == history.REF:
                 raise sd.DoneWithThisFrameRef(self.cpu, resultbox.getref_base())
             elif result_type == history.FLOAT:
-                raise sd.DoneWithThisFrameFloat(resultbox.getfloat())
+                raise sd.DoneWithThisFrameFloat(resultbox.getfloatstorage())
             else:
                 assert False
 
@@ -2151,7 +2151,7 @@
             # warmstate.py.
             virtualizable_box = self.virtualizable_boxes[-1]
             virtualizable = vinfo.unwrap_virtualizable_box(virtualizable_box)
-            assert not virtualizable.vable_token
+            assert not vinfo.gettoken(virtualizable)
             # fill the virtualizable with the local boxes
             self.synchronize_virtualizable()
         #

diff --git a/pypy/module/imp/importing.py b/pypy/module/imp/importing.py
--- a/pypy/module/imp/importing.py
+++ b/pypy/module/imp/importing.py
@@ -291,7 +291,8 @@
         if space.is_true(w_loader):
             return w_loader
 
-def find_in_path_hooks(space, w_modulename, w_pathitem):
+def _getimporter(space, w_pathitem):
+    # the function 'imp._getimporter' is a pypy-only extension
     w_path_importer_cache = space.sys.get("path_importer_cache")
     w_importer = space.finditem(w_path_importer_cache, w_pathitem)
     if w_importer is None:
@@ -311,11 +312,15 @@
                 )
             except OperationError, e:
                 if e.match(space, space.w_ImportError):
-                    return
+                    return None
                 raise
         if space.is_true(w_importer):
             space.setitem(w_path_importer_cache, w_pathitem, w_importer)
-    if space.is_true(w_importer):
+    return w_importer
+
+def find_in_path_hooks(space, w_modulename, w_pathitem):
+    w_importer = _getimporter(space, w_pathitem)
+    if w_importer is not None and space.is_true(w_importer):
         w_loader = space.call_method(w_importer, "find_module", w_modulename)
         if space.is_true(w_loader):
             return w_loader

diff --git a/pypy/jit/metainterp/warmstate.py b/pypy/jit/metainterp/warmstate.py
--- a/pypy/jit/metainterp/warmstate.py
+++ b/pypy/jit/metainterp/warmstate.py
@@ -12,13 +12,13 @@
 from pypy.rlib.jit import BaseJitCell
 from pypy.rlib.debug import debug_start, debug_stop, debug_print
 from pypy.jit.metainterp import history
-from pypy.jit.codewriter import support, heaptracker
+from pypy.jit.codewriter import support, heaptracker, longlong
 
 # ____________________________________________________________
 
 @specialize.arg(0)
 def specialize_value(TYPE, x):
-    """'x' must be a Signed, a GCREF or a Float.
+    """'x' must be a Signed, a GCREF or a FLOATSTORAGE.
     This function casts it to a more specialized type, like Char or Ptr(..).
     """
     INPUT = lltype.typeOf(x)
@@ -28,15 +28,15 @@
             return rffi.cast(TYPE, x)
         else:
             return lltype.cast_primitive(TYPE, x)
-    elif INPUT is lltype.Float:
+    elif INPUT is longlong.FLOATSTORAGE:
         assert TYPE is lltype.Float
-        return x
+        return longlong.getrealfloat(x)
     else:
         return lltype.cast_opaque_ptr(TYPE, x)
 
 @specialize.ll()
 def unspecialize_value(value):
-    """Casts 'value' to a Signed, a GCREF or a Float."""
+    """Casts 'value' to a Signed, a GCREF or a FLOATSTORAGE."""
     if isinstance(lltype.typeOf(value), lltype.Ptr):
         if lltype.typeOf(value).TO._gckind == 'gc':
             return lltype.cast_opaque_ptr(llmemory.GCREF, value)
@@ -46,7 +46,7 @@
     elif isinstance(lltype.typeOf(value), ootype.OOType):
         return ootype.cast_to_object(value)
     elif isinstance(value, float):
-        return value
+        return longlong.getfloatstorage(value)
     else:
         return intmask(value)
 
@@ -83,6 +83,7 @@
         else:
             return history.BoxObj(value)
     elif isinstance(value, float):
+        value = longlong.getfloatstorage(value)
         if in_const_box:
             return history.ConstFloat(value)
         else:
@@ -138,7 +139,11 @@
         intvalue = lltype.cast_primitive(lltype.Signed, value)
         cpu.set_future_value_int(j, intvalue)
     elif typecode == 'float':
-        assert isinstance(value, float)
+        if lltype.typeOf(value) is lltype.Float:
+            value = longlong.getfloatstorage(value)
+        else:
+            assert longlong.is_longlong(lltype.typeOf(value))
+            value = rffi.cast(lltype.SignedLongLong, value)
         cpu.set_future_value_float(j, value)
     else:
         assert False

diff --git a/pypy/jit/backend/x86/runner.py b/pypy/jit/backend/x86/runner.py
--- a/pypy/jit/backend/x86/runner.py
+++ b/pypy/jit/backend/x86/runner.py
@@ -13,7 +13,6 @@
 class AbstractX86CPU(AbstractLLCPU):
     debug = True
     supports_floats = True
-    supports_longlong = True
 
     BOOTSTRAP_TP = lltype.FuncType([], lltype.Signed)
     dont_keepalive_stuff = False # for tests
@@ -150,12 +149,15 @@
     CALLEE_SAVE_REGISTERS = [regloc.ebx, regloc.esi, regloc.edi]
     FRAME_FIXED_SIZE = len(CALLEE_SAVE_REGISTERS) + 2
 
+    supports_longlong = True
+
     def __init__(self, *args, **kwargs):
         assert sys.maxint == (2**31 - 1)
         super(CPU386, self).__init__(*args, **kwargs)
 
 class CPU386_NO_SSE2(CPU386):
     supports_floats = False
+    supports_longlong = False
 
 class CPU_X86_64(AbstractX86CPU):
     WORD = 8

diff --git a/pypy/jit/backend/test/runner_test.py b/pypy/jit/backend/test/runner_test.py
--- a/pypy/jit/backend/test/runner_test.py
+++ b/pypy/jit/backend/test/runner_test.py
@@ -14,9 +14,15 @@
 from pypy.rpython.ootypesystem import ootype
 from pypy.rpython.annlowlevel import llhelper
 from pypy.rpython.llinterp import LLException
-from pypy.jit.codewriter import heaptracker
+from pypy.jit.codewriter import heaptracker, longlong
 from pypy.rlib.rarithmetic import intmask
 
+def boxfloat(x):
+    return BoxFloat(longlong.getfloatstorage(x))
+
+def constfloat(x):
+    return ConstFloat(longlong.getfloatstorage(x))
+
 
 class Runner(object):
 
@@ -36,7 +42,7 @@
                 self.cpu.set_future_value_ref(j, box.getref_base())
                 j += 1
             elif isinstance(box, BoxFloat):
-                self.cpu.set_future_value_float(j, box.getfloat())
+                self.cpu.set_future_value_float(j, box.getfloatstorage())
                 j += 1
             else:
                 raise NotImplementedError(box)
@@ -344,7 +350,10 @@
         from pypy.jit.metainterp.test.test_executor import get_float_tests
         for opnum, boxargs, rettype, retvalue in get_float_tests(self.cpu):
             res = self.execute_operation(opnum, boxargs, rettype)
-            assert res.value == retvalue
+            if isinstance(res, BoxFloat):
+                assert res.getfloat() == retvalue
+            else:
+                assert res.value == retvalue
 
     def test_ovf_operations(self, reversed=False):
         minint = -sys.maxint-1
@@ -416,6 +425,8 @@
         assert x == ord('B')
         if cpu.supports_floats:
             def func(f, i):
+                assert isinstance(f, float)
+                assert isinstance(i, int)
                 return f - float(i)
             FPTR = self.Ptr(self.FuncType([lltype.Float, lltype.Signed],
                                           lltype.Float))
@@ -424,8 +435,8 @@
             calldescr = cpu.calldescrof(FTP, FTP.ARGS, FTP.RESULT)
             x = cpu.bh_call_f(self.get_funcbox(cpu, func_ptr).value,
                               calldescr,
-                              [42], None, [3.5])
-            assert x == 3.5 - 42
+                              [42], None, [longlong.getfloatstorage(3.5)])
+            assert longlong.getrealfloat(x) == 3.5 - 42
 
     def test_call(self):
         from pypy.rlib.libffi import types
@@ -479,13 +490,13 @@
             func_ptr = llhelper(FPTR, func)
             calldescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
             funcbox = self.get_funcbox(cpu, func_ptr)
-            args = ([BoxFloat(.1) for i in range(7)] +
-                    [BoxInt(1), BoxInt(2), BoxFloat(.2), BoxFloat(.3),
-                     BoxFloat(.4)])
+            args = ([boxfloat(.1) for i in range(7)] +
+                    [BoxInt(1), BoxInt(2), boxfloat(.2), boxfloat(.3),
+                     boxfloat(.4)])
             res = self.execute_operation(rop.CALL,
                                          [funcbox] + args,
                                          'float', descr=calldescr)
-            assert abs(res.value - 4.6) < 0.0001
+            assert abs(res.getfloat() - 4.6) < 0.0001
 
     def test_call_many_arguments(self):
         # Test calling a function with a large number of arguments (more than
@@ -547,6 +558,20 @@
                                      descr=calldescr)
         assert res.value == ord('a')
 
+    def test_call_with_const_floats(self):
+        def func(f1, f2):
+            return f1 + f2
+
+        FUNC = self.FuncType([lltype.Float, lltype.Float], lltype.Float)
+        FPTR = self.Ptr(FUNC)
+        calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
+        func_ptr = llhelper(FPTR, func)
+        funcbox = self.get_funcbox(self.cpu, func_ptr)
+        res = self.execute_operation(rop.CALL, [funcbox, constfloat(1.5),
+                                                constfloat(2.5)], 'float',
+                                     descr=calldescr)
+        assert res.getfloat() == 4.0
+
 
     def test_field_basic(self):
         t_box, T_box = self.alloc_instance(self.T)
@@ -599,17 +624,17 @@
         assert res.value == null_const.value
         if self.cpu.supports_floats:
             floatdescr = self.cpu.fielddescrof(self.S, 'float')
-            self.execute_operation(rop.SETFIELD_GC, [t_box, BoxFloat(3.4)],
+            self.execute_operation(rop.SETFIELD_GC, [t_box, boxfloat(3.4)],
                                    'void', descr=floatdescr)
             res = self.execute_operation(rop.GETFIELD_GC, [t_box],
                                          'float', descr=floatdescr)
-            assert res.value == 3.4
+            assert res.getfloat() == 3.4
             #
-            self.execute_operation(rop.SETFIELD_GC, [t_box, ConstFloat(-3.6)],
+            self.execute_operation(rop.SETFIELD_GC, [t_box, constfloat(-3.6)],
                                    'void', descr=floatdescr)
             res = self.execute_operation(rop.GETFIELD_GC, [t_box],
                                          'float', descr=floatdescr)
-            assert res.value == -3.6
+            assert res.getfloat() == -3.6
 
 
     def test_passing_guards(self):
@@ -625,7 +650,7 @@
                (rop.GUARD_ISNULL, [nullbox])
                ])
         if self.cpu.supports_floats:
-            all.append((rop.GUARD_VALUE, [BoxFloat(3.5), ConstFloat(3.5)]))
+            all.append((rop.GUARD_VALUE, [boxfloat(3.5), constfloat(3.5)]))
         for (opname, args) in all:
             assert self.execute_operation(opname, args, 'void') == None
             assert not self.guard_failed
@@ -651,7 +676,7 @@
                (rop.GUARD_NONNULL, [nullbox]),
                (rop.GUARD_ISNULL, [t_box])])
         if self.cpu.supports_floats:
-            all.append((rop.GUARD_VALUE, [BoxFloat(-1.0), ConstFloat(1.0)]))
+            all.append((rop.GUARD_VALUE, [boxfloat(-1.0), constfloat(1.0)]))
         for opname, args in all:
             assert self.execute_operation(opname, args, 'void') == None
             assert self.guard_failed
@@ -816,17 +841,17 @@
             a_box, A = self.alloc_array_of(lltype.Float, 31)
             arraydescr = self.cpu.arraydescrof(A)
             self.execute_operation(rop.SETARRAYITEM_GC, [a_box, BoxInt(1),
-                                                         BoxFloat(3.5)],
+                                                         boxfloat(3.5)],
                                    'void', descr=arraydescr)
             self.execute_operation(rop.SETARRAYITEM_GC, [a_box, BoxInt(2),
-                                                         ConstFloat(4.5)],
+                                                         constfloat(4.5)],
                                    'void', descr=arraydescr)
             r = self.execute_operation(rop.GETARRAYITEM_GC, [a_box, BoxInt(1)],
                                        'float', descr=arraydescr)
-            assert r.value == 3.5
+            assert r.getfloat() == 3.5
             r = self.execute_operation(rop.GETARRAYITEM_GC, [a_box, BoxInt(2)],
                                        'float', descr=arraydescr)
-            assert r.value == 4.5
+            assert r.getfloat() == 4.5
 
         # For platforms where sizeof(INT) != sizeof(Signed) (ie, x86-64)
         a_box, A = self.alloc_array_of(rffi.INT, 342)
@@ -926,10 +951,10 @@
         assert r.value == u_box.value
 
         if self.cpu.supports_floats:
-            r = self.execute_operation(rop.SAME_AS, [ConstFloat(5.5)], 'float')
-            assert r.value == 5.5
-            r = self.execute_operation(rop.SAME_AS, [BoxFloat(5.5)], 'float')
-            assert r.value == 5.5
+            r = self.execute_operation(rop.SAME_AS, [constfloat(5.5)], 'float')
+            assert r.getfloat() == 5.5
+            r = self.execute_operation(rop.SAME_AS, [boxfloat(5.5)], 'float')
+            assert r.getfloat() == 5.5
 
     def test_virtual_ref(self):
         pass   # VIRTUAL_REF must not reach the backend nowadays
@@ -1000,7 +1025,7 @@
                     p = lltype.malloc(S)
                     values.append(lltype.cast_opaque_ptr(llmemory.GCREF, p))
                 elif isinstance(box, BoxFloat):
-                    values.append(r.random())
+                    values.append(longlong.getfloatstorage(r.random()))
                 else:
                     assert 0
             values[index_counter] = 11
@@ -1048,7 +1073,7 @@
         faildescr1 = BasicFailDescr(1)
         faildescr2 = BasicFailDescr(2)
         operations = [
-            ResOperation(rop.FLOAT_LE, [fboxes[0], ConstFloat(9.2)], i2),
+            ResOperation(rop.FLOAT_LE, [fboxes[0], constfloat(9.2)], i2),
             ResOperation(rop.GUARD_TRUE, [i2], None, descr=faildescr1),
             ResOperation(rop.FINISH, fboxes, None, descr=faildescr2),
             ]
@@ -1059,20 +1084,22 @@
         fboxes2 = [BoxFloat() for i in range(12)]
         f3 = BoxFloat()
         bridge = [
-            ResOperation(rop.FLOAT_SUB, [fboxes2[0], ConstFloat(1.0)], f3),
+            ResOperation(rop.FLOAT_SUB, [fboxes2[0], constfloat(1.0)], f3),
             ResOperation(rop.JUMP, [f3] + fboxes2[1:], None, descr=looptoken),
         ]
 
         self.cpu.compile_bridge(faildescr1, fboxes2, bridge, looptoken)
 
         for i in range(len(fboxes)):
-            self.cpu.set_future_value_float(i, 13.5 + 6.73 * i)
+            x = 13.5 + 6.73 * i
+            self.cpu.set_future_value_float(i, longlong.getfloatstorage(x))
         fail = self.cpu.execute_token(looptoken)
         assert fail.identifier == 2
         res = self.cpu.get_latest_value_float(0)
-        assert res == 8.5
+        assert longlong.getrealfloat(res) == 8.5
         for i in range(1, len(fboxes)):
-            assert self.cpu.get_latest_value_float(i) == 13.5 + 6.73 * i
+            got = longlong.getrealfloat(self.cpu.get_latest_value_float(i))
+            assert got == 13.5 + 6.73 * i
 
     def test_integers_and_guards(self):
         for opname, compare in [
@@ -1149,11 +1176,11 @@
                     if combinaison[0] == 'b':
                         fbox1 = BoxFloat()
                     else:
-                        fbox1 = ConstFloat(-4.5)
+                        fbox1 = constfloat(-4.5)
                     if combinaison[1] == 'b':
                         fbox2 = BoxFloat()
                     else:
-                        fbox2 = ConstFloat(-4.5)
+                        fbox2 = constfloat(-4.5)
                     b1 = BoxInt()
                     faildescr1 = BasicFailDescr(1)
                     faildescr2 = BasicFailDescr(2)
@@ -1177,10 +1204,12 @@
                                 if test2 == -4.5 or combinaison[1] == 'b':
                                     n = 0
                                     if combinaison[0] == 'b':
-                                        cpu.set_future_value_float(n, test1)
+                                        cpu.set_future_value_float(
+                                            n, longlong.getfloatstorage(test1))
                                         n += 1
                                     if combinaison[1] == 'b':
-                                        cpu.set_future_value_float(n, test2)
+                                        cpu.set_future_value_float(
+                                            n, longlong.getfloatstorage(test2))
                                         n += 1
                                     fail = cpu.execute_token(looptoken)
                                     #
@@ -1230,7 +1259,7 @@
             if isinstance(box, BoxInt):
                 self.cpu.set_future_value_int(i, box.getint())
             elif isinstance(box, BoxFloat):
-                self.cpu.set_future_value_float(i, box.getfloat())
+                self.cpu.set_future_value_float(i, box.getfloatstorage())
             else:
                 assert 0
         #
@@ -1244,12 +1273,12 @@
         from pypy.rlib.rarithmetic import INFINITY, NAN, isinf, isnan
         from pypy.jit.metainterp.resoperation import opname
 
-        fzer = BoxFloat(0.0)
-        fone = BoxFloat(1.0)
-        fmqr = BoxFloat(-0.25)
-        finf = BoxFloat(INFINITY)
-        fmnf = BoxFloat(-INFINITY)
-        fnan = BoxFloat(NAN)
+        fzer = boxfloat(0.0)
+        fone = boxfloat(1.0)
+        fmqr = boxfloat(-0.25)
+        finf = boxfloat(INFINITY)
+        fmnf = boxfloat(-INFINITY)
+        fnan = boxfloat(NAN)
 
         all_cases_unary =  [(a,)   for a in [fzer,fone,fmqr,finf,fmnf,fnan]]
         all_cases_binary = [(a, b) for a in [fzer,fone,fmqr,finf,fmnf,fnan]
@@ -1259,7 +1288,7 @@
 
         def nan_and_infinity(opnum, realoperation, testcases):
             for testcase in testcases:
-                realvalues = [b.value for b in testcase]
+                realvalues = [b.getfloat() for b in testcase]
                 expected = realoperation(*realvalues)
                 if isinstance(expected, float):
                     expectedtype = 'float'
@@ -1268,15 +1297,17 @@
                 got = self.execute_operation(opnum, list(testcase),
                                              expectedtype)
                 if isnan(expected):
-                    ok = isnan(got.value)
+                    ok = isnan(got.getfloat())
                 elif isinf(expected):
-                    ok = isinf(got.value)
+                    ok = isinf(got.getfloat())
+                elif isinstance(got, BoxFloat):
+                    ok = (got.getfloat() == expected)
                 else:
-                    ok = (got.value == expected)
+                    ok = got.value == expected
                 if not ok:
                     raise AssertionError("%s(%s): got %r, expected %r" % (
                         opname[opnum], ', '.join(map(repr, realvalues)),
-                        got.value, expected))
+                        got.getfloat(), expected))
                 # if we expect a boolean, also check the combination with
                 # a GUARD_TRUE or GUARD_FALSE
                 if isinstance(expected, bool):
@@ -1296,7 +1327,8 @@
                         self.cpu.compile_loop(unique_testcase_list, operations,
                                               looptoken)
                         for i, box in enumerate(unique_testcase_list):
-                            self.cpu.set_future_value_float(i, box.value)
+                            self.cpu.set_future_value_float(
+                                i, box.getfloatstorage())
                         fail = self.cpu.execute_token(looptoken)
                         if fail.identifier != 5 - (expected_id^expected):
                             if fail.identifier == 4:
@@ -1767,7 +1799,8 @@
         self.cpu.set_future_value_int(1, 0)
         fail = self.cpu.execute_token(looptoken)
         assert fail.identifier == 0
-        assert self.cpu.get_latest_value_float(0) == 42.5
+        x = self.cpu.get_latest_value_float(0)
+        assert longlong.getrealfloat(x) == 42.5
         assert values == []
 
         self.cpu.set_future_value_int(0, 10)
@@ -1775,7 +1808,8 @@
         fail = self.cpu.execute_token(looptoken)
         assert fail.identifier == 1
         assert self.cpu.get_latest_value_int(0) == 1
-        assert self.cpu.get_latest_value_float(1) == 42.5
+        x = self.cpu.get_latest_value_float(1)
+        assert longlong.getrealfloat(x) == 42.5
         assert self.cpu.get_latest_value_int(2) == 10
         assert values == [1, 10]
 
@@ -1810,9 +1844,10 @@
             descr_C = cpu.arraydescrof(C)
             x = cpu.bh_getarrayitem_gc_f(
                 descr_C, lltype.cast_opaque_ptr(llmemory.GCREF, c), 3)
-            assert x == 3.5
+            assert longlong.getrealfloat(x) == 3.5
             cpu.bh_setarrayitem_gc_f(
-                descr_C, lltype.cast_opaque_ptr(llmemory.GCREF, c), 4, 4.5)
+                descr_C, lltype.cast_opaque_ptr(llmemory.GCREF, c), 4,
+                longlong.getfloatstorage(4.5))
             assert c[4] == 4.5
         s = rstr.mallocstr(6)
         x = cpu.bh_strlen(lltype.cast_opaque_ptr(llmemory.GCREF, s))
@@ -1866,13 +1901,13 @@
             descrfld_z = cpu.fielddescrof(S, 'z')
             cpu.bh_setfield_gc_f(
                 lltype.cast_opaque_ptr(llmemory.GCREF, s),
-                descrfld_z, 3.5)
+                descrfld_z, longlong.getfloatstorage(3.5))
             assert s.z == 3.5
             s.z = 3.2
             x = cpu.bh_getfield_gc_f(
                 lltype.cast_opaque_ptr(llmemory.GCREF, s),
                 descrfld_z)
-            assert x == 3.2
+            assert longlong.getrealfloat(x) == 3.2
         ### we don't support in the JIT for now GC pointers
         ### stored inside non-GC structs.
         #descrfld_ry = cpu.fielddescrof(RS, 'y')
@@ -2028,7 +2063,8 @@
             py.test.skip("requires floats")
         called = []
         def assembler_helper(failindex, virtualizable):
-            assert self.cpu.get_latest_value_float(0) == 1.2 + 3.2
+            x = self.cpu.get_latest_value_float(0)
+            assert longlong.getrealfloat(x) == 1.2 + 3.2
             called.append(failindex)
             return 13.5
 
@@ -2054,10 +2090,11 @@
         looptoken = LoopToken()
         looptoken.outermost_jitdriver_sd = FakeJitDriverSD()
         self.cpu.compile_loop(loop.inputargs, loop.operations, looptoken)
-        self.cpu.set_future_value_float(0, 1.2)
-        self.cpu.set_future_value_float(1, 2.3)
+        self.cpu.set_future_value_float(0, longlong.getfloatstorage(1.2))
+        self.cpu.set_future_value_float(1, longlong.getfloatstorage(2.3))
         res = self.cpu.execute_token(looptoken)
-        assert self.cpu.get_latest_value_float(0) == 1.2 + 2.3
+        x = self.cpu.get_latest_value_float(0)
+        assert longlong.getrealfloat(x) == 1.2 + 2.3
         ops = '''
         [f4, f5]
         f3 = call_assembler(f4, f5, descr=looptoken)
@@ -2067,10 +2104,11 @@
         loop = parse(ops, namespace=locals())
         othertoken = LoopToken()
         self.cpu.compile_loop(loop.inputargs, loop.operations, othertoken)
-        self.cpu.set_future_value_float(0, 1.2)
-        self.cpu.set_future_value_float(1, 3.2)
+        self.cpu.set_future_value_float(0, longlong.getfloatstorage(1.2))
+        self.cpu.set_future_value_float(1, longlong.getfloatstorage(3.2))
         res = self.cpu.execute_token(othertoken)
-        assert self.cpu.get_latest_value_float(0) == 13.5
+        x = self.cpu.get_latest_value_float(0)
+        assert longlong.getrealfloat(x) == 13.5
         assert called
 
         # test the fast path, which should not call assembler_helper()
@@ -2079,10 +2117,11 @@
         try:
             othertoken = LoopToken()
             self.cpu.compile_loop(loop.inputargs, loop.operations, othertoken)
-            self.cpu.set_future_value_float(0, 1.2)
-            self.cpu.set_future_value_float(1, 3.2)
+            self.cpu.set_future_value_float(0, longlong.getfloatstorage(1.2))
+            self.cpu.set_future_value_float(1, longlong.getfloatstorage(3.2))
             res = self.cpu.execute_token(othertoken)
-            assert self.cpu.get_latest_value_float(0) == 1.2 + 3.2
+            x = self.cpu.get_latest_value_float(0)
+            assert longlong.getrealfloat(x) == 1.2 + 3.2
             assert not called
         finally:
             del self.cpu.done_with_this_frame_float_v
@@ -2116,7 +2155,8 @@
             py.test.skip("requires floats")
         called = []
         def assembler_helper(failindex, virtualizable):
-            assert self.cpu.get_latest_value_float(0) == 1.25 + 3.25
+            x = self.cpu.get_latest_value_float(0)
+            assert longlong.getrealfloat(x) == 1.25 + 3.25
             called.append(failindex)
             return 13.5
 
@@ -2141,10 +2181,11 @@
         looptoken = LoopToken()
         looptoken.outermost_jitdriver_sd = FakeJitDriverSD()
         self.cpu.compile_loop(loop.inputargs, loop.operations, looptoken)
-        self.cpu.set_future_value_float(0, 1.25)
-        self.cpu.set_future_value_float(1, 2.35)
+        self.cpu.set_future_value_float(0, longlong.getfloatstorage(1.25))
+        self.cpu.set_future_value_float(1, longlong.getfloatstorage(2.35))
         res = self.cpu.execute_token(looptoken)
-        assert self.cpu.get_latest_value_float(0) == 1.25 + 2.35
+        x = self.cpu.get_latest_value_float(0)
+        assert longlong.getrealfloat(x) == 1.25 + 2.35
         assert not called
 
         ops = '''
@@ -2158,10 +2199,11 @@
         self.cpu.compile_loop(loop.inputargs, loop.operations, othertoken)
 
         # normal call_assembler: goes to looptoken
-        self.cpu.set_future_value_float(0, 1.25)
-        self.cpu.set_future_value_float(1, 3.25)
+        self.cpu.set_future_value_float(0, longlong.getfloatstorage(1.25))
+        self.cpu.set_future_value_float(1, longlong.getfloatstorage(3.25))
         res = self.cpu.execute_token(othertoken)
-        assert self.cpu.get_latest_value_float(0) == 13.5
+        x = self.cpu.get_latest_value_float(0)
+        assert longlong.getrealfloat(x) == 13.5
         assert called
         del called[:]
 
@@ -2179,10 +2221,12 @@
         self.cpu.redirect_call_assembler(looptoken, looptoken2)
 
         # now, our call_assembler should go to looptoken2
-        self.cpu.set_future_value_float(0, 6.0)
-        self.cpu.set_future_value_float(1, 1.5)    # 6.0-1.5 == 1.25+3.25
+        self.cpu.set_future_value_float(0, longlong.getfloatstorage(6.0))
+        self.cpu.set_future_value_float(1, longlong.getfloatstorage(1.5))
+                                                       # 6.0-1.5 == 1.25+3.25
         res = self.cpu.execute_token(othertoken)
-        assert self.cpu.get_latest_value_float(0) == 13.5
+        x = self.cpu.get_latest_value_float(0)
+        assert longlong.getrealfloat(x) == 13.5
         assert called
 
     def test_short_result_of_getfield_direct(self):
@@ -2379,6 +2423,66 @@
             assert res.value == expected, (
                 "%r: got %r, expected %r" % (RESTYPE, res.value, expected))
 
+    def test_supports_longlong(self):
+        if sys.maxint > 2147483647:
+            assert not self.cpu.supports_longlong, (
+                "supports_longlong should be False on 64-bit platforms")
+
+    def test_longlong_result_of_call_direct(self):
+        if not self.cpu.supports_longlong:
+            py.test.skip("longlong test")
+        from pypy.translator.tool.cbuild import ExternalCompilationInfo
+        from pypy.rlib.rarithmetic import r_longlong
+        eci = ExternalCompilationInfo(
+            separate_module_sources=["""
+            long long fn_test_result_of_call(long long x)
+            {
+                return x - 100000000000000;
+            }
+            """],
+            export_symbols=['fn_test_result_of_call'])
+        f = rffi.llexternal('fn_test_result_of_call', [lltype.SignedLongLong],
+                            lltype.SignedLongLong,
+                            compilation_info=eci, _nowrapper=True)
+        value = r_longlong(0x7ff05af3307a3fff)
+        expected = r_longlong(0x7ff000001fffffff)
+        assert f(value) == expected
+        #
+        FUNC = self.FuncType([lltype.SignedLongLong], lltype.SignedLongLong)
+        FPTR = self.Ptr(FUNC)
+        calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
+        x = self.cpu.bh_call_f(self.get_funcbox(self.cpu, f).value,
+                               calldescr, None, None, [value])
+        assert x == expected
+
+    def test_longlong_result_of_call_compiled(self):
+        if not self.cpu.supports_longlong:
+            py.test.skip("test of longlong result")
+        from pypy.translator.tool.cbuild import ExternalCompilationInfo
+        from pypy.rlib.rarithmetic import r_longlong
+        eci = ExternalCompilationInfo(
+            separate_module_sources=["""
+            long long fn_test_result_of_call(long long x)
+            {
+                return x - 100000000000000;
+            }
+            """],
+            export_symbols=['fn_test_result_of_call'])
+        f = rffi.llexternal('fn_test_result_of_call', [lltype.SignedLongLong],
+                            lltype.SignedLongLong,
+                            compilation_info=eci, _nowrapper=True)
+        value = r_longlong(0x7ff05af3307a3fff)
+        expected = r_longlong(0x7ff000001fffffff)
+        assert f(value) == expected
+        #
+        FUNC = self.FuncType([lltype.SignedLongLong], lltype.SignedLongLong)
+        FPTR = self.Ptr(FUNC)
+        calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
+        funcbox = self.get_funcbox(self.cpu, f)
+        res = self.execute_operation(rop.CALL, [funcbox, BoxFloat(value)],
+                                     'float', descr=calldescr)
+        assert res.getfloatstorage() == expected
+
     def test_free_loop_and_bridges(self):
         from pypy.jit.backend.llsupport.llmodel import AbstractLLCPU
         if not isinstance(self.cpu, AbstractLLCPU):

diff --git a/pypy/jit/backend/llsupport/test/test_descr.py b/pypy/jit/backend/llsupport/test/test_descr.py
--- a/pypy/jit/backend/llsupport/test/test_descr.py
+++ b/pypy/jit/backend/llsupport/test/test_descr.py
@@ -5,6 +5,7 @@
 from pypy.rpython.annlowlevel import llhelper
 from pypy.jit.metainterp.history import BoxInt, BoxFloat, BoxPtr
 from pypy.jit.metainterp import history
+from pypy.jit.codewriter import longlong
 import sys, struct, py
 
 def test_get_size_descr():
@@ -248,7 +249,7 @@
     #
     descr6 = get_call_descr(c0, [lltype.Signed], lltype.SignedLongLong)
     assert descr6.get_result_size(False) == 8
-    assert descr6.get_return_type() == history.FLOAT
+    assert descr6.get_return_type() == "L"
     assert descr6.arg_classes == "i"
 
 def test_get_call_descr_translated():
@@ -351,5 +352,5 @@
     opaquea = lltype.cast_opaque_ptr(llmemory.GCREF, a)
     a[0] = 1
     res = descr2.call_stub(rffi.cast(lltype.Signed, fnptr),
-                           [], [opaquea], [3.5])
-    assert res == 4.5
+                           [], [opaquea], [longlong.getfloatstorage(3.5)])
+    assert longlong.getrealfloat(res) == 4.5

diff --git a/lib-python/conftest.py b/lib-python/conftest.py
--- a/lib-python/conftest.py
+++ b/lib-python/conftest.py
@@ -696,6 +696,8 @@
                 cmd += ' --pdb'
             if self.config.option.capture == 'no':
                 status = os.system(cmd)
+                stdout.write('')
+                stderr.write('')
             else:
                 status = os.system("%s >>%s 2>>%s" %(cmd, stdout, stderr))
             if os.WIFEXITED(status):

diff --git a/pypy/jit/backend/llsupport/descr.py b/pypy/jit/backend/llsupport/descr.py
--- a/pypy/jit/backend/llsupport/descr.py
+++ b/pypy/jit/backend/llsupport/descr.py
@@ -5,9 +5,8 @@
 from pypy.jit.metainterp.history import BasicFailDescr, LoopToken, BoxFloat
 from pypy.jit.metainterp import history
 from pypy.jit.metainterp.resoperation import ResOperation, rop
-from pypy.jit.codewriter import heaptracker
+from pypy.jit.codewriter import heaptracker, longlong
 from pypy.rlib.rarithmetic import r_longlong, r_ulonglong
-from pypy.rlib.longlong2float import longlong2float, float2longlong
 
 # The point of the class organization in this file is to make instances
 # as compact as possible.  This is done by not storing the field size or
@@ -275,7 +274,10 @@
     def create_call_stub(self, rtyper, RESULT):
         def process(c):
             if c == 'L':
-                return 'float2longlong(%s)' % (process('f'),)
+                assert longlong.supports_longlong
+                c = 'f'
+            elif c == 'f' and longlong.supports_longlong:
+                return 'longlong.getrealfloat(%s)' % (process('L'),)
             arg = 'args_%s[%d]' % (c, seen[c])
             seen[c] += 1
             return arg
@@ -302,7 +304,9 @@
         elif self.get_return_type() == history.REF:
             result = 'lltype.cast_opaque_ptr(llmemory.GCREF, res)'
         elif self.get_return_type() == history.FLOAT:
-            result = 'cast_to_float(res)'
+            result = 'longlong.getfloatstorage(res)'
+        elif self.get_return_type() == 'L':
+            result = 'rffi.cast(lltype.SignedLongLong, res)'
         elif self.get_return_type() == history.VOID:
             result = 'None'
         else:
@@ -321,7 +325,7 @@
         self.call_stub = d['call_stub']
 
     def verify_types(self, args_i, args_r, args_f, return_type):
-        assert self._return_type == return_type
+        assert self._return_type in return_type
         assert self.arg_classes.count('i') == len(args_i or ())
         assert self.arg_classes.count('r') == len(args_r or ())
         assert (self.arg_classes.count('f') +
@@ -330,15 +334,6 @@
     def repr_of_descr(self):
         return '<%s>' % self._clsname
 
-def cast_to_float(x):
-    if isinstance(x, r_longlong):
-        return longlong2float(x)
-    if isinstance(x, r_ulonglong):
-        return longlong2float(rffi.cast(lltype.SignedLongLong, x))
-    assert isinstance(x, float)
-    return x
-cast_to_float._annspecialcase_ = 'specialize:argtype(0)'
-
 
 class BaseIntCallDescr(BaseCallDescr):
     # Base class of the various subclasses of descrs corresponding to
@@ -393,12 +388,13 @@
 class FloatCallDescr(BaseCallDescr):
     _clsname = 'FloatCallDescr'
     _return_type = history.FLOAT
-    call_stub = staticmethod(lambda func, args_i, args_r, args_f: 0.0)
+    call_stub = staticmethod(lambda func,args_i,args_r,args_f: longlong.ZEROF)
     def get_result_size(self, translate_support_code):
         return symbolic.get_size(lltype.Float, translate_support_code)
 
 class LongLongCallDescr(FloatCallDescr):
     _clsname = 'LongLongCallDescr'
+    _return_type = 'L'
 
 class VoidCallDescr(BaseCallDescr):
     _clsname = 'VoidCallDescr'

diff --git a/pypy/objspace/std/test/helper.py b/pypy/objspace/std/test/helper.py
deleted file mode 100644
--- a/pypy/objspace/std/test/helper.py
+++ /dev/null
@@ -1,69 +0,0 @@
-def raises(excp, func, *args):
-    try:
-        func(*args)
-        assert 1 == 0
-    except excp:pass
-
-def assertEqual(a, b):
-    assert a == b
-
-def assertNotEqual(a, b):
-    assert a != b
-
-def assertIs(a, b):
-    assert a is b
-
-# complex specific tests
-
-EPS = 1e-9
-
-def assertAlmostEqual(a, b):
-    if isinstance(a, complex):
-        if isinstance(b, complex):
-            assert a.real - b.real < EPS
-            assert a.imag - b.imag < EPS
-        else:
-            assert a.real - b < EPS
-            assert a.imag < EPS
-    else:
-        if isinstance(b, complex):
-            assert a - b.real < EPS
-            assert b.imag < EPS
-        else:
-            assert a - b < EPS
-
-def assertCloseAbs(x, y, eps=1e-9):
-    """Return true iff floats x and y "are close\""""
-    # put the one with larger magnitude second
-    if abs(x) > abs(y):
-        x, y = y, x
-    if y == 0:
-        return abs(x) < eps
-    if x == 0:
-        return abs(y) < eps
-    # check that relative difference < eps
-    assert abs((x-y)/y) < eps
-
-def assertClose(x, y, eps=1e-9):
-    """Return true iff complexes x and y "are close\""""
-    assertCloseAbs(x.real, y.real, eps)
-    assertCloseAbs(x.imag, y.imag, eps)
-
-
-def check_div(x, y):
-    """Compute complex z=x*y, and check that z/x==y and z/y==x."""
-    z = x * y
-    if x != 0:
-        q = z / x
-        assertClose(q, y)
-        q = z.__div__(x)
-        assertClose(q, y)
-        q = z.__truediv__(x)
-        assertClose(q, y)
-    if y != 0:
-        q = z / y
-        assertClose(q, x)
-        q = z.__div__(y)
-        assertClose(q, x)
-        q = z.__truediv__(y)
-        assertClose(q, x)

diff --git a/pypy/jit/metainterp/blackhole.py b/pypy/jit/metainterp/blackhole.py
--- a/pypy/jit/metainterp/blackhole.py
+++ b/pypy/jit/metainterp/blackhole.py
@@ -7,7 +7,7 @@
 from pypy.rpython.lltypesystem.lloperation import llop
 from pypy.rpython.llinterp import LLException
 from pypy.jit.codewriter.jitcode import JitCode, SwitchDictDescr
-from pypy.jit.codewriter import heaptracker
+from pypy.jit.codewriter import heaptracker, longlong
 from pypy.jit.metainterp.jitexc import JitException, get_llexception, reraise
 from pypy.jit.metainterp.compile import ResumeAtPositionDescr
 
@@ -204,7 +204,7 @@
                 assert argcodes[next_argcode] == '>'
                 assert argcodes[next_argcode + 1] == 'f'
                 next_argcode = next_argcode + 2
-                assert lltype.typeOf(result) is lltype.Float
+                assert lltype.typeOf(result) is longlong.FLOATSTORAGE
                 self.registers_f[ord(code[position])] = result
                 position += 1
             elif resulttype == 'L':
@@ -252,7 +252,7 @@
         if we_are_translated():
             default_i = 0
             default_r = NULL
-            default_f = 0.0
+            default_f = longlong.ZEROF
         else:
             default_i = MissingValue()
             default_r = MissingValue()
@@ -281,12 +281,15 @@
         self.position = position
 
     def setarg_i(self, index, value):
+        assert lltype.typeOf(value) is lltype.Signed
         self.registers_i[index] = value
 
     def setarg_r(self, index, value):
+        assert lltype.typeOf(value) == llmemory.GCREF
         self.registers_r[index] = value
 
     def setarg_f(self, index, value):
+        assert lltype.typeOf(value) is longlong.FLOATSTORAGE
         self.registers_f[index] = value
 
     def run(self):
@@ -535,52 +538,82 @@
 
     @arguments("f", returns="f")
     def bhimpl_float_neg(a):
-        return -a
+        a = longlong.getrealfloat(a)
+        x = -a
+        return longlong.getfloatstorage(x)
     @arguments("f", returns="f")
     def bhimpl_float_abs(a):
-        return abs(a)
+        a = longlong.getrealfloat(a)
+        x = abs(a)
+        return longlong.getfloatstorage(x)
 
     @arguments("f", "f", returns="f")
     def bhimpl_float_add(a, b):
-        return a + b
+        a = longlong.getrealfloat(a)
+        b = longlong.getrealfloat(b)
+        x = a + b
+        return longlong.getfloatstorage(x)
     @arguments("f", "f", returns="f")
     def bhimpl_float_sub(a, b):
-        return a - b
+        a = longlong.getrealfloat(a)
+        b = longlong.getrealfloat(b)
+        x = a - b
+        return longlong.getfloatstorage(x)
     @arguments("f", "f", returns="f")
     def bhimpl_float_mul(a, b):
-        return a * b
+        a = longlong.getrealfloat(a)
+        b = longlong.getrealfloat(b)
+        x = a * b
+        return longlong.getfloatstorage(x)
     @arguments("f", "f", returns="f")
     def bhimpl_float_truediv(a, b):
-        return a / b
+        a = longlong.getrealfloat(a)
+        b = longlong.getrealfloat(b)
+        x = a / b
+        return longlong.getfloatstorage(x)
 
     @arguments("f", "f", returns="i")
     def bhimpl_float_lt(a, b):
+        a = longlong.getrealfloat(a)
+        b = longlong.getrealfloat(b)
         return a < b
     @arguments("f", "f", returns="i")
     def bhimpl_float_le(a, b):
+        a = longlong.getrealfloat(a)
+        b = longlong.getrealfloat(b)
         return a <= b
     @arguments("f", "f", returns="i")
     def bhimpl_float_eq(a, b):
+        a = longlong.getrealfloat(a)
+        b = longlong.getrealfloat(b)
         return a == b
     @arguments("f", "f", returns="i")
     def bhimpl_float_ne(a, b):
+        a = longlong.getrealfloat(a)
+        b = longlong.getrealfloat(b)
         return a != b
     @arguments("f", "f", returns="i")
     def bhimpl_float_gt(a, b):
+        a = longlong.getrealfloat(a)
+        b = longlong.getrealfloat(b)
         return a > b
     @arguments("f", "f", returns="i")
     def bhimpl_float_ge(a, b):
+        a = longlong.getrealfloat(a)
+        b = longlong.getrealfloat(b)
         return a >= b
 
     @arguments("f", returns="i")
     def bhimpl_cast_float_to_int(a):
+        a = longlong.getrealfloat(a)
         # note: we need to call int() twice to care for the fact that
         # int(-2147483648.0) returns a long :-(
         return int(int(a))
 
     @arguments("i", returns="f")
     def bhimpl_cast_int_to_float(a):
-        return float(a)
+        x = float(a)
+        return longlong.getfloatstorage(x)
 
     # ----------
     # control flow operations
@@ -1262,10 +1295,13 @@
     # connect the return of values from the called frame to the
     # 'xxx_call_yyy' instructions from the caller frame
     def _setup_return_value_i(self, result):
+        assert lltype.typeOf(result) is lltype.Signed
         self.registers_i[ord(self.jitcode.code[self.position-1])] = result
     def _setup_return_value_r(self, result):
+        assert lltype.typeOf(result) == llmemory.GCREF
         self.registers_r[ord(self.jitcode.code[self.position-1])] = result
     def _setup_return_value_f(self, result):
+        assert lltype.typeOf(result) is longlong.FLOATSTORAGE
         self.registers_f[ord(self.jitcode.code[self.position-1])] = result
 
     def _done_with_this_frame(self):
@@ -1329,7 +1365,7 @@
         for i in range(self.jitcode.num_regs_f()):
             box = miframe.registers_f[i]
             if box is not None:
-                self.setarg_f(i, box.getfloat())
+                self.setarg_f(i, box.getfloatstorage())
 
 # ____________________________________________________________
 

diff --git a/.hgignore b/.hgignore
--- a/.hgignore
+++ b/.hgignore
@@ -17,6 +17,7 @@
 ^pypy/doc/.+\.html$
 ^pypy/doc/basicblock\.asc$
 ^pypy/doc/.+\.svninfo$
+^pypy/translator/c/src/dtoa.o$
 ^pypy/translator/c/src/libffi_msvc/.+\.obj$
 ^pypy/translator/c/src/libffi_msvc/.+\.dll$
 ^pypy/translator/c/src/libffi_msvc/.+\.lib$
@@ -60,4 +61,4 @@
 ^pypy/doc/image/lattice3\.png$
 ^pypy/doc/image/stackless_informal\.png$
 ^pypy/doc/image/parsing_example.+\.png$
-^compiled
\ No newline at end of file
+^compiled

diff --git a/pypy/jit/backend/llsupport/llmodel.py b/pypy/jit/backend/llsupport/llmodel.py
--- a/pypy/jit/backend/llsupport/llmodel.py
+++ b/pypy/jit/backend/llsupport/llmodel.py
@@ -7,7 +7,7 @@
 from pypy.jit.metainterp.history import BoxInt, BoxPtr, set_future_values,\
      BoxFloat
 from pypy.jit.metainterp import history
-from pypy.jit.codewriter import heaptracker
+from pypy.jit.codewriter import heaptracker, longlong
 from pypy.jit.backend.model import AbstractCPU
 from pypy.jit.backend.llsupport import symbolic
 from pypy.jit.backend.llsupport.symbolic import WORD, unroll_basic_sizes
@@ -315,7 +315,7 @@
         ofs = self.unpack_arraydescr(arraydescr)
         # --- start of GC unsafe code (no GC operation!) ---
         items = rffi.ptradd(rffi.cast(rffi.CCHARP, gcref), ofs)
-        items = rffi.cast(rffi.CArrayPtr(lltype.Float), items)
+        items = rffi.cast(rffi.CArrayPtr(longlong.FLOATSTORAGE), items)
         fval = items[itemindex]
         # --- end of GC unsafe code ---
         return fval
@@ -348,7 +348,7 @@
         ofs = self.unpack_arraydescr(arraydescr)
         # --- start of GC unsafe code (no GC operation!) ---
         items = rffi.ptradd(rffi.cast(rffi.CCHARP, gcref), ofs)
-        items = rffi.cast(rffi.CArrayPtr(lltype.Float), items)
+        items = rffi.cast(rffi.CArrayPtr(longlong.FLOATSTORAGE), items)
         items[itemindex] = newvalue
         # --- end of GC unsafe code ---
 
@@ -410,7 +410,7 @@
         ofs = self.unpack_fielddescr(fielddescr)
         # --- start of GC unsafe code (no GC operation!) ---
         fieldptr = rffi.ptradd(rffi.cast(rffi.CCHARP, struct), ofs)
-        fval = rffi.cast(rffi.CArrayPtr(lltype.Float), fieldptr)[0]
+        fval = rffi.cast(rffi.CArrayPtr(longlong.FLOATSTORAGE), fieldptr)[0]
         # --- end of GC unsafe code ---
         return fval
 
@@ -452,7 +452,7 @@
         ofs = self.unpack_fielddescr(fielddescr)
         # --- start of GC unsafe code (no GC operation!) ---
         fieldptr = rffi.ptradd(rffi.cast(rffi.CCHARP, struct), ofs)
-        fieldptr = rffi.cast(rffi.CArrayPtr(lltype.Float), fieldptr)
+        fieldptr = rffi.cast(rffi.CArrayPtr(longlong.FLOATSTORAGE), fieldptr)
         fieldptr[0] = newvalue
         # --- end of GC unsafe code ---
 
@@ -509,9 +509,9 @@
         return calldescr.call_stub(func, args_i, args_r, args_f)
 
     def bh_call_f(self, func, calldescr, args_i, args_r, args_f):
-        assert isinstance(calldescr, FloatCallDescr)
+        assert isinstance(calldescr, FloatCallDescr)  # or LongLongCallDescr
         if not we_are_translated():
-            calldescr.verify_types(args_i, args_r, args_f, history.FLOAT)
+            calldescr.verify_types(args_i, args_r, args_f, history.FLOAT + 'L')
         return calldescr.call_stub(func, args_i, args_r, args_f)
 
     def bh_call_v(self, func, calldescr, args_i, args_r, args_f):

diff --git a/pypy/jit/metainterp/compile.py b/pypy/jit/metainterp/compile.py
--- a/pypy/jit/metainterp/compile.py
+++ b/pypy/jit/metainterp/compile.py
@@ -15,7 +15,7 @@
 from pypy.jit.metainterp.typesystem import llhelper, oohelper
 from pypy.jit.metainterp.optimizeutil import InvalidLoop
 from pypy.jit.metainterp.resume import NUMBERING
-from pypy.jit.codewriter import heaptracker
+from pypy.jit.codewriter import heaptracker, longlong
 
 def giveup():
     from pypy.jit.metainterp.pyjitpl import SwitchToBlackhole
@@ -528,7 +528,7 @@
 class ResumeGuardCountersFloat(AbstractResumeGuardCounters):
     def __init__(self):
         self.counters = [0] * 5
-        self.values = [0.0] * 5
+        self.values = [longlong.ZEROF] * 5
     see_float = func_with_new_name(_see, 'see_float')
 
 

diff --git a/pypy/jit/metainterp/test/test_resume.py b/pypy/jit/metainterp/test/test_resume.py
--- a/pypy/jit/metainterp/test/test_resume.py
+++ b/pypy/jit/metainterp/test/test_resume.py
@@ -8,7 +8,7 @@
 from pypy.jit.metainterp.history import ConstPtr, ConstFloat
 from pypy.jit.metainterp.test.test_optimizeutil import LLtypeMixin
 from pypy.jit.metainterp import executor
-from pypy.jit.codewriter import heaptracker
+from pypy.jit.codewriter import heaptracker, longlong
 
 class Storage:
     rd_frame_info_list = None
@@ -114,7 +114,7 @@
                         callback_i(index, count_i); count_i += 1
                     elif ARG == llmemory.GCREF:
                         callback_r(index, count_r); count_r += 1
-                    elif ARG == lltype.Float:
+                    elif ARG == longlong.FLOATSTORAGE:
                         callback_f(index, count_f); count_f += 1
                     else:
                         assert 0
@@ -137,7 +137,8 @@
     reader.consume_one_section(bh)
     expected_i = [x for x in expected if lltype.typeOf(x) == lltype.Signed]
     expected_r = [x for x in expected if lltype.typeOf(x) == llmemory.GCREF]
-    expected_f = [x for x in expected if lltype.typeOf(x) == lltype.Float]
+    expected_f = [x for x in expected if lltype.typeOf(x) ==
+                                                      longlong.FLOATSTORAGE]
     assert bh.written_i == expected_i
     assert bh.written_r == expected_r
     assert bh.written_f == expected_f
@@ -749,7 +750,7 @@
 
 def test_ResumeDataLoopMemo_other():
     memo = ResumeDataLoopMemo(FakeMetaInterpStaticData())
-    const = ConstFloat(-1.0)
+    const = ConstFloat(longlong.getfloatstorage(-1.0))
     tagged = memo.getconst(const)
     index, tagbits = untag(tagged)
     assert tagbits == TAGCONST

diff --git a/pypy/jit/metainterp/warmspot.py b/pypy/jit/metainterp/warmspot.py
--- a/pypy/jit/metainterp/warmspot.py
+++ b/pypy/jit/metainterp/warmspot.py
@@ -23,7 +23,7 @@
 from pypy.jit.metainterp.jitprof import Profiler, EmptyProfiler
 from pypy.jit.metainterp.jitexc import JitException
 from pypy.jit.metainterp.jitdriver import JitDriverStaticData
-from pypy.jit.codewriter import support, codewriter
+from pypy.jit.codewriter import support, codewriter, longlong
 from pypy.jit.codewriter.policy import JitPolicy
 
 # ____________________________________________________________
@@ -344,7 +344,7 @@
 
         class DoneWithThisFrameFloat(JitException):
             def __init__(self, result):
-                assert lltype.typeOf(result) is lltype.Float
+                assert lltype.typeOf(result) is longlong.FLOATSTORAGE
                 self.result = result
             def __str__(self):
                 return 'DoneWithThisFrameFloat(%s)' % (self.result,)

diff --git a/pypy/jit/metainterp/resume.py b/pypy/jit/metainterp/resume.py
--- a/pypy/jit/metainterp/resume.py
+++ b/pypy/jit/metainterp/resume.py
@@ -1007,17 +1007,16 @@
             return len(numb.nums)
         index = len(numb.nums) - 1
         virtualizable = self.decode_ref(numb.nums[index])
-        virtualizable = vinfo.cast_gcref_to_vtype(virtualizable)
         if self.resume_after_guard_not_forced == 1:
             # in the middle of handle_async_forcing()
-            assert virtualizable.vable_token
-            virtualizable.vable_token = vinfo.TOKEN_NONE
+            assert vinfo.gettoken(virtualizable)
+            vinfo.settoken(virtualizable, vinfo.TOKEN_NONE)
         else:
             # just jumped away from assembler (case 4 in the comment in
             # virtualizable.py) into tracing (case 2); check that vable_token
             # is and stays 0.  Note the call to reset_vable_token() in
             # warmstate.py.
-            assert not virtualizable.vable_token
+            assert not vinfo.gettoken(virtualizable)
         return vinfo.write_from_resume_data_partial(virtualizable, self, numb)
 
     def load_value_of_type(self, TYPE, tagged):
@@ -1158,7 +1157,7 @@
     def decode_float(self, tagged):
         num, tag = untag(tagged)
         if tag == TAGCONST:
-            return self.consts[num].getfloat()
+            return self.consts[num].getfloatstorage()
         else:
             assert tag == TAGBOX
             if num < 0:

diff --git a/pypy/jit/backend/model.py b/pypy/jit/backend/model.py
--- a/pypy/jit/backend/model.py
+++ b/pypy/jit/backend/model.py
@@ -189,12 +189,6 @@
     def typedescrof(TYPE):
         raise NotImplementedError
 
-    #def cast_adr_to_int(self, adr):
-    #    raise NotImplementedError
-
-    #def cast_int_to_adr(self, int):
-    #    raise NotImplementedError
-
     # ---------- the backend-dependent operations ----------
 
     # lltype specific operations

diff --git a/pypy/module/signal/interp_signal.py b/pypy/module/signal/interp_signal.py
--- a/pypy/module/signal/interp_signal.py
+++ b/pypy/module/signal/interp_signal.py
@@ -1,5 +1,5 @@
 from __future__ import with_statement
-from pypy.interpreter.error import OperationError
+from pypy.interpreter.error import OperationError, exception_from_errno
 from pypy.interpreter.baseobjspace import W_Root, ObjSpace
 from pypy.interpreter.executioncontext import AsyncAction, AbstractActionFlag
 from pypy.interpreter.executioncontext import PeriodicAsyncAction
@@ -35,6 +35,7 @@
     include_dirs = [str(py.path.local(autopath.pypydir).join('translator', 'c'))],
     export_symbols = ['pypysig_poll', 'pypysig_default',
                       'pypysig_ignore', 'pypysig_setflag',
+                      'pypysig_reinstall',
                       'pypysig_set_wakeup_fd',
                       'pypysig_getaddr_occurred'],
 )
@@ -65,6 +66,7 @@
 pypysig_ignore = external('pypysig_ignore', [rffi.INT], lltype.Void)
 pypysig_default = external('pypysig_default', [rffi.INT], lltype.Void)
 pypysig_setflag = external('pypysig_setflag', [rffi.INT], lltype.Void)
+pypysig_reinstall = external('pypysig_reinstall', [rffi.INT], lltype.Void)
 pypysig_set_wakeup_fd = external('pypysig_set_wakeup_fd', [rffi.INT], rffi.INT)
 pypysig_poll = external('pypysig_poll', [], rffi.INT, threadsafe=False)
 # don't bother releasing the GIL around a call to pypysig_poll: it's
@@ -150,7 +152,7 @@
         except KeyError:
             return    # no handler, ignore signal
         # re-install signal handler, for OSes that clear it
-        pypysig_setflag(n)
+        pypysig_reinstall(n)
         # invoke the app-level handler
         space = self.space
         ec = space.getexecutioncontext()
@@ -303,6 +305,10 @@
     w_interval = space.wrap(double_from_timeval(val.c_it_interval))
     return space.newtuple([w_value, w_interval])
 
+def get_itimer_error(space):
+    mod = space.getbuiltinmodule("signal")
+    return space.getattr(mod, space.wrap("ItimerError"))
+
 @jit.dont_look_inside
 @unwrap_spec(ObjSpace, int, float, float)
 def setitimer(space, which, first, interval=0):
@@ -313,7 +319,10 @@
 
         with lltype.scoped_alloc(itimervalP.TO, 1) as old:
 
-            c_setitimer(which, new, old)
+            ret = c_setitimer(which, new, old)
+            if ret != 0:
+                raise exception_from_errno(space, get_itimer_error(space))
+
 
             return itimer_retval(space, old[0])
 

diff --git a/pypy/jit/tool/oparser.py b/pypy/jit/tool/oparser.py
--- a/pypy/jit/tool/oparser.py
+++ b/pypy/jit/tool/oparser.py
@@ -9,6 +9,7 @@
 from pypy.jit.metainterp.resoperation import rop, ResOperation, ResOpWithDescr, N_aryOp
 from pypy.jit.metainterp.typesystem import llhelper
 from pypy.jit.codewriter.heaptracker import adr2int
+from pypy.jit.codewriter import longlong
 from pypy.rpython.lltypesystem import lltype, llmemory
 from pypy.rpython.ootypesystem import ootype
 
@@ -160,7 +161,7 @@
             return ConstInt(int(arg))
         except ValueError:
             if self.is_float(arg):
-                return ConstFloat(float(arg))
+                return ConstFloat(longlong.getfloatstorage(float(arg)))
             if (arg.startswith('"') or arg.startswith("'") or
                 arg.startswith('s"')):
                 # XXX ootype

diff --git a/pypy/jit/backend/x86/assembler.py b/pypy/jit/backend/x86/assembler.py
--- a/pypy/jit/backend/x86/assembler.py
+++ b/pypy/jit/backend/x86/assembler.py
@@ -38,6 +38,7 @@
 from pypy.jit.backend.x86.jump import remap_frame_layout
 from pypy.jit.metainterp.history import ConstInt, BoxInt
 from pypy.jit.codewriter.effectinfo import EffectInfo
+from pypy.jit.codewriter import longlong
 
 # darwin requires the stack to be 16 bytes aligned on calls. Same for gcc 4.5.0,
 # better safe than sorry
@@ -71,7 +72,8 @@
         self.malloc_unicode_func_addr = 0
         self.fail_boxes_int = values_array(lltype.Signed, failargs_limit)
         self.fail_boxes_ptr = values_array(llmemory.GCREF, failargs_limit)
-        self.fail_boxes_float = values_array(lltype.Float, failargs_limit)
+        self.fail_boxes_float = values_array(longlong.FLOATSTORAGE,
+                                             failargs_limit)
         self.fail_ebp = 0
         self.loop_run_counters = []
         self.float_const_neg_addr = 0
@@ -1151,24 +1153,11 @@
             self.mc.MOVD_xr(loc2.value, edx.value)
             self.mc.PUNPCKLDQ_xx(resloc.value, loc2.value)
 
-    def genop_llong_from_two_ints(self, op, arglocs, resloc):
+    def genop_llong_from_uint(self, op, arglocs, resloc):
+        loc1, = arglocs
         assert isinstance(resloc, RegLoc)
-        loc1, loc2, loc3 = arglocs
-        #
-        if isinstance(loc1, ConstFloatLoc):
-            self.mc.MOVSD(resloc, loc1)
-        else:
-            assert isinstance(loc1, RegLoc)
-            self.mc.MOVD_xr(resloc.value, loc1.value)
-        #
-        if loc2 is not None:
-            assert isinstance(loc3, RegLoc)
-            if isinstance(loc2, ConstFloatLoc):
-                self.mc.MOVSD(loc3, loc2)
-            else:
-                assert isinstance(loc2, RegLoc)
-                self.mc.MOVD_xr(loc3.value, loc2.value)
-            self.mc.PUNPCKLDQ_xx(resloc.value, loc3.value)
+        assert isinstance(loc1, RegLoc)
+        self.mc.MOVD_xr(resloc.value, loc1.value)
 
     def genop_llong_eq(self, op, arglocs, resloc):
         loc1, loc2, locxtmp = arglocs
@@ -1814,11 +1803,13 @@
 
         if IS_X86_32 and isinstance(resloc, StackLoc) and resloc.width == 8:
             # a float or a long long return
-            from pypy.jit.backend.llsupport.descr import LongLongCallDescr
-            if isinstance(op.getdescr(), LongLongCallDescr):
+            if op.getdescr().get_return_type() == 'L':
                 self.mc.MOV_br(resloc.value, eax.value)      # long long
                 self.mc.MOV_br(resloc.value + 4, edx.value)
-                # XXX should ideally not move the result on the stack
+                # XXX should ideally not move the result on the stack,
+                #     but it's a mess to load eax/edx into a xmm register
+                #     and this way is simpler also because the result loc
+                #     can just be always a stack location
             else:
                 self.mc.FSTP_b(resloc.value)   # float return
         elif size == WORD:

diff --git a/pypy/jit/codewriter/support.py b/pypy/jit/codewriter/support.py
--- a/pypy/jit/codewriter/support.py
+++ b/pypy/jit/codewriter/support.py
@@ -300,9 +300,8 @@
 def _ll_1_llong_from_int(x):
     return r_longlong(intmask(x))
 
-def _ll_2_llong_from_two_ints(x_lo, x_hi):
-    z = (r_ulonglong(r_uint(x_hi)) << 32) | r_ulonglong(r_uint(x_lo))
-    return u_to_longlong(z)
+def _ll_1_llong_from_uint(x):
+    return r_longlong(r_uint(x))
 
 def _ll_1_llong_to_int(xll):
     return intmask(xll)

diff --git a/pypy/translator/tool/cbuild.py b/pypy/translator/tool/cbuild.py
--- a/pypy/translator/tool/cbuild.py
+++ b/pypy/translator/tool/cbuild.py
@@ -281,7 +281,8 @@
                 return self
             basepath = udir.join('module_cache')
         else:
-            basepath = py.path.local(self.separate_module_files[0]).dirpath()
+            #basepath = py.path.local(self.separate_module_files[0]).dirpath()
+            basepath = udir.join('shared_cache')
         if outputfilename is None:
             # find more or less unique name there
             pth = basepath.join('externmod').new(ext=host.so_ext)
@@ -290,7 +291,8 @@
                 pth = basepath.join(
                     'externmod_%d' % (num,)).new(ext=host.so_ext)
                 num += 1
-            outputfilename=pth.purebasename
+            basepath.ensure(dir=1)
+            outputfilename = str(pth.dirpath().join(pth.purebasename))
         lib = str(host.compile([], self, outputfilename=outputfilename,
                                standalone=False))
         d = self._copy_attributes()

From commits-noreply at bitbucket.org  Wed Feb 16 11:50:44 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Wed, 16 Feb 2011 11:50:44 +0100 (CET)
Subject: [pypy-svn] pypy default: make sure that the co_filename attribute
	of code	objects inside zip files is in the form
	myfile.zip/mymodule.py
Message-ID: <20110216105044.6C043282BEA@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42032:f1f804016710
Date: 2011-02-16 11:16 +0100
http://bitbucket.org/pypy/pypy/changeset/f1f804016710/

Log:	make sure that the co_filename attribute of code objects inside zip
	files is in the form myfile.zip/mymodule.py

diff --git a/pypy/module/zipimport/interp_zipimport.py b/pypy/module/zipimport/interp_zipimport.py
--- a/pypy/module/zipimport/interp_zipimport.py
+++ b/pypy/module/zipimport/interp_zipimport.py
@@ -149,7 +149,8 @@
         real_name = self.filename + os.path.sep + self.corr_zname(filename)
         space.setattr(w_mod, w('__loader__'), space.wrap(self))
         importing._prepare_module(space, w_mod, real_name, pkgpath)
-        code_w = importing.parse_source_module(space, filename, buf)
+        co_filename = self.make_co_filename(filename)
+        code_w = importing.parse_source_module(space, co_filename, buf)
         importing.exec_code_module(space, w_mod, code_w)
         return w_mod
 
@@ -231,6 +232,14 @@
         subname = fullname[startpos:]
         return self.prefix + subname.replace('.', '/')
 
+    def make_co_filename(self, filename):
+        """
+        Return the filename to be used for compiling the module, i.e. what
+        gets in code_object.co_filename. Something like
+        'myfile.zip/mymodule.py'
+        """
+        return self.filename + os.path.sep + filename
+
     def load_module(self, space, fullname):
         w = space.wrap
         filename = self.make_filename(fullname)
@@ -297,8 +306,9 @@
                     code_w = importing.read_compiled_module(
                         space, filename + ext, source[8:])
                 else:
+                    co_filename = self.make_co_filename(filename+ext)
                     code_w = importing.parse_source_module(
-                        space, filename + ext, source)
+                        space, co_filename, source)
                 return space.wrap(code_w)
         raise operationerrfmt(self.w_ZipImportError,
             "Cannot find source or code for %s in %s", filename, self.name)

diff --git a/pypy/module/zipimport/test/test_zipimport.py b/pypy/module/zipimport/test/test_zipimport.py
--- a/pypy/module/zipimport/test/test_zipimport.py
+++ b/pypy/module/zipimport/test/test_zipimport.py
@@ -34,6 +34,10 @@
         return pyc
     make_pyc = classmethod(make_pyc)
     def make_class(cls):
+        # XXX: this is (mostly) wrong: .compile() compiles the code object
+        # using the host python compiler, but then in the tests we load it
+        # with py.py. It works (mostly by chance) because the two functions
+        # are very simple and the bytecodes are compatible enough.
         co = py.code.Source("""
         def get_name():
             return __name__
@@ -338,6 +342,25 @@
         import zipimport
         assert sys.path_hooks.count(zipimport.zipimporter) == 1
 
+    def test_co_filename(self):
+        self.writefile('mymodule.py', """
+def get_co_filename():
+    return get_co_filename.func_code.co_filename
+""")
+        import os
+        expected = self.zipfile + os.sep + 'mymodule.py'
+        #
+        import mymodule
+        co_filename = mymodule.get_co_filename()
+        assert co_filename == expected
+        #
+        import zipimport
+        z = zipimport.zipimporter(self.zipfile)
+        code = z.get_code('mymodule')
+        co_filename = code.co_filename
+        assert co_filename == expected
+
+
 class AppTestZipimportDeflated(AppTestZipimport):
     compression = ZIP_DEFLATED
 

From commits-noreply at bitbucket.org  Wed Feb 16 14:14:15 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Wed, 16 Feb 2011 14:14:15 +0100 (CET)
Subject: [pypy-svn] pypy default: bah,
	hack the lib-python conftest failure detection logic,
	else it uncorrectly
Message-ID: <20110216131415.C3919282BEC@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42033:b4e9b52d2357
Date: 2011-02-16 14:13 +0100
http://bitbucket.org/pypy/pypy/changeset/b4e9b52d2357/

Log:	bah, hack the lib-python conftest failure detection logic, else it
	uncorrectly reports as failing a test that is passing but contains
	the string "FAILURE"

diff --git a/lib-python/conftest.py b/lib-python/conftest.py
--- a/lib-python/conftest.py
+++ b/lib-python/conftest.py
@@ -716,8 +716,10 @@
         if test_stderr.rfind(26*"=" + "skipped" + 26*"=") != -1:
             skipped = True
         outcome = 'OK'
-        if not exit_status: 
-            if 'FAIL' in test_stdout or re.search('[^:]ERROR', test_stderr):
+        if not exit_status:
+            # match "FAIL" but not e.g. "FAILURE", which is in the output of a
+            # test in test_zipimport_support.py
+            if re.search(r'\bFAIL\b', test_stdout) or re.search('[^:]ERROR', test_stderr):
                 outcome = 'FAIL'
                 exit_status = 2  
         elif timedout: 

From commits-noreply at bitbucket.org  Wed Feb 16 14:20:56 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Wed, 16 Feb 2011 14:20:56 +0100 (CET)
Subject: [pypy-svn] pypy arm-backend-2: (arigo,
	david) when generating code for a call,
	force the variable stored in r0 to be spilled if the call has
	a	return value and ensure a register can not be bound to
	two	boxes after a call.
Message-ID: <20110216132056.9BC0D282BEC@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backend-2
Changeset: r42034:95434737666e
Date: 2011-02-16 14:20 +0100
http://bitbucket.org/pypy/pypy/changeset/95434737666e/

Log:	(arigo, david) when generating code for a call, force the variable
	stored in r0 to be spilled if the call has a return value and ensure
	a register can not be bound to two boxes after a call.

diff --git a/pypy/jit/backend/arm/opassembler.py b/pypy/jit/backend/arm/opassembler.py
--- a/pypy/jit/backend/arm/opassembler.py
+++ b/pypy/jit/backend/arm/opassembler.py
@@ -268,9 +268,13 @@
             regalloc.before_call(save_all_regs=spill_all_regs)
         else:
             if result:
-                # XXX maybe move instance check to llsupport/regalloc
-                if reg_args > 0 and isinstance(args[0], Box) and regalloc.stays_alive(args[0]):
-                    regalloc.force_spill_var(args[0])
+                # XXX hack if the call has a result force the value in r0 to be
+                # spilled
+                if reg_args == 0 or (isinstance(args[0], Box) and
+                        regalloc.stays_alive(args[0])):
+                    t = TempBox()
+                    regalloc.force_allocate_reg(t, selected_reg=regalloc.call_result_location(t))
+                    regalloc.possibly_free_var(t)
                 self.mc.PUSH([reg.value for reg in r.caller_resp][1:])
             else:
                 self.mc.PUSH([reg.value for reg in r.caller_resp])

diff --git a/pypy/jit/backend/llsupport/regalloc.py b/pypy/jit/backend/llsupport/regalloc.py
--- a/pypy/jit/backend/llsupport/regalloc.py
+++ b/pypy/jit/backend/llsupport/regalloc.py
@@ -324,6 +324,8 @@
         """
         self._check_type(v)
         r = self.call_result_location(v)
+        if not we_are_translated():
+            assert r not in self.reg_bindings.values()
         self.reg_bindings[v] = r
         self.free_regs = [fr for fr in self.free_regs if fr is not r]
         return r

From commits-noreply at bitbucket.org  Wed Feb 16 14:37:19 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed, 16 Feb 2011 14:37:19 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix 7 of the 8 failures of test_pypy_c
	just by not	printing the args, which
Message-ID: <20110216133719.C43692A202C@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r42035:ae7f5eddf443
Date: 2011-02-16 14:37 +0100
http://bitbucket.org/pypy/pypy/changeset/ae7f5eddf443/

Log:	Fix 7 of the 8 failures of test_pypy_c just by not printing the
	args, which ends up in the jit compiling the loop of listrepr()...

diff --git a/pypy/module/pypyjit/test/test_pypy_c.py b/pypy/module/pypyjit/test/test_pypy_c.py
--- a/pypy/module/pypyjit/test/test_pypy_c.py
+++ b/pypy/module/pypyjit/test/test_pypy_c.py
@@ -105,9 +105,9 @@
                 pass
 
             def check(args, expected):
-                print >> sys.stderr, 'trying:', args
+                #print >> sys.stderr, 'trying:', args
                 result = main(*args)
-                print >> sys.stderr, 'got:', repr(result)
+                #print >> sys.stderr, 'got:', repr(result)
                 assert result == expected
                 assert type(result) is type(expected)
         """ % threshold)

From commits-noreply at bitbucket.org  Wed Feb 16 15:20:25 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Wed, 16 Feb 2011 15:20:25 +0100 (CET)
Subject: [pypy-svn] virtualenv-pypy trunk: ignore exec_prefix for pypy. This
	fixes a	crash that you get when running virtualenv inside a
	pypy virtualenv
Message-ID: <20110216142025.6F139282BEC@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: trunk
Changeset: r363:352a9bf21c85
Date: 2010-12-09 19:26 +0100
http://bitbucket.org/pypy/virtualenv-pypy/changeset/352a9bf21c85/

Log:	ignore exec_prefix for pypy. This fixes a crash that you get when
	running virtualenv inside a pypy virtualenv

diff --git a/virtualenv.py b/virtualenv.py
--- a/virtualenv.py
+++ b/virtualenv.py
@@ -799,7 +799,8 @@
     else:
         logger.debug('No include dir %s' % stdinc_dir)
 
-    if sys.exec_prefix != prefix:
+    # pypy never uses exec_prefix, just ignore it
+    if sys.exec_prefix != prefix and not is_pypy:
         if sys.platform == 'win32':
             exec_dir = join(sys.exec_prefix, 'lib')
         elif is_jython:

From commits-noreply at bitbucket.org  Wed Feb 16 15:20:25 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Wed, 16 Feb 2011 15:20:25 +0100 (CET)
Subject: [pypy-svn] virtualenv-pypy trunk: use the same executable name as
	you get with	pypy releases
Message-ID: <20110216142025.C6A11282BF2@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: trunk
Changeset: r364:95a9f95cfa1b
Date: 2010-12-09 19:27 +0100
http://bitbucket.org/pypy/virtualenv-pypy/changeset/95a9f95cfa1b/

Log:	use the same executable name as you get with pypy releases

diff --git a/virtualenv.py b/virtualenv.py
--- a/virtualenv.py
+++ b/virtualenv.py
@@ -34,7 +34,7 @@
 is_pypy = hasattr(sys, 'pypy_version_info')
 
 if is_pypy:
-    expected_exe = 'pypy-c'
+    expected_exe = 'pypy'
 elif is_jython:
     expected_exe = 'jython'
 else:

From commits-noreply at bitbucket.org  Wed Feb 16 15:24:46 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Wed, 16 Feb 2011 15:24:46 +0100 (CET)
Subject: [pypy-svn] pypy default: add the subrepo virtualenv-pypy inside
	contrib
Message-ID: <20110216142446.31335282BEC@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42036:8fd190160a21
Date: 2011-02-16 15:23 +0100
http://bitbucket.org/pypy/pypy/changeset/8fd190160a21/

Log:	add the subrepo virtualenv-pypy inside contrib

diff --git a/.hgsub b/.hgsub
--- a/.hgsub
+++ b/.hgsub
@@ -1,3 +1,4 @@
 greenlet = [svn]http://codespeak.net/svn/greenlet/trunk/c
 testrunner = [svn]http://codespeak.net/svn/pypy/build/testrunner
 lib_pypy/pyrepl = [svn]http://codespeak.net/svn/pyrepl/trunk/pyrepl/pyrepl
+contrib/virtualenv-pypy = https://bitbucket.org/pypy/virtualenv-pypy

diff --git a/.hgsubstate b/.hgsubstate
--- a/.hgsubstate
+++ b/.hgsubstate
@@ -1,3 +1,4 @@
+95a9f95cfa1bad8f87f5866edb8a06159c1f38c7 contrib/virtualenv-pypy
 80037 greenlet
 80348 lib_pypy/pyrepl
 80037 testrunner

From commits-noreply at bitbucket.org  Wed Feb 16 15:36:41 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Wed, 16 Feb 2011 15:36:41 +0100 (CET)
Subject: [pypy-svn] virtualenv-pypy trunk: try hard to find all the needed
	plat-{darwin, mac} directories and put
Message-ID: <20110216143641.956E72A202C@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: trunk
Changeset: r365:792c971f4114
Date: 2010-12-10 10:34 +0100
http://bitbucket.org/pypy/virtualenv-pypy/changeset/792c971f4114/

Log:	try hard to find all the needed plat-{darwin,mac} directories and
	put the in sys.path when running on pypy.

	This fixes pypy issue604 i.e. using virtualenv --distribute with
	pypy: https://codespeak.net/issue/pypy-dev/issue604

diff --git a/virtualenv.py b/virtualenv.py
--- a/virtualenv.py
+++ b/virtualenv.py
@@ -1241,139 +1241,140 @@
 
 ##file site.py
 SITE_PY = """
-eJzVPP1z2zaWv/OvQOXJUEplOh/dzo5T98ZJnNZ7buJt0mluXY+WkiCJNUWyBGlZe3P3t9/7AECA
-pGS77f5wmkwskcDDw8P7xgMGg8FpUchsLtb5vE6lUDIuZytRxNVKiUVeimqVlPPDIi6rLTyd3cRL
-qUSVC7VVEbaKguDpH/wET8WnVaIMCvAtrqt8HVfJLE7TrUjWRV5Wci7mdZlkS5FkSZXEafIvaJFn
-kXj6xzEIzjMBM08TWYpbWSqAq0S+EJfbapVnYlgXOOfn0V/il6OxULMyKSpoUGqcgSKruAoyKeeA
-JrSsFZAyqeShKuQsWSQz23CT1+lcFGk8k+Kf/+SpUdMwDFS+lpuVLKXIABmAKQFWgXjA16QUs3wu
-IyFey1mMA/DzhlgBQxvjmikkY5aLNM+WMKdMzqRScbkVw2ldESBCWcxzwCkBDKokTYNNXt6oESwp
-rccGHomY2cOfDLMHzBPH73IO4PghC37KkrsxwwbuQXDVitmmlIvkTsQIFn7KOzmb6GfDZCHmyWIB
-NMiqETYJGAEl0mR6VNByfKNX6NsjwspyZQxjSESZG/NL6hEF55WIUwVsWxdII0WYv5XTJM6AGtkt
-DAcQgaRB3zjzRFV2HJqdyAFAietYgZSslRiu4yQDZv0hnhHaPyfZPN+oEVEAVkuJX2tVufMf9hAA
-WjsEGAe4WGY16yxNbmS6HQECnwD7Uqo6rVAg5kkpZ1VeJlIRAEBtK+QdID0WcSk1CZkzjdyOif5E
-kyTDhUUBQ4HHl0iSRbKsS5IwsUiAc4Er3n34Ubw9e31++l7zmAHGMrtcA84AhRbawQkGEEe1Ko/S
-HAQ6Ci7wj4jncxSyJY4PeDUNju5d6WAIcy+idh9nwYHsenH1MDDHCpQJjRVQv/+GLmO1Avr8zz3r
-HQSnu6hCE+dvm1UOMpnFaylWMfMXckbwjYbzbVRUq1fADQrhVEAqhYuDCCYID0ji0myYZ1IUwGJp
-kslRABSaUlt/FYEV3ufZIa11ixMAQhlk8NJ5NqIRMwkT7cJ6hfrCNN7SzHSTwK7zOi9JcQD/ZzPS
-RWmc3RCOihiKv03lMskyRAh5IQgPQhpY3STAifNIXFAr0gumkQhZe3FLFIkaeAmZDnhS3sXrIpVj
-Fl/UrfvVCA0mK2HWOmWOg5YVqVdatWaqvbz3Ivrc4jpCs1qVEoDXU0/oFnk+FlPQ2YRNEa9ZvKpN
-TpwT9MgTdUKeoJbQF78DRU+VqtfSvkReAc1CDBUs8jTNN0Cy4yAQ4gAbGaPsMye8hXfwP8DF/1NZ
-zVZB4IxkAWtQiPwuUAgETILMNFdrJDxu06zcVjJJxpoiL+eypKEeRuwjRvyBjXGuwfu80kaNp4ur
-nK+TClXSVJvMhC1eFlasH1/xvGEaYLkV0cw0bei0xumlxSqeSuOSTOUCJUEv0iu77DBm0DMm2eJK
-rNnKwDsgi0zYgvQrFlQ6i0qSEwAwWPjiLCnqlBopZDARw0DrguCvYzTpuXaWgL3ZLAeokNh8z8D+
-AG7/AjHarBKgzwwggIZBLQXLN02qEh2ERh8FvtE3/Xl84NTzhbZNPOQiTlJt5eMsOKeHZ2VJ4juT
-BfYaa2IomGFWoWu3zICOKOaDwSAIjDu0VeZrbr9NJtM6QXs3mQRVuT0G7hAo5AFDF+9hojQcv1mU
-+RpfW/Q+gj4AvYw9ggNxSYpCso/rMdMrpICrlQvTFM2vw5ECVUlw+ePZu/PPZx/FibhqtNK4rZKu
-YcyzLAbOJKUOfNEatlFH0BJ1V4LqS7wDC03rCiaJepMEyriqgf0A9U9lTa9hGjPvZXD2/vT1xdnk
-p49nP04+nn86AwTBVMjggKaMFq4Gn09FwN/AWHMVaRMZdHrQg9enH+2DYJKoSbEttvAAbB1wYTmE
-+Y5FiA8n2oxOkmyRhyNq/Cv70SesGbTTdHX81bU4ORHhr/FtHAbguDRNeRF/IB7+tC0kdK3gzzBX
-oyCYywXw+41EqRg+JWd0xB2AiNAy18bx1zzJzHt67Q1BQjukHoDDZDJLY6Ww8WQSAmmpQ88HOkTs
-0SKrD6FjsXW7jjQq+CklLEWGXcb4Xw+K8ZT6IRqMotvFNAIZWc9iJbkVTR/6TSaoKCaToR4QJIh4
-HLwclv1QmCaoKMoEnEniFVQcU5Wn+BPho+iRyGA8g6oJF0nHK9FtnNZSDZ1JARGHwxYZUbslijgI
-/IIhmL9m6UajNjUNz0AzIF+ag+oqW5TDzwE4GaAjTOSE0RUHPEwzxPRv7N4TDuDnhahjlWpBYZUk
-Ls8uxctnLw7Rh4BAb26p4zVHs5hktbQPF7BaS1k5CHOvcEzCMHLpskDlhk+P98NcR3Zluqyw0Etc
-ynV+K+eALTKws8riR3oD4TDMYxbDKoIyJSPMSs84azEGfzx7kBY02EC9NUEx62+W/oAjcJkpUB0c
-zRKpdajN9qco89sELfx0q1+CgQL1hmbKeBOBs3Aek6EdAg0BrmeGlNrIEBRYWbOXSHgjSFTx80YV
-RgTuAnXrNX29yfJNNuHw8wTV5HBkWRcFSzMvNmiW4EC8A8MBSOYQTTVEYyjgZwuUrUNAHqYP0wXK
-kkMPgMC6KoqRHFgmvqIpcqiGwyKM0StBwltKNNK3ZgiKbwwxHEj0NrIPjJZASDA5q+CsatBMhrJm
-msHADkl8rruIOO7zAbSoGIGhG2po3MjQ7+oYlLO4cJWS0w9t6OfPn5lt1IqSGojYFCeNdntB5i0q
-tmAKE9AJxg3iFAmxwQY8SgBTK82a4vCjyAt2gWA9L7Vsg+WGkKqqiuOjo81mE+mQPi+XR2px9Je/
-fv31X5+xTpzPiX9gOo606PxWdETv0I2MvjEW6Fuzci1+TDKfGwnWUJIrRP4f4vddncxzcXw4svoT
-ubgxrPi/cT5AgUzMoExloO2gweiJOnwSvVQD8UQM3bbDEXsS2qRaK+ZbXehR5WC7wdOY5XVWhY4i
-VeJLsG4QFs/ltF6GdnDPRpofMFWU06HlgcPn14iBzxmGr4wpnqCWILZAi++Q/kdmm5j8Ga0hkLxo
-ojoh67Zfixnizh8u79Y7dITGzDBRyB0oEX6TBwugbdyVHPxoZxTtnuOMmo9nCIylDwzzaldwiIJD
-uOBajF2pc7gafVSQpg2rZlAwrmoEBQ1u3ZSprcGRjQwRJHo3JsLmhdUtgE6tdJ0Jys0qQAt3nI61
-a7OC4wkhD5yI5/REglN73Hn3jJe2TlPKorR41KMKA/YWGu10Dnw5NADGYlD+NOCWelnOP7QWhdeg
-B1jOiRdksEWHmfCN6wMODgY97NSx+rt6M437QOAiUfuHASeMT3iAUoEwFUOfcXdxuKUtJ5taCO82
-OMRTZpVIotUO2Wrrjl6Z2muXFkmGqtdZo2iW5uAUW6VIfNS8930FClzwcZ8t0wKoydCQw2l0Qs6e
-J3+hbocpq2WNwb2b+0CM1oki44ZkWsF/4FVQToESQEBLgmbBPFTI/In9CSJn56u/7GAPS2hkCLfp
-Li+kYzA0HPP+QCAZdQYEhCADEnZlkTxH1gYpcJizQJ5sw2u5U7gJRqRAzBwDQloGcKeXXnyDTyLc
-dSABRch3lZKF+FIMYPnakvow1f2ncqnJGgydBuQp6HTDiZuKcNIQJ620hM/QfkKC9ieKHDh4Ch6P
-m1x32dwwrc2SgK/u622LFChkSpwMRi6q14YwbgL3ixOnRUMsM4hhKG8gbxvFjDQK7HJr0LDgBoy3
-5u2x9GM3YYF9h2GuXsj1HYR/YZmoWa5CjG87qQv3o7miSxuL7UUyHcAfbwEGo2sPkkx1+gKTLL9j
-kNCDHvZB9yaLWZF5XG6SLCQFpul34i9NBw9LSs/GHX2kaOoIJopZxqN3JQgIbTcegTihJoCgXIZK
-e/1dsHunOLBwufvA85qvjl9ed4k73pXgsZ/+pTq7q8pY4WqlvGgsFLhaXfuNShcmF2dbvWGoN5Qx
-SihzBUGk+PDxs0BCcC51E28fN/WG4RGbe+fkfQzqoNfuJVdrdsQugAhqRWSUo/DxHPlwZB87uT0T
-ewSQRzHMnkUxkDSf/B44+xYKxjicbzNMo7VVBn7g9ddfTXoSoy6SX381uGeUFjH6xH7Y8gTtyLSR
-L3qnbbqUMk7J13A6UVIxa3jHtilGrNAp/NNMdt3jdOLHvDcmo4Hfad6JG83ngOgBUXY+/RViVaXT
-W7dxklJOHtA4PEQ9Z8Jszhz04+NB2o8ypqTAY3k27o2E1NUzWJiQ4/pRdzraLzo1qd+eeNR8ilh1
-UTnQW+jNDpC3Le7u/u2W/V5L/W/SWY8E5M1m0EPAB87B7E7+/58JKyuGppXVqKX1ldyv5w2wB6jD
-HW7OHjekOzRvZi2MM8Fyp8RTFNCnYkNb0pTKw40JgDJnP6MHDi6j3th8U5clb0+SnBeyPMT9urHA
-ahzjaVCRTxfM0XtZISa22YxSo07tRt6nOkOd7LQzCRs/tV9kV7lJkcjsNimhL2iVYfj9hx/Owi4D
-6GGwUz84dx0NlzzcTiHcRzBtqIkTPqYPU+gxXX6/VLVdZZ+gZsvYJCA12bqE7eQdTdzavwb3ZCC8
-/UHeh8WIcLaSs5uJpL1lZFPs6uRg3+BrxMRuOfs1PipeUKESzGSW1kgrdvSwwmxRZzNKx1cS7Lku
-B8XyENox5nTTIo2XYkid55jq0NxI2ZDbuNTeTlHmWIAo6mR+tEzmQv5WxymGkXKxAFxwr0S/inh4
-yniIt7zpzYVpSs7qMqm2QIJY5XqrifbHnYbTLU906CHJuwpMQNwxPxYfcdr4ngk3N+QywaifYMdJ
-YpyHHcxeIHIXPYf3WT7BUSdUxzlmpLrbwPQ4aI+QA4ABAIX5D0Y6U+S/kfTK3c+iNXeJilrSI6Ub
-2ebkcSCU4Qgja/5NP31GdHlrB5bL3Vgu92O5bGO57MVy6WO53I+lKxK4sDZJYiShL1HSzqL3FmS4
-OQ4e5iyerbgd1vdhHR9AFIUJ6IxMcZmrl0nh7SQCQmrb2d+kh02BRcKFg2XOKVcNErkf90x08GgK
-lJ3OVK6hO/NUjM+2q8jE73sURVQONKXuLG/zuIojTy6WaT4FsbXojhsAY9GuN+HcXHY7mXI2sWWp
-Bpf/9en7D++xOYIamN106oaLiIYFpzJ8GpdL1ZWmJtgogB2ppV/3Qd00wIMHZnJ4lAP+7y0VFCDj
-iA1tiOeiAA+Ayn5sM7c4Jgxbz3UVjX7OTM57GydikFWDZlI7iHR6efn29NPpgFJMg/8duAJjaOtL
-h4uPaWEbdP03t7mlOPYBoda5lMb4uXPyaN1wxP021oBtub3PrlsPXjzEYPeGpf4s/62UgiUBQkU6
-2fgYQj04+PlDYUKHPoYRO9Vh7k4OOyv2nSN7joviiH5fmrs9gL+3hjHGBAigXaihiQyaYKql9K15
-3UNRB+gDfb0/HIK1Q692JONT1E6ixwF0KGub7Xb/vH0BNnpKVq/Pvjt/f3H++vL00/eOC4iu3IeP
-Ry/E2Q+fBZUjoAFjnyjGnfgKC1/AsLiHWcQ8h381pjfmdcVJSej19uJC7wys8TgD1reizYngOVfN
-WGico+Gsp32oy10Qo1QHSM65EaoOoXMlGC+t+cyCynUNLB1HmaKzWuvQS58HMueGaBs1AumDxi4p
-GARXNMErqlSuTFRY8o6TPkvTg5S20bYOIaUcVGd32tlvMdl8LzFHneFJ01kr+qvQxTW8jlSRJhDJ
-vQqtLOluWI3RMI5+aDdUGa8+Deh0h5F1Q571TizQar0KeW66/6hhtN9qwLBhsLcw70xSNQLV6GIt
-lQixEe8chPIOvtql12ugYMFwY6nCRTRMl8DsYwiuxSqBAAJ4cgXWF+MEgNBaCT8BfexkB2SOxQDh
-m/X88O+hJojf+pdfeppXZXr4D1FAFCS4ciXsIabb+C0EPpGMxNmHd6OQkaNKUPH3GkvAwSGhLJ8j
-7VQuwzu2k6GS6UKXM/j6AF9oP4Fet7qXsih1937XOEQJeKKG5DU8UYZ+IVYXWdhjnMqoBRqr2y1m
-eErM3fY2nwPxcSXTVBdEn7+9OAPfEQvuUYJ4n+cMhuN8CW7Z6lovPsXWAoUbuvC6RDYu0YWlTf15
-5DXrzcyiyFFvrw7ArhNlP7u9OqnOMk6Ui/YQp82wnJLzCLkZlsOsLHN3txnS2W1GdEfJYcaYXJZU
-NelzBnA0PY05MIKICYv6TbKZ9y6TrDJlcmkyA20KihfU6hhEBUmMJ9eI//KM0715qcyBF3hYbMtk
-uaowpQ6dIyq2x+Y/nH6+OH9P1esvXja+dw+LjikeGHPpwgnWpWHOA764tWbIW5NJH+fqVwgDdRD8
-ab/imogTHqDTj9OL+Kf9ik8cnTjxIM8A1FRdtIUEwwCnW5/0NBLBuNpoGD9u3VmDmQ+GMpJ4wEGX
-F7jz6/KjbdkyKJT9MS8fsVexKDQNh6azWwfV/ug5LgrcXJkP+xvB2z4JM58pdL3pvNlVceV+OrKI
-hx8Bo25rfwxTk9RpqqfjMNsubqHgVlvaXzInY+q0m2UoykDEodt55DJZvyrWzZkDvdrdDjDxjUbX
-SGKvQh/8kg20n+FhYondiVZMRzo7QaYA8xlSHxGpwZNCuwAKhEpOh47kjkdPX3hzdGzC/XPUugss
-5PegCHUBKB0syEvgRPjyG7uP/IrQQlV6LELHX8lkltvqJPxsVuhbPvfn2CsDlMpEsSvjbCmHDGts
-YH7pE3tHIpa0rccxV0mrWkJzN3iodzsYvCsW/bsnBrMWH3Ta3chtWxv51MEGvccPfAhlvAHtXtTV
-kNdq52YBNtdbsMMQkyS/hTvodQ96Ghb6Xb/17OHgh4ll3Etrr1pHW0L7QvuVsxICpkrRZoljhY2H
-6BrmxgaeNFZ4YJ/qihH7u+e8kFPl6sJlFFyo3gwHukEr1B/wyRU+uZdQZXRzsEK/m8tbmebgFkHE
-hYXvv9rC91FkUx29NUF/BoKX28ttP3r0pkHu2BTno+OkCljIKJPVEWLUm5C5B7kGH1z2X3TQEGc3
-5Me++fl8LN68/xH+fy0/QOSD59fG4h+AiXiTlxAB8hlKOtyOpf0Vh3Z5rfCQG0GjzQS+BwBdqkuP
-2rhxoc8c+IcNrBYTWGdZrvnyCUCR50jnihsbbirp4bc56tN1Fo0j17c0A/0SybD7AAQeGjjSLaNV
-tU5RnTupjGZNrwYX52/O3n88i6o75Hbzc+CkOvwqHZyR3sgtcdNqLOyTWY1Prh2/9nuZFj1urY4M
-zWEKjAxFCMFDYaNBvtsgthFAXGJ4L4rtPJ9F2BJ4n89vVRvwc0dOEHivHfaMIMIajvRWV+Ns42Og
-hvilrZcG0JD66DlRT0IonuJBIn4cDfot5VhQ/hn+PL3ZzN30tT4RQhNsY9rMeuh3t6pxxXTW8Fxm
-ItRO7EqYc4JpEqv1dOaeH/uQCX07BSg92o+Qi7hOKyEzEGEKxumaAND97pEvlhPmFrY4dA6K0inp
-Jt4qpyImVmKAow7opDNunFBmD2LlH+IbthB4Fk3UfKgVoBOiFOHkTldVz1Ysxxy0EAF7CgQ2Sfby
-RdghMg/KkeyscTVhnujYMUZLWen584Ph6Op5Y+wpezzzDnzOCrCDLqccgA4tnj59OhD/cb9/wqhE
-aZ7fgOMEsPvCVnFBr3d4FnpydrW6vrd5EwFLzlbyCh5cU5bbPq8zSiHu6UoLIu1fAyPEtQktP5r2
-LUvNybWSN4S5BW8saRPyU5bQHTSYApKocvVVPpgeMgJFLAm6IYzVLElCTifAemzzGs9qYTpQ84u8
-A45PEMwY3+JOFgfDK/QBqbDSco9F50QMCPCACp14NDrsSqeVAM/J5VajOTnPkqo5Z/DM3eTUh7or
-e7WM5isRb1AyzDxaxHCO/Xms2vjA+V4W9WKKfHblJgZbs+TX9+EOrA2Sli8WBlN4aBZplstyZowq
-rlgySyoHjGmHcLgz3ahDBigKelAagIYnwzC3Em3ffmHXxcX0A+33HpqRdJlPZW8p4iROnLWq3aKo
-GZ/SRZaQlm/NlxGM8p7Sz9of8MYSX+jkJxaZe5cpuMfd6kxfksB1Fs3NCQCHLuaxCtKyo6cjnNug
-LHxmWh1uNHcqODXxGEQTbrdJWdVxOtEH+SfouU3sBrjG0x6T2nsA0Pos4Pbn4BAf6pJu8B1MNQzS
-EysyTcn+iVjoJELkHj3yT+kUOfp6Lzw9jqnpZ3wRgKPBseWX5vDKQ1S+OULROX3gYjmm2qNw1K6o
-7LTCfQ5TIm+d7HYc8KghW7B8h31WbPFOHpjWk3lE/0LfkaPLFHBj6tGDp8mUBgv7Co/v76srATH+
-W4OgLBI5P3yiEDvG+Y9C1VAMddxA4REzDOnuCQL5ZWsnzykv5NrfXds3HaBff7UPrKuCewufac/E
-V8v6aJtbidxs2uDnwHrEK3C6UW/MzWFkrZb43CbqEDaI9qy5qVdpH5mB1w+f8p4JP2BHNMTBNHe4
-8rqPVha/faRqGgW/i0q6Vz+t0AnGUtFVzG9QmdXFsQ0V+TBfRmn2oVtAhJ/qpre0Psa7j4jRq5tw
-3/S5/7656xaBnbnZP+vM3T9C49JA993NL300YAddE+JBVbkWo8mfI7pjvbXbn6LSn4W9hZEzVcSD
-GrWxZsl1PHO/Y4HBIV/i6B6HClyQZtVbc+qcD2uzc5eTu9zMm6n43J6QpB3yuWYvNud0pc+Ea64m
-crlUkxhvhJqQD0j1AR3jbryKd3QbkIzV1jgDeOcCgDCsoiu53GJNWHXwM/lmSt5edw7XCxqaitCc
-qjaVzDm2154HgIs4pqf+JnPEZWmDVGI2RtVlUYKzNtD3F/K+b1+pXAPUxJfrWN0Y1E2Psb7ODofg
-YgNzhIozCewAetQBQvDJCudmF67znEzsO+CXZ81R0WRsGUJm9VqWcdXckuDvLyXiW2cEOjiHC+xE
-kI3YtTjFRSyx/OEghTGc/f6ldo4832/P+dCRVWkPZyvqoZMTjzl66ki54ebkzt6S5N7OMadrMSle
-5Ns1hG3WcJ+9GQKWwlz5Q4pQh3T8Vl9DwvfTcc4Jq+ocPgK5d4+t+NWNVmexw2DRcJ65iqF77wSe
-fCRD23edVIcLuhdH+czQjO/rDcssnd2EHY0tFU+4Ra/iaUYbNYEOFiLdE+j4xaaPDHQ8+A8MdPTl
-X2BNND5aH/SWn94TEbGacG/SahgB+kyASLhh0rqHydjDoVvMCeFKcjewl1GyznROiBgzgRzZvWKF
-QPCNWcqtfPNutDHj9kUivnTR4+8uPrw+vSBaTC5P3/zn6Xe0zY9ZvZbNenAkmOWHTO1Dr6zQjQr1
-1mzf4A22PVfTcW28htB539nW6oHQfw6ib0Hbisx9vatDp5682wkQ3z/tFtRdKrsXcsf50rXL7oZs
-q/4v0E+5WMv8cvbWzCOTU2ZxaBLG5n2T49My2kmB7Fo4p2yqq060U6ovM9uRnhnZ4j1aAUztIX/Z
-zJ6pxLb5I3ZU2leEU8UhnmIxNwGAFM6kcyEV3UXFoCr/LvISlF2MOxTsMI7tvZ7UjrOYyl5Yi7sU
-MxkZgnjHSAbd+bnCPpfpDioEASs8fd0SI2L0n877272yJ0pcHdKBtUNUNtf2F66ZdnJ/TnBHrLL3
-liiz5Y27AdB4UafuLpft0+lAzh8lTfOFUyENmu8I6NyIpwL2Rp+JFeJ0K0KIEvVWDhZdER31nUMO
-8mg3HewNrZ6Jw13HmdzjPEI8391w3joxpHu84B7qnh6qNodGHAuMdT+7zimJbwkyZ90FXVTiOR+4
-26Ovx4Svt1fPj23KFvkdX7vXYCDtB45hv2pOBuy9GsvpTbxSjqn+A4uNRm3w1wOHNRdid4DTqXPe
-EQSZ7TiGNPDe99dGmB7enb2DNqKW745hQmL4RI1oUk5luMbdPhl1JtuorC4MLnK/H0ZH+wEohNLv
-m+CHb2MB9fxMx4PTmu4TtA4nHg115IEKHXxe4B7G62uwa3eno2kP6k4l//agADdo855ebxBr9hq4
-lZfo2G0L2jNveGCH7edDfv39nz+gf7ckxnZ/sc+htq1e9h4sYScWi6hw87pFIfM4AusCCnNIahrr
-b42E4+H9howONzVTQ65Ah4/qsvCuUAosyImdaMtvjUHwf71Zz9M=
+eJzVPP1z2zaWv/OvwNKToZTKdD66nR2n7o2TOK3v3MTbpLO5dTM6SoIk1hTJEqRl7c3d337vAwAB
+kvLHdvvDaTKxRAIPDw/vGw8Iw/C0LGW+EJti0WRSKJlU87Uok3qtxLKoRL1Oq8VhmVT1Dp7Or5OV
+VKIuhNqpGFvFQfD0d36Cp+LTOlUGBfiWNHWxSep0nmTZTqSbsqhquRCLpkrzlUjztE6TLP0HtCjy
+WDz9/RgE57mAmWeprMSNrBTAVaJYistdvS5yMWpKnPPz+M/Jy/FEqHmVljU0qDTOQJF1Uge5lAtA
+E1o2CkiZ1vJQlXKeLtO5bbgtmmwhyiyZS/Ff/8VTo6ZRFKhiI7drWUmRAzIAUwKsEvGAr2kl5sVC
+xkK8lvMEB+DnLbEChjbBNVNIxrwQWZGvYE65nEulkmonRrOmJkCEslgUgFMKGNRplgXborpWY1hS
+Wo8tPBIJs4c/GWYPmCeO3+ccwPFDHvycp7cThg3cg+DqNbNNJZfprUgQLPyUt3I+1c9G6VIs0uUS
+aJDXY2wSMAJKZOnsqKTl+Fav0HdHhJXlygTGkIgyN+aX1CMOzmuRZArYtimRRoowfytnaZIDNfIb
+GA4gAkmDoXEWqartODQ7UQCACtexBinZKDHaJGkOzPpjMie0/5bmi2KrxkQBWC0lfm1U7c5/NEAA
+aO0QYBLgYpnVbPIsvZbZbgwIfALsK6marEaBWKSVnNdFlUpFAAC1nZC3gPREJJXUJGTONHI7IfoT
+TdIcFxYFDAUeXyJJlumqqUjCxDIFzgWuePfhJ/H27PX56XvNYwYYy+xqAzgDFFpoBycYQBw1qjrK
+ChDoOLjAPyJZLFDIVjg+4NU2OLp3pYMRzL2Mu32cBQey68XVw8Aca1AmNFZA/f4bukzUGujzP/es
+dxCc7qMKTZy/bdcFyGSebKRYJ8xfyBnBtxrOd3FZr18BNyiEUwOpFC4OIpgiPCCJS7NRkUtRAotl
+aS7HAVBoRm39VQRWeF/kh7TWHU4ACFWQw0vn2ZhGzCVMtA/rFeoL03hHM9NNArvOm6IixQH8n89J
+F2VJfk04KmIo/jaTqzTPESHkhSA6iGhgdZ0CJy5icUGtSC+YRiJi7cUtUSQa4CVkOuBJeZtsykxO
+WHxRt96tRmgwWQuz1hlzHLSsSb3SqrVTHeS9F/HnDtcRmvW6kgC8mXlCtyyKiZiBziZsymTD4lVv
+C+KcYECeqBPyBLWEvvgdKHqqVLOR9iXyCmgWYqhgWWRZsQWSHQeBEAfYyBhlnznhLbyD/wEu/p/J
+er4OAmckC1iDQuT3gUIgYBJkrrlaI+Fxm2blrpJJc9YURbWQFQ31MGIfMeIPbIxzDd4XtTZqPF1c
+5WKT1qiSZtpkpmzx8qhm/fiK5w3TAMutiGamaUunDU4vK9fJTBqXZCaXKAl6kV7ZZYcxg4ExyRbX
+YsNWBt4BWWTKFmRYsaDSWdaSnACAwcKX5GnZZNRIIYOJBAbalAR/k6BJL7SzBOzNZjlAhcTmew72
+B3D7B4jRdp0CfeYAATQMailYvllaV+ggtPoo8I2+6c/jA6eeL7Vt4iGXSZppK5/kwTk9PKsqEt+5
+LLHXRBNDwQzzGl27VQ50RDEPwzAIjDu0U+ZrYb9Np7MmRXs3nQZ1tTsG7hAo5AFDF+9hojQcv1lW
+xQZfW/Q+gj4AvYw9ggNxSYpCso/rMdMrpICrlUvTFM2vw5ECVUlw+dPZu/PPZx/FibhqtdKkq5K+
+wJhneQKcSUod+KIzbKuOoCXqrhTVl3gHFprWFUwS9SYJlEndAPsB6p+qhl7DNObey+Ds/enri7Pp
+zx/Pfpp+PP90BgiCqZDBAU0ZLVwDPp+Kgb+BsRYq1iYy6PWgB69PP9oHwTRV03JX7uAB2DrgwmoE
+852ICB9OtRmdpvmyiMbU+Ff2o09YM2in6er46y/i5EREvyY3SRSA49I25UX8kXj4066U0LWGP6NC
+jYNgIZfA79cSpWL0lJzRMXcAIkLLQhvHX4s0N+/ptTcECe2IegAO0+k8S5TCxtNpBKSlDgMf6BCz
+R4usPoKO5c7tOtao4KeSsBQ5dpngfwMoJjPqh2gwim4X0whkZDNPlORWNH3oN52iophOR3pAkCDi
+cfByWPYjYZqgoqhScCaJV1BxzFSR4U+Ej6JHIoPxDKomXCQdr8Q3SdZINXImBUQcjTpkRO2WKuIg
+8AtGYP7apRuPu9Q0PAPNgHxZAaqr6lAOPwfgZICOMJETRlcc8DDNENN/Z/eecAA/L0Idq1QHCqsk
+cXl2KV4+e3GIPgQEegtLHa85msU0b6R9uITVWsnaQZh7RRMShrFLlyUqN3x6fDfMTWxXps8KS73E
+ldwUN3IB2CIDO6ssfqI3EA7DPOYJrCIoUzLCrPSMs5Zg8MezB2lBgw3U2xAUs/5m6Q84Ape5AtXB
+0SyRWofabH/KqrhJ0cLPdvolGChQb2imjDcROAvnMRnaIdAQ4HrmSKmtjECBVQ17iYQ3gkQVv2hV
+YUzgLlC3fqGv13mxzaccfp6gmhyNLeuiYGnmxQbtEhyId2A4AMkCoqmWaAwF/GyBsnUIyMP0YbpA
+WXLoARBYV0UxkgPLxFc0RQ7VcFiEMX4lSHgriUb6xgxB8Y0hhgOJ3sb2gdESCAkmZxWcVQ2ayVDW
+TDMY2CGJz3UXMcd9PoAOFWMwdCMNjRsZ+l0dg3IWF65ScvqhDf38+TOzjVpTUgMRm+Gk0W4vybzF
+5Q5MYQo6wbhBnCIhNtiCRwlgGqVZUxx+FEXJLhCs56WWbbDcEFLVdXl8dLTdbmMd0hfV6kgtj/78
+l2+++csz1omLBfEPTMeRFp3fio/oHbqR8bfGAn1nVq7Dj2nucyPBGklyhcj/Q/y+b9JFIY4Px1Z/
+Ihe3hhX/N84HKJCpGZSpDLQNW4yeqMMn8UsViidi5LYdjdmT0CbVWjHf6kKPugDbDZ7GvGjyOnIU
+qRJfgXWDsHghZ80qsoN7NtL8gKminI4sDxw+/4IY+Jxh+MqY4ilqCWILtPgO6X9itknIn9EaAsmL
+JqoXsu6GtZgh7uLh8m69Q0dozAxThdyBEuE3ebAA2sZ9ycGPdkbR7jnOqPl4hsBY+sAwr3YFRyg4
+hAuuxcSVOoer0UcFadqyagYF46pGUNDg1s2Y2hoc2cgIQaJ3YyJsXljdAujUSdeZoNysArRwx+lZ
+uy4rOJ4Q8sCJeE5PJDi1x713z3hpmyyjLEqHRz2qMGBvodFOF8CXIwNgIsLq55Bb6mU5/9BZFF6D
+AWAFJ16QwZY9ZsI3rg8YHoQD7NSz+vt6M42HQOAiUfuHASeMT3iASoEwlSOfcfdxuKUtJ5s6CO83
+OMRTZpVIotUe2erqjkGZutMuLdMcVa+zRvE8K8AptkqR+Kh97/sKFLjg4yFbpgVQk6Elh9PohJw9
+T/4i3Q5TVqsGg3s394EYbVJFxg3JtIb/wKugnAIlgICWBM2CeaiQ+RP7F4icna/+soc9LKGRIdym
++7yQnsHQcMz7A4Fk1BkQEIIcSNiXRfIcWRtkwGHOAnmyDa/lXuEmGLECMXMMCGkZwJ1eevENPolx
+14EEFCHf1kqW4isRwvJ1JfVhqvtfyqUmazByGpCnoNMNJ24qwklDnHTSEj5D+wkJ2p8oC+DgGXg8
+bnLdZXPDtDZLAr66r7ctUqCQKXESjl1UvxjCuAncP504LVpimUEMQ3kDedsoZqRxYJdbg4YFN2C8
+Ne+OpR+7CQvsO4oK9UJubiH8i6pUzQsVYXzbS124H80VfdpYbC/SWQh/vAUIx188SDLT6QtMsvwT
+g0Qe9GgIujdZzIoskmqb5hEpME2/E39penhYUno27ugjRVNHMFHMMh69q0BAaLvxCMQJNQEE5TJS
+2uvvg71ziqGFy91Dz2u+On75pU/cyb4Ej/0ML9XZbV0lClcr40VjocDV6ttvVLowuSTf6Q1DvaGM
+UUJVKAgixYePnwUSgnOp22T3uKm3DI/Y3Dsn72NQB712L7k6syN2AURQKyKjHEWP58iHI/vYyd0x
+sUcAeRTD3LEoBpLmk38Gzl0LBWMcLnY5ptG6KgM/8Pqbr6cDiVEXyW++Du8ZpUOMIbEfdTxBOzJt
+5IvBaZsulUwy8jWcTpRUzFvesW3KMSt0Cv80k30ZcDrxY94bk9HC7zXvxY3mc0D0gCi7mP0KsarS
+6a2bJM0oJw9oHB6injNhNmcOhvHxIN2NMqakwGN5NhmMhNTVM1iYiOP6cX862i86NanfgXjUfMpE
+9VE50Fvo7Q6Qty3u7v7tl/1BS/0H6axHAvJmEw4Q8IFzMLuT//9nwsqKoWllNe5ofSXv1vMG2APU
+4R435w43pD80b2YtjTPBcqfEUxTQp2JLW9KUysONCYCyYD9jAA4uo97YfNNUFW9PkpyXsjrE/bqJ
+wGoc42lQkU8fzNF7WSMmttmcUqNO7UYxpDojney0M4laP3VYZNeFSZHI/CatoC9olVH0w4cfz6I+
+A+hhsNMwOHcdDZc83E4h3EcwbaSJEz2mD1PoMV3+eanquso+Qc2WsUlAarL1CdvLO5q4dXgN7slA
+ePuDvA+LEeF8LefXU0l7y8im2NXJwb7B14iJ3XL2a3xUsqRCJZjJPGuQVuzoYYXZssnnlI6vJdhz
+XQ6K5SG0Y8zppmWWrMSIOi8w1aG5kbIhN0mlvZ2yKrAAUTTp4miVLoT8rUkyDCPlcgm44F6JfhXz
+8JTxEG9505sL05ScN1Va74AEiSr0VhPtjzsNZzue6MhDkncVmIC4Y34sPuK08T0TbmHIZYJRP8GO
+k8Q4DzuYvUDkLnoO7/NiiqNOqY5zwkj1t4HpcdAdoQAAIQCF+YdjnSny30h65e5n0Zq7REUt6ZHS
+jWwL8jgQymiMkTX/pp8+I7q8tQfL1X4sV3djuepiuRrEcuVjubobS1ckcGFtksRIwlCipJtFHyzI
+cHMcPMxZMl9zO6zvwzo+gChKE9AZmeIyVy+TwttJBITUtrO/SQ/bAouUCwerglOuGiRyP+6Z6ODR
+FCg7nalcQ3fmqRifbV+Rid/3KI6pHGhG3VneFkmdxJ5crLJiBmJr0Z20ACaiW2/Cubn8ZjrjbGLH
+UoWX//nphw/vsTmCCs1uOnXDRUTDglMZPU2qlepLUxtslMCO1NKv+6BuGuDBAzM5PMoB//eWCgqQ
+ccSWNsQLUYIHQGU/tplbHBNFnee6ikY/ZybnvY0TEeZ12E5qD5FOLy/fnn46DSnFFP5v6AqMoa0v
+HS4+poVt0Pff3OaW4tgHhFrnUlrj587Jo3XLEffbWAO24/Y++9J58OIhBnswLPVn+YdSCpYECBXr
+ZONjCPXg4Od3hQk9+hhG7FWHuTs57KzYd47sOS6KI/pDae7uAP7eGsYYUyCAdqFGJjJog6mO0rfm
+9Q6KOkAf6Ov97hCsG3p1IxmfonYSAw6gQ1nbbL/75+0LsNFTsn599v35+4vz15enn35wXEB05T58
+PHohzn78LKgcAQ0Y+0QJ7sTXWPgChsU9zCIWBfxrML2xaGpOSkKvtxcXemdgg8cZsL4VbU4Mz7lq
+xkLjHA1nPe1DXe6CGGU6QHLOjVB1CJ0rwXhpw2cWVKFrYOk4ygyd1UaHXvo8kDk3RNuoMUgfNHZJ
+wSC4ogleUaVybaLCinec9FmaAaS0jbZ1CBnloHq7085+i8nme4k56gxP2s5a0V9FLq7Rl1iVWQqR
+3KvIypLuhtUYLePoh3ZDlfEa0oBOdxhZN+RZ78UCrdariOem+49bRvutAQxbBnsL884lVSNQjS7W
+UokIG/HOQSRv4atder0GChYMN5ZqXETDdCnMPoHgWqxTCCCAJ9dgfTFOAAidlfAT0MdOdkAWWAwQ
+vdksDv8aaYL4rX/5ZaB5XWWHfxclREGCK1eiAWK6jd9C4BPLWJx9eDeOGDmqBBV/bbAEHBwSyvI5
+0k7lMrxjOx0pmS11OYOvD/CF9hPodad7JctKdx92jSOUgCdqRF7DE2XoF2F1kYU9wamMO6Cxut1i
+hqfE3G1v8zkQH9cyy3RB9PnbizPwHbHgHiWI93nOYDjOl+CWra714lNsHVC4oQuvK2TjCl1Y2tRf
+xF6zwcwsihz19uoA7DpR9rPfq5fqrJJUuWiPcNoMyyk5j5GbYTnMyjJ395shnd1mRHeUHGaM6WVF
+VZM+ZwBH09OEAyOImLCo3ySbee8yzWtTJpelc9CmoHhBrU5AVJDEeHKN+K/IOd1bVMoceIGH5a5K
+V+saU+rQOaZie2z+4+nni/P3VL3+4mXrew+w6ITigQmXLpxgXRrmPOCLW2uGvDWdDnGufoUwUAfB
+n+4rrok44QF6/Ti9iH+6r/jE0YkTD/IMQE01ZVdIMAxwug1JTysRjKuNhvHj1p21mPlgKCOJBxx0
+eYE7vz4/2pYdg0LZH/PyEXsVy1LTcGQ6u3VQ3Y+e47LEzZXFaLgRvB2SMPOZQdfr3pt9FVfupyeL
+ePgRMOq39scwNUm9pno6DrPt4xYKbrWl/SV3MqZOu3mOogxEHLmdxy6TDati3Zw50Kvd7QET32p0
+jSQOKvTwlzzUfoaHiSV2L1oxHensBJkCzGdIfUSkAU8K7QIoECo5HTmSOxk/feHN0bEJ989R6y6w
+kD+AItQFoHSwoKiAE+HLb+w+8itCC1XpsYgcfyWXeWGrk/CzXaNv+dyf46AMUCoTxa5K8pUcMayJ
+gfmVT+w9iVjSth7HXKWdagnN3eCh3u5h8L5YDO+eGMw6fNBrdy13XW3kUwcbDB4/8CFUyRa0e9nU
+I16rvZsF2FxvwY4iTJL8Fu2h1z3oaVjod/02sIeDHyaWcS+tveocbYnsC+1XzisImGpFmyWOFTYe
+omuYWxt40lrh0D7VFSP298B5IafK1YXLKLhQvRmGukEn1A/55Aqf3EupMro9WKHfLeSNzApwiyDi
+wsL3X23h+zi2qY7BmqB/BYKXu8vdMHr0pkXu2BTno+OkSljIOJf1EWI0mJC5B7kWH1z2X3TQkOTX
+5Me++dv5RLx5/xP8/1p+gMgHz69NxN8BE/GmqCAC5DOUdLgdS/trDu2KRuEhN4JGmwl8DwC6VJce
+tXHjQp858A8bWC0msM6y2vDlE4Aiz5HOFbc23FTSw29z1KfvLBpHbmhpQv0SybD/AAQeGjjSLeN1
+vclQnTupjHZNr8KL8zdn7z+exfUtcrv5GTqpDr9KB2ekN3Ir3LSaCPtk3uCTL45f+4PMygG3VkeG
+5jAFRoYiguChtNEg322Q2AggqTC8F+VuUcxjbAm8z+e36i34uWMnCLzXDntGEGGNxnqrq3W28TFQ
+Q/zS1UshNKQ+ek7UkxBKZniQiB/H4bClnAjKP8Ofp9fbhZu+1idCaIJdTNtZj/zuVjWumc4anstM
+hNqJXQlzTjBLE7WZzd3zYx9yoW+nAKVH+xFymTRZLWQOIkzBOF0TALrfPfLFcsLcwhaHzkFROiXb
+JjvlVMQkSoQ4akgnnXHjhDJ7ECv/mFyzhcCzaKLhQ60AnRClCKdwuqpmvmY55qCFCDhQILBN85cv
+oh6ReVCOZOetqwnzRMeOMVrJWs+fH4zGV89bY0/Z47l34HNegh10OeUAdGj59OnTUPzb/f4JoxJn
+RXENjhPAHgpbxQW93uNZ6MnZ1er73uZNDCw5X8srePCFstz2eZNTCvGOrrQg0v41MCJcm8jyo2nf
+sdScXKt4Q5hb8MaSNiE/5yndQYMpIIkqV1/lg+khI1DEkqAbokTN0zTidAKsx65o8KwWpgM1v8hb
+4PgUwUzwLe5kcTC8Rh+QCist91h0TkRIgEMqdOLR6LArnVYCPKeXO43m9DxP6/acwTN3k1Mf6q7t
+1TKar0SyRckw8+gQwzn257Fq6wMXd7KoF1MU8ys3MdiZJb++D3dgbZC0Yrk0mMJDs0jzQlZzY1Rx
+xdJ5WjtgTDuEw53pRh0yQHEwgFIIGp4Mw8JKtH37J7suLqYfaL/30Iyky3xqe0sRJ3GSvFPtFsft
++JQusoS0fGu+jGGU95R+1v6AN5b4k05+YpG5d5mCe9ytyfUlCVxn0d6cAHDoYh6rIC07ejrCuQ3K
+wmem1eFGe6eCUxOPQTThdpNWdZNkU32Qf4qe29RugGs87TGpOw8AWp8F3P4CHOJDXdINvoOphkF6
+YkWmKdk/EUudRIjdo0f+KZ2yQF/vBXtM4FYgZRZTc17WpHmuLLvag4jP+KYAR8UjqK/M6ZaH2ARz
+xqJ3PMGdxoSKk6Jxt+Sy1wo3QkwNvfXCu4HCo4bswPI9+nm5w0t7YFpPFjH9i3xPj25bwJ2rRw+e
+pTMaLBqqTL6/ry4VxABxA5K0TOXi8IlC7Bjn3wtVQ2nntZ9vaMZ4bvnA1irpI7vtSQY6sDQDT3/b
+qhdH0RiH1A5izgJrrcDHJlGE2R7Zo96mXKRTsmdv1MJyPo2er6qRZ6dDW0r4Da/UgPeaoC6L791q
+1MXXFuxAzK5P2PHGUdtwIFZ7BA9FdP0HLdpXnc3UP3LxuJ5735ZcD8tvvr4LT9esDhazDxDUI6bT
+6g9kqr0s89iFecAu9/3893Deu4vvHk0l3WuYVhjYYPnvOuE3aKCa8tiG/0Ykcetk5BaF4ae+Hjwu
+weKIuqm+ju6aPve/a+66RWBnbvZEe3P3j0W5NNB99/PLEA046NKEeFCldSuww7rAFeOJ3oofTinq
+D5Lf7YQz3aMV7gNjNv4xQUlsqrE3WnOTzN3vWFdyyHd3uqfgBnV1Z9q9Y4Fdjn+IouU9dHxuD8ZS
+YcRCcyB7cXST05RL7aZytVLTBC8Cm5LrT2UhPZ/OOJPv6BIomaid8QHxqg0AYbhJF/C5NbrAGBBe
+8IWkXFXh3KkgaGiqPXSKGVW64JSOdjgBXMypHOpvEoZcjRhmEpNwqqnKCnz0UF9bydv9QxWSLVCT
+Vtgk6tqgbnpM9C2GOATXmJiTc5xAYr/fow4Qgg/UOBf6cHnvdGrfAb88a08IpxPLEDJvNrJK6vZy
+DH9bMRXfOSPQeUlcYCdx0Epmh1NcxFLLHw5SGLrb719pl9dz+e84Fjy2Wu/hbEU9dE7qMSeOHdtj
+uDm9tZdjuZeyLOg2VNfI22Yt99kLQWApzE1PpCt1JM9v9e0zfC0hpxrRTXD4COTePa3kF7VatcZO
+ikXDeeYqhv51I3jglWzx0C1iPS7o3xfmM0M7vq83LLP0NpH2NLZU1K7VoOJpRxu38S3Wn90T3/o1
+xo+Mbz34D4xv9Z1vYHA0PlofDFYd3xMIs5pwL1BrGQH6TIFIuE/WuX7LmMyRW8MLQWh6G9o7SFln
+OgeDjJlAjuzfrEMg+KI05RY8ehcZmXGH4ktfuujx9xcfXp9eEC2ml6dv/uP0e6ruwGRux2Y9OAGQ
+F4dM7UOvmtRNBugd+aHBW2wHbiTkIxEaQu99bzdzAMLw8ZehBe0qMvf1vg69YwT9ToD43dPuQN2n
+sgch9/wzXbLu7sN3yj4D/ZRr9MwvZ0vVPDJbCSwO7T6Bed+mdrWM9jJf+xbOqZbrqxPtt+o77PZk
+5ca2ZpNWADO6yF82oWsK8G3akB2V7s3wVGiKh5fMBRAghXPp3ENGV5AxqNq/gr4CZZfgxhQ7jBN7
+nSu14+S1svcU4+bUXMaGIN7pobA/P1fYFzLbQ4UgYIWnb9liRIz+09s9dov0iRJXh3RO8RCVzRf7
+C9dMO7l/S3EjtLbX1ShT6YCbQNB42WTu5qbt0+tAzh/lyoulUxgPmu8I6NyKpwL2Rp+JFeJsJyII
+JPUOHtbaER31VVMO8mg3HewNrZ6Jw32n2NxTXEI8399w0Tkopnu84B7qnh6qMWeFHAuM5V77jqeJ
+7wgyb7YIup/Gcz5wk0/figpfb66eH9vUJ/I7vnYUCdE+dAz7VXsg5M4b0ZzexCvVhMp+MGQbd8F/
+CR3WXIr9AU6vvH1PEGR2YRlS6L0fLokxPbyrmsMuopbvjmFCYvREjWlSzoEAjbt9Mu5NtlVZfRh8
+tuF+GD3tB6AQyrBvgh++hAfU8zMdD84aukbSzXxHjjxQfYvPC9zDeH0tdt3udCLxQd3ppIc9H8IN
+uryn1xvEmr0GbuXlQvbbgu7MWx7YY/v5bOdw/+cP6N+vhLLdX9zlUNtWLwfPE7ETi7VzWLPQoZB5
+HIN1AYU5IjWNZddGwvHOhpaMDje1U0OuQIePyvHwilgKLMiJnWrLb41B8H8KTGoz
 """.decode("base64").decode("zlib")
 
 ##file ez_setup.py

diff --git a/virtualenv_support/site.py b/virtualenv_support/site.py
--- a/virtualenv_support/site.py
+++ b/virtualenv_support/site.py
@@ -539,6 +539,7 @@
     sys.real_prefix = f.read().strip()
     f.close()
     pos = 2
+    hardcoded_relative_dirs = []
     if sys.path[0] == '':
         pos += 1
     if sys.platform == 'win32':
@@ -550,8 +551,16 @@
         paths = [os.path.join(sys.real_prefix, 'lib_pypy'),
                  os.path.join(sys.real_prefix, 'lib-python', 'modified-%s' % cpyver),
                  os.path.join(sys.real_prefix, 'lib-python', cpyver)]
+        hardcoded_relative_dirs = paths[:] # for the special 'darwin' case below
+        #
+        # This is hardcoded in the Python executable, but relative to sys.prefix:
+        for path in paths[:]:
+            plat_path = os.path.join(path, 'plat-%s' % sys.platform)
+            if os.path.exists(plat_path):
+                paths.append(plat_path)
     else:
         paths = [os.path.join(sys.real_prefix, 'lib', 'python'+sys.version[:3])]
+        hardcoded_relative_dirs = paths[:] # for the special 'darwin' case below
         lib64_path = os.path.join(sys.real_prefix, 'lib64', 'python'+sys.version[:3])
         if os.path.exists(lib64_path):
             paths.append(lib64_path)
@@ -570,7 +579,8 @@
     # These are hardcoded in the Apple's Python executable,
     # but relative to sys.prefix, so we have to fix them up:
     if sys.platform == 'darwin':
-        hardcoded_paths = [os.path.join(sys.real_prefix, 'lib', 'python'+sys.version[:3], module)
+        hardcoded_paths = [os.path.join(relative_dir, module)
+                           for relative_dir in hardcoded_relative_dirs
                            for module in ('plat-darwin', 'plat-mac', 'plat-mac/lib-scriptpackages')]
 
         for path in hardcoded_paths:

From commits-noreply at bitbucket.org  Wed Feb 16 15:37:18 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Wed, 16 Feb 2011 15:37:18 +0100 (CET)
Subject: [pypy-svn] pypy default: update the subrepo
Message-ID: <20110216143718.74755282BF2@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42037:8a7e75a991f8
Date: 2011-02-16 15:36 +0100
http://bitbucket.org/pypy/pypy/changeset/8a7e75a991f8/

Log:	update the subrepo

diff --git a/.hgsubstate b/.hgsubstate
--- a/.hgsubstate
+++ b/.hgsubstate
@@ -1,4 +1,4 @@
-95a9f95cfa1bad8f87f5866edb8a06159c1f38c7 contrib/virtualenv-pypy
+792c971f4114520f0773e3474e2516600af60ee0 contrib/virtualenv-pypy
 80037 greenlet
 80348 lib_pypy/pyrepl
 80037 testrunner

From commits-noreply at bitbucket.org  Wed Feb 16 15:44:35 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed, 16 Feb 2011 15:44:35 +0100 (CET)
Subject: [pypy-svn] pypy default: One case where the test is not xfail()ing,
	but fails because of the presence of xfail() on another test
Message-ID: <20110216144435.BFF672A202C@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r42038:8aa8cdbe6d40
Date: 2011-02-16 15:07 +0100
http://bitbucket.org/pypy/pypy/changeset/8aa8cdbe6d40/

Log:	One case where the test is not xfail()ing, but fails because of the
	presence of xfail() on another test

diff --git a/lib-python/modified-2.7.0/ctypes/test/__init__.py b/lib-python/modified-2.7.0/ctypes/test/__init__.py
--- a/lib-python/modified-2.7.0/ctypes/test/__init__.py
+++ b/lib-python/modified-2.7.0/ctypes/test/__init__.py
@@ -211,9 +211,9 @@
     """
     Poor's man xfail: remove it when all the failures have been fixed
     """
-    def new_method(self):
+    def new_method(self, *args, **kwds):
         try:
-            method(self)
+            method(self, *args, **kwds)
         except:
             pass
         else:

diff --git a/lib-python/modified-2.7.0/ctypes/test/test_strings.py b/lib-python/modified-2.7.0/ctypes/test/test_strings.py
--- a/lib-python/modified-2.7.0/ctypes/test/test_strings.py
+++ b/lib-python/modified-2.7.0/ctypes/test/test_strings.py
@@ -46,7 +46,6 @@
         self.assertRaises(TypeError, setattr, buf, "value", memoryview("abc"))
         self.assertRaises(ValueError, setattr, buf, "raw", memoryview("x" * 100))
 
-    @xfail
     def test_c_buffer_deprecated(self):
         # Compatibility with 2.x
         with test_support.check_py3k_warnings():

From commits-noreply at bitbucket.org  Wed Feb 16 15:44:36 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed, 16 Feb 2011 15:44:36 +0100 (CET)
Subject: [pypy-svn] pypy default: We get a TypeError on CPython,
	not a ValueError.
Message-ID: <20110216144436.A209A2A202C@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r42039:23785f57e97a
Date: 2011-02-16 15:40 +0100
http://bitbucket.org/pypy/pypy/changeset/23785f57e97a/

Log:	We get a TypeError on CPython, not a ValueError.

diff --git a/pypy/module/_rawffi/test/test__rawffi.py b/pypy/module/_rawffi/test/test__rawffi.py
--- a/pypy/module/_rawffi/test/test__rawffi.py
+++ b/pypy/module/_rawffi/test/test__rawffi.py
@@ -494,9 +494,9 @@
 
     def test_invalid_bitfields(self):
         import _rawffi
-        raises(ValueError, _rawffi.Structure, [('A', 'c', 1)])
-        raises(ValueError, _rawffi.Structure, [('A', 'I', 129)])
-        raises(ValueError, _rawffi.Structure, [('A', 'I', -1)])
+        raises(TypeError, _rawffi.Structure, [('A', 'c', 1)])
+        raises(TypeError, _rawffi.Structure, [('A', 'I', 129)])
+        raises(TypeError, _rawffi.Structure, [('A', 'I', -1)])
 
     def test_packed_structure(self):
         import _rawffi

diff --git a/pypy/module/_rawffi/structure.py b/pypy/module/_rawffi/structure.py
--- a/pypy/module/_rawffi/structure.py
+++ b/pypy/module/_rawffi/structure.py
@@ -43,7 +43,7 @@
                 if c == tp.itemcode:
                     break
             else:
-                raise OperationError(space.w_ValueError, space.wrap(
+                raise OperationError(space.w_TypeError, space.wrap(
                     "bit fields not allowed for type"))
         else:
             bitsize = 0

From commits-noreply at bitbucket.org  Wed Feb 16 15:44:37 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed, 16 Feb 2011 15:44:37 +0100 (CET)
Subject: [pypy-svn] pypy default: Oups.
Message-ID: <20110216144437.409262A202C@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r42040:56861f9c183a
Date: 2011-02-16 15:42 +0100
http://bitbucket.org/pypy/pypy/changeset/56861f9c183a/

Log:	Oups.

diff --git a/pypy/module/_rawffi/test/test__rawffi.py b/pypy/module/_rawffi/test/test__rawffi.py
--- a/pypy/module/_rawffi/test/test__rawffi.py
+++ b/pypy/module/_rawffi/test/test__rawffi.py
@@ -495,8 +495,8 @@
     def test_invalid_bitfields(self):
         import _rawffi
         raises(TypeError, _rawffi.Structure, [('A', 'c', 1)])
-        raises(TypeError, _rawffi.Structure, [('A', 'I', 129)])
-        raises(TypeError, _rawffi.Structure, [('A', 'I', -1)])
+        raises(ValueError, _rawffi.Structure, [('A', 'I', 129)])
+        raises(ValueError, _rawffi.Structure, [('A', 'I', -1)])
 
     def test_packed_structure(self):
         import _rawffi

From commits-noreply at bitbucket.org  Wed Feb 16 15:44:38 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed, 16 Feb 2011 15:44:38 +0100 (CET)
Subject: [pypy-svn] pypy default: merge heads
Message-ID: <20110216144438.0736E2A2033@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r42041:74ee6879744f
Date: 2011-02-16 15:44 +0100
http://bitbucket.org/pypy/pypy/changeset/74ee6879744f/

Log:	merge heads


From commits-noreply at bitbucket.org  Wed Feb 16 15:52:31 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed, 16 Feb 2011 15:52:31 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix.
Message-ID: <20110216145231.A7A88282BF2@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r42042:9b5294bba4db
Date: 2011-02-16 15:49 +0100
http://bitbucket.org/pypy/pypy/changeset/9b5294bba4db/

Log:	Fix.

diff --git a/pypy/module/_rawffi/test/test__rawffi.py b/pypy/module/_rawffi/test/test__rawffi.py
--- a/pypy/module/_rawffi/test/test__rawffi.py
+++ b/pypy/module/_rawffi/test/test__rawffi.py
@@ -497,6 +497,7 @@
         raises(TypeError, _rawffi.Structure, [('A', 'c', 1)])
         raises(ValueError, _rawffi.Structure, [('A', 'I', 129)])
         raises(ValueError, _rawffi.Structure, [('A', 'I', -1)])
+        raises(ValueError, _rawffi.Structure, [('A', 'I', 0)])
 
     def test_packed_structure(self):
         import _rawffi

diff --git a/pypy/module/_rawffi/structure.py b/pypy/module/_rawffi/structure.py
--- a/pypy/module/_rawffi/structure.py
+++ b/pypy/module/_rawffi/structure.py
@@ -36,7 +36,7 @@
         if len_l == 3:
             bitsize = space.int_w(l_w[2])
 
-            if bitsize < 0 or bitsize > tp.size * 8:
+            if bitsize <= 0 or bitsize > tp.size * 8:
                 raise OperationError(space.w_ValueError, space.wrap(
                     "number of bits invalid for bit field"))
             for c in unroll_letters_for_numbers:

From commits-noreply at bitbucket.org  Wed Feb 16 15:52:32 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed, 16 Feb 2011 15:52:32 +0100 (CET)
Subject: [pypy-svn] pypy default: Add a case where the list of bases is
	empty (hence	"object").
Message-ID: <20110216145232.40F3B282BF2@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r42043:d484be65901c
Date: 2011-02-16 15:50 +0100
http://bitbucket.org/pypy/pypy/changeset/d484be65901c/

Log:	Add a case where the list of bases is empty (hence "object").

diff --git a/lib_pypy/_ctypes/structure.py b/lib_pypy/_ctypes/structure.py
--- a/lib_pypy/_ctypes/structure.py
+++ b/lib_pypy/_ctypes/structure.py
@@ -105,6 +105,7 @@
         res = type.__new__(self, name, cls, typedict)
         if "_abstract_" in typedict:
             return res
+        cls = cls or (object,)
         if isinstance(cls[0], StructOrUnionMeta):
             cls[0]._make_final()
         if '_fields_' in typedict:

From commits-noreply at bitbucket.org  Wed Feb 16 15:52:32 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed, 16 Feb 2011 15:52:32 +0100 (CET)
Subject: [pypy-svn] pypy default: merge heads
Message-ID: <20110216145232.7E65B282BF7@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r42044:1b3f25bf90e7
Date: 2011-02-16 15:52 +0100
http://bitbucket.org/pypy/pypy/changeset/1b3f25bf90e7/

Log:	merge heads


From commits-noreply at bitbucket.org  Wed Feb 16 16:21:32 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed, 16 Feb 2011 16:21:32 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix. Change the tests from xfail()
	to	check_impl_detail()
Message-ID: <20110216152132.3261B282BF2@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r42045:d2f98245d92f
Date: 2011-02-16 16:21 +0100
http://bitbucket.org/pypy/pypy/changeset/d2f98245d92f/

Log:	Fix. Change the tests from xfail() to check_impl_detail() as it is,
	as far as I can tell, really an impl detail.

diff --git a/lib-python/modified-2.7.0/ctypes/test/test_strings.py b/lib-python/modified-2.7.0/ctypes/test/test_strings.py
--- a/lib-python/modified-2.7.0/ctypes/test/test_strings.py
+++ b/lib-python/modified-2.7.0/ctypes/test/test_strings.py
@@ -1,7 +1,6 @@
 import unittest
 from ctypes import *
 from test import test_support
-from ctypes.test import xfail
 
 class StringArrayTestCase(unittest.TestCase):
     def test(self):
@@ -26,24 +25,24 @@
         self.assertRaises(ValueError, setattr, buf, "value", "aaaaaaaa")
         self.assertRaises(TypeError, setattr, buf, "value", 42)
 
-    @xfail
     def test_c_buffer_value(self, memoryview=memoryview):
         buf = c_buffer(32)
 
         buf.value = "Hello, World"
         self.assertEqual(buf.value, "Hello, World")
 
-        self.assertRaises(TypeError, setattr, buf, "value", memoryview("Hello, World"))
-        self.assertRaises(TypeError, setattr, buf, "value", memoryview("abc"))
+        if test_support.check_impl_detail():
+            self.assertRaises(TypeError, setattr, buf, "value", memoryview("Hello, World"))
+            self.assertRaises(TypeError, setattr, buf, "value", memoryview("abc"))
         self.assertRaises(ValueError, setattr, buf, "raw", memoryview("x" * 100))
 
-    @xfail
     def test_c_buffer_raw(self, memoryview=memoryview):
         buf = c_buffer(32)
 
         buf.raw = memoryview("Hello, World")
         self.assertEqual(buf.value, "Hello, World")
-        self.assertRaises(TypeError, setattr, buf, "value", memoryview("abc"))
+        if test_support.check_impl_detail():
+            self.assertRaises(TypeError, setattr, buf, "value", memoryview("abc"))
         self.assertRaises(ValueError, setattr, buf, "raw", memoryview("x" * 100))
 
     def test_c_buffer_deprecated(self):

diff --git a/lib_pypy/_ctypes/array.py b/lib_pypy/_ctypes/array.py
--- a/lib_pypy/_ctypes/array.py
+++ b/lib_pypy/_ctypes/array.py
@@ -31,6 +31,8 @@
                                                    self._length_)
 
                 def setraw(self, buffer):
+                    if len(buffer) > self._length_:
+                        raise ValueError("%r too long" % (buffer,))
                     for i in range(len(buffer)):
                         self[i] = buffer[i]
                 res.raw = property(getraw, setraw)

From commits-noreply at bitbucket.org  Wed Feb 16 16:35:38 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed, 16 Feb 2011 16:35:38 +0100 (CET)
Subject: [pypy-svn] pypy default: Skip this test. No chance to even import
	as long as we	don't
Message-ID: <20110216153538.346E836C053@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r42046:2037e373ba16
Date: 2011-02-16 16:35 +0100
http://bitbucket.org/pypy/pypy/changeset/2037e373ba16/

Log:	Skip this test. No chance to even import as long as we don't have
	'disassembler'.

diff --git a/pypy/module/pypyjit/test/test_pypy_c_new.py b/pypy/module/pypyjit/test/test_pypy_c_new.py
--- a/pypy/module/pypyjit/test/test_pypy_c_new.py
+++ b/pypy/module/pypyjit/test/test_pypy_c_new.py
@@ -1,5 +1,6 @@
 
 import py, sys, re
+py.test.skip("in-progress?")
 import subprocess
 import disassembler
 from pypy.tool.udir import udir

From commits-noreply at bitbucket.org  Wed Feb 16 17:28:32 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed, 16 Feb 2011 17:28:32 +0100 (CET)
Subject: [pypy-svn] pypy default: These tests now (seem to) pass.
Message-ID: <20110216162832.D5FAD282BF2@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r42047:b6ee4b86e547
Date: 2011-02-16 17:13 +0100
http://bitbucket.org/pypy/pypy/changeset/b6ee4b86e547/

Log:	These tests now (seem to) pass.

diff --git a/lib-python/modified-2.7.0/ctypes/test/test_refcounts.py b/lib-python/modified-2.7.0/ctypes/test/test_refcounts.py
--- a/lib-python/modified-2.7.0/ctypes/test/test_refcounts.py
+++ b/lib-python/modified-2.7.0/ctypes/test/test_refcounts.py
@@ -1,7 +1,6 @@
 import unittest
 import ctypes
 import gc
-from ctypes.test import xfail
 
 MyCallback = ctypes.CFUNCTYPE(ctypes.c_int, ctypes.c_int)
 OtherCallback = ctypes.CFUNCTYPE(ctypes.c_int, ctypes.c_int, ctypes.c_ulonglong)
@@ -83,7 +82,6 @@
         self.assertEqual(grc(func), 2)
 
 class AnotherLeak(unittest.TestCase):
-    @xfail
     def test_callback(self):
         import sys
 

diff --git a/lib-python/modified-2.7.0/ctypes/test/test_bitfields.py b/lib-python/modified-2.7.0/ctypes/test/test_bitfields.py
--- a/lib-python/modified-2.7.0/ctypes/test/test_bitfields.py
+++ b/lib-python/modified-2.7.0/ctypes/test/test_bitfields.py
@@ -143,7 +143,6 @@
         result = self.fail_fields(("a", Dummy, 1))
         self.assertEqual(result, (TypeError, 'bit fields not allowed for type Dummy'))
 
-    @xfail
     def test_single_bitfield_size(self):
         for c_typ in int_types:
             result = self.fail_fields(("a", c_typ, -1))

From commits-noreply at bitbucket.org  Wed Feb 16 17:28:33 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed, 16 Feb 2011 17:28:33 +0100 (CET)
Subject: [pypy-svn] pypy default: Make another xfail test pass, probably.
Message-ID: <20110216162833.BA7B5282BF2@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r42048:90fbed3c6314
Date: 2011-02-16 16:28 +0000
http://bitbucket.org/pypy/pypy/changeset/90fbed3c6314/

Log:	Make another xfail test pass, probably.

diff --git a/lib-python/modified-2.7.0/ctypes/test/test_bitfields.py b/lib-python/modified-2.7.0/ctypes/test/test_bitfields.py
--- a/lib-python/modified-2.7.0/ctypes/test/test_bitfields.py
+++ b/lib-python/modified-2.7.0/ctypes/test/test_bitfields.py
@@ -4,7 +4,6 @@
 
 import ctypes
 import _ctypes_test
-from ctypes.test import xfail
 
 class BITS(Structure):
     _fields_ = [("A", c_int, 1),
@@ -113,21 +112,24 @@
         return self.get_except(type(Structure), "X", (),
                                {"_fields_": fields})
 
-    @xfail
     def test_nonint_types(self):
         # bit fields are not allowed on non-integer types.
         result = self.fail_fields(("a", c_char_p, 1))
-        self.assertEqual(result, (TypeError, 'bit fields not allowed for type c_char_p'))
+        self.assertEqual(result[0], TypeError)
+        self.assertIn('bit fields not allowed for type', result[1])
 
         result = self.fail_fields(("a", c_void_p, 1))
-        self.assertEqual(result, (TypeError, 'bit fields not allowed for type c_void_p'))
+        self.assertEqual(result[0], TypeError)
+        self.assertIn('bit fields not allowed for type', result[1])
 
         if c_int != c_long:
             result = self.fail_fields(("a", POINTER(c_int), 1))
-            self.assertEqual(result, (TypeError, 'bit fields not allowed for type LP_c_int'))
+            self.assertEqual(result[0], TypeError)
+            self.assertIn('bit fields not allowed for type', result[1])
 
         result = self.fail_fields(("a", c_char, 1))
-        self.assertEqual(result, (TypeError, 'bit fields not allowed for type c_char'))
+        self.assertEqual(result[0], TypeError)
+        self.assertIn('bit fields not allowed for type', result[1])
 
         try:
             c_wchar
@@ -135,13 +137,15 @@
             pass
         else:
             result = self.fail_fields(("a", c_wchar, 1))
-            self.assertEqual(result, (TypeError, 'bit fields not allowed for type c_wchar'))
+            self.assertEqual(result[0], TypeError)
+            self.assertIn('bit fields not allowed for type', result[1])
 
         class Dummy(Structure):
             _fields_ = []
 
         result = self.fail_fields(("a", Dummy, 1))
-        self.assertEqual(result, (TypeError, 'bit fields not allowed for type Dummy'))
+        self.assertEqual(result[0], TypeError)
+        self.assertIn('bit fields not allowed for type', result[1])
 
     def test_single_bitfield_size(self):
         for c_typ in int_types:

diff --git a/pypy/module/_rawffi/structure.py b/pypy/module/_rawffi/structure.py
--- a/pypy/module/_rawffi/structure.py
+++ b/pypy/module/_rawffi/structure.py
@@ -34,17 +34,16 @@
         tp = unpack_shape_with_length(space, l_w[1])
 
         if len_l == 3:
-            bitsize = space.int_w(l_w[2])
-
-            if bitsize <= 0 or bitsize > tp.size * 8:
-                raise OperationError(space.w_ValueError, space.wrap(
-                    "number of bits invalid for bit field"))
             for c in unroll_letters_for_numbers:
                 if c == tp.itemcode:
                     break
             else:
                 raise OperationError(space.w_TypeError, space.wrap(
                     "bit fields not allowed for type"))
+            bitsize = space.int_w(l_w[2])
+            if bitsize <= 0 or bitsize > tp.size * 8:
+                raise OperationError(space.w_ValueError, space.wrap(
+                    "number of bits invalid for bit field"))
         else:
             bitsize = 0
 

From commits-noreply at bitbucket.org  Wed Feb 16 18:46:13 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Wed, 16 Feb 2011 18:46:13 +0100 (CET)
Subject: [pypy-svn] pypy default: Document len_w.
Message-ID: <20110216174613.CB50C36C374@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r42049:e8cfe2e69afe
Date: 2011-02-16 12:45 -0500
http://bitbucket.org/pypy/pypy/changeset/e8cfe2e69afe/

Log:	Document len_w.

diff --git a/pypy/doc/objspace.txt b/pypy/doc/objspace.txt
--- a/pypy/doc/objspace.txt
+++ b/pypy/doc/objspace.txt
@@ -142,6 +142,9 @@
 ``hash_w(w_obj):``
   Shortcut for space.int_w(space.hash(w_obj))
 
+``len_w(w_obj):``
+  Shortcut for space.int_w(space.len(w_obj))
+
 ``not_(w_obj):``
   Shortcut for space.newbool(not space.is_true(w_obj))
 

From commits-noreply at bitbucket.org  Wed Feb 16 19:19:11 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 19:19:11 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: modernize unwrap_spec in the
	__builtin__	module
Message-ID: <20110216181911.14E79282BF2@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42050:3ed721ed67a1
Date: 2011-02-16 11:02 +0100
http://bitbucket.org/pypy/pypy/changeset/3ed721ed67a1/

Log:	modernize unwrap_spec in the __builtin__ module

diff --git a/pypy/module/__builtin__/interp_memoryview.py b/pypy/module/__builtin__/interp_memoryview.py
--- a/pypy/module/__builtin__/interp_memoryview.py
+++ b/pypy/module/__builtin__/interp_memoryview.py
@@ -3,7 +3,7 @@
 """
 from pypy.interpreter.baseobjspace import Wrappable
 from pypy.interpreter import gateway, buffer
-from pypy.interpreter.gateway import interp2app, unwrap_spec, ObjSpace, W_Root
+from pypy.interpreter.gateway import interp2app, unwrap_spec
 from pypy.interpreter.typedef import TypeDef, GetSetProperty
 from pypy.interpreter.error import OperationError
 import operator
@@ -39,7 +39,6 @@
                 str1 = self.as_str()
                 str2 = space.buffer_w(w_buf).as_str()
                 return space.wrap(getattr(operator, name)(str1, str2))
-        descr__cmp.unwrap_spec = ['self', ObjSpace, W_Root]
         descr__cmp.func_name = name
         return descr__cmp
 
@@ -69,15 +68,12 @@
             buf = buffer.SubBuffer(buf, start, size)
         return W_MemoryView(buf)
 
-    @unwrap_spec('self', ObjSpace)
     def descr_buffer(self, space):
         return space.wrap(self.buf)
 
-    @unwrap_spec('self', ObjSpace)
     def descr_tobytes(self, space):
         return space.wrap(self.as_str())
 
-    @unwrap_spec('self', ObjSpace)
     def descr_tolist(self, space):
         buf = self.buf
         result = []
@@ -85,7 +81,6 @@
             result.append(space.wrap(ord(buf.getitem(i))))
         return space.newlist(result)
 
-    @unwrap_spec('self', ObjSpace, W_Root)
     def descr_getitem(self, space, w_index):
         start, stop, step = space.decode_index(w_index, self.getlength())
         if step == 0:  # index only
@@ -98,7 +93,7 @@
                 space.wrap("memoryview object does not support"
                            " slicing with a step"))
 
-    @unwrap_spec('self', ObjSpace, W_Root, 'bufferstr')
+    @unwrap_spec(newstring='bufferstr')
     def descr_setitem(self, space, w_index, newstring):
         buf = self.buf
         if isinstance(buf, buffer.RWBuffer):
@@ -107,7 +102,6 @@
             raise OperationError(space.w_TypeError,
                                  space.wrap("cannot modify read-only memory"))
 
-    @unwrap_spec('self', ObjSpace)
     def descr_len(self, space):
         return self.buf.descr_len(space)
 
@@ -128,7 +122,6 @@
         return space.w_None
 
 
- at unwrap_spec(ObjSpace, W_Root, W_Root)
 def descr_new(space, w_subtype, w_object):
     memoryview = W_MemoryView(space.buffer(w_object))
     return space.wrap(memoryview)

diff --git a/pypy/module/__builtin__/compiling.py b/pypy/module/__builtin__/compiling.py
--- a/pypy/module/__builtin__/compiling.py
+++ b/pypy/module/__builtin__/compiling.py
@@ -3,11 +3,11 @@
 """
 
 from pypy.interpreter.pycode import PyCode
-from pypy.interpreter.baseobjspace import W_Root, ObjSpace
 from pypy.interpreter.error import OperationError
 from pypy.interpreter.astcompiler import consts, ast
-from pypy.interpreter.gateway import NoneNotWrapped
+from pypy.interpreter.gateway import NoneNotWrapped, unwrap_spec
 
+ at unwrap_spec(filename=str, mode=str, flags=int, dont_inherit=int)
 def compile(space, w_source, filename, mode, flags=0, dont_inherit=0):
     """Compile the source string (a Python module, statement or expression)
 into a code object that can be executed by the exec statement or eval().
@@ -61,8 +61,6 @@
     else:
         code = ec.compiler.compile_ast(ast_node, filename, mode, flags)
     return space.wrap(code)
-#
-compile.unwrap_spec = [ObjSpace,W_Root,str,str,int,int]
 
 
 def eval(space, w_code, w_globals=None, w_locals=None):

diff --git a/pypy/module/__builtin__/interp_classobj.py b/pypy/module/__builtin__/interp_classobj.py
--- a/pypy/module/__builtin__/interp_classobj.py
+++ b/pypy/module/__builtin__/interp_classobj.py
@@ -1,9 +1,7 @@
 import new
 from pypy.interpreter.error import OperationError, operationerrfmt
-from pypy.interpreter.gateway import (ObjSpace, W_Root, NoneNotWrapped,
-    applevel, interp2app, ObjSpace, unwrap_spec)
+from pypy.interpreter.gateway import NoneNotWrapped, applevel, interp2app
 from pypy.interpreter.typedef import TypeDef, make_weakref_descr
-from pypy.interpreter.argument import Arguments
 from pypy.interpreter.baseobjspace import Wrappable
 from pypy.interpreter.typedef import GetSetProperty, descr_get_dict
 from pypy.interpreter.typedef import descr_set_dict
@@ -233,18 +231,12 @@
 
 W_ClassObject.typedef = TypeDef("classobj",
     __new__ = interp2app(descr_classobj_new),
-    __repr__ = interp2app(W_ClassObject.descr_repr,
-                          unwrap_spec=['self', ObjSpace]),
-    __str__ = interp2app(W_ClassObject.descr_str,
-                         unwrap_spec=['self', ObjSpace]),
-    __call__ = interp2app(class_descr_call,
-                          unwrap_spec=[ObjSpace, W_Root, Arguments]),
-    __getattribute__ = interp2app(W_ClassObject.descr_getattribute,
-                             unwrap_spec=['self', ObjSpace, W_Root]),
-    __setattr__ = interp2app(W_ClassObject.descr_setattr,
-                             unwrap_spec=['self', ObjSpace, W_Root, W_Root]),
-    __delattr__ = interp2app(W_ClassObject.descr_delattr,
-                             unwrap_spec=['self', ObjSpace, W_Root]),
+    __repr__ = interp2app(W_ClassObject.descr_repr),
+    __str__ = interp2app(W_ClassObject.descr_str),
+    __call__ = interp2app(class_descr_call),
+    __getattribute__ = interp2app(W_ClassObject.descr_getattribute),
+    __setattr__ = interp2app(W_ClassObject.descr_setattr),
+    __delattr__ = interp2app(W_ClassObject.descr_delattr),
     __weakref__ = make_weakref_descr(W_ClassObject),
 )
 W_ClassObject.typedef.acceptable_as_base_class = False
@@ -454,7 +446,6 @@
             return self.descr_str(space)
         return space.call_function(w_meth)
 
-    @unwrap_spec("self", ObjSpace, W_Root)
     def descr_format(self, space, w_format_spec):
         w_meth = self.getattr(space, "__format__", False)
         if w_meth is not None:
@@ -737,9 +728,7 @@
         make_unary_instance_method(specialname),
         None,
         W_InstanceObject)
-    rawdict[specialname] = interp2app(
-        meth,
-        unwrap_spec=["self", ObjSpace])
+    rawdict[specialname] = interp2app(meth)
 
 # binary operations that return NotImplemented if they fail
 # e.g. rich comparisons, coerce and inplace ops
@@ -750,9 +739,7 @@
         make_binary_returning_notimplemented_instance_method(specialname),
         None,
         W_InstanceObject)
-    rawdict[specialname] = interp2app(
-        meth,
-        unwrap_spec=["self", ObjSpace, W_Root])
+    rawdict[specialname] = interp2app(meth)
 
 for op in "or and xor lshift rshift add sub mul div mod divmod floordiv truediv".split():
     specialname = "__%s__" % (op, )
@@ -760,13 +747,9 @@
     func, rfunc = make_binary_instance_method(op)
     # fool the gateway logic by giving it a real unbound method
     meth = new.instancemethod(func, None, W_InstanceObject)
-    rawdict[specialname] = interp2app(
-        meth,
-        unwrap_spec=["self", ObjSpace, W_Root])
+    rawdict[specialname] = interp2app(meth)
     rmeth = new.instancemethod(rfunc, None, W_InstanceObject)
-    rawdict[rspecialname] = interp2app(
-        rmeth,
-        unwrap_spec=["self", ObjSpace, W_Root])
+    rawdict[rspecialname] = interp2app(meth)
 
 
 def descr_del_dict(space, w_inst):
@@ -778,62 +761,34 @@
 
 W_InstanceObject.typedef = TypeDef("instance",
     __new__ = interp2app(descr_instance_new),
-    __getattribute__ = interp2app(W_InstanceObject.descr_getattribute,
-                                  unwrap_spec=['self', ObjSpace, W_Root]),
-    __setattr__ = interp2app(W_InstanceObject.descr_setattr,
-                             unwrap_spec=['self', ObjSpace, W_Root, W_Root]),
-    __delattr__ = interp2app(W_InstanceObject.descr_delattr,
-                             unwrap_spec=['self', ObjSpace, W_Root]),
-    __repr__ = interp2app(W_InstanceObject.descr_repr,
-                          unwrap_spec=['self', ObjSpace]),
-    __str__ = interp2app(W_InstanceObject.descr_str,
-                         unwrap_spec=['self', ObjSpace]),
-    __unicode__ = interp2app(W_InstanceObject.descr_unicode,
-                         unwrap_spec=['self', ObjSpace]),
+    __getattribute__ = interp2app(W_InstanceObject.descr_getattribute),
+    __setattr__ = interp2app(W_InstanceObject.descr_setattr),
+    __delattr__ = interp2app(W_InstanceObject.descr_delattr),
+    __repr__ = interp2app(W_InstanceObject.descr_repr),
+    __str__ = interp2app(W_InstanceObject.descr_str),
+    __unicode__ = interp2app(W_InstanceObject.descr_unicode),
     __format__ = interp2app(W_InstanceObject.descr_format),
-    __len__ = interp2app(W_InstanceObject.descr_len,
-                         unwrap_spec=['self', ObjSpace]),
-    __getitem__ = interp2app(W_InstanceObject.descr_getitem,
-                             unwrap_spec=['self', ObjSpace, W_Root]),
-    __setitem__ = interp2app(W_InstanceObject.descr_setitem,
-                             unwrap_spec=['self', ObjSpace, W_Root, W_Root]),
-    __delitem__ = interp2app(W_InstanceObject.descr_delitem,
-                             unwrap_spec=['self', ObjSpace, W_Root]),
-    __iter__ = interp2app(W_InstanceObject.descr_iter,
-                          unwrap_spec=['self', ObjSpace]),
-    __getslice__ = interp2app(W_InstanceObject.descr_getslice,
-                             unwrap_spec=['self', ObjSpace, W_Root, W_Root]),
-    __setslice__ = interp2app(W_InstanceObject.descr_setslice,
-                             unwrap_spec=['self', ObjSpace, W_Root,
-                                          W_Root, W_Root]),
-    __delslice__ = interp2app(W_InstanceObject.descr_delslice,
-                             unwrap_spec=['self', ObjSpace, W_Root, W_Root]),
-    __call__ = interp2app(W_InstanceObject.descr_call,
-                          unwrap_spec=['self', ObjSpace, Arguments]),
-    __nonzero__ = interp2app(W_InstanceObject.descr_nonzero,
-                             unwrap_spec=['self', ObjSpace]),
-    __cmp__ = interp2app(W_InstanceObject.descr_cmp,
-                         unwrap_spec=['self', ObjSpace, W_Root]),
-    __hash__ = interp2app(W_InstanceObject.descr_hash,
-                          unwrap_spec=['self', ObjSpace]),
-    __int__ = interp2app(W_InstanceObject.descr_int,
-                           unwrap_spec=['self', ObjSpace]),
-    __long__ = interp2app(W_InstanceObject.descr_long,
-                           unwrap_spec=['self', ObjSpace]),
-    __index__ = interp2app(W_InstanceObject.descr_index,
-                           unwrap_spec=['self', ObjSpace]),
-    __contains__ = interp2app(W_InstanceObject.descr_contains,
-                         unwrap_spec=['self', ObjSpace, W_Root]),
-    __pow__ = interp2app(W_InstanceObject.descr_pow,
-                         unwrap_spec=['self', ObjSpace, W_Root, W_Root]),
-    __rpow__ = interp2app(W_InstanceObject.descr_rpow,
-                         unwrap_spec=['self', ObjSpace, W_Root, W_Root]),
-    next = interp2app(W_InstanceObject.descr_next,
-                      unwrap_spec=['self', ObjSpace]),
-    __del__ = interp2app(W_InstanceObject.descr_del,
-                         unwrap_spec=['self', ObjSpace]),
-    __exit__ = interp2app(W_InstanceObject.descr_exit,
-                          unwrap_spec=['self', ObjSpace, W_Root, W_Root, W_Root]),
+    __len__ = interp2app(W_InstanceObject.descr_len),
+    __getitem__ = interp2app(W_InstanceObject.descr_getitem),
+    __setitem__ = interp2app(W_InstanceObject.descr_setitem),
+    __delitem__ = interp2app(W_InstanceObject.descr_delitem),
+    __iter__ = interp2app(W_InstanceObject.descr_iter),
+    __getslice__ = interp2app(W_InstanceObject.descr_getslice),
+    __setslice__ = interp2app(W_InstanceObject.descr_setslice),
+    __delslice__ = interp2app(W_InstanceObject.descr_delslice),
+    __call__ = interp2app(W_InstanceObject.descr_call),
+    __nonzero__ = interp2app(W_InstanceObject.descr_nonzero),
+    __cmp__ = interp2app(W_InstanceObject.descr_cmp),
+    __hash__ = interp2app(W_InstanceObject.descr_hash),
+    __int__ = interp2app(W_InstanceObject.descr_int),
+    __long__ = interp2app(W_InstanceObject.descr_long),
+    __index__ = interp2app(W_InstanceObject.descr_index),
+    __contains__ = interp2app(W_InstanceObject.descr_contains),
+    __pow__ = interp2app(W_InstanceObject.descr_pow),
+    __rpow__ = interp2app(W_InstanceObject.descr_rpow),
+    next = interp2app(W_InstanceObject.descr_next),
+    __del__ = interp2app(W_InstanceObject.descr_del),
+    __exit__ = interp2app(W_InstanceObject.descr_exit),
     __dict__ = dict_descr,
     **rawdict
 )

diff --git a/pypy/module/__builtin__/operation.py b/pypy/module/__builtin__/operation.py
--- a/pypy/module/__builtin__/operation.py
+++ b/pypy/module/__builtin__/operation.py
@@ -3,9 +3,8 @@
 """
 
 from pypy.interpreter import gateway
-from pypy.interpreter.baseobjspace import ObjSpace, W_Root
 from pypy.interpreter.error import OperationError
-from pypy.interpreter.gateway import interp2app
+from pypy.interpreter.gateway import interp2app, unwrap_spec
 from pypy.interpreter.typedef import TypeDef
 from pypy.rlib.runicode import UNICHR
 from pypy.rlib.rarithmetic import isnan, isinf, round_double
@@ -27,6 +26,7 @@
                              space.wrap("character code not in range(256)"))
     return space.wrap(char)
 
+ at unwrap_spec(code=int)
 def unichr(space, code):
     "Return a Unicode string of one character with the given ordinal."
     # XXX range checking!
@@ -36,7 +36,6 @@
         raise OperationError(space.w_ValueError,
                              space.wrap("unichr() arg out of range"))
     return space.wrap(c)
-unichr.unwrap_spec = [ObjSpace, int]
 
 def len(space, w_obj):
     "len(object) -> integer\n\nReturn the number of items of a sequence or mapping."
@@ -128,6 +127,7 @@
 NDIGITS_MAX = int((rfloat.DBL_MANT_DIG - rfloat.DBL_MIN_EXP) * 0.30103)
 NDIGITS_MIN = -int((rfloat.DBL_MAX_EXP + 1) * 0.30103)
 
+ at unwrap_spec(number=float)
 def round(space, number, w_ndigits=0):
     """round(number[, ndigits]) -> floating point number
 
@@ -157,8 +157,6 @@
         raise OperationError(space.w_OverflowError,
                              space.wrap("rounded value too large to represent"))
     return space.wrap(z)
-#
-round.unwrap_spec = [ObjSpace, float, W_Root]
 
 # ____________________________________________________________
 

diff --git a/pypy/module/__builtin__/descriptor.py b/pypy/module/__builtin__/descriptor.py
--- a/pypy/module/__builtin__/descriptor.py
+++ b/pypy/module/__builtin__/descriptor.py
@@ -1,8 +1,7 @@
 
 from pypy.interpreter.typedef import TypeDef, GetSetProperty
-from pypy.interpreter.baseobjspace import W_Root, ObjSpace, Wrappable, \
-     Arguments
-from pypy.interpreter.gateway import interp2app
+from pypy.interpreter.baseobjspace import Wrappable
+from pypy.interpreter.gateway import interp2app, unwrap_spec
 from pypy.interpreter.error import OperationError, operationerrfmt
 from pypy.objspace.descroperation import object_getattribute, object_setattr
 from pypy.interpreter.function import StaticMethod, ClassMethod
@@ -24,8 +23,8 @@
             #     XXX write a fast path for this common case
             w_selftype = space.type(w(self))
             return space.call_function(w_selftype, self.w_starttype, w_obj)
-    get.unwrap_spec = ['self', ObjSpace, W_Root, W_Root]
 
+    @unwrap_spec(name=str)
     def getattribute(self, space, name):
         w = space.wrap
         # only use a special logic for bound super objects and not for
@@ -49,7 +48,6 @@
         # fallback to object.__getattribute__()
         return space.call_function(object_getattribute(space),
                                    w(self), w(name))
-    getattribute.unwrap_spec = ['self', ObjSpace, str]
 
 def descr_new_super(space, w_subtype, w_starttype, w_obj_or_type=None):
     if space.is_w(w_obj_or_type, space.w_None):
@@ -77,7 +75,6 @@
     w_result = space.allocate_instance(W_Super, w_subtype)
     W_Super.__init__(w_result, space, w_starttype, w_type, w_obj_or_type)
     return w_result
-descr_new_super.unwrap_spec = [ObjSpace, W_Root, W_Root, W_Root]
 
 W_Super.typedef = TypeDef(
     'super',
@@ -115,12 +112,10 @@
                     space.setattr(space.wrap(self), space.wrap("__doc__"),
                                   w_getter_doc)
                 self.getter_doc = True
-    init.unwrap_spec = ['self', ObjSpace, W_Root, W_Root, W_Root, W_Root]
 
     def new(space, w_subtype, w_fget=None, w_fset=None, w_fdel=None, w_doc=None):
         w_result = space.allocate_instance(W_Property, w_subtype)
         return w_result
-    new.unwrap_spec = [ObjSpace, W_Root, W_Root, W_Root, W_Root, W_Root]
 
     def get(self, space, w_obj, w_objtype=None):
         if space.is_w(w_obj, space.w_None):
@@ -129,7 +124,6 @@
             raise OperationError(space.w_AttributeError, space.wrap(
                 "unreadable attribute"))
         return space.call_function(self.w_fget, w_obj)
-    get.unwrap_spec = ['self', ObjSpace, W_Root, W_Root]
 
     def set(self, space, w_obj, w_value):
         if space.is_w(self.w_fset, space.w_None):
@@ -137,7 +131,6 @@
                 "can't set attribute"))
         space.call_function(self.w_fset, w_obj, w_value)
         return space.w_None
-    set.unwrap_spec = ['self', ObjSpace, W_Root, W_Root]
 
     def delete(self, space, w_obj):
         if space.is_w(self.w_fdel, space.w_None):
@@ -145,19 +138,15 @@
                 "can't delete attribute"))
         space.call_function(self.w_fdel, w_obj)
         return space.w_None
-    delete.unwrap_spec = ['self', ObjSpace, W_Root]
 
     def getter(self, space, w_getter):
         return self._copy(space, w_getter=w_getter)
-    getter.unwrap_spec = ['self', ObjSpace, W_Root]
 
     def setter(self, space, w_setter):
         return self._copy(space, w_setter=w_setter)
-    setter.unwrap_spec = ['self', ObjSpace, W_Root]
 
     def deleter(self, space, w_deleter):
         return self._copy(space, w_deleter=w_deleter)
-    deleter.unwrap_spec = ['self', ObjSpace, W_Root]
 
     def _copy(self, space, w_getter=None, w_setter=None, w_deleter=None):
         if w_getter is None:

diff --git a/pypy/module/__builtin__/functional.py b/pypy/module/__builtin__/functional.py
--- a/pypy/module/__builtin__/functional.py
+++ b/pypy/module/__builtin__/functional.py
@@ -4,11 +4,10 @@
 """
 
 from pypy.interpreter.error import OperationError
-from pypy.interpreter.gateway import ObjSpace, W_Root, NoneNotWrapped, applevel
-from pypy.interpreter.gateway import interp2app
+from pypy.interpreter.gateway import NoneNotWrapped, applevel
+from pypy.interpreter.gateway import interp2app, unwrap_spec
 from pypy.interpreter.typedef import TypeDef
 from pypy.interpreter.baseobjspace import Wrappable
-from pypy.interpreter.argument import Arguments
 from pypy.rlib.rarithmetic import r_uint, intmask
 from pypy.rlib.objectmodel import specialize
 from inspect import getsource, getfile
@@ -96,7 +95,6 @@
         res_w[idx] = space.wrap(v)
         v += step
     return space.newlist(res_w)
-range_int.unwrap_spec = [ObjSpace, W_Root, W_Root, W_Root]
 
 
 def range_withspecialized_implementation(space, start, step, howmany):
@@ -211,7 +209,6 @@
     With two or more arguments, return the largest argument.
     """
     return min_max(space, __args__, "max")
-max.unwrap_spec = [ObjSpace, Arguments]
 
 def min(space, __args__):
     """Return the smallest item in a sequence.
@@ -219,8 +216,8 @@
     If more than one argument is passed, return the minimum of them.
     """
     return min_max(space, __args__, "min")
-min.unwrap_spec = [ObjSpace, Arguments]
 
+ at unwrap_spec(collections_w="args_w")
 def map(space, w_func, collections_w):
     """does 3 separate things, hence this enormous docstring.
        1.  if function is None, return a list of tuples, each with one
@@ -250,7 +247,6 @@
         result_w = map_multiple_collections(space, w_func, collections_w,
                                             none_func)
     return space.newlist(result_w)
-map.unwrap_spec = [ObjSpace, W_Root, "args_w"]
 
 def map_single_collection(space, w_func, w_collection):
     """Special case for 'map(func, coll)', where 'func' is not None and there
@@ -350,8 +346,8 @@
             break
         w_last = space.add(w_last, w_next)
     return w_last
-sum.unwrap_spec = [ObjSpace, W_Root, W_Root]
 
+ at unwrap_spec(sequences_w="args_w")
 def zip(space, sequences_w):
     """Return a list of tuples, where the nth tuple contains every nth item of
     each collection.
@@ -371,7 +367,6 @@
                 raise
             return space.newlist(result_w)
         result_w.append(space.newtuple(items_w))
-zip.unwrap_spec = [ObjSpace, "args_w"]
 
 def reduce(space, w_func, w_sequence, w_initial=NoneNotWrapped):
     """ Apply function of two arguments cumulatively to the items of sequence,
@@ -397,7 +392,6 @@
             break
         w_result = space.call_function(w_func, w_result, w_next)
     return w_result
-reduce.unwrap_spec = [ObjSpace, W_Root, W_Root, W_Root]
 
 def filter(space, w_func, w_seq):
     """construct a list of those elements of collection for which function
@@ -473,7 +467,6 @@
         if not space.is_true(w_next):
             return space.w_False
     return space.w_True
-all.unwrap_spec = [ObjSpace, W_Root]
 
 
 def any(space, w_S):
@@ -491,7 +484,6 @@
         if space.is_true(w_next):
             return space.w_True
     return space.w_False
-any.unwrap_spec = [ObjSpace, W_Root]
 
 
 class W_Enumerate(Wrappable):
@@ -511,14 +503,12 @@
 
     def descr___iter__(self, space):
         return space.wrap(self)
-    descr___iter__.unwrap_spec = ["self", ObjSpace]
 
     def descr_next(self, space):
         w_item = space.next(self.w_iter)
         w_index = self.w_index
         self.w_index = space.add(w_index, space.wrap(1))
         return space.newtuple([w_index, w_item])
-    descr_next.unwrap_spec = ["self", ObjSpace]
 
     def descr___reduce__(self, space):
         from pypy.interpreter.mixedmodule import MixedModule
@@ -527,7 +517,6 @@
         w_new_inst = mod.get('enumerate_new')
         w_info = space.newtuple([self.w_iter, self.w_index])
         return space.newtuple([w_new_inst, w_info])
-    descr___reduce__.unwrap_spec = ["self", ObjSpace]
 
 # exported through _pickle_support
 def _make_enumerate(space, w_iter, w_index):
@@ -553,7 +542,6 @@
     if w_reversed is not None:
         return space.call_function(w_reversed)
     return space.wrap(W_ReversedIterator(space, w_sequence))
-reversed.unwrap_spec = [ObjSpace, W_Root]
 
 class W_ReversedIterator(Wrappable):
 
@@ -566,7 +554,6 @@
 
     def descr___iter__(self, space):
         return space.wrap(self)
-    descr___iter__.unwrap_spec = ["self", ObjSpace]
 
     def descr_next(self, space):
         if self.remaining >= 0:
@@ -583,7 +570,6 @@
         # Done
         self.remaining = -1
         raise OperationError(space.w_StopIteration, space.w_None)
-    descr_next.unwrap_spec = ["self", ObjSpace]
 
     def descr___reduce__(self, space):
         from pypy.interpreter.mixedmodule import MixedModule
@@ -593,7 +579,6 @@
         info_w = [self.w_sequence, space.wrap(self.remaining)]
         w_info = space.newtuple(info_w)
         return space.newtuple([w_new_inst, w_info])
-    descr___reduce__.unwrap_spec = ["self", ObjSpace]
 
 W_ReversedIterator.typedef = TypeDef("reversed",
     __iter__=interp2app(W_ReversedIterator.descr___iter__),
@@ -643,6 +628,7 @@
     def descr_len(self):
         return self.space.wrap(self.len)
 
+    @unwrap_spec(i='index')
     def descr_getitem(self, i):
         # xrange does NOT support slicing
         space = self.space
@@ -680,8 +666,7 @@
 W_XRange.typedef = TypeDef("xrange",
     __new__          = interp2app(W_XRange.descr_new.im_func),
     __repr__         = interp2app(W_XRange.descr_repr),
-    __getitem__      = interp2app(W_XRange.descr_getitem,
-                                  unwrap_spec=['self', 'index']),
+    __getitem__      = interp2app(W_XRange.descr_getitem),
     __iter__         = interp2app(W_XRange.descr_iter),
     __len__          = interp2app(W_XRange.descr_len),
     __reversed__     = interp2app(W_XRange.descr_reversed),

From commits-noreply at bitbucket.org  Wed Feb 16 19:19:13 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 19:19:13 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: More progress
Message-ID: <20110216181913.2B94E282BF2@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42051:0f2ff1b4d199
Date: 2011-02-16 11:15 +0100
http://bitbucket.org/pypy/pypy/changeset/0f2ff1b4d199/

Log:	More progress

diff --git a/pypy/module/_demo/demo.py b/pypy/module/_demo/demo.py
--- a/pypy/module/_demo/demo.py
+++ b/pypy/module/_demo/demo.py
@@ -18,6 +18,7 @@
     return space.getattr(w_module, space.wrap(name))
 
 
+ at unwrap_spec(repetitions=int)
 def measuretime(space, repetitions, w_callable):
     if repetitions <= 0:
         w_DemoError = get(space, 'DemoError')
@@ -28,8 +29,8 @@
         space.call_function(w_callable)
     endtime = time(0)
     return space.wrap(endtime - starttime)
-measuretime.unwrap_spec = [ObjSpace, int, W_Root]
 
+ at unwrap_spec(n=int)
 def sieve(space, n):
     lst = range(2, n + 1)
     head = 0
@@ -46,7 +47,6 @@
                 newlst.append(element)
         lst = newlst
         head += 1
-sieve.unwrap_spec = [ObjSpace, int]
  
 class W_MyType(Wrappable):
     def __init__(self, space, x=1):
@@ -64,11 +64,11 @@
     def fset_x(space, self, w_value):
         self.x = space.int_w(w_value)
 
+ at unwrap_spec(x=int)
 def mytype_new(space, w_subtype, x):
     if x == 3:
         return space.wrap(MySubType(space, x))
     return space.wrap(W_MyType(space, x))
-mytype_new.unwrap_spec = [ObjSpace, W_Root, int]
 
 getset_x = GetSetProperty(W_MyType.fget_x, W_MyType.fset_x, cls=W_MyType)
 

diff --git a/pypy/module/__pypy__/interp_magic.py b/pypy/module/__pypy__/interp_magic.py
--- a/pypy/module/__pypy__/interp_magic.py
+++ b/pypy/module/__pypy__/interp_magic.py
@@ -1,5 +1,5 @@
 from pypy.interpreter.error import OperationError
-from pypy.interpreter.gateway import ObjSpace
+from pypy.interpreter.gateway import unwrap_spec
 from pypy.rlib.objectmodel import we_are_translated
 from pypy.objspace.std.typeobject import MethodCache
 from pypy.objspace.std.mapdict import IndexCache
@@ -22,6 +22,7 @@
     import pdb
     pdb.set_trace()
 
+ at unwrap_spec(name=str)
 def method_cache_counter(space, name):
     """Return a tuple (method_cache_hits, method_cache_misses) for calls to
     methods with the name."""
@@ -29,7 +30,6 @@
     cache = space.fromcache(MethodCache)
     return space.newtuple([space.newint(cache.hits.get(name, 0)),
                            space.newint(cache.misses.get(name, 0))])
-method_cache_counter.unwrap_spec = [ObjSpace, str]
 
 def reset_method_cache_counter(space):
     """Reset the method cache counter to zero for all method names."""
@@ -42,6 +42,7 @@
         cache.misses = {}
         cache.hits = {}
 
+ at unwrap_spec(name=str)
 def mapdict_cache_counter(space, name):
     """Return a tuple (index_cache_hits, index_cache_misses) for lookups
     in the mapdict cache with the given attribute name."""
@@ -50,7 +51,6 @@
     cache = space.fromcache(IndexCache)
     return space.newtuple([space.newint(cache.hits.get(name, 0)),
                            space.newint(cache.misses.get(name, 0))])
-mapdict_cache_counter.unwrap_spec = [ObjSpace, str]
 
 def builtinify(space, w_func):
     from pypy.interpreter.function import Function, BuiltinFunction

diff --git a/pypy/module/__pypy__/interp_identitydict.py b/pypy/module/__pypy__/interp_identitydict.py
--- a/pypy/module/__pypy__/interp_identitydict.py
+++ b/pypy/module/__pypy__/interp_identitydict.py
@@ -1,59 +1,47 @@
 from pypy.interpreter.error import OperationError
 from pypy.interpreter.typedef import TypeDef
-from pypy.interpreter.gateway import ObjSpace, W_Root, NoneNotWrapped, interp2app
-from pypy.interpreter.gateway import Arguments, unwrap_spec
+from pypy.interpreter.gateway import NoneNotWrapped, interp2app, unwrap_spec
 from pypy.interpreter.baseobjspace import Wrappable
 
 class W_IdentityDict(Wrappable):
     def __init__(self, space):
         self.dict = {}
-    __init__.unwrap_spec = ['self', ObjSpace]
 
-    @unwrap_spec(ObjSpace, W_Root)
     def descr_new(space, w_subtype):
         self = space.allocate_instance(W_IdentityDict, w_subtype)
         W_IdentityDict.__init__(self, space)
         return space.wrap(self)
 
-    @unwrap_spec('self', ObjSpace)
     def descr_len(self, space):
         return space.wrap(len(self.dict))
 
-    @unwrap_spec('self', ObjSpace, W_Root)
     def descr_contains(self, space, w_key):
         return space.wrap(w_key in self.dict)
 
-    @unwrap_spec('self', ObjSpace, W_Root, W_Root)
     def descr_setitem(self, space, w_key, w_value):
         self.dict[w_key] = w_value
 
-    @unwrap_spec('self', ObjSpace, W_Root)
     def descr_getitem(self, space, w_key):
         try:
             return self.dict[w_key]
         except KeyError:
             raise OperationError(space.w_KeyError, w_key)
 
-    @unwrap_spec('self', ObjSpace, W_Root)
     def descr_delitem(self, space, w_key):
         try:
             del self.dict[w_key]
         except KeyError:
             raise OperationError(space.w_KeyError, w_key)
 
-    @unwrap_spec('self', ObjSpace, W_Root, W_Root)
     def get(self, space, w_key, w_default=None):
         return self.dict.get(w_key, w_default)
 
-    @unwrap_spec('self', ObjSpace)
     def keys(self, space):
         return space.newlist(self.dict.keys())
 
-    @unwrap_spec('self', ObjSpace)
     def values(self, space):
         return space.newlist(self.dict.values())
 
-    @unwrap_spec('self', ObjSpace)
     def clear(self, space):
         self.dict.clear()
 

diff --git a/pypy/module/__pypy__/interp_debug.py b/pypy/module/__pypy__/interp_debug.py
--- a/pypy/module/__pypy__/interp_debug.py
+++ b/pypy/module/__pypy__/interp_debug.py
@@ -1,22 +1,22 @@
-from pypy.interpreter.gateway import interp2app, NoneNotWrapped, unwrap_spec, ObjSpace
+from pypy.interpreter.gateway import interp2app, NoneNotWrapped, unwrap_spec
 from pypy.interpreter.error import OperationError
 from pypy.rlib import debug
 
- at unwrap_spec(ObjSpace, str)
+ at unwrap_spec(category=str)
 def debug_start(space, category):
     debug.debug_start(category)
 
- at unwrap_spec(ObjSpace, 'args_w')
+ at unwrap_spec(args_w='args_w')
 def debug_print(space, args_w):
     parts = [space.str_w(space.str(w_item)) for w_item in args_w]
     debug.debug_print(' '.join(parts))
 
- at unwrap_spec(ObjSpace, str)
+ at unwrap_spec(category=str)
 def debug_stop(space, category):
     debug.debug_stop(category)
 
 
- at unwrap_spec(ObjSpace, str, 'args_w')
+ at unwrap_spec(category=str, args_w='args_w')
 def debug_print_once(space, category, args_w):
     debug_start(space, category)
     debug_print(space, args_w)

diff --git a/pypy/module/_codecs/interp_codecs.py b/pypy/module/_codecs/interp_codecs.py
--- a/pypy/module/_codecs/interp_codecs.py
+++ b/pypy/module/_codecs/interp_codecs.py
@@ -1,7 +1,5 @@
 from pypy.interpreter.error import OperationError, operationerrfmt
-from pypy.interpreter.gateway import ObjSpace, NoneNotWrapped, interp2app
-from pypy.interpreter.gateway import unwrap_spec
-from pypy.interpreter.baseobjspace import W_Root
+from pypy.interpreter.gateway import NoneNotWrapped, interp2app, unwrap_spec
 from pypy.rlib.rstring import StringBuilder, UnicodeBuilder
 from pypy.rlib.objectmodel import we_are_translated
 
@@ -89,9 +87,9 @@
         raise OperationError(
             space.w_TypeError,
             space.wrap("argument must be callable"))
-register_codec.unwrap_spec = [ObjSpace, W_Root]
 
 
+ at unwrap_spec(encoding=str)
 def lookup_codec(space, encoding):
     """lookup(encoding) -> (encoder, decoder, stream_reader, stream_writer)
     Looks up a codec tuple in the Python codec registry and returns
@@ -130,7 +128,6 @@
     raise operationerrfmt(
         space.w_LookupError,
         "unknown encoding: %s", encoding)
-lookup_codec.unwrap_spec = [ObjSpace, str]
 
 # ____________________________________________________________
 # Register standard error handlers
@@ -255,6 +252,7 @@
         state.codec_error_registry[error] = space.wrap(interp2app(globals()[name]))
 
 
+ at unwrap_spec(errors=str)
 def lookup_error(space, errors):
     """lookup_error(errors) -> handler
 
@@ -270,9 +268,9 @@
             space.w_LookupError,
             "unknown error handler name %s", errors)
     return w_err_handler
-lookup_error.unwrap_spec = [ObjSpace, str]
 
 
+ at unwrap_spec(errors=str)
 def encode(space, w_obj, w_encoding=NoneNotWrapped, errors='strict'):
     """encode(obj, [encoding[,errors]]) -> object
 
@@ -290,12 +288,12 @@
     w_encoder = space.getitem(lookup_codec(space, encoding), space.wrap(0))
     w_res = space.call_function(w_encoder, w_obj, space.wrap(errors))
     return space.getitem(w_res, space.wrap(0))
-encode.unwrap_spec = [ObjSpace, W_Root, W_Root, str]
 
+ at unwrap_spec(s='bufferstr', errors='str_or_None')
 def buffer_encode(space, s, errors='strict'):
     return space.newtuple([space.wrap(s), space.wrap(len(s))])
-buffer_encode.unwrap_spec = [ObjSpace, 'bufferstr', 'str_or_None']
 
+ at unwrap_spec(errors=str)
 def decode(space, w_obj, w_encoding=NoneNotWrapped, errors='strict'):
     """decode(obj, [encoding[,errors]]) -> object
 
@@ -321,8 +319,8 @@
         return space.getitem(w_res, space.wrap(0))
     else:
         assert 0, "XXX, what to do here?"
-decode.unwrap_spec = [ObjSpace, W_Root, W_Root, str]
 
+ at unwrap_spec(errors=str)
 def register_error(space, errors, w_handler):
     """register_error(errors, handler)
 
@@ -339,7 +337,6 @@
         raise OperationError(
             space.w_TypeError,
             space.wrap("handler must be callable"))
-register_error.unwrap_spec = [ObjSpace, str, W_Root]
 
 # ____________________________________________________________
 # delegation to runicode
@@ -381,6 +378,7 @@
 def make_encoder_wrapper(name):
     rname = "unicode_encode_%s" % (name.replace("_encode", ""), )
     assert hasattr(runicode, rname)
+    @unwrap_spec(uni=unicode, errors='str_or_None')
     def wrap_encoder(space, uni, errors="strict"):
         if errors is None:
             errors = 'strict'
@@ -389,12 +387,12 @@
         result = func(uni, len(uni), errors, state.encode_error_handler)
         return space.newtuple([space.wrap(result), space.wrap(len(uni))])
     wrap_encoder.func_name = rname
-    wrap_encoder.unwrap_spec = [ObjSpace, unicode, 'str_or_None']
     globals()[name] = wrap_encoder
 
 def make_decoder_wrapper(name):
     rname = "str_decode_%s" % (name.replace("_decode", ""), )
     assert hasattr(runicode, rname)
+    @unwrap_spec(string='bufferstr', errors='str_or_None')
     def wrap_decoder(space, string, errors="strict", w_final=False):
         if errors is None:
             errors = 'strict'
@@ -405,7 +403,6 @@
                                 final, state.decode_error_handler)
         return space.newtuple([space.wrap(result), space.wrap(consumed)])
     wrap_decoder.func_name = rname
-    wrap_decoder.unwrap_spec = [ObjSpace, 'bufferstr', 'str_or_None', W_Root]
     globals()[name] = wrap_decoder
 
 for encoders in [
@@ -444,6 +441,7 @@
     make_encoder_wrapper('mbcs_encode')
     make_decoder_wrapper('mbcs_decode')
 
+ at unwrap_spec(data=str, errors='str_or_None', byteorder=int)
 def utf_16_ex_decode(space, data, errors='strict', byteorder=0, w_final=False):
     if errors is None:
         errors = 'strict'
@@ -462,8 +460,8 @@
         data, len(data), errors, final, state.decode_error_handler, byteorder)
     return space.newtuple([space.wrap(res), space.wrap(consumed),
                            space.wrap(byteorder)])
-utf_16_ex_decode.unwrap_spec = [ObjSpace, str, 'str_or_None', int, W_Root]
 
+ at unwrap_spec(data=str, errors='str_or_None', byteorder=int)
 def utf_32_ex_decode(space, data, errors='strict', byteorder=0, w_final=False):
     final = space.is_true(w_final)
     state = space.fromcache(CodecState)
@@ -480,7 +478,6 @@
         data, len(data), errors, final, state.decode_error_handler, byteorder)
     return space.newtuple([space.wrap(res), space.wrap(consumed),
                            space.wrap(byteorder)])
-utf_32_ex_decode.unwrap_spec = [ObjSpace, str, str, int, W_Root]
 
 # ____________________________________________________________
 # Charmap
@@ -583,7 +580,7 @@
         raise OperationError(space.w_TypeError, space.wrap("invalid mapping"))
 
 
- at unwrap_spec(ObjSpace, str, 'str_or_None', W_Root)
+ at unwrap_spec(string=str, errors='str_or_None')
 def charmap_decode(space, string, errors="strict", w_mapping=None):
     if errors is None:
         errors = 'strict'
@@ -602,7 +599,7 @@
         final, state.decode_error_handler, mapping)
     return space.newtuple([space.wrap(result), space.wrap(consumed)])
 
- at unwrap_spec(ObjSpace, unicode, 'str_or_None', W_Root)
+ at unwrap_spec(uni=unicode, errors='str_or_None')
 def charmap_encode(space, uni, errors="strict", w_mapping=None):
     if errors is None:
         errors = 'strict'
@@ -618,7 +615,7 @@
     return space.newtuple([space.wrap(result), space.wrap(len(uni))])
 
 
- at unwrap_spec(ObjSpace, unicode)
+ at unwrap_spec(chars=unicode)
 def charmap_build(space, chars):
     # XXX CPython sometimes uses a three-level trie
     w_charmap = space.newdict()
@@ -645,7 +642,7 @@
             return -1
         return space.int_w(w_code)
 
- at unwrap_spec(ObjSpace, 'bufferstr', 'str_or_None', W_Root)
+ at unwrap_spec(string='bufferstr', errors='str_or_None')
 def unicode_escape_decode(space, string, errors="strict", w_final=False):
     if errors is None:
         errors = 'strict'
@@ -665,7 +662,7 @@
 # ____________________________________________________________
 # Unicode-internal
 
- at unwrap_spec(ObjSpace, W_Root, 'str_or_None')
+ at unwrap_spec(errors='str_or_None')
 def unicode_internal_decode(space, w_string, errors="strict"):
     if errors is None:
         errors = 'strict'
@@ -689,13 +686,13 @@
 # support for the "string escape" codec
 # This is a bytes-to bytes transformation
 
- at unwrap_spec(ObjSpace, W_Root, 'str_or_None')
+ at unwrap_spec(errors='str_or_None')
 def escape_encode(space, w_string, errors='strict'):
     w_repr = space.repr(w_string)
     w_result = space.getslice(w_repr, space.wrap(1), space.wrap(-1))
     return space.newtuple([w_result, space.len(w_string)])
 
- at unwrap_spec(ObjSpace, str, 'str_or_None')
+ at unwrap_spec(data=str, errors='str_or_None')
 def escape_decode(space, data, errors='strict'):
     from pypy.interpreter.pyparser.parsestring import PyString_DecodeEscape
     result = PyString_DecodeEscape(space, data, None)

diff --git a/pypy/module/_bisect/interp_bisect.py b/pypy/module/_bisect/interp_bisect.py
--- a/pypy/module/_bisect/interp_bisect.py
+++ b/pypy/module/_bisect/interp_bisect.py
@@ -1,7 +1,8 @@
 from pypy.interpreter.error import OperationError
-from pypy.interpreter.gateway import ObjSpace, W_Root
+from pypy.interpreter.gateway import unwrap_spec
 
 
+ at unwrap_spec(lo=int, hi=int)
 def bisect_left(space, w_a, w_x, lo=0, hi=-1):
     """Return the index where to insert item x in list a, assuming a is sorted.
 
@@ -24,9 +25,9 @@
         else:
             hi = mid
     return space.wrap(lo)
-bisect_left.unwrap_spec = [ObjSpace, W_Root, W_Root, int, int]
 
 
+ at unwrap_spec(lo=int, hi=int)
 def bisect_right(space, w_a, w_x, lo=0, hi=-1):
     """Return the index where to insert item x in list a, assuming a is sorted.
 
@@ -49,4 +50,3 @@
         else:
             lo = mid + 1
     return space.wrap(lo)
-bisect_right.unwrap_spec = [ObjSpace, W_Root, W_Root, int, int]

diff --git a/pypy/module/__pypy__/bytebuffer.py b/pypy/module/__pypy__/bytebuffer.py
--- a/pypy/module/__pypy__/bytebuffer.py
+++ b/pypy/module/__pypy__/bytebuffer.py
@@ -3,7 +3,7 @@
 #
 
 from pypy.interpreter.buffer import RWBuffer
-from pypy.interpreter.gateway import ObjSpace
+from pypy.interpreter.gateway import unwrap_spec
 
 
 class ByteBuffer(RWBuffer):
@@ -21,6 +21,6 @@
         self.data[index] = char
 
 
+ at unwrap_spec(length=int)
 def bytebuffer(space, length):
     return space.wrap(ByteBuffer(length))
-bytebuffer.unwrap_spec = [ObjSpace, int]

From commits-noreply at bitbucket.org  Wed Feb 16 19:19:14 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 19:19:14 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: This file is probably not
	tested :-(
Message-ID: <20110216181914.285902A2030@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42052:623ae973d6b4
Date: 2011-02-16 11:15 +0100
http://bitbucket.org/pypy/pypy/changeset/623ae973d6b4/

Log:	This file is probably not tested :-(

diff --git a/pypy/module/_demo/demo.py b/pypy/module/_demo/demo.py
--- a/pypy/module/_demo/demo.py
+++ b/pypy/module/_demo/demo.py
@@ -1,6 +1,6 @@
 from pypy.interpreter.error import OperationError
-from pypy.interpreter.baseobjspace import ObjSpace, W_Root, Wrappable
-from pypy.interpreter.gateway import interp2app
+from pypy.interpreter.baseobjspace import Wrappable
+from pypy.interpreter.gateway import interp2app, unwrap_spec
 from pypy.interpreter.typedef import TypeDef, GetSetProperty
 from pypy.rpython.lltypesystem import rffi, lltype
 from pypy.rpython.tool import rffi_platform

From commits-noreply at bitbucket.org  Wed Feb 16 19:19:15 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 19:19:15 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: update the _ffi module
Message-ID: <20110216181915.70FCE2A2030@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42053:6394db00ef4d
Date: 2011-02-16 11:24 +0100
http://bitbucket.org/pypy/pypy/changeset/6394db00ef4d/

Log:	update the _ffi module

diff --git a/pypy/module/_ffi/interp_ffi.py b/pypy/module/_ffi/interp_ffi.py
--- a/pypy/module/_ffi/interp_ffi.py
+++ b/pypy/module/_ffi/interp_ffi.py
@@ -1,6 +1,5 @@
 import sys
-from pypy.interpreter.baseobjspace import W_Root, ObjSpace, Wrappable, \
-    Arguments
+from pypy.interpreter.baseobjspace import Wrappable, Arguments
 from pypy.interpreter.error import OperationError, wrap_oserror, \
     operationerrfmt
 from pypy.interpreter.gateway import interp2app, NoneNotWrapped, unwrap_spec
@@ -18,7 +17,6 @@
         self.name = name
         self.ffitype = ffitype
 
-    @unwrap_spec('self', ObjSpace)
     def str(self, space):
         return space.wrap('<ffi type %s>' % self.name)
 
@@ -83,7 +81,7 @@
                 assert False, "Argument kind '%s' not supported" % kind
         return argchain
 
-    @unwrap_spec('self', ObjSpace, 'args_w')
+    @unwrap_spec(args_w='args_w')
     def call(self, space, args_w):
         self = jit.hint(self, promote=True)
         argchain = self.build_argchain(space, self.func.argtypes, args_w)
@@ -152,7 +150,6 @@
                                  space.wrap('Unsupported restype'))
         return space.wrap(intres)
 
-    @unwrap_spec('self', ObjSpace)
     def getaddr(self, space):
         """
         Return the physical address in memory of the function
@@ -187,7 +184,7 @@
             raise OperationError(space.w_TypeError, space.wrap(msg))
         return res
 
-    @unwrap_spec('self', ObjSpace, str, W_Root, W_Root)
+    @unwrap_spec(name=str)
     def getfunc(self, space, name, w_argtypes, w_restype):
         argtypes = [self.ffitype(w_argtype) for w_argtype in
                     space.listview(w_argtypes)]
@@ -196,7 +193,7 @@
         return W_FuncPtr(func)
 
 
- at unwrap_spec(ObjSpace, W_Root, str)
+ at unwrap_spec(name=str)
 def descr_new_cdll(space, w_type, name):
     return space.wrap(W_CDLL(space, name))
 

From commits-noreply at bitbucket.org  Wed Feb 16 19:19:17 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 19:19:17 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: args_w is now automatically
	unwrapped
Message-ID: <20110216181917.3AE18282BF7@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42054:8036e3d9cc7c
Date: 2011-02-16 11:28 +0100
http://bitbucket.org/pypy/pypy/changeset/8036e3d9cc7c/

Log:	args_w is now automatically unwrapped

diff --git a/pypy/interpreter/gateway.py b/pypy/interpreter/gateway.py
--- a/pypy/interpreter/gateway.py
+++ b/pypy/interpreter/gateway.py
@@ -464,6 +464,8 @@
                 unwrap_spec.append(ObjSpace)
             elif argname == '__args__':
                 unwrap_spec.append(Arguments)
+            elif argname == 'args_w':
+                unwrap_spec.append('args_w')
             elif argname.startswith('w_'):
                 unwrap_spec.append(W_Root)
             else:

diff --git a/pypy/module/__pypy__/interp_debug.py b/pypy/module/__pypy__/interp_debug.py
--- a/pypy/module/__pypy__/interp_debug.py
+++ b/pypy/module/__pypy__/interp_debug.py
@@ -6,7 +6,6 @@
 def debug_start(space, category):
     debug.debug_start(category)
 
- at unwrap_spec(args_w='args_w')
 def debug_print(space, args_w):
     parts = [space.str_w(space.str(w_item)) for w_item in args_w]
     debug.debug_print(' '.join(parts))
@@ -16,7 +15,7 @@
     debug.debug_stop(category)
 
 
- at unwrap_spec(category=str, args_w='args_w')
+ at unwrap_spec(category=str)
 def debug_print_once(space, category, args_w):
     debug_start(space, category)
     debug_print(space, args_w)

diff --git a/pypy/module/_ffi/interp_ffi.py b/pypy/module/_ffi/interp_ffi.py
--- a/pypy/module/_ffi/interp_ffi.py
+++ b/pypy/module/_ffi/interp_ffi.py
@@ -81,7 +81,6 @@
                 assert False, "Argument kind '%s' not supported" % kind
         return argchain
 
-    @unwrap_spec(args_w='args_w')
     def call(self, space, args_w):
         self = jit.hint(self, promote=True)
         argchain = self.build_argchain(space, self.func.argtypes, args_w)

From commits-noreply at bitbucket.org  Wed Feb 16 19:19:18 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 19:19:18 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: Remove most unwrap_spec from
	interp_file.py
Message-ID: <20110216181918.3808F282BF7@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42055:972d8e705dc8
Date: 2011-02-16 11:39 +0100
http://bitbucket.org/pypy/pypy/changeset/972d8e705dc8/

Log:	Remove most unwrap_spec from interp_file.py

diff --git a/pypy/module/_file/interp_file.py b/pypy/module/_file/interp_file.py
--- a/pypy/module/_file/interp_file.py
+++ b/pypy/module/_file/interp_file.py
@@ -8,11 +8,10 @@
 from pypy.module._file.interp_stream import StreamErrors, wrap_streamerror, wrap_oserror_as_ioerror
 from pypy.module.posix.interp_posix import dispatch_filename
 from pypy.interpreter.error import OperationError, operationerrfmt
-from pypy.interpreter.gateway import ObjSpace, W_Root, Arguments, unwrap_spec
 from pypy.interpreter.typedef import TypeDef, GetSetProperty
 from pypy.interpreter.typedef import interp_attrproperty, make_weakref_descr
 from pypy.interpreter.typedef import interp_attrproperty_w
-from pypy.interpreter.gateway import interp2app
+from pypy.interpreter.gateway import interp2app, unwrap_spec
 
 
 class W_File(W_AbstractStream):
@@ -97,6 +96,7 @@
     # The 'direct_' methods assume that the caller already acquired the
     # file lock.  They don't convert StreamErrors to OperationErrors, too.
 
+    @unwrap_spec(mode=str, buffering=int)
     def direct___init__(self, w_name, mode='r', buffering=-1):
         self.direct_close()
         self.w_name = w_name
@@ -111,7 +111,6 @@
         self.check_closed()
         return self
 
-    @unwrap_spec("self", Arguments)
     def file__exit__(self, __args__):
         """__exit__(*excinfo) -> None. Closes the file."""
         self.space.call_method(self, "close")
@@ -152,6 +151,7 @@
             raise OperationError(self.space.w_StopIteration, self.space.w_None)
         return line
 
+    @unwrap_spec(n=int)
     def direct_read(self, n=-1):
         stream = self.getstream()
         if n < 0:
@@ -166,6 +166,7 @@
                 result.append(data)
             return ''.join(result)
 
+    @unwrap_spec(size=int)
     def direct_readline(self, size=-1):
         stream = self.getstream()
         if size < 0:
@@ -189,6 +190,7 @@
                 size -= len(c)
             return ''.join(result)
 
+    @unwrap_spec(size=int)
     def direct_readlines(self, size=0):
         stream = self.getstream()
         # NB. this implementation is very inefficient for unbuffered
@@ -211,6 +213,7 @@
                 size -= len(line)
         return result
 
+    @unwrap_spec(offset=r_longlong, whence=int)
     def direct_seek(self, offset, whence=0):
         self.getstream().seek(offset, whence)
 
@@ -226,6 +229,7 @@
             size = space.r_longlong_w(w_size)
         stream.truncate(size)
 
+    @unwrap_spec(data='bufferstr')
     def direct_write(self, data):
         self.softspace = 0
         self.getstream().write(data)
@@ -251,7 +255,7 @@
 
     _exposed_method_names = []
 
-    def _decl(class_scope, name, unwrap_spec, docstring,
+    def _decl(class_scope, name, docstring,
               wrapresult="space.wrap(result)"):
         # hack hack to build a wrapper around the direct_xxx methods.
         # The wrapper adds lock/unlock calls and a space.wrap() on
@@ -261,6 +265,7 @@
         co = direct_fn.func_code
         argnames = co.co_varnames[:co.co_argcount]
         defaults = direct_fn.func_defaults or ()
+        unwrap_spec = getattr(direct_fn, 'unwrap_spec', None)
 
         args = []
         for i, argname in enumerate(argnames):
@@ -289,16 +294,16 @@
                 return %(wrapresult)s
         """ % locals())
         exec str(src) in globals(), class_scope
-        class_scope['file_' + name].unwrap_spec = unwrap_spec
+        if unwrap_spec is not None:
+            class_scope['file_' + name].unwrap_spec = unwrap_spec
         class_scope['_exposed_method_names'].append(name)
 
 
-    _decl(locals(), "__init__", ['self', W_Root, str, int],
-          """Opens a file.""")
+    _decl(locals(), "__init__", """Opens a file.""")
 
-    _decl(locals(), "__enter__", ['self'], """__enter__() -> self.""")
+    _decl(locals(), "__enter__", """__enter__() -> self.""")
 
-    _decl(locals(), "close", ['self'],
+    _decl(locals(), "close",
         """close() -> None or (perhaps) an integer.  Close the file.
 
 Sets data attribute .closed to True.  A closed file cannot be used for
@@ -309,35 +314,35 @@
         # close-while-another-thread-uses-it.
 
 
-    _decl(locals(), "fileno", ['self'],
+    _decl(locals(), "fileno",
         '''fileno() -> integer "file descriptor".
 
 This is needed for lower-level file interfaces, such os.read().''')
     
-    _decl(locals(), "flush", ['self'],
+    _decl(locals(), "flush",
         """flush() -> None.  Flush the internal I/O buffer.""")
 
-    _decl(locals(), "isatty", ['self'],
+    _decl(locals(), "isatty",
         """isatty() -> true or false.  True if the file is connected to a tty device.""")
 
-    _decl(locals(), "next", ['self'],
+    _decl(locals(), "next",
         """next() -> the next line in the file, or raise StopIteration""")
 
-    _decl(locals(), "read", ['self', int],
+    _decl(locals(), "read",
         """read([size]) -> read at most size bytes, returned as a string.
 
 If the size argument is negative or omitted, read until EOF is reached.
 Notice that when in non-blocking mode, less data than what was requested
 may be returned, even if no size parameter was given.""")
 
-    _decl(locals(), "readline", ['self', int],
+    _decl(locals(), "readline",
         """readlines([size]) -> list of strings, each a line from the file.
 
 Call readline() repeatedly and return a list of the lines so read.
 The optional size argument, if given, is an approximate bound on the
 total number of bytes in the lines returned.""")
 
-    _decl(locals(), "readlines", ['self', int],
+    _decl(locals(), "readlines",
         """readlines([size]) -> list of strings, each a line from the file.
 
 Call readline() repeatedly and return a list of the lines so read.
@@ -345,7 +350,7 @@
 total number of bytes in the lines returned.""",
         wrapresult = "wrap_list_of_str(space, result)")
 
-    _decl(locals(), "seek", ['self', r_longlong, int],
+    _decl(locals(), "seek",
         """seek(offset[, whence]) -> None.  Move to new file position.
 
 Argument offset is a byte count.  Optional argument whence defaults to
@@ -357,25 +362,25 @@
 undefined behavior.
 Note that not all file objects are seekable.""")
 
-    _decl(locals(), "tell", ['self'],
+    _decl(locals(), "tell",
         "tell() -> current file position, an integer (may be a long integer).")
 
-    _decl(locals(), "truncate", ['self', W_Root],
+    _decl(locals(), "truncate",
         """truncate([size]) -> None.  Truncate the file to at most size bytes.
 
 Size defaults to the current file position, as returned by tell().""")
 
-    _decl(locals(), "write", ['self', 'bufferstr'],
+    _decl(locals(), "write",
         """write(str) -> None.  Write string str to file.
 
 Note that due to buffering, flush() or close() may be needed before
 the file on disk reflects the data written.""")
 
-    _decl(locals(), "__iter__", ['self'],
+    _decl(locals(), "__iter__",
         """Iterating over files, as in 'for line in f:', returns each line of
 the file one by one.""")
 
-    _decl(locals(), "xreadlines", ['self'],
+    _decl(locals(), "xreadlines",
         """xreadlines() -> returns self.
 
 For backward compatibility. File objects now include the performance
@@ -391,7 +396,6 @@
             self.getdisplayname(),
             self.mode)
         return self.getrepr(self.space, info)
-    file__repr__.unwrap_spec = ['self']
 
     def getdisplayname(self):
         w_name = self.w_name
@@ -420,7 +424,6 @@
                     raise
                 break  # done
             self.file_write(space.str_w(w_line))
-    file_writelines.unwrap_spec = ['self', W_Root]
 
     def file_readinto(self, w_rwbuffer):
         """readinto() -> Undocumented.  Don't use this; it may go away."""
@@ -431,24 +434,22 @@
         data = space.str_w(w_data)
         rwbuffer.setslice(0, data)
         return space.wrap(len(data))
-    file_readinto.unwrap_spec = ['self', W_Root]
 
 
 # ____________________________________________________________
 
 
-def descr_file__new__(space, w_subtype, args):
+def descr_file__new__(space, w_subtype, __args__):
     file = space.allocate_instance(W_File, w_subtype)
     W_File.__init__(file, space)
     return space.wrap(file)
-descr_file__new__.unwrap_spec = [ObjSpace, W_Root, Arguments]
 
+ at unwrap_spec(fd=int, mode=str, buffering=int)
 def descr_file_fdopen(space, w_subtype, fd, mode='r', buffering=-1):
     file = space.allocate_instance(W_File, w_subtype)
     W_File.__init__(file, space)
     file.file_fdopen(fd, mode, buffering)
     return space.wrap(file)
-descr_file_fdopen.unwrap_spec = [ObjSpace, W_Root, int, str, int]
 
 def descr_file_closed(space, file):
     return space.wrap(file.stream is None)

From commits-noreply at bitbucket.org  Wed Feb 16 19:19:21 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 19:19:21 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: progress in "hash" modules
Message-ID: <20110216181921.D4F4C2A202E@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42056:8bc2084a78fe
Date: 2011-02-16 11:46 +0100
http://bitbucket.org/pypy/pypy/changeset/8bc2084a78fe/

Log:	progress in "hash" modules

diff --git a/pypy/module/_hashlib/interp_hashlib.py b/pypy/module/_hashlib/interp_hashlib.py
--- a/pypy/module/_hashlib/interp_hashlib.py
+++ b/pypy/module/_hashlib/interp_hashlib.py
@@ -3,7 +3,7 @@
 from pypy.interpreter.typedef import TypeDef, GetSetProperty
 from pypy.interpreter.error import OperationError
 from pypy.tool.sourcetools import func_renamer
-from pypy.interpreter.baseobjspace import Wrappable, W_Root, ObjSpace
+from pypy.interpreter.baseobjspace import Wrappable
 from pypy.rpython.lltypesystem import lltype, rffi
 from pypy.rlib.objectmodel import keepalive_until_here
 from pypy.rlib import ropenssl
@@ -33,20 +33,18 @@
         ropenssl.EVP_MD_CTX_cleanup(self.ctx)
         lltype.free(self.ctx, flavor='raw')
 
-    @unwrap_spec('self', ObjSpace)
     def descr_repr(self, space):
         addrstring = self.getaddrstring(space)
         return space.wrap("<%s HASH object at 0x%s>" % (
             self.name, addrstring))
 
-    @unwrap_spec('self', ObjSpace, 'bufferstr')
+    @unwrap_spec(string='bufferstr')
     def update(self, space, string):
         with rffi.scoped_nonmovingbuffer(string) as buf:
             with self.lock:
                 # XXX try to not release the GIL for small requests
                 ropenssl.EVP_DigestUpdate(self.ctx, buf, len(string))
 
-    @unwrap_spec('self', ObjSpace)
     def copy(self, space):
         "Return a copy of the hash object."
         w_hash = W_Hash(space, self.name)
@@ -54,13 +52,11 @@
             ropenssl.EVP_MD_CTX_copy(w_hash.ctx, self.ctx)
         return w_hash
 
-    @unwrap_spec('self', ObjSpace)
     def digest(self, space):
         "Return the digest value as a string of binary data."
         digest = self._digest(space)
         return space.wrap(digest)
 
-    @unwrap_spec('self', ObjSpace)
     def hexdigest(self, space):
         "Return the digest value as a string of hexadecimal digits."
         digest = self._digest(space)
@@ -132,7 +128,7 @@
     block_size=GetSetProperty(W_Hash.get_block_size),
     )
 
- at unwrap_spec(ObjSpace, str, 'bufferstr')
+ at unwrap_spec(name=str, string='bufferstr')
 def new(space, name, string=''):
     w_hash = W_Hash(space, name)
     w_hash.update(space, string)
@@ -141,7 +137,7 @@
 # shortcut functions
 def make_new_hash(name, funcname):
     @func_renamer(funcname)
-    @unwrap_spec(ObjSpace, 'bufferstr')
+    @unwrap_spec(string='bufferstr')
     def new_hash(space, string=''):
         return new(space, name, string)
     return new_hash

diff --git a/pypy/module/_md5/interp_md5.py b/pypy/module/_md5/interp_md5.py
--- a/pypy/module/_md5/interp_md5.py
+++ b/pypy/module/_md5/interp_md5.py
@@ -1,7 +1,7 @@
 from pypy.rlib import rmd5
 from pypy.interpreter.baseobjspace import Wrappable
 from pypy.interpreter.typedef import TypeDef
-from pypy.interpreter.gateway import interp2app, ObjSpace, W_Root
+from pypy.interpreter.gateway import interp2app, unwrap_spec
 
 
 class W_MD5(Wrappable, rmd5.RMD5):
@@ -13,6 +13,7 @@
         self.space = space
         self._init()
 
+    @unwrap_spec(string='bufferstr')
     def update_w(self, string):
         self.update(string)
 
@@ -28,6 +29,7 @@
         return self.space.wrap(clone)
 
 
+ at unwrap_spec(initialdata='bufferstr')
 def W_MD5___new__(space, w_subtype, initialdata=''):
     """
     Create a new md5 object and call its initializer.
@@ -41,12 +43,11 @@
 
 W_MD5.typedef = TypeDef(
     'MD5Type',
-    __new__   = interp2app(W_MD5___new__, unwrap_spec=[ObjSpace, W_Root,
-                                                       'bufferstr']),
-    update    = interp2app(W_MD5.update_w, unwrap_spec=['self', 'bufferstr']),
-    digest    = interp2app(W_MD5.digest_w, unwrap_spec=['self']),
-    hexdigest = interp2app(W_MD5.hexdigest_w, unwrap_spec=['self']),
-    copy      = interp2app(W_MD5.copy_w, unwrap_spec=['self']),
+    __new__   = interp2app(W_MD5___new__),
+    update    = interp2app(W_MD5.update_w),
+    digest    = interp2app(W_MD5.digest_w),
+    hexdigest = interp2app(W_MD5.hexdigest_w),
+    copy      = interp2app(W_MD5.copy_w),
     digest_size = 16,
     digestsize = 16,
     block_size = 64,

diff --git a/pypy/module/_io/interp_iobase.py b/pypy/module/_io/interp_iobase.py
--- a/pypy/module/_io/interp_iobase.py
+++ b/pypy/module/_io/interp_iobase.py
@@ -2,7 +2,7 @@
 from pypy.interpreter.typedef import (
     TypeDef, GetSetProperty, generic_new_descr, descr_get_dict, descr_set_dict,
     make_weakref_descr)
-from pypy.interpreter.gateway import interp2app, unwrap_spec
+from pypy.interpreter.gateway import interp2app
 from pypy.interpreter.error import OperationError, operationerrfmt
 from pypy.rlib.rstring import StringBuilder
 

diff --git a/pypy/module/_sha/interp_sha.py b/pypy/module/_sha/interp_sha.py
--- a/pypy/module/_sha/interp_sha.py
+++ b/pypy/module/_sha/interp_sha.py
@@ -1,7 +1,7 @@
 from pypy.rlib import rsha
 from pypy.interpreter.baseobjspace import Wrappable
 from pypy.interpreter.typedef import TypeDef
-from pypy.interpreter.gateway import interp2app, ObjSpace, W_Root
+from pypy.interpreter.gateway import interp2app, unwrap_spec
 
 
 class W_SHA(Wrappable, rsha.RSHA):
@@ -13,6 +13,7 @@
         self.space = space
         self._init()
 
+    @unwrap_spec(string='bufferstr')
     def update_w(self, string):
         self.update(string)
 
@@ -28,6 +29,7 @@
         return self.space.wrap(clone)
 
 
+ at unwrap_spec(initialdata='bufferstr')
 def W_SHA___new__(space, w_subtype, initialdata=''):
     """
     Create a new sha object and call its initializer.
@@ -41,12 +43,11 @@
 
 W_SHA.typedef = TypeDef(
     'SHAType',
-    __new__   = interp2app(W_SHA___new__, unwrap_spec=[ObjSpace, W_Root,
-                                                       'bufferstr']),
-    update    = interp2app(W_SHA.update_w, unwrap_spec=['self', 'bufferstr']),
-    digest    = interp2app(W_SHA.digest_w, unwrap_spec=['self']),
-    hexdigest = interp2app(W_SHA.hexdigest_w, unwrap_spec=['self']),
-    copy      = interp2app(W_SHA.copy_w, unwrap_spec=['self']),
+    __new__   = interp2app(W_SHA___new__),
+    update    = interp2app(W_SHA.update_w),
+    digest    = interp2app(W_SHA.digest_w),
+    hexdigest = interp2app(W_SHA.hexdigest_w),
+    copy      = interp2app(W_SHA.copy_w),
     digest_size = 20,
     digestsize = 20,
     block_size = 64,

From commits-noreply at bitbucket.org  Wed Feb 16 19:19:23 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 19:19:23 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: progress: _locale module.
Message-ID: <20110216181923.8AFDE2A202E@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42057:91821c757735
Date: 2011-02-16 11:50 +0100
http://bitbucket.org/pypy/pypy/changeset/91821c757735/

Log:	progress: _locale module.

diff --git a/pypy/module/_locale/interp_locale.py b/pypy/module/_locale/interp_locale.py
--- a/pypy/module/_locale/interp_locale.py
+++ b/pypy/module/_locale/interp_locale.py
@@ -3,7 +3,7 @@
 from pypy.rlib.rarithmetic import intmask
 
 from pypy.interpreter.error import OperationError
-from pypy.interpreter.gateway import ObjSpace, W_Root
+from pypy.interpreter.gateway import unwrap_spec
 
 from pypy.rlib import rlocale
 from pypy.module.exceptions.interp_exceptions import _new_exception, W_Exception
@@ -45,6 +45,7 @@
             ul.append(chr(c))
     space.setattr(stringmod, space.wrap('letters'), space.wrap(''.join(ul)))
 
+ at unwrap_spec(category=int)
 def setlocale(space, category, w_locale=None):
     "(integer,string=None) -> string. Activates/queries locale processing."
 
@@ -63,8 +64,6 @@
 
     return space.wrap(result)
 
-setlocale.unwrap_spec = [ObjSpace, int, W_Root]
-
 def _w_copy_grouping(space, text):
     groups = [ space.wrap(ord(group)) for group in text ]
     if groups:
@@ -117,8 +116,6 @@
 
     return w_result
 
-localeconv.unwrap_spec = [ObjSpace]
-
 _strcoll = rlocale.external('strcoll', [rffi.CCHARP, rffi.CCHARP], rffi.INT)
 _wcscoll = rlocale.external('wcscoll', [rffi.CWCHARP, rffi.CWCHARP], rffi.INT)
 
@@ -154,11 +151,10 @@
 
     return space.wrap(result)
 
-strcoll.unwrap_spec = [ObjSpace, W_Root, W_Root]
-
 _strxfrm = rlocale.external('strxfrm',
                     [rffi.CCHARP, rffi.CCHARP, rffi.SIZE_T], rffi.SIZE_T)
 
+ at unwrap_spec(s=str)
 def strxfrm(space, s):
     "string -> string. Returns a string that behaves for cmp locale-aware."
     n1 = len(s) + 1
@@ -185,10 +181,9 @@
 
     return space.wrap(val)
 
-strxfrm.unwrap_spec = [ObjSpace, str]
-
 if rlocale.HAVE_LANGINFO:
 
+    @unwrap_spec(key=int)
     def nl_langinfo(space, key):
         """nl_langinfo(key) -> string
         Return the value for the locale information associated with key."""
@@ -199,14 +194,13 @@
             raise OperationError(space.w_ValueError,
                                  space.wrap("unsupported langinfo constant"))
 
-    nl_langinfo.unwrap_spec = [ObjSpace, int]
-
 #___________________________________________________________________
 # HAVE_LIBINTL dependence
 
 if rlocale.HAVE_LIBINTL:
     _gettext = rlocale.external('gettext', [rffi.CCHARP], rffi.CCHARP)
 
+    @unwrap_spec(msg=str)
     def gettext(space, msg):
         """gettext(msg) -> string
         Return translation of msg."""
@@ -216,10 +210,9 @@
         finally:
             rffi.free_charp(msg_c)
 
-    gettext.unwrap_spec = [ObjSpace, str]
-
     _dgettext = rlocale.external('dgettext', [rffi.CCHARP, rffi.CCHARP], rffi.CCHARP)
 
+    @unwrap_spec(msg=str)
     def dgettext(space, w_domain, msg):
         """dgettext(domain, msg) -> string
         Return translation of msg in domain."""
@@ -242,11 +235,10 @@
 
         return space.wrap(rffi.charp2str(result))
 
-    dgettext.unwrap_spec = [ObjSpace, W_Root, str]
-
     _dcgettext = rlocale.external('dcgettext', [rffi.CCHARP, rffi.CCHARP, rffi.INT],
                                                                 rffi.CCHARP)
 
+    @unwrap_spec(msg=str, category=int)
     def dcgettext(space, w_domain, msg, category):
         """dcgettext(domain, msg, category) -> string
         Return translation of msg in domain and category."""
@@ -271,8 +263,6 @@
 
         return space.wrap(rffi.charp2str(result))
 
-    dcgettext.unwrap_spec = [ObjSpace, W_Root, str, int]
-
 
     _textdomain = rlocale.external('textdomain', [rffi.CCHARP], rffi.CCHARP)
 
@@ -293,11 +283,10 @@
 
         return space.wrap(rffi.charp2str(result))
 
-    textdomain.unwrap_spec = [ObjSpace, W_Root]
-
     _bindtextdomain = rlocale.external('bindtextdomain', [rffi.CCHARP, rffi.CCHARP],
                                                                 rffi.CCHARP)
 
+    @unwrap_spec(domain=str)
     def bindtextdomain(space, domain, w_dir):
         """bindtextdomain(domain, dir) -> string
         Bind the C library's domain to dir."""
@@ -324,12 +313,11 @@
             raise OperationError(space.w_OSError, space.wrap(errno))
         return space.wrap(rffi.charp2str(dirname))
 
-    bindtextdomain.unwrap_spec = [ObjSpace, str, W_Root]
-
     _bind_textdomain_codeset = rlocale.external('bind_textdomain_codeset',
                                     [rffi.CCHARP, rffi.CCHARP], rffi.CCHARP)
 
     if rlocale.HAVE_BIND_TEXTDOMAIN_CODESET:
+        @unwrap_spec(domain=str)
         def bind_textdomain_codeset(space, domain, w_codeset):
             """bind_textdomain_codeset(domain, codeset) -> string
             Bind the C library's domain to codeset."""
@@ -356,8 +344,6 @@
             else:
                 return space.wrap(rffi.charp2str(result))
 
-        bind_textdomain_codeset.unwrap_spec = [ObjSpace, str, W_Root]
-
 if sys.platform == 'win32':
     def getdefaultlocale(space):
         language, encoding = rlocale.getdefaultlocale()

From commits-noreply at bitbucket.org  Wed Feb 16 19:19:24 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 19:19:24 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: Update unwrap_spec in zlib and
	bz2 modules
Message-ID: <20110216181924.ECAEF282BF7@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42058:4e1db4dc3384
Date: 2011-02-16 11:59 +0100
http://bitbucket.org/pypy/pypy/changeset/4e1db4dc3384/

Log:	Update unwrap_spec in zlib and bz2 modules

diff --git a/pypy/module/bz2/interp_bz2.py b/pypy/module/bz2/interp_bz2.py
--- a/pypy/module/bz2/interp_bz2.py
+++ b/pypy/module/bz2/interp_bz2.py
@@ -6,7 +6,7 @@
 from pypy.interpreter.baseobjspace import Wrappable
 from pypy.interpreter.typedef import TypeDef, GetSetProperty
 from pypy.interpreter.typedef import interp_attrproperty
-from pypy.interpreter.gateway import ObjSpace, W_Root, NoneNotWrapped, interp2app, Arguments
+from pypy.interpreter.gateway import NoneNotWrapped, interp2app, unwrap_spec
 from pypy.rlib.streamio import Stream
 from pypy.translator.tool.cbuild import ExternalCompilationInfo
 from pypy.translator.platform import platform as compiler
@@ -246,6 +246,7 @@
             raise operationerrfmt(space.w_ValueError,
                                   "invalid mode: '%s'", mode)
 
+    @unwrap_spec(mode=str, buffering=int, compresslevel=int)
     def direct_bz2__init__(self, w_name, mode='r', buffering=-1,
                            compresslevel=9):
         self.direct_close()
@@ -259,8 +260,8 @@
         self.fdopenstream(stream, fd, mode, w_name)
 
     _exposed_method_names = []
-    W_File._decl.im_func(locals(), "bz2__init__", ['self', W_Root, str, int, int],
-          """Opens a BZ2-compressed file.""")
+    W_File._decl.im_func(locals(), "bz2__init__",
+                         """Opens a BZ2-compressed file.""")
     # XXX ^^^ hacking hacking... can't just use the name "__init__" again
     # because the RTyper is confused about the two direct__init__() with
     # a different signature, confusion caused by the fact that
@@ -278,13 +279,11 @@
         info = "%s bz2.BZ2File %s, mode '%s'" % (head, self.getdisplayname(),
                                                  self.mode)
         return self.getrepr(self.space, info)
-    file_bz2__repr__.unwrap_spec = ['self']
 
-def descr_bz2file__new__(space, w_subtype, args):
+def descr_bz2file__new__(space, w_subtype, __args__):
     bz2file = space.allocate_instance(W_BZ2File, w_subtype)
     W_BZ2File.__init__(bz2file, space)
     return space.wrap(bz2file)
-descr_bz2file__new__.unwrap_spec = [ObjSpace, W_Root, Arguments]
 
 same_attributes_as_in_file = list(W_File._exposed_method_names)
 same_attributes_as_in_file.remove('__init__')
@@ -485,12 +484,12 @@
     def try_to_find_file_descriptor(self):
         return self.stream.try_to_find_file_descriptor()
 
+ at unwrap_spec(compresslevel=int)
 def descr_compressor__new__(space, w_subtype, compresslevel=9):
     x = space.allocate_instance(W_BZ2Compressor, w_subtype)
     x = space.interp_w(W_BZ2Compressor, x)
     W_BZ2Compressor.__init__(x, space, compresslevel)
     return space.wrap(x)
-descr_compressor__new__.unwrap_spec = [ObjSpace, W_Root, int]
 
 class W_BZ2Compressor(Wrappable):
     """BZ2Compressor([compresslevel=9]) -> compressor object
@@ -504,7 +503,6 @@
         self.bzs = lltype.malloc(bz_stream.TO, flavor='raw', zero=True)
         self.running = False
         self._init_bz2comp(compresslevel)
-    __init__.unwrap_spec = ['self', ObjSpace, int]
         
     def _init_bz2comp(self, compresslevel):
         if compresslevel < 1 or compresslevel > 9:
@@ -521,6 +519,7 @@
         BZ2_bzCompressEnd(self.bzs)
         lltype.free(self.bzs, flavor='raw')
     
+    @unwrap_spec(data='bufferstr')
     def compress(self, data):
         """compress(data) -> string
 
@@ -561,8 +560,6 @@
 
                 res = out.make_result_string()
                 return self.space.wrap(res)
-
-    compress.unwrap_spec = ['self', 'bufferstr']
     
     def flush(self):
         if not self.running:
@@ -583,7 +580,6 @@
 
             res = out.make_result_string()
             return self.space.wrap(res)
-    flush.unwrap_spec = ['self']
 
 W_BZ2Compressor.typedef = TypeDef("BZ2Compressor",
     __doc__ = W_BZ2Compressor.__doc__,
@@ -598,7 +594,6 @@
     x = space.interp_w(W_BZ2Decompressor, x)
     W_BZ2Decompressor.__init__(x, space)
     return space.wrap(x)
-descr_decompressor__new__.unwrap_spec = [ObjSpace, W_Root]
 
 class W_BZ2Decompressor(Wrappable):
     """BZ2Decompressor() -> decompressor object
@@ -626,7 +621,8 @@
     def __del__(self):
         BZ2_bzDecompressEnd(self.bzs)
         lltype.free(self.bzs, flavor='raw')
-    
+
+    @unwrap_spec(data='bufferstr')
     def decompress(self, data):
         """decompress(data) -> string
 
@@ -673,8 +669,6 @@
                 res = out.make_result_string()
                 return self.space.wrap(res)
 
-    decompress.unwrap_spec = ['self', 'bufferstr']
-
 
 W_BZ2Decompressor.typedef = TypeDef("BZ2Decompressor",
     __doc__ = W_BZ2Decompressor.__doc__,
@@ -684,6 +678,7 @@
 )
 
 
+ at unwrap_spec(data='bufferstr', compresslevel=int)
 def compress(space, data, compresslevel=9):
     """compress(data [, compresslevel=9]) -> string
 
@@ -727,8 +722,8 @@
                 res = out.make_result_string()
                 BZ2_bzCompressEnd(bzs)
                 return space.wrap(res)
-compress.unwrap_spec = [ObjSpace, 'bufferstr', int]
 
+ at unwrap_spec(data='bufferstr')
 def decompress(space, data):
     """decompress(data) -> decompressed data
 
@@ -769,4 +764,3 @@
                 res = out.make_result_string()
                 BZ2_bzDecompressEnd(bzs)
                 return space.wrap(res)
-decompress.unwrap_spec = [ObjSpace, 'bufferstr']

diff --git a/pypy/module/zlib/interp_zlib.py b/pypy/module/zlib/interp_zlib.py
--- a/pypy/module/zlib/interp_zlib.py
+++ b/pypy/module/zlib/interp_zlib.py
@@ -1,5 +1,5 @@
 import sys
-from pypy.interpreter.gateway import ObjSpace, W_Root, interp2app
+from pypy.interpreter.gateway import interp2app, unwrap_spec
 from pypy.interpreter.baseobjspace import Wrappable
 from pypy.interpreter.typedef import TypeDef, interp_attrproperty
 from pypy.interpreter.error import OperationError
@@ -20,6 +20,7 @@
         return intmask((x ^ SIGN_EXTEND2) - SIGN_EXTEND2)
 
 
+ at unwrap_spec(string='bufferstr')
 def crc32(space, string, w_start = rzlib.CRC32_DEFAULT_START):
     """
     crc32(string[, start]) -- Compute a CRC-32 checksum of string.
@@ -48,10 +49,10 @@
     checksum = unsigned_to_signed_32bit(checksum)
 
     return space.wrap(checksum)
-crc32.unwrap_spec = [ObjSpace, 'bufferstr', W_Root]
 
 
-def adler32(space, string, start = rzlib.ADLER32_DEFAULT_START):
+ at unwrap_spec(string='bufferstr', start=r_uint)
+def adler32(space, string, start=rzlib.ADLER32_DEFAULT_START):
     """
     adler32(string[, start]) -- Compute an Adler-32 checksum of string.
 
@@ -63,7 +64,6 @@
     checksum = unsigned_to_signed_32bit(checksum)
 
     return space.wrap(checksum)
-adler32.unwrap_spec = [ObjSpace, 'bufferstr', r_uint]
 
 
 def zlib_error(space, msg):
@@ -72,6 +72,7 @@
     return OperationError(w_error, space.wrap(msg))
 
 
+ at unwrap_spec(string='bufferstr', level=int)
 def compress(space, string, level=rzlib.Z_DEFAULT_COMPRESSION):
     """
     compress(string[, level]) -- Returned compressed string.
@@ -90,9 +91,9 @@
     except rzlib.RZlibError, e:
         raise zlib_error(space, e.msg)
     return space.wrap(result)
-compress.unwrap_spec = [ObjSpace, 'bufferstr', int]
 
 
+ at unwrap_spec(string='bufferstr', wbits=int, bufsize=int)
 def decompress(space, string, wbits=rzlib.MAX_WBITS, bufsize=0):
     """
     decompress(string[, wbits[, bufsize]]) -- Return decompressed string.
@@ -112,7 +113,6 @@
     except rzlib.RZlibError, e:
         raise zlib_error(space, e.msg)
     return space.wrap(result)
-decompress.unwrap_spec = [ObjSpace, 'bufferstr', int, int]
 
 
 class ZLibObject(Wrappable):
@@ -164,6 +164,7 @@
             self.stream = rzlib.null_stream
 
 
+    @unwrap_spec(data='bufferstr')
     def compress(self, data):
         """
         compress(data) -- Return a string containing data compressed.
@@ -185,9 +186,9 @@
         except rzlib.RZlibError, e:
             raise zlib_error(self.space, e.msg)
         return self.space.wrap(result)
-    compress.unwrap_spec = ['self', 'bufferstr']
 
 
+    @unwrap_spec(mode=int)
     def flush(self, mode=rzlib.Z_FINISH):
         """
         flush( [mode] ) -- Return a string containing any remaining compressed
@@ -215,9 +216,9 @@
         except rzlib.RZlibError, e:
             raise zlib_error(self.space, e.msg)
         return self.space.wrap(result)
-    flush.unwrap_spec = ['self', int]
 
 
+ at unwrap_spec(level=int, method=int, wbits=int, memLevel=int, strategy=int)
 def Compress___new__(space, w_subtype, level=rzlib.Z_DEFAULT_COMPRESSION,
                      method=rzlib.Z_DEFLATED,             # \
                      wbits=rzlib.MAX_WBITS,               #  \   undocumented
@@ -231,7 +232,6 @@
     Compress.__init__(stream, space, level,
                       method, wbits, memLevel, strategy)
     return space.wrap(stream)
-Compress___new__.unwrap_spec = [ObjSpace, W_Root, int, int, int, int, int]
 
 
 Compress.typedef = TypeDef(
@@ -277,6 +277,7 @@
             self.stream = rzlib.null_stream
 
 
+    @unwrap_spec(data='bufferstr', max_length=int)
     def decompress(self, data, max_length=0):
         """
         decompress(data[, max_length]) -- Return a string containing the
@@ -312,9 +313,9 @@
         else:
             self.unconsumed_tail = tail
         return self.space.wrap(string)
-    decompress.unwrap_spec = ['self', 'bufferstr', int]
 
 
+    @unwrap_spec(length=int)
     def flush(self, length=sys.maxint):
         """
         flush( [length] ) -- This is kept for backward compatibility,
@@ -330,9 +331,9 @@
         # I could not figure out a case in which flush() in CPython
         # doesn't simply return an empty string without complaining.
         return self.space.wrap("")
-    flush.unwrap_spec = ['self', int]
 
 
+ at unwrap_spec(wbits=int)
 def Decompress___new__(space, w_subtype, wbits=rzlib.MAX_WBITS):
     """
     Create a new Decompress and call its initializer.
@@ -341,7 +342,6 @@
     stream = space.interp_w(Decompress, stream)
     Decompress.__init__(stream, space, wbits)
     return space.wrap(stream)
-Decompress___new__.unwrap_spec = [ObjSpace, W_Root, int]
 
 
 Decompress.typedef = TypeDef(

From commits-noreply at bitbucket.org  Wed Feb 16 19:19:26 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 19:19:26 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: More progress
Message-ID: <20110216181926.C4216282BF2@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42059:1ac8b35e03d0
Date: 2011-02-16 12:05 +0100
http://bitbucket.org/pypy/pypy/changeset/1ac8b35e03d0/

Log:	More progress

diff --git a/pypy/module/_lsprof/interp_lsprof.py b/pypy/module/_lsprof/interp_lsprof.py
--- a/pypy/module/_lsprof/interp_lsprof.py
+++ b/pypy/module/_lsprof/interp_lsprof.py
@@ -1,9 +1,8 @@
 
-from pypy.interpreter.baseobjspace import (W_Root, ObjSpace, Wrappable,
-                                           Arguments)
+from pypy.interpreter.baseobjspace import Wrappable
 from pypy.interpreter.typedef import (TypeDef, GetSetProperty,
                                       interp_attrproperty)
-from pypy.interpreter.gateway import interp2app, NoneNotWrapped
+from pypy.interpreter.gateway import interp2app, unwrap_spec, NoneNotWrapped
 from pypy.interpreter.function import Method, Function
 from pypy.interpreter.error import OperationError
 import time, sys
@@ -30,7 +29,6 @@
         return space.wrap('("%s", %d, %d, %f, %f, %s)' % (
             frame_repr, self.callcount, self.reccallcount,
             self.tt, self.it, calls_repr))
-    repr.unwrap_spec = ['self', ObjSpace]
 
     def get_code(space, self):
         return self.frame
@@ -58,7 +56,6 @@
         frame_repr = space.str_w(space.repr(self.frame))
         return space.wrap('("%s", %d, %d, %f, %f)' % (
             frame_repr, self.callcount, self.reccallcount, self.tt, self.it))
-    repr.unwrap_spec = ['self', ObjSpace]
 
     def get_code(space, self):
         return self.frame
@@ -241,7 +238,6 @@
             self.builtins = space.bool_w(w_builtins)
         # set profiler hook
         space.getexecutioncontext().setllprofile(lsprof_call, space.wrap(self))
-    enable.unwrap_spec = ['self', ObjSpace, W_Root, W_Root]
 
     def _enter_call(self, f_code):
         # we have a superb gc, no point in freelist :)
@@ -295,7 +291,6 @@
         # unset profiler hook
         space.getexecutioncontext().setllprofile(None, None)
         self._flush_unmatched()
-    disable.unwrap_spec = ['self', ObjSpace]
 
     def getstats(self, space):
         if self.w_callable is None:
@@ -306,14 +301,13 @@
             factor = 1.0 / sys.maxint
         return stats(space, self.data.values() + self.builtin_data.values(),
                      factor)
-    getstats.unwrap_spec = ['self', ObjSpace]
 
+ at unwrap_spec(time_unit=float, subcalls=bool, builtins=bool)
 def descr_new_profile(space, w_type, w_callable=NoneNotWrapped, time_unit=0.0,
                       subcalls=True, builtins=True):
     p = space.allocate_instance(W_Profiler, w_type)
     p.__init__(space, w_callable, time_unit, subcalls, builtins)
     return space.wrap(p)
-descr_new_profile.unwrap_spec = [ObjSpace, W_Root, W_Root, float, bool, bool]
 
 W_Profiler.typedef = TypeDef(
     'Profiler',

diff --git a/pypy/module/gc/interp_gc.py b/pypy/module/gc/interp_gc.py
--- a/pypy/module/gc/interp_gc.py
+++ b/pypy/module/gc/interp_gc.py
@@ -1,4 +1,4 @@
-from pypy.interpreter.gateway import ObjSpace
+from pypy.interpreter.gateway import unwrap_spec
 from pypy.interpreter.error import OperationError
 from pypy.rlib import rgc
 from pypy.rlib.streamio import open_file_as_stream
@@ -16,8 +16,6 @@
             cache.clear()
     rgc.collect()
     return space.wrap(0)
-    
-collect.unwrap_spec = [ObjSpace]
 
 def enable_finalizers(space):
     if space.user_del_action.finalizers_lock_count == 0:
@@ -25,14 +23,13 @@
                              space.wrap("finalizers are already enabled"))
     space.user_del_action.finalizers_lock_count -= 1
     space.user_del_action.fire()
-enable_finalizers.unwrap_spec = [ObjSpace]
 
 def disable_finalizers(space):
     space.user_del_action.finalizers_lock_count += 1
-disable_finalizers.unwrap_spec = [ObjSpace]
 
 # ____________________________________________________________
 
+ at unwrap_spec(filename=str)
 def dump_heap_stats(space, filename):
     tb = rgc._heap_stats()
     if not tb:
@@ -43,4 +40,3 @@
         f.write("%d %d " % (tb[i].count, tb[i].size))
         f.write(",".join([str(tb[i].links[j]) for j in range(len(tb))]) + "\n")
     f.close()
-dump_heap_stats.unwrap_spec = [ObjSpace, str]

diff --git a/pypy/module/gc/referents.py b/pypy/module/gc/referents.py
--- a/pypy/module/gc/referents.py
+++ b/pypy/module/gc/referents.py
@@ -1,7 +1,7 @@
 from pypy.rlib import rgc
 from pypy.interpreter.baseobjspace import W_Root, Wrappable
 from pypy.interpreter.typedef import TypeDef
-from pypy.interpreter.gateway import ObjSpace
+from pypy.interpreter.gateway import unwrap_spec
 from pypy.interpreter.error import wrap_oserror, OperationError
 from pypy.rlib.objectmodel import we_are_translated
 
@@ -140,7 +140,6 @@
         gcref = rgc.cast_instance_to_gcref(w_obj)
         _list_w_obj_referents(gcref, result)
     return space.newlist(result)
-get_referents.unwrap_spec = [ObjSpace, 'args_w']
 
 def get_referrers(space, args_w):
     """Return the list of objects that directly refer to any of objs."""
@@ -167,8 +166,8 @@
                         result_w[w_obj] = None
                 pending_w += referents_w
     return space.newlist(result_w.keys())
-get_referrers.unwrap_spec = [ObjSpace, 'args_w']
 
+ at unwrap_spec(fd=int)
 def _dump_rpy_heap(space, fd):
     try:
         ok = rgc.dump_rpy_heap(fd)
@@ -176,10 +175,8 @@
         raise wrap_oserror(space, e)
     if not ok:
         raise missing_operation(space)
-_dump_rpy_heap.unwrap_spec = [ObjSpace, int]
 
 def get_typeids_z(space):
     a = rgc.get_typeids_z()
     s = ''.join([a[i] for i in range(len(a))])
     return space.wrap(s)
-get_typeids_z.unwrap_spec = [ObjSpace]

From commits-noreply at bitbucket.org  Wed Feb 16 19:19:28 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 19:19:28 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: progress: minimal_curses and
	termios
Message-ID: <20110216181928.42710282BF2@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42060:029e0a0bba86
Date: 2011-02-16 12:08 +0100
http://bitbucket.org/pypy/pypy/changeset/029e0a0bba86/

Log:	progress: minimal_curses and termios

diff --git a/pypy/module/_minimal_curses/interp_curses.py b/pypy/module/_minimal_curses/interp_curses.py
--- a/pypy/module/_minimal_curses/interp_curses.py
+++ b/pypy/module/_minimal_curses/interp_curses.py
@@ -1,5 +1,5 @@
 
-from pypy.interpreter.baseobjspace import ObjSpace, W_Root
+from pypy.interpreter.baseobjspace import unwrap_spec
 from pypy.interpreter.error import OperationError
 from pypy.module._minimal_curses import _curses
 
@@ -40,6 +40,7 @@
     except _curses.error, e:
         raise curses_error(e.args[0])
 
+ at unwrap_spec(fd=int)
 def setupterm(space, w_termname=None, fd=-1):
     if fd == -1:
         w_stdout = space.getattr(space.getbuiltinmodule('sys'),
@@ -53,7 +54,6 @@
             _curses_setupterm(space.str_w(w_termname), fd)
     except curses_error, e:
         raise convert_error(space, e)
-setupterm.unwrap_spec = [ObjSpace, W_Root, int]
 
 class TermError(Exception):
     pass
@@ -75,6 +75,7 @@
     except _curses.error, e:
         raise curses_error(e.args[0])
 
+ at unwrap_spec(capname=str)
 def tigetstr(space, capname):
     try:
         result = _curses_tigetstr(capname)
@@ -83,12 +84,11 @@
     except curses_error, e:
         raise convert_error(space, e)
     return space.wrap(result)
-tigetstr.unwrap_spec = [ObjSpace, str]
 
+ at unwrap_spec(s=str)
 def tparm(space, s, args_w):
     args = [space.int_w(a) for a in args_w]
     try:
         return space.wrap(_curses_tparm(s, args))
     except curses_error, e:
         raise convert_error(space, e)
-tparm.unwrap_spec = [ObjSpace, str, 'args_w']

diff --git a/pypy/module/termios/interp_termios.py b/pypy/module/termios/interp_termios.py
--- a/pypy/module/termios/interp_termios.py
+++ b/pypy/module/termios/interp_termios.py
@@ -3,7 +3,7 @@
 little use of termios module on RPython level by itself
 """
 
-from pypy.interpreter.baseobjspace import ObjSpace, W_Root
+from pypy.interpreter.gateway import unwrap_spec
 from pypy.interpreter.error import OperationError, wrap_oserror
 from pypy.rpython.module import ll_termios
 from pypy.rlib.objectmodel import we_are_translated
@@ -20,6 +20,7 @@
     w_exception_class = space.getattr(w_module, space.wrap('error'))
     return wrap_oserror(space, error, w_exception_class=w_exception_class)
 
+ at unwrap_spec(fd=int, when=int)
 def tcsetattr(space, fd, when, w_attributes):
     w_iflag, w_oflag, w_cflag, w_lflag, w_ispeed, w_ospeed, w_cc = \
              space.unpackiterable(w_attributes, expected_length=7)
@@ -39,8 +40,8 @@
         rtermios.tcsetattr(fd, when, tup)
     except OSError, e:
         raise convert_error(space, e)
-tcsetattr.unwrap_spec = [ObjSpace, int, int, W_Root]
 
+ at unwrap_spec(fd=int)
 def tcgetattr(space, fd):
     try:
         tup = rtermios.tcgetattr(fd)
@@ -56,32 +57,31 @@
     w_cc = space.newlist(cc_w)
     l_w.append(w_cc)
     return space.newlist(l_w)
-tcgetattr.unwrap_spec = [ObjSpace, int]
 
+ at unwrap_spec(fd=int, duration=int)
 def tcsendbreak(space, fd, duration):
     try:
         termios.tcsendbreak(fd, duration)
     except OSError, e:
         raise convert_error(space, e)
-tcsendbreak.unwrap_spec = [ObjSpace, int, int]
 
+ at unwrap_spec(fd=int)
 def tcdrain(space, fd):
     try:
         termios.tcdrain(fd)
     except OSError, e:
         raise convert_error(space, e)
-tcdrain.unwrap_spec = [ObjSpace, int]
 
+ at unwrap_spec(fd=int, queue=int)
 def tcflush(space, fd, queue):
     try:
         termios.tcflush(fd, queue)
     except OSError, e:
         raise convert_error(space, e)
-tcflush.unwrap_spec = [ObjSpace, int, int]
 
+ at unwrap_spec(fd=int, action=int)
 def tcflow(space, fd, action):
     try:
         termios.tcflow(fd, action)
     except OSError, e:
         raise convert_error(space, e)
-tcflow.unwrap_spec = [ObjSpace, int, int]

From commits-noreply at bitbucket.org  Wed Feb 16 19:19:29 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 19:19:29 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: Update unwrap_spec in
	_multiprocessing	module
Message-ID: <20110216181929.E039D282BF2@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42061:c9bd9c16dade
Date: 2011-02-16 13:03 +0100
http://bitbucket.org/pypy/pypy/changeset/c9bd9c16dade/

Log:	Update unwrap_spec in _multiprocessing module

diff --git a/pypy/module/_multiprocessing/interp_semaphore.py b/pypy/module/_multiprocessing/interp_semaphore.py
--- a/pypy/module/_multiprocessing/interp_semaphore.py
+++ b/pypy/module/_multiprocessing/interp_semaphore.py
@@ -1,7 +1,7 @@
 from __future__ import with_statement
-from pypy.interpreter.baseobjspace import ObjSpace, Wrappable, W_Root
+from pypy.interpreter.baseobjspace import Wrappable
 from pypy.interpreter.typedef import TypeDef, GetSetProperty
-from pypy.interpreter.gateway import interp2app, Arguments, unwrap_spec
+from pypy.interpreter.gateway import interp2app, unwrap_spec
 from pypy.interpreter.error import wrap_oserror, OperationError
 from pypy.rpython.lltypesystem import rffi, lltype
 from pypy.rlib.rarithmetic import r_uint
@@ -409,18 +409,15 @@
     def handle_get(space, self):
         return w_handle(space, self.handle)
 
-    @unwrap_spec('self', ObjSpace)
     def get_count(self, space):
         return space.wrap(self.count)
 
     def _ismine(self):
         return self.count > 0 and ll_thread.get_ident() == self.last_tid
 
-    @unwrap_spec('self', ObjSpace)
     def is_mine(self, space):
         return space.wrap(self._ismine())
 
-    @unwrap_spec('self', ObjSpace)
     def is_zero(self, space):
         try:
             res = semlock_iszero(self, space)
@@ -428,7 +425,6 @@
             raise wrap_oserror(space, e)
         return space.wrap(res)
 
-    @unwrap_spec('self', ObjSpace)
     def get_value(self, space):
         try:
             val = semlock_getvalue(self, space)
@@ -436,7 +432,7 @@
             raise wrap_oserror(space, e)
         return space.wrap(val)
 
-    @unwrap_spec('self', ObjSpace, bool, W_Root)
+    @unwrap_spec(block=bool)
     def acquire(self, space, block=True, w_timeout=None):
         # check whether we already own the lock
         if self.kind == RECURSIVE_MUTEX and self._ismine():
@@ -455,7 +451,6 @@
         else:
             return space.w_False
 
-    @unwrap_spec('self', ObjSpace)
     def release(self, space):
         if self.kind == RECURSIVE_MUTEX:
             if not self._ismine():
@@ -474,21 +469,19 @@
 
         self.count -= 1
 
-    @unwrap_spec(ObjSpace, W_Root, W_Root, int, int)
+    @unwrap_spec(kind=int, maxvalue=int)
     def rebuild(space, w_cls, w_handle, kind, maxvalue):
         self = space.allocate_instance(W_SemLock, w_cls)
         self.__init__(handle_w(space, w_handle), kind, maxvalue)
         return space.wrap(self)
 
-    @unwrap_spec('self', ObjSpace)
     def enter(self, space):
         return self.acquire(space, w_timeout=space.w_None)
 
-    @unwrap_spec('self', ObjSpace, Arguments)
     def exit(self, space, __args__):
         self.release(space)
 
- at unwrap_spec(ObjSpace, W_Root, int, int, int)
+ at unwrap_spec(kind=int, value=int, maxvalue=int)
 def descr_new(space, w_subtype, kind, value, maxvalue):
     if kind != RECURSIVE_MUTEX and kind != SEMAPHORE:
         raise OperationError(space.w_ValueError,

diff --git a/pypy/module/_multiprocessing/interp_connection.py b/pypy/module/_multiprocessing/interp_connection.py
--- a/pypy/module/_multiprocessing/interp_connection.py
+++ b/pypy/module/_multiprocessing/interp_connection.py
@@ -1,5 +1,5 @@
 from __future__ import with_statement
-from pypy.interpreter.baseobjspace import ObjSpace, Wrappable, W_Root
+from pypy.interpreter.baseobjspace import Wrappable
 from pypy.interpreter.typedef import TypeDef, GetSetProperty
 from pypy.interpreter.gateway import interp2app, unwrap_spec
 from pypy.interpreter.error import (
@@ -69,7 +69,7 @@
             raise OperationError(space.w_IOError,
                                  space.wrap("connection is read-only"))
 
-    @unwrap_spec('self', ObjSpace, 'bufferstr', 'index', 'index')
+    @unwrap_spec(buffer='bufferstr', offset='index', size='index')
     def send_bytes(self, space, buffer, offset=0, size=PY_SSIZE_T_MIN):
         length = len(buffer)
         self._check_writable(space)
@@ -91,7 +91,7 @@
 
         self.do_send_string(space, buffer, offset, size)
 
-    @unwrap_spec('self', ObjSpace, 'index')
+    @unwrap_spec(maxlength='index')
     def recv_bytes(self, space, maxlength=PY_SSIZE_T_MAX):
         self._check_readable(space)
         if maxlength < 0:
@@ -110,7 +110,7 @@
             if newbuf:
                 rffi.free_charp(newbuf)
 
-    @unwrap_spec('self', ObjSpace, W_Root, 'index')
+    @unwrap_spec(offset='index')
     def recv_bytes_into(self, space, w_buffer, offset=0):
         rwbuffer = space.rwbuffer_w(w_buffer)
         length = rwbuffer.getlength()
@@ -129,7 +129,6 @@
 
         return space.wrap(res)
 
-    @unwrap_spec('self', ObjSpace, W_Root)
     def send(self, space, w_obj):
         self._check_writable(space)
 
@@ -144,7 +143,6 @@
         buffer = space.bufferstr_w(w_pickled)
         self.do_send_string(space, buffer, 0, len(buffer))
 
-    @unwrap_spec('self', ObjSpace)
     def recv(self, space):
         self._check_readable(space)
 
@@ -168,7 +166,6 @@
 
         return w_unpickled
 
-    @unwrap_spec('self', ObjSpace, W_Root)
     def poll(self, space, w_timeout=0.0):
         self._check_readable(space)
         if space.is_w(w_timeout, space.w_None):
@@ -232,7 +229,7 @@
         W_BaseConnection.__init__(self, flags)
         self.fd = fd
 
-    @unwrap_spec(ObjSpace, W_Root, int, bool, bool)
+    @unwrap_spec(fd=int, readable=bool, writable=bool)
     def descr_new_file(space, w_subtype, fd, readable=True, writable=True):
         flags = (readable and READABLE) | (writable and WRITABLE)
 
@@ -240,7 +237,6 @@
         W_FileConnection.__init__(self, fd, flags)
         return space.wrap(self)
 
-    @unwrap_spec('self', ObjSpace)
     def fileno(self, space):
         return space.wrap(self.fd)
 
@@ -350,7 +346,7 @@
         W_BaseConnection.__init__(self, flags)
         self.handle = handle
 
-    @unwrap_spec(ObjSpace, W_Root, W_Root, bool, bool)
+    @unwrap_spec(readable=bool, writable=bool)
     def descr_new_pipe(space, w_subtype, w_handle, readable=True, writable=True):
         from pypy.module._multiprocessing.interp_win32 import handle_w
         handle = handle_w(space, w_handle)
@@ -369,7 +365,6 @@
     def is_valid(self):
         return self.handle != self.INVALID_HANDLE_VALUE
 
-    @unwrap_spec('self', ObjSpace)
     def fileno(self, space):
         return w_handle(space, self.handle)
 

diff --git a/pypy/module/_multiprocessing/interp_memory.py b/pypy/module/_multiprocessing/interp_memory.py
--- a/pypy/module/_multiprocessing/interp_memory.py
+++ b/pypy/module/_multiprocessing/interp_memory.py
@@ -1,10 +1,7 @@
-from pypy.interpreter.baseobjspace import ObjSpace, Wrappable, W_Root
-from pypy.interpreter.gateway import unwrap_spec
 from pypy.interpreter.error import OperationError
 from pypy.rpython.lltypesystem import rffi
 from pypy.module.mmap.interp_mmap import W_MMap
 
- at unwrap_spec(ObjSpace, W_Root)
 def address_of_buffer(space, w_obj):
     if space.config.objspace.usemodules.mmap:
         mmap = space.interp_w(W_MMap, w_obj)

diff --git a/pypy/module/_multiprocessing/interp_win32.py b/pypy/module/_multiprocessing/interp_win32.py
--- a/pypy/module/_multiprocessing/interp_win32.py
+++ b/pypy/module/_multiprocessing/interp_win32.py
@@ -1,4 +1,4 @@
-from pypy.interpreter.gateway import ObjSpace, W_Root, unwrap_spec, interp2app
+from pypy.interpreter.gateway import unwrap_spec, interp2app
 from pypy.interpreter.function import StaticMethod
 from pypy.interpreter.error import wrap_windowserror, OperationError
 from pypy.rlib import rwin32
@@ -104,7 +104,8 @@
 # __________________________________________________________
 # functions for the "win32" namespace
 
- at unwrap_spec(ObjSpace, str, r_uint, r_uint, r_uint, r_uint, r_uint, r_uint, W_Root)
+ at unwrap_spec(name=str, openmode=r_uint, pipemode=r_uint, maxinstances=r_uint,
+             outputsize=r_uint, inputsize=r_uint, timeout=r_uint)
 def CreateNamedPipe(space, name, openmode, pipemode, maxinstances,
                     outputsize, inputsize, timeout, w_security):
     security = space.int_w(w_security)
@@ -129,7 +130,6 @@
     if not _ConnectNamedPipe(handle, rffi.NULL):
         raise wrap_windowserror(space, rwin32.lastWindowsError())
 
- at unwrap_spec(ObjSpace, W_Root, W_Root, W_Root, W_Root)
 def SetNamedPipeHandleState(space, w_handle, w_pipemode, w_maxinstances, w_timeout):
     handle = handle_w(space, w_handle)
     state = lltype.malloc(rffi.CArrayPtr(rffi.UINT).TO, 3, flavor='raw')
@@ -150,13 +150,14 @@
         lltype.free(state, flavor='raw')
         lltype.free(statep, flavor='raw')
 
- at unwrap_spec(ObjSpace, str, r_uint)
+ at unwrap_spec(name=str, timeout=r_uint)
 def WaitNamedPipe(space, name, timeout):
     # Careful: zero means "default value specified by CreateNamedPipe()"
     if not _WaitNamedPipe(name, timeout):
         raise wrap_windowserror(space, rwin32.lastWindowsError())
 
- at unwrap_spec(ObjSpace, str, r_uint, r_uint, W_Root, r_uint, r_uint, W_Root)
+ at unwrap_spec(filename=str, access=r_uint, share=r_uint,
+             disposition=r_uint, flags=r_uint)
 def CreateFile(space, filename, access, share, w_security,
                disposition, flags, w_templatefile):
     security = space.int_w(w_security)
@@ -173,7 +174,7 @@
 
     return w_handle(space, handle)
 
- at unwrap_spec(ObjSpace, r_uint)
+ at unwrap_spec(code=r_uint)
 def ExitProcess(space, code):
     _ExitProcess(code)
 

From commits-noreply at bitbucket.org  Wed Feb 16 19:19:30 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 19:19:30 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: _random, _pickle_support
Message-ID: <20110216181930.DD209282BF2@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42062:a05535fd772b
Date: 2011-02-16 13:08 +0100
http://bitbucket.org/pypy/pypy/changeset/a05535fd772b/

Log:	_random, _pickle_support

diff --git a/pypy/module/_random/interp_random.py b/pypy/module/_random/interp_random.py
--- a/pypy/module/_random/interp_random.py
+++ b/pypy/module/_random/interp_random.py
@@ -1,15 +1,14 @@
 from pypy.interpreter.error import OperationError
 from pypy.interpreter.typedef import TypeDef
-from pypy.interpreter.gateway import ObjSpace, W_Root, NoneNotWrapped, interp2app
-from pypy.interpreter.gateway import Arguments
+from pypy.interpreter.gateway import NoneNotWrapped, interp2app, unwrap_spec
 from pypy.interpreter.baseobjspace import Wrappable
 from pypy.rlib.rarithmetic import r_uint, intmask
 from pypy.rlib import rrandom
 
 import time
 
-def descr_new__(space, w_subtype, args):
-    w_anything = args.firstarg()
+def descr_new__(space, w_subtype, __args__):
+    w_anything = __args__.firstarg()
     x = space.allocate_instance(W_Random, w_subtype)
     x = space.interp_w(W_Random, x)
     W_Random.__init__(x, space, w_anything)
@@ -19,11 +18,9 @@
     def __init__(self, space, w_anything):
         self._rnd = rrandom.Random()
         self.seed(space, w_anything)
-    __init__.unwrap_spec = ['self', ObjSpace, W_Root]
 
     def random(self, space):
         return space.newfloat(self._rnd.random())
-    random.unwrap_spec = ['self', ObjSpace]
 
     def seed(self, space, w_n=NoneNotWrapped):
         if w_n is None:
@@ -51,7 +48,6 @@
         if not key:
             key = [r_uint(0)]
         self._rnd.init_by_array(key)
-    seed.unwrap_spec = ['self', ObjSpace, W_Root]
 
     def getstate(self, space):
         state = [None] * (rrandom.N + 1)
@@ -59,7 +55,6 @@
             state[i] = space.newint(intmask(self._rnd.state[i]))
         state[rrandom.N] = space.newint(self._rnd.index)
         return space.newtuple(state)
-    getstate.unwrap_spec = ['self', ObjSpace]
 
     def setstate(self, space, w_state):
         if not space.is_true(space.isinstance(w_state, space.w_tuple)):
@@ -79,7 +74,6 @@
             self._rnd.state[i] = space.uint_w(w_item)
         w_item = space.getitem(w_state, space.newint(rrandom.N))
         self._rnd.index = space.int_w(w_item)
-    setstate.unwrap_spec = ['self', ObjSpace, W_Root]
 
     def jumpahead(self, space, w_n):
         if space.is_true(space.isinstance(w_n, space.w_long)):
@@ -88,8 +82,8 @@
         else:
             n = space.int_w(w_n)
         self._rnd.jumpahead(n)
-    jumpahead.unwrap_spec = ['self', ObjSpace, W_Root]
 
+    @unwrap_spec(k=int)
     def getrandbits(self, space, k):
         if k <= 0:
             strerror = space.wrap("number of bits must be greater than zero")
@@ -114,11 +108,10 @@
             w_result = space.or_(space.lshift(w_result, w_eight),
                                  space.newint(intmask(byte)))
         return w_result
-    getrandbits.unwrap_spec = ['self', ObjSpace, int]
 
 
 W_Random.typedef = TypeDef("Random",
-    __new__ = interp2app(descr_new__, unwrap_spec=[ObjSpace, W_Root, Arguments]),
+    __new__ = interp2app(descr_new__),
     random = interp2app(W_Random.random),
     seed = interp2app(W_Random.seed),
     getstate = interp2app(W_Random.getstate),

diff --git a/pypy/module/_pickle_support/maker.py b/pypy/module/_pickle_support/maker.py
--- a/pypy/module/_pickle_support/maker.py
+++ b/pypy/module/_pickle_support/maker.py
@@ -7,8 +7,7 @@
 from pypy.interpreter.pytraceback import PyTraceback
 from pypy.interpreter.generator import GeneratorIterator
 from pypy.rlib.objectmodel import instantiate
-from pypy.interpreter.argument import Arguments
-from pypy.interpreter.baseobjspace import ObjSpace, W_Root
+from pypy.interpreter.gateway import unwrap_spec
 from pypy.objspace.std.dicttype import dictiter_typedef
 from pypy.objspace.std.iterobject import W_SeqIterObject, W_ReverseSeqIterObject
 
@@ -23,13 +22,11 @@
 def code_new(space, __args__):
     w_type = space.gettypeobject(PyCode.typedef)
     return space.call_args(w_type, __args__)
-code_new.unwrap_spec = [ObjSpace, Arguments]
 
 def func_new(space):
     fu = instantiate(Function)
     fu.w_func_dict = space.newdict()
     return space.wrap(fu)
-func_new.unwrap_spec = [ObjSpace]
 
 def module_new(space, w_name, w_dict):
     new_mod = Module(space, w_name, w_dict, add_package=False)
@@ -38,7 +35,6 @@
 def method_new(space, __args__):
     w_type = space.gettypeobject(Method.typedef)
     return space.call_args(w_type, __args__)
-method_new.unwrap_spec = [ObjSpace, Arguments]
 
 def builtin_method_new(space, w_instance, w_name):
     return space.getattr(w_instance, w_name)
@@ -47,7 +43,6 @@
     # we got a listobject.
     # simply create an iterator and that's it.
     return space.iter(w_lis)
-dictiter_surrogate_new.unwrap_spec = [ObjSpace, W_Root]
 
 def seqiter_new(space, w_seq, w_index):
     return W_SeqIterObject(w_seq, space.int_w(w_index))
@@ -60,26 +55,25 @@
 def frame_new(space):
     new_frame = instantiate(space.FrameClass)   # XXX fish
     return space.wrap(new_frame)
-frame_new.unwrap_spec = [ObjSpace]
 
 def traceback_new(space):
     tb = instantiate(PyTraceback)
     return space.wrap(tb)
-traceback_new.unwrap_spec = [ObjSpace]
 
+ at unwrap_spec(runnint=int)
 def generator_new(space, w_frame, running):
     frame = space.interp_w(PyFrame, w_frame, can_be_None=True)
     new_generator = GeneratorIterator(frame)
     new_generator.running = running
     return space.wrap(new_generator)
-generator_new.unwrap_spec = [ObjSpace, W_Root, int]
 
+ at unwrap_spec(current=int, remaining=int, step=int)
 def xrangeiter_new(space, current, remaining, step):
     from pypy.module.__builtin__.functional import W_XRangeIterator
     new_iter = W_XRangeIterator(space, current, remaining, step)
     return space.wrap(new_iter)
-xrangeiter_new.unwrap_spec = [ObjSpace, int, int, int]
 
+ at unwrap_spec(identifier=str)
 def builtin_code(space, identifier):
     from pypy.interpreter import gateway
     try:
@@ -88,8 +82,8 @@
         raise OperationError(space.w_RuntimeError, 
                              space.wrap("cannot unpickle builtin code: "+
                                         identifier))
-builtin_code.unwrap_spec = [ObjSpace, str]
-        
+
+ at unwrap_spec(identifier=str)
 def builtin_function(space, identifier):
     from pypy.interpreter import function
     try:
@@ -98,18 +92,15 @@
         raise OperationError(space.w_RuntimeError, 
                              space.wrap("cannot unpickle builtin function: "+
                                         identifier))
-builtin_function.unwrap_spec = [ObjSpace, str]
 
 
 def enumerate_new(space, w_iter, w_index):
     from pypy.module.__builtin__.functional import _make_enumerate
     return _make_enumerate(space, w_iter, w_index)
-enumerate_new.unwrap_spec = [ObjSpace, W_Root, W_Root]
 
 def reversed_new(space, w_seq, w_remaining):
     from pypy.module.__builtin__.functional import _make_reversed
     return _make_reversed(space, w_seq, w_remaining)
-reversed_new.unwrap_spec = [ObjSpace, W_Root, W_Root]
 
 
 # ___________________________________________________________________

From commits-noreply at bitbucket.org  Wed Feb 16 19:19:33 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 19:19:33 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: simplify _rawffi
Message-ID: <20110216181933.EB682282BF2@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42063:023eba0c4a01
Date: 2011-02-16 13:23 +0100
http://bitbucket.org/pypy/pypy/changeset/023eba0c4a01/

Log:	simplify _rawffi

diff --git a/pypy/module/_rawffi/interp_rawffi.py b/pypy/module/_rawffi/interp_rawffi.py
--- a/pypy/module/_rawffi/interp_rawffi.py
+++ b/pypy/module/_rawffi/interp_rawffi.py
@@ -1,8 +1,7 @@
 import sys
-from pypy.interpreter.baseobjspace import W_Root, ObjSpace, Wrappable, \
-     Arguments
+from pypy.interpreter.baseobjspace import Wrappable
 from pypy.interpreter.error import OperationError, wrap_oserror, operationerrfmt
-from pypy.interpreter.gateway import interp2app, NoneNotWrapped
+from pypy.interpreter.gateway import interp2app, NoneNotWrapped, unwrap_spec
 from pypy.interpreter.typedef import TypeDef, GetSetProperty
 
 from pypy.rlib.clibffi import *
@@ -142,6 +141,7 @@
         self.w_cache = space.newdict()
         self.space = space
 
+    @unwrap_spec(flags=int)
     def ptr(self, space, w_name, w_argtypes, w_restype, flags=FUNCFLAG_CDECL):
         """ Get a pointer for function name with provided argtypes
         and restype
@@ -193,8 +193,8 @@
         w_funcptr = W_FuncPtr(space, ptr, argshapes, resshape)
         space.setitem(self.w_cache, w_key, w_funcptr)
         return w_funcptr
-    ptr.unwrap_spec = ['self', ObjSpace, W_Root, W_Root, W_Root, int]
 
+    @unwrap_spec(name=str)
     def getaddressindll(self, space, name):
         try:
             address_as_uint = rffi.cast(lltype.Unsigned,
@@ -203,8 +203,8 @@
             raise operationerrfmt(space.w_ValueError,
                                   "Cannot find symbol %s", name)
         return space.wrap(address_as_uint)
-    getaddressindll.unwrap_spec = ['self', ObjSpace, str]
 
+ at unwrap_spec(name='str_or_None')
 def descr_new_cdll(space, w_type, name):
     try:
         cdll = CDLL(name)
@@ -214,7 +214,6 @@
     except OSError, e:
         raise wrap_oserror(space, e)
     return space.wrap(W_CDLL(space, name, cdll))
-descr_new_cdll.unwrap_spec = [ObjSpace, W_Root, 'str_or_None']
 
 W_CDLL.typedef = TypeDef(
     'CDLL',
@@ -251,10 +250,10 @@
     def get_basic_ffi_type(self):
         raise NotImplementedError
 
+    @unwrap_spec(n=int)
     def descr_size_alignment(self, space, n=1):
         return space.newtuple([space.wrap(self.size * n),
                                space.wrap(self.alignment)])
-    descr_size_alignment.unwrap_spec = ['self', ObjSpace, int]
     
 
 class W_DataInstance(Wrappable):
@@ -276,13 +275,11 @@
         array = ARRAY_OF_PTRS.allocate(space, 1)
         array.setitem(space, 0, space.wrap(self))
         return space.wrap(array)
-    byptr.unwrap_spec = ['self', ObjSpace]
 
     def free(self, space):
         if not self.ll_buffer:
             raise segfault_exception(space, "freeing NULL pointer")
         self._free()
-    free.unwrap_spec = ['self', ObjSpace]
 
     def _free(self):
         if tracker.DO_TRACING:
@@ -294,7 +291,6 @@
     def descr_buffer(self, space):
         from pypy.module._rawffi.buffer import RawFFIBuffer
         return space.wrap(RawFFIBuffer(self))
-    descr_buffer.unwrap_spec = ['self', ObjSpace]
 
     def getrawsize(self):
         raise NotImplementedError("abstract base class")
@@ -391,7 +387,6 @@
             #     hence our testing is not performing that well
             del tracker.alloced[rffi.cast(lltype.Signed, array.ll_buffer)]
         return space.wrap(array)
-    byptr.unwrap_spec = ['self', ObjSpace]
 
     def call(self, space, args_w):
         from pypy.module._rawffi.array import W_ArrayInstance
@@ -445,8 +440,8 @@
                 return space.w_None
         except StackCheckError, e:
             raise OperationError(space.w_ValueError, space.wrap(e.message))
-    call.unwrap_spec = ['self', ObjSpace, 'args_w']
 
+ at unwrap_spec(addr=r_uint, flags=int)
 def descr_new_funcptr(space, w_tp, addr, w_args, w_res, flags=FUNCFLAG_CDECL):
     argshapes = unpack_argshapes(space, w_args)
     resshape = unpack_resshape(space, w_res)
@@ -455,7 +450,6 @@
     ptr = RawFuncPtr('???', ffi_args, ffi_res, rffi.cast(rffi.VOIDP, addr),
                      flags)
     return space.wrap(W_FuncPtr(space, ptr, argshapes, resshape))
-descr_new_funcptr.unwrap_spec = [ObjSpace, W_Root, r_uint, W_Root, W_Root, int]
 
 W_FuncPtr.typedef = TypeDef(
     'FuncPtr',
@@ -467,6 +461,7 @@
 W_FuncPtr.typedef.acceptable_as_base_class = False
 
 def _create_new_accessor(func_name, name):
+    @unwrap_spec(tp_letter=str)
     def accessor(space, tp_letter):
         if len(tp_letter) != 1:
             raise OperationError(space.w_ValueError, space.wrap(
@@ -477,50 +472,49 @@
         except KeyError:
             raise operationerrfmt(space.w_ValueError,
                         "Unknown type specification %s", tp_letter)
-    accessor.unwrap_spec = [ObjSpace, str]
     return func_with_new_name(accessor, func_name)
 
 sizeof = _create_new_accessor('sizeof', 'c_size')
 alignment = _create_new_accessor('alignment', 'c_alignment')
 
+ at unwrap_spec(address=r_uint, maxlength=int)
 def charp2string(space, address, maxlength=sys.maxint):
     if address == 0:
         return space.w_None
     s = rffi.charp2strn(rffi.cast(rffi.CCHARP, address), maxlength)
     return space.wrap(s)
-charp2string.unwrap_spec = [ObjSpace, r_uint, int]
 
+ at unwrap_spec(address=r_uint, maxlength=int)
 def wcharp2unicode(space, address, maxlength=sys.maxint):
     if address == 0:
         return space.w_None
     s = rffi.wcharp2unicoden(rffi.cast(rffi.CWCHARP, address), maxlength)
     return space.wrap(s)
-wcharp2unicode.unwrap_spec = [ObjSpace, r_uint, int]
 
+ at unwrap_spec(address=r_uint, maxlength=int)
 def charp2rawstring(space, address, maxlength=-1):
     if maxlength == -1:
         return charp2string(space, address)
     s = rffi.charpsize2str(rffi.cast(rffi.CCHARP, address), maxlength)
     return space.wrap(s)
-charp2rawstring.unwrap_spec = [ObjSpace, r_uint, int]
 
+ at unwrap_spec(address=r_uint, maxlength=int)
 def wcharp2rawunicode(space, address, maxlength=-1):
     if maxlength == -1:
         return wcharp2unicode(space, address)
     s = rffi.wcharpsize2unicode(rffi.cast(rffi.CWCHARP, address), maxlength)
     return space.wrap(s)
-wcharp2rawunicode.unwrap_spec = [ObjSpace, r_uint, int]
 
 if _MS_WINDOWS:
+    @unwrap_spec(code=int)
     def FormatError(space, code):
         return space.wrap(rwin32.FormatError(code))
-    FormatError.unwrap_spec = [ObjSpace, int]
 
+    @unwrap_spec(hresult=int)
     def check_HRESULT(space, hresult):
         if rwin32.FAILED(hresult):
             raise OperationError(space.w_WindowsError, space.wrap(hresult))
         return space.wrap(hresult)
-    check_HRESULT.unwrap_spec = [ObjSpace, int]
 
 def get_libc(space):
     name = get_libc_name()

diff --git a/pypy/module/_rawffi/callback.py b/pypy/module/_rawffi/callback.py
--- a/pypy/module/_rawffi/callback.py
+++ b/pypy/module/_rawffi/callback.py
@@ -1,7 +1,5 @@
 
-from pypy.interpreter.baseobjspace import W_Root, ObjSpace, Wrappable,\
-     Arguments
-from pypy.interpreter.gateway import interp2app
+from pypy.interpreter.gateway import interp2app, unwrap_spec
 from pypy.interpreter.typedef import TypeDef, GetSetProperty
 from pypy.rpython.lltypesystem import lltype, rffi
 from pypy.module._rawffi.array import get_elem, push_elem
@@ -93,13 +91,11 @@
             addr = rffi.cast(lltype.Signed, self.ll_callback.ll_closure)
             tracker.trace_free(addr)
         del self.global_counter.CallbackPtr_by_number[self.number]
-    free.unwrap_spec = ['self']
 
+ at unwrap_spec(flags=int)
 def descr_new_callbackptr(space, w_type, w_callable, w_args, w_result,
                           flags=FUNCFLAG_CDECL):
     return W_CallbackPtr(space, w_callable, w_args, w_result, flags)
-descr_new_callbackptr.unwrap_spec = [ObjSpace, W_Root, W_Root, W_Root, W_Root,
-                                     int]
 
 W_CallbackPtr.typedef = TypeDef(
     'CallbackPtr',

diff --git a/pypy/module/_rawffi/array.py b/pypy/module/_rawffi/array.py
--- a/pypy/module/_rawffi/array.py
+++ b/pypy/module/_rawffi/array.py
@@ -3,9 +3,7 @@
 to app-level with apropriate interface
 """
 
-from pypy.interpreter.baseobjspace import W_Root, ObjSpace, Wrappable,\
-     Arguments
-from pypy.interpreter.gateway import interp2app
+from pypy.interpreter.gateway import interp2app, unwrap_spec
 from pypy.interpreter.typedef import TypeDef, GetSetProperty, interp_attrproperty
 from pypy.rpython.lltypesystem import lltype, rffi
 from pypy.interpreter.error import OperationError
@@ -47,6 +45,7 @@
     def get_basic_ffi_type(self):
         return self.basicffitype
 
+    @unwrap_spec(length=int, autofree=bool)
     def descr_call(self, space, length, w_items=None, autofree=False):
         result = self.allocate(space, length, autofree)
         if not space.is_w(w_items, space.w_None):
@@ -66,11 +65,10 @@
         return space.wrap("<_rawffi.Array '%s' (%d, %d)>" % (self.itemcode,
                                                              self.size,
                                                              self.alignment))
-    descr_repr.unwrap_spec = ['self', ObjSpace]
 
+    @unwrap_spec(address=r_uint, length=int)
     def fromaddress(self, space, address, length):
         return space.wrap(W_ArrayInstance(space, self, length, address))
-    fromaddress.unwrap_spec = ['self', ObjSpace, r_uint, int]
 
 PRIMITIVE_ARRAY_TYPES = {}
 for _code in TYPEMAP:
@@ -84,10 +82,8 @@
 
 W_Array.typedef = TypeDef(
     'Array',
-    __new__  = interp2app(descr_new_array,
-                          unwrap_spec=[ObjSpace, W_Root, W_Root]),
-    __call__ = interp2app(W_Array.descr_call,
-                          unwrap_spec=['self', ObjSpace, int, W_Root, int]),
+    __new__  = interp2app(descr_new_array),
+    __call__ = interp2app(W_Array.descr_call),
     __repr__ = interp2app(W_Array.descr_repr),
     fromaddress = interp2app(W_Array.fromaddress),
     size_alignment = interp2app(W_Array.descr_size_alignment)
@@ -113,7 +109,6 @@
         addr = rffi.cast(lltype.Unsigned, self.ll_buffer)
         return space.wrap("<_rawffi array %x of length %d>" % (addr,
                                                                self.length))
-    descr_repr.unwrap_spec = ['self', ObjSpace]
 
     # This only allows non-negative indexes.  Arrays of shape 'c' also
     # support simple slices.
@@ -135,7 +130,6 @@
             self.setslice(space, w_index, w_value)
         else:
             self.setitem(space, num, w_value)
-    descr_setitem.unwrap_spec = ['self', ObjSpace, W_Root, W_Root]
 
     def getitem(self, space, num):
         if not self.ll_buffer:
@@ -154,17 +148,15 @@
             return self.getslice(space, w_index)
         else:
             return self.getitem(space, num)
-    descr_getitem.unwrap_spec = ['self', ObjSpace, W_Root]
 
     def getlength(self, space):
         return space.wrap(self.length)
-    getlength.unwrap_spec = ['self', ObjSpace]
 
+    @unwrap_spec(num=int)
     def descr_itemaddress(self, space, num):
         itemsize = self.shape.size
         ptr = rffi.ptradd(self.ll_buffer, itemsize * num)
         return space.wrap(rffi.cast(lltype.Unsigned, ptr))
-    descr_itemaddress.unwrap_spec = ['self', ObjSpace, int]
 
     def getrawsize(self):
         itemsize = self.shape.size

diff --git a/pypy/module/_rawffi/tracker.py b/pypy/module/_rawffi/tracker.py
--- a/pypy/module/_rawffi/tracker.py
+++ b/pypy/module/_rawffi/tracker.py
@@ -2,8 +2,6 @@
 """ The file that keeps track about freed/kept-alive objects allocated
 by _rawffi. Used for debugging ctypes
 """
-from pypy.interpreter.baseobjspace import W_Root, ObjSpace, Wrappable, \
-     Arguments
 
 class Tracker(object):
     DO_TRACING = True
@@ -23,7 +21,6 @@
 
 def num_of_allocated_objects(space):
     return space.wrap(len(tracker.alloced))
-num_of_allocated_objects.unwrap_spec = [ObjSpace]
 
 def print_alloced_objects(space):
     xxx

diff --git a/pypy/module/_rawffi/structure.py b/pypy/module/_rawffi/structure.py
--- a/pypy/module/_rawffi/structure.py
+++ b/pypy/module/_rawffi/structure.py
@@ -3,10 +3,8 @@
 to app-level with apropriate interface
 """
 
-from pypy.interpreter.baseobjspace import W_Root, Wrappable
-from pypy.interpreter.gateway import interp2app, ObjSpace
+from pypy.interpreter.gateway import interp2app, unwrap_spec
 from pypy.interpreter.typedef import interp_attrproperty
-from pypy.interpreter.argument import Arguments
 from pypy.interpreter.typedef import TypeDef, GetSetProperty
 from pypy.rpython.lltypesystem import lltype, rffi
 from pypy.interpreter.error import OperationError, operationerrfmt
@@ -169,33 +167,32 @@
             raise operationerrfmt(space.w_AttributeError,
                 "C Structure has no attribute %s", attr)
 
+    @unwrap_spec(autofree=bool)
     def descr_call(self, space, autofree=False):
         return space.wrap(self.allocate(space, 1, autofree))
-    descr_call.unwrap_spec = ['self', ObjSpace, int]
 
     def descr_repr(self, space):
         fieldnames = ' '.join(["'%s'" % name for name, _, _ in self.fields])
         return space.wrap("<_rawffi.Structure %s (%d, %d)>" % (fieldnames,
                                                                self.size,
                                                                self.alignment))
-    descr_repr.unwrap_spec = ['self', ObjSpace]
 
+    @unwrap_spec(address=r_uint)
     def fromaddress(self, space, address):
         return space.wrap(W_StructureInstance(space, self, address))
-    fromaddress.unwrap_spec = ['self', ObjSpace, r_uint]
 
+    @unwrap_spec(attr=str)
     def descr_fieldoffset(self, space, attr):
         index = self.getindex(space, attr)
         return space.wrap(self.ll_positions[index])
-    descr_fieldoffset.unwrap_spec = ['self', ObjSpace, str]
 
+    @unwrap_spec(attr=str)
     def descr_fieldsize(self, space, attr):
         index = self.getindex(space, attr)
         if self.ll_bitsizes and index < len(self.ll_bitsizes):
             return space.wrap(self.ll_bitsizes[index])
         else:
             return space.wrap(self.fields[index][1].size)
-    descr_fieldsize.unwrap_spec = ['self', ObjSpace, str]
 
     # get the corresponding ffi_type
     ffi_struct = lltype.nullptr(clibffi.FFI_STRUCT_P.TO)
@@ -227,21 +224,20 @@
     
 
 
-def descr_new_structure(space, w_type, w_shapeinfo, union=0, pack=0):
+ at unwrap_spec(union=bool, pack=int)
+def descr_new_structure(space, w_type, w_shapeinfo, union=False, pack=0):
     if pack < 0:
         raise OperationError(space.w_ValueError, space.wrap(
             "_pack_ must be a non-negative integer"))
 
-    is_union = bool(union)
     if space.is_true(space.isinstance(w_shapeinfo, space.w_tuple)):
         w_size, w_alignment = space.fixedview(w_shapeinfo, expected_length=2)
         S = W_Structure(space, None, space.int_w(w_size),
-                                     space.int_w(w_alignment), is_union)
+                                     space.int_w(w_alignment), union)
     else:
         fields = unpack_fields(space, w_shapeinfo)
-        S = W_Structure(space, fields, 0, 0, is_union, pack)
+        S = W_Structure(space, fields, 0, 0, union, pack)
     return space.wrap(S)
-descr_new_structure.unwrap_spec = [ObjSpace, W_Root, W_Root, int, int]
 
 W_Structure.typedef = TypeDef(
     'Structure',
@@ -321,29 +317,28 @@
     def descr_repr(self, space):
         addr = rffi.cast(lltype.Unsigned, self.ll_buffer)
         return space.wrap("<_rawffi struct %x>" % (addr,))
-    descr_repr.unwrap_spec = ['self', ObjSpace]
 
+    @unwrap_spec(attr=str)
     def getattr(self, space, attr):
         if not self.ll_buffer:
             raise segfault_exception(space, "accessing NULL pointer")
         i = self.shape.getindex(space, attr)
         _, tp, _ = self.shape.fields[i]
         return wrap_value(space, cast_pos, self, i, tp.itemcode)
-    getattr.unwrap_spec = ['self', ObjSpace, str]
 
+    @unwrap_spec(attr=str)
     def setattr(self, space, attr, w_value):
         if not self.ll_buffer:
             raise segfault_exception(space, "accessing NULL pointer")
         i = self.shape.getindex(space, attr)
         _, tp, _ = self.shape.fields[i]
         unwrap_value(space, push_field, self, i, tp.itemcode, w_value)
-    setattr.unwrap_spec = ['self', ObjSpace, str, W_Root]
 
+    @unwrap_spec(attr=str)
     def descr_fieldaddress(self, space, attr):
         i = self.shape.getindex(space, attr)
         ptr = rffi.ptradd(self.ll_buffer, self.shape.ll_positions[i])
         return space.wrap(rffi.cast(lltype.Unsigned, ptr))
-    descr_fieldaddress.unwrap_spec = ['self', ObjSpace, str]
 
     def getrawsize(self):
         return self.shape.size

From commits-noreply at bitbucket.org  Wed Feb 16 19:19:36 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 19:19:36 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: modern unwrap_spec in _socket
	module
Message-ID: <20110216181936.41C80282C22@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42064:1e57fd2c5b76
Date: 2011-02-16 13:38 +0100
http://bitbucket.org/pypy/pypy/changeset/1e57fd2c5b76/

Log:	modern unwrap_spec in _socket module

diff --git a/pypy/module/_socket/interp_socket.py b/pypy/module/_socket/interp_socket.py
--- a/pypy/module/_socket/interp_socket.py
+++ b/pypy/module/_socket/interp_socket.py
@@ -2,7 +2,7 @@
 from pypy.interpreter.typedef import TypeDef, make_weakref_descr,\
      interp_attrproperty
 from pypy.interpreter.gateway import ObjSpace, W_Root, NoneNotWrapped
-from pypy.interpreter.gateway import interp2app
+from pypy.interpreter.gateway import interp2app, unwrap_spec
 from pypy.rlib.rarithmetic import intmask
 from pypy.rlib import rsocket
 from pypy.rlib.rsocket import RSocket, AF_INET, SOCK_STREAM
@@ -35,7 +35,6 @@
                                    addr.as_object(sock.fd, space)])
         except SocketError, e:
             raise converted_error(space, e)
-    accept_w.unwrap_spec = ['self', ObjSpace]
 
     def bind_w(self, space, w_addr):
         """bind(address)
@@ -48,7 +47,6 @@
             self.bind(self.addr_from_object(space, w_addr))
         except SocketError, e:
             raise converted_error(space, e)
-    bind_w.unwrap_spec = ['self', ObjSpace, W_Root]
 
     def close_w(self, space):
         """close()
@@ -59,7 +57,6 @@
             self.close()
         except SocketError, e:
             raise converted_error(space, e)
-    close_w.unwrap_spec = ['self', ObjSpace]
 
     def connect_w(self, space, w_addr):
         """connect(address)
@@ -74,7 +71,6 @@
         except TypeError, e:
             raise OperationError(space.w_TypeError,
                                  space.wrap(str(e)))
-    connect_w.unwrap_spec = ['self', ObjSpace, W_Root]
 
     def connect_ex_w(self, space, w_addr):
         """connect_ex(address) -> errno
@@ -88,14 +84,12 @@
             raise converted_error(space, e)
         error = self.connect_ex(addr)
         return space.wrap(error)
-    connect_ex_w.unwrap_spec = ['self', ObjSpace, W_Root]
 
     def dup_w(self, space):
         try:
             return self.dup(W_RSocket)
         except SocketError, e:
             raise converted_error(space, e)
-    dup_w.unwrap_spec = ['self', ObjSpace]
     
     def fileno_w(self, space):
         """fileno() -> integer
@@ -103,7 +97,6 @@
         Return the integer file descriptor of the socket.
         """
         return space.wrap(intmask(self.fd))
-    fileno_w.unwrap_spec = ['self', ObjSpace]
 
     def getpeername_w(self, space):
         """getpeername() -> address info
@@ -116,7 +109,6 @@
             return addr.as_object(self.fd, space)
         except SocketError, e:
             raise converted_error(space, e)
-    getpeername_w.unwrap_spec = ['self', ObjSpace]
 
     def getsockname_w(self, space):
         """getsockname() -> address info
@@ -129,8 +121,8 @@
             return addr.as_object(self.fd, space)
         except SocketError, e:
             raise converted_error(space, e)
-    getsockname_w.unwrap_spec = ['self', ObjSpace]
 
+    @unwrap_spec(level=int, optname=int)
     def getsockopt_w(self, space, level, optname, w_buflen=NoneNotWrapped):
         """getsockopt(level, option[, buffersize]) -> value
 
@@ -145,7 +137,6 @@
                 raise converted_error(space, e)
         buflen = space.int_w(w_buflen)
         return space.wrap(self.getsockopt(level, optname, buflen))
-    getsockopt_w.unwrap_spec = ['self', ObjSpace, int, int, W_Root]
 
     def gettimeout_w(self, space):
         """gettimeout() -> timeout
@@ -157,8 +148,8 @@
         if timeout < 0.0:
             return space.w_None
         return space.wrap(timeout)
-    gettimeout_w.unwrap_spec = ['self', ObjSpace]
-    
+
+    @unwrap_spec(backlog=int)
     def listen_w(self, space, backlog):
         """listen(backlog)
 
@@ -170,7 +161,6 @@
             self.listen(backlog)
         except SocketError, e:
             raise converted_error(space, e)
-    listen_w.unwrap_spec = ['self', ObjSpace, int]
 
     def makefile_w(self, space, w_mode="r", w_buffsize=-1):
         """makefile([mode[, buffersize]]) -> file object
@@ -179,10 +169,8 @@
         The mode and buffersize arguments are as for the built-in open() function.
         """
         return app_makefile(space, self, w_mode, w_buffsize)
-    makefile_w.unwrap_spec = ['self', ObjSpace, W_Root, W_Root]
-        
-        
-                   
+
+    @unwrap_spec(buffersize='nonnegint', flags=int)
     def recv_w(self, space, buffersize, flags=0):
         """recv(buffersize[, flags]) -> data
 
@@ -196,8 +184,8 @@
         except SocketError, e:
             raise converted_error(space, e)
         return space.wrap(data)
-    recv_w.unwrap_spec = ['self', ObjSpace, 'nonnegint', int]
 
+    @unwrap_spec(buffersize='nonnegint', flags=int)
     def recvfrom_w(self, space, buffersize, flags=0):
         """recvfrom(buffersize[, flags]) -> (data, address info)
 
@@ -212,8 +200,8 @@
             return space.newtuple([space.wrap(data), w_addr])
         except SocketError, e:
             raise converted_error(space, e)
-    recvfrom_w.unwrap_spec = ['self', ObjSpace, 'nonnegint', int]
 
+    @unwrap_spec(data='bufferstr', flags=int)
     def send_w(self, space, data, flags=0):
         """send(data[, flags]) -> count
 
@@ -226,8 +214,8 @@
         except SocketError, e:
             raise converted_error(space, e)
         return space.wrap(count)
-    send_w.unwrap_spec = ['self', ObjSpace, 'bufferstr', int]
 
+    @unwrap_spec(data='bufferstr', flags=int)
     def sendall_w(self, space, data, flags=0):
         """sendall(data[, flags])
 
@@ -240,8 +228,8 @@
             count = self.sendall(data, flags, SignalChecker(space))
         except SocketError, e:
             raise converted_error(space, e)
-    sendall_w.unwrap_spec = ['self', ObjSpace, 'bufferstr', int]
 
+    @unwrap_spec(data='bufferstr')
     def sendto_w(self, space, data, w_param2, w_param3=NoneNotWrapped):
         """sendto(data[, flags], address) -> count
 
@@ -262,18 +250,18 @@
         except SocketError, e:
             raise converted_error(space, e)
         return space.wrap(count)
-    sendto_w.unwrap_spec = ['self', ObjSpace, 'bufferstr', W_Root, W_Root]
 
-    def setblocking_w(self, space, flag):
+    @unwrap_spec(flag=bool)
+    def setblocking_w(self, flag):
         """setblocking(flag)
 
         Set the socket to blocking (flag is true) or non-blocking (false).
         setblocking(True) is equivalent to settimeout(None);
         setblocking(False) is equivalent to settimeout(0.0).
         """
-        self.setblocking(bool(flag))
-    setblocking_w.unwrap_spec = ['self', ObjSpace, int]
+        self.setblocking(flag)
 
+    @unwrap_spec(level=int, optname=int)
     def setsockopt_w(self, space, level, optname, w_optval):
         """setsockopt(level, option, value)
 
@@ -293,9 +281,7 @@
             self.setsockopt_int(level, optname, optval)
         except SocketError, e:
             raise converted_error(space, e)
-            
-    setsockopt_w.unwrap_spec = ['self', ObjSpace, int, int, W_Root]
-    
+
     def settimeout_w(self, space, w_timeout):
         """settimeout(timeout)
 
@@ -312,8 +298,8 @@
                 raise OperationError(space.w_ValueError,
                                      space.wrap('Timeout value out of range'))
         self.settimeout(timeout)
-    settimeout_w.unwrap_spec = ['self', ObjSpace, W_Root]
 
+    @unwrap_spec(nbytes=int, flags=int)
     def recv_into_w(self, space, w_buffer, nbytes=0, flags=0):
         rwbuffer = space.rwbuffer_w(w_buffer)
         lgt = rwbuffer.getlength()
@@ -324,8 +310,7 @@
         except SocketError, e:
             raise converted_error(space, e)
 
-    recv_into_w.unwrap_spec = ['self', ObjSpace, W_Root, int, int]
-
+    @unwrap_spec(nbytes=int, flags=int)
     def recvfrom_into_w(self, space, w_buffer, nbytes=0, flags=0):
         rwbuffer = space.rwbuffer_w(w_buffer)
         lgt = rwbuffer.getlength()
@@ -340,8 +325,8 @@
             return space.newtuple([space.wrap(readlgt), w_addr])
         except SocketError, e:
             raise converted_error(space, e)        
-    recvfrom_into_w.unwrap_spec = ['self', ObjSpace, W_Root, int, int]
 
+    @unwrap_spec(cmd=int)
     def ioctl_w(self, space, cmd, w_option):
         from pypy.rpython.lltypesystem import rffi, lltype
         from pypy.rlib import rwin32
@@ -381,8 +366,8 @@
             return space.wrap(recv_ptr[0])
         finally:
             lltype.free(recv_ptr, flavor='raw')
-    ioctl_w.unwrap_spec = ['self', ObjSpace, int, W_Root]
 
+    @unwrap_spec(how=int)
     def shutdown_w(self, space, how):
         """shutdown(flag)
 
@@ -394,7 +379,6 @@
             self.shutdown(how)
         except SocketError, e:
             raise converted_error(space, e)
-    shutdown_w.unwrap_spec = ['self', ObjSpace, int]
 
     #------------------------------------------------------------
     # Support functions for socket._socketobject
@@ -406,7 +390,6 @@
         Intended only to be used by socket._socketobject
         """
         self.usecount += 1
-    _reuse_w.unwrap_spec = ['self']
 
     def _drop_w(self, space):
         """_drop()
@@ -419,7 +402,6 @@
         if self.usecount > 0:
             return
         self.close_w(space)
-    _drop_w.unwrap_spec = ['self', ObjSpace]
 
 app_makefile = gateway.applevel(r'''
 def makefile(self, mode="r", buffersize=-1):
@@ -433,6 +415,7 @@
     return os.fdopen(newfd, mode, buffersize)
 ''', filename =__file__).interphook('makefile')
 
+ at unwrap_spec(family=int, type=int, proto=int)
 def newsocket(space, w_subtype, family=AF_INET,
               type=SOCK_STREAM, proto=0):
     # XXX If we want to support subclassing the socket type we will need
@@ -445,8 +428,7 @@
     except SocketError, e:
         raise converted_error(space, e)
     return space.wrap(sock)
-descr_socket_new = interp2app(newsocket,
-                               unwrap_spec=[ObjSpace, W_Root, int, int, int])
+descr_socket_new = interp2app(newsocket)
 
 # ____________________________________________________________
 # Error handling
@@ -480,9 +462,7 @@
 socketmethods = {}
 for methodname in socketmethodnames:
     method = getattr(W_RSocket, methodname + '_w')
-    assert hasattr(method,'unwrap_spec'), methodname
-    assert method.im_func.func_code.co_argcount == len(method.unwrap_spec), methodname
-    socketmethods[methodname] = interp2app(method, unwrap_spec=method.unwrap_spec)
+    socketmethods[methodname] = interp2app(method)
 
 W_RSocket.typedef = TypeDef("_socket.socket",
     __doc__ = """\

diff --git a/pypy/module/_socket/interp_func.py b/pypy/module/_socket/interp_func.py
--- a/pypy/module/_socket/interp_func.py
+++ b/pypy/module/_socket/interp_func.py
@@ -1,4 +1,4 @@
-from pypy.interpreter.gateway import ObjSpace, W_Root, NoneNotWrapped
+from pypy.interpreter.gateway import NoneNotWrapped, unwrap_spec
 from pypy.module._socket.interp_socket import converted_error, W_RSocket
 from pypy.rlib import rsocket
 from pypy.rlib.rsocket import SocketError
@@ -15,20 +15,19 @@
     except SocketError, e:
         raise converted_error(space, e)
     return space.wrap(res)
-gethostname.unwrap_spec = [ObjSpace]
 
-def gethostbyname(space, hostname):
+ at unwrap_spec(host=str)
+def gethostbyname(space, host):
     """gethostbyname(host) -> address
 
     Return the IP address (a string of the form '255.255.255.255') for a host.
     """
     try:
-        addr = rsocket.gethostbyname(hostname)
+        addr = rsocket.gethostbyname(host)
         ip = addr.get_host()
     except SocketError, e:
         raise converted_error(space, e)
     return space.wrap(ip)
-gethostbyname.unwrap_spec = [ObjSpace, str]
 
 def common_wrapgethost(space, (name, aliases, address_list)):
     aliases = [space.wrap(alias) for alias in aliases]
@@ -37,6 +36,7 @@
                            space.newlist(aliases),
                            space.newlist(address_list)])
 
+ at unwrap_spec(host=str)
 def gethostbyname_ex(space, host):
     """gethostbyname_ex(host) -> (name, aliaslist, addresslist)
 
@@ -48,8 +48,8 @@
     except SocketError, e:
         raise converted_error(space, e)
     return common_wrapgethost(space, res)
-gethostbyname_ex.unwrap_spec = [ObjSpace, str]
 
+ at unwrap_spec(host=str)
 def gethostbyaddr(space, host):
     """gethostbyaddr(host) -> (name, aliaslist, addresslist)
 
@@ -61,8 +61,8 @@
     except SocketError, e:
         raise converted_error(space, e)
     return common_wrapgethost(space, res)
-gethostbyaddr.unwrap_spec = [ObjSpace, str]
 
+ at unwrap_spec(name=str)
 def getservbyname(space, name, w_proto=None):
     """getservbyname(servicename[, protocolname]) -> integer
 
@@ -79,8 +79,8 @@
     except SocketError, e:
         raise converted_error(space, e)
     return space.wrap(port)
-getservbyname.unwrap_spec = [ObjSpace, str, W_Root]
 
+ at unwrap_spec(port=int)
 def getservbyport(space, port, w_proto=None):
     """getservbyport(port[, protocolname]) -> string
 
@@ -102,8 +102,8 @@
     except SocketError, e:
         raise converted_error(space, e)
     return space.wrap(service)
-getservbyport.unwrap_spec = [ObjSpace, int, W_Root]
 
+ at unwrap_spec(name=str)
 def getprotobyname(space, name):
     """getprotobyname(name) -> integer
 
@@ -114,8 +114,8 @@
     except SocketError, e:
         raise converted_error(space, e)
     return space.wrap(proto)
-getprotobyname.unwrap_spec = [ObjSpace, str]
 
+ at unwrap_spec(flags=int)
 def getnameinfo(space, w_sockaddr, flags):
     """getnameinfo(sockaddr, flags) --> (host, port)
 
@@ -126,8 +126,8 @@
     except SocketError, e:
         raise converted_error(space, e)
     return space.newtuple([space.wrap(host), space.wrap(servport)])
-getnameinfo.unwrap_spec = [ObjSpace, W_Root, int]
 
+ at unwrap_spec(fd=int, family=int, type=int, proto=int)
 def fromfd(space, fd, family, type, proto=0):
     """fromfd(fd, family, type[, proto]) -> socket object
 
@@ -139,11 +139,11 @@
     except SocketError, e:
         raise converted_error(space, e)
     return space.wrap(sock)
-fromfd.unwrap_spec = [ObjSpace, int, int, int, int]
 
-def socketpair(space, family = rsocket.socketpair_default_family,
-                      type   = rsocket.SOCK_STREAM,
-                      proto  = 0):
+ at unwrap_spec(family=int, type=int, proto=int)
+def socketpair(space, family=rsocket.socketpair_default_family,
+                      type  =rsocket.SOCK_STREAM,
+                      proto =0):
     """socketpair([family[, type[, proto]]]) -> (socket object, socket object)
 
     Create a pair of socket objects from the sockets returned by the platform
@@ -156,43 +156,43 @@
     except SocketError, e:
         raise converted_error(space, e)
     return space.newtuple([space.wrap(sock1), space.wrap(sock2)])
-socketpair.unwrap_spec = [ObjSpace, int, int, int]
 
 # The following 4 functions refuse all negative numbers, like CPython 2.6.
 # They could also check that the argument is not too large, but CPython 2.6
 # is not doing that consistently.
+ at unwrap_spec(x="c_uint")
 def ntohs(space, x):
     """ntohs(integer) -> integer
 
     Convert a 16-bit integer from network to host byte order.
     """
     return space.wrap(rsocket.ntohs(x))
-ntohs.unwrap_spec = [ObjSpace, "c_uint"]
 
+ at unwrap_spec(x="c_uint")
 def ntohl(space, x):
     """ntohl(integer) -> integer
 
     Convert a 32-bit integer from network to host byte order.
     """
     return space.wrap(rsocket.ntohl(x))
-ntohl.unwrap_spec = [ObjSpace, "c_uint"]
 
+ at unwrap_spec(x="c_uint")
 def htons(space, x):
     """htons(integer) -> integer
 
     Convert a 16-bit integer from host to network byte order.
     """
     return space.wrap(rsocket.htons(x))
-htons.unwrap_spec = [ObjSpace, "c_uint"]
 
+ at unwrap_spec(x="c_uint")
 def htonl(space, x):
     """htonl(integer) -> integer
 
     Convert a 32-bit integer from host to network byte order.
     """
     return space.wrap(rsocket.htonl(x))
-htonl.unwrap_spec = [ObjSpace, "c_uint"]
 
+ at unwrap_spec(ip=str)
 def inet_aton(space, ip):
     """inet_aton(string) -> packed 32-bit IP representation
 
@@ -204,8 +204,8 @@
     except SocketError, e:
         raise converted_error(space, e)
     return space.wrap(buf)
-inet_aton.unwrap_spec = [ObjSpace, str]
 
+ at unwrap_spec(packed=str)
 def inet_ntoa(space, packed):
     """inet_ntoa(packed_ip) -> ip_address_string
 
@@ -216,8 +216,8 @@
     except SocketError, e:
         raise converted_error(space, e)
     return space.wrap(ip)
-inet_ntoa.unwrap_spec = [ObjSpace, str]
 
+ at unwrap_spec(family=int, ip=str)
 def inet_pton(space, family, ip):
     """inet_pton(family, ip) -> packed IP address string
 
@@ -229,8 +229,8 @@
     except SocketError, e:
         raise converted_error(space, e)
     return space.wrap(buf)
-inet_pton.unwrap_spec = [ObjSpace, int, str]
 
+ at unwrap_spec(family=int, packed=str)
 def inet_ntop(space, family, packed):
     """inet_ntop(family, packed_ip) -> string formatted IP address
 
@@ -244,8 +244,8 @@
         raise OperationError(space.w_ValueError,
                   space.wrap(str(e)))
     return space.wrap(ip)
-inet_ntop.unwrap_spec = [ObjSpace, int, str]
 
+ at unwrap_spec(family=int, socktype=int, proto=int, flags=int)
 def getaddrinfo(space, w_host, w_port,
                 family=rsocket.AF_UNSPEC, socktype=0, proto=0, flags=0):
     """getaddrinfo(host, port [, family, socktype, proto, flags])
@@ -288,7 +288,6 @@
                             addr.as_object(-1, space)]) # -1 as per cpython
             for (family, socktype, protocol, canonname, addr) in lst]
     return space.newlist(lst1)
-getaddrinfo.unwrap_spec = [ObjSpace, W_Root, W_Root, int, int, int, int]
 
 def getdefaulttimeout(space):
     """getdefaulttimeout() -> timeout
@@ -301,7 +300,6 @@
     if timeout < 0.0:
         return space.w_None
     return space.wrap(timeout)
-getdefaulttimeout.unwrap_spec = [ObjSpace]
 
 def setdefaulttimeout(space, w_timeout):
     if space.is_w(w_timeout, space.w_None):

From commits-noreply at bitbucket.org  Wed Feb 16 19:19:38 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 19:19:38 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: modern unwrap_spec for _sre
	and _ssl
Message-ID: <20110216181938.05E652A202C@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42065:da911ae5e59b
Date: 2011-02-16 13:46 +0100
http://bitbucket.org/pypy/pypy/changeset/da911ae5e59b/

Log:	modern unwrap_spec for _sre and _ssl

diff --git a/pypy/module/_socket/interp_socket.py b/pypy/module/_socket/interp_socket.py
--- a/pypy/module/_socket/interp_socket.py
+++ b/pypy/module/_socket/interp_socket.py
@@ -1,8 +1,7 @@
 from pypy.interpreter.baseobjspace import Wrappable
 from pypy.interpreter.typedef import TypeDef, make_weakref_descr,\
      interp_attrproperty
-from pypy.interpreter.gateway import ObjSpace, W_Root, NoneNotWrapped
-from pypy.interpreter.gateway import interp2app, unwrap_spec
+from pypy.interpreter.gateway import NoneNotWrapped, interp2app, unwrap_spec
 from pypy.rlib.rarithmetic import intmask
 from pypy.rlib import rsocket
 from pypy.rlib.rsocket import RSocket, AF_INET, SOCK_STREAM

diff --git a/pypy/module/_sre/interp_sre.py b/pypy/module/_sre/interp_sre.py
--- a/pypy/module/_sre/interp_sre.py
+++ b/pypy/module/_sre/interp_sre.py
@@ -3,7 +3,7 @@
 from pypy.interpreter.typedef import GetSetProperty, TypeDef
 from pypy.interpreter.typedef import interp_attrproperty, interp_attrproperty_w
 from pypy.interpreter.typedef import make_weakref_descr
-from pypy.interpreter.gateway import interp2app, ObjSpace, W_Root
+from pypy.interpreter.gateway import interp2app, unwrap_spec
 from pypy.interpreter.error import OperationError
 from pypy.rlib.rarithmetic import intmask
 from pypy.tool.pairtype import extendabletype
@@ -16,9 +16,9 @@
 from pypy.rlib.rsre import rsre_core
 from pypy.rlib.rsre.rsre_char import MAGIC, CODESIZE, getlower, set_unicode_db
 
+ at unwrap_spec(char_ord=int, flags=int)
 def w_getlower(space, char_ord, flags):
     return space.wrap(getlower(char_ord, flags))
-w_getlower.unwrap_spec = [ObjSpace, int, int]
 
 def w_getcodesize(space):
     return space.wrap(CODESIZE)
@@ -130,16 +130,17 @@
         else:
             return self.space.w_None
 
+    @unwrap_spec(pos=int, endpos=int)
     def match_w(self, w_string, pos=0, endpos=sys.maxint):
         ctx = self.make_ctx(w_string, pos, endpos)
         return self.getmatch(ctx, matchcontext(self.space, ctx))
-    match_w.unwrap_spec = ['self', W_Root, int, int]
 
+    @unwrap_spec(pos=int, endpos=int)
     def search_w(self, w_string, pos=0, endpos=sys.maxint):
         ctx = self.make_ctx(w_string, pos, endpos)
         return self.getmatch(ctx, searchcontext(self.space, ctx))
-    search_w.unwrap_spec = ['self', W_Root, int, int]
 
+    @unwrap_spec(pos=int, endpos=int)
     def findall_w(self, w_string, pos=0, endpos=sys.maxint):
         space = self.space
         matchlist_w = []
@@ -164,16 +165,16 @@
             no_progress = (ctx.match_start == ctx.match_end)
             ctx.reset(ctx.match_end + no_progress)
         return space.newlist(matchlist_w)
-    findall_w.unwrap_spec = ['self', W_Root, int, int]
 
+    @unwrap_spec(pos=int, endpos=int)
     def finditer_w(self, w_string, pos=0, endpos=sys.maxint):
         # this also works as the implementation of the undocumented
         # scanner() method.
         ctx = self.make_ctx(w_string, pos, endpos)
         scanner = W_SRE_Scanner(self, ctx)
         return self.space.wrap(scanner)
-    finditer_w.unwrap_spec = ['self', W_Root, int, int]
 
+    @unwrap_spec(maxsplit=int)
     def split_w(self, w_string, maxsplit=0):
         space = self.space
         splitlist = []
@@ -201,18 +202,17 @@
             ctx.reset(last)
         splitlist.append(slice_w(space, ctx, last, ctx.end, space.w_None))
         return space.newlist(splitlist)
-    split_w.unwrap_spec = ['self', W_Root, int]
 
+    @unwrap_spec(count=int)
     def sub_w(self, w_repl, w_string, count=0):
         w_item, n = self.subx(w_repl, w_string, count)
         return w_item
-    sub_w.unwrap_spec = ['self', W_Root, W_Root, int]
 
+    @unwrap_spec(count=int)
     def subn_w(self, w_repl, w_string, count=0):
         w_item, n = self.subx(w_repl, w_string, count)
         space = self.space
         return space.newtuple([w_item, space.wrap(n)])
-    subn_w.unwrap_spec = ['self', W_Root, W_Root, int]
 
     def subx(self, w_ptemplate, w_string, count):
         space = self.space
@@ -287,6 +287,7 @@
         return w_item, n
 
 
+ at unwrap_spec(flags=int, groups=int)
 def SRE_Pattern__new__(space, w_subtype, w_pattern, flags, w_code,
               groups=0, w_groupindex=None, w_indexgroup=None):
     n = space.len_w(w_code)
@@ -303,8 +304,6 @@
     srepat.w_groupindex = w_groupindex
     srepat.w_indexgroup = w_indexgroup
     return w_srepat
-SRE_Pattern__new__.unwrap_spec = [ObjSpace, W_Root, W_Root, int, W_Root,
-                                  int, W_Root, W_Root]
 
 
 W_SRE_Pattern.typedef = TypeDef(
@@ -359,7 +358,6 @@
                 start, end = self.do_span(args_w[i])
                 results[i] = slice_w(space, ctx, start, end, space.w_None)
             return space.newtuple(results)
-    group_w.unwrap_spec = ['self', 'args_w']
 
     def groups_w(self, w_default=None):
         fmarks = self.flatten_marks()
@@ -542,9 +540,9 @@
 
 W_SRE_Scanner.typedef = TypeDef(
     'SRE_Scanner',
-    __iter__ = interp2app(W_SRE_Scanner.iter_w,   unwrap_spec=['self']),
-    next     = interp2app(W_SRE_Scanner.next_w,   unwrap_spec=['self']),
-    match    = interp2app(W_SRE_Scanner.match_w,  unwrap_spec=['self']),
-    search   = interp2app(W_SRE_Scanner.search_w, unwrap_spec=['self']),
+    __iter__ = interp2app(W_SRE_Scanner.iter_w),
+    next     = interp2app(W_SRE_Scanner.next_w),
+    match    = interp2app(W_SRE_Scanner.match_w),
+    search   = interp2app(W_SRE_Scanner.search_w),
     pattern  = interp_attrproperty('srepat', W_SRE_Scanner),
 )

diff --git a/pypy/module/_ssl/interp_ssl.py b/pypy/module/_ssl/interp_ssl.py
--- a/pypy/module/_ssl/interp_ssl.py
+++ b/pypy/module/_ssl/interp_ssl.py
@@ -2,7 +2,7 @@
 from pypy.interpreter.error import OperationError
 from pypy.interpreter.baseobjspace import W_Root, ObjSpace, Wrappable
 from pypy.interpreter.typedef import TypeDef
-from pypy.interpreter.gateway import interp2app
+from pypy.interpreter.gateway import interp2app, unwrap_spec
 
 from pypy.rlib import rpoll, rsocket
 from pypy.rlib.ropenssl import *
@@ -78,6 +78,7 @@
 
 if HAVE_OPENSSL_RAND:
     # helper routines for seeding the SSL PRNG
+    @unwrap_spec(string=str, entropy=float)
     def RAND_add(space, string, entropy):
         """RAND_add(string, entropy)
 
@@ -90,7 +91,6 @@
             libssl_RAND_add(buf, len(string), entropy)
         finally:
             rffi.free_charp(buf)
-    RAND_add.unwrap_spec = [ObjSpace, str, float]
 
     def RAND_status(space):
         """RAND_status() -> 0 or 1
@@ -101,8 +101,8 @@
 
         res = libssl_RAND_status()
         return space.wrap(res)
-    RAND_status.unwrap_spec = [ObjSpace]
 
+    @unwrap_spec(path=str)
     def RAND_egd(space, path):
         """RAND_egd(path) -> bytes
 
@@ -120,7 +120,6 @@
             msg += " enough data to seed the PRNG"
             raise ssl_error(space, msg)
         return space.wrap(bytes)
-    RAND_egd.unwrap_spec = [ObjSpace, str]
 
 class SSLObject(Wrappable):
     def __init__(self, space):
@@ -137,11 +136,9 @@
     
     def server(self):
         return self.space.wrap(rffi.charp2str(self._server))
-    server.unwrap_spec = ['self']
     
     def issuer(self):
         return self.space.wrap(rffi.charp2str(self._issuer))
-    issuer.unwrap_spec = ['self']
     
     def __del__(self):
         if self.peer_cert:
@@ -153,6 +150,7 @@
         lltype.free(self._server, flavor='raw')
         lltype.free(self._issuer, flavor='raw')
     
+    @unwrap_spec(data='bufferstr')
     def write(self, data):
         """write(s) -> len
 
@@ -201,8 +199,8 @@
             return self.space.wrap(num_bytes)
         else:
             raise _ssl_seterror(self.space, self, num_bytes)
-    write.unwrap_spec = ['self', 'bufferstr']
-    
+
+    @unwrap_spec(num_bytes=int)
     def read(self, num_bytes=1024):
         """read([len]) -> string
 
@@ -256,7 +254,6 @@
         result = rffi.str_from_buffer(raw_buf, gc_buf, num_bytes, count)
         rffi.keep_buffer_alive_until_here(raw_buf, gc_buf)
         return self.space.wrap(result)
-    read.unwrap_spec = ['self', int]
 
     def _refresh_nonblocking(self, space):
         # just in case the blocking state of the socket has been changed
@@ -375,17 +372,12 @@
 
 
 SSLObject.typedef = TypeDef("SSLObject",
-    server = interp2app(SSLObject.server,
-        unwrap_spec=SSLObject.server.unwrap_spec),
-    issuer = interp2app(SSLObject.issuer,
-        unwrap_spec=SSLObject.issuer.unwrap_spec),
-    write = interp2app(SSLObject.write,
-        unwrap_spec=SSLObject.write.unwrap_spec),
-    read = interp2app(SSLObject.read, unwrap_spec=SSLObject.read.unwrap_spec),
-    do_handshake=interp2app(SSLObject.do_handshake,
-                            unwrap_spec=['self', ObjSpace]),
-    shutdown=interp2app(SSLObject.shutdown,
-                        unwrap_spec=['self', ObjSpace]),
+    server = interp2app(SSLObject.server),
+    issuer = interp2app(SSLObject.issuer),
+    write = interp2app(SSLObject.write),
+    read = interp2app(SSLObject.read),
+    do_handshake=interp2app(SSLObject.do_handshake),
+    shutdown=interp2app(SSLObject.shutdown),
 )
 
 
@@ -452,7 +444,6 @@
 
     ss.w_socket = w_sock
     return ss
-new_sslobject.unwrap_spec = [ObjSpace, W_Root, str, str]
 
 def check_socket_and_wait_for_timeout(space, w_sock, writing):
     """If the socket has a timeout, do a select()/poll() on the socket.
@@ -553,12 +544,11 @@
     return ssl_error(space, errstr, errval)
 
 
+ at unwrap_spec(side=int, cert_mode=int, protocol=int)
 def sslwrap(space, w_socket, side, w_key_file=None, w_cert_file=None,
             cert_mode=PY_SSL_CERT_NONE, protocol=PY_SSL_VERSION_SSL23,
             w_cacerts_file=None, w_cipher=None):
     """sslwrap(socket, side, [keyfile, certfile]) -> sslobject"""
     return space.wrap(new_sslobject(
         space, w_socket, side, w_key_file, w_cert_file))
-sslwrap.unwrap_spec = [ObjSpace, W_Root, int, W_Root, W_Root,
-                       int, int, W_Root, W_Root]
 

From commits-noreply at bitbucket.org  Wed Feb 16 19:19:42 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 19:19:42 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: unwrap_spec in _warnings,
	_stackless
Message-ID: <20110216181942.920672A202C@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42066:899187b5c78c
Date: 2011-02-16 13:52 +0100
http://bitbucket.org/pypy/pypy/changeset/899187b5c78c/

Log:	unwrap_spec in _warnings, _stackless

diff --git a/pypy/module/_stackless/interp_stackless.py b/pypy/module/_stackless/interp_stackless.py
--- a/pypy/module/_stackless/interp_stackless.py
+++ b/pypy/module/_stackless/interp_stackless.py
@@ -1,7 +1,7 @@
 from pypy.interpreter.baseobjspace import Wrappable
 from pypy.interpreter.typedef import GetSetProperty, TypeDef
 from pypy.interpreter.typedef import interp_attrproperty, interp_attrproperty_w
-from pypy.interpreter.gateway import interp2app, ObjSpace, W_Root
+from pypy.interpreter.gateway import interp2app
 from pypy.interpreter.error import OperationError
 from pypy.rlib.rarithmetic import intmask
 import os

diff --git a/pypy/module/_stackless/interp_clonable.py b/pypy/module/_stackless/interp_clonable.py
--- a/pypy/module/_stackless/interp_clonable.py
+++ b/pypy/module/_stackless/interp_clonable.py
@@ -1,6 +1,6 @@
 from pypy.interpreter.error import OperationError
 from pypy.interpreter.typedef import TypeDef
-from pypy.interpreter.gateway import interp2app, ObjSpace, W_Root
+from pypy.interpreter.gateway import interp2app
 from pypy.module._stackless.interp_coroutine import AppCoroutine, AppCoState
 from pypy.module._stackless.interp_coroutine import makeStaticMethod
 from pypy.module._stackless.rcoroutine import AbstractThunk
@@ -58,8 +58,7 @@
     __new__    = interp2app(AppClonableCoroutine.descr_method__new__.im_func),
     getcurrent = interp2app(AppClonableCoroutine.w_getcurrent),
     clone      = interp2app(AppClonableCoroutine.w_clone),
-    __reduce__ = interp2app(AppClonableCoroutine.descr__reduce__,
-                            unwrap_spec=['self', ObjSpace]),
+    __reduce__ = interp2app(AppClonableCoroutine.descr__reduce__),
 )
 
 class AppClonableCoState(AppCoState):
@@ -105,4 +104,3 @@
     # we resume here twice.  The following would need explanations about
     # why it returns the correct thing in both the parent and the child...
     return space.wrap(thunk.newcoroutine)
-fork.unwrap_spec = [ObjSpace]

diff --git a/pypy/module/_stackless/interp_coroutine.py b/pypy/module/_stackless/interp_coroutine.py
--- a/pypy/module/_stackless/interp_coroutine.py
+++ b/pypy/module/_stackless/interp_coroutine.py
@@ -19,7 +19,7 @@
 from pypy.interpreter.argument import Arguments
 from pypy.interpreter.typedef import GetSetProperty, TypeDef
 from pypy.interpreter.typedef import interp_attrproperty, interp_attrproperty_w
-from pypy.interpreter.gateway import interp2app, ObjSpace, W_Root
+from pypy.interpreter.gateway import interp2app, unwrap_spec
 from pypy.interpreter.error import OperationError, operationerrfmt
 from pypy.interpreter.function import StaticMethod
 
@@ -348,8 +348,7 @@
 
 AppCoroutine.typedef = TypeDef("coroutine",
     __new__ = interp2app(AppCoroutine.descr_method__new__.im_func),
-    bind = interp2app(AppCoroutine.w_bind,
-                      unwrap_spec=['self', W_Root, Arguments]),
+    bind = interp2app(AppCoroutine.w_bind),
     switch = interp2app(AppCoroutine.w_switch),
     kill = interp2app(AppCoroutine.w_kill),
     throw = interp2app(AppCoroutine.w_throw),
@@ -362,10 +361,8 @@
     _framestack = GetSetProperty(w_descr__framestack),
     getcurrent = interp2app(AppCoroutine.w_getcurrent),
     getmain = interp2app(AppCoroutine.w_getmain),
-    __reduce__   = interp2app(AppCoroutine.descr__reduce__,
-                              unwrap_spec=['self', ObjSpace]),
-    __setstate__ = interp2app(AppCoroutine.descr__setstate__,
-                              unwrap_spec=['self', ObjSpace, W_Root]),
+    __reduce__   = interp2app(AppCoroutine.descr__reduce__),
+    __setstate__ = interp2app(AppCoroutine.descr__setstate__),
     __module__ = '_stackless',
 )
 
@@ -401,12 +398,10 @@
 
 def return_main(space):
     return AppCoroutine._get_state(space).main
-return_main.unwrap_spec = [ObjSpace]
 
 def get_stack_depth_limit(space):
     return space.wrap(rstack.get_stack_depth_limit())
-get_stack_depth_limit.unwrap_spec = [ObjSpace]
 
+ at unwrap_spec(limit=int)
 def set_stack_depth_limit(space, limit):
     rstack.set_stack_depth_limit(limit)
-set_stack_depth_limit.unwrap_spec = [ObjSpace, int]

diff --git a/pypy/module/_warnings/interp_warnings.py b/pypy/module/_warnings/interp_warnings.py
--- a/pypy/module/_warnings/interp_warnings.py
+++ b/pypy/module/_warnings/interp_warnings.py
@@ -1,5 +1,4 @@
 from pypy.interpreter.gateway import unwrap_spec
-from pypy.interpreter.baseobjspace import W_Root, ObjSpace
 from pypy.interpreter.error import OperationError
 
 def create_filter(space, w_category, action):
@@ -307,7 +306,7 @@
         space.call_function(
             w_show_fxn, w_message, w_category, w_filename, w_lineno)
 
- at unwrap_spec(ObjSpace, W_Root, W_Root, int)
+ at unwrap_spec(stacklevel=int)
 def warn(space, w_message, w_category=None, stacklevel=1):
     w_category = get_category(space, w_message, w_category);
     do_warn(space, w_message, w_category, stacklevel)
@@ -346,7 +345,7 @@
     w_source_line = space.getitem(w_source_list, space.wrap(lineno - 1))
     return w_source_line
 
- at unwrap_spec(ObjSpace, W_Root, W_Root, W_Root, int, W_Root, W_Root, W_Root)
+ at unwrap_spec(lineno=int)
 def warn_explicit(space, w_message, w_category, w_filename, lineno,
                   w_module=None, w_registry=None, w_module_globals=None):
 

diff --git a/pypy/module/_stackless/interp_greenlet.py b/pypy/module/_stackless/interp_greenlet.py
--- a/pypy/module/_stackless/interp_greenlet.py
+++ b/pypy/module/_stackless/interp_greenlet.py
@@ -1,6 +1,6 @@
 from pypy.interpreter.argument import Arguments
 from pypy.interpreter.typedef import GetSetProperty, TypeDef
-from pypy.interpreter.gateway import interp2app, ObjSpace, W_Root
+from pypy.interpreter.gateway import interp2app
 from pypy.interpreter.gateway import NoneNotWrapped
 from pypy.interpreter.error import OperationError
 
@@ -220,11 +220,9 @@
     w_greenlet.__flags__ = old_flags
 
 AppGreenlet.typedef = TypeDef("greenlet",
-    __new__ = interp2app(AppGreenlet.descr_method__new__.im_func,
-                         unwrap_spec=[ObjSpace, W_Root, Arguments]),
+    __new__ = interp2app(AppGreenlet.descr_method__new__.im_func),
     __init__ = interp2app(AppGreenlet.descr_method__init__),
-    switch = interp2app(AppGreenlet.w_switch,
-                        unwrap_spec=['self', 'args_w']),
+    switch = interp2app(AppGreenlet.w_switch),
     dead = GetSetProperty(AppGreenlet.w_get_is_dead),
     run = GetSetProperty(AppGreenlet.w_get_run,
                          AppGreenlet.w_set_run,

From commits-noreply at bitbucket.org  Wed Feb 16 19:19:43 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 19:19:43 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: modern unwrap_spec in _winreg
Message-ID: <20110216181943.876942A202C@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42067:f3f94934cbe1
Date: 2011-02-16 14:04 +0100
http://bitbucket.org/pypy/pypy/changeset/f3f94934cbe1/

Log:	modern unwrap_spec in _winreg

diff --git a/pypy/module/_winreg/interp_winreg.py b/pypy/module/_winreg/interp_winreg.py
--- a/pypy/module/_winreg/interp_winreg.py
+++ b/pypy/module/_winreg/interp_winreg.py
@@ -1,7 +1,7 @@
 from __future__ import with_statement
 from pypy.interpreter.baseobjspace import Wrappable
 from pypy.interpreter.baseobjspace import ObjSpace, W_Root
-from pypy.interpreter.gateway import interp2app, Arguments
+from pypy.interpreter.gateway import interp2app, unwrap_spec
 from pypy.interpreter.typedef import TypeDef, GetSetProperty
 from pypy.interpreter.error import OperationError, wrap_windowserror
 from pypy.rpython.lltypesystem import rffi, lltype
@@ -20,39 +20,32 @@
 
     def descr_del(self, space):
         self.Close(space)
-    descr_del.unwrap_spec = ['self', ObjSpace]
 
     def as_int(self):
         return rffi.cast(rffi.SIZE_T, self.hkey)
 
     def descr_nonzero(self, space):
         return space.wrap(self.as_int() != 0)
-    descr_nonzero.unwrap_spec = ['self', ObjSpace]
 
     def descr_handle_get(space, self):
         return space.wrap(self.as_int())
 
     def descr_repr(self, space):
         return space.wrap("<PyHKEY:0x%x>" % (self.as_int(),))
-    descr_repr.unwrap_spec = ['self', ObjSpace]
 
     def descr_int(self, space):
         return space.wrap(self.as_int())
-    descr_int.unwrap_spec = ['self', ObjSpace]
 
     def descr__enter__(self, space):
         return self
-    descr__enter__.unwrap_spec = ['self', ObjSpace]
 
     def descr__exit__(self, space, __args__):
         CloseKey(space, self)
-    descr__exit__.unwrap_spec = ['self', ObjSpace, Arguments]
 
     def Close(self, space):
         """key.Close() - Closes the underlying Windows handle.
 If the handle is already closed, no error is raised."""
         CloseKey(space, self)
-    Close.unwrap_spec = ['self', ObjSpace]
 
     def Detach(self, space):
         """int = key.Detach() - Detaches the Windows handle from the handle object.
@@ -68,13 +61,12 @@
         key = self.as_int()
         self.hkey = rwin32.NULL_HANDLE
         return space.wrap(key)
-    Detach.unwrap_spec = ['self', ObjSpace]
 
+ at unwrap_spec(key=int)
 def new_HKEY(space, w_subtype, key):
     hkey = rffi.cast(rwinreg.HKEY, key)
     return space.wrap(W_HKEY(hkey))
-descr_HKEY_new = interp2app(new_HKEY,
-                            unwrap_spec=[ObjSpace, W_Root, int])
+descr_HKEY_new = interp2app(new_HKEY)
 
 W_HKEY.typedef = TypeDef(
     "_winreg.HKEYType",
@@ -139,7 +131,6 @@
             raiseWindowsError(space, ret, 'RegCloseKey')
     if isinstance(w_hkey, W_HKEY):
         space.interp_w(W_HKEY, w_hkey).hkey = rwin32.NULL_HANDLE
-CloseKey.unwrap_spec = [ObjSpace, W_Root]
 
 def FlushKey(space, w_hkey):
     """FlushKey(key) - Writes all the attributes of a key to the registry.
@@ -158,8 +149,8 @@
         ret = rwinreg.RegFlushKey(hkey)
         if ret != 0:
             raiseWindowsError(space, ret, 'RegFlushKey')
-FlushKey.unwrap_spec = [ObjSpace, W_Root]
 
+ at unwrap_spec(subkey=str, filename=str)
 def LoadKey(space, w_hkey, subkey, filename):
     """LoadKey(key, sub_key, file_name) - Creates a subkey under the specified key
 and stores registration information from a specified file into that subkey.
@@ -182,8 +173,8 @@
     ret = rwinreg.RegLoadKey(hkey, subkey, filename)
     if ret != 0:
         raiseWindowsError(space, ret, 'RegLoadKey')
-LoadKey.unwrap_spec = [ObjSpace, W_Root, str, str]
 
+ at unwrap_spec(filename=str)
 def SaveKey(space, w_hkey, filename):
     """SaveKey(key, file_name) - Saves the specified key, and all its subkeys to the specified file.
 
@@ -202,8 +193,8 @@
     ret = rwinreg.RegSaveKey(hkey, filename, None)
     if ret != 0:
         raiseWindowsError(space, ret, 'RegSaveKey')
-SaveKey.unwrap_spec = [ObjSpace, W_Root, str]
 
+ at unwrap_spec(typ=int, value=str)
 def SetValue(space, w_hkey, w_subkey, typ, value):
     """SetValue(key, sub_key, type, value) - Associates a value with a specified key.
 
@@ -234,7 +225,6 @@
         ret = rwinreg.RegSetValue(hkey, subkey, rwinreg.REG_SZ, dataptr, len(value))
         if ret != 0:
             raiseWindowsError(space, ret, 'RegSetValue')
-SetValue.unwrap_spec = [ObjSpace, W_Root, W_Root, int, str]
 
 def QueryValue(space, w_hkey, w_subkey):
     """string = QueryValue(key, sub_key) - retrieves the unnamed value for a key.
@@ -272,7 +262,6 @@
                 if ret != 0:
                     raiseWindowsError(space, ret, 'RegQueryValue')
                 return space.wrap(rffi.charp2strn(buf, bufsize_p[0] - 1))
-QueryValue.unwrap_spec = [ObjSpace, W_Root, W_Root]
 
 def convert_to_regdata(space, w_value, typ):
     buf = None
@@ -382,6 +371,7 @@
     else: # REG_BINARY and all other types
         return space.wrap(rffi.charpsize2str(buf, buflen))
 
+ at unwrap_spec(value_name=str, typ=int)
 def SetValueEx(space, w_hkey, value_name, w_reserved, typ, w_value):
     """SetValueEx(key, value_name, reserved, type, value) - Stores data in the value field of an open registry key.
 
@@ -421,7 +411,6 @@
         lltype.free(buf, flavor='raw')
     if ret != 0:
         raiseWindowsError(space, ret, 'RegSetValueEx')
-SetValueEx.unwrap_spec = [ObjSpace, W_Root, str, W_Root, int, W_Root]
 
 def QueryValueEx(space, w_hkey, w_subkey):
     """value,type_id = QueryValueEx(key, value_name) - Retrieves the type and data for a specified value name associated with an open registry key.
@@ -462,8 +451,7 @@
                         space.wrap(retType[0]),
                         ])
 
-QueryValueEx.unwrap_spec = [ObjSpace, W_Root, W_Root]
-
+ at unwrap_spec(subkey=str)
 def CreateKey(space, w_hkey, subkey):
     """key = CreateKey(key, sub_key) - Creates or opens the specified key.
 
@@ -482,8 +470,8 @@
         if ret != 0:
             raiseWindowsError(space, ret, 'CreateKey')
         return space.wrap(W_HKEY(rethkey[0]))
-CreateKey.unwrap_spec = [ObjSpace, W_Root, str]
 
+ at unwrap_spec(subkey=str, res=int, sam=rffi.r_uint)
 def CreateKeyEx(space, w_hkey, subkey, res=0, sam=rwinreg.KEY_WRITE):
     """key = CreateKey(key, sub_key) - Creates or opens the specified key.
 
@@ -504,8 +492,8 @@
         if ret != 0:
             raiseWindowsError(space, ret, 'CreateKeyEx')
         return space.wrap(W_HKEY(rethkey[0]))
-CreateKeyEx.unwrap_spec = [ObjSpace, W_Root, str, int, rffi.r_uint]
 
+ at unwrap_spec(subkey=str)
 def DeleteKey(space, w_hkey, subkey):
     """DeleteKey(key, sub_key) - Deletes the specified key.
 
@@ -521,8 +509,8 @@
     ret = rwinreg.RegDeleteKey(hkey, subkey)
     if ret != 0:
         raiseWindowsError(space, ret, 'RegDeleteKey')
-DeleteKey.unwrap_spec = [ObjSpace, W_Root, str]
 
+ at unwrap_spec(subkey=str)
 def DeleteValue(space, w_hkey, subkey):
     """DeleteValue(key, value) - Removes a named value from a registry key.
 
@@ -532,8 +520,8 @@
     ret = rwinreg.RegDeleteValue(hkey, subkey)
     if ret != 0:
         raiseWindowsError(space, ret, 'RegDeleteValue')
-DeleteValue.unwrap_spec = [ObjSpace, W_Root, str]
 
+ at unwrap_spec(subkey=str, res=int, sam=rffi.r_uint)
 def OpenKey(space, w_hkey, subkey, res=0, sam=rwinreg.KEY_READ):
     """key = OpenKey(key, sub_key, res = 0, sam = KEY_READ) - Opens the specified key.
 
@@ -551,8 +539,8 @@
         if ret != 0:
             raiseWindowsError(space, ret, 'RegOpenKeyEx')
         return space.wrap(W_HKEY(rethkey[0]))
-OpenKey.unwrap_spec = [ObjSpace, W_Root, str, int, rffi.r_uint]
 
+ at unwrap_spec(index=int)
 def EnumValue(space, w_hkey, index):
     """tuple = EnumValue(key, index) - Enumerates values of an open registry key.
 key is an already open key, or any one of the predefined HKEY_* constants.
@@ -615,8 +603,7 @@
                                 space.wrap(retType[0]),
                                 ])
 
-EnumValue.unwrap_spec = [ObjSpace, W_Root, int]
-
+ at unwrap_spec(index=int)
 def EnumKey(space, w_hkey, index):
     """string = EnumKey(key, index) - Enumerates subkeys of an open registry key.
 
@@ -645,8 +632,6 @@
                 raiseWindowsError(space, ret, 'RegEnumKeyEx')
             return space.wrap(rffi.charp2str(buf))
 
-EnumKey.unwrap_spec = [ObjSpace, W_Root, int]
-
 def QueryInfoKey(space, w_hkey):
     """tuple = QueryInfoKey(key) - Returns information about a key.
 
@@ -674,7 +659,6 @@
                 return space.newtuple([space.wrap(nSubKeys[0]),
                                        space.wrap(nValues[0]),
                                        space.wrap(l)])
-QueryInfoKey.unwrap_spec = [ObjSpace, W_Root]
 
 def str_or_None_w(space, w_obj):
     if space.is_w(w_obj, space.w_None):
@@ -699,15 +683,14 @@
         if ret != 0:
             raiseWindowsError(space, ret, 'RegConnectRegistry')
         return space.wrap(W_HKEY(rethkey[0]))
-ConnectRegistry.unwrap_spec = [ObjSpace, W_Root, W_Root]
 
+ at unwrap_spec(source=unicode)
 def ExpandEnvironmentStrings(space, source):
     "string = ExpandEnvironmentStrings(string) - Expand environment vars."
     try:
         return space.wrap(rwinreg.ExpandEnvironmentStrings(source))
     except WindowsError, e:
         raise wrap_windowserror(space, e)
-ExpandEnvironmentStrings.unwrap_spec = [ObjSpace, unicode]
 
 def DisableReflectionKey(space, w_key):
     """Disables registry reflection for 32-bit processes running on a 64-bit
@@ -717,7 +700,6 @@
     Disabling reflection for a key does not affect reflection of any subkeys."""
     raise OperationError(space.w_NotImplementedError, space.wrap(
         "not implemented on this platform"))
-DisableReflectionKey.unwrap_spec = [ObjSpace, W_Root]
 
 def EnableReflectionKey(space, w_key):
     """Restores registry reflection for the specified disabled key.
@@ -725,15 +707,14 @@
     Restoring reflection for a key does not affect reflection of any subkeys."""
     raise OperationError(space.w_NotImplementedError, space.wrap(
         "not implemented on this platform"))
-EnableReflectionKey.unwrap_spec = [ObjSpace, W_Root]
 
 def QueryReflectionKey(space, w_key):
     """bool = QueryReflectionKey(hkey) - Determines the reflection state for the specified key.
     Will generally raise NotImplemented if executed on a 32-bit Operating System."""
     raise OperationError(space.w_NotImplementedError, space.wrap(
         "not implemented on this platform"))
-QueryReflectionKey.unwrap_spec = [ObjSpace, W_Root]
 
+ at unwrap_spec(subkey=str)
 def DeleteKeyEx(space, w_key, subkey):
     """DeleteKeyEx(key, sub_key, sam, res) - Deletes the specified key.
 
@@ -750,4 +731,3 @@
     On unsupported Windows versions, NotImplementedError is raised."""
     raise OperationError(space.w_NotImplementedError, space.wrap(
         "not implemented on this platform"))
-DeleteKeyEx.unwrap_spec = [ObjSpace, W_Root, str]

From commits-noreply at bitbucket.org  Wed Feb 16 19:19:44 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 19:19:44 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: Fix a typo in a module which
	has no direct	test
Message-ID: <20110216181944.5D452282C1B@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42068:cd4477731b0a
Date: 2011-02-16 15:51 +0100
http://bitbucket.org/pypy/pypy/changeset/cd4477731b0a/

Log:	Fix a typo in a module which has no direct test

diff --git a/pypy/module/_pickle_support/maker.py b/pypy/module/_pickle_support/maker.py
--- a/pypy/module/_pickle_support/maker.py
+++ b/pypy/module/_pickle_support/maker.py
@@ -60,7 +60,7 @@
     tb = instantiate(PyTraceback)
     return space.wrap(tb)
 
- at unwrap_spec(runnint=int)
+ at unwrap_spec(running=int)
 def generator_new(space, w_frame, running):
     frame = space.interp_w(PyFrame, w_frame, can_be_None=True)
     new_generator = GeneratorIterator(frame)

From commits-noreply at bitbucket.org  Wed Feb 16 19:19:45 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 19:19:45 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: Modern,
	shorter unwrap_spec in _weakref module
Message-ID: <20110216181945.423B2282BF2@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42069:713176c770ff
Date: 2011-02-16 16:27 +0100
http://bitbucket.org/pypy/pypy/changeset/713176c770ff/

Log:	Modern, shorter unwrap_spec in _weakref module

diff --git a/pypy/module/_weakref/interp__weakref.py b/pypy/module/_weakref/interp__weakref.py
--- a/pypy/module/_weakref/interp__weakref.py
+++ b/pypy/module/_weakref/interp__weakref.py
@@ -120,12 +120,30 @@
             except OperationError, e:
                 e.write_unraisable(w_self.space, 'weakref callback ', w_self.w_callable)
 
+    def descr__repr__(self, space):
+        w_obj = self.dereference()
+        if w_obj is None:
+            state = '; dead'
+        else:
+            typename = space.type(w_obj).getname(space, '?')
+            objname = w_obj.getname(space, '')
+            if objname:
+                state = "; to '%s' (%s)" % (typename, objname)
+            else:
+                state = "; to '%s'" % (typename,)
+        return self.getrepr(space, self.typedef.name, state)
 
 class W_Weakref(W_WeakrefBase):
     def __init__(w_self, space, w_obj, w_callable):
         W_WeakrefBase.__init__(w_self, space, w_obj, w_callable)
         w_self.w_hash = None
 
+    def descr__init__weakref(self, space, w_obj, w_callable=None,
+                             __args__=None):
+        if __args__.arguments_w:
+            raise OperationError(space.w_TypeError, space.wrap(
+                "__init__ expected at most 2 arguments"))
+
     def descr_hash(self):
         if self.w_hash is not None:
             return self.w_hash
@@ -142,6 +160,21 @@
             return self.space.w_None
         return w_obj
 
+    def descr__eq__(self, space, w_ref2):
+        if not isinstance(w_ref2, W_Weakref):
+            return space.w_NotImplemented
+        ref1 = self
+        ref2 = w_ref2
+        w_obj1 = ref1.dereference()
+        w_obj2 = ref2.dereference()
+        if (w_obj1 is None or
+            w_obj2 is None):
+            return space.is_(ref1, ref2)
+        return space.eq(w_obj1, w_obj2)
+
+    def descr__ne__(self, space, w_ref2):
+        return space.not_(space.eq(self, w_ref2))
+
 def descr__new__weakref(space, w_subtype, w_obj, w_callable=None,
                         __args__=None):
     if __args__.arguments_w:
@@ -153,57 +186,16 @@
         w_obj.setweakref(space, lifeline)
     return lifeline.get_or_make_weakref(space, w_subtype, w_obj, w_callable)
 
-def descr__init__weakref(space, self, w_obj, w_callable=None,
-                         __args__=None):
-    if __args__.arguments_w:
-        raise OperationError(space.w_TypeError, space.wrap(
-            "__init__ expected at most 2 arguments"))
-
-def descr__eq__(space, ref1, w_ref2):
-    if not isinstance(w_ref2, W_Weakref):
-        return space.w_NotImplemented
-    ref2 = w_ref2
-    w_obj1 = ref1.dereference()
-    w_obj2 = ref2.dereference()
-    if (w_obj1 is None or
-        w_obj2 is None):
-        return space.is_(ref1, ref2)
-    return space.eq(w_obj1, w_obj2)
-
-def descr__ne__(space, ref1, w_ref2):
-    return space.not_(space.eq(ref1, w_ref2))
-
-def descr__repr__(space, ref):
-    w_obj = ref.dereference()
-    if w_obj is None:
-        state = '; dead'
-    else:
-        typename = space.type(w_obj).getname(space, '?')
-        objname = w_obj.getname(space, '')
-        if objname:
-            state = "; to '%s' (%s)" % (typename, objname)
-        else:
-            state = "; to '%s'" % (typename,)
-    return ref.getrepr(space, ref.typedef.name, state)
-
-reprdescr = interp2app(descr__repr__, unwrap_spec=[ObjSpace, W_WeakrefBase])
-
 W_Weakref.typedef = TypeDef("weakref",
     __doc__ = """A weak reference to an object 'obj'.  A 'callback' can be given,
 which is called with 'obj' as an argument when it is about to be finalized.""",
-    __new__ = interp2app(descr__new__weakref,
-                         unwrap_spec=[ObjSpace, W_Root, W_Root, W_Root,
-                                      Arguments]),
-    __init__ = interp2app(descr__init__weakref,
-                         unwrap_spec=[ObjSpace, W_Weakref, W_Root, W_Root,
-                                      Arguments]),
-    __eq__ = interp2app(descr__eq__,
-                        unwrap_spec=[ObjSpace, W_Weakref, W_Root]),
-    __ne__ = interp2app(descr__ne__,
-                        unwrap_spec=[ObjSpace, W_Weakref, W_Root]),
-    __hash__ = interp2app(W_Weakref.descr_hash, unwrap_spec=['self']),
-    __repr__ = reprdescr,
-    __call__ = interp2app(W_Weakref.descr_call, unwrap_spec=['self'])
+    __new__ = interp2app(descr__new__weakref),
+    __init__ = interp2app(W_Weakref.descr__init__weakref),
+    __eq__ = interp2app(W_Weakref.descr__eq__),
+    __ne__ = interp2app(W_Weakref.descr__ne__),
+    __hash__ = interp2app(W_Weakref.descr_hash),
+    __call__ = interp2app(W_Weakref.descr_call),
+    __repr__ = interp2app(W_WeakrefBase.descr__repr__),
 )
 
 
@@ -292,33 +284,28 @@
 
     func.func_name = opname
     for special_method in special_methods:
-        proxy_typedef_dict[special_method] = interp2app(
-            func, unwrap_spec=[ObjSpace] + [W_Root] * arity)
-        callable_proxy_typedef_dict[special_method] = interp2app(
-            func, unwrap_spec=[ObjSpace] + [W_Root] * arity)
+        proxy_typedef_dict[special_method] = interp2app(func)
+        callable_proxy_typedef_dict[special_method] = interp2app(func)
 
 # __unicode__ is not yet a space operation
 def proxy_unicode(space, w_obj):
     w_obj = force(space, w_obj)
     return space.call_method(w_obj, '__unicode__')
-proxy_typedef_dict['__unicode__'] = interp2app(
-    proxy_unicode, unwrap_spec=[ObjSpace, W_Root])
-callable_proxy_typedef_dict['__unicode__'] = interp2app(
-    proxy_unicode, unwrap_spec=[ObjSpace, W_Root])
+proxy_typedef_dict['__unicode__'] = interp2app(proxy_unicode)
+callable_proxy_typedef_dict['__unicode__'] = interp2app(proxy_unicode)
 
 
 W_Proxy.typedef = TypeDef("weakproxy",
     __new__ = interp2app(descr__new__proxy),
-    __hash__ = interp2app(W_Proxy.descr__hash__, unwrap_spec=['self', ObjSpace]),
-    __repr__ = reprdescr,
+    __hash__ = interp2app(W_Proxy.descr__hash__),
+    __repr__ = interp2app(W_WeakrefBase.descr__repr__),
     **proxy_typedef_dict)
 W_Proxy.typedef.acceptable_as_base_class = False
 
 W_CallableProxy.typedef = TypeDef("weakcallableproxy",
     __new__ = interp2app(descr__new__callableproxy),
-    __hash__ = interp2app(W_Proxy.descr__hash__, unwrap_spec=['self', ObjSpace]),
-    __repr__ = reprdescr,
-    __call__ = interp2app(W_CallableProxy.descr__call__,
-                          unwrap_spec=['self', ObjSpace, Arguments]), 
+    __hash__ = interp2app(W_Proxy.descr__hash__),
+    __repr__ = interp2app(W_WeakrefBase.descr__repr__),
+    __call__ = interp2app(W_CallableProxy.descr__call__),
     **callable_proxy_typedef_dict)
 W_CallableProxy.typedef.acceptable_as_base_class = False

From commits-noreply at bitbucket.org  Wed Feb 16 19:19:45 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 19:19:45 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: Merge default
Message-ID: <20110216181945.90597282C1A@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42070:88ce1a0b3c4f
Date: 2011-02-16 16:27 +0100
http://bitbucket.org/pypy/pypy/changeset/88ce1a0b3c4f/

Log:	Merge default


From commits-noreply at bitbucket.org  Wed Feb 16 19:19:46 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 19:19:46 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: simpler unwrap_specs in array
	module
Message-ID: <20110216181946.673F6282BF2@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42071:3876f4ed51f3
Date: 2011-02-16 16:35 +0100
http://bitbucket.org/pypy/pypy/changeset/3876f4ed51f3/

Log:	simpler unwrap_specs in array module

diff --git a/pypy/module/array/interp_array.py b/pypy/module/array/interp_array.py
--- a/pypy/module/array/interp_array.py
+++ b/pypy/module/array/interp_array.py
@@ -3,25 +3,20 @@
 from pypy.interpreter.error import OperationError
 from pypy.interpreter.typedef import TypeDef, GetSetProperty, make_weakref_descr
 from pypy.rpython.lltypesystem import lltype, rffi
-from pypy.interpreter.gateway import interp2app, ObjSpace, W_Root, \
-     ApplevelClass
-from pypy.rlib.jit import dont_look_inside
-from pypy.rlib import rgc
+from pypy.interpreter.gateway import interp2app, unwrap_spec
 from pypy.rlib.unroll import unrolling_iterable
 from pypy.rlib.rarithmetic import ovfcheck
-from pypy.rlib.rstruct.runpack import runpack
-from pypy.interpreter.argument import Arguments, Signature
-from pypy.interpreter.baseobjspace import ObjSpace, W_Root, Wrappable
+from pypy.interpreter.baseobjspace import Wrappable
 from pypy.objspace.std.stdtypedef import SMM, StdTypeDef
 from pypy.objspace.std.register_all import register_all
 from pypy.objspace.std.model import W_Object
-from pypy.interpreter.argument import Arguments, Signature
 from pypy.module._file.interp_file import W_File
 from pypy.interpreter.buffer import RWBuffer
 from pypy.objspace.std.multimethod import FailedToImplement
 
-def w_array(space, w_cls, typecode, w_args=None):
-    if len(w_args.arguments_w) > 1:
+ at unwrap_spec(typecode=str)
+def w_array(space, w_cls, typecode, __args__):
+    if len(__args__.arguments_w) > 1:
         msg = 'array() takes at most 2 arguments'
         raise OperationError(space.w_TypeError, space.wrap(msg))
     if len(typecode) != 1:
@@ -30,7 +25,7 @@
     typecode = typecode[0]
 
     if space.is_w(w_cls, space.gettypeobject(W_ArrayBase.typedef)):
-        if w_args.keywords: # XXX this might be forbidden fishing
+        if __args__.keywords:
             msg = 'array.array() does not take keyword arguments'
             raise OperationError(space.w_TypeError, space.wrap(msg))
 
@@ -39,8 +34,8 @@
             a = space.allocate_instance(types[tc].w_class, w_cls)
             a.__init__(space)
 
-            if len(w_args.arguments_w) > 0:
-                w_initializer = w_args.arguments_w[0]
+            if len(__args__.arguments_w) > 0:
+                w_initializer = __args__.arguments_w[0]
                 if space.type(w_initializer) is space.w_str:
                     a.fromstring(w_initializer)
                 elif space.type(w_initializer) is space.w_unicode:
@@ -55,7 +50,6 @@
         raise OperationError(space.w_ValueError, space.wrap(msg))
 
     return a
-w_array.unwrap_spec = (ObjSpace, W_Root, str, Arguments)
 
 
 array_append = SMM('append', 2)

From commits-noreply at bitbucket.org  Wed Feb 16 19:19:49 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 19:19:49 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: Modern unwrap_spec in binascii
Message-ID: <20110216181949.7B65A282C20@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42072:85b507b48d1a
Date: 2011-02-16 16:42 +0100
http://bitbucket.org/pypy/pypy/changeset/85b507b48d1a/

Log:	Modern unwrap_spec in binascii

diff --git a/pypy/module/binascii/interp_qp.py b/pypy/module/binascii/interp_qp.py
--- a/pypy/module/binascii/interp_qp.py
+++ b/pypy/module/binascii/interp_qp.py
@@ -1,4 +1,4 @@
-from pypy.interpreter.gateway import ObjSpace
+from pypy.interpreter.gateway import unwrap_spec
 from pypy.rlib.rstring import StringBuilder
 
 MAXLINESIZE = 76
@@ -14,6 +14,7 @@
         return ord(c) - (ord('a') - 10)
 hexval._always_inline_ = True
 
+ at unwrap_spec(data='bufferstr', header=int)
 def a2b_qp(space, data, header=0):
     "Decode a string of qp-encoded data."
 
@@ -56,7 +57,6 @@
                 c = ' '
             odata.append(c)
     return space.wrap(odata.build())
-a2b_qp.unwrap_spec = [ObjSpace, 'bufferstr', int]
 
 # ____________________________________________________________
 
@@ -92,6 +92,7 @@
         self._flush()
         return self.builder.build()
 
+ at unwrap_spec(data='bufferstr', quotetabs=int, istext=int, header=int)
 def b2a_qp(space, data, quotetabs=0, istext=1, header=0):
     """Encode a string using quoted-printable encoding.
 
@@ -159,4 +160,3 @@
                 inp += 1
 
     return space.wrap(odata.build())
-b2a_qp.unwrap_spec = [ObjSpace, 'bufferstr', int, int, int]

diff --git a/pypy/module/binascii/interp_uu.py b/pypy/module/binascii/interp_uu.py
--- a/pypy/module/binascii/interp_uu.py
+++ b/pypy/module/binascii/interp_uu.py
@@ -1,4 +1,4 @@
-from pypy.interpreter.gateway import ObjSpace
+from pypy.interpreter.gateway import unwrap_spec
 from pypy.rlib.rstring import StringBuilder
 from pypy.module.binascii.interp_binascii import raise_Error
 
@@ -29,6 +29,7 @@
 _a2b_write._always_inline_ = True
 
 
+ at unwrap_spec(ascii='bufferstr')
 def a2b_uu(space, ascii):
     "Decode a line of uuencoded data."
 
@@ -52,7 +53,6 @@
     if remaining > 0:
         res.append_multiple_char('\x00', remaining)
     return space.wrap(res.build())
-a2b_uu.unwrap_spec = [ObjSpace, 'bufferstr']
 
 # ____________________________________________________________
 
@@ -63,6 +63,7 @@
         return 0
 _a2b_read._always_inline_ = True
 
+ at unwrap_spec(bin='bufferstr')
 def b2a_uu(space, bin):
     "Uuencode a line of data."
 
@@ -84,4 +85,3 @@
 
     res.append('\n')
     return space.wrap(res.build())
-b2a_uu.unwrap_spec = [ObjSpace, 'bufferstr']

diff --git a/pypy/module/binascii/interp_base64.py b/pypy/module/binascii/interp_base64.py
--- a/pypy/module/binascii/interp_base64.py
+++ b/pypy/module/binascii/interp_base64.py
@@ -1,5 +1,5 @@
 from pypy.interpreter.error import OperationError
-from pypy.interpreter.gateway import ObjSpace
+from pypy.interpreter.gateway import unwrap_spec
 from pypy.rlib.rstring import StringBuilder
 from pypy.module.binascii.interp_binascii import raise_Error
 from pypy.rlib.rarithmetic import ovfcheck
@@ -35,6 +35,7 @@
 assert len(table_a2b_base64) == 256
 
 
+ at unwrap_spec(ascii='bufferstr')
 def a2b_base64(space, ascii):
     "Decode a line of base64 data."
 
@@ -71,13 +72,13 @@
             raise_Error(space, "Incorrect padding")
 
     return space.wrap(res.build())
-a2b_base64.unwrap_spec = [ObjSpace, 'bufferstr']
 
 # ____________________________________________________________
 
 table_b2a_base64 = (
     "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/")
 
+ at unwrap_spec(bin='bufferstr')
 def b2a_base64(space, bin):
     "Base64-code line of data."
 
@@ -110,4 +111,3 @@
         res.append(PAD)
     res.append('\n')
     return space.wrap(res.build())
-b2a_base64.unwrap_spec = [ObjSpace, 'bufferstr']

diff --git a/pypy/module/binascii/interp_crc32.py b/pypy/module/binascii/interp_crc32.py
--- a/pypy/module/binascii/interp_crc32.py
+++ b/pypy/module/binascii/interp_crc32.py
@@ -1,4 +1,4 @@
-from pypy.interpreter.gateway import ObjSpace
+from pypy.interpreter.gateway import unwrap_spec
 from pypy.rlib.rarithmetic import r_uint, intmask
 from pypy.rpython.lltypesystem import rffi
 
@@ -61,6 +61,7 @@
 crc_32_tab = map(r_uint, crc_32_tab)
 
 
+ at unwrap_spec(data='bufferstr', oldcrc='c_int')
 def crc32(space, data, oldcrc=0):
     "Compute the CRC-32 incrementally."
 
@@ -72,4 +73,3 @@
 
     crc = ~intmask(rffi.cast(rffi.INT, crc))   # unsigned => 32-bit signed
     return space.wrap(crc)
-crc32.unwrap_spec = [ObjSpace, 'bufferstr', 'c_int']

diff --git a/pypy/module/binascii/interp_hexlify.py b/pypy/module/binascii/interp_hexlify.py
--- a/pypy/module/binascii/interp_hexlify.py
+++ b/pypy/module/binascii/interp_hexlify.py
@@ -1,5 +1,5 @@
 from pypy.interpreter.error import OperationError
-from pypy.interpreter.gateway import ObjSpace
+from pypy.interpreter.gateway import unwrap_spec
 from pypy.rlib.rstring import StringBuilder
 from pypy.rlib.rarithmetic import ovfcheck
 
@@ -12,6 +12,7 @@
         return chr((ord('a')-10) + value)
 _value2char._always_inline_ = True
 
+ at unwrap_spec(data='bufferstr')
 def hexlify(space, data):
     '''Hexadecimal representation of binary data.
 This function is also available as "hexlify()".'''
@@ -24,7 +25,6 @@
         res.append(_value2char(ord(c) >> 4))
         res.append(_value2char(ord(c) & 0xf))
     return space.wrap(res.build())
-hexlify.unwrap_spec = [ObjSpace, 'bufferstr']
 
 # ____________________________________________________________
 
@@ -42,6 +42,7 @@
                          space.wrap('Non-hexadecimal digit found'))
 _char2value._always_inline_ = True
 
+ at unwrap_spec(hexstr='bufferstr')
 def unhexlify(space, hexstr):
     '''Binary data of hexadecimal representation.
 hexstr must contain an even number of hex digits (upper or lower case).
@@ -55,4 +56,3 @@
         b = _char2value(space, hexstr[i+1])
         res.append(chr((a << 4) | b))
     return space.wrap(res.build())
-unhexlify.unwrap_spec = [ObjSpace, 'bufferstr']

diff --git a/pypy/module/binascii/interp_hqx.py b/pypy/module/binascii/interp_hqx.py
--- a/pypy/module/binascii/interp_hqx.py
+++ b/pypy/module/binascii/interp_hqx.py
@@ -1,5 +1,5 @@
 from pypy.interpreter.error import OperationError
-from pypy.interpreter.gateway import ObjSpace
+from pypy.interpreter.gateway import unwrap_spec
 from pypy.rlib.rstring import StringBuilder
 from pypy.module.binascii.interp_binascii import raise_Error, raise_Incomplete
 from pypy.rlib.rarithmetic import ovfcheck
@@ -62,6 +62,7 @@
 ]
 table_a2b_hqx = ''.join(map(chr, table_a2b_hqx))
 
+ at unwrap_spec(ascii='bufferstr')
 def a2b_hqx(space, ascii):
     """Decode .hqx coding.  Returns (bin, done)."""
 
@@ -97,13 +98,13 @@
         if pending_bits > 0:
             raise_Incomplete(space, 'String has incomplete number of bytes')
     return space.newtuple([space.wrap(res.build()), space.wrap(done)])
-a2b_hqx.unwrap_spec = [ObjSpace, 'bufferstr']
 
 # ____________________________________________________________
 
 hqx_encoding = (
     '!"#$%&\'()*+,-012345689 at ABCDEFGHIJKLMNPQRSTUVXYZ[`abcdefhijklmpqr')
 
+ at unwrap_spec(bin='bufferstr')
 def b2a_hqx(space, bin):
     "Encode .hqx data."
     extra = (len(bin) + 2) // 3
@@ -128,10 +129,10 @@
         leftchar <<= (6 - leftbits)
         res.append(hqx_encoding[leftchar & 0x3f])
     return space.wrap(res.build())
-b2a_hqx.unwrap_spec = [ObjSpace, 'bufferstr']
 
 # ____________________________________________________________
 
+ at unwrap_spec(hexbin='bufferstr')
 def rledecode_hqx(space, hexbin):
     "Decode hexbin RLE-coded string."
 
@@ -160,10 +161,10 @@
                     raise_Error(space, 'String starts with the RLE code \x90')
                 res.append_multiple_char(chr(lastpushed), count)
     return space.wrap(res.build())
-rledecode_hqx.unwrap_spec = [ObjSpace, 'bufferstr']
 
 # ____________________________________________________________
 
+ at unwrap_spec(data='bufferstr')
 def rlecode_hqx(space, data):
     "Binhex RLE-code binary data."
 
@@ -197,7 +198,6 @@
     # some programs somewhere that would start failing obscurely in rare
     # cases.
     return space.wrap(res.build())
-rlecode_hqx.unwrap_spec = [ObjSpace, 'bufferstr']
 
 # ____________________________________________________________
 
@@ -236,10 +236,10 @@
         0x6e17, 0x7e36, 0x4e55, 0x5e74, 0x2e93, 0x3eb2, 0x0ed1, 0x1ef0,
 ]
 
+ at unwrap_spec(data='bufferstr', oldcrc=int)
 def crc_hqx(space, data, oldcrc):
     "Compute hqx CRC incrementally."
     crc = oldcrc
     for c in data:
         crc = ((crc << 8) & 0xff00) ^ crctab_hqx[((crc >> 8) & 0xff) ^ ord(c)]
     return space.wrap(crc)
-crc_hqx.unwrap_spec = [ObjSpace, 'bufferstr', int]

From commits-noreply at bitbucket.org  Wed Feb 16 19:19:53 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 19:19:53 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: modern unwrap_spec in
	cStringIO, clr, cmath
Message-ID: <20110216181953.B5745282C1E@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42073:58cf02bab704
Date: 2011-02-16 16:50 +0100
http://bitbucket.org/pypy/pypy/changeset/58cf02bab704/

Log:	modern unwrap_spec in cStringIO, clr, cmath

diff --git a/pypy/module/cStringIO/interp_stringio.py b/pypy/module/cStringIO/interp_stringio.py
--- a/pypy/module/cStringIO/interp_stringio.py
+++ b/pypy/module/cStringIO/interp_stringio.py
@@ -2,7 +2,7 @@
 from pypy.interpreter.error import OperationError
 from pypy.interpreter.baseobjspace import Wrappable
 from pypy.interpreter.typedef import TypeDef, GetSetProperty
-from pypy.interpreter.gateway import interp2app, W_Root, ObjSpace
+from pypy.interpreter.gateway import interp2app, unwrap_spec
 from pypy.rlib.rStringIO import RStringIO
 
 
@@ -12,11 +12,9 @@
     def descr___iter__(self):
         self.check_closed()
         return self
-    descr___iter__.unwrap_spec = ['self']
 
     def descr_close(self):
         self.close()
-    descr_close.unwrap_spec = ['self']
 
     def check_closed(self):
         if self.is_closed():
@@ -26,17 +24,14 @@
 
     def descr_flush(self):
         self.check_closed()
-    descr_flush.unwrap_spec = ['self']
 
     def descr_getvalue(self):
         self.check_closed()
         return self.space.wrap(self.getvalue())
-    descr_getvalue.unwrap_spec = ['self']
 
     def descr_isatty(self):
         self.check_closed()
         return self.space.w_False
-    descr_isatty.unwrap_spec = ['self']
 
     def descr_next(self):
         space = self.space
@@ -46,16 +41,17 @@
             raise OperationError(space.w_StopIteration, space.w_None)
         return space.wrap(line)
 
+    @unwrap_spec(n=int)
     def descr_read(self, n=-1):
         self.check_closed()
         return self.space.wrap(self.read(n))
-    descr_read.unwrap_spec = ['self', int]
 
+    @unwrap_spec(size=int)
     def descr_readline(self, size=-1):
         self.check_closed()
         return self.space.wrap(self.readline(size))
-    descr_readline.unwrap_spec = ['self', int]
 
+    @unwrap_spec(size=int)
     def descr_readlines(self, size=0):
         self.check_closed()
         lines_w = []
@@ -69,22 +65,19 @@
                 if size <= 0:
                     break
         return self.space.newlist(lines_w)
-    descr_readlines.unwrap_spec = ['self', int]
 
     def descr_reset(self):
         self.check_closed()
         self.seek(0)
-    descr_reset.unwrap_spec = ['self']
 
+    @unwrap_spec(position=int, mode=int)
     def descr_seek(self, position, mode=0):
         self.check_closed()
         self.seek(position, mode)
-    descr_seek.unwrap_spec = ['self', int, int]
 
     def descr_tell(self):
         self.check_closed()
         return self.space.wrap(self.tell())
-    descr_tell.unwrap_spec = ['self']
 
     # abstract methods
     def close(self):                  assert False, "abstract"
@@ -183,12 +176,11 @@
         if size < 0:
             raise OperationError(space.w_IOError, space.wrap("negative size"))
         self.truncate(size)
-    descr_truncate.unwrap_spec = ['self', W_Root]
 
+    @unwrap_spec(buffer='bufferstr')
     def descr_write(self, buffer):
         self.check_closed()
         self.write(buffer)
-    descr_write.unwrap_spec = ['self', 'bufferstr']
 
     def descr_writelines(self, w_lines):
         self.check_closed()
@@ -202,7 +194,6 @@
                     raise
                 break  # done
             self.write(space.str_w(w_line))
-    descr_writelines.unwrap_spec = ['self', W_Root]
 
 # ____________________________________________________________
 
@@ -262,4 +253,3 @@
     else:
         string = space.bufferstr_w(w_string)
         return space.wrap(W_InputType(space, string))
-StringIO.unwrap_spec = [ObjSpace, W_Root]

diff --git a/pypy/module/clr/interp_clr.py b/pypy/module/clr/interp_clr.py
--- a/pypy/module/clr/interp_clr.py
+++ b/pypy/module/clr/interp_clr.py
@@ -1,8 +1,8 @@
 import os.path
 from pypy.module.clr import assemblyname
-from pypy.interpreter.baseobjspace import ObjSpace, W_Root, Wrappable
+from pypy.interpreter.baseobjspace import W_Root, Wrappable
 from pypy.interpreter.error import OperationError, operationerrfmt
-from pypy.interpreter.gateway import interp2app, ApplevelClass
+from pypy.interpreter.gateway import interp2app, unwrap_spec, ApplevelClass
 from pypy.interpreter.typedef import TypeDef
 from pypy.rpython.ootypesystem import ootype
 from pypy.translator.cli.dotnet import CLR, box, unbox, NativeException, native_exc,\
@@ -67,6 +67,7 @@
     else:
         return cli2py(space, b_res)
 
+ at unwrap_spec(typename=str, methname=str)
 def call_staticmethod(space, typename, methname, w_args):
     """
     Call a .NET static method.
@@ -83,7 +84,6 @@
     """
     b_type = System.Type.GetType(typename) # XXX: cache this!
     return call_method(space, None, b_type, methname, w_args, 0)
-call_staticmethod.unwrap_spec = [ObjSpace, str, str, W_Root]
 
 def py2cli(space, w_obj):
     try:
@@ -263,20 +263,20 @@
         info.w_info = space.newtuple([info.w_namespaces, info.w_classes, info.w_generics])
         save_info_for_std_assemblies(space)
     return info.w_info
-get_assemblies_info.unwrap_spec = [ObjSpace]
 
 #_______________________________________________________________________________
 # AddReference* methods
 
 # AddReference', 'AddReferenceByName', 'AddReferenceByPartialName', 'AddReferenceToFile', 'AddReferenceToFileAndPath'
 
+ at unwrap_spec(name=str)
 def AddReferenceByPartialName(space, name):
     b_assembly = Assembly.LoadWithPartialName(name)
     if b_assembly is not None:
         save_info_for_assembly(space, b_assembly)
-AddReferenceByPartialName.unwrap_spec = [ObjSpace, str]
 
 
+ at unwrap_spec(assemblyname=str, namespace=str, classname=str)
 def load_cli_class(space, assemblyname, namespace, classname):
     """
     Load the given .NET class into the PyPy interpreter and return a
@@ -295,7 +295,6 @@
         w_cls = build_cli_class(space, namespace, classname, fullname, assemblyname)
         CliClassCache.put(fullname, w_cls)
     return w_cls
-load_cli_class.unwrap_spec = [ObjSpace, str, str, str]
 
 def build_cli_class(space, namespace, classname, fullname, assemblyname):
     assembly_qualified_name = '%s, %s' % (fullname, assemblyname)
@@ -333,10 +332,11 @@
         self.space = space
         self.b_obj = b_obj
 
+    @unwrap_spec(name=str, startfrom=int)
     def call_method(self, name, w_args, startfrom=0):
         return call_method(self.space, self.b_obj, self.b_obj.GetType(), name, w_args, startfrom)
-    call_method.unwrap_spec = ['self', str, W_Root, int]
 
+ at unwrap_spec(typename=str)
 def cli_object_new(space, w_subtype, typename, w_args):
     b_type = System.Type.GetType(typename)
     b_args, b_paramtypes = rewrap_args(space, w_args, 0)
@@ -349,7 +349,6 @@
         # TODO: use the appropriate exception, not StandardError
         raise OperationError(space.w_StandardError, space.wrap(message))
     return space.wrap(W_CliObject(space, b_obj))
-cli_object_new.unwrap_spec = [ObjSpace, W_Root, str, W_Root]
 
 W_CliObject.typedef = TypeDef(
     '_CliObject_internal',

diff --git a/pypy/module/cmath/interp_cmath.py b/pypy/module/cmath/interp_cmath.py
--- a/pypy/module/cmath/interp_cmath.py
+++ b/pypy/module/cmath/interp_cmath.py
@@ -4,7 +4,7 @@
 from pypy.rlib.rarithmetic import copysign, asinh, log1p, isinf, isnan
 from pypy.tool.sourcetools import func_with_new_name
 from pypy.interpreter.error import OperationError
-from pypy.interpreter.gateway import ObjSpace, W_Root, NoneNotWrapped
+from pypy.interpreter.gateway import NoneNotWrapped
 from pypy.module.cmath import Module, names_and_docstrings
 from pypy.module.cmath.constant import DBL_MIN, CM_SCALE_UP, CM_SCALE_DOWN
 from pypy.module.cmath.constant import CM_LARGE_DOUBLE, DBL_MANT_DIG
@@ -49,7 +49,6 @@
     #
     name = c_func.func_name
     assert name.startswith('c_')
-    wrapper.unwrap_spec = [ObjSpace, W_Root]
     wrapper.func_doc = names_and_docstrings[name[2:]]
     fnname = 'wrapped_' + name[2:]
     globals()[fnname] = func_with_new_name(wrapper, fnname)
@@ -296,7 +295,6 @@
         return space.truediv(w_logz, w_logbase)
     else:
         return w_logz
-wrapped_log.unwrap_spec = [ObjSpace, W_Root, W_Root]
 wrapped_log.func_doc = _inner_wrapped_log.func_doc
 
 
@@ -498,7 +496,6 @@
     y = space.float_w(w_y)
     resx, resy = call_c_func(c_rect, space, x, y)
     return space.newcomplex(resx, resy)
-wrapped_rect.unwrap_spec = [ObjSpace, W_Root, W_Root]
 wrapped_rect.func_doc = names_and_docstrings['rect']
 
 
@@ -530,7 +527,6 @@
     x, y = space.unpackcomplex(w_z)
     result = call_c_func(c_phase, space, x, y)
     return space.newfloat(result)
-wrapped_phase.unwrap_spec = [ObjSpace, W_Root]
 wrapped_phase.func_doc = names_and_docstrings['phase']
 
 
@@ -562,7 +558,6 @@
     x, y = space.unpackcomplex(w_z)
     resx, resy = call_c_func(c_polar, space, x, y)
     return space.newtuple([space.newfloat(resx), space.newfloat(resy)])
-wrapped_polar.unwrap_spec = [ObjSpace, W_Root]
 wrapped_polar.func_doc = names_and_docstrings['polar']
 
 
@@ -573,7 +568,6 @@
     x, y = space.unpackcomplex(w_z)
     res = c_isinf(x, y)
     return space.newbool(res)
-wrapped_isinf.unwrap_spec = [ObjSpace, W_Root]
 wrapped_isinf.func_doc = names_and_docstrings['isinf']
 
 
@@ -584,5 +578,4 @@
     x, y = space.unpackcomplex(w_z)
     res = c_isnan(x, y)
     return space.newbool(res)
-wrapped_isnan.unwrap_spec = [ObjSpace, W_Root]
 wrapped_isnan.func_doc = names_and_docstrings['isnan']

From commits-noreply at bitbucket.org  Wed Feb 16 19:19:55 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 19:19:55 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: Simpler unwrap_spec in
	exception, fcntl
Message-ID: <20110216181955.1EB2B282BF7@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42074:567dbdc42138
Date: 2011-02-16 16:55 +0100
http://bitbucket.org/pypy/pypy/changeset/567dbdc42138/

Log:	Simpler unwrap_spec in exception, fcntl

diff --git a/pypy/module/exceptions/interp_exceptions.py b/pypy/module/exceptions/interp_exceptions.py
--- a/pypy/module/exceptions/interp_exceptions.py
+++ b/pypy/module/exceptions/interp_exceptions.py
@@ -72,11 +72,11 @@
            +-- BytesWarning
 """
 
-from pypy.interpreter.baseobjspace import ObjSpace, Wrappable, W_Root
+from pypy.interpreter.baseobjspace import Wrappable
 from pypy.interpreter.typedef import TypeDef, interp_attrproperty_w,\
      GetSetProperty, interp_attrproperty, descr_get_dict, descr_set_dict,\
      descr_del_dict
-from pypy.interpreter.gateway import interp2app, Arguments
+from pypy.interpreter.gateway import interp2app
 from pypy.interpreter.error import OperationError
 from pypy.rlib import rwin32
 
@@ -106,7 +106,6 @@
             self.w_message = args_w[0]
         else:
             self.w_message = space.wrap("")
-    descr_init.unwrap_spec = ['self', ObjSpace, 'args_w']
 
     def descr_str(self, space):
         lgt = len(self.args_w)
@@ -116,7 +115,6 @@
             return space.str(self.args_w[0])
         else:
             return space.str(space.newtuple(self.args_w))
-    descr_str.unwrap_spec = ['self', ObjSpace]
 
     def descr_unicode(self, space):
         w_str = space.lookup(self, "__str__")
@@ -133,7 +131,6 @@
         else:
             w_tup = space.newtuple(self.args_w)
             return space.call_function(space.w_unicode, w_tup)
-    descr_unicode.unwrap_spec = ['self', ObjSpace]
 
     def descr_repr(self, space):
         if self.args_w:
@@ -142,7 +139,6 @@
             args_repr = "()"
         clsname = self.getclass(space).getname(space, '?')
         return space.wrap(clsname + args_repr)
-    descr_repr.unwrap_spec = ['self', ObjSpace]
 
     def descr_getargs(space, self):
         return space.newtuple(self.args_w)
@@ -152,7 +148,6 @@
 
     def descr_getitem(self, space, w_index):
         return space.getitem(space.newtuple(self.args_w), w_index)
-    descr_getitem.unwrap_spec = ['self', ObjSpace, W_Root]
 
     def getdict(self):
         if self.w_dict is None:
@@ -169,12 +164,10 @@
         if self.w_dict is not None and space.is_true(self.w_dict):
             lst = lst + [self.w_dict]
         return space.newtuple(lst)
-    descr_reduce.unwrap_spec = ['self', ObjSpace]
 
     def descr_setstate(self, space, w_dict):
         w_olddict = self.getdict()
         space.call_method(w_olddict, 'update', w_dict)
-    descr_setstate.unwrap_spec = ['self', ObjSpace, W_Root]
 
     def descr_message_get(space, self):
         w_dict = self.w_dict
@@ -209,7 +202,6 @@
         exc = space.allocate_instance(cls, w_subtype)
         basecls.__init__(exc, space)
         return space.wrap(exc)
-    descr_new_base_exception.unwrap_spec = [ObjSpace, W_Root, Arguments]
     descr_new_base_exception.func_name = 'descr_new_' + cls.__name__
     return interp2app(descr_new_base_exception)
 
@@ -305,8 +297,6 @@
         self.w_reason = w_reason
         W_BaseException.descr_init(self, space, [w_object, w_start,
                                                  w_end, w_reason])
-    descr_init.unwrap_spec = ['self', ObjSpace, W_Root, W_Root, W_Root,
-                              W_Root]
 
     def descr_str(self, space):
         return space.appexec([space.wrap(self)], r"""(self):
@@ -319,7 +309,6 @@
                 return "can't translate character u'\\U%08x' in position %d: %s"%(badchar, self.start, self.reason)
             return "can't translate characters in position %d-%d: %s" % (self.start, self.end - 1, self.reason)
         """)
-    descr_str.unwrap_spec = ['self', ObjSpace]
 
 W_UnicodeTranslateError.typedef = TypeDef(
     'UnicodeTranslateError',
@@ -345,7 +334,6 @@
         return space.repr(self.args_w[0])
     else:
         return space.str(space.newtuple(self.args_w))
-key_error_str.unwrap_spec = ['self', ObjSpace]
 
 W_KeyError = _new_exception('KeyError', W_LookupError,
                             """Mapping key not found.""",
@@ -378,7 +366,6 @@
         if len(args_w) == 3:
             self.w_filename = args_w[2]
             self.args_w = [args_w[0], args_w[1]]
-    descr_init.unwrap_spec = ['self', ObjSpace, 'args_w']
 
     # since we rebind args_w, we need special reduce, grump
     def descr_reduce(self, space):
@@ -390,7 +377,6 @@
         if self.w_dict is not None and space.is_true(self.w_dict):
             lst = lst + [self.w_dict]
         return space.newtuple(lst)
-    descr_reduce.unwrap_spec = ['self', ObjSpace]
 
     def descr_str(self, space):
         if (not space.is_w(self.w_errno, space.w_None) and
@@ -404,7 +390,6 @@
                               (space.str_w(space.str(self.w_errno)),
                                space.str_w(self.w_strerror)))
         return W_BaseException.descr_str(self, space)
-    descr_str.unwrap_spec = ['self', ObjSpace]
 
 W_EnvironmentError.typedef = TypeDef(
     'EnvironmentError',
@@ -442,7 +427,6 @@
             errno = self._winerror_to_errno.get(errno, self._default_errno)
         self.w_winerror = self.w_errno
         self.w_errno = space.wrap(errno)
-    descr_init.unwrap_spec = ['self', ObjSpace, 'args_w']
 
     def descr_str(self, space):
         if (not space.is_w(self.w_winerror, space.w_None) and
@@ -455,7 +439,6 @@
             return space.wrap("[Error %d] %s" % (space.int_w(self.w_winerror),
                                                  space.str_w(self.w_strerror)))
         return W_BaseException.descr_str(self, space)
-    descr_str.unwrap_spec = ['self', ObjSpace]
 
     if hasattr(rwin32, 'build_winerror_to_errno'):
         _winerror_to_errno, _default_errno = rwin32.build_winerror_to_errno()
@@ -524,7 +507,6 @@
                 args_w = args_w[:]
                 args_w[1] = space.newtuple(values_w[:4])
         W_BaseException.descr_init(self, space, args_w)
-    descr_init.unwrap_spec = ['self', ObjSpace, 'args_w']
 
     def descr_str(self, space):
         return space.appexec([self], """(self):
@@ -552,8 +534,6 @@
             return buffer
         """)
 
-    descr_str.unwrap_spec = ['self', ObjSpace]
-
     def descr_repr(self, space):
         if (len(self.args_w) == 2
             and not space.is_w(self.w_lastlineno, space.w_None)
@@ -568,7 +548,6 @@
             return space.wrap(clsname + args_repr)
         else:
             return W_StandardError.descr_repr(self, space)
-    descr_repr.unwrap_spec = ['self', ObjSpace]
 
 W_SyntaxError.typedef = TypeDef(
     'SyntaxError',
@@ -605,7 +584,6 @@
         elif len(args_w) > 1:
             self.w_code = space.newtuple(args_w)
         W_BaseException.descr_init(self, space, args_w)
-    descr_init.unwrap_spec = ['self', ObjSpace, 'args_w']
 
 W_SystemExit.typedef = TypeDef(
     'SystemExit',
@@ -661,8 +639,6 @@
         self.w_reason = w_reason
         W_BaseException.descr_init(self, space, [w_encoding, w_object,
                                                  w_start, w_end, w_reason])
-    descr_init.unwrap_spec = ['self', ObjSpace, W_Root, W_Root, W_Root, W_Root,
-                              W_Root]
 
     def descr_str(self, space):
         return space.appexec([self], """(self):
@@ -673,7 +649,6 @@
             return "'%s' codec can't decode bytes in position %d-%d: %s" % (
                 self.encoding, self.start, self.end - 1, self.reason)
         """)
-    descr_str.unwrap_spec = ['self', ObjSpace]
 
 W_UnicodeDecodeError.typedef = TypeDef(
     'UnicodeDecodeError',
@@ -753,8 +728,6 @@
         self.w_reason = w_reason
         W_BaseException.descr_init(self, space, [w_encoding, w_object,
                                                  w_start, w_end, w_reason])
-    descr_init.unwrap_spec = ['self', ObjSpace, W_Root, W_Root, W_Root, W_Root,
-                              W_Root]
 
     def descr_str(self, space):
         return space.appexec([self], r"""(self):
@@ -771,7 +744,6 @@
             return "'%s' codec can't encode characters in position %d-%d: %s" % (
                 self.encoding, self.start, self.end - 1, self.reason)
         """)
-    descr_str.unwrap_spec = ['self', ObjSpace]
 
 W_UnicodeEncodeError.typedef = TypeDef(
     'UnicodeEncodeError',

diff --git a/pypy/module/fcntl/interp_fcntl.py b/pypy/module/fcntl/interp_fcntl.py
--- a/pypy/module/fcntl/interp_fcntl.py
+++ b/pypy/module/fcntl/interp_fcntl.py
@@ -1,7 +1,7 @@
 from pypy.rpython.tool import rffi_platform as platform
 from pypy.rpython.lltypesystem import rffi, lltype
 from pypy.interpreter.error import OperationError, wrap_oserror
-from pypy.interpreter.baseobjspace import W_Root, ObjSpace
+from pypy.interpreter.gateway import unwrap_spec
 from pypy.rlib import rposix
 from pypy.translator.tool.cbuild import ExternalCompilationInfo
 import sys
@@ -100,6 +100,7 @@
     l.c_l_type = rffi.cast(rffi.SHORT, l_type)
     return l
 
+ at unwrap_spec(op=int)
 def fcntl(space, w_fd, op, w_arg=0):
     """fcntl(fd, op, [arg])
 
@@ -144,8 +145,8 @@
 
     raise OperationError(space.w_TypeError,
                          space.wrap("int or string or buffer required"))
-fcntl.unwrap_spec = [ObjSpace, W_Root, int, W_Root]
 
+ at unwrap_spec(op=int)
 def flock(space, w_fd, op):
     """flock(fd, operation)
 
@@ -168,8 +169,8 @@
         op = rffi.cast(rffi.INT, op)        # C long => C int
         fcntl_flock(fd, op, l)
         lltype.free(l, flavor='raw')
-flock.unwrap_spec = [ObjSpace, W_Root, int]
 
+ at unwrap_spec(op=int, length=int, start=int, whence=int)
 def lockf(space, w_fd, op, length=0, start=0, whence=0):
     """lockf (fd, operation, length=0, start=0, whence=0)
 
@@ -215,8 +216,8 @@
         fcntl_flock(fd, op, l)
     finally:
         lltype.free(l, flavor='raw')
-lockf.unwrap_spec = [ObjSpace, W_Root, int, int, int, int]
 
+ at unwrap_spec(op=int, mutate_flag=int)
 def ioctl(space, w_fd, op, w_arg=0, mutate_flag=-1):
     """ioctl(fd, opt[, arg[, mutate_flag]])
 
@@ -277,4 +278,3 @@
 
     raise OperationError(space.w_TypeError,
                          space.wrap("int or string or buffer required"))
-ioctl.unwrap_spec = [ObjSpace, W_Root, int, W_Root, int]

From commits-noreply at bitbucket.org  Wed Feb 16 19:19:56 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 19:19:56 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: modern unwrap_spec in imp
	module
Message-ID: <20110216181956.63D17282BF7@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42075:866835341f94
Date: 2011-02-16 17:00 +0100
http://bitbucket.org/pypy/pypy/changeset/866835341f94/

Log:	modern unwrap_spec in imp module

diff --git a/pypy/module/imp/interp_imp.py b/pypy/module/imp/interp_imp.py
--- a/pypy/module/imp/interp_imp.py
+++ b/pypy/module/imp/interp_imp.py
@@ -3,7 +3,7 @@
 from pypy.rlib import streamio
 from pypy.interpreter.error import OperationError, operationerrfmt
 from pypy.interpreter.module import Module
-from pypy.interpreter.gateway import NoneNotWrapped, W_Root, ObjSpace
+from pypy.interpreter.gateway import NoneNotWrapped, unwrap_spec
 from pypy.module._file.interp_stream import StreamErrors, wrap_streamerror
 import struct
 
@@ -105,6 +105,7 @@
         stream.close()
     return w_mod
 
+ at unwrap_spec(filename=str)
 def _run_compiled_module(space, w_modulename, filename, w_file, w_module):
     # the function 'imp._run_compiled_module' is a pypy-only extension
     stream = get_file(space, w_file, filename, 'rb')
@@ -117,14 +118,13 @@
         stream.readall())
     if space.is_w(w_file, space.w_None):
         stream.close()
-_run_compiled_module.unwrap_spec = [ObjSpace, W_Root, str, W_Root, W_Root]
 
+ at unwrap_spec(filename=str)
 def load_compiled(space, w_modulename, filename, w_file=None):
     w_mod = space.wrap(Module(space, w_modulename))
     importing._prepare_module(space, w_mod, filename, None)
     _run_compiled_module(space, w_modulename, filename, w_file, w_mod)
     return w_mod
-load_compiled.unwrap_spec = [ObjSpace, W_Root, str, W_Root]
 
 def new_module(space, w_name):
     return space.wrap(Module(space, w_name))

diff --git a/pypy/module/imp/importing.py b/pypy/module/imp/importing.py
--- a/pypy/module/imp/importing.py
+++ b/pypy/module/imp/importing.py
@@ -5,10 +5,10 @@
 import sys, os, stat
 
 from pypy.interpreter.module import Module
-from pypy.interpreter.gateway import Arguments, interp2app, unwrap_spec
+from pypy.interpreter.gateway import interp2app, unwrap_spec
 from pypy.interpreter.typedef import TypeDef, generic_new_descr
 from pypy.interpreter.error import OperationError, operationerrfmt
-from pypy.interpreter.baseobjspace import W_Root, ObjSpace, Wrappable
+from pypy.interpreter.baseobjspace import Wrappable
 from pypy.interpreter.eval import Code
 from pypy.rlib import streamio, jit, rposix
 from pypy.rlib.streamio import StreamErrors
@@ -108,6 +108,7 @@
 def check_sys_modules_w(space, modulename):
     return space.finditem_str(space.sys.get('modules'), modulename)
 
+ at unwrap_spec(name=str, level=int)
 def importhook(space, name, w_globals=None,
                w_locals=None, w_fromlist=None, level=-1):
     modulename = name
@@ -186,8 +187,6 @@
         space.setitem(space.sys.get('modules'), w(rel_modulename), space.w_None)
     space.timer.stop_name("importhook", modulename)
     return w_mod
-#
-importhook.unwrap_spec = [ObjSpace, str, W_Root, W_Root, W_Root, int]
 
 @jit.dont_look_inside
 def absolute_import(space, modulename, baselevel, fromlist_w, tentative):
@@ -330,7 +329,7 @@
     def __init__(self, space):
         pass
 
-    @unwrap_spec('self', ObjSpace, str)
+    @unwrap_spec(path=str)
     def descr_init(self, space, path):
         if not path:
             raise OperationError(space.w_ImportError, space.wrap(
@@ -346,7 +345,6 @@
                 raise OperationError(space.w_ImportError, space.wrap(
                     "existing directory"))
 
-    @unwrap_spec('self', ObjSpace, Arguments)
     def find_module_w(self, space, __args__):
         return space.wrap(None)
 

From commits-noreply at bitbucket.org  Wed Feb 16 19:19:57 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 19:19:57 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: Remove almost all unwrap_spec
	from	itertools module
Message-ID: <20110216181957.5AA79282BF7@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42076:0fff9c5ee796
Date: 2011-02-16 17:06 +0100
http://bitbucket.org/pypy/pypy/changeset/0fff9c5ee796/

Log:	Remove almost all unwrap_spec from itertools module

diff --git a/pypy/module/itertools/interp_itertools.py b/pypy/module/itertools/interp_itertools.py
--- a/pypy/module/itertools/interp_itertools.py
+++ b/pypy/module/itertools/interp_itertools.py
@@ -1,8 +1,7 @@
 from pypy.interpreter.baseobjspace import Wrappable
 from pypy.interpreter.error import OperationError
 from pypy.interpreter.typedef import TypeDef, make_weakref_descr
-from pypy.interpreter.gateway import interp2app, ObjSpace, W_Root, unwrap_spec
-from pypy.interpreter.argument import Arguments
+from pypy.interpreter.gateway import interp2app, unwrap_spec
 from pypy.rlib.rarithmetic import ovfcheck
 
 class W_Count(Wrappable):
@@ -51,7 +50,6 @@
         raise OperationError(space.w_TypeError,
                              space.wrap("expected a number"))
 
- at unwrap_spec(ObjSpace, W_Root, W_Root, W_Root)
 def W_Count___new__(space, w_subtype, w_start=0, w_step=1):
     check_number(space, w_start)
     check_number(space, w_step)
@@ -63,10 +61,10 @@
         'count',
         __module__ = 'itertools',
         __new__ = interp2app(W_Count___new__),
-        __iter__ = interp2app(W_Count.iter_w, unwrap_spec=['self']),
-        next = interp2app(W_Count.next_w, unwrap_spec=['self']),
-        __reduce__ = interp2app(W_Count.reduce_w, unwrap_spec=['self']),
-        __repr__ = interp2app(W_Count.repr_w, unwrap_spec=['self']),
+        __iter__ = interp2app(W_Count.iter_w),
+        next = interp2app(W_Count.next_w),
+        __reduce__ = interp2app(W_Count.reduce_w),
+        __repr__ = interp2app(W_Count.repr_w),
         __doc__ = """Make an iterator that returns consecutive integers starting
     with n.  If not specified n defaults to zero. Does not currently
     support python long integers. Often used as an argument to imap()
@@ -123,10 +121,10 @@
 W_Repeat.typedef = TypeDef(
         'repeat',
         __module__ = 'itertools',
-        __new__  = interp2app(W_Repeat___new__, unwrap_spec=[ObjSpace, W_Root, W_Root, W_Root]),
-        __iter__ = interp2app(W_Repeat.iter_w, unwrap_spec=['self']),
-        next     = interp2app(W_Repeat.next_w, unwrap_spec=['self']),
-        __repr__ = interp2app(W_Repeat.repr_w, unwrap_spec=['self']),
+        __new__  = interp2app(W_Repeat___new__),
+        __iter__ = interp2app(W_Repeat.iter_w),
+        next     = interp2app(W_Repeat.next_w),
+        __repr__ = interp2app(W_Repeat.repr_w),
         __doc__  = """Make an iterator that returns object over and over again.
     Runs indefinitely unless the times argument is specified.  Used
     as argument to imap() for invariant parameters to the called
@@ -176,9 +174,9 @@
 W_TakeWhile.typedef = TypeDef(
         'takewhile',
         __module__ = 'itertools',
-        __new__  = interp2app(W_TakeWhile___new__, unwrap_spec=[ObjSpace, W_Root, W_Root, W_Root]),
-        __iter__ = interp2app(W_TakeWhile.iter_w, unwrap_spec=['self']),
-        next     = interp2app(W_TakeWhile.next_w, unwrap_spec=['self']),
+        __new__  = interp2app(W_TakeWhile___new__),
+        __iter__ = interp2app(W_TakeWhile.iter_w),
+        next     = interp2app(W_TakeWhile.next_w),
         __doc__  = """Make an iterator that returns elements from the iterable as
     long as the predicate is true.
 
@@ -225,9 +223,9 @@
 W_DropWhile.typedef = TypeDef(
         'dropwhile',
         __module__ = 'itertools',
-        __new__  = interp2app(W_DropWhile___new__, unwrap_spec=[ObjSpace, W_Root, W_Root, W_Root]),
-        __iter__ = interp2app(W_DropWhile.iter_w, unwrap_spec=['self']),
-        next     = interp2app(W_DropWhile.next_w, unwrap_spec=['self']),
+        __new__  = interp2app(W_DropWhile___new__),
+        __iter__ = interp2app(W_DropWhile.iter_w),
+        next     = interp2app(W_DropWhile.next_w),
         __doc__  = """Make an iterator that drops elements from the iterable as long
     as the predicate is true; afterwards, returns every
     element. Note, the iterator does not produce any output until the
@@ -282,9 +280,9 @@
 W_IFilter.typedef = TypeDef(
         'ifilter',
         __module__ = 'itertools',
-        __new__  = interp2app(W_IFilter___new__, unwrap_spec=[ObjSpace, W_Root, W_Root, W_Root]),
-        __iter__ = interp2app(W_IFilter.iter_w, unwrap_spec=['self']),
-        next     = interp2app(W_IFilter.next_w, unwrap_spec=['self']),
+        __new__  = interp2app(W_IFilter___new__),
+        __iter__ = interp2app(W_IFilter.iter_w),
+        next     = interp2app(W_IFilter.next_w),
         __doc__  = """Make an iterator that filters elements from iterable returning
     only those for which the predicate is True.  If predicate is
     None, return the items that are true.
@@ -310,9 +308,9 @@
 W_IFilterFalse.typedef = TypeDef(
         'ifilterfalse',
         __module__ = 'itertools',
-        __new__  = interp2app(W_IFilterFalse___new__, unwrap_spec=[ObjSpace, W_Root, W_Root, W_Root]),
-        __iter__ = interp2app(W_IFilterFalse.iter_w, unwrap_spec=['self']),
-        next     = interp2app(W_IFilterFalse.next_w, unwrap_spec=['self']),
+        __new__  = interp2app(W_IFilterFalse___new__),
+        __iter__ = interp2app(W_IFilterFalse.iter_w),
+        next     = interp2app(W_IFilterFalse.next_w),
         __doc__  = """Make an iterator that filters elements from iterable returning
     only those for which the predicate is False.  If predicate is
     None, return the items that are false.
@@ -401,9 +399,9 @@
 W_ISlice.typedef = TypeDef(
         'islice',
         __module__ = 'itertools',
-        __new__  = interp2app(W_ISlice___new__, unwrap_spec=[ObjSpace, W_Root, W_Root, W_Root, 'args_w']),
-        __iter__ = interp2app(W_ISlice.iter_w, unwrap_spec=['self']),
-        next     = interp2app(W_ISlice.next_w, unwrap_spec=['self']),
+        __new__  = interp2app(W_ISlice___new__),
+        __iter__ = interp2app(W_ISlice.iter_w),
+        next     = interp2app(W_ISlice.next_w),
         __doc__  = """Make an iterator that returns selected elements from the
     iterable.  If start is non-zero, then elements from the iterable
     are skipped until start is reached. Afterward, elements are
@@ -469,11 +467,10 @@
 W_Chain.typedef = TypeDef(
         'chain',
         __module__ = 'itertools',
-        __new__  = interp2app(W_Chain___new__, unwrap_spec=[ObjSpace, W_Root, 'args_w']),
-        __iter__ = interp2app(W_Chain.iter_w, unwrap_spec=['self']),
-        next     = interp2app(W_Chain.next_w, unwrap_spec=['self']),
-        from_iterable = interp2app(chain_from_iterable, unwrap_spec=[ObjSpace, W_Root, W_Root],
-                                   as_classmethod=True),
+        __new__  = interp2app(W_Chain___new__),
+        __iter__ = interp2app(W_Chain.iter_w),
+        next     = interp2app(W_Chain.next_w),
+        from_iterable = interp2app(chain_from_iterable, as_classmethod=True),
         __doc__  = """Make an iterator that returns elements from the first iterable
     until it is exhausted, then proceeds to the next iterable, until
     all of the iterables are exhausted. Used for treating consecutive
@@ -550,9 +547,9 @@
 W_IMap.typedef = TypeDef(
         'imap',
         __module__ = 'itertools',
-        __new__  = interp2app(W_IMap___new__, unwrap_spec=[ObjSpace, W_Root, W_Root, 'args_w']),
-        __iter__ = interp2app(W_IMap.iter_w, unwrap_spec=['self']),
-        next     = interp2app(W_IMap.next_w, unwrap_spec=['self']),
+        __new__  = interp2app(W_IMap___new__),
+        __iter__ = interp2app(W_IMap.iter_w),
+        next     = interp2app(W_IMap.next_w),
         __doc__  = """Make an iterator that computes the function using arguments
     from each of the iterables. If function is set to None, then
     imap() returns the arguments as a tuple. Like map() but stops
@@ -595,9 +592,9 @@
 W_IZip.typedef = TypeDef(
         'izip',
         __module__ = 'itertools',
-        __new__  = interp2app(W_IZip___new__, unwrap_spec=[ObjSpace, W_Root, 'args_w']),
-        __iter__ = interp2app(W_IZip.iter_w, unwrap_spec=['self']),
-        next     = interp2app(W_IZip.next_w, unwrap_spec=['self']),
+        __new__  = interp2app(W_IZip___new__),
+        __iter__ = interp2app(W_IZip.iter_w),
+        next     = interp2app(W_IZip.next_w),
         __doc__  = """Make an iterator that aggregates elements from each of the
     iterables.  Like zip() except that it returns an iterator instead
     of a list. Used for lock-step iteration over several iterables at
@@ -643,7 +640,6 @@
             objects_w[index] = w_value
         return space.newtuple(objects_w)
 
- at unwrap_spec(ObjSpace, W_Root, Arguments)
 def W_IZipLongest___new__(space, w_subtype, __args__):
     arguments_w, kwds_w = __args__.unpack()
     w_fillvalue = space.w_None
@@ -666,8 +662,8 @@
         'izip_longest',
         __module__ = 'itertools',
         __new__  = interp2app(W_IZipLongest___new__),
-        __iter__ = interp2app(W_IZipLongest.iter_w, unwrap_spec=['self']),
-        next     = interp2app(W_IZipLongest.next_w, unwrap_spec=['self']),
+        __iter__ = interp2app(W_IZipLongest.iter_w),
+        next     = interp2app(W_IZipLongest.next_w),
         __doc__  = """Return an izip_longest object whose .next() method returns a tuple where
     the i-th element comes from the i-th iterable argument.  The .next()
     method continues until the longest iterable in the argument sequence
@@ -725,9 +721,9 @@
 W_Cycle.typedef = TypeDef(
         'cycle',
         __module__ = 'itertools',
-        __new__  = interp2app(W_Cycle___new__, unwrap_spec=[ObjSpace, W_Root, W_Root]),
-        __iter__ = interp2app(W_Cycle.iter_w, unwrap_spec=['self']),
-        next     = interp2app(W_Cycle.next_w, unwrap_spec=['self']),
+        __new__  = interp2app(W_Cycle___new__),
+        __iter__ = interp2app(W_Cycle.iter_w),
+        next     = interp2app(W_Cycle.next_w),
         __doc__  = """Make an iterator returning elements from the iterable and
     saving a copy of each. When the iterable is exhausted, return
     elements from the saved copy. Repeats indefinitely.
@@ -766,10 +762,9 @@
 W_StarMap.typedef = TypeDef(
         'starmap',
         __module__ = 'itertools',
-        __new__  = interp2app(W_StarMap___new__, unwrap_spec=[ObjSpace, W_Root, W_Root, W_Root]),
-
-        __iter__ = interp2app(W_StarMap.iter_w, unwrap_spec=['self']),
-        next     = interp2app(W_StarMap.next_w, unwrap_spec=['self']),
+        __new__  = interp2app(W_StarMap___new__),
+        __iter__ = interp2app(W_StarMap.iter_w),
+        next     = interp2app(W_StarMap.next_w),
         __doc__  = """Make an iterator that computes the function using arguments
     tuples obtained from the iterable. Used instead of imap() when
     argument parameters are already grouped in tuples from a single
@@ -786,6 +781,7 @@
     """)
 
 
+ at unwrap_spec(n=int)
 def tee(space, w_iterable, n=2):
     """Return n independent iterators from a single iterable.
     Note : once tee() has made a split, the original iterable
@@ -825,7 +821,6 @@
         tee_state = TeeState(space, w_iterable)
         iterators_w = [space.wrap(W_TeeIterable(space, tee_state)) for x in range(n)]
     return space.newtuple(iterators_w)
-tee.unwrap_spec = [ObjSpace, W_Root, int]
 
 class TeeState(object):
     def __init__(self, space, w_iterable):
@@ -872,11 +867,9 @@
 W_TeeIterable.typedef = TypeDef(
         '_tee',
         __module__ = 'itertools',
-        __new__ = interp2app(W_TeeIterable___new__, unwrap_spec=[ObjSpace,
-                                                                 W_Root,
-                                                                 W_Root]),
-        __iter__ = interp2app(W_TeeIterable.iter_w, unwrap_spec=['self']),
-        next     = interp2app(W_TeeIterable.next_w, unwrap_spec=['self']),
+        __new__ = interp2app(W_TeeIterable___new__),
+        __iter__ = interp2app(W_TeeIterable.iter_w),
+        next     = interp2app(W_TeeIterable.next_w),
         __weakref__ = make_weakref_descr(W_TeeIterable),
         )
 W_TeeIterable.typedef.acceptable_as_base_class = False
@@ -977,9 +970,9 @@
 W_GroupBy.typedef = TypeDef(
         'groupby',
         __module__ = 'itertools',
-        __new__  = interp2app(W_GroupBy___new__, unwrap_spec=[ObjSpace, W_Root, W_Root, W_Root]),
-        __iter__ = interp2app(W_GroupBy.iter_w, unwrap_spec=['self']),
-        next     = interp2app(W_GroupBy.next_w, unwrap_spec=['self']),
+        __new__  = interp2app(W_GroupBy___new__),
+        __iter__ = interp2app(W_GroupBy.iter_w),
+        next     = interp2app(W_GroupBy.next_w),
         __doc__  = """Make an iterator that returns consecutive keys and groups from the
     iterable. The key is a function computing a key value for each
     element. If not specified or is None, key defaults to an identity
@@ -1024,8 +1017,8 @@
 W_GroupByIterator.typedef = TypeDef(
         '_groupby',
         __module__ = 'itertools',
-        __iter__ = interp2app(W_GroupByIterator.iter_w, unwrap_spec=['self']),
-        next     = interp2app(W_GroupByIterator.next_w, unwrap_spec=['self']))
+        __iter__ = interp2app(W_GroupByIterator.iter_w),
+        next     = interp2app(W_GroupByIterator.next_w))
 W_GroupByIterator.typedef.acceptable_as_base_class = False
 
 
@@ -1056,10 +1049,9 @@
 W_Compress.typedef = TypeDef(
     'compress',
     __module__ = 'itertools',
-    __new__ = interp2app(W_Compress__new__,
-                         unwrap_spec=[ObjSpace, W_Root, W_Root, W_Root]),
-    __iter__ = interp2app(W_Compress.iter_w, unwrap_spec=['self']),
-    next     = interp2app(W_Compress.next_w, unwrap_spec=['self']),
+    __new__ = interp2app(W_Compress__new__),
+    __iter__ = interp2app(W_Compress.iter_w),
+    next     = interp2app(W_Compress.next_w),
     __doc__ = """Make an iterator that filters elements from *data* returning
    only those that have a corresponding element in *selectors* that evaluates to
    ``True``.  Stops when either the *data* or *selectors* iterables has been
@@ -1115,11 +1107,9 @@
             else:
                 break
 
-    @unwrap_spec("self", ObjSpace)
     def iter_w(self, space):
         return space.wrap(self)
 
-    @unwrap_spec("self", ObjSpace)
     def next_w(self, space):
         if not self.cont:
             raise OperationError(space.w_StopIteration, space.w_None)
@@ -1131,7 +1121,6 @@
         return space.newtuple(l)
 
 
- at unwrap_spec(ObjSpace, W_Root, Arguments)
 def W_Product__new__(space, w_subtype, __args__):
     arguments_w, kwds_w = __args__.unpack()
     w_repeat = space.wrap(1)
@@ -1197,11 +1186,9 @@
     def max_index(self, j):
         return self.indices[j - 1] + 1
 
-    @unwrap_spec("self", ObjSpace)
     def descr__iter__(self, space):
         return self
 
-    @unwrap_spec("self", ObjSpace)
     def descr_next(self, space):
         if self.stopped:
             raise OperationError(space.w_StopIteration, space.w_None)
@@ -1242,7 +1229,7 @@
         self.last_result_w = result_w
         return space.newtuple(result_w)
 
- at unwrap_spec(ObjSpace, W_Root, W_Root, int)
+ at unwrap_spec(r=int)
 def W_Combinations__new__(space, w_subtype, w_iterable, r):
     pool_w = space.fixedview(w_iterable)
     if r < 0:
@@ -1278,7 +1265,7 @@
     def max_index(self, j):
         return self.indices[j - 1]
 
- at unwrap_spec(ObjSpace, W_Root, W_Root, int)
+ at unwrap_spec(r=int)
 def W_CombinationsWithReplacement__new__(space, w_subtype, w_iterable, r):
     pool_w = space.fixedview(w_iterable)
     if r < 0:
@@ -1316,11 +1303,9 @@
             self.indices = range(n)
             self.cycles = range(n, n_minus_r, -1)
 
-    @unwrap_spec("self", ObjSpace)
     def descr__iter__(self, space):
         return self
 
-    @unwrap_spec("self", ObjSpace)
     def descr_next(self, space):
         if self.stopped:
             raise OperationError(space.w_StopIteration, space.w_None)
@@ -1347,7 +1332,6 @@
         self.stopped = True
         return w_result
 
- at unwrap_spec(ObjSpace, W_Root, W_Root, W_Root)
 def W_Permutations__new__(space, w_subtype, w_iterable, w_r=None):
     pool_w = space.fixedview(w_iterable)
     if space.is_w(w_r, space.w_None):

From commits-noreply at bitbucket.org  Wed Feb 16 19:19:58 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 19:19:58 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: Remove this script,
	it does not generate a correct module anymore.
Message-ID: <20110216181958.2C631282BF7@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42077:4216d4770c90
Date: 2011-02-16 17:12 +0100
http://bitbucket.org/pypy/pypy/changeset/4216d4770c90/

Log:	Remove this script, it does not generate a correct module anymore.

diff --git a/pypy/module/math/_genmath.py b/pypy/module/math/_genmath.py
deleted file mode 100644
--- a/pypy/module/math/_genmath.py
+++ /dev/null
@@ -1,62 +0,0 @@
-# ONESHOT SCRIPT (probably can go away soon)
-# to generate the mixed module 'math' (see same directory) 
-import py
-import math
-import re
-import sys
-rex_arg = re.compile(".*\((.*)\).*")
-
-if __name__ == '__main__': 
-    print py.code.Source("""
-        import math 
-        from pypy.interpreter.gateway import ObjSpace
-
-    """)
-    names = []
-    for name, func in math.__dict__.items(): 
-        if not callable(func): 
-            continue
-        sig = func.__doc__.split('\n')[0].strip()
-        sig = sig.split('->')[0].strip()
-        m = rex_arg.match(sig) 
-        assert m
-        args = m.group(1)
-        args = ", ".join(args.split(','))
-        sig = sig.replace('(', '(space,')
-        sig = ", ".join(sig.split(','))
-        argc = len(args.split(','))
-        unwrap_spec = ['ObjSpace']
-        unwrap_spec += ['float'] * argc 
-        unwrap_spec = ", ".join(unwrap_spec)
-        doc = func.__doc__.replace('\n', '\n       ')
-        
-        print py.code.Source('''
-            def %(sig)s: 
-                """%(doc)s
-                """
-                return space.wrap(math.%(name)s(%(args)s))
-            %(name)s.unwrap_spec = [%(unwrap_spec)s]
-        ''' % locals())
-        names.append(name) 
-
-    print >>sys.stderr, py.code.Source("""
-        # Package initialisation
-        from pypy.interpreter.mixedmodule import MixedModule
-
-        class Module(MixedModule):
-            appleveldefs = {
-            }
-            interpleveldefs = {
-    """)
-        
-    for name in names: 
-        space = " " * (15-len(name))
-        print >>sys.stderr, (
-            "       %(name)r%(space)s: 'interp_math.%(name)s'," % locals())
-    print >>sys.stderr, py.code.Source("""
-        }
-    """) 
-            
-        
-        
-        

From commits-noreply at bitbucket.org  Wed Feb 16 19:19:59 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 19:19:59 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: Remove all unwrap_spec from
	the math module
Message-ID: <20110216181959.1720D282BF7@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42078:97ec08e562c4
Date: 2011-02-16 17:18 +0100
http://bitbucket.org/pypy/pypy/changeset/97ec08e562c4/

Log:	Remove all unwrap_spec from the math module

diff --git a/pypy/module/math/interp_math.py b/pypy/module/math/interp_math.py
--- a/pypy/module/math/interp_math.py
+++ b/pypy/module/math/interp_math.py
@@ -3,7 +3,7 @@
 
 from pypy.rlib import rarithmetic, unroll
 from pypy.interpreter.error import OperationError
-from pypy.interpreter.gateway import ObjSpace, W_Root, NoneNotWrapped
+from pypy.interpreter.gateway import NoneNotWrapped
 
 class State: 
     def __init__(self, space): 
@@ -61,7 +61,6 @@
 def trunc(space, w_x):
     """Truncate x."""
     return space.trunc(w_x)
-trunc.unwrap_spec = [ObjSpace, W_Root]
 
 def copysign(space, w_x, w_y):
     """Return x with the sign of y."""
@@ -69,17 +68,14 @@
     x = _get_double(space, w_x)
     y = _get_double(space, w_y)
     return space.wrap(rarithmetic.copysign(x, y))
-copysign.unwrap_spec = [ObjSpace, W_Root, W_Root]
 
 def isinf(space, w_x):
     """Return True if x is infinity."""
     return space.wrap(rarithmetic.isinf(_get_double(space, w_x)))
-isinf.unwrap_spec = [ObjSpace, W_Root]
 
 def isnan(space, w_x):
     """Return True if x is not a number."""
     return space.wrap(rarithmetic.isnan(_get_double(space, w_x)))
-isnan.unwrap_spec = [ObjSpace, W_Root]
 
 def pow(space, w_x, w_y):
     """pow(x,y)
@@ -87,7 +83,6 @@
        Return x**y (x to the power of y).
     """
     return math2(space, math.pow, w_x, w_y)
-pow.unwrap_spec = [ObjSpace, W_Root, W_Root]
 
 def cosh(space, w_x):
     """cosh(x)
@@ -95,7 +90,6 @@
        Return the hyperbolic cosine of x.
     """
     return math1(space, math.cosh, w_x)
-cosh.unwrap_spec = [ObjSpace, W_Root]
 
 def ldexp(space, w_x,  w_i):
     """ldexp(x, i) -> x * (2**i)
@@ -124,7 +118,6 @@
         raise OperationError(space.w_ValueError,
                              space.wrap("math domain error"))
     return space.wrap(r)
-ldexp.unwrap_spec = [ObjSpace, W_Root, W_Root]
 
 def hypot(space, w_x, w_y):
     """hypot(x,y)
@@ -132,7 +125,6 @@
        Return the Euclidean distance, sqrt(x*x + y*y).
     """
     return math2(space, math.hypot, w_x, w_y)
-hypot.unwrap_spec = [ObjSpace, W_Root, W_Root]
 
 def tan(space, w_x):
     """tan(x)
@@ -140,7 +132,6 @@
        Return the tangent of x (measured in radians).
     """
     return math1(space, math.tan, w_x)
-tan.unwrap_spec = [ObjSpace, W_Root]
 
 def asin(space, w_x):
     """asin(x)
@@ -148,7 +139,6 @@
        Return the arc sine (measured in radians) of x.
     """
     return math1(space, math.asin, w_x)
-asin.unwrap_spec = [ObjSpace, W_Root]
 
 def fabs(space, w_x):
     """fabs(x)
@@ -156,7 +146,6 @@
        Return the absolute value of the float x.
     """
     return math1(space, math.fabs, w_x)
-fabs.unwrap_spec = [ObjSpace, W_Root]
 
 def floor(space, w_x):
     """floor(x)
@@ -165,7 +154,6 @@
        This is the largest integral value <= x.
     """
     return math1(space, math.floor, w_x)
-floor.unwrap_spec = [ObjSpace, W_Root]
 
 def sqrt(space, w_x):
     """sqrt(x)
@@ -173,7 +161,6 @@
        Return the square root of x.
     """
     return math1(space, math.sqrt, w_x)
-sqrt.unwrap_spec = [ObjSpace, W_Root]
 
 def frexp(space, w_x):
     """frexp(x)
@@ -184,7 +171,6 @@
     """
     mant, expo = math1_w(space, math.frexp, w_x)
     return space.newtuple([space.wrap(mant), space.wrap(expo)])
-frexp.unwrap_spec = [ObjSpace, W_Root]
 
 degToRad = math.pi / 180.0
 
@@ -192,7 +178,6 @@
     """degrees(x) -> converts angle x from radians to degrees
     """
     return space.wrap(_get_double(space, w_x) / degToRad)
-degrees.unwrap_spec = [ObjSpace, W_Root]
 
 def _log_any(space, w_x, base):
     # base is supposed to be positive or 0.0, which means we use e
@@ -230,13 +215,11 @@
             # just for raising the proper errors
             return math1(space, math.log, w_base)
     return _log_any(space, w_x, base)
-log.unwrap_spec = [ObjSpace, W_Root, W_Root]
 
 def log10(space, w_x):
     """log10(x) -> the base 10 logarithm of x.
     """
     return _log_any(space, w_x, 10.0)
-log10.unwrap_spec = [ObjSpace, W_Root]
 
 def fmod(space, w_x, w_y):
     """fmod(x,y)
@@ -244,7 +227,6 @@
        Return fmod(x, y), according to platform C.  x % y may differ.
     """
     return math2(space, math.fmod, w_x, w_y)
-fmod.unwrap_spec = [ObjSpace, W_Root, W_Root]
 
 def atan(space, w_x):
     """atan(x)
@@ -252,7 +234,6 @@
        Return the arc tangent (measured in radians) of x.
     """
     return math1(space, math.atan, w_x)
-atan.unwrap_spec = [ObjSpace, W_Root]
 
 def ceil(space, w_x):
     """ceil(x)
@@ -261,7 +242,6 @@
        This is the smallest integral value >= x.
     """
     return math1(space, math.ceil, w_x)
-ceil.unwrap_spec = [ObjSpace, W_Root]
 
 def sinh(space, w_x):
     """sinh(x)
@@ -269,7 +249,6 @@
        Return the hyperbolic sine of x.
     """
     return math1(space, math.sinh, w_x)
-sinh.unwrap_spec = [ObjSpace, W_Root]
 
 def cos(space, w_x):
     """cos(x)
@@ -277,7 +256,6 @@
        Return the cosine of x (measured in radians).
     """
     return math1(space, math.cos, w_x)
-cos.unwrap_spec = [ObjSpace, W_Root]
 
 def tanh(space, w_x):
     """tanh(x)
@@ -285,13 +263,11 @@
        Return the hyperbolic tangent of x.
     """
     return math1(space, math.tanh, w_x)
-tanh.unwrap_spec = [ObjSpace, W_Root]
 
 def radians(space, w_x):
     """radians(x) -> converts angle x from degrees to radians
     """
     return space.wrap(_get_double(space, w_x) * degToRad)
-radians.unwrap_spec = [ObjSpace, W_Root]
 
 def sin(space, w_x):
     """sin(x)
@@ -299,7 +275,6 @@
        Return the sine of x (measured in radians).
     """
     return math1(space, math.sin, w_x)
-sin.unwrap_spec = [ObjSpace, W_Root]
 
 def atan2(space, w_y, w_x):
     """atan2(y, x)
@@ -308,7 +283,6 @@
        Unlike atan(y/x), the signs of both x and y are considered.
     """
     return math2(space, math.atan2, w_y,  w_x)
-atan2.unwrap_spec = [ObjSpace, W_Root, W_Root]
 
 def modf(space, w_x):
     """modf(x)
@@ -318,7 +292,6 @@
     """
     frac, intpart = math1_w(space, math.modf, w_x)
     return space.newtuple([space.wrap(frac), space.wrap(intpart)])
-modf.unwrap_spec = [ObjSpace, W_Root]
 
 def exp(space, w_x):
     """exp(x)
@@ -326,7 +299,6 @@
        Return e raised to the power of x.
     """
     return math1(space, math.exp, w_x)
-exp.unwrap_spec = [ObjSpace, W_Root]
 
 def acos(space, w_x):
     """acos(x)
@@ -334,7 +306,6 @@
        Return the arc cosine (measured in radians) of x.
     """
     return math1(space, math.acos, w_x)
-acos.unwrap_spec = [ObjSpace, W_Root]
 
 def fsum(space, w_iterable):
     """Sum an iterable of floats, trying to keep precision."""
@@ -400,7 +371,6 @@
             if y == yr:
                 hi = v
     return space.wrap(hi)
-fsum.unwrap_spec = [ObjSpace, W_Root]
 
 def factorial(space, w_x):
     """Find x!."""
@@ -421,47 +391,38 @@
 def log1p(space, w_x):
     """Find log(x + 1)."""
     return math1(space, rarithmetic.log1p, w_x)
-log1p.unwrap_spec = [ObjSpace, W_Root]
 
 def acosh(space, w_x):
     """Inverse hyperbolic cosine"""
     return math1(space, rarithmetic.acosh, w_x)
-acosh.unwrap_spec = [ObjSpace, W_Root]
 
 def asinh(space, w_x):
     """Inverse hyperbolic sine"""
     return math1(space, rarithmetic.asinh, w_x)
-asinh.unwrap_spec = [ObjSpace, W_Root]
 
 def atanh(space, w_x):
     """Inverse hyperbolic tangent"""
     return math1(space, rarithmetic.atanh, w_x)
-atanh.unwrap_spec = [ObjSpace, W_Root]
 
 def expm1(space, w_x):
     """exp(x) - 1"""
     return math1(space, rarithmetic.expm1, w_x)
-expm1.unwrap_spec = [ObjSpace, W_Root]
 
 def erf(space, w_x):
     """The error function"""
     return math1(space, _erf, w_x)
-erf.unwrap_spec = [ObjSpace, W_Root]
 
 def erfc(space, w_x):
     """The complementary error function"""
     return math1(space, _erfc, w_x)
-erfc.unwrap_spec = [ObjSpace, W_Root]
 
 def gamma(space, w_x):
     """Compute the gamma function for x."""
     return math1(space, _gamma, w_x)
-gamma.unwrap_spec = [ObjSpace, W_Root]
 
 def lgamma(space, w_x):
     """Compute the natural logarithm of the gamma function for x."""
     return math1(space, _lgamma, w_x)
-lgamma.unwrap_spec = [ObjSpace, W_Root]
 
 # Implementation of the error function, the complimentary error function, the
 # gamma function, and the natural log of the gamma function.  These exist in

From commits-noreply at bitbucket.org  Wed Feb 16 19:20:00 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 19:20:00 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: simpler unwrap_spec in
	micronumpy, mmap modules
Message-ID: <20110216182000.C76BE282BF2@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42079:b2516cce1853
Date: 2011-02-16 17:27 +0100
http://bitbucket.org/pypy/pypy/changeset/b2516cce1853/

Log:	simpler unwrap_spec in micronumpy, mmap modules

diff --git a/pypy/module/micronumpy/numarray.py b/pypy/module/micronumpy/numarray.py
--- a/pypy/module/micronumpy/numarray.py
+++ b/pypy/module/micronumpy/numarray.py
@@ -1,8 +1,8 @@
 
-from pypy.interpreter.baseobjspace import ObjSpace, W_Root, Wrappable
+from pypy.interpreter.baseobjspace import Wrappable
 from pypy.interpreter.error import OperationError
 from pypy.interpreter.typedef import TypeDef
-from pypy.interpreter.gateway import interp2app, NoneNotWrapped
+from pypy.interpreter.gateway import interp2app, unwrap_spec, NoneNotWrapped
 from pypy.rlib.debug import make_sure_not_resized
 
 class BaseNumArray(Wrappable):
@@ -16,6 +16,7 @@
         self.storage = [0] * dim
         make_sure_not_resized(self.storage)
 
+    @unwrap_spec(index=int)
     def descr_getitem(self, index):
         space = self.space
         try:
@@ -23,8 +24,8 @@
         except IndexError:
             raise OperationError(space.w_IndexError,
                                  space.wrap("list index out of range"))
-    descr_getitem.unwrap_spec = ['self', int]
 
+    @unwrap_spec(index=int, value=int)
     def descr_setitem(self, index, value):
         space = self.space
         try:
@@ -33,11 +34,9 @@
             raise OperationError(space.w_IndexError,
                                  space.wrap("list index out of range"))
         return space.w_None
-    descr_setitem.unwrap_spec = ['self', int, int]
 
     def descr_len(self):
         return self.space.wrap(len(self.storage))
-    descr_len.unwrap_spec = ['self']
 
 NumArray.typedef = TypeDef(
     'NumArray',
@@ -84,19 +83,17 @@
         indexes = self._unpack_indexes(space, w_index)
         pos = compute_pos(space, indexes, self.dim)
         return space.wrap(self.storage[pos])
-    descr_getitem.unwrap_spec = ['self', W_Root]
 
+    @unwrap_spec(value=int)
     def descr_setitem(self, w_index, value):
         space = self.space
         indexes = self._unpack_indexes(space, w_index)
         pos = compute_pos(space, indexes, self.dim)
         self.storage[pos] = value
         return space.w_None
-    descr_setitem.unwrap_spec = ['self', W_Root, int]
 
     def descr_len(self):
         return self.space.wrap(self.dim[0])
-    descr_len.unwrap_spec = ['self']
 
 MultiDimArray.typedef = TypeDef(
     'NumArray',
@@ -124,4 +121,3 @@
         return space.wrap(NumArray(space, dim[0], dtype))
     else:
         return space.wrap(MultiDimArray(space, dim, dtype))
-zeros.unwrap_spec = [ObjSpace, W_Root, W_Root]

diff --git a/pypy/module/micronumpy/ufunc.py b/pypy/module/micronumpy/ufunc.py
--- a/pypy/module/micronumpy/ufunc.py
+++ b/pypy/module/micronumpy/ufunc.py
@@ -19,4 +19,3 @@
         else:
             res.storage[i] = two
     return space.wrap(res)
-minimum.unwrap_spec = [ObjSpace, W_Root, W_Root]

diff --git a/pypy/module/mmap/interp_mmap.py b/pypy/module/mmap/interp_mmap.py
--- a/pypy/module/mmap/interp_mmap.py
+++ b/pypy/module/mmap/interp_mmap.py
@@ -1,9 +1,9 @@
 from pypy.rpython.tool import rffi_platform
 from pypy.rpython.lltypesystem import rffi, lltype
 from pypy.interpreter.error import OperationError, wrap_oserror
-from pypy.interpreter.baseobjspace import W_Root, ObjSpace, Wrappable
+from pypy.interpreter.baseobjspace import Wrappable
 from pypy.interpreter.typedef import TypeDef
-from pypy.interpreter.gateway import interp2app, NoneNotWrapped
+from pypy.interpreter.gateway import interp2app, unwrap_spec, NoneNotWrapped
 from pypy.rlib import rmmap
 from pypy.rlib.rmmap import RValueError, RTypeError, ROverflowError
 import sys
@@ -18,7 +18,6 @@
         
     def close(self):
         self.mmap.close()
-    close.unwrap_spec = ['self']
 
     def read_byte(self):
         try:
@@ -26,17 +25,16 @@
         except RValueError, v:
             raise OperationError(self.space.w_ValueError,
                                  self.space.wrap(v.message))
-    read_byte.unwrap_spec = ['self']
-    
+
     def readline(self):
         return self.space.wrap(self.mmap.readline())
-    readline.unwrap_spec = ['self']
-    
+
+    @unwrap_spec(num=int)
     def read(self, num=-1):
         self.check_valid()
         return self.space.wrap(self.mmap.read(num))
-    read.unwrap_spec = ['self', int]
 
+    @unwrap_spec(tofind='bufferstr')
     def find(self, tofind, w_start=NoneNotWrapped, w_end=NoneNotWrapped):
         space = self.space
         if w_start is None:
@@ -48,8 +46,8 @@
         else:
             end = space.getindex_w(w_end, None)
         return space.wrap(self.mmap.find(tofind, start, end))
-    find.unwrap_spec = ['self', 'bufferstr', W_Root, W_Root]
 
+    @unwrap_spec(tofind='bufferstr')
     def rfind(self, tofind, w_start=NoneNotWrapped, w_end=NoneNotWrapped):
         space = self.space
         if w_start is None:
@@ -61,27 +59,25 @@
         else:
             end = space.getindex_w(w_end, None)
         return space.wrap(self.mmap.find(tofind, start, end, True))
-    rfind.unwrap_spec = ['self', 'bufferstr', W_Root, W_Root]
 
+    @unwrap_spec(pos='index', whence=int)
     def seek(self, pos, whence=0):
         try:
             self.mmap.seek(pos, whence)
         except RValueError, v:
             raise OperationError(self.space.w_ValueError,
-                                 self.space.wrap(v.message))            
-    seek.unwrap_spec = ['self', 'index', int]
-    
+                                 self.space.wrap(v.message))
+
     def tell(self):
         return self.space.wrap(self.mmap.tell())
-    tell.unwrap_spec = ['self']
-    
+
     def descr_size(self):
         try:
             return self.space.wrap(self.mmap.file_size())
         except OSError, e:
             raise mmap_error(self.space, e)
-    descr_size.unwrap_spec = ['self']
-    
+
+    @unwrap_spec(data='bufferstr')
     def write(self, data):
         self.check_writeable()
         try:
@@ -89,8 +85,8 @@
         except RValueError, v:
             raise OperationError(self.space.w_ValueError,
                                  self.space.wrap(v.message))
-    write.unwrap_spec = ['self', 'bufferstr']
-    
+
+    @unwrap_spec(byte=str)
     def write_byte(self, byte):
         try:
             self.mmap.write_byte(byte)
@@ -100,8 +96,8 @@
         except RTypeError, v:
             raise OperationError(self.space.w_TypeError,
                                  self.space.wrap(v.message))
-    write_byte.unwrap_spec = ['self', str]
 
+    @unwrap_spec(offset=int, size=int)
     def flush(self, offset=0, size=0):
         try:
             return self.space.wrap(self.mmap.flush(offset, size))
@@ -110,16 +106,16 @@
                                  self.space.wrap(v.message))
         except OSError, e:
             raise mmap_error(self.space, e)
-    flush.unwrap_spec = ['self', int, int]
-    
+
+    @unwrap_spec(dest=int, src=int, count=int)
     def move(self, dest, src, count):
         try:
             self.mmap.move(dest, src, count)
         except RValueError, v:
             raise OperationError(self.space.w_ValueError,
                                  self.space.wrap(v.message))
-    move.unwrap_spec = ['self', int, int, int]
-    
+
+    @unwrap_spec(newsize=int)
     def resize(self, newsize):
         self.check_valid()
         self.check_resizeable()
@@ -127,11 +123,9 @@
             self.mmap.resize(newsize)
         except OSError, e:
             raise mmap_error(self.space, e)
-    resize.unwrap_spec = ['self', int]
-    
+
     def __len__(self):
         return self.space.wrap(self.mmap.size)
-    __len__.unwrap_spec = ['self']
 
     def check_valid(self):
         try:
@@ -171,7 +165,6 @@
             res = "".join([self.mmap.getitem(i)
                            for i in range(start, stop, step)])
             return space.wrap(res)
-    descr_getitem.unwrap_spec = ['self', W_Root]
 
     def descr_setitem(self, w_index, w_value):
         space = self.space
@@ -194,17 +187,17 @@
             for i in range(length):
                 self.mmap.setitem(start, value[i])
                 start += step
-    descr_setitem.unwrap_spec = ['self', W_Root, W_Root]
 
     def descr_buffer(self):
         # XXX improve to work directly on the low-level address
         from pypy.interpreter.buffer import StringLikeBuffer
         space = self.space
         return space.wrap(StringLikeBuffer(space, space.wrap(self)))
-    descr_buffer.unwrap_spec = ['self']
 
 if rmmap._POSIX:
 
+    @unwrap_spec(fileno=int, length='index', flags=int,
+                 prot=int, access=int, offset='index')
     def mmap(space, w_subtype, fileno, length, flags=rmmap.MAP_SHARED,
              prot=rmmap.PROT_WRITE | rmmap.PROT_READ,
              access=rmmap._ACCESS_DEFAULT, offset=0):
@@ -222,10 +215,11 @@
         except ROverflowError, e:
             raise OperationError(space.w_OverflowError, space.wrap(e.message))
         return space.wrap(self)
-    mmap.unwrap_spec = [ObjSpace, W_Root, int, 'index', int, int, int, 'index']
 
 elif rmmap._MS_WINDOWS:
 
+    @unwrap_spec(fileno=int, length='index', tagname=str,
+                 access=int, offset='index')
     def mmap(space, w_subtype, fileno, length, tagname="",
              access=rmmap._ACCESS_DEFAULT, offset=0):
         self = space.allocate_instance(W_MMap, w_subtype)
@@ -242,7 +236,6 @@
         except ROverflowError, e:
             raise OperationError(space.w_OverflowError, space.wrap(e.message))
         return space.wrap(self)
-    mmap.unwrap_spec = [ObjSpace, W_Root, int, 'index', str, int, 'index']
 
 W_MMap.typedef = TypeDef("mmap",
     __new__ = interp2app(mmap),

From commits-noreply at bitbucket.org  Wed Feb 16 19:20:05 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 19:20:05 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: Simplify unwrap_spec in the
	oracle module
Message-ID: <20110216182005.328EE282BF2@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42080:dc7c35233392
Date: 2011-02-16 17:47 +0100
http://bitbucket.org/pypy/pypy/changeset/dc7c35233392/

Log:	Simplify unwrap_spec in the oracle module

diff --git a/pypy/module/oracle/interp_pool.py b/pypy/module/oracle/interp_pool.py
--- a/pypy/module/oracle/interp_pool.py
+++ b/pypy/module/oracle/interp_pool.py
@@ -1,7 +1,7 @@
 from pypy.interpreter.baseobjspace import Wrappable
 from pypy.interpreter.argument import Arguments, Signature
-from pypy.interpreter.gateway import ObjSpace, W_Root, NoneNotWrapped
-from pypy.interpreter.gateway import interp2app
+from pypy.interpreter.gateway import NoneNotWrapped
+from pypy.interpreter.gateway import interp2app, unwrap_spec
 from pypy.interpreter.typedef import TypeDef, GetSetProperty
 from pypy.interpreter.typedef import interp_attrproperty, interp_attrproperty_w
 from pypy.interpreter.error import OperationError
@@ -17,6 +17,8 @@
     def __init__(self):
         self.environment = None
 
+    @unwrap_spec(min=int, max=int, increment=int,
+                 threaded=bool, getmode=int, events=bool, homogeneous=bool)
     def descr_new(space, w_subtype,
                   w_user, w_password, w_dsn,
                   min, max, increment,
@@ -105,11 +107,6 @@
             dsn_buf.clear()
 
         return space.wrap(self)
-    descr_new.unwrap_spec = [ObjSpace, W_Root,
-                             W_Root, W_Root, W_Root,
-                             int, int, int,
-                             W_Root,
-                             bool, int, bool, bool]
 
     def checkConnected(self, space):
         if not self.handle:
@@ -136,15 +133,12 @@
                             __args__.keywords + ["pool"],
                             __args__.keywords_w + [space.wrap(self)])
         return space.call_args(self.w_connectionType, newargs)
-    acquire.unwrap_spec = ['self', ObjSpace, Arguments]
 
     def release(self, space, w_connection):
         self._release(space, w_connection, roci.OCI_DEFAULT)
-    release.unwrap_spec = ['self', ObjSpace, W_Root]
 
     def drop(self, space, w_connection):
         self._release(space, w_connection, roci.OCI_SESSRLS_DROPSESS)
-    drop.unwrap_spec = ['self', ObjSpace, W_Root]
 
     def _release(self, space, w_connection, mode):
         from pypy.module.oracle.interp_connect import W_Connection

diff --git a/pypy/module/oracle/interp_object.py b/pypy/module/oracle/interp_object.py
--- a/pypy/module/oracle/interp_object.py
+++ b/pypy/module/oracle/interp_object.py
@@ -1,8 +1,7 @@
 from pypy.interpreter.baseobjspace import Wrappable
 from pypy.interpreter.typedef import TypeDef, GetSetProperty
 from pypy.interpreter.typedef import interp_attrproperty
-from pypy.interpreter.gateway import ObjSpace
-from pypy.interpreter.gateway import interp2app
+from pypy.interpreter.gateway import interp2app, unwrap_spec
 from pypy.interpreter.error import OperationError
 from pypy.rpython.lltypesystem import rffi, lltype
 
@@ -344,6 +343,7 @@
         self.indicator = indicator
         self.isIndependent = isIndependent
 
+    @unwrap_spec(attr=str)
     def getattr(self, space, attr):
         try:
             attribute = self.objectType.attributesByName[attr]
@@ -405,8 +405,6 @@
             lltype.free(nameptr, flavor='raw')
             lltype.free(namelenptr, flavor='raw')
 
-    getattr.unwrap_spec = ['self', ObjSpace, str]
-
 W_ExternalObject.typedef = TypeDef(
     'ExternalObject',
     type = interp_attrproperty('objectType', W_ExternalObject),

diff --git a/pypy/module/oracle/interp_variable.py b/pypy/module/oracle/interp_variable.py
--- a/pypy/module/oracle/interp_variable.py
+++ b/pypy/module/oracle/interp_variable.py
@@ -1,8 +1,7 @@
 from pypy.interpreter.baseobjspace import Wrappable
 from pypy.interpreter.typedef import TypeDef
 from pypy.interpreter.typedef import interp_attrproperty
-from pypy.interpreter.gateway import ObjSpace, W_Root
-from pypy.interpreter.gateway import interp2app
+from pypy.interpreter.gateway import interp2app, unwrap_spec
 from pypy.interpreter.error import OperationError
 from pypy.rpython.lltypesystem import rffi, lltype
 from pypy.rlib.rarithmetic import ovfcheck
@@ -390,11 +389,11 @@
             [self.getSingleValue(space, i)
              for i in range(numElements)])
 
+    @unwrap_spec(pos=int)
     def getValue(self, space, pos=0):
         if self.isArray:
             return self.getArrayValue(space, self.actualElementsPtr[0])
         return self.getSingleValue(space, pos)
-    getValue.unwrap_spec = ['self', ObjSpace, int]
 
     def setSingleValue(self, space, pos, w_value):
         # ensure we do not exceed the number of allocated elements
@@ -437,12 +436,12 @@
         for i in range(len(elements_w)):
             self.setSingleValue(space, i, elements_w[i])
 
+    @unwrap_spec(pos=int)
     def setValue(self, space, pos, w_value):
         if self.isArray:
             self.setArrayValue(space, w_value)
         else:
             self.setSingleValue(space, pos, w_value)
-    setValue.unwrap_spec = ['self', ObjSpace, int, W_Root]
 
 
 W_Variable.typedef = TypeDef(

diff --git a/pypy/module/oracle/interp_lob.py b/pypy/module/oracle/interp_lob.py
--- a/pypy/module/oracle/interp_lob.py
+++ b/pypy/module/oracle/interp_lob.py
@@ -1,7 +1,6 @@
 from pypy.interpreter.baseobjspace import Wrappable
 from pypy.interpreter.typedef import TypeDef
-from pypy.interpreter.gateway import ObjSpace
-from pypy.interpreter.gateway import interp2app
+from pypy.interpreter.gateway import interp2app, unwrap_spec
 from pypy.interpreter.error import OperationError
 
 from pypy.module.oracle.interp_error import get
@@ -19,24 +18,22 @@
                 space.wrap(
                     "LOB variable no longer valid after subsequent fetch"))
 
+    @unwrap_spec(offset=int, amount=int)
     def read(self, space, offset=-1, amount=-1):
         self._verify(space)
         return self.lobVar.read(space, self.pos, offset, amount)
-    read.unwrap_spec = ['self', ObjSpace, int, int]
 
     def size(self, space):
         self._verify(space)
         return space.wrap(self.lobVar.getLength(space, self.pos))
-    size.unwrap_spec = ['self', ObjSpace]
 
+    @unwrap_spec(newSize=int)
     def trim(self, space, newSize=0):
         self._verify(space)
         self.lobVar.trim(space, self.pos, newSize)
-    trim.unwrap_spec = ['self', ObjSpace, int]
 
     def desc_str(self, space):
         return self.read(space, offset=1, amount=-1)
-    desc_str.unwrap_spec = ['self', ObjSpace]
 
 W_ExternalLob.typedef = TypeDef(
     'ExternalLob',

diff --git a/pypy/module/oracle/interp_connect.py b/pypy/module/oracle/interp_connect.py
--- a/pypy/module/oracle/interp_connect.py
+++ b/pypy/module/oracle/interp_connect.py
@@ -1,5 +1,5 @@
 from pypy.interpreter.baseobjspace import Wrappable
-from pypy.interpreter.gateway import ObjSpace, W_Root, NoneNotWrapped
+from pypy.interpreter.gateway import unwrap_spec, NoneNotWrapped
 from pypy.interpreter.typedef import (TypeDef, interp_attrproperty_w,
                                       GetSetProperty)
 from pypy.interpreter.gateway import interp2app
@@ -30,6 +30,10 @@
         self.w_version = None
         self.release = False
 
+
+    @unwrap_spec(mode=int, handle=int,
+                 threaded=bool, twophase=bool, events=bool,
+                 purity=bool)
     def descr_new(space, w_subtype,
                   w_user=NoneNotWrapped,
                   w_password=NoneNotWrapped,
@@ -77,15 +81,6 @@
             self.connect(space, mode, twophase)
         return space.wrap(self)
 
-    descr_new.unwrap_spec = [ObjSpace, W_Root,
-                             W_Root, W_Root, W_Root,
-                             int, int,
-                             W_Root,
-                             bool, bool, bool,
-                             W_Root,
-                             int,
-                             W_Root]
-
     def __del__(self):
         if self.release:
             roci.OCITransRollback(
@@ -418,7 +413,6 @@
             roci.OCIHandleFree(self.handle, roci.OCI_HTYPE_SVCCTX)
 
         self.handle = lltype.nullptr(roci.OCISvcCtx.TO)
-    close.unwrap_spec = ['self', ObjSpace]
 
     def commit(self, space):
         # make sure we are actually connected
@@ -431,7 +425,6 @@
             status, "Connection_Commit()")
 
         self.commitMode = roci.OCI_DEFAULT
-    commit.unwrap_spec = ['self', ObjSpace]
 
     def rollback(self, space):
         # make sure we are actually connected
@@ -442,11 +435,9 @@
             roci.OCI_DEFAULT)
         self.environment.checkForError(
             status, "Connection_Rollback()")
-    rollback.unwrap_spec = ['self', ObjSpace]
 
     def newCursor(self, space):
         return space.wrap(W_Cursor(space, self))
-    newCursor.unwrap_spec = ['self', ObjSpace]
 
     def _getCharacterSetName(self, space, attribute):
         # get character set id
@@ -525,8 +516,7 @@
 
 W_Connection.typedef = TypeDef(
     "Connection",
-    __new__ = interp2app(W_Connection.descr_new.im_func,
-                         unwrap_spec=W_Connection.descr_new.unwrap_spec),
+    __new__ = interp2app(W_Connection.descr_new.im_func),
     username = interp_attrproperty_w('w_username', W_Connection),
     password = interp_attrproperty_w('w_password', W_Connection),
     tnsentry = interp_attrproperty_w('w_tnsentry', W_Connection),

diff --git a/pypy/module/oracle/interp_cursor.py b/pypy/module/oracle/interp_cursor.py
--- a/pypy/module/oracle/interp_cursor.py
+++ b/pypy/module/oracle/interp_cursor.py
@@ -1,9 +1,8 @@
 from pypy.interpreter.baseobjspace import Wrappable
-from pypy.interpreter.gateway import ObjSpace, W_Root, NoneNotWrapped
-from pypy.interpreter.argument import Arguments
+from pypy.interpreter.gateway import NoneNotWrapped
 from pypy.interpreter.typedef import TypeDef, GetSetProperty
 from pypy.interpreter.typedef import interp_attrproperty, interp_attrproperty_w
-from pypy.interpreter.gateway import interp2app
+from pypy.interpreter.gateway import interp2app, unwrap_spec
 from pypy.interpreter.error import OperationError
 from pypy.rpython.lltypesystem import rffi, lltype
 
@@ -68,7 +67,6 @@
         self._checkOpen(space)
 
         return self._execute(space, w_stmt, w_vars)
-    execute.unwrap_spec = ['self', ObjSpace, W_Root, Arguments]
 
     def prepare(self, space, w_stmt, w_tag=None):
         # make sure the cursor is open
@@ -76,7 +74,6 @@
 
         # prepare the statement
         self._internalPrepare(space, w_stmt, w_tag)
-    prepare.unwrap_spec = ['self', ObjSpace, W_Root, W_Root]
 
     def _execute(self, space, w_stmt, w_vars):
 
@@ -153,7 +150,6 @@
         # zero since Oracle raises an error otherwise
         if numrows > 0:
             self._internalExecute(space, numIters=numrows)
-    executemany.unwrap_spec = ['self', ObjSpace, W_Root, W_Root]
 
     def close(self, space):
         # make sure we are actually open
@@ -164,8 +160,8 @@
 
         self.isOpen = False
         self.handle = lltype.nullptr(roci.OCIStmt.TO)
-    close.unwrap_spec = ['self', ObjSpace]
 
+    @unwrap_spec(name=str)
     def callfunc(self, space, name, w_returnType, w_parameters=None):
         retvar = interp_variable.newVariableByType(space, self, w_returnType, 1)
         if space.is_w(w_parameters, space.w_None):
@@ -175,8 +171,8 @@
 
         # determine the results
         return retvar.getValue(space, 0)
-    callfunc.unwrap_spec = ['self', ObjSpace, str, W_Root, W_Root]
 
+    @unwrap_spec(name=str)
     def callproc(self, space, name, w_parameters=None):
         if space.is_w(w_parameters, space.w_None):
             w_parameters = None
@@ -191,8 +187,6 @@
                 ret_w.append(v.getValue(space, 0))
         return space.newlist(ret_w)
 
-    callproc.unwrap_spec = ['self', ObjSpace, str, W_Root]
-
     def _call(self, space, name, retvar, w_args):
         # determine the number of arguments passed
         if w_args:
@@ -768,7 +762,6 @@
             return self._createRow(space)
 
         return space.w_None
-    fetchone.unwrap_spec = ['self', ObjSpace]
 
     def fetchmany(self, space, w_numRows=NoneNotWrapped):
         if w_numRows is not None:
@@ -780,19 +773,16 @@
         self._verifyFetch(space)
 
         return self._multiFetch(space, limit=numRows)
-    fetchmany.unwrap_spec = ['self', ObjSpace, W_Root]
 
     def fetchall(self, space):
         # verify fetch can be performed
         self._verifyFetch(space)
 
         return self._multiFetch(space, limit=0)
-    fetchall.unwrap_spec = ['self', ObjSpace]
 
     def descr_iter(self, space):
         self._verifyFetch(space)
         return space.wrap(self)
-    descr_iter.unwrap_spec = ['self', ObjSpace]
 
     def descr_next(self, space):
         # verify fetch can be performed
@@ -803,7 +793,6 @@
             return self._createRow(space)
 
         raise OperationError(space.w_StopIteration, space.w_None)
-    descr_next.unwrap_spec = ['self', ObjSpace]
 
     def _moreRows(self, space):
         if self.rowNum < self.actualRows:
@@ -964,8 +953,8 @@
         if nbElements:
             _, names = self._get_bind_info(space, nbElements)
         return space.newlist(names)
-    bindnames.unwrap_spec = ['self', ObjSpace]
 
+    @unwrap_spec(size=int)
     def var(self, space, w_type, size=0, w_arraysize=None,
             w_inconverter=None, w_outconverter=None):
         if space.is_w(w_arraysize, space.w_None):
@@ -984,8 +973,8 @@
         var.w_outconverter = w_outconverter
 
         return space.wrap(var)
-    var.unwrap_spec = ['self', ObjSpace, W_Root, int, W_Root, W_Root, W_Root]
 
+    @unwrap_spec(size=int)
     def arrayvar(self, space, w_type, w_value, size=0):
         # determine the type of variable
         varType = interp_variable.typeByPythonType(space, self, w_type)
@@ -1011,7 +1000,6 @@
             var.setArrayValue(space, w_value)
 
         return var
-    arrayvar.unwrap_spec = ['self', ObjSpace, W_Root, W_Root, int]
 
     def setinputsizes(self, space, __args__):
         args_w, kw_w = __args__.unpack()
@@ -1051,12 +1039,11 @@
                         space, self, w_value, self.bindArraySize)
                 self.bindList[i] = var
             return space.newlist(self.bindList)
-    setinputsizes.unwrap_spec = ['self', ObjSpace, Arguments]
 
+    @unwrap_spec(outputSize=int, outputSizeColumn=int)
     def setoutputsize(self, space, outputSize, outputSizeColumn=-1):
         self.outputSize = outputSize
         self.outputSizeColumn = outputSizeColumn
-    setoutputsize.unwrap_spec = ['self', ObjSpace, int, int]
 
 
     def arraysize_get(space, self):

From commits-noreply at bitbucket.org  Wed Feb 16 19:20:14 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 19:20:14 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: Simplify unwrap_spec for
	parser, pyexpat, pypyjit
Message-ID: <20110216182014.6077A282C1D@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42081:4de80533c51b
Date: 2011-02-16 17:55 +0100
http://bitbucket.org/pypy/pypy/changeset/4de80533c51b/

Log:	Simplify unwrap_spec for parser, pyexpat, pypyjit

diff --git a/pypy/module/pyexpat/interp_pyexpat.py b/pypy/module/pyexpat/interp_pyexpat.py
--- a/pypy/module/pyexpat/interp_pyexpat.py
+++ b/pypy/module/pyexpat/interp_pyexpat.py
@@ -1,7 +1,7 @@
 from pypy.interpreter.baseobjspace import Wrappable
 from pypy.interpreter.typedef import TypeDef, GetSetProperty
-from pypy.interpreter.gateway import ObjSpace, W_Root, NoneNotWrapped
-from pypy.interpreter.gateway import interp2app
+from pypy.interpreter.gateway import NoneNotWrapped
+from pypy.interpreter.gateway import interp2app, unwrap_spec
 from pypy.interpreter.error import OperationError
 from pypy.objspace.descroperation import object_setattr
 from pypy.rpython.lltypesystem import rffi, lltype
@@ -357,6 +357,7 @@
             global_storage.free_nonmoving_id(
                 rffi.cast(lltype.Signed, self.itself))
 
+    @unwrap_spec(flag=int)
     def SetParamEntityParsing(self, space, flag):
         """SetParamEntityParsing(flag) -> success
 Controls parsing of parameter entities (including the external DTD
@@ -365,7 +366,6 @@
 XML_PARAM_ENTITY_PARSING_ALWAYS. Returns true if setting the flag
 was successful."""
         XML_SetParamEntityParsing(self.itself, flag)
-    SetParamEntityParsing.unwrap_spec = ['self', ObjSpace, int]
 
     def UseForeignDTD(self, space, w_flag=True):
         """UseForeignDTD([flag])
@@ -376,7 +376,6 @@
 'flag' defaults to True if not provided."""
         flag = space.is_true(w_flag)
         XML_UseForeignDTD(self.itself, flag)
-    UseForeignDTD.unwrap_spec = ['self', ObjSpace, W_Root]
 
     # Handlers management
 
@@ -499,6 +498,7 @@
         return True
 
 
+    @unwrap_spec(name=str)
     def setattr(self, space, name, w_value):
         if name == "namespace_prefixes":
             XML_SetReturnNSTriplet(self.itself, space.int_w(w_value))
@@ -513,15 +513,15 @@
         return space.call_function(
             object_setattr(space),
             space.wrap(self), space.wrap(name), w_value)
-    setattr.unwrap_spec = ['self', ObjSpace, str, W_Root]
 
     # Parse methods
 
+    @unwrap_spec(data=str, isfinal=bool)
     def Parse(self, space, data, isfinal=False):
         """Parse(data[, isfinal])
 Parse XML data.  `isfinal' should be true at end of input."""
 
-        res = XML_Parse(self.itself, data, len(data), bool(isfinal))
+        res = XML_Parse(self.itself, data, len(data), isfinal)
         if self._exc_info:
             e = self._exc_info
             self._exc_info = None
@@ -531,7 +531,6 @@
             raise exc
         self.flush_character_buffer(space)
         return space.wrap(res)
-    Parse.unwrap_spec = ['self', ObjSpace, str, int]
 
     def ParseFile(self, space, w_file):
         """ParseFile(file)
@@ -540,11 +539,10 @@
         w_data = space.call_method(w_file, 'read')
         data = space.str_w(w_data)
         return self.Parse(space, data, isfinal=True)
-    ParseFile.unwrap_spec = ['self', ObjSpace, W_Root]
 
+    @unwrap_spec(base=str)
     def SetBase(self, space, base):
         XML_SetBase(self.itself, base)
-    SetBase.unwrap_spec = ['self', ObjSpace, str]
 
     def ExternalEntityParserCreate(self, space, w_context, w_encoding=None):
         """ExternalEntityParserCreate(context[, encoding])
@@ -572,7 +570,6 @@
             parser.handlers[i] = self.handlers[i]
 
         return space.wrap(parser)
-    ExternalEntityParserCreate.unwrap_spec = ['self', ObjSpace, W_Root, W_Root]
 
     def flush_character_buffer(self, space):
         if not self.buffer_w:
@@ -676,9 +673,7 @@
     CurrentColumnNumber = GetSetProperty(W_XMLParserType.descr_ErrorColumnNumber, cls=W_XMLParserType),
     CurrentByteIndex = GetSetProperty(W_XMLParserType.descr_ErrorByteIndex, cls=W_XMLParserType),
 
-    **dict((name, interp2app(getattr(W_XMLParserType, name),
-                             unwrap_spec=getattr(W_XMLParserType,
-                                                 name).unwrap_spec))
+    **dict((name, interp2app(getattr(W_XMLParserType, name)))
            for name in XMLParser_methods)
     )
 
@@ -740,11 +735,10 @@
         parser.itself, UnknownEncodingHandlerData_callback,
         rffi.cast(rffi.VOIDP, parser.id))
     return space.wrap(parser)
-ParserCreate.unwrap_spec = [ObjSpace, W_Root, W_Root, W_Root]
 
+ at unwrap_spec(code=int)
 def ErrorString(space, code):
     """ErrorString(errno) -> string
 Returns string error for given number."""
     return space.wrap(rffi.charp2str(XML_ErrorString(code)))
-ErrorString.unwrap_spec = [ObjSpace, int]
 

diff --git a/pypy/module/parser/pyparser.py b/pypy/module/parser/pyparser.py
--- a/pypy/module/parser/pyparser.py
+++ b/pypy/module/parser/pyparser.py
@@ -1,7 +1,6 @@
-from pypy.interpreter.baseobjspace import ObjSpace, Wrappable
+from pypy.interpreter.baseobjspace import Wrappable
 from pypy.interpreter.typedef import TypeDef
-from pypy.interpreter.gateway import interp2app
-from pypy.interpreter.argument import Arguments
+from pypy.interpreter.gateway import interp2app, unwrap_spec
 from pypy.interpreter.error import OperationError
 from pypy.interpreter.pyparser import pyparse, pygram, error
 from pypy.interpreter.astcompiler.astbuilder import ast_from_node
@@ -36,22 +35,21 @@
 
     def descr_issuite(self, space):
         return space.wrap(self.tree.type == pygram.syms.file_input)
-    descr_issuite.unwrap_spec = ["self", ObjSpace]
 
     def descr_isexpr(self, space):
         return space.wrap(self.tree.type == pygram.syms.eval_input)
-    descr_isexpr.unwrap_spec = ["self", ObjSpace]
 
+    @unwrap_spec(line_info=bool, col_info=bool)
     def descr_totuple(self, space, line_info=False, col_info=False):
         return self._build_app_tree(space, self.tree, space.newtuple,
                                     line_info, col_info)
-    descr_totuple.unwrap_spec = ["self", ObjSpace, bool, bool]
 
+    @unwrap_spec(line_info=bool, col_info=bool)
     def descr_tolist(self, space, line_info=False, col_info=False):
         return self._build_app_tree(space, self.tree, space.newlist,
                                     line_info, col_info)
-    descr_tolist.unwrap_spec = ["self", ObjSpace, bool, bool]
 
+    @unwrap_spec(filename=str)
     def descr_compile(self, space, filename="<syntax-tree>"):
         info = pyparse.CompileInfo(filename, self.mode)
         try:
@@ -64,7 +62,6 @@
             raise OperationError(space.w_SyntaxError,
                                  e.wrap_info(space))
         return space.wrap(result)
-    descr_compile.unwrap_spec = ["self", ObjSpace, str]
 
 STType.typedef = TypeDef("parser.st",
     issuite=interp2app(STType.descr_issuite),
@@ -89,32 +86,32 @@
     return space.wrap(STType(tree, mode))
 
 
+ at unwrap_spec(source=str)
 def suite(space, source):
     return parse_python(space, source, 'exec')
-suite.unwrap_spec = [ObjSpace, str]
 
 
+ at unwrap_spec(source=str)
 def expr(space, source):
     return parse_python(space, source, 'eval')
-expr.unwrap_spec = [ObjSpace, str]
 
 
+ at unwrap_spec(st=STType)
 def isexpr(space, st):
     return space.call_method(st, "isexpr")
-isexpr.unwrap_spec = [ObjSpace, STType]
 
+ at unwrap_spec(st=STType)
 def issuite(space, st):
     return space.call_method(st, "issuite")
-issuite.unwrap_spec = [ObjSpace, STType]
 
+ at unwrap_spec(st=STType)
 def st2tuple(space, st, __args__):
     return space.call_args(space.getattr(st, space.wrap("totuple")), __args__)
-st2tuple.unwrap_spec = [ObjSpace, STType, Arguments]
 
+ at unwrap_spec(st=STType)
 def st2list(space, st, __args__):
     return space.call_args(space.getattr(st, space.wrap("tolist")), __args__)
-st2list.unwrap_spec = [ObjSpace, STType, Arguments]
 
+ at unwrap_spec(st=STType)
 def compilest(space, st, __args__):
     return space.call_args(space.getattr(st, space.wrap("compile")), __args__)
-compilest.unwrap_spec = [ObjSpace, STType, Arguments]

diff --git a/pypy/module/pypyjit/interp_jit.py b/pypy/module/pypyjit/interp_jit.py
--- a/pypy/module/pypyjit/interp_jit.py
+++ b/pypy/module/pypyjit/interp_jit.py
@@ -9,7 +9,6 @@
 from pypy.rlib.jit import current_trace_length
 import pypy.interpreter.pyopcode   # for side-effects
 from pypy.interpreter.error import OperationError, operationerrfmt
-from pypy.interpreter.gateway import ObjSpace, Arguments, W_Root
 from pypy.interpreter.pycode import PyCode, CO_GENERATOR
 from pypy.interpreter.pyframe import PyFrame
 from pypy.interpreter.pyopcode import ExitFrame
@@ -119,13 +118,13 @@
 #
 # Public interface    
 
-def set_param(space, args):
+def set_param(space, __args__):
     '''Configure the tunable JIT parameters.
         * set_param(name=value, ...)            # as keyword arguments
         * set_param("name=value,name=value")    # as a user-supplied string
     '''
     # XXXXXXXXX
-    args_w, kwds_w = args.unpack()
+    args_w, kwds_w = __args__.unpack()
     if len(args_w) > 1:
         msg = "set_param() takes at most 1 non-keyword argument, %d given"
         raise operationerrfmt(space.w_TypeError, msg, len(args_w))
@@ -144,11 +143,8 @@
             raise operationerrfmt(space.w_TypeError,
                                   "no JIT parameter '%s'", key)
 
-set_param.unwrap_spec = [ObjSpace, Arguments]
-
 @dont_look_inside
-def residual_call(space, w_callable, args):
+def residual_call(space, w_callable, __args__):
     '''For testing.  Invokes callable(...), but without letting
     the JIT follow the call.'''
-    return space.call_args(w_callable, args)
-residual_call.unwrap_spec = [ObjSpace, W_Root, Arguments]
+    return space.call_args(w_callable, __args__)

From commits-noreply at bitbucket.org  Wed Feb 16 19:20:15 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 19:20:15 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: update posix module to the new
	unwrap_spec
Message-ID: <20110216182015.D53F4282BF7@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42082:a3a837102fdf
Date: 2011-02-16 18:18 +0100
http://bitbucket.org/pypy/pypy/changeset/a3a837102fdf/

Log:	update posix module to the new unwrap_spec

diff --git a/pypy/module/posix/interp_posix.py b/pypy/module/posix/interp_posix.py
--- a/pypy/module/posix/interp_posix.py
+++ b/pypy/module/posix/interp_posix.py
@@ -1,4 +1,4 @@
-from pypy.interpreter.gateway import ObjSpace, W_Root, NoneNotWrapped
+from pypy.interpreter.gateway import unwrap_spec, NoneNotWrapped
 from pypy.rlib import rposix, objectmodel
 from pypy.rlib.objectmodel import specialize
 from pypy.rlib.rarithmetic import r_longlong
@@ -14,6 +14,9 @@
 import os, sys
 _WIN = sys.platform == 'win32'
 
+c_int = "c_int"
+c_nonnegint = "c_nonnegint"
+
 class FileEncoder:
     def __init__(self, space, w_obj):
         self.space = space
@@ -76,6 +79,7 @@
                 return func(fname1, fname2, *args)
     return dispatch
 
+ at unwrap_spec(flag=c_int, mode=c_int)
 def open(space, w_fname, flag, mode=0777):
     """Open a file (for low level IO).
 Return a file descriptor (a small integer)."""
@@ -85,8 +89,8 @@
     except OSError, e: 
         raise wrap_oserror2(space, e, w_fname)
     return space.wrap(fd)
-open.unwrap_spec = [ObjSpace, W_Root, "c_int", "c_int"]
 
+ at unwrap_spec(fd=c_int, pos=r_longlong, how=c_int)
 def lseek(space, fd, pos, how):
     """Set the current position of a file descriptor.  Return the new position.
 If how == 0, 'pos' is relative to the start of the file; if how == 1, to the
@@ -97,8 +101,8 @@
         raise wrap_oserror(space, e) 
     else: 
         return space.wrap(pos) 
-lseek.unwrap_spec = [ObjSpace, "c_int", r_longlong, "c_int"]
 
+ at unwrap_spec(fd=c_int)
 def isatty(space, fd):
     """Return True if 'fd' is an open file descriptor connected to the
 slave end of a terminal."""
@@ -108,8 +112,8 @@
         raise wrap_oserror(space, e) 
     else:  
         return space.wrap(res) 
-isatty.unwrap_spec = [ObjSpace, "c_int"]
 
+ at unwrap_spec(fd=c_int, buffersize=int)
 def read(space, fd, buffersize):
     """Read data from a file descriptor."""
     try: 
@@ -118,8 +122,8 @@
         raise wrap_oserror(space, e) 
     else: 
         return space.wrap(s) 
-read.unwrap_spec = [ObjSpace, "c_int", int]
 
+ at unwrap_spec(fd=c_int, data='bufferstr')
 def write(space, fd, data):
     """Write a string to a file descriptor.  Return the number of bytes
 actually written, which may be smaller than len(data)."""
@@ -129,21 +133,21 @@
         raise wrap_oserror(space, e) 
     else: 
         return space.wrap(res) 
-write.unwrap_spec = [ObjSpace, "c_int", 'bufferstr']
 
+ at unwrap_spec(fd=c_int)
 def close(space, fd):
     """Close a file descriptor (for low level IO)."""
     try: 
         os.close(fd)
     except OSError, e: 
         raise wrap_oserror(space, e) 
-close.unwrap_spec = [ObjSpace, "c_int"]
 
+ at unwrap_spec(fd_low=c_int, fd_high=c_int)
 def closerange(fd_low, fd_high):
     """Closes all file descriptors in [fd_low, fd_high), ignoring errors."""
     rposix.closerange(fd_low, fd_high)
-closerange.unwrap_spec = ["c_int", "c_int"]
 
+ at unwrap_spec(fd=c_int, length=r_longlong)
 def ftruncate(space, fd, length):
     """Truncate a file to a specified length."""
     try:
@@ -158,7 +162,6 @@
         raise AssertionError
     except OSError, e: 
         raise wrap_oserror(space, e) 
-ftruncate.unwrap_spec = [ObjSpace, "c_int", r_longlong]
 
 def fsync(space, w_fd):
     """Force write of file with filedescriptor to disk."""
@@ -167,7 +170,6 @@
         os.fsync(fd)
     except OSError, e:
         raise wrap_oserror(space, e)
-fsync.unwrap_spec = [ObjSpace, W_Root]
 
 def fdatasync(space, w_fd):
     """Force write of file with filedescriptor to disk.
@@ -177,7 +179,6 @@
         os.fdatasync(fd)
     except OSError, e:
         raise wrap_oserror(space, e)
-fdatasync.unwrap_spec = [ObjSpace, W_Root]
 
 def fchdir(space, w_fd):
     """Change to the directory of the given file descriptor.  fildes must be
@@ -187,7 +188,6 @@
         os.fchdir(fd)
     except OSError, e:
         raise wrap_oserror(space, e)
-fchdir.unwrap_spec = [ObjSpace, W_Root]
 
 # ____________________________________________________________
 
@@ -236,6 +236,7 @@
                                   space.wrap('stat_result'))
     return space.call_function(w_stat_result, w_tuple, w_keywords)
 
+ at unwrap_spec(fd=c_int)
 def fstat(space, fd):
     """Perform a stat system call on the file referenced to by an open
 file descriptor."""
@@ -245,7 +246,6 @@
         raise wrap_oserror(space, e) 
     else:
         return build_stat_result(space, st)
-fstat.unwrap_spec = [ObjSpace, "c_int"]
 
 def stat(space, w_path):
     """Perform a stat system call on the given path.  Return an object
@@ -268,7 +268,6 @@
         raise wrap_oserror2(space, e, w_path)
     else: 
         return build_stat_result(space, st)
-stat.unwrap_spec = [ObjSpace, W_Root]
 
 def lstat(space, w_path):
     "Like stat(path), but do no follow symbolic links."
@@ -278,7 +277,6 @@
         raise wrap_oserror2(space, e, w_path)
     else:
         return build_stat_result(space, st)
-lstat.unwrap_spec = [ObjSpace, W_Root]
 
 class StatState(object):
     def __init__(self, space):
@@ -298,8 +296,8 @@
         return space.wrap(state.stat_float_times)
     else:
         state.stat_float_times = space.bool_w(w_value)
-stat_float_times.unwrap_spec = [ObjSpace, W_Root]
 
+ at unwrap_spec(fd=c_int)
 def dup(space, fd):
     """Create a copy of the file descriptor.  Return the new file
 descriptor."""
@@ -309,16 +307,16 @@
         raise wrap_oserror(space, e) 
     else:
         return space.wrap(newfd)
-dup.unwrap_spec = [ObjSpace, "c_int"]
 
+ at unwrap_spec(old_fd=c_int, new_fd=c_int)
 def dup2(space, old_fd, new_fd):
     """Duplicate a file descriptor."""
     try:
         os.dup2(old_fd, new_fd)
     except OSError, e: 
         raise wrap_oserror(space, e) 
-dup2.unwrap_spec = [ObjSpace, "c_int", "c_int"]
 
+ at unwrap_spec(mode=c_int)
 def access(space, w_path, mode):
     """
     access(path, mode) -> 1 if granted, 0 otherwise
@@ -335,7 +333,6 @@
         raise wrap_oserror2(space, e, w_path)
     else:
         return space.wrap(ok)
-access.unwrap_spec = [ObjSpace, W_Root, "c_int"]
 
 
 def times(space):
@@ -354,8 +351,8 @@
                                space.wrap(times[2]),
                                space.wrap(times[3]),
                                space.wrap(times[4])])
-times.unwrap_spec = [ObjSpace]
 
+ at unwrap_spec(cmd=str)
 def system(space, cmd):
     """Execute the command (a string) in a subshell."""
     try:
@@ -364,7 +361,6 @@
         raise wrap_oserror(space, e) 
     else: 
         return space.wrap(rc)
-system.unwrap_spec = [ObjSpace, str]
 
 def unlink(space, w_path):
     """Remove a file (same as remove(path))."""
@@ -372,7 +368,6 @@
         dispatch_filename(rposix.unlink)(space, w_path)
     except OSError, e:
         raise wrap_oserror2(space, e, w_path)
-unlink.unwrap_spec = [ObjSpace, W_Root]
 
 def remove(space, w_path):
     """Remove a file (same as unlink(path))."""
@@ -380,7 +375,6 @@
         dispatch_filename(rposix.unlink)(space, w_path)
     except OSError, e:
         raise wrap_oserror2(space, e, w_path)
-remove.unwrap_spec = [ObjSpace, W_Root]
 
 def _getfullpathname(space, w_path):
     """helper for ntpath.abspath """
@@ -397,7 +391,6 @@
         raise wrap_oserror2(space, e, w_path)
     else:
         return w_fullpath
-_getfullpathname.unwrap_spec = [ObjSpace, W_Root]
 
 def getcwd(space):
     """Return the current working directory."""
@@ -407,7 +400,6 @@
         raise wrap_oserror(space, e) 
     else: 
         return space.wrap(cur)
-getcwd.unwrap_spec = [ObjSpace]
 
 if sys.platform == 'win32':
     def getcwdu(space):
@@ -424,7 +416,6 @@
         filesystemencoding = space.sys.filesystemencoding
         return space.call_method(getcwd(space), 'decode',
                                  space.wrap(filesystemencoding))
-getcwdu.unwrap_spec = [ObjSpace]
 
 def chdir(space, w_path):
     """Change the current working directory to the specified path."""
@@ -432,15 +423,14 @@
         dispatch_filename(rposix.chdir)(space, w_path)
     except OSError, e:
         raise wrap_oserror2(space, e, w_path)
-chdir.unwrap_spec = [ObjSpace, W_Root]
 
+ at unwrap_spec(mode=c_int)
 def mkdir(space, w_path, mode=0777):
     """Create a directory."""
     try:
         dispatch_filename(rposix.mkdir)(space, w_path, mode)
     except OSError, e:
         raise wrap_oserror2(space, e, w_path)
-mkdir.unwrap_spec = [ObjSpace, W_Root, "c_int"]
 
 def rmdir(space, w_path):
     """Remove a directory."""
@@ -448,8 +438,8 @@
         dispatch_filename(rposix.rmdir)(space, w_path)
     except OSError, e:
         raise wrap_oserror2(space, e, w_path)
-rmdir.unwrap_spec = [ObjSpace, W_Root]
 
+ at unwrap_spec(errno=c_int)
 def strerror(space, errno):
     """Translate an error code to a message string."""
     try:
@@ -458,7 +448,6 @@
         raise OperationError(space.w_ValueError,
                              space.wrap("strerror() argument out of range"))
     return space.wrap(text)
-strerror.unwrap_spec = [ObjSpace, "c_int"]
 
 # ____________________________________________________________
 
@@ -497,14 +486,15 @@
     for key, value in os.environ.items():
         space.setitem(w_env, space.wrap(key), space.wrap(value))
 
+ at unwrap_spec(name=str, value=str)
 def putenv(space, name, value):
     """Change or add an environment variable."""
     try:
         os.environ[name] = value
     except OSError, e:
         raise wrap_oserror(space, e) 
-putenv.unwrap_spec = [ObjSpace, str, str]
 
+ at unwrap_spec(name=str)
 def unsetenv(space, name):
     """Delete an environment variable."""
     try:
@@ -513,7 +503,6 @@
         pass
     except OSError, e:
         raise wrap_oserror(space, e) 
-unsetenv.unwrap_spec = [ObjSpace, str]
 
 
 def listdir(space, w_dirname):
@@ -535,7 +524,6 @@
     except OSError, e:
         raise wrap_oserror2(space, e, w_dirname)
     return space.newlist(result_w)
-listdir.unwrap_spec = [ObjSpace, W_Root]
 
 def pipe(space):
     "Create a pipe.  Returns (read_end, write_end)."
@@ -544,15 +532,14 @@
     except OSError, e: 
         raise wrap_oserror(space, e) 
     return space.newtuple([space.wrap(fd1), space.wrap(fd2)])
-pipe.unwrap_spec = [ObjSpace]
 
+ at unwrap_spec(mode=c_int)
 def chmod(space, w_path, mode):
     "Change the access permissions of a file."
     try:
         dispatch_filename(rposix.chmod)(space, w_path, mode)
     except OSError, e:
         raise wrap_oserror2(space, e, w_path)
-chmod.unwrap_spec = [ObjSpace, W_Root, "c_int"]
 
 def rename(space, w_old, w_new):
     "Rename a file or directory."
@@ -560,16 +547,16 @@
         dispatch_filename_2(rposix.rename)(space, w_old, w_new)
     except OSError, e:
         raise wrap_oserror(space, e) 
-rename.unwrap_spec = [ObjSpace, W_Root, W_Root]
 
+ at unwrap_spec(mode=c_int)
 def mkfifo(space, w_filename, mode=0666):
     """Create a FIFO (a POSIX named pipe)."""
     try:
         dispatch_filename(rposix.mkfifo)(space, w_filename, mode)
     except OSError, e: 
         raise wrap_oserror2(space, e, w_filename)
-mkfifo.unwrap_spec = [ObjSpace, W_Root, "c_int"]
 
+ at unwrap_spec(mode=c_int, device=c_int)
 def mknod(space, w_filename, mode=0600, device=0):
     """Create a filesystem node (file, device special file or named pipe)
 named filename. mode specifies both the permissions to use and the
@@ -581,13 +568,12 @@
         dispatch_filename(rposix.mknod)(space, w_filename, mode, device)
     except OSError, e: 
         raise wrap_oserror2(space, e, w_filename)
-mknod.unwrap_spec = [ObjSpace, W_Root, "c_int", "c_int"]
 
+ at unwrap_spec(mask=c_int)
 def umask(space, mask):
     "Set the current numeric umask and return the previous umask."
     prevmask = os.umask(mask)
     return space.wrap(prevmask)
-umask.unwrap_spec = [ObjSpace, "c_int"]
 
 def getpid(space):
     "Return the current process id."
@@ -596,47 +582,46 @@
     except OSError, e: 
         raise wrap_oserror(space, e) 
     return space.wrap(pid)
-getpid.unwrap_spec = [ObjSpace]
 
+ at unwrap_spec(pid=c_int, sig=c_int)
 def kill(space, pid, sig):
     "Kill a process with a signal."
     try:
         os.kill(pid, sig)
     except OSError, e:
         raise wrap_oserror(space, e)
-kill.unwrap_spec = [ObjSpace, "c_int", "c_int"]
 
+ at unwrap_spec(pgid=c_int, sig=c_int)
 def killpg(space, pgid, sig):
     "Kill a process group with a signal."
     try:
         os.killpg(pgid, sig)
     except OSError, e:
         raise wrap_oserror(space, e)
-killpg.unwrap_spec = [ObjSpace, "c_int", "c_int"]
 
 def abort(space):
     """Abort the interpreter immediately.  This 'dumps core' or otherwise fails
 in the hardest way possible on the hosting operating system."""
     import signal
     os.kill(os.getpid(), signal.SIGABRT)
-abort.unwrap_spec = [ObjSpace]
 
+ at unwrap_spec(src=str, dst=str)
 def link(space, src, dst):
     "Create a hard link to a file."
     try: 
         os.link(src, dst)
     except OSError, e: 
         raise wrap_oserror(space, e) 
-link.unwrap_spec = [ObjSpace, str, str]
 
+ at unwrap_spec(src=str, dst=str)
 def symlink(space, src, dst):
     "Create a symbolic link pointing to src named dst."
     try: 
         os.symlink(src, dst)
     except OSError, e: 
         raise wrap_oserror(space, e) 
-symlink.unwrap_spec = [ObjSpace, str, str]
 
+ at unwrap_spec(path=str)
 def readlink(space, path):
     "Return a string representing the path to which the symbolic link points."
     try:
@@ -644,7 +629,6 @@
     except OSError, e: 
         raise wrap_oserror(space, e, path)
     return space.wrap(result)
-readlink.unwrap_spec = [ObjSpace, str]
 
 before_fork_hooks = []
 after_fork_child_hooks = []
@@ -706,6 +690,7 @@
     return space.newtuple([space.wrap(pid),
                            space.wrap(master_fd)])
 
+ at unwrap_spec(pid=c_int, options=c_int)
 def waitpid(space, pid, options):
     """ waitpid(pid, options) -> (pid, status)
     
@@ -716,12 +701,12 @@
     except OSError, e: 
         raise wrap_oserror(space, e) 
     return space.newtuple([space.wrap(pid), space.wrap(status)])
-waitpid.unwrap_spec = [ObjSpace, "c_int", "c_int"]
 
+ at unwrap_spec(status=c_int)
 def _exit(space, status):
     os._exit(status)
-_exit.unwrap_spec = [ObjSpace, "c_int"]
 
+ at unwrap_spec(command=str)
 def execv(space, command, w_args):
     """ execv(path, args)
 
@@ -745,8 +730,8 @@
         os.execv(command, args)
     except OSError, e:
         raise wrap_oserror(space, e)
-execv.unwrap_spec = [ObjSpace, str, W_Root]
 
+ at unwrap_spec(command=str)
 def execve(space, command, w_args, w_env):
     """ execve(path, args, env)
 
@@ -766,8 +751,8 @@
         os.execve(command, args, env)
     except OSError, e:
         raise wrap_oserror(space, e)
-execve.unwrap_spec = [ObjSpace, str, W_Root, W_Root]
 
+ at unwrap_spec(mode=int, path=str)
 def spawnv(space, mode, path, w_args):
     args = [space.str_w(w_arg) for w_arg in space.unpackiterable(w_args)]
     try:
@@ -775,7 +760,6 @@
     except OSError, e:
         raise wrap_oserror(space, e)
     return space.wrap(ret)
-spawnv.unwrap_spec = [ObjSpace, int, str, W_Root]
 
 def utime(space, w_path, w_tuple):
     """ utime(path, (atime, mtime))
@@ -804,7 +788,6 @@
         if not e.match(space, space.w_TypeError):
             raise
         raise OperationError(space.w_TypeError, space.wrap(msg))
-utime.unwrap_spec = [ObjSpace, W_Root, W_Root]
 
 def setsid(space):
     """setsid() -> pid
@@ -816,7 +799,6 @@
     except OSError, e:
         raise wrap_oserror(space, e)
     return space.wrap(result)
-setsid.unwrap_spec = [ObjSpace]
 
 def uname(space):
     """ uname() -> (sysname, nodename, release, version, machine)
@@ -829,7 +811,6 @@
         raise wrap_oserror(space, e)
     l_w = [space.wrap(i) for i in [r[0], r[1], r[2], r[3], r[4]]]
     return space.newtuple(l_w)
-uname.unwrap_spec = [ObjSpace]
 
 def getuid(space):
     """ getuid() -> uid
@@ -837,8 +818,8 @@
     Return the current process's user id.
     """
     return space.wrap(os.getuid())
-getuid.unwrap_spec = [ObjSpace]
 
+ at unwrap_spec(arg=c_nonnegint)
 def setuid(space, arg):
     """ setuid(uid)
 
@@ -849,8 +830,8 @@
     except OSError, e:
         raise wrap_oserror(space, e)
     return space.w_None
-setuid.unwrap_spec = [ObjSpace, "c_nonnegint"]
 
+ at unwrap_spec(arg=c_nonnegint)
 def seteuid(space, arg):
     """ seteuid(uid)
 
@@ -861,8 +842,8 @@
     except OSError, e:
         raise wrap_oserror(space, e)
     return space.w_None
-seteuid.unwrap_spec = [ObjSpace, "c_nonnegint"]
 
+ at unwrap_spec(arg=c_nonnegint)
 def setgid(space, arg):
     """ setgid(gid)
 
@@ -873,8 +854,8 @@
     except OSError, e:
         raise wrap_oserror(space, e)
     return space.w_None
-setgid.unwrap_spec = [ObjSpace, "c_nonnegint"]
 
+ at unwrap_spec(arg=c_nonnegint)
 def setegid(space, arg):
     """ setegid(gid)
 
@@ -885,8 +866,8 @@
     except OSError, e:
         raise wrap_oserror(space, e)
     return space.w_None
-setegid.unwrap_spec = [ObjSpace, "c_nonnegint"]
 
+ at unwrap_spec(path=str)
 def chroot(space, path):
     """ chroot(path)
 
@@ -897,7 +878,6 @@
     except OSError, e:
         raise wrap_oserror(space, e, path)
     return space.w_None
-chroot.unwrap_spec = [ObjSpace, str]
 
 def getgid(space):
     """ getgid() -> gid
@@ -905,7 +885,6 @@
     Return the current process's group id.
     """
     return space.wrap(os.getgid())
-getgid.unwrap_spec = [ObjSpace]
 
 def getegid(space):
     """ getegid() -> gid
@@ -913,7 +892,6 @@
     Return the current process's effective group id.
     """
     return space.wrap(os.getegid())
-getgid.unwrap_spec = [ObjSpace]
 
 def geteuid(space):
     """ geteuid() -> euid
@@ -921,7 +899,6 @@
     Return the current process's effective user id.
     """
     return space.wrap(os.geteuid())
-geteuid.unwrap_spec = [ObjSpace]
 
 def getgroups(space):
     """ getgroups() -> list of group IDs
@@ -929,7 +906,6 @@
     Return list of supplemental group IDs for the process.
     """
     return space.newlist([space.wrap(e) for e in os.getgroups()])
-getgroups.unwrap_spec = [ObjSpace]
 
 def getpgrp(space):
     """ getpgrp() -> pgrp
@@ -937,7 +913,6 @@
     Return the current process group id.
     """
     return space.wrap(os.getpgrp())
-getpgrp.unwrap_spec = [ObjSpace]
 
 def setpgrp(space):
     """ setpgrp()
@@ -949,7 +924,6 @@
     except OSError, e:
         raise wrap_oserror(space, e)
     return space.w_None        
-setpgrp.unwrap_spec = [ObjSpace]
 
 def getppid(space):
     """ getppid() -> ppid
@@ -957,8 +931,8 @@
     Return the parent's process id.
     """
     return space.wrap(os.getppid())
-getppid.unwrap_spec = [ObjSpace]
 
+ at unwrap_spec(pid=c_int)
 def getpgid(space, pid):
     """ getpgid(pid) -> pgid
 
@@ -969,8 +943,8 @@
     except OSError, e:
         raise wrap_oserror(space, e)
     return space.wrap(pgid)
-getpgid.unwrap_spec = [ObjSpace, "c_int"]
 
+ at unwrap_spec(pid=c_int, pgrp=c_int)
 def setpgid(space, pid, pgrp):
     """ setpgid(pid, pgrp)
 
@@ -981,8 +955,8 @@
     except OSError, e:
         raise wrap_oserror(space, e)
     return space.w_None                
-setpgid.unwrap_spec = [ObjSpace, "c_int", "c_int"]
 
+ at unwrap_spec(ruid=c_int, euid=c_int)
 def setreuid(space, ruid, euid):
     """ setreuid(ruid, euid)
 
@@ -993,8 +967,8 @@
     except OSError, e:
         raise wrap_oserror(space, e)
     return space.w_None                
-setreuid.unwrap_spec = [ObjSpace, "c_int", "c_int"]
 
+ at unwrap_spec(rgid=c_int, egid=c_int)
 def setregid(space, rgid, egid):
     """ setregid(rgid, egid)
 
@@ -1005,8 +979,8 @@
     except OSError, e:
         raise wrap_oserror(space, e)
     return space.w_None
-setregid.unwrap_spec = [ObjSpace, "c_int", "c_int"]
 
+ at unwrap_spec(pid=c_int)
 def getsid(space, pid):
     """ getsid(pid) -> sid
 
@@ -1017,7 +991,6 @@
     except OSError, e:
         raise wrap_oserror(space, e)
     return space.wrap(sid)
-getsid.unwrap_spec = [ObjSpace, "c_int"]
 
 def setsid(space):
     """ setsid()
@@ -1029,17 +1002,17 @@
     except OSError, e:
         raise wrap_oserror(space, e)
     return space.w_None                
-setsid.unwrap_spec = [ObjSpace]        
 
 def declare_new_w_star(name):
     if name in RegisterOs.w_star_returning_int:
+        @unwrap_spec(status=c_int)
         def WSTAR(space, status):
             return space.wrap(getattr(os, name)(status))
     else:
+        @unwrap_spec(status=c_int)
         def WSTAR(space, status):
             return space.newbool(getattr(os, name)(status))
     WSTAR.__doc__ = getattr(os, name).__doc__
-    WSTAR.unwrap_spec = [ObjSpace, "c_int"]
     WSTAR.func_name = name
     return WSTAR
 
@@ -1048,12 +1021,12 @@
         func = declare_new_w_star(name)
         globals()[name] = func
 
+ at unwrap_spec(fd=c_int)
 def ttyname(space, fd):
     try:
         return space.wrap(os.ttyname(fd))
     except OSError, e:
         raise wrap_oserror(space, e)
-ttyname.unwrap_spec = [ObjSpace, "c_int"]
 
 def confname_w(space, w_name, namespace):
     # XXX slightly non-nice, reuses the sysconf of the underlying os module
@@ -1070,31 +1043,30 @@
 def sysconf(space, w_name):
     num = confname_w(space, w_name, os.sysconf_names)
     return space.wrap(os.sysconf(num))
-sysconf.unwrap_spec = [ObjSpace, W_Root]
 
+ at unwrap_spec(fd=c_int)
 def fpathconf(space, fd, w_name):
     num = confname_w(space, w_name, os.pathconf_names)
     try:
         return space.wrap(os.fpathconf(fd, num))
     except OSError, e:
         raise wrap_oserror(space, e)
-fpathconf.unwrap_spec = [ObjSpace, 'c_int', W_Root]
 
+ at unwrap_spec(path=str, uid=c_int, gid=c_int)
 def chown(space, path, uid, gid):
     try:
         os.chown(path, uid, gid)
     except OSError, e:
         raise wrap_oserror(space, e, path)
     return space.w_None
-chown.unwrap_spec = [ObjSpace, str, "c_int", "c_int"]
 
+ at unwrap_spec(path=str, uid=c_int, gid=c_int)
 def lchown(space, path, uid, gid):
     try:
         os.lchown(path, uid, gid)
     except OSError, e:
         raise wrap_oserror(space, e, path)
     return space.w_None
-lchown.unwrap_spec = [ObjSpace, str, "c_int", "c_int"]
 
 def getloadavg(space):
     try:
@@ -1105,23 +1077,23 @@
     return space.newtuple([space.wrap(load[0]),
                            space.wrap(load[1]),
                            space.wrap(load[2])])
-getloadavg.unwrap_spec = [ObjSpace]
 
+ at unwrap_spec(major=c_int, minor=c_int)
 def makedev(space, major, minor):
     result = os.makedev(major, minor)
     return space.wrap(result)
-makedev.unwrap_spec = [ObjSpace, 'c_int', 'c_int']
 
+ at unwrap_spec(device=c_int)
 def major(space, device):
     result = os.major(device)
     return space.wrap(result)
-major.unwrap_spec = [ObjSpace, 'c_int']
 
+ at unwrap_spec(device=c_int)
 def minor(space, device):
     result = os.minor(device)
     return space.wrap(result)
-minor.unwrap_spec = [ObjSpace, 'c_int']
 
+ at unwrap_spec(inc=c_int)
 def nice(space, inc):
     "Decrease the priority of process by inc and return the new priority."
     try:
@@ -1129,7 +1101,6 @@
     except OSError, e:
         raise wrap_oserror(space, e)
     return space.wrap(res)
-nice.unwrap_spec = [ObjSpace, "c_int"]
 
 if _WIN:
     from pypy.rlib import rwin32
@@ -1165,6 +1136,7 @@
         calling_conv='win',
         compilation_info=eci)
 
+    @unwrap_spec(n=int)
     def win32_urandom(space, n):
         """urandom(n) -> str
 
@@ -1199,4 +1171,3 @@
                 rffi.charpsize2str(rffi.cast(rffi.CCHARP, buf), n))
         finally:
             lltype.free(buf, flavor='raw')
-    win32_urandom.unwrap_spec = [ObjSpace, int]

From commits-noreply at bitbucket.org  Wed Feb 16 19:20:18 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 19:20:18 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: update modules rctime, sys
Message-ID: <20110216182018.959042A202C@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42083:4d86f5056570
Date: 2011-02-16 18:22 +0100
http://bitbucket.org/pypy/pypy/changeset/4d86f5056570/

Log:	update modules rctime, sys

diff --git a/pypy/module/sys/vm.py b/pypy/module/sys/vm.py
--- a/pypy/module/sys/vm.py
+++ b/pypy/module/sys/vm.py
@@ -2,7 +2,7 @@
 Implementation of interpreter-level 'sys' routines.
 """
 from pypy.interpreter.error import OperationError
-from pypy.interpreter.gateway import ObjSpace, NoneNotWrapped
+from pypy.interpreter.gateway import unwrap_spec, NoneNotWrapped
 from pypy.rlib.runicode import MAXUNICODE
 from pypy.rlib import jit
 import sys
@@ -64,11 +64,11 @@
     """
     return space.wrap(space.sys.recursionlimit)
 
+ at unwrap_spec(interval=int)
 def setcheckinterval(space, interval):
     """Tell the Python interpreter to check for asynchronous events every
     n instructions.  This also affects how often thread switches occur."""
     space.actionflag.setcheckinterval(interval)
-setcheckinterval.unwrap_spec = [ObjSpace, int]
 
 def getcheckinterval(space):
     """Return the current check interval; see setcheckinterval()."""

diff --git a/pypy/module/sys/state.py b/pypy/module/sys/state.py
--- a/pypy/module/sys/state.py
+++ b/pypy/module/sys/state.py
@@ -3,7 +3,7 @@
 """
 import pypy
 from pypy.interpreter.error import OperationError
-from pypy.interpreter.gateway import ObjSpace
+from pypy.interpreter.gateway import unwrap_spec
 
 import sys, os, stat, errno
 
@@ -64,6 +64,7 @@
     #
     return importlist
 
+ at unwrap_spec(srcdir=str)
 def pypy_initial_path(space, srcdir):
     try:
         path = getinitialpath(srcdir)
@@ -76,8 +77,6 @@
                                         space.wrap(srcdir))
         return space.newlist([space.wrap(p) for p in path])
 
-pypy_initial_path.unwrap_spec = [ObjSpace, str]
-
 def get(space):
     return space.fromcache(State)
 

diff --git a/pypy/module/rctime/interp_time.py b/pypy/module/rctime/interp_time.py
--- a/pypy/module/rctime/interp_time.py
+++ b/pypy/module/rctime/interp_time.py
@@ -1,7 +1,7 @@
 from pypy.rpython.tool import rffi_platform as platform
 from pypy.rpython.lltypesystem import rffi
 from pypy.interpreter.error import OperationError, operationerrfmt
-from pypy.interpreter.baseobjspace import W_Root, ObjSpace
+from pypy.interpreter.gateway import unwrap_spec
 from pypy.rpython.lltypesystem import lltype
 from pypy.rlib.rarithmetic import ovfcheck_float_to_int
 from pypy.rlib import rposix
@@ -244,6 +244,7 @@
     return os.strerror(errno)
 
 if sys.platform != 'win32':
+    @unwrap_spec(secs=float)
     def sleep(space, secs):
         pytime.sleep(secs)
 else:
@@ -263,6 +264,7 @@
                 pytime.sleep(0.001)
                 raise wrap_oserror(space,
                                    OSError(EINTR, "sleep() interrupted"))
+    @unwrap_spec(secs=float)
     def sleep(space, secs):
         # as decreed by Guido, only the main thread can be
         # interrupted.
@@ -273,7 +275,6 @@
             _simple_sleep(space, MAX, interruptible)
             secs -= MAX
         _simple_sleep(space, secs, interruptible)
-sleep.unwrap_spec = [ObjSpace, float]
 
 def _get_module_object(space, obj_name):
     w_module = space.getbuiltinmodule('time')
@@ -429,7 +430,6 @@
             space.wrap("unconvertible time"))
 
     return space.wrap(rffi.charp2str(p)[:-1]) # get rid of new line
-ctime.unwrap_spec = [ObjSpace, W_Root]
 
 # by now w_tup is an optional argument (and not *args)
 # because of the ext. compiler bugs in handling such arguments (*args, **kwds)
@@ -446,7 +446,6 @@
             space.wrap("unconvertible time"))
     
     return space.wrap(rffi.charp2str(p)[:-1]) # get rid of new line
-asctime.unwrap_spec = [ObjSpace, W_Root]
 
 def gmtime(space, w_seconds=None):
     """gmtime([seconds]) -> (tm_year, tm_mon, tm_day, tm_hour, tm_min,
@@ -467,7 +466,6 @@
     if not p:
         raise OperationError(space.w_ValueError, space.wrap(_get_error_msg()))
     return _tm_to_tuple(space, p)
-gmtime.unwrap_spec = [ObjSpace, W_Root]
 
 def localtime(space, w_seconds=None):
     """localtime([seconds]) -> (tm_year, tm_mon, tm_day, tm_hour, tm_min,
@@ -485,7 +483,6 @@
     if not p:
         raise OperationError(space.w_ValueError, space.wrap(_get_error_msg()))
     return _tm_to_tuple(space, p)
-localtime.unwrap_spec = [ObjSpace, W_Root]
 
 def mktime(space, w_tup):
     """mktime(tuple) -> floating point number
@@ -499,7 +496,6 @@
             space.wrap("mktime argument out of range"))
 
     return space.wrap(float(tt))
-mktime.unwrap_spec = [ObjSpace, W_Root]
 
 if _POSIX:
     def tzset(space):
@@ -519,8 +515,8 @@
 
         # reset timezone, altzone, daylight and tzname
         _init_timezone(space)
-    tzset.unwrap_spec = [ObjSpace]
 
+ at unwrap_spec(format=str)
 def strftime(space, format, w_tup=None):
     """strftime(format[, tuple]) -> string
 
@@ -586,4 +582,3 @@
         finally:
             lltype.free(outbuf, flavor='raw')
         i += i
-strftime.unwrap_spec = [ObjSpace, str, W_Root]

From commits-noreply at bitbucket.org  Wed Feb 16 19:20:20 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 19:20:20 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: simplify unwrap_spec for
	modules select, signal
Message-ID: <20110216182020.71327282BF2@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42084:6ac5da276e32
Date: 2011-02-16 18:30 +0100
http://bitbucket.org/pypy/pypy/changeset/6ac5da276e32/

Log:	simplify unwrap_spec for modules select, signal

diff --git a/pypy/module/select/interp_select.py b/pypy/module/select/interp_select.py
--- a/pypy/module/select/interp_select.py
+++ b/pypy/module/select/interp_select.py
@@ -1,7 +1,7 @@
 import math
 from pypy.interpreter.typedef import TypeDef
 from pypy.interpreter.baseobjspace import Wrappable
-from pypy.interpreter.gateway import W_Root, ObjSpace, interp2app
+from pypy.interpreter.gateway import interp2app, unwrap_spec
 from pypy.interpreter.error import (
     OperationError, operationerrfmt, wrap_oserror)
 from pypy.rlib import rpoll
@@ -18,18 +18,18 @@
     def __init__(self):
         self.fddict = {}
 
+    @unwrap_spec(events=int)
     def register(self, space, w_fd, events=defaultevents):
         fd = space.c_filedescriptor_w(w_fd)
         self.fddict[fd] = events
-    register.unwrap_spec = ['self', ObjSpace, W_Root, int]
 
+    @unwrap_spec(events=int)
     def modify(self, space, w_fd, events):
         fd = space.c_filedescriptor_w(w_fd)
         if fd not in self.fddict:
             raise wrap_oserror(space, OSError(errno.ENOENT, "poll.modify"),
                                exception_name='w_IOError')
         self.fddict[fd] = events
-    modify.unwrap_spec = ['self', ObjSpace, W_Root, int]
 
     def unregister(self, space, w_fd):
         fd = space.c_filedescriptor_w(w_fd)
@@ -38,7 +38,6 @@
         except KeyError:
             raise OperationError(space.w_KeyError,
                                  space.wrap(fd)) # XXX should this maybe be w_fd?
-    unregister.unwrap_spec = ['self', ObjSpace, W_Root]
 
     def poll(self, space, w_timeout=None):
         if space.is_w(w_timeout, space.w_None):
@@ -68,14 +67,10 @@
             retval_w.append(space.newtuple([space.wrap(fd),
                                             space.wrap(revents)]))
         return space.newlist(retval_w)
-    poll.unwrap_spec = ['self', ObjSpace, W_Root]
 
 pollmethods = {}
 for methodname in 'register modify unregister poll'.split():
-    method = getattr(Poll, methodname)
-    assert hasattr(method,'unwrap_spec'), methodname
-    assert method.im_func.func_code.co_argcount == len(method.unwrap_spec), methodname
-    pollmethods[methodname] = interp2app(method, unwrap_spec=method.unwrap_spec)
+    pollmethods[methodname] = interp2app(getattr(Poll, methodname))
 Poll.typedef = TypeDef('select.poll', **pollmethods)
 
 def select(space, w_iwtd, w_owtd, w_ewtd, w_timeout=None):

diff --git a/pypy/module/select/interp_epoll.py b/pypy/module/select/interp_epoll.py
--- a/pypy/module/select/interp_epoll.py
+++ b/pypy/module/select/interp_epoll.py
@@ -3,7 +3,7 @@
 import errno
 
 from pypy.interpreter.baseobjspace import Wrappable
-from pypy.interpreter.gateway import interp2app, unwrap_spec, ObjSpace, W_Root
+from pypy.interpreter.gateway import interp2app, unwrap_spec
 from pypy.interpreter.error import OperationError, operationerrfmt, exception_from_errno
 from pypy.interpreter.typedef import TypeDef, GetSetProperty
 from pypy.rpython.lltypesystem import lltype, rffi
@@ -60,7 +60,7 @@
         self.space = space
         self.epfd = epfd
 
-    @unwrap_spec(ObjSpace, W_Root, int)
+    @unwrap_spec(sizehint=int)
     def descr__new__(space, w_subtype, sizehint=-1):
         if sizehint == -1:
             sizehint = FD_SETSIZE - 1
@@ -74,7 +74,7 @@
 
         return space.wrap(W_Epoll(space, epfd))
 
-    @unwrap_spec(ObjSpace, W_Root, int)
+    @unwrap_spec(fd=int)
     def descr_fromfd(space, w_cls, fd):
         return space.wrap(W_Epoll(space, fd))
 
@@ -107,32 +107,29 @@
     def descr_get_closed(space, self):
         return space.wrap(self.epfd < 0)
 
-    @unwrap_spec("self", ObjSpace)
     def descr_fileno(self, space):
         self.check_closed()
         return space.wrap(self.epfd)
 
-    @unwrap_spec("self", ObjSpace)
     def descr_close(self, space):
         self.check_closed()
         self.close()
 
-    @unwrap_spec("self", ObjSpace, W_Root, int)
+    @unwrap_spec(eventmask=int)
     def descr_register(self, space, w_fd, eventmask=-1):
         self.check_closed()
         self.epoll_ctl(EPOLL_CTL_ADD, w_fd, eventmask)
 
-    @unwrap_spec("self", ObjSpace, W_Root)
     def descr_unregister(self, space, w_fd):
         self.check_closed()
         self.epoll_ctl(EPOLL_CTL_DEL, w_fd, 0, ignore_ebadf=True)
 
-    @unwrap_spec("self", ObjSpace, W_Root, int)
+    @unwrap_spec(eventmask=int)
     def descr_modify(self, space, w_fd, eventmask=-1):
         self.check_closed()
         self.epoll_ctl(EPOLL_CTL_MOD, w_fd, eventmask)
 
-    @unwrap_spec("self", ObjSpace, float, int)
+    @unwrap_spec(timeout=float, maxevents=int)
     def descr_poll(self, space, timeout=-1.0, maxevents=-1):
         self.check_closed()
         if timeout < 0:

diff --git a/pypy/module/signal/interp_signal.py b/pypy/module/signal/interp_signal.py
--- a/pypy/module/signal/interp_signal.py
+++ b/pypy/module/signal/interp_signal.py
@@ -1,6 +1,5 @@
 from __future__ import with_statement
 from pypy.interpreter.error import OperationError, exception_from_errno
-from pypy.interpreter.baseobjspace import W_Root, ObjSpace
 from pypy.interpreter.executioncontext import AsyncAction, AbstractActionFlag
 from pypy.interpreter.executioncontext import PeriodicAsyncAction
 from pypy.interpreter.gateway import unwrap_spec
@@ -192,6 +191,7 @@
             self.fire_after_thread_switch()
 
 
+ at unwrap_spec(signum=int)
 def getsignal(space, signum):
     """
     getsignal(sig) -> action
@@ -207,18 +207,16 @@
     if signum in action.handlers_w:
         return action.handlers_w[signum]
     return space.wrap(SIG_DFL)
-getsignal.unwrap_spec = [ObjSpace, int]
 
 @jit.dont_look_inside
+ at unwrap_spec(timeout=int)
 def alarm(space, timeout):
     return space.wrap(c_alarm(timeout))
-alarm.unwrap_spec = [ObjSpace, int]
 
 @jit.dont_look_inside
 def pause(space):
     c_pause()
     return space.w_None
-pause.unwrap_spec = [ObjSpace]
 
 def check_signum(space, signum):
     if signum < 1 or signum >= NSIG:
@@ -226,6 +224,7 @@
                              space.wrap("signal number out of range"))
 
 @jit.dont_look_inside
+ at unwrap_spec(signum=int)
 def signal(space, signum, w_handler):
     """
     signal(sig, action) -> action
@@ -262,8 +261,8 @@
         pypysig_setflag(signum)
         action.handlers_w[signum] = w_handler
     return old_handler
-signal.unwrap_spec = [ObjSpace, int, W_Root]
 
+ at unwrap_spec(fd=int)
 def set_wakeup_fd(space, fd):
     """Sets the fd to be written to (with '\0') when a signal
     comes in.  Returns the old fd.  A library can use this to
@@ -280,14 +279,13 @@
                 space.wrap("set_wakeup_fd only works in main thread"))
     old_fd = pypysig_set_wakeup_fd(fd)
     return space.wrap(intmask(old_fd))
-set_wakeup_fd.unwrap_spec = [ObjSpace, int]
 
+ at unwrap_spec(signum=int, flag=int)
 def siginterrupt(space, signum, flag):
     check_signum(space, signum)
     if rffi.cast(lltype.Signed, c_siginterrupt(signum, flag)) < 0:
         errno = rposix.get_errno()
         raise OperationError(space.w_RuntimeError, space.wrap(errno))
-siginterrupt.unwrap_spec = [ObjSpace, int, int]
 
 
 #__________________________________________________________
@@ -310,7 +308,7 @@
     return space.getattr(mod, space.wrap("ItimerError"))
 
 @jit.dont_look_inside
- at unwrap_spec(ObjSpace, int, float, float)
+ at unwrap_spec(which=int, first=float, interval=float)
 def setitimer(space, which, first, interval=0):
     with lltype.scoped_alloc(itimervalP.TO, 1) as new:
 
@@ -327,7 +325,7 @@
             return itimer_retval(space, old[0])
 
 @jit.dont_look_inside
- at unwrap_spec(ObjSpace, int)
+ at unwrap_spec(which=int)
 def getitimer(space, which):
     with lltype.scoped_alloc(itimervalP.TO, 1) as old:
 

From commits-noreply at bitbucket.org  Wed Feb 16 19:20:23 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 19:20:23 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: newer unwrap_spec in modules
	thread, time, token
Message-ID: <20110216182023.4A0EF282BF2@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42085:cfb157c1cfd7
Date: 2011-02-16 18:40 +0100
http://bitbucket.org/pypy/pypy/changeset/cfb157c1cfd7/

Log:	newer unwrap_spec in modules thread, time, token

diff --git a/pypy/module/thread/os_local.py b/pypy/module/thread/os_local.py
--- a/pypy/module/thread/os_local.py
+++ b/pypy/module/thread/os_local.py
@@ -4,7 +4,6 @@
 from pypy.interpreter.typedef import TypeDef, interp2app
 from pypy.interpreter.typedef import GetSetProperty, descr_get_dict
 from pypy.interpreter.typedef import descr_set_dict
-from pypy.interpreter.gateway import ObjSpace, W_Root, Arguments
 
 
 class Local(Wrappable):
@@ -42,12 +41,10 @@
         local = space.allocate_instance(Local, w_subtype)
         Local.__init__(local, space, __args__)
         return space.wrap(local)
-    descr_local__new__.unwrap_spec=[ObjSpace, W_Root, Arguments]
 
     def descr_local__init__(self, space):
         # No arguments allowed
         pass
-    descr_local__init__.unwrap_spec=['self', ObjSpace]
 
 Local.typedef = TypeDef("thread._local",
                         __doc__ = "Thread-local data",

diff --git a/pypy/module/time/interp_time.py b/pypy/module/time/interp_time.py
--- a/pypy/module/time/interp_time.py
+++ b/pypy/module/time/interp_time.py
@@ -1,5 +1,5 @@
 import time
-from pypy.interpreter.gateway import ObjSpace
+from pypy.interpreter.gateway import unwrap_spec
 
 
 def clock(space):
@@ -13,8 +13,8 @@
 second may be present if the system clock provides them."""
     return space.wrap(time.time())
 
+ at unwrap_spec(seconds=float)
 def sleep(space, seconds):
     """Delay execution for a given number of seconds.  The argument may
 be a floating point number for subsecond precision."""
     time.sleep(seconds)
-sleep.unwrap_spec = [ObjSpace, float]

diff --git a/pypy/module/token/__init__.py b/pypy/module/token/__init__.py
--- a/pypy/module/token/__init__.py
+++ b/pypy/module/token/__init__.py
@@ -1,5 +1,5 @@
 from pypy.interpreter.mixedmodule import MixedModule
-from pypy.interpreter.baseobjspace import ObjSpace
+from pypy.interpreter.gateway import unwrap_spec
 from pypy.interpreter.pyparser import pytoken, pygram
 
 
@@ -25,14 +25,14 @@
 _init_tokens()
 
 
+ at unwrap_spec(tok=int)
 def isterminal(space, tok):
     return space.wrap(tok < 256)
-isterminal.unwrap_spec = [ObjSpace, int]
 
+ at unwrap_spec(tok=int)
 def isnonterminal(space, tok):
     return space.wrap(tok >= 256)
-isnonterminal.unwrap_spec = [ObjSpace, int]
 
+ at unwrap_spec(tok=int)
 def iseof(space, tok):
     return space.wrap(tok == pygram.tokens.ENDMARKER)
-iseof.unwrap_spec = [ObjSpace, int]

diff --git a/pypy/module/thread/os_thread.py b/pypy/module/thread/os_thread.py
--- a/pypy/module/thread/os_thread.py
+++ b/pypy/module/thread/os_thread.py
@@ -5,8 +5,7 @@
 from pypy.module.thread import ll_thread as thread
 from pypy.module.thread.error import wrap_thread_error
 from pypy.interpreter.error import OperationError, operationerrfmt
-from pypy.interpreter.gateway import NoneNotWrapped
-from pypy.interpreter.gateway import ObjSpace, W_Root, Arguments
+from pypy.interpreter.gateway import unwrap_spec, NoneNotWrapped, Arguments
 from pypy.rlib.objectmodel import free_non_gc_object
 
 # Here are the steps performed to start a new thread:
@@ -202,6 +201,7 @@
     ident = thread.get_ident()
     return space.wrap(ident)
 
+ at unwrap_spec(size=int)
 def stack_size(space, size=0):
     """stack_size([size]) -> size
 
@@ -232,7 +232,6 @@
     if error == -2:
         raise wrap_thread_error(space, "setting stack size not supported")
     return space.wrap(old_size)
-stack_size.unwrap_spec = [ObjSpace, int]
 
 def _count(space):
     """_count() -> integer

diff --git a/pypy/module/thread/os_lock.py b/pypy/module/thread/os_lock.py
--- a/pypy/module/thread/os_lock.py
+++ b/pypy/module/thread/os_lock.py
@@ -5,7 +5,7 @@
 from pypy.module.thread import ll_thread as thread
 from pypy.module.thread.error import wrap_thread_error
 from pypy.interpreter.baseobjspace import Wrappable
-from pypy.interpreter.gateway import ObjSpace, interp2app, Arguments
+from pypy.interpreter.gateway import interp2app, unwrap_spec
 from pypy.interpreter.typedef import TypeDef
 
 # Force the declaration of the type 'thread.LockType' for RPython
@@ -36,6 +36,7 @@
         except thread.error:
             raise wrap_thread_error(space, "out of resources")
 
+    @unwrap_spec(waitflag=int)
     def descr_lock_acquire(self, space, waitflag=1):
         """Lock the lock.  Without argument, this blocks if the lock is already
 locked (even by the same thread), waiting for another thread to release
@@ -78,16 +79,11 @@
     def __exit__(self, *args):
         self.descr_lock_release(self.space)
 
-descr_acquire = interp2app(Lock.descr_lock_acquire,
-                           unwrap_spec=['self', ObjSpace, int])
-descr_release = interp2app(Lock.descr_lock_release,
-                           unwrap_spec=['self', ObjSpace])
-descr_locked  = interp2app(Lock.descr_lock_locked,
-                           unwrap_spec=['self', ObjSpace])
-descr__enter__ = interp2app(Lock.descr__enter__,
-                            unwrap_spec=['self', ObjSpace])
-descr__exit__ = interp2app(Lock.descr__exit__,
-                            unwrap_spec=['self', ObjSpace, Arguments])
+descr_acquire = interp2app(Lock.descr_lock_acquire)
+descr_release = interp2app(Lock.descr_lock_release)
+descr_locked  = interp2app(Lock.descr_lock_locked)
+descr__enter__ = interp2app(Lock.descr__enter__)
+descr__exit__ = interp2app(Lock.descr__exit__)
 
 
 Lock.typedef = TypeDef("thread.lock",

From commits-noreply at bitbucket.org  Wed Feb 16 19:20:24 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 19:20:24 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: Update unicode data with the
	new unwrap_spec
Message-ID: <20110216182024.72A5F282BF2@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42086:470b28590ad9
Date: 2011-02-16 18:46 +0100
http://bitbucket.org/pypy/pypy/changeset/470b28590ad9/

Log:	Update unicode data with the new unwrap_spec

diff --git a/pypy/module/unicodedata/interp_ucd.py b/pypy/module/unicodedata/interp_ucd.py
--- a/pypy/module/unicodedata/interp_ucd.py
+++ b/pypy/module/unicodedata/interp_ucd.py
@@ -1,8 +1,7 @@
 """
 Implementation of the interpreter-level functions in the module unicodedata.
 """
-from pypy.interpreter.gateway import W_Root, ObjSpace, NoneNotWrapped
-from pypy.interpreter.gateway import  interp2app
+from pypy.interpreter.gateway import  interp2app, unwrap_spec, NoneNotWrapped
 from pypy.interpreter.baseobjspace import Wrappable
 from pypy.interpreter.error import OperationError
 from pypy.interpreter.typedef import TypeDef, interp_attrproperty
@@ -122,6 +121,7 @@
 
         self.version = unicodedb.version
 
+    @unwrap_spec(name=str)
     def _get_code(self, space, name):
         try:
             code = self._lookup(name.upper())
@@ -129,8 +129,8 @@
             msg = space.mod(space.wrap("undefined character name '%s'"), space.wrap(name))
             raise OperationError(space.w_KeyError, msg)
         return space.wrap(code)
-    _get_code.unwrap_spec = ['self', ObjSpace, str]
 
+    @unwrap_spec(name=str)
     def lookup(self, space, name):
         try:
             code = self._lookup(name.upper())
@@ -138,7 +138,6 @@
             msg = space.mod(space.wrap("undefined character name '%s'"), space.wrap(name))
             raise OperationError(space.w_KeyError, msg)
         return space.wrap(code_to_unichr(code))
-    lookup.unwrap_spec = ['self', ObjSpace, str]
 
     def name(self, space, w_unichr, w_default=NoneNotWrapped):
         code = unichr_to_code_w(space, w_unichr)
@@ -149,7 +148,6 @@
                 return w_default
             raise OperationError(space.w_ValueError, space.wrap('no such name'))
         return space.wrap(name)
-    name.unwrap_spec = ['self', ObjSpace, W_Root, W_Root]
 
 
     def decimal(self, space, w_unichr, w_default=NoneNotWrapped):
@@ -161,7 +159,6 @@
         if w_default is not None:
             return w_default
         raise OperationError(space.w_ValueError, space.wrap('not a decimal'))
-    decimal.unwrap_spec = ['self', ObjSpace, W_Root, W_Root]
 
     def digit(self, space, w_unichr, w_default=NoneNotWrapped):
         code = unichr_to_code_w(space, w_unichr)
@@ -172,7 +169,6 @@
         if w_default is not None:
             return w_default
         raise OperationError(space.w_ValueError, space.wrap('not a digit'))
-    digit.unwrap_spec = ['self', ObjSpace, W_Root, W_Root]
 
     def numeric(self, space, w_unichr, w_default=NoneNotWrapped):
         code = unichr_to_code_w(space, w_unichr)
@@ -184,39 +180,33 @@
             return w_default
         raise OperationError(space.w_ValueError,
                              space.wrap('not a numeric character'))
-    numeric.unwrap_spec = ['self', ObjSpace, W_Root, W_Root]
 
     def category(self, space, w_unichr):
         code = unichr_to_code_w(space, w_unichr)
         return space.wrap(self._category(code))
-    category.unwrap_spec = ['self', ObjSpace, W_Root]
 
     def east_asian_width(self, space, w_unichr):
         code = unichr_to_code_w(space, w_unichr)
         return space.wrap(self._east_asian_width(code))
-    east_asian_width.unwrap_spec = ['self', ObjSpace, W_Root]
 
     def bidirectional(self, space, w_unichr):
         code = unichr_to_code_w(space, w_unichr)
         return space.wrap(self._bidirectional(code))
-    bidirectional.unwrap_spec = ['self', ObjSpace, W_Root]
 
     def combining(self, space, w_unichr):
         code = unichr_to_code_w(space, w_unichr)
         return space.wrap(self._combining(code))
-    combining.unwrap_spec = ['self', ObjSpace, W_Root]
 
     def mirrored(self, space, w_unichr):
         code = unichr_to_code_w(space, w_unichr)
         # For no reason, unicodedata.mirrored() returns an int, not a bool
         return space.wrap(int(self._mirrored(code)))
-    mirrored.unwrap_spec = ['self', ObjSpace, W_Root]
 
     def decomposition(self, space, w_unichr):
         code = unichr_to_code_w(space, w_unichr)
         return space.wrap(self._decomposition(code))
-    decomposition.unwrap_spec = ['self', ObjSpace, W_Root]
 
+    @unwrap_spec(form=str)
     def normalize(self, space, form, w_unistr):
         if not space.is_true(space.isinstance(w_unistr, space.w_unicode)):
             raise OperationError(space.w_TypeError, space.wrap('argument 2 must be unicode'))
@@ -349,16 +339,14 @@
         result[starter_pos] = current
 
         return space.wrap(u''.join([unichr(i) for i in result[:next_insert]]))
-    normalize.unwrap_spec = ['self', ObjSpace, str, W_Root]
 
 
 methods = {}
-for methodname in UCD.__dict__:
-    method = getattr(UCD, methodname)
-    if not hasattr(method,'unwrap_spec'):
-        continue
-    assert method.im_func.func_code.co_argcount == len(method.unwrap_spec), methodname
-    methods[methodname] = interp2app(method, unwrap_spec=method.unwrap_spec)
+for methodname in """
+        _get_code lookup name decimal digit numeric category east_asian_width
+        bidirectional combining mirrored decomposition normalize
+        """.split():
+    methods[methodname] = interp2app(getattr(UCD, methodname))
 
 
 UCD.typedef = TypeDef("unicodedata.UCD",

From commits-noreply at bitbucket.org  Wed Feb 16 19:20:25 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 19:20:25 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: update zipimport to the new
	unwrap_spec
Message-ID: <20110216182025.5BF912A202C@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42087:318d10258aa1
Date: 2011-02-16 18:50 +0100
http://bitbucket.org/pypy/pypy/changeset/318d10258aa1/

Log:	update zipimport to the new unwrap_spec

diff --git a/pypy/module/zipimport/interp_zipimport.py b/pypy/module/zipimport/interp_zipimport.py
--- a/pypy/module/zipimport/interp_zipimport.py
+++ b/pypy/module/zipimport/interp_zipimport.py
@@ -1,8 +1,7 @@
 
-from pypy.interpreter.baseobjspace import W_Root, ObjSpace, Wrappable, \
-     Arguments
+from pypy.interpreter.baseobjspace import Wrappable
 from pypy.interpreter.error import OperationError, operationerrfmt
-from pypy.interpreter.gateway import interp2app
+from pypy.interpreter.gateway import interp2app, unwrap_spec
 from pypy.interpreter.typedef import TypeDef, GetSetProperty
 from pypy.interpreter.module import Module
 from pypy.module.imp import importing
@@ -36,7 +35,8 @@
     # -------------- dict-like interface -----------------
     # I don't care about speed of those, they're obscure anyway
     # THIS IS A TERRIBLE HACK TO BE CPYTHON COMPATIBLE
-        
+
+    @unwrap_spec(name=str)
     def getitem(self, space, name):
         try:
             w_zipimporter = self.cache[name]
@@ -53,49 +53,41 @@
                 w(info.file_size), w(info.file_offset), w(info.dostime),
                 w(info.dosdate), w(info.CRC)]))
         return w_d
-    getitem.unwrap_spec = ['self', ObjSpace, str]
 
     def keys(self, space):
         return space.newlist([space.wrap(s)
                               for s in self.cache.keys()])
-    keys.unwrap_spec = ['self', ObjSpace]
 
     def values(self, space):
         keys = self.cache.keys()
         values_w = [self.getitem(space, key) for key in keys]
         return space.newlist(values_w)
-    values.unwrap_spec = ['self', ObjSpace]
 
     def items(self, space):
         w = space.wrap
         items_w = [space.newtuple([w(key), self.getitem(space, key)])
                    for key in self.cache.keys()]
         return space.newlist(items_w)
-    items.unwrap_spec = ['self', ObjSpace]
 
     def iterkeys(self, space):
         return space.iter(self.keys(space))
-    iterkeys.unwrap_spec = ['self', ObjSpace]
 
     def itervalues(self, space):
         return space.iter(self.values(space))
-    itervalues.unwrap_spec = ['self', ObjSpace]
 
     def iteritems(self, space):
         return space.iter(self.items(space))
-    iteritems.unwrap_spec = ['self', ObjSpace]
 
+    @unwrap_spec(name=str)
     def contains(self, space, name):
         return space.newbool(name in self.cache)
-    contains.unwrap_spec = ['self', ObjSpace, str]
 
     def clear(self, space):
         self.cache = {}
-    clear.unwrap_spec = ['self', ObjSpace]
 
+    @unwrap_spec(name=str)
     def delitem(self, space, name):
         del self.cache[name]
-    delitem.unwrap_spec = ['self', ObjSpace, str]
 
 W_ZipCache.typedef = TypeDef(
     'zip_dict',
@@ -218,12 +210,12 @@
         except KeyError:
             return False
 
+    @unwrap_spec(fullname=str)
     def find_module(self, space, fullname, w_path=None):
         filename = self.make_filename(fullname)
         for _, _, ext in ENUMERATE_EXTS:
             if self.have_modulefile(space, filename + ext):
                 return space.wrap(self)
-    find_module.unwrap_spec = ['self', ObjSpace, str, W_Root]
 
     def make_filename(self, fullname):
         startpos = fullname.rfind('.') + 1 # 0 when not found
@@ -231,6 +223,7 @@
         subname = fullname[startpos:]
         return self.prefix + subname.replace('.', '/')
 
+    @unwrap_spec(fullname=str)
     def load_module(self, space, fullname):
         w = space.wrap
         filename = self.make_filename(fullname)
@@ -266,8 +259,8 @@
             raise OperationError(self.w_ZipImportError, last_exc.get_w_value(space))
         # should never happen I think
         return space.w_None
-    load_module.unwrap_spec = ['self', ObjSpace, str]
 
+    @unwrap_spec(filename=str)
     def get_data(self, space, filename):
         filename = self._find_relative_path(filename)
         w = space.wrap
@@ -280,8 +273,8 @@
             return w(data)
         except (KeyError, OSError):
             raise OperationError(space.w_IOError, space.wrap("Error reading file"))
-    get_data.unwrap_spec = ['self', ObjSpace, str]
 
+    @unwrap_spec(fullname=str)
     def get_code(self, space, fullname):
         filename = self.make_filename(fullname)
         for compiled, _, ext in ENUMERATE_EXTS:
@@ -302,8 +295,8 @@
                 return space.wrap(code_w)
         raise operationerrfmt(self.w_ZipImportError,
             "Cannot find source or code for %s in %s", filename, self.name)
-    get_code.unwrap_spec = ['self', ObjSpace, str]
 
+    @unwrap_spec(fullname=str)
     def get_source(self, space, fullname):
         filename = self.make_filename(fullname)
         found = False
@@ -318,8 +311,8 @@
             return space.w_None
         raise operationerrfmt(self.w_ZipImportError,
             "Cannot find source for %s in %s", filename, self.name)
-    get_source.unwrap_spec = ['self', ObjSpace, str]
 
+    @unwrap_spec(fullname=str)
     def get_filename(self, space, fullname):
         filename = self.make_filename(fullname)
         for _, is_package, ext in ENUMERATE_EXTS:
@@ -328,8 +321,8 @@
                                   self.corr_zname(filename + ext))
         raise operationerrfmt(self.w_ZipImportError,
             "Cannot find module %s in %s", filename, self.name)
-    get_filename.unwrap_spec = ['self', ObjSpace, str]
 
+    @unwrap_spec(fullname=str)
     def is_package(self, space, fullname):
         filename = self.make_filename(fullname)
         for _, is_package, ext in ENUMERATE_EXTS:
@@ -337,12 +330,12 @@
                 return space.wrap(is_package)
         raise operationerrfmt(self.w_ZipImportError,
             "Cannot find module %s in %s", filename, self.name)
-    is_package.unwrap_spec = ['self', ObjSpace, str]
 
     def getarchive(space, self):
         space = self.space
         return space.wrap(self.filename)
 
+ at unwrap_spec(name=str)
 def descr_new_zipimporter(space, w_type, name):
     w = space.wrap
     w_ZipImportError = space.getattr(space.getbuiltinmodule('zipimport'),
@@ -391,8 +384,6 @@
     zip_cache.set(filename, w_result)
     return w_result
 
-descr_new_zipimporter.unwrap_spec = [ObjSpace, W_Root, str]
-
 W_ZipImporter.typedef = TypeDef(
     'zipimporter',
     __new__     = interp2app(descr_new_zipimporter),

From commits-noreply at bitbucket.org  Wed Feb 16 19:20:26 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 19:20:26 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: Simplify unwrap_spec in the
	cpyext module
Message-ID: <20110216182026.BE92A282BF2@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42088:42336ab54b93
Date: 2011-02-16 18:56 +0100
http://bitbucket.org/pypy/pypy/changeset/42336ab54b93/

Log:	Simplify unwrap_spec in the cpyext module

diff --git a/pypy/module/cpyext/methodobject.py b/pypy/module/cpyext/methodobject.py
--- a/pypy/module/cpyext/methodobject.py
+++ b/pypy/module/cpyext/methodobject.py
@@ -1,8 +1,8 @@
-from pypy.interpreter.baseobjspace import Wrappable, W_Root, ObjSpace
+from pypy.interpreter.baseobjspace import Wrappable
 from pypy.interpreter.typedef import TypeDef, GetSetProperty
 from pypy.interpreter.argument import Arguments
 from pypy.interpreter.typedef import interp_attrproperty, interp_attrproperty_w
-from pypy.interpreter.gateway import interp2app, unwrap_spec
+from pypy.interpreter.gateway import interp2app
 from pypy.interpreter.error import OperationError, operationerrfmt
 from pypy.interpreter.function import BuiltinFunction, Method, StaticMethod
 from pypy.rpython.lltypesystem import rffi, lltype
@@ -153,7 +153,6 @@
         return self.space.wrap("<slot wrapper '%s' of '%s' objects>" % (self.method_name,
             self.w_objclass.getname(self.space, '?')))
 
- at unwrap_spec(ObjSpace, W_Root, Arguments)
 def cwrapper_descr_call(space, w_self, __args__):
     self = space.interp_w(W_PyCWrapperObject, w_self)
     args_w, kw_w = __args__.unpack()
@@ -165,7 +164,6 @@
     return self.call(space, w_self, w_args, w_kw)
 
 
- at unwrap_spec(ObjSpace, W_Root, Arguments)
 def cfunction_descr_call(space, w_self, __args__):
     self = space.interp_w(W_PyCFunctionObject, w_self)
     args_w, kw_w = __args__.unpack()
@@ -176,7 +174,6 @@
     ret = self.call(space, None, w_args, w_kw)
     return ret
 
- at unwrap_spec(ObjSpace, W_Root, Arguments)
 def cmethod_descr_call(space, w_self, __args__):
     self = space.interp_w(W_PyCFunctionObject, w_self)
     args_w, kw_w = __args__.unpack()

diff --git a/pypy/module/cpyext/api.py b/pypy/module/cpyext/api.py
--- a/pypy/module/cpyext/api.py
+++ b/pypy/module/cpyext/api.py
@@ -16,7 +16,7 @@
 from pypy.translator import platform
 from pypy.module.cpyext.state import State
 from pypy.interpreter.error import OperationError, operationerrfmt
-from pypy.interpreter.baseobjspace import W_Root, ObjSpace
+from pypy.interpreter.baseobjspace import W_Root
 from pypy.interpreter.gateway import unwrap_spec
 from pypy.interpreter.nestedscope import Cell
 from pypy.interpreter.module import Module
@@ -939,7 +939,7 @@
     copy_header_files(trunk_include)
 
 initfunctype = lltype.Ptr(lltype.FuncType([], lltype.Void))
- at unwrap_spec(ObjSpace, str, str)
+ at unwrap_spec(path=str, name=str)
 def load_extension_module(space, path, name):
     if os.sep not in path:
         path = os.curdir + os.sep + path      # force a '/' in the path

From commits-noreply at bitbucket.org  Wed Feb 16 19:20:29 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 19:20:29 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: Small unwrap_spec
	simplifications in	various places
Message-ID: <20110216182029.517502A202C@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42089:ff6aa900d761
Date: 2011-02-16 19:04 +0100
http://bitbucket.org/pypy/pypy/changeset/ff6aa900d761/

Log:	Small unwrap_spec simplifications in various places

diff --git a/pypy/translator/goal/nanos.py b/pypy/translator/goal/nanos.py
--- a/pypy/translator/goal/nanos.py
+++ b/pypy/translator/goal/nanos.py
@@ -25,7 +25,7 @@
 more from os and os.path.
 """
 
-from pypy.interpreter.gateway import applevel, ObjSpace, W_Root, interp2app
+from pypy.interpreter.gateway import applevel, interp2app
 import os, py
 
 if os.name == 'posix':
@@ -268,7 +268,7 @@
 def getenv(space, w_name):
     name = space.str_w(w_name)
     return space.wrap(os.environ.get(name))
-getenv_w = interp2app(getenv, unwrap_spec=[ObjSpace, W_Root])
+getenv_w = interp2app(getenv)
 
 def setup_nanos(space):
     w_os = space.wrap(app_os.buildmodule(space, 'os'))

diff --git a/pypy/module/struct/interp_struct.py b/pypy/module/struct/interp_struct.py
--- a/pypy/module/struct/interp_struct.py
+++ b/pypy/module/struct/interp_struct.py
@@ -1,4 +1,4 @@
-from pypy.interpreter.gateway import ObjSpace
+from pypy.interpreter.gateway import unwrap_spec
 from pypy.interpreter.error import OperationError
 from pypy.rlib.rstruct.error import StructError
 from pypy.module.struct.formatiterator import CalcSizeFormatIterator
@@ -6,6 +6,7 @@
 from pypy.module.struct.formatiterator import UnpackFormatIterator
 
 
+ at unwrap_spec(format=str)
 def calcsize(space, format):
     fmtiter = CalcSizeFormatIterator()
     try:
@@ -13,9 +14,9 @@
     except StructError, e:
         raise e.at_applevel(space)
     return space.wrap(fmtiter.totalsize)
-calcsize.unwrap_spec = [ObjSpace, str]
 
 
+ at unwrap_spec(format=str)
 def pack(space, format, args_w):
     fmtiter = PackFormatIterator(space, args_w)
     try:
@@ -24,9 +25,9 @@
         raise e.at_applevel(space)
     result = ''.join(fmtiter.result)
     return space.wrap(result)
-pack.unwrap_spec = [ObjSpace, str, 'args_w']
 
 
+ at unwrap_spec(format=str, input='bufferstr')
 def unpack(space, format, input):
     fmtiter = UnpackFormatIterator(space, input)
     try:
@@ -34,4 +35,3 @@
     except StructError, e:
         raise e.at_applevel(space)
     return space.newtuple(fmtiter.result_w[:])
-unpack.unwrap_spec = [ObjSpace, str, 'bufferstr']

diff --git a/pypy/module/crypt/interp_crypt.py b/pypy/module/crypt/interp_crypt.py
--- a/pypy/module/crypt/interp_crypt.py
+++ b/pypy/module/crypt/interp_crypt.py
@@ -1,5 +1,5 @@
 from pypy.interpreter.error import OperationError
-from pypy.interpreter.baseobjspace import ObjSpace, W_Root
+from pypy.interpreter.baseobjspace import unwrap_spec
 from pypy.rpython.lltypesystem import rffi, lltype
 from pypy.translator.tool.cbuild import ExternalCompilationInfo
 import sys
@@ -11,6 +11,7 @@
 c_crypt = rffi.llexternal('crypt', [rffi.CCHARP, rffi.CCHARP], rffi.CCHARP,
                           compilation_info=eci, threadsafe=False)
 
+ at unwrap_spec(word=str, salt=str)
 def crypt(space, word, salt):
     """word will usually be a user's password. salt is a 2-character string
     which will be used to select one of 4096 variations of DES. The characters
@@ -22,5 +23,3 @@
         return space.w_None
     str_res = rffi.charp2str(res)
     return space.wrap(str_res) 
-
-crypt.unwrap_spec = [ObjSpace, str, str]

diff --git a/pypy/tool/pytest/appsupport.py b/pypy/tool/pytest/appsupport.py
--- a/pypy/tool/pytest/appsupport.py
+++ b/pypy/tool/pytest/appsupport.py
@@ -176,10 +176,7 @@
     w_BuiltinAssertionError = space.getitem(space.builtin.w_dict, 
                                             space.wrap('AssertionError'))
     w_metaclass = space.type(w_BuiltinAssertionError)
-    w_init = space.wrap(gateway.interp2app_temp(my_init,
-                                                unwrap_spec=[gateway.ObjSpace,
-                                                             gateway.W_Root,
-                                                             gateway.Arguments]))
+    w_init = space.wrap(gateway.interp2app_temp(my_init)
     w_dict = space.newdict()
     space.setitem(w_dict, space.wrap('__init__'), w_init)
     return space.call_function(w_metaclass,
@@ -247,11 +244,7 @@
     raise OperationError(space.w_AssertionError,
                          space.wrap("DID NOT RAISE"))
 
-app_raises = gateway.interp2app_temp(pypyraises,
-                                     unwrap_spec=[gateway.ObjSpace,
-                                                  gateway.W_Root,
-                                                  gateway.W_Root,
-                                                  gateway.Arguments])
+app_raises = gateway.interp2app_temp(pypyraises)
 
 def pypyskip(space, w_message): 
     """skip a test at app-level. """ 

From commits-noreply at bitbucket.org  Wed Feb 16 19:20:30 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 19:20:30 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: Oops
Message-ID: <20110216182030.3FC8D282BF2@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42090:d7173c0d195d
Date: 2011-02-16 19:07 +0100
http://bitbucket.org/pypy/pypy/changeset/d7173c0d195d/

Log:	Oops

diff --git a/pypy/tool/pytest/appsupport.py b/pypy/tool/pytest/appsupport.py
--- a/pypy/tool/pytest/appsupport.py
+++ b/pypy/tool/pytest/appsupport.py
@@ -176,7 +176,7 @@
     w_BuiltinAssertionError = space.getitem(space.builtin.w_dict, 
                                             space.wrap('AssertionError'))
     w_metaclass = space.type(w_BuiltinAssertionError)
-    w_init = space.wrap(gateway.interp2app_temp(my_init)
+    w_init = space.wrap(gateway.interp2app_temp(my_init))
     w_dict = space.newdict()
     space.setitem(w_dict, space.wrap('__init__'), w_init)
     return space.call_function(w_metaclass,

From commits-noreply at bitbucket.org  Wed Feb 16 19:20:31 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Wed, 16 Feb 2011 19:20:31 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: New unwrap_spec in
	interpreter/buffer.py
Message-ID: <20110216182031.2B6CE282BF2@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42091:510c7d13e5b9
Date: 2011-02-16 19:08 +0100
http://bitbucket.org/pypy/pypy/changeset/510c7d13e5b9/

Log:	New unwrap_spec in interpreter/buffer.py

diff --git a/pypy/interpreter/buffer.py b/pypy/interpreter/buffer.py
--- a/pypy/interpreter/buffer.py
+++ b/pypy/interpreter/buffer.py
@@ -17,7 +17,7 @@
 import operator
 from pypy.interpreter.baseobjspace import Wrappable
 from pypy.interpreter.typedef import TypeDef
-from pypy.interpreter.gateway import interp2app, ObjSpace, W_Root
+from pypy.interpreter.gateway import interp2app, unwrap_spec
 from pypy.interpreter.error import OperationError
 from pypy.rlib.objectmodel import compute_hash
 
@@ -47,7 +47,6 @@
 
     def descr_len(self, space):
         return space.wrap(self.getlength())
-    descr_len.unwrap_spec = ['self', ObjSpace]
 
     def descr_getitem(self, space, w_index):
         start, stop, step, size = space.decode_index4(w_index, self.getlength())
@@ -55,8 +54,8 @@
             return space.wrap(self.getitem(start))
         res = self.getslice(start, stop, step, size)
         return space.wrap(res)
-    descr_getitem.unwrap_spec = ['self', ObjSpace, W_Root]
 
+    @unwrap_spec(newstring='bufferstr')
     def descr_setitem(self, space, w_index, newstring):
         if not isinstance(self, RWBuffer):
             raise OperationError(space.w_TypeError,
@@ -78,19 +77,16 @@
             raise OperationError(space.w_ValueError,
                                  space.wrap("buffer object does not support"
                                             " slicing with a step"))
-    descr_setitem.unwrap_spec = ['self', ObjSpace, W_Root, 'bufferstr']
 
     def descr__buffer__(self, space):
         return space.wrap(self)
-    descr__buffer__.unwrap_spec = ['self', ObjSpace]
 
     def descr_str(self, space):
         return space.wrap(self.as_str())
-    descr_str.unwrap_spec = ['self', ObjSpace]
 
+    @unwrap_spec(other='bufferstr')
     def descr_add(self, space, other):
         return space.wrap(self.as_str() + other)
-    descr_add.unwrap_spec = ['self', ObjSpace, 'bufferstr']
 
     def _make_descr__cmp(name):
         def descr__cmp(self, space, w_other):
@@ -101,7 +97,6 @@
             str1 = self.as_str()
             str2 = other.as_str()
             return space.wrap(getattr(operator, name)(str1, str2))
-        descr__cmp.unwrap_spec = ['self', ObjSpace, W_Root]
         descr__cmp.func_name = name
         return descr__cmp
 
@@ -114,7 +109,6 @@
 
     def descr_hash(self, space):
         return space.wrap(compute_hash(self.as_str()))
-    descr_hash.unwrap_spec = ['self', ObjSpace]
 
     def descr_mul(self, space, w_times):
         # xxx not the most efficient implementation
@@ -122,7 +116,6 @@
         # use the __mul__ method instead of space.mul() so that we
         # return NotImplemented instead of raising a TypeError
         return space.call_method(w_string, '__mul__', w_times)
-    descr_mul.unwrap_spec = ['self', ObjSpace, W_Root]
 
     def descr_repr(self, space):
         if isinstance(self, RWBuffer):
@@ -133,7 +126,6 @@
 
         return space.wrap("<%s for 0x%s, size %d>" %
                           (info, addrstring, self.getlength()))
-    descr_repr.unwrap_spec = ['self', ObjSpace]
 
 
 class RWBuffer(Buffer):
@@ -150,7 +142,7 @@
         for i in range(len(string)):
             self.setitem(start + i, string[i])
 
-
+ at unwrap_spec(offset=int, size=int)
 def descr_buffer__new__(space, w_subtype, w_object, offset=0, size=-1):
     # w_subtype can only be exactly 'buffer' for now
     if not space.is_w(w_subtype, space.gettypefor(Buffer)):
@@ -184,7 +176,6 @@
     else:
         buffer = SubBuffer(buffer, offset, size)
     return space.wrap(buffer)
-descr_buffer__new__.unwrap_spec = [ObjSpace, W_Root, W_Root, int, int]
 
 
 Buffer.typedef = TypeDef(

From commits-noreply at bitbucket.org  Thu Feb 17 00:42:24 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Thu, 17 Feb 2011 00:42:24 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: GetSetProperty now accept
	arguments in both	order: (space, self) or (self, space)
Message-ID: <20110216234224.59B08282BF2@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42092:fe81f52a772c
Date: 2011-02-17 00:36 +0100
http://bitbucket.org/pypy/pypy/changeset/fe81f52a772c/

Log:	GetSetProperty now accept arguments in both order: (space, self) or
	(self, space) Use the more natural order when the getter is a class
	method

diff --git a/pypy/module/_demo/demo.py b/pypy/module/_demo/demo.py
--- a/pypy/module/_demo/demo.py
+++ b/pypy/module/_demo/demo.py
@@ -58,10 +58,10 @@
         y = space.int_w(w_y)
         return space.wrap(self.x * y)
 
-    def fget_x(space, self):
+    def fget_x(self, space):
         return space.wrap(self.x)
 
-    def fset_x(space, self, w_value):
+    def fset_x(self, space, w_value):
         self.x = space.int_w(w_value)
 
 @unwrap_spec(x=int)

diff --git a/pypy/interpreter/pycode.py b/pypy/interpreter/pycode.py
--- a/pypy/interpreter/pycode.py
+++ b/pypy/interpreter/pycode.py
@@ -267,19 +267,19 @@
         co = self._to_code()
         dis.dis(co)
 
-    def fget_co_consts(space, self):
+    def fget_co_consts(self, space):
         return space.newtuple(self.co_consts_w)
     
-    def fget_co_names(space, self):
+    def fget_co_names(self, space):
         return space.newtuple(self.co_names_w)
 
-    def fget_co_varnames(space, self):
+    def fget_co_varnames(self, space):
         return space.newtuple([space.wrap(name) for name in self.co_varnames])
 
-    def fget_co_cellvars(space, self):
+    def fget_co_cellvars(self, space):
         return space.newtuple([space.wrap(name) for name in self.co_cellvars])
 
-    def fget_co_freevars(space, self):
+    def fget_co_freevars(self, space):
         return space.newtuple([space.wrap(name) for name in self.co_freevars])    
 
     def descr_code__eq__(self, w_other):

diff --git a/pypy/interpreter/pytraceback.py b/pypy/interpreter/pytraceback.py
--- a/pypy/interpreter/pytraceback.py
+++ b/pypy/interpreter/pytraceback.py
@@ -21,7 +21,7 @@
     def get_lineno(self):
         return offset2lineno(self.frame.pycode, self.lasti)
 
-    def descr_tb_lineno(space, self):
+    def descr_tb_lineno(self, space):
         return space.wrap(self.get_lineno())
 
     def descr__reduce__(self, space):

diff --git a/pypy/module/exceptions/interp_exceptions.py b/pypy/module/exceptions/interp_exceptions.py
--- a/pypy/module/exceptions/interp_exceptions.py
+++ b/pypy/module/exceptions/interp_exceptions.py
@@ -140,10 +140,10 @@
         clsname = self.getclass(space).getname(space, '?')
         return space.wrap(clsname + args_repr)
 
-    def descr_getargs(space, self):
+    def descr_getargs(self, space):
         return space.newtuple(self.args_w)
 
-    def descr_setargs(space, self, w_newargs):
+    def descr_setargs(self, space, w_newargs):
         self.args_w = space.fixedview(w_newargs)
 
     def descr_getitem(self, space, w_index):
@@ -169,7 +169,7 @@
         w_olddict = self.getdict()
         space.call_method(w_olddict, 'update', w_dict)
 
-    def descr_message_get(space, self):
+    def descr_message_get(self, space):
         w_dict = self.w_dict
         if w_dict is not None:
             w_msg = space.finditem(w_dict, space.wrap("message"))
@@ -182,10 +182,10 @@
                    space.w_DeprecationWarning)
         return self.w_message
 
-    def descr_message_set(space, self, w_new):
+    def descr_message_set(self, space, w_new):
         space.setitem(self.getdict(), space.wrap("message"), w_new)
 
-    def descr_message_del(space, self):
+    def descr_message_del(self, space):
         w_dict = self.w_dict
         if w_dict is not None:
             try:

diff --git a/pypy/module/_stackless/interp_coroutine.py b/pypy/module/_stackless/interp_coroutine.py
--- a/pypy/module/_stackless/interp_coroutine.py
+++ b/pypy/module/_stackless/interp_coroutine.py
@@ -294,15 +294,15 @@
         setattr(AppCoroutine, meth.__name__, meth.im_func)
 del meth, methname
 
-def w_get_is_zombie(space, self):
+def w_get_is_zombie(self, space):
     return space.wrap(self.get_is_zombie())
 AppCoroutine.w_get_is_zombie = w_get_is_zombie
 
-def w_get_is_alive(space, self):
+def w_get_is_alive(self, space):
     return space.wrap(self.is_alive())
 AppCoroutine.w_get_is_alive = w_get_is_alive
 
-def w_descr__framestack(space, self):
+def w_descr__framestack(self, space):
     assert isinstance(self, AppCoroutine)
     counter = 0
     f = self.subctx.topframe

diff --git a/pypy/module/_winreg/interp_winreg.py b/pypy/module/_winreg/interp_winreg.py
--- a/pypy/module/_winreg/interp_winreg.py
+++ b/pypy/module/_winreg/interp_winreg.py
@@ -27,7 +27,7 @@
     def descr_nonzero(self, space):
         return space.wrap(self.as_int() != 0)
 
-    def descr_handle_get(space, self):
+    def descr_handle_get(self, space):
         return space.wrap(self.as_int())
 
     def descr_repr(self, space):

diff --git a/pypy/module/oracle/interp_cursor.py b/pypy/module/oracle/interp_cursor.py
--- a/pypy/module/oracle/interp_cursor.py
+++ b/pypy/module/oracle/interp_cursor.py
@@ -369,7 +369,7 @@
 
         self.fetchVariables = None
 
-    def getDescription(space, self):
+    def getDescription(self, space):
         "Return a list of 7-tuples consisting of the description of "
         "the define variables"
 
@@ -1046,23 +1046,23 @@
         self.outputSizeColumn = outputSizeColumn
 
 
-    def arraysize_get(space, self):
+    def arraysize_get(self, space):
         return space.wrap(self.arraySize)
-    def arraysize_set(space, self, w_value):
+    def arraysize_set(self, space, w_value):
         self.arraySize = space.int_w(w_value)
 
-    def bindarraysize_get(space, self):
+    def bindarraysize_get(self, space):
         return space.wrap(self.bindArraySize)
-    def bindarraysize_set(space, self, w_value):
+    def bindarraysize_set(self, space, w_value):
         self.bindArraySize = space.int_w(w_value)
 
-    def bindvars_get(space, self):
+    def bindvars_get(self, space):
         if self.bindList:
             return space.newlist(self.bindList)
         if self.bindDict:
             return self.bindDict
 
-    def fetchvars_get(space, self):
+    def fetchvars_get(self, space):
         return space.newlist(self.fetchVariables)
 
 W_Cursor.typedef = TypeDef(

diff --git a/pypy/module/cpyext/methodobject.py b/pypy/module/cpyext/methodobject.py
--- a/pypy/module/cpyext/methodobject.py
+++ b/pypy/module/cpyext/methodobject.py
@@ -102,7 +102,7 @@
                 w_arg = w_args
             return generic_cpy_call(space, func, w_self, w_arg)
 
-    def get_doc(space, self):
+    def get_doc(self, space):
         doc = self.ml.c_ml_doc
         if doc:
             return space.wrap(rffi.charp2str(doc))

diff --git a/pypy/interpreter/nestedscope.py b/pypy/interpreter/nestedscope.py
--- a/pypy/interpreter/nestedscope.py
+++ b/pypy/interpreter/nestedscope.py
@@ -67,7 +67,7 @@
         return "<%s(%s) at 0x%x>" % (self.__class__.__name__,
                                      content, uid(self))
 
-    def descr__cell_contents(space, self):
+    def descr__cell_contents(self, space):
         try:
             return self.get()
         except ValueError:

diff --git a/pypy/module/zipimport/interp_zipimport.py b/pypy/module/zipimport/interp_zipimport.py
--- a/pypy/module/zipimport/interp_zipimport.py
+++ b/pypy/module/zipimport/interp_zipimport.py
@@ -117,7 +117,7 @@
             space.getbuiltinmodule('zipimport'),
             space.wrap('ZipImportError'))
 
-    def getprefix(space, self):
+    def getprefix(self, space):
         return space.wrap(self.prefix)
 
     def _find_relative_path(self, filename):
@@ -331,7 +331,7 @@
         raise operationerrfmt(self.w_ZipImportError,
             "Cannot find module %s in %s", filename, self.name)
 
-    def getarchive(space, self):
+    def getarchive(self, space):
         space = self.space
         return space.wrap(self.filename)
 

diff --git a/pypy/module/pyexpat/interp_pyexpat.py b/pypy/module/pyexpat/interp_pyexpat.py
--- a/pypy/module/pyexpat/interp_pyexpat.py
+++ b/pypy/module/pyexpat/interp_pyexpat.py
@@ -600,21 +600,21 @@
         self.w_error = w_error
         return OperationError(w_errorcls, w_error)
 
-    def descr_ErrorCode(space, self):
+    def descr_ErrorCode(self, space):
         return space.wrap(XML_GetErrorCode(self.itself))
 
-    def descr_ErrorLineNumber(space, self):
+    def descr_ErrorLineNumber(self, space):
         return space.wrap(XML_GetErrorLineNumber(self.itself))
 
-    def descr_ErrorColumnNumber(space, self):
+    def descr_ErrorColumnNumber(self, space):
         return space.wrap(XML_GetErrorColumnNumber(self.itself))
 
-    def descr_ErrorByteIndex(space, self):
+    def descr_ErrorByteIndex(self, space):
         return space.wrap(XML_GetErrorByteIndex(self.itself))
 
-    def get_buffer_size(space, self):
+    def get_buffer_size(self, space):
         return space.wrap(self.buffer_size)
-    def set_buffer_size(space, self, w_value):
+    def set_buffer_size(self, space, w_value):
         value = space.getindex_w(w_value, space.w_TypeError)
         if value <= 0:
             raise OperationError(space.w_ValueError, space.wrap(
@@ -622,9 +622,9 @@
         self.flush_character_buffer(space)
         self.buffer_size = value
 
-    def get_buffer_text(space, self):
+    def get_buffer_text(self, space):
         return space.wrap(self.buffer_w is not None)
-    def set_buffer_text(space, self, w_value):
+    def set_buffer_text(self, space, w_value):
         if space.is_true(w_value):
             self.buffer_w = []
             self.buffer_used = 0
@@ -632,7 +632,7 @@
             self.flush_character_buffer(space)
             self.buffer_w = None
 
-    def get_intern(space, self):
+    def get_intern(self, space):
         if self.w_intern:
             return self.w_intern
         else:

diff --git a/pypy/module/_lsprof/interp_lsprof.py b/pypy/module/_lsprof/interp_lsprof.py
--- a/pypy/module/_lsprof/interp_lsprof.py
+++ b/pypy/module/_lsprof/interp_lsprof.py
@@ -17,7 +17,7 @@
         self.tt = tt
         self.w_calls = w_sublist
 
-    def get_calls(space, self):
+    def get_calls(self, space):
         return self.w_calls
 
     def repr(self, space):
@@ -30,7 +30,7 @@
             frame_repr, self.callcount, self.reccallcount,
             self.tt, self.it, calls_repr))
 
-    def get_code(space, self):
+    def get_code(self, space):
         return self.frame
 
 W_StatsEntry.typedef = TypeDef(
@@ -57,7 +57,7 @@
         return space.wrap('("%s", %d, %d, %f, %f)' % (
             frame_repr, self.callcount, self.reccallcount, self.tt, self.it))
 
-    def get_code(space, self):
+    def get_code(self, space):
         return self.frame
 
 W_StatsSubEntry.typedef = TypeDef(

diff --git a/pypy/module/_io/interp_textio.py b/pypy/module/_io/interp_textio.py
--- a/pypy/module/_io/interp_textio.py
+++ b/pypy/module/_io/interp_textio.py
@@ -964,7 +964,7 @@
         self._check_init(space)
         return space.wrap(self.chunk_size)
 
-    def chunk_size_set_w(space, self, w_size):
+    def chunk_size_set_w(self, space, w_size):
         self._check_init(space)
         size = space.int_w(w_size)
         if size <= 0:

diff --git a/pypy/module/_multiprocessing/interp_semaphore.py b/pypy/module/_multiprocessing/interp_semaphore.py
--- a/pypy/module/_multiprocessing/interp_semaphore.py
+++ b/pypy/module/_multiprocessing/interp_semaphore.py
@@ -402,11 +402,11 @@
         self.count = 0
         self.maxvalue = maxvalue
 
-    def kind_get(space, self):
+    def kind_get(self, space):
         return space.newint(self.kind)
-    def maxvalue_get(space, self):
+    def maxvalue_get(self, space):
         return space.newint(self.maxvalue)
-    def handle_get(space, self):
+    def handle_get(self, space):
         return w_handle(space, self.handle)
 
     def get_count(self, space):

diff --git a/pypy/module/_multiprocessing/interp_connection.py b/pypy/module/_multiprocessing/interp_connection.py
--- a/pypy/module/_multiprocessing/interp_connection.py
+++ b/pypy/module/_multiprocessing/interp_connection.py
@@ -53,11 +53,11 @@
     def close(self):
         self.do_close()
 
-    def closed_get(space, self):
+    def closed_get(self, space):
         return space.newbool(not self.is_valid())
-    def readable_get(space, self):
+    def readable_get(self, space):
         return space.newbool(bool(self.flags & READABLE))
-    def writable_get(space, self):
+    def writable_get(self, space):
         return space.newbool(bool(self.flags & WRITABLE))
 
     def _check_readable(self, space):

diff --git a/pypy/module/_stackless/interp_greenlet.py b/pypy/module/_stackless/interp_greenlet.py
--- a/pypy/module/_stackless/interp_greenlet.py
+++ b/pypy/module/_stackless/interp_greenlet.py
@@ -139,13 +139,13 @@
     def isdead(self):
         return self.thunk is None and not self.active
 
-    def w_get_is_dead(space, self):
+    def w_get_is_dead(self, space):
         return space.newbool(self.isdead())
 
     def descr__nonzero__(self):
         return self.space.newbool(self.active)
 
-    def w_get_run(space, self):
+    def w_get_run(self, space):
         w_run = self.w_callable
         if w_run is None:
             raise OperationError(space.w_AttributeError, space.wrap("run"))
@@ -159,15 +159,15 @@
                                             "after the start of the greenlet"))
         self.w_callable = w_run
 
-    def w_set_run(space, self, w_run):
+    def w_set_run(self, space, w_run):
         self.set_run(w_run)
 
-    def w_del_run(space, self):
+    def w_del_run(self, space):
         if self.w_callable is None:
             raise OperationError(space.w_AttributeError, space.wrap("run"))
         self.w_callable = None
 
-    def w_get_parent(space, self):
+    def w_get_parent(self, space):
         return space.wrap(self.parent)
 
     def set_parent(self, w_parent):
@@ -184,10 +184,10 @@
             curr = curr.parent
         self.parent = newparent
 
-    def w_set_parent(space, self, w_parent):
+    def w_set_parent(self, space, w_parent):
         self.set_parent(w_parent)
 
-    def w_get_frame(space, self):
+    def w_get_frame(self, space):
         if not self.active or self.costate.current is self:
             f = None
         else:

diff --git a/pypy/module/_hashlib/interp_hashlib.py b/pypy/module/_hashlib/interp_hashlib.py
--- a/pypy/module/_hashlib/interp_hashlib.py
+++ b/pypy/module/_hashlib/interp_hashlib.py
@@ -67,10 +67,10 @@
             result.append(hexdigits[ ord(c)       & 0xf])
         return space.wrap(result.build())
 
-    def get_digest_size(space, self):
+    def get_digest_size(self, space):
         return space.wrap(self._digest_size())
 
-    def get_block_size(space, self):
+    def get_block_size(self, space):
         return space.wrap(self._block_size())
 
     def _digest(self, space):

diff --git a/pypy/module/_sre/interp_sre.py b/pypy/module/_sre/interp_sre.py
--- a/pypy/module/_sre/interp_sre.py
+++ b/pypy/module/_sre/interp_sre.py
@@ -431,7 +431,7 @@
             return mark.gid // 2 + 1
         return -1
 
-    def fget_lastgroup(space, self):
+    def fget_lastgroup(self, space):
         lastindex = self._last_index()
         if lastindex < 0:
             return space.w_None
@@ -441,19 +441,19 @@
             return space.w_None
         return w_result
 
-    def fget_lastindex(space, self):
+    def fget_lastindex(self, space):
         lastindex = self._last_index()
         if lastindex >= 0:
             return space.wrap(lastindex)
         return space.w_None
 
-    def fget_pos(space, self):
+    def fget_pos(self, space):
         return space.wrap(self.ctx.original_pos)
 
-    def fget_endpos(space, self):
+    def fget_endpos(self, space):
         return space.wrap(self.ctx.end)
 
-    def fget_regs(space, self):
+    def fget_regs(self, space):
         space = self.space
         fmarks = self.flatten_marks()
         num_groups = self.srepat.num_groups
@@ -466,7 +466,7 @@
                                               space.wrap(fmarks[i*2+1])])
         return space.newtuple(result_w)
 
-    def fget_string(space, self):
+    def fget_string(self, space):
         return self.ctx._w_string(space)
 
 

diff --git a/pypy/interpreter/generator.py b/pypy/interpreter/generator.py
--- a/pypy/interpreter/generator.py
+++ b/pypy/interpreter/generator.py
@@ -127,16 +127,16 @@
             msg = "generator ignored GeneratorExit"
             raise OperationError(space.w_RuntimeError, space.wrap(msg))
 
-    def descr_gi_frame(space, self):
+    def descr_gi_frame(self, space):
         if self.frame is not None and not self.frame.frame_finished_execution:
             return self.frame
         else:
             return space.w_None
 
-    def descr_gi_code(space, self):
+    def descr_gi_code(self, space):
         return self.pycode
 
-    def descr__name__(space, self):
+    def descr__name__(self, space):
         code_name = self.pycode.co_name
         return space.wrap(code_name)
 

diff --git a/pypy/module/_rawffi/interp_rawffi.py b/pypy/module/_rawffi/interp_rawffi.py
--- a/pypy/module/_rawffi/interp_rawffi.py
+++ b/pypy/module/_rawffi/interp_rawffi.py
@@ -267,7 +267,7 @@
                 ll_buf = rffi.cast(lltype.Signed, self.ll_buffer)
                 tracker.trace_allocation(ll_buf, self)
 
-    def getbuffer(space, self):
+    def getbuffer(self, space):
         return space.wrap(rffi.cast(lltype.Unsigned, self.ll_buffer))
 
     def byptr(self, space):
@@ -370,18 +370,13 @@
         self.argshapes = argshapes
         self.resshape = resshape
 
-    def getbuffer(space, self):
-        return space.wrap(rffi.cast(lltype.Unsigned, self.ptr.funcsym))
-
-    # XXX exactly the same as previous one, but arguments are suitable
-    #     for calling with python
-    def _getbuffer(self, space):
+    def getbuffer(self, space):
         return space.wrap(rffi.cast(lltype.Unsigned, self.ptr.funcsym))
 
     def byptr(self, space):
         from pypy.module._rawffi.array import ARRAY_OF_PTRS
         array = ARRAY_OF_PTRS.allocate(space, 1)
-        array.setitem(space, 0, self._getbuffer(space))
+        array.setitem(space, 0, self.getbuffer(space))
         if tracker.DO_TRACING:
             # XXX this is needed, because functions tend to live forever
             #     hence our testing is not performing that well

diff --git a/pypy/module/oracle/interp_pool.py b/pypy/module/oracle/interp_pool.py
--- a/pypy/module/oracle/interp_pool.py
+++ b/pypy/module/oracle/interp_pool.py
@@ -172,7 +172,7 @@
         connection.handle = lltype.nullptr(roci.OCISvcCtx.TO)
 
 def computedProperty(oci_attr_code, oci_value_type):
-    def fget(space, self):
+    def fget(self, space):
         self.checkConnected(space)
 
         valueptr = lltype.malloc(rffi.CArrayPtr(oci_value_type).TO,

diff --git a/pypy/interpreter/function.py b/pypy/interpreter/function.py
--- a/pypy/interpreter/function.py
+++ b/pypy/interpreter/function.py
@@ -324,13 +324,13 @@
         self.defs_w    = space.fixedview(w_defs_w)
         self.w_module = w_module
 
-    def fget_func_defaults(space, self):
+    def fget_func_defaults(self, space):
         values_w = self.defs_w
         if not values_w or None in values_w:
             return space.w_None
         return space.newtuple(values_w)
 
-    def fset_func_defaults(space, self, w_defaults):
+    def fset_func_defaults(self, space, w_defaults):
         if space.is_w(w_defaults, space.w_None):
             self.defs_w = []
             return
@@ -338,21 +338,21 @@
             raise OperationError( space.w_TypeError, space.wrap("func_defaults must be set to a tuple object or None") )
         self.defs_w = space.fixedview(w_defaults)
 
-    def fdel_func_defaults(space, self):
+    def fdel_func_defaults(self, space):
         self.defs_w = []
 
-    def fget_func_doc(space, self):
+    def fget_func_doc(self, space):
         if self.w_doc is None:
             self.w_doc = self.code.getdocstring(space)
         return self.w_doc
 
-    def fset_func_doc(space, self, w_doc):
+    def fset_func_doc(self, space, w_doc):
         self.w_doc = w_doc
 
-    def fget_func_name(space, self):
+    def fget_func_name(self, space):
         return space.wrap(self.name)
 
-    def fset_func_name(space, self, w_name):
+    def fset_func_name(self, space, w_name):
         try:
             self.name = space.str_w(w_name)
         except OperationError, e:
@@ -363,10 +363,10 @@
             raise
 
 
-    def fdel_func_doc(space, self):
+    def fdel_func_doc(self, space):
         self.w_doc = space.w_None
 
-    def fget___module__(space, self):
+    def fget___module__(self, space):
         if self.w_module is None:
             if self.w_func_globals is not None and not space.is_w(self.w_func_globals, space.w_None):
                 self.w_module = space.call_method( self.w_func_globals, "get", space.wrap("__name__") )
@@ -374,16 +374,16 @@
                 self.w_module = space.w_None
         return self.w_module
 
-    def fset___module__(space, self, w_module):
+    def fset___module__(self, space, w_module):
         self.w_module = w_module
 
-    def fdel___module__(space, self):
+    def fdel___module__(self, space):
         self.w_module = space.w_None
 
-    def fget_func_code(space, self):
+    def fget_func_code(self, space):
         return space.wrap(self.code)
 
-    def fset_func_code(space, self, w_code):
+    def fset_func_code(self, space, w_code):
         from pypy.interpreter.pycode import PyCode
         if not self.can_change_code:
             raise OperationError(space.w_TypeError,
@@ -398,7 +398,7 @@
                 self.name, closure_len, len(code.co_freevars))
         self.code = code
 
-    def fget_func_closure(space, self):
+    def fget_func_closure(self, space):
         if self.closure is not None:
             w_res = space.newtuple( [ space.wrap(i) for i in self.closure ] )
         else:

diff --git a/pypy/module/oracle/interp_object.py b/pypy/module/oracle/interp_object.py
--- a/pypy/module/oracle/interp_object.py
+++ b/pypy/module/oracle/interp_object.py
@@ -271,7 +271,7 @@
             self.attributes.append(attribute)
             self.attributesByName[attribute.name] = attribute
 
-    def get_attributes(space, self):
+    def get_attributes(self, space):
         return space.newlist([space.wrap(attr) for attr in self.attributes])
 
 W_ObjectType.typedef = TypeDef(

diff --git a/pypy/module/__builtin__/interp_memoryview.py b/pypy/module/__builtin__/interp_memoryview.py
--- a/pypy/module/__builtin__/interp_memoryview.py
+++ b/pypy/module/__builtin__/interp_memoryview.py
@@ -105,19 +105,19 @@
     def descr_len(self, space):
         return self.buf.descr_len(space)
 
-    def w_get_format(space, self):
+    def w_get_format(self, space):
         return space.wrap("B")
-    def w_get_itemsize(space, self):
+    def w_get_itemsize(self, space):
         return space.wrap(1)
-    def w_get_ndim(space, self):
+    def w_get_ndim(self, space):
         return space.wrap(1)
-    def w_is_readonly(space, self):
+    def w_is_readonly(self, space):
         return space.wrap(not isinstance(self.buf, buffer.RWBuffer))
-    def w_get_shape(space, self):
+    def w_get_shape(self, space):
         return space.newtuple([space.wrap(self.getlength())])
-    def w_get_strides(space, self):
+    def w_get_strides(self, space):
         return space.newtuple([space.wrap(1)])
-    def w_get_suboffsets(space, self):
+    def w_get_suboffsets(self, space):
         # I've never seen anyone filling this field
         return space.w_None
 

diff --git a/pypy/module/select/interp_epoll.py b/pypy/module/select/interp_epoll.py
--- a/pypy/module/select/interp_epoll.py
+++ b/pypy/module/select/interp_epoll.py
@@ -104,7 +104,7 @@
             if result < 0:
                 raise exception_from_errno(self.space, self.space.w_IOError)
 
-    def descr_get_closed(space, self):
+    def descr_get_closed(self, space):
         return space.wrap(self.epfd < 0)
 
     def descr_fileno(self, space):

diff --git a/pypy/interpreter/eval.py b/pypy/interpreter/eval.py
--- a/pypy/interpreter/eval.py
+++ b/pypy/interpreter/eval.py
@@ -76,10 +76,10 @@
     def getcode(self):
         return None
 
-    def fget_code(space, self):
+    def fget_code(self, space):
         return space.wrap(self.getcode())
 
-    def fget_getdictscope(space, self): # unwrapping through unwrap_spec in typedef.py
+    def fget_getdictscope(self, space):
         return self.getdictscope()
 
     def setdictscope(self, w_locals):

diff --git a/pypy/module/cStringIO/interp_stringio.py b/pypy/module/cStringIO/interp_stringio.py
--- a/pypy/module/cStringIO/interp_stringio.py
+++ b/pypy/module/cStringIO/interp_stringio.py
@@ -197,13 +197,13 @@
 
 # ____________________________________________________________
 
-def descr_closed(space, self):
+def descr_closed(self, space):
     return space.wrap(self.is_closed())
 
-def descr_softspace(space, self):
+def descr_softspace(self, space):
     return space.wrap(self.softspace)
 
-def descr_setsoftspace(space, self, w_newvalue):
+def descr_setsoftspace(self, space, w_newvalue):
     self.softspace = space.int_w(w_newvalue)
 
 common_descrs = {

diff --git a/pypy/module/oracle/interp_connect.py b/pypy/module/oracle/interp_connect.py
--- a/pypy/module/oracle/interp_connect.py
+++ b/pypy/module/oracle/interp_connect.py
@@ -486,14 +486,14 @@
             rffi.keep_buffer_alive_until_here(charsetname_buf, charsetname)
         return space.wrap(charset)
 
-    def get_encoding(space, self):
+    def get_encoding(self, space):
         return self._getCharacterSetName(space, roci.OCI_ATTR_ENV_CHARSET_ID)
-    def get_nationalencoding(space, self):
+    def get_nationalencoding(self, space):
         return self._getCharacterSetName(space, roci.OCI_ATTR_ENV_CHARSET_ID)
-    def get_maxbytespercharacter(space, self):
+    def get_maxbytespercharacter(self, space):
         return space.wrap(self.environment.maxBytesPerCharacter)
 
-    def get_version(space, self):
+    def get_version(self, space):
         # if version has already been determined, no need to determine again
         if self.w_version:
             return self.w_version

diff --git a/pypy/interpreter/pyframe.py b/pypy/interpreter/pyframe.py
--- a/pypy/interpreter/pyframe.py
+++ b/pypy/interpreter/pyframe.py
@@ -442,16 +442,14 @@
 
     ### line numbers ###
 
-    # for f*_f_* unwrapping through unwrap_spec in typedef.py
-
-    def fget_f_lineno(space, self): 
+    def fget_f_lineno(self, space): 
         "Returns the line number of the instruction currently being executed."
         if self.w_f_trace is None:
             return space.wrap(self.get_last_lineno())
         else:
             return space.wrap(self.f_lineno)
 
-    def fset_f_lineno(space, self, w_new_lineno):
+    def fset_f_lineno(self, space, w_new_lineno):
         "Returns the line number of the instruction currently being executed."
         try:
             new_lineno = space.int_w(w_new_lineno)
@@ -585,19 +583,19 @@
         "Returns the line number of the instruction currently being executed."
         return pytraceback.offset2lineno(self.pycode, self.last_instr)
 
-    def fget_f_builtins(space, self):
+    def fget_f_builtins(self, space):
         return self.get_builtin().getdict()
 
-    def fget_f_back(space, self):
+    def fget_f_back(self, space):
         return self.space.wrap(self.f_backref())
 
-    def fget_f_lasti(space, self):
+    def fget_f_lasti(self, space):
         return self.space.wrap(self.last_instr)
 
-    def fget_f_trace(space, self):
+    def fget_f_trace(self, space):
         return self.w_f_trace
 
-    def fset_f_trace(space, self, w_trace):
+    def fset_f_trace(self, space, w_trace):
         if space.is_w(w_trace, space.w_None):
             self.w_f_trace = None
         else:
@@ -605,10 +603,10 @@
             self.f_lineno = self.get_last_lineno()
             space.frame_trace_action.fire()
 
-    def fdel_f_trace(space, self): 
+    def fdel_f_trace(self, space): 
         self.w_f_trace = None 
 
-    def fget_f_exc_type(space, self):
+    def fget_f_exc_type(self, space):
         if self.last_exception is not None:
             f = self.f_backref()
             while f is not None and f.last_exception is None:
@@ -617,7 +615,7 @@
                 return f.last_exception.w_type
         return space.w_None
          
-    def fget_f_exc_value(space, self):
+    def fget_f_exc_value(self, space):
         if self.last_exception is not None:
             f = self.f_backref()
             while f is not None and f.last_exception is None:
@@ -626,7 +624,7 @@
                 return f.last_exception.get_w_value(space)
         return space.w_None
 
-    def fget_f_exc_traceback(space, self):
+    def fget_f_exc_traceback(self, space):
         if self.last_exception is not None:
             f = self.f_backref()
             while f is not None and f.last_exception is None:
@@ -635,7 +633,7 @@
                 return space.wrap(f.last_exception.application_traceback)
         return space.w_None
          
-    def fget_f_restricted(space, self):
+    def fget_f_restricted(self, space):
         if space.config.objspace.honor__builtins__:
             return space.wrap(self.builtin is not space.builtin)
         return space.w_False

From commits-noreply at bitbucket.org  Thu Feb 17 00:42:24 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Thu, 17 Feb 2011 00:42:24 +0100 (CET)
Subject: [pypy-svn] pypy default: Avoid warning messages when the test suite
	finishes:
Message-ID: <20110216234224.00313282BF2@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r42093:d52d8b46fe48
Date: 2011-02-17 00:42 +0100
http://bitbucket.org/pypy/pypy/changeset/d52d8b46fe48/

Log:	Avoid warning messages when the test suite finishes: Exception
	AttributeError: "'NoneType' object has no attribute
	'nested_hash_level'" ignored

diff --git a/pypy/rpython/lltypesystem/lltype.py b/pypy/rpython/lltypesystem/lltype.py
--- a/pypy/rpython/lltypesystem/lltype.py
+++ b/pypy/rpython/lltypesystem/lltype.py
@@ -21,6 +21,8 @@
         weakref.WeakValueDictionary.__init__(self, *args, **kwargs)
         remove_base = self._remove
         def remove(*args):
+            if TLS is None: # Happens when the interpreter is shutting down
+                return remove_base(*args)
             nested_hash_level = TLS.nested_hash_level
             try:
                 # The 'remove' function is called when an object dies.  This

From agaynor at codespeak.net  Thu Feb 17 05:38:50 2011
From: agaynor at codespeak.net (agaynor at codespeak.net)
Date: Thu, 17 Feb 2011 05:38:50 +0100 (CET)
Subject: [pypy-svn] r80354 - pypy/extradoc/talk/ustour2011
Message-ID: <20110217043850.0C71A282BF2@codespeak.net>

Author: agaynor
Date: Thu Feb 17 05:38:47 2011
New Revision: 80354

Modified:
   pypy/extradoc/talk/ustour2011/mozilla-abstract.txt
Log:
Added my bio.


Modified: pypy/extradoc/talk/ustour2011/mozilla-abstract.txt
==============================================================================
--- pypy/extradoc/talk/ustour2011/mozilla-abstract.txt	(original)
+++ pypy/extradoc/talk/ustour2011/mozilla-abstract.txt	Thu Feb 17 05:38:47 2011
@@ -1,5 +1,5 @@
 PyPy Mozilla Talk:  Monday March 7th, 2011 at 2:30
-Speakers: Armin Rigo, Maciej Fijalkowski
+Speakers: Armin Rigo, Maciej Fijalkowski, Alex Gaynor
 
 Bios:
 Armin Rigo is a researcher at Heinrich-Heine-Universit?t in D?sseldorf,
@@ -27,6 +27,14 @@
 Karoo desert. The infrastructure is largely python-based and employs
 Twisted and PyPy among other technologies.
 
+Alex Gaynor is a core developer of PyPy. Since 2009 he has contributed to the
+JIT, the Python interpreter, working on issues of Python 2.7 compatibility and
+the CPython C-API, and implementing third party modules such as PostgreSQL
+bindings for PyPy. He is also a core developer of Django (a popular Python web
+framework), where he works on all elements on the framework. He is presently
+an independent contractor, primarily working in web development where he uses
+Python exclusively.
+
 Abstract:
 
 The PyPy project has recently gathered a lot of attention for its


From agaynor at codespeak.net  Thu Feb 17 06:06:57 2011
From: agaynor at codespeak.net (agaynor at codespeak.net)
Date: Thu, 17 Feb 2011 06:06:57 +0100 (CET)
Subject: [pypy-svn] r80355 - pypy/extradoc/talk/ustour2011
Message-ID: <20110217050657.A087C282BF2@codespeak.net>

Author: agaynor
Date: Thu Feb 17 06:06:56 2011
New Revision: 80355

Modified:
   pypy/extradoc/talk/ustour2011/mozilla-abstract.txt
Log:
Mention I'm a student.


Modified: pypy/extradoc/talk/ustour2011/mozilla-abstract.txt
==============================================================================
--- pypy/extradoc/talk/ustour2011/mozilla-abstract.txt	(original)
+++ pypy/extradoc/talk/ustour2011/mozilla-abstract.txt	Thu Feb 17 06:06:56 2011
@@ -33,7 +33,8 @@
 bindings for PyPy. He is also a core developer of Django (a popular Python web
 framework), where he works on all elements on the framework. He is presently
 an independent contractor, primarily working in web development where he uses
-Python exclusively.
+Python exclusively. Alex a student at Rensselaer Polytechnic Institute where
+he is a Junior pursuing a bachelors degree in Computer Science.
 
 Abstract:
 


From commits-noreply at bitbucket.org  Thu Feb 17 06:15:06 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Thu, 17 Feb 2011 06:15:06 +0100 (CET)
Subject: [pypy-svn] pypy default: Added a doc file for _collections (why
	does nobody run	the config tests!).
Message-ID: <20110217051506.3C79D282BF2@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r42094:323043b0914b
Date: 2011-02-17 00:14 -0500
http://bitbucket.org/pypy/pypy/changeset/323043b0914b/

Log:	Added a doc file for _collections (why does nobody run the config
	tests!).

diff --git a/pypy/doc/config/objspace.usemodules._collections.txt b/pypy/doc/config/objspace.usemodules._collections.txt
new file mode 100644
--- /dev/null
+++ b/pypy/doc/config/objspace.usemodules._collections.txt
@@ -0,0 +1,2 @@
+Use the '_collections' module.
+Used by the 'collections' standard lib module. This module is expected to be working and is included by default.

From commits-noreply at bitbucket.org  Thu Feb 17 06:15:06 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Thu, 17 Feb 2011 06:15:06 +0100 (CET)
Subject: [pypy-svn] pypy default: Merged upstream.
Message-ID: <20110217051506.7BDA3282BF7@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r42095:4dcb06422e19
Date: 2011-02-17 00:14 -0500
http://bitbucket.org/pypy/pypy/changeset/4dcb06422e19/

Log:	Merged upstream.


From commits-noreply at bitbucket.org  Thu Feb 17 06:44:08 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Thu, 17 Feb 2011 06:44:08 +0100 (CET)
Subject: [pypy-svn] pypy default: Rewrite defaultdict.__missing__ at
	interp-level.
Message-ID: <20110217054408.D3D7E282BF2@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r42096:054a832a4398
Date: 2011-02-17 04:53 +0100
http://bitbucket.org/pypy/pypy/changeset/054a832a4398/

Log:	Rewrite defaultdict.__missing__ at interp-level.

diff --git a/pypy/module/_collections/__init__.py b/pypy/module/_collections/__init__.py
--- a/pypy/module/_collections/__init__.py
+++ b/pypy/module/_collections/__init__.py
@@ -14,4 +14,5 @@
 
     interpleveldefs = {
         'deque' : 'interp_deque.W_Deque',
+        '__missing__': 'interp_defaultdict.missing',
         }

diff --git a/pypy/module/_collections/app_defaultdict.py b/pypy/module/_collections/app_defaultdict.py
--- a/pypy/module/_collections/app_defaultdict.py
+++ b/pypy/module/_collections/app_defaultdict.py
@@ -4,7 +4,10 @@
 #
 # The issue is that for now we don't support writing interp-level
 # subclasses of Wrappable that inherit at app-level from a type like
-# 'dict'.
+# 'dict'.  But what we can do is write individual methods at
+# interp-level.
+
+import _collections
 
 
 class defaultdict(dict):
@@ -19,11 +22,8 @@
         super(defaultdict, self).__init__(*args, **kwds)
  
     def __missing__(self, key):
-        # from defaultdict docs
-        if self.default_factory is None: 
-            raise KeyError(key)
-        self[key] = value = self.default_factory()
-        return value
+        pass    # this method is written at interp-level
+    __missing__.func_code = _collections.__missing__.func_code
 
     def __repr__(self, recurse=set()):
         # XXX not thread-safe, but good enough

diff --git a/pypy/module/_collections/interp_defaultdict.py b/pypy/module/_collections/interp_defaultdict.py
new file mode 100644
--- /dev/null
+++ b/pypy/module/_collections/interp_defaultdict.py
@@ -0,0 +1,12 @@
+from pypy.interpreter.error import OperationError
+
+def missing(space, w_self, w_key):
+    # An interp-level version of this method.  This is mostly only
+    # useful because it can be executed atomically in the presence of
+    # threads.
+    w_default_factory = space.getattr(w_self, space.wrap('default_factory'))
+    if space.is_w(w_default_factory, space.w_None):
+        raise OperationError(space.w_KeyError, w_key)
+    w_value = space.call_function(w_default_factory)
+    space.setitem(w_self, w_key, w_value)
+    return w_value

From lac at codespeak.net  Thu Feb 17 07:08:01 2011
From: lac at codespeak.net (lac at codespeak.net)
Date: Thu, 17 Feb 2011 07:08:01 +0100 (CET)
Subject: [pypy-svn] r80356 - pypy/extradoc/talk/ustour2011
Message-ID: <20110217060801.D16A8282BF2@codespeak.net>

Author: lac
Date: Thu Feb 17 07:08:00 2011
New Revision: 80356

Modified:
   pypy/extradoc/talk/ustour2011/mozilla-abstract.txt
Log:
Fix minor typos and missing words.


Modified: pypy/extradoc/talk/ustour2011/mozilla-abstract.txt
==============================================================================
--- pypy/extradoc/talk/ustour2011/mozilla-abstract.txt	(original)
+++ pypy/extradoc/talk/ustour2011/mozilla-abstract.txt	Thu Feb 17 07:08:00 2011
@@ -27,13 +27,14 @@
 Karoo desert. The infrastructure is largely python-based and employs
 Twisted and PyPy among other technologies.
 
-Alex Gaynor is a core developer of PyPy. Since 2009 he has contributed to the
-JIT, the Python interpreter, working on issues of Python 2.7 compatibility and
-the CPython C-API, and implementing third party modules such as PostgreSQL
-bindings for PyPy. He is also a core developer of Django (a popular Python web
-framework), where he works on all elements on the framework. He is presently
-an independent contractor, primarily working in web development where he uses
-Python exclusively. Alex a student at Rensselaer Polytechnic Institute where
+Alex Gaynor is a core developer of PyPy. Since 2009 he has contributed
+to the JIT, and the Python interpreter, working on issues of Python
+2.7 compatibility and the CPython C-API, and implementing third party
+modules such as PostgreSQL bindings for PyPy. He is also a core
+developer of Django (a popular Python web framework), where he works
+on all elements of the framework. He is presently an independent
+contractor, primarily working in web development where he uses Python
+exclusively. Alex is a student at Rensselaer Polytechnic Institute where
 he is a Junior pursuing a bachelors degree in Computer Science.
 
 Abstract:


From commits-noreply at bitbucket.org  Thu Feb 17 07:13:40 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Thu, 17 Feb 2011 07:13:40 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix translation
Message-ID: <20110217061340.CBF45282BF7@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r42097:8ff85bca8bce
Date: 2011-02-17 07:13 +0100
http://bitbucket.org/pypy/pypy/changeset/8ff85bca8bce/

Log:	Fix translation

diff --git a/pypy/module/_collections/__init__.py b/pypy/module/_collections/__init__.py
--- a/pypy/module/_collections/__init__.py
+++ b/pypy/module/_collections/__init__.py
@@ -16,3 +16,12 @@
         'deque' : 'interp_deque.W_Deque',
         '__missing__': 'interp_defaultdict.missing',
         }
+
+    def setup_after_space_initialization(self):
+        """NOT_RPYTHON"""
+        # must remove the interp-level name '__missing__' after it has
+        # been used...  otherwise, some code is not happy about seeing
+        # this code object twice
+        space = self.space
+        space.getattr(self, space.wrap('defaultdict'))  # force importing
+        space.delattr(self, space.wrap('__missing__'))

From commits-noreply at bitbucket.org  Thu Feb 17 07:38:01 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Thu, 17 Feb 2011 07:38:01 +0100 (CET)
Subject: [pypy-svn] pypy default: Test and fix.
Message-ID: <20110217063801.4A16F282BF2@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r42098:3649edf6e8ae
Date: 2011-02-17 07:37 +0100
http://bitbucket.org/pypy/pypy/changeset/3649edf6e8ae/

Log:	Test and fix.

diff --git a/pypy/module/_collections/test/test_defaultdict.py b/pypy/module/_collections/test/test_defaultdict.py
--- a/pypy/module/_collections/test/test_defaultdict.py
+++ b/pypy/module/_collections/test/test_defaultdict.py
@@ -16,3 +16,14 @@
         d[5] = l2
         d[5].append(44)
         assert l == [42, 43] and l2 == [44]
+
+    def test_keyerror_without_factory(self):
+        from _collections import defaultdict
+        d1 = defaultdict()
+        for key in ['foo', (1,)]:
+            try:
+                d1[key]
+            except KeyError, err:
+                assert err.args[0] == key
+            else:
+                assert 0, "expected KeyError"

diff --git a/pypy/module/_collections/interp_defaultdict.py b/pypy/module/_collections/interp_defaultdict.py
--- a/pypy/module/_collections/interp_defaultdict.py
+++ b/pypy/module/_collections/interp_defaultdict.py
@@ -6,7 +6,7 @@
     # threads.
     w_default_factory = space.getattr(w_self, space.wrap('default_factory'))
     if space.is_w(w_default_factory, space.w_None):
-        raise OperationError(space.w_KeyError, w_key)
+        raise OperationError(space.w_KeyError, space.newtuple([w_key]))
     w_value = space.call_function(w_default_factory)
     space.setitem(w_self, w_key, w_value)
     return w_value

From commits-noreply at bitbucket.org  Thu Feb 17 10:19:23 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Thu, 17 Feb 2011 10:19:23 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: Use args_w instead of w_args
	for dict.pop()
Message-ID: <20110217091923.B8677282BF7@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42099:d3970feb8301
Date: 2011-02-16 19:40 +0100
http://bitbucket.org/pypy/pypy/changeset/d3970feb8301/

Log:	Use args_w instead of w_args for dict.pop()

diff --git a/pypy/objspace/std/dictmultiobject.py b/pypy/objspace/std/dictmultiobject.py
--- a/pypy/objspace/std/dictmultiobject.py
+++ b/pypy/objspace/std/dictmultiobject.py
@@ -794,9 +794,8 @@
         w_dict.setitem(w_key, w_default)
         return w_default
 
-def dict_pop__DictMulti_ANY(space, w_dict, w_key, w_defaults):
-    defaults = space.listview(w_defaults)
-    len_defaults = len(defaults)
+def dict_pop__DictMulti_ANY(space, w_dict, w_key, defaults_w):
+    len_defaults = len(defaults_w)
     if len_defaults > 1:
         raise operationerrfmt(space.w_TypeError,
                               "pop expected at most 2 arguments, got %d",
@@ -804,7 +803,7 @@
     w_item = w_dict.getitem(w_key)
     if w_item is None:
         if len_defaults > 0:
-            return defaults[0]
+            return defaults_w[0]
         else:
             space.raise_key_error(w_key)
     else:

diff --git a/pypy/objspace/std/dicttype.py b/pypy/objspace/std/dicttype.py
--- a/pypy/objspace/std/dicttype.py
+++ b/pypy/objspace/std/dicttype.py
@@ -20,7 +20,7 @@
 dict_get        = SMM('get',           3, defaults=(None,),
                       doc='D.get(k[,d]) -> D[k] if k in D, else d.  d defaults'
                           ' to None.')
-dict_pop        = SMM('pop',           2, w_varargs=True,
+dict_pop        = SMM('pop',           2, varargs_w=True,
                       doc='D.pop(k[,d]) -> v, remove specified key and return'
                           ' the corresponding value\nIf key is not found, d is'
                           ' returned if given, otherwise KeyError is raised')

diff --git a/pypy/objspace/std/proxy_helpers.py b/pypy/objspace/std/proxy_helpers.py
--- a/pypy/objspace/std/proxy_helpers.py
+++ b/pypy/objspace/std/proxy_helpers.py
@@ -44,6 +44,8 @@
         return install_general_args_trampoline(type_, mm, is_local, op_name)
     if ['w_args'] == mm.argnames_after:
         return install_w_args_trampoline(type_, mm, is_local, op_name)
+    if ['args_w'] == mm.argnames_after:
+        return install_w_args_trampoline(type_, mm, is_local, op_name)
     assert not mm.argnames_after
     # we search here for special-cased stuff
     def function(space, w_transparent_list, *args_w):

From commits-noreply at bitbucket.org  Thu Feb 17 10:19:24 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Thu, 17 Feb 2011 10:19:24 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: Remove support for w_args in
	multimethods; args_w is probably better
Message-ID: <20110217091924.D48E6282BF7@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42100:e17a25858e4d
Date: 2011-02-16 19:43 +0100
http://bitbucket.org/pypy/pypy/changeset/e17a25858e4d/

Log:	Remove support for w_args in multimethods; args_w is probably better

diff --git a/pypy/objspace/std/model.py b/pypy/objspace/std/model.py
--- a/pypy/objspace/std/model.py
+++ b/pypy/objspace/std/model.py
@@ -404,8 +404,6 @@
 
         if extras.get('general__args__', False):
             self.argnames_after = ['__args__']
-        if extras.get('w_varargs', False):
-            self.argnames_after = ['w_args']
         if extras.get('varargs_w', False):
             self.argnames_after = ['args_w']
         self.argnames_after += extras.get('extra_args', [])

diff --git a/pypy/objspace/std/proxy_helpers.py b/pypy/objspace/std/proxy_helpers.py
--- a/pypy/objspace/std/proxy_helpers.py
+++ b/pypy/objspace/std/proxy_helpers.py
@@ -28,7 +28,7 @@
     function = func_with_new_name(function, mm.name)
     mm.register(function, type_)
 
-def install_w_args_trampoline(type_, mm, is_local, op_name):
+def install_args_w_trampoline(type_, mm, is_local, op_name):
     def function(space, w_transparent_list, *args_w):
         args = Arguments(space, [space.wrap(op_name)] + list(args_w[:-1]), w_stararg=args_w[-1])
         return space.call_args(w_transparent_list.w_controller, args)
@@ -42,10 +42,8 @@
     
     if ['__args__'] == mm.argnames_after:
         return install_general_args_trampoline(type_, mm, is_local, op_name)
-    if ['w_args'] == mm.argnames_after:
-        return install_w_args_trampoline(type_, mm, is_local, op_name)
     if ['args_w'] == mm.argnames_after:
-        return install_w_args_trampoline(type_, mm, is_local, op_name)
+        return install_args_w_trampoline(type_, mm, is_local, op_name)
     assert not mm.argnames_after
     # we search here for special-cased stuff
     def function(space, w_transparent_list, *args_w):

diff --git a/pypy/objspace/std/stdtypedef.py b/pypy/objspace/std/stdtypedef.py
--- a/pypy/objspace/std/stdtypedef.py
+++ b/pypy/objspace/std/stdtypedef.py
@@ -171,8 +171,6 @@
     wrapper_arglist = solid_arglist[:]
     if multimethod.extras.get('varargs_w', False):
         wrapper_arglist.append('args_w')
-    if multimethod.extras.get('w_varargs', False):
-        wrapper_arglist.append('w_args')        
     if multimethod.extras.get('keywords', False):
         raise Exception, "no longer supported, use __args__"
     if multimethod.extras.get('general__args__', False):
@@ -245,8 +243,6 @@
     unwrap_spec = [baseobjspace.ObjSpace] + [baseobjspace.W_Root]*multimethod.arity
     if multimethod.extras.get('varargs_w', False):
         unwrap_spec.append('args_w')
-    if multimethod.extras.get('w_varargs', False):
-        unwrap_spec.append('w_args')        
     if multimethod.extras.get('general__args__', False):
         unwrap_spec.append(argument.Arguments)
     if 'doc' in multimethod.extras:

From commits-noreply at bitbucket.org  Thu Feb 17 10:19:26 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Thu, 17 Feb 2011 10:19:26 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: Since the trampoline function
	names its	parameters consistently,
	unwrap_spec is not needed anymore.
Message-ID: <20110217091926.A1CA22A202D@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42101:c9ec1322984c
Date: 2011-02-16 19:45 +0100
http://bitbucket.org/pypy/pypy/changeset/c9ec1322984c/

Log:	Since the trampoline function names its parameters consistently,
	unwrap_spec is not needed anymore.

diff --git a/pypy/objspace/std/slicetype.py b/pypy/objspace/std/slicetype.py
--- a/pypy/objspace/std/slicetype.py
+++ b/pypy/objspace/std/slicetype.py
@@ -48,7 +48,6 @@
 
 # ____________________________________________________________
 
- at gateway.unwrap_spec(args_w='args_w')
 def descr__new__(space, w_slicetype, args_w):
     from pypy.objspace.std.sliceobject import W_SliceObject
     w_start = space.w_None

diff --git a/pypy/objspace/std/stdtypedef.py b/pypy/objspace/std/stdtypedef.py
--- a/pypy/objspace/std/stdtypedef.py
+++ b/pypy/objspace/std/stdtypedef.py
@@ -240,14 +240,9 @@
 
 def wrap_trampoline_in_gateway(func, methname, multimethod):
     """NOT_RPYTHON"""
-    unwrap_spec = [baseobjspace.ObjSpace] + [baseobjspace.W_Root]*multimethod.arity
-    if multimethod.extras.get('varargs_w', False):
-        unwrap_spec.append('args_w')
-    if multimethod.extras.get('general__args__', False):
-        unwrap_spec.append(argument.Arguments)
     if 'doc' in multimethod.extras:
         func.__doc__ = multimethod.extras['doc']
-    return gateway.interp2app(func, app_name=methname, unwrap_spec=unwrap_spec)
+    return gateway.interp2app(func, app_name=methname)
 
 def slicemultimethod(space, multimethod, typedef, result, local=False):
     """NOT_RPYTHON"""    

From commits-noreply at bitbucket.org  Thu Feb 17 11:21:30 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Thu, 17 Feb 2011 11:21:30 +0100 (CET)
Subject: [pypy-svn] pypy default: remove this subrepo,
	there is cleary no consensus on having it here
Message-ID: <20110217102130.BE9E236C226@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42102:d9c9e250dce5
Date: 2011-02-17 11:20 +0100
http://bitbucket.org/pypy/pypy/changeset/d9c9e250dce5/

Log:	remove this subrepo, there is cleary no consensus on having it here

diff --git a/.hgsub b/.hgsub
--- a/.hgsub
+++ b/.hgsub
@@ -1,4 +1,3 @@
 greenlet = [svn]http://codespeak.net/svn/greenlet/trunk/c
 testrunner = [svn]http://codespeak.net/svn/pypy/build/testrunner
 lib_pypy/pyrepl = [svn]http://codespeak.net/svn/pyrepl/trunk/pyrepl/pyrepl
-contrib/virtualenv-pypy = https://bitbucket.org/pypy/virtualenv-pypy

diff --git a/.hgsubstate b/.hgsubstate
--- a/.hgsubstate
+++ b/.hgsubstate
@@ -1,4 +1,3 @@
-792c971f4114520f0773e3474e2516600af60ee0 contrib/virtualenv-pypy
 80037 greenlet
 80348 lib_pypy/pyrepl
 80037 testrunner

From commits-noreply at bitbucket.org  Thu Feb 17 11:36:22 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Thu, 17 Feb 2011 11:36:22 +0100 (CET)
Subject: [pypy-svn] pypy default: kill relative imports and use absolute
	ones instead;
	the relative imports were added in the sys-prefix branch because
	at	some point lib_pypy was placed in a non-importable
	directory,	but not they are no longer necessary
Message-ID: <20110217103622.98EEC282BF7@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42103:e6ebdfa1fdcb
Date: 2011-02-17 11:36 +0100
http://bitbucket.org/pypy/pypy/changeset/e6ebdfa1fdcb/

Log:	kill relative imports and use absolute ones instead; the relative
	imports were added in the sys-prefix branch because at some point
	lib_pypy was placed in a non-importable directory, but not they are
	no longer necessary

diff --git a/lib_pypy/pypy_test/test_marshal_extra.py b/lib_pypy/pypy_test/test_marshal_extra.py
--- a/lib_pypy/pypy_test/test_marshal_extra.py
+++ b/lib_pypy/pypy_test/test_marshal_extra.py
@@ -2,7 +2,7 @@
 import py
 import sys
 import marshal as cpy_marshal
-from .. import _marshal as marshal
+from lib_pypy import _marshal as marshal
 
 from pypy.tool.udir import udir 
 

diff --git a/lib_pypy/pypy_test/test_defaultdict.py b/lib_pypy/pypy_test/test_defaultdict.py
--- a/lib_pypy/pypy_test/test_defaultdict.py
+++ b/lib_pypy/pypy_test/test_defaultdict.py
@@ -10,7 +10,7 @@
 
 import copy
 
-from .._collections import defaultdict
+from lib_pypy._collections import defaultdict
 
 def foobar():
     return list

diff --git a/lib_pypy/pypy_test/test_deque_extra.py b/lib_pypy/pypy_test/test_deque_extra.py
--- a/lib_pypy/pypy_test/test_deque_extra.py
+++ b/lib_pypy/pypy_test/test_deque_extra.py
@@ -7,7 +7,7 @@
 class Test_deque:
     def setup_method(self,method):
         
-        from .._collections import deque
+        from lib_pypy._collections import deque
         self.deque = deque
         self.d = deque(range(n))
         

diff --git a/lib_pypy/pypy_test/test_binascii.py b/lib_pypy/pypy_test/test_binascii.py
--- a/lib_pypy/pypy_test/test_binascii.py
+++ b/lib_pypy/pypy_test/test_binascii.py
@@ -1,6 +1,6 @@
 from __future__ import absolute_import
 import py
-from .. import binascii
+from lib_pypy import binascii
 
 # Create binary test data
 data = "The quick brown fox jumps over the lazy dog.\r\n"

diff --git a/lib_pypy/pypy_test/test_stackless_pickling.py b/lib_pypy/pypy_test/test_stackless_pickling.py
--- a/lib_pypy/pypy_test/test_stackless_pickling.py
+++ b/lib_pypy/pypy_test/test_stackless_pickling.py
@@ -8,7 +8,7 @@
     import stackless
 except ImportError:
     try:
-        from .. import stackless as stackless
+        from lib_pypy import stackless as stackless
     except ImportError, e:
         skip('cannot import stackless: %s' % (e,))
 

diff --git a/lib_pypy/pypy_test/test_struct_extra.py b/lib_pypy/pypy_test/test_struct_extra.py
--- a/lib_pypy/pypy_test/test_struct_extra.py
+++ b/lib_pypy/pypy_test/test_struct_extra.py
@@ -1,5 +1,5 @@
 from __future__ import absolute_import
-from .. import struct 
+from lib_pypy import struct 
 
 def test_simple():
     morezeros = '\x00' * (struct.calcsize('l')-4)

diff --git a/lib_pypy/pypy_test/test_structseq.py b/lib_pypy/pypy_test/test_structseq.py
--- a/lib_pypy/pypy_test/test_structseq.py
+++ b/lib_pypy/pypy_test/test_structseq.py
@@ -1,6 +1,6 @@
 from __future__ import absolute_import
 import py
-from .._structseq import structseqfield, structseqtype
+from lib_pypy._structseq import structseqfield, structseqtype
 
 
 class mydata:

diff --git a/lib_pypy/pypy_test/test_collections.py b/lib_pypy/pypy_test/test_collections.py
--- a/lib_pypy/pypy_test/test_collections.py
+++ b/lib_pypy/pypy_test/test_collections.py
@@ -1,5 +1,5 @@
 from __future__ import absolute_import
-from .. import _collections as collections
+from lib_pypy import _collections as collections
 import py
 
 class TestDeque:

diff --git a/lib_pypy/pypy_test/test_pyexpat.py b/lib_pypy/pypy_test/test_pyexpat.py
--- a/lib_pypy/pypy_test/test_pyexpat.py
+++ b/lib_pypy/pypy_test/test_pyexpat.py
@@ -5,10 +5,10 @@
 import StringIO, sys
 import unittest, py
 
-from ..ctypes_config_cache import rebuild
+from lib_pypy.ctypes_config_cache import rebuild
 rebuild.rebuild_one('pyexpat.ctc.py')
 
-from .. import pyexpat
+from lib_pypy import pyexpat
 #from xml.parsers import expat
 expat = pyexpat
 

diff --git a/lib_pypy/pypy_test/test_ctypes_support.py b/lib_pypy/pypy_test/test_ctypes_support.py
--- a/lib_pypy/pypy_test/test_ctypes_support.py
+++ b/lib_pypy/pypy_test/test_ctypes_support.py
@@ -5,7 +5,7 @@
 try:
     from ctypes_support import standard_c_lib, get_errno, set_errno
 except ImportError:    # on top of cpython
-    from ..ctypes_support import standard_c_lib, get_errno, set_errno
+    from lib_pypy.ctypes_support import standard_c_lib, get_errno, set_errno
 
 
 def test_stdlib_and_errno():

diff --git a/lib_pypy/pypy_test/test_datetime.py b/lib_pypy/pypy_test/test_datetime.py
--- a/lib_pypy/pypy_test/test_datetime.py
+++ b/lib_pypy/pypy_test/test_datetime.py
@@ -1,7 +1,7 @@
 from __future__ import absolute_import
 import py
 
-from .. import datetime
+from lib_pypy import datetime
 
 def test_repr():
     print datetime

diff --git a/lib_pypy/pypy_test/test_stackless.py b/lib_pypy/pypy_test/test_stackless.py
--- a/lib_pypy/pypy_test/test_stackless.py
+++ b/lib_pypy/pypy_test/test_stackless.py
@@ -10,7 +10,7 @@
     import stackless
 except ImportError:
     try:
-        from .. import stackless
+        from lib_pypy import stackless
     except ImportError, e:
         skip('cannot import stackless: %s' % (e,))
 

diff --git a/lib_pypy/pypy_test/test_syslog.py b/lib_pypy/pypy_test/test_syslog.py
--- a/lib_pypy/pypy_test/test_syslog.py
+++ b/lib_pypy/pypy_test/test_syslog.py
@@ -1,10 +1,10 @@
 from __future__ import absolute_import
 # XXX very minimal test
 
-from ..ctypes_config_cache import rebuild
+from lib_pypy.ctypes_config_cache import rebuild
 rebuild.rebuild_one('syslog.ctc.py')
 
-from .. import syslog
+from lib_pypy import syslog
 
 
 def test_syslog():

diff --git a/lib_pypy/pypy_test/test_locale.py b/lib_pypy/pypy_test/test_locale.py
--- a/lib_pypy/pypy_test/test_locale.py
+++ b/lib_pypy/pypy_test/test_locale.py
@@ -2,10 +2,10 @@
 import py
 import sys
 
-from ..ctypes_config_cache import rebuild
+from lib_pypy.ctypes_config_cache import rebuild
 rebuild.rebuild_one('locale.ctc.py')
 
-from .. import _locale
+from lib_pypy import _locale
 
 
 def setup_module(mod):

diff --git a/lib_pypy/pypy_test/inprogress_test_binascii_extra.py b/lib_pypy/pypy_test/inprogress_test_binascii_extra.py
--- a/lib_pypy/pypy_test/inprogress_test_binascii_extra.py
+++ b/lib_pypy/pypy_test/inprogress_test_binascii_extra.py
@@ -1,5 +1,5 @@
 from __future__ import absolute_import
-from .. import binascii
+from lib_pypy import binascii
 
 def test_uu():
     assert binascii.b2a_uu('1234567') == "',3(S-#4V-P  \n"

diff --git a/lib_pypy/pypy_test/test_coroutine.py b/lib_pypy/pypy_test/test_coroutine.py
--- a/lib_pypy/pypy_test/test_coroutine.py
+++ b/lib_pypy/pypy_test/test_coroutine.py
@@ -2,7 +2,7 @@
 from py.test import skip, raises
 
 try:
-    from ..stackless import coroutine, CoroutineExit
+    from lib_pypy.stackless import coroutine, CoroutineExit
 except ImportError, e:
     skip('cannot import stackless: %s' % (e,))
 

diff --git a/lib_pypy/pypy_test/test_resource.py b/lib_pypy/pypy_test/test_resource.py
--- a/lib_pypy/pypy_test/test_resource.py
+++ b/lib_pypy/pypy_test/test_resource.py
@@ -1,8 +1,8 @@
 from __future__ import absolute_import
-from ..ctypes_config_cache import rebuild
+from lib_pypy.ctypes_config_cache import rebuild
 rebuild.rebuild_one('resource.ctc.py')
 
-from .. import resource
+from lib_pypy import resource
 
 def test_resource():
     x = resource.getrusage(resource.RUSAGE_SELF)

diff --git a/lib_pypy/pypy_test/test_sha_extra.py b/lib_pypy/pypy_test/test_sha_extra.py
--- a/lib_pypy/pypy_test/test_sha_extra.py
+++ b/lib_pypy/pypy_test/test_sha_extra.py
@@ -4,7 +4,7 @@
 # Publication 180-1, Secure Hash Standard,  1995 April 17
 # http://www.itl.nist.gov/div897/pubs/fip180-1.htm
 from __future__ import absolute_import
-from .. import _sha as pysha
+from lib_pypy import _sha as pysha
 
 class TestSHA: 
     def check(self, data, digest):

diff --git a/lib_pypy/pypy_test/test_grp_extra.py b/lib_pypy/pypy_test/test_grp_extra.py
--- a/lib_pypy/pypy_test/test_grp_extra.py
+++ b/lib_pypy/pypy_test/test_grp_extra.py
@@ -1,7 +1,7 @@
 from __future__ import absolute_import
 import py
 try:
-    from .. import grp
+    from lib_pypy import grp
 except ImportError:
     py.test.skip("No grp module on this platform")
 

diff --git a/lib_pypy/pypy_test/test_md5_extra.py b/lib_pypy/pypy_test/test_md5_extra.py
--- a/lib_pypy/pypy_test/test_md5_extra.py
+++ b/lib_pypy/pypy_test/test_md5_extra.py
@@ -6,7 +6,7 @@
 
 from __future__ import absolute_import
 import md5                              # CPython's implementation in C.
-from .. import _md5 as pymd5  
+from lib_pypy import _md5 as pymd5  
 
 
 # Helpers...

diff --git a/lib_pypy/pypy_test/test_dbm_extra.py b/lib_pypy/pypy_test/test_dbm_extra.py
--- a/lib_pypy/pypy_test/test_dbm_extra.py
+++ b/lib_pypy/pypy_test/test_dbm_extra.py
@@ -2,7 +2,7 @@
 import py
 from pypy.tool.udir import udir
 try:
-    from .. import dbm
+    from lib_pypy import dbm
 except ImportError, e:
     py.test.skip(e)
 

From commits-noreply at bitbucket.org  Thu Feb 17 11:58:55 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Thu, 17 Feb 2011 11:58:55 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: Merge default
Message-ID: <20110217105855.B0037282C18@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42104:fab2b29e23bc
Date: 2011-02-17 10:49 +0100
http://bitbucket.org/pypy/pypy/changeset/fab2b29e23bc/

Log:	Merge default

diff --git a/pypy/module/zipimport/interp_zipimport.py b/pypy/module/zipimport/interp_zipimport.py
--- a/pypy/module/zipimport/interp_zipimport.py
+++ b/pypy/module/zipimport/interp_zipimport.py
@@ -141,7 +141,8 @@
         real_name = self.filename + os.path.sep + self.corr_zname(filename)
         space.setattr(w_mod, w('__loader__'), space.wrap(self))
         importing._prepare_module(space, w_mod, real_name, pkgpath)
-        code_w = importing.parse_source_module(space, filename, buf)
+        co_filename = self.make_co_filename(filename)
+        code_w = importing.parse_source_module(space, co_filename, buf)
         importing.exec_code_module(space, w_mod, code_w)
         return w_mod
 
@@ -223,6 +224,14 @@
         subname = fullname[startpos:]
         return self.prefix + subname.replace('.', '/')
 
+    def make_co_filename(self, filename):
+        """
+        Return the filename to be used for compiling the module, i.e. what
+        gets in code_object.co_filename. Something like
+        'myfile.zip/mymodule.py'
+        """
+        return self.filename + os.path.sep + filename
+
     @unwrap_spec(fullname=str)
     def load_module(self, space, fullname):
         w = space.wrap
@@ -290,8 +299,9 @@
                     code_w = importing.read_compiled_module(
                         space, filename + ext, source[8:])
                 else:
+                    co_filename = self.make_co_filename(filename+ext)
                     code_w = importing.parse_source_module(
-                        space, filename + ext, source)
+                        space, co_filename, source)
                 return space.wrap(code_w)
         raise operationerrfmt(self.w_ZipImportError,
             "Cannot find source or code for %s in %s", filename, self.name)

diff --git a/pypy/module/_rawffi/structure.py b/pypy/module/_rawffi/structure.py
--- a/pypy/module/_rawffi/structure.py
+++ b/pypy/module/_rawffi/structure.py
@@ -32,17 +32,16 @@
         tp = unpack_shape_with_length(space, l_w[1])
 
         if len_l == 3:
-            bitsize = space.int_w(l_w[2])
-
-            if bitsize < 0 or bitsize > tp.size * 8:
-                raise OperationError(space.w_ValueError, space.wrap(
-                    "number of bits invalid for bit field"))
             for c in unroll_letters_for_numbers:
                 if c == tp.itemcode:
                     break
             else:
+                raise OperationError(space.w_TypeError, space.wrap(
+                    "bit fields not allowed for type"))
+            bitsize = space.int_w(l_w[2])
+            if bitsize <= 0 or bitsize > tp.size * 8:
                 raise OperationError(space.w_ValueError, space.wrap(
-                    "bit fields not allowed for type"))
+                    "number of bits invalid for bit field"))
         else:
             bitsize = 0
 

From commits-noreply at bitbucket.org  Thu Feb 17 11:58:56 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Thu, 17 Feb 2011 11:58:56 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: Fix imports
Message-ID: <20110217105856.6F66C282C18@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42105:d0e201b0e61c
Date: 2011-02-17 11:57 +0100
http://bitbucket.org/pypy/pypy/changeset/d0e201b0e61c/

Log:	Fix imports

diff --git a/pypy/module/_minimal_curses/interp_curses.py b/pypy/module/_minimal_curses/interp_curses.py
--- a/pypy/module/_minimal_curses/interp_curses.py
+++ b/pypy/module/_minimal_curses/interp_curses.py
@@ -1,5 +1,5 @@
 
-from pypy.interpreter.baseobjspace import unwrap_spec
+from pypy.interpreter.gateway import unwrap_spec
 from pypy.interpreter.error import OperationError
 from pypy.module._minimal_curses import _curses
 

diff --git a/pypy/module/crypt/interp_crypt.py b/pypy/module/crypt/interp_crypt.py
--- a/pypy/module/crypt/interp_crypt.py
+++ b/pypy/module/crypt/interp_crypt.py
@@ -1,5 +1,5 @@
 from pypy.interpreter.error import OperationError
-from pypy.interpreter.baseobjspace import unwrap_spec
+from pypy.interpreter.gateway import unwrap_spec
 from pypy.rpython.lltypesystem import rffi, lltype
 from pypy.translator.tool.cbuild import ExternalCompilationInfo
 import sys

From commits-noreply at bitbucket.org  Thu Feb 17 11:58:56 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Thu, 17 Feb 2011 11:58:56 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: Remove most unwrap_specs in
	the _collection	module
Message-ID: <20110217105856.F1DFC282C18@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42106:c73a2d9029bd
Date: 2011-02-17 11:57 +0100
http://bitbucket.org/pypy/pypy/changeset/c73a2d9029bd/

Log:	Remove most unwrap_specs in the _collection module

diff --git a/pypy/module/_collections/interp_deque.py b/pypy/module/_collections/interp_deque.py
--- a/pypy/module/_collections/interp_deque.py
+++ b/pypy/module/_collections/interp_deque.py
@@ -1,10 +1,9 @@
 import sys
 from pypy.interpreter import gateway
 from pypy.interpreter.baseobjspace import Wrappable
-from pypy.interpreter.typedef import TypeDef, interp2app, make_weakref_descr
+from pypy.interpreter.typedef import TypeDef, make_weakref_descr
 from pypy.interpreter.typedef import GetSetProperty
-from pypy.interpreter.gateway import ObjSpace, W_Root, unwrap_spec, Arguments
-from pypy.interpreter.gateway import NoneNotWrapped
+from pypy.interpreter.gateway import interp2app, unwrap_spec, NoneNotWrapped
 from pypy.interpreter.error import OperationError
 from pypy.rlib.debug import check_nonneg
 
@@ -81,7 +80,6 @@
                 self.space.w_RuntimeError,
                 self.space.wrap("deque mutated during iteration"))
 
-    @unwrap_spec('self', W_Root, W_Root)
     def init(self, w_iterable=NoneNotWrapped, w_maxlen=None):
         space = self.space
         if space.is_w(w_maxlen, space.w_None):
@@ -104,7 +102,6 @@
             self.pop()
             assert self.len == self.maxlen
 
-    @unwrap_spec('self', W_Root)
     def append(self, w_x):
         "Add an element to the right side of the deque."
         ri = self.rightindex + 1
@@ -119,7 +116,6 @@
         self.trimleft()
         self.modified()
 
-    @unwrap_spec('self', W_Root)
     def appendleft(self, w_x):
         "Add an element to the left side of the deque."
         li = self.leftindex - 1
@@ -134,7 +130,6 @@
         self.trimright()
         self.modified()
 
-    @unwrap_spec('self')
     def clear(self):
         "Remove all elements from the deque."
         self.leftblock = Block(None, None)
@@ -144,7 +139,6 @@
         self.len = 0
         self.modified()
 
-    @unwrap_spec('self', W_Root)
     def count(self, w_x):
         "Return number of occurrences of value."
         space = self.space
@@ -164,7 +158,6 @@
                 index = 0
         return space.wrap(result)
 
-    @unwrap_spec('self', W_Root)
     def extend(self, w_iterable):
         "Extend the right side of the deque with elements from the iterable"
         # Handle case where id(deque) == id(iterable)
@@ -182,12 +175,10 @@
                 raise
             self.append(w_obj)
 
-    @unwrap_spec('self', W_Root)
     def iadd(self, w_iterable):
         self.extend(w_iterable)
         return self.space.wrap(self)
 
-    @unwrap_spec('self', W_Root)
     def extendleft(self, w_iterable):
         "Extend the left side of the deque with elements from the iterable"
         # Handle case where id(deque) == id(iterable)
@@ -206,7 +197,6 @@
                 raise
             self.appendleft(w_obj)
 
-    @unwrap_spec('self')
     def pop(self):
         "Remove and return the rightmost element."
         if self.len == 0:
@@ -231,7 +221,6 @@
         self.modified()
         return w_obj
 
-    @unwrap_spec('self')
     def popleft(self):
         "Remove and return the leftmost element."
         if self.len == 0:
@@ -256,7 +245,6 @@
         self.modified()
         return w_obj
 
-    @unwrap_spec('self', W_Root)
     def remove(self, w_x):
         "Remove first occurrence of value."
         space = self.space
@@ -278,7 +266,6 @@
         raise OperationError(space.w_ValueError,
                              space.wrap("deque.remove(x): x not in deque"))
 
-    @unwrap_spec('self')
     def reverse(self):
         "Reverse *IN PLACE*."
         li = self.leftindex
@@ -296,7 +283,7 @@
                 rb = rb.leftlink
                 ri = BLOCKLEN - 1
 
-    @unwrap_spec('self', int)
+    @unwrap_spec(n=int)
     def rotate(self, n=1):
         "Rotate the deque n steps to the right (default n=1).  If n is negative, rotates left."
         len = self.len
@@ -315,20 +302,16 @@
             self.append(self.popleft())
             i -= 1
 
-    @unwrap_spec('self')
     def iter(self):
         return W_DequeIter(self)
 
-    @unwrap_spec('self')
     def reviter(self):
         "Return a reverse iterator over the deque."
         return W_DequeRevIter(self)
 
-    @unwrap_spec('self')
     def length(self):
         return self.space.wrap(self.len)
 
-    @unwrap_spec('self')
     def repr(self):
         space = self.space
         ec = space.getexecutioncontext()
@@ -344,22 +327,16 @@
         return space.compare_by_iteration(space.wrap(self), w_other, op)
     compare._annspecialcase_ = 'specialize:arg(2)'
 
-    @unwrap_spec('self', W_Root)
     def lt(self, w_other):
         return self.compare(w_other, 'lt')
-    @unwrap_spec('self', W_Root)
     def le(self, w_other):
         return self.compare(w_other, 'le')
-    @unwrap_spec('self', W_Root)
     def eq(self, w_other):
         return self.compare(w_other, 'eq')
-    @unwrap_spec('self', W_Root)
     def ne(self, w_other):
         return self.compare(w_other, 'ne')
-    @unwrap_spec('self', W_Root)
     def gt(self, w_other):
         return self.compare(w_other, 'gt')
-    @unwrap_spec('self', W_Root)
     def ge(self, w_other):
         return self.compare(w_other, 'ge')
 
@@ -387,7 +364,6 @@
         self.popleft()
         self.rotate(i)
 
-    @unwrap_spec('self', W_Root)
     def getitem(self, w_index):
         space = self.space
         start, stop, step = space.decode_index(w_index, self.len)
@@ -398,7 +374,6 @@
             raise OperationError(space.w_TypeError,
                                  space.wrap("deque[:] is not supported"))
 
-    @unwrap_spec('self', W_Root, W_Root)
     def setitem(self, w_index, w_newobj):
         space = self.space
         start, stop, step = space.decode_index(w_index, self.len)
@@ -409,7 +384,6 @@
             raise OperationError(space.w_TypeError,
                                  space.wrap("deque[:] is not supported"))
 
-    @unwrap_spec('self', W_Root)
     def delitem(self, w_index):
         space = self.space
         start, stop, step = space.decode_index(w_index, self.len)
@@ -419,7 +393,6 @@
             raise OperationError(space.w_TypeError,
                                  space.wrap("deque[:] is not supported"))
 
-    @unwrap_spec('self')
     def copy(self):
         "Return a shallow copy of a deque."
         space = self.space
@@ -430,7 +403,6 @@
             return space.call_function(space.type(w_self), w_self,
                                        space.wrap(self.maxlen))
 
-    @unwrap_spec('self')
     def reduce(self):
         "Return state information for pickling."
         space = self.space
@@ -486,8 +458,7 @@
 dequerepr = app.interphook("dequerepr")
 
 
- at unwrap_spec(ObjSpace, W_Root, Arguments)
-def descr__new__(space, w_subtype, args):
+def descr__new__(space, w_subtype, __args__):
     w_self = space.allocate_instance(W_Deque, w_subtype)
     W_Deque.__init__(space.interp_w(W_Deque, w_self), space)
     return w_self
@@ -543,11 +514,9 @@
         self.lock = deque.getlock()
         check_nonneg(self.index)
 
-    @unwrap_spec('self')
     def iter(self):
         return self.space.wrap(self)
 
-    @unwrap_spec('self')
     def next(self):
         self.deque.checklock(self.lock)
         if self.counter == 0:
@@ -580,11 +549,9 @@
         self.lock = deque.getlock()
         check_nonneg(self.index)
 
-    @unwrap_spec('self')
     def iter(self):
         return self.space.wrap(self)
 
-    @unwrap_spec('self')
     def next(self):
         self.deque.checklock(self.lock)
         if self.counter == 0:

From commits-noreply at bitbucket.org  Thu Feb 17 14:46:42 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Thu, 17 Feb 2011 14:46:42 +0100 (CET)
Subject: [pypy-svn] pypy default: Pick variable to spill based on the
	longest longevity
Message-ID: <20110217134642.DC5CF282C1A@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: 
Changeset: r42107:519673a58add
Date: 2011-02-17 14:27 +0100
http://bitbucket.org/pypy/pypy/changeset/519673a58add/

Log:	Pick variable to spill based on the longest longevity

diff --git a/pypy/jit/backend/llsupport/regalloc.py b/pypy/jit/backend/llsupport/regalloc.py
--- a/pypy/jit/backend/llsupport/regalloc.py
+++ b/pypy/jit/backend/llsupport/regalloc.py
@@ -160,7 +160,8 @@
                                 need_lower_byte=False):
         """ Silly algorithm.
         """
-        candidates = []
+        cur_max_age = -1
+        candidate = None
         for next in self.reg_bindings:
             reg = self.reg_bindings[next]
             if next in forbidden_vars:
@@ -172,8 +173,13 @@
                     continue
             if need_lower_byte and reg in self.no_lower_byte_regs:
                 continue
-            return next
-        raise NoVariableToSpill
+            max_age = self.longevity[next][1]
+            if cur_max_age < max_age:
+                cur_max_age = max_age
+                candidate = next
+        if candidate is None:
+            raise NoVariableToSpill
+        return candidate
 
     def force_allocate_reg(self, v, forbidden_vars=[], selected_reg=None,
                            need_lower_byte=False):

diff --git a/pypy/jit/backend/llsupport/test/test_regalloc.py b/pypy/jit/backend/llsupport/test/test_regalloc.py
--- a/pypy/jit/backend/llsupport/test/test_regalloc.py
+++ b/pypy/jit/backend/llsupport/test/test_regalloc.py
@@ -16,9 +16,12 @@
     return res, longevity
 
 class FakeReg(object):
-    pass
+    def __init__(self, i):
+        self.n = i
+    def __repr__(self):
+        return 'r%d' % self.n
 
-r0, r1, r2, r3 = [FakeReg() for _ in range(4)]
+r0, r1, r2, r3 = [FakeReg(i) for i in range(4)]
 regs = [r0, r1, r2, r3]
 
 class RegisterManager(BaseRegMan):
@@ -326,3 +329,21 @@
         assert fm.frame_depth == 3
         
         
+
+    def test_spilling(self):
+        b0, b1, b2, b3, b4, b5 = newboxes(0, 1, 2, 3, 4, 5)
+        longevity = {b0: (0, 3), b1: (0, 3), b3: (0, 5), b2: (0, 2), b4: (1, 4), b5: (1, 3)}
+        fm = TFrameManager()
+        asm = MockAsm()
+        rm = RegisterManager(longevity, frame_manager=fm, assembler=asm)
+        rm.next_instruction()
+        for b in b0, b1, b2, b3:
+            rm.force_allocate_reg(b)
+        assert len(rm.free_regs) == 0
+        rm.next_instruction()
+        loc = rm.loc(b3)
+        spilled = rm.force_allocate_reg(b4)
+        assert spilled is loc
+        spilled2 = rm.force_allocate_reg(b5)
+        assert spilled2 is loc
+        rm._check_invariants()

From commits-noreply at bitbucket.org  Thu Feb 17 14:55:22 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Thu, 17 Feb 2011 14:55:22 +0100 (CET)
Subject: [pypy-svn] pypy default: Update comment for variable picking
	algorithm
Message-ID: <20110217135522.441882A202C@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: 
Changeset: r42108:c920da595e27
Date: 2011-02-17 14:52 +0100
http://bitbucket.org/pypy/pypy/changeset/c920da595e27/

Log:	Update comment for variable picking algorithm

diff --git a/pypy/jit/backend/llsupport/regalloc.py b/pypy/jit/backend/llsupport/regalloc.py
--- a/pypy/jit/backend/llsupport/regalloc.py
+++ b/pypy/jit/backend/llsupport/regalloc.py
@@ -158,7 +158,7 @@
 
     def _pick_variable_to_spill(self, v, forbidden_vars, selected_reg=None,
                                 need_lower_byte=False):
-        """ Silly algorithm.
+        """ Slightly less silly algorithm.
         """
         cur_max_age = -1
         candidate = None

From commits-noreply at bitbucket.org  Thu Feb 17 16:07:24 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Thu, 17 Feb 2011 16:07:24 +0100 (CET)
Subject: [pypy-svn] pypy numpy-exp: At least on 64bit this change looks much
	better
Message-ID: <20110217150724.7F1AA282C18@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: numpy-exp
Changeset: r42109:90ab0a59c1e1
Date: 2011-02-16 07:37 -0700
http://bitbucket.org/pypy/pypy/changeset/90ab0a59c1e1/

Log:	At least on 64bit this change looks much better

diff --git a/pypy/jit/backend/x86/regalloc.py b/pypy/jit/backend/x86/regalloc.py
--- a/pypy/jit/backend/x86/regalloc.py
+++ b/pypy/jit/backend/x86/regalloc.py
@@ -146,7 +146,7 @@
         self._prepare(inputargs, operations)
         jump = operations[-1]
         loop_consts = self._compute_loop_consts(inputargs, jump, looptoken)
-        self.loop_consts = loop_consts
+        self.loop_consts = {}#loop_consts
         return self._process_inputargs(inputargs)
 
     def prepare_bridge(self, prev_depths, inputargs, arglocs, operations):

From commits-noreply at bitbucket.org  Thu Feb 17 16:07:25 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Thu, 17 Feb 2011 16:07:25 +0100 (CET)
Subject: [pypy-svn] pypy out-of-line-guards: Remove obsolete stuff
Message-ID: <20110217150725.3BB96282C18@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: out-of-line-guards
Changeset: r42110:81c3f74205c7
Date: 2011-02-16 07:40 -0700
http://bitbucket.org/pypy/pypy/changeset/81c3f74205c7/

Log:	Remove obsolete stuff

diff --git a/pypy/jit/metainterp/history.py b/pypy/jit/metainterp/history.py
--- a/pypy/jit/metainterp/history.py
+++ b/pypy/jit/metainterp/history.py
@@ -770,16 +770,9 @@
     def __init__(self):
         # For memory management of assembled loops
         self._keepalive_target_looktokens = {}      # set of other LoopTokens
-        self._back_looptokens = []
-        self._back_looptokens_call_asm = []
-        # the reverse of the _keepalive_target_looktokens dict
 
     def record_jump_to(self, target_loop_token, is_call_asm=False):
         self._keepalive_target_looktokens[target_loop_token] = None
-        if is_call_asm:
-            target_loop_token._back_looptokens_call_asm.append(weakref.ref(self))
-        else:
-            target_loop_token._back_looptokens.append(weakref.ref(self))
 
     def __repr__(self):
         return '<Loop %d, gen=%d>' % (self.number, self.generation)

From commits-noreply at bitbucket.org  Thu Feb 17 16:07:28 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Thu, 17 Feb 2011 16:07:28 +0100 (CET)
Subject: [pypy-svn] pypy out-of-line-guards: Emit extra -live- at the
	beginning of each	entry point
Message-ID: <20110217150728.468C3282C20@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: out-of-line-guards
Changeset: r42111:25925a2a13cd
Date: 2011-02-17 06:41 -0700
http://bitbucket.org/pypy/pypy/changeset/25925a2a13cd/

Log:	Emit extra -live- at the beginning of each entry point

diff --git a/pypy/jit/codewriter/call.py b/pypy/jit/codewriter/call.py
--- a/pypy/jit/codewriter/call.py
+++ b/pypy/jit/codewriter/call.py
@@ -148,7 +148,7 @@
 
     def grab_initial_jitcodes(self):
         for jd in self.jitdrivers_sd:
-            jd.mainjitcode = self.get_jitcode(jd.portal_graph)
+            jd.mainjitcode = self.get_jitcode(jd.portal_graph, entry_point=True)
             jd.mainjitcode.is_portal = True
 
     def enum_pending_graphs(self):
@@ -156,14 +156,14 @@
             graph = self.unfinished_graphs.pop()
             yield graph, self.jitcodes[graph]
 
-    def get_jitcode(self, graph, called_from=None):
+    def get_jitcode(self, graph, called_from=None, entry_point=False):
         # 'called_from' is only one of the callers, used for debugging.
         try:
             return self.jitcodes[graph]
         except KeyError:
             fnaddr, calldescr = self.get_jitcode_calldescr(graph)
             jitcode = JitCode(graph.name, fnaddr, calldescr,
-                              called_from=called_from)
+                              called_from=called_from, entry_point=entry_point)
             self.jitcodes[graph] = jitcode
             self.unfinished_graphs.append(graph)
             return jitcode

diff --git a/pypy/jit/codewriter/codewriter.py b/pypy/jit/codewriter/codewriter.py
--- a/pypy/jit/codewriter/codewriter.py
+++ b/pypy/jit/codewriter/codewriter.py
@@ -49,7 +49,7 @@
         # which means mostly producing a linear list of operations and
         # inserting jumps or conditional jumps.  This is a list of tuples
         # of the shape ("opname", arg1, ..., argN) or (Label(...),).
-        ssarepr = flatten_graph(graph, regallocs)
+        ssarepr = flatten_graph(graph, regallocs, jitcode.entry_point)
         #
         # step 3b: compute the liveness around certain operations
         compute_liveness(ssarepr)

diff --git a/pypy/jit/codewriter/jitcode.py b/pypy/jit/codewriter/jitcode.py
--- a/pypy/jit/codewriter/jitcode.py
+++ b/pypy/jit/codewriter/jitcode.py
@@ -9,13 +9,15 @@
     _empty_r = []
     _empty_f = []
 
-    def __init__(self, name, fnaddr=None, calldescr=None, called_from=None):
+    def __init__(self, name, fnaddr=None, calldescr=None, called_from=None,
+                 entry_point=False):
         self.name = name
         self.fnaddr = fnaddr
         self.calldescr = calldescr
         self.is_portal = False
         self._called_from = called_from   # debugging
         self._ssarepr     = None          # debugging
+        self.entry_point = entry_point
 
     def setup(self, code='', constants_i=[], constants_r=[], constants_f=[],
               num_regs_i=255, num_regs_r=255, num_regs_f=255,

diff --git a/pypy/jit/codewriter/flatten.py b/pypy/jit/codewriter/flatten.py
--- a/pypy/jit/codewriter/flatten.py
+++ b/pypy/jit/codewriter/flatten.py
@@ -60,10 +60,12 @@
 
 # ____________________________________________________________
 
-def flatten_graph(graph, regallocs, _include_all_exc_links=False):
+def flatten_graph(graph, regallocs, _include_all_exc_links=False,
+                  entry_point=False):
     """Flatten the graph into an SSARepr, with already-computed register
     allocations.  'regallocs' in a dict {kind: RegAlloc}."""
-    flattener = GraphFlattener(graph, regallocs, _include_all_exc_links)
+    flattener = GraphFlattener(graph, regallocs, _include_all_exc_links,
+                               entry_point)
     flattener.enforce_input_args()
     flattener.generate_ssa_form()
     return flattener.ssarepr
@@ -71,7 +73,8 @@
 
 class GraphFlattener(object):
 
-    def __init__(self, graph, regallocs, _include_all_exc_links=False):
+    def __init__(self, graph, regallocs, _include_all_exc_links=False,
+                 entry_point=False):
         self.graph = graph
         self.regallocs = regallocs
         self._include_all_exc_links = _include_all_exc_links
@@ -81,6 +84,7 @@
         else:
             name = '?'
         self.ssarepr = SSARepr(name)
+        self.entry_point = entry_point
 
     def enforce_input_args(self):
         inputargs = self.graph.startblock.inputargs
@@ -98,6 +102,8 @@
 
     def generate_ssa_form(self):
         self.seen_blocks = {}
+        if self.entry_point:
+            self.emitline('-live-')
         self.make_bytecode_block(self.graph.startblock)
 
     def make_bytecode_block(self, block):

diff --git a/pypy/jit/codewriter/test/test_flatten.py b/pypy/jit/codewriter/test/test_flatten.py
--- a/pypy/jit/codewriter/test/test_flatten.py
+++ b/pypy/jit/codewriter/test/test_flatten.py
@@ -115,7 +115,8 @@
         return self.rtyper.annotator.translator.graphs
 
     def encoding_test(self, func, args, expected,
-                      transform=False, liveness=False, cc=None, jd=None):
+                      transform=False, liveness=False, cc=None, jd=None,
+                      entry_point=False):
         graphs = self.make_graphs(func, args)
         #graphs[0].show()
         if transform:
@@ -123,7 +124,8 @@
             cc = cc or FakeCallControl()
             transform_graph(graphs[0], FakeCPU(self.rtyper), cc, jd)
         ssarepr = flatten_graph(graphs[0], fake_regallocs(),
-                                _include_all_exc_links=not transform)
+                                _include_all_exc_links=not transform,
+                                entry_point=entry_point)
         if liveness:
             from pypy.jit.codewriter.liveness import compute_liveness
             compute_liveness(ssarepr)
@@ -628,6 +630,15 @@
             int_return %i0
         """, transform=True, liveness=True)
 
+    def test_live_at_the_beginning(self):
+        def f(x):
+            return 3 + x
+        self.encoding_test(f, [5], """
+        -live-
+        int_add $3, %i0 -> %i1
+        int_return %i1
+        """, entry_point=True)
+
     def test_ptr_nonzero(self):
         def f(p):
             if p:

From commits-noreply at bitbucket.org  Thu Feb 17 16:07:30 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Thu, 17 Feb 2011 16:07:30 +0100 (CET)
Subject: [pypy-svn] pypy out-of-line-guards: Some general progress on
	out-of-line guards.	Merge with preambles and start
Message-ID: <20110217150730.DC7DC2A202C@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: out-of-line-guards
Changeset: r42112:9a10c0355d10
Date: 2011-02-17 07:55 -0700
http://bitbucket.org/pypy/pypy/changeset/9a10c0355d10/

Log:	Some general progress on out-of-line guards. Merge with preambles
	and start doing the right thing when we fail such a guard

diff --git a/pypy/jit/codewriter/codewriter.py b/pypy/jit/codewriter/codewriter.py
--- a/pypy/jit/codewriter/codewriter.py
+++ b/pypy/jit/codewriter/codewriter.py
@@ -49,7 +49,8 @@
         # which means mostly producing a linear list of operations and
         # inserting jumps or conditional jumps.  This is a list of tuples
         # of the shape ("opname", arg1, ..., argN) or (Label(...),).
-        ssarepr = flatten_graph(graph, regallocs, jitcode.entry_point)
+        ssarepr = flatten_graph(graph, regallocs,
+                                entry_point=jitcode.entry_point)
         #
         # step 3b: compute the liveness around certain operations
         compute_liveness(ssarepr)

diff --git a/pypy/jit/metainterp/compile.py b/pypy/jit/metainterp/compile.py
--- a/pypy/jit/metainterp/compile.py
+++ b/pypy/jit/metainterp/compile.py
@@ -105,6 +105,7 @@
     loop.preamble = create_empty_loop(metainterp, 'Preamble ')
     loop.preamble.inputargs = loop.inputargs
     loop.preamble.token = make_loop_token(len(loop.inputargs), jitdriver_sd)
+    loop.preamble.token.invalidated_array = metainterp.invalidated_array
     loop.preamble.start_resumedescr = start_resumedescr
 
     try:
@@ -400,6 +401,23 @@
         self.copy_all_attrbutes_into(res)
         return res
 
+class ResumeGuardInvalidatedDescr(ResumeGuardDescr):
+    def __init__(self, metainterp_sd, jitdriver_sd):
+        self.metainterp_sd = metainterp_sd
+        self.jitdriver_sd = jitdriver_sd
+
+    def handle_fail(self, metainterp_sd, jitdriver_sd):
+        # Failures of GUARD_NOT_INVALIDATED are never compiled,
+        # but always blackhole.
+        from pypy.jit.metainterp.blackhole import resume_in_blackhole
+        resume_in_blackhole(metainterp_sd, jitdriver_sd, self)
+        assert 0
+
+    def _clone_if_mutable(self):
+        res = ResumeGuardInvalidatedDescr(self.metainterp_sd, self.jitdriver_sd)
+        self.copy_all_attrbutes_into(res)
+        return res
+
 class ResumeGuardForcedDescr(ResumeGuardDescr):
 
     def __init__(self, metainterp_sd, jitdriver_sd):

diff --git a/pypy/jit/backend/llgraph/runner.py b/pypy/jit/backend/llgraph/runner.py
--- a/pypy/jit/backend/llgraph/runner.py
+++ b/pypy/jit/backend/llgraph/runner.py
@@ -123,7 +123,8 @@
         clt = original_loop_token.compiled_loop_token
         clt.loop_and_bridges.append(c)
         clt.compiling_a_bridge()
-        self._compile_loop_or_bridge(c, inputargs, operations)
+        self._compile_loop_or_bridge(c, inputargs, operations,
+                                     original_loop_token.invalidated_array)
         old, oldindex = faildescr._compiled_fail
         llimpl.compile_redirect_fail(old, oldindex, c)
 
@@ -138,14 +139,16 @@
         clt.loop_and_bridges = [c]
         clt.compiled_version = c
         looptoken.compiled_loop_token = clt
-        self._compile_loop_or_bridge(c, inputargs, operations)
+        self._compile_loop_or_bridge(c, inputargs, operations,
+                                     looptoken.invalidated_array)
 
     def free_loop_and_bridges(self, compiled_loop_token):
         for c in compiled_loop_token.loop_and_bridges:
             llimpl.mark_as_free(c)
         model.AbstractCPU.free_loop_and_bridges(self, compiled_loop_token)
 
-    def _compile_loop_or_bridge(self, c, inputargs, operations):
+    def _compile_loop_or_bridge(self, c, inputargs, operations,
+                                invalidated_array):
         var2index = {}
         for box in inputargs:
             if isinstance(box, history.BoxInt):
@@ -158,6 +161,7 @@
             else:
                 raise Exception("box is: %r" % (box,))
         self._compile_operations(c, operations, var2index)
+        llimpl.compile_add_inv_array(c, invalidated_array)
         return c
 
     def _compile_operations(self, c, operations, var2index):

diff --git a/pypy/jit/backend/llgraph/llimpl.py b/pypy/jit/backend/llgraph/llimpl.py
--- a/pypy/jit/backend/llgraph/llimpl.py
+++ b/pypy/jit/backend/llgraph/llimpl.py
@@ -168,7 +168,6 @@
 
 class CompiledLoop(object):
     has_been_freed = False
-    invalidated = False
 
     def __init__(self):
         self.inputargs = []
@@ -294,34 +293,15 @@
     del _variables[:]
     return _to_opaque(CompiledLoop())
 
+def compile_add_inv_array(loop, inv_array):
+    loop = _from_opaque(loop)
+    loop.invalidated_array = inv_array
+
 def mark_as_free(loop):
     loop = _from_opaque(loop)
     assert not loop.has_been_freed
     loop.has_been_freed = True
 
-def mark_as_invalid(loop):
-    _mark_as_invalid(_from_opaque(loop))
-
-def _mark_as_invalid(loop):
-    assert not loop.has_been_freed
-    for op in loop.operations:
-        if op.is_guard() and op.jump_target is not None:
-            _mark_as_invalid(op.jump_target)
-    loop.invalidated = True
-
-def invalidate_call_asm(from_loop, ctl):
-    from_loop = _from_opaque(from_loop)
-    _invalidate_call_asm(from_loop, ctl)
-
-def _invalidate_call_asm(from_loop, ctl):
-    for op in from_loop.operations:
-        if op.opnum == rop.CALL_ASSEMBLER:
-            call_target = op.descr().compiled_loop_token
-            if call_target is ctl:
-                op.descr = weakref.ref(op.descr()._tmp_token)
-        if op.is_guard() and op.jump_target is not None:
-            _invalidate_call_asm(op.jump_target, to_loop)
-
 def compile_start_int_var(loop):
     return compile_start_ref_var(loop, lltype.Signed)
 
@@ -901,7 +881,6 @@
         ctl = loop_token.compiled_loop_token
         if hasattr(ctl, 'redirected'):
             return self._do_call_assembler(ctl.redirected, *args)
-        assert not loop_token.invalidated
         assert not self._forced
         self._may_force = self.opindex
         try:
@@ -960,7 +939,7 @@
             raise GuardFailed
 
     def op_guard_not_invalidated(self, descr):
-        if self.loop.invalidated:
+        if self.loop.invalidated_array[0]:
             raise GuardFailed
 
 class OOFrame(Frame):
@@ -1693,9 +1672,8 @@
 setannotation(compile_add_fail, annmodel.SomeInteger())
 setannotation(compile_add_fail_arg, annmodel.s_None)
 setannotation(compile_redirect_fail, annmodel.s_None)
+setannotation(compile_add_inv_array, annmodel.s_None)
 setannotation(mark_as_free, annmodel.s_None)
-setannotation(mark_as_invalid, annmodel.s_None)
-setannotation(invalidate_call_asm, annmodel.s_None)
 
 setannotation(new_frame, s_Frame)
 setannotation(frame_clear, annmodel.s_None)

diff --git a/pypy/jit/metainterp/pyjitpl.py b/pypy/jit/metainterp/pyjitpl.py
--- a/pypy/jit/metainterp/pyjitpl.py
+++ b/pypy/jit/metainterp/pyjitpl.py
@@ -1085,6 +1085,9 @@
         if opnum == rop.GUARD_NOT_FORCED:
             resumedescr = compile.ResumeGuardForcedDescr(metainterp_sd,
                                                    metainterp.jitdriver_sd)
+        elif opnum == rop.GUARD_NOT_INVALIDATED:
+            resumedescr = compile.ResumeGuardInvalidatedDescr(metainterp_sd,
+                                                      metainterp.jitdriver_sd)
         else:
             resumedescr = compile.ResumeGuardDescr()
         guard_op = metainterp.history.record(opnum, moreargs, None,
@@ -1696,6 +1699,7 @@
         self.resumekey = compile.ResumeFromInterpDescr(original_greenkey)
         self.history.inputargs = original_boxes[num_green_args:]
         self.seen_loop_header_for_jdindex = -1
+        self.record_guard_not_invalidated()
         try:
             self.interpret()
         except GenerateMergePoint, gmp:
@@ -2302,6 +2306,9 @@
         self.history.operations[-1] = newop
         return resbox
 
+    def record_guard_not_invalidated(self):
+        self.framestack[-1].generate_guard(rop.GUARD_NOT_INVALIDATED, None)
+
     def direct_assembler_call(self, targetjitdriver_sd):
         """ Generate a direct call to assembler for portal entry point,
         patching the CALL_MAY_FORCE that occurred just now.

From commits-noreply at bitbucket.org  Thu Feb 17 16:07:32 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Thu, 17 Feb 2011 16:07:32 +0100 (CET)
Subject: [pypy-svn] pypy out-of-line-guards: Fix all tests. Part of logic
	can (and	should) be moved from warmstate to
Message-ID: <20110217150732.A6DFD282C19@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: out-of-line-guards
Changeset: r42113:a22af9a1af46
Date: 2011-02-17 08:07 -0700
http://bitbucket.org/pypy/pypy/changeset/a22af9a1af46/

Log:	Fix all tests. Part of logic can (and should) be moved from
	warmstate to the ResumeDescrNotInvalidated. There are also missing
	tests about call_assembler being correctly redirected

diff --git a/pypy/jit/metainterp/compile.py b/pypy/jit/metainterp/compile.py
--- a/pypy/jit/metainterp/compile.py
+++ b/pypy/jit/metainterp/compile.py
@@ -375,6 +375,7 @@
         # to the corresponding guard_op and compile from there
         assert metainterp.resumekey_original_loop_token is not None
         new_loop.token = metainterp.resumekey_original_loop_token
+        new_loop.token.invalidated_array = metainterp.invalidated_array
         inputargs = metainterp.history.inputargs
         if not we_are_translated():
             self._debug_suboperations = new_loop.operations
@@ -569,6 +570,7 @@
         # to every guard in the loop.
         new_loop_token = make_loop_token(len(redargs), jitdriver_sd)
         new_loop.token = new_loop_token
+        new_loop_token.invalidated_array = metainterp.invalidated_array
         send_loop_to_backend(metainterp_sd, new_loop, "entry bridge")
         # send the new_loop to warmspot.py, to be called directly the next time
         jitdriver_sd.warmstate.attach_unoptimized_bridge_from_interp(
@@ -659,6 +661,7 @@
     # 'redboxes' is only used to know the types of red arguments.
     inputargs = [box.clonebox() for box in redboxes]
     loop_token = make_loop_token(len(inputargs), jitdriver_sd)
+    loop_token.invalidated_array = cpu.create_invalidate_array()
     # 'nb_red_args' might be smaller than len(redboxes),
     # because it doesn't include the virtualizable boxes.
     nb_red_args = jitdriver_sd.num_red_args

diff --git a/pypy/jit/metainterp/warmstate.py b/pypy/jit/metainterp/warmstate.py
--- a/pypy/jit/metainterp/warmstate.py
+++ b/pypy/jit/metainterp/warmstate.py
@@ -158,7 +158,7 @@
         if self.compiled_merge_points_wref is not None:
             for wref in self.compiled_merge_points_wref:
                 looptoken = wref()
-                if looptoken is not None and not looptoken.invalidated:
+                if looptoken is not None and not looptoken.invalidated_array[0]:
                     result.append(looptoken)
         return result
 
@@ -169,7 +169,7 @@
     def get_entry_loop_token(self):
         if self.wref_entry_loop_token is not None:
             looptoken = self.wref_entry_loop_token()
-            if looptoken is not None and looptoken.invalidated:
+            if looptoken is not None and looptoken.invalidated_array[0]:
                 self.wref_entry_loop_token = None
             else:
                 return looptoken
@@ -336,7 +336,7 @@
                 if not confirm_enter_jit(*args):
                     return
                 loop_token = cell.get_entry_loop_token()
-                if loop_token is None or loop_token.invalidated:
+                if loop_token is None or loop_token.invalidated_array[0]:
                     # it was a weakref that has been freed or invalidated
                     cell.counter = 0
                     return

diff --git a/pypy/jit/metainterp/test/test_outofline.py b/pypy/jit/metainterp/test/test_outofline.py
--- a/pypy/jit/metainterp/test/test_outofline.py
+++ b/pypy/jit/metainterp/test/test_outofline.py
@@ -79,7 +79,7 @@
             return total
 
         assert self.meta_interp(f, []) == f()
-        self.check_loop_count(3)
+        self.check_loop_count(4)
         self.check_history(getfield_gc=0, getfield_gc_pure=0)        
 
     def test_jit_invariant_entry_bridge(self):
@@ -111,7 +111,7 @@
             return total
 
         assert self.meta_interp(main, []) == main()
-        self.check_loop_count(4)
+        self.check_loop_count(2)
         self.check_history(getfield_gc=0, getfield_gc_pure=0)
 
     def test_jit_invariant_invalidate_bridge(self):

diff --git a/pypy/jit/metainterp/memmgr.py b/pypy/jit/metainterp/memmgr.py
--- a/pypy/jit/metainterp/memmgr.py
+++ b/pypy/jit/metainterp/memmgr.py
@@ -68,7 +68,7 @@
         debug_print("Loop tokens before:", oldtotal)
         max_generation = self.current_generation - (self.max_age-1)
         for looptoken in self.alive_loops.keys():
-            if (looptoken.invalidated or
+            if (looptoken.invalidated_array[0] or
                 0 <= looptoken.generation < max_generation):
                 # XXX free looptoken invalidated ptr
                 del self.alive_loops[looptoken]

From commits-noreply at bitbucket.org  Thu Feb 17 17:10:58 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Thu, 17 Feb 2011 17:10:58 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix a NameError.
Message-ID: <20110217161058.219022A202C@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r42114:bf8dc9c9c134
Date: 2011-02-17 11:10 -0500
http://bitbucket.org/pypy/pypy/changeset/bf8dc9c9c134/

Log:	Fix a NameError.

diff --git a/lib-python/modified-2.7.0/distutils/tests/test_install.py b/lib-python/modified-2.7.0/distutils/tests/test_install.py
--- a/lib-python/modified-2.7.0/distutils/tests/test_install.py
+++ b/lib-python/modified-2.7.0/distutils/tests/test_install.py
@@ -2,6 +2,7 @@
 
 import os
 import unittest
+from test import test_support
 
 from distutils.command.install import install
 from distutils.core import Distribution
@@ -38,7 +39,7 @@
             expected = os.path.normpath(expected)
             self.assertEqual(got, expected)
 
-        if check_impl_detail():
+        if test_support.check_impl_detail():
             libdir = os.path.join(destination, "lib", "python")
             check_path(cmd.install_lib, libdir)
             check_path(cmd.install_platlib, libdir)

From commits-noreply at bitbucket.org  Thu Feb 17 17:18:11 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Thu, 17 Feb 2011 17:18:11 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix tests checking the extension of .so
	files
Message-ID: <20110217161811.21E97282C18@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r42115:07bdbec7a699
Date: 2011-02-17 11:17 -0500
http://bitbucket.org/pypy/pypy/changeset/07bdbec7a699/

Log:	Fix tests checking the extension of .so files

diff --git a/lib-python/modified-2.7.0/distutils/tests/test_build_ext.py b/lib-python/modified-2.7.0/distutils/tests/test_build_ext.py
--- a/lib-python/modified-2.7.0/distutils/tests/test_build_ext.py
+++ b/lib-python/modified-2.7.0/distutils/tests/test_build_ext.py
@@ -292,7 +292,7 @@
         finally:
             os.chdir(old_wd)
         self.assert_(os.path.exists(so_file))
-        self.assertEquals(os.path.splitext(so_file)[-1],
+        self.assertEquals(so_file[so_file.index(os.path.extsep):],
                           sysconfig.get_config_var('SO'))
         so_dir = os.path.dirname(so_file)
         self.assertEquals(so_dir, other_tmp_dir)
@@ -301,7 +301,7 @@
         cmd.run()
         so_file = cmd.get_outputs()[0]
         self.assert_(os.path.exists(so_file))
-        self.assertEquals(os.path.splitext(so_file)[-1],
+        self.assertEquals(so_file[so_file.index(os.path.extsep):],
                           sysconfig.get_config_var('SO'))
         so_dir = os.path.dirname(so_file)
         self.assertEquals(so_dir, cmd.build_lib)

From commits-noreply at bitbucket.org  Thu Feb 17 20:07:51 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Thu, 17 Feb 2011 20:07:51 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: Fix a typo and failures
	in	test_descroperation.pt
Message-ID: <20110217190751.98C3C282BE3@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42116:2d9dc38c523d
Date: 2011-02-17 14:40 +0100
http://bitbucket.org/pypy/pypy/changeset/2d9dc38c523d/

Log:	Fix a typo and failures in test_descroperation.pt

diff --git a/pypy/module/__builtin__/interp_classobj.py b/pypy/module/__builtin__/interp_classobj.py
--- a/pypy/module/__builtin__/interp_classobj.py
+++ b/pypy/module/__builtin__/interp_classobj.py
@@ -749,7 +749,7 @@
     meth = new.instancemethod(func, None, W_InstanceObject)
     rawdict[specialname] = interp2app(meth)
     rmeth = new.instancemethod(rfunc, None, W_InstanceObject)
-    rawdict[rspecialname] = interp2app(meth)
+    rawdict[rspecialname] = interp2app(rmeth)
 
 
 def descr_del_dict(space, w_inst):

From commits-noreply at bitbucket.org  Thu Feb 17 20:07:52 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Thu, 17 Feb 2011 20:07:52 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: Translation fix
Message-ID: <20110217190752.2D412282BE3@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42117:3083febe5edc
Date: 2011-02-17 17:14 +0100
http://bitbucket.org/pypy/pypy/changeset/3083febe5edc/

Log:	Translation fix

diff --git a/pypy/objspace/std/proxy_helpers.py b/pypy/objspace/std/proxy_helpers.py
--- a/pypy/objspace/std/proxy_helpers.py
+++ b/pypy/objspace/std/proxy_helpers.py
@@ -30,7 +30,7 @@
 
 def install_args_w_trampoline(type_, mm, is_local, op_name):
     def function(space, w_transparent_list, *args_w):
-        args = Arguments(space, [space.wrap(op_name)] + list(args_w[:-1]), w_stararg=args_w[-1])
+        args = Arguments(space, [space.wrap(op_name)] + list(args_w[:-1]) + args_w[-1])
         return space.call_args(w_transparent_list.w_controller, args)
     
     function = func_with_new_name(function, mm.name)

From commits-noreply at bitbucket.org  Thu Feb 17 20:07:53 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Thu, 17 Feb 2011 20:07:53 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: Test and fix
Message-ID: <20110217190753.55FFB282C19@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42118:17c2817d710a
Date: 2011-02-17 17:41 +0100
http://bitbucket.org/pypy/pypy/changeset/17c2817d710a/

Log:	Test and fix

diff --git a/pypy/objspace/std/objecttype.py b/pypy/objspace/std/objecttype.py
--- a/pypy/objspace/std/objecttype.py
+++ b/pypy/objspace/std/objecttype.py
@@ -210,6 +210,7 @@
     __reduce_ex__ = gateway.interp2app(descr__reduce_ex__),
     __reduce__ = gateway.interp2app(descr__reduce__),
     __format__ = gateway.interp2app(descr___format__),
-    __subclasshook__ = gateway.interp2app(descr___subclasshook__),
+    __subclasshook__ = gateway.interp2app(descr___subclasshook__,
+                                          as_classmethod=True),
     __init__ = gateway.interp2app(descr__init__),
     )

diff --git a/pypy/objspace/std/test/test_obj.py b/pypy/objspace/std/test/test_obj.py
--- a/pypy/objspace/std/test/test_obj.py
+++ b/pypy/objspace/std/test/test_obj.py
@@ -68,3 +68,4 @@
         class x(object):
             pass
         assert x().__subclasshook__(object()) is NotImplemented
+        assert x.__subclasshook__(object()) is NotImplemented

From commits-noreply at bitbucket.org  Thu Feb 17 20:07:55 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Thu, 17 Feb 2011 20:07:55 +0100 (CET)
Subject: [pypy-svn] pypy default: Properly convert WindowsErrors coming from
	the streamio	module.
Message-ID: <20110217190755.57872282C19@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r42119:ff019dd9c604
Date: 2011-02-17 18:45 +0100
http://bitbucket.org/pypy/pypy/changeset/ff019dd9c604/

Log:	Properly convert WindowsErrors coming from the streamio module.
	Otherwise all you get is a IOError: [Errno 0] No error: 'foo'

diff --git a/pypy/module/_file/test/test_file_extra.py b/pypy/module/_file/test/test_file_extra.py
--- a/pypy/module/_file/test/test_file_extra.py
+++ b/pypy/module/_file/test/test_file_extra.py
@@ -520,6 +520,14 @@
         assert data == 'hel'
         f.close()
 
+        import errno, sys
+        f = open(fn)
+        exc = raises(OSError, f.truncate, 3)
+        assert exc.value.errno == errno.EACCES
+        if sys.platform == 'win32':
+            assert exc.value.winerror == 5 # ERROR_ACCESS_DENIED
+        f.close()
+
     def test_readinto(self):
         from array import array
         a = array('c')

diff --git a/pypy/module/_file/interp_stream.py b/pypy/module/_file/interp_stream.py
--- a/pypy/module/_file/interp_stream.py
+++ b/pypy/module/_file/interp_stream.py
@@ -2,7 +2,7 @@
 from pypy.rlib import streamio
 from pypy.rlib.streamio import StreamErrors
 
-from pypy.interpreter.error import OperationError
+from pypy.interpreter.error import OperationError, wrap_oserror2
 from pypy.interpreter.gateway import ObjSpace
 from pypy.interpreter.baseobjspace import Wrappable
 from pypy.interpreter.typedef import TypeDef
@@ -17,23 +17,13 @@
     elif isinstance(e, OSError):
         return wrap_oserror_as_ioerror(space, e, w_filename)
     else:
+        # should not happen: wrap_streamerror() is only called when
+        # StreamErrors = (OSError, StreamError) are raised
         return OperationError(space.w_IOError, space.w_None)
 
 def wrap_oserror_as_ioerror(space, e, w_filename=None):
-    assert isinstance(e, OSError)
-    errno = e.errno
-    try:
-        msg = os.strerror(errno)
-    except ValueError:
-        msg = 'error %d' % errno
-    if w_filename is None:
-        w_filename = space.w_None
-    w_error = space.call_function(space.w_IOError,
-                                  space.wrap(errno),
-                                  space.wrap(msg),
-                                  w_filename)
-    return OperationError(space.w_IOError, w_error)
-
+    return wrap_oserror2(space, e, w_filename,
+                         w_exception_class=space.w_IOError)
 
 class W_AbstractStream(Wrappable):
     """Base class for interp-level objects that expose streams to app-level"""

From commits-noreply at bitbucket.org  Thu Feb 17 20:07:56 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Thu, 17 Feb 2011 20:07:56 +0100 (CET)
Subject: [pypy-svn] pypy default: Be sure to close all mmaps and files,
	otherwise os.unlink fails on Windows
Message-ID: <20110217190756.89850282C19@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r42120:b480be79614b
Date: 2011-02-17 19:04 +0100
http://bitbucket.org/pypy/pypy/changeset/b480be79614b/

Log:	Be sure to close all mmaps and files, otherwise os.unlink fails on
	Windows

diff --git a/lib-python/modified-2.7.0/test/test_mmap.py b/lib-python/modified-2.7.0/test/test_mmap.py
--- a/lib-python/modified-2.7.0/test/test_mmap.py
+++ b/lib-python/modified-2.7.0/test/test_mmap.py
@@ -168,9 +168,11 @@
         else:
             self.fail("Able to resize readonly memory map")
         f.close()
+        m.close()
         del m, f
-        self.assertEqual(open(TESTFN, "rb").read(), 'a'*mapsize,
-               "Readonly memory map data file was modified")
+        with open(TESTFN, "rb") as f:
+            self.assertEqual(f.read(), 'a'*mapsize,
+                "Readonly memory map data file was modified")
 
         # Opening mmap with size too big
         import sys
@@ -220,11 +222,13 @@
         self.assertEqual(m[:], 'd' * mapsize,
                "Copy-on-write memory map data not written correctly.")
         m.flush()
-        self.assertEqual(open(TESTFN, "rb").read(), 'c'*mapsize,
-               "Copy-on-write test data file should not be modified.")
+        f.close()
+        with open(TESTFN, "rb") as f:
+            self.assertEqual(f.read(), 'c'*mapsize,
+                "Copy-on-write test data file should not be modified.")
         # Ensuring copy-on-write maps cannot be resized
         self.assertRaises(TypeError, m.resize, 2*mapsize)
-        f.close()
+        m.close()
         del m, f
 
         # Ensuring invalid access parameter raises exception
@@ -279,6 +283,7 @@
         self.assertEqual(m.find('one', 1), 8)
         self.assertEqual(m.find('one', 1, -1), 8)
         self.assertEqual(m.find('one', 1, -2), -1)
+        m.close()
 
 
     def test_rfind(self):
@@ -297,6 +302,7 @@
         self.assertEqual(m.rfind('one', 0, -2), 0)
         self.assertEqual(m.rfind('one', 1, -1), 8)
         self.assertEqual(m.rfind('one', 1, -2), -1)
+        m.close()
 
 
     def test_double_close(self):
@@ -538,6 +544,7 @@
         self.assertEquals(m[:], "012bar6789")
         m.seek(8)
         self.assertRaises(ValueError, m.write, "bar")
+        m.close()
 
     if os.name == 'nt':
         def test_tagname(self):

From commits-noreply at bitbucket.org  Thu Feb 17 20:07:58 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Thu, 17 Feb 2011 20:07:58 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix two crashers in mmap on windows
Message-ID: <20110217190758.AD6B6282C22@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r42121:f1b2c255611b
Date: 2011-02-17 20:01 +0100
http://bitbucket.org/pypy/pypy/changeset/f1b2c255611b/

Log:	Fix two crashers in mmap on windows

diff --git a/pypy/rlib/test/test_rmmap.py b/pypy/rlib/test/test_rmmap.py
--- a/pypy/rlib/test/test_rmmap.py
+++ b/pypy/rlib/test/test_rmmap.py
@@ -1,6 +1,7 @@
 from pypy.tool.udir import udir
-import os
+import os, sys
 from pypy.rpython.test.test_llinterp import interpret
+from pypy.rlib.rarithmetic import intmask
 from pypy.rlib import rmmap as mmap
 from pypy.rlib.rmmap import RTypeError, RValueError, alloc, free
 
@@ -312,7 +313,6 @@
         f.close()
     
     def test_resize(self):
-        import sys
         if ("darwin" in sys.platform) or ("freebsd" in sys.platform):
             skip("resize does not work under OSX or FreeBSD")
         
@@ -325,10 +325,10 @@
         def func(no):
             m = mmap.mmap(no, 6, access=mmap.ACCESS_WRITE)
             f_size = os.fstat(no).st_size
-            assert m.file_size() == f_size == 6
+            assert intmask(m.file_size()) == f_size == 6
             m.resize(10)
             f_size = os.fstat(no).st_size
-            assert m.file_size() == f_size == 10
+            assert intmask(m.file_size()) == f_size == 10
             m.close()
 
         interpret(func, [f.fileno()])
@@ -417,6 +417,34 @@
 
         compile(func, [int])
 
+    def test_windows_crasher_1(self):
+        if sys.platform != "win32":
+            skip("Windows-only test")
+
+        m = mmap.mmap(-1, 1000, tagname="foo")
+        # same tagname, but larger size
+        try:
+            m2 = mmap.mmap(-1, 5000, tagname="foo")
+            m2.getitem(4500)
+        except WindowsError:
+            pass
+        m.close()
+
+    def test_windows_crasher_2(self):
+        if sys.platform != "win32":
+            skip("Windows-only test")
+
+        f = open(self.tmpname + "t", "w+")
+        f.write("foobar")
+        f.flush()
+
+        f = open(self.tmpname + "t", "r+b")
+        m = mmap.mmap(f.fileno(), 0)
+        f.close()
+        raises(WindowsError, m.resize, 0)
+        raises(RValueError, m.getitem, 0)
+        m.close()
+
 def test_alloc_free():
     map_size = 65536
     data = alloc(map_size)

diff --git a/pypy/rlib/rmmap.py b/pypy/rlib/rmmap.py
--- a/pypy/rlib/rmmap.py
+++ b/pypy/rlib/rmmap.py
@@ -561,7 +561,11 @@
                     charp = rffi.cast(LPCSTR, data)
                     self.setdata(charp, newsize)
                     return
-            raise rwin32.lastWindowsError()
+            winerror = rwin32.lastWindowsError()
+            if self.map_handle:
+                rwin32.CloseHandle(self.map_handle)
+            self.map_handle = INVALID_HANDLE
+            raise winerror
 
     def len(self):
         self.check_valid()
@@ -784,13 +788,17 @@
 
         if m.map_handle:
             data = MapViewOfFile(m.map_handle, dwDesiredAccess,
-                                 offset_hi, offset_lo, 0)
+                                 offset_hi, offset_lo, length)
             if data:
                 # XXX we should have a real LPVOID which must always be casted
                 charp = rffi.cast(LPCSTR, data)
                 m.setdata(charp, map_size)
                 return m
-        raise rwin32.lastWindowsError()
+        winerror = rwin32.lastWindowsError()
+        if m.map_handle:
+            rwin32.CloseHandle(m.map_handle)
+        m.map_handle = INVALID_HANDLE
+        raise winerror
 
     def alloc(map_size):
         """Allocate memory.  This is intended to be used by the JIT,

From commits-noreply at bitbucket.org  Thu Feb 17 22:08:53 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Thu, 17 Feb 2011 22:08:53 +0100 (CET)
Subject: [pypy-svn] pypy interplevel-exception-classes: A branch to
	define	module-specific exceptions in RPython
Message-ID: <20110217210853.C081C282BE3@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: interplevel-exception-classes
Changeset: r42122:2ece37cdcbff
Date: 2011-02-17 22:08 +0100
http://bitbucket.org/pypy/pypy/changeset/2ece37cdcbff/

Log:	A branch to define module-specific exceptions in RPython and get rid
	of most appleveldefs.


From agaynor at codespeak.net  Thu Feb 17 22:43:22 2011
From: agaynor at codespeak.net (agaynor at codespeak.net)
Date: Thu, 17 Feb 2011 22:43:22 +0100 (CET)
Subject: [pypy-svn] r80359 - pypy/extradoc/talk/ustour2011
Message-ID: <20110217214322.8DC8B282BE3@codespeak.net>

Author: agaynor
Date: Thu Feb 17 22:43:20 2011
New Revision: 80359

Modified:
   pypy/extradoc/talk/ustour2011/mozilla-abstract.txt
Log:
Typo.


Modified: pypy/extradoc/talk/ustour2011/mozilla-abstract.txt
==============================================================================
--- pypy/extradoc/talk/ustour2011/mozilla-abstract.txt	(original)
+++ pypy/extradoc/talk/ustour2011/mozilla-abstract.txt	Thu Feb 17 22:43:20 2011
@@ -65,5 +65,5 @@
 please send a note to lac at openend.se and we will make sure that we
 cover this in our presentation.
 
-The PyPy project has a blog which make be of interest.  You can read
-it at http://morepypy.blogspot.com/ .
+The PyPy project has a blog which may be of interest.  You can read it
+at http://morepypy.blogspot.com/ .


From commits-noreply at bitbucket.org  Fri Feb 18 00:48:02 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri, 18 Feb 2011 00:48:02 +0100 (CET)
Subject: [pypy-svn] pypy interplevel-exception-classes: New
	function:	error.new_exception_class(space, name, w_base)
Message-ID: <20110217234802.67F2D282BE3@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: interplevel-exception-classes
Changeset: r42123:c9d190210332
Date: 2011-02-17 23:15 +0100
http://bitbucket.org/pypy/pypy/changeset/c9d190210332/

Log:	New function: error.new_exception_class(space, name, w_base)

diff --git a/pypy/interpreter/test/test_error.py b/pypy/interpreter/test/test_error.py
--- a/pypy/interpreter/test/test_error.py
+++ b/pypy/interpreter/test/test_error.py
@@ -1,7 +1,7 @@
 import py, os, errno
 from pypy.interpreter.error import OperationError, operationerrfmt
 from pypy.interpreter.error import decompose_valuefmt, get_operrcls2
-from pypy.interpreter.error import wrap_oserror
+from pypy.interpreter.error import wrap_oserror, new_exception_class
 
 
 def test_decompose_valuefmt():
@@ -74,3 +74,26 @@
     assert e.get_w_value(space) == ([SystemError], [errno.EBADF],
                                     [os.strerror(errno.EBADF)],
                                     ["test.py"])
+
+def test_new_exception(space):
+    w_error = new_exception_class(space, 'error', module="_socket")
+    assert w_error.getname(space) == 'error'
+    assert space.str_w(space.repr(w_error)) == "<class '_socket.error'>"
+    operr = OperationError(w_error, space.wrap("message"))
+    assert operr.match(space, w_error)
+    assert operr.match(space, space.w_Exception)
+
+    # subclass of ValueError
+    w_error = new_exception_class(space, 'error', space.w_ValueError)
+    operr = OperationError(w_error, space.wrap("message"))
+    assert operr.match(space, w_error)
+    assert operr.match(space, space.w_ValueError)
+
+    # subclass of (ValueError, TypeError)
+    w_bases = space.newtuple([space.w_ValueError, space.w_TypeError])
+    w_error = new_exception_class(space, 'error', w_bases)
+    operr = OperationError(w_error, space.wrap("message"))
+    assert operr.match(space, w_error)
+    assert operr.match(space, space.w_ValueError)
+    assert operr.match(space, space.w_TypeError)
+

diff --git a/pypy/interpreter/error.py b/pypy/interpreter/error.py
--- a/pypy/interpreter/error.py
+++ b/pypy/interpreter/error.py
@@ -408,3 +408,23 @@
     msg = os.strerror(errno)
     w_error = space.call_function(w_type, space.wrap(errno), space.wrap(msg))
     return OperationError(w_type, w_error)
+
+def new_exception_class(space, name, w_bases=None, module=None, w_dict=None):
+    """Create a new exception type.
+    @param name: the name of the type.
+    @param w_bases: Either an exception type, or a wrapped tuple of
+                    exception types.  default is space.w_Exception.
+    @param module: optional module name.
+    @param w_dict: an optional dictionary to populate the class __dict__.
+    """
+    if w_bases is None:
+        w_bases = space.newtuple([space.w_Exception])
+    elif not space.isinstance_w(w_bases, space.w_tuple):
+        w_bases = space.newtuple([w_bases])
+    if w_dict is None:
+        w_dict = space.newdict()
+    w_exc = space.call_function(
+        space.w_type, space.wrap(name), w_bases, w_dict)
+    if module is not None:
+        space.setattr(w_exc, space.wrap("__module__"), space.wrap(module))
+    return w_exc

From commits-noreply at bitbucket.org  Fri Feb 18 00:48:03 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri, 18 Feb 2011 00:48:03 +0100 (CET)
Subject: [pypy-svn] pypy interplevel-exception-classes: Try a slightly
	different	interface, class name can be "module.name"
Message-ID: <20110217234803.34BF8282BE3@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: interplevel-exception-classes
Changeset: r42124:1ae0fbf48f5a
Date: 2011-02-17 23:42 +0100
http://bitbucket.org/pypy/pypy/changeset/1ae0fbf48f5a/

Log:	Try a slightly different interface, class name can be "module.name"

diff --git a/pypy/interpreter/test/test_error.py b/pypy/interpreter/test/test_error.py
--- a/pypy/interpreter/test/test_error.py
+++ b/pypy/interpreter/test/test_error.py
@@ -76,7 +76,7 @@
                                     ["test.py"])
 
 def test_new_exception(space):
-    w_error = new_exception_class(space, 'error', module="_socket")
+    w_error = new_exception_class(space, '_socket.error')
     assert w_error.getname(space) == 'error'
     assert space.str_w(space.repr(w_error)) == "<class '_socket.error'>"
     operr = OperationError(w_error, space.wrap("message"))

diff --git a/pypy/interpreter/error.py b/pypy/interpreter/error.py
--- a/pypy/interpreter/error.py
+++ b/pypy/interpreter/error.py
@@ -409,14 +409,17 @@
     w_error = space.call_function(w_type, space.wrap(errno), space.wrap(msg))
     return OperationError(w_type, w_error)
 
-def new_exception_class(space, name, w_bases=None, module=None, w_dict=None):
+def new_exception_class(space, name, w_bases=None, w_dict=None):
     """Create a new exception type.
     @param name: the name of the type.
     @param w_bases: Either an exception type, or a wrapped tuple of
                     exception types.  default is space.w_Exception.
-    @param module: optional module name.
     @param w_dict: an optional dictionary to populate the class __dict__.
     """
+    if '.' in name:
+        module, name = name.rsplit('.', 1)
+    else:
+        module = None
     if w_bases is None:
         w_bases = space.newtuple([space.w_Exception])
     elif not space.isinstance_w(w_bases, space.w_tuple):
@@ -425,6 +428,6 @@
         w_dict = space.newdict()
     w_exc = space.call_function(
         space.w_type, space.wrap(name), w_bases, w_dict)
-    if module is not None:
+    if module:
         space.setattr(w_exc, space.wrap("__module__"), space.wrap(module))
     return w_exc

From commits-noreply at bitbucket.org  Fri Feb 18 00:48:08 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri, 18 Feb 2011 00:48:08 +0100 (CET)
Subject: [pypy-svn] pypy interplevel-exception-classes: Implement
	_socket._error and	_ssl.SSLError at interp_level.
Message-ID: <20110217234808.1FB65282C18@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: interplevel-exception-classes
Changeset: r42125:2b0d0b8bb55e
Date: 2011-02-18 00:48 +0100
http://bitbucket.org/pypy/pypy/changeset/2b0d0b8bb55e/

Log:	Implement _socket._error and _ssl.SSLError at interp_level. The
	app_* files are no longer needed,

	Flowgraphs of the the _ssl module don't need to build the _socket
	Module anymore. Now let's try a translation.

diff --git a/pypy/module/_socket/interp_socket.py b/pypy/module/_socket/interp_socket.py
--- a/pypy/module/_socket/interp_socket.py
+++ b/pypy/module/_socket/interp_socket.py
@@ -4,10 +4,12 @@
 from pypy.interpreter.gateway import ObjSpace, W_Root, NoneNotWrapped
 from pypy.interpreter.gateway import interp2app
 from pypy.rlib.rarithmetic import intmask
+from pypy.rlib.objectmodel import specialize
 from pypy.rlib import rsocket
 from pypy.rlib.rsocket import RSocket, AF_INET, SOCK_STREAM
 from pypy.rlib.rsocket import SocketError, SocketErrorWithErrno
 from pypy.interpreter.error import OperationError, operationerrfmt
+from pypy.interpreter.error import new_exception_class
 from pypy.interpreter import gateway
 
 class SignalChecker:
@@ -451,10 +453,27 @@
 # ____________________________________________________________
 # Error handling
 
+class SocketAPI:
+    def __init__(self, space):
+        self.w_error = new_exception_class(
+            space, "_socket.error", space.w_IOError)
+        self.w_herror = new_exception_class(
+            space, "_socket.herror", self.w_error)
+        self.w_gaierror = new_exception_class(
+            space, "_socket.gaierror", self.w_error)
+        self.w_timeout = new_exception_class(
+            space, "_socket.timeout", self.w_error)
+
+    @specialize.memo()
+    def get_exception(self, applevelerrcls):
+        return getattr(self, 'w_' + applevelerrcls)
+
+def get_error(space, name):
+    return space.fromcache(SocketAPI).get_exception(name)
+
 def converted_error(space, e):
     message = e.get_msg()
-    w_module = space.getbuiltinmodule('_socket')
-    w_exception_class = space.getattr(w_module, space.wrap(e.applevelerrcls))
+    w_exception_class = get_error(space, e.applevelerrcls)
     if isinstance(e, SocketErrorWithErrno):
         w_exception = space.call_function(w_exception_class, space.wrap(e.errno),
                                       space.wrap(message))

diff --git a/pypy/module/_ssl/app_ssl.py b/pypy/module/_ssl/app_ssl.py
deleted file mode 100644
--- a/pypy/module/_ssl/app_ssl.py
+++ /dev/null
@@ -1,7 +0,0 @@
-import _socket
-
-class SSLError(_socket.error):
-    pass
-
-__doc__ = """Implementation module for SSL socket operations. 
-See the socket module for documentation."""

diff --git a/pypy/module/_ssl/interp_ssl.py b/pypy/module/_ssl/interp_ssl.py
--- a/pypy/module/_ssl/interp_ssl.py
+++ b/pypy/module/_ssl/interp_ssl.py
@@ -1,5 +1,5 @@
 from pypy.rpython.lltypesystem import rffi, lltype
-from pypy.interpreter.error import OperationError
+from pypy.interpreter.error import OperationError, new_exception_class
 from pypy.interpreter.baseobjspace import W_Root, ObjSpace, Wrappable
 from pypy.interpreter.typedef import TypeDef
 from pypy.interpreter.gateway import interp2app
@@ -562,3 +562,11 @@
 sslwrap.unwrap_spec = [ObjSpace, W_Root, int, W_Root, W_Root,
                        int, int, W_Root, W_Root]
 
+class Cache:
+    def __init__(self, space):
+        w_socketerror = interp_socket.get_error(space, "error")
+        self.w_error = new_exception_class(
+            space, "_ssl.SSLError", w_socketerror)
+
+def get_error(space):
+    return space.fromcache(Cache).w_error

diff --git a/pypy/module/_socket/app_socket.py b/pypy/module/_socket/app_socket.py
deleted file mode 100644
--- a/pypy/module/_socket/app_socket.py
+++ /dev/null
@@ -1,15 +0,0 @@
-"""Implementation module for socket operations.
-
-See the socket module for documentation."""
-
-class error(IOError):
-    pass
-
-class herror(error):
-    pass
-
-class gaierror(error):
-    pass
-
-class timeout(error):
-    pass

diff --git a/pypy/module/_ssl/__init__.py b/pypy/module/_ssl/__init__.py
--- a/pypy/module/_ssl/__init__.py
+++ b/pypy/module/_ssl/__init__.py
@@ -1,13 +1,15 @@
 from pypy.interpreter.mixedmodule import MixedModule
 
 class Module(MixedModule):
+    """Implementation module for SSL socket operations.
+    See the socket module for documentation."""
+
     interpleveldefs = {
         'sslwrap': 'interp_ssl.sslwrap',
+        'SSLError': 'interp_ssl.get_error(space)',
     }
 
     appleveldefs = {
-        '__doc__': 'app_ssl.__doc__',
-        'SSLError': 'app_ssl.SSLError',
     }
 
     @classmethod

diff --git a/pypy/module/_socket/__init__.py b/pypy/module/_socket/__init__.py
--- a/pypy/module/_socket/__init__.py
+++ b/pypy/module/_socket/__init__.py
@@ -5,15 +5,15 @@
 class Module(MixedModule):
 
     appleveldefs = {
-        'error'      : 'app_socket.error',
-        'herror'     : 'app_socket.herror',
-        'gaierror'   : 'app_socket.gaierror',
-        'timeout'    : 'app_socket.timeout',
     }
 
     interpleveldefs = {
         'SocketType':  'interp_socket.W_RSocket',
         'socket'    :  'interp_socket.W_RSocket',
+        'error'     :  'interp_socket.get_error(space, "error")',
+        'herror'    :  'interp_socket.get_error(space, "herror")',
+        'gaierror'  :  'interp_socket.get_error(space, "gaierror")',
+        'timeout'   :  'interp_socket.get_error(space, "timeout")',
     }
 
     def startup(self, space):

diff --git a/pypy/module/_ssl/test/test_ssl.py b/pypy/module/_ssl/test/test_ssl.py
--- a/pypy/module/_ssl/test/test_ssl.py
+++ b/pypy/module/_ssl/test/test_ssl.py
@@ -11,8 +11,10 @@
         import _ssl
     
     def test_sslerror(self):
-        import _ssl
+        import _ssl, _socket
         assert issubclass(_ssl.SSLError, Exception)
+        assert issubclass(_ssl.SSLError, IOError)
+        assert issubclass(_ssl.SSLError, _socket.error)
 
     def test_constants(self):
         import _ssl

From commits-noreply at bitbucket.org  Fri Feb 18 01:01:41 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri, 18 Feb 2011 01:01:41 +0100 (CET)
Subject: [pypy-svn] pypy default: Merge branch improve-unwrap_spec.
Message-ID: <20110218000141.AD60C282BE3@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r42126:132d8c39f628
Date: 2011-02-18 00:59 +0100
http://bitbucket.org/pypy/pypy/changeset/132d8c39f628/

Log:	Merge branch improve-unwrap_spec.

	The @unwrap_spec decorator is now the preferred way to describe how
	a RPython function is exposed to applevel. it now accepts keyword
	arguments, and it's not necessary to specify the parameters which
	have a default conversion. It is often possible to even omit the
	unwrap_spec! Default conversions are:
	- parameters which name starts with "w_" are considered as W_Root.
	- 'self' gets the "self" unwrap spec
	- 'space' is the ObjSpace
	- '__args__' receives the Arguments object
	- 'args_w' receives the list of (wrapped) remaining args

	In addition, GetSetProperty getter/setter functions can be
	indifferently written with the two signatures: (space, self) or
	(self, space). The latter makes more sense when the getter is
	defined as a method!

diff --git a/pypy/module/math/_genmath.py b/pypy/module/math/_genmath.py
deleted file mode 100644
--- a/pypy/module/math/_genmath.py
+++ /dev/null
@@ -1,62 +0,0 @@
-# ONESHOT SCRIPT (probably can go away soon)
-# to generate the mixed module 'math' (see same directory) 
-import py
-import math
-import re
-import sys
-rex_arg = re.compile(".*\((.*)\).*")
-
-if __name__ == '__main__': 
-    print py.code.Source("""
-        import math 
-        from pypy.interpreter.gateway import ObjSpace
-
-    """)
-    names = []
-    for name, func in math.__dict__.items(): 
-        if not callable(func): 
-            continue
-        sig = func.__doc__.split('\n')[0].strip()
-        sig = sig.split('->')[0].strip()
-        m = rex_arg.match(sig) 
-        assert m
-        args = m.group(1)
-        args = ", ".join(args.split(','))
-        sig = sig.replace('(', '(space,')
-        sig = ", ".join(sig.split(','))
-        argc = len(args.split(','))
-        unwrap_spec = ['ObjSpace']
-        unwrap_spec += ['float'] * argc 
-        unwrap_spec = ", ".join(unwrap_spec)
-        doc = func.__doc__.replace('\n', '\n       ')
-        
-        print py.code.Source('''
-            def %(sig)s: 
-                """%(doc)s
-                """
-                return space.wrap(math.%(name)s(%(args)s))
-            %(name)s.unwrap_spec = [%(unwrap_spec)s]
-        ''' % locals())
-        names.append(name) 
-
-    print >>sys.stderr, py.code.Source("""
-        # Package initialisation
-        from pypy.interpreter.mixedmodule import MixedModule
-
-        class Module(MixedModule):
-            appleveldefs = {
-            }
-            interpleveldefs = {
-    """)
-        
-    for name in names: 
-        space = " " * (15-len(name))
-        print >>sys.stderr, (
-            "       %(name)r%(space)s: 'interp_math.%(name)s'," % locals())
-    print >>sys.stderr, py.code.Source("""
-        }
-    """) 
-            
-        
-        
-        

From commits-noreply at bitbucket.org  Fri Feb 18 01:01:41 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri, 18 Feb 2011 01:01:41 +0100 (CET)
Subject: [pypy-svn] pypy improve-unwrap_spec: Close branch
Message-ID: <20110218000141.F06FE282C18@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: improve-unwrap_spec
Changeset: r42127:96f57362b7b2
Date: 2011-02-18 01:00 +0100
http://bitbucket.org/pypy/pypy/changeset/96f57362b7b2/

Log:	Close branch


From commits-noreply at bitbucket.org  Fri Feb 18 01:12:44 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri, 18 Feb 2011 01:12:44 +0100 (CET)
Subject: [pypy-svn] pypy interplevel-exception-classes: Try to fix
	translation
Message-ID: <20110218001244.58F00282BE3@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: interplevel-exception-classes
Changeset: r42128:1e842fd10e8e
Date: 2011-02-18 01:12 +0100
http://bitbucket.org/pypy/pypy/changeset/1e842fd10e8e/

Log:	Try to fix translation

diff --git a/pypy/module/_socket/interp_socket.py b/pypy/module/_socket/interp_socket.py
--- a/pypy/module/_socket/interp_socket.py
+++ b/pypy/module/_socket/interp_socket.py
@@ -464,10 +464,10 @@
         self.w_timeout = new_exception_class(
             space, "_socket.timeout", self.w_error)
 
-    @specialize.memo()
     def get_exception(self, applevelerrcls):
         return getattr(self, 'w_' + applevelerrcls)
 
+ at specialize.arg(1)
 def get_error(space, name):
     return space.fromcache(SocketAPI).get_exception(name)
 

From commits-noreply at bitbucket.org  Fri Feb 18 01:59:27 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri, 18 Feb 2011 01:59:27 +0100 (CET)
Subject: [pypy-svn] pypy interplevel-exception-classes: Another attempt to
	translate
Message-ID: <20110218005927.90943282BE3@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: interplevel-exception-classes
Changeset: r42129:4d475a8682c5
Date: 2011-02-18 01:59 +0100
http://bitbucket.org/pypy/pypy/changeset/4d475a8682c5/

Log:	Another attempt to translate

diff --git a/pypy/module/_socket/interp_socket.py b/pypy/module/_socket/interp_socket.py
--- a/pypy/module/_socket/interp_socket.py
+++ b/pypy/module/_socket/interp_socket.py
@@ -4,7 +4,6 @@
 from pypy.interpreter.gateway import ObjSpace, W_Root, NoneNotWrapped
 from pypy.interpreter.gateway import interp2app
 from pypy.rlib.rarithmetic import intmask
-from pypy.rlib.objectmodel import specialize
 from pypy.rlib import rsocket
 from pypy.rlib.rsocket import RSocket, AF_INET, SOCK_STREAM
 from pypy.rlib.rsocket import SocketError, SocketErrorWithErrno
@@ -464,10 +463,15 @@
         self.w_timeout = new_exception_class(
             space, "_socket.timeout", self.w_error)
 
+        self.errors_w = {'error': self.w_error,
+                         'herror': self.w_herror,
+                         'gaierror': self.w_gaierror,
+                         'timeout': self.w_timeout,
+                         }
+
     def get_exception(self, applevelerrcls):
-        return getattr(self, 'w_' + applevelerrcls)
+        return self.errors_w[applevelerrcls]
 
- at specialize.arg(1)
 def get_error(space, name):
     return space.fromcache(SocketAPI).get_exception(name)
 

From commits-noreply at bitbucket.org  Fri Feb 18 02:20:58 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Fri, 18 Feb 2011 02:20:58 +0100 (CET)
Subject: [pypy-svn] pypy default: Allow property.__new__ to accept arbitrary
	args.
Message-ID: <20110218012058.817FF282BE3@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r42130:e58a70d39e19
Date: 2011-02-17 20:15 -0500
http://bitbucket.org/pypy/pypy/changeset/e58a70d39e19/

Log:	Allow property.__new__ to accept arbitrary args.

diff --git a/pypy/module/__builtin__/descriptor.py b/pypy/module/__builtin__/descriptor.py
--- a/pypy/module/__builtin__/descriptor.py
+++ b/pypy/module/__builtin__/descriptor.py
@@ -7,7 +7,7 @@
 from pypy.objspace.descroperation import object_getattribute, object_setattr
 from pypy.interpreter.function import StaticMethod, ClassMethod
 from pypy.interpreter.typedef import GetSetProperty, descr_get_dict, \
-     descr_set_dict, interp_attrproperty_w
+     descr_set_dict, interp_attrproperty_w, generic_new_descr
 
 class W_Super(Wrappable):
     def __init__(self, space, w_starttype, w_objtype, w_self):
@@ -98,6 +98,9 @@
 class W_Property(Wrappable):
     _immutable_fields_ = ["w_fget", "w_fset", "w_fdel"]
 
+    def __init__(self, space):
+        pass
+
     def init(self, space, w_fget=None, w_fset=None, w_fdel=None, w_doc=None):
         self.w_fget = w_fget
         self.w_fset = w_fset
@@ -117,11 +120,6 @@
                 self.getter_doc = True
     init.unwrap_spec = ['self', ObjSpace, W_Root, W_Root, W_Root, W_Root]
 
-    def new(space, w_subtype, w_fget=None, w_fset=None, w_fdel=None, w_doc=None):
-        w_result = space.allocate_instance(W_Property, w_subtype)
-        return w_result
-    new.unwrap_spec = [ObjSpace, W_Root, W_Root, W_Root, W_Root, W_Root]
-
     def get(self, space, w_obj, w_objtype=None):
         if space.is_w(w_obj, space.w_None):
             return space.wrap(self)
@@ -185,7 +183,7 @@
     def setx(self, value): self.__x = value
     def delx(self): del self.__x
     x = property(getx, setx, delx, "I am the 'x' property.")''',
-    __new__ = interp2app(W_Property.new.im_func),
+    __new__ = generic_new_descr(W_Property),
     __init__ = interp2app(W_Property.init),
     __get__ = interp2app(W_Property.get),
     __set__ = interp2app(W_Property.set),

diff --git a/pypy/module/__builtin__/test/test_descriptor.py b/pypy/module/__builtin__/test/test_descriptor.py
--- a/pypy/module/__builtin__/test/test_descriptor.py
+++ b/pypy/module/__builtin__/test/test_descriptor.py
@@ -328,6 +328,19 @@
         X().x
         assert l
 
+        class P(property):
+            def __init__(self, awesome):
+                property.__init__(self, x)
+                self.awesome = awesome
+
+        l[:] = []
+        class X(object):
+            x = P(awesome=True)
+
+        X().x
+        assert l
+        assert X.x.awesome
+
     def test_property_decorator(self):
         class X(object):
             @property

From commits-noreply at bitbucket.org  Fri Feb 18 02:20:59 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Fri, 18 Feb 2011 02:20:59 +0100 (CET)
Subject: [pypy-svn] pypy default: Resolved merge conflicts (I hope).
Message-ID: <20110218012059.3D7E5282BE3@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r42131:906869603389
Date: 2011-02-17 20:20 -0500
http://bitbucket.org/pypy/pypy/changeset/906869603389/

Log:	Resolved merge conflicts (I hope).


diff --git a/pypy/module/math/_genmath.py b/pypy/module/math/_genmath.py
deleted file mode 100644
--- a/pypy/module/math/_genmath.py
+++ /dev/null
@@ -1,62 +0,0 @@
-# ONESHOT SCRIPT (probably can go away soon)
-# to generate the mixed module 'math' (see same directory) 
-import py
-import math
-import re
-import sys
-rex_arg = re.compile(".*\((.*)\).*")
-
-if __name__ == '__main__': 
-    print py.code.Source("""
-        import math 
-        from pypy.interpreter.gateway import ObjSpace
-
-    """)
-    names = []
-    for name, func in math.__dict__.items(): 
-        if not callable(func): 
-            continue
-        sig = func.__doc__.split('\n')[0].strip()
-        sig = sig.split('->')[0].strip()
-        m = rex_arg.match(sig) 
-        assert m
-        args = m.group(1)
-        args = ", ".join(args.split(','))
-        sig = sig.replace('(', '(space,')
-        sig = ", ".join(sig.split(','))
-        argc = len(args.split(','))
-        unwrap_spec = ['ObjSpace']
-        unwrap_spec += ['float'] * argc 
-        unwrap_spec = ", ".join(unwrap_spec)
-        doc = func.__doc__.replace('\n', '\n       ')
-        
-        print py.code.Source('''
-            def %(sig)s: 
-                """%(doc)s
-                """
-                return space.wrap(math.%(name)s(%(args)s))
-            %(name)s.unwrap_spec = [%(unwrap_spec)s]
-        ''' % locals())
-        names.append(name) 
-
-    print >>sys.stderr, py.code.Source("""
-        # Package initialisation
-        from pypy.interpreter.mixedmodule import MixedModule
-
-        class Module(MixedModule):
-            appleveldefs = {
-            }
-            interpleveldefs = {
-    """)
-        
-    for name in names: 
-        space = " " * (15-len(name))
-        print >>sys.stderr, (
-            "       %(name)r%(space)s: 'interp_math.%(name)s'," % locals())
-    print >>sys.stderr, py.code.Source("""
-        }
-    """) 
-            
-        
-        
-        

From commits-noreply at bitbucket.org  Fri Feb 18 04:44:18 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Fri, 18 Feb 2011 04:44:18 +0100 (CET)
Subject: [pypy-svn] pypy default: Reapply 3ed721ed67a1, I lost it somehow.
Message-ID: <20110218034418.075FB282BE3@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r42132:bb209126c7f5
Date: 2011-02-17 22:44 -0500
http://bitbucket.org/pypy/pypy/changeset/bb209126c7f5/

Log:	Reapply 3ed721ed67a1, I lost it somehow.

diff --git a/pypy/module/__builtin__/descriptor.py b/pypy/module/__builtin__/descriptor.py
--- a/pypy/module/__builtin__/descriptor.py
+++ b/pypy/module/__builtin__/descriptor.py
@@ -1,8 +1,7 @@
 
 from pypy.interpreter.typedef import TypeDef, GetSetProperty
-from pypy.interpreter.baseobjspace import W_Root, ObjSpace, Wrappable, \
-     Arguments
-from pypy.interpreter.gateway import interp2app
+from pypy.interpreter.baseobjspace import Wrappable
+from pypy.interpreter.gateway import interp2app, unwrap_spec
 from pypy.interpreter.error import OperationError, operationerrfmt
 from pypy.objspace.descroperation import object_getattribute, object_setattr
 from pypy.interpreter.function import StaticMethod, ClassMethod
@@ -24,8 +23,8 @@
             #     XXX write a fast path for this common case
             w_selftype = space.type(w(self))
             return space.call_function(w_selftype, self.w_starttype, w_obj)
-    get.unwrap_spec = ['self', ObjSpace, W_Root, W_Root]
 
+    @unwrap_spec(name=str)
     def getattribute(self, space, name):
         w = space.wrap
         # only use a special logic for bound super objects and not for
@@ -49,7 +48,6 @@
         # fallback to object.__getattribute__()
         return space.call_function(object_getattribute(space),
                                    w(self), w(name))
-    getattribute.unwrap_spec = ['self', ObjSpace, str]
 
 def descr_new_super(space, w_subtype, w_starttype, w_obj_or_type=None):
     if space.is_w(w_obj_or_type, space.w_None):
@@ -77,7 +75,6 @@
     w_result = space.allocate_instance(W_Super, w_subtype)
     W_Super.__init__(w_result, space, w_starttype, w_type, w_obj_or_type)
     return w_result
-descr_new_super.unwrap_spec = [ObjSpace, W_Root, W_Root, W_Root]
 
 W_Super.typedef = TypeDef(
     'super',
@@ -118,7 +115,6 @@
                     space.setattr(space.wrap(self), space.wrap("__doc__"),
                                   w_getter_doc)
                 self.getter_doc = True
-    init.unwrap_spec = ['self', ObjSpace, W_Root, W_Root, W_Root, W_Root]
 
     def get(self, space, w_obj, w_objtype=None):
         if space.is_w(w_obj, space.w_None):
@@ -127,7 +123,6 @@
             raise OperationError(space.w_AttributeError, space.wrap(
                 "unreadable attribute"))
         return space.call_function(self.w_fget, w_obj)
-    get.unwrap_spec = ['self', ObjSpace, W_Root, W_Root]
 
     def set(self, space, w_obj, w_value):
         if space.is_w(self.w_fset, space.w_None):
@@ -135,7 +130,6 @@
                 "can't set attribute"))
         space.call_function(self.w_fset, w_obj, w_value)
         return space.w_None
-    set.unwrap_spec = ['self', ObjSpace, W_Root, W_Root]
 
     def delete(self, space, w_obj):
         if space.is_w(self.w_fdel, space.w_None):
@@ -143,19 +137,15 @@
                 "can't delete attribute"))
         space.call_function(self.w_fdel, w_obj)
         return space.w_None
-    delete.unwrap_spec = ['self', ObjSpace, W_Root]
 
     def getter(self, space, w_getter):
         return self._copy(space, w_getter=w_getter)
-    getter.unwrap_spec = ['self', ObjSpace, W_Root]
 
     def setter(self, space, w_setter):
         return self._copy(space, w_setter=w_setter)
-    setter.unwrap_spec = ['self', ObjSpace, W_Root]
 
     def deleter(self, space, w_deleter):
         return self._copy(space, w_deleter=w_deleter)
-    deleter.unwrap_spec = ['self', ObjSpace, W_Root]
 
     def _copy(self, space, w_getter=None, w_setter=None, w_deleter=None):
         if w_getter is None:

From commits-noreply at bitbucket.org  Fri Feb 18 09:04:08 2011
From: commits-noreply at bitbucket.org (ademan)
Date: Fri, 18 Feb 2011 09:04:08 +0100 (CET)
Subject: [pypy-svn] pypy default: Added support for Py_UNICODE_COPY and test.
Message-ID: <20110218080408.A27A12A202D@codespeak.net>

Author: Daniel Roberts <Ademan555 at gmail.com>
Branch: 
Changeset: r42133:fb1b5246077b
Date: 2011-02-18 00:03 -0800
http://bitbucket.org/pypy/pypy/changeset/fb1b5246077b/

Log:	Added support for Py_UNICODE_COPY and test.

diff --git a/pypy/module/cpyext/test/test_unicodeobject.py b/pypy/module/cpyext/test/test_unicodeobject.py
--- a/pypy/module/cpyext/test/test_unicodeobject.py
+++ b/pypy/module/cpyext/test/test_unicodeobject.py
@@ -225,3 +225,23 @@
 
     def test_compare(self, space, api):
         assert api.PyUnicode_Compare(space.wrap('a'), space.wrap('b')) == -1
+
+    def test_copy(self, space, api):
+        w_x = space.wrap(u"abcd\u0660")
+        target_chunk, _ = rffi.alloc_unicodebuffer(space.int_w(space.len(w_x)))
+        #lltype.malloc(Py_UNICODE, space.int_w(space.len(w_x)), flavor='raw')
+
+        x_chunk = api.PyUnicode_AS_UNICODE(w_x)
+        api.Py_UNICODE_COPY(target_chunk, x_chunk, 4)
+        w_y = api.PyUnicode_FromUnicode(target_chunk, 4)
+
+        assert space.eq_w(w_y, space.wrap(u"abcd"))
+
+        size = api.PyUnicode_GET_SIZE(w_x)
+        api.Py_UNICODE_COPY(target_chunk, x_chunk, size)
+        w_y = api.PyUnicode_FromUnicode(target_chunk, size)
+
+        assert space.eq_w(w_y, w_x)
+
+        lltype.free(target_chunk, flavor='raw')
+

diff --git a/pypy/module/cpyext/unicodeobject.py b/pypy/module/cpyext/unicodeobject.py
--- a/pypy/module/cpyext/unicodeobject.py
+++ b/pypy/module/cpyext/unicodeobject.py
@@ -1,5 +1,6 @@
 from pypy.interpreter.error import OperationError
 from pypy.rpython.lltypesystem import rffi, lltype
+from pypy.rpython.lltypesystem import llmemory
 from pypy.module.unicodedata import unicodedb
 from pypy.module.cpyext.api import (
     CANNOT_FAIL, Py_ssize_t, build_type_checkers, cpython_api,
@@ -439,3 +440,10 @@
     """Compare two strings and return -1, 0, 1 for less than, equal, and greater
     than, respectively."""
     return space.int_w(space.cmp(w_left, w_right))
+
+ at cpython_api([rffi.CWCHARP, rffi.CWCHARP, Py_ssize_t], lltype.Void)
+def Py_UNICODE_COPY(space, target, source, length):
+    """Roughly equivalent to memcpy() only the base size is Py_UNICODE
+    copies sizeof(Py_UNICODE) * length bytes from source to target"""
+    for i in range(0, length):
+        target[i] = source[i]

From commits-noreply at bitbucket.org  Fri Feb 18 09:39:14 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Fri, 18 Feb 2011 09:39:14 +0100 (CET)
Subject: [pypy-svn] pypy arm-backend-2: rewrite the resoperation dispatch
	loop a bit and	call regalloc._check_invariants on each iteration
Message-ID: <20110218083914.552AD282BE3@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backend-2
Changeset: r42134:51665f35de78
Date: 2011-02-17 20:54 +0100
http://bitbucket.org/pypy/pypy/changeset/51665f35de78/

Log:	rewrite the resoperation dispatch loop a bit and call
	regalloc._check_invariants on each iteration

diff --git a/pypy/jit/backend/arm/assembler.py b/pypy/jit/backend/arm/assembler.py
--- a/pypy/jit/backend/arm/assembler.py
+++ b/pypy/jit/backend/arm/assembler.py
@@ -476,26 +476,26 @@
 
     def _walk_operations(self, operations, regalloc):
         fcond=c.AL
-        i = 0
-        while i < len(operations):
-            regalloc.position = i
+        while regalloc.position < len(operations) - 1:
+            regalloc.next_instruction()
+            i = regalloc.position
             op = operations[i]
             opnum = op.getopnum()
             if op.has_no_side_effect() and op.result not in regalloc.longevity:
-                i += 1
                 regalloc.possibly_free_vars_for_op(op)
-                continue
             elif self.can_merge_with_next_guard(op, i, operations):
                 arglocs = regalloc.operations_with_guard[opnum](regalloc, op,
                                         operations[i+1], fcond)
                 fcond = self.operations_with_guard[opnum](self, op,
                                         operations[i+1], arglocs, regalloc, fcond)
-                i += 1
-                regalloc.position = i
+                regalloc.next_instruction()
             else:
                 arglocs = regalloc.operations[opnum](regalloc, op, fcond)
                 fcond = self.operations[opnum](self, op, arglocs, regalloc, fcond)
-            i += 1
+            if op.result:
+                regalloc.possibly_free_var(op.result)
+            regalloc.possibly_free_vars_for_op(op)
+            regalloc._check_invariants()
 
     def can_merge_with_next_guard(self, op, i, operations):
         if op.getopnum() == rop.CALL_MAY_FORCE or op.getopnum() == rop.CALL_ASSEMBLER:

From commits-noreply at bitbucket.org  Fri Feb 18 09:39:15 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Fri, 18 Feb 2011 09:39:15 +0100 (CET)
Subject: [pypy-svn] pypy arm-backend-2: Improve freeing of boxes and also
	free the	failargs of guards
Message-ID: <20110218083915.2132F282BE3@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backend-2
Changeset: r42135:f10a4fc7e222
Date: 2011-02-17 21:00 +0100
http://bitbucket.org/pypy/pypy/changeset/f10a4fc7e222/

Log:	Improve freeing of boxes and also free the failargs of guards

diff --git a/pypy/jit/backend/arm/opassembler.py b/pypy/jit/backend/arm/opassembler.py
--- a/pypy/jit/backend/arm/opassembler.py
+++ b/pypy/jit/backend/arm/opassembler.py
@@ -297,11 +297,10 @@
             self._adjust_sp(-n, fcond=fcond)
 
         # restore the argumets stored on the stack
-        if spill_all_regs:
+        if result is not None:
             regalloc.after_call(result)
-        else:
+        if not spill_all_regs:
             if result is not None:
-                regalloc.after_call(result)
                 self.mc.POP([reg.value for reg in r.caller_resp][1:])
             else:
                 self.mc.POP([reg.value for reg in r.caller_resp])
@@ -678,7 +677,7 @@
         assert resloc is r.r0
         self.mc.gen_load_int(r.ip.value, value)
         self.mc.CMP_rr(resloc.value, r.ip.value)
-
+        regalloc.possibly_free_var(resbox)
 
         fast_jmp_pos = self.mc.currpos()
         self.mc.NOP()
@@ -695,7 +694,8 @@
         self.mov_loc_loc(arglocs[1], r.r1)
         self.mc.BL(asm_helper_adr)
         self.mc.POP([reg.value for reg in r.caller_resp][1:])
-        regalloc.after_call(op.result)
+        if op.result:
+            regalloc.after_call(op.result)
         # jump to merge point
         jmp_pos = self.mc.currpos()
         #jmp_location = self.mc.curraddr()
@@ -715,10 +715,11 @@
             fielddescr = jd.vable_token_descr
             assert isinstance(fielddescr, BaseFieldDescr)
             ofs = fielddescr.offset
+            resloc = regalloc.force_allocate_reg(resbox)
             self.mov_loc_loc(arglocs[1], r.ip, cond=c.MI)
             self.mc.MOV_ri(resloc.value, 0, cond=c.MI)
             self.mc.STR_ri(resloc.value, r.ip.value, ofs*WORD, cond=c.MI)
-        regalloc.possibly_free_var(resbox)
+            regalloc.possibly_free_var(resbox)
 
         if op.result is not None:
             # load the return value from fail_boxes_xxx[0]

diff --git a/pypy/jit/backend/arm/regalloc.py b/pypy/jit/backend/arm/regalloc.py
--- a/pypy/jit/backend/arm/regalloc.py
+++ b/pypy/jit/backend/arm/regalloc.py
@@ -211,6 +211,7 @@
         args = self._prepare_guard(guard, args)
         self.possibly_free_vars(boxes)
         self.possibly_free_var(op.result)
+        self.possibly_free_vars(guard.getfailargs())
         return args
 
 
@@ -277,6 +278,7 @@
             arg = op.getarg(i)
             if arg:
                 args.append(self.loc(arg))
+                self.possibly_free_var(arg)
             else:
                 args.append(None)
         return args
@@ -285,6 +287,7 @@
         l0, box = self._ensure_value_is_boxed(op.getarg(0))
         args = self._prepare_guard(op, [l0])
         self.possibly_free_var(box)
+        self.possibly_free_vars(op.getfailargs())
         return args
 
     prepare_op_guard_false = prepare_op_guard_true
@@ -305,10 +308,13 @@
         assert op.result is None
         arglocs = self._prepare_guard(op, [l0, l1])
         self.possibly_free_vars(boxes)
+        self.possibly_free_vars(op.getfailargs())
         return arglocs
 
     def prepare_op_guard_no_overflow(self, op, fcond):
-        return  self._prepare_guard(op)
+        locs = self._prepare_guard(op)
+        self.possibly_free_vars(op.getfailargs())
+        return locs
 
     prepare_op_guard_overflow = prepare_op_guard_no_overflow
 
@@ -329,6 +335,7 @@
         pos_exception = imm(self.cpu.pos_exception())
         arglocs = self._prepare_guard(op, [loc, loc1, resloc, pos_exc_value, pos_exception])
         self.possibly_free_vars(boxes)
+        self.possibly_free_vars(op.getfailargs())
         return arglocs
 
     def prepare_op_guard_no_exception(self, op, fcond):
@@ -336,6 +343,7 @@
                     ConstInt(self.cpu.pos_exception()))
         arglocs = self._prepare_guard(op, [loc])
         self.possibly_free_var(box)
+        self.possibly_free_vars(op.getfailargs())
         return arglocs
 
     def prepare_op_guard_class(self, op, fcond):
@@ -361,6 +369,7 @@
         boxes.append(offset_box)
         arglocs = self._prepare_guard(op, [x, y, offset_loc])
         self.possibly_free_vars(boxes)
+        self.possibly_free_vars(op.getfailargs())
 
         return arglocs
 
@@ -368,7 +377,8 @@
     def prepare_op_jump(self, op, fcond):
         descr = op.getdescr()
         assert isinstance(descr, LoopToken)
-        return [self.loc(op.getarg(i)) for i in range(op.numargs())]
+        locs = [self.loc(op.getarg(i)) for i in range(op.numargs())]
+        return locs
 
 
     def prepare_op_setfield_gc(self, op, fcond):
@@ -577,7 +587,6 @@
         else:
             argloc, box = self._ensure_value_is_boxed(arg)
             self.possibly_free_var(box)
-        self.possibly_free_vars_for_op(op)
 
         resloc = self.force_allocate_reg(op.result)
         self.possibly_free_var(op.result)
@@ -672,7 +681,9 @@
         # force all reg values to be spilled when calling
         self.assembler.emit_op_call(op, args, self, fcond, spill_all_regs=True)
 
-        return self._prepare_guard(guard_op)
+        locs = self._prepare_guard(guard_op)
+        self.possibly_free_vars(guard_op.getfailargs())
+        return locs
 
     def prepare_guard_call_assembler(self, op, guard_op, fcond):
         descr = op.getdescr()
@@ -686,6 +697,7 @@
             vable = self.frame_manager.loc(op.getarg(vable_index))
         else:
             vable = imm(0)
+        self.possibly_free_vars(guard_op.getfailargs())
         return [imm(size), vable]
 
     def _prepare_args_for_new_op(self, new_args):

From commits-noreply at bitbucket.org  Fri Feb 18 09:39:16 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Fri, 18 Feb 2011 09:39:16 +0100 (CET)
Subject: [pypy-svn] pypy arm-backend-2: disable test if the backend does not
	support	floats
Message-ID: <20110218083916.2F040282C1A@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backend-2
Changeset: r42136:c1f3a65d5139
Date: 2011-02-17 21:05 +0100
http://bitbucket.org/pypy/pypy/changeset/c1f3a65d5139/

Log:	disable test if the backend does not support floats

diff --git a/pypy/jit/backend/test/runner_test.py b/pypy/jit/backend/test/runner_test.py
--- a/pypy/jit/backend/test/runner_test.py
+++ b/pypy/jit/backend/test/runner_test.py
@@ -559,6 +559,8 @@
         assert res.value == ord('a')
 
     def test_call_with_const_floats(self):
+        if not self.cpu.supports_floats:
+            py.test.skip("requires floats")
         def func(f1, f2):
             return f1 + f2
 

From commits-noreply at bitbucket.org  Fri Feb 18 11:49:04 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri, 18 Feb 2011 11:49:04 +0100 (CET)
Subject: [pypy-svn] pypy shorter-float-repr: Close merged branch
Message-ID: <20110218104904.BBA3D282BE3@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: shorter-float-repr
Changeset: r42137:14fcefc8d8e4
Date: 2011-02-18 09:55 +0100
http://bitbucket.org/pypy/pypy/changeset/14fcefc8d8e4/

Log:	Close merged branch


From commits-noreply at bitbucket.org  Fri Feb 18 11:49:07 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri, 18 Feb 2011 11:49:07 +0100 (CET)
Subject: [pypy-svn] pypy interplevel-exception-classes: Merge default
Message-ID: <20110218104907.43D18282C1A@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: interplevel-exception-classes
Changeset: r42138:631267330434
Date: 2011-02-18 10:40 +0100
http://bitbucket.org/pypy/pypy/changeset/631267330434/

Log:	Merge default

diff --git a/pypy/module/_socket/interp_socket.py b/pypy/module/_socket/interp_socket.py
--- a/pypy/module/_socket/interp_socket.py
+++ b/pypy/module/_socket/interp_socket.py
@@ -1,8 +1,7 @@
 from pypy.interpreter.baseobjspace import Wrappable
 from pypy.interpreter.typedef import TypeDef, make_weakref_descr,\
      interp_attrproperty
-from pypy.interpreter.gateway import ObjSpace, W_Root, NoneNotWrapped
-from pypy.interpreter.gateway import interp2app
+from pypy.interpreter.gateway import NoneNotWrapped, interp2app, unwrap_spec
 from pypy.rlib.rarithmetic import intmask
 from pypy.rlib import rsocket
 from pypy.rlib.rsocket import RSocket, AF_INET, SOCK_STREAM
@@ -36,7 +35,6 @@
                                    addr.as_object(sock.fd, space)])
         except SocketError, e:
             raise converted_error(space, e)
-    accept_w.unwrap_spec = ['self', ObjSpace]
 
     def bind_w(self, space, w_addr):
         """bind(address)
@@ -49,7 +47,6 @@
             self.bind(self.addr_from_object(space, w_addr))
         except SocketError, e:
             raise converted_error(space, e)
-    bind_w.unwrap_spec = ['self', ObjSpace, W_Root]
 
     def close_w(self, space):
         """close()
@@ -60,7 +57,6 @@
             self.close()
         except SocketError, e:
             raise converted_error(space, e)
-    close_w.unwrap_spec = ['self', ObjSpace]
 
     def connect_w(self, space, w_addr):
         """connect(address)
@@ -75,7 +71,6 @@
         except TypeError, e:
             raise OperationError(space.w_TypeError,
                                  space.wrap(str(e)))
-    connect_w.unwrap_spec = ['self', ObjSpace, W_Root]
 
     def connect_ex_w(self, space, w_addr):
         """connect_ex(address) -> errno
@@ -89,14 +84,12 @@
             raise converted_error(space, e)
         error = self.connect_ex(addr)
         return space.wrap(error)
-    connect_ex_w.unwrap_spec = ['self', ObjSpace, W_Root]
 
     def dup_w(self, space):
         try:
             return self.dup(W_RSocket)
         except SocketError, e:
             raise converted_error(space, e)
-    dup_w.unwrap_spec = ['self', ObjSpace]
     
     def fileno_w(self, space):
         """fileno() -> integer
@@ -104,7 +97,6 @@
         Return the integer file descriptor of the socket.
         """
         return space.wrap(intmask(self.fd))
-    fileno_w.unwrap_spec = ['self', ObjSpace]
 
     def getpeername_w(self, space):
         """getpeername() -> address info
@@ -117,7 +109,6 @@
             return addr.as_object(self.fd, space)
         except SocketError, e:
             raise converted_error(space, e)
-    getpeername_w.unwrap_spec = ['self', ObjSpace]
 
     def getsockname_w(self, space):
         """getsockname() -> address info
@@ -130,8 +121,8 @@
             return addr.as_object(self.fd, space)
         except SocketError, e:
             raise converted_error(space, e)
-    getsockname_w.unwrap_spec = ['self', ObjSpace]
 
+    @unwrap_spec(level=int, optname=int)
     def getsockopt_w(self, space, level, optname, w_buflen=NoneNotWrapped):
         """getsockopt(level, option[, buffersize]) -> value
 
@@ -146,7 +137,6 @@
                 raise converted_error(space, e)
         buflen = space.int_w(w_buflen)
         return space.wrap(self.getsockopt(level, optname, buflen))
-    getsockopt_w.unwrap_spec = ['self', ObjSpace, int, int, W_Root]
 
     def gettimeout_w(self, space):
         """gettimeout() -> timeout
@@ -158,8 +148,8 @@
         if timeout < 0.0:
             return space.w_None
         return space.wrap(timeout)
-    gettimeout_w.unwrap_spec = ['self', ObjSpace]
-    
+
+    @unwrap_spec(backlog=int)
     def listen_w(self, space, backlog):
         """listen(backlog)
 
@@ -171,7 +161,6 @@
             self.listen(backlog)
         except SocketError, e:
             raise converted_error(space, e)
-    listen_w.unwrap_spec = ['self', ObjSpace, int]
 
     def makefile_w(self, space, w_mode="r", w_buffsize=-1):
         """makefile([mode[, buffersize]]) -> file object
@@ -180,10 +169,8 @@
         The mode and buffersize arguments are as for the built-in open() function.
         """
         return app_makefile(space, self, w_mode, w_buffsize)
-    makefile_w.unwrap_spec = ['self', ObjSpace, W_Root, W_Root]
-        
-        
-                   
+
+    @unwrap_spec(buffersize='nonnegint', flags=int)
     def recv_w(self, space, buffersize, flags=0):
         """recv(buffersize[, flags]) -> data
 
@@ -197,8 +184,8 @@
         except SocketError, e:
             raise converted_error(space, e)
         return space.wrap(data)
-    recv_w.unwrap_spec = ['self', ObjSpace, 'nonnegint', int]
 
+    @unwrap_spec(buffersize='nonnegint', flags=int)
     def recvfrom_w(self, space, buffersize, flags=0):
         """recvfrom(buffersize[, flags]) -> (data, address info)
 
@@ -213,8 +200,8 @@
             return space.newtuple([space.wrap(data), w_addr])
         except SocketError, e:
             raise converted_error(space, e)
-    recvfrom_w.unwrap_spec = ['self', ObjSpace, 'nonnegint', int]
 
+    @unwrap_spec(data='bufferstr', flags=int)
     def send_w(self, space, data, flags=0):
         """send(data[, flags]) -> count
 
@@ -227,8 +214,8 @@
         except SocketError, e:
             raise converted_error(space, e)
         return space.wrap(count)
-    send_w.unwrap_spec = ['self', ObjSpace, 'bufferstr', int]
 
+    @unwrap_spec(data='bufferstr', flags=int)
     def sendall_w(self, space, data, flags=0):
         """sendall(data[, flags])
 
@@ -241,8 +228,8 @@
             count = self.sendall(data, flags, SignalChecker(space))
         except SocketError, e:
             raise converted_error(space, e)
-    sendall_w.unwrap_spec = ['self', ObjSpace, 'bufferstr', int]
 
+    @unwrap_spec(data='bufferstr')
     def sendto_w(self, space, data, w_param2, w_param3=NoneNotWrapped):
         """sendto(data[, flags], address) -> count
 
@@ -263,18 +250,18 @@
         except SocketError, e:
             raise converted_error(space, e)
         return space.wrap(count)
-    sendto_w.unwrap_spec = ['self', ObjSpace, 'bufferstr', W_Root, W_Root]
 
-    def setblocking_w(self, space, flag):
+    @unwrap_spec(flag=bool)
+    def setblocking_w(self, flag):
         """setblocking(flag)
 
         Set the socket to blocking (flag is true) or non-blocking (false).
         setblocking(True) is equivalent to settimeout(None);
         setblocking(False) is equivalent to settimeout(0.0).
         """
-        self.setblocking(bool(flag))
-    setblocking_w.unwrap_spec = ['self', ObjSpace, int]
+        self.setblocking(flag)
 
+    @unwrap_spec(level=int, optname=int)
     def setsockopt_w(self, space, level, optname, w_optval):
         """setsockopt(level, option, value)
 
@@ -294,9 +281,7 @@
             self.setsockopt_int(level, optname, optval)
         except SocketError, e:
             raise converted_error(space, e)
-            
-    setsockopt_w.unwrap_spec = ['self', ObjSpace, int, int, W_Root]
-    
+
     def settimeout_w(self, space, w_timeout):
         """settimeout(timeout)
 
@@ -313,8 +298,8 @@
                 raise OperationError(space.w_ValueError,
                                      space.wrap('Timeout value out of range'))
         self.settimeout(timeout)
-    settimeout_w.unwrap_spec = ['self', ObjSpace, W_Root]
 
+    @unwrap_spec(nbytes=int, flags=int)
     def recv_into_w(self, space, w_buffer, nbytes=0, flags=0):
         rwbuffer = space.rwbuffer_w(w_buffer)
         lgt = rwbuffer.getlength()
@@ -325,8 +310,7 @@
         except SocketError, e:
             raise converted_error(space, e)
 
-    recv_into_w.unwrap_spec = ['self', ObjSpace, W_Root, int, int]
-
+    @unwrap_spec(nbytes=int, flags=int)
     def recvfrom_into_w(self, space, w_buffer, nbytes=0, flags=0):
         rwbuffer = space.rwbuffer_w(w_buffer)
         lgt = rwbuffer.getlength()
@@ -341,8 +325,8 @@
             return space.newtuple([space.wrap(readlgt), w_addr])
         except SocketError, e:
             raise converted_error(space, e)        
-    recvfrom_into_w.unwrap_spec = ['self', ObjSpace, W_Root, int, int]
 
+    @unwrap_spec(cmd=int)
     def ioctl_w(self, space, cmd, w_option):
         from pypy.rpython.lltypesystem import rffi, lltype
         from pypy.rlib import rwin32
@@ -382,8 +366,8 @@
             return space.wrap(recv_ptr[0])
         finally:
             lltype.free(recv_ptr, flavor='raw')
-    ioctl_w.unwrap_spec = ['self', ObjSpace, int, W_Root]
 
+    @unwrap_spec(how=int)
     def shutdown_w(self, space, how):
         """shutdown(flag)
 
@@ -395,7 +379,6 @@
             self.shutdown(how)
         except SocketError, e:
             raise converted_error(space, e)
-    shutdown_w.unwrap_spec = ['self', ObjSpace, int]
 
     #------------------------------------------------------------
     # Support functions for socket._socketobject
@@ -407,7 +390,6 @@
         Intended only to be used by socket._socketobject
         """
         self.usecount += 1
-    _reuse_w.unwrap_spec = ['self']
 
     def _drop_w(self, space):
         """_drop()
@@ -420,7 +402,6 @@
         if self.usecount > 0:
             return
         self.close_w(space)
-    _drop_w.unwrap_spec = ['self', ObjSpace]
 
 app_makefile = gateway.applevel(r'''
 def makefile(self, mode="r", buffersize=-1):
@@ -434,6 +415,7 @@
     return os.fdopen(newfd, mode, buffersize)
 ''', filename =__file__).interphook('makefile')
 
+ at unwrap_spec(family=int, type=int, proto=int)
 def newsocket(space, w_subtype, family=AF_INET,
               type=SOCK_STREAM, proto=0):
     # XXX If we want to support subclassing the socket type we will need
@@ -446,8 +428,7 @@
     except SocketError, e:
         raise converted_error(space, e)
     return space.wrap(sock)
-descr_socket_new = interp2app(newsocket,
-                               unwrap_spec=[ObjSpace, W_Root, int, int, int])
+descr_socket_new = interp2app(newsocket)
 
 # ____________________________________________________________
 # Error handling
@@ -503,9 +484,7 @@
 socketmethods = {}
 for methodname in socketmethodnames:
     method = getattr(W_RSocket, methodname + '_w')
-    assert hasattr(method,'unwrap_spec'), methodname
-    assert method.im_func.func_code.co_argcount == len(method.unwrap_spec), methodname
-    socketmethods[methodname] = interp2app(method, unwrap_spec=method.unwrap_spec)
+    socketmethods[methodname] = interp2app(method)
 
 W_RSocket.typedef = TypeDef("_socket.socket",
     __doc__ = """\

diff --git a/pypy/module/math/_genmath.py b/pypy/module/math/_genmath.py
deleted file mode 100644
--- a/pypy/module/math/_genmath.py
+++ /dev/null
@@ -1,62 +0,0 @@
-# ONESHOT SCRIPT (probably can go away soon)
-# to generate the mixed module 'math' (see same directory) 
-import py
-import math
-import re
-import sys
-rex_arg = re.compile(".*\((.*)\).*")
-
-if __name__ == '__main__': 
-    print py.code.Source("""
-        import math 
-        from pypy.interpreter.gateway import ObjSpace
-
-    """)
-    names = []
-    for name, func in math.__dict__.items(): 
-        if not callable(func): 
-            continue
-        sig = func.__doc__.split('\n')[0].strip()
-        sig = sig.split('->')[0].strip()
-        m = rex_arg.match(sig) 
-        assert m
-        args = m.group(1)
-        args = ", ".join(args.split(','))
-        sig = sig.replace('(', '(space,')
-        sig = ", ".join(sig.split(','))
-        argc = len(args.split(','))
-        unwrap_spec = ['ObjSpace']
-        unwrap_spec += ['float'] * argc 
-        unwrap_spec = ", ".join(unwrap_spec)
-        doc = func.__doc__.replace('\n', '\n       ')
-        
-        print py.code.Source('''
-            def %(sig)s: 
-                """%(doc)s
-                """
-                return space.wrap(math.%(name)s(%(args)s))
-            %(name)s.unwrap_spec = [%(unwrap_spec)s]
-        ''' % locals())
-        names.append(name) 
-
-    print >>sys.stderr, py.code.Source("""
-        # Package initialisation
-        from pypy.interpreter.mixedmodule import MixedModule
-
-        class Module(MixedModule):
-            appleveldefs = {
-            }
-            interpleveldefs = {
-    """)
-        
-    for name in names: 
-        space = " " * (15-len(name))
-        print >>sys.stderr, (
-            "       %(name)r%(space)s: 'interp_math.%(name)s'," % locals())
-    print >>sys.stderr, py.code.Source("""
-        }
-    """) 
-            
-        
-        
-        

diff --git a/pypy/module/_ssl/interp_ssl.py b/pypy/module/_ssl/interp_ssl.py
--- a/pypy/module/_ssl/interp_ssl.py
+++ b/pypy/module/_ssl/interp_ssl.py
@@ -2,7 +2,7 @@
 from pypy.interpreter.error import OperationError, new_exception_class
 from pypy.interpreter.baseobjspace import W_Root, ObjSpace, Wrappable
 from pypy.interpreter.typedef import TypeDef
-from pypy.interpreter.gateway import interp2app
+from pypy.interpreter.gateway import interp2app, unwrap_spec
 
 from pypy.rlib import rpoll, rsocket
 from pypy.rlib.ropenssl import *
@@ -78,6 +78,7 @@
 
 if HAVE_OPENSSL_RAND:
     # helper routines for seeding the SSL PRNG
+    @unwrap_spec(string=str, entropy=float)
     def RAND_add(space, string, entropy):
         """RAND_add(string, entropy)
 
@@ -90,7 +91,6 @@
             libssl_RAND_add(buf, len(string), entropy)
         finally:
             rffi.free_charp(buf)
-    RAND_add.unwrap_spec = [ObjSpace, str, float]
 
     def RAND_status(space):
         """RAND_status() -> 0 or 1
@@ -101,8 +101,8 @@
 
         res = libssl_RAND_status()
         return space.wrap(res)
-    RAND_status.unwrap_spec = [ObjSpace]
 
+    @unwrap_spec(path=str)
     def RAND_egd(space, path):
         """RAND_egd(path) -> bytes
 
@@ -120,7 +120,6 @@
             msg += " enough data to seed the PRNG"
             raise ssl_error(space, msg)
         return space.wrap(bytes)
-    RAND_egd.unwrap_spec = [ObjSpace, str]
 
 class SSLObject(Wrappable):
     def __init__(self, space):
@@ -137,11 +136,9 @@
     
     def server(self):
         return self.space.wrap(rffi.charp2str(self._server))
-    server.unwrap_spec = ['self']
     
     def issuer(self):
         return self.space.wrap(rffi.charp2str(self._issuer))
-    issuer.unwrap_spec = ['self']
     
     def __del__(self):
         if self.peer_cert:
@@ -153,6 +150,7 @@
         lltype.free(self._server, flavor='raw')
         lltype.free(self._issuer, flavor='raw')
     
+    @unwrap_spec(data='bufferstr')
     def write(self, data):
         """write(s) -> len
 
@@ -201,8 +199,8 @@
             return self.space.wrap(num_bytes)
         else:
             raise _ssl_seterror(self.space, self, num_bytes)
-    write.unwrap_spec = ['self', 'bufferstr']
-    
+
+    @unwrap_spec(num_bytes=int)
     def read(self, num_bytes=1024):
         """read([len]) -> string
 
@@ -256,7 +254,6 @@
         result = rffi.str_from_buffer(raw_buf, gc_buf, num_bytes, count)
         rffi.keep_buffer_alive_until_here(raw_buf, gc_buf)
         return self.space.wrap(result)
-    read.unwrap_spec = ['self', int]
 
     def _refresh_nonblocking(self, space):
         # just in case the blocking state of the socket has been changed
@@ -375,17 +372,12 @@
 
 
 SSLObject.typedef = TypeDef("SSLObject",
-    server = interp2app(SSLObject.server,
-        unwrap_spec=SSLObject.server.unwrap_spec),
-    issuer = interp2app(SSLObject.issuer,
-        unwrap_spec=SSLObject.issuer.unwrap_spec),
-    write = interp2app(SSLObject.write,
-        unwrap_spec=SSLObject.write.unwrap_spec),
-    read = interp2app(SSLObject.read, unwrap_spec=SSLObject.read.unwrap_spec),
-    do_handshake=interp2app(SSLObject.do_handshake,
-                            unwrap_spec=['self', ObjSpace]),
-    shutdown=interp2app(SSLObject.shutdown,
-                        unwrap_spec=['self', ObjSpace]),
+    server = interp2app(SSLObject.server),
+    issuer = interp2app(SSLObject.issuer),
+    write = interp2app(SSLObject.write),
+    read = interp2app(SSLObject.read),
+    do_handshake=interp2app(SSLObject.do_handshake),
+    shutdown=interp2app(SSLObject.shutdown),
 )
 
 
@@ -452,7 +444,6 @@
 
     ss.w_socket = w_sock
     return ss
-new_sslobject.unwrap_spec = [ObjSpace, W_Root, str, str]
 
 def check_socket_and_wait_for_timeout(space, w_sock, writing):
     """If the socket has a timeout, do a select()/poll() on the socket.
@@ -553,14 +544,13 @@
     return ssl_error(space, errstr, errval)
 
 
+ at unwrap_spec(side=int, cert_mode=int, protocol=int)
 def sslwrap(space, w_socket, side, w_key_file=None, w_cert_file=None,
             cert_mode=PY_SSL_CERT_NONE, protocol=PY_SSL_VERSION_SSL23,
             w_cacerts_file=None, w_cipher=None):
     """sslwrap(socket, side, [keyfile, certfile]) -> sslobject"""
     return space.wrap(new_sslobject(
         space, w_socket, side, w_key_file, w_cert_file))
-sslwrap.unwrap_spec = [ObjSpace, W_Root, int, W_Root, W_Root,
-                       int, int, W_Root, W_Root]
 
 class Cache:
     def __init__(self, space):

From commits-noreply at bitbucket.org  Fri Feb 18 11:49:08 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri, 18 Feb 2011 11:49:08 +0100 (CET)
Subject: [pypy-svn] pypy interplevel-exception-classes: Directly access the
	exception	class defined in this module
Message-ID: <20110218104908.5048E282C1A@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: interplevel-exception-classes
Changeset: r42139:173118f7592b
Date: 2011-02-18 10:47 +0100
http://bitbucket.org/pypy/pypy/changeset/173118f7592b/

Log:	Directly access the exception class defined in this module

diff --git a/pypy/module/_ssl/interp_ssl.py b/pypy/module/_ssl/interp_ssl.py
--- a/pypy/module/_ssl/interp_ssl.py
+++ b/pypy/module/_ssl/interp_ssl.py
@@ -67,8 +67,7 @@
 constants["OPENSSL_VERSION"] = SSLEAY_VERSION
 
 def ssl_error(space, msg, errno=0):
-    w_module = space.getbuiltinmodule('_ssl')
-    w_exception_class = space.getattr(w_module, space.wrap('SSLError'))
+    w_exception_class = get_error(space)
     if errno:
         w_exception = space.call_function(w_exception_class,
                                           space.wrap(errno), space.wrap(msg))

From commits-noreply at bitbucket.org  Fri Feb 18 11:49:10 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri, 18 Feb 2011 11:49:10 +0100 (CET)
Subject: [pypy-svn] pypy interplevel-exception-classes: add
	space.new_exception_class(),
	and use it to remove applevel code in _rawffi
Message-ID: <20110218104910.3B5E62A202D@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: interplevel-exception-classes
Changeset: r42140:607197252e64
Date: 2011-02-18 10:57 +0100
http://bitbucket.org/pypy/pypy/changeset/607197252e64/

Log:	add space.new_exception_class(), and use it to remove applevel code
	in _rawffi

diff --git a/pypy/module/_rawffi/error.py b/pypy/module/_rawffi/error.py
deleted file mode 100644
--- a/pypy/module/_rawffi/error.py
+++ /dev/null
@@ -1,2 +0,0 @@
-class SegfaultException(Exception):
-    pass

diff --git a/pypy/interpreter/baseobjspace.py b/pypy/interpreter/baseobjspace.py
--- a/pypy/interpreter/baseobjspace.py
+++ b/pypy/interpreter/baseobjspace.py
@@ -3,6 +3,7 @@
 from pypy.interpreter.executioncontext import ExecutionContext, ActionFlag
 from pypy.interpreter.executioncontext import UserDelAction, FrameTraceAction
 from pypy.interpreter.error import OperationError, operationerrfmt
+from pypy.interpreter.error import new_exception_class
 from pypy.interpreter.argument import Arguments
 from pypy.interpreter.miscutils import ThreadLocals
 from pypy.tool.cache import Cache
@@ -958,6 +959,10 @@
     def exception_issubclass_w(self, w_cls1, w_cls2):
         return self.is_true(self.issubtype(w_cls1, w_cls2))
 
+    def new_exception_class(self, *args, **kwargs):
+        "NOT_RPYTHON; convenience method to create excceptions in modules"
+        return new_exception_class(self, *args, **kwargs)
+
     # end of special support code
 
     def eval(self, expression, w_globals, w_locals, hidden_applevel=False):

diff --git a/pypy/module/_socket/interp_socket.py b/pypy/module/_socket/interp_socket.py
--- a/pypy/module/_socket/interp_socket.py
+++ b/pypy/module/_socket/interp_socket.py
@@ -7,7 +7,6 @@
 from pypy.rlib.rsocket import RSocket, AF_INET, SOCK_STREAM
 from pypy.rlib.rsocket import SocketError, SocketErrorWithErrno
 from pypy.interpreter.error import OperationError, operationerrfmt
-from pypy.interpreter.error import new_exception_class
 from pypy.interpreter import gateway
 
 class SignalChecker:
@@ -435,14 +434,14 @@
 
 class SocketAPI:
     def __init__(self, space):
-        self.w_error = new_exception_class(
-            space, "_socket.error", space.w_IOError)
-        self.w_herror = new_exception_class(
-            space, "_socket.herror", self.w_error)
-        self.w_gaierror = new_exception_class(
-            space, "_socket.gaierror", self.w_error)
-        self.w_timeout = new_exception_class(
-            space, "_socket.timeout", self.w_error)
+        self.w_error = space.new_exception_class(
+            "_socket.error", space.w_IOError)
+        self.w_herror = space.new_exception_class(
+            "_socket.herror", self.w_error)
+        self.w_gaierror = space.new_exception_class(
+            "_socket.gaierror", self.w_error)
+        self.w_timeout = space.new_exception_class(
+            "_socket.timeout", self.w_error)
 
         self.errors_w = {'error': self.w_error,
                          'herror': self.w_herror,

diff --git a/pypy/module/_rawffi/__init__.py b/pypy/module/_rawffi/__init__.py
--- a/pypy/module/_rawffi/__init__.py
+++ b/pypy/module/_rawffi/__init__.py
@@ -30,6 +30,7 @@
         'get_libc'           : 'interp_rawffi.get_libc',
         'get_errno'          : 'interp_rawffi.get_errno',
         'set_errno'          : 'interp_rawffi.set_errno',
+        'SegfaultException'  : 'space.new_exception_class("_rawffi.SegfaultException")',
     }
 
     if sys.platform == 'win32':
@@ -37,7 +38,6 @@
         interpleveldefs['set_last_error'] = 'interp_rawffi.set_last_error'
 
     appleveldefs = {
-        'SegfaultException'  : 'error.SegfaultException',
     }
 
     def buildloaders(cls):

diff --git a/pypy/module/_ssl/interp_ssl.py b/pypy/module/_ssl/interp_ssl.py
--- a/pypy/module/_ssl/interp_ssl.py
+++ b/pypy/module/_ssl/interp_ssl.py
@@ -1,5 +1,5 @@
 from pypy.rpython.lltypesystem import rffi, lltype
-from pypy.interpreter.error import OperationError, new_exception_class
+from pypy.interpreter.error import OperationError
 from pypy.interpreter.baseobjspace import W_Root, ObjSpace, Wrappable
 from pypy.interpreter.typedef import TypeDef
 from pypy.interpreter.gateway import interp2app, unwrap_spec
@@ -554,8 +554,8 @@
 class Cache:
     def __init__(self, space):
         w_socketerror = interp_socket.get_error(space, "error")
-        self.w_error = new_exception_class(
-            space, "_ssl.SSLError", w_socketerror)
+        self.w_error = space.new_exception_class(
+            "_ssl.SSLError", w_socketerror)
 
 def get_error(space):
     return space.fromcache(Cache).w_error

From commits-noreply at bitbucket.org  Fri Feb 18 11:49:14 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri, 18 Feb 2011 11:49:14 +0100 (CET)
Subject: [pypy-svn] pypy interplevel-exception-classes: Kill
	parser/app_helpers.py; the exception is not used anyway :-/
Message-ID: <20110218104914.2DE6B282C18@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: interplevel-exception-classes
Changeset: r42141:0aeb7d7f94ba
Date: 2011-02-18 11:19 +0100
http://bitbucket.org/pypy/pypy/changeset/0aeb7d7f94ba/

Log:	Kill parser/app_helpers.py; the exception is not used anyway :-/

diff --git a/pypy/module/parser/__init__.py b/pypy/module/parser/__init__.py
--- a/pypy/module/parser/__init__.py
+++ b/pypy/module/parser/__init__.py
@@ -7,7 +7,6 @@
      applevel_name = 'parser'
 
      appleveldefs = {
-         'ParserError' : 'app_helpers.ParserError'
          }
 
      interpleveldefs = {
@@ -25,5 +24,6 @@
          'ast2tuple'    : 'pyparser.st2tuple',
          'ASTType'      : 'pyparser.STType',
          'compilest'    : 'pyparser.compilest',
-         'compileast'   : 'pyparser.compilest'
+         'compileast'   : 'pyparser.compilest',
+         'ParserError'  : 'space.new_exception_class("parser.ParserError")',
          }

diff --git a/pypy/module/parser/test/test_parser.py b/pypy/module/parser/test/test_parser.py
--- a/pypy/module/parser/test/test_parser.py
+++ b/pypy/module/parser/test/test_parser.py
@@ -58,3 +58,6 @@
                      self.m.compilest(self.m.suite("x = 4"))):
             assert isinstance(code, types.CodeType)
             assert code.co_filename == "<syntax-tree>"
+
+    def test_error(self):
+        assert repr(self.m.ParserError) == "<class 'parser.ParserError'>"

diff --git a/pypy/module/parser/app_helpers.py b/pypy/module/parser/app_helpers.py
deleted file mode 100644
--- a/pypy/module/parser/app_helpers.py
+++ /dev/null
@@ -1,2 +0,0 @@
-class ParserError(Exception):
-    pass

From commits-noreply at bitbucket.org  Fri Feb 18 11:49:16 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri, 18 Feb 2011 11:49:16 +0100 (CET)
Subject: [pypy-svn] pypy interplevel-exception-classes: Kill
	select/app_select.py and	rewrite the exception at interp_level
Message-ID: <20110218104916.11755282C18@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: interplevel-exception-classes
Changeset: r42142:e9e95d67f547
Date: 2011-02-18 11:19 +0100
http://bitbucket.org/pypy/pypy/changeset/e9e95d67f547/

Log:	Kill select/app_select.py and rewrite the exception at interp_level

diff --git a/pypy/module/select/__init__.py b/pypy/module/select/__init__.py
--- a/pypy/module/select/__init__.py
+++ b/pypy/module/select/__init__.py
@@ -7,12 +7,12 @@
 
 class Module(MixedModule):
     appleveldefs = {
-        'error': 'app_select.error',
     }
 
     interpleveldefs = {
         'poll'  : 'interp_select.poll',
         'select': 'interp_select.select',
+        'error' : 'space.new_exception_class("select.error")',
     }
 
     # TODO: this doesn't feel right...

diff --git a/pypy/module/select/app_select.py b/pypy/module/select/app_select.py
deleted file mode 100644
--- a/pypy/module/select/app_select.py
+++ /dev/null
@@ -1,2 +0,0 @@
-class error(Exception):
-    pass

diff --git a/pypy/module/select/interp_select.py b/pypy/module/select/interp_select.py
--- a/pypy/module/select/interp_select.py
+++ b/pypy/module/select/interp_select.py
@@ -9,6 +9,10 @@
 
 defaultevents = rpoll.POLLIN | rpoll.POLLOUT | rpoll.POLLPRI
 
+class Cache:
+    def __init__(self, space):
+        self.w_error = space.new_exception_class("select.error")
+
 def poll(space):
     """Returns a polling object, which supports registering and
 unregistering file descriptors, and then polling them for I/O events."""
@@ -55,8 +59,7 @@
         try:
             retval = rpoll.poll(self.fddict, timeout)
         except rpoll.PollError, e:
-            w_module = space.getbuiltinmodule('select')
-            w_errortype = space.getattr(w_module, space.wrap('error'))
+            w_errortype = space.fromcache(Cache).w_error
             message = e.get_msg()
             raise OperationError(w_errortype,
                                  space.newtuple([space.wrap(e.errno),
@@ -116,8 +119,7 @@
         else:
             iwtd, owtd, ewtd = rpoll.select(iwtd, owtd, ewtd, space.float_w(w_timeout))
     except rpoll.SelectError, s:
-        w_module = space.getbuiltinmodule('select')
-        w_errortype = space.getattr(w_module, space.wrap('error'))
+        w_errortype = space.fromcache(Cache).w_error
         raise OperationError(w_errortype, space.newtuple([
             space.wrap(s.errno), space.wrap(s.get_msg())]))
     

From commits-noreply at bitbucket.org  Fri Feb 18 11:49:17 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri, 18 Feb 2011 11:49:17 +0100 (CET)
Subject: [pypy-svn] pypy interplevel-exception-classes: oops
Message-ID: <20110218104917.0C082282BE3@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: interplevel-exception-classes
Changeset: r42143:0954a68b1a24
Date: 2011-02-18 11:20 +0100
http://bitbucket.org/pypy/pypy/changeset/0954a68b1a24/

Log:	oops

diff --git a/pypy/module/select/__init__.py b/pypy/module/select/__init__.py
--- a/pypy/module/select/__init__.py
+++ b/pypy/module/select/__init__.py
@@ -12,7 +12,7 @@
     interpleveldefs = {
         'poll'  : 'interp_select.poll',
         'select': 'interp_select.select',
-        'error' : 'space.new_exception_class("select.error")',
+        'error' : 'space.fromcache(interp_select.Cache).w_error'
     }
 
     # TODO: this doesn't feel right...

From commits-noreply at bitbucket.org  Fri Feb 18 11:49:19 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri, 18 Feb 2011 11:49:19 +0100 (CET)
Subject: [pypy-svn] pypy interplevel-exception-classes: interp-level
	implementation of	thread.exit()
Message-ID: <20110218104919.0C647282BE3@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: interplevel-exception-classes
Changeset: r42144:4ca9870b73ff
Date: 2011-02-18 11:33 +0100
http://bitbucket.org/pypy/pypy/changeset/4ca9870b73ff/

Log:	interp-level implementation of thread.exit()

diff --git a/pypy/module/thread/__init__.py b/pypy/module/thread/__init__.py
--- a/pypy/module/thread/__init__.py
+++ b/pypy/module/thread/__init__.py
@@ -4,8 +4,6 @@
 
 class Module(MixedModule):
     appleveldefs = {
-        'exit':                   'app_thread.exit',
-        'exit_thread':            'app_thread.exit',   # obsolete synonym
         'error':                  'app_thread.error',
     }
 
@@ -13,6 +11,8 @@
         'start_new_thread':       'os_thread.start_new_thread',
         'start_new':              'os_thread.start_new_thread', # obsolete syn.
         'get_ident':              'os_thread.get_ident',
+        'exit':                   'os_thread.exit',
+        'exit_thread':            'os_thread.exit', # obsolete synonym
         'stack_size':             'os_thread.stack_size',
         '_count':                 'os_thread._count',
         'allocate_lock':          'os_lock.allocate_lock',

diff --git a/pypy/module/thread/app_thread.py b/pypy/module/thread/app_thread.py
--- a/pypy/module/thread/app_thread.py
+++ b/pypy/module/thread/app_thread.py
@@ -1,7 +1,2 @@
 class error(Exception):
     pass
-
-def exit():
-    """This is synonymous to ``raise SystemExit''.  It will cause the current
-thread to exit silently unless the exception is caught."""
-    raise SystemExit

diff --git a/pypy/module/thread/os_thread.py b/pypy/module/thread/os_thread.py
--- a/pypy/module/thread/os_thread.py
+++ b/pypy/module/thread/os_thread.py
@@ -244,3 +244,7 @@
 In most applications `threading.enumerate()` should be used instead."""
     return space.wrap(bootstrapper.nbthreads)
 
+def exit(space):
+    """This is synonymous to ``raise SystemExit''.  It will cause the current
+thread to exit silently unless the exception is caught."""
+    raise OperationError(space.w_SystemExit, space.w_None)

diff --git a/pypy/module/thread/test/test_thread.py b/pypy/module/thread/test/test_thread.py
--- a/pypy/module/thread/test/test_thread.py
+++ b/pypy/module/thread/test/test_thread.py
@@ -132,6 +132,7 @@
             result = sys.stderr.getvalue()
             assert "ValueError" in result
             assert "hello world" in result
+            assert len(result.splitlines()) == 1
         finally:
             sys.stderr = prev
 

From commits-noreply at bitbucket.org  Fri Feb 18 11:49:23 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri, 18 Feb 2011 11:49:23 +0100 (CET)
Subject: [pypy-svn] pypy interplevel-exception-classes: rewrite thread.error
	at	interp-level and get rid of app_thread.py
Message-ID: <20110218104923.6FE3D282C24@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: interplevel-exception-classes
Changeset: r42145:5ce5fe1cf557
Date: 2011-02-18 11:48 +0100
http://bitbucket.org/pypy/pypy/changeset/5ce5fe1cf557/

Log:	rewrite thread.error at interp-level and get rid of app_thread.py

diff --git a/pypy/module/thread/__init__.py b/pypy/module/thread/__init__.py
--- a/pypy/module/thread/__init__.py
+++ b/pypy/module/thread/__init__.py
@@ -4,7 +4,6 @@
 
 class Module(MixedModule):
     appleveldefs = {
-        'error':                  'app_thread.error',
     }
 
     interpleveldefs = {
@@ -19,6 +18,7 @@
         'allocate':               'os_lock.allocate_lock',  # obsolete synonym
         'LockType':               'os_lock.getlocktype(space)',
         '_local':                 'os_local.getlocaltype(space)',
+        'error':                  'space.fromcache(error.Cache).w_error',
     }
 
     def __init__(self, space, *args):

diff --git a/pypy/module/thread/app_thread.py b/pypy/module/thread/app_thread.py
deleted file mode 100644
--- a/pypy/module/thread/app_thread.py
+++ /dev/null
@@ -1,2 +0,0 @@
-class error(Exception):
-    pass

diff --git a/pypy/module/thread/test/test_fork.py b/pypy/module/thread/test/test_fork.py
--- a/pypy/module/thread/test/test_fork.py
+++ b/pypy/module/thread/test/test_fork.py
@@ -41,6 +41,9 @@
         "Checks that a forked interpreter can start a thread"
         import os, thread, time
 
+        if not hasattr(os, 'fork'):
+            skip("No fork on this platform")
+
         # pre-allocate some locks
         thread.start_new_thread(lambda: None, ())
 

diff --git a/pypy/module/thread/error.py b/pypy/module/thread/error.py
--- a/pypy/module/thread/error.py
+++ b/pypy/module/thread/error.py
@@ -1,6 +1,9 @@
 from pypy.interpreter.error import OperationError
 
+class Cache:
+    def __init__(self, space):
+        self.w_error = space.new_exception_class("thread.error")
+
 def wrap_thread_error(space, msg):
-    w_module = space.getbuiltinmodule('thread')
-    w_error = space.getattr(w_module, space.wrap('error'))
+    w_error = space.fromcache(Cache).w_error
     return OperationError(w_error, space.wrap(msg))

diff --git a/pypy/module/thread/test/test_gil.py b/pypy/module/thread/test/test_gil.py
--- a/pypy/module/thread/test/test_gil.py
+++ b/pypy/module/thread/test/test_gil.py
@@ -22,7 +22,7 @@
         return True
     def getexecutioncontext(self):
         return FakeEC()
-    def getbuiltinmodule(self, name):
+    def fromcache(self, key):
         raise NotImplementedError
 
 

From commits-noreply at bitbucket.org  Fri Feb 18 13:12:57 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri, 18 Feb 2011 13:12:57 +0100 (CET)
Subject: [pypy-svn] pypy interplevel-exception-classes: Rewrite zlib.error
	at interp-level
Message-ID: <20110218121257.2856A282BE3@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: interplevel-exception-classes
Changeset: r42146:5b7acbb6d2ae
Date: 2011-02-18 12:54 +0100
http://bitbucket.org/pypy/pypy/changeset/5b7acbb6d2ae/

Log:	Rewrite zlib.error at interp-level

diff --git a/pypy/module/zlib/app_zlib.py b/pypy/module/zlib/app_zlib.py
deleted file mode 100644
--- a/pypy/module/zlib/app_zlib.py
+++ /dev/null
@@ -1,11 +0,0 @@
-
-"""
-Application-level definitions for the zlib module.
-
-NOT_RPYTHON
-"""
-
-class error(Exception):
-    """
-    Raised by zlib operations.
-    """

diff --git a/pypy/module/zlib/__init__.py b/pypy/module/zlib/__init__.py
--- a/pypy/module/zlib/__init__.py
+++ b/pypy/module/zlib/__init__.py
@@ -31,10 +31,10 @@
         'compress': 'interp_zlib.compress',
         'decompress': 'interp_zlib.decompress',
         '__version__': 'space.wrap("1.0")',
+        'error': 'space.fromcache(interp_zlib.Cache).w_error',
         }
 
     appleveldefs = {
-        'error': 'app_zlib.error',
         }
 
 

diff --git a/pypy/module/zlib/interp_zlib.py b/pypy/module/zlib/interp_zlib.py
--- a/pypy/module/zlib/interp_zlib.py
+++ b/pypy/module/zlib/interp_zlib.py
@@ -66,9 +66,12 @@
     return space.wrap(checksum)
 
 
+class Cache:
+    def __init__(self, space):
+        self.w_error = space.new_exception_class("zlib.error")
+
 def zlib_error(space, msg):
-    w_module = space.getbuiltinmodule('zlib')
-    w_error = space.getattr(w_module, space.wrap('error'))
+    w_error = space.fromcache(Cache).w_error
     return OperationError(w_error, space.wrap(msg))
 
 

From commits-noreply at bitbucket.org  Fri Feb 18 13:12:58 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri, 18 Feb 2011 13:12:58 +0100 (CET)
Subject: [pypy-svn] pypy interplevel-exception-classes: termios.error at
	interp-level
Message-ID: <20110218121258.3841B282BE3@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: interplevel-exception-classes
Changeset: r42147:7b9198ad3e51
Date: 2011-02-18 13:02 +0100
http://bitbucket.org/pypy/pypy/changeset/7b9198ad3e51/

Log:	termios.error at interp-level

diff --git a/pypy/module/termios/__init__.py b/pypy/module/termios/__init__.py
--- a/pypy/module/termios/__init__.py
+++ b/pypy/module/termios/__init__.py
@@ -14,7 +14,6 @@
     sys.stdin.fileno(), or a file object, such as sys.stdin itself."
 
     appleveldefs = {
-        'error'       : 'app_termios.error',
     }
     
     interpleveldefs = {
@@ -24,6 +23,7 @@
         'tcgetattr'   : 'interp_termios.tcgetattr',
         'tcsendbreak' : 'interp_termios.tcsendbreak',
         'tcsetattr'   : 'interp_termios.tcsetattr',
+        'error'       : 'space.fromcache(interp_termios.Cache).w_error',
     }
 
 import termios

diff --git a/pypy/module/termios/app_termios.py b/pypy/module/termios/app_termios.py
deleted file mode 100644
--- a/pypy/module/termios/app_termios.py
+++ /dev/null
@@ -1,3 +0,0 @@
-
-class error(Exception):
-    pass

diff --git a/pypy/module/termios/interp_termios.py b/pypy/module/termios/interp_termios.py
--- a/pypy/module/termios/interp_termios.py
+++ b/pypy/module/termios/interp_termios.py
@@ -11,13 +11,12 @@
 from pypy.rlib import rtermios
 import termios
 
-# proper semantics are to have termios.error, but since it's not documented
-# anyway, let's have it as OSError on interplevel. We need to have
-# some details what is missing in RPython modules though
+class Cache:
+    def __init__(self):
+        self.w_error = space.new_exception_class("termios.error")
 
 def convert_error(space, error):
-    w_module = space.getbuiltinmodule('termios')
-    w_exception_class = space.getattr(w_module, space.wrap('error'))
+    w_exception_class = space.fromcache(Cache).w_error
     return wrap_oserror(space, error, w_exception_class=w_exception_class)
 
 @unwrap_spec(fd=int, when=int)

From commits-noreply at bitbucket.org  Fri Feb 18 13:13:00 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri, 18 Feb 2011 13:13:00 +0100 (CET)
Subject: [pypy-svn] pypy interplevel-exception-classes: pyexpat.error at
	interp-level
Message-ID: <20110218121300.A1BCE282C1E@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: interplevel-exception-classes
Changeset: r42148:4142c25d9610
Date: 2011-02-18 13:07 +0100
http://bitbucket.org/pypy/pypy/changeset/4142c25d9610/

Log:	pyexpat.error at interp-level

diff --git a/pypy/module/pyexpat/__init__.py b/pypy/module/pyexpat/__init__.py
--- a/pypy/module/pyexpat/__init__.py
+++ b/pypy/module/pyexpat/__init__.py
@@ -6,8 +6,6 @@
     "Python wrapper for Expat parser."
 
     appleveldefs = {
-        'ExpatError' : 'app_pyexpat.ExpatError',
-        'error'      : 'app_pyexpat.ExpatError',
         }
 
     interpleveldefs = {
@@ -15,6 +13,9 @@
         'XMLParserType': 'interp_pyexpat.W_XMLParserType',
         'ErrorString':   'interp_pyexpat.ErrorString',
 
+        'ExpatError':    'space.fromcache(interp_pyexpat.Cache).w_error',
+        'error':         'space.fromcache(interp_pyexpat.Cache).w_error',
+
         '__version__':   'space.wrap("85819")',
         'EXPAT_VERSION': 'interp_pyexpat.get_expat_version(space)',
         'version_info':  'interp_pyexpat.get_expat_version_info(space)',

diff --git a/pypy/module/pyexpat/app_pyexpat.py b/pypy/module/pyexpat/app_pyexpat.py
deleted file mode 100644
--- a/pypy/module/pyexpat/app_pyexpat.py
+++ /dev/null
@@ -1,2 +0,0 @@
-class ExpatError(Exception):
-    pass

diff --git a/pypy/module/pyexpat/interp_pyexpat.py b/pypy/module/pyexpat/interp_pyexpat.py
--- a/pypy/module/pyexpat/interp_pyexpat.py
+++ b/pypy/module/pyexpat/interp_pyexpat.py
@@ -325,6 +325,10 @@
         space.wrap(XML_MINOR_VERSION),
         space.wrap(XML_MICRO_VERSION)])
 
+class Cache:
+    def __init__(self, space):
+        self.w_error = space.new_exception_class("pyexpat.ExpatError")
+
 class W_XMLParserType(Wrappable):
 
     def __init__(self, space, parser, w_intern):
@@ -590,8 +594,7 @@
         lineno = XML_GetCurrentLineNumber(self.itself)
         colno = XML_GetCurrentColumnNumber(self.itself)
         msg = "%s: line %d, column %d" % (err, lineno, colno)
-        w_module = space.getbuiltinmodule('pyexpat')
-        w_errorcls = space.getattr(w_module, space.wrap('error'))
+        w_errorcls = space.fromcache(Cache).w_error
         w_error = space.call_function(w_errorcls, space.wrap(msg))
         space.setattr(w_error, space.wrap("code"), space.wrap(code))
         space.setattr(w_error, space.wrap("offset"), space.wrap(colno))

From commits-noreply at bitbucket.org  Fri Feb 18 13:13:03 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri, 18 Feb 2011 13:13:03 +0100 (CET)
Subject: [pypy-svn] pypy interplevel-exception-classes: rewrite
	binascii.Error, binascii.Incomplete at interp-level
Message-ID: <20110218121303.2D69B2A202D@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: interplevel-exception-classes
Changeset: r42149:bb687bd5ab3e
Date: 2011-02-18 13:11 +0100
http://bitbucket.org/pypy/pypy/changeset/bb687bd5ab3e/

Log:	rewrite binascii.Error, binascii.Incomplete at interp-level

diff --git a/pypy/module/binascii/interp_binascii.py b/pypy/module/binascii/interp_binascii.py
--- a/pypy/module/binascii/interp_binascii.py
+++ b/pypy/module/binascii/interp_binascii.py
@@ -1,11 +1,14 @@
 from pypy.interpreter.error import OperationError
 
+class Cache:
+    def __init__(self, space):
+        self.w_error = space.new_exception_class("binascii.Error")
+        self.w_incomplete = space.new_exception_class("binascii.Incomplete")
+
 def raise_Error(space, msg):
-    w_module = space.getbuiltinmodule('binascii')
-    w_Error = space.getattr(w_module, space.wrap('Error'))
-    raise OperationError(w_Error, space.wrap(msg))
+    w_error = space.fromcache(Cache).w_error
+    raise OperationError(w_error, space.wrap(msg))
 
 def raise_Incomplete(space, msg):
-    w_module = space.getbuiltinmodule('binascii')
-    w_Error = space.getattr(w_module, space.wrap('Incomplete'))
-    raise OperationError(w_Error, space.wrap(msg))
+    w_error = space.fromcache(Cache).w_incomplete
+    raise OperationError(w_error, space.wrap(msg))

diff --git a/pypy/module/binascii/app_binascii.py b/pypy/module/binascii/app_binascii.py
deleted file mode 100644
--- a/pypy/module/binascii/app_binascii.py
+++ /dev/null
@@ -1,6 +0,0 @@
-
-class Error(Exception):
-    pass
-
-class Incomplete(Exception):
-    pass

diff --git a/pypy/module/binascii/__init__.py b/pypy/module/binascii/__init__.py
--- a/pypy/module/binascii/__init__.py
+++ b/pypy/module/binascii/__init__.py
@@ -12,8 +12,6 @@
     """binascii - Conversion between binary data and ASCII"""
 
     appleveldefs = {
-        'Error': 'app_binascii.Error',
-        'Incomplete': 'app_binascii.Incomplete',
         }
 
     interpleveldefs = {
@@ -33,4 +31,6 @@
         'hexlify': 'interp_hexlify.hexlify',
         'a2b_hex': 'interp_hexlify.unhexlify',
         'unhexlify': 'interp_hexlify.unhexlify',
+        'Error'     : 'space.fromcache(interp_binascii.Cache).w_error',
+        'Incomplete': 'space.fromcache(interp_binascii.Cache).w_incomplete',
         }

From commits-noreply at bitbucket.org  Fri Feb 18 13:54:31 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Fri, 18 Feb 2011 13:54:31 +0100 (CET)
Subject: [pypy-svn] pypy default: make this test file working again (more or
	less)
Message-ID: <20110218125431.A5D99282BE3@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42150:d989053c409f
Date: 2011-02-17 11:46 +0100
http://bitbucket.org/pypy/pypy/changeset/d989053c409f/

Log:	make this test file working again (more or less)

diff --git a/pypy/module/pypyjit/test/test_pypy_c_new.py b/pypy/module/pypyjit/test/test_pypy_c_new.py
--- a/pypy/module/pypyjit/test/test_pypy_c_new.py
+++ b/pypy/module/pypyjit/test/test_pypy_c_new.py
@@ -1,8 +1,6 @@
-
 import py, sys, re
-py.test.skip("in-progress?")
 import subprocess
-import disassembler
+from lib_pypy import disassembler
 from pypy.tool.udir import udir
 from pypy.tool import logparser
 
@@ -11,6 +9,13 @@
 
 class BaseTestPyPyC(object):
     def setup_class(cls):
+        from pypy.conftest import option
+        from pypy.module.pypyjit.test.test_pypy_c import has_info
+        if option.pypy_c is None:
+            py.test.skip("pass --pypy!")
+        if not has_info(option.pypy_c, 'translation.jit'):
+            py.test.skip("must give a pypy-c with the jit enabled")
+        cls.pypy_c = option.pypy_c
         cls.tmpdir = udir.join('test-pypy-jit')
         cls.tmpdir.ensure(dir=True)
 
@@ -46,7 +51,7 @@
             f.write(str(py.code.Source(func)) + "\n")
             f.write("print %s()\n" % func.func_name)
         logfile = self.filepath.new(ext='.log')
-        pipe = subprocess.Popen([sys.executable, str(self.filepath)],
+        pipe = subprocess.Popen([self.pypy_c, str(self.filepath)],
                                 stdout=subprocess.PIPE, stderr=subprocess.PIPE,
                                 env={'PYPYLOG': "jit-log-opt,jit-summary:" + str(logfile)})
         pipe.wait()
@@ -74,6 +79,7 @@
         assert len(res['name']) == 6
 
     def test_full(self):
+        py.test.skip('in-progress')
         def f():
             i = 0
             while i < 1003:

From commits-noreply at bitbucket.org  Fri Feb 18 13:54:32 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Fri, 18 Feb 2011 13:54:32 +0100 (CET)
Subject: [pypy-svn] pypy default: improve this test
Message-ID: <20110218125432.2F510282BE3@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42151:a1d0450b3cf6
Date: 2011-02-17 11:54 +0100
http://bitbucket.org/pypy/pypy/changeset/a1d0450b3cf6/

Log:	improve this test

diff --git a/pypy/module/pypyjit/test/test_pypy_c_new.py b/pypy/module/pypyjit/test/test_pypy_c_new.py
--- a/pypy/module/pypyjit/test/test_pypy_c_new.py
+++ b/pypy/module/pypyjit/test/test_pypy_c_new.py
@@ -62,6 +62,7 @@
         assert res == func()
         log = logparser.parse_log_file(str(logfile))
         parts = logparser.extract_category(log, 'jit-log-opt-')
+        import pdb;pdb.set_trace()
         log.xxx
         return Trace()
 
@@ -70,13 +71,16 @@
         def f():
             i = 0
             x = 0
-            # LOOP name
+            # LOOP my_loop
             z = x + 3
             return z
 
         res = self.parse_func(f)
         assert len(res) == 1
-        assert len(res['name']) == 6
+        my_loop = res['my_loop']
+        opcodes_names = [opcode.__class__.__name__ for opcode in my_loop]
+        assert opcodes_names == ['LOAD_CONST', 'STORE_FAST', 'LOAD_FAST',
+                                 'LOAD_CONST', 'BINARY_ADD', 'STORE_FAST']
 
     def test_full(self):
         py.test.skip('in-progress')

From commits-noreply at bitbucket.org  Fri Feb 18 13:54:32 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Fri, 18 Feb 2011 13:54:32 +0100 (CET)
Subject: [pypy-svn] pypy default: the only supported way to run these tests
	is `pypy-c	py.test ...`,
	because we need to disassemble the snippets with the very same
	pypy	used for JITting them
Message-ID: <20110218125432.A7849282BE3@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42152:60b09f3d13a5
Date: 2011-02-17 16:04 +0100
http://bitbucket.org/pypy/pypy/changeset/60b09f3d13a5/

Log:	the only supported way to run these tests is `pypy-c py.test ...`,
	because we need to disassemble the snippets with the very same pypy
	used for JITting them

diff --git a/pypy/module/pypyjit/test/test_pypy_c_new.py b/pypy/module/pypyjit/test/test_pypy_c_new.py
--- a/pypy/module/pypyjit/test/test_pypy_c_new.py
+++ b/pypy/module/pypyjit/test/test_pypy_c_new.py
@@ -9,13 +9,10 @@
 
 class BaseTestPyPyC(object):
     def setup_class(cls):
-        from pypy.conftest import option
-        from pypy.module.pypyjit.test.test_pypy_c import has_info
-        if option.pypy_c is None:
-            py.test.skip("pass --pypy!")
-        if not has_info(option.pypy_c, 'translation.jit'):
+        if '__pypy__' not in sys.builtin_module_names:
+            py.test.skip("must run this test with pypy")
+        if not sys.pypy_translation_info['translation.jit']:
             py.test.skip("must give a pypy-c with the jit enabled")
-        cls.pypy_c = option.pypy_c
         cls.tmpdir = udir.join('test-pypy-jit')
         cls.tmpdir.ensure(dir=True)
 
@@ -51,7 +48,7 @@
             f.write(str(py.code.Source(func)) + "\n")
             f.write("print %s()\n" % func.func_name)
         logfile = self.filepath.new(ext='.log')
-        pipe = subprocess.Popen([self.pypy_c, str(self.filepath)],
+        pipe = subprocess.Popen([sys.executable, str(self.filepath)],
                                 stdout=subprocess.PIPE, stderr=subprocess.PIPE,
                                 env={'PYPYLOG': "jit-log-opt,jit-summary:" + str(logfile)})
         pipe.wait()

From commits-noreply at bitbucket.org  Fri Feb 18 13:54:34 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Fri, 18 Feb 2011 13:54:34 +0100 (CET)
Subject: [pypy-svn] pypy default: refactoring: the idea is that you can mark
	some lines	in the source code,
	and then later retrieve the corresponding operations
Message-ID: <20110218125434.0C5D0282C1E@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42153:dd7a2052c216
Date: 2011-02-17 16:14 +0100
http://bitbucket.org/pypy/pypy/changeset/dd7a2052c216/

Log:	refactoring: the idea is that you can mark some lines in the source
	code, and then later retrieve the corresponding operations

diff --git a/pypy/module/pypyjit/test/test_pypy_c_new.py b/pypy/module/pypyjit/test/test_pypy_c_new.py
--- a/pypy/module/pypyjit/test/test_pypy_c_new.py
+++ b/pypy/module/pypyjit/test/test_pypy_c_new.py
@@ -28,20 +28,38 @@
         except ValueError:
             return out
 
-    def parse_func(self, func):
-        # find lines such as # LOOP <name> is in a line
-        code = disassembler.dis(func)
+    def find_chunks_range(self, func):
+        """
+        Parse the given function and return a dictionary mapping "chunk
+        names" to "line ranges".  Chunks are identified by comments with a
+        special syntax::
+
+            # the chunk "myid" corresponds to the whole line
+            print 'foo' # ID: myid
+        """
         result = {}
+        start_lineno = func.func_code.co_firstlineno
         for i, line in enumerate(py.code.Source(func)):
-            m = re.search('# LOOP (\w+)', line)
+            m = re.search('# ID: (\w+)', line)
             if m:
                 name = m.group(1)
-                result[name] = []
-                for opcode in code.opcodes:
-                    no = opcode.lineno - func.func_code.co_firstlineno
-                    if i - 1 <= no <= i + 1:
-                        result[name].append(opcode)
+                lineno = start_lineno+i
+                result[name] = xrange(lineno, lineno+1)
         return result
+
+    def find_chunks(self, func):
+        """
+        Parse the given function and return a dictionary mapping "chunk names"
+        to "opcodes".
+        """
+        chunks = {}
+        code = disassembler.dis(func)
+        ranges = self.find_chunks_range(func)
+        for name, linerange in ranges.iteritems():
+            opcodes = [opcode for opcode in code.opcodes
+                       if opcode.lineno in linerange]
+            chunks[name] = opcodes
+        return chunks
     
     def run(self, func):
         with self.filepath.open("w") as f:
@@ -64,20 +82,30 @@
         return Trace()
 
 class TestInfrastructure(BaseTestPyPyC):
-    def test_parse_func(self):
+
+    def test_find_chunks_range(self):
+        def f():
+            a = 0 # ID: myline
+            return a
+        #
+        start_lineno = f.func_code.co_firstlineno
+        ids = self.find_chunks_range(f)
+        assert len(ids) == 1
+        myline_range = ids['myline']
+        assert list(myline_range) == range(start_lineno+1, start_lineno+2)
+
+    def test_find_chunks(self):
         def f():
             i = 0
             x = 0
-            # LOOP my_loop
-            z = x + 3
+            z = x + 3 # ID: myline
             return z
-
-        res = self.parse_func(f)
-        assert len(res) == 1
-        my_loop = res['my_loop']
-        opcodes_names = [opcode.__class__.__name__ for opcode in my_loop]
-        assert opcodes_names == ['LOAD_CONST', 'STORE_FAST', 'LOAD_FAST',
-                                 'LOAD_CONST', 'BINARY_ADD', 'STORE_FAST']
+        #
+        chunks = self.find_chunks(f)
+        assert len(chunks) == 1
+        myline = chunks['myline']
+        opcodes_names = [opcode.__class__.__name__ for opcode in myline]
+        assert opcodes_names == ['LOAD_FAST', 'LOAD_CONST', 'BINARY_ADD', 'STORE_FAST']
 
     def test_full(self):
         py.test.skip('in-progress')

From commits-noreply at bitbucket.org  Fri Feb 18 13:54:36 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Fri, 18 Feb 2011 13:54:36 +0100 (CET)
Subject: [pypy-svn] pypy default: move test_pypy_c_new to its own directory,
	because the file is getting too big
Message-ID: <20110218125436.C74062A202D@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42154:41f62bfd60c1
Date: 2011-02-18 10:09 +0100
http://bitbucket.org/pypy/pypy/changeset/41f62bfd60c1/

Log:	move test_pypy_c_new to its own directory, because the file is
	getting too big

diff --git a/pypy/module/pypyjit/test/test_pypy_c_new.py b/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py
copy from pypy/module/pypyjit/test/test_pypy_c_new.py
copy to pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py
--- a/pypy/module/pypyjit/test/test_pypy_c_new.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py
@@ -3,9 +3,18 @@
 from lib_pypy import disassembler
 from pypy.tool.udir import udir
 from pypy.tool import logparser
+from pypy.jit.tool import oparser
+
+class Log(object):
+    def __init__(self, func, rawtraces):
+        traces = map(Trace, rawtraces)
 
 class Trace(object):
-    pass
+    def __init__(self, rawtrace):
+        # "low level trace", i.e. an instance of history.TreeLoop
+        self.lltrace = oparser.parse(rawtrace, no_namespace=True)
+        
+
 
 class BaseTestPyPyC(object):
     def setup_class(cls):
@@ -19,15 +28,6 @@
     def setup_method(self, meth):
         self.filepath = self.tmpdir.join(meth.im_func.func_name + '.py')
 
-    def parse_out(self, out):
-        out = out.strip("\n")
-        if out == 'None':
-            return None
-        try:
-            return int(out)
-        except ValueError:
-            return out
-
     def find_chunks_range(self, func):
         """
         Parse the given function and return a dictionary mapping "chunk
@@ -61,25 +61,33 @@
             chunks[name] = opcodes
         return chunks
     
-    def run(self, func):
+    def run(self, func, threshold=1000):
+        # write the snippet
         with self.filepath.open("w") as f:
             f.write(str(py.code.Source(func)) + "\n")
-            f.write("print %s()\n" % func.func_name)
+            f.write("%s()\n" % func.func_name)
+        #
+        # run a child pypy-c with logging enabled
         logfile = self.filepath.new(ext='.log')
-        pipe = subprocess.Popen([sys.executable, str(self.filepath)],
-                                stdout=subprocess.PIPE, stderr=subprocess.PIPE,
-                                env={'PYPYLOG': "jit-log-opt,jit-summary:" + str(logfile)})
+        env={'PYPYLOG': 'jit-log-opt,jit-summary:' + str(logfile)}
+        cmdline = [sys.executable,
+                   '--jit', 'threshold=%d' % threshold,
+                   str(self.filepath)]
+        pipe = subprocess.Popen(cmdline,
+                                env=env,
+                                stdout=subprocess.PIPE,
+                                stderr=subprocess.PIPE)
         pipe.wait()
         stderr = pipe.stderr.read()
+        stdout = pipe.stdout.read()
         assert not stderr
-        res = self.parse_out(pipe.stdout.read())
-        bytecodes = self.parse_func(func)
-        assert res == func()
-        log = logparser.parse_log_file(str(logfile))
-        parts = logparser.extract_category(log, 'jit-log-opt-')
-        import pdb;pdb.set_trace()
-        log.xxx
-        return Trace()
+        #
+        # parse the JIT log
+        chunks = self.find_chunks(func)
+        rawlog = logparser.parse_log_file(str(logfile))
+        rawtraces = logparser.extract_category(rawlog, 'jit-log-opt-')
+        log = Log(func, rawtraces)
+        return log
 
 class TestInfrastructure(BaseTestPyPyC):
 
@@ -107,6 +115,16 @@
         opcodes_names = [opcode.__class__.__name__ for opcode in myline]
         assert opcodes_names == ['LOAD_FAST', 'LOAD_CONST', 'BINARY_ADD', 'STORE_FAST']
 
+    def test_parse_jitlog(self):
+        def f():
+            i = 0
+            while i < 1003: # default threshold is 10
+                i += 1 # ID: increment
+            return i
+        #
+        trace = self.run(f)
+
+
     def test_full(self):
         py.test.skip('in-progress')
         def f():

diff --git a/pypy/module/pypyjit/test_pypy_c/__init__.py b/pypy/module/pypyjit/test_pypy_c/__init__.py
new file mode 100644

diff --git a/pypy/module/pypyjit/test/test_pypy_c_new.py b/pypy/module/pypyjit/test/test_pypy_c_new.py
deleted file mode 100644
--- a/pypy/module/pypyjit/test/test_pypy_c_new.py
+++ /dev/null
@@ -1,134 +0,0 @@
-import py, sys, re
-import subprocess
-from lib_pypy import disassembler
-from pypy.tool.udir import udir
-from pypy.tool import logparser
-
-class Trace(object):
-    pass
-
-class BaseTestPyPyC(object):
-    def setup_class(cls):
-        if '__pypy__' not in sys.builtin_module_names:
-            py.test.skip("must run this test with pypy")
-        if not sys.pypy_translation_info['translation.jit']:
-            py.test.skip("must give a pypy-c with the jit enabled")
-        cls.tmpdir = udir.join('test-pypy-jit')
-        cls.tmpdir.ensure(dir=True)
-
-    def setup_method(self, meth):
-        self.filepath = self.tmpdir.join(meth.im_func.func_name + '.py')
-
-    def parse_out(self, out):
-        out = out.strip("\n")
-        if out == 'None':
-            return None
-        try:
-            return int(out)
-        except ValueError:
-            return out
-
-    def find_chunks_range(self, func):
-        """
-        Parse the given function and return a dictionary mapping "chunk
-        names" to "line ranges".  Chunks are identified by comments with a
-        special syntax::
-
-            # the chunk "myid" corresponds to the whole line
-            print 'foo' # ID: myid
-        """
-        result = {}
-        start_lineno = func.func_code.co_firstlineno
-        for i, line in enumerate(py.code.Source(func)):
-            m = re.search('# ID: (\w+)', line)
-            if m:
-                name = m.group(1)
-                lineno = start_lineno+i
-                result[name] = xrange(lineno, lineno+1)
-        return result
-
-    def find_chunks(self, func):
-        """
-        Parse the given function and return a dictionary mapping "chunk names"
-        to "opcodes".
-        """
-        chunks = {}
-        code = disassembler.dis(func)
-        ranges = self.find_chunks_range(func)
-        for name, linerange in ranges.iteritems():
-            opcodes = [opcode for opcode in code.opcodes
-                       if opcode.lineno in linerange]
-            chunks[name] = opcodes
-        return chunks
-    
-    def run(self, func):
-        with self.filepath.open("w") as f:
-            f.write(str(py.code.Source(func)) + "\n")
-            f.write("print %s()\n" % func.func_name)
-        logfile = self.filepath.new(ext='.log')
-        pipe = subprocess.Popen([sys.executable, str(self.filepath)],
-                                stdout=subprocess.PIPE, stderr=subprocess.PIPE,
-                                env={'PYPYLOG': "jit-log-opt,jit-summary:" + str(logfile)})
-        pipe.wait()
-        stderr = pipe.stderr.read()
-        assert not stderr
-        res = self.parse_out(pipe.stdout.read())
-        bytecodes = self.parse_func(func)
-        assert res == func()
-        log = logparser.parse_log_file(str(logfile))
-        parts = logparser.extract_category(log, 'jit-log-opt-')
-        import pdb;pdb.set_trace()
-        log.xxx
-        return Trace()
-
-class TestInfrastructure(BaseTestPyPyC):
-
-    def test_find_chunks_range(self):
-        def f():
-            a = 0 # ID: myline
-            return a
-        #
-        start_lineno = f.func_code.co_firstlineno
-        ids = self.find_chunks_range(f)
-        assert len(ids) == 1
-        myline_range = ids['myline']
-        assert list(myline_range) == range(start_lineno+1, start_lineno+2)
-
-    def test_find_chunks(self):
-        def f():
-            i = 0
-            x = 0
-            z = x + 3 # ID: myline
-            return z
-        #
-        chunks = self.find_chunks(f)
-        assert len(chunks) == 1
-        myline = chunks['myline']
-        opcodes_names = [opcode.__class__.__name__ for opcode in myline]
-        assert opcodes_names == ['LOAD_FAST', 'LOAD_CONST', 'BINARY_ADD', 'STORE_FAST']
-
-    def test_full(self):
-        py.test.skip('in-progress')
-        def f():
-            i = 0
-            while i < 1003:
-                # LOOP one
-                i += 1
-
-        trace = self.run(f)
-        loop = trace.get_loops('one')
-        loop.get_bytecode(3, 'LOAD_FAST').match('''
-        int_add
-        guard_true
-        ''')
-        loop.get_bytecode(4, 'LOAD_CONST').match_stats(
-            guard='3', call='1-2', call_may_force='0'
-        )
-        # this would make operations that are "costly" obligatory to pass
-        # like new
-        loo.get_bytecode(5, 'INPLACE_ADD').match_stats(
-            allocs='5-10'
-            )
-
-class TestPyPyCNew(BaseTestPyPyC):
-    pass

From commits-noreply at bitbucket.org  Fri Feb 18 13:54:38 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Fri, 18 Feb 2011 13:54:38 +0100 (CET)
Subject: [pypy-svn] pypy default: move some code into model.py
Message-ID: <20110218125438.9FFEC2A202D@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42155:2bc86912a181
Date: 2011-02-18 10:26 +0100
http://bitbucket.org/pypy/pypy/changeset/2bc86912a181/

Log:	move some code into model.py

diff --git a/pypy/module/pypyjit/test_pypy_c/model.py b/pypy/module/pypyjit/test_pypy_c/model.py
new file mode 100644
--- /dev/null
+++ b/pypy/module/pypyjit/test_pypy_c/model.py
@@ -0,0 +1,49 @@
+import py
+import re
+from lib_pypy import disassembler
+from pypy.jit.tool import oparser
+
+class Log(object):
+    def __init__(self, func, rawtraces):
+        traces = map(Trace, rawtraces)
+
+    @classmethod
+    def find_chunks_range(cls, func):
+        """
+        Parse the given function and return a dictionary mapping "chunk
+        names" to "line ranges".  Chunks are identified by comments with a
+        special syntax::
+
+            # the chunk "myid" corresponds to the whole line
+            print 'foo' # ID: myid
+        """
+        result = {}
+        start_lineno = func.func_code.co_firstlineno
+        for i, line in enumerate(py.code.Source(func)):
+            m = re.search('# ID: (\w+)', line)
+            if m:
+                name = m.group(1)
+                lineno = start_lineno+i
+                result[name] = xrange(lineno, lineno+1)
+        return result
+
+    @classmethod
+    def find_chunks(cls, func):
+        """
+        Parse the given function and return a dictionary mapping "chunk names"
+        to "opcodes".
+        """
+        chunks = {}
+        code = disassembler.dis(func)
+        ranges = cls.find_chunks_range(func)
+        for name, linerange in ranges.iteritems():
+            opcodes = [opcode for opcode in code.opcodes
+                       if opcode.lineno in linerange]
+            chunks[name] = opcodes
+        return chunks
+
+
+class Trace(object):
+    def __init__(self, rawtrace):
+        # "low level trace", i.e. an instance of history.TreeLoop
+        self.lltrace = oparser.parse(rawtrace, no_namespace=True)

diff --git a/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py b/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py
--- a/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py
@@ -3,16 +3,8 @@
 from lib_pypy import disassembler
 from pypy.tool.udir import udir
 from pypy.tool import logparser
-from pypy.jit.tool import oparser
+from pypy.module.pypyjit.test_pypy_c.model import Log
 
-class Log(object):
-    def __init__(self, func, rawtraces):
-        traces = map(Trace, rawtraces)
-
-class Trace(object):
-    def __init__(self, rawtrace):
-        # "low level trace", i.e. an instance of history.TreeLoop
-        self.lltrace = oparser.parse(rawtrace, no_namespace=True)
         
 
 
@@ -28,38 +20,6 @@
     def setup_method(self, meth):
         self.filepath = self.tmpdir.join(meth.im_func.func_name + '.py')
 
-    def find_chunks_range(self, func):
-        """
-        Parse the given function and return a dictionary mapping "chunk
-        names" to "line ranges".  Chunks are identified by comments with a
-        special syntax::
-
-            # the chunk "myid" corresponds to the whole line
-            print 'foo' # ID: myid
-        """
-        result = {}
-        start_lineno = func.func_code.co_firstlineno
-        for i, line in enumerate(py.code.Source(func)):
-            m = re.search('# ID: (\w+)', line)
-            if m:
-                name = m.group(1)
-                lineno = start_lineno+i
-                result[name] = xrange(lineno, lineno+1)
-        return result
-
-    def find_chunks(self, func):
-        """
-        Parse the given function and return a dictionary mapping "chunk names"
-        to "opcodes".
-        """
-        chunks = {}
-        code = disassembler.dis(func)
-        ranges = self.find_chunks_range(func)
-        for name, linerange in ranges.iteritems():
-            opcodes = [opcode for opcode in code.opcodes
-                       if opcode.lineno in linerange]
-            chunks[name] = opcodes
-        return chunks
     
     def run(self, func, threshold=1000):
         # write the snippet
@@ -83,7 +43,6 @@
         assert not stderr
         #
         # parse the JIT log
-        chunks = self.find_chunks(func)
         rawlog = logparser.parse_log_file(str(logfile))
         rawtraces = logparser.extract_category(rawlog, 'jit-log-opt-')
         log = Log(func, rawtraces)
@@ -91,30 +50,6 @@
 
 class TestInfrastructure(BaseTestPyPyC):
 
-    def test_find_chunks_range(self):
-        def f():
-            a = 0 # ID: myline
-            return a
-        #
-        start_lineno = f.func_code.co_firstlineno
-        ids = self.find_chunks_range(f)
-        assert len(ids) == 1
-        myline_range = ids['myline']
-        assert list(myline_range) == range(start_lineno+1, start_lineno+2)
-
-    def test_find_chunks(self):
-        def f():
-            i = 0
-            x = 0
-            z = x + 3 # ID: myline
-            return z
-        #
-        chunks = self.find_chunks(f)
-        assert len(chunks) == 1
-        myline = chunks['myline']
-        opcodes_names = [opcode.__class__.__name__ for opcode in myline]
-        assert opcodes_names == ['LOAD_FAST', 'LOAD_CONST', 'BINARY_ADD', 'STORE_FAST']
-
     def test_parse_jitlog(self):
         def f():
             i = 0

diff --git a/pypy/module/pypyjit/test_pypy_c/test_model.py b/pypy/module/pypyjit/test_pypy_c/test_model.py
new file mode 100644
--- /dev/null
+++ b/pypy/module/pypyjit/test_pypy_c/test_model.py
@@ -0,0 +1,27 @@
+from pypy.module.pypyjit.test_pypy_c.model import Log
+
+class TestLog(object):
+
+    def test_find_chunks_range(self):
+        def f():
+            a = 0 # ID: myline
+            return a
+        #
+        start_lineno = f.func_code.co_firstlineno
+        ids = Log.find_chunks_range(f)
+        assert len(ids) == 1
+        myline_range = ids['myline']
+        assert list(myline_range) == range(start_lineno+1, start_lineno+2)
+
+    def test_find_chunks(self):
+        def f():
+            i = 0
+            x = 0
+            z = x + 3 # ID: myline
+            return z
+        #
+        chunks = Log.find_chunks(f)
+        assert len(chunks) == 1
+        myline = chunks['myline']
+        opcodes_names = [opcode.__class__.__name__ for opcode in myline]
+        assert opcodes_names == ['LOAD_FAST', 'LOAD_CONST', 'BINARY_ADD', 'STORE_FAST']

From commits-noreply at bitbucket.org  Fri Feb 18 13:54:41 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Fri, 18 Feb 2011 13:54:41 +0100 (CET)
Subject: [pypy-svn] pypy default: more refactoring, still work in progress
Message-ID: <20110218125441.65AFC282C2B@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42156:be7854456771
Date: 2011-02-18 13:11 +0100
http://bitbucket.org/pypy/pypy/changeset/be7854456771/

Log:	more refactoring, still work in progress

diff --git a/pypy/module/pypyjit/test_pypy_c/model.py b/pypy/module/pypyjit/test_pypy_c/model.py
--- a/pypy/module/pypyjit/test_pypy_c/model.py
+++ b/pypy/module/pypyjit/test_pypy_c/model.py
@@ -5,7 +5,8 @@
 
 class Log(object):
     def __init__(self, func, rawtraces):
-        traces = map(Trace, rawtraces)
+        chunks = self.find_chunks(func)
+        self.traces = [Trace(rawtrace, chunks) for rawtrace in rawtraces]
 
     @classmethod
     def find_chunks_range(cls, func):
@@ -44,6 +45,19 @@
 
 
 class Trace(object):
-    def __init__(self, rawtrace):
+    def __init__(self, rawtrace, chunks):
         # "low level trace", i.e. an instance of history.TreeLoop
         self.lltrace = oparser.parse(rawtrace, no_namespace=True)
+        self.split_into_opcodes()
+
+    def split_into_opcodes(self):
+        self.opcodes = []
+        for op in self.lltrace.operations:
+            if op.getopname() == "debug_merge_point":
+                opcode = TraceForOpcode(op) # XXX
+                self.opcodes.append(opcode)
+            else:
+                opcode.append(op)
+
+class TraceForOpcode(list):
+    pass

diff --git a/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py b/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py
--- a/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py
@@ -4,62 +4,11 @@
 from pypy.tool.udir import udir
 from pypy.tool import logparser
 from pypy.module.pypyjit.test_pypy_c.model import Log
+from pypy.module.pypyjit.test_pypy_c.test_model import BaseTestPyPyC
 
-        
-
-
-class BaseTestPyPyC(object):
-    def setup_class(cls):
-        if '__pypy__' not in sys.builtin_module_names:
-            py.test.skip("must run this test with pypy")
-        if not sys.pypy_translation_info['translation.jit']:
-            py.test.skip("must give a pypy-c with the jit enabled")
-        cls.tmpdir = udir.join('test-pypy-jit')
-        cls.tmpdir.ensure(dir=True)
-
-    def setup_method(self, meth):
-        self.filepath = self.tmpdir.join(meth.im_func.func_name + '.py')
-
-    
-    def run(self, func, threshold=1000):
-        # write the snippet
-        with self.filepath.open("w") as f:
-            f.write(str(py.code.Source(func)) + "\n")
-            f.write("%s()\n" % func.func_name)
-        #
-        # run a child pypy-c with logging enabled
-        logfile = self.filepath.new(ext='.log')
-        env={'PYPYLOG': 'jit-log-opt,jit-summary:' + str(logfile)}
-        cmdline = [sys.executable,
-                   '--jit', 'threshold=%d' % threshold,
-                   str(self.filepath)]
-        pipe = subprocess.Popen(cmdline,
-                                env=env,
-                                stdout=subprocess.PIPE,
-                                stderr=subprocess.PIPE)
-        pipe.wait()
-        stderr = pipe.stderr.read()
-        stdout = pipe.stdout.read()
-        assert not stderr
-        #
-        # parse the JIT log
-        rawlog = logparser.parse_log_file(str(logfile))
-        rawtraces = logparser.extract_category(rawlog, 'jit-log-opt-')
-        log = Log(func, rawtraces)
-        return log
 
 class TestInfrastructure(BaseTestPyPyC):
 
-    def test_parse_jitlog(self):
-        def f():
-            i = 0
-            while i < 1003: # default threshold is 10
-                i += 1 # ID: increment
-            return i
-        #
-        trace = self.run(f)
-
-
     def test_full(self):
         py.test.skip('in-progress')
         def f():

diff --git a/pypy/module/pypyjit/test_pypy_c/test_model.py b/pypy/module/pypyjit/test_pypy_c/test_model.py
--- a/pypy/module/pypyjit/test_pypy_c/test_model.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_model.py
@@ -1,5 +1,50 @@
+import sys
+import subprocess
+import py
+from pypy.tool.udir import udir
+from pypy.tool import logparser
 from pypy.module.pypyjit.test_pypy_c.model import Log
 
+class BaseTestPyPyC(object):
+    def setup_class(cls):
+        if '__pypy__' not in sys.builtin_module_names:
+            py.test.skip("must run this test with pypy")
+        if not sys.pypy_translation_info['translation.jit']:
+            py.test.skip("must give a pypy-c with the jit enabled")
+        cls.tmpdir = udir.join('test-pypy-jit')
+        cls.tmpdir.ensure(dir=True)
+
+    def setup_method(self, meth):
+        self.filepath = self.tmpdir.join(meth.im_func.func_name + '.py')
+
+    def run(self, func, threshold=1000):
+        # write the snippet
+        with self.filepath.open("w") as f:
+            f.write(str(py.code.Source(func)) + "\n")
+            f.write("%s()\n" % func.func_name)
+        #
+        # run a child pypy-c with logging enabled
+        logfile = self.filepath.new(ext='.log')
+        env={'PYPYLOG': 'jit-log-opt,jit-summary:' + str(logfile)}
+        cmdline = [sys.executable,
+                   '--jit', 'threshold=%d' % threshold,
+                   str(self.filepath)]
+        pipe = subprocess.Popen(cmdline,
+                                env=env,
+                                stdout=subprocess.PIPE,
+                                stderr=subprocess.PIPE)
+        pipe.wait()
+        stderr = pipe.stderr.read()
+        stdout = pipe.stdout.read()
+        assert not stderr
+        #
+        # parse the JIT log
+        rawlog = logparser.parse_log_file(str(logfile))
+        rawtraces = logparser.extract_category(rawlog, 'jit-log-opt-')
+        log = Log(func, rawtraces)
+        return log
+
+
 class TestLog(object):
 
     def test_find_chunks_range(self):
@@ -25,3 +70,15 @@
         myline = chunks['myline']
         opcodes_names = [opcode.__class__.__name__ for opcode in myline]
         assert opcodes_names == ['LOAD_FAST', 'LOAD_CONST', 'BINARY_ADD', 'STORE_FAST']
+
+class TestRunPyPyC(BaseTestPyPyC):
+
+    def test_parse_jitlog(self):
+        py.test.skip('in-progress')
+        def f():
+            i = 0
+            while i < 1003: # default threshold is 10
+                i += 1 # ID: increment
+            return i
+        #
+        log = self.run(f)

From commits-noreply at bitbucket.org  Fri Feb 18 13:54:46 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Fri, 18 Feb 2011 13:54:46 +0100 (CET)
Subject: [pypy-svn] pypy default: import the "jitlogparser" from the
	external repo	pypy/jitviewer. So far it is
Message-ID: <20110218125446.29292282BE3@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42157:e97f2618bc93
Date: 2011-02-18 13:23 +0100
http://bitbucket.org/pypy/pypy/changeset/e97f2618bc93/

Log:	import the "jitlogparser" from the external repo pypy/jitviewer. So
	far it is just a plain copy of the files with adapted imports, later
	the HTML-related parts will be removed

diff --git a/pypy/tool/jitlogparser/test/test_parser.py b/pypy/tool/jitlogparser/test/test_parser.py
new file mode 100644
--- /dev/null
+++ b/pypy/tool/jitlogparser/test/test_parser.py
@@ -0,0 +1,216 @@
+from pypy.jit.metainterp.resoperation import ResOperation, rop
+from pypy.jit.metainterp.history import ConstInt, Const
+from pypy.tool.jitlogparser.parser import parse, Bytecode, Function,\
+     slice_debug_merge_points,\
+     adjust_bridges, parse_log_counts, cssclass
+from pypy.tool.jitlogparser.storage import LoopStorage
+import py
+
+
+def test_parse():
+    ops = parse('''
+    [i7]
+    i9 = int_lt(i7, 1003)
+    guard_true(i9, descr=<Guard2>) []
+    i13 = getfield_raw(151937600, descr=<SignedFieldDescr pypysig_long_struct.c_value 0>)
+    ''').operations
+    assert len(ops) == 3
+    assert ops[0].name == 'int_lt'
+    assert ops[1].name == 'guard_true'
+    assert ops[1].descr is not None
+    assert ops[0].res == 'i9'
+    assert ops[0].html_repr().plaintext() == 'i9 = i7 < 1003'
+    assert ops[2].descr is not None
+    assert len(ops[2].args) == 1
+    assert ops[2].html_repr().plaintext() == 'i13 = ((pypysig_long_struct)151937600).value'
+
+def test_parse_non_code():
+    ops = parse('''
+    []
+    debug_merge_point("SomeRandomStuff", 0)
+    ''')
+    res = slice_debug_merge_points(ops.operations, LoopStorage())
+    assert len(res.chunks) == 1
+    assert res.chunks[0].html_repr()
+
+def test_split():
+    ops = parse('''
+    [i0]
+    debug_merge_point("<code object stuff, file '/tmp/x.py', line 200> #10 ADD", 0)
+    debug_merge_point("<code object stuff, file '/tmp/x.py', line 200> #11 SUB", 0)
+    i1 = int_add(i0, 1)
+    debug_merge_point("<code object stuff, file '/tmp/x.py', line 200> #11 SUB", 0)
+    i2 = int_add(i1, 1)
+    ''')
+    res = slice_debug_merge_points(ops.operations, LoopStorage())
+    assert len(res.chunks) == 3
+    assert len(res.chunks[0].operations) == 1
+    assert len(res.chunks[1].operations) == 2
+    assert len(res.chunks[2].operations) == 2
+    assert res.chunks[2].bytecode_no == 11
+
+def test_inlined_call():
+    ops = parse("""
+    []
+    debug_merge_point('<code object inlined_call, file 'source.py', line 12> #28 CALL_FUNCTION', 0)
+    i18 = getfield_gc(p0, descr=<BoolFieldDescr pypy.interpreter.pyframe.PyFrame.inst_is_being_profiled 89>)
+    debug_merge_point('<code object inner, file 'source.py', line 9> #0 LOAD_FAST', 1)
+    debug_merge_point('<code object inner, file 'source.py', line 9> #3 LOAD_CONST', 1)
+    debug_merge_point('<code object inner, file 'source.py', line 9> #7 RETURN_VALUE', 1)
+    debug_merge_point('<code object inlined_call, file 'source.py', line 12> #31 STORE_FAST', 0)
+    """)
+    res = slice_debug_merge_points(ops.operations, LoopStorage())
+    assert len(res.chunks) == 3 # two chunks + inlined call
+    assert isinstance(res.chunks[0], Bytecode)
+    assert isinstance(res.chunks[1], Function)
+    assert isinstance(res.chunks[2], Bytecode)
+    assert res.chunks[1].path == "1"
+    assert len(res.chunks[1].chunks) == 3
+    
+def test_name():
+    ops = parse('''
+    [i0]
+    debug_merge_point("<code object stuff, file '/tmp/x.py', line 200> #10 ADD", 0)
+    debug_merge_point("<code object stuff, file '/tmp/x.py', line 201> #11 SUB", 0)
+    i1 = int_add(i0, 1)
+    debug_merge_point("<code object stuff, file '/tmp/x.py', line 202> #11 SUB", 0)
+    i2 = int_add(i1, 1)
+    ''')
+    res = slice_debug_merge_points(ops.operations, LoopStorage())
+    assert res.repr() == res.chunks[0].repr()
+    assert res.repr() == "stuff, file '/tmp/x.py', line 200"
+    assert res.startlineno == 200
+    assert res.filename == '/tmp/x.py'
+    assert res.name == 'stuff'
+
+def test_name_no_first():
+    ops = parse('''
+    [i0]
+    i3 = int_add(i0, 1)
+    debug_merge_point("<code object stuff, file '/tmp/x.py', line 200> #10 ADD", 0)
+    debug_merge_point("<code object stuff, file '/tmp/x.py', line 201> #11 SUB", 0)
+    i1 = int_add(i0, 1)
+    debug_merge_point("<code object stuff, file '/tmp/x.py', line 202> #11 SUB", 0)
+    i2 = int_add(i1, 1)
+    ''')
+    res = slice_debug_merge_points(ops.operations, LoopStorage())
+    assert res.repr() == res.chunks[1].repr()
+
+def test_lineno():
+    fname = str(py.path.local(__file__).join('..', 'x.py'))
+    ops = parse('''
+    [i0, i1]
+    debug_merge_point("<code object f, file '%(fname)s', line 2> #0 LOAD_FAST", 0)
+    debug_merge_point("<code object f, file '%(fname)s', line 2> #3 LOAD_FAST", 0)
+    debug_merge_point("<code object f, file '%(fname)s', line 2> #6 BINARY_ADD", 0)
+    debug_merge_point("<code object f, file '%(fname)s', line 2> #7 RETURN_VALUE", 0)
+    ''' % locals())
+    res = slice_debug_merge_points(ops.operations, LoopStorage())
+    assert res.chunks[1].lineno == 3
+
+def test_linerange():
+    fname = str(py.path.local(__file__).join('..', 'x.py'))
+    ops = parse('''
+    [i0, i1]
+    debug_merge_point("<code object f, file '%(fname)s', line 5> #9 LOAD_FAST", 0)
+    debug_merge_point("<code object f, file '%(fname)s', line 5> #12 LOAD_CONST", 0)
+    debug_merge_point("<code object f, file '%(fname)s', line 5> #22 LOAD_CONST", 0)
+    debug_merge_point("<code object f, file '%(fname)s', line 5> #28 LOAD_CONST", 0)
+    debug_merge_point("<code object f, file '%(fname)s', line 5> #6 SETUP_LOOP", 0)
+    ''' % locals())
+    res = slice_debug_merge_points(ops.operations, LoopStorage())
+    assert res.linerange == (7, 9)
+    assert res.lineset == set([7, 8, 9])
+
+def test_linerange_notstarts():
+    fname = str(py.path.local(__file__).join('..', 'x.py'))
+    ops = parse("""
+    [p6, p1]
+    debug_merge_point('<code object h, file '%(fname)s', line 11> #17 FOR_ITER', 0)
+    guard_class(p6, 144264192, descr=<Guard2>)
+    p12 = getfield_gc(p6, descr=<GcPtrFieldDescr pypy.objspace.std.iterobject.W_AbstractSeqIterObject.inst_w_seq 12>)
+    """ % locals())
+    res = slice_debug_merge_points(ops.operations, LoopStorage())
+    assert res.lineset
+
+def test_reassign_loops():
+    main = parse('''
+    [v0]
+    guard_false(v0, descr=<Guard18>) []
+    ''')
+    main.count = 10
+    bridge = parse('''
+    # bridge out of Guard 18 with 13 ops
+    [i0, i1]
+    int_add(i0, i1)
+    ''')
+    bridge.count = 3
+    entry_bridge = parse('''
+    # Loop 3 : entry bridge
+    []
+    ''')
+    loops = LoopStorage().reconnect_loops([main, bridge, entry_bridge])
+    assert len(loops) == 2
+    assert len(loops[0].operations[0].bridge.operations) == 1
+    assert loops[0].operations[0].bridge.no == 18
+    assert loops[0].operations[0].percentage == 30
+
+def test_adjust_bridges():
+    main = parse('''
+    [v0]
+    guard_false(v0, descr=<Guard13>)
+    guard_true(v0, descr=<Guard5>)
+    ''')
+    bridge = parse('''
+    # bridge out of Guard 13
+    []
+    int_add(0, 1)
+    ''')
+    loops = LoopStorage().reconnect_loops([main, bridge])
+    assert adjust_bridges(main, {})[1].name == 'guard_true'
+    assert adjust_bridges(main, {'loop-13': True})[1].name == 'int_add'
+
+def test_parsing_strliteral():
+    loop = parse("""
+    debug_merge_point('StrLiteralSearch at 11/51 [17, 8, 3, 1, 1, 1, 1, 51, 0, 19, 51, 1]', 0)
+    """)
+    ops = slice_debug_merge_points(loop.operations, LoopStorage())
+    chunk = ops.chunks[0]
+    assert chunk.bytecode_name == 'StrLiteralSearch'
+
+LINES = '''
+0:3
+1:3
+2:604
+3:396
+4:102
+5:2000
+6:3147
+7:2445
+8:2005
+9:2000
+10:1420
+11:40
+12:0
+'''.split("\n")
+
+def test_parse_log_count():
+    class Loop(object):
+        pass
+    
+    loops = [Loop() for i in range(13)]
+    nums = parse_log_counts(LINES, loops)
+    assert nums[5] == 2000
+    assert loops[9].count == 2000
+
+def test_highlight_var():
+    ops = parse('''
+    [p0]
+    guard_class(p0, 144264192, descr=<Guard0>)
+    ''').operations
+    assert len(ops) == 1
+    op = ops[0]
+    assert op.name == 'guard_class'
+    html = op.html_repr()
+    p0 = cssclass('p0', 'p0', onmouseover="highlight_var(this)", onmouseout="disable_var(this)")
+    assert p0 in html

diff --git a/pypy/tool/jitlogparser/test/x.py b/pypy/tool/jitlogparser/test/x.py
new file mode 100644
--- /dev/null
+++ b/pypy/tool/jitlogparser/test/x.py
@@ -0,0 +1,12 @@
+
+def f(a, b):
+    return a + b
+
+def g():
+    i = 0
+    while i < 10:
+        a = 'foo'
+        i += 1
+
+def h():
+    [x for x in range(10)]

diff --git a/pypy/tool/jitlogparser/storage.py b/pypy/tool/jitlogparser/storage.py
new file mode 100644
--- /dev/null
+++ b/pypy/tool/jitlogparser/storage.py
@@ -0,0 +1,56 @@
+
+""" This file represents a storage mechanism that let us invent unique names
+for all loops and bridges, so http requests can refer to them by name
+"""
+
+import os
+from pypy.tool.jitlogparser.parser import Function, Bytecode
+from pypy.tool.jitlogparser.module_finder import gather_all_code_objs
+
+class LoopStorage(object):
+    def __init__(self, extrapath=None):
+        self.loops = None
+        self.functions = {}
+        self.codes = {}
+        self.extrapath = extrapath
+
+    def load_code(self, fname):
+        try:
+            return self.codes[fname]
+        except KeyError:
+            if os.path.isabs(fname):
+                res = gather_all_code_objs(fname)
+            else:
+                if self.extrapath is None:
+                    raise IOError("Cannot find %s" % fname)
+                res = gather_all_code_objs(os.path.join(self.extrapath, fname))
+            self.codes[fname] = res
+            return res
+
+    def reconnect_loops(self, loops):
+        """ Re-connect loops in a way that entry bridges are filtered out
+        and normal bridges are associated with guards. Returning list of
+        normal loops.
+        """
+        res = []
+        guard_dict = {}
+        for loop_no, loop in enumerate(loops):
+            for op in loop.operations:
+                if op.name.startswith('guard_'):
+                    guard_dict[int(op.descr[len('<Guard'):-1])] = (op, loop)
+        for loop in loops:
+            if loop.comment:
+                comment = loop.comment.strip()
+                if 'entry bridge' in comment:
+                    pass
+                elif comment.startswith('# bridge out of'):
+                    no = int(comment[len('# bridge out of Guard '):].split(' ', 1)[0])
+                    op, parent = guard_dict[no]
+                    op.bridge = loop
+                    op.percentage = ((getattr(loop, 'count', 1) * 100) /
+                                     max(getattr(parent, 'count', 1), 1))
+                    loop.no = no
+                    continue
+            res.append(loop)
+        self.loops = res
+        return res


diff --git a/pypy/tool/jitlogparser/module_finder.py b/pypy/tool/jitlogparser/module_finder.py
new file mode 100644
--- /dev/null
+++ b/pypy/tool/jitlogparser/module_finder.py
@@ -0,0 +1,25 @@
+
+import os, sys, marshal, types, struct, imp
+
+def _all_codes_from(code):
+    res = {}
+    more = [code]
+    while more:
+        next = more.pop()
+        res[next.co_firstlineno] = next
+        more += [co for co in next.co_consts
+                 if isinstance(co, types.CodeType)]
+    return res
+
+def gather_all_code_objs(fname):
+    """ Gathers all code objects from a give fname and sorts them by
+    starting lineno
+    """
+    fname = str(fname)
+    if fname.endswith('.pyc'):
+        code = compile(open(fname[:-1]).read(), fname, 'exec')
+    elif fname.endswith('.py'):
+        code = compile(open(fname).read(), fname, 'exec')
+    else:
+        raise Exception("Unknown file extension: %s" % fname)
+    return _all_codes_from(code)

diff --git a/pypy/tool/jitlogparser/parser.py b/pypy/tool/jitlogparser/parser.py
new file mode 100644
--- /dev/null
+++ b/pypy/tool/jitlogparser/parser.py
@@ -0,0 +1,372 @@
+import re, sys
+from lib_pypy.disassembler import dis
+from pypy.jit.metainterp.resoperation import rop, opname
+from pypy.jit.tool.oparser import OpParser
+
+class Html(str):
+    def __html__(self):
+        return self
+
+    def plaintext(self):
+        # This is not a general way to strip tags, but it's good enough to use
+        # in tests
+        import re
+        s = re.sub('<.*?>', '', self)
+        s = s.replace("&lt;", "<")
+        s = s.replace("&gt;", ">")
+        s = s.replace("&amp;", "&")
+        return s
+
+
+def cssclass(cls, s, **kwds):
+    attrs = ['%s="%s"' % (name, value) for name, value in kwds.iteritems()]
+    return '<span class="%s" %s>%s</span>' % (cls, ' '.join(attrs), s)
+
+
+def _new_binop(name):
+    import cgi
+    name = cgi.escape(name)
+    def f(self):
+        return '%s = %s %s %s' % (self.getres(), self.getarg(0), name, self.getarg(1))
+    return f
+
+class Op(object):
+    bridge = None
+    
+    def __init__(self, name, args, res, descr):
+        self.name = name
+        self.args = args
+        self.res = res
+        self.descr = descr
+        self._is_guard = name.startswith('guard_')
+        if self._is_guard:
+            self.guard_no = int(self.descr[len('<Guard'):-1])
+
+    def setfailargs(self, _):
+        pass
+
+    def html_repr(self):
+        s = getattr(self, 'repr_' + self.name, self.generic_repr)()
+        if self.is_guard():
+            s = '<span class="guard">guard</span>(' + s + ')'
+        return Html(s)
+
+    def getarg(self, i):
+        return self._getvar(self.args[i])
+
+    def getargs(self):
+        return [self._getvar(v) for v in self.args]
+
+    def getres(self):
+        return self._getvar(self.res)
+
+    def _getvar(self, v):
+        return cssclass(v, v, onmouseover='highlight_var(this)', onmouseout='disable_var(this)')
+
+    def is_guard(self):
+        return self._is_guard
+
+    for bin_op, name in [('==', 'int_eq'),
+                         ('!=', 'int_ne'),
+                         ('==', 'float_eq'),
+                         ('!=', 'float_ne'),
+                         ('>', 'int_gt'),
+                         ('<', 'int_lt'),
+                         ('<=', 'int_le'),
+                         ('>=', 'int_ge'),
+                         ('+', 'int_add'),
+                         ('+', 'float_add'),
+                         ('-', 'int_sub'),
+                         ('-', 'float_sub'),
+                         ('*', 'int_mul'),
+                         ('*', 'float_mul'),
+                         ('&', 'int_and')]:
+        locals()['repr_' + name] = _new_binop(bin_op)
+
+    def repr_guard_true(self):
+        return '%s is true' % self.getarg(0)
+
+    def repr_guard_false(self):
+        return '%s is false' % self.getarg(0)
+
+    def repr_guard_value(self):
+        return '%s is %s' % (self.getarg(0), self.getarg(1))
+
+    def repr_guard_isnull(self):
+        return '%s is null' % self.getarg(0)
+
+    def repr_getfield_raw(self):
+        name, field = self.descr.split(' ')[1].rsplit('.', 1)
+        return '%s = ((%s)%s).%s' % (self.getres(), name, self.getarg(0), field[2:])
+
+    def repr_getfield_gc(self):
+        fullname, field = self.descr.split(' ')[1].rsplit('.', 1)
+        names = fullname.rsplit('.', 1)
+        if len(names) == 2:
+            namespace, classname = names
+        else:
+            namespace = ''
+            classname = names[0]
+        namespace = cssclass('namespace', namespace)
+        classname = cssclass('classname', classname)
+        field = cssclass('fieldname', field)
+            
+        obj = self.getarg(0)
+        return '%s = ((%s.%s)%s).%s' % (self.getres(), namespace, classname, obj, field)
+    repr_getfield_gc_pure = repr_getfield_gc
+
+    def repr_setfield_raw(self):
+        name, field = self.descr.split(' ')[1].rsplit('.', 1)
+        return '((%s)%s).%s = %s' % (name, self.getarg(0), field[2:], self.getarg(1))
+
+    def repr_setfield_gc(self):
+        name, field = self.descr.split(' ')[1].rsplit('.', 1)
+        return '((%s)%s).%s = %s' % (name, self.getarg(0), field, self.getarg(1))
+
+    def generic_repr(self):
+        arglist = ', '.join(self.getargs())
+        if self.res is not None:
+            return '%s = %s(%s)' % (self.getres(), self.name, arglist)
+        else:
+            return '%s(%s)' % (self.name, arglist)
+
+    def __repr__(self):
+        return '<%s (%s)>' % (self.name, ', '.join([repr(a)
+                                                    for a in self.args]))
+
+class SimpleParser(OpParser):
+    def parse_args(self, opname, argspec):
+        if not argspec.strip():
+            return [], None
+        if opname == 'debug_merge_point':
+            return argspec.rsplit(", ", 1), None
+        else:
+            args = argspec.split(', ')
+            descr = None
+            if args[-1].startswith('descr='):
+                descr = args[-1][len('descr='):]
+                args = args[:-1]
+            return (args, descr)
+
+    def box_for_var(self, res):
+        return res
+
+    def create_op(self, opnum, args, res, descr):
+        return Op(intern(opname[opnum].lower()), args, res, descr)
+
+class NonCodeError(Exception):
+    pass
+
+class Bytecode(object):
+    filename = None
+    startlineno = 0
+    name = None
+    code = None
+    bytecode_no = 0
+    bytecode_name = None
+    is_bytecode = True
+    inline_level = None
+    
+    def __init__(self, operations, storage):
+        if operations[0].name == 'debug_merge_point':
+            self.inline_level = int(operations[0].args[1])
+            m = re.search('<code object ([<>\w]+), file \'(.+?)\', line (\d+)> #(\d+) (\w+)',
+                         operations[0].getarg(0))
+            if m is None:
+                # a non-code loop, like StrLiteralSearch or something
+                self.bytecode_name = operations[0].args[0].split(" ")[0][1:]
+            else:
+                self.name, self.filename, lineno, bytecode_no, self.bytecode_name = m.groups()
+                self.startlineno = int(lineno)
+                self.bytecode_no = int(bytecode_no)
+        self.operations = operations
+        self.storage = storage
+
+    def repr(self):
+        if self.filename is None:
+            return "Unknown"
+        return "%s, file '%s', line %d" % (self.name, self.filename,
+                                           self.startlineno)
+
+    def getcode(self):
+        if self.code is None:
+            self.code = dis(self.storage.load_code(self.filename)[self.startlineno])
+        return self.code
+
+    def getlineno(self):
+        code = self.getcode()
+        return code.map[self.bytecode_no].lineno
+    lineno = property(getlineno)
+
+    def getline_starts_here(self):
+        code = self.getcode()
+        return code.map[self.bytecode_no].line_starts_here
+    line_starts_here = property(getline_starts_here)
+
+    def __repr__(self):
+        return "[%s]" % ", ".join([repr(op) for op in self.operations])
+
+    def pretty_print(self, out):
+        pass
+
+    def html_repr(self):
+        if self.filename is not None:
+            code = self.getcode()
+            opcode = self.code.map[self.bytecode_no]
+            return '%s %s' % (self.bytecode_name, opcode.argstr)
+        else:
+            return self.bytecode_name
+
+class Function(object):
+    filename = None
+    name = None
+    startlineno = 0
+    _linerange = None
+    _lineset = None
+    is_bytecode = False
+    inline_level = None
+    
+    def __init__(self, chunks, path, storage):
+        self.path = path
+        self.chunks = chunks
+        for chunk in self.chunks:
+            if chunk.filename is not None:
+                self.startlineno = chunk.startlineno
+                self.filename = chunk.filename
+                self.name = chunk.name
+                self.inline_level = chunk.inline_level
+                break
+        self.storage = storage
+
+    def getlinerange(self):
+        if self._linerange is None:
+            self._compute_linerange()
+        return self._linerange
+    linerange = property(getlinerange)
+
+    def getlineset(self):
+        if self._lineset is None:
+            self._compute_linerange()
+        return self._lineset
+    lineset = property(getlineset)
+
+    def _compute_linerange(self):
+        self._lineset = set()
+        minline = sys.maxint
+        maxline = -1
+        for chunk in self.chunks:
+            if chunk.is_bytecode and chunk.filename is not None:
+                lineno = chunk.lineno
+                minline = min(minline, lineno)
+                maxline = max(maxline, lineno)
+                if chunk.line_starts_here or len(chunk.operations) > 1:
+                    self._lineset.add(lineno)
+        if minline == sys.maxint:
+            minline = 0
+            maxline = 0
+        self._linerange = minline, maxline
+
+    def html_repr(self):
+        return "inlined call to %s in %s" % (self.name, self.filename)
+
+    def repr(self):
+        if self.filename is None:
+            return "Unknown"
+        return "%s, file '%s', line %d" % (self.name, self.filename,
+                                           self.startlineno)
+        
+    def __repr__(self):
+        return "[%s]" % ", ".join([repr(chunk) for chunk in self.chunks])
+
+    def pretty_print(self, out):
+        print >>out, "Loop starting at %s in %s at %d" % (self.name,
+                                        self.filename, self.startlineno)
+        lineno = -1
+        for chunk in self.chunks:
+            if chunk.filename is not None and chunk.lineno != lineno:
+                lineno = chunk.lineno
+                source = chunk.getcode().source[chunk.lineno -
+                                                chunk.startlineno]
+                print >>out, "  ", source
+            chunk.pretty_print(out)
+
+def parse_log_counts(input, loops):
+    if not input:
+        return
+    lines = input[-1].splitlines()
+    nums = []
+    i = 0
+    for line in lines:
+        if line:
+            num, count = line.split(':')
+            assert int(num) == i
+            count = int(count)
+            nums.append(count)
+            loops[i].count = count
+            i += 1
+    return nums
+
+def parse(input):
+    return SimpleParser(input, None, {}, 'lltype', None,
+                        nonstrict=True).parse()
+
+def slice_debug_merge_points(operations, storage, limit=None):
+    """ Slice given operation list into a chain of Bytecode chunks.
+    Also detect inlined functions and make them Function
+    """
+    stack = []
+
+    def getpath(stack):
+        return ",".join([str(len(v)) for v in stack])
+
+    def append_to_res(bc):
+        if not stack:
+            stack.append([])
+        else:
+            if bc.inline_level is not None and bc.inline_level + 1 != len(stack):
+                if bc.inline_level < len(stack):
+                    last = stack.pop()
+                    stack[-1].append(Function(last, getpath(stack), storage))
+                else:
+                    stack.append([])
+        stack[-1].append(bc)
+
+    so_far = []
+    stack = []
+    for op in operations:
+        if op.name == 'debug_merge_point':
+            if so_far:
+                append_to_res(Bytecode(so_far, storage))
+                if limit:
+                    break
+                so_far = []
+        so_far.append(op)
+    if so_far:
+        append_to_res(Bytecode(so_far, storage))
+    # wrap stack back up
+    if not stack:
+        # no ops whatsoever
+        return Function([], getpath(stack), storage)
+    while True:
+        next = stack.pop()
+        if not stack:
+            return Function(next, getpath(stack), storage)
+        stack[-1].append(Function(next, getpath(stack), storage))
+
+def adjust_bridges(loop, bridges):
+    """ Slice given loop according to given bridges to follow. Returns a plain
+    list of operations.
+    """
+    ops = loop.operations
+    res = []
+    i = 0
+    while i < len(ops):
+        op = ops[i]
+        if op.is_guard() and bridges.get('loop-' + str(op.guard_no), None):
+            res.append(op)
+            i = 0
+            ops = op.bridge.operations
+        else:
+            res.append(op)
+            i += 1
+    return res

diff --git a/pypy/tool/jitlogparser/__init__.py b/pypy/tool/jitlogparser/__init__.py
new file mode 100644

From commits-noreply at bitbucket.org  Fri Feb 18 13:54:48 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Fri, 18 Feb 2011 13:54:48 +0100 (CET)
Subject: [pypy-svn] pypy default: kill unneeded function parse_log_counts
Message-ID: <20110218125448.2249C2A202D@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42158:98d1401a7c2e
Date: 2011-02-18 13:24 +0100
http://bitbucket.org/pypy/pypy/changeset/98d1401a7c2e/

Log:	kill unneeded function parse_log_counts


diff --git a/pypy/tool/jitlogparser/test/test_parser.py b/pypy/tool/jitlogparser/test/test_parser.py
--- a/pypy/tool/jitlogparser/test/test_parser.py
+++ b/pypy/tool/jitlogparser/test/test_parser.py
@@ -2,7 +2,7 @@
 from pypy.jit.metainterp.history import ConstInt, Const
 from pypy.tool.jitlogparser.parser import parse, Bytecode, Function,\
      slice_debug_merge_points,\
-     adjust_bridges, parse_log_counts, cssclass
+     adjust_bridges, cssclass
 from pypy.tool.jitlogparser.storage import LoopStorage
 import py
 
@@ -178,30 +178,6 @@
     chunk = ops.chunks[0]
     assert chunk.bytecode_name == 'StrLiteralSearch'
 
-LINES = '''
-0:3
-1:3
-2:604
-3:396
-4:102
-5:2000
-6:3147
-7:2445
-8:2005
-9:2000
-10:1420
-11:40
-12:0
-'''.split("\n")
-
-def test_parse_log_count():
-    class Loop(object):
-        pass
-    
-    loops = [Loop() for i in range(13)]
-    nums = parse_log_counts(LINES, loops)
-    assert nums[5] == 2000
-    assert loops[9].count == 2000
 
 def test_highlight_var():
     ops = parse('''

diff --git a/pypy/tool/jitlogparser/parser.py b/pypy/tool/jitlogparser/parser.py
--- a/pypy/tool/jitlogparser/parser.py
+++ b/pypy/tool/jitlogparser/parser.py
@@ -290,22 +290,6 @@
                 print >>out, "  ", source
             chunk.pretty_print(out)
 
-def parse_log_counts(input, loops):
-    if not input:
-        return
-    lines = input[-1].splitlines()
-    nums = []
-    i = 0
-    for line in lines:
-        if line:
-            num, count = line.split(':')
-            assert int(num) == i
-            count = int(count)
-            nums.append(count)
-            loops[i].count = count
-            i += 1
-    return nums
-
 def parse(input):
     return SimpleParser(input, None, {}, 'lltype', None,
                         nonstrict=True).parse()

From commits-noreply at bitbucket.org  Fri Feb 18 13:54:49 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Fri, 18 Feb 2011 13:54:49 +0100 (CET)
Subject: [pypy-svn] pypy default: kill all the html-related features
Message-ID: <20110218125449.C0AAE282BE3@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42159:9a4c1a56401e
Date: 2011-02-18 13:31 +0100
http://bitbucket.org/pypy/pypy/changeset/9a4c1a56401e/

Log:	kill all the html-related features


diff --git a/pypy/tool/jitlogparser/test/test_parser.py b/pypy/tool/jitlogparser/test/test_parser.py
--- a/pypy/tool/jitlogparser/test/test_parser.py
+++ b/pypy/tool/jitlogparser/test/test_parser.py
@@ -1,8 +1,7 @@
 from pypy.jit.metainterp.resoperation import ResOperation, rop
 from pypy.jit.metainterp.history import ConstInt, Const
 from pypy.tool.jitlogparser.parser import parse, Bytecode, Function,\
-     slice_debug_merge_points,\
-     adjust_bridges, cssclass
+     slice_debug_merge_points, adjust_bridges
 from pypy.tool.jitlogparser.storage import LoopStorage
 import py
 
@@ -19,10 +18,9 @@
     assert ops[1].name == 'guard_true'
     assert ops[1].descr is not None
     assert ops[0].res == 'i9'
-    assert ops[0].html_repr().plaintext() == 'i9 = i7 < 1003'
+    assert ops[0].repr() == 'i9 = int_lt(i7, 1003)'
     assert ops[2].descr is not None
     assert len(ops[2].args) == 1
-    assert ops[2].html_repr().plaintext() == 'i13 = ((pypysig_long_struct)151937600).value'
 
 def test_parse_non_code():
     ops = parse('''
@@ -31,7 +29,7 @@
     ''')
     res = slice_debug_merge_points(ops.operations, LoopStorage())
     assert len(res.chunks) == 1
-    assert res.chunks[0].html_repr()
+    assert res.chunks[0].repr()
 
 def test_split():
     ops = parse('''
@@ -177,16 +175,3 @@
     ops = slice_debug_merge_points(loop.operations, LoopStorage())
     chunk = ops.chunks[0]
     assert chunk.bytecode_name == 'StrLiteralSearch'
-
-
-def test_highlight_var():
-    ops = parse('''
-    [p0]
-    guard_class(p0, 144264192, descr=<Guard0>)
-    ''').operations
-    assert len(ops) == 1
-    op = ops[0]
-    assert op.name == 'guard_class'
-    html = op.html_repr()
-    p0 = cssclass('p0', 'p0', onmouseover="highlight_var(this)", onmouseout="disable_var(this)")
-    assert p0 in html

diff --git a/pypy/tool/jitlogparser/parser.py b/pypy/tool/jitlogparser/parser.py
--- a/pypy/tool/jitlogparser/parser.py
+++ b/pypy/tool/jitlogparser/parser.py
@@ -3,33 +3,6 @@
 from pypy.jit.metainterp.resoperation import rop, opname
 from pypy.jit.tool.oparser import OpParser
 
-class Html(str):
-    def __html__(self):
-        return self
-
-    def plaintext(self):
-        # This is not a general way to strip tags, but it's good enough to use
-        # in tests
-        import re
-        s = re.sub('<.*?>', '', self)
-        s = s.replace("&lt;", "<")
-        s = s.replace("&gt;", ">")
-        s = s.replace("&amp;", "&")
-        return s
-
-
-def cssclass(cls, s, **kwds):
-    attrs = ['%s="%s"' % (name, value) for name, value in kwds.iteritems()]
-    return '<span class="%s" %s>%s</span>' % (cls, ' '.join(attrs), s)
-
-
-def _new_binop(name):
-    import cgi
-    name = cgi.escape(name)
-    def f(self):
-        return '%s = %s %s %s' % (self.getres(), self.getarg(0), name, self.getarg(1))
-    return f
-
 class Op(object):
     bridge = None
     
@@ -45,12 +18,6 @@
     def setfailargs(self, _):
         pass
 
-    def html_repr(self):
-        s = getattr(self, 'repr_' + self.name, self.generic_repr)()
-        if self.is_guard():
-            s = '<span class="guard">guard</span>(' + s + ')'
-        return Html(s)
-
     def getarg(self, i):
         return self._getvar(self.args[i])
 
@@ -61,69 +28,12 @@
         return self._getvar(self.res)
 
     def _getvar(self, v):
-        return cssclass(v, v, onmouseover='highlight_var(this)', onmouseout='disable_var(this)')
+        return v
 
     def is_guard(self):
         return self._is_guard
 
-    for bin_op, name in [('==', 'int_eq'),
-                         ('!=', 'int_ne'),
-                         ('==', 'float_eq'),
-                         ('!=', 'float_ne'),
-                         ('>', 'int_gt'),
-                         ('<', 'int_lt'),
-                         ('<=', 'int_le'),
-                         ('>=', 'int_ge'),
-                         ('+', 'int_add'),
-                         ('+', 'float_add'),
-                         ('-', 'int_sub'),
-                         ('-', 'float_sub'),
-                         ('*', 'int_mul'),
-                         ('*', 'float_mul'),
-                         ('&', 'int_and')]:
-        locals()['repr_' + name] = _new_binop(bin_op)
-
-    def repr_guard_true(self):
-        return '%s is true' % self.getarg(0)
-
-    def repr_guard_false(self):
-        return '%s is false' % self.getarg(0)
-
-    def repr_guard_value(self):
-        return '%s is %s' % (self.getarg(0), self.getarg(1))
-
-    def repr_guard_isnull(self):
-        return '%s is null' % self.getarg(0)
-
-    def repr_getfield_raw(self):
-        name, field = self.descr.split(' ')[1].rsplit('.', 1)
-        return '%s = ((%s)%s).%s' % (self.getres(), name, self.getarg(0), field[2:])
-
-    def repr_getfield_gc(self):
-        fullname, field = self.descr.split(' ')[1].rsplit('.', 1)
-        names = fullname.rsplit('.', 1)
-        if len(names) == 2:
-            namespace, classname = names
-        else:
-            namespace = ''
-            classname = names[0]
-        namespace = cssclass('namespace', namespace)
-        classname = cssclass('classname', classname)
-        field = cssclass('fieldname', field)
-            
-        obj = self.getarg(0)
-        return '%s = ((%s.%s)%s).%s' % (self.getres(), namespace, classname, obj, field)
-    repr_getfield_gc_pure = repr_getfield_gc
-
-    def repr_setfield_raw(self):
-        name, field = self.descr.split(' ')[1].rsplit('.', 1)
-        return '((%s)%s).%s = %s' % (name, self.getarg(0), field[2:], self.getarg(1))
-
-    def repr_setfield_gc(self):
-        name, field = self.descr.split(' ')[1].rsplit('.', 1)
-        return '((%s)%s).%s = %s' % (name, self.getarg(0), field, self.getarg(1))
-
-    def generic_repr(self):
+    def repr(self):
         arglist = ', '.join(self.getargs())
         if self.res is not None:
             return '%s = %s(%s)' % (self.getres(), self.name, arglist)
@@ -209,14 +119,6 @@
     def pretty_print(self, out):
         pass
 
-    def html_repr(self):
-        if self.filename is not None:
-            code = self.getcode()
-            opcode = self.code.map[self.bytecode_no]
-            return '%s %s' % (self.bytecode_name, opcode.argstr)
-        else:
-            return self.bytecode_name
-
 class Function(object):
     filename = None
     name = None
@@ -266,9 +168,6 @@
             maxline = 0
         self._linerange = minline, maxline
 
-    def html_repr(self):
-        return "inlined call to %s in %s" % (self.name, self.filename)
-
     def repr(self):
         if self.filename is None:
             return "Unknown"

From commits-noreply at bitbucket.org  Fri Feb 18 13:54:53 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Fri, 18 Feb 2011 13:54:53 +0100 (CET)
Subject: [pypy-svn] pypy default: import these two test files from jitviewer
Message-ID: <20110218125453.07468282C23@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42160:cb3a7879df1d
Date: 2011-02-18 13:34 +0100
http://bitbucket.org/pypy/pypy/changeset/cb3a7879df1d/

Log:	import these two test files from jitviewer

diff --git a/pypy/tool/jitlogparser/test/test_modulefinder.py b/pypy/tool/jitlogparser/test/test_modulefinder.py
new file mode 100644
--- /dev/null
+++ b/pypy/tool/jitlogparser/test/test_modulefinder.py
@@ -0,0 +1,20 @@
+import py
+from pypy.tool.jitlogparser.module_finder import gather_all_code_objs
+import re, sys
+
+def setup_module(mod):
+    if sys.version_info[:2] != (2.6):
+        py.test.skip("Specific python 2.6 tests")
+
+def test_gather_code_py():
+    fname = re.__file__
+    codes = gather_all_code_objs(fname)
+    assert len(codes) == 21
+    assert sorted(codes.keys()) == [102, 134, 139, 144, 153, 164, 169, 181, 188, 192, 197, 206, 229, 251, 266, 271, 277, 285, 293, 294, 308]
+
+def test_load_code():
+    fname = re.__file__
+    code = gather_all_code_objs(fname)[144]
+    assert code.co_name == 'sub'
+    assert code.co_filename == '/usr/lib/python2.6/re.py'
+    assert code.co_firstlineno == 144


diff --git a/pypy/tool/jitlogparser/test/test_storage.py b/pypy/tool/jitlogparser/test/test_storage.py
new file mode 100644
--- /dev/null
+++ b/pypy/tool/jitlogparser/test/test_storage.py
@@ -0,0 +1,8 @@
+import py
+from pypy.tool.jitlogparser.storage import LoopStorage
+
+def test_load_codes():
+    tmppath = py.test.ensuretemp('load_codes')
+    tmppath.join("x.py").write("def f(): pass") # one code
+    s = LoopStorage(str(tmppath))
+    assert s.load_code(str(tmppath.join('x.py'))) == s.load_code('x.py')

From commits-noreply at bitbucket.org  Fri Feb 18 13:54:54 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Fri, 18 Feb 2011 13:54:54 +0100 (CET)
Subject: [pypy-svn] pypy default: (in-progress) start to integrate the new
	jitlogparser	with test_pypy_c
Message-ID: <20110218125454.5EC472A202D@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42161:ec880fe41dc4
Date: 2011-02-18 13:53 +0100
http://bitbucket.org/pypy/pypy/changeset/ec880fe41dc4/

Log:	(in-progress) start to integrate the new jitlogparser with
	test_pypy_c

diff --git a/pypy/module/pypyjit/test_pypy_c/model.py b/pypy/module/pypyjit/test_pypy_c/model.py
--- a/pypy/module/pypyjit/test_pypy_c/model.py
+++ b/pypy/module/pypyjit/test_pypy_c/model.py
@@ -1,7 +1,8 @@
 import py
 import re
 from lib_pypy import disassembler
-from pypy.jit.tool import oparser
+from pypy.tool.jitlogparser.parser import parse, slice_debug_merge_points
+from pypy.tool.jitlogparser.storage import LoopStorage
 
 class Log(object):
     def __init__(self, func, rawtraces):
@@ -47,7 +48,10 @@
 class Trace(object):
     def __init__(self, rawtrace, chunks):
         # "low level trace", i.e. an instance of history.TreeLoop
-        self.lltrace = oparser.parse(rawtrace, no_namespace=True)
+        self.lltrace = parse(rawtrace)
+        storage = LoopStorage()
+        function = slice_debug_merge_points(self.lltrace.operations, storage)
+        import pdb;pdb.set_trace()
         self.split_into_opcodes()
 
     def split_into_opcodes(self):


From commits-noreply at bitbucket.org  Fri Feb 18 14:03:36 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri, 18 Feb 2011 14:03:36 +0100 (CET)
Subject: [pypy-svn] pypy default: Close the file explicitely to let the test
	pass on	Windows
Message-ID: <20110218130336.2D0F0282BE3@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r42162:c61667a961ac
Date: 2011-02-18 13:35 +0100
http://bitbucket.org/pypy/pypy/changeset/c61667a961ac/

Log:	Close the file explicitely to let the test pass on Windows

diff --git a/lib-python/modified-2.7.0/test/test_mmap.py b/lib-python/modified-2.7.0/test/test_mmap.py
--- a/lib-python/modified-2.7.0/test/test_mmap.py
+++ b/lib-python/modified-2.7.0/test/test_mmap.py
@@ -582,7 +582,8 @@
             m.close()
 
             # Should not crash (Issue 5385)
-            open(TESTFN, "wb").write("x"*10)
+            with open(TESTFN, "wb") as f:
+                f.write("x"*10)
             f = open(TESTFN, "r+b")
             m = mmap.mmap(f.fileno(), 0)
             f.close()

From commits-noreply at bitbucket.org  Fri Feb 18 14:03:36 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri, 18 Feb 2011 14:03:36 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix mmap test on Windows: offset must be a
	multiple of	allocation granularity
Message-ID: <20110218130336.B9C40282BE3@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r42163:d97f79177ee8
Date: 2011-02-18 13:42 +0100
http://bitbucket.org/pypy/pypy/changeset/d97f79177ee8/

Log:	Fix mmap test on Windows: offset must be a multiple of allocation
	granularity

diff --git a/pypy/module/mmap/test/test_mmap.py b/pypy/module/mmap/test/test_mmap.py
--- a/pypy/module/mmap/test/test_mmap.py
+++ b/pypy/module/mmap/test/test_mmap.py
@@ -519,13 +519,15 @@
         assert b[:] == "foobar"
 
     def test_offset(self):
-        from mmap import mmap
+        from mmap import mmap, ALLOCATIONGRANULARITY
         f = open(self.tmpname + "y", "w+")
-        f.write("foobar" * 3000)
+        f.write("foobar" * ALLOCATIONGRANULARITY)
         f.flush()
-        m = mmap(f.fileno(), 4, offset=8192)
-        assert m[:] == "obar"
-        assert len(m) == 4
+        size = ALLOCATIONGRANULARITY
+        offset = 2 * ALLOCATIONGRANULARITY
+        m = mmap(f.fileno(), size, offset=offset)
+        assert m[:] == ("foobar" * ALLOCATIONGRANULARITY)[offset:offset+size]
+        assert len(m) == size
         m.close()
         f.close()
 

From commits-noreply at bitbucket.org  Fri Feb 18 14:03:39 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri, 18 Feb 2011 14:03:39 +0100 (CET)
Subject: [pypy-svn] pypy default: write mmap constants at interp-level
Message-ID: <20110218130339.47935282C1A@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r42164:6f34d2c9c7b9
Date: 2011-02-18 13:46 +0100
http://bitbucket.org/pypy/pypy/changeset/6f34d2c9c7b9/

Log:	write mmap constants at interp-level

diff --git a/pypy/module/mmap/interp_mmap.py b/pypy/module/mmap/interp_mmap.py
--- a/pypy/module/mmap/interp_mmap.py
+++ b/pypy/module/mmap/interp_mmap.py
@@ -264,6 +264,9 @@
 constants = rmmap.constants
 PAGESIZE = rmmap.PAGESIZE
 ALLOCATIONGRANULARITY = rmmap.ALLOCATIONGRANULARITY
+ACCESS_READ  = rmmap.ACCESS_READ
+ACCESS_WRITE = rmmap.ACCESS_WRITE
+ACCESS_COPY  = rmmap.ACCESS_COPY
 
 def mmap_error(space, e):
     w_module = space.getbuiltinmodule('mmap')

diff --git a/pypy/module/mmap/app_mmap.py b/pypy/module/mmap/app_mmap.py
--- a/pypy/module/mmap/app_mmap.py
+++ b/pypy/module/mmap/app_mmap.py
@@ -1,6 +1,2 @@
-ACCESS_READ = 1
-ACCESS_WRITE = 2
-ACCESS_COPY = 3
-
 class error(EnvironmentError):
     pass

diff --git a/pypy/module/mmap/__init__.py b/pypy/module/mmap/__init__.py
--- a/pypy/module/mmap/__init__.py
+++ b/pypy/module/mmap/__init__.py
@@ -5,13 +5,13 @@
     interpleveldefs = {
         'PAGESIZE': 'space.wrap(interp_mmap.PAGESIZE)',
         'ALLOCATIONGRANULARITY': 'space.wrap(interp_mmap.ALLOCATIONGRANULARITY)',
+        'ACCESS_READ' : 'space.wrap(interp_mmap.ACCESS_READ)',
+        'ACCESS_WRITE': 'space.wrap(interp_mmap.ACCESS_WRITE)',
+        'ACCESS_COPY' : 'space.wrap(interp_mmap.ACCESS_COPY)',
         'mmap': 'interp_mmap.W_MMap'
     }
 
     appleveldefs = {
-        'ACCESS_READ': 'app_mmap.ACCESS_READ',
-        'ACCESS_WRITE': 'app_mmap.ACCESS_WRITE',
-        'ACCESS_COPY': 'app_mmap.ACCESS_COPY',
         'error': 'app_mmap.error'
     }
     

From commits-noreply at bitbucket.org  Fri Feb 18 14:03:39 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri, 18 Feb 2011 14:03:39 +0100 (CET)
Subject: [pypy-svn] pypy interplevel-exception-classes: merge default
Message-ID: <20110218130339.9CD4E282C1B@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: interplevel-exception-classes
Changeset: r42165:8df6f54dd5a4
Date: 2011-02-18 13:48 +0100
http://bitbucket.org/pypy/pypy/changeset/8df6f54dd5a4/

Log:	merge default


From commits-noreply at bitbucket.org  Fri Feb 18 14:03:41 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri, 18 Feb 2011 14:03:41 +0100 (CET)
Subject: [pypy-svn] pypy interplevel-exception-classes: mmap.error at
	interp-level
Message-ID: <20110218130341.C40952A2039@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: interplevel-exception-classes
Changeset: r42166:c9dae975baef
Date: 2011-02-18 13:53 +0100
http://bitbucket.org/pypy/pypy/changeset/c9dae975baef/

Log:	mmap.error at interp-level

diff --git a/pypy/module/mmap/interp_mmap.py b/pypy/module/mmap/interp_mmap.py
--- a/pypy/module/mmap/interp_mmap.py
+++ b/pypy/module/mmap/interp_mmap.py
@@ -268,7 +268,11 @@
 ACCESS_WRITE = rmmap.ACCESS_WRITE
 ACCESS_COPY  = rmmap.ACCESS_COPY
 
+class Cache:
+    def __init__(self, space):
+        self.w_error = space.new_exception_class("mmap.error",
+                                                 space.w_EnvironmentError)
+
 def mmap_error(space, e):
-    w_module = space.getbuiltinmodule('mmap')
-    w_error = space.getattr(w_module, space.wrap('error'))
+    w_error = space.fromcache(Cache).w_error
     return wrap_oserror(space, e, w_exception_class=w_error)

diff --git a/pypy/module/mmap/app_mmap.py b/pypy/module/mmap/app_mmap.py
deleted file mode 100644
--- a/pypy/module/mmap/app_mmap.py
+++ /dev/null
@@ -1,2 +0,0 @@
-class error(EnvironmentError):
-    pass

diff --git a/pypy/module/mmap/__init__.py b/pypy/module/mmap/__init__.py
--- a/pypy/module/mmap/__init__.py
+++ b/pypy/module/mmap/__init__.py
@@ -8,11 +8,11 @@
         'ACCESS_READ' : 'space.wrap(interp_mmap.ACCESS_READ)',
         'ACCESS_WRITE': 'space.wrap(interp_mmap.ACCESS_WRITE)',
         'ACCESS_COPY' : 'space.wrap(interp_mmap.ACCESS_COPY)',
-        'mmap': 'interp_mmap.W_MMap'
+        'mmap': 'interp_mmap.W_MMap',
+        'error': 'space.fromcache(interp_mmap.Cache).w_error',
     }
 
     appleveldefs = {
-        'error': 'app_mmap.error'
     }
     
     def buildloaders(cls):

From commits-noreply at bitbucket.org  Fri Feb 18 14:03:44 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri, 18 Feb 2011 14:03:44 +0100 (CET)
Subject: [pypy-svn] pypy interplevel-exception-classes:
	zipimport.ZipImportError at	interp-level
Message-ID: <20110218130344.997602A2030@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: interplevel-exception-classes
Changeset: r42167:af791a1b3462
Date: 2011-02-18 14:00 +0100
http://bitbucket.org/pypy/pypy/changeset/af791a1b3462/

Log:	zipimport.ZipImportError at interp-level

diff --git a/pypy/module/zipimport/app_zipimport.py b/pypy/module/zipimport/app_zipimport.py
deleted file mode 100644
--- a/pypy/module/zipimport/app_zipimport.py
+++ /dev/null
@@ -1,4 +0,0 @@
-
-class ZipImportError(ImportError):
-    pass
-

diff --git a/pypy/module/zipimport/__init__.py b/pypy/module/zipimport/__init__.py
--- a/pypy/module/zipimport/__init__.py
+++ b/pypy/module/zipimport/__init__.py
@@ -8,11 +8,11 @@
 
     interpleveldefs = {
         'zipimporter':'interp_zipimport.W_ZipImporter',
-        '_zip_directory_cache' : 'space.wrap(interp_zipimport.zip_cache)'
+        '_zip_directory_cache' : 'space.wrap(interp_zipimport.zip_cache)',
+        'ZipImportError': 'space.fromcache(interp_zipimport.Cache).w_error',
     }
 
     appleveldefs = {
-        'ZipImportError'      : 'app_zipimport.ZipImportError',
     }
 
     def setup_after_space_initialization(self):

diff --git a/pypy/module/zipimport/interp_zipimport.py b/pypy/module/zipimport/interp_zipimport.py
--- a/pypy/module/zipimport/interp_zipimport.py
+++ b/pypy/module/zipimport/interp_zipimport.py
@@ -22,6 +22,14 @@
      (True, False, '.pyo'),
      (False, False, '.py')])
 
+class Cache:
+    def __init__(self, space):
+        self.w_error = space.new_exception_class("zipimport.ZipImportError",
+                                                 space.w_ImportError)
+
+def get_error(space):
+    return space.fromcache(Cache).w_error
+
 class W_ZipCache(Wrappable):
     def __init__(self):
         self.cache = {}
@@ -113,9 +121,6 @@
         self.filename = filename
         self.dir = dir
         self.prefix = prefix
-        self.w_ZipImportError = space.getattr(
-            space.getbuiltinmodule('zipimport'),
-            space.wrap('ZipImportError'))
 
     def getprefix(self, space):
         return space.wrap(self.prefix)
@@ -265,7 +270,7 @@
                     w_mods = space.sys.get('modules')
                 space.call_method(w_mods, 'pop', w(fullname), space.w_None)
         if last_exc:
-            raise OperationError(self.w_ZipImportError, last_exc.get_w_value(space))
+            raise OperationError(get_error(space), last_exc.get_w_value(space))
         # should never happen I think
         return space.w_None
 
@@ -303,7 +308,7 @@
                     code_w = importing.parse_source_module(
                         space, co_filename, source)
                 return space.wrap(code_w)
-        raise operationerrfmt(self.w_ZipImportError,
+        raise operationerrfmt(get_error(space),
             "Cannot find source or code for %s in %s", filename, self.name)
 
     @unwrap_spec(fullname=str)
@@ -319,7 +324,7 @@
                     found = True
         if found:
             return space.w_None
-        raise operationerrfmt(self.w_ZipImportError,
+        raise operationerrfmt(get_error(space),
             "Cannot find source for %s in %s", filename, self.name)
 
     @unwrap_spec(fullname=str)
@@ -329,7 +334,7 @@
             if self.have_modulefile(space, filename + ext):
                 return space.wrap(self.filename + os.path.sep +
                                   self.corr_zname(filename + ext))
-        raise operationerrfmt(self.w_ZipImportError,
+        raise operationerrfmt(get_error(space),
             "Cannot find module %s in %s", filename, self.name)
 
     @unwrap_spec(fullname=str)
@@ -338,7 +343,7 @@
         for _, is_package, ext in ENUMERATE_EXTS:
             if self.have_modulefile(space, filename + ext):
                 return space.wrap(is_package)
-        raise operationerrfmt(self.w_ZipImportError,
+        raise operationerrfmt(get_error(space),
             "Cannot find module %s in %s", filename, self.name)
 
     def getarchive(self, space):

From commits-noreply at bitbucket.org  Fri Feb 18 14:03:44 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri, 18 Feb 2011 14:03:44 +0100 (CET)
Subject: [pypy-svn] pypy default: Merge heads
Message-ID: <20110218130344.F0E97282C18@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r42168:84321e49b3c0
Date: 2011-02-18 14:02 +0100
http://bitbucket.org/pypy/pypy/changeset/84321e49b3c0/

Log:	Merge heads


From commits-noreply at bitbucket.org  Fri Feb 18 14:24:35 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri, 18 Feb 2011 14:24:35 +0100 (CET)
Subject: [pypy-svn] pypy interplevel-exception-classes:
	signal.default_int_handler at	interp-level
Message-ID: <20110218132435.EE51F282BE3@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: interplevel-exception-classes
Changeset: r42169:bbda366711b8
Date: 2011-02-18 14:12 +0100
http://bitbucket.org/pypy/pypy/changeset/bbda366711b8/

Log:	signal.default_int_handler at interp-level

diff --git a/pypy/module/signal/__init__.py b/pypy/module/signal/__init__.py
--- a/pypy/module/signal/__init__.py
+++ b/pypy/module/signal/__init__.py
@@ -11,6 +11,7 @@
         'NSIG':                'space.wrap(interp_signal.NSIG)',
         'SIG_DFL':             'space.wrap(interp_signal.SIG_DFL)',
         'SIG_IGN':             'space.wrap(interp_signal.SIG_IGN)',
+        'default_int_handler': 'interp_signal.default_int_handler',
     }
 
     if os.name == 'posix':
@@ -25,7 +26,6 @@
             interpleveldefs[name] = 'space.wrap(interp_signal.%s)' % (name,)
 
     appleveldefs = {
-        'default_int_handler': 'app_signal.default_int_handler',
         'ItimerError': 'app_signal.ItimerError',
     }
 

diff --git a/pypy/module/signal/app_signal.py b/pypy/module/signal/app_signal.py
--- a/pypy/module/signal/app_signal.py
+++ b/pypy/module/signal/app_signal.py
@@ -1,14 +1,2 @@
-
-
-def default_int_handler(signum, frame):
-    """
-    default_int_handler(...)
-
-    The default handler for SIGINT installed by Python.
-    It raises KeyboardInterrupt.
-    """
-    raise KeyboardInterrupt()
-
-
 class ItimerError(IOError):
-    pass
\ No newline at end of file
+    pass

diff --git a/pypy/module/signal/interp_signal.py b/pypy/module/signal/interp_signal.py
--- a/pypy/module/signal/interp_signal.py
+++ b/pypy/module/signal/interp_signal.py
@@ -208,6 +208,16 @@
         return action.handlers_w[signum]
     return space.wrap(SIG_DFL)
 
+def default_int_handler(space, w_signum, w_frame):
+    """
+    default_int_handler(...)
+
+    The default handler for SIGINT installed by Python.
+    It raises KeyboardInterrupt.
+    """
+    raise OperationError(space.w_KeyboardInterrupt,
+                         space.w_None)
+
 @jit.dont_look_inside
 @unwrap_spec(timeout=int)
 def alarm(space, timeout):

From commits-noreply at bitbucket.org  Fri Feb 18 14:24:37 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri, 18 Feb 2011 14:24:37 +0100 (CET)
Subject: [pypy-svn] pypy interplevel-exception-classes: signal.ItimerError
	at interp_level
Message-ID: <20110218132437.01D61282BE3@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: interplevel-exception-classes
Changeset: r42170:1426ed71866b
Date: 2011-02-18 14:22 +0100
http://bitbucket.org/pypy/pypy/changeset/1426ed71866b/

Log:	signal.ItimerError at interp_level

diff --git a/pypy/module/signal/__init__.py b/pypy/module/signal/__init__.py
--- a/pypy/module/signal/__init__.py
+++ b/pypy/module/signal/__init__.py
@@ -12,6 +12,7 @@
         'SIG_DFL':             'space.wrap(interp_signal.SIG_DFL)',
         'SIG_IGN':             'space.wrap(interp_signal.SIG_IGN)',
         'default_int_handler': 'interp_signal.default_int_handler',
+        'ItimerError':         'interp_signal.get_itimer_error(space)',
     }
 
     if os.name == 'posix':
@@ -26,7 +27,6 @@
             interpleveldefs[name] = 'space.wrap(interp_signal.%s)' % (name,)
 
     appleveldefs = {
-        'ItimerError': 'app_signal.ItimerError',
     }
 
     def buildloaders(cls):

diff --git a/pypy/module/signal/app_signal.py b/pypy/module/signal/app_signal.py
deleted file mode 100644
--- a/pypy/module/signal/app_signal.py
+++ /dev/null
@@ -1,2 +0,0 @@
-class ItimerError(IOError):
-    pass

diff --git a/pypy/module/signal/interp_signal.py b/pypy/module/signal/interp_signal.py
--- a/pypy/module/signal/interp_signal.py
+++ b/pypy/module/signal/interp_signal.py
@@ -313,9 +313,13 @@
     w_interval = space.wrap(double_from_timeval(val.c_it_interval))
     return space.newtuple([w_value, w_interval])
 
+class Cache:
+    def __init__(self, space):
+        self.w_itimererror = space.new_exception_class("signal.ItimerError",
+                                                       space.w_IOError)
+
 def get_itimer_error(space):
-    mod = space.getbuiltinmodule("signal")
-    return space.getattr(mod, space.wrap("ItimerError"))
+    return space.fromcache(Cache).w_itimererror
 
 @jit.dont_look_inside
 @unwrap_spec(which=int, first=float, interval=float)

From commits-noreply at bitbucket.org  Fri Feb 18 17:16:10 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri, 18 Feb 2011 17:16:10 +0100 (CET)
Subject: [pypy-svn] pypy interplevel-exception-classes: Run tests, and fix.
Message-ID: <20110218161610.810E9282BE3@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: interplevel-exception-classes
Changeset: r42171:f0de43c98c04
Date: 2011-02-18 17:15 +0100
http://bitbucket.org/pypy/pypy/changeset/f0de43c98c04/

Log:	Run tests, and fix.

diff --git a/pypy/module/termios/interp_termios.py b/pypy/module/termios/interp_termios.py
--- a/pypy/module/termios/interp_termios.py
+++ b/pypy/module/termios/interp_termios.py
@@ -12,7 +12,7 @@
 import termios
 
 class Cache:
-    def __init__(self):
+    def __init__(self, space):
         self.w_error = space.new_exception_class("termios.error")
 
 def convert_error(space, error):

From commits-noreply at bitbucket.org  Fri Feb 18 17:57:19 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri, 18 Feb 2011 17:57:19 +0100 (CET)
Subject: [pypy-svn] pypy default: Adapt the test for Linux
Message-ID: <20110218165719.3D9072A202D@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r42172:111e770bec9e
Date: 2011-02-18 17:56 +0100
http://bitbucket.org/pypy/pypy/changeset/111e770bec9e/

Log:	Adapt the test for Linux

diff --git a/pypy/module/_file/test/test_file_extra.py b/pypy/module/_file/test/test_file_extra.py
--- a/pypy/module/_file/test/test_file_extra.py
+++ b/pypy/module/_file/test/test_file_extra.py
@@ -522,10 +522,14 @@
 
         import errno, sys
         f = open(fn)
-        exc = raises(OSError, f.truncate, 3)
-        assert exc.value.errno == errno.EACCES
+        exc = raises(EnvironmentError, f.truncate, 3)
         if sys.platform == 'win32':
             assert exc.value.winerror == 5 # ERROR_ACCESS_DENIED
+        else:
+            # CPython explicitely checks the file mode
+            # PyPy relies on the libc to raise the error
+            assert (exc.value.message == "File not open for writing" or
+                    exc.value.errno == errno.EINVAL)
         f.close()
 
     def test_readinto(self):

From commits-noreply at bitbucket.org  Fri Feb 18 18:30:53 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Fri, 18 Feb 2011 18:30:53 +0100 (CET)
Subject: [pypy-svn] pypy default: make slice_debug_merge_points a
	classmethod	(alternative constructor) of Function
Message-ID: <20110218173053.B6AA1282C1A@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42173:8da3130dcb94
Date: 2011-02-18 16:07 +0100
http://bitbucket.org/pypy/pypy/changeset/8da3130dcb94/

Log:	make slice_debug_merge_points a classmethod (alternative
	constructor) of Function

diff --git a/pypy/tool/jitlogparser/test/test_parser.py b/pypy/tool/jitlogparser/test/test_parser.py
--- a/pypy/tool/jitlogparser/test/test_parser.py
+++ b/pypy/tool/jitlogparser/test/test_parser.py
@@ -1,7 +1,7 @@
 from pypy.jit.metainterp.resoperation import ResOperation, rop
 from pypy.jit.metainterp.history import ConstInt, Const
 from pypy.tool.jitlogparser.parser import parse, Bytecode, Function,\
-     slice_debug_merge_points, adjust_bridges
+     adjust_bridges
 from pypy.tool.jitlogparser.storage import LoopStorage
 import py
 
@@ -27,7 +27,7 @@
     []
     debug_merge_point("SomeRandomStuff", 0)
     ''')
-    res = slice_debug_merge_points(ops.operations, LoopStorage())
+    res = Function.from_operations(ops.operations, LoopStorage())
     assert len(res.chunks) == 1
     assert res.chunks[0].repr()
 
@@ -40,7 +40,7 @@
     debug_merge_point("<code object stuff, file '/tmp/x.py', line 200> #11 SUB", 0)
     i2 = int_add(i1, 1)
     ''')
-    res = slice_debug_merge_points(ops.operations, LoopStorage())
+    res = Function.from_operations(ops.operations, LoopStorage())
     assert len(res.chunks) == 3
     assert len(res.chunks[0].operations) == 1
     assert len(res.chunks[1].operations) == 2
@@ -57,7 +57,7 @@
     debug_merge_point('<code object inner, file 'source.py', line 9> #7 RETURN_VALUE', 1)
     debug_merge_point('<code object inlined_call, file 'source.py', line 12> #31 STORE_FAST', 0)
     """)
-    res = slice_debug_merge_points(ops.operations, LoopStorage())
+    res = Function.from_operations(ops.operations, LoopStorage())
     assert len(res.chunks) == 3 # two chunks + inlined call
     assert isinstance(res.chunks[0], Bytecode)
     assert isinstance(res.chunks[1], Function)
@@ -74,7 +74,7 @@
     debug_merge_point("<code object stuff, file '/tmp/x.py', line 202> #11 SUB", 0)
     i2 = int_add(i1, 1)
     ''')
-    res = slice_debug_merge_points(ops.operations, LoopStorage())
+    res = Function.from_operations(ops.operations, LoopStorage())
     assert res.repr() == res.chunks[0].repr()
     assert res.repr() == "stuff, file '/tmp/x.py', line 200"
     assert res.startlineno == 200
@@ -91,7 +91,7 @@
     debug_merge_point("<code object stuff, file '/tmp/x.py', line 202> #11 SUB", 0)
     i2 = int_add(i1, 1)
     ''')
-    res = slice_debug_merge_points(ops.operations, LoopStorage())
+    res = Function.from_operations(ops.operations, LoopStorage())
     assert res.repr() == res.chunks[1].repr()
 
 def test_lineno():
@@ -103,7 +103,7 @@
     debug_merge_point("<code object f, file '%(fname)s', line 2> #6 BINARY_ADD", 0)
     debug_merge_point("<code object f, file '%(fname)s', line 2> #7 RETURN_VALUE", 0)
     ''' % locals())
-    res = slice_debug_merge_points(ops.operations, LoopStorage())
+    res = Function.from_operations(ops.operations, LoopStorage())
     assert res.chunks[1].lineno == 3
 
 def test_linerange():
@@ -116,7 +116,7 @@
     debug_merge_point("<code object f, file '%(fname)s', line 5> #28 LOAD_CONST", 0)
     debug_merge_point("<code object f, file '%(fname)s', line 5> #6 SETUP_LOOP", 0)
     ''' % locals())
-    res = slice_debug_merge_points(ops.operations, LoopStorage())
+    res = Function.from_operations(ops.operations, LoopStorage())
     assert res.linerange == (7, 9)
     assert res.lineset == set([7, 8, 9])
 
@@ -128,7 +128,7 @@
     guard_class(p6, 144264192, descr=<Guard2>)
     p12 = getfield_gc(p6, descr=<GcPtrFieldDescr pypy.objspace.std.iterobject.W_AbstractSeqIterObject.inst_w_seq 12>)
     """ % locals())
-    res = slice_debug_merge_points(ops.operations, LoopStorage())
+    res = Function.from_operations(ops.operations, LoopStorage())
     assert res.lineset
 
 def test_reassign_loops():
@@ -172,6 +172,6 @@
     loop = parse("""
     debug_merge_point('StrLiteralSearch at 11/51 [17, 8, 3, 1, 1, 1, 1, 51, 0, 19, 51, 1]', 0)
     """)
-    ops = slice_debug_merge_points(loop.operations, LoopStorage())
+    ops = Function.from_operations(loop.operations, LoopStorage())
     chunk = ops.chunks[0]
     assert chunk.bytecode_name == 'StrLiteralSearch'

From commits-noreply at bitbucket.org  Fri Feb 18 18:30:54 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Fri, 18 Feb 2011 18:30:54 +0100 (CET)
Subject: [pypy-svn] pypy default: rename "chunks" to "ids",
	to avoid confusion with the "chunks" used by the parser
Message-ID: <20110218173054.7DBA6282C1A@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42174:9267807b5260
Date: 2011-02-18 16:42 +0100
http://bitbucket.org/pypy/pypy/changeset/9267807b5260/

Log:	rename "chunks" to "ids", to avoid confusion with the "chunks" used
	by the parser

diff --git a/pypy/module/pypyjit/test_pypy_c/model.py b/pypy/module/pypyjit/test_pypy_c/model.py
--- a/pypy/module/pypyjit/test_pypy_c/model.py
+++ b/pypy/module/pypyjit/test_pypy_c/model.py
@@ -10,13 +10,12 @@
         self.traces = [Trace(rawtrace, chunks) for rawtrace in rawtraces]
 
     @classmethod
-    def find_chunks_range(cls, func):
+    def find_ids_range(cls, func):
         """
-        Parse the given function and return a dictionary mapping "chunk
-        names" to "line ranges".  Chunks are identified by comments with a
-        special syntax::
+        Parse the given function and return a dictionary mapping "ids" to
+        "line ranges".  Ids are identified by comments with a special syntax::
 
-            # the chunk "myid" corresponds to the whole line
+            # "myid" corresponds to the whole line
             print 'foo' # ID: myid
         """
         result = {}
@@ -30,19 +29,19 @@
         return result
 
     @classmethod
-    def find_chunks(cls, func):
+    def find_ids(cls, func):
         """
-        Parse the given function and return a dictionary mapping "chunk names"
-        to "opcodes".
+        Parse the given function and return a dictionary mapping "ids" to
+        "opcodes".
         """
-        chunks = {}
+        ids = {}
         code = disassembler.dis(func)
-        ranges = cls.find_chunks_range(func)
+        ranges = cls.find_ids_range(func)
         for name, linerange in ranges.iteritems():
             opcodes = [opcode for opcode in code.opcodes
                        if opcode.lineno in linerange]
-            chunks[name] = opcodes
-        return chunks
+            ids[name] = opcodes
+        return ids
 
 
 class Trace(object):

diff --git a/pypy/module/pypyjit/test_pypy_c/test_model.py b/pypy/module/pypyjit/test_pypy_c/test_model.py
--- a/pypy/module/pypyjit/test_pypy_c/test_model.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_model.py
@@ -47,27 +47,27 @@
 
 class TestLog(object):
 
-    def test_find_chunks_range(self):
+    def test_find_ids_range(self):
         def f():
             a = 0 # ID: myline
             return a
         #
         start_lineno = f.func_code.co_firstlineno
-        ids = Log.find_chunks_range(f)
+        ids = Log.find_ids_range(f)
         assert len(ids) == 1
         myline_range = ids['myline']
         assert list(myline_range) == range(start_lineno+1, start_lineno+2)
 
-    def test_find_chunks(self):
+    def test_find_ids(self):
         def f():
             i = 0
             x = 0
             z = x + 3 # ID: myline
             return z
         #
-        chunks = Log.find_chunks(f)
-        assert len(chunks) == 1
-        myline = chunks['myline']
+        ids = Log.find_ids(f)
+        assert len(ids) == 1
+        myline = ids['myline']
         opcodes_names = [opcode.__class__.__name__ for opcode in myline]
         assert opcodes_names == ['LOAD_FAST', 'LOAD_CONST', 'BINARY_ADD', 'STORE_FAST']
 

From commits-noreply at bitbucket.org  Fri Feb 18 18:30:55 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Fri, 18 Feb 2011 18:30:55 +0100 (CET)
Subject: [pypy-svn] pypy default: this belongs to 93a209dfaa72
Message-ID: <20110218173055.0987E282C1A@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42175:f6b15c701d96
Date: 2011-02-18 16:43 +0100
http://bitbucket.org/pypy/pypy/changeset/f6b15c701d96/

Log:	this belongs to 93a209dfaa72

diff --git a/pypy/tool/jitlogparser/parser.py b/pypy/tool/jitlogparser/parser.py
--- a/pypy/tool/jitlogparser/parser.py
+++ b/pypy/tool/jitlogparser/parser.py
@@ -140,6 +140,51 @@
                 break
         self.storage = storage
 
+    @classmethod
+    def from_operations(cls, operations, storage, limit=None):
+        """ Slice given operation list into a chain of Bytecode chunks.
+        Also detect inlined functions and make them Function
+        """
+        stack = []
+
+        def getpath(stack):
+            return ",".join([str(len(v)) for v in stack])
+
+        def append_to_res(bc):
+            if not stack:
+                stack.append([])
+            else:
+                if bc.inline_level is not None and bc.inline_level + 1 != len(stack):
+                    if bc.inline_level < len(stack):
+                        last = stack.pop()
+                        stack[-1].append(cls(last, getpath(stack), storage))
+                    else:
+                        stack.append([])
+            stack[-1].append(bc)
+
+        so_far = []
+        stack = []
+        for op in operations:
+            if op.name == 'debug_merge_point':
+                if so_far:
+                    append_to_res(Bytecode(so_far, storage))
+                    if limit:
+                        break
+                    so_far = []
+            so_far.append(op)
+        if so_far:
+            append_to_res(Bytecode(so_far, storage))
+        # wrap stack back up
+        if not stack:
+            # no ops whatsoever
+            return cls([], getpath(stack), storage)
+        while True:
+            next = stack.pop()
+            if not stack:
+                return cls(next, getpath(stack), storage)
+            stack[-1].append(cls(next, getpath(stack), storage))
+
+
     def getlinerange(self):
         if self._linerange is None:
             self._compute_linerange()
@@ -193,48 +238,6 @@
     return SimpleParser(input, None, {}, 'lltype', None,
                         nonstrict=True).parse()
 
-def slice_debug_merge_points(operations, storage, limit=None):
-    """ Slice given operation list into a chain of Bytecode chunks.
-    Also detect inlined functions and make them Function
-    """
-    stack = []
-
-    def getpath(stack):
-        return ",".join([str(len(v)) for v in stack])
-
-    def append_to_res(bc):
-        if not stack:
-            stack.append([])
-        else:
-            if bc.inline_level is not None and bc.inline_level + 1 != len(stack):
-                if bc.inline_level < len(stack):
-                    last = stack.pop()
-                    stack[-1].append(Function(last, getpath(stack), storage))
-                else:
-                    stack.append([])
-        stack[-1].append(bc)
-
-    so_far = []
-    stack = []
-    for op in operations:
-        if op.name == 'debug_merge_point':
-            if so_far:
-                append_to_res(Bytecode(so_far, storage))
-                if limit:
-                    break
-                so_far = []
-        so_far.append(op)
-    if so_far:
-        append_to_res(Bytecode(so_far, storage))
-    # wrap stack back up
-    if not stack:
-        # no ops whatsoever
-        return Function([], getpath(stack), storage)
-    while True:
-        next = stack.pop()
-        if not stack:
-            return Function(next, getpath(stack), storage)
-        stack[-1].append(Function(next, getpath(stack), storage))
 
 def adjust_bridges(loop, bridges):
     """ Slice given loop according to given bridges to follow. Returns a plain

From commits-noreply at bitbucket.org  Fri Feb 18 18:30:57 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Fri, 18 Feb 2011 18:30:57 +0100 (CET)
Subject: [pypy-svn] pypy default: use the jitlogparser to parse the log and
	produce	structured descriptions of the loops,
	which can be easiliy used by the tests
Message-ID: <20110218173057.4C216282C23@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42176:7070fe1ed663
Date: 2011-02-18 16:52 +0100
http://bitbucket.org/pypy/pypy/changeset/7070fe1ed663/

Log:	use the jitlogparser to parse the log and produce structured
	descriptions of the loops, which can be easiliy used by the tests

diff --git a/pypy/module/pypyjit/test_pypy_c/model.py b/pypy/module/pypyjit/test_pypy_c/model.py
--- a/pypy/module/pypyjit/test_pypy_c/model.py
+++ b/pypy/module/pypyjit/test_pypy_c/model.py
@@ -1,13 +1,17 @@
 import py
 import re
+import os.path
 from lib_pypy import disassembler
-from pypy.tool.jitlogparser.parser import parse, slice_debug_merge_points
+from pypy.tool.jitlogparser.parser import parse, Function
 from pypy.tool.jitlogparser.storage import LoopStorage
 
 class Log(object):
     def __init__(self, func, rawtraces):
-        chunks = self.find_chunks(func)
-        self.traces = [Trace(rawtrace, chunks) for rawtrace in rawtraces]
+        storage = LoopStorage()
+        storage.ids = self.find_ids(func)
+        traces = [parse(rawtrace) for rawtrace in rawtraces]
+        traces = storage.reconnect_loops(traces)
+        self.functions = [FunctionWithIds.from_trace(trace, storage) for trace in traces]
 
     @classmethod
     def find_ids_range(cls, func):
@@ -43,24 +47,11 @@
             ids[name] = opcodes
         return ids
 
+    def by_filename(self, filename):
+        return [func for func in self.functions if func.filename == filename]
 
-class Trace(object):
-    def __init__(self, rawtrace, chunks):
-        # "low level trace", i.e. an instance of history.TreeLoop
-        self.lltrace = parse(rawtrace)
-        storage = LoopStorage()
-        function = slice_debug_merge_points(self.lltrace.operations, storage)
-        import pdb;pdb.set_trace()
-        self.split_into_opcodes()
+class FunctionWithIds(Function):
 
-    def split_into_opcodes(self):
-        self.opcodes = []
-        for op in self.lltrace.operations:
-            if op.getopname() == "debug_merge_point":
-                opcode = TraceForOpcode(op) # XXX
-                self.opcodes.append(opcode)
-            else:
-                opcode.append(op)
-
-class TraceForOpcode(list):
-    pass
+    @classmethod
+    def from_trace(cls, trace, storage):
+        return cls.from_operations(trace.operations, storage)

diff --git a/pypy/module/pypyjit/test_pypy_c/test_model.py b/pypy/module/pypyjit/test_pypy_c/test_model.py
--- a/pypy/module/pypyjit/test_pypy_c/test_model.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_model.py
@@ -74,7 +74,6 @@
 class TestRunPyPyC(BaseTestPyPyC):
 
     def test_parse_jitlog(self):
-        py.test.skip('in-progress')
         def f():
             i = 0
             while i < 1003: # default threshold is 10
@@ -82,3 +81,6 @@
             return i
         #
         log = self.run(f)
+        loops = log.by_filename(self.filepath)
+        assert len(loops) == 2 # loop and entry bridge
+        

From commits-noreply at bitbucket.org  Fri Feb 18 18:30:59 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Fri, 18 Feb 2011 18:30:59 +0100 (CET)
Subject: [pypy-svn] pypy default: add a way to distinguish loops and entry
	bridges
Message-ID: <20110218173059.2D7412A202D@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42177:2b39e1dc17af
Date: 2011-02-18 17:09 +0100
http://bitbucket.org/pypy/pypy/changeset/2b39e1dc17af/

Log:	add a way to distinguish loops and entry bridges

diff --git a/pypy/module/pypyjit/test_pypy_c/model.py b/pypy/module/pypyjit/test_pypy_c/model.py
--- a/pypy/module/pypyjit/test_pypy_c/model.py
+++ b/pypy/module/pypyjit/test_pypy_c/model.py
@@ -47,11 +47,20 @@
             ids[name] = opcodes
         return ids
 
-    def by_filename(self, filename):
-        return [func for func in self.functions if func.filename == filename]
+    def _filter(self, function, is_entry_bridge=False):
+        return is_entry_bridge == '*' or function.is_entry_bridge == is_entry_bridge
+
+    def by_filename(self, filename, **kwds):
+        return [func for func in self.functions
+                if func.filename == filename and self._filter(func, **kwds)]
 
 class FunctionWithIds(Function):
 
+    is_entry_bridge = False
+
     @classmethod
     def from_trace(cls, trace, storage):
-        return cls.from_operations(trace.operations, storage)
+        res = cls.from_operations(trace.operations, storage)
+        if 'entry bridge' in trace.comment:
+            res.is_entry_bridge = True
+        return res

diff --git a/pypy/module/pypyjit/test_pypy_c/test_model.py b/pypy/module/pypyjit/test_pypy_c/test_model.py
--- a/pypy/module/pypyjit/test_pypy_c/test_model.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_model.py
@@ -82,5 +82,13 @@
         #
         log = self.run(f)
         loops = log.by_filename(self.filepath)
-        assert len(loops) == 2 # loop and entry bridge
-        
+        assert len(loops) == 1
+        assert loops[0].filename == self.filepath
+        assert not loops[0].is_entry_bridge
+        #
+        loops = log.by_filename(self.filepath, is_entry_bridge=True)
+        assert len(loops) == 1
+        assert loops[0].is_entry_bridge
+        #
+        loops = log.by_filename(self.filepath, is_entry_bridge='*')
+        assert len(loops) == 2

From commits-noreply at bitbucket.org  Fri Feb 18 18:31:00 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Fri, 18 Feb 2011 18:31:00 +0100 (CET)
Subject: [pypy-svn] pypy default: rename "function" to "loop",
	it's just easier to understand
Message-ID: <20110218173100.28D1B282C1B@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42178:4b7dbb6625d2
Date: 2011-02-18 17:17 +0100
http://bitbucket.org/pypy/pypy/changeset/4b7dbb6625d2/

Log:	rename "function" to "loop", it's just easier to understand

diff --git a/pypy/module/pypyjit/test_pypy_c/model.py b/pypy/module/pypyjit/test_pypy_c/model.py
--- a/pypy/module/pypyjit/test_pypy_c/model.py
+++ b/pypy/module/pypyjit/test_pypy_c/model.py
@@ -11,7 +11,7 @@
         storage.ids = self.find_ids(func)
         traces = [parse(rawtrace) for rawtrace in rawtraces]
         traces = storage.reconnect_loops(traces)
-        self.functions = [FunctionWithIds.from_trace(trace, storage) for trace in traces]
+        self.loops = [LoopWithIds.from_trace(trace, storage) for trace in traces]
 
     @classmethod
     def find_ids_range(cls, func):
@@ -47,14 +47,15 @@
             ids[name] = opcodes
         return ids
 
-    def _filter(self, function, is_entry_bridge=False):
-        return is_entry_bridge == '*' or function.is_entry_bridge == is_entry_bridge
+    def _filter(self, loop, is_entry_bridge=False):
+        return is_entry_bridge == '*' or loop.is_entry_bridge == is_entry_bridge
 
     def by_filename(self, filename, **kwds):
-        return [func for func in self.functions
-                if func.filename == filename and self._filter(func, **kwds)]
+        return [loop for loop in self.loops
+                if loop.filename == filename and self._filter(loop, **kwds)]
 
-class FunctionWithIds(Function):
+
+class LoopWithIds(Function):
 
     is_entry_bridge = False
 

From commits-noreply at bitbucket.org  Fri Feb 18 18:31:01 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Fri, 18 Feb 2011 18:31:01 +0100 (CET)
Subject: [pypy-svn] pypy default: move find_ids* functions outside of the
	Log class: they	now take a disassembler.Code object instead
	of a python function
Message-ID: <20110218173101.9756F282C1B@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42179:35f1d3dd3264
Date: 2011-02-18 17:55 +0100
http://bitbucket.org/pypy/pypy/changeset/35f1d3dd3264/

Log:	move find_ids* functions outside of the Log class: they now take a
	disassembler.Code object instead of a python function

diff --git a/lib_pypy/disassembler.py b/lib_pypy/disassembler.py
--- a/lib_pypy/disassembler.py
+++ b/lib_pypy/disassembler.py
@@ -32,8 +32,9 @@
 class CodeRepresentation(object):
     """ Representation of opcodes
     """
-    def __init__(self, opcodes, source):
+    def __init__(self, opcodes, co, source):
         self.opcodes = opcodes
+        self.co = co
         self.map = {}
         current_lineno = None
         for opcode in opcodes:
@@ -162,7 +163,7 @@
             opargstr = ''
         opcls = globals()[opname[op].replace('+', '_')]
         res.append(opcls(pos, lastline, oparg, opargstr))
-    return CodeRepresentation(res, source)
+    return CodeRepresentation(res, co, source)
 
 def disassemble_string(code, lasti=-1, varnames=None, names=None,
                        constants=None):

diff --git a/pypy/module/pypyjit/test_pypy_c/model.py b/pypy/module/pypyjit/test_pypy_c/model.py
--- a/pypy/module/pypyjit/test_pypy_c/model.py
+++ b/pypy/module/pypyjit/test_pypy_c/model.py
@@ -1,10 +1,42 @@
 import py
 import re
 import os.path
-from lib_pypy import disassembler
 from pypy.tool.jitlogparser.parser import parse, Function
 from pypy.tool.jitlogparser.storage import LoopStorage
 
+
+def find_ids_range(code):
+    """
+    Parse the given function and return a dictionary mapping "ids" to
+    "line ranges".  Ids are identified by comments with a special syntax::
+
+        # "myid" corresponds to the whole line
+        print 'foo' # ID: myid
+    """
+    result = {}
+    start_lineno = code.co.co_firstlineno
+    for i, line in enumerate(py.code.Source(code.source)):
+        m = re.search('# ID: (\w+)', line)
+        if m:
+            name = m.group(1)
+            lineno = start_lineno+i
+            result[name] = xrange(lineno, lineno+1)
+    return result
+
+def find_ids(code):
+    """
+    Parse the given function and return a dictionary mapping "ids" to
+    "opcodes".
+    """
+    ids = {}
+    ranges = find_ids_range(code)
+    for name, linerange in ranges.iteritems():
+        opcodes = [opcode for opcode in code.opcodes
+                   if opcode.lineno in linerange]
+        ids[name] = opcodes
+    return ids
+
+
 class Log(object):
     def __init__(self, func, rawtraces):
         storage = LoopStorage()
@@ -13,40 +45,6 @@
         traces = storage.reconnect_loops(traces)
         self.loops = [LoopWithIds.from_trace(trace, storage) for trace in traces]
 
-    @classmethod
-    def find_ids_range(cls, func):
-        """
-        Parse the given function and return a dictionary mapping "ids" to
-        "line ranges".  Ids are identified by comments with a special syntax::
-
-            # "myid" corresponds to the whole line
-            print 'foo' # ID: myid
-        """
-        result = {}
-        start_lineno = func.func_code.co_firstlineno
-        for i, line in enumerate(py.code.Source(func)):
-            m = re.search('# ID: (\w+)', line)
-            if m:
-                name = m.group(1)
-                lineno = start_lineno+i
-                result[name] = xrange(lineno, lineno+1)
-        return result
-
-    @classmethod
-    def find_ids(cls, func):
-        """
-        Parse the given function and return a dictionary mapping "ids" to
-        "opcodes".
-        """
-        ids = {}
-        code = disassembler.dis(func)
-        ranges = cls.find_ids_range(func)
-        for name, linerange in ranges.iteritems():
-            opcodes = [opcode for opcode in code.opcodes
-                       if opcode.lineno in linerange]
-            ids[name] = opcodes
-        return ids
-
     def _filter(self, loop, is_entry_bridge=False):
         return is_entry_bridge == '*' or loop.is_entry_bridge == is_entry_bridge
 

diff --git a/pypy/module/pypyjit/test_pypy_c/test_model.py b/pypy/module/pypyjit/test_pypy_c/test_model.py
--- a/pypy/module/pypyjit/test_pypy_c/test_model.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_model.py
@@ -1,9 +1,10 @@
 import sys
 import subprocess
 import py
+from lib_pypy import disassembler
 from pypy.tool.udir import udir
 from pypy.tool import logparser
-from pypy.module.pypyjit.test_pypy_c.model import Log
+from pypy.module.pypyjit.test_pypy_c.model import Log, find_ids_range, find_ids
 
 class BaseTestPyPyC(object):
     def setup_class(cls):
@@ -53,7 +54,8 @@
             return a
         #
         start_lineno = f.func_code.co_firstlineno
-        ids = Log.find_ids_range(f)
+        code = disassembler.dis(f)
+        ids = find_ids_range(code)
         assert len(ids) == 1
         myline_range = ids['myline']
         assert list(myline_range) == range(start_lineno+1, start_lineno+2)
@@ -65,7 +67,8 @@
             z = x + 3 # ID: myline
             return z
         #
-        ids = Log.find_ids(f)
+        code = disassembler.dis(f)
+        ids = find_ids(code)
         assert len(ids) == 1
         myline = ids['myline']
         opcodes_names = [opcode.__class__.__name__ for opcode in myline]

From commits-noreply at bitbucket.org  Fri Feb 18 18:31:04 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Fri, 18 Feb 2011 18:31:04 +0100 (CET)
Subject: [pypy-svn] pypy default: add a way to get the code by the id
Message-ID: <20110218173104.17854282C22@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42180:fabd77a12b07
Date: 2011-02-18 18:19 +0100
http://bitbucket.org/pypy/pypy/changeset/fabd77a12b07/

Log:	add a way to get the code by the id

diff --git a/pypy/module/pypyjit/test_pypy_c/model.py b/pypy/module/pypyjit/test_pypy_c/model.py
--- a/pypy/module/pypyjit/test_pypy_c/model.py
+++ b/pypy/module/pypyjit/test_pypy_c/model.py
@@ -40,7 +40,6 @@
 class Log(object):
     def __init__(self, func, rawtraces):
         storage = LoopStorage()
-        storage.ids = self.find_ids(func)
         traces = [parse(rawtrace) for rawtrace in rawtraces]
         traces = storage.reconnect_loops(traces)
         self.loops = [LoopWithIds.from_trace(trace, storage) for trace in traces]
@@ -52,14 +51,45 @@
         return [loop for loop in self.loops
                 if loop.filename == filename and self._filter(loop, **kwds)]
 
+    def by_id(self, id, **kwds):
+        return [loop for loop in self.loops
+                if loop.has_id(id) and self._filter(loop, **kwds)]
+
 
 class LoopWithIds(Function):
 
     is_entry_bridge = False
 
+    def __init__(self, *args, **kwds):
+        Function.__init__(self, *args, **kwds)
+        self.compute_ids()
+
     @classmethod
     def from_trace(cls, trace, storage):
         res = cls.from_operations(trace.operations, storage)
-        if 'entry bridge' in trace.comment:
-            res.is_entry_bridge = True
+        res.is_entry_bridge = 'entry bridge' in trace.comment
         return res
+
+    def compute_ids(self):
+        self.ids = set()
+        self.code = None
+        if not self.filename:
+            return
+        self.code = self.chunks[0].getcode()
+        ids = find_ids(self.code)
+        all_my_opcodes = self.get_set_of_opcodes()
+        # XXX: for now, we just look for the first opcode in the id range
+        for id, opcodes in ids.iteritems():
+            targetop = opcodes[0]
+            if targetop in all_my_opcodes:
+                self.ids.add(id)
+
+    def get_set_of_opcodes(self):
+        res = set()
+        for chunk in self.chunks:
+            opcode = self.code.map[chunk.bytecode_no]
+            res.add(opcode)
+        return res
+
+    def has_id(self, id):
+        return id in self.ids

diff --git a/pypy/module/pypyjit/test_pypy_c/test_model.py b/pypy/module/pypyjit/test_pypy_c/test_model.py
--- a/pypy/module/pypyjit/test_pypy_c/test_model.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_model.py
@@ -79,8 +79,8 @@
     def test_parse_jitlog(self):
         def f():
             i = 0
-            while i < 1003: # default threshold is 10
-                i += 1 # ID: increment
+            while i < 1003:
+                i += 1
             return i
         #
         log = self.run(f)
@@ -95,3 +95,15 @@
         #
         loops = log.by_filename(self.filepath, is_entry_bridge='*')
         assert len(loops) == 2
+
+    def test_by_id(self):
+        def f():
+            i = 0
+            while i < 1003:
+                i += 1 # ID: increment
+            return i
+        #
+        log = self.run(f)
+        loop, = log.by_id('increment')
+        assert loop.filename == self.filepath
+        assert loop.code.co.co_name == 'f'

From commits-noreply at bitbucket.org  Fri Feb 18 18:58:02 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri, 18 Feb 2011 18:58:02 +0100 (CET)
Subject: [pypy-svn] pypy default: Rewrite most module specific exceptions at
	interp-level.
Message-ID: <20110218175802.6EF86282BE3@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r42181:76f4a40761e1
Date: 2011-02-18 18:56 +0100
http://bitbucket.org/pypy/pypy/changeset/76f4a40761e1/

Log:	Rewrite most module specific exceptions at interp-level. many
	applevel files were removed.

	This also fixes the _socket issue on Windows, where it's important
	that the _socket module is not imported during translation and
	stored in the prebuilt sys.modules.

diff --git a/pypy/module/zlib/app_zlib.py b/pypy/module/zlib/app_zlib.py
deleted file mode 100644
--- a/pypy/module/zlib/app_zlib.py
+++ /dev/null
@@ -1,11 +0,0 @@
-
-"""
-Application-level definitions for the zlib module.
-
-NOT_RPYTHON
-"""
-
-class error(Exception):
-    """
-    Raised by zlib operations.
-    """

diff --git a/pypy/module/_ssl/app_ssl.py b/pypy/module/_ssl/app_ssl.py
deleted file mode 100644
--- a/pypy/module/_ssl/app_ssl.py
+++ /dev/null
@@ -1,7 +0,0 @@
-import _socket
-
-class SSLError(_socket.error):
-    pass
-
-__doc__ = """Implementation module for SSL socket operations. 
-See the socket module for documentation."""

diff --git a/pypy/module/parser/app_helpers.py b/pypy/module/parser/app_helpers.py
deleted file mode 100644
--- a/pypy/module/parser/app_helpers.py
+++ /dev/null
@@ -1,2 +0,0 @@
-class ParserError(Exception):
-    pass

diff --git a/pypy/module/pyexpat/app_pyexpat.py b/pypy/module/pyexpat/app_pyexpat.py
deleted file mode 100644
--- a/pypy/module/pyexpat/app_pyexpat.py
+++ /dev/null
@@ -1,2 +0,0 @@
-class ExpatError(Exception):
-    pass

diff --git a/pypy/module/thread/app_thread.py b/pypy/module/thread/app_thread.py
deleted file mode 100644
--- a/pypy/module/thread/app_thread.py
+++ /dev/null
@@ -1,7 +0,0 @@
-class error(Exception):
-    pass
-
-def exit():
-    """This is synonymous to ``raise SystemExit''.  It will cause the current
-thread to exit silently unless the exception is caught."""
-    raise SystemExit

diff --git a/pypy/module/_socket/app_socket.py b/pypy/module/_socket/app_socket.py
deleted file mode 100644
--- a/pypy/module/_socket/app_socket.py
+++ /dev/null
@@ -1,15 +0,0 @@
-"""Implementation module for socket operations.
-
-See the socket module for documentation."""
-
-class error(IOError):
-    pass
-
-class herror(error):
-    pass
-
-class gaierror(error):
-    pass
-
-class timeout(error):
-    pass

diff --git a/pypy/module/_rawffi/error.py b/pypy/module/_rawffi/error.py
deleted file mode 100644
--- a/pypy/module/_rawffi/error.py
+++ /dev/null
@@ -1,2 +0,0 @@
-class SegfaultException(Exception):
-    pass

diff --git a/pypy/module/signal/app_signal.py b/pypy/module/signal/app_signal.py
deleted file mode 100644
--- a/pypy/module/signal/app_signal.py
+++ /dev/null
@@ -1,14 +0,0 @@
-
-
-def default_int_handler(signum, frame):
-    """
-    default_int_handler(...)
-
-    The default handler for SIGINT installed by Python.
-    It raises KeyboardInterrupt.
-    """
-    raise KeyboardInterrupt()
-
-
-class ItimerError(IOError):
-    pass
\ No newline at end of file

diff --git a/pypy/module/zipimport/app_zipimport.py b/pypy/module/zipimport/app_zipimport.py
deleted file mode 100644
--- a/pypy/module/zipimport/app_zipimport.py
+++ /dev/null
@@ -1,4 +0,0 @@
-
-class ZipImportError(ImportError):
-    pass
-

diff --git a/pypy/module/binascii/app_binascii.py b/pypy/module/binascii/app_binascii.py
deleted file mode 100644
--- a/pypy/module/binascii/app_binascii.py
+++ /dev/null
@@ -1,6 +0,0 @@
-
-class Error(Exception):
-    pass
-
-class Incomplete(Exception):
-    pass

diff --git a/pypy/module/select/app_select.py b/pypy/module/select/app_select.py
deleted file mode 100644
--- a/pypy/module/select/app_select.py
+++ /dev/null
@@ -1,2 +0,0 @@
-class error(Exception):
-    pass

diff --git a/pypy/module/termios/app_termios.py b/pypy/module/termios/app_termios.py
deleted file mode 100644
--- a/pypy/module/termios/app_termios.py
+++ /dev/null
@@ -1,3 +0,0 @@
-
-class error(Exception):
-    pass

diff --git a/pypy/module/mmap/app_mmap.py b/pypy/module/mmap/app_mmap.py
deleted file mode 100644
--- a/pypy/module/mmap/app_mmap.py
+++ /dev/null
@@ -1,2 +0,0 @@
-class error(EnvironmentError):
-    pass

From commits-noreply at bitbucket.org  Fri Feb 18 18:58:02 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Fri, 18 Feb 2011 18:58:02 +0100 (CET)
Subject: [pypy-svn] pypy interplevel-exception-classes: Close merged branch
Message-ID: <20110218175802.AD180282C1A@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: interplevel-exception-classes
Changeset: r42182:6f87e9cb78e9
Date: 2011-02-18 18:57 +0100
http://bitbucket.org/pypy/pypy/changeset/6f87e9cb78e9/

Log:	Close merged branch


From lac at codespeak.net  Sat Feb 19 04:51:06 2011
From: lac at codespeak.net (lac at codespeak.net)
Date: Sat, 19 Feb 2011 04:51:06 +0100 (CET)
Subject: [pypy-svn] r80362 - pypy/extradoc/talk/ustour2011
Message-ID: <20110219035106.B4790282C1A@codespeak.net>

Author: lac
Date: Sat Feb 19 04:51:03 2011
New Revision: 80362

Added:
   pypy/extradoc/talk/ustour2011/dropbox.txt   (contents, props changed)
Log:
Begin with google abstract, take out Maciej, add some points based on Rian
Hunter's mail.  Points need tightening up -- they are more for 'make sure
we address these issues' than anything else.


Added: pypy/extradoc/talk/ustour2011/dropbox.txt
==============================================================================
--- (empty file)
+++ pypy/extradoc/talk/ustour2011/dropbox.txt	Sat Feb 19 04:51:03 2011
@@ -0,0 +1,82 @@
+Talk at Dropbox.  Friday March 4th at 2 p.m.
+Speaker: Armin Rigo
+
+Bio:
+Armin Rigo is a researcher at Heinrich-Heine-Universit?t in D?sseldorf,
+Germany.  His academic interests include Programming Languages and
+Implementation Techniques.
+
+He is the lead designer of the PyPy project and one of its
+original founders.  He is also the author of Psyco, a hand-coded
+Just-in-Time specializing compiler for Python, which can be used
+transparently with 32-bit x86 versions of CPython.  Since 2003 he has
+worked on all aspects of PyPy: its Python interpreter (written in
+Python), its translation toolchain (which produces C code), its
+garbage collectors, and its Tracing Just-in-Time compiler generator.
+Since the end of 2010, the Just in Time compiler generated by PyPy
+has outperformed Psyco, while being much more general and robust.
+
+Abstract:
+The PyPy project has recently gathered a lot of attention for its
+progress in speeding up the Python language -- it is the fastest,
+most compatible and most stable 'alternative? Python interpreter.  No
+longer merely a research curiosity, PyPy is now suitable for production
+use.  Since 2009, the PyPy project has received funding from the Eurostars 
+Eureka program, which targets small European firms which produce research.
+(See: http://www.eurostars-eureka.eu/)
+
+This talk will being with an introduction to PyPy and then a more detailed
+discussion of what we have accomplished with the past two years' work in 
+the area of dynamic (Just-in-Time) interpreters.  We have had some very
+useful comments from Rian Hunter about areas of interest to dropbox, and
+we will finally discuss those bits which weren't already discussed.:
+
+* PyPy - the architecture XXX expand this
+
+* most Python benchmarks run much faster than with CPython or Psyco
+  "10x speedup in the average case for a tight loop in python"
+  speed.pypy.org
+* the real-world PyPy compiler toolchain itself (200 KLocs) runs twice as fast
+* already supports 64bit and is in the process of supporting ARM
+* full compatibility with  CPython (more than Jython/IronPython)
+* full (and JIT-ed) ctypes support to call C libraries from Python
+* supports Stackless Python (in-progress)
+* new "cpyext" layer which integrates existing CPython C extensions
+* an experimental super-fast JIT-compilation of calls to C++ libraries
+*A memory conservative JIT in steady state i.e. python right now  
+requires about 10MB of steady-state memory (all bytecodes / loaded  
+modules) not counting app usage. how much will that overhead increase  
+when using pypy? 
+     Maemo experiments.
+     Things we have done to shrink the memory usage.
+     But its huge now, anyway, because of the JIT.
+     What could be done to change this.
+     And do you really care about absolute size?  Or just that it can
+     scale and doesn't grow without bounds?
+
+* MIT license "so we can modify the runtime to generate obfuscated  
+pyc files" -- if your business really depends on obsfuscated code, and
+reverse engineering is a real worry for you, then we can do much better
+with pypy than what you are doing now.  (Anto cracked it already)
+
+*Compatibility with the major python executable packagers, py2exe,  
+py2app, and bfreeze (in that order)
+We're not going to embed a Cpython stub into an exe as py2exe does.
+We can produce executables with the python files embedded.  A PyPy packager.
+what it would look like.  What would be its advantages?
+
+*Future directions (what to do when Eurostars funding runs out)
+re: "The engineering resources to migrate won't require >1 month of work"
+-- one option is to hire us to do the migration.
+
+
+*Any other aspect of interest to audience members.  We already got a nice 
+start on understanding your interests through a list of questons from
+Rian Hunter.  But if anybody else has ideas about what they would like
+us to speak about, (or if Rian gets some more ideas) could  they please
+send a note to lac at openend.se and we will make sure that we cover this
+in our presentation.
+
+The PyPy project has a blog which may be of interest.  You can read
+it at http://morepypy.blogspot.com/ .
+


From arigo at codespeak.net  Sat Feb 19 10:55:35 2011
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sat, 19 Feb 2011 10:55:35 +0100 (CET)
Subject: [pypy-svn] r80363 - pypy/extradoc/talk/ustour2011
Message-ID: <20110219095535.0330D2A202C@codespeak.net>

Author: arigo
Date: Sat Feb 19 10:55:33 2011
New Revision: 80363

Modified:
   pypy/extradoc/talk/ustour2011/dropbox.txt
Log:
Typo.  Add a comment.


Modified: pypy/extradoc/talk/ustour2011/dropbox.txt
==============================================================================
--- pypy/extradoc/talk/ustour2011/dropbox.txt	(original)
+++ pypy/extradoc/talk/ustour2011/dropbox.txt	Sat Feb 19 10:55:33 2011
@@ -25,7 +25,7 @@
 Eureka program, which targets small European firms which produce research.
 (See: http://www.eurostars-eureka.eu/)
 
-This talk will being with an introduction to PyPy and then a more detailed
+This talk will begin with an introduction to PyPy and then a more detailed
 discussion of what we have accomplished with the past two years' work in 
 the area of dynamic (Just-in-Time) interpreters.  We have had some very
 useful comments from Rian Hunter about areas of interest to dropbox, and
@@ -64,6 +64,10 @@
 We're not going to embed a Cpython stub into an exe as py2exe does.
 We can produce executables with the python files embedded.  A PyPy packager.
 what it would look like.  What would be its advantages?
+(Actually we can produce a single exe that contains all
+necessary .py files already loaded; then there are no .py/.pyc files
+to distribute and obfuscate any more, and it's all in just one .exe.
+Such an .exe is maybe relatively large but compresses a lot.)
 
 *Future directions (what to do when Eurostars funding runs out)
 re: "The engineering resources to migrate won't require >1 month of work"


From commits-noreply at bitbucket.org  Sun Feb 20 11:35:10 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Sun, 20 Feb 2011 11:35:10 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: hg merge default
Message-ID: <20110220103510.91273282BAD@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r42183:c47f551e245f
Date: 2011-02-20 09:38 +0100
http://bitbucket.org/pypy/pypy/changeset/c47f551e245f/

Log:	hg merge default

diff --git a/pypy/module/_socket/app_socket.py b/pypy/module/_socket/app_socket.py
deleted file mode 100644
--- a/pypy/module/_socket/app_socket.py
+++ /dev/null
@@ -1,15 +0,0 @@
-"""Implementation module for socket operations.
-
-See the socket module for documentation."""
-
-class error(IOError):
-    pass
-
-class herror(error):
-    pass
-
-class gaierror(error):
-    pass
-
-class timeout(error):
-    pass

diff --git a/pypy/module/math/_genmath.py b/pypy/module/math/_genmath.py
deleted file mode 100644
--- a/pypy/module/math/_genmath.py
+++ /dev/null
@@ -1,62 +0,0 @@
-# ONESHOT SCRIPT (probably can go away soon)
-# to generate the mixed module 'math' (see same directory) 
-import py
-import math
-import re
-import sys
-rex_arg = re.compile(".*\((.*)\).*")
-
-if __name__ == '__main__': 
-    print py.code.Source("""
-        import math 
-        from pypy.interpreter.gateway import ObjSpace
-
-    """)
-    names = []
-    for name, func in math.__dict__.items(): 
-        if not callable(func): 
-            continue
-        sig = func.__doc__.split('\n')[0].strip()
-        sig = sig.split('->')[0].strip()
-        m = rex_arg.match(sig) 
-        assert m
-        args = m.group(1)
-        args = ", ".join(args.split(','))
-        sig = sig.replace('(', '(space,')
-        sig = ", ".join(sig.split(','))
-        argc = len(args.split(','))
-        unwrap_spec = ['ObjSpace']
-        unwrap_spec += ['float'] * argc 
-        unwrap_spec = ", ".join(unwrap_spec)
-        doc = func.__doc__.replace('\n', '\n       ')
-        
-        print py.code.Source('''
-            def %(sig)s: 
-                """%(doc)s
-                """
-                return space.wrap(math.%(name)s(%(args)s))
-            %(name)s.unwrap_spec = [%(unwrap_spec)s]
-        ''' % locals())
-        names.append(name) 
-
-    print >>sys.stderr, py.code.Source("""
-        # Package initialisation
-        from pypy.interpreter.mixedmodule import MixedModule
-
-        class Module(MixedModule):
-            appleveldefs = {
-            }
-            interpleveldefs = {
-    """)
-        
-    for name in names: 
-        space = " " * (15-len(name))
-        print >>sys.stderr, (
-            "       %(name)r%(space)s: 'interp_math.%(name)s'," % locals())
-    print >>sys.stderr, py.code.Source("""
-        }
-    """) 
-            
-        
-        
-        

diff --git a/pypy/module/_ssl/app_ssl.py b/pypy/module/_ssl/app_ssl.py
deleted file mode 100644
--- a/pypy/module/_ssl/app_ssl.py
+++ /dev/null
@@ -1,7 +0,0 @@
-import _socket
-
-class SSLError(_socket.error):
-    pass
-
-__doc__ = """Implementation module for SSL socket operations. 
-See the socket module for documentation."""

diff --git a/pypy/module/parser/app_helpers.py b/pypy/module/parser/app_helpers.py
deleted file mode 100644
--- a/pypy/module/parser/app_helpers.py
+++ /dev/null
@@ -1,2 +0,0 @@
-class ParserError(Exception):
-    pass

diff --git a/pypy/module/pyexpat/app_pyexpat.py b/pypy/module/pyexpat/app_pyexpat.py
deleted file mode 100644
--- a/pypy/module/pyexpat/app_pyexpat.py
+++ /dev/null
@@ -1,2 +0,0 @@
-class ExpatError(Exception):
-    pass

diff --git a/pypy/module/thread/app_thread.py b/pypy/module/thread/app_thread.py
deleted file mode 100644
--- a/pypy/module/thread/app_thread.py
+++ /dev/null
@@ -1,7 +0,0 @@
-class error(Exception):
-    pass
-
-def exit():
-    """This is synonymous to ``raise SystemExit''.  It will cause the current
-thread to exit silently unless the exception is caught."""
-    raise SystemExit

diff --git a/pypy/module/zlib/app_zlib.py b/pypy/module/zlib/app_zlib.py
deleted file mode 100644
--- a/pypy/module/zlib/app_zlib.py
+++ /dev/null
@@ -1,11 +0,0 @@
-
-"""
-Application-level definitions for the zlib module.
-
-NOT_RPYTHON
-"""
-
-class error(Exception):
-    """
-    Raised by zlib operations.
-    """

diff --git a/pypy/module/_rawffi/error.py b/pypy/module/_rawffi/error.py
deleted file mode 100644
--- a/pypy/module/_rawffi/error.py
+++ /dev/null
@@ -1,2 +0,0 @@
-class SegfaultException(Exception):
-    pass

diff --git a/pypy/module/signal/app_signal.py b/pypy/module/signal/app_signal.py
deleted file mode 100644
--- a/pypy/module/signal/app_signal.py
+++ /dev/null
@@ -1,14 +0,0 @@
-
-
-def default_int_handler(signum, frame):
-    """
-    default_int_handler(...)
-
-    The default handler for SIGINT installed by Python.
-    It raises KeyboardInterrupt.
-    """
-    raise KeyboardInterrupt()
-
-
-class ItimerError(IOError):
-    pass
\ No newline at end of file

diff --git a/pypy/module/zipimport/app_zipimport.py b/pypy/module/zipimport/app_zipimport.py
deleted file mode 100644
--- a/pypy/module/zipimport/app_zipimport.py
+++ /dev/null
@@ -1,4 +0,0 @@
-
-class ZipImportError(ImportError):
-    pass
-

diff --git a/pypy/module/binascii/app_binascii.py b/pypy/module/binascii/app_binascii.py
deleted file mode 100644
--- a/pypy/module/binascii/app_binascii.py
+++ /dev/null
@@ -1,6 +0,0 @@
-
-class Error(Exception):
-    pass
-
-class Incomplete(Exception):
-    pass

diff --git a/pypy/module/select/app_select.py b/pypy/module/select/app_select.py
deleted file mode 100644
--- a/pypy/module/select/app_select.py
+++ /dev/null
@@ -1,2 +0,0 @@
-class error(Exception):
-    pass

diff --git a/pypy/module/termios/app_termios.py b/pypy/module/termios/app_termios.py
deleted file mode 100644
--- a/pypy/module/termios/app_termios.py
+++ /dev/null
@@ -1,3 +0,0 @@
-
-class error(Exception):
-    pass

diff --git a/pypy/module/mmap/app_mmap.py b/pypy/module/mmap/app_mmap.py
deleted file mode 100644
--- a/pypy/module/mmap/app_mmap.py
+++ /dev/null
@@ -1,6 +0,0 @@
-ACCESS_READ = 1
-ACCESS_WRITE = 2
-ACCESS_COPY = 3
-
-class error(EnvironmentError):
-    pass

diff --git a/pypy/module/pypyjit/test/test_pypy_c_new.py b/pypy/module/pypyjit/test/test_pypy_c_new.py
deleted file mode 100644
--- a/pypy/module/pypyjit/test/test_pypy_c_new.py
+++ /dev/null
@@ -1,98 +0,0 @@
-
-import py, sys, re
-import subprocess
-import disassembler
-from pypy.tool.udir import udir
-from pypy.tool import logparser
-
-class Trace(object):
-    pass
-
-class BaseTestPyPyC(object):
-    def setup_class(cls):
-        cls.tmpdir = udir.join('test-pypy-jit')
-        cls.tmpdir.ensure(dir=True)
-
-    def setup_method(self, meth):
-        self.filepath = self.tmpdir.join(meth.im_func.func_name + '.py')
-
-    def parse_out(self, out):
-        out = out.strip("\n")
-        if out == 'None':
-            return None
-        try:
-            return int(out)
-        except ValueError:
-            return out
-
-    def parse_func(self, func):
-        # find lines such as # LOOP <name> is in a line
-        code = disassembler.dis(func)
-        result = {}
-        for i, line in enumerate(py.code.Source(func)):
-            m = re.search('# LOOP (\w+)', line)
-            if m:
-                name = m.group(1)
-                result[name] = []
-                for opcode in code.opcodes:
-                    no = opcode.lineno - func.func_code.co_firstlineno
-                    if i - 1 <= no <= i + 1:
-                        result[name].append(opcode)
-        return result
-    
-    def run(self, func):
-        with self.filepath.open("w") as f:
-            f.write(str(py.code.Source(func)) + "\n")
-            f.write("print %s()\n" % func.func_name)
-        logfile = self.filepath.new(ext='.log')
-        pipe = subprocess.Popen([sys.executable, str(self.filepath)],
-                                stdout=subprocess.PIPE, stderr=subprocess.PIPE,
-                                env={'PYPYLOG': "jit-log-opt,jit-summary:" + str(logfile)})
-        pipe.wait()
-        stderr = pipe.stderr.read()
-        assert not stderr
-        res = self.parse_out(pipe.stdout.read())
-        bytecodes = self.parse_func(func)
-        assert res == func()
-        log = logparser.parse_log_file(str(logfile))
-        parts = logparser.extract_category(log, 'jit-log-opt-')
-        log.xxx
-        return Trace()
-
-class TestInfrastructure(BaseTestPyPyC):
-    def test_parse_func(self):
-        def f():
-            i = 0
-            x = 0
-            # LOOP name
-            z = x + 3
-            return z
-
-        res = self.parse_func(f)
-        assert len(res) == 1
-        assert len(res['name']) == 6
-
-    def test_full(self):
-        def f():
-            i = 0
-            while i < 1003:
-                # LOOP one
-                i += 1
-
-        trace = self.run(f)
-        loop = trace.get_loops('one')
-        loop.get_bytecode(3, 'LOAD_FAST').match('''
-        int_add
-        guard_true
-        ''')
-        loop.get_bytecode(4, 'LOAD_CONST').match_stats(
-            guard='3', call='1-2', call_may_force='0'
-        )
-        # this would make operations that are "costly" obligatory to pass
-        # like new
-        loo.get_bytecode(5, 'INPLACE_ADD').match_stats(
-            allocs='5-10'
-            )
-
-class TestPyPyCNew(BaseTestPyPyC):
-    pass

From commits-noreply at bitbucket.org  Sun Feb 20 11:35:11 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Sun, 20 Feb 2011 11:35:11 +0100 (CET)
Subject: [pypy-svn] pypy jit-usable_retrace: proper cloning of values
Message-ID: <20110220103511.AC188282BAD@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-usable_retrace
Changeset: r42184:078265599215
Date: 2011-02-20 11:34 +0100
http://bitbucket.org/pypy/pypy/changeset/078265599215/

Log:	proper cloning of values

diff --git a/pypy/jit/metainterp/optimizeopt/heap.py b/pypy/jit/metainterp/optimizeopt/heap.py
--- a/pypy/jit/metainterp/optimizeopt/heap.py
+++ b/pypy/jit/metainterp/optimizeopt/heap.py
@@ -39,15 +39,15 @@
             newd = {}
             new.cached_fields[descr] = newd
             for value, fieldvalue in d.items():
-                newd[value.get_reconstructed(optimizer, valuemap)] = \
-                                       fieldvalue.get_reconstructed(optimizer, valuemap)
+                newd[value.get_cloned(optimizer, valuemap)] = \
+                                       fieldvalue.get_cloned(optimizer, valuemap)
             
         for descr, d in self.known_heap_fields.items():
             newd = {}
             new.known_heap_fields[descr] = newd
             for value, fieldvalue in d.items():
-                newd[value.get_reconstructed(optimizer, valuemap)] = \
-                                       fieldvalue.get_reconstructed(optimizer, valuemap)
+                newd[value.get_cloned(optimizer, valuemap)] = \
+                                       fieldvalue.get_cloned(optimizer, valuemap)
             
         new.cached_arrayitems = {}
         for descr, d in self.cached_arrayitems.items():
@@ -55,16 +55,16 @@
             new.cached_arrayitems[descr] = newd
             for value, cache in d.items():
                 newcache = CachedArrayItems()
-                newd[value.get_reconstructed(optimizer, valuemap)] = newcache
+                newd[value.get_cloned(optimizer, valuemap)] = newcache
                 if cache.var_index_item:
                     newcache.var_index_item = \
-                          cache.var_index_item.get_reconstructed(optimizer, valuemap)
+                          cache.var_index_item.get_cloned(optimizer, valuemap)
                 if cache.var_index_indexvalue:
                     newcache.var_index_indexvalue = \
-                          cache.var_index_indexvalue.get_reconstructed(optimizer, valuemap)
+                          cache.var_index_indexvalue.get_cloned(optimizer, valuemap)
                 for index, fieldvalue in cache.fixed_index_items.items():
                     newcache.fixed_index_items[index] = \
-                           fieldvalue.get_reconstructed(optimizer, valuemap)
+                           fieldvalue.get_cloned(optimizer, valuemap)
 
         return new
 

diff --git a/pypy/jit/metainterp/optimizeopt/optimizer.py b/pypy/jit/metainterp/optimizeopt/optimizer.py
--- a/pypy/jit/metainterp/optimizeopt/optimizer.py
+++ b/pypy/jit/metainterp/optimizeopt/optimizer.py
@@ -32,9 +32,15 @@
     known_class = None
     intbound = None
 
-    def __init__(self, box):
+    def __init__(self, box, level=None, known_class=None, intbound=None):
         self.box = box
-        self.intbound = IntBound(MININT, MAXINT) #IntUnbounded()
+        self.level = level
+        self.known_class = known_class
+        if intbound:
+            self.intbound = intbound
+        else:
+            self.intbound = IntBound(MININT, MAXINT) #IntUnbounded()
+
         if isinstance(box, Const):
             self.make_constant(box)
         # invariant: box is a Const if and only if level == LEVEL_CONSTANT
@@ -51,18 +57,21 @@
             boxes.append(self.force_box())
             already_seen[self.get_key_box()] = None
 
-    def get_reconstructed(self, optimizer, valuemap):
+    def get_cloned(self, optimizer, valuemap):
         if self in valuemap:
             return valuemap[self]
-        new = self.reconstruct_for_next_iteration(optimizer)
+        new = self.clone_for_next_iteration(optimizer)
+        assert new.__class__ is self.__class__
+        assert new.is_virtual() == self.is_virtual()
         valuemap[self] = new
-        self.reconstruct_childs(new, valuemap)
+        self.clone_childs(new, valuemap)
         return new
 
-    def reconstruct_for_next_iteration(self, optimizer):
-        return self
+    def clone_for_next_iteration(self, optimizer):
+        return OptValue(self.box, self.level, self.known_class,
+                        self.intbound.clone())
 
-    def reconstruct_childs(self, new, valuemap):
+    def clone_childs(self, new, valuemap):
         pass
 
     def get_args_for_fail(self, modifier):
@@ -161,6 +170,9 @@
     def __init__(self, box):
         self.make_constant(box)
 
+    def clone_for_next_iteration(self, optimizer):
+        return self
+    
 CONST_0      = ConstInt(0)
 CONST_1      = ConstInt(1)
 CVAL_ZERO    = ConstantValue(CONST_0)
@@ -281,17 +293,17 @@
 
         new.values = {}
         for box, value in self.values.items():
-            new.values[box] = value.get_reconstructed(new, valuemap)
+            new.values[box] = value.get_cloned(new, valuemap)
         new.interned_refs = self.interned_refs
         new.bool_boxes = {}
         for value in new.bool_boxes.keys():
-            new.bool_boxes[value.get_reconstructed(new, valuemap)] = None
+            new.bool_boxes[value.get_cloned(new, valuemap)] = None
 
         # FIXME: Move to rewrite.py
         new.loop_invariant_results = {}
         for key, value in self.loop_invariant_results.items():
             new.loop_invariant_results[key] = \
-                                 value.get_reconstructed(new, valuemap)
+                                 value.get_cloned(new, valuemap)
             
         new.pure_operations = self.pure_operations
         new.producer = self.producer

diff --git a/pypy/jit/metainterp/optimizeopt/virtualize.py b/pypy/jit/metainterp/optimizeopt/virtualize.py
--- a/pypy/jit/metainterp/optimizeopt/virtualize.py
+++ b/pypy/jit/metainterp/optimizeopt/virtualize.py
@@ -46,7 +46,7 @@
     def _really_force(self):
         raise NotImplementedError("abstract base")
 
-    def reconstruct_for_next_iteration(self, _optimizer):
+    def clone_for_next_iteration(self, _optimizer):
         return optimizer.OptValue(self.force_box())
 
 def get_fielddescrlist_cache(cpu):
@@ -144,17 +144,16 @@
         else:
             boxes.append(self.box)
 
-    def reconstruct_for_next_iteration(self, optimizer):
-        self.optimizer = optimizer
-        return self
+    def clone_for_next_iteration(self, optimizer):
+        raise NotImplementedError
 
-    def reconstruct_childs(self, new, valuemap):
+    def clone_childs(self, new, valuemap):
         assert isinstance(new, AbstractVirtualStructValue)
         if new.box is None:
             lst = self._get_field_descr_list()
             for ofs in lst:
                 new._fields[ofs] = \
-                      self._fields[ofs].get_reconstructed(new.optimizer, valuemap)
+                      self._fields[ofs].get_cloned(new.optimizer, valuemap)
 
 class VirtualValue(AbstractVirtualStructValue):
     level = optimizer.LEVEL_KNOWNCLASS
@@ -175,6 +174,13 @@
         field_names = [field.name for field in self._fields]
         return "<VirtualValue cls=%s fields=%s>" % (cls_name, field_names)
 
+    def clone_for_next_iteration(self, optimizer):
+        new = VirtualValue(optimizer, self.known_class, self.keybox,
+                           self.source_op)
+        new.box = self.box
+        return new
+            
+
 class VStructValue(AbstractVirtualStructValue):
 
     def __init__(self, optimizer, structdescr, keybox, source_op=None):
@@ -185,6 +191,13 @@
         fielddescrs = self._get_field_descr_list()
         return modifier.make_vstruct(self.structdescr, fielddescrs)
 
+    def clone_for_next_iteration(self, optimizer):
+        new = VStructValue(optimizer, self.structdescr, self.keybox,
+                           self.source_op)
+        new.box = self.box
+        return new
+
+    
 class VArrayValue(AbstractVirtualValue):
 
     def __init__(self, optimizer, arraydescr, size, keybox, source_op=None):
@@ -247,16 +260,18 @@
         else:
             boxes.append(self.box)
 
-    def reconstruct_for_next_iteration(self, optimizer):
-        self.optimizer = optimizer
-        return self
+    def clone_for_next_iteration(self, optimizer):
+        new = VArrayValue(optimizer, self.arraydescr, len(self._items),
+                          self.keybox, self.source_op)
+        new.box = self.box
+        return new        
 
-    def reconstruct_childs(self, new, valuemap):
+    def clone_childs(self, new, valuemap):
         assert isinstance(new, VArrayValue)
         if new.box is None:
             for i in range(len(self._items)):
-                new._items[i] = self._items[i].get_reconstructed(new.optimizer,
-                                                                 valuemap)
+                new._items[i] = self._items[i].get_cloned(new.optimizer,
+                                                          valuemap)
 
 class OptVirtualize(optimizer.Optimization):
     "Virtualize objects until they escape."

From commits-noreply at bitbucket.org  Sun Feb 20 13:09:24 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sun, 20 Feb 2011 13:09:24 +0100 (CET)
Subject: [pypy-svn] pypy default: Write a test that fails on OSX.
Message-ID: <20110220120924.7B8562A2031@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r42185:915ed143c2e5
Date: 2011-02-20 13:04 +0100
http://bitbucket.org/pypy/pypy/changeset/915ed143c2e5/

Log:	Write a test that fails on OSX.

diff --git a/pypy/rlib/test/test_rlocale.py b/pypy/rlib/test/test_rlocale.py
--- a/pypy/rlib/test/test_rlocale.py
+++ b/pypy/rlib/test/test_rlocale.py
@@ -1,10 +1,11 @@
 
 # -*- coding: utf-8 -*-
 
-import py
+import py, sys
 import locale as cpython_locale
 from pypy.rlib.rlocale import setlocale, LC_ALL, LocaleError, isupper, \
-     islower, isalpha, tolower, isalnum, numeric_formatting
+     islower, isalpha, tolower, isalnum, numeric_formatting, external
+from pypy.rpython.lltypesystem import rffi
 
 class TestLocale(object):
     def setup_class(cls):
@@ -34,3 +35,10 @@
     assert isinstance(dec, str)
     assert isinstance(th, str)
     assert isinstance(grouping, str)
+
+def test_libintl():
+    if sys.platform not in ("linux2", "darwin"):
+        py.test.skip("there is (maybe) no libintl here")
+    _gettext = external('gettext', [rffi.CCHARP], rffi.CCHARP)
+    res = _gettext("1234")
+    assert rffi.charp2str(res) == "1234"

From commits-noreply at bitbucket.org  Sun Feb 20 13:09:25 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sun, 20 Feb 2011 13:09:25 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix.  Thanks dcolish for reporting this.
Message-ID: <20110220120925.19F822A2031@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r42186:c0b31ff7151e
Date: 2011-02-20 13:08 +0100
http://bitbucket.org/pypy/pypy/changeset/c0b31ff7151e/

Log:	Fix. Thanks dcolish for reporting this.

diff --git a/pypy/rlib/rlocale.py b/pypy/rlib/rlocale.py
--- a/pypy/rlib/rlocale.py
+++ b/pypy/rlib/rlocale.py
@@ -14,15 +14,22 @@
 
 HAVE_LANGINFO = sys.platform != 'win32'
 HAVE_LIBINTL  = sys.platform != 'win32'
+libraries = []
 
 if HAVE_LIBINTL:
     try:
-        platform.verify_eci(ExternalCompilationInfo(includes=['libintl.h']))
+        platform.verify_eci(ExternalCompilationInfo(includes=['libintl.h'],
+                                                    libraries=['intl']))
+        libraries.append('intl')
     except platform.CompilationError:
-        HAVE_LIBINTL = False
+        try:
+            platform.verify_eci(ExternalCompilationInfo(includes=['libintl.h']))
+        except platform.CompilationError:
+            HAVE_LIBINTL = False
 
 class CConfig:
     includes = ['locale.h', 'limits.h', 'ctype.h']
+    libraries = libraries
 
     if HAVE_LANGINFO:
         includes += ['langinfo.h']
@@ -31,7 +38,7 @@
     if sys.platform == 'win32':
         includes += ['windows.h']
     _compilation_info_ = ExternalCompilationInfo(
-        includes=includes
+        includes=includes, libraries=libraries
     )
     HAVE_BIND_TEXTDOMAIN_CODESET = platform.Has('bind_textdomain_codeset')
     lconv = platform.Struct("struct lconv", [

From arigo at codespeak.net  Sun Feb 20 15:09:32 2011
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sun, 20 Feb 2011 15:09:32 +0100 (CET)
Subject: [pypy-svn] r80364 - pypy/extradoc/talk/stanford-ee380-2011
Message-ID: <20110220140932.BDFA5282BAD@codespeak.net>

Author: arigo
Date: Sun Feb 20 15:09:30 2011
New Revision: 80364

Added:
   pypy/extradoc/talk/stanford-ee380-2011/
   pypy/extradoc/talk/stanford-ee380-2011/abstract.txt
Log:
Abstract, first version.


Added: pypy/extradoc/talk/stanford-ee380-2011/abstract.txt
==============================================================================
--- (empty file)
+++ pypy/extradoc/talk/stanford-ee380-2011/abstract.txt	Sun Feb 20 15:09:30 2011
@@ -0,0 +1,27 @@
+---------------------------------
+Python in Python: the PyPy system
+---------------------------------
+
+PyPy is a complete Python implementation in Python, in the
+old tradition of Squeak and Scheme48 --- or so it seems.
+
+During this talk I will describe what PyPy more precisely is,
+contrasting it with the other Python implementations, and showing
+results: faster execution of most programs (by a factor between
+1.5x and 20x) and a lower memory usage, while being fully
+compatible.
+
+I will then focus on the architecture of PyPy: on the one hand,
+Python itself is written as a straightforward interpreter,
+written in a (large) subset of Python called RPython.  On the
+other hand, we have a complex translation toolchain able to
+compile interpreters from RPython to efficient code (either C, or
+experimentally JVM/.NET).  This toolchain "weaves" into the final
+executable various aspects not present in the source of the
+interpreter: the object model, garbage collection, and even the
+Just-in-Time Compiler.  I will explain how this is done,
+particularly the JIT Compiler, which is a "meta" tracing JIT.
+
+I will conclude by comparing PyPy with other projects, old and
+new: Squeak, CPython, Jython and IronPython, as well as the
+various recent tracing JITs like TraceMonkey.


From arigo at codespeak.net  Sun Feb 20 16:15:45 2011
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sun, 20 Feb 2011 16:15:45 +0100 (CET)
Subject: [pypy-svn] r80365 - pypy/extradoc/talk/stanford-ee380-2011
Message-ID: <20110220151545.A1B5C282BAD@codespeak.net>

Author: arigo
Date: Sun Feb 20 16:15:39 2011
New Revision: 80365

Modified:
   pypy/extradoc/talk/stanford-ee380-2011/abstract.txt
Log:
Updates and various expansions from the discussion with lac.


Modified: pypy/extradoc/talk/stanford-ee380-2011/abstract.txt
==============================================================================
--- pypy/extradoc/talk/stanford-ee380-2011/abstract.txt	(original)
+++ pypy/extradoc/talk/stanford-ee380-2011/abstract.txt	Sun Feb 20 16:15:39 2011
@@ -5,23 +5,33 @@
 PyPy is a complete Python implementation in Python, in the
 old tradition of Squeak and Scheme48 --- or so it seems.
 
-During this talk I will describe what PyPy more precisely is,
-contrasting it with the other Python implementations, and showing
+During this talk I will describe what PyPy is: a mature,
+8-years-old project of roughly 200K lines of code and 150K lines
+of tests, implementing the full Python language.  I will show our
 results: faster execution of most programs (by a factor between
-1.5x and 20x) and a lower memory usage, while being fully
-compatible.
+1.5x and 20x) and a lower memory usage.
 
-I will then focus on the architecture of PyPy: on the one hand,
-Python itself is written as a straightforward interpreter,
-written in a (large) subset of Python called RPython.  On the
-other hand, we have a complex translation toolchain able to
-compile interpreters from RPython to efficient code (either C, or
-experimentally JVM/.NET).  This toolchain "weaves" into the final
-executable various aspects not present in the source of the
-interpreter: the object model, garbage collection, and even the
-Just-in-Time Compiler.  I will explain how this is done,
-particularly the JIT Compiler, which is a "meta" tracing JIT.
+I will then focus on the architecture of PyPy.  On the one hand,
+we have written a straightforward interpreter for the Python
+language, using a (large) subset of Python called RPython.  On
+the other hand, we have a complex translation toolchain which is
+able to compile interpreters from RPython to efficient code
+(either C, or experimentally JVM/.NET).
+
+This separation has two benefits.  On the one hand, we can easily
+write interpreters for other languages too -- we have a Prolog
+interpreter and played with at least Smalltalk and JavaScript.
+On the other hand, it allows us to keep architectural choices
+independent from the original interpreter's source code, instead
+of getting stuck with one particular set of choices.  Our
+toolchain "weaves" into the final executable various aspects not
+present in the source of the interpreter: the object model,
+garbage collection (GC), and even the Just-in-Time Compiler
+(JIT).  For example, CPython's GC is stuck to be reference
+counting, while we offer a number of choices.  I will explain how
+this is done, and describe in particular the JIT Compiler, which
+is a "meta" tracing JIT --- it works for any language.
 
 I will conclude by comparing PyPy with other projects, old and
-new: Squeak, CPython, Jython and IronPython, as well as the
-various recent tracing JITs like TraceMonkey.
+new: Squeak, CPython, Jython and IronPython, the Jikes RVM, as
+well as the various recent tracing JITs such as TraceMonkey.


From arigo at codespeak.net  Sun Feb 20 17:06:20 2011
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sun, 20 Feb 2011 17:06:20 +0100 (CET)
Subject: [pypy-svn] r80366 - pypy/extradoc/talk/stanford-ee380-2011
Message-ID: <20110220160620.61522282BAD@codespeak.net>

Author: arigo
Date: Sun Feb 20 17:06:19 2011
New Revision: 80366

Modified:
   pypy/extradoc/talk/stanford-ee380-2011/abstract.txt
Log:
Updates.

Modified: pypy/extradoc/talk/stanford-ee380-2011/abstract.txt
==============================================================================
--- pypy/extradoc/talk/stanford-ee380-2011/abstract.txt	(original)
+++ pypy/extradoc/talk/stanford-ee380-2011/abstract.txt	Sun Feb 20 17:06:19 2011
@@ -9,28 +9,31 @@
 8-years-old project of roughly 200K lines of code and 150K lines
 of tests, implementing the full Python language.  I will show our
 results: faster execution of most programs (by a factor between
-1.5x and 20x) and a lower memory usage.
+1.5x and 20x), and smaller total memory usage for large programs.
 
 I will then focus on the architecture of PyPy.  On the one hand,
 we have written a straightforward interpreter for the Python
 language, using a (large) subset of Python called RPython.  On
 the other hand, we have a complex translation toolchain which is
-able to compile interpreters from RPython to efficient code
-(either C, or experimentally JVM/.NET).
+able to compile interpreters from RPython to efficient C code.
+(We also have experimental backends for producing JVM/.NET code.)
 
-This separation has two benefits.  On the one hand, we can easily
-write interpreters for other languages too -- we have a Prolog
-interpreter and played with at least Smalltalk and JavaScript.
+There are two distinct benefits from keeping the interpreter and
+the translation toolchain separate.  On the one hand, we keep our
+interpreter simple, and we can easily write interpreters for
+other languages -- we have a Prolog interpreter and played with
+at least Smalltalk and JavaScript.
 On the other hand, it allows us to keep architectural choices
 independent from the original interpreter's source code, instead
 of getting stuck with one particular set of choices.  Our
 toolchain "weaves" into the final executable various aspects not
 present in the source of the interpreter: the object model,
-garbage collection (GC), and even the Just-in-Time Compiler
-(JIT).  For example, CPython's GC is stuck to be reference
-counting, while we offer a number of choices.  I will explain how
-this is done, and describe in particular the JIT Compiler, which
-is a "meta" tracing JIT --- it works for any language.
+garbage collection (GC), the backend (C/JVM/.NET), and even the
+Just-in-Time Compiler (JIT).  For example, CPython's GC is stuck
+to be reference counting, while we offer a number of choices.  I
+will explain how this is done, and describe in particular the JIT
+Compiler, which is a "meta" tracing JIT --- it works for any
+language.
 
 I will conclude by comparing PyPy with other projects, old and
 new: Squeak, CPython, Jython and IronPython, the Jikes RVM, as


From arigo at codespeak.net  Sun Feb 20 17:56:07 2011
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sun, 20 Feb 2011 17:56:07 +0100 (CET)
Subject: [pypy-svn] r80367 - pypy/extradoc/talk/stanford-ee380-2011
Message-ID: <20110220165607.25F20282BAD@codespeak.net>

Author: arigo
Date: Sun Feb 20 17:56:05 2011
New Revision: 80367

Modified:
   pypy/extradoc/talk/stanford-ee380-2011/abstract.txt
Log:
Updates: discussion with lac.

Modified: pypy/extradoc/talk/stanford-ee380-2011/abstract.txt
==============================================================================
--- pypy/extradoc/talk/stanford-ee380-2011/abstract.txt	(original)
+++ pypy/extradoc/talk/stanford-ee380-2011/abstract.txt	Sun Feb 20 17:56:05 2011
@@ -16,25 +16,25 @@
 language, using a (large) subset of Python called RPython.  On
 the other hand, we have a complex translation toolchain which is
 able to compile interpreters from RPython to efficient C code.
-(We also have experimental backends for producing JVM/.NET code.)
+(We also have experimental backends for producing JVM and .NET
+code.)
 
 There are two distinct benefits from keeping the interpreter and
 the translation toolchain separate.  On the one hand, we keep our
 interpreter simple, and we can easily write interpreters for
-other languages -- we have a Prolog interpreter and played with
-at least Smalltalk and JavaScript.
-On the other hand, it allows us to keep architectural choices
-independent from the original interpreter's source code, instead
-of getting stuck with one particular set of choices.  Our
-toolchain "weaves" into the final executable various aspects not
-present in the source of the interpreter: the object model,
-garbage collection (GC), the backend (C/JVM/.NET), and even the
-Just-in-Time Compiler (JIT).  For example, CPython's GC is stuck
-to be reference counting, while we offer a number of choices.  I
-will explain how this is done, and describe in particular the JIT
-Compiler, which is a "meta" tracing JIT --- it works for any
-language.
+other languages.  We have a complete Prolog interpreter and have
+at least played with versions for Smalltalk and Javascript.  On
+the other hand, the fact that our source interpreter does not
+contain any architectural choices makes for unprecedented
+flexibility.  Our toolchain "weaves" into the final executable
+various aspects including the object model, choice of garbage
+collection (GC), choice of backend (C/JVM/.NET), and even the
+Just-in-Time Compiler (JIT).  There are great practical benefits
+to this.  For example, CPython's GC is stuck with using reference
+counting, while we offer a number of choices.  I will explain how
+this is done, and describe in particular the JIT Compiler, which
+is a "meta" tracing JIT --- it works for any language.
 
-I will conclude by comparing PyPy with other projects, old and
-new: Squeak, CPython, Jython and IronPython, the Jikes RVM, as
-well as the various recent tracing JITs such as TraceMonkey.
+I will conclude by comparing PyPy to other projects, old and new:
+Squeak, CPython, Jython and IronPython, the Jikes RVM, as well as
+the various recent tracing JITs such as TraceMonkey.


From commits-noreply at bitbucket.org  Sun Feb 20 17:59:29 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Sun, 20 Feb 2011 17:59:29 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix for str(IOError(1, 2))
Message-ID: <20110220165929.80A822A202B@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r42187:e4ce179aeac2
Date: 2011-02-18 19:07 -0500
http://bitbucket.org/pypy/pypy/changeset/e4ce179aeac2/

Log:	Fix for str(IOError(1, 2))

diff --git a/pypy/module/exceptions/interp_exceptions.py b/pypy/module/exceptions/interp_exceptions.py
--- a/pypy/module/exceptions/interp_exceptions.py
+++ b/pypy/module/exceptions/interp_exceptions.py
@@ -381,14 +381,17 @@
     def descr_str(self, space):
         if (not space.is_w(self.w_errno, space.w_None) and
             not space.is_w(self.w_strerror, space.w_None)):
+            errno = space.str_w(space.str(self.w_errno))
+            strerror = space.str_w(space.str(self.w_strerror))
             if not space.is_w(self.w_filename, space.w_None):
                 return space.wrap("[Errno %s] %s: %s" % (
-                    space.str_w(space.str(self.w_errno)),
-                    space.str_w(self.w_strerror),
+                    errno,
+                    strerror,
                     space.str_w(space.repr(self.w_filename))))
-            return space.wrap("[Errno %s] %s" %
-                              (space.str_w(space.str(self.w_errno)),
-                               space.str_w(self.w_strerror)))
+            return space.wrap("[Errno %s] %s" % (
+                errno,
+                strerror,
+            ))
         return W_BaseException.descr_str(self, space)
 
 W_EnvironmentError.typedef = TypeDef(

diff --git a/pypy/module/exceptions/test/test_exc.py b/pypy/module/exceptions/test/test_exc.py
--- a/pypy/module/exceptions/test/test_exc.py
+++ b/pypy/module/exceptions/test/test_exc.py
@@ -69,6 +69,7 @@
         assert repr(Exception(3, "x")) == "Exception(3, 'x')"
         assert str(IOError("foo", "bar")) == "[Errno foo] bar"
         assert isinstance(IOError("foo", "bar"), IOError)
+        assert str(IOError(1, 2)) == "[Errno 1] 2"
 
     def test_custom_class(self):
         from exceptions import Exception, BaseException, LookupError

From commits-noreply at bitbucket.org  Sun Feb 20 17:59:29 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Sun, 20 Feb 2011 17:59:29 +0100 (CET)
Subject: [pypy-svn] pypy default: Merged upstream (workin on train is fun!).
Message-ID: <20110220165929.BBA292A2031@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r42188:04e742a4554d
Date: 2011-02-20 11:59 -0500
http://bitbucket.org/pypy/pypy/changeset/04e742a4554d/

Log:	Merged upstream (workin on train is fun!).


From arigo at codespeak.net  Sun Feb 20 18:01:33 2011
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sun, 20 Feb 2011 18:01:33 +0100 (CET)
Subject: [pypy-svn] r80368 - pypy/extradoc/talk/stanford-ee380-2011
Message-ID: <20110220170133.CEA81282BAD@codespeak.net>

Author: arigo
Date: Sun Feb 20 18:01:32 2011
New Revision: 80368

Modified:
   pypy/extradoc/talk/stanford-ee380-2011/abstract.txt
Log:
Improve intro paragraph (lac).


Modified: pypy/extradoc/talk/stanford-ee380-2011/abstract.txt
==============================================================================
--- pypy/extradoc/talk/stanford-ee380-2011/abstract.txt	(original)
+++ pypy/extradoc/talk/stanford-ee380-2011/abstract.txt	Sun Feb 20 18:01:32 2011
@@ -2,8 +2,9 @@
 Python in Python: the PyPy system
 ---------------------------------
 
-PyPy is a complete Python implementation in Python, in the
-old tradition of Squeak and Scheme48 --- or so it seems.
+PyPy is a complete Python implementation in Python, in the old
+tradition of Squeak and Scheme48 --- but there is more to PyPy
+than just this.
 
 During this talk I will describe what PyPy is: a mature,
 8-years-old project of roughly 200K lines of code and 150K lines


From arigo at codespeak.net  Sun Feb 20 18:06:06 2011
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sun, 20 Feb 2011 18:06:06 +0100 (CET)
Subject: [pypy-svn] r80369 - pypy/extradoc/talk/stanford-ee380-2011
Message-ID: <20110220170606.04E0E282BAD@codespeak.net>

Author: arigo
Date: Sun Feb 20 18:06:05 2011
New Revision: 80369

Modified:
   pypy/extradoc/talk/stanford-ee380-2011/abstract.txt
Log:
Mention Stackless.

Modified: pypy/extradoc/talk/stanford-ee380-2011/abstract.txt
==============================================================================
--- pypy/extradoc/talk/stanford-ee380-2011/abstract.txt	(original)
+++ pypy/extradoc/talk/stanford-ee380-2011/abstract.txt	Sun Feb 20 18:06:05 2011
@@ -29,7 +29,8 @@
 contain any architectural choices makes for unprecedented
 flexibility.  Our toolchain "weaves" into the final executable
 various aspects including the object model, choice of garbage
-collection (GC), choice of backend (C/JVM/.NET), and even the
+collection (GC), of execution model (regular vs. "Stackless"),
+choice of backend (C/JVM/.NET), and even the
 Just-in-Time Compiler (JIT).  There are great practical benefits
 to this.  For example, CPython's GC is stuck with using reference
 counting, while we offer a number of choices.  I will explain how


From arigo at codespeak.net  Sun Feb 20 18:32:06 2011
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sun, 20 Feb 2011 18:32:06 +0100 (CET)
Subject: [pypy-svn] r80370 - pypy/extradoc/talk/stanford-ee380-2011
Message-ID: <20110220173206.AEFFD2A202B@codespeak.net>

Author: arigo
Date: Sun Feb 20 18:32:05 2011
New Revision: 80370

Modified:
   pypy/extradoc/talk/stanford-ee380-2011/abstract.txt
Log:
Expand a bit the sentence about JITs.

Modified: pypy/extradoc/talk/stanford-ee380-2011/abstract.txt
==============================================================================
--- pypy/extradoc/talk/stanford-ee380-2011/abstract.txt	(original)
+++ pypy/extradoc/talk/stanford-ee380-2011/abstract.txt	Sun Feb 20 18:32:05 2011
@@ -33,9 +33,15 @@
 choice of backend (C/JVM/.NET), and even the
 Just-in-Time Compiler (JIT).  There are great practical benefits
 to this.  For example, CPython's GC is stuck with using reference
-counting, while we offer a number of choices.  I will explain how
-this is done, and describe in particular the JIT Compiler, which
-is a "meta" tracing JIT --- it works for any language.
+counting, while we offer a number of choices.
+
+I will explain how this is done, and describe in more details the
+JIT Compiler.  It is a "tracing JIT", as pioneered in recent
+years for Java and Javascript.  However, it is also a "meta JIT":
+it works for any language, by tracing at the level of the
+language's interpreter.  In other words, from an interpreter for
+any language, we produce quasi automatically a JIT suited to this
+language.
 
 I will conclude by comparing PyPy to other projects, old and new:
 Squeak, CPython, Jython and IronPython, the Jikes RVM, as well as


From lac at codespeak.net  Sun Feb 20 18:37:53 2011
From: lac at codespeak.net (lac at codespeak.net)
Date: Sun, 20 Feb 2011 18:37:53 +0100 (CET)
Subject: [pypy-svn] r80371 - pypy/extradoc/talk/stanford-ee380-2011
Message-ID: <20110220173753.E18DF282BAD@codespeak.net>

Author: lac
Date: Sun Feb 20 18:37:52 2011
New Revision: 80371

Modified:
   pypy/extradoc/talk/stanford-ee380-2011/abstract.txt
Log:
fix typo


Modified: pypy/extradoc/talk/stanford-ee380-2011/abstract.txt
==============================================================================
--- pypy/extradoc/talk/stanford-ee380-2011/abstract.txt	(original)
+++ pypy/extradoc/talk/stanford-ee380-2011/abstract.txt	Sun Feb 20 18:37:52 2011
@@ -40,7 +40,7 @@
 years for Java and Javascript.  However, it is also a "meta JIT":
 it works for any language, by tracing at the level of the
 language's interpreter.  In other words, from an interpreter for
-any language, we produce quasi automatically a JIT suited to this
+any language, we produce quasi-automatically a JIT suited to this
 language.
 
 I will conclude by comparing PyPy to other projects, old and new:


From commits-noreply at bitbucket.org  Sun Feb 20 19:10:52 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Sun, 20 Feb 2011 19:10:52 +0100 (CET)
Subject: [pypy-svn] pypy jit-usable_retrace: Allow a retraced loop to be
	passed	information from the bridge causing the retrace.
Message-ID: <20110220181052.8A5C32A202B@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-usable_retrace
Changeset: r42189:92b25ad0e9c0
Date: 2011-02-20 19:10 +0100
http://bitbucket.org/pypy/pypy/changeset/92b25ad0e9c0/

Log:	Allow a retraced loop to be passed information from the bridge
	causing the retrace.

diff --git a/pypy/jit/metainterp/optimizeopt/unroll.py b/pypy/jit/metainterp/optimizeopt/unroll.py
--- a/pypy/jit/metainterp/optimizeopt/unroll.py
+++ b/pypy/jit/metainterp/optimizeopt/unroll.py
@@ -621,8 +621,8 @@
         return self.map[loopbox]
 
 class OptInlineShortPreamble(Optimization):
-    def __init__(self, retraced):
-        self.retraced = retraced
+    def __init__(self, retrace):
+        self.retrace = retrace
         self.inliner = None
         
     
@@ -638,6 +638,10 @@
             # memory. This should also allow better behaviour in
             # situations that the is_emittable() chain currently cant
             # handle and the inlining fails unexpectedly belwo.
+            if self.retrace:
+                jump_args = op.getarglist()
+                jump_values = [self.getvalue(a) for a in jump_args]
+                self.retrace.compile(jump_values)
             short = descr.short_preamble
             if short:
                 args = op.getarglist()
@@ -683,7 +687,7 @@
                 if descr.failed_states:
                     retraced_count += len(descr.failed_states)
                 limit = self.optimizer.metainterp_sd.warmrunnerdesc.memory_manager.retrace_limit
-                if not self.retraced and retraced_count<limit:
+                if not self.retrace and retraced_count<limit:
                     if not descr.failed_states:
                         raise RetraceLoop
                     for failed in descr.failed_states:

diff --git a/pypy/jit/metainterp/compile.py b/pypy/jit/metainterp/compile.py
--- a/pypy/jit/metainterp/compile.py
+++ b/pypy/jit/metainterp/compile.py
@@ -80,19 +80,17 @@
 
 # ____________________________________________________________
 
-def compile_new_loop(metainterp, old_loop_tokens, greenkey, start,
+def compile_new_loop(metainterp, old_loop_tokens, greenkey, inputargs, h_ops,
                      start_resumedescr, full_preamble_needed=True):
     """Try to compile a new loop by closing the current history back
     to the first operation.
     """
-    history = metainterp.history
     loop = create_empty_loop(metainterp)
-    loop.inputargs = history.inputargs
+    loop.inputargs = inputargs
     for box in loop.inputargs:
         assert isinstance(box, Box)
     # make a copy, because optimize_loop can mutate the ops and descrs
-    h_ops = history.operations
-    loop.operations = [h_ops[i].clone() for i in range(start, len(h_ops))]
+    loop.operations = [h_ops[i].clone() for i in range(len(h_ops))]
     metainterp_sd = metainterp.staticdata
     jitdriver_sd = metainterp.jitdriver_sd
     loop_token = make_loop_token(len(loop.inputargs), jitdriver_sd)
@@ -566,7 +564,7 @@
         pass
 
 
-def compile_new_bridge(metainterp, old_loop_tokens, resumekey, retraced=False):
+def compile_new_bridge(metainterp, old_loop_tokens, resumekey, retrace=None):
     """Try to compile a new bridge leading from the beginning of the history
     to some existing place.
     """
@@ -589,7 +587,7 @@
         target_loop_token = state.optimize_bridge(metainterp_sd,
                                                   old_loop_tokens, new_loop,
                                                   inline_short_preamble,
-                                                  retraced)
+                                                  retrace)
     except InvalidLoop:
         # XXX I am fairly convinced that optimize_bridge cannot actually raise
         # InvalidLoop

diff --git a/pypy/jit/metainterp/pyjitpl.py b/pypy/jit/metainterp/pyjitpl.py
--- a/pypy/jit/metainterp/pyjitpl.py
+++ b/pypy/jit/metainterp/pyjitpl.py
@@ -1878,7 +1878,10 @@
         old_loop_tokens = self.get_compiled_merge_points(greenkey)
         self.history.record(rop.JUMP, live_arg_boxes[num_green_args:], None)
         loop_token = compile.compile_new_loop(self, old_loop_tokens,
-                                              greenkey, start, start_resumedescr)
+                                              greenkey,
+                                              self.history.inputargs,
+                                              self.history.operations[start:],
+                                              start_resumedescr)
         if loop_token is not None: # raise if it *worked* correctly
             self.set_compiled_merge_points(greenkey, old_loop_tokens)
             raise GenerateMergePoint(live_arg_boxes, loop_token)
@@ -1913,14 +1916,11 @@
         self.history.inputargs = original_boxes[num_green_args:]
         greenkey = original_boxes[:num_green_args]
         self.history.record(rop.JUMP, live_arg_boxes[num_green_args:], None)
-        loop_token = compile.compile_new_loop(self, [], greenkey, start,
-                                              start_resumedescr, False)
+        retrace = RetraceCompiler(self, greenkey,
+                                  original_boxes[num_green_args:],
+                                  self.history.operations[start:],
+                                  start_resumedescr, old_loop_tokens[0])
         self.history.operations.pop()     # remove the JUMP
-        if loop_token is None:
-            return
-
-        if loop_token.short_preamble:
-            old_loop_tokens[0].short_preamble.extend(loop_token.short_preamble)
 
         self.history.inputargs = original_inputargs
         self.history.operations = self.history.operations[:start]
@@ -1931,7 +1931,7 @@
                                                            #[loop_token],
                                                            old_loop_tokens,
                                                            self.resumekey,
-                                                           True)
+                                                           retrace)
         except RetraceLoop:
             assert False
         assert target_loop_token is not None
@@ -2469,3 +2469,29 @@
     frame._put_back_list_of_boxes(newvalue, 0, position)
     frame._put_back_list_of_boxes(newvalue, length1, position2)
     frame._put_back_list_of_boxes(newvalue, length1 + length2, position3)
+
+class RetraceCompiler(object):
+    def __init__(self, metainterp, greenkey, inputargs, h_ops,
+                 start_resumedescr, parent):
+        self.metainterp = metainterp
+        self.greenkey = greenkey
+        self.inputargs = inputargs
+        self.h_ops = h_ops
+        self.start_resumedescr = start_resumedescr
+        self.parent = parent
+
+    def compile(self, values):
+        assert len(values) == len(self.inputargs)
+        loop_token = compile.compile_new_loop(self.metainterp, [],
+                                              self.greenkey,
+                                              self.inputargs, self.h_ops,
+                                              self.start_resumedescr, False)
+        if loop_token is None:
+            return
+
+        if loop_token.short_preamble:
+            self.parent.short_preamble.extend(loop_token.short_preamble)
+        
+
+        
+        

diff --git a/pypy/jit/metainterp/nounroll_optimize.py b/pypy/jit/metainterp/nounroll_optimize.py
--- a/pypy/jit/metainterp/nounroll_optimize.py
+++ b/pypy/jit/metainterp/nounroll_optimize.py
@@ -18,7 +18,7 @@
     return None
 
 def optimize_bridge(metainterp_sd, old_loop_tokens, bridge,
-                    inline_short_preamble, retraced=False):
+                    inline_short_preamble, retraced=None):
     debug_start("jit-optimize")
     try:
         return _optimize_bridge(metainterp_sd, old_loop_tokens, bridge)

diff --git a/pypy/jit/metainterp/optimize.py b/pypy/jit/metainterp/optimize.py
--- a/pypy/jit/metainterp/optimize.py
+++ b/pypy/jit/metainterp/optimize.py
@@ -23,7 +23,7 @@
 # ____________________________________________________________
 
 def optimize_bridge(metainterp_sd, old_loop_tokens, bridge,
-                    inline_short_preamble=True, retraced=False):
+                    inline_short_preamble=True, retraced=None):
     debug_start("jit-optimize")
     try:
         return _optimize_bridge(metainterp_sd, old_loop_tokens, bridge,
@@ -32,7 +32,7 @@
         debug_stop("jit-optimize")
 
 def _optimize_bridge(metainterp_sd, old_loop_tokens, bridge,
-                     inline_short_preamble, retraced=False):
+                     inline_short_preamble, retraced=None):
     cpu = metainterp_sd.cpu
     metainterp_sd.logger_noopt.log_loop(bridge.inputargs, bridge.operations)
     if old_loop_tokens:

diff --git a/pypy/jit/metainterp/optimizeopt/__init__.py b/pypy/jit/metainterp/optimizeopt/__init__.py
--- a/pypy/jit/metainterp/optimizeopt/__init__.py
+++ b/pypy/jit/metainterp/optimizeopt/__init__.py
@@ -7,7 +7,7 @@
 from pypy.jit.metainterp.optimizeopt.unroll import optimize_unroll, OptInlineShortPreamble
 
 def optimize_loop_1(metainterp_sd, loop, unroll=True,
-                    inline_short_preamble=True, retraced=False):
+                    inline_short_preamble=True, retraced=None):
     """Optimize loop.operations to remove internal overheadish operations. 
     """
     opt_str = OptString()
@@ -35,7 +35,7 @@
         optimizer.propagate_all_forward()
 
 def optimize_bridge_1(metainterp_sd, bridge, inline_short_preamble=True,
-                      retraced=False):
+                      retraced=None):
     """The same, but for a bridge. """
     optimize_loop_1(metainterp_sd, bridge, False, inline_short_preamble,
                     retraced)

From lac at codespeak.net  Mon Feb 21 02:12:51 2011
From: lac at codespeak.net (lac at codespeak.net)
Date: Mon, 21 Feb 2011 02:12:51 +0100 (CET)
Subject: [pypy-svn] r80372 - pypy/extradoc/talk/stanford-ee380-2011
Message-ID: <20110221011251.2454D282BAD@codespeak.net>

Author: lac
Date: Mon Feb 21 02:12:48 2011
New Revision: 80372

Modified:
   pypy/extradoc/talk/stanford-ee380-2011/abstract.txt
Log:
why didn't I notice this earlier?  fix english


Modified: pypy/extradoc/talk/stanford-ee380-2011/abstract.txt
==============================================================================
--- pypy/extradoc/talk/stanford-ee380-2011/abstract.txt	(original)
+++ pypy/extradoc/talk/stanford-ee380-2011/abstract.txt	Mon Feb 21 02:12:48 2011
@@ -7,7 +7,7 @@
 than just this.
 
 During this talk I will describe what PyPy is: a mature,
-8-years-old project of roughly 200K lines of code and 150K lines
+8 year old project of roughly 200K lines of code and 150K lines
 of tests, implementing the full Python language.  I will show our
 results: faster execution of most programs (by a factor between
 1.5x and 20x), and smaller total memory usage for large programs.


From lac at codespeak.net  Mon Feb 21 02:37:18 2011
From: lac at codespeak.net (lac at codespeak.net)
Date: Mon, 21 Feb 2011 02:37:18 +0100 (CET)
Subject: [pypy-svn] r80373 - pypy/extradoc/talk/ustour2011
Message-ID: <20110221013718.291D6282BAD@codespeak.net>

Author: lac
Date: Mon Feb 21 02:37:16 2011
New Revision: 80373

Modified:
   pypy/extradoc/talk/ustour2011/dropbox.txt
Log:
canabalise the Stanford text.  tighten text.  Still one XXX left but I think
its nearly ready to ship, and about time too.


Modified: pypy/extradoc/talk/ustour2011/dropbox.txt
==============================================================================
--- pypy/extradoc/talk/ustour2011/dropbox.txt	(original)
+++ pypy/extradoc/talk/ustour2011/dropbox.txt	Mon Feb 21 02:37:16 2011
@@ -17,69 +17,73 @@
 has outperformed Psyco, while being much more general and robust.
 
 Abstract:
-The PyPy project has recently gathered a lot of attention for its
-progress in speeding up the Python language -- it is the fastest,
-most compatible and most stable 'alternative? Python interpreter.  No
-longer merely a research curiosity, PyPy is now suitable for production
-use.  Since 2009, the PyPy project has received funding from the Eurostars 
-Eureka program, which targets small European firms which produce research.
-(See: http://www.eurostars-eureka.eu/)
-
-This talk will begin with an introduction to PyPy and then a more detailed
-discussion of what we have accomplished with the past two years' work in 
-the area of dynamic (Just-in-Time) interpreters.  We have had some very
-useful comments from Rian Hunter about areas of interest to dropbox, and
-we will finally discuss those bits which weren't already discussed.:
+PyPy is a complete Python implementation in Python, in the old
+tradition of Squeak and Scheme48 --- but there is more to PyPy
+than just this.
+
+During this talk I will describe what PyPy is: a mature,
+8 year old project of roughly 200K lines of code and 150K lines
+of tests, implementing the full Python language.  I will show our
+results: faster execution of most programs (by a factor between
+1.5x and 20x), and smaller total memory usage for large programs.
+
+I will then briefly focus on the architecture of PyPy.  On the one hand,
+we have written a straightforward interpreter for the Python
+language, using a (large) subset of Python called RPython.  On
+the other hand, we have a complex translation toolchain which is
+able to compile interpreters from RPython to efficient C code.
+(We also have experimental backends for producing JVM and .NET
+code.)
+
+There are two distinct benefits from keeping the interpreter and
+the translation toolchain separate.  On the one hand, we keep our
+interpreter simple, and we can easily write interpreters for
+other languages.  We have a complete Prolog interpreter and have
+at least played with versions for Smalltalk and Javascript.  On
+the other hand, the fact that our source interpreter does not
+contain any architectural choices makes for unprecedented
+flexibility.  Our toolchain "weaves" into the final executable
+various aspects including the object model, choice of garbage
+collection (GC), of execution model (regular vs. "Stackless"),
+choice of backend (C/JVM/.NET), and even the
+Just-in-Time Compiler (JIT).  There are great practical benefits
+to this.  For example, CPython's GC is stuck with using reference
+counting, while we offer a number of choices.
+
+We will then move to a more detailed discussion of what we have
+accomplished with the past two years' work in the area of dynamic
+(Just-in-Time) interpreters.  Lastly, we have had some very useful comments
+from Rian Hunter about subject matter that is of particular interest to 
+dropbox, and we will discuss these issues and any others brought up by
+members of the audience.
 
-* PyPy - the architecture XXX expand this
+Highlights include:
 
+* speed.pypy.org
 * most Python benchmarks run much faster than with CPython or Psyco
-  "10x speedup in the average case for a tight loop in python"
-  speed.pypy.org
+* tight loops run much faster, how much faster varies but XXX can we say on average  XXX faster?
 * the real-world PyPy compiler toolchain itself (200 KLocs) runs twice as fast
-* already supports 64bit and is in the process of supporting ARM
+* pypy already supports 64bit and is in the process of supporting ARM
 * full compatibility with  CPython (more than Jython/IronPython)
 * full (and JIT-ed) ctypes support to call C libraries from Python
 * supports Stackless Python (in-progress)
 * new "cpyext" layer which integrates existing CPython C extensions
 * an experimental super-fast JIT-compilation of calls to C++ libraries
-*A memory conservative JIT in steady state i.e. python right now  
-requires about 10MB of steady-state memory (all bytecodes / loaded  
-modules) not counting app usage. how much will that overhead increase  
-when using pypy? 
-     Maemo experiments.
-     Things we have done to shrink the memory usage.
-     But its huge now, anyway, because of the JIT.
-     What could be done to change this.
-     And do you really care about absolute size?  Or just that it can
-     scale and doesn't grow without bounds?
-
-* MIT license "so we can modify the runtime to generate obfuscated  
-pyc files" -- if your business really depends on obsfuscated code, and
-reverse engineering is a real worry for you, then we can do much better
-with pypy than what you are doing now.  (Anto cracked it already)
-
-*Compatibility with the major python executable packagers, py2exe,  
-py2app, and bfreeze (in that order)
-We're not going to embed a Cpython stub into an exe as py2exe does.
-We can produce executables with the python files embedded.  A PyPy packager.
-what it would look like.  What would be its advantages?
-(Actually we can produce a single exe that contains all
-necessary .py files already loaded; then there are no .py/.pyc files
-to distribute and obfuscate any more, and it's all in just one .exe.
-Such an .exe is maybe relatively large but compresses a lot.)
-
-*Future directions (what to do when Eurostars funding runs out)
-re: "The engineering resources to migrate won't require >1 month of work"
--- one option is to hire us to do the migration.
-
-
-*Any other aspect of interest to audience members.  We already got a nice 
-start on understanding your interests through a list of questons from
-Rian Hunter.  But if anybody else has ideas about what they would like
-us to speak about, (or if Rian gets some more ideas) could  they please
-send a note to lac at openend.se and we will make sure that we cover this
-in our presentation.
+* smaller total memory usage for large and long-running programs.
+* MIT license 
+  - with a digression on code-obsfuctation and pypy if there is
+    sufficient interest
+* pypy and binary packagers. Embedding pypy. One large .exe anyone?
+* Future directions.   Since 2009, the PyPy project has received funding 
+  from the Eurostars  Eureka program, which targets small European firms 
+  which produce research.  What should we do when the funding runs out?
+  One thing we would like is to be hired to migrate projects from CPython 
+  to PyPy.
+*Any other aspect of interest to audience members. 
+
+If there is something you are interested in which isn't mentioned in this
+abstract, please send a note to lac at openend.se and we will make sure that 
+we cover this in our presentation.
 
 The PyPy project has a blog which may be of interest.  You can read
 it at http://morepypy.blogspot.com/ .


From cfbolz at codespeak.net  Mon Feb 21 13:03:50 2011
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Mon, 21 Feb 2011 13:03:50 +0100 (CET)
Subject: [pypy-svn] r80374 - pypy/extradoc/talk/stanford-ee380-2011
Message-ID: <20110221120350.77065282BE3@codespeak.net>

Author: cfbolz
Date: Mon Feb 21 13:03:46 2011
New Revision: 80374

Modified:
   pypy/extradoc/talk/stanford-ee380-2011/abstract.txt
Log:
a typo and a precisification


Modified: pypy/extradoc/talk/stanford-ee380-2011/abstract.txt
==============================================================================
--- pypy/extradoc/talk/stanford-ee380-2011/abstract.txt	(original)
+++ pypy/extradoc/talk/stanford-ee380-2011/abstract.txt	Mon Feb 21 13:03:46 2011
@@ -24,11 +24,11 @@
 the translation toolchain separate.  On the one hand, we keep our
 interpreter simple, and we can easily write interpreters for
 other languages.  We have a complete Prolog interpreter and have
-at least played with versions for Smalltalk and Javascript.  On
+at least played with versions for Smalltalk and JavaScript.  On
 the other hand, the fact that our source interpreter does not
 contain any architectural choices makes for unprecedented
 flexibility.  Our toolchain "weaves" into the final executable
-various aspects including the object model, choice of garbage
+various aspects including the object memory layout, choice of garbage
 collection (GC), of execution model (regular vs. "Stackless"),
 choice of backend (C/JVM/.NET), and even the
 Just-in-Time Compiler (JIT).  There are great practical benefits


From arigo at codespeak.net  Mon Feb 21 13:05:56 2011
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Mon, 21 Feb 2011 13:05:56 +0100 (CET)
Subject: [pypy-svn] r80375 - pypy/extradoc/talk/stanford-ee380-2011
Message-ID: <20110221120556.9225B282BE3@codespeak.net>

Author: arigo
Date: Mon Feb 21 13:05:55 2011
New Revision: 80375

Modified:
   pypy/extradoc/talk/stanford-ee380-2011/abstract.txt
Log:
JavaScript is mentioned twice :-)


Modified: pypy/extradoc/talk/stanford-ee380-2011/abstract.txt
==============================================================================
--- pypy/extradoc/talk/stanford-ee380-2011/abstract.txt	(original)
+++ pypy/extradoc/talk/stanford-ee380-2011/abstract.txt	Mon Feb 21 13:05:55 2011
@@ -37,7 +37,7 @@
 
 I will explain how this is done, and describe in more details the
 JIT Compiler.  It is a "tracing JIT", as pioneered in recent
-years for Java and Javascript.  However, it is also a "meta JIT":
+years for Java and JavaScript.  However, it is also a "meta JIT":
 it works for any language, by tracing at the level of the
 language's interpreter.  In other words, from an interpreter for
 any language, we produce quasi-automatically a JIT suited to this


From commits-noreply at bitbucket.org  Mon Feb 21 13:37:19 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon, 21 Feb 2011 13:37:19 +0100 (CET)
Subject: [pypy-svn] pypy default: Add more precise asserts,
	designed to catch most cases where 'obj' is nonsense (it
	points	inside an object, where there is a pointer).
Message-ID: <20110221123719.2630A282BE3@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r42190:375a03b5f02e
Date: 2011-02-20 14:17 +0100
http://bitbucket.org/pypy/pypy/changeset/375a03b5f02e/

Log:	Add more precise asserts, designed to catch most cases where 'obj'
	is nonsense (it points inside an object, where there is a pointer).

diff --git a/pypy/rpython/memory/gc/minimark.py b/pypy/rpython/memory/gc/minimark.py
--- a/pypy/rpython/memory/gc/minimark.py
+++ b/pypy/rpython/memory/gc/minimark.py
@@ -108,6 +108,8 @@
 GCFLAG_HAS_CARDS    = first_gcflag << 5
 GCFLAG_CARDS_SET    = first_gcflag << 6     # <- at least one card bit is set
 
+TID_MASK            = (first_gcflag << 7) - 1
+
 
 FORWARDSTUB = lltype.GcStruct('forwarding_stub',
                               ('forw', llmemory.Address))
@@ -820,9 +822,13 @@
         that can never be set on a young object -- except if tid == -42.
         """
         assert self.is_in_nursery(obj)
-        result = (self.header(obj).tid & GCFLAG_FINALIZATION_ORDERING != 0)
+        tid = self.header(obj).tid
+        result = (tid & GCFLAG_FINALIZATION_ORDERING != 0)
         if result:
-            ll_assert(self.header(obj).tid == -42, "bogus header for young obj")
+            ll_assert(tid == -42, "bogus header for young obj")
+        else:
+            ll_assert(tid != 0, "bogus header (1)")
+            ll_assert(tid & ~TID_MASK == 0, "bogus header (2)")
         return result
 
     def get_forwarding_address(self, obj):

From commits-noreply at bitbucket.org  Mon Feb 21 13:37:20 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon, 21 Feb 2011 13:37:20 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix comment.
Message-ID: <20110221123720.F2966282BE3@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r42191:9ba1de2fdb88
Date: 2011-02-20 23:06 +0100
http://bitbucket.org/pypy/pypy/changeset/9ba1de2fdb88/

Log:	Fix comment.

diff --git a/pypy/tool/udir.py b/pypy/tool/udir.py
--- a/pypy/tool/udir.py
+++ b/pypy/tool/udir.py
@@ -14,7 +14,7 @@
 #
 # The default value for $PYPY_USESSION_DIR is the system tmp.
 # The default value for $PYPY_USESSION_BASENAME is the name
-# of the current subversion branch.
+# of the current Mercurial branch.
 #
 
 import autopath

From commits-noreply at bitbucket.org  Mon Feb 21 13:37:21 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon, 21 Feb 2011 13:37:21 +0100 (CET)
Subject: [pypy-svn] pypy default: Ignore the PCMP{EQ, GT}{B, W,
	D} instructions.
Message-ID: <20110221123721.E93CE282BE7@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r42192:0cea2c2108ac
Date: 2011-02-21 13:36 +0100
http://bitbucket.org/pypy/pypy/changeset/0cea2c2108ac/

Log:	Ignore the PCMP{EQ,GT}{B,W,D} instructions.

diff --git a/pypy/translator/c/gcc/trackgcroot.py b/pypy/translator/c/gcc/trackgcroot.py
--- a/pypy/translator/c/gcc/trackgcroot.py
+++ b/pypy/translator/c/gcc/trackgcroot.py
@@ -456,7 +456,7 @@
         'inc', 'dec', 'not', 'neg', 'or', 'and', 'sbb', 'adc',
         'shl', 'shr', 'sal', 'sar', 'rol', 'ror', 'mul', 'imul', 'div', 'idiv',
         'bswap', 'bt', 'rdtsc',
-        'punpck', 'pshufd', 
+        'punpck', 'pshufd', 'pcmp',
         # zero-extending moves should not produce GC pointers
         'movz', 
         ])

From commits-noreply at bitbucket.org  Mon Feb 21 13:37:23 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon, 21 Feb 2011 13:37:23 +0100 (CET)
Subject: [pypy-svn] pypy default: merge heads
Message-ID: <20110221123723.37966282BEA@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r42193:136e24f8c357
Date: 2011-02-21 13:36 +0100
http://bitbucket.org/pypy/pypy/changeset/136e24f8c357/

Log:	merge heads


From commits-noreply at bitbucket.org  Mon Feb 21 14:20:14 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Mon, 21 Feb 2011 14:20:14 +0100 (CET)
Subject: [pypy-svn] pypy arm-backend-2: merge default
Message-ID: <20110221132014.4C35B36C227@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backend-2
Changeset: r42194:84a9fd2d73a8
Date: 2011-02-18 09:45 +0100
http://bitbucket.org/pypy/pypy/changeset/84a9fd2d73a8/

Log:	merge default

diff --git a/pypy/interpreter/function.py b/pypy/interpreter/function.py
--- a/pypy/interpreter/function.py
+++ b/pypy/interpreter/function.py
@@ -186,10 +186,8 @@
             raise OperationError( space.w_TypeError, space.wrap("setting function's dictionary to a non-dict") )
         self.w_func_dict = w_dict
 
-    # unwrapping is done through unwrap_specs in typedef.py
-
     def descr_function__new__(space, w_subtype, w_code, w_globals,
-                            w_name=None, w_argdefs=None, w_closure=None):
+                              w_name=None, w_argdefs=None, w_closure=None):
         code = space.interp_w(Code, w_code)
         if not space.is_true(space.isinstance(w_globals, space.w_dict)):
             raise OperationError(space.w_TypeError, space.wrap("expected dict"))
@@ -326,13 +324,13 @@
         self.defs_w    = space.fixedview(w_defs_w)
         self.w_module = w_module
 
-    def fget_func_defaults(space, self):
+    def fget_func_defaults(self, space):
         values_w = self.defs_w
         if not values_w or None in values_w:
             return space.w_None
         return space.newtuple(values_w)
 
-    def fset_func_defaults(space, self, w_defaults):
+    def fset_func_defaults(self, space, w_defaults):
         if space.is_w(w_defaults, space.w_None):
             self.defs_w = []
             return
@@ -340,21 +338,21 @@
             raise OperationError( space.w_TypeError, space.wrap("func_defaults must be set to a tuple object or None") )
         self.defs_w = space.fixedview(w_defaults)
 
-    def fdel_func_defaults(space, self):
+    def fdel_func_defaults(self, space):
         self.defs_w = []
 
-    def fget_func_doc(space, self):
+    def fget_func_doc(self, space):
         if self.w_doc is None:
             self.w_doc = self.code.getdocstring(space)
         return self.w_doc
 
-    def fset_func_doc(space, self, w_doc):
+    def fset_func_doc(self, space, w_doc):
         self.w_doc = w_doc
 
-    def fget_func_name(space, self):
+    def fget_func_name(self, space):
         return space.wrap(self.name)
 
-    def fset_func_name(space, self, w_name):
+    def fset_func_name(self, space, w_name):
         try:
             self.name = space.str_w(w_name)
         except OperationError, e:
@@ -365,10 +363,10 @@
             raise
 
 
-    def fdel_func_doc(space, self):
+    def fdel_func_doc(self, space):
         self.w_doc = space.w_None
 
-    def fget___module__(space, self):
+    def fget___module__(self, space):
         if self.w_module is None:
             if self.w_func_globals is not None and not space.is_w(self.w_func_globals, space.w_None):
                 self.w_module = space.call_method( self.w_func_globals, "get", space.wrap("__name__") )
@@ -376,16 +374,16 @@
                 self.w_module = space.w_None
         return self.w_module
 
-    def fset___module__(space, self, w_module):
+    def fset___module__(self, space, w_module):
         self.w_module = w_module
 
-    def fdel___module__(space, self):
+    def fdel___module__(self, space):
         self.w_module = space.w_None
 
-    def fget_func_code(space, self):
+    def fget_func_code(self, space):
         return space.wrap(self.code)
 
-    def fset_func_code(space, self, w_code):
+    def fset_func_code(self, space, w_code):
         from pypy.interpreter.pycode import PyCode
         if not self.can_change_code:
             raise OperationError(space.w_TypeError,
@@ -400,7 +398,7 @@
                 self.name, closure_len, len(code.co_freevars))
         self.code = code
 
-    def fget_func_closure(space, self):
+    def fget_func_closure(self, space):
         if self.closure is not None:
             w_res = space.newtuple( [ space.wrap(i) for i in self.closure ] )
         else:

diff --git a/pypy/jit/backend/llsupport/regalloc.py b/pypy/jit/backend/llsupport/regalloc.py
--- a/pypy/jit/backend/llsupport/regalloc.py
+++ b/pypy/jit/backend/llsupport/regalloc.py
@@ -158,9 +158,10 @@
 
     def _pick_variable_to_spill(self, v, forbidden_vars, selected_reg=None,
                                 need_lower_byte=False):
-        """ Silly algorithm.
+        """ Slightly less silly algorithm.
         """
-        candidates = []
+        cur_max_age = -1
+        candidate = None
         for next in self.reg_bindings:
             reg = self.reg_bindings[next]
             if next in forbidden_vars:
@@ -172,8 +173,13 @@
                     continue
             if need_lower_byte and reg in self.no_lower_byte_regs:
                 continue
-            return next
-        raise NoVariableToSpill
+            max_age = self.longevity[next][1]
+            if cur_max_age < max_age:
+                cur_max_age = max_age
+                candidate = next
+        if candidate is None:
+            raise NoVariableToSpill
+        return candidate
 
     def force_allocate_reg(self, v, forbidden_vars=[], selected_reg=None,
                            need_lower_byte=False):

diff --git a/pypy/interpreter/pycode.py b/pypy/interpreter/pycode.py
--- a/pypy/interpreter/pycode.py
+++ b/pypy/interpreter/pycode.py
@@ -9,8 +9,7 @@
 from pypy.interpreter import eval
 from pypy.interpreter.argument import Signature
 from pypy.interpreter.error import OperationError
-from pypy.interpreter.gateway import NoneNotWrapped 
-from pypy.interpreter.baseobjspace import ObjSpace, W_Root
+from pypy.interpreter.gateway import NoneNotWrapped, unwrap_spec
 from pypy.interpreter.astcompiler.consts import (CO_OPTIMIZED,
     CO_OPTIMIZED, CO_NEWLOCALS, CO_VARARGS, CO_VARKEYWORDS, CO_NESTED,
     CO_GENERATOR, CO_CONTAINSGLOBALS)
@@ -268,19 +267,19 @@
         co = self._to_code()
         dis.dis(co)
 
-    def fget_co_consts(space, self):
+    def fget_co_consts(self, space):
         return space.newtuple(self.co_consts_w)
     
-    def fget_co_names(space, self):
+    def fget_co_names(self, space):
         return space.newtuple(self.co_names_w)
 
-    def fget_co_varnames(space, self):
+    def fget_co_varnames(self, space):
         return space.newtuple([space.wrap(name) for name in self.co_varnames])
 
-    def fget_co_cellvars(space, self):
+    def fget_co_cellvars(self, space):
         return space.newtuple([space.wrap(name) for name in self.co_cellvars])
 
-    def fget_co_freevars(space, self):
+    def fget_co_freevars(self, space):
         return space.newtuple([space.wrap(name) for name in self.co_freevars])    
 
     def descr_code__eq__(self, w_other):
@@ -330,14 +329,10 @@
             w_result = space.xor(w_result, space.hash(w_const))
         return w_result
 
-    unwrap_spec =        [ObjSpace, W_Root, 
-                          int, int, int, int,
-                          str, W_Root, W_Root, 
-                          W_Root, str, str, int, 
-                          str, W_Root, 
-                          W_Root, int]
-
-
+    @unwrap_spec(argcount=int, nlocals=int, stacksize=int, flags=int,
+                 codestring=str,
+                 filename=str, name=str, firstlineno=int,
+                 lnotab=str, magic=int)
     def descr_code__new__(space, w_subtype,
                           argcount, nlocals, stacksize, flags,
                           codestring, w_constants, w_names,
@@ -369,7 +364,6 @@
         PyCode.__init__(code, space, argcount, nlocals, stacksize, flags, codestring, consts_w[:], names,
                       varnames, filename, name, firstlineno, lnotab, freevars, cellvars, magic=magic)
         return space.wrap(code)
-    descr_code__new__.unwrap_spec = unwrap_spec 
 
     def descr__reduce__(self, space):
         from pypy.interpreter.mixedmodule import MixedModule
@@ -402,4 +396,3 @@
 
     def repr(self, space):
         return space.wrap(self.get_repr())
-    repr.unwrap_spec = ['self', ObjSpace]

diff --git a/pypy/module/cpyext/api.py b/pypy/module/cpyext/api.py
--- a/pypy/module/cpyext/api.py
+++ b/pypy/module/cpyext/api.py
@@ -16,7 +16,7 @@
 from pypy.translator import platform
 from pypy.module.cpyext.state import State
 from pypy.interpreter.error import OperationError, operationerrfmt
-from pypy.interpreter.baseobjspace import W_Root, ObjSpace
+from pypy.interpreter.baseobjspace import W_Root
 from pypy.interpreter.gateway import unwrap_spec
 from pypy.interpreter.nestedscope import Cell
 from pypy.interpreter.module import Module
@@ -939,7 +939,7 @@
     copy_header_files(trunk_include)
 
 initfunctype = lltype.Ptr(lltype.FuncType([], lltype.Void))
- at unwrap_spec(ObjSpace, str, str)
+ at unwrap_spec(path=str, name=str)
 def load_extension_module(space, path, name):
     if os.sep not in path:
         path = os.curdir + os.sep + path      # force a '/' in the path

diff --git a/pypy/module/imp/importing.py b/pypy/module/imp/importing.py
--- a/pypy/module/imp/importing.py
+++ b/pypy/module/imp/importing.py
@@ -5,10 +5,10 @@
 import sys, os, stat
 
 from pypy.interpreter.module import Module
-from pypy.interpreter.gateway import Arguments, interp2app, unwrap_spec
+from pypy.interpreter.gateway import interp2app, unwrap_spec
 from pypy.interpreter.typedef import TypeDef, generic_new_descr
 from pypy.interpreter.error import OperationError, operationerrfmt
-from pypy.interpreter.baseobjspace import W_Root, ObjSpace, Wrappable
+from pypy.interpreter.baseobjspace import Wrappable
 from pypy.interpreter.eval import Code
 from pypy.rlib import streamio, jit, rposix
 from pypy.rlib.streamio import StreamErrors
@@ -108,6 +108,7 @@
 def check_sys_modules_w(space, modulename):
     return space.finditem_str(space.sys.get('modules'), modulename)
 
+ at unwrap_spec(name=str, level=int)
 def importhook(space, name, w_globals=None,
                w_locals=None, w_fromlist=None, level=-1):
     modulename = name
@@ -186,8 +187,6 @@
         space.setitem(space.sys.get('modules'), w(rel_modulename), space.w_None)
     space.timer.stop_name("importhook", modulename)
     return w_mod
-#
-importhook.unwrap_spec = [ObjSpace, str, W_Root, W_Root, W_Root, int]
 
 @jit.dont_look_inside
 def absolute_import(space, modulename, baselevel, fromlist_w, tentative):
@@ -330,7 +329,7 @@
     def __init__(self, space):
         pass
 
-    @unwrap_spec('self', ObjSpace, str)
+    @unwrap_spec(path=str)
     def descr_init(self, space, path):
         if not path:
             raise OperationError(space.w_ImportError, space.wrap(
@@ -346,7 +345,6 @@
                 raise OperationError(space.w_ImportError, space.wrap(
                     "existing directory"))
 
-    @unwrap_spec('self', ObjSpace, Arguments)
     def find_module_w(self, space, __args__):
         return space.wrap(None)
 

diff --git a/pypy/objspace/std/mapdict.py b/pypy/objspace/std/mapdict.py
--- a/pypy/objspace/std/mapdict.py
+++ b/pypy/objspace/std/mapdict.py
@@ -685,8 +685,9 @@
 
     @jit.dont_look_inside
     def is_valid_for_map(self, map):
+        # note that 'map' can be None here
         mymap = self.map_wref()
-        if mymap is map:     # also handles the case self.map_wref()->None
+        if mymap is not None and mymap is map:
             version_tag = map.terminator.w_cls.version_tag()
             if version_tag is self.version_tag:
                 # everything matches, it's incredibly fast

diff --git a/lib-python/conftest.py b/lib-python/conftest.py
--- a/lib-python/conftest.py
+++ b/lib-python/conftest.py
@@ -195,7 +195,7 @@
     RegrTest('test_dbm.py'),
     RegrTest('test_decimal.py'),
     RegrTest('test_decorators.py', core=True),
-    RegrTest('test_deque.py', core=True),
+    RegrTest('test_deque.py', core=True, usemodules='_collections'),
     RegrTest('test_descr.py', core=True, usemodules='_weakref'),
     RegrTest('test_descrtut.py', core=True),
     RegrTest('test_dict.py', core=True),
@@ -499,7 +499,7 @@
     RegrTest('test_complex_args.py'),
     RegrTest('test_contextlib.py', usemodules="thread"),
     RegrTest('test_ctypes.py', usemodules="_rawffi"),
-    RegrTest('test_defaultdict.py'),
+    RegrTest('test_defaultdict.py', usemodules='_collections'),
     RegrTest('test_email_renamed.py'),
     RegrTest('test_exception_variations.py'),
     RegrTest('test_float.py'),
@@ -716,8 +716,10 @@
         if test_stderr.rfind(26*"=" + "skipped" + 26*"=") != -1:
             skipped = True
         outcome = 'OK'
-        if not exit_status: 
-            if 'FAIL' in test_stdout or re.search('[^:]ERROR', test_stderr):
+        if not exit_status:
+            # match "FAIL" but not e.g. "FAILURE", which is in the output of a
+            # test in test_zipimport_support.py
+            if re.search(r'\bFAIL\b', test_stdout) or re.search('[^:]ERROR', test_stderr):
                 outcome = 'FAIL'
                 exit_status = 2  
         elif timedout: 

diff --git a/pypy/module/array/interp_array.py b/pypy/module/array/interp_array.py
--- a/pypy/module/array/interp_array.py
+++ b/pypy/module/array/interp_array.py
@@ -3,25 +3,20 @@
 from pypy.interpreter.error import OperationError
 from pypy.interpreter.typedef import TypeDef, GetSetProperty, make_weakref_descr
 from pypy.rpython.lltypesystem import lltype, rffi
-from pypy.interpreter.gateway import interp2app, ObjSpace, W_Root, \
-     ApplevelClass
-from pypy.rlib.jit import dont_look_inside
-from pypy.rlib import rgc
+from pypy.interpreter.gateway import interp2app, unwrap_spec
 from pypy.rlib.unroll import unrolling_iterable
 from pypy.rlib.rarithmetic import ovfcheck
-from pypy.rlib.rstruct.runpack import runpack
-from pypy.interpreter.argument import Arguments, Signature
-from pypy.interpreter.baseobjspace import ObjSpace, W_Root, Wrappable
+from pypy.interpreter.baseobjspace import Wrappable
 from pypy.objspace.std.stdtypedef import SMM, StdTypeDef
 from pypy.objspace.std.register_all import register_all
 from pypy.objspace.std.model import W_Object
-from pypy.interpreter.argument import Arguments, Signature
 from pypy.module._file.interp_file import W_File
 from pypy.interpreter.buffer import RWBuffer
 from pypy.objspace.std.multimethod import FailedToImplement
 
-def w_array(space, w_cls, typecode, w_args=None):
-    if len(w_args.arguments_w) > 1:
+ at unwrap_spec(typecode=str)
+def w_array(space, w_cls, typecode, __args__):
+    if len(__args__.arguments_w) > 1:
         msg = 'array() takes at most 2 arguments'
         raise OperationError(space.w_TypeError, space.wrap(msg))
     if len(typecode) != 1:
@@ -30,7 +25,7 @@
     typecode = typecode[0]
 
     if space.is_w(w_cls, space.gettypeobject(W_ArrayBase.typedef)):
-        if w_args.keywords: # XXX this might be forbidden fishing
+        if __args__.keywords:
             msg = 'array.array() does not take keyword arguments'
             raise OperationError(space.w_TypeError, space.wrap(msg))
 
@@ -39,8 +34,8 @@
             a = space.allocate_instance(types[tc].w_class, w_cls)
             a.__init__(space)
 
-            if len(w_args.arguments_w) > 0:
-                w_initializer = w_args.arguments_w[0]
+            if len(__args__.arguments_w) > 0:
+                w_initializer = __args__.arguments_w[0]
                 if space.type(w_initializer) is space.w_str:
                     a.fromstring(w_initializer)
                 elif space.type(w_initializer) is space.w_unicode:
@@ -55,7 +50,6 @@
         raise OperationError(space.w_ValueError, space.wrap(msg))
 
     return a
-w_array.unwrap_spec = (ObjSpace, W_Root, str, Arguments)
 
 
 array_append = SMM('append', 2)

diff --git a/pypy/config/pypyoption.py b/pypy/config/pypyoption.py
--- a/pypy/config/pypyoption.py
+++ b/pypy/config/pypyoption.py
@@ -32,7 +32,8 @@
      "crypt", "signal", "_rawffi", "termios", "zlib", "bz2",
      "struct", "_hashlib", "_md5", "_sha", "_minimal_curses", "cStringIO",
      "thread", "itertools", "pyexpat", "_ssl", "cpyext", "array",
-     "_bisect", "binascii", "_multiprocessing", '_warnings']
+     "_bisect", "binascii", "_multiprocessing", '_warnings',
+     "_collections"]
 ))
 
 translation_modules = default_modules.copy()

diff --git a/pypy/objspace/std/dictmultiobject.py b/pypy/objspace/std/dictmultiobject.py
--- a/pypy/objspace/std/dictmultiobject.py
+++ b/pypy/objspace/std/dictmultiobject.py
@@ -794,9 +794,8 @@
         w_dict.setitem(w_key, w_default)
         return w_default
 
-def dict_pop__DictMulti_ANY(space, w_dict, w_key, w_defaults):
-    defaults = space.listview(w_defaults)
-    len_defaults = len(defaults)
+def dict_pop__DictMulti_ANY(space, w_dict, w_key, defaults_w):
+    len_defaults = len(defaults_w)
     if len_defaults > 1:
         raise operationerrfmt(space.w_TypeError,
                               "pop expected at most 2 arguments, got %d",
@@ -804,7 +803,7 @@
     w_item = w_dict.getitem(w_key)
     if w_item is None:
         if len_defaults > 0:
-            return defaults[0]
+            return defaults_w[0]
         else:
             space.raise_key_error(w_key)
     else:

diff --git a/pypy/interpreter/baseobjspace.py b/pypy/interpreter/baseobjspace.py
--- a/pypy/interpreter/baseobjspace.py
+++ b/pypy/interpreter/baseobjspace.py
@@ -1009,6 +1009,38 @@
         return self.call_args(w_func, args)
     appexec._annspecialcase_ = 'specialize:arg(2)'
 
+    def _next_or_none(self, w_it):
+        try:
+            return self.next(w_it)
+        except OperationError, e:
+            if not e.match(self, self.w_StopIteration):
+                raise
+            return None
+
+    def compare_by_iteration(self, w_iterable1, w_iterable2, op):
+        w_it1 = self.iter(w_iterable1)
+        w_it2 = self.iter(w_iterable2)
+        while True:
+            w_x1 = self._next_or_none(w_it1)
+            w_x2 = self._next_or_none(w_it2)
+            if w_x1 is None or w_x2 is None:
+                if op == 'eq': return self.newbool(w_x1 is w_x2)  # both None
+                if op == 'ne': return self.newbool(w_x1 is not w_x2)
+                if op == 'lt': return self.newbool(w_x2 is not None)
+                if op == 'le': return self.newbool(w_x1 is None)
+                if op == 'gt': return self.newbool(w_x1 is not None)
+                if op == 'ge': return self.newbool(w_x2 is None)
+                assert False, "bad value for op"
+            if not self.eq_w(w_x1, w_x2):
+                if op == 'eq': return self.w_False
+                if op == 'ne': return self.w_True
+                if op == 'lt': return self.lt(w_x1, w_x2)
+                if op == 'le': return self.le(w_x1, w_x2)
+                if op == 'gt': return self.gt(w_x1, w_x2)
+                if op == 'ge': return self.ge(w_x1, w_x2)
+                assert False, "bad value for op"
+    compare_by_iteration._annspecialcase_ = 'specialize:arg(3)'
+
     def decode_index(self, w_index_or_slice, seqlength):
         """Helper for custom sequence implementations
              -> (index, 0, 0) or

diff --git a/pypy/module/gc/interp_gc.py b/pypy/module/gc/interp_gc.py
--- a/pypy/module/gc/interp_gc.py
+++ b/pypy/module/gc/interp_gc.py
@@ -1,4 +1,4 @@
-from pypy.interpreter.gateway import ObjSpace
+from pypy.interpreter.gateway import unwrap_spec
 from pypy.interpreter.error import OperationError
 from pypy.rlib import rgc
 from pypy.rlib.streamio import open_file_as_stream
@@ -16,8 +16,6 @@
             cache.clear()
     rgc.collect()
     return space.wrap(0)
-    
-collect.unwrap_spec = [ObjSpace]
 
 def enable_finalizers(space):
     if space.user_del_action.finalizers_lock_count == 0:
@@ -25,14 +23,13 @@
                              space.wrap("finalizers are already enabled"))
     space.user_del_action.finalizers_lock_count -= 1
     space.user_del_action.fire()
-enable_finalizers.unwrap_spec = [ObjSpace]
 
 def disable_finalizers(space):
     space.user_del_action.finalizers_lock_count += 1
-disable_finalizers.unwrap_spec = [ObjSpace]
 
 # ____________________________________________________________
 
+ at unwrap_spec(filename=str)
 def dump_heap_stats(space, filename):
     tb = rgc._heap_stats()
     if not tb:
@@ -43,4 +40,3 @@
         f.write("%d %d " % (tb[i].count, tb[i].size))
         f.write(",".join([str(tb[i].links[j]) for j in range(len(tb))]) + "\n")
     f.close()
-dump_heap_stats.unwrap_spec = [ObjSpace, str]

diff --git a/pypy/module/_rawffi/test/test__rawffi.py b/pypy/module/_rawffi/test/test__rawffi.py
--- a/pypy/module/_rawffi/test/test__rawffi.py
+++ b/pypy/module/_rawffi/test/test__rawffi.py
@@ -494,9 +494,10 @@
 
     def test_invalid_bitfields(self):
         import _rawffi
-        raises(ValueError, _rawffi.Structure, [('A', 'c', 1)])
+        raises(TypeError, _rawffi.Structure, [('A', 'c', 1)])
         raises(ValueError, _rawffi.Structure, [('A', 'I', 129)])
         raises(ValueError, _rawffi.Structure, [('A', 'I', -1)])
+        raises(ValueError, _rawffi.Structure, [('A', 'I', 0)])
 
     def test_packed_structure(self):
         import _rawffi

diff --git a/pypy/module/pypyjit/test/test_pypy_c.py b/pypy/module/pypyjit/test/test_pypy_c.py
--- a/pypy/module/pypyjit/test/test_pypy_c.py
+++ b/pypy/module/pypyjit/test/test_pypy_c.py
@@ -105,9 +105,9 @@
                 pass
 
             def check(args, expected):
-                print >> sys.stderr, 'trying:', args
+                #print >> sys.stderr, 'trying:', args
                 result = main(*args)
-                print >> sys.stderr, 'got:', repr(result)
+                #print >> sys.stderr, 'got:', repr(result)
                 assert result == expected
                 assert type(result) is type(expected)
         """ % threshold)

diff --git a/pypy/module/pypyjit/interp_jit.py b/pypy/module/pypyjit/interp_jit.py
--- a/pypy/module/pypyjit/interp_jit.py
+++ b/pypy/module/pypyjit/interp_jit.py
@@ -9,7 +9,6 @@
 from pypy.rlib.jit import current_trace_length
 import pypy.interpreter.pyopcode   # for side-effects
 from pypy.interpreter.error import OperationError, operationerrfmt
-from pypy.interpreter.gateway import ObjSpace, Arguments, W_Root
 from pypy.interpreter.pycode import PyCode, CO_GENERATOR
 from pypy.interpreter.pyframe import PyFrame
 from pypy.interpreter.pyopcode import ExitFrame
@@ -119,13 +118,13 @@
 #
 # Public interface    
 
-def set_param(space, args):
+def set_param(space, __args__):
     '''Configure the tunable JIT parameters.
         * set_param(name=value, ...)            # as keyword arguments
         * set_param("name=value,name=value")    # as a user-supplied string
     '''
     # XXXXXXXXX
-    args_w, kwds_w = args.unpack()
+    args_w, kwds_w = __args__.unpack()
     if len(args_w) > 1:
         msg = "set_param() takes at most 1 non-keyword argument, %d given"
         raise operationerrfmt(space.w_TypeError, msg, len(args_w))
@@ -144,11 +143,8 @@
             raise operationerrfmt(space.w_TypeError,
                                   "no JIT parameter '%s'", key)
 
-set_param.unwrap_spec = [ObjSpace, Arguments]
-
 @dont_look_inside
-def residual_call(space, w_callable, args):
+def residual_call(space, w_callable, __args__):
     '''For testing.  Invokes callable(...), but without letting
     the JIT follow the call.'''
-    return space.call_args(w_callable, args)
-residual_call.unwrap_spec = [ObjSpace, W_Root, Arguments]
+    return space.call_args(w_callable, __args__)

diff --git a/pypy/module/gc/referents.py b/pypy/module/gc/referents.py
--- a/pypy/module/gc/referents.py
+++ b/pypy/module/gc/referents.py
@@ -1,7 +1,7 @@
 from pypy.rlib import rgc
 from pypy.interpreter.baseobjspace import W_Root, Wrappable
 from pypy.interpreter.typedef import TypeDef
-from pypy.interpreter.gateway import ObjSpace
+from pypy.interpreter.gateway import unwrap_spec
 from pypy.interpreter.error import wrap_oserror, OperationError
 from pypy.rlib.objectmodel import we_are_translated
 
@@ -140,7 +140,6 @@
         gcref = rgc.cast_instance_to_gcref(w_obj)
         _list_w_obj_referents(gcref, result)
     return space.newlist(result)
-get_referents.unwrap_spec = [ObjSpace, 'args_w']
 
 def get_referrers(space, args_w):
     """Return the list of objects that directly refer to any of objs."""
@@ -167,8 +166,8 @@
                         result_w[w_obj] = None
                 pending_w += referents_w
     return space.newlist(result_w.keys())
-get_referrers.unwrap_spec = [ObjSpace, 'args_w']
 
+ at unwrap_spec(fd=int)
 def _dump_rpy_heap(space, fd):
     try:
         ok = rgc.dump_rpy_heap(fd)
@@ -176,10 +175,8 @@
         raise wrap_oserror(space, e)
     if not ok:
         raise missing_operation(space)
-_dump_rpy_heap.unwrap_spec = [ObjSpace, int]
 
 def get_typeids_z(space):
     a = rgc.get_typeids_z()
     s = ''.join([a[i] for i in range(len(a))])
     return space.wrap(s)
-get_typeids_z.unwrap_spec = [ObjSpace]

diff --git a/pypy/rpython/lltypesystem/lltype.py b/pypy/rpython/lltypesystem/lltype.py
--- a/pypy/rpython/lltypesystem/lltype.py
+++ b/pypy/rpython/lltypesystem/lltype.py
@@ -13,6 +13,29 @@
 
 TLS = tlsobject()
 
+class WeakValueDictionary(weakref.WeakValueDictionary):
+    """A subclass of weakref.WeakValueDictionary
+    which resets the 'nested_hash_level' when keys are being deleted.
+    """
+    def __init__(self, *args, **kwargs):
+        weakref.WeakValueDictionary.__init__(self, *args, **kwargs)
+        remove_base = self._remove
+        def remove(*args):
+            if TLS is None: # Happens when the interpreter is shutting down
+                return remove_base(*args)
+            nested_hash_level = TLS.nested_hash_level
+            try:
+                # The 'remove' function is called when an object dies.  This
+                # can happen anywhere when they are reference cycles,
+                # especially when we are already computing another __hash__
+                # value.  It's not really a recursion in this case, so we
+                # reset the counter; otherwise the hash value may be be
+                # incorrect and the key won't be deleted.
+                TLS.nested_hash_level = 0
+                remove_base(*args)
+            finally:
+                TLS.nested_hash_level = nested_hash_level
+        self._remove = remove
 
 class _uninitialized(object):
     def __init__(self, TYPE):
@@ -397,7 +420,7 @@
     # behaves more or less like a Struct with fields item0, item1, ...
     # but also supports __getitem__(), __setitem__(), __len__().
 
-    _cache = weakref.WeakValueDictionary() # cache the length-1 FixedSizeArrays
+    _cache = WeakValueDictionary() # cache the length-1 FixedSizeArrays
     def __new__(cls, OF, length, **kwds):
         if length == 1 and not kwds:
             try:
@@ -620,7 +643,7 @@
 class Ptr(LowLevelType):
     __name__ = property(lambda self: '%sPtr' % self.TO.__name__)
 
-    _cache = weakref.WeakValueDictionary()  # cache the Ptrs
+    _cache = WeakValueDictionary()  # cache the Ptrs
     def __new__(cls, TO, use_cache=True):
         if not isinstance(TO, ContainerType):
             raise TypeError, ("can only point to a Container type, "

diff --git a/pypy/module/signal/interp_signal.py b/pypy/module/signal/interp_signal.py
--- a/pypy/module/signal/interp_signal.py
+++ b/pypy/module/signal/interp_signal.py
@@ -1,6 +1,5 @@
 from __future__ import with_statement
 from pypy.interpreter.error import OperationError, exception_from_errno
-from pypy.interpreter.baseobjspace import W_Root, ObjSpace
 from pypy.interpreter.executioncontext import AsyncAction, AbstractActionFlag
 from pypy.interpreter.executioncontext import PeriodicAsyncAction
 from pypy.interpreter.gateway import unwrap_spec
@@ -192,6 +191,7 @@
             self.fire_after_thread_switch()
 
 
+ at unwrap_spec(signum=int)
 def getsignal(space, signum):
     """
     getsignal(sig) -> action
@@ -207,18 +207,16 @@
     if signum in action.handlers_w:
         return action.handlers_w[signum]
     return space.wrap(SIG_DFL)
-getsignal.unwrap_spec = [ObjSpace, int]
 
 @jit.dont_look_inside
+ at unwrap_spec(timeout=int)
 def alarm(space, timeout):
     return space.wrap(c_alarm(timeout))
-alarm.unwrap_spec = [ObjSpace, int]
 
 @jit.dont_look_inside
 def pause(space):
     c_pause()
     return space.w_None
-pause.unwrap_spec = [ObjSpace]
 
 def check_signum(space, signum):
     if signum < 1 or signum >= NSIG:
@@ -226,6 +224,7 @@
                              space.wrap("signal number out of range"))
 
 @jit.dont_look_inside
+ at unwrap_spec(signum=int)
 def signal(space, signum, w_handler):
     """
     signal(sig, action) -> action
@@ -262,8 +261,8 @@
         pypysig_setflag(signum)
         action.handlers_w[signum] = w_handler
     return old_handler
-signal.unwrap_spec = [ObjSpace, int, W_Root]
 
+ at unwrap_spec(fd=int)
 def set_wakeup_fd(space, fd):
     """Sets the fd to be written to (with '\0') when a signal
     comes in.  Returns the old fd.  A library can use this to
@@ -280,14 +279,13 @@
                 space.wrap("set_wakeup_fd only works in main thread"))
     old_fd = pypysig_set_wakeup_fd(fd)
     return space.wrap(intmask(old_fd))
-set_wakeup_fd.unwrap_spec = [ObjSpace, int]
 
+ at unwrap_spec(signum=int, flag=int)
 def siginterrupt(space, signum, flag):
     check_signum(space, signum)
     if rffi.cast(lltype.Signed, c_siginterrupt(signum, flag)) < 0:
         errno = rposix.get_errno()
         raise OperationError(space.w_RuntimeError, space.wrap(errno))
-siginterrupt.unwrap_spec = [ObjSpace, int, int]
 
 
 #__________________________________________________________
@@ -310,7 +308,7 @@
     return space.getattr(mod, space.wrap("ItimerError"))
 
 @jit.dont_look_inside
- at unwrap_spec(ObjSpace, int, float, float)
+ at unwrap_spec(which=int, first=float, interval=float)
 def setitimer(space, which, first, interval=0):
     with lltype.scoped_alloc(itimervalP.TO, 1) as new:
 
@@ -327,7 +325,7 @@
             return itimer_retval(space, old[0])
 
 @jit.dont_look_inside
- at unwrap_spec(ObjSpace, int)
+ at unwrap_spec(which=int)
 def getitimer(space, which):
     with lltype.scoped_alloc(itimervalP.TO, 1) as old:
 

diff --git a/pypy/module/sys/state.py b/pypy/module/sys/state.py
--- a/pypy/module/sys/state.py
+++ b/pypy/module/sys/state.py
@@ -3,7 +3,7 @@
 """
 import pypy
 from pypy.interpreter.error import OperationError
-from pypy.interpreter.gateway import ObjSpace
+from pypy.interpreter.gateway import unwrap_spec
 
 import sys, os, stat, errno
 
@@ -64,6 +64,7 @@
     #
     return importlist
 
+ at unwrap_spec(srcdir=str)
 def pypy_initial_path(space, srcdir):
     try:
         path = getinitialpath(srcdir)
@@ -76,8 +77,6 @@
                                         space.wrap(srcdir))
         return space.newlist([space.wrap(p) for p in path])
 
-pypy_initial_path.unwrap_spec = [ObjSpace, str]
-
 def get(space):
     return space.fromcache(State)
 

diff --git a/pypy/module/math/_genmath.py b/pypy/module/math/_genmath.py
deleted file mode 100644
--- a/pypy/module/math/_genmath.py
+++ /dev/null
@@ -1,62 +0,0 @@
-# ONESHOT SCRIPT (probably can go away soon)
-# to generate the mixed module 'math' (see same directory) 
-import py
-import math
-import re
-import sys
-rex_arg = re.compile(".*\((.*)\).*")
-
-if __name__ == '__main__': 
-    print py.code.Source("""
-        import math 
-        from pypy.interpreter.gateway import ObjSpace
-
-    """)
-    names = []
-    for name, func in math.__dict__.items(): 
-        if not callable(func): 
-            continue
-        sig = func.__doc__.split('\n')[0].strip()
-        sig = sig.split('->')[0].strip()
-        m = rex_arg.match(sig) 
-        assert m
-        args = m.group(1)
-        args = ", ".join(args.split(','))
-        sig = sig.replace('(', '(space,')
-        sig = ", ".join(sig.split(','))
-        argc = len(args.split(','))
-        unwrap_spec = ['ObjSpace']
-        unwrap_spec += ['float'] * argc 
-        unwrap_spec = ", ".join(unwrap_spec)
-        doc = func.__doc__.replace('\n', '\n       ')
-        
-        print py.code.Source('''
-            def %(sig)s: 
-                """%(doc)s
-                """
-                return space.wrap(math.%(name)s(%(args)s))
-            %(name)s.unwrap_spec = [%(unwrap_spec)s]
-        ''' % locals())
-        names.append(name) 
-
-    print >>sys.stderr, py.code.Source("""
-        # Package initialisation
-        from pypy.interpreter.mixedmodule import MixedModule
-
-        class Module(MixedModule):
-            appleveldefs = {
-            }
-            interpleveldefs = {
-    """)
-        
-    for name in names: 
-        space = " " * (15-len(name))
-        print >>sys.stderr, (
-            "       %(name)r%(space)s: 'interp_math.%(name)s'," % locals())
-    print >>sys.stderr, py.code.Source("""
-        }
-    """) 
-            
-        
-        
-        

From commits-noreply at bitbucket.org  Mon Feb 21 14:20:15 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Mon, 21 Feb 2011 14:20:15 +0100 (CET)
Subject: [pypy-svn] pypy arm-backend-2: improve freeing of boxes in
	operations merged	with guards
Message-ID: <20110221132015.253FE36C227@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backend-2
Changeset: r42195:a84286a29f72
Date: 2011-02-19 15:47 +0100
http://bitbucket.org/pypy/pypy/changeset/a84286a29f72/

Log:	improve freeing of boxes in operations merged with guards

diff --git a/pypy/jit/backend/arm/assembler.py b/pypy/jit/backend/arm/assembler.py
--- a/pypy/jit/backend/arm/assembler.py
+++ b/pypy/jit/backend/arm/assembler.py
@@ -484,11 +484,11 @@
             if op.has_no_side_effect() and op.result not in regalloc.longevity:
                 regalloc.possibly_free_vars_for_op(op)
             elif self.can_merge_with_next_guard(op, i, operations):
+                regalloc.next_instruction()
                 arglocs = regalloc.operations_with_guard[opnum](regalloc, op,
                                         operations[i+1], fcond)
                 fcond = self.operations_with_guard[opnum](self, op,
                                         operations[i+1], arglocs, regalloc, fcond)
-                regalloc.next_instruction()
             else:
                 arglocs = regalloc.operations[opnum](regalloc, op, fcond)
                 fcond = self.operations[opnum](self, op, arglocs, regalloc, fcond)

diff --git a/pypy/jit/backend/arm/regalloc.py b/pypy/jit/backend/arm/regalloc.py
--- a/pypy/jit/backend/arm/regalloc.py
+++ b/pypy/jit/backend/arm/regalloc.py
@@ -195,21 +195,18 @@
         return [reg1, reg2, res]
 
     def prepare_guard_int_mul_ovf(self, op, guard, fcond):
-        args = []
         boxes = list(op.getarglist())
         a0, a1 = boxes
 
-        reg1, box = self._ensure_value_is_boxed(a0,forbidden_vars=boxes)
+        reg1, box = self._ensure_value_is_boxed(a0, forbidden_vars=boxes)
         boxes.append(box)
-        reg2, box = self._ensure_value_is_boxed(a1,forbidden_vars=boxes)
+        reg2, box = self._ensure_value_is_boxed(a1, forbidden_vars=boxes)
         boxes.append(box)
-        res = self.force_allocate_reg(op.result, boxes)
+        res = self.force_allocate_reg(op.result)
+        args = self._prepare_guard(guard, [reg1, reg2, res])
 
-        args.append(reg1)
-        args.append(reg2)
-        args.append(res)
-        args = self._prepare_guard(guard, args)
         self.possibly_free_vars(boxes)
+        self.possibly_free_vars_for_op(op)
         self.possibly_free_var(op.result)
         self.possibly_free_vars(guard.getfailargs())
         return args

From commits-noreply at bitbucket.org  Mon Feb 21 14:20:15 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Mon, 21 Feb 2011 14:20:15 +0100 (CET)
Subject: [pypy-svn] pypy arm-backend-2: do not spill all registers around
	call_may_force	and get rid of the flag for it
Message-ID: <20110221132015.DAC5136C227@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backend-2
Changeset: r42196:203780ab1b87
Date: 2011-02-19 18:30 +0100
http://bitbucket.org/pypy/pypy/changeset/203780ab1b87/

Log:	do not spill all registers around call_may_force and get rid of the
	flag for it

diff --git a/pypy/jit/backend/arm/opassembler.py b/pypy/jit/backend/arm/opassembler.py
--- a/pypy/jit/backend/arm/opassembler.py
+++ b/pypy/jit/backend/arm/opassembler.py
@@ -237,11 +237,11 @@
         self._gen_path_to_exit_path(op, op.getarglist(), arglocs, c.AL)
         return fcond
 
-    def emit_op_call(self, op, args, regalloc, fcond, spill_all_regs=False):
+    def emit_op_call(self, op, args, regalloc, fcond):
         adr = args[0].value
         arglist = op.getarglist()[1:]
         cond =  self._emit_call(adr, arglist, regalloc, fcond,
-                                op.result, spill_all_regs=spill_all_regs)
+                                op.result)
         descr = op.getdescr()
         #XXX Hack, Hack, Hack
         if op.result and not we_are_translated() and not isinstance(descr, LoopToken):
@@ -252,10 +252,9 @@
         return cond
 
     # XXX improve this interface
-    # XXX and get rid of spill_all_regs in favor of pushing them in
     # emit_op_call_may_force
     # XXX improve freeing of stuff here
-    def _emit_call(self, adr, args, regalloc, fcond=c.AL, result=None, spill_all_regs=False):
+    def _emit_call(self, adr, args, regalloc, fcond=c.AL, result=None):
         n = 0
         n_args = len(args)
         reg_args = min(n_args, 4)
@@ -264,20 +263,17 @@
             l = regalloc.make_sure_var_in_reg(args[i],
                                             selected_reg=r.all_regs[i])
         # save caller saved registers
-        if spill_all_regs:
-            regalloc.before_call(save_all_regs=spill_all_regs)
+        if result:
+            # XXX hack if the call has a result force the value in r0 to be
+            # spilled
+            if reg_args == 0 or (isinstance(args[0], Box) and
+                    regalloc.stays_alive(args[0])):
+                t = TempBox()
+                regalloc.force_allocate_reg(t, selected_reg=regalloc.call_result_location(t))
+                regalloc.possibly_free_var(t)
+            self.mc.PUSH([reg.value for reg in r.caller_resp][1:])
         else:
-            if result:
-                # XXX hack if the call has a result force the value in r0 to be
-                # spilled
-                if reg_args == 0 or (isinstance(args[0], Box) and
-                        regalloc.stays_alive(args[0])):
-                    t = TempBox()
-                    regalloc.force_allocate_reg(t, selected_reg=regalloc.call_result_location(t))
-                    regalloc.possibly_free_var(t)
-                self.mc.PUSH([reg.value for reg in r.caller_resp][1:])
-            else:
-                self.mc.PUSH([reg.value for reg in r.caller_resp])
+            self.mc.PUSH([reg.value for reg in r.caller_resp])
 
         # all arguments past the 4th go on the stack
         if n_args > 4:
@@ -299,11 +295,9 @@
         # restore the argumets stored on the stack
         if result is not None:
             regalloc.after_call(result)
-        if not spill_all_regs:
-            if result is not None:
-                self.mc.POP([reg.value for reg in r.caller_resp][1:])
-            else:
-                self.mc.POP([reg.value for reg in r.caller_resp])
+            self.mc.POP([reg.value for reg in r.caller_resp][1:])
+        else:
+            self.mc.POP([reg.value for reg in r.caller_resp])
         return fcond
 
     def emit_op_same_as(self, op, arglocs, regalloc, fcond):

diff --git a/pypy/jit/backend/arm/regalloc.py b/pypy/jit/backend/arm/regalloc.py
--- a/pypy/jit/backend/arm/regalloc.py
+++ b/pypy/jit/backend/arm/regalloc.py
@@ -190,6 +190,7 @@
         boxes.append(box)
 
         self.possibly_free_vars(boxes)
+        self.possibly_free_vars_for_op(op)
         res = self.force_allocate_reg(op.result)
         self.possibly_free_var(op.result)
         return [reg1, reg2, res]
@@ -675,9 +676,10 @@
         fail_index = self.cpu.get_fail_descr_number(faildescr)
         self.assembler._write_fail_index(fail_index)
         args = [imm(rffi.cast(lltype.Signed, op.getarg(0).getint()))]
-        # force all reg values to be spilled when calling
-        self.assembler.emit_op_call(op, args, self, fcond, spill_all_regs=True)
-
+        for v in guard_op.getfailargs():
+            if v in self.reg_bindings:
+                self.force_spill_var(v)
+        self.assembler.emit_op_call(op, args, self, fcond)
         locs = self._prepare_guard(guard_op)
         self.possibly_free_vars(guard_op.getfailargs())
         return locs

From commits-noreply at bitbucket.org  Mon Feb 21 14:20:17 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Mon, 21 Feb 2011 14:20:17 +0100 (CET)
Subject: [pypy-svn] pypy arm-backend-2: provide a c function wrapping div,
	unsigned div and mod which are called from the jit for
	the	corresponding operations because arm's application
	profile does	not implement these operations.
Message-ID: <20110221132017.DB5D92A2038@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backend-2
Changeset: r42197:2fc7eab2a9ba
Date: 2011-02-20 21:19 +0100
http://bitbucket.org/pypy/pypy/changeset/2fc7eab2a9ba/

Log:	provide a c function wrapping div, unsigned div and mod which are
	called from the jit for the corresponding operations because arm's
	application profile does not implement these operations.

diff --git a/pypy/jit/backend/arm/arch.py b/pypy/jit/backend/arm/arch.py
--- a/pypy/jit/backend/arm/arch.py
+++ b/pypy/jit/backend/arm/arch.py
@@ -1,3 +1,4 @@
+from pypy.rpython.lltypesystem import lltype, rffi
 from pypy.rlib.rarithmetic import r_uint
 from pypy.rpython.lltypesystem import lltype
 
@@ -6,16 +7,40 @@
 # The Address in the PC points two words befind the current instruction
 PC_OFFSET = 8
 
+from pypy.translator.tool.cbuild import ExternalCompilationInfo
+eci = ExternalCompilationInfo(post_include_bits=["""
+static int pypy__arm_int_div(int a, int b) {
+    return a/b;
+}
+static uint pypy__arm_uint_div(uint a, uint b) {
+    return a/b;
+}
+static int pypy__arm_int_mod(uint a, uint b) {
+    return a % b;
+}
+"""])
+
 arm_int_div_sign = lltype.Ptr(lltype.FuncType([lltype.Signed, lltype.Signed], lltype.Signed))
-def arm_int_div(a, b):
+def arm_int_div_emulator(a, b):
     return int(a/float(b))
+arm_int_div = rffi.llexternal(
+    "pypy__arm_int_div", [lltype.Signed, lltype.Signed], lltype.Signed,
+                        _callable=arm_int_div_emulator,
+                        compilation_info=eci,
+                        _nowrapper=True, pure_function=True)
 
 arm_uint_div_sign = lltype.Ptr(lltype.FuncType([lltype.Unsigned, lltype.Unsigned], lltype.Unsigned))
-def arm_uint_div(a, b):
+def arm_uint_div_emulator(a, b):
     return r_uint(a)/r_uint(b)
+arm_uint_div = rffi.llexternal(
+    "pypy__arm_uint_div", [lltype.Unsigned, lltype.Unsigned], lltype.Unsigned,
+                        _callable=arm_uint_div_emulator,
+                        compilation_info=eci,
+                        _nowrapper=True, pure_function=True)
+
 
 arm_int_mod_sign = arm_int_div_sign
-def arm_int_mod(a, b):
+def arm_int_mod_emulator(a, b):
     sign = 1
     if a < 0:
         a = -1 * a
@@ -24,4 +49,9 @@
         b = -1 * b
     res = a % b
     return sign * res
+arm_int_mod = rffi.llexternal(
+    "pypy__arm_int_mod", [lltype.Signed, lltype.Signed], lltype.Signed,
+                        _callable=arm_int_mod_emulator,
+                        compilation_info=eci,
+                        _nowrapper=True, pure_function=True)
 

diff --git a/pypy/jit/backend/arm/test/test_arch.py b/pypy/jit/backend/arm/test/test_arch.py
--- a/pypy/jit/backend/arm/test/test_arch.py
+++ b/pypy/jit/backend/arm/test/test_arch.py
@@ -15,3 +15,9 @@
     assert arch.arm_int_mod(11, -2) == 1
     assert arch.arm_int_mod(11, -3) == 2
 
+
+def test_div():
+    assert arch.arm_int_div(-7, 2) == -3
+    assert arch.arm_int_div(9, 2) == 4
+    assert arch.arm_int_div(10, 5) == 2
+

diff --git a/pypy/jit/backend/arm/codebuilder.py b/pypy/jit/backend/arm/codebuilder.py
--- a/pypy/jit/backend/arm/codebuilder.py
+++ b/pypy/jit/backend/arm/codebuilder.py
@@ -18,7 +18,7 @@
     def f(self, c=cond.AL):
         """Generates a call to a helper function, takes its
         arguments in r0 and r1, result is placed in r0"""
-        addr = rffi.cast(lltype.Signed, llhelper(signature, function))
+        addr = rffi.cast(lltype.Signed, function)
         if c == cond.AL:
             self.BL(addr)
         else:

From commits-noreply at bitbucket.org  Mon Feb 21 14:20:19 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Mon, 21 Feb 2011 14:20:19 +0100 (CET)
Subject: [pypy-svn] pypy arm-backend-2: Move some redundant parts of guards
	to their own	procedures
Message-ID: <20110221132019.E7D6C2A2038@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backend-2
Changeset: r42198:c3de83bc0f2b
Date: 2011-02-21 14:15 +0100
http://bitbucket.org/pypy/pypy/changeset/c3de83bc0f2b/

Log:	Move some redundant parts of guards to their own procedures

diff --git a/pypy/jit/backend/arm/assembler.py b/pypy/jit/backend/arm/assembler.py
--- a/pypy/jit/backend/arm/assembler.py
+++ b/pypy/jit/backend/arm/assembler.py
@@ -92,6 +92,7 @@
                                                       ll_new_unicode)
         self.memcpy_addr = self.cpu.cast_ptr_to_int(memcpy_fn)
         self._exit_code_addr = self._gen_exit_path()
+        self._gen_leave_jitted_hook()
 
     def setup_failure_recovery(self):
 
@@ -199,12 +200,26 @@
         mem[i+2] = chr((n >> 16) & 0xFF)
         mem[i+3] = chr((n >> 24) & 0xFF)
 
+    def _gen_leave_jitted_hook(self):
+        def gen_code(save_exc=False):
+            mc = ARMv7Builder()
+            mc.PUSH([reg.value for reg in r.caller_resp] + [r.ip.value])
+            addr = self.cpu.get_on_leave_jitted_int(save_exception=save_exc)
+            mc.BL(addr)
+            mc.POP([reg.value for reg in r.caller_resp]+[r.ip.value])
+            assert self._exit_code_addr != 0
+            mc.B(self._exit_code_addr)
+            return mc.materialize(self.cpu.asmmemmgr, [],
+                                   self.cpu.gc_ll_descr.gcrootmap)
+        self._leave_jitted_jook_save_exc = gen_code(True)
+        self._leave_jitted_jook = gen_code(False)
+
     def _gen_exit_path(self):
         mc = ARMv7Builder()
         decode_registers_addr = llhelper(self.recovery_func_sign, self.failure_recovery_func)
 
         mc.PUSH([reg.value for reg in r.all_regs])     # registers r0 .. r10
-        mc.MOV_rr(r.r0.value, r.lr.value) # move mem block address, to r0 to pass as
+        mc.MOV_rr(r.r0.value, r.ip.value) # move mem block address, to r0 to pass as
         mc.MOV_rr(r.r1.value, r.fp.value) # pass the current frame pointer as second param
         mc.MOV_rr(r.r2.value, r.sp.value) # pass the current stack pointer as third param
 
@@ -216,12 +231,11 @@
         return mc.materialize(self.cpu.asmmemmgr, [],
                                    self.cpu.gc_ll_descr.gcrootmap)
 
-    def _gen_path_to_exit_path(self, op, args, arglocs, fcond=c.AL):
+    def _gen_path_to_exit_path(self, op, args, arglocs, fcond=c.AL, save_exc=False):
         descr = op.getdescr()
         if op.getopnum() != rop.FINISH:
             assert isinstance(descr, AbstractFailDescr)
             descr._arm_frame_depth = arglocs[0].getint()
-        reg = r.lr
         # The size of the allocated memory is based on the following sizes
         # first argloc is the frame depth and not considered for the memory
         # allocation
@@ -272,8 +286,12 @@
 
         n = self.cpu.get_fail_descr_number(descr)
         self.encode32(mem, j+1, n)
-        self.mc.LDR_ri(r.lr.value, r.pc.value, imm=WORD)
-        self.mc.B(self._exit_code_addr)
+        self.mc.LDR_ri(r.ip.value, r.pc.value, imm=WORD)
+        if save_exc:
+            path = self._leave_jitted_jook_save_exc
+        else:
+            path = self._leave_jitted_jook
+        self.mc.B(path)
         self.mc.write32(memaddr)
 
         return memaddr

diff --git a/pypy/jit/backend/arm/opassembler.py b/pypy/jit/backend/arm/opassembler.py
--- a/pypy/jit/backend/arm/opassembler.py
+++ b/pypy/jit/backend/arm/opassembler.py
@@ -129,7 +129,7 @@
 
     _mixin_ = True
 
-    guard_size = 10*WORD
+    guard_size = 5*WORD
     def _emit_guard(self, op, arglocs, fcond, save_exc=False):
         descr = op.getdescr()
         assert isinstance(descr, AbstractFailDescr)
@@ -139,12 +139,9 @@
 
         self.mc.ADD_ri(r.pc.value, r.pc.value, self.guard_size-PC_OFFSET, cond=fcond)
         descr._arm_guard_pos = self.mc.currpos()
-        self.mc.PUSH([reg.value for reg in r.caller_resp])
-        addr = self.cpu.get_on_leave_jitted_int(save_exception=save_exc)
-        self.mc.BL(addr)
-        self.mc.POP([reg.value for reg in r.caller_resp])
 
-        memaddr = self._gen_path_to_exit_path(op, op.getfailargs(), arglocs)
+        memaddr = self._gen_path_to_exit_path(op, op.getfailargs(),
+                                            arglocs, save_exc=save_exc)
         descr._failure_recovery_code = memaddr
         return c.AL
 

From antocuni at codespeak.net  Mon Feb 21 15:42:39 2011
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Mon, 21 Feb 2011 15:42:39 +0100 (CET)
Subject: [pypy-svn] r80376 - pypy/extradoc/talk/ustour2011
Message-ID: <20110221144239.69782282BE3@codespeak.net>

Author: antocuni
Date: Mon Feb 21 15:42:36 2011
New Revision: 80376

Modified:
   pypy/extradoc/talk/ustour2011/dropbox.txt
Log:
fix an XXX

Modified: pypy/extradoc/talk/ustour2011/dropbox.txt
==============================================================================
--- pypy/extradoc/talk/ustour2011/dropbox.txt	(original)
+++ pypy/extradoc/talk/ustour2011/dropbox.txt	Mon Feb 21 15:42:36 2011
@@ -61,7 +61,7 @@
 
 * speed.pypy.org
 * most Python benchmarks run much faster than with CPython or Psyco
-* tight loops run much faster, how much faster varies but XXX can we say on average  XXX faster?
+* tight loops run much faster, up to 20-60 times faster
 * the real-world PyPy compiler toolchain itself (200 KLocs) runs twice as fast
 * pypy already supports 64bit and is in the process of supporting ARM
 * full compatibility with  CPython (more than Jython/IronPython)


From commits-noreply at bitbucket.org  Mon Feb 21 16:13:26 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon, 21 Feb 2011 16:13:26 +0100 (CET)
Subject: [pypy-svn] pypy default: Test and fix.
Message-ID: <20110221151326.48E15282BE3@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r42199:f185f9ad6796
Date: 2011-02-21 14:07 +0000
http://bitbucket.org/pypy/pypy/changeset/f185f9ad6796/

Log:	Test and fix.

diff --git a/pypy/jit/codewriter/test/test_longlong.py b/pypy/jit/codewriter/test/test_longlong.py
--- a/pypy/jit/codewriter/test/test_longlong.py
+++ b/pypy/jit/codewriter/test/test_longlong.py
@@ -176,12 +176,18 @@
     def test_casts(self):
         self.do_check('cast_int_to_longlong', EffectInfo.OS_LLONG_FROM_INT,
                       [lltype.Signed], lltype.SignedLongLong)
+        self.do_check('cast_uint_to_longlong', EffectInfo.OS_LLONG_FROM_UINT,
+                      [lltype.Unsigned], lltype.SignedLongLong)
         self.do_check('truncate_longlong_to_int', EffectInfo.OS_LLONG_TO_INT,
                       [lltype.SignedLongLong], lltype.Signed)
         self.do_check('cast_float_to_longlong', EffectInfo.OS_LLONG_FROM_FLOAT,
                       [lltype.Float], lltype.SignedLongLong)
         self.do_check('cast_longlong_to_float', EffectInfo.OS_LLONG_TO_FLOAT,
                       [lltype.SignedLongLong], lltype.Float)
+        for T1 in [lltype.SignedLongLong, lltype.UnsignedLongLong]:
+            for T2 in [lltype.Signed, lltype.Unsigned]:
+                self.do_check('cast_primitive', EffectInfo.OS_LLONG_TO_INT,
+                              [T1], T2)
 
     def test_constants(self):
         for TYPE in [lltype.SignedLongLong, lltype.UnsignedLongLong]:

diff --git a/pypy/jit/codewriter/jtransform.py b/pypy/jit/codewriter/jtransform.py
--- a/pypy/jit/codewriter/jtransform.py
+++ b/pypy/jit/codewriter/jtransform.py
@@ -834,8 +834,10 @@
                 op2 = self._handle_oopspec_call(op1, args,
                                                 EffectInfo.OS_LLONG_%s,
                                                 EffectInfo.EF_PURE)
+                if %r == "TO_INT":
+                    assert op2.result.concretetype == lltype.Signed
                 return op2
-        ''' % (_op, _oopspec.lower(), _oopspec)).compile()
+        ''' % (_op, _oopspec.lower(), _oopspec, _oopspec)).compile()
 
     def _normalize(self, oplist):
         if isinstance(oplist, SpaceOperation):
@@ -873,14 +875,24 @@
             args = op.args
             if fromll:
                 opname = 'truncate_longlong_to_int'
+                RESULT = lltype.Signed
             else:
                 from pypy.rpython.lltypesystem import rffi
                 if rffi.cast(op.args[0].concretetype, -1) < 0:
                     opname = 'cast_int_to_longlong'
                 else:
                     opname = 'cast_uint_to_longlong'
-            op1 = SpaceOperation(opname, args, op.result)
-            return self.rewrite_operation(op1)
+                RESULT = lltype.SignedLongLong
+            v = varoftype(RESULT)
+            op1 = SpaceOperation(opname, args, v)
+            op2 = self.rewrite_operation(op1)
+            #
+            # force a renaming to put the correct result in place, even though
+            # it might be slightly mistyped (e.g. Signed versus Unsigned)
+            assert op2.result is v
+            op2.result = op.result
+            #
+            return op2
 
     # ----------
     # Renames, from the _old opname to the _new one.

From commits-noreply at bitbucket.org  Mon Feb 21 16:13:26 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Mon, 21 Feb 2011 16:13:26 +0100 (CET)
Subject: [pypy-svn] pypy default: merge heads
Message-ID: <20110221151326.7FDFF282BE7@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r42200:986fc8b4c759
Date: 2011-02-21 15:13 +0000
http://bitbucket.org/pypy/pypy/changeset/986fc8b4c759/

Log:	merge heads


From lac at codespeak.net  Mon Feb 21 16:14:57 2011
From: lac at codespeak.net (lac at codespeak.net)
Date: Mon, 21 Feb 2011 16:14:57 +0100 (CET)
Subject: [pypy-svn] r80377 - pypy/extradoc/talk/ustour2011
Message-ID: <20110221151457.DC997282BE3@codespeak.net>

Author: lac
Date: Mon Feb 21 16:14:55 2011
New Revision: 80377

Modified:
   pypy/extradoc/talk/ustour2011/dropbox.txt
Log:
de-Stanfordise this talk


Modified: pypy/extradoc/talk/ustour2011/dropbox.txt
==============================================================================
--- pypy/extradoc/talk/ustour2011/dropbox.txt	(original)
+++ pypy/extradoc/talk/ustour2011/dropbox.txt	Mon Feb 21 16:14:55 2011
@@ -17,49 +17,19 @@
 has outperformed Psyco, while being much more general and robust.
 
 Abstract:
-PyPy is a complete Python implementation in Python, in the old
-tradition of Squeak and Scheme48 --- but there is more to PyPy
-than just this.
-
-During this talk I will describe what PyPy is: a mature,
-8 year old project of roughly 200K lines of code and 150K lines
-of tests, implementing the full Python language.  I will show our
-results: faster execution of most programs (by a factor between
-1.5x and 20x), and smaller total memory usage for large programs.
-
-I will then briefly focus on the architecture of PyPy.  On the one hand,
-we have written a straightforward interpreter for the Python
-language, using a (large) subset of Python called RPython.  On
-the other hand, we have a complex translation toolchain which is
-able to compile interpreters from RPython to efficient C code.
-(We also have experimental backends for producing JVM and .NET
-code.)
-
-There are two distinct benefits from keeping the interpreter and
-the translation toolchain separate.  On the one hand, we keep our
-interpreter simple, and we can easily write interpreters for
-other languages.  We have a complete Prolog interpreter and have
-at least played with versions for Smalltalk and Javascript.  On
-the other hand, the fact that our source interpreter does not
-contain any architectural choices makes for unprecedented
-flexibility.  Our toolchain "weaves" into the final executable
-various aspects including the object model, choice of garbage
-collection (GC), of execution model (regular vs. "Stackless"),
-choice of backend (C/JVM/.NET), and even the
-Just-in-Time Compiler (JIT).  There are great practical benefits
-to this.  For example, CPython's GC is stuck with using reference
-counting, while we offer a number of choices.
-
-We will then move to a more detailed discussion of what we have
-accomplished with the past two years' work in the area of dynamic
-(Just-in-Time) interpreters.  Lastly, we have had some very useful comments
-from Rian Hunter about subject matter that is of particular interest to 
-dropbox, and we will discuss these issues and any others brought up by
-members of the audience.
 
-Highlights include:
+The PyPy project has recently gathered a lot of attention for its
+progress in speeding up the Python language -- it is the fastest,
+most compatible and most stable 'alternative? Python interpreter.  No
+longer merely a research curiosity, PyPy is now suitable for production
+use.  Since 2009, the PyPy project has received funding from the Eurostars 
+Eureka program, which targets small European firms which produce research.
+(See: http://www.eurostars-eureka.eu/)
+
+We will discuss what the PyPy project has achieved, with a particular
+focus on the past two years' work  in the area of dynamic (Just-in-Time) 
+interpreters:
 
-* speed.pypy.org
 * most Python benchmarks run much faster than with CPython or Psyco
 * tight loops run much faster, up to 20-60 times faster
 * the real-world PyPy compiler toolchain itself (200 KLocs) runs twice as fast


From lac at codespeak.net  Mon Feb 21 16:18:24 2011
From: lac at codespeak.net (lac at codespeak.net)
Date: Mon, 21 Feb 2011 16:18:24 +0100 (CET)
Subject: [pypy-svn] r80378 - pypy/extradoc/talk/ustour2011
Message-ID: <20110221151824.2EFD42A202B@codespeak.net>

Author: lac
Date: Mon Feb 21 16:18:22 2011
New Revision: 80378

Modified:
   pypy/extradoc/talk/ustour2011/dropbox.txt
Log:
add speed.pypy.org which was mistakenly deleted last time


Modified: pypy/extradoc/talk/ustour2011/dropbox.txt
==============================================================================
--- pypy/extradoc/talk/ustour2011/dropbox.txt	(original)
+++ pypy/extradoc/talk/ustour2011/dropbox.txt	Mon Feb 21 16:18:22 2011
@@ -32,6 +32,7 @@
 
 * most Python benchmarks run much faster than with CPython or Psyco
 * tight loops run much faster, up to 20-60 times faster
+* speed.pypy.org
 * the real-world PyPy compiler toolchain itself (200 KLocs) runs twice as fast
 * pypy already supports 64bit and is in the process of supporting ARM
 * full compatibility with  CPython (more than Jython/IronPython)


From arigo at codespeak.net  Mon Feb 21 16:19:10 2011
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Mon, 21 Feb 2011 16:19:10 +0100 (CET)
Subject: [pypy-svn] r80379 - pypy/extradoc/talk/ustour2011
Message-ID: <20110221151910.6E1BE282BE3@codespeak.net>

Author: arigo
Date: Mon Feb 21 16:19:09 2011
New Revision: 80379

Modified:
   pypy/extradoc/talk/ustour2011/dropbox.txt
Log:
url-ify

Modified: pypy/extradoc/talk/ustour2011/dropbox.txt
==============================================================================
--- pypy/extradoc/talk/ustour2011/dropbox.txt	(original)
+++ pypy/extradoc/talk/ustour2011/dropbox.txt	Mon Feb 21 16:19:09 2011
@@ -32,7 +32,7 @@
 
 * most Python benchmarks run much faster than with CPython or Psyco
 * tight loops run much faster, up to 20-60 times faster
-* speed.pypy.org
+* http://speed.pypy.org/
 * the real-world PyPy compiler toolchain itself (200 KLocs) runs twice as fast
 * pypy already supports 64bit and is in the process of supporting ARM
 * full compatibility with  CPython (more than Jython/IronPython)


From arigo at codespeak.net  Mon Feb 21 16:20:18 2011
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Mon, 21 Feb 2011 16:20:18 +0100 (CET)
Subject: [pypy-svn] r80380 - pypy/extradoc/talk/ustour2011
Message-ID: <20110221152018.260D92A202B@codespeak.net>

Author: arigo
Date: Mon Feb 21 16:20:16 2011
New Revision: 80380

Modified:
   pypy/extradoc/talk/ustour2011/dropbox.txt
Log:
Add the main url too.

Modified: pypy/extradoc/talk/ustour2011/dropbox.txt
==============================================================================
--- pypy/extradoc/talk/ustour2011/dropbox.txt	(original)
+++ pypy/extradoc/talk/ustour2011/dropbox.txt	Mon Feb 21 16:20:16 2011
@@ -18,7 +18,7 @@
 
 Abstract:
 
-The PyPy project has recently gathered a lot of attention for its
+The PyPy project (http://pypy.org/) has recently gathered a lot of attention for its
 progress in speeding up the Python language -- it is the fastest,
 most compatible and most stable 'alternative? Python interpreter.  No
 longer merely a research curiosity, PyPy is now suitable for production


From lac at codespeak.net  Mon Feb 21 16:22:49 2011
From: lac at codespeak.net (lac at codespeak.net)
Date: Mon, 21 Feb 2011 16:22:49 +0100 (CET)
Subject: [pypy-svn] r80381 - pypy/extradoc/talk/ustour2011
Message-ID: <20110221152249.D8040282BE3@codespeak.net>

Author: lac
Date: Mon Feb 21 16:22:48 2011
New Revision: 80381

Modified:
   pypy/extradoc/talk/ustour2011/dropbox.txt
Log:
make the line not wrap


Modified: pypy/extradoc/talk/ustour2011/dropbox.txt
==============================================================================
--- pypy/extradoc/talk/ustour2011/dropbox.txt	(original)
+++ pypy/extradoc/talk/ustour2011/dropbox.txt	Mon Feb 21 16:22:48 2011
@@ -18,8 +18,8 @@
 
 Abstract:
 
-The PyPy project (http://pypy.org/) has recently gathered a lot of attention for its
-progress in speeding up the Python language -- it is the fastest,
+The PyPy project (http://pypy.org/) has recently gathered a lot of attention
+for its progress in speeding up the Python language -- it is the fastest,
 most compatible and most stable 'alternative? Python interpreter.  No
 longer merely a research curiosity, PyPy is now suitable for production
 use.  Since 2009, the PyPy project has received funding from the Eurostars 
@@ -50,7 +50,7 @@
   which produce research.  What should we do when the funding runs out?
   One thing we would like is to be hired to migrate projects from CPython 
   to PyPy.
-*Any other aspect of interest to audience members. 
+* Any other aspect of interest to audience members. 
 
 If there is something you are interested in which isn't mentioned in this
 abstract, please send a note to lac at openend.se and we will make sure that 


From cfbolz at codespeak.net  Mon Feb 21 17:35:45 2011
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Mon, 21 Feb 2011 17:35:45 +0100 (CET)
Subject: [pypy-svn] r80382 - pypy/extradoc/planning
Message-ID: <20110221163545.4C8D62A2031@codespeak.net>

Author: cfbolz
Date: Mon Feb 21 17:35:43 2011
New Revision: 80382

Modified:
   pypy/extradoc/planning/jit.txt
Log:
a new todo


Modified: pypy/extradoc/planning/jit.txt
==============================================================================
--- pypy/extradoc/planning/jit.txt	(original)
+++ pypy/extradoc/planning/jit.txt	Mon Feb 21 17:35:43 2011
@@ -45,6 +45,9 @@
 - geninterp fun :-(  geninterp'ed functions are not JITted,
   unlike plain app-level functions.  How about we just kill geninterp?
 
+- local imports should be jitted more efficiently, right now they produce a
+  long trace and they are rather common (e.g. in translate.py)
+
 OPTIMIZATIONS
 -------------
 


From commits-noreply at bitbucket.org  Tue Feb 22 12:24:23 2011
From: commits-noreply at bitbucket.org (ademan)
Date: Tue, 22 Feb 2011 12:24:23 +0100 (CET)
Subject: [pypy-svn] pypy default: Added PyUnicode_{Encode,
	Decode}ASCII and tests.
Message-ID: <20110222112423.1D0B636C227@codespeak.net>

Author: Daniel Roberts <Ademan555 at gmail.com>
Branch: 
Changeset: r42201:574f2be3da7b
Date: 2011-02-22 03:23 -0800
http://bitbucket.org/pypy/pypy/changeset/574f2be3da7b/

Log:	Added PyUnicode_{Encode,Decode}ASCII and tests.

diff --git a/pypy/module/cpyext/test/test_unicodeobject.py b/pypy/module/cpyext/test/test_unicodeobject.py
--- a/pypy/module/cpyext/test/test_unicodeobject.py
+++ b/pypy/module/cpyext/test/test_unicodeobject.py
@@ -245,3 +245,28 @@
 
         lltype.free(target_chunk, flavor='raw')
 
+    def test_ascii_codec(self, space, api):
+        s = 'abcdefg'
+        data = rffi.str2charp(s)
+        w_u = api.PyUnicode_DecodeASCII(data, len(s), lltype.nullptr(rffi.CCHARP.TO))
+        assert space.eq_w(w_u, space.wrap(u"abcdefg"))
+        rffi.free_charp(data)
+
+        s = 'abcd\xFF'
+        data = rffi.str2charp(s)
+        w_u = api.PyUnicode_DecodeASCII(data, len(s), lltype.nullptr(rffi.CCHARP.TO))
+        assert w_u is None
+        rffi.free_charp(data)
+
+        uni = u'abcdefg'
+        data = rffi.unicode2wcharp(uni)
+        w_s = api.PyUnicode_EncodeASCII(data, len(uni), lltype.nullptr(rffi.CCHARP.TO))
+        assert space.eq_w(space.wrap("abcdefg"), w_s)
+        rffi.free_wcharp(data)
+
+        u = u'&#65533;bcd&#65533;fg'
+        data = rffi.unicode2wcharp(u)
+        w_s = api.PyUnicode_EncodeASCII(data, len(u), lltype.nullptr(rffi.CCHARP.TO))
+        assert w_s is None
+        rffi.free_wcharp(data)
+

diff --git a/pypy/module/cpyext/stubs.py b/pypy/module/cpyext/stubs.py
--- a/pypy/module/cpyext/stubs.py
+++ b/pypy/module/cpyext/stubs.py
@@ -2654,24 +2654,6 @@
     by the codec."""
     raise NotImplementedError
 
- at cpython_api([rffi.CCHARP, Py_ssize_t, rffi.CCHARP], PyObject)
-def PyUnicode_DecodeASCII(space, s, size, errors):
-    """Create a Unicode object by decoding size bytes of the ASCII encoded string
-    s.  Return NULL if an exception was raised by the codec.
-    
-    This function used an int type for size. This might require
-    changes in your code for properly supporting 64-bit systems."""
-    raise NotImplementedError
-
- at cpython_api([rffi.CWCHARP, Py_ssize_t, rffi.CCHARP], PyObject)
-def PyUnicode_EncodeASCII(space, s, size, errors):
-    """Encode the Py_UNICODE buffer of the given size using ASCII and return a
-    Python string object.  Return NULL if an exception was raised by the codec.
-    
-    This function used an int type for size. This might require
-    changes in your code for properly supporting 64-bit systems."""
-    raise NotImplementedError
-
 @cpython_api([rffi.CCHARP, Py_ssize_t, PyObject, rffi.CCHARP], PyObject)
 def PyUnicode_DecodeCharmap(space, s, size, mapping, errors):
     """Create a Unicode object by decoding size bytes of the encoded string s using

diff --git a/pypy/module/cpyext/unicodeobject.py b/pypy/module/cpyext/unicodeobject.py
--- a/pypy/module/cpyext/unicodeobject.py
+++ b/pypy/module/cpyext/unicodeobject.py
@@ -408,6 +408,39 @@
         else:
             raise
 
+ at cpython_api([rffi.CCHARP, Py_ssize_t, rffi.CCHARP], PyObject)
+def PyUnicode_DecodeASCII(space, s, size, errors):
+    """Create a Unicode object by decoding size bytes of the ASCII encoded string
+    s.  Return NULL if an exception was raised by the codec.
+    
+    This function used an int type for size. This might require
+    changes in your code for properly supporting 64-bit systems."""
+    w_s = space.wrap(rffi.charpsize2str(s, size))
+    try:
+        return space.call_method(w_s, 'decode', space.wrap('ascii'))
+    except OperationError, e:
+        if e.match(space, space.w_UnicodeDecodeError):
+            return None
+        else:
+            raise
+
+ at cpython_api([rffi.CWCHARP, Py_ssize_t, rffi.CCHARP], PyObject)
+def PyUnicode_EncodeASCII(space, s, size, errors):
+    """Encode the Py_UNICODE buffer of the given size using ASCII and return a
+    Python string object.  Return NULL if an exception was raised by the codec.
+    
+    This function used an int type for size. This might require
+    changes in your code for properly supporting 64-bit systems."""
+
+    w_s = space.wrap(rffi.wcharpsize2unicode(s, size))
+    try:
+        return space.call_method(w_s, 'encode', space.wrap('ascii'))
+    except OperationError, e:
+        if e.match(space, space.w_UnicodeEncodeError):
+            return None
+        else:
+            raise
+
 if sys.platform == 'win32':
     @cpython_api([CONST_WSTRING, Py_ssize_t, CONST_STRING], PyObject)
     def PyUnicode_EncodeMBCS(space, wchar_p, length, errors):

From commits-noreply at bitbucket.org  Tue Feb 22 14:45:11 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Tue, 22 Feb 2011 14:45:11 +0100 (CET)
Subject: [pypy-svn] pypy default: add a method for getting all the
	operations in a loop
Message-ID: <20110222134511.EB75F2A2033@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42202:cd51a50500a9
Date: 2011-02-22 11:09 +0100
http://bitbucket.org/pypy/pypy/changeset/cd51a50500a9/

Log:	add a method for getting all the operations in a loop

diff --git a/pypy/module/pypyjit/test_pypy_c/model.py b/pypy/module/pypyjit/test_pypy_c/model.py
--- a/pypy/module/pypyjit/test_pypy_c/model.py
+++ b/pypy/module/pypyjit/test_pypy_c/model.py
@@ -93,3 +93,9 @@
 
     def has_id(self, id):
         return id in self.ids
+
+    def allops(self, include_debug_merge_points=False):
+        for chunk in self.chunks:
+            for op in chunk.operations:
+                if op.name != 'debug_merge_point' or include_debug_merge_points:
+                    yield op

diff --git a/pypy/tool/jitlogparser/parser.py b/pypy/tool/jitlogparser/parser.py
--- a/pypy/tool/jitlogparser/parser.py
+++ b/pypy/tool/jitlogparser/parser.py
@@ -41,8 +41,9 @@
             return '%s(%s)' % (self.name, arglist)
 
     def __repr__(self):
-        return '<%s (%s)>' % (self.name, ', '.join([repr(a)
-                                                    for a in self.args]))
+        return self.repr()
+        ## return '<%s (%s)>' % (self.name, ', '.join([repr(a)
+        ##                                             for a in self.args]))
 
 class SimpleParser(OpParser):
     def parse_args(self, opname, argspec):

diff --git a/pypy/module/pypyjit/test_pypy_c/test_model.py b/pypy/module/pypyjit/test_pypy_c/test_model.py
--- a/pypy/module/pypyjit/test_pypy_c/test_model.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_model.py
@@ -107,3 +107,13 @@
         loop, = log.by_id('increment')
         assert loop.filename == self.filepath
         assert loop.code.co.co_name == 'f'
+        #
+        ops = list(loop.allops())
+        opnames = [op.name for op in ops]
+        assert opnames == [
+            # this is the actual loop
+            'int_lt', 'guard_true', 'int_add',
+            # this is the signal checking stuff
+            'getfield_raw', 'int_sub', 'setfield_raw', 'int_lt', 'guard_false',
+            'jump'
+            ]

From commits-noreply at bitbucket.org  Tue Feb 22 14:45:12 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Tue, 22 Feb 2011 14:45:12 +0100 (CET)
Subject: [pypy-svn] pypy default: add a way to do a very simple
	pattern-matching on the	operations of a loop,
	to ensure that variable matches among different operations
Message-ID: <20110222134512.BFF772A2033@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42203:f3c1f1d24190
Date: 2011-02-22 11:50 +0100
http://bitbucket.org/pypy/pypy/changeset/f3c1f1d24190/

Log:	add a way to do a very simple pattern-matching on the operations of
	a loop, to ensure that variable matches among different operations

diff --git a/pypy/module/pypyjit/test_pypy_c/model.py b/pypy/module/pypyjit/test_pypy_c/model.py
--- a/pypy/module/pypyjit/test_pypy_c/model.py
+++ b/pypy/module/pypyjit/test_pypy_c/model.py
@@ -99,3 +99,50 @@
             for op in chunk.operations:
                 if op.name != 'debug_merge_point' or include_debug_merge_points:
                     yield op
+
+    @classmethod
+    def parse_ops(cls, src):
+        ops = [cls.parse_op(line) for line in src.splitlines()]
+        return [op for op in ops if op is not None]
+
+    @classmethod
+    def parse_op(cls, line):
+        # strip comment
+        if '#' in line:
+            line = line[:line.index('#')]
+        # find the resvar, if any
+        if '=' in line:
+            resvar, _, line = line.partition('=')
+            resvar = resvar.strip()
+        else:
+            resvar = None
+        line = line.strip()
+        if not line:
+            return None
+        opname, _, args = line.partition('(')
+        opname = opname.strip()
+        assert args.endswith(')')
+        args = args[:-1]
+        args = args.split(',')
+        args = map(str.strip, args)
+        return opname, resvar, args
+
+    def match(self, expected_src):
+        alpha_map = {}
+        def match_var(v1, v2):
+            if v1 not in alpha_map:
+                alpha_map[v1] = v2
+            assert alpha_map[v1] == v2
+        #
+        expected_ops = self.parse_ops(expected_src)
+        ops = list(self.allops())
+        assert len(ops) == len(expected_ops)
+        for op, (exp_opname, exp_res, exp_args) in zip(ops, expected_ops):
+            assert op.name == exp_opname
+            match_var(op.res, exp_res)
+            assert len(op.args) == len(exp_args)
+            for arg, exp_arg in zip(op.args, exp_args):
+                match_var(arg, exp_arg)
+        return True
+
+                   

diff --git a/pypy/module/pypyjit/test_pypy_c/test_model.py b/pypy/module/pypyjit/test_pypy_c/test_model.py
--- a/pypy/module/pypyjit/test_pypy_c/test_model.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_model.py
@@ -4,7 +4,7 @@
 from lib_pypy import disassembler
 from pypy.tool.udir import udir
 from pypy.tool import logparser
-from pypy.module.pypyjit.test_pypy_c.model import Log, find_ids_range, find_ids
+from pypy.module.pypyjit.test_pypy_c.model import Log, find_ids_range, find_ids, LoopWithIds
 
 class BaseTestPyPyC(object):
     def setup_class(cls):
@@ -117,3 +117,43 @@
             'getfield_raw', 'int_sub', 'setfield_raw', 'int_lt', 'guard_false',
             'jump'
             ]
+
+    def test_parse_op(self):
+        res = LoopWithIds.parse_op("  a =   int_add(  b,  3 ) # foo")
+        assert res == ("int_add", "a", ["b", "3"])
+        res = LoopWithIds.parse_op("guard_true(a)")
+        assert res == ("guard_true", None, ["a"])
+
+    def test_match(self):
+        def f():
+            i = 0
+            while i < 1003:
+                i += 1 # ID: increment
+            return i
+        #
+        log = self.run(f)
+        loop, = log.by_id('increment')
+        assert loop.match("""
+            i6 = int_lt(i4, 1003)
+            guard_true(i6)
+            i8 = int_add(i4, 1)
+            # signal checking stuff
+            i10 = getfield_raw(37212896)
+            i12 = int_sub(i10, 1)
+            setfield_raw(37212896, i12)
+            i14 = int_lt(i12, 0)
+            guard_false(i14)
+            jump(p0, p1, p2, p3, i8)
+        """)
+
+        py.test.raises(AssertionError, loop.match, """
+            i6 = int_lt(i4, 1003)
+            guard_true(i6)
+            i8 = int_add(i5, 1) # variable mismatch
+            i10 = getfield_raw(37212896)
+            i12 = int_sub(i10, 1)
+            setfield_raw(37212896, i12)
+            i14 = int_lt(i12, 0)
+            guard_false(i14)
+            jump(p0, p1, p2, p3, i8)
+        """)

From commits-noreply at bitbucket.org  Tue Feb 22 14:45:13 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Tue, 22 Feb 2011 14:45:13 +0100 (CET)
Subject: [pypy-svn] pypy default: cosmetic changes
Message-ID: <20110222134513.E00522A2033@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42204:876d7c247938
Date: 2011-02-22 11:54 +0100
http://bitbucket.org/pypy/pypy/changeset/876d7c247938/

Log:	cosmetic changes

diff --git a/pypy/module/pypyjit/test_pypy_c/model.py b/pypy/module/pypyjit/test_pypy_c/model.py
--- a/pypy/module/pypyjit/test_pypy_c/model.py
+++ b/pypy/module/pypyjit/test_pypy_c/model.py
@@ -47,11 +47,17 @@
     def _filter(self, loop, is_entry_bridge=False):
         return is_entry_bridge == '*' or loop.is_entry_bridge == is_entry_bridge
 
-    def by_filename(self, filename, **kwds):
+    def loops_by_filename(self, filename, **kwds):
+        """
+        Return all loops which start in the file ``filename``
+        """
         return [loop for loop in self.loops
                 if loop.filename == filename and self._filter(loop, **kwds)]
 
-    def by_id(self, id, **kwds):
+    def loops_by_id(self, id, **kwds):
+        """
+        Return all loops which contain the ID ``id``
+        """
         return [loop for loop in self.loops
                 if loop.has_id(id) and self._filter(loop, **kwds)]
 
@@ -132,17 +138,15 @@
         def match_var(v1, v2):
             if v1 not in alpha_map:
                 alpha_map[v1] = v2
-            assert alpha_map[v1] == v2
+            assert alpha_map[v1] == v2, "variable mismatch"
         #
         expected_ops = self.parse_ops(expected_src)
         ops = list(self.allops())
-        assert len(ops) == len(expected_ops)
+        assert len(ops) == len(expected_ops), "wrong number of operations"
         for op, (exp_opname, exp_res, exp_args) in zip(ops, expected_ops):
             assert op.name == exp_opname
             match_var(op.res, exp_res)
-            assert len(op.args) == len(exp_args)
+            assert len(op.args) == len(exp_args), "wrong number of arguments"
             for arg, exp_arg in zip(op.args, exp_args):
                 match_var(arg, exp_arg)
         return True
-
-                   

diff --git a/pypy/module/pypyjit/test_pypy_c/test_model.py b/pypy/module/pypyjit/test_pypy_c/test_model.py
--- a/pypy/module/pypyjit/test_pypy_c/test_model.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_model.py
@@ -84,16 +84,16 @@
             return i
         #
         log = self.run(f)
-        loops = log.by_filename(self.filepath)
+        loops = log.loops_by_filename(self.filepath)
         assert len(loops) == 1
         assert loops[0].filename == self.filepath
         assert not loops[0].is_entry_bridge
         #
-        loops = log.by_filename(self.filepath, is_entry_bridge=True)
+        loops = log.loops_by_filename(self.filepath, is_entry_bridge=True)
         assert len(loops) == 1
         assert loops[0].is_entry_bridge
         #
-        loops = log.by_filename(self.filepath, is_entry_bridge='*')
+        loops = log.loops_by_filename(self.filepath, is_entry_bridge='*')
         assert len(loops) == 2
 
     def test_by_id(self):
@@ -104,7 +104,7 @@
             return i
         #
         log = self.run(f)
-        loop, = log.by_id('increment')
+        loop, = log.loops_by_id('increment')
         assert loop.filename == self.filepath
         assert loop.code.co.co_name == 'f'
         #
@@ -132,7 +132,7 @@
             return i
         #
         log = self.run(f)
-        loop, = log.by_id('increment')
+        loop, = log.loops_by_id('increment')
         assert loop.match("""
             i6 = int_lt(i4, 1003)
             guard_true(i6)
@@ -145,7 +145,7 @@
             guard_false(i14)
             jump(p0, p1, p2, p3, i8)
         """)
-
+        #
         py.test.raises(AssertionError, loop.match, """
             i6 = int_lt(i4, 1003)
             guard_true(i6)

From commits-noreply at bitbucket.org  Tue Feb 22 14:45:21 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Tue, 22 Feb 2011 14:45:21 +0100 (CET)
Subject: [pypy-svn] pypy default: add a way to get only the operations that
	correspond to	the specified ID
Message-ID: <20110222134521.351B2282C23@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42205:7b2b0d388008
Date: 2011-02-22 14:34 +0100
http://bitbucket.org/pypy/pypy/changeset/7b2b0d388008/

Log:	add a way to get only the operations that correspond to the
	specified ID

diff --git a/pypy/module/pypyjit/test_pypy_c/model.py b/pypy/module/pypyjit/test_pypy_c/model.py
--- a/pypy/module/pypyjit/test_pypy_c/model.py
+++ b/pypy/module/pypyjit/test_pypy_c/model.py
@@ -77,7 +77,7 @@
         return res
 
     def compute_ids(self):
-        self.ids = set()
+        self.ids = {}
         self.code = None
         if not self.filename:
             return
@@ -88,7 +88,7 @@
         for id, opcodes in ids.iteritems():
             targetop = opcodes[0]
             if targetop in all_my_opcodes:
-                self.ids.add(id)
+                self.ids[id] = opcodes
 
     def get_set_of_opcodes(self):
         res = set()
@@ -100,12 +100,25 @@
     def has_id(self, id):
         return id in self.ids
 
+    def _ops_for_chunk(self, chunk, include_debug_merge_points):
+        for op in chunk.operations:
+            if op.name != 'debug_merge_point' or include_debug_merge_points:
+                yield op
+
     def allops(self, include_debug_merge_points=False):
         for chunk in self.chunks:
-            for op in chunk.operations:
-                if op.name != 'debug_merge_point' or include_debug_merge_points:
+            for op in self._ops_for_chunk(chunk, include_debug_merge_points):
+                yield op
+
+    def ops_by_id(self, id, include_debug_merge_points=False):
+        target_opcodes = self.ids[id]
+        for chunk in self.chunks:
+            opcode = self.code.map[chunk.bytecode_no]
+            if opcode in target_opcodes:
+                for op in self._ops_for_chunk(chunk, include_debug_merge_points):
                     yield op
 
+
     @classmethod
     def parse_ops(cls, src):
         ops = [cls.parse_op(line) for line in src.splitlines()]

diff --git a/pypy/module/pypyjit/test_pypy_c/test_model.py b/pypy/module/pypyjit/test_pypy_c/test_model.py
--- a/pypy/module/pypyjit/test_pypy_c/test_model.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_model.py
@@ -96,7 +96,7 @@
         loops = log.loops_by_filename(self.filepath, is_entry_bridge='*')
         assert len(loops) == 2
 
-    def test_by_id(self):
+    def test_loops_by_id(self):
         def f():
             i = 0
             while i < 1003:
@@ -118,6 +118,21 @@
             'jump'
             ]
 
+    def test_ops_by_id(self):
+        def f():
+            i = 0
+            while i < 1003:
+                i += 1 # ID: increment
+                a = 0  # to make sure that JUMP_ABSOLUTE is not part of the ID
+            return i
+        #
+        log = self.run(f)
+        loop, = log.loops_by_id('increment')
+        #
+        ops = list(loop.ops_by_id('increment'))
+        opnames = [op.name for op in ops]
+        assert opnames == ['int_add']
+
     def test_parse_op(self):
         res = LoopWithIds.parse_op("  a =   int_add(  b,  3 ) # foo")
         assert res == ("int_add", "a", ["b", "3"])

From commits-noreply at bitbucket.org  Tue Feb 22 14:45:22 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Tue, 22 Feb 2011 14:45:22 +0100 (CET)
Subject: [pypy-svn] pypy default: implement match_by_id
Message-ID: <20110222134522.AAD0F282C23@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42206:a3f1c6e8b3d3
Date: 2011-02-22 14:44 +0100
http://bitbucket.org/pypy/pypy/changeset/a3f1c6e8b3d3/

Log:	implement match_by_id

diff --git a/pypy/module/pypyjit/test_pypy_c/model.py b/pypy/module/pypyjit/test_pypy_c/model.py
--- a/pypy/module/pypyjit/test_pypy_c/model.py
+++ b/pypy/module/pypyjit/test_pypy_c/model.py
@@ -146,7 +146,7 @@
         args = map(str.strip, args)
         return opname, resvar, args
 
-    def match(self, expected_src):
+    def match_ops(self, ops, expected_src):
         alpha_map = {}
         def match_var(v1, v2):
             if v1 not in alpha_map:
@@ -154,7 +154,6 @@
             assert alpha_map[v1] == v2, "variable mismatch"
         #
         expected_ops = self.parse_ops(expected_src)
-        ops = list(self.allops())
         assert len(ops) == len(expected_ops), "wrong number of operations"
         for op, (exp_opname, exp_res, exp_args) in zip(ops, expected_ops):
             assert op.name == exp_opname
@@ -163,3 +162,11 @@
             for arg, exp_arg in zip(op.args, exp_args):
                 match_var(arg, exp_arg)
         return True
+
+    def match(self, expected_src):
+        ops = list(self.allops())
+        return self.match_ops(ops, expected_src)
+
+    def match_by_id(self, id, expected_src):
+        ops = list(self.ops_by_id(id))
+        return self.match_ops(ops, expected_src)

diff --git a/pypy/module/pypyjit/test_pypy_c/test_model.py b/pypy/module/pypyjit/test_pypy_c/test_model.py
--- a/pypy/module/pypyjit/test_pypy_c/test_model.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_model.py
@@ -172,3 +172,17 @@
             guard_false(i14)
             jump(p0, p1, p2, p3, i8)
         """)
+
+    def test_match_by_id(self):
+        def f():
+            i = 0
+            while i < 1003:
+                i += 1 # ID: increment
+                a = 0  # to make sure that JUMP_ABSOLUTE is not part of the ID
+            return i
+        #
+        log = self.run(f)
+        loop, = log.loops_by_id('increment')
+        assert loop.match_by_id('increment', """
+            i1 = int_add(i0, 1)
+        """)

From commits-noreply at bitbucket.org  Tue Feb 22 14:59:46 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 22 Feb 2011 14:59:46 +0100 (CET)
Subject: [pypy-svn] pypy lltrace: A branch in which to add an "lltrace"
	option to the	generated C code, 
Message-ID: <20110222135946.A9504282BF2@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: lltrace
Changeset: r42207:e18bdb04d352
Date: 2011-02-22 11:38 +0100
http://bitbucket.org/pypy/pypy/changeset/e18bdb04d352/

Log:	A branch in which to add an "lltrace" option to the generated C
	code, tracing all memory writes. Should give us a cheap equivalent
	of undodb-gdb.


From commits-noreply at bitbucket.org  Tue Feb 22 14:59:47 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 22 Feb 2011 14:59:47 +0100 (CET)
Subject: [pypy-svn] pypy lltrace: Quick fix.
Message-ID: <20110222135947.4B41B282BF2@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: lltrace
Changeset: r42208:e65a70f4c55f
Date: 2011-02-22 12:03 +0100
http://bitbucket.org/pypy/pypy/changeset/e65a70f4c55f/

Log:	Quick fix.

diff --git a/pypy/translator/goal/targetrpystonedalone.py b/pypy/translator/goal/targetrpystonedalone.py
--- a/pypy/translator/goal/targetrpystonedalone.py
+++ b/pypy/translator/goal/targetrpystonedalone.py
@@ -2,11 +2,12 @@
 from pypy.translator.test import rpystone
 from pypy.translator.goal import richards
 import pypy.interpreter.gateway # needed before sys, order of imports !!!
-from pypy.module.sys.version import svn_revision
+#from pypy.module.sys.version import svn_revision
 
 # __________  Entry point  __________
 
-VERSION = svn_revision()
+#VERSION = svn_revision()
+VERSION = ''
 
 # note that we have %f but no length specifiers in RPython
 

From commits-noreply at bitbucket.org  Tue Feb 22 14:59:48 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 22 Feb 2011 14:59:48 +0100 (CET)
Subject: [pypy-svn] pypy lltrace: Start implementing "make lltrace". Right
	now with	fprintf.
Message-ID: <20110222135948.5DFB6282BF2@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: lltrace
Changeset: r42209:3f26c4717b82
Date: 2011-02-22 13:04 +0100
http://bitbucket.org/pypy/pypy/changeset/3f26c4717b82/

Log:	Start implementing "make lltrace". Right now with fprintf.

diff --git a/pypy/translator/c/genc.py b/pypy/translator/c/genc.py
--- a/pypy/translator/c/genc.py
+++ b/pypy/translator/c/genc.py
@@ -557,6 +557,7 @@
             ('linuxmemchk', '', '$(MAKE) CFLAGS="$(DEBUGFLAGS) -DRPY_ASSERT -DLINUXMEMCHK" $(TARGET)'),
             ('llsafer', '', '$(MAKE) CFLAGS="-O2 -DRPY_LL_ASSERT" $(TARGET)'),
             ('lldebug', '', '$(MAKE) CFLAGS="$(DEBUGFLAGS) -DRPY_ASSERT -DRPY_LL_ASSERT" $(TARGET)'),
+            ('lltrace', '', '$(MAKE) CFLAGS="$(DEBUGFLAGS) -DRPY_ASSERT -DRPY_LL_ASSERT -DRPY_LL_TRACE" $(TARGET)'),
             ('profile', '', '$(MAKE) CFLAGS="-g -O1 -pg $(CFLAGS) -fno-omit-frame-pointer" LDFLAGS="-pg $(LDFLAGS)" $(TARGET)'),
             ]
         if self.has_profopt():

diff --git a/pypy/translator/c/src/support.h b/pypy/translator/c/src/support.h
--- a/pypy/translator/c/src/support.h
+++ b/pypy/translator/c/src/support.h
@@ -103,6 +103,27 @@
 #  define RPyBareItem(array, index)          ((array)[index])
 #endif
 
+
+#ifdef RPY_LL_TRACE
+#  define RPyTraceSet(ptr)   _RPyTraceSet(&(ptr), (long)(ptr))
+#else
+#  define RPyTraceSet(ptr)   /* nothing */
+#endif
+void _RPyTraceSet(void *addr, long newvalue);
+#ifndef PYPY_NOT_MAIN_FILE
+static void *_RPyTrace_start;
+static void *_RPyTrace_stop;
+static void *_RPyTrace_current = NULL;
+void _RPyTraceCreateBuffer(void)
+{
+}
+void _RPyTraceSet(void *addr, long newvalue)
+{
+  fprintf(stderr, "set 0x%x into %p\n", newvalue, addr);
+}
+#endif
+
+
 #ifndef PYPY_STANDALONE
 
 /* prototypes */

diff --git a/pypy/translator/c/funcgen.py b/pypy/translator/c/funcgen.py
--- a/pypy/translator/c/funcgen.py
+++ b/pypy/translator/c/funcgen.py
@@ -484,12 +484,14 @@
             result = '/* %s */' % result
         return result
 
-    def generic_set(self, op, targetexpr):
+    def generic_set(self, op, targetexpr, gckind):
         newvalue = self.expr(op.args[-1], special_case_void=False)
         result = '%s = %s;' % (targetexpr, newvalue)
         T = self.lltypemap(op.args[-1])
         if T is Void:
             result = '/* %s */' % result
+        elif gckind == 'gc':
+            result = '%s RPyTraceSet(%s);' % (result, targetexpr)
         return result
 
     def OP_GETFIELD(self, op, ampersand=''):
@@ -510,7 +512,7 @@
         expr = structdef.ptr_access_expr(self.expr(op.args[0]),
                                          op.args[1].value,
                                          baseexpr_is_const)
-        return self.generic_set(op, expr)
+        return self.generic_set(op, expr, STRUCT._gckind)
 
     def OP_GETSUBSTRUCT(self, op):
         RESULT = self.lltypemap(op.result).TO
@@ -540,7 +542,8 @@
         ptr = self.expr(op.args[0])
         index = self.expr(op.args[1])
         arraydef = self.db.gettypedefnode(ARRAY)
-        return self.generic_set(op, arraydef.itemindex_access_expr(ptr, index))
+        return self.generic_set(op, arraydef.itemindex_access_expr(ptr, index),
+                                ARRAY._gckind)
     OP_BARE_SETARRAYITEM = OP_SETARRAYITEM
 
     def OP_GETARRAYSUBSTRUCT(self, op):
@@ -582,7 +585,9 @@
         return self.generic_get(op, self.interior_expr(op.args))
 
     def OP_BARE_SETINTERIORFIELD(self, op):
-        return self.generic_set(op, self.interior_expr(op.args[:-1]))
+        STRUCT = self.lltypemap(op.args[0]).TO
+        return self.generic_set(op, self.interior_expr(op.args[:-1]),
+                                STRUCT._gckind)
 
     def OP_GETINTERIORARRAYSIZE(self, op):
         expr, ARRAY = self.interior_expr(op.args, True)

From commits-noreply at bitbucket.org  Tue Feb 22 14:59:51 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 22 Feb 2011 14:59:51 +0100 (CET)
Subject: [pypy-svn] pypy lltrace: Write the recording logic in its own file.
	Add a number	that identifies
Message-ID: <20110222135951.E5003282C1D@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: lltrace
Changeset: r42210:7cd9093a53f9
Date: 2011-02-22 13:25 +0100
http://bitbucket.org/pypy/pypy/changeset/7cd9093a53f9/

Log:	Write the recording logic in its own file. Add a number that
	identifies the location of the setfield.

diff --git a/pypy/translator/c/src/debug_lltrace.h b/pypy/translator/c/src/debug_lltrace.h
new file mode 100644
--- /dev/null
+++ b/pypy/translator/c/src/debug_lltrace.h
@@ -0,0 +1,62 @@
+
+void _RPyTraceSet(void *addr, long newvalue, int mark);
+
+
+#ifndef RPY_LL_TRACE /****************************************/
+
+#  define RPyTraceSet(ptr, mark)   /* nothing */
+#  ifndef PYPY_NOT_MAIN_FILE
+void _RPyTraceSet(void *addr, long newvalue, int mark) { }
+#  endif
+
+
+#else /*******************************************************/
+
+
+#  define RPyTraceSet(ptr, mark)   _RPyTraceSet(&(ptr), (long)(ptr), mark)
+
+#  ifndef PYPY_NOT_MAIN_FILE
+
+struct _RPyTrace_s {
+  long mark;
+  void *addr;
+  long newvalue;
+};
+
+static struct _RPyTrace_s *_RPyTrace_start   = NULL;
+static struct _RPyTrace_s *_RPyTrace_stop    = NULL;
+static struct _RPyTrace_s *_RPyTrace_current = NULL;
+static int _RPyTrace_default_size = 134217728;
+
+void _RPyTrace_WrapAround(void)
+{
+  if (_RPyTrace_start == NULL)
+    {
+      char *csize = getenv("PYPYTRACE");
+      int size = csize ? atoi(csize) : 0;
+      if (size <= 0)
+        size = _RPyTrace_default_size;
+      _RPyTrace_start = malloc(size * sizeof(struct _RPyTrace_s));
+      RPyAssert(_RPyTrace_start, "not enough memory to allocate the trace");
+      _RPyTrace_stop = _RPyTrace_start + size;
+    }
+  _RPyTrace_current = _RPyTrace_start;
+  fprintf(stderr, "lltrace: buffer from %p to %p, size %ld entries\n",
+          _RPyTrace_start, _RPyTrace_stop,
+          (long)(_RPyTrace_stop - _RPyTrace_start));
+}
+
+void _RPyTraceSet(void *addr, long newvalue, int mark)
+{
+  if (_RPyTrace_current == _RPyTrace_stop)
+    _RPyTrace_WrapAround();
+  _RPyTrace_current->mark = mark;
+  _RPyTrace_current->addr = addr;
+  _RPyTrace_current->newvalue = newvalue;
+  ++_RPyTrace_current;
+}
+
+#  endif
+
+
+#endif /******************************************************/

diff --git a/pypy/translator/c/src/g_include.h b/pypy/translator/c/src/g_include.h
--- a/pypy/translator/c/src/g_include.h
+++ b/pypy/translator/c/src/g_include.h
@@ -46,6 +46,7 @@
 #  include "src/debug_print.h"
 #  include "src/debug_traceback.h"
 #  include "src/debug_alloc.h"
+#  include "src/debug_lltrace.h"
 #ifndef AVR
 #  include "src/ll_os.h"
 #  include "src/ll_strtod.h"

diff --git a/pypy/translator/c/src/support.h b/pypy/translator/c/src/support.h
--- a/pypy/translator/c/src/support.h
+++ b/pypy/translator/c/src/support.h
@@ -103,27 +103,6 @@
 #  define RPyBareItem(array, index)          ((array)[index])
 #endif
 
-
-#ifdef RPY_LL_TRACE
-#  define RPyTraceSet(ptr)   _RPyTraceSet(&(ptr), (long)(ptr))
-#else
-#  define RPyTraceSet(ptr)   /* nothing */
-#endif
-void _RPyTraceSet(void *addr, long newvalue);
-#ifndef PYPY_NOT_MAIN_FILE
-static void *_RPyTrace_start;
-static void *_RPyTrace_stop;
-static void *_RPyTrace_current = NULL;
-void _RPyTraceCreateBuffer(void)
-{
-}
-void _RPyTraceSet(void *addr, long newvalue)
-{
-  fprintf(stderr, "set 0x%x into %p\n", newvalue, addr);
-}
-#endif
-
-
 #ifndef PYPY_STANDALONE
 
 /* prototypes */

diff --git a/pypy/translator/c/funcgen.py b/pypy/translator/c/funcgen.py
--- a/pypy/translator/c/funcgen.py
+++ b/pypy/translator/c/funcgen.py
@@ -491,8 +491,11 @@
         if T is Void:
             result = '/* %s */' % result
         elif gckind == 'gc':
-            result = '%s RPyTraceSet(%s);' % (result, targetexpr)
+            mark = FunctionCodeGenerator._TRACE_MARK + 1
+            FunctionCodeGenerator._TRACE_MARK = mark
+            result = '%s RPyTraceSet(%s, %d);' % (result, targetexpr, mark)
         return result
+    _TRACE_MARK = 0
 
     def OP_GETFIELD(self, op, ampersand=''):
         assert isinstance(op.args[1], Constant)

From commits-noreply at bitbucket.org  Tue Feb 22 14:59:54 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Tue, 22 Feb 2011 14:59:54 +0100 (CET)
Subject: [pypy-svn] pypy lltrace: Support for generating trace events from
	inside the	assembler
Message-ID: <20110222135954.A0735282C1D@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: lltrace
Changeset: r42211:296023b39a57
Date: 2011-02-22 14:58 +0100
http://bitbucket.org/pypy/pypy/changeset/296023b39a57/

Log:	Support for generating trace events from inside the assembler
	produced by the JIT.

diff --git a/pypy/jit/backend/llsupport/trace.py b/pypy/jit/backend/llsupport/trace.py
new file mode 100644
--- /dev/null
+++ b/pypy/jit/backend/llsupport/trace.py
@@ -0,0 +1,17 @@
+from pypy.rpython.lltypesystem import lltype, rffi
+from pypy.rlib.objectmodel import CDefinedIntSymbolic, we_are_translated
+
+
+# Calling this function adds an entry in the buffer maintained by
+# src/debug_lltrace.h.  Arguments: addr, newvalue, mark.
+trace_set = rffi.llexternal("_RPyTraceSet",
+                            [rffi.CCHARP, rffi.LONG, rffi.LONG],
+                            lltype.Void,
+                            _callable = lambda a, n, m: None,
+                            _nowrapper = True)
+
+_is_tracing = CDefinedIntSymbolic('RPY_IS_TRACING', default=0)
+addr_of_trace_set = CDefinedIntSymbolic('((long)&_RPyTraceSet)', default=0)
+
+def is_tracing():
+    return we_are_translated() and _is_tracing != 0

diff --git a/pypy/jit/backend/x86/assembler.py b/pypy/jit/backend/x86/assembler.py
--- a/pypy/jit/backend/x86/assembler.py
+++ b/pypy/jit/backend/x86/assembler.py
@@ -39,6 +39,7 @@
 from pypy.jit.metainterp.history import ConstInt, BoxInt
 from pypy.jit.codewriter.effectinfo import EffectInfo
 from pypy.jit.codewriter import longlong
+from pypy.jit.backend.llsupport import trace
 
 # darwin requires the stack to be 16 bytes aligned on calls. Same for gcc 4.5.0,
 # better safe than sorry
@@ -1253,7 +1254,30 @@
         else:
             not_implemented("load_from_mem size = %d" % size)
 
-    def save_into_mem(self, dest_addr, value_loc, size_loc):
+    def generate_ll_trace(self, dest_addr, value_loc):
+        # XXX for now, we only trace 32-bit pointer-sized writes
+        self.mc.PUSH_r(eax.value)
+        self.mc.PUSH_r(ecx.value)
+        self.mc.PUSH_r(edx.value)
+        self.mc.PUSH_r(edx.value)
+        self.mc.PUSH_r(edx.value) # 5 pushes + 3 args, keeps 16-bytes alignment
+        # use the current address as the mark
+        self.mc.CALL_l(0)    # this is equivalent to "PUSH(IP)"
+        # push the newvalue
+        self.mc.PUSH(value_loc)
+        # push the address in which we are about to write
+        self.mc.LEA(eax, dest_addr)
+        self.mc.PUSH_r(eax.value)
+        # call (as an indirect call, to avoid needing a relative displacement)
+        self.mc.MOV_ri(eax.value, trace.addr_of_trace_set)
+        self.mc.CALL_r(eax.value)
+        # cancel the 8 pushes
+        self.mc.ADD_ri(esp.value, 5 * WORD)
+        self.mc.POP_r(edx.value)
+        self.mc.POP_r(ecx.value)
+        self.mc.POP_r(eax.value)
+
+    def save_into_mem(self, dest_addr, value_loc, size_loc, is_gc):
         size = size_loc.value
         if isinstance(value_loc, RegLoc) and value_loc.is_xmm:
             self.mc.MOVSD(dest_addr, value_loc)
@@ -1262,6 +1286,8 @@
         elif size == 2:
             self.mc.MOV16(dest_addr, value_loc)
         elif size == 4:
+            if IS_X86_32 and trace.is_tracing() and is_gc:
+                self.generate_ll_trace(dest_addr, value_loc)
             self.mc.MOV32(dest_addr, value_loc)
         elif size == 8:
             if IS_X86_64:
@@ -1298,7 +1324,8 @@
         base_loc, ofs_loc, size_loc, value_loc = arglocs
         assert isinstance(size_loc, ImmedLoc)
         dest_addr = AddressLoc(base_loc, ofs_loc)
-        self.save_into_mem(dest_addr, value_loc, size_loc)
+        self.save_into_mem(dest_addr, value_loc, size_loc,
+                           op.getopnum() == rop.SETFIELD_GC)
 
     def genop_discard_setarrayitem_gc(self, op, arglocs):
         base_loc, ofs_loc, value_loc, size_loc, baseofs = arglocs
@@ -1306,7 +1333,8 @@
         assert isinstance(size_loc, ImmedLoc)
         scale = _get_scale(size_loc.value)
         dest_addr = AddressLoc(base_loc, ofs_loc, scale, baseofs.value)
-        self.save_into_mem(dest_addr, value_loc, size_loc)
+        self.save_into_mem(dest_addr, value_loc, size_loc,
+                           op.getopnum() == rop.SETARRAYITEM_GC)
 
     def genop_discard_strsetitem(self, op, arglocs):
         base_loc, ofs_loc, val_loc = arglocs

diff --git a/pypy/jit/backend/llsupport/llmodel.py b/pypy/jit/backend/llsupport/llmodel.py
--- a/pypy/jit/backend/llsupport/llmodel.py
+++ b/pypy/jit/backend/llsupport/llmodel.py
@@ -18,6 +18,7 @@
 from pypy.jit.backend.llsupport.descr import BaseIntCallDescr, GcPtrCallDescr
 from pypy.jit.backend.llsupport.descr import FloatCallDescr, VoidCallDescr
 from pypy.jit.backend.llsupport.asmmemmgr import AsmMemoryManager
+from pypy.jit.backend.llsupport.trace import trace_set
 from pypy.rpython.annlowlevel import cast_instance_to_base_ptr
 
 
@@ -325,6 +326,9 @@
         ofs, size, sign = self.unpack_arraydescr_size(arraydescr)
         # --- start of GC unsafe code (no GC operation!) ---
         items = rffi.ptradd(rffi.cast(rffi.CCHARP, gcref), ofs)
+        if lltype.typeOf(gcref) is not lltype.Signed:
+            trace_set(rffi.ptradd(items, itemindex * size),
+                      rffi.cast(rffi.LONG, newvalue), -100-size)
         for TYPE, _, itemsize in unroll_basic_sizes:
             if size == itemsize:
                 items = rffi.cast(rffi.CArrayPtr(TYPE), items)
@@ -339,6 +343,8 @@
         self.gc_ll_descr.do_write_barrier(gcref, newvalue)
         # --- start of GC unsafe code (no GC operation!) ---
         items = rffi.ptradd(rffi.cast(rffi.CCHARP, gcref), ofs)
+        trace_set(rffi.ptradd(items, itemindex * WORD),
+                  rffi.cast(rffi.LONG, newvalue), -99)
         items = rffi.cast(rffi.CArrayPtr(lltype.Signed), items)
         items[itemindex] = self.cast_gcref_to_int(newvalue)
         # --- end of GC unsafe code ---
@@ -348,6 +354,9 @@
         ofs = self.unpack_arraydescr(arraydescr)
         # --- start of GC unsafe code (no GC operation!) ---
         items = rffi.ptradd(rffi.cast(rffi.CCHARP, gcref), ofs)
+        if lltype.typeOf(gcref) is not lltype.Signed:
+            trace_set(rffi.ptradd(items, itemindex * symbolic.SIZEOF_FLOAT),
+                      rffi.cast(rffi.LONG, newvalue), -98)
         items = rffi.cast(rffi.CArrayPtr(longlong.FLOATSTORAGE), items)
         items[itemindex] = newvalue
         # --- end of GC unsafe code ---
@@ -426,6 +435,8 @@
         ofs, size, sign = self.unpack_fielddescr_size(fielddescr)
         # --- start of GC unsafe code (no GC operation!) ---
         fieldptr = rffi.ptradd(rffi.cast(rffi.CCHARP, struct), ofs)
+        if lltype.typeOf(struct) is not lltype.Signed:
+            trace_set(fieldptr, rffi.cast(rffi.LONG, newvalue), -10)
         for TYPE, _, itemsize in unroll_basic_sizes:
             if size == itemsize:
                 fieldptr = rffi.cast(rffi.CArrayPtr(TYPE), fieldptr)
@@ -443,6 +454,7 @@
         self.gc_ll_descr.do_write_barrier(struct, newvalue)
         # --- start of GC unsafe code (no GC operation!) ---
         fieldptr = rffi.ptradd(rffi.cast(rffi.CCHARP, struct), ofs)
+        trace_set(fieldptr, rffi.cast(rffi.LONG, newvalue), -9)
         fieldptr = rffi.cast(rffi.CArrayPtr(lltype.Signed), fieldptr)
         fieldptr[0] = self.cast_gcref_to_int(newvalue)
         # --- end of GC unsafe code ---
@@ -452,6 +464,8 @@
         ofs = self.unpack_fielddescr(fielddescr)
         # --- start of GC unsafe code (no GC operation!) ---
         fieldptr = rffi.ptradd(rffi.cast(rffi.CCHARP, struct), ofs)
+        if lltype.typeOf(struct) is not lltype.Signed:
+            trace_set(fieldptr, rffi.cast(rffi.LONG, newvalue), -8)
         fieldptr = rffi.cast(rffi.CArrayPtr(longlong.FLOATSTORAGE), fieldptr)
         fieldptr[0] = newvalue
         # --- end of GC unsafe code ---

diff --git a/pypy/translator/c/src/debug_lltrace.h b/pypy/translator/c/src/debug_lltrace.h
--- a/pypy/translator/c/src/debug_lltrace.h
+++ b/pypy/translator/c/src/debug_lltrace.h
@@ -1,18 +1,21 @@
 
-void _RPyTraceSet(void *addr, long newvalue, int mark);
+void _RPyTraceSet(void *addr, long newvalue, long mark);
 
 
 #ifndef RPY_LL_TRACE /****************************************/
 
+
+#  define RPY_IS_TRACING           0
 #  define RPyTraceSet(ptr, mark)   /* nothing */
 #  ifndef PYPY_NOT_MAIN_FILE
-void _RPyTraceSet(void *addr, long newvalue, int mark) { }
+void _RPyTraceSet(void *addr, long newvalue, long mark) { }
 #  endif
 
 
 #else /*******************************************************/
 
 
+#  define RPY_IS_TRACING           1
 #  define RPyTraceSet(ptr, mark)   _RPyTraceSet(&(ptr), (long)(ptr), mark)
 
 #  ifndef PYPY_NOT_MAIN_FILE
@@ -26,15 +29,15 @@
 static struct _RPyTrace_s *_RPyTrace_start   = NULL;
 static struct _RPyTrace_s *_RPyTrace_stop    = NULL;
 static struct _RPyTrace_s *_RPyTrace_current = NULL;
-static int _RPyTrace_default_size = 134217728;
+static const long _RPyTrace_default_size = 134217728;
 
 void _RPyTrace_WrapAround(void)
 {
   if (_RPyTrace_start == NULL)
     {
-      char *csize = getenv("PYPYTRACE");
-      int size = csize ? atoi(csize) : 0;
-      if (size <= 0)
+      char *csize = getenv("PYPYTRACEBUF");
+      long size = csize ? atol(csize) : 0;
+      if (size <= 1)
         size = _RPyTrace_default_size;
       _RPyTrace_start = malloc(size * sizeof(struct _RPyTrace_s));
       RPyAssert(_RPyTrace_start, "not enough memory to allocate the trace");
@@ -46,7 +49,7 @@
           (long)(_RPyTrace_stop - _RPyTrace_start));
 }
 
-void _RPyTraceSet(void *addr, long newvalue, int mark)
+void _RPyTraceSet(void *addr, long newvalue, long mark)
 {
   if (_RPyTrace_current == _RPyTrace_stop)
     _RPyTrace_WrapAround();

diff --git a/pypy/jit/backend/x86/test/test_ztranslation.py b/pypy/jit/backend/x86/test/test_ztranslation.py
--- a/pypy/jit/backend/x86/test/test_ztranslation.py
+++ b/pypy/jit/backend/x86/test/test_ztranslation.py
@@ -36,6 +36,11 @@
 
             def __init__(self, i):
                 self.i = i
+                self.a = None
+
+        class A(object):
+            def __init__(self, next):
+                self.next = next
 
         @dont_look_inside
         def myabs(x):
@@ -64,6 +69,7 @@
                 k = myabs(j)
                 if k - abs(j):  raise ValueError
                 if k - abs(-j): raise ValueError
+                frame.a = A(frame.a)
             return total * 10
         #
         from pypy.rpython.lltypesystem import lltype, rffi

From commits-noreply at bitbucket.org  Tue Feb 22 17:41:32 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Tue, 22 Feb 2011 17:41:32 +0100 (CET)
Subject: [pypy-svn] pypy default: improve this test
Message-ID: <20110222164132.49D7C2A2036@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42212:2e06bb0dd762
Date: 2011-02-22 17:19 +0100
http://bitbucket.org/pypy/pypy/changeset/2e06bb0dd762/

Log:	improve this test

diff --git a/pypy/module/pypyjit/test_pypy_c/test_model.py b/pypy/module/pypyjit/test_pypy_c/test_model.py
--- a/pypy/module/pypyjit/test_pypy_c/test_model.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_model.py
@@ -176,8 +176,10 @@
     def test_match_by_id(self):
         def f():
             i = 0
+            j = 2000
             while i < 1003:
                 i += 1 # ID: increment
+                j -= 1 # ID: product
                 a = 0  # to make sure that JUMP_ABSOLUTE is not part of the ID
             return i
         #
@@ -186,3 +188,7 @@
         assert loop.match_by_id('increment', """
             i1 = int_add(i0, 1)
         """)
+        assert loop.match_by_id('product', """
+            i4 = int_sub_ovf(i3, 1)
+            guard_no_overflow()
+        """)

From commits-noreply at bitbucket.org  Tue Feb 22 17:41:32 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Tue, 22 Feb 2011 17:41:32 +0100 (CET)
Subject: [pypy-svn] pypy default: add the possibility to return a value
Message-ID: <20110222164132.D9CE02A2036@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42213:19438e0d8289
Date: 2011-02-22 17:25 +0100
http://bitbucket.org/pypy/pypy/changeset/19438e0d8289/

Log:	add the possibility to return a value

diff --git a/pypy/module/pypyjit/test_pypy_c/test_model.py b/pypy/module/pypyjit/test_pypy_c/test_model.py
--- a/pypy/module/pypyjit/test_pypy_c/test_model.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_model.py
@@ -22,7 +22,7 @@
         # write the snippet
         with self.filepath.open("w") as f:
             f.write(str(py.code.Source(func)) + "\n")
-            f.write("%s()\n" % func.func_name)
+            f.write("print %s()\n" % func.func_name)
         #
         # run a child pypy-c with logging enabled
         logfile = self.filepath.new(ext='.log')
@@ -43,6 +43,7 @@
         rawlog = logparser.parse_log_file(str(logfile))
         rawtraces = logparser.extract_category(rawlog, 'jit-log-opt-')
         log = Log(func, rawtraces)
+        log.result = eval(stdout)
         return log
 
 
@@ -84,6 +85,7 @@
             return i
         #
         log = self.run(f)
+        assert log.result == 1003
         loops = log.loops_by_filename(self.filepath)
         assert len(loops) == 1
         assert loops[0].filename == self.filepath

From commits-noreply at bitbucket.org  Tue Feb 22 17:41:33 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Tue, 22 Feb 2011 17:41:33 +0100 (CET)
Subject: [pypy-svn] pypy default: add the possibility to pass parameters
Message-ID: <20110222164133.59E652A2036@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42214:c7277f58c1f5
Date: 2011-02-22 17:30 +0100
http://bitbucket.org/pypy/pypy/changeset/c7277f58c1f5/

Log:	add the possibility to pass parameters

diff --git a/pypy/module/pypyjit/test_pypy_c/test_model.py b/pypy/module/pypyjit/test_pypy_c/test_model.py
--- a/pypy/module/pypyjit/test_pypy_c/test_model.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_model.py
@@ -18,11 +18,12 @@
     def setup_method(self, meth):
         self.filepath = self.tmpdir.join(meth.im_func.func_name + '.py')
 
-    def run(self, func, threshold=1000):
+    def run(self, func, args=[], threshold=1000):
         # write the snippet
+        arglist = ', '.join(map(repr, args))
         with self.filepath.open("w") as f:
             f.write(str(py.code.Source(func)) + "\n")
-            f.write("print %s()\n" % func.func_name)
+            f.write("print %s(%s)\n" % (func.func_name, arglist))
         #
         # run a child pypy-c with logging enabled
         logfile = self.filepath.new(ext='.log')
@@ -77,6 +78,12 @@
 
 class TestRunPyPyC(BaseTestPyPyC):
 
+    def test_run_function(self):
+        def f(a, b):
+            return a+b
+        log = self.run(f, [30, 12])
+        assert log.result == 42
+
     def test_parse_jitlog(self):
         def f():
             i = 0

From commits-noreply at bitbucket.org  Tue Feb 22 17:41:33 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Tue, 22 Feb 2011 17:41:33 +0100 (CET)
Subject: [pypy-svn] pypy default: port the first test from test_pypy_c to
	test_pypy_c_new	:-)
Message-ID: <20110222164133.CF9112A2037@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42215:9cb986673405
Date: 2011-02-22 17:41 +0100
http://bitbucket.org/pypy/pypy/changeset/9cb986673405/

Log:	port the first test from test_pypy_c to test_pypy_c_new :-)

diff --git a/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py b/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py
--- a/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py
@@ -33,4 +33,34 @@
             )
 
 class TestPyPyCNew(BaseTestPyPyC):
-    pass
+
+    def test_f1(self):
+        def f1(n):
+            "Arbitrary test function."
+            i = 0
+            x = 1
+            while i<n:
+                j = 0
+                while j<=i:
+                    j = j + 1
+                    x = x + (i&j)
+                i = i + 1
+            return x
+        log = self.run(f1, [2117])
+        assert log.result == 1083876708
+        loop, = log.loops_by_filename(self.filepath)
+        assert loop.match("""
+            i9 = int_le(i7, i8)
+            guard_true(i9)
+            i11 = int_add_ovf(i7, 1)
+            guard_no_overflow()
+            i12 = int_and(i8, i11)
+            i13 = int_add_ovf(i6, i12)
+            guard_no_overflow()
+            i16 = getfield_raw(37212896)
+            i18 = int_sub(i16, 1)
+            setfield_raw(37212896, i18)
+            i20 = int_lt(i18, 0)
+            guard_false(i20)
+            jump(p0, p1, p2, p3, p4, p5, i13, i11, i8)
+        """)

From commits-noreply at bitbucket.org  Tue Feb 22 18:45:31 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Tue, 22 Feb 2011 18:45:31 +0100 (CET)
Subject: [pypy-svn] pypy arm-backend-2: move procedure generation a bit
	around
Message-ID: <20110222174531.CF76C2A2036@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backend-2
Changeset: r42216:8926c712630e
Date: 2011-02-21 21:49 +0100
http://bitbucket.org/pypy/pypy/changeset/8926c712630e/

Log:	move procedure generation a bit around

diff --git a/pypy/jit/backend/arm/assembler.py b/pypy/jit/backend/arm/assembler.py
--- a/pypy/jit/backend/arm/assembler.py
+++ b/pypy/jit/backend/arm/assembler.py
@@ -92,7 +92,8 @@
                                                       ll_new_unicode)
         self.memcpy_addr = self.cpu.cast_ptr_to_int(memcpy_fn)
         self._exit_code_addr = self._gen_exit_path()
-        self._gen_leave_jitted_hook()
+        self._leave_jitted_jook_save_exc = self._gen_leave_jitted_hook_code(True)
+        self._leave_jitted_jook = self._gen_leave_jitted_hook_code(False)
 
     def setup_failure_recovery(self):
 
@@ -200,20 +201,16 @@
         mem[i+2] = chr((n >> 16) & 0xFF)
         mem[i+3] = chr((n >> 24) & 0xFF)
 
-    def _gen_leave_jitted_hook(self):
-        def gen_code(save_exc=False):
-            mc = ARMv7Builder()
-            mc.PUSH([reg.value for reg in r.caller_resp] + [r.ip.value])
-            addr = self.cpu.get_on_leave_jitted_int(save_exception=save_exc)
-            mc.BL(addr)
-            mc.POP([reg.value for reg in r.caller_resp]+[r.ip.value])
-            assert self._exit_code_addr != 0
-            mc.B(self._exit_code_addr)
-            return mc.materialize(self.cpu.asmmemmgr, [],
-                                   self.cpu.gc_ll_descr.gcrootmap)
-        self._leave_jitted_jook_save_exc = gen_code(True)
-        self._leave_jitted_jook = gen_code(False)
-
+    def _gen_leave_jitted_hook_code(self, save_exc=False):
+        mc = ARMv7Builder()
+        mc.PUSH([reg.value for reg in r.caller_resp] + [r.ip.value])
+        addr = self.cpu.get_on_leave_jitted_int(save_exception=save_exc)
+        mc.BL(addr)
+        mc.POP([reg.value for reg in r.caller_resp]+[r.ip.value])
+        assert self._exit_code_addr != 0
+        mc.B(self._exit_code_addr)
+        return mc.materialize(self.cpu.asmmemmgr, [],
+                               self.cpu.gc_ll_descr.gcrootmap)
     def _gen_exit_path(self):
         mc = ARMv7Builder()
         decode_registers_addr = llhelper(self.recovery_func_sign, self.failure_recovery_func)

From commits-noreply at bitbucket.org  Tue Feb 22 18:45:32 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Tue, 22 Feb 2011 18:45:32 +0100 (CET)
Subject: [pypy-svn] pypy arm-backend-2: allow to move const values to stack
	locations
Message-ID: <20110222174532.9D2952A2036@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backend-2
Changeset: r42217:7834f275ce27
Date: 2011-02-22 18:21 +0100
http://bitbucket.org/pypy/pypy/changeset/7834f275ce27/

Log:	allow to move const values to stack locations

diff --git a/pypy/jit/backend/arm/assembler.py b/pypy/jit/backend/arm/assembler.py
--- a/pypy/jit/backend/arm/assembler.py
+++ b/pypy/jit/backend/arm/assembler.py
@@ -559,15 +559,23 @@
 
     def regalloc_mov(self, prev_loc, loc, cond=c.AL):
         if prev_loc.is_imm():
+            if loc.is_reg():
+                new_loc = loc
+            else:
+                assert loc is not r.ip
+                new_loc = r.ip
             if _check_imm_arg(ConstInt(prev_loc.getint())):
-                self.mc.MOV_ri(loc.value, prev_loc.getint(), cond=cond)
+                self.mc.MOV_ri(new_loc.value, prev_loc.getint(), cond=cond)
             else:
-                self.mc.gen_load_int(loc.value, prev_loc.getint(), cond=cond)
-        elif loc.is_stack():
+                self.mc.gen_load_int(new_loc.value, prev_loc.getint(), cond=cond)
+            prev_loc = new_loc
+            if not loc.is_stack():
+                return 
+        if loc.is_stack() and prev_loc.is_reg():
             self.mc.STR_ri(prev_loc.value, r.fp.value, loc.position*-WORD, cond=cond)
-        elif prev_loc.is_stack():
+        elif loc.is_reg() and prev_loc.is_stack():
             self.mc.LDR_ri(loc.value, r.fp.value, prev_loc.position*-WORD, cond=cond)
-        else:
+        elif loc.is_reg() and prev_loc.is_reg():
             self.mc.MOV_rr(loc.value, prev_loc.value, cond=cond)
     mov_loc_loc = regalloc_mov
 

diff --git a/pypy/jit/backend/arm/test/test_regalloc.py b/pypy/jit/backend/arm/test/test_regalloc.py
--- a/pypy/jit/backend/arm/test/test_regalloc.py
+++ b/pypy/jit/backend/arm/test/test_regalloc.py
@@ -631,3 +631,11 @@
         self.run(loop)
         assert self.getint(0) == 29
 
+    def test_jump_with_consts(self):
+        loop = """ 
+        [i0, i1, i2, i3, i4, i5, i6, i7, i8, i9, i10, i11, i12, i13, i14]
+        jump(i1, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15)
+        """ 
+        large = self.interpret(loop, range(15), run=False)
+        # ensure compiling this loop works
+        assert 1

From commits-noreply at bitbucket.org  Tue Feb 22 18:45:33 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Tue, 22 Feb 2011 18:45:33 +0100 (CET)
Subject: [pypy-svn] pypy arm-backend-2: generate code to set the SP at a
	position where	it is executed before the frame is used
Message-ID: <20110222174533.429892A2036@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backend-2
Changeset: r42218:774fcb133e04
Date: 2011-02-22 18:29 +0100
http://bitbucket.org/pypy/pypy/changeset/774fcb133e04/

Log:	generate code to set the SP at a position where it is executed
	before the frame is used

diff --git a/pypy/jit/backend/arm/assembler.py b/pypy/jit/backend/arm/assembler.py
--- a/pypy/jit/backend/arm/assembler.py
+++ b/pypy/jit/backend/arm/assembler.py
@@ -371,8 +371,8 @@
 
         self.align()
         self.gen_func_prolog()
+        sp_patch_location = self._prepare_sp_patch_position()
         arglocs = self.gen_bootstrap_code(inputargs, regalloc, looptoken)
-        sp_patch_location = self._prepare_sp_patch_position()
 
         loop_head = self.mc.currpos()
 
@@ -411,8 +411,8 @@
 
         frame_depth = faildescr._arm_frame_depth
         locs = self.decode_inputargs(enc, inputargs, regalloc)
+        sp_patch_location = self._prepare_sp_patch_position()
         regalloc.update_bindings(locs, frame_depth, inputargs)
-        sp_patch_location = self._prepare_sp_patch_position()
 
         self._walk_operations(operations, regalloc)
 

From commits-noreply at bitbucket.org  Tue Feb 22 18:45:36 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Tue, 22 Feb 2011 18:45:36 +0100 (CET)
Subject: [pypy-svn] pypy arm-backend-2: (arigo,
	david) patch the frame size of a loop or bridge with a larger
	value	if it jumps to another location that requires a larger
	frame	than the current one would need
Message-ID: <20110222174536.0422A2A203B@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backend-2
Changeset: r42219:65a51f86470e
Date: 2011-02-22 18:42 +0100
http://bitbucket.org/pypy/pypy/changeset/65a51f86470e/

Log:	(arigo, david) patch the frame size of a loop or bridge with a
	larger value if it jumps to another location that requires a larger
	frame than the current one would need

diff --git a/pypy/jit/backend/arm/assembler.py b/pypy/jit/backend/arm/assembler.py
--- a/pypy/jit/backend/arm/assembler.py
+++ b/pypy/jit/backend/arm/assembler.py
@@ -326,7 +326,7 @@
         looptoken._arm_arglocs = arglocs
         return arglocs
 
-    def gen_direct_bootstrap_code(self, arglocs, loop_head, regalloc):
+    def gen_direct_bootstrap_code(self, arglocs, loop_head, looptoken):
         self.gen_func_prolog()
         if len(arglocs) > 4:
             reg_args = 4
@@ -351,7 +351,7 @@
                 assert 0, 'invalid location'
         sp_patch_location = self._prepare_sp_patch_position()
         self.mc.B_offs(loop_head)
-        self._patch_sp_offset(sp_patch_location, regalloc)
+        self._patch_sp_offset(sp_patch_location, looptoken)
 
     def _dump(self, ops, type='loop'):
         debug_start('jit-backend-ops')
@@ -381,12 +381,13 @@
 
         self._walk_operations(operations, regalloc)
 
-        self._patch_sp_offset(sp_patch_location, regalloc)
+        looptoken._arm_frame_depth = regalloc.frame_manager.frame_depth 
+        self._patch_sp_offset(sp_patch_location, looptoken)
 
         self.align()
 
         direct_bootstrap_code = self.mc.currpos()
-        self.gen_direct_bootstrap_code(arglocs, loop_head, regalloc)
+        self.gen_direct_bootstrap_code(arglocs, loop_head, looptoken)
 
         loop_start = self.materialize_loop(looptoken)
         looptoken._arm_bootstrap_code = loop_start
@@ -416,7 +417,8 @@
 
         self._walk_operations(operations, regalloc)
 
-        self._patch_sp_offset(sp_patch_location, regalloc)
+        original_loop_token._arm_frame_depth = regalloc.frame_manager.frame_depth 
+        self._patch_sp_offset(sp_patch_location, original_loop_token)
 
         bridge_start = self.materialize_loop(original_loop_token)
         self.update_descrs_for_bridges(bridge_start)
@@ -460,13 +462,13 @@
             self.mc.MOV_rr(r.r0.value, r.r0.value)
         return l
 
-    def _patch_sp_offset(self, pos, regalloc):
+    def _patch_sp_offset(self, pos, looptoken):
         cb = OverwritingBuilder(self.mc, pos, OverwritingBuilder.size_of_gen_load_int)
         # Note: the frame_depth is one less than the value stored in the frame
         # manager
-        if regalloc.frame_manager.frame_depth == 1:
+        if looptoken._arm_frame_depth == 1:
             return
-        n = (regalloc.frame_manager.frame_depth-1)*WORD
+        n = (looptoken._arm_frame_depth-1)*WORD
         self._adjust_sp(n, cb, base_reg=r.fp)
 
     def _adjust_sp(self, n, cb=None, fcond=c.AL, base_reg=r.sp):

diff --git a/pypy/jit/backend/arm/opassembler.py b/pypy/jit/backend/arm/opassembler.py
--- a/pypy/jit/backend/arm/opassembler.py
+++ b/pypy/jit/backend/arm/opassembler.py
@@ -228,6 +228,8 @@
         else:
             target = descr._arm_bootstrap_code + descr._arm_loop_code
             self.mc.B(target, fcond)
+            new_fd = max(regalloc.frame_manager.frame_depth, descr._arm_frame_depth)
+            regalloc.frame_manager.frame_depth = new_fd
         return fcond
 
     def emit_op_finish(self, op, arglocs, regalloc, fcond):

diff --git a/pypy/jit/backend/test/test_frame_size.py b/pypy/jit/backend/test/test_frame_size.py
new file mode 100644
--- /dev/null
+++ b/pypy/jit/backend/test/test_frame_size.py
@@ -0,0 +1,98 @@
+import py, sys, random, os, struct, operator
+from pypy.jit.metainterp.history import (AbstractFailDescr,
+                                         AbstractDescr,
+                                         BasicFailDescr,
+                                         BoxInt, Box, BoxPtr,
+                                         LoopToken,
+                                         ConstInt, ConstPtr,
+                                         BoxObj, Const,
+                                         ConstObj, BoxFloat, ConstFloat)
+from pypy.jit.metainterp.resoperation import ResOperation, rop
+from pypy.jit.metainterp.typesystem import deref
+from pypy.jit.tool.oparser import parse
+from pypy.rpython.lltypesystem import lltype, llmemory, rstr, rffi, rclass
+from pypy.rpython.ootypesystem import ootype
+from pypy.rpython.annlowlevel import llhelper
+from pypy.rpython.llinterp import LLException
+from pypy.jit.codewriter import heaptracker, longlong
+from pypy.rlib.rarithmetic import intmask
+from pypy.jit.backend.detect_cpu import getcpuclass
+
+CPU = getcpuclass()
+
+class TestFrameSize(object):
+    cpu = CPU(None, None)
+    cpu.setup_once()
+
+    looptoken = None
+    
+    def f1(x):
+        return x+1
+
+    F1PTR = lltype.Ptr(lltype.FuncType([lltype.Signed], lltype.Signed))
+    f1ptr = llhelper(F1PTR, f1)
+    f1_calldescr = cpu.calldescrof(F1PTR.TO, F1PTR.TO.ARGS, F1PTR.TO.RESULT)
+    namespace = locals().copy()
+    type_system = 'lltype'
+
+    def parse(self, s, boxkinds=None):
+        return parse(s, self.cpu, self.namespace,
+                     type_system=self.type_system,
+                     boxkinds=boxkinds)
+
+    def interpret(self, ops, args, run=True):
+        loop = self.parse(ops)
+        self.cpu.compile_loop(loop.inputargs, loop.operations, loop.token)
+        for i, arg in enumerate(args):
+            if isinstance(arg, int):
+                self.cpu.set_future_value_int(i, arg)
+            elif isinstance(arg, float):
+                self.cpu.set_future_value_float(i, arg)
+            else:
+                assert isinstance(lltype.typeOf(arg), lltype.Ptr)
+                llgcref = lltype.cast_opaque_ptr(llmemory.GCREF, arg)
+                self.cpu.set_future_value_ref(i, llgcref)
+        if run:
+            self.cpu.execute_token(loop.token)
+        return loop
+
+    def getint(self, index):
+        return self.cpu.get_latest_value_int(index)
+
+    def getfloat(self, index):
+        return self.cpu.get_latest_value_float(index)
+
+    def getints(self, end):
+        return [self.cpu.get_latest_value_int(index) for
+                index in range(0, end)]
+
+    def getfloats(self, end):
+        return [self.cpu.get_latest_value_float(index) for
+                index in range(0, end)]
+
+    def getptr(self, index, T):
+        gcref = self.cpu.get_latest_value_ref(index)
+        return lltype.cast_opaque_ptr(T, gcref)
+
+    
+
+    def test_call_loop_from_loop(self): 
+
+        large_frame_loop = """ 
+        [i0, i1, i2, i3, i4, i5, i6, i7, i8, i9, i10, i11, i12, i13, i14]
+        i15 = call(ConstClass(f1ptr), i0, descr=f1_calldescr)
+        finish(i0, i1, i2, i3, i4, i5, i6, i7, i8, i9, i10, i11, i12, i13, i14, i15)
+        """ 
+        large = self.interpret(large_frame_loop, range(15), run=False)
+        self.namespace['looptoken'] = large.token
+        assert self.namespace['looptoken']._arm_bootstrap_code != 0
+        small_frame_loop = """
+        [i0]
+        i1 = int_add(i0, 1)
+        jump(i1, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, descr=looptoken)
+        """
+
+        self.interpret(small_frame_loop, [110])
+        expected = [111, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 112]
+        assert self.getints(16) == expected
+

From commits-noreply at bitbucket.org  Tue Feb 22 23:00:07 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Tue, 22 Feb 2011 23:00:07 +0100 (CET)
Subject: [pypy-svn] jitviewer default: Allow specifying an optional port
	number.
Message-ID: <20110222220007.37543282BF2@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r85:225538520836
Date: 2011-02-22 16:59 -0500
http://bitbucket.org/pypy/jitviewer/changeset/225538520836/

Log:	Allow specifying an optional port number.

diff --git a/bin/jitviewer.py b/bin/jitviewer.py
--- a/bin/jitviewer.py
+++ b/bin/jitviewer.py
@@ -1,7 +1,7 @@
 #!/usr/bin/env pypy-c
 """ A web-based browser of your log files. Run by
 
-jitviewer.py <path to your log file>
+jitviewer.py <path to your log file> [port]
 
 and point your browser to http://localhost:5000
 
@@ -85,7 +85,7 @@
                 path_so_far.append(e)
         callstack.append((','.join(path_so_far), '%s in %s at %d' % (loop.name,
                                         loop.filename, loop.startlineno)))
-        
+
         startline, endline = loop.linerange
         if loop.filename is not None:
             code = self.storage.load_code(loop.filename)[loop.startlineno]
@@ -114,7 +114,7 @@
 
 class OverrideFlask(flask.Flask):
     root_path = property(lambda self: self._root_path, lambda *args: None)
-    
+
     def __init__(self, *args, **kwargs):
         self._root_path = kwargs.pop('root_path')
         flask.Flask.__init__(self, *args, **kwargs)
@@ -126,11 +126,15 @@
     if not '__pypy__' in sys.builtin_module_names:
         print "Please run it using pypy-c"
         sys.exit(1)
-    if len(sys.argv) != 2:
+    if len(sys.argv) != 2 and len(sys.argv) != 3:
         print __doc__
         sys.exit(1)
     log = parse_log_file(sys.argv[1])
     extra_path = os.path.dirname(sys.argv[1])
+    if len(sys.argv) != 3:
+        port = 5000
+    else:
+        port = int(sys.argv[2])
     storage = LoopStorage(extra_path)
     loops = [parse(l) for l in extract_category(log, "jit-log-opt-")]
     parse_log_counts(extract_category(log, 'jit-backend-count'), loops)
@@ -141,7 +145,7 @@
     app.route('/')(server.index)
     app.route('/loop')(server.loop)
     #th = start_browser('http://localhost:5000/')
-    app.run(use_reloader=False, host='0.0.0.0')
+    app.run(use_reloader=False, host='0.0.0.0', port=port)
     #th.join()
 
 if __name__ == '__main__':

From commits-noreply at bitbucket.org  Wed Feb 23 00:31:01 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Wed, 23 Feb 2011 00:31:01 +0100 (CET)
Subject: [pypy-svn] pypy default: A broken test for test_pypy_c_new
Message-ID: <20110222233101.9F0FE282BF2@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r42220:403262457eab
Date: 2011-02-22 18:30 -0500
http://bitbucket.org/pypy/pypy/changeset/403262457eab/

Log:	A broken test for test_pypy_c_new

diff --git a/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py b/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py
--- a/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py
@@ -33,7 +33,6 @@
             )
 
 class TestPyPyCNew(BaseTestPyPyC):
-
     def test_f1(self):
         def f1(n):
             "Arbitrary test function."
@@ -64,3 +63,25 @@
             guard_false(i20)
             jump(p0, p1, p2, p3, p4, p5, i13, i11, i8)
         """)
+
+    def test_cmp_exc(self):
+        py.test.skip("WIP")
+
+        def f1(n):
+            def f():
+                raise KeyError
+
+            i = 0
+            while i < n:
+                try:
+                    f()
+                except KeyError:
+                    i += 1
+            return i
+
+        log = self.run(f1, [10000])
+        assert log.result == 10000
+        loop, = log.loops_by_filename(self.filepath)
+        assert loop.get_bytecode("COMPARE_OP").match_stats(
+            call=0
+        )
\ No newline at end of file

From lac at codespeak.net  Wed Feb 23 14:02:36 2011
From: lac at codespeak.net (lac at codespeak.net)
Date: Wed, 23 Feb 2011 14:02:36 +0100 (CET)
Subject: [pypy-svn] r80384 - pypy/extradoc/pycon-advertisement
Message-ID: <20110223130236.C66D5282BE7@codespeak.net>

Author: lac
Date: Wed Feb 23 14:02:34 2011
New Revision: 80384

Added:
   pypy/extradoc/pycon-advertisement/
   pypy/extradoc/pycon-advertisement/draft.1
Log:
First draft, copy-pasted from email from jacob


Added: pypy/extradoc/pycon-advertisement/draft.1
==============================================================================
--- (empty file)
+++ pypy/extradoc/pycon-advertisement/draft.1	Wed Feb 23 14:02:34 2011
@@ -0,0 +1,22 @@
+Support PyPy
+============
+
+While we are happy to help you getting your code to work with PyPy,
+you may want something more than a bit of guidance over IRC. Some
+members of the PyPy development team are available for consulting
+work. Here are some examples of what they could do for you:
+
+- Commercial support contracts
+- On site mentoring of developers
+- Make your codebase compatible with PyPy
+- Port your favourite C/C++ extension library to PyPy
+- Build tools that are compatible with SWIG/Boost/Cython/whatever
+- Make Stackless Python work with the JIT
+- Write a backend for your favourite CPU architecture (X86, amd64 and ARM are 
+already there)
+- Make code that works under PyPy go even faster
+- Write an interpreter for your second favourite dynamic programming language
+- Implement Python 3.x features
+- Build a code obfuscator that is not trivial to work around
+
+Send an email to pypy at openend.se and we will get in touch with you.


From commits-noreply at bitbucket.org  Wed Feb 23 14:54:31 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed, 23 Feb 2011 14:54:31 +0100 (CET)
Subject: [pypy-svn] pypy default: Add another combination of this
	instruction.
Message-ID: <20110223135431.CCC57282BE7@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r42221:9c984a48fff6
Date: 2011-02-22 10:14 +0000
http://bitbucket.org/pypy/pypy/changeset/9c984a48fff6/

Log:	Add another combination of this instruction.

diff --git a/pypy/jit/backend/x86/rx86.py b/pypy/jit/backend/x86/rx86.py
--- a/pypy/jit/backend/x86/rx86.py
+++ b/pypy/jit/backend/x86/rx86.py
@@ -455,12 +455,12 @@
     CMP_mi8 = insn(rex_w, '\x83', orbyte(7<<3), mem_reg_plus_const(1), immediate(2, 'b'))
     CMP_mi32 = insn(rex_w, '\x81', orbyte(7<<3), mem_reg_plus_const(1), immediate(2))
     CMP_mi = select_8_or_32_bit_immed(CMP_mi8, CMP_mi32)
-
     CMP_mr = insn(rex_w, '\x39', register(2, 8), mem_reg_plus_const(1))
 
     CMP_ji8 = insn(rex_w, '\x83', '\x3D', immediate(1), immediate(2, 'b'))
     CMP_ji32 = insn(rex_w, '\x81', '\x3D', immediate(1), immediate(2))
     CMP_ji = select_8_or_32_bit_immed(CMP_ji8, CMP_ji32)
+    CMP_jr = insn(rex_w, '\x39', register(2, 8), '\x05', immediate(1))
 
     CMP32_mi = insn(rex_nw, '\x81', orbyte(7<<3), mem_reg_plus_const(1), immediate(2))
 

From commits-noreply at bitbucket.org  Wed Feb 23 14:54:32 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed, 23 Feb 2011 14:54:32 +0100 (CET)
Subject: [pypy-svn] pypy lltrace: Fix.
Message-ID: <20110223135432.8C7C7282BE7@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: lltrace
Changeset: r42222:324614f81b83
Date: 2011-02-22 17:12 +0100
http://bitbucket.org/pypy/pypy/changeset/324614f81b83/

Log:	Fix.

diff --git a/pypy/jit/backend/x86/assembler.py b/pypy/jit/backend/x86/assembler.py
--- a/pypy/jit/backend/x86/assembler.py
+++ b/pypy/jit/backend/x86/assembler.py
@@ -1256,6 +1256,7 @@
 
     def generate_ll_trace(self, dest_addr, value_loc):
         # XXX for now, we only trace 32-bit pointer-sized writes
+        assert IS_X86_32    # <--- for PUSH_i32 below
         self.mc.PUSH_r(eax.value)
         self.mc.PUSH_r(ecx.value)
         self.mc.PUSH_r(edx.value)
@@ -1264,7 +1265,11 @@
         # use the current address as the mark
         self.mc.CALL_l(0)    # this is equivalent to "PUSH(IP)"
         # push the newvalue
-        self.mc.PUSH(value_loc)
+        if isinstance(value_loc, RegLoc):
+            self.mc.PUSH_r(value_loc.value)
+        else:
+            assert isinstance(value_loc, ImmedLoc)
+            self.mc.PUSH_i32(value_loc.value)
         # push the address in which we are about to write
         self.mc.LEA(eax, dest_addr)
         self.mc.PUSH_r(eax.value)

From commits-noreply at bitbucket.org  Wed Feb 23 14:54:33 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed, 23 Feb 2011 14:54:33 +0100 (CET)
Subject: [pypy-svn] pypy lltrace: aaAAAah. It's not a SETFIELD_GC. Because
	it was	replaced with
Message-ID: <20110223135433.80DE1282BE7@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: lltrace
Changeset: r42223:62ea17f625a4
Date: 2011-02-23 14:48 +0100
http://bitbucket.org/pypy/pypy/changeset/62ea17f625a4/

Log:	aaAAAah. It's not a SETFIELD_GC. Because it was replaced with
	SETFIELD_RAW earlier. Grrrr.

diff --git a/pypy/jit/backend/x86/assembler.py b/pypy/jit/backend/x86/assembler.py
--- a/pypy/jit/backend/x86/assembler.py
+++ b/pypy/jit/backend/x86/assembler.py
@@ -1282,7 +1282,7 @@
         self.mc.POP_r(ecx.value)
         self.mc.POP_r(eax.value)
 
-    def save_into_mem(self, dest_addr, value_loc, size_loc, is_gc):
+    def save_into_mem(self, dest_addr, value_loc, size_loc):
         size = size_loc.value
         if isinstance(value_loc, RegLoc) and value_loc.is_xmm:
             self.mc.MOVSD(dest_addr, value_loc)
@@ -1291,7 +1291,7 @@
         elif size == 2:
             self.mc.MOV16(dest_addr, value_loc)
         elif size == 4:
-            if IS_X86_32 and trace.is_tracing() and is_gc:
+            if IS_X86_32 and trace.is_tracing():
                 self.generate_ll_trace(dest_addr, value_loc)
             self.mc.MOV32(dest_addr, value_loc)
         elif size == 8:
@@ -1329,8 +1329,7 @@
         base_loc, ofs_loc, size_loc, value_loc = arglocs
         assert isinstance(size_loc, ImmedLoc)
         dest_addr = AddressLoc(base_loc, ofs_loc)
-        self.save_into_mem(dest_addr, value_loc, size_loc,
-                           op.getopnum() == rop.SETFIELD_GC)
+        self.save_into_mem(dest_addr, value_loc, size_loc)
 
     def genop_discard_setarrayitem_gc(self, op, arglocs):
         base_loc, ofs_loc, value_loc, size_loc, baseofs = arglocs
@@ -1338,8 +1337,7 @@
         assert isinstance(size_loc, ImmedLoc)
         scale = _get_scale(size_loc.value)
         dest_addr = AddressLoc(base_loc, ofs_loc, scale, baseofs.value)
-        self.save_into_mem(dest_addr, value_loc, size_loc,
-                           op.getopnum() == rop.SETARRAYITEM_GC)
+        self.save_into_mem(dest_addr, value_loc, size_loc)
 
     def genop_discard_strsetitem(self, op, arglocs):
         base_loc, ofs_loc, val_loc = arglocs

From commits-noreply at bitbucket.org  Wed Feb 23 14:54:33 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed, 23 Feb 2011 14:54:33 +0100 (CET)
Subject: [pypy-svn] pypy default: merge heads
Message-ID: <20110223135433.01744282BEB@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r42224:fb46285d3360
Date: 2011-02-23 13:54 +0000
http://bitbucket.org/pypy/pypy/changeset/fb46285d3360/

Log:	merge heads


From lac at codespeak.net  Wed Feb 23 15:13:02 2011
From: lac at codespeak.net (lac at codespeak.net)
Date: Wed, 23 Feb 2011 15:13:02 +0100 (CET)
Subject: [pypy-svn] r80385 - pypy/extradoc/pycon-advertisement
Message-ID: <20110223141302.48A76282BDE@codespeak.net>

Author: lac
Date: Wed Feb 23 15:13:00 2011
New Revision: 80385

Modified:
   pypy/extradoc/pycon-advertisement/draft.1
Log:
Expand what I started with


Modified: pypy/extradoc/pycon-advertisement/draft.1
==============================================================================
--- pypy/extradoc/pycon-advertisement/draft.1	(original)
+++ pypy/extradoc/pycon-advertisement/draft.1	Wed Feb 23 15:13:00 2011
@@ -4,19 +4,32 @@
 While we are happy to help you getting your code to work with PyPy,
 you may want something more than a bit of guidance over IRC. Some
 members of the PyPy development team are available for consulting
-work. Here are some examples of what they could do for you:
+work. Here are some examples of what we could do for you:
+
+- community development: On site mentoring of developers, company sprints
+- support: Commercial support contracts -- if it breaks, we will fix it
+- speed improvements for you:  We could profile and analyse your code, 
+  looking for ways  to make it more 'jit-friendly', and then either simply 
+  report what we found, or make the suggested changes in your codebase.
+- speed improvements for you: There are many ways we could improve our jit.
+  You could pay us to make the improvements which do the most for your 
+  codebase first.
+- speed improvements: work on making our generational GC more sophisticated
+- porting: Make PyPy support Python 3.0
+- porting: We could port  your favourite C/C++ extension library to RPython, 
+  so that PyPy can jit it.
+- porting: Build a facility for separate compilation, so you can more easily 
+  port your own extension libraries to RPython.
+- integration: integrate PyPy with Cython
+- integration: integrate PyPy with NumPy
+- whole new world: Make Stackless Python work with the JIT
+- whole new world: Write a backend for your favourite CPU architecture (the 
+  X86, amd64 and ARM are already there)
+- whole new world: Make the JIT work with the JVM
+- whole new world: Make the JIT work with .NET
+- whole new world: Write an interpreter for your second favourite dynamic 
+  programming language, so it can be sped up as well.
+- whole new world: Build a code obfuscator that is not trivial to work around
 
-- Commercial support contracts
-- On site mentoring of developers
-- Make your codebase compatible with PyPy
-- Port your favourite C/C++ extension library to PyPy
-- Build tools that are compatible with SWIG/Boost/Cython/whatever
-- Make Stackless Python work with the JIT
-- Write a backend for your favourite CPU architecture (X86, amd64 and ARM are 
-already there)
-- Make code that works under PyPy go even faster
-- Write an interpreter for your second favourite dynamic programming language
-- Implement Python 3.x features
-- Build a code obfuscator that is not trivial to work around
 
 Send an email to pypy at openend.se and we will get in touch with you.


From commits-noreply at bitbucket.org  Wed Feb 23 15:34:28 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Wed, 23 Feb 2011 15:34:28 +0100 (CET)
Subject: [pypy-svn] pypy default: Rename Bytecode to OpCode
Message-ID: <20110223143428.1063C2A2033@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42225:459181aa134c
Date: 2011-02-23 10:32 +0100
http://bitbucket.org/pypy/pypy/changeset/459181aa134c/

Log:	Rename Bytecode to OpCode

diff --git a/pypy/tool/jitlogparser/parser.py b/pypy/tool/jitlogparser/parser.py
--- a/pypy/tool/jitlogparser/parser.py
+++ b/pypy/tool/jitlogparser/parser.py
@@ -68,7 +68,7 @@
 class NonCodeError(Exception):
     pass
 
-class Bytecode(object):
+class OpCode(object):
     filename = None
     startlineno = 0
     name = None
@@ -143,7 +143,7 @@
 
     @classmethod
     def from_operations(cls, operations, storage, limit=None):
-        """ Slice given operation list into a chain of Bytecode chunks.
+        """ Slice given operation list into a chain of OpCode chunks.
         Also detect inlined functions and make them Function
         """
         stack = []
@@ -168,13 +168,13 @@
         for op in operations:
             if op.name == 'debug_merge_point':
                 if so_far:
-                    append_to_res(Bytecode(so_far, storage))
+                    append_to_res(OpCode(so_far, storage))
                     if limit:
                         break
                     so_far = []
             so_far.append(op)
         if so_far:
-            append_to_res(Bytecode(so_far, storage))
+            append_to_res(OpCode(so_far, storage))
         # wrap stack back up
         if not stack:
             # no ops whatsoever

diff --git a/pypy/tool/jitlogparser/storage.py b/pypy/tool/jitlogparser/storage.py
--- a/pypy/tool/jitlogparser/storage.py
+++ b/pypy/tool/jitlogparser/storage.py
@@ -4,7 +4,7 @@
 """
 
 import os
-from pypy.tool.jitlogparser.parser import Function, Bytecode
+from pypy.tool.jitlogparser.parser import Function
 from pypy.tool.jitlogparser.module_finder import gather_all_code_objs
 
 class LoopStorage(object):

diff --git a/pypy/tool/jitlogparser/test/test_parser.py b/pypy/tool/jitlogparser/test/test_parser.py
--- a/pypy/tool/jitlogparser/test/test_parser.py
+++ b/pypy/tool/jitlogparser/test/test_parser.py
@@ -1,6 +1,6 @@
 from pypy.jit.metainterp.resoperation import ResOperation, rop
 from pypy.jit.metainterp.history import ConstInt, Const
-from pypy.tool.jitlogparser.parser import parse, Bytecode, Function,\
+from pypy.tool.jitlogparser.parser import parse, OpCode, Function,\
      adjust_bridges
 from pypy.tool.jitlogparser.storage import LoopStorage
 import py
@@ -59,9 +59,9 @@
     """)
     res = Function.from_operations(ops.operations, LoopStorage())
     assert len(res.chunks) == 3 # two chunks + inlined call
-    assert isinstance(res.chunks[0], Bytecode)
+    assert isinstance(res.chunks[0], OpCode)
     assert isinstance(res.chunks[1], Function)
-    assert isinstance(res.chunks[2], Bytecode)
+    assert isinstance(res.chunks[2], OpCode)
     assert res.chunks[1].path == "1"
     assert len(res.chunks[1].chunks) == 3
     

From commits-noreply at bitbucket.org  Wed Feb 23 15:34:28 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Wed, 23 Feb 2011 15:34:28 +0100 (CET)
Subject: [pypy-svn] pypy default: rename OpCode to TraceForOpcode once more,
	to distinguish it from lib_pypy.disassembler.Opcode
Message-ID: <20110223143428.D2DC12A2033@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42226:1e7c2cf4e620
Date: 2011-02-23 14:42 +0100
http://bitbucket.org/pypy/pypy/changeset/1e7c2cf4e620/

Log:	rename OpCode to TraceForOpcode once more, to distinguish it from
	lib_pypy.disassembler.Opcode

diff --git a/pypy/tool/jitlogparser/parser.py b/pypy/tool/jitlogparser/parser.py
--- a/pypy/tool/jitlogparser/parser.py
+++ b/pypy/tool/jitlogparser/parser.py
@@ -68,7 +68,7 @@
 class NonCodeError(Exception):
     pass
 
-class OpCode(object):
+class TraceForOpcode(object):
     filename = None
     startlineno = 0
     name = None
@@ -100,7 +100,7 @@
                                            self.startlineno)
 
     def getcode(self):
-        if self.code is None:
+        if self.code is None and self.filename is not None:
             self.code = dis(self.storage.load_code(self.filename)[self.startlineno])
         return self.code
 
@@ -143,7 +143,7 @@
 
     @classmethod
     def from_operations(cls, operations, storage, limit=None):
-        """ Slice given operation list into a chain of OpCode chunks.
+        """ Slice given operation list into a chain of TraceForOpcode chunks.
         Also detect inlined functions and make them Function
         """
         stack = []
@@ -168,13 +168,13 @@
         for op in operations:
             if op.name == 'debug_merge_point':
                 if so_far:
-                    append_to_res(OpCode(so_far, storage))
+                    append_to_res(TraceForOpcode(so_far, storage))
                     if limit:
                         break
                     so_far = []
             so_far.append(op)
         if so_far:
-            append_to_res(OpCode(so_far, storage))
+            append_to_res(TraceForOpcode(so_far, storage))
         # wrap stack back up
         if not stack:
             # no ops whatsoever

diff --git a/pypy/tool/jitlogparser/test/test_parser.py b/pypy/tool/jitlogparser/test/test_parser.py
--- a/pypy/tool/jitlogparser/test/test_parser.py
+++ b/pypy/tool/jitlogparser/test/test_parser.py
@@ -1,6 +1,6 @@
 from pypy.jit.metainterp.resoperation import ResOperation, rop
 from pypy.jit.metainterp.history import ConstInt, Const
-from pypy.tool.jitlogparser.parser import parse, OpCode, Function,\
+from pypy.tool.jitlogparser.parser import parse, TraceForOpcode, Function,\
      adjust_bridges
 from pypy.tool.jitlogparser.storage import LoopStorage
 import py
@@ -59,9 +59,9 @@
     """)
     res = Function.from_operations(ops.operations, LoopStorage())
     assert len(res.chunks) == 3 # two chunks + inlined call
-    assert isinstance(res.chunks[0], OpCode)
+    assert isinstance(res.chunks[0], TraceForOpcode)
     assert isinstance(res.chunks[1], Function)
-    assert isinstance(res.chunks[2], OpCode)
+    assert isinstance(res.chunks[2], TraceForOpcode)
     assert res.chunks[1].path == "1"
     assert len(res.chunks[1].chunks) == 3
     

From commits-noreply at bitbucket.org  Wed Feb 23 15:34:31 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Wed, 23 Feb 2011 15:34:31 +0100 (CET)
Subject: [pypy-svn] pypy default: in-progress work toward supporting inlined
	calls
Message-ID: <20110223143431.7082C2A2038@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42227:5cee3bd30e32
Date: 2011-02-23 14:53 +0100
http://bitbucket.org/pypy/pypy/changeset/5cee3bd30e32/

Log:	in-progress work toward supporting inlined calls

diff --git a/pypy/module/pypyjit/test_pypy_c/model.py b/pypy/module/pypyjit/test_pypy_c/model.py
--- a/pypy/module/pypyjit/test_pypy_c/model.py
+++ b/pypy/module/pypyjit/test_pypy_c/model.py
@@ -1,7 +1,7 @@
 import py
 import re
 import os.path
-from pypy.tool.jitlogparser.parser import parse, Function
+from pypy.tool.jitlogparser.parser import parse, Function, TraceForOpcode
 from pypy.tool.jitlogparser.storage import LoopStorage
 
 
@@ -68,7 +68,10 @@
 
     def __init__(self, *args, **kwds):
         Function.__init__(self, *args, **kwds)
-        self.compute_ids()
+        self.ids = {}
+        self.code = self.chunks[0].getcode()
+        if self.code:
+            self.compute_ids(self.ids)
 
     @classmethod
     def from_trace(cls, trace, storage):
@@ -76,26 +79,43 @@
         res.is_entry_bridge = 'entry bridge' in trace.comment
         return res
 
-    def compute_ids(self):
-        self.ids = {}
-        self.code = None
-        if not self.filename:
-            return
-        self.code = self.chunks[0].getcode()
-        ids = find_ids(self.code)
+    def flatten_chunks(self):
+        """
+        return a flat sequence of TraceForOpcode objects, including the ones
+        inside inlined functions
+        """
+        for chunk in self.chunks:
+            if isinstance(chunk, TraceForOpcode):
+                yield chunk
+            else:
+                for subchunk in chunk.flatten_chunks():
+                    yield subchunk
+
+    def compute_ids(self, ids):
+        #
+        # 1. compute the ids of self, i.e. the outer function
+        id2opcodes = find_ids(self.code)
         all_my_opcodes = self.get_set_of_opcodes()
         # XXX: for now, we just look for the first opcode in the id range
-        for id, opcodes in ids.iteritems():
-            targetop = opcodes[0]
-            if targetop in all_my_opcodes:
-                self.ids[id] = opcodes
+        for id, opcodes in id2opcodes.iteritems():
+            if not opcodes:
+                continue
+            target_opcode = opcodes[0]
+            if target_opcode in all_my_opcodes:
+                ids[id] = opcodes
+        #
+        # 2. compute the ids of all the inlined functions
+        for chunk in self.chunks:
+            if isinstance(chunk, LoopWithIds):
+                chunk.compute_ids(ids)
 
     def get_set_of_opcodes(self):
-        res = set()
+        result = set()
         for chunk in self.chunks:
-            opcode = self.code.map[chunk.bytecode_no]
-            res.add(opcode)
-        return res
+            if isinstance(chunk, TraceForOpcode):
+                opcode = self.code.map[chunk.bytecode_no]
+                result.add(opcode)
+        return result
 
     def has_id(self, id):
         return id in self.ids
@@ -106,13 +126,13 @@
                 yield op
 
     def allops(self, include_debug_merge_points=False):
-        for chunk in self.chunks:
+        for chunk in self.flatten_chunks():
             for op in self._ops_for_chunk(chunk, include_debug_merge_points):
                 yield op
 
     def ops_by_id(self, id, include_debug_merge_points=False):
         target_opcodes = self.ids[id]
-        for chunk in self.chunks:
+        for chunk in self.flatten_chunks():
             opcode = self.code.map[chunk.bytecode_no]
             if opcode in target_opcodes:
                 for op in self._ops_for_chunk(chunk, include_debug_merge_points):

diff --git a/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py b/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py
--- a/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py
@@ -84,4 +84,4 @@
         loop, = log.loops_by_filename(self.filepath)
         assert loop.get_bytecode("COMPARE_OP").match_stats(
             call=0
-        )
\ No newline at end of file
+        )

diff --git a/pypy/module/pypyjit/test_pypy_c/test_model.py b/pypy/module/pypyjit/test_pypy_c/test_model.py
--- a/pypy/module/pypyjit/test_pypy_c/test_model.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_model.py
@@ -105,6 +105,20 @@
         loops = log.loops_by_filename(self.filepath, is_entry_bridge='*')
         assert len(loops) == 2
 
+    def test_inlined_function(self):
+        py.test.skip('in-progress')
+        def f():
+            def g(x):
+                return x+1 # ID: add
+            i = 0
+            while i < 1003:
+                i = g(i) # ID: call
+            return i
+        #
+        log = self.run(f)
+        loop, = log.loops_by_filename(self.filepath)
+        call_ops = list(loop.ops_by_id('call'))
+
     def test_loops_by_id(self):
         def f():
             i = 0

From commits-noreply at bitbucket.org  Wed Feb 23 15:34:32 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Wed, 23 Feb 2011 15:34:32 +0100 (CET)
Subject: [pypy-svn] pypy default: always store the .code attribute on
	TraceForOpcode	instances, it makes everything much simpler
Message-ID: <20110223143432.A9B92282BDE@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42228:54d9ff95c7af
Date: 2011-02-23 15:12 +0100
http://bitbucket.org/pypy/pypy/changeset/54d9ff95c7af/

Log:	always store the .code attribute on TraceForOpcode instances, it
	makes everything much simpler

diff --git a/pypy/tool/jitlogparser/parser.py b/pypy/tool/jitlogparser/parser.py
--- a/pypy/tool/jitlogparser/parser.py
+++ b/pypy/tool/jitlogparser/parser.py
@@ -1,5 +1,4 @@
 import re, sys
-from lib_pypy.disassembler import dis
 from pypy.jit.metainterp.resoperation import rop, opname
 from pypy.jit.tool.oparser import OpParser
 
@@ -92,6 +91,7 @@
                 self.bytecode_no = int(bytecode_no)
         self.operations = operations
         self.storage = storage
+        self.code = storage.disassemble_code(self.filename, self.startlineno)
 
     def repr(self):
         if self.filename is None:
@@ -100,8 +100,6 @@
                                            self.startlineno)
 
     def getcode(self):
-        if self.code is None and self.filename is not None:
-            self.code = dis(self.storage.load_code(self.filename)[self.startlineno])
         return self.code
 
     def getlineno(self):

diff --git a/pypy/tool/jitlogparser/storage.py b/pypy/tool/jitlogparser/storage.py
--- a/pypy/tool/jitlogparser/storage.py
+++ b/pypy/tool/jitlogparser/storage.py
@@ -3,7 +3,9 @@
 for all loops and bridges, so http requests can refer to them by name
 """
 
+import py
 import os
+from lib_pypy.disassembler import dis
 from pypy.tool.jitlogparser.parser import Function
 from pypy.tool.jitlogparser.module_finder import gather_all_code_objs
 
@@ -12,6 +14,7 @@
         self.loops = None
         self.functions = {}
         self.codes = {}
+        self.disassembled_codes = {}
         self.extrapath = extrapath
 
     def load_code(self, fname):
@@ -27,6 +30,17 @@
             self.codes[fname] = res
             return res
 
+    def disassemble_code(self, fname, startlineno):
+        if py.path.local(fname).check(file=False):
+            return None # cannot find source file
+        key = (fname, startlineno)
+        try:
+            return self.disassembled_codes[key]
+        except KeyError:
+            res = dis(self.load_code(fname)[startlineno])
+            self.disassembled_codes[key] = res
+            return res
+
     def reconnect_loops(self, loops):
         """ Re-connect loops in a way that entry bridges are filtered out
         and normal bridges are associated with guards. Returning list of

diff --git a/pypy/tool/jitlogparser/test/test_parser.py b/pypy/tool/jitlogparser/test/test_parser.py
--- a/pypy/tool/jitlogparser/test/test_parser.py
+++ b/pypy/tool/jitlogparser/test/test_parser.py
@@ -34,10 +34,10 @@
 def test_split():
     ops = parse('''
     [i0]
-    debug_merge_point("<code object stuff, file '/tmp/x.py', line 200> #10 ADD", 0)
-    debug_merge_point("<code object stuff, file '/tmp/x.py', line 200> #11 SUB", 0)
+    debug_merge_point("<code object stuff, file '/I/dont/exist.py', line 200> #10 ADD", 0)
+    debug_merge_point("<code object stuff, file '/I/dont/exist.py', line 200> #11 SUB", 0)
     i1 = int_add(i0, 1)
-    debug_merge_point("<code object stuff, file '/tmp/x.py', line 200> #11 SUB", 0)
+    debug_merge_point("<code object stuff, file '/I/dont/exist.py', line 200> #11 SUB", 0)
     i2 = int_add(i1, 1)
     ''')
     res = Function.from_operations(ops.operations, LoopStorage())
@@ -68,27 +68,27 @@
 def test_name():
     ops = parse('''
     [i0]
-    debug_merge_point("<code object stuff, file '/tmp/x.py', line 200> #10 ADD", 0)
-    debug_merge_point("<code object stuff, file '/tmp/x.py', line 201> #11 SUB", 0)
+    debug_merge_point("<code object stuff, file '/I/dont/exist.py', line 200> #10 ADD", 0)
+    debug_merge_point("<code object stuff, file '/I/dont/exist.py', line 201> #11 SUB", 0)
     i1 = int_add(i0, 1)
-    debug_merge_point("<code object stuff, file '/tmp/x.py', line 202> #11 SUB", 0)
+    debug_merge_point("<code object stuff, file '/I/dont/exist.py', line 202> #11 SUB", 0)
     i2 = int_add(i1, 1)
     ''')
     res = Function.from_operations(ops.operations, LoopStorage())
     assert res.repr() == res.chunks[0].repr()
-    assert res.repr() == "stuff, file '/tmp/x.py', line 200"
+    assert res.repr() == "stuff, file '/I/dont/exist.py', line 200"
     assert res.startlineno == 200
-    assert res.filename == '/tmp/x.py'
+    assert res.filename == '/I/dont/exist.py'
     assert res.name == 'stuff'
 
 def test_name_no_first():
     ops = parse('''
     [i0]
     i3 = int_add(i0, 1)
-    debug_merge_point("<code object stuff, file '/tmp/x.py', line 200> #10 ADD", 0)
-    debug_merge_point("<code object stuff, file '/tmp/x.py', line 201> #11 SUB", 0)
+    debug_merge_point("<code object stuff, file '/I/dont/exist.py', line 200> #10 ADD", 0)
+    debug_merge_point("<code object stuff, file '/I/dont/exist.py', line 201> #11 SUB", 0)
     i1 = int_add(i0, 1)
-    debug_merge_point("<code object stuff, file '/tmp/x.py', line 202> #11 SUB", 0)
+    debug_merge_point("<code object stuff, file '/I/dont/exist.py', line 202> #11 SUB", 0)
     i2 = int_add(i1, 1)
     ''')
     res = Function.from_operations(ops.operations, LoopStorage())

From commits-noreply at bitbucket.org  Wed Feb 23 15:34:36 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Wed, 23 Feb 2011 15:34:36 +0100 (CET)
Subject: [pypy-svn] pypy default: inlined functions are finally handled
	correctly
Message-ID: <20110223143436.090C4282C20@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42229:fd989ec07b4c
Date: 2011-02-23 15:31 +0100
http://bitbucket.org/pypy/pypy/changeset/fd989ec07b4c/

Log:	inlined functions are finally handled correctly

diff --git a/pypy/module/pypyjit/test_pypy_c/model.py b/pypy/module/pypyjit/test_pypy_c/model.py
--- a/pypy/module/pypyjit/test_pypy_c/model.py
+++ b/pypy/module/pypyjit/test_pypy_c/model.py
@@ -113,7 +113,7 @@
         result = set()
         for chunk in self.chunks:
             if isinstance(chunk, TraceForOpcode):
-                opcode = self.code.map[chunk.bytecode_no]
+                opcode = chunk.getopcode()
                 result.add(opcode)
         return result
 
@@ -133,7 +133,7 @@
     def ops_by_id(self, id, include_debug_merge_points=False):
         target_opcodes = self.ids[id]
         for chunk in self.flatten_chunks():
-            opcode = self.code.map[chunk.bytecode_no]
+            opcode = chunk.getopcode()
             if opcode in target_opcodes:
                 for op in self._ops_for_chunk(chunk, include_debug_merge_points):
                     yield op

diff --git a/pypy/tool/jitlogparser/parser.py b/pypy/tool/jitlogparser/parser.py
--- a/pypy/tool/jitlogparser/parser.py
+++ b/pypy/tool/jitlogparser/parser.py
@@ -102,14 +102,15 @@
     def getcode(self):
         return self.code
 
+    def getopcode(self):
+        return self.code.map[self.bytecode_no]
+
     def getlineno(self):
-        code = self.getcode()
-        return code.map[self.bytecode_no].lineno
+        return self.getopcode().lineno
     lineno = property(getlineno)
 
     def getline_starts_here(self):
-        code = self.getcode()
-        return code.map[self.bytecode_no].line_starts_here
+        return self.getopcode().line_starts_here
     line_starts_here = property(getline_starts_here)
 
     def __repr__(self):

diff --git a/pypy/module/pypyjit/test_pypy_c/test_model.py b/pypy/module/pypyjit/test_pypy_c/test_model.py
--- a/pypy/module/pypyjit/test_pypy_c/test_model.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_model.py
@@ -106,18 +106,31 @@
         assert len(loops) == 2
 
     def test_inlined_function(self):
-        py.test.skip('in-progress')
         def f():
             def g(x):
                 return x+1 # ID: add
             i = 0
             while i < 1003:
                 i = g(i) # ID: call
+                a = 0    # to make sure that JUMP_ABSOLUTE is not part of the ID
             return i
         #
         log = self.run(f)
         loop, = log.loops_by_filename(self.filepath)
-        call_ops = list(loop.ops_by_id('call'))
+        call_ops = [op.name for op in loop.ops_by_id('call')]
+        assert call_ops == ['force_token'] # it does not follow inlining
+        #
+        add_ops = [op.name for op in loop.ops_by_id('add')]
+        assert add_ops == ['int_add']
+        #
+        ops = [op.name for op in loop.allops()]
+        assert ops == [
+            # this is the actual loop
+            'int_lt', 'guard_true', 'force_token', 'int_add',
+            # this is the signal checking stuff
+            'getfield_raw', 'int_sub', 'setfield_raw', 'int_lt', 'guard_false',
+            'jump'
+            ]
 
     def test_loops_by_id(self):
         def f():

From commits-noreply at bitbucket.org  Wed Feb 23 15:34:37 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Wed, 23 Feb 2011 15:34:37 +0100 (CET)
Subject: [pypy-svn] pypy default: move this test later
Message-ID: <20110223143437.09D29282C20@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42230:5b870186ab6f
Date: 2011-02-23 15:32 +0100
http://bitbucket.org/pypy/pypy/changeset/5b870186ab6f/

Log:	move this test later

diff --git a/pypy/module/pypyjit/test_pypy_c/test_model.py b/pypy/module/pypyjit/test_pypy_c/test_model.py
--- a/pypy/module/pypyjit/test_pypy_c/test_model.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_model.py
@@ -105,33 +105,6 @@
         loops = log.loops_by_filename(self.filepath, is_entry_bridge='*')
         assert len(loops) == 2
 
-    def test_inlined_function(self):
-        def f():
-            def g(x):
-                return x+1 # ID: add
-            i = 0
-            while i < 1003:
-                i = g(i) # ID: call
-                a = 0    # to make sure that JUMP_ABSOLUTE is not part of the ID
-            return i
-        #
-        log = self.run(f)
-        loop, = log.loops_by_filename(self.filepath)
-        call_ops = [op.name for op in loop.ops_by_id('call')]
-        assert call_ops == ['force_token'] # it does not follow inlining
-        #
-        add_ops = [op.name for op in loop.ops_by_id('add')]
-        assert add_ops == ['int_add']
-        #
-        ops = [op.name for op in loop.allops()]
-        assert ops == [
-            # this is the actual loop
-            'int_lt', 'guard_true', 'force_token', 'int_add',
-            # this is the signal checking stuff
-            'getfield_raw', 'int_sub', 'setfield_raw', 'int_lt', 'guard_false',
-            'jump'
-            ]
-
     def test_loops_by_id(self):
         def f():
             i = 0
@@ -169,6 +142,34 @@
         opnames = [op.name for op in ops]
         assert opnames == ['int_add']
 
+
+    def test_inlined_function(self):
+        def f():
+            def g(x):
+                return x+1 # ID: add
+            i = 0
+            while i < 1003:
+                i = g(i) # ID: call
+                a = 0    # to make sure that JUMP_ABSOLUTE is not part of the ID
+            return i
+        #
+        log = self.run(f)
+        loop, = log.loops_by_filename(self.filepath)
+        call_ops = [op.name for op in loop.ops_by_id('call')]
+        assert call_ops == ['force_token'] # it does not follow inlining
+        #
+        add_ops = [op.name for op in loop.ops_by_id('add')]
+        assert add_ops == ['int_add']
+        #
+        ops = [op.name for op in loop.allops()]
+        assert ops == [
+            # this is the actual loop
+            'int_lt', 'guard_true', 'force_token', 'int_add',
+            # this is the signal checking stuff
+            'getfield_raw', 'int_sub', 'setfield_raw', 'int_lt', 'guard_false',
+            'jump'
+            ]
+
     def test_parse_op(self):
         res = LoopWithIds.parse_op("  a =   int_add(  b,  3 ) # foo")
         assert res == ("int_add", "a", ["b", "3"])

From commits-noreply at bitbucket.org  Wed Feb 23 18:01:54 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Wed, 23 Feb 2011 18:01:54 +0100 (CET)
Subject: [pypy-svn] pypy default: Unroll the cmp_exc_match loop.
Message-ID: <20110223170154.9E43B282BDE@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r42231:cd575e762be0
Date: 2011-02-23 12:01 -0500
http://bitbucket.org/pypy/pypy/changeset/cd575e762be0/

Log:	Unroll the cmp_exc_match loop.

diff --git a/pypy/interpreter/pyopcode.py b/pypy/interpreter/pyopcode.py
--- a/pypy/interpreter/pyopcode.py
+++ b/pypy/interpreter/pyopcode.py
@@ -754,6 +754,7 @@
     def cmp_is_not(self, w_1, w_2):
         return self.space.not_(self.space.is_(w_1, w_2))
 
+    @jit.unroll_safe
     def cmp_exc_match(self, w_1, w_2):
         if self.space.is_true(self.space.isinstance(w_2, self.space.w_tuple)):
             for w_t in self.space.fixedview(w_2):

diff --git a/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py b/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py
--- a/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py
@@ -65,8 +65,6 @@
         """)
 
     def test_cmp_exc(self):
-        py.test.skip("WIP")
-
         def f1(n):
             def f():
                 raise KeyError
@@ -75,13 +73,12 @@
             while i < n:
                 try:
                     f()
-                except KeyError:
+                except KeyError: # ID: except
                     i += 1
             return i
 
         log = self.run(f1, [10000])
         assert log.result == 10000
-        loop, = log.loops_by_filename(self.filepath)
-        assert loop.get_bytecode("COMPARE_OP").match_stats(
-            call=0
-        )
+        loop, = log.loops_by_id("except")
+        ops = [o.name for o in loop.ops_by_id("except")]
+        assert "call_may_force" not in ops
\ No newline at end of file

From commits-noreply at bitbucket.org  Wed Feb 23 18:13:56 2011
From: commits-noreply at bitbucket.org (ademan)
Date: Wed, 23 Feb 2011 18:13:56 +0100 (CET)
Subject: [pypy-svn] pypy default: Changed PyUnicode_*ASCII functions to
	raise exceptions	since strict codec functioning is default.
	Changed tests to match.
Message-ID: <20110223171356.710B9282BDE@codespeak.net>

Author: Daniel Roberts <Ademan555 at gmail.com>
Branch: 
Changeset: r42232:5994bfbae1f7
Date: 2011-02-23 09:13 -0800
http://bitbucket.org/pypy/pypy/changeset/5994bfbae1f7/

Log:	Changed PyUnicode_*ASCII functions to raise exceptions since strict
	codec functioning is default. Changed tests to match.

diff --git a/pypy/module/cpyext/test/test_unicodeobject.py b/pypy/module/cpyext/test/test_unicodeobject.py
--- a/pypy/module/cpyext/test/test_unicodeobject.py
+++ b/pypy/module/cpyext/test/test_unicodeobject.py
@@ -184,8 +184,7 @@
         assert space.eq_w(space.wrap(ustr), result)
 
         w_ustr = space.wrap(u"abcd\xe9f")
-        result = api.PyUnicode_AsASCIIString(w_ustr)
-        assert result is None
+        self.raises(space, api, UnicodeEncodeError, api.PyUnicode_AsASCIIString, w_ustr)
 
     def test_decode_utf16(self, space, api):
         def test(encoded, endian, realendian=None):
@@ -254,8 +253,8 @@
 
         s = 'abcd\xFF'
         data = rffi.str2charp(s)
-        w_u = api.PyUnicode_DecodeASCII(data, len(s), lltype.nullptr(rffi.CCHARP.TO))
-        assert w_u is None
+        self.raises(space, api, UnicodeDecodeError, api.PyUnicode_DecodeASCII,
+                    data, len(s), lltype.nullptr(rffi.CCHARP.TO))
         rffi.free_charp(data)
 
         uni = u'abcdefg'
@@ -267,6 +266,7 @@
         u = u'&#65533;bcd&#65533;fg'
         data = rffi.unicode2wcharp(u)
         w_s = api.PyUnicode_EncodeASCII(data, len(u), lltype.nullptr(rffi.CCHARP.TO))
-        assert w_s is None
+        self.raises(space, api, UnicodeEncodeError, api.PyUnicode_EncodeASCII,
+                    data, len(u), lltype.nullptr(rffi.CCHARP.TO))
         rffi.free_wcharp(data)
 

diff --git a/pypy/module/cpyext/unicodeobject.py b/pypy/module/cpyext/unicodeobject.py
--- a/pypy/module/cpyext/unicodeobject.py
+++ b/pypy/module/cpyext/unicodeobject.py
@@ -400,13 +400,7 @@
     """Encode a Unicode object using ASCII and return the result as Python string
     object.  Error handling is "strict".  Return NULL if an exception was raised
     by the codec."""
-    try:
-        return space.call_method(w_unicode, 'encode', space.wrap('ascii')) #space.w_None for errors?
-    except OperationError, e:
-        if e.match(space, space.w_UnicodeEncodeError):
-            return None
-        else:
-            raise
+    return space.call_method(w_unicode, 'encode', space.wrap('ascii')) #space.w_None for errors?
 
 @cpython_api([rffi.CCHARP, Py_ssize_t, rffi.CCHARP], PyObject)
 def PyUnicode_DecodeASCII(space, s, size, errors):
@@ -416,13 +410,7 @@
     This function used an int type for size. This might require
     changes in your code for properly supporting 64-bit systems."""
     w_s = space.wrap(rffi.charpsize2str(s, size))
-    try:
-        return space.call_method(w_s, 'decode', space.wrap('ascii'))
-    except OperationError, e:
-        if e.match(space, space.w_UnicodeDecodeError):
-            return None
-        else:
-            raise
+    return space.call_method(w_s, 'decode', space.wrap('ascii'))
 
 @cpython_api([rffi.CWCHARP, Py_ssize_t, rffi.CCHARP], PyObject)
 def PyUnicode_EncodeASCII(space, s, size, errors):
@@ -433,13 +421,7 @@
     changes in your code for properly supporting 64-bit systems."""
 
     w_s = space.wrap(rffi.wcharpsize2unicode(s, size))
-    try:
-        return space.call_method(w_s, 'encode', space.wrap('ascii'))
-    except OperationError, e:
-        if e.match(space, space.w_UnicodeEncodeError):
-            return None
-        else:
-            raise
+    return space.call_method(w_s, 'encode', space.wrap('ascii'))
 
 if sys.platform == 'win32':
     @cpython_api([CONST_WSTRING, Py_ssize_t, CONST_STRING], PyObject)

From commits-noreply at bitbucket.org  Wed Feb 23 18:33:31 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Wed, 23 Feb 2011 18:33:31 +0100 (CET)
Subject: [pypy-svn] pypy default: kill this test,
	we decided to use a different way of testing
Message-ID: <20110223173331.74620282BDE@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42233:095d33ac5756
Date: 2011-02-23 15:54 +0100
http://bitbucket.org/pypy/pypy/changeset/095d33ac5756/

Log:	kill this test, we decided to use a different way of testing

diff --git a/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py b/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py
--- a/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py
@@ -7,31 +7,6 @@
 from pypy.module.pypyjit.test_pypy_c.test_model import BaseTestPyPyC
 
 
-class TestInfrastructure(BaseTestPyPyC):
-
-    def test_full(self):
-        py.test.skip('in-progress')
-        def f():
-            i = 0
-            while i < 1003:
-                # LOOP one
-                i += 1
-
-        trace = self.run(f)
-        loop = trace.get_loops('one')
-        loop.get_bytecode(3, 'LOAD_FAST').match('''
-        int_add
-        guard_true
-        ''')
-        loop.get_bytecode(4, 'LOAD_CONST').match_stats(
-            guard='3', call='1-2', call_may_force='0'
-        )
-        # this would make operations that are "costly" obligatory to pass
-        # like new
-        loo.get_bytecode(5, 'INPLACE_ADD').match_stats(
-            allocs='5-10'
-            )
-
 class TestPyPyCNew(BaseTestPyPyC):
     def test_f1(self):
         def f1(n):

From commits-noreply at bitbucket.org  Wed Feb 23 18:33:32 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Wed, 23 Feb 2011 18:33:32 +0100 (CET)
Subject: [pypy-svn] pypy default: don't import site: it's fastern,
	and avoids creating tons of regex-related useless loops
Message-ID: <20110223173332.0EFA0282BDE@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42234:cf3b7b61b179
Date: 2011-02-23 17:30 +0100
http://bitbucket.org/pypy/pypy/changeset/cf3b7b61b179/

Log:	don't import site: it's fastern, and avoids creating tons of regex-
	related useless loops

diff --git a/pypy/module/pypyjit/test_pypy_c/test_model.py b/pypy/module/pypyjit/test_pypy_c/test_model.py
--- a/pypy/module/pypyjit/test_pypy_c/test_model.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_model.py
@@ -28,7 +28,7 @@
         # run a child pypy-c with logging enabled
         logfile = self.filepath.new(ext='.log')
         env={'PYPYLOG': 'jit-log-opt,jit-summary:' + str(logfile)}
-        cmdline = [sys.executable,
+        cmdline = [sys.executable, '-S',
                    '--jit', 'threshold=%d' % threshold,
                    str(self.filepath)]
         pipe = subprocess.Popen(cmdline,

From commits-noreply at bitbucket.org  Wed Feb 23 18:33:34 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Wed, 23 Feb 2011 18:33:34 +0100 (CET)
Subject: [pypy-svn] pypy default: find a way not to repeat the same code for
	ticker check	every time
Message-ID: <20110223173334.75C63282BF2@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42235:35e02ea78e2e
Date: 2011-02-23 18:28 +0100
http://bitbucket.org/pypy/pypy/changeset/35e02ea78e2e/

Log:	find a way not to repeat the same code for ticker check every time

diff --git a/pypy/module/pypyjit/test_pypy_c/model.py b/pypy/module/pypyjit/test_pypy_c/model.py
--- a/pypy/module/pypyjit/test_pypy_c/model.py
+++ b/pypy/module/pypyjit/test_pypy_c/model.py
@@ -166,7 +166,23 @@
         args = map(str.strip, args)
         return opname, resvar, args
 
+    def preprocess_expected_src(self, src):
+        # all loops decrement the tick-counter at the end. The rpython code is
+        # in jump_absolute() in pypyjit/interp.py. The string --TICK-- is
+        # replaced with the corresponding operations, so that tests don't have
+        # to repeat it every time
+        ticker_check = """
+            ticker0 = getfield_raw(ticker_address)
+            ticker1 = int_sub(ticker0, 1)
+            setfield_raw(ticker_address, ticker1)
+            ticker_cond = int_lt(ticker1, 0)
+            guard_false(ticker_cond)
+        """
+        src = src.replace('--TICK--', ticker_check)
+        return src
+
     def match_ops(self, ops, expected_src):
+        expected_src = self.preprocess_expected_src(expected_src)
         alpha_map = {}
         def match_var(v1, v2):
             if v1 not in alpha_map:

diff --git a/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py b/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py
--- a/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py
@@ -31,11 +31,7 @@
             i12 = int_and(i8, i11)
             i13 = int_add_ovf(i6, i12)
             guard_no_overflow()
-            i16 = getfield_raw(37212896)
-            i18 = int_sub(i16, 1)
-            setfield_raw(37212896, i18)
-            i20 = int_lt(i18, 0)
-            guard_false(i20)
+            --TICK--
             jump(p0, p1, p2, p3, p4, p5, i13, i11, i8)
         """)
 

diff --git a/pypy/module/pypyjit/test_pypy_c/test_model.py b/pypy/module/pypyjit/test_pypy_c/test_model.py
--- a/pypy/module/pypyjit/test_pypy_c/test_model.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_model.py
@@ -198,15 +198,19 @@
             jump(p0, p1, p2, p3, i8)
         """)
         #
+        assert loop.match("""
+            i6 = int_lt(i4, 1003)
+            guard_true(i6)
+            i8 = int_add(i4, 1)
+            --TICK--
+            jump(p0, p1, p2, p3, i8)
+        """)
+        #
         py.test.raises(AssertionError, loop.match, """
             i6 = int_lt(i4, 1003)
             guard_true(i6)
             i8 = int_add(i5, 1) # variable mismatch
-            i10 = getfield_raw(37212896)
-            i12 = int_sub(i10, 1)
-            setfield_raw(37212896, i12)
-            i14 = int_lt(i12, 0)
-            guard_false(i14)
+            --TICK--
             jump(p0, p1, p2, p3, i8)
         """)
 

From commits-noreply at bitbucket.org  Wed Feb 23 18:33:35 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Wed, 23 Feb 2011 18:33:35 +0100 (CET)
Subject: [pypy-svn] pypy default: port test_factorial from test_pypy_c
Message-ID: <20110223173335.7C92C282BF2@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42236:695b175d4cff
Date: 2011-02-23 18:31 +0100
http://bitbucket.org/pypy/pypy/changeset/695b175d4cff/

Log:	port test_factorial from test_pypy_c

diff --git a/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py b/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py
--- a/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py
@@ -35,6 +35,27 @@
             jump(p0, p1, p2, p3, p4, p5, i13, i11, i8)
         """)
 
+    def test_factorial(self):
+        def fact(n):
+            r = 1
+            while n > 1:
+                r *= n
+                n -= 1
+            return r
+        log = self.run(fact, [7], threshold=5)
+        assert log.result == 5040
+        loop, = log.loops_by_filename(self.filepath)
+        assert loop.match("""
+            i7 = int_gt(i4, 1)
+            guard_true(i7)
+            i8 = int_mul_ovf(i5, i4)
+            guard_no_overflow()
+            i10 = int_sub(i4, 1)
+            --TICK--
+            jump(p0, p1, p2, p3, i10, i8)
+        """)
+
+
     def test_cmp_exc(self):
         def f1(n):
             def f():

From commits-noreply at bitbucket.org  Wed Feb 23 18:56:07 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Wed, 23 Feb 2011 18:56:07 +0100 (CET)
Subject: [pypy-svn] pypy arm-backend-2: Fix an off-by-one error when doing a
	jump to a	location that is more than 255 bytes away
Message-ID: <20110223175607.84BDA36C21D@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backend-2
Changeset: r42237:1aa4e6cf725e
Date: 2011-02-23 18:42 +0100
http://bitbucket.org/pypy/pypy/changeset/1aa4e6cf725e/

Log:	Fix an off-by-one error when doing a jump to a location that is more
	than 255 bytes away

diff --git a/pypy/jit/backend/arm/test/test_regalloc.py b/pypy/jit/backend/arm/test/test_regalloc.py
--- a/pypy/jit/backend/arm/test/test_regalloc.py
+++ b/pypy/jit/backend/arm/test/test_regalloc.py
@@ -72,6 +72,14 @@
                               zero_division_value)
     FPTR = lltype.Ptr(lltype.FuncType([lltype.Signed], lltype.Void))
     raising_fptr = llhelper(FPTR, raising_func)
+
+    def f(a):
+        return 23
+
+    FPTR = lltype.Ptr(lltype.FuncType([lltype.Signed], lltype.Signed))
+    f_fptr = llhelper(FPTR, f)
+    f_calldescr = cpu.calldescrof(FPTR.TO, FPTR.TO.ARGS, FPTR.TO.RESULT)
+
     zero_division_tp, zero_division_value = cpu.get_zero_division_error()
     zd_addr = cpu.cast_int_to_adr(zero_division_tp)
     zero_division_error = llmemory.cast_adr_to_ptr(zd_addr,
@@ -421,6 +429,7 @@
     fielddescr = cpu.fielddescrof(S, 'field')
 
     A = lltype.GcArray(lltype.Char)
+    I = lltype.GcArray(lltype.Signed)
     arraydescr = cpu.arraydescrof(A)
 
     namespace = locals().copy()
@@ -485,6 +494,36 @@
         self.interpret(ops, [s, ord('a')])
         assert s[1] == 'a'
 
+    def test_setarrayitem2_gc(self):
+        ops = '''
+        [p0, i, i1]
+        setarrayitem_gc(p0, i1, i, descr=arraydescr)
+        finish()
+        '''
+        s = lltype.malloc(self.A, 3)
+        self.interpret(ops, [s, ord('a'), 1])
+        assert s[1] == 'a'
+
+    def test_setarrayitem3_gc(self):
+        ops = '''
+        [p0, i0, i1]
+        setarrayitem_gc(p0, i1, i0, descr=arraydescr)
+        finish()
+        '''
+        s = lltype.malloc(self.I, 3)
+        self.interpret(ops, [s, 1234567890, 1])
+        assert s[1] == 1234567890
+
+    def test_setarrayitem4_gc(self):
+        ops = '''
+        [p0, i0]
+        setarrayitem_gc(p0, 1, i0, descr=arraydescr)
+        finish()
+        '''
+        s = lltype.malloc(self.I, 3)
+        self.interpret(ops, [s, 1234567890])
+        assert s[1] == 1234567890
+
     def test_division_optimized(self):
         ops = '''
         [i7, i6]
@@ -626,6 +665,68 @@
         bridge = self.attach_bridge(ops, loop, -2)
 
 
+    def test_from_loop_to_loop(self):
+        def assembler_helper(failindex, virtualizable):
+            return 1
+
+        FUNCPTR = lltype.Ptr(lltype.FuncType([lltype.Signed, llmemory.GCREF],
+                                             lltype.Signed))
+        class FakeJitDriverSD:
+            index_of_virtualizable = -1
+            _assembler_helper_ptr = llhelper(FUNCPTR, assembler_helper)
+            assembler_helper_adr = llmemory.cast_ptr_to_adr(
+                _assembler_helper_ptr)
+
+        FakeJitDriverSD.portal_calldescr = self.cpu.calldescrof(
+            lltype.Ptr(lltype.FuncType([lltype.Signed], lltype.Signed)), [lltype.Signed], lltype.Signed)
+        loop1 = """
+        [i0, i1, i2, i3, i4, i5, i6, i7, i8, i9, i10]
+        i11 = int_add(i0, i1)
+        finish(i0, i1, i2, i3, i4, i5, i6, i7, i8, i9, i10, i11)
+        """
+        large = self.interpret(loop1, range(11), run=False)
+        large.token.outermost_jitdriver_sd = FakeJitDriverSD()
+        self.namespace['looptoken'] = large.token
+        assert self.namespace['looptoken']._arm_bootstrap_code != 0
+        loop2 = """
+       [i0]
+        i1 = force_token()
+        call_assembler(1,2,3,4,5,6,7,8,9,10,11, descr=looptoken)
+        guard_not_forced() [i0]
+        finish(i0)
+        """
+
+        self.interpret(loop2, [110])
+        assert self.getint(0) == 0
+
+    def test_far_far_jump(self):
+        ops = """
+        [i0, i1, i2, i3, i4, i5, i6, i7, i8, i9, i10]
+        i11 = int_add(i0, 1)
+        i12 = int_lt(i11, 2)
+        i13 = call(ConstClass(f_fptr), i12, descr=f_calldescr)
+        i14 = call(ConstClass(f_fptr), i12, descr=f_calldescr)
+        i15 = call(ConstClass(f_fptr), i12, descr=f_calldescr)
+        i16 = call(ConstClass(f_fptr), i12, descr=f_calldescr)
+        i17 = call(ConstClass(f_fptr), i12, descr=f_calldescr)
+        i18 = call(ConstClass(f_fptr), i12, descr=f_calldescr)
+        i19 = call(ConstClass(f_fptr), i12, descr=f_calldescr)
+        i20 = call(ConstClass(f_fptr), i12, descr=f_calldescr)
+        i21 = call(ConstClass(f_fptr), i12, descr=f_calldescr)
+        i22 = call(ConstClass(f_fptr), i12, descr=f_calldescr)
+        i23 = call(ConstClass(f_fptr), i12, descr=f_calldescr)
+        i24 = call(ConstClass(f_fptr), i12, descr=f_calldescr)
+        i26 = call(ConstClass(f_fptr), i12, descr=f_calldescr)
+        i27 = call(ConstClass(f_fptr), i12, descr=f_calldescr)
+        i28 = call(ConstClass(f_fptr), i12, descr=f_calldescr)
+        i29 = call(ConstClass(f_fptr), i12, descr=f_calldescr)
+        i30 = call(ConstClass(f_fptr), i12, descr=f_calldescr)
+        guard_true(i12) [i11, i1, i2, i3, i4, i5, i6, i7, i8, i9, i10]
+        jump(i11, i1, i2, i3, i4, i5, i6, i7, i8, i9, i10)
+        """
+        self.interpret(ops, range(11))
+        assert self.getint(0) == 2 # and not segfault()
+
         self.cpu.set_future_value_int(0, 4)
         self.cpu.set_future_value_int(1, 7)        
         self.run(loop)

diff --git a/pypy/jit/backend/arm/codebuilder.py b/pypy/jit/backend/arm/codebuilder.py
--- a/pypy/jit/backend/arm/codebuilder.py
+++ b/pypy/jit/backend/arm/codebuilder.py
@@ -87,7 +87,7 @@
                 assert c == cond.AL
                 self.LDR_ri(reg.ip.value, reg.pc.value, cond=c)
                 self.SUB_rr(reg.pc.value, reg.pc.value, reg.ip.value, cond=c)
-                target += 2 * WORD
+                target += WORD
                 self.write32(target)
 
     def BL(self, target, c=cond.AL):

From commits-noreply at bitbucket.org  Wed Feb 23 18:56:08 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Wed, 23 Feb 2011 18:56:08 +0100 (CET)
Subject: [pypy-svn] pypy arm-backend-2: Add two methods required by the
	frontend
Message-ID: <20110223175608.5305636C21D@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backend-2
Changeset: r42238:6771be37c413
Date: 2011-02-23 18:45 +0100
http://bitbucket.org/pypy/pypy/changeset/6771be37c413/

Log:	Add two methods required by the frontend

diff --git a/pypy/jit/backend/arm/assembler.py b/pypy/jit/backend/arm/assembler.py
--- a/pypy/jit/backend/arm/assembler.py
+++ b/pypy/jit/backend/arm/assembler.py
@@ -56,6 +56,8 @@
         self.cpu = cpu
         self.fail_boxes_int = values_array(lltype.Signed, failargs_limit)
         self.fail_boxes_ptr = values_array(llmemory.GCREF, failargs_limit)
+        self.fail_boxes_count = 0
+        self.fail_force_index = 0
         self.setup_failure_recovery()
         self.mc = None
         self.malloc_func_addr = 0
@@ -157,6 +159,7 @@
         assert enc[i] == self.END_OF_LOCS
         descr = self.decode32(enc, i+1)
         self.fail_boxes_count = fail_index
+        self.fail_force_index = frame_loc
         return descr
 
     def decode_inputargs(self, enc, inputargs, regalloc):

diff --git a/pypy/jit/backend/arm/runner.py b/pypy/jit/backend/arm/runner.py
--- a/pypy/jit/backend/arm/runner.py
+++ b/pypy/jit/backend/arm/runner.py
@@ -54,6 +54,12 @@
     def get_latest_value_count(self):
         return self.assembler.fail_boxes_count
 
+    def get_latest_value_count(self):
+        return self.assembler.fail_boxes_count
+
+    def get_latest_force_token(self):
+        return self.assembler.fail_force_index
+
     def clear_latest_values(self, count):
         setitem = self.assembler.fail_boxes_ptr.setitem
         null = lltype.nullptr(llmemory.GCREF.TO)

From commits-noreply at bitbucket.org  Wed Feb 23 18:56:15 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Wed, 23 Feb 2011 18:56:15 +0100 (CET)
Subject: [pypy-svn] pypy arm-backend-2: Fix some issues with call_assembler
Message-ID: <20110223175615.4022B2A2033@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backend-2
Changeset: r42239:5b2748fcc600
Date: 2011-02-23 18:49 +0100
http://bitbucket.org/pypy/pypy/changeset/5b2748fcc600/

Log:	Fix some issues with call_assembler

diff --git a/pypy/jit/backend/arm/assembler.py b/pypy/jit/backend/arm/assembler.py
--- a/pypy/jit/backend/arm/assembler.py
+++ b/pypy/jit/backend/arm/assembler.py
@@ -376,7 +376,8 @@
         self.gen_func_prolog()
         sp_patch_location = self._prepare_sp_patch_position()
         arglocs = self.gen_bootstrap_code(inputargs, regalloc, looptoken)
-
+        #for x in range(5):
+        #    self.mc.NOP()
         loop_head = self.mc.currpos()
 
         looptoken._arm_loop_code = loop_head
@@ -384,7 +385,7 @@
 
         self._walk_operations(operations, regalloc)
 
-        looptoken._arm_frame_depth = regalloc.frame_manager.frame_depth 
+        looptoken._arm_frame_depth = regalloc.frame_manager.frame_depth
         self._patch_sp_offset(sp_patch_location, looptoken)
 
         self.align()
@@ -420,7 +421,7 @@
 
         self._walk_operations(operations, regalloc)
 
-        original_loop_token._arm_frame_depth = regalloc.frame_manager.frame_depth 
+        original_loop_token._arm_frame_depth = regalloc.frame_manager.frame_depth
         self._patch_sp_offset(sp_patch_location, original_loop_token)
 
         bridge_start = self.materialize_loop(original_loop_token)
@@ -575,7 +576,7 @@
                 self.mc.gen_load_int(new_loc.value, prev_loc.getint(), cond=cond)
             prev_loc = new_loc
             if not loc.is_stack():
-                return 
+                return
         if loc.is_stack() and prev_loc.is_reg():
             self.mc.STR_ri(prev_loc.value, r.fp.value, loc.position*-WORD, cond=cond)
         elif loc.is_reg() and prev_loc.is_stack():

diff --git a/pypy/jit/backend/arm/opassembler.py b/pypy/jit/backend/arm/opassembler.py
--- a/pypy/jit/backend/arm/opassembler.py
+++ b/pypy/jit/backend/arm/opassembler.py
@@ -666,7 +666,7 @@
             else:
                 raise AssertionError(kind)
         # check value
-        resloc = regalloc.force_allocate_reg(resbox)
+        resloc = regalloc.try_allocate_reg(resbox)
         assert resloc is r.r0
         self.mc.gen_load_int(r.ip.value, value)
         self.mc.CMP_rr(resloc.value, r.ip.value)
@@ -709,14 +709,13 @@
             assert isinstance(fielddescr, BaseFieldDescr)
             ofs = fielddescr.offset
             resloc = regalloc.force_allocate_reg(resbox)
-            self.mov_loc_loc(arglocs[1], r.ip, cond=c.MI)
-            self.mc.MOV_ri(resloc.value, 0, cond=c.MI)
-            self.mc.STR_ri(resloc.value, r.ip.value, ofs*WORD, cond=c.MI)
+            self.mov_loc_loc(arglocs[1], r.ip)
+            self.mc.MOV_ri(resloc.value, 0)
+            self.mc.STR_ri(resloc.value, r.ip.value, ofs)
             regalloc.possibly_free_var(resbox)
 
         if op.result is not None:
             # load the return value from fail_boxes_xxx[0]
-            resloc = regalloc.force_allocate_reg(op.result)
             kind = op.result.type
             if kind == INT:
                 adr = self.fail_boxes_int.get_addr_for_num(0)
@@ -724,20 +723,21 @@
                 adr = self.fail_boxes_ptr.get_addr_for_num(0)
             else:
                 raise AssertionError(kind)
+            resloc = regalloc.force_allocate_reg(op.result)
+            regalloc.possibly_free_var(resbox)
             self.mc.gen_load_int(r.ip.value, adr)
             self.mc.LDR_ri(resloc.value, r.ip.value)
 
+        # merge point
         offset = self.mc.currpos() - jmp_pos
-        pmc = OverwritingBuilder(self.mc, jmp_pos, WORD)
-        pmc.ADD_ri(r.pc.value, r.pc.value, offset - PC_OFFSET)
+        if offset - PC_OFFSET >= 0:
+            pmc = OverwritingBuilder(self.mc, jmp_pos, WORD)
+            pmc.ADD_ri(r.pc.value, r.pc.value, offset - PC_OFFSET)
 
         self.mc.LDR_ri(r.ip.value, r.fp.value)
         self.mc.CMP_ri(r.ip.value, 0)
 
         self._emit_guard(guard_op, regalloc._prepare_guard(guard_op), c.GE)
-        regalloc.possibly_free_vars_for_op(op)
-        if op.result:
-            regalloc.possibly_free_var(op.result)
         return fcond
 
 

diff --git a/pypy/jit/backend/arm/codebuilder.py b/pypy/jit/backend/arm/codebuilder.py
--- a/pypy/jit/backend/arm/codebuilder.py
+++ b/pypy/jit/backend/arm/codebuilder.py
@@ -44,15 +44,15 @@
         self.write32(instr)
 
     def VPUSH(self, regs, cond=cond.AL):
-        nregs = len(regs) 
+        nregs = len(regs)
         assert nregs > 0 and nregs <= 16
         freg = regs[0]
         D = (freg & 0x10) >> 4
         Dd = (freg & 0xF)
-        nregs *= 2 
-        instr = (cond << 28 
-                | 0xD2D << 16 
-                | D << 22 
+        nregs *= 2
+        instr = (cond << 28
+                | 0xD2D << 16
+                | D << 22
                 | Dd << 12
                 | 0xB << 8
                 | nregs)

From commits-noreply at bitbucket.org  Wed Feb 23 18:56:16 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Wed, 23 Feb 2011 18:56:16 +0100 (CET)
Subject: [pypy-svn] pypy arm-backend-2: tests and test fixes
Message-ID: <20110223175616.7F4F62A2033@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backend-2
Changeset: r42240:405092e4e2e3
Date: 2011-02-23 18:54 +0100
http://bitbucket.org/pypy/pypy/changeset/405092e4e2e3/

Log:	tests and test fixes

diff --git a/pypy/jit/backend/arm/test/test_regalloc.py b/pypy/jit/backend/arm/test/test_regalloc.py
--- a/pypy/jit/backend/arm/test/test_regalloc.py
+++ b/pypy/jit/backend/arm/test/test_regalloc.py
@@ -665,6 +665,20 @@
         bridge = self.attach_bridge(ops, loop, -2)
 
 
+        self.cpu.set_future_value_int(0, 4)
+        self.cpu.set_future_value_int(1, 7)
+        self.run(loop)
+        assert self.getint(0) == 29
+
+    def test_jump_with_consts(self):
+        loop = """
+        [i0, i1, i2, i3, i4, i5, i6, i7, i8, i9, i10, i11, i12, i13, i14]
+        jump(i1, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15)
+        """
+        large = self.interpret(loop, range(15), run=False)
+        # ensure compiling this loop works
+        assert 1
+
     def test_from_loop_to_loop(self):
         def assembler_helper(failindex, virtualizable):
             return 1
@@ -726,17 +740,3 @@
         """
         self.interpret(ops, range(11))
         assert self.getint(0) == 2 # and not segfault()
-
-        self.cpu.set_future_value_int(0, 4)
-        self.cpu.set_future_value_int(1, 7)        
-        self.run(loop)
-        assert self.getint(0) == 29
-
-    def test_jump_with_consts(self):
-        loop = """ 
-        [i0, i1, i2, i3, i4, i5, i6, i7, i8, i9, i10, i11, i12, i13, i14]
-        jump(i1, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15)
-        """ 
-        large = self.interpret(loop, range(15), run=False)
-        # ensure compiling this loop works
-        assert 1

diff --git a/pypy/jit/backend/arm/test/test_assembler.py b/pypy/jit/backend/arm/test/test_assembler.py
--- a/pypy/jit/backend/arm/test/test_assembler.py
+++ b/pypy/jit/backend/arm/test/test_assembler.py
@@ -163,7 +163,7 @@
 
         # call to div
         self.a.mc.PUSH(range(2, 12))
-        div_addr = rffi.cast(lltype.Signed, llhelper(arm_int_div_sign, arm_int_div))
+        div_addr = rffi.cast(lltype.Signed, arm_int_div)
         self.a.mc.BL(div_addr)
         self.a.mc.POP(range(2, 12))
         self.a.gen_func_epilog()
@@ -217,8 +217,16 @@
         self.a.mov_loc_loc(imm(2478), r.r0)
         self.a.gen_func_epilog()
         assert run_asm(self.a) == 2478
-      
 
+    def test_load_store(self):
+        x =  0x60002224
+        self.a.gen_func_prolog()
+        self.a.mc.gen_load_int(r.r1.value, x)
+        self.a.mc.MOV_ri(r.r3.value, 8)
+        self.a.mc.STR_rr(r.r1.value, r.fp.value, r.r3.value)
+        self.a.mc.LDR_ri(r.r0.value, r.fp.value, 8)
+        self.a.gen_func_epilog()
+        assert run_asm(self.a) == x
 
 def callme(inp):
     i = inp + 10

From commits-noreply at bitbucket.org  Wed Feb 23 18:59:11 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Wed, 23 Feb 2011 18:59:11 +0100 (CET)
Subject: [pypy-svn] pypy default: add a way to get the ops for just the
	specified opcode
Message-ID: <20110223175911.5127B36C21D@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42241:6466e4ce87f0
Date: 2011-02-23 18:58 +0100
http://bitbucket.org/pypy/pypy/changeset/6466e4ce87f0/

Log:	add a way to get the ops for just the specified opcode

diff --git a/pypy/module/pypyjit/test_pypy_c/model.py b/pypy/module/pypyjit/test_pypy_c/model.py
--- a/pypy/module/pypyjit/test_pypy_c/model.py
+++ b/pypy/module/pypyjit/test_pypy_c/model.py
@@ -130,11 +130,13 @@
             for op in self._ops_for_chunk(chunk, include_debug_merge_points):
                 yield op
 
-    def ops_by_id(self, id, include_debug_merge_points=False):
+    def ops_by_id(self, id, include_debug_merge_points=False, opcode=None):
+        opcode_name = opcode
         target_opcodes = self.ids[id]
         for chunk in self.flatten_chunks():
             opcode = chunk.getopcode()
-            if opcode in target_opcodes:
+            if opcode in target_opcodes and (opcode_name is None or
+                                             opcode.__class__.__name__ == opcode_name):
                 for op in self._ops_for_chunk(chunk, include_debug_merge_points):
                     yield op
 

diff --git a/pypy/module/pypyjit/test_pypy_c/test_model.py b/pypy/module/pypyjit/test_pypy_c/test_model.py
--- a/pypy/module/pypyjit/test_pypy_c/test_model.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_model.py
@@ -142,6 +142,26 @@
         opnames = [op.name for op in ops]
         assert opnames == ['int_add']
 
+    def test_ops_by_id_and_opcode(self):
+        def f():
+            i = 0
+            j = 0
+            while i < 1003:
+                i += 1; j -= 1 # ID: foo
+                a = 0  # to make sure that JUMP_ABSOLUTE is not part of the ID
+            return i
+        #
+        log = self.run(f)
+        loop, = log.loops_by_id('foo')
+        #
+        ops = list(loop.ops_by_id('foo', opcode='INPLACE_ADD'))
+        opnames = [op.name for op in ops]
+        assert opnames == ['int_add']
+        #
+        ops = list(loop.ops_by_id('foo', opcode='INPLACE_SUBTRACT'))
+        opnames = [op.name for op in ops]
+        assert opnames == ['int_sub_ovf', 'guard_no_overflow']
+        
 
     def test_inlined_function(self):
         def f():

From commits-noreply at bitbucket.org  Wed Feb 23 19:01:23 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Wed, 23 Feb 2011 19:01:23 +0100 (CET)
Subject: [pypy-svn] pypy default: Be much more precise about this test.
Message-ID: <20110223180123.D05DA36C21D@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r42242:4336427adba2
Date: 2011-02-23 13:01 -0500
http://bitbucket.org/pypy/pypy/changeset/4336427adba2/

Log:	Be much more precise about this test.

diff --git a/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py b/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py
--- a/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py
@@ -58,19 +58,18 @@
 
     def test_cmp_exc(self):
         def f1(n):
-            def f():
-                raise KeyError
-
+            # So we don't get a LOAD_GLOBAL op
+            KE = KeyError
             i = 0
             while i < n:
                 try:
-                    f()
-                except KeyError: # ID: except
+                    raise KE
+                except KE: # ID: except
                     i += 1
             return i
 
         log = self.run(f1, [10000])
         assert log.result == 10000
         loop, = log.loops_by_id("except")
-        ops = [o.name for o in loop.ops_by_id("except")]
-        assert "call_may_force" not in ops
\ No newline at end of file
+        ops = list(loop.ops_by_id("except", opcode="COMPARE_OP"))
+        assert ops == []

From commits-noreply at bitbucket.org  Wed Feb 23 19:06:24 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Wed, 23 Feb 2011 19:06:24 +0100 (CET)
Subject: [pypy-svn] pypy default: add a convenience function to get the
	opnames
Message-ID: <20110223180624.C617236C21D@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42243:47e29caac238
Date: 2011-02-23 19:06 +0100
http://bitbucket.org/pypy/pypy/changeset/47e29caac238/

Log:	add a convenience function to get the opnames

diff --git a/pypy/module/pypyjit/test_pypy_c/model.py b/pypy/module/pypyjit/test_pypy_c/model.py
--- a/pypy/module/pypyjit/test_pypy_c/model.py
+++ b/pypy/module/pypyjit/test_pypy_c/model.py
@@ -61,6 +61,9 @@
         return [loop for loop in self.loops
                 if loop.has_id(id) and self._filter(loop, **kwds)]
 
+    @classmethod
+    def opnames(self, oplist):
+        return [op.name for op in oplist]
 
 class LoopWithIds(Function):
 

diff --git a/pypy/module/pypyjit/test_pypy_c/test_model.py b/pypy/module/pypyjit/test_pypy_c/test_model.py
--- a/pypy/module/pypyjit/test_pypy_c/test_model.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_model.py
@@ -117,9 +117,8 @@
         assert loop.filename == self.filepath
         assert loop.code.co.co_name == 'f'
         #
-        ops = list(loop.allops())
-        opnames = [op.name for op in ops]
-        assert opnames == [
+        ops = loop.allops()
+        assert log.opnames(ops) == [
             # this is the actual loop
             'int_lt', 'guard_true', 'int_add',
             # this is the signal checking stuff
@@ -138,9 +137,8 @@
         log = self.run(f)
         loop, = log.loops_by_id('increment')
         #
-        ops = list(loop.ops_by_id('increment'))
-        opnames = [op.name for op in ops]
-        assert opnames == ['int_add']
+        ops = loop.ops_by_id('increment')
+        assert log.opnames(ops) == ['int_add']
 
     def test_ops_by_id_and_opcode(self):
         def f():
@@ -154,13 +152,11 @@
         log = self.run(f)
         loop, = log.loops_by_id('foo')
         #
-        ops = list(loop.ops_by_id('foo', opcode='INPLACE_ADD'))
-        opnames = [op.name for op in ops]
-        assert opnames == ['int_add']
+        ops = loop.ops_by_id('foo', opcode='INPLACE_ADD')
+        assert log.opnames(ops) == ['int_add']
         #
-        ops = list(loop.ops_by_id('foo', opcode='INPLACE_SUBTRACT'))
-        opnames = [op.name for op in ops]
-        assert opnames == ['int_sub_ovf', 'guard_no_overflow']
+        ops = loop.ops_by_id('foo', opcode='INPLACE_SUBTRACT')
+        assert log.opnames(ops) == ['int_sub_ovf', 'guard_no_overflow']
         
 
     def test_inlined_function(self):
@@ -175,13 +171,13 @@
         #
         log = self.run(f)
         loop, = log.loops_by_filename(self.filepath)
-        call_ops = [op.name for op in loop.ops_by_id('call')]
+        call_ops = log.opnames(loop.ops_by_id('call'))
         assert call_ops == ['force_token'] # it does not follow inlining
         #
-        add_ops = [op.name for op in loop.ops_by_id('add')]
+        add_ops = log.opnames(loop.ops_by_id('add'))
         assert add_ops == ['int_add']
         #
-        ops = [op.name for op in loop.allops()]
+        ops = log.opnames(loop.allops())
         assert ops == [
             # this is the actual loop
             'int_lt', 'guard_true', 'force_token', 'int_add',

From commits-noreply at bitbucket.org  Wed Feb 23 21:14:15 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed, 23 Feb 2011 21:14:15 +0100 (CET)
Subject: [pypy-svn] pypy default: Add a line about how to use it (I have to
	look it up	every time).
Message-ID: <20110223201415.CA05B282BDE@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r42244:7379011bbdae
Date: 2011-02-23 21:10 +0100
http://bitbucket.org/pypy/pypy/changeset/7379011bbdae/

Log:	Add a line about how to use it (I have to look it up every time).

diff --git a/pypy/jit/tool/pypytrace.vim b/pypy/jit/tool/pypytrace.vim
--- a/pypy/jit/tool/pypytrace.vim
+++ b/pypy/jit/tool/pypytrace.vim
@@ -1,5 +1,6 @@
 " Language   : PyPy JIT traces
 " Maintainer : Armin Rigo
+" Usage      : set syntax=pypytrace
 
 if exists("b:current_syntax")
  finish

From commits-noreply at bitbucket.org  Wed Feb 23 21:42:48 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Wed, 23 Feb 2011 21:42:48 +0100 (CET)
Subject: [pypy-svn] pypy default: (vincele) Fix utf7 decoding
Message-ID: <20110223204248.F3969282BDE@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: 
Changeset: r42245:faefe0c03a55
Date: 2011-02-23 21:40 +0100
http://bitbucket.org/pypy/pypy/changeset/faefe0c03a55/

Log:	(vincele) Fix utf7 decoding

	At the end of a base64 sequence parsing, reset base64 state
	variables, to avoid screwing next utf7 char decoding.

diff --git a/pypy/rlib/test/test_runicode.py b/pypy/rlib/test/test_runicode.py
--- a/pypy/rlib/test/test_runicode.py
+++ b/pypy/rlib/test/test_runicode.py
@@ -1,3 +1,5 @@
+# -*- coding: utf8 -*-
+
 import py
 import sys, random
 from pypy.rlib import runicode
@@ -288,6 +290,30 @@
         u = u'A\u2262\u0391.'
         assert runicode.unicode_encode_utf_7(u, len(u), None) == 'A+ImIDkQ.'
 
+    def test_utf7_tofrom_utf8_bug(self):
+        def _assert_decu7(input, expected):
+            assert runicode.str_decode_utf_7(input, len(input), None) == (expected, len(input))
+
+        _assert_decu7('+-', u'+')
+        _assert_decu7('+-+-', u'++')
+        _assert_decu7('+-+AOQ-', u'+\xe4')
+        _assert_decu7('+AOQ-', u'\xe4')
+        _assert_decu7('+AOQ-', u'\xe4')
+        _assert_decu7('+AOQ- ', u'\xe4 ')
+        _assert_decu7(' +AOQ-', u' \xe4')
+        _assert_decu7(' +AOQ- ', u' \xe4 ')
+        _assert_decu7('+AOQ-+AOQ-', u'\xe4\xe4')
+
+        s_utf7 = 'Die M+AOQ-nner +AOQ-rgen sich!'
+        s_utf8 = u'Die M&#228;nner &#228;rgen sich!'
+        s_utf8_esc = u'Die M\xe4nner \xe4rgen sich!'
+
+        _assert_decu7(s_utf7, s_utf8_esc)
+        _assert_decu7(s_utf7, s_utf8)
+
+        assert runicode.unicode_encode_utf_7(s_utf8_esc, len(s_utf8_esc), None) == s_utf7
+        assert runicode.unicode_encode_utf_7(s_utf8,     len(s_utf8_esc), None) == s_utf7
+
     def test_utf7_partial(self):
         s = u"a+-b".encode('utf-7')
         assert s == "a+--b"

diff --git a/pypy/rlib/runicode.py b/pypy/rlib/runicode.py
--- a/pypy/rlib/runicode.py
+++ b/pypy/rlib/runicode.py
@@ -732,7 +732,7 @@
 
             else:
                 # now leaving a base-64 section
-                inShift = 0
+                inShift = False
                 pos += 1
 
                 if surrogate:
@@ -762,7 +762,9 @@
                 if ch == '-':
                     # '-' is absorbed; other terminating characters are
                     # preserved
-                    pass
+                    base64bits = 0
+                    base64buffer = 0
+                    surrogate = 0
                 else:
                     result.append(unichr(ord(ch)))
 

From commits-noreply at bitbucket.org  Wed Feb 23 21:42:50 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Wed, 23 Feb 2011 21:42:50 +0100 (CET)
Subject: [pypy-svn] pypy default: (vincele) Fix AST_Tests.test_snippets
	from	lib-python/2.7.0/test/test_ast.py
Message-ID: <20110223204250.A52BA282BDE@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: 
Changeset: r42246:8759aa9651bd
Date: 2011-02-23 21:41 +0100
http://bitbucket.org/pypy/pypy/changeset/8759aa9651bd/

Log:	(vincele) Fix AST_Tests.test_snippets from lib-
	python/2.7.0/test/test_ast.py

	CPython changed lineno & col_offset reporting, see:
	http://bugs.python.org/issue6704 (Fix Commited in r74464)

	The attached patch fixes this, and it covers other cases not
	currently tested. A modified test is available in the patch that add
	missing test snippets.

diff --git a/lib-python/modified-2.7.0/test/test_ast.py b/lib-python/modified-2.7.0/test/test_ast.py
new file mode 100644
--- /dev/null
+++ b/lib-python/modified-2.7.0/test/test_ast.py
@@ -0,0 +1,423 @@
+import sys, itertools, unittest
+from test import test_support
+import ast
+
+def to_tuple(t):
+    if t is None or isinstance(t, (basestring, int, long, complex)):
+        return t
+    elif isinstance(t, list):
+        return [to_tuple(e) for e in t]
+    result = [t.__class__.__name__]
+    if hasattr(t, 'lineno') and hasattr(t, 'col_offset'):
+        result.append((t.lineno, t.col_offset))
+    if t._fields is None:
+        return tuple(result)
+    for f in t._fields:
+        result.append(to_tuple(getattr(t, f)))
+    return tuple(result)
+
+
+# These tests are compiled through "exec"
+# There should be atleast one test per statement
+exec_tests = [
+    # FunctionDef
+    "def f(): pass",
+    # ClassDef
+    "class C:pass",
+    # Return
+    "def f():return 1",
+    # Delete
+    "del v",
+    # Assign
+    "v = 1",
+    # AugAssign
+    "v += 1",
+    # Print
+    "print >>f, 1, ",
+    # For
+    "for v in v:pass",
+    # While
+    "while v:pass",
+    # If
+    "if v:pass",
+    # Raise
+    "raise Exception, 'string'",
+    # TryExcept
+    "try:\n  pass\nexcept Exception:\n  pass",
+    # TryFinally
+    "try:\n  pass\nfinally:\n  pass",
+    # Assert
+    "assert v",
+    # Import
+    "import sys",
+    # ImportFrom
+    "from sys import v",
+    # Exec
+    "exec 'v'",
+    # Global
+    "global v",
+    # Expr
+    "1",
+    # Pass,
+    "pass",
+    # Break
+    "break",
+    # Continue
+    "continue",
+    # for statements with naked tuples (see http://bugs.python.org/issue6704)
+    "for a,b in c: pass",
+    "[(a,b) for a,b in c]",
+    "((a,b) for a,b in c)",
+    "((a,b) for (a,b) in c)",
+    # Multiline generator expression
+    """(
+    (
+    Aa
+    ,
+    Bb
+    )
+    for
+    Aa
+    ,
+    Bb in Cc
+    )""",
+    # dictcomp
+    "{a : b for w in x for m in p if g}",
+    # dictcomp
+    "{a : b for v,w in x}",
+    # setcomp
+    "{r for l in x if g}",
+    # setcomp
+    "{r for l,m in x}",
+]
+
+# These are compiled through "single"
+# because of overlap with "eval", it just tests what
+# can't be tested with "eval"
+single_tests = [
+    "1+2"
+]
+
+# These are compiled through "eval"
+# It should test all expressions
+eval_tests = [
+  # BoolOp
+  "a and b",
+  # BinOp
+  "a + b",
+  # UnaryOp
+  "not v",
+  # Lambda
+  "lambda:None",
+  # Dict
+  "{ 1:2 }",
+  # Empty dict
+  "{}",
+  # Set
+  "{None,}",
+  # Multiline dict
+  """{
+1
+ :
+  2
+    }""",
+  # ListComp
+  "[a for b in c if d]",
+  # GeneratorExp
+  "(a for b in c if d)",
+  # Yield - yield expressions can't work outside a function
+  #
+  # Compare
+  "1 < 2 < 3",
+  # Call
+  "f(1,2,c=3,*d,**e)",
+  # Repr
+  "`v`",
+  # Num
+  "10L",
+  # Str
+  "'string'",
+  # Attribute
+  "a.b",
+  # Subscript
+  "a[b:c]",
+  # Name
+  "v",
+  # List
+  "[1,2,3]",
+  # Empty list
+  "[]",
+  # Tuple
+  "1,2,3",
+  # Tuple
+  "(1,2,3)",
+  # Empty tuple
+  "()",
+  # Combination
+  "a.b.c.d(a.b[1:2])",
+
+]
+
+# TODO: expr_context, slice, boolop, operator, unaryop, cmpop, comprehension
+# excepthandler, arguments, keywords, alias
+
+class AST_Tests(unittest.TestCase):
+
+    def _assertTrueorder(self, ast_node, parent_pos):
+        if not isinstance(ast_node, ast.AST) or ast_node._fields is None:
+            return
+        if isinstance(ast_node, (ast.expr, ast.stmt, ast.excepthandler)):
+            node_pos = (ast_node.lineno, ast_node.col_offset)
+            self.assertTrue(node_pos >= parent_pos)
+            parent_pos = (ast_node.lineno, ast_node.col_offset)
+        for name in ast_node._fields:
+            value = getattr(ast_node, name)
+            if isinstance(value, list):
+                for child in value:
+                    self._assertTrueorder(child, parent_pos)
+            elif value is not None:
+                self._assertTrueorder(value, parent_pos)
+
+    def test_snippets(self):
+        for input, output, kind in ((exec_tests, exec_results, "exec"),
+                                    (single_tests, single_results, "single"),
+                                    (eval_tests, eval_results, "eval")):
+            for i, o in itertools.izip(input, output):
+                ast_tree = compile(i, "?", kind, ast.PyCF_ONLY_AST)
+                self.assertEquals(to_tuple(ast_tree), o)
+                self._assertTrueorder(ast_tree, (0, 0))
+
+    def test_slice(self):
+        slc = ast.parse("x[::]").body[0].value.slice
+        self.assertIsNone(slc.upper)
+        self.assertIsNone(slc.lower)
+        self.assertIsInstance(slc.step, ast.Name)
+        self.assertEqual(slc.step.id, "None")
+
+    def test_from_import(self):
+        im = ast.parse("from . import y").body[0]
+        self.assertIsNone(im.module)
+
+    def test_base_classes(self):
+        self.assertTrue(issubclass(ast.For, ast.stmt))
+        self.assertTrue(issubclass(ast.Name, ast.expr))
+        self.assertTrue(issubclass(ast.stmt, ast.AST))
+        self.assertTrue(issubclass(ast.expr, ast.AST))
+        self.assertTrue(issubclass(ast.comprehension, ast.AST))
+        self.assertTrue(issubclass(ast.Gt, ast.AST))
+
+    def test_nodeclasses(self):
+        x = ast.BinOp(1, 2, 3, lineno=0)
+        self.assertEquals(x.left, 1)
+        self.assertEquals(x.op, 2)
+        self.assertEquals(x.right, 3)
+        self.assertEquals(x.lineno, 0)
+
+        # node raises exception when not given enough arguments
+        self.assertRaises(TypeError, ast.BinOp, 1, 2)
+
+        # can set attributes through kwargs too
+        x = ast.BinOp(left=1, op=2, right=3, lineno=0)
+        self.assertEquals(x.left, 1)
+        self.assertEquals(x.op, 2)
+        self.assertEquals(x.right, 3)
+        self.assertEquals(x.lineno, 0)
+
+        # this used to fail because Sub._fields was None
+        x = ast.Sub()
+
+    def test_pickling(self):
+        import pickle
+        mods = [pickle]
+        try:
+            import cPickle
+            mods.append(cPickle)
+        except ImportError:
+            pass
+        protocols = [0, 1, 2]
+        for mod in mods:
+            for protocol in protocols:
+                for ast in (compile(i, "?", "exec", 0x400) for i in exec_tests):
+                    ast2 = mod.loads(mod.dumps(ast, protocol))
+                    self.assertEquals(to_tuple(ast2), to_tuple(ast))
+
+
+class ASTHelpers_Test(unittest.TestCase):
+
+    def test_parse(self):
+        a = ast.parse('foo(1 + 1)')
+        b = compile('foo(1 + 1)', '<unknown>', 'exec', ast.PyCF_ONLY_AST)
+        self.assertEqual(ast.dump(a), ast.dump(b))
+
+    def test_dump(self):
+        node = ast.parse('spam(eggs, "and cheese")')
+        self.assertEqual(ast.dump(node),
+            "Module(body=[Expr(value=Call(func=Name(id='spam', ctx=Load()), "
+            "args=[Name(id='eggs', ctx=Load()), Str(s='and cheese')], "
+            "keywords=[], starargs=None, kwargs=None))])"
+        )
+        self.assertEqual(ast.dump(node, annotate_fields=False),
+            "Module([Expr(Call(Name('spam', Load()), [Name('eggs', Load()), "
+            "Str('and cheese')], [], None, None))])"
+        )
+        self.assertEqual(ast.dump(node, include_attributes=True),
+            "Module(body=[Expr(value=Call(func=Name(id='spam', ctx=Load(), "
+            "lineno=1, col_offset=0), args=[Name(id='eggs', ctx=Load(), "
+            "lineno=1, col_offset=5), Str(s='and cheese', lineno=1, "
+            "col_offset=11)], keywords=[], starargs=None, kwargs=None, "
+            "lineno=1, col_offset=0), lineno=1, col_offset=0)])"
+        )
+
+    def test_copy_location(self):
+        src = ast.parse('1 + 1', mode='eval')
+        src.body.right = ast.copy_location(ast.Num(2), src.body.right)
+        self.assertEqual(ast.dump(src, include_attributes=True),
+            'Expression(body=BinOp(left=Num(n=1, lineno=1, col_offset=0), '
+            'op=Add(), right=Num(n=2, lineno=1, col_offset=4), lineno=1, '
+            'col_offset=0))'
+        )
+
+    def test_fix_missing_locations(self):
+        src = ast.parse('write("spam")')
+        src.body.append(ast.Expr(ast.Call(ast.Name('spam', ast.Load()),
+                                          [ast.Str('eggs')], [], None, None)))
+        self.assertEqual(src, ast.fix_missing_locations(src))
+        self.assertEqual(ast.dump(src, include_attributes=True),
+            "Module(body=[Expr(value=Call(func=Name(id='write', ctx=Load(), "
+            "lineno=1, col_offset=0), args=[Str(s='spam', lineno=1, "
+            "col_offset=6)], keywords=[], starargs=None, kwargs=None, "
+            "lineno=1, col_offset=0), lineno=1, col_offset=0), "
+            "Expr(value=Call(func=Name(id='spam', ctx=Load(), lineno=1, "
+            "col_offset=0), args=[Str(s='eggs', lineno=1, col_offset=0)], "
+            "keywords=[], starargs=None, kwargs=None, lineno=1, "
+            "col_offset=0), lineno=1, col_offset=0)])"
+        )
+
+    def test_increment_lineno(self):
+        src = ast.parse('1 + 1', mode='eval')
+        self.assertEqual(ast.increment_lineno(src, n=3), src)
+        self.assertEqual(ast.dump(src, include_attributes=True),
+            'Expression(body=BinOp(left=Num(n=1, lineno=4, col_offset=0), '
+            'op=Add(), right=Num(n=1, lineno=4, col_offset=4), lineno=4, '
+            'col_offset=0))'
+        )
+
+    def test_iter_fields(self):
+        node = ast.parse('foo()', mode='eval')
+        d = dict(ast.iter_fields(node.body))
+        self.assertEqual(d.pop('func').id, 'foo')
+        self.assertEqual(d, {'keywords': [], 'kwargs': None,
+                             'args': [], 'starargs': None})
+
+    def test_iter_child_nodes(self):
+        node = ast.parse("spam(23, 42, eggs='leek')", mode='eval')
+        self.assertEqual(len(list(ast.iter_child_nodes(node.body))), 4)
+        iterator = ast.iter_child_nodes(node.body)
+        self.assertEqual(next(iterator).id, 'spam')
+        self.assertEqual(next(iterator).n, 23)
+        self.assertEqual(next(iterator).n, 42)
+        self.assertEqual(ast.dump(next(iterator)),
+            "keyword(arg='eggs', value=Str(s='leek'))"
+        )
+
+    def __test_get_docstring(self):
+        node = ast.parse('def foo():\n  """line one\n  line two"""')
+        self.assertEqual(ast.get_docstring(node.body[0]),
+                         'line one\nline two')
+
+    def test_literal_eval(self):
+        self.assertEqual(ast.literal_eval('[1, 2, 3]'), [1, 2, 3])
+        self.assertEqual(ast.literal_eval('{"foo": 42}'), {"foo": 42})
+        self.assertEqual(ast.literal_eval('(True, False, None)'), (True, False, None))
+        self.assertRaises(ValueError, ast.literal_eval, 'foo()')
+
+    def test_literal_eval_issue4907(self):
+        self.assertEqual(ast.literal_eval('2j'), 2j)
+        self.assertEqual(ast.literal_eval('10 + 2j'), 10 + 2j)
+        self.assertEqual(ast.literal_eval('1.5 - 2j'), 1.5 - 2j)
+        self.assertRaises(ValueError, ast.literal_eval, '2 + (3 + 4j)')
+
+
+def test_main():
+    with test_support.check_py3k_warnings(("backquote not supported",
+                                             SyntaxWarning)):
+        test_support.run_unittest(AST_Tests, ASTHelpers_Test)
+
+def main():
+    if __name__ != '__main__':
+        return
+    if sys.argv[1:] == ['-g']:
+        for statements, kind in ((exec_tests, "exec"), (single_tests, "single"),
+                                 (eval_tests, "eval")):
+            print kind+"_results = ["
+            for s in statements:
+                print repr(to_tuple(compile(s, "?", kind, 0x400)))+","
+            print "]"
+        print "main()"
+        raise SystemExit
+    test_main()
+
+#### EVERYTHING BELOW IS GENERATED #####
+exec_results = [
+('Module', [('FunctionDef', (1, 0), 'f', ('arguments', [], None, None, []), [('Pass', (1, 9))], [])]),
+('Module', [('ClassDef', (1, 0), 'C', [], [('Pass', (1, 8))], [])]),
+('Module', [('FunctionDef', (1, 0), 'f', ('arguments', [], None, None, []), [('Return', (1, 8), ('Num', (1, 15), 1))], [])]),
+('Module', [('Delete', (1, 0), [('Name', (1, 4), 'v', ('Del',))])]),
+('Module', [('Assign', (1, 0), [('Name', (1, 0), 'v', ('Store',))], ('Num', (1, 4), 1))]),
+('Module', [('AugAssign', (1, 0), ('Name', (1, 0), 'v', ('Store',)), ('Add',), ('Num', (1, 5), 1))]),
+('Module', [('Print', (1, 0), ('Name', (1, 8), 'f', ('Load',)), [('Num', (1, 11), 1)], False)]),
+('Module', [('For', (1, 0), ('Name', (1, 4), 'v', ('Store',)), ('Name', (1, 9), 'v', ('Load',)), [('Pass', (1, 11))], [])]),
+('Module', [('While', (1, 0), ('Name', (1, 6), 'v', ('Load',)), [('Pass', (1, 8))], [])]),
+('Module', [('If', (1, 0), ('Name', (1, 3), 'v', ('Load',)), [('Pass', (1, 5))], [])]),
+('Module', [('Raise', (1, 0), ('Name', (1, 6), 'Exception', ('Load',)), ('Str', (1, 17), 'string'), None)]),
+('Module', [('TryExcept', (1, 0), [('Pass', (2, 2))], [('ExceptHandler', (3, 0), ('Name', (3, 7), 'Exception', ('Load',)), None, [('Pass', (4, 2))])], [])]),
+('Module', [('TryFinally', (1, 0), [('Pass', (2, 2))], [('Pass', (4, 2))])]),
+('Module', [('Assert', (1, 0), ('Name', (1, 7), 'v', ('Load',)), None)]),
+('Module', [('Import', (1, 0), [('alias', 'sys', None)])]),
+('Module', [('ImportFrom', (1, 0), 'sys', [('alias', 'v', None)], 0)]),
+('Module', [('Exec', (1, 0), ('Str', (1, 5), 'v'), None, None)]),
+('Module', [('Global', (1, 0), ['v'])]),
+('Module', [('Expr', (1, 0), ('Num', (1, 0), 1))]),
+('Module', [('Pass', (1, 0))]),
+('Module', [('Break', (1, 0))]),
+('Module', [('Continue', (1, 0))]),
+('Module', [('For', (1, 0), ('Tuple', (1, 4), [('Name', (1, 4), 'a', ('Store',)), ('Name', (1, 6), 'b', ('Store',))], ('Store',)), ('Name', (1, 11), 'c', ('Load',)), [('Pass', (1, 14))], [])]),
+('Module', [('Expr', (1, 0), ('ListComp', (1, 1), ('Tuple', (1, 2), [('Name', (1, 2), 'a', ('Load',)), ('Name', (1, 4), 'b', ('Load',))], ('Load',)), [('comprehension', ('Tuple', (1, 11), [('Name', (1, 11), 'a', ('Store',)), ('Name', (1, 13), 'b', ('Store',))], ('Store',)), ('Name', (1, 18), 'c', ('Load',)), [])]))]),
+('Module', [('Expr', (1, 0), ('GeneratorExp', (1, 1), ('Tuple', (1, 2), [('Name', (1, 2), 'a', ('Load',)), ('Name', (1, 4), 'b', ('Load',))], ('Load',)), [('comprehension', ('Tuple', (1, 11), [('Name', (1, 11), 'a', ('Store',)), ('Name', (1, 13), 'b', ('Store',))], ('Store',)), ('Name', (1, 18), 'c', ('Load',)), [])]))]),
+('Module', [('Expr', (1, 0), ('GeneratorExp', (1, 1), ('Tuple', (1, 2), [('Name', (1, 2), 'a', ('Load',)), ('Name', (1, 4), 'b', ('Load',))], ('Load',)), [('comprehension', ('Tuple', (1, 12), [('Name', (1, 12), 'a', ('Store',)), ('Name', (1, 14), 'b', ('Store',))], ('Store',)), ('Name', (1, 20), 'c', ('Load',)), [])]))]),
+('Module', [('Expr', (1, 0), ('GeneratorExp', (2, 4), ('Tuple', (3, 4), [('Name', (3, 4), 'Aa', ('Load',)), ('Name', (5, 4), 'Bb', ('Load',))], ('Load',)), [('comprehension', ('Tuple', (8, 4), [('Name', (8, 4), 'Aa', ('Store',)), ('Name', (10, 4), 'Bb', ('Store',))], ('Store',)), ('Name', (10, 10), 'Cc', ('Load',)), [])]))]),
+('Module', [('Expr', (1, 0), ('DictComp', (1, 1), ('Name', (1, 1), 'a', ('Load',)), ('Name', (1, 5), 'b', ('Load',)), [('comprehension', ('Name', (1, 11), 'w', ('Store',)), ('Name', (1, 16), 'x', ('Load',)), []), ('comprehension', ('Name', (1, 22), 'm', ('Store',)), ('Name', (1, 27), 'p', ('Load',)), [('Name', (1, 32), 'g', ('Load',))])]))]),
+('Module', [('Expr', (1, 0), ('DictComp', (1, 1), ('Name', (1, 1), 'a', ('Load',)), ('Name', (1, 5), 'b', ('Load',)), [('comprehension', ('Tuple', (1, 11), [('Name', (1, 11), 'v', ('Store',)), ('Name', (1, 13), 'w', ('Store',))], ('Store',)), ('Name', (1, 18), 'x', ('Load',)), [])]))]),
+('Module', [('Expr', (1, 0), ('SetComp', (1, 1), ('Name', (1, 1), 'r', ('Load',)), [('comprehension', ('Name', (1, 7), 'l', ('Store',)), ('Name', (1, 12), 'x', ('Load',)), [('Name', (1, 17), 'g', ('Load',))])]))]),
+('Module', [('Expr', (1, 0), ('SetComp', (1, 1), ('Name', (1, 1), 'r', ('Load',)), [('comprehension', ('Tuple', (1, 7), [('Name', (1, 7), 'l', ('Store',)), ('Name', (1, 9), 'm', ('Store',))], ('Store',)), ('Name', (1, 14), 'x', ('Load',)), [])]))]),
+]
+single_results = [
+('Interactive', [('Expr', (1, 0), ('BinOp', (1, 0), ('Num', (1, 0), 1), ('Add',), ('Num', (1, 2), 2)))]),
+]
+eval_results = [
+('Expression', ('BoolOp', (1, 0), ('And',), [('Name', (1, 0), 'a', ('Load',)), ('Name', (1, 6), 'b', ('Load',))])),
+('Expression', ('BinOp', (1, 0), ('Name', (1, 0), 'a', ('Load',)), ('Add',), ('Name', (1, 4), 'b', ('Load',)))),
+('Expression', ('UnaryOp', (1, 0), ('Not',), ('Name', (1, 4), 'v', ('Load',)))),
+('Expression', ('Lambda', (1, 0), ('arguments', [], None, None, []), ('Name', (1, 7), 'None', ('Load',)))),
+('Expression', ('Dict', (1, 0), [('Num', (1, 2), 1)], [('Num', (1, 4), 2)])),
+('Expression', ('Dict', (1, 0), [], [])),
+('Expression', ('Set', (1, 0), [('Name', (1, 1), 'None', ('Load',))])),
+('Expression', ('Dict', (1, 0), [('Num', (2, 0), 1)], [('Num', (4, 2), 2)])),
+('Expression', ('ListComp', (1, 1), ('Name', (1, 1), 'a', ('Load',)), [('comprehension', ('Name', (1, 7), 'b', ('Store',)), ('Name', (1, 12), 'c', ('Load',)), [('Name', (1, 17), 'd', ('Load',))])])),
+('Expression', ('GeneratorExp', (1, 1), ('Name', (1, 1), 'a', ('Load',)), [('comprehension', ('Name', (1, 7), 'b', ('Store',)), ('Name', (1, 12), 'c', ('Load',)), [('Name', (1, 17), 'd', ('Load',))])])),
+('Expression', ('Compare', (1, 0), ('Num', (1, 0), 1), [('Lt',), ('Lt',)], [('Num', (1, 4), 2), ('Num', (1, 8), 3)])),
+('Expression', ('Call', (1, 0), ('Name', (1, 0), 'f', ('Load',)), [('Num', (1, 2), 1), ('Num', (1, 4), 2)], [('keyword', 'c', ('Num', (1, 8), 3))], ('Name', (1, 11), 'd', ('Load',)), ('Name', (1, 15), 'e', ('Load',)))),
+('Expression', ('Repr', (1, 0), ('Name', (1, 1), 'v', ('Load',)))),
+('Expression', ('Num', (1, 0), 10L)),
+('Expression', ('Str', (1, 0), 'string')),
+('Expression', ('Attribute', (1, 0), ('Name', (1, 0), 'a', ('Load',)), 'b', ('Load',))),
+('Expression', ('Subscript', (1, 0), ('Name', (1, 0), 'a', ('Load',)), ('Slice', ('Name', (1, 2), 'b', ('Load',)), ('Name', (1, 4), 'c', ('Load',)), None), ('Load',))),
+('Expression', ('Name', (1, 0), 'v', ('Load',))),
+('Expression', ('List', (1, 0), [('Num', (1, 1), 1), ('Num', (1, 3), 2), ('Num', (1, 5), 3)], ('Load',))),
+('Expression', ('List', (1, 0), [], ('Load',))),
+('Expression', ('Tuple', (1, 0), [('Num', (1, 0), 1), ('Num', (1, 2), 2), ('Num', (1, 4), 3)], ('Load',))),
+('Expression', ('Tuple', (1, 1), [('Num', (1, 1), 1), ('Num', (1, 3), 2), ('Num', (1, 5), 3)], ('Load',))),
+('Expression', ('Tuple', (1, 0), [], ('Load',))),
+('Expression', ('Call', (1, 0), ('Attribute', (1, 0), ('Attribute', (1, 0), ('Attribute', (1, 0), ('Name', (1, 0), 'a', ('Load',)), 'b', ('Load',)), 'c', ('Load',)), 'd', ('Load',)), [('Subscript', (1, 8), ('Attribute', (1, 8), ('Name', (1, 8), 'a', ('Load',)), 'b', ('Load',)), ('Slice', ('Num', (1, 12), 1), ('Num', (1, 14), 2), None), ('Load',))], [], None, None)),
+]
+main()

diff --git a/pypy/interpreter/astcompiler/astbuilder.py b/pypy/interpreter/astcompiler/astbuilder.py
--- a/pypy/interpreter/astcompiler/astbuilder.py
+++ b/pypy/interpreter/astcompiler/astbuilder.py
@@ -1130,13 +1130,13 @@
         elif first_child_type == tokens.LBRACE:
             maker = atom_node.children[1]
             if maker.type == tokens.RBRACE:
-                return ast.Dict(None, None, maker.lineno, maker.column)
+                return ast.Dict(None, None, atom_node.lineno, atom_node.column)
             n_maker_children = len(maker.children)
             if n_maker_children == 1 or maker.children[1].type == tokens.COMMA:
                 elts = []
                 for i in range(0, n_maker_children, 2):
                     elts.append(self.handle_expr(maker.children[i]))
-                return ast.Set(elts, maker.lineno, maker.column)
+                return ast.Set(elts, atom_node.lineno, atom_node.column)
             if maker.children[1].type == syms.comp_for:
                 return self.handle_setcomp(maker)
             if (n_maker_children > 3 and
@@ -1147,7 +1147,7 @@
             for i in range(0, n_maker_children, 4):
                 keys.append(self.handle_expr(maker.children[i]))
                 values.append(self.handle_expr(maker.children[i + 2]))
-            return ast.Dict(keys, values, maker.lineno, maker.column)
+            return ast.Dict(keys, values, atom_node.lineno, atom_node.column)
         elif first_child_type == tokens.BACKQUOTE:
             expr = self.handle_testlist(atom_node.children[1])
             return ast.Repr(expr, atom_node.lineno, atom_node.column)
@@ -1197,7 +1197,8 @@
     def comprehension_helper(self, comp_node,
                              handle_source_expr_meth="handle_expr",
                              for_type=syms.comp_for, if_type=syms.comp_if,
-                             iter_type=syms.comp_iter):
+                             iter_type=syms.comp_iter,
+                             comp_fix_unamed_tuple_location=False):
         handle_source_expression = getattr(self, handle_source_expr_meth)
         fors_count = self.count_comp_fors(comp_node, for_type, if_type)
         comps = []
@@ -1209,8 +1210,13 @@
             if len(for_node.children) == 1:
                 comp = ast.comprehension(for_targets[0], expr, None)
             else:
-                target = ast.Tuple(for_targets, ast.Store, comp_node.lineno,
-                                   comp_node.column)
+                col = comp_node.column
+                line = comp_node.lineno
+                # Modified in python2.7, see http://bugs.python.org/issue6704
+                if comp_fix_unamed_tuple_location:
+                    col = for_targets[0].col_offset
+                    line = for_targets[0].lineno
+                target = ast.Tuple(for_targets, ast.Store, line, col)
                 comp = ast.comprehension(target, expr, None)
             if len(comp_node.children) == 5:
                 comp_node = comp_iter = comp_node.children[4]
@@ -1232,7 +1238,8 @@
 
     def handle_genexp(self, genexp_node):
         elt = self.handle_expr(genexp_node.children[0])
-        comps = self.comprehension_helper(genexp_node.children[1])
+        comps = self.comprehension_helper(genexp_node.children[1],
+                                          comp_fix_unamed_tuple_location=True)
         return ast.GeneratorExp(elt, comps, genexp_node.lineno,
                                 genexp_node.column)
 
@@ -1241,19 +1248,22 @@
         comps = self.comprehension_helper(listcomp_node.children[1],
                                           "handle_testlist",
                                           syms.list_for, syms.list_if,
-                                          syms.list_iter)
+                                          syms.list_iter,
+                                          comp_fix_unamed_tuple_location=True)
         return ast.ListComp(elt, comps, listcomp_node.lineno,
                             listcomp_node.column)
 
     def handle_setcomp(self, set_maker):
         elt = self.handle_expr(set_maker.children[0])
-        comps = self.comprehension_helper(set_maker.children[1])
+        comps = self.comprehension_helper(set_maker.children[1],
+                                          comp_fix_unamed_tuple_location=True)
         return ast.SetComp(elt, comps, set_maker.lineno, set_maker.column)
 
     def handle_dictcomp(self, dict_maker):
         key = self.handle_expr(dict_maker.children[0])
         value = self.handle_expr(dict_maker.children[2])
-        comps = self.comprehension_helper(dict_maker.children[3])
+        comps = self.comprehension_helper(dict_maker.children[3],
+                                          comp_fix_unamed_tuple_location=True)
         return ast.DictComp(key, value, comps, dict_maker.lineno,
                             dict_maker.column)
 

From commits-noreply at bitbucket.org  Wed Feb 23 22:54:43 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Wed, 23 Feb 2011 22:54:43 +0100 (CET)
Subject: [pypy-svn] pypy default: Change the definition of "_immutable_ =
	True" to be	explicitly
Message-ID: <20110223215443.8FB38282BDE@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r42247:72ff127c6643
Date: 2011-02-23 22:52 +0100
http://bitbucket.org/pypy/pypy/changeset/72ff127c6643/

Log:	Change the definition of "_immutable_ = True" to be explicitly
	required on all subclasses when we put it on a class. Previously, we
	got this effect implicitly. However, it made an obscure bug that
	took me three days to locate: app-level subclasses of the immutable
	built-in types, like 'long', have their fields flagged as immutable
	too, including the 'storage' from the mapdict, which is nonsense.

diff --git a/pypy/objspace/std/unicodeobject.py b/pypy/objspace/std/unicodeobject.py
--- a/pypy/objspace/std/unicodeobject.py
+++ b/pypy/objspace/std/unicodeobject.py
@@ -21,7 +21,7 @@
 
 class W_UnicodeObject(W_Object):
     from pypy.objspace.std.unicodetype import unicode_typedef as typedef
-    _immutable_ = True
+    _immutable_fields_ = ['_value']
 
     def __init__(w_self, unistr):
         assert isinstance(unistr, unicode)

diff --git a/pypy/interpreter/function.py b/pypy/interpreter/function.py
--- a/pypy/interpreter/function.py
+++ b/pypy/interpreter/function.py
@@ -563,7 +563,7 @@
 
 class StaticMethod(Wrappable):
     """The staticmethod objects."""
-    _immutable_ = True
+    _immutable_fields_ = ['w_function']
 
     def __init__(self, w_function):
         self.w_function = w_function
@@ -577,7 +577,7 @@
 
 class ClassMethod(Wrappable):
     """The classmethod objects."""
-    _immutable_ = True
+    _immutable_fields_ = ['w_function']
 
     def __init__(self, w_function):
         self.w_function = w_function

diff --git a/pypy/objspace/std/intobject.py b/pypy/objspace/std/intobject.py
--- a/pypy/objspace/std/intobject.py
+++ b/pypy/objspace/std/intobject.py
@@ -17,9 +17,8 @@
 
 class W_IntObject(W_Object):
     __slots__ = 'intval'
+    _immutable_fields_ = ['intval']
 
-    _immutable_ = True
-    
     from pypy.objspace.std.inttype import int_typedef as typedef
     
     def __init__(w_self, intval):

diff --git a/pypy/objspace/std/complexobject.py b/pypy/objspace/std/complexobject.py
--- a/pypy/objspace/std/complexobject.py
+++ b/pypy/objspace/std/complexobject.py
@@ -14,7 +14,7 @@
     """This is a reimplementation of the CPython "PyComplexObject"
     """
     from pypy.objspace.std.complextype import complex_typedef as typedef
-    _immutable_ = True
+    _immutable_fields_ = ['realval', 'imagval']
 
     def __init__(w_self, realval=0.0, imgval=0.0):
         w_self.realval = float(realval)

diff --git a/pypy/objspace/std/smalllongobject.py b/pypy/objspace/std/smalllongobject.py
--- a/pypy/objspace/std/smalllongobject.py
+++ b/pypy/objspace/std/smalllongobject.py
@@ -19,7 +19,7 @@
 
 class W_SmallLongObject(W_Object):
     from pypy.objspace.std.longtype import long_typedef as typedef
-    _immutable_ = True
+    _immutable_fields_ = ['longlong']
 
     def __init__(w_self, value):
         assert isinstance(value, r_longlong)

diff --git a/pypy/objspace/std/ropeunicodeobject.py b/pypy/objspace/std/ropeunicodeobject.py
--- a/pypy/objspace/std/ropeunicodeobject.py
+++ b/pypy/objspace/std/ropeunicodeobject.py
@@ -78,7 +78,7 @@
 
 class W_RopeUnicodeObject(W_Object):
     from pypy.objspace.std.unicodetype import unicode_typedef as typedef
-    _immutable_ = True
+    _immutable_fields_ = ['_node']
 
     def __init__(w_self, node):
         w_self._node = node

diff --git a/pypy/objspace/std/longobject.py b/pypy/objspace/std/longobject.py
--- a/pypy/objspace/std/longobject.py
+++ b/pypy/objspace/std/longobject.py
@@ -11,7 +11,7 @@
 class W_LongObject(W_Object):
     """This is a wrapper of rbigint."""
     from pypy.objspace.std.longtype import long_typedef as typedef
-    _immutable_ = True
+    _immutable_fields_ = ['num']
 
     def __init__(w_self, l):
         w_self.num = l # instance of rbigint

diff --git a/pypy/rpython/rclass.py b/pypy/rpython/rclass.py
--- a/pypy/rpython/rclass.py
+++ b/pypy/rpython/rclass.py
@@ -158,7 +158,13 @@
         pass
 
     def _check_for_immutable_hints(self, hints):
-        if self.classdef.classdesc.lookup('_immutable_') is not None:
+        loc = self.classdef.classdesc.lookup('_immutable_')
+        if loc is not None:
+            if loc is not self.classdef.classdesc:
+                raise ImmutableConflictError(
+                    "class %r inherits from its parent _immutable_=True, "
+                    "so it should also declare _immutable_=True" % (
+                    self.classdef,))
             hints = hints.copy()
             hints['immutable'] = True
         self.immutable_field_list = []  # unless overwritten below

diff --git a/pypy/objspace/std/ropeobject.py b/pypy/objspace/std/ropeobject.py
--- a/pypy/objspace/std/ropeobject.py
+++ b/pypy/objspace/std/ropeobject.py
@@ -21,7 +21,7 @@
 
 class W_RopeObject(W_Object):
     from pypy.objspace.std.stringtype import str_typedef as typedef
-    _immutable_ = True
+    _immutable_fields_ = ['_node']
 
     def __init__(w_self, node):
         if not we_are_translated():

diff --git a/pypy/objspace/std/stringobject.py b/pypy/objspace/std/stringobject.py
--- a/pypy/objspace/std/stringobject.py
+++ b/pypy/objspace/std/stringobject.py
@@ -21,7 +21,7 @@
 
 class W_StringObject(W_Object):
     from pypy.objspace.std.stringtype import str_typedef as typedef
-    _immutable_ = True
+    _immutable_fields_ = ['_value']
 
     def __init__(w_self, str):
         w_self._value = str

diff --git a/pypy/rpython/test/test_rclass.py b/pypy/rpython/test/test_rclass.py
--- a/pypy/rpython/test/test_rclass.py
+++ b/pypy/rpython/test/test_rclass.py
@@ -853,17 +853,17 @@
                accessor.fields == {"ov" : ""} # for ootype
 
     def test_immutable_subclass_1(self):
+        from pypy.rpython.rclass import ImmutableConflictError
         from pypy.jit.metainterp.typesystem import deref
         class A(object):
             _immutable_ = True
         class B(A):
             pass
         def f():
+            A()
             B().v = 123
             return B()
-        t, typer, graph = self.gengraph(f, [])
-        B_TYPE = deref(graph.getreturnvar().concretetype)
-        assert B_TYPE._hints["immutable"]    # inherited from A
+        py.test.raises(ImmutableConflictError, self.gengraph, f, [])
 
     def test_immutable_subclass_2(self):
         from pypy.jit.metainterp.typesystem import deref
@@ -872,6 +872,7 @@
         class B(A):
             _immutable_ = True
         def f():
+            A()
             B().v = 123
             return B()
         t, typer, graph = self.gengraph(f, [])

diff --git a/pypy/objspace/std/floatobject.py b/pypy/objspace/std/floatobject.py
--- a/pypy/objspace/std/floatobject.py
+++ b/pypy/objspace/std/floatobject.py
@@ -25,7 +25,7 @@
        it is assumed that the constructor takes a real Python float as
        an argument"""
     from pypy.objspace.std.floattype import float_typedef as typedef
-    _immutable_ = True
+    _immutable_fields_ = ['floatval']
 
     def __init__(w_self, floatval):
         w_self.floatval = floatval

diff --git a/pypy/objspace/std/sliceobject.py b/pypy/objspace/std/sliceobject.py
--- a/pypy/objspace/std/sliceobject.py
+++ b/pypy/objspace/std/sliceobject.py
@@ -8,7 +8,7 @@
 
 class W_SliceObject(W_Object):
     from pypy.objspace.std.slicetype import slice_typedef as typedef
-    _immutable_ = True
+    _immutable_fields_ = ['w_start', 'w_stop', 'w_step']
 
     def __init__(w_self, w_start, w_stop, w_step):
         assert w_start is not None

From commits-noreply at bitbucket.org  Wed Feb 23 23:36:41 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Wed, 23 Feb 2011 23:36:41 +0100 (CET)
Subject: [pypy-svn] pypy default: be sure that we don't allow alpha-renaming
	for	ConstClass(...) in match()
Message-ID: <20110223223641.8CBDB2A2033@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42248:8b7fa8e31c35
Date: 2011-02-23 23:20 +0100
http://bitbucket.org/pypy/pypy/changeset/8b7fa8e31c35/

Log:	be sure that we don't allow alpha-renaming for ConstClass(...) in
	match()

diff --git a/pypy/module/pypyjit/test_pypy_c/model.py b/pypy/module/pypyjit/test_pypy_c/model.py
--- a/pypy/module/pypyjit/test_pypy_c/model.py
+++ b/pypy/module/pypyjit/test_pypy_c/model.py
@@ -133,6 +133,13 @@
             for op in self._ops_for_chunk(chunk, include_debug_merge_points):
                 yield op
 
+    def print_ops(self, id=None):
+        if id is None:
+            ops = self.allops()
+        else:
+            ops = self.ops_by_id(id)
+        print '\n'.join(map(str, ops))
+
     def ops_by_id(self, id, include_debug_merge_points=False, opcode=None):
         opcode_name = opcode
         target_opcodes = self.ids[id]
@@ -186,13 +193,22 @@
         src = src.replace('--TICK--', ticker_check)
         return src
 
+    @classmethod
+    def _get_match_var(cls):
+        def is_const(v1):
+            return isinstance(v1, str) and v1.startswith('ConstClass(')
+        alpha_map = {}
+        def match_var(v1, v2):
+            if is_const(v1) or is_const(v2):
+                return v1 == v2
+            if v1 not in alpha_map:
+                alpha_map[v1] = v2
+            return alpha_map[v1] == v2
+        return match_var
+
     def match_ops(self, ops, expected_src):
         expected_src = self.preprocess_expected_src(expected_src)
-        alpha_map = {}
-        def match_var(v1, v2):
-            if v1 not in alpha_map:
-                alpha_map[v1] = v2
-            assert alpha_map[v1] == v2, "variable mismatch"
+        match_var = self._get_match_var()
         #
         expected_ops = self.parse_ops(expected_src)
         assert len(ops) == len(expected_ops), "wrong number of operations"
@@ -201,7 +217,7 @@
             match_var(op.res, exp_res)
             assert len(op.args) == len(exp_args), "wrong number of arguments"
             for arg, exp_arg in zip(op.args, exp_args):
-                match_var(arg, exp_arg)
+                assert match_var(arg, exp_arg), "variable mismatch"
         return True
 
     def match(self, expected_src):

diff --git a/pypy/module/pypyjit/test_pypy_c/test_model.py b/pypy/module/pypyjit/test_pypy_c/test_model.py
--- a/pypy/module/pypyjit/test_pypy_c/test_model.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_model.py
@@ -76,6 +76,22 @@
         opcodes_names = [opcode.__class__.__name__ for opcode in myline]
         assert opcodes_names == ['LOAD_FAST', 'LOAD_CONST', 'BINARY_ADD', 'STORE_FAST']
 
+    def test_match_var(self):
+        match_var = LoopWithIds._get_match_var()
+        assert match_var('v0', 'V0')
+        assert not match_var('v0', 'V1')
+        assert match_var('v0', 'V0')
+        #
+        # for ConstPtr, we allow the same alpha-renaming as for variables
+        assert match_var('ConstPtr(ptr0)', 'PTR0')
+        assert not match_var('ConstPtr(ptr0)', 'PTR1')
+        assert match_var('ConstPtr(ptr0)', 'PTR0')
+        #
+        # for ConstClass, we want the exact matching
+        assert match_var('ConstClass(foo)', 'ConstClass(foo)')
+        assert not match_var('ConstClass(bar)', 'v1')
+        assert not match_var('v2', 'ConstClass(baz)')
+
 class TestRunPyPyC(BaseTestPyPyC):
 
     def test_run_function(self):
@@ -249,3 +265,23 @@
             i4 = int_sub_ovf(i3, 1)
             guard_no_overflow()
         """)
+
+    def test_match_constants(self):
+        def f():
+            i = 0L # force it to long, so that we get calls to rbigint
+            while i < 1003:
+                i += 1L # ID: increment
+                a = 0
+            return i
+        log = self.run(f)
+        loop, = log.loops_by_id('increment')
+        assert loop.match_by_id('increment', """
+            p12 = call(ConstClass(rbigint.add), p4, ConstPtr(ptr11))
+            guard_no_exception()
+        """)
+        #
+        py.test.raises(AssertionError, loop.match_by_id, 'increment', """
+            p12 = call(ConstClass(rbigint.SUB), p4, ConstPtr(ptr11))
+            guard_no_exception()
+        """)
+        

From commits-noreply at bitbucket.org  Wed Feb 23 23:36:42 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Wed, 23 Feb 2011 23:36:42 +0100 (CET)
Subject: [pypy-svn] pypy default: add the testcase for longs
Message-ID: <20110223223642.364D72A2033@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42249:2ae5d1fc0a1d
Date: 2011-02-23 23:24 +0100
http://bitbucket.org/pypy/pypy/changeset/2ae5d1fc0a1d/

Log:	add the testcase for longs

diff --git a/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py b/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py
--- a/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py
@@ -54,7 +54,21 @@
             --TICK--
             jump(p0, p1, p2, p3, i10, i8)
         """)
-
+        #
+        log = self.run(fact, [25], threshold=20)
+        assert log.result == 15511210043330985984000000L
+        loop, = log.loops_by_filename(self.filepath)
+        assert loop.match("""
+            i7 = int_gt(i4, 1)
+            guard_true(i7)
+            p9 = call(ConstClass(fromint), i4)
+            guard_no_exception()
+            p11 = call(ConstClass(rbigint.mul), p5, p9)
+            guard_no_exception()
+            i13 = int_sub(i4, 1)
+            --TICK--
+            jump(p0, p1, p2, p3, i13, p11)
+        """)
 
     def test_cmp_exc(self):
         def f1(n):

From commits-noreply at bitbucket.org  Thu Feb 24 00:06:50 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Thu, 24 Feb 2011 00:06:50 +0100 (CET)
Subject: [pypy-svn] pypy default: Revert 8759aa9651bd. Breaks translation.
Message-ID: <20110223230650.2C237282BDE@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r42250:745e67a1466d
Date: 2011-02-23 23:06 +0000
http://bitbucket.org/pypy/pypy/changeset/745e67a1466d/

Log:	Revert 8759aa9651bd. Breaks translation.

diff --git a/lib-python/modified-2.7.0/test/test_ast.py b/lib-python/modified-2.7.0/test/test_ast.py
deleted file mode 100644
--- a/lib-python/modified-2.7.0/test/test_ast.py
+++ /dev/null
@@ -1,423 +0,0 @@
-import sys, itertools, unittest
-from test import test_support
-import ast
-
-def to_tuple(t):
-    if t is None or isinstance(t, (basestring, int, long, complex)):
-        return t
-    elif isinstance(t, list):
-        return [to_tuple(e) for e in t]
-    result = [t.__class__.__name__]
-    if hasattr(t, 'lineno') and hasattr(t, 'col_offset'):
-        result.append((t.lineno, t.col_offset))
-    if t._fields is None:
-        return tuple(result)
-    for f in t._fields:
-        result.append(to_tuple(getattr(t, f)))
-    return tuple(result)
-
-
-# These tests are compiled through "exec"
-# There should be atleast one test per statement
-exec_tests = [
-    # FunctionDef
-    "def f(): pass",
-    # ClassDef
-    "class C:pass",
-    # Return
-    "def f():return 1",
-    # Delete
-    "del v",
-    # Assign
-    "v = 1",
-    # AugAssign
-    "v += 1",
-    # Print
-    "print >>f, 1, ",
-    # For
-    "for v in v:pass",
-    # While
-    "while v:pass",
-    # If
-    "if v:pass",
-    # Raise
-    "raise Exception, 'string'",
-    # TryExcept
-    "try:\n  pass\nexcept Exception:\n  pass",
-    # TryFinally
-    "try:\n  pass\nfinally:\n  pass",
-    # Assert
-    "assert v",
-    # Import
-    "import sys",
-    # ImportFrom
-    "from sys import v",
-    # Exec
-    "exec 'v'",
-    # Global
-    "global v",
-    # Expr
-    "1",
-    # Pass,
-    "pass",
-    # Break
-    "break",
-    # Continue
-    "continue",
-    # for statements with naked tuples (see http://bugs.python.org/issue6704)
-    "for a,b in c: pass",
-    "[(a,b) for a,b in c]",
-    "((a,b) for a,b in c)",
-    "((a,b) for (a,b) in c)",
-    # Multiline generator expression
-    """(
-    (
-    Aa
-    ,
-    Bb
-    )
-    for
-    Aa
-    ,
-    Bb in Cc
-    )""",
-    # dictcomp
-    "{a : b for w in x for m in p if g}",
-    # dictcomp
-    "{a : b for v,w in x}",
-    # setcomp
-    "{r for l in x if g}",
-    # setcomp
-    "{r for l,m in x}",
-]
-
-# These are compiled through "single"
-# because of overlap with "eval", it just tests what
-# can't be tested with "eval"
-single_tests = [
-    "1+2"
-]
-
-# These are compiled through "eval"
-# It should test all expressions
-eval_tests = [
-  # BoolOp
-  "a and b",
-  # BinOp
-  "a + b",
-  # UnaryOp
-  "not v",
-  # Lambda
-  "lambda:None",
-  # Dict
-  "{ 1:2 }",
-  # Empty dict
-  "{}",
-  # Set
-  "{None,}",
-  # Multiline dict
-  """{
-1
- :
-  2
-    }""",
-  # ListComp
-  "[a for b in c if d]",
-  # GeneratorExp
-  "(a for b in c if d)",
-  # Yield - yield expressions can't work outside a function
-  #
-  # Compare
-  "1 < 2 < 3",
-  # Call
-  "f(1,2,c=3,*d,**e)",
-  # Repr
-  "`v`",
-  # Num
-  "10L",
-  # Str
-  "'string'",
-  # Attribute
-  "a.b",
-  # Subscript
-  "a[b:c]",
-  # Name
-  "v",
-  # List
-  "[1,2,3]",
-  # Empty list
-  "[]",
-  # Tuple
-  "1,2,3",
-  # Tuple
-  "(1,2,3)",
-  # Empty tuple
-  "()",
-  # Combination
-  "a.b.c.d(a.b[1:2])",
-
-]
-
-# TODO: expr_context, slice, boolop, operator, unaryop, cmpop, comprehension
-# excepthandler, arguments, keywords, alias
-
-class AST_Tests(unittest.TestCase):
-
-    def _assertTrueorder(self, ast_node, parent_pos):
-        if not isinstance(ast_node, ast.AST) or ast_node._fields is None:
-            return
-        if isinstance(ast_node, (ast.expr, ast.stmt, ast.excepthandler)):
-            node_pos = (ast_node.lineno, ast_node.col_offset)
-            self.assertTrue(node_pos >= parent_pos)
-            parent_pos = (ast_node.lineno, ast_node.col_offset)
-        for name in ast_node._fields:
-            value = getattr(ast_node, name)
-            if isinstance(value, list):
-                for child in value:
-                    self._assertTrueorder(child, parent_pos)
-            elif value is not None:
-                self._assertTrueorder(value, parent_pos)
-
-    def test_snippets(self):
-        for input, output, kind in ((exec_tests, exec_results, "exec"),
-                                    (single_tests, single_results, "single"),
-                                    (eval_tests, eval_results, "eval")):
-            for i, o in itertools.izip(input, output):
-                ast_tree = compile(i, "?", kind, ast.PyCF_ONLY_AST)
-                self.assertEquals(to_tuple(ast_tree), o)
-                self._assertTrueorder(ast_tree, (0, 0))
-
-    def test_slice(self):
-        slc = ast.parse("x[::]").body[0].value.slice
-        self.assertIsNone(slc.upper)
-        self.assertIsNone(slc.lower)
-        self.assertIsInstance(slc.step, ast.Name)
-        self.assertEqual(slc.step.id, "None")
-
-    def test_from_import(self):
-        im = ast.parse("from . import y").body[0]
-        self.assertIsNone(im.module)
-
-    def test_base_classes(self):
-        self.assertTrue(issubclass(ast.For, ast.stmt))
-        self.assertTrue(issubclass(ast.Name, ast.expr))
-        self.assertTrue(issubclass(ast.stmt, ast.AST))
-        self.assertTrue(issubclass(ast.expr, ast.AST))
-        self.assertTrue(issubclass(ast.comprehension, ast.AST))
-        self.assertTrue(issubclass(ast.Gt, ast.AST))
-
-    def test_nodeclasses(self):
-        x = ast.BinOp(1, 2, 3, lineno=0)
-        self.assertEquals(x.left, 1)
-        self.assertEquals(x.op, 2)
-        self.assertEquals(x.right, 3)
-        self.assertEquals(x.lineno, 0)
-
-        # node raises exception when not given enough arguments
-        self.assertRaises(TypeError, ast.BinOp, 1, 2)
-
-        # can set attributes through kwargs too
-        x = ast.BinOp(left=1, op=2, right=3, lineno=0)
-        self.assertEquals(x.left, 1)
-        self.assertEquals(x.op, 2)
-        self.assertEquals(x.right, 3)
-        self.assertEquals(x.lineno, 0)
-
-        # this used to fail because Sub._fields was None
-        x = ast.Sub()
-
-    def test_pickling(self):
-        import pickle
-        mods = [pickle]
-        try:
-            import cPickle
-            mods.append(cPickle)
-        except ImportError:
-            pass
-        protocols = [0, 1, 2]
-        for mod in mods:
-            for protocol in protocols:
-                for ast in (compile(i, "?", "exec", 0x400) for i in exec_tests):
-                    ast2 = mod.loads(mod.dumps(ast, protocol))
-                    self.assertEquals(to_tuple(ast2), to_tuple(ast))
-
-
-class ASTHelpers_Test(unittest.TestCase):
-
-    def test_parse(self):
-        a = ast.parse('foo(1 + 1)')
-        b = compile('foo(1 + 1)', '<unknown>', 'exec', ast.PyCF_ONLY_AST)
-        self.assertEqual(ast.dump(a), ast.dump(b))
-
-    def test_dump(self):
-        node = ast.parse('spam(eggs, "and cheese")')
-        self.assertEqual(ast.dump(node),
-            "Module(body=[Expr(value=Call(func=Name(id='spam', ctx=Load()), "
-            "args=[Name(id='eggs', ctx=Load()), Str(s='and cheese')], "
-            "keywords=[], starargs=None, kwargs=None))])"
-        )
-        self.assertEqual(ast.dump(node, annotate_fields=False),
-            "Module([Expr(Call(Name('spam', Load()), [Name('eggs', Load()), "
-            "Str('and cheese')], [], None, None))])"
-        )
-        self.assertEqual(ast.dump(node, include_attributes=True),
-            "Module(body=[Expr(value=Call(func=Name(id='spam', ctx=Load(), "
-            "lineno=1, col_offset=0), args=[Name(id='eggs', ctx=Load(), "
-            "lineno=1, col_offset=5), Str(s='and cheese', lineno=1, "
-            "col_offset=11)], keywords=[], starargs=None, kwargs=None, "
-            "lineno=1, col_offset=0), lineno=1, col_offset=0)])"
-        )
-
-    def test_copy_location(self):
-        src = ast.parse('1 + 1', mode='eval')
-        src.body.right = ast.copy_location(ast.Num(2), src.body.right)
-        self.assertEqual(ast.dump(src, include_attributes=True),
-            'Expression(body=BinOp(left=Num(n=1, lineno=1, col_offset=0), '
-            'op=Add(), right=Num(n=2, lineno=1, col_offset=4), lineno=1, '
-            'col_offset=0))'
-        )
-
-    def test_fix_missing_locations(self):
-        src = ast.parse('write("spam")')
-        src.body.append(ast.Expr(ast.Call(ast.Name('spam', ast.Load()),
-                                          [ast.Str('eggs')], [], None, None)))
-        self.assertEqual(src, ast.fix_missing_locations(src))
-        self.assertEqual(ast.dump(src, include_attributes=True),
-            "Module(body=[Expr(value=Call(func=Name(id='write', ctx=Load(), "
-            "lineno=1, col_offset=0), args=[Str(s='spam', lineno=1, "
-            "col_offset=6)], keywords=[], starargs=None, kwargs=None, "
-            "lineno=1, col_offset=0), lineno=1, col_offset=0), "
-            "Expr(value=Call(func=Name(id='spam', ctx=Load(), lineno=1, "
-            "col_offset=0), args=[Str(s='eggs', lineno=1, col_offset=0)], "
-            "keywords=[], starargs=None, kwargs=None, lineno=1, "
-            "col_offset=0), lineno=1, col_offset=0)])"
-        )
-
-    def test_increment_lineno(self):
-        src = ast.parse('1 + 1', mode='eval')
-        self.assertEqual(ast.increment_lineno(src, n=3), src)
-        self.assertEqual(ast.dump(src, include_attributes=True),
-            'Expression(body=BinOp(left=Num(n=1, lineno=4, col_offset=0), '
-            'op=Add(), right=Num(n=1, lineno=4, col_offset=4), lineno=4, '
-            'col_offset=0))'
-        )
-
-    def test_iter_fields(self):
-        node = ast.parse('foo()', mode='eval')
-        d = dict(ast.iter_fields(node.body))
-        self.assertEqual(d.pop('func').id, 'foo')
-        self.assertEqual(d, {'keywords': [], 'kwargs': None,
-                             'args': [], 'starargs': None})
-
-    def test_iter_child_nodes(self):
-        node = ast.parse("spam(23, 42, eggs='leek')", mode='eval')
-        self.assertEqual(len(list(ast.iter_child_nodes(node.body))), 4)
-        iterator = ast.iter_child_nodes(node.body)
-        self.assertEqual(next(iterator).id, 'spam')
-        self.assertEqual(next(iterator).n, 23)
-        self.assertEqual(next(iterator).n, 42)
-        self.assertEqual(ast.dump(next(iterator)),
-            "keyword(arg='eggs', value=Str(s='leek'))"
-        )
-
-    def __test_get_docstring(self):
-        node = ast.parse('def foo():\n  """line one\n  line two"""')
-        self.assertEqual(ast.get_docstring(node.body[0]),
-                         'line one\nline two')
-
-    def test_literal_eval(self):
-        self.assertEqual(ast.literal_eval('[1, 2, 3]'), [1, 2, 3])
-        self.assertEqual(ast.literal_eval('{"foo": 42}'), {"foo": 42})
-        self.assertEqual(ast.literal_eval('(True, False, None)'), (True, False, None))
-        self.assertRaises(ValueError, ast.literal_eval, 'foo()')
-
-    def test_literal_eval_issue4907(self):
-        self.assertEqual(ast.literal_eval('2j'), 2j)
-        self.assertEqual(ast.literal_eval('10 + 2j'), 10 + 2j)
-        self.assertEqual(ast.literal_eval('1.5 - 2j'), 1.5 - 2j)
-        self.assertRaises(ValueError, ast.literal_eval, '2 + (3 + 4j)')
-
-
-def test_main():
-    with test_support.check_py3k_warnings(("backquote not supported",
-                                             SyntaxWarning)):
-        test_support.run_unittest(AST_Tests, ASTHelpers_Test)
-
-def main():
-    if __name__ != '__main__':
-        return
-    if sys.argv[1:] == ['-g']:
-        for statements, kind in ((exec_tests, "exec"), (single_tests, "single"),
-                                 (eval_tests, "eval")):
-            print kind+"_results = ["
-            for s in statements:
-                print repr(to_tuple(compile(s, "?", kind, 0x400)))+","
-            print "]"
-        print "main()"
-        raise SystemExit
-    test_main()
-
-#### EVERYTHING BELOW IS GENERATED #####
-exec_results = [
-('Module', [('FunctionDef', (1, 0), 'f', ('arguments', [], None, None, []), [('Pass', (1, 9))], [])]),
-('Module', [('ClassDef', (1, 0), 'C', [], [('Pass', (1, 8))], [])]),
-('Module', [('FunctionDef', (1, 0), 'f', ('arguments', [], None, None, []), [('Return', (1, 8), ('Num', (1, 15), 1))], [])]),
-('Module', [('Delete', (1, 0), [('Name', (1, 4), 'v', ('Del',))])]),
-('Module', [('Assign', (1, 0), [('Name', (1, 0), 'v', ('Store',))], ('Num', (1, 4), 1))]),
-('Module', [('AugAssign', (1, 0), ('Name', (1, 0), 'v', ('Store',)), ('Add',), ('Num', (1, 5), 1))]),
-('Module', [('Print', (1, 0), ('Name', (1, 8), 'f', ('Load',)), [('Num', (1, 11), 1)], False)]),
-('Module', [('For', (1, 0), ('Name', (1, 4), 'v', ('Store',)), ('Name', (1, 9), 'v', ('Load',)), [('Pass', (1, 11))], [])]),
-('Module', [('While', (1, 0), ('Name', (1, 6), 'v', ('Load',)), [('Pass', (1, 8))], [])]),
-('Module', [('If', (1, 0), ('Name', (1, 3), 'v', ('Load',)), [('Pass', (1, 5))], [])]),
-('Module', [('Raise', (1, 0), ('Name', (1, 6), 'Exception', ('Load',)), ('Str', (1, 17), 'string'), None)]),
-('Module', [('TryExcept', (1, 0), [('Pass', (2, 2))], [('ExceptHandler', (3, 0), ('Name', (3, 7), 'Exception', ('Load',)), None, [('Pass', (4, 2))])], [])]),
-('Module', [('TryFinally', (1, 0), [('Pass', (2, 2))], [('Pass', (4, 2))])]),
-('Module', [('Assert', (1, 0), ('Name', (1, 7), 'v', ('Load',)), None)]),
-('Module', [('Import', (1, 0), [('alias', 'sys', None)])]),
-('Module', [('ImportFrom', (1, 0), 'sys', [('alias', 'v', None)], 0)]),
-('Module', [('Exec', (1, 0), ('Str', (1, 5), 'v'), None, None)]),
-('Module', [('Global', (1, 0), ['v'])]),
-('Module', [('Expr', (1, 0), ('Num', (1, 0), 1))]),
-('Module', [('Pass', (1, 0))]),
-('Module', [('Break', (1, 0))]),
-('Module', [('Continue', (1, 0))]),
-('Module', [('For', (1, 0), ('Tuple', (1, 4), [('Name', (1, 4), 'a', ('Store',)), ('Name', (1, 6), 'b', ('Store',))], ('Store',)), ('Name', (1, 11), 'c', ('Load',)), [('Pass', (1, 14))], [])]),
-('Module', [('Expr', (1, 0), ('ListComp', (1, 1), ('Tuple', (1, 2), [('Name', (1, 2), 'a', ('Load',)), ('Name', (1, 4), 'b', ('Load',))], ('Load',)), [('comprehension', ('Tuple', (1, 11), [('Name', (1, 11), 'a', ('Store',)), ('Name', (1, 13), 'b', ('Store',))], ('Store',)), ('Name', (1, 18), 'c', ('Load',)), [])]))]),
-('Module', [('Expr', (1, 0), ('GeneratorExp', (1, 1), ('Tuple', (1, 2), [('Name', (1, 2), 'a', ('Load',)), ('Name', (1, 4), 'b', ('Load',))], ('Load',)), [('comprehension', ('Tuple', (1, 11), [('Name', (1, 11), 'a', ('Store',)), ('Name', (1, 13), 'b', ('Store',))], ('Store',)), ('Name', (1, 18), 'c', ('Load',)), [])]))]),
-('Module', [('Expr', (1, 0), ('GeneratorExp', (1, 1), ('Tuple', (1, 2), [('Name', (1, 2), 'a', ('Load',)), ('Name', (1, 4), 'b', ('Load',))], ('Load',)), [('comprehension', ('Tuple', (1, 12), [('Name', (1, 12), 'a', ('Store',)), ('Name', (1, 14), 'b', ('Store',))], ('Store',)), ('Name', (1, 20), 'c', ('Load',)), [])]))]),
-('Module', [('Expr', (1, 0), ('GeneratorExp', (2, 4), ('Tuple', (3, 4), [('Name', (3, 4), 'Aa', ('Load',)), ('Name', (5, 4), 'Bb', ('Load',))], ('Load',)), [('comprehension', ('Tuple', (8, 4), [('Name', (8, 4), 'Aa', ('Store',)), ('Name', (10, 4), 'Bb', ('Store',))], ('Store',)), ('Name', (10, 10), 'Cc', ('Load',)), [])]))]),
-('Module', [('Expr', (1, 0), ('DictComp', (1, 1), ('Name', (1, 1), 'a', ('Load',)), ('Name', (1, 5), 'b', ('Load',)), [('comprehension', ('Name', (1, 11), 'w', ('Store',)), ('Name', (1, 16), 'x', ('Load',)), []), ('comprehension', ('Name', (1, 22), 'm', ('Store',)), ('Name', (1, 27), 'p', ('Load',)), [('Name', (1, 32), 'g', ('Load',))])]))]),
-('Module', [('Expr', (1, 0), ('DictComp', (1, 1), ('Name', (1, 1), 'a', ('Load',)), ('Name', (1, 5), 'b', ('Load',)), [('comprehension', ('Tuple', (1, 11), [('Name', (1, 11), 'v', ('Store',)), ('Name', (1, 13), 'w', ('Store',))], ('Store',)), ('Name', (1, 18), 'x', ('Load',)), [])]))]),
-('Module', [('Expr', (1, 0), ('SetComp', (1, 1), ('Name', (1, 1), 'r', ('Load',)), [('comprehension', ('Name', (1, 7), 'l', ('Store',)), ('Name', (1, 12), 'x', ('Load',)), [('Name', (1, 17), 'g', ('Load',))])]))]),
-('Module', [('Expr', (1, 0), ('SetComp', (1, 1), ('Name', (1, 1), 'r', ('Load',)), [('comprehension', ('Tuple', (1, 7), [('Name', (1, 7), 'l', ('Store',)), ('Name', (1, 9), 'm', ('Store',))], ('Store',)), ('Name', (1, 14), 'x', ('Load',)), [])]))]),
-]
-single_results = [
-('Interactive', [('Expr', (1, 0), ('BinOp', (1, 0), ('Num', (1, 0), 1), ('Add',), ('Num', (1, 2), 2)))]),
-]
-eval_results = [
-('Expression', ('BoolOp', (1, 0), ('And',), [('Name', (1, 0), 'a', ('Load',)), ('Name', (1, 6), 'b', ('Load',))])),
-('Expression', ('BinOp', (1, 0), ('Name', (1, 0), 'a', ('Load',)), ('Add',), ('Name', (1, 4), 'b', ('Load',)))),
-('Expression', ('UnaryOp', (1, 0), ('Not',), ('Name', (1, 4), 'v', ('Load',)))),
-('Expression', ('Lambda', (1, 0), ('arguments', [], None, None, []), ('Name', (1, 7), 'None', ('Load',)))),
-('Expression', ('Dict', (1, 0), [('Num', (1, 2), 1)], [('Num', (1, 4), 2)])),
-('Expression', ('Dict', (1, 0), [], [])),
-('Expression', ('Set', (1, 0), [('Name', (1, 1), 'None', ('Load',))])),
-('Expression', ('Dict', (1, 0), [('Num', (2, 0), 1)], [('Num', (4, 2), 2)])),
-('Expression', ('ListComp', (1, 1), ('Name', (1, 1), 'a', ('Load',)), [('comprehension', ('Name', (1, 7), 'b', ('Store',)), ('Name', (1, 12), 'c', ('Load',)), [('Name', (1, 17), 'd', ('Load',))])])),
-('Expression', ('GeneratorExp', (1, 1), ('Name', (1, 1), 'a', ('Load',)), [('comprehension', ('Name', (1, 7), 'b', ('Store',)), ('Name', (1, 12), 'c', ('Load',)), [('Name', (1, 17), 'd', ('Load',))])])),
-('Expression', ('Compare', (1, 0), ('Num', (1, 0), 1), [('Lt',), ('Lt',)], [('Num', (1, 4), 2), ('Num', (1, 8), 3)])),
-('Expression', ('Call', (1, 0), ('Name', (1, 0), 'f', ('Load',)), [('Num', (1, 2), 1), ('Num', (1, 4), 2)], [('keyword', 'c', ('Num', (1, 8), 3))], ('Name', (1, 11), 'd', ('Load',)), ('Name', (1, 15), 'e', ('Load',)))),
-('Expression', ('Repr', (1, 0), ('Name', (1, 1), 'v', ('Load',)))),
-('Expression', ('Num', (1, 0), 10L)),
-('Expression', ('Str', (1, 0), 'string')),
-('Expression', ('Attribute', (1, 0), ('Name', (1, 0), 'a', ('Load',)), 'b', ('Load',))),
-('Expression', ('Subscript', (1, 0), ('Name', (1, 0), 'a', ('Load',)), ('Slice', ('Name', (1, 2), 'b', ('Load',)), ('Name', (1, 4), 'c', ('Load',)), None), ('Load',))),
-('Expression', ('Name', (1, 0), 'v', ('Load',))),
-('Expression', ('List', (1, 0), [('Num', (1, 1), 1), ('Num', (1, 3), 2), ('Num', (1, 5), 3)], ('Load',))),
-('Expression', ('List', (1, 0), [], ('Load',))),
-('Expression', ('Tuple', (1, 0), [('Num', (1, 0), 1), ('Num', (1, 2), 2), ('Num', (1, 4), 3)], ('Load',))),
-('Expression', ('Tuple', (1, 1), [('Num', (1, 1), 1), ('Num', (1, 3), 2), ('Num', (1, 5), 3)], ('Load',))),
-('Expression', ('Tuple', (1, 0), [], ('Load',))),
-('Expression', ('Call', (1, 0), ('Attribute', (1, 0), ('Attribute', (1, 0), ('Attribute', (1, 0), ('Name', (1, 0), 'a', ('Load',)), 'b', ('Load',)), 'c', ('Load',)), 'd', ('Load',)), [('Subscript', (1, 8), ('Attribute', (1, 8), ('Name', (1, 8), 'a', ('Load',)), 'b', ('Load',)), ('Slice', ('Num', (1, 12), 1), ('Num', (1, 14), 2), None), ('Load',))], [], None, None)),
-]
-main()

diff --git a/pypy/interpreter/astcompiler/astbuilder.py b/pypy/interpreter/astcompiler/astbuilder.py
--- a/pypy/interpreter/astcompiler/astbuilder.py
+++ b/pypy/interpreter/astcompiler/astbuilder.py
@@ -1130,13 +1130,13 @@
         elif first_child_type == tokens.LBRACE:
             maker = atom_node.children[1]
             if maker.type == tokens.RBRACE:
-                return ast.Dict(None, None, atom_node.lineno, atom_node.column)
+                return ast.Dict(None, None, maker.lineno, maker.column)
             n_maker_children = len(maker.children)
             if n_maker_children == 1 or maker.children[1].type == tokens.COMMA:
                 elts = []
                 for i in range(0, n_maker_children, 2):
                     elts.append(self.handle_expr(maker.children[i]))
-                return ast.Set(elts, atom_node.lineno, atom_node.column)
+                return ast.Set(elts, maker.lineno, maker.column)
             if maker.children[1].type == syms.comp_for:
                 return self.handle_setcomp(maker)
             if (n_maker_children > 3 and
@@ -1147,7 +1147,7 @@
             for i in range(0, n_maker_children, 4):
                 keys.append(self.handle_expr(maker.children[i]))
                 values.append(self.handle_expr(maker.children[i + 2]))
-            return ast.Dict(keys, values, atom_node.lineno, atom_node.column)
+            return ast.Dict(keys, values, maker.lineno, maker.column)
         elif first_child_type == tokens.BACKQUOTE:
             expr = self.handle_testlist(atom_node.children[1])
             return ast.Repr(expr, atom_node.lineno, atom_node.column)
@@ -1197,8 +1197,7 @@
     def comprehension_helper(self, comp_node,
                              handle_source_expr_meth="handle_expr",
                              for_type=syms.comp_for, if_type=syms.comp_if,
-                             iter_type=syms.comp_iter,
-                             comp_fix_unamed_tuple_location=False):
+                             iter_type=syms.comp_iter):
         handle_source_expression = getattr(self, handle_source_expr_meth)
         fors_count = self.count_comp_fors(comp_node, for_type, if_type)
         comps = []
@@ -1210,13 +1209,8 @@
             if len(for_node.children) == 1:
                 comp = ast.comprehension(for_targets[0], expr, None)
             else:
-                col = comp_node.column
-                line = comp_node.lineno
-                # Modified in python2.7, see http://bugs.python.org/issue6704
-                if comp_fix_unamed_tuple_location:
-                    col = for_targets[0].col_offset
-                    line = for_targets[0].lineno
-                target = ast.Tuple(for_targets, ast.Store, line, col)
+                target = ast.Tuple(for_targets, ast.Store, comp_node.lineno,
+                                   comp_node.column)
                 comp = ast.comprehension(target, expr, None)
             if len(comp_node.children) == 5:
                 comp_node = comp_iter = comp_node.children[4]
@@ -1238,8 +1232,7 @@
 
     def handle_genexp(self, genexp_node):
         elt = self.handle_expr(genexp_node.children[0])
-        comps = self.comprehension_helper(genexp_node.children[1],
-                                          comp_fix_unamed_tuple_location=True)
+        comps = self.comprehension_helper(genexp_node.children[1])
         return ast.GeneratorExp(elt, comps, genexp_node.lineno,
                                 genexp_node.column)
 
@@ -1248,22 +1241,19 @@
         comps = self.comprehension_helper(listcomp_node.children[1],
                                           "handle_testlist",
                                           syms.list_for, syms.list_if,
-                                          syms.list_iter,
-                                          comp_fix_unamed_tuple_location=True)
+                                          syms.list_iter)
         return ast.ListComp(elt, comps, listcomp_node.lineno,
                             listcomp_node.column)
 
     def handle_setcomp(self, set_maker):
         elt = self.handle_expr(set_maker.children[0])
-        comps = self.comprehension_helper(set_maker.children[1],
-                                          comp_fix_unamed_tuple_location=True)
+        comps = self.comprehension_helper(set_maker.children[1])
         return ast.SetComp(elt, comps, set_maker.lineno, set_maker.column)
 
     def handle_dictcomp(self, dict_maker):
         key = self.handle_expr(dict_maker.children[0])
         value = self.handle_expr(dict_maker.children[2])
-        comps = self.comprehension_helper(dict_maker.children[3],
-                                          comp_fix_unamed_tuple_location=True)
+        comps = self.comprehension_helper(dict_maker.children[3])
         return ast.DictComp(key, value, comps, dict_maker.lineno,
                             dict_maker.column)
 

From commits-noreply at bitbucket.org  Thu Feb 24 00:43:11 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Thu, 24 Feb 2011 00:43:11 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix MiniMark debug assertions when run on
	top of Python.
Message-ID: <20110223234311.8EEF936C224@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r42251:3db3cb5ba7e6
Date: 2011-02-23 18:42 -0500
http://bitbucket.org/pypy/pypy/changeset/3db3cb5ba7e6/

Log:	Fix MiniMark debug assertions when run on top of Python.

diff --git a/pypy/rpython/memory/gc/minimark.py b/pypy/rpython/memory/gc/minimark.py
--- a/pypy/rpython/memory/gc/minimark.py
+++ b/pypy/rpython/memory/gc/minimark.py
@@ -1,7 +1,7 @@
 """ MiniMark GC.
 
 Environment variables can be used to fine-tune the following parameters:
-    
+
  PYPY_GC_NURSERY        The nursery size.  Defaults to half the size of
                         the L2 cache.  Try values like '1.2MB'.  Small values
                         (like 1 or 1KB) are useful for debugging.
@@ -827,7 +827,7 @@
         if result:
             ll_assert(tid == -42, "bogus header for young obj")
         else:
-            ll_assert(tid != 0, "bogus header (1)")
+            ll_assert(bool(tid), "bogus header (1)")
             ll_assert(tid & ~TID_MASK == 0, "bogus header (2)")
         return result
 

diff --git a/pypy/rpython/lltypesystem/llgroup.py b/pypy/rpython/lltypesystem/llgroup.py
--- a/pypy/rpython/lltypesystem/llgroup.py
+++ b/pypy/rpython/lltypesystem/llgroup.py
@@ -76,6 +76,9 @@
         self.index = memberindex
         self.member = grp.members[memberindex]._as_ptr()
 
+    def __nonzero__(self):
+        return True
+
     def _get_group_member(self, grpptr):
         assert grpptr == self.grpptr, "get_group_member: wrong group!"
         return self.member
@@ -117,6 +120,9 @@
     def __repr__(self):
         return '<CombinedSymbolic %r|%s>' % (self.lowpart, self.rest)
 
+    def __nonzero__(self):
+        return True
+
     def __and__(self, other):
         if (other & CombinedSymbolic.MASK) == 0:
             return self.rest & other

From commits-noreply at bitbucket.org  Thu Feb 24 02:24:39 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Thu, 24 Feb 2011 02:24:39 +0100 (CET)
Subject: [pypy-svn] pypy default: Make directories with ? disappear
Message-ID: <20110224012439.847BE282BDE@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: 
Changeset: r42252:a8c4588929dd
Date: 2011-02-24 02:23 +0100
http://bitbucket.org/pypy/pypy/changeset/a8c4588929dd/

Log:	Make directories with ? disappear

diff --git a/pypy/tool/udir.py b/pypy/tool/udir.py
--- a/pypy/tool/udir.py
+++ b/pypy/tool/udir.py
@@ -34,6 +34,9 @@
         if info:
             project, hgtag, hgid = info
             basename = hgtag
+            if basename == '?':
+                basename = 'unknown' # directories with ? are not fun
+                # especially on windows
             if isinstance(basename, unicode):
                 basename = basename.encode(sys.getdefaultencoding())
         else:

From commits-noreply at bitbucket.org  Thu Feb 24 08:15:38 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Thu, 24 Feb 2011 08:15:38 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: hg merge default
Message-ID: <20110224071538.96C032A2033@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r42253:796e283415ba
Date: 2011-02-24 07:38 +0100
http://bitbucket.org/pypy/pypy/changeset/796e283415ba/

Log:	hg merge default


From commits-noreply at bitbucket.org  Thu Feb 24 10:13:26 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Thu, 24 Feb 2011 10:13:26 +0100 (CET)
Subject: [pypy-svn] pypy default: make sure that we insert '' in sys.path
	also when	running with -m, as cpython does.
Message-ID: <20110224091326.6DDE9282BEB@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42254:da5639f83041
Date: 2011-02-24 10:12 +0100
http://bitbucket.org/pypy/pypy/changeset/da5639f83041/

Log:	make sure that we insert '' in sys.path also when running with -m,
	as cpython does. Thanks Ademan for the original bug report

diff --git a/pypy/translator/goal/app_main.py b/pypy/translator/goal/app_main.py
--- a/pypy/translator/goal/app_main.py
+++ b/pypy/translator/goal/app_main.py
@@ -507,6 +507,8 @@
             success = run_toplevel(run_it)
         elif run_module:
             # handle the "-m" command
+            # '' on sys.path is required also here
+            sys.path.insert(0, '')
             import runpy
             success = run_toplevel(runpy._run_module_as_main, sys.argv[0])
         elif run_stdin:

diff --git a/pypy/translator/goal/test2/test_app_main.py b/pypy/translator/goal/test2/test_app_main.py
--- a/pypy/translator/goal/test2/test_app_main.py
+++ b/pypy/translator/goal/test2/test_app_main.py
@@ -435,6 +435,8 @@
         child.expect('True')
         child.sendline('"pypy.translator.goal.test2.mymodule" in sys.modules')
         child.expect('False')
+        child.sendline('sys.path[0]')
+        child.expect("''")
 
     def test_option_i_noexit(self):
         child = self.spawn(['-i', '-c', 'import sys; sys.exit(1)'])

From commits-noreply at bitbucket.org  Thu Feb 24 15:00:56 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Thu, 24 Feb 2011 15:00:56 +0100 (CET)
Subject: [pypy-svn] pypy default: That's the one place where using
	py.io.saferepr() is	not a good idea.
Message-ID: <20110224140056.4DF6A2A2033@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r42255:17bd126d3fee
Date: 2011-02-24 15:00 +0100
http://bitbucket.org/pypy/pypy/changeset/17bd126d3fee/

Log:	That's the one place where using py.io.saferepr() is not a good
	idea. The result was: when translate.py fails, we have a huge amount
	of output, followed by an error message that hides in "..." the real
	error. Let's just keep the full error message.

diff --git a/pypy/translator/platform/test/test_platform.py b/pypy/translator/platform/test/test_platform.py
--- a/pypy/translator/platform/test/test_platform.py
+++ b/pypy/translator/platform/test/test_platform.py
@@ -6,12 +6,12 @@
 from pypy.translator.tool.cbuild import ExternalCompilationInfo
 
 def test_compilationerror_repr():
-    # compilation error output/stdout may be large, make sure
-    # repr creates a limited version
+    # compilation error output/stdout may be large, but we don't want
+    # repr to create a limited version
     c = CompilationError('', '*'*1000)
-    assert len(repr(c)) < 500
+    assert repr(c) == 'CompilationError(err="""\n\t%s""")' % ('*'*1000,)
     c = CompilationError('*'*1000, '')
-    assert len(repr(c)) < 500
+    assert repr(c) == 'CompilationError(out="""\n\t%s""")' % ('*'*1000,)
 
 class TestPlatform(object):
     platform = host

diff --git a/pypy/translator/platform/__init__.py b/pypy/translator/platform/__init__.py
--- a/pypy/translator/platform/__init__.py
+++ b/pypy/translator/platform/__init__.py
@@ -19,9 +19,11 @@
 
     def __repr__(self):
         if self.err:
-            return "<CompilationError err=%s>" % py.io.saferepr(self.err)
+            attr = 'err'
         else:
-            return "<CompilationError out=%s>" % py.io.saferepr(self.out)
+            attr = 'out'
+        text = getattr(self, attr).replace('\n', '\n\t')
+        return 'CompilationError(%s="""\n\t%s""")' % (attr, text)
 
     __str__ = __repr__
 

From commits-noreply at bitbucket.org  Thu Feb 24 15:23:06 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Thu, 24 Feb 2011 15:23:06 +0100 (CET)
Subject: [pypy-svn] pypy default: (bivab, arigo)
Message-ID: <20110224142306.C80E8282BF2@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r42256:c4568a7b5237
Date: 2011-02-24 15:22 +0100
http://bitbucket.org/pypy/pypy/changeset/c4568a7b5237/

Log:	(bivab, arigo)

	Oups. A bug in the C translation backend. Only shows up when we
	have a switch on characters, whose cases are >= 128.

diff --git a/pypy/translator/c/test/test_backendoptimized.py b/pypy/translator/c/test/test_backendoptimized.py
--- a/pypy/translator/c/test/test_backendoptimized.py
+++ b/pypy/translator/c/test/test_backendoptimized.py
@@ -190,6 +190,22 @@
             y = ord(x)
             assert fn(y) == f(y)
 
+    def test_char_may_be_signed(self):
+        def f(n):
+            case = chr(n)
+            if case == '\xFF': return 1
+            if case == '\xFE': return 2
+            if case == '\xFD': return 3
+            if case == '\xFC': return 4
+            if case == '\xFB': return 5
+            if case == '\xFA': return 6
+            return 7
+        codegenerator = self.CodeGenerator()
+        fn = codegenerator.getcompiled(f, [int])
+        for input, expected in [(255, 1), (253, 3), (251, 5), (161, 7)]:
+            res = fn(input)
+            assert res == expected
+
     def test_unichr_switch(self):
         def f(y):
             x = unichr(y)

diff --git a/pypy/translator/c/primitive.py b/pypy/translator/c/primitive.py
--- a/pypy/translator/c/primitive.py
+++ b/pypy/translator/c/primitive.py
@@ -120,7 +120,7 @@
     if ' ' <= value < '\x7f':
         return "'%s'" % (value.replace("\\", r"\\").replace("'", r"\'"),)
     else:
-        return '%d' % ord(value)
+        return '((char)%d)' % ord(value)
 
 def name_bool(value, db):
     return '%d' % value

From commits-noreply at bitbucket.org  Thu Feb 24 15:24:53 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Thu, 24 Feb 2011 15:24:53 +0100 (CET)
Subject: [pypy-svn] pypy default: For completeness, "fix" also name_unichar(),
	even though
Message-ID: <20110224142453.3615B282BF2@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r42257:cb0805d8b7ac
Date: 2011-02-24 15:24 +0100
http://bitbucket.org/pypy/pypy/changeset/cb0805d8b7ac/

Log:	For completeness, "fix" also name_unichar(), even though it was
	probably not really broken.

diff --git a/pypy/translator/c/primitive.py b/pypy/translator/c/primitive.py
--- a/pypy/translator/c/primitive.py
+++ b/pypy/translator/c/primitive.py
@@ -130,7 +130,7 @@
 
 def name_unichar(value, db):
     assert type(value) is unicode and len(value) == 1
-    return '%d' % ord(value)
+    return '((wchar_t)%d)' % ord(value)
 
 def name_address(value, db):
     if value:

From commits-noreply at bitbucket.org  Thu Feb 24 16:28:11 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Thu, 24 Feb 2011 16:28:11 +0100 (CET)
Subject: [pypy-svn] pypy lltrace: Checkin of in-progress changes.
Message-ID: <20110224152811.3D9002A2033@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: lltrace
Changeset: r42258:994df461b5d5
Date: 2011-02-24 15:33 +0100
http://bitbucket.org/pypy/pypy/changeset/994df461b5d5/

Log:	Checkin of in-progress changes.

diff --git a/pypy/jit/backend/x86/assembler.py b/pypy/jit/backend/x86/assembler.py
--- a/pypy/jit/backend/x86/assembler.py
+++ b/pypy/jit/backend/x86/assembler.py
@@ -1335,6 +1335,25 @@
         base_loc, ofs_loc, value_loc, size_loc, baseofs = arglocs
         assert isinstance(baseofs, ImmedLoc)
         assert isinstance(size_loc, ImmedLoc)
+        #
+        # XXX
+        if op.getopname() == 'setarrayitem_gc':
+            # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ but we may get
+            # it as a setarrayitem_raw!!!!!!  fix this
+            from pypy.jit.backend.llsupport.descr import BaseArrayDescr
+            arraydescr = op.getdescr()
+            assert isinstance(arraydescr, BaseArrayDescr)
+            ofs = arraydescr.get_ofs_length(self.cpu.translate_support_code)
+            self.mc.CMP(addr_add_const(base_loc, ofs), ofs_loc)
+            self.mc.J_il8(rx86.Conditions['A'], 0)
+            ja_location = self.mc.get_relative_pos()
+            self.mc.writechar(chr(0xCC))    # INT3
+            # patch the JA above
+            offset = self.mc.get_relative_pos() - ja_location
+            assert 0 < offset <= 127
+            self.mc.overwrite(ja_location-1, chr(offset))
+        # XXX
+        #
         scale = _get_scale(size_loc.value)
         dest_addr = AddressLoc(base_loc, ofs_loc, scale, baseofs.value)
         self.save_into_mem(dest_addr, value_loc, size_loc)

diff --git a/pypy/jit/backend/llsupport/llmodel.py b/pypy/jit/backend/llsupport/llmodel.py
--- a/pypy/jit/backend/llsupport/llmodel.py
+++ b/pypy/jit/backend/llsupport/llmodel.py
@@ -4,6 +4,7 @@
 from pypy.rpython.llinterp import LLInterpreter, LLException
 from pypy.rpython.annlowlevel import llhelper
 from pypy.rlib.objectmodel import we_are_translated, specialize
+from pypy.rlib.debug import ll_assert
 from pypy.jit.metainterp.history import BoxInt, BoxPtr, set_future_values,\
      BoxFloat
 from pypy.jit.metainterp import history
@@ -277,6 +278,13 @@
 
     # ____________________________________________________________
 
+    @specialize.argtype(2)
+    def _checkarraybound(self, arraydescr, array, itemindex):
+        if lltype.typeOf(array) == llmemory.GCREF:
+            length = self.bh_arraylen_gc(arraydescr, array)
+            ll_assert(0 <= itemindex < length,
+                      "bh_setarrayitem_gc: index out of bound")
+
     def bh_arraylen_gc(self, arraydescr, array):
         assert isinstance(arraydescr, BaseArrayDescr)
         ofs = arraydescr.get_ofs_length(self.translate_support_code)
@@ -284,6 +292,7 @@
 
     @specialize.argtype(2)
     def bh_getarrayitem_gc_i(self, arraydescr, gcref, itemindex):
+        self._checkarraybound(arraydescr, gcref, itemindex)
         ofs, size, sign = self.unpack_arraydescr_size(arraydescr)
         # --- start of GC unsafe code (no GC operation!) ---
         items = rffi.ptradd(rffi.cast(rffi.CCHARP, gcref), ofs)
@@ -303,6 +312,7 @@
             raise NotImplementedError("size = %d" % size)
 
     def bh_getarrayitem_gc_r(self, arraydescr, gcref, itemindex):
+        self._checkarraybound(arraydescr, gcref, itemindex)
         ofs = self.unpack_arraydescr(arraydescr)
         # --- start of GC unsafe code (no GC operation!) ---
         items = rffi.ptradd(rffi.cast(rffi.CCHARP, gcref), ofs)
@@ -313,6 +323,7 @@
 
     @specialize.argtype(2)
     def bh_getarrayitem_gc_f(self, arraydescr, gcref, itemindex):
+        self._checkarraybound(arraydescr, gcref, itemindex)
         ofs = self.unpack_arraydescr(arraydescr)
         # --- start of GC unsafe code (no GC operation!) ---
         items = rffi.ptradd(rffi.cast(rffi.CCHARP, gcref), ofs)
@@ -323,6 +334,7 @@
 
     @specialize.argtype(2)
     def bh_setarrayitem_gc_i(self, arraydescr, gcref, itemindex, newvalue):
+        self._checkarraybound(arraydescr, gcref, itemindex)
         ofs, size, sign = self.unpack_arraydescr_size(arraydescr)
         # --- start of GC unsafe code (no GC operation!) ---
         items = rffi.ptradd(rffi.cast(rffi.CCHARP, gcref), ofs)
@@ -339,6 +351,7 @@
             raise NotImplementedError("size = %d" % size)
 
     def bh_setarrayitem_gc_r(self, arraydescr, gcref, itemindex, newvalue):
+        self._checkarraybound(arraydescr, gcref, itemindex)
         ofs = self.unpack_arraydescr(arraydescr)
         self.gc_ll_descr.do_write_barrier(gcref, newvalue)
         # --- start of GC unsafe code (no GC operation!) ---
@@ -351,6 +364,7 @@
 
     @specialize.argtype(2)
     def bh_setarrayitem_gc_f(self, arraydescr, gcref, itemindex, newvalue):
+        self._checkarraybound(arraydescr, gcref, itemindex)
         ofs = self.unpack_arraydescr(arraydescr)
         # --- start of GC unsafe code (no GC operation!) ---
         items = rffi.ptradd(rffi.cast(rffi.CCHARP, gcref), ofs)

From commits-noreply at bitbucket.org  Thu Feb 24 16:28:12 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Thu, 24 Feb 2011 16:28:12 +0100 (CET)
Subject: [pypy-svn] pypy default: (vincele)
Message-ID: <20110224152812.AB84F2A2034@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r42259:61ba51102c34
Date: 2011-02-24 16:27 +0100
http://bitbucket.org/pypy/pypy/changeset/61ba51102c34/

Log:	(vincele)

	changed lineno & col_offset reporting, as CPython did:
	http://bugs.python.org/issue6704

diff --git a/lib-python/modified-2.7.0/test/test_ast.py b/lib-python/modified-2.7.0/test/test_ast.py
new file mode 100644
--- /dev/null
+++ b/lib-python/modified-2.7.0/test/test_ast.py
@@ -0,0 +1,578 @@
+import sys, itertools, unittest
+from test import test_support
+import ast
+
+def to_tuple(t):
+    if t is None or isinstance(t, (basestring, int, long, complex)):
+        return t
+    elif isinstance(t, list):
+        return [to_tuple(e) for e in t]
+    result = [t.__class__.__name__]
+    if hasattr(t, 'lineno') and hasattr(t, 'col_offset'):
+        result.append((t.lineno, t.col_offset))
+    if t._fields is None:
+        return tuple(result)
+    for f in t._fields:
+        result.append(to_tuple(getattr(t, f)))
+    return tuple(result)
+
+
+# These tests are compiled through "exec"
+# There should be atleast one test per statement
+exec_tests = [
+    # None
+    "None",
+    # FunctionDef
+    "def f(): pass",
+    # FunctionDef with arg
+    "def f(a): pass",
+    # FunctionDef with arg and default value
+    "def f(a=0): pass",
+    # FunctionDef with varargs
+    "def f(*args): pass",
+    # FunctionDef with kwargs
+    "def f(**kwargs): pass",
+    # FunctionDef with all kind of args
+    "def f(a, b=1, c=None, d=[], e={}, *args, **kwargs): pass",
+    # ClassDef
+    "class C:pass",
+    # ClassDef, new style class
+    "class C(object): pass",
+    # Return
+    "def f():return 1",
+    # Delete
+    "del v",
+    # Assign
+    "v = 1",
+    # AugAssign
+    "v += 1",
+    # Print
+    "print >>f, 1, ",
+    # For
+    "for v in v:pass",
+    # While
+    "while v:pass",
+    # If
+    "if v:pass",
+    # Raise
+    "raise Exception, 'string'",
+    # TryExcept
+    "try:\n  pass\nexcept Exception:\n  pass",
+    # TryFinally
+    "try:\n  pass\nfinally:\n  pass",
+    # Assert
+    "assert v",
+    # Import
+    "import sys",
+    # ImportFrom
+    "from sys import v",
+    # Exec
+    "exec 'v'",
+    # Global
+    "global v",
+    # Expr
+    "1",
+    # Pass,
+    "pass",
+    # Break
+    "break",
+    # Continue
+    "continue",
+    # for statements with naked tuples (see http://bugs.python.org/issue6704)
+    "for a,b in c: pass",
+    "[(a,b) for a,b in c]",
+    "((a,b) for a,b in c)",
+    "((a,b) for (a,b) in c)",
+    # Multiline generator expression
+    """(
+    (
+    Aa
+    ,
+       Bb
+    )
+    for
+    Aa
+    ,
+    Bb in Cc
+    )""",
+    # dictcomp
+    "{a : b for w in x for m in p if g}",
+    # dictcomp with naked tuple
+    "{a : b for v,w in x}",
+    # setcomp
+    "{r for l in x if g}",
+    # setcomp with naked tuple
+    "{r for l,m in x}",
+]
+
+# These are compiled through "single"
+# because of overlap with "eval", it just tests what
+# can't be tested with "eval"
+single_tests = [
+    "1+2"
+]
+
+# These are compiled through "eval"
+# It should test all expressions
+eval_tests = [
+  # None
+  "None",
+  # BoolOp
+  "a and b",
+  # BinOp
+  "a + b",
+  # UnaryOp
+  "not v",
+  # Lambda
+  "lambda:None",
+  # Dict
+  "{ 1:2 }",
+  # Empty dict
+  "{}",
+  # Set
+  "{None,}",
+  # Multiline dict
+  """{
+      1
+        :
+          2
+     }""",
+  # ListComp
+  "[a for b in c if d]",
+  # GeneratorExp
+  "(a for b in c if d)",
+  # Yield - yield expressions can't work outside a function
+  #
+  # Compare
+  "1 < 2 < 3",
+  # Call
+  "f(1,2,c=3,*d,**e)",
+  # Repr
+  "`v`",
+  # Num
+  "10L",
+  # Str
+  "'string'",
+  # Attribute
+  "a.b",
+  # Subscript
+  "a[b:c]",
+  # Name
+  "v",
+  # List
+  "[1,2,3]",
+  # Empty list
+  "[]",
+  # Tuple
+  "1,2,3",
+  # Tuple
+  "(1,2,3)",
+  # Empty tuple
+  "()",
+  # Combination
+  "a.b.c.d(a.b[1:2])",
+
+]
+
+# TODO: expr_context, slice, boolop, operator, unaryop, cmpop, comprehension
+# excepthandler, arguments, keywords, alias
+
+class AST_Tests(unittest.TestCase):
+
+    def _assertTrueorder(self, ast_node, parent_pos):
+        if not isinstance(ast_node, ast.AST) or ast_node._fields is None:
+            return
+        if isinstance(ast_node, (ast.expr, ast.stmt, ast.excepthandler)):
+            node_pos = (ast_node.lineno, ast_node.col_offset)
+            self.assertTrue(node_pos >= parent_pos)
+            parent_pos = (ast_node.lineno, ast_node.col_offset)
+        for name in ast_node._fields:
+            value = getattr(ast_node, name)
+            if isinstance(value, list):
+                for child in value:
+                    self._assertTrueorder(child, parent_pos)
+            elif value is not None:
+                self._assertTrueorder(value, parent_pos)
+
+    def test_AST_objects(self):
+        x = ast.AST()
+        try:
+            x.foobar = 21
+        except AttributeError, e:
+            self.assertEquals(e.args[0],
+                              "'_ast.AST' object has no attribute 'foobar'")
+        else:
+            self.assert_(False)
+
+        try:
+            ast.AST(lineno=2)
+        except AttributeError, e:
+            self.assertEquals(e.args[0],
+                              "'_ast.AST' object has no attribute 'lineno'")
+        else:
+            self.assert_(False)
+        try:
+            ast.AST(2)
+        except TypeError, e:
+            self.assertEquals(e.args[0],
+                              "_ast.AST constructor takes 0 positional arguments")
+        else:
+            self.assert_(False)
+
+    def test_snippets(self):
+        for input, output, kind in ((exec_tests, exec_results, "exec"),
+                                    (single_tests, single_results, "single"),
+                                    (eval_tests, eval_results, "eval")):
+            for i, o in itertools.izip(input, output):
+                ast_tree = compile(i, "?", kind, ast.PyCF_ONLY_AST)
+                self.assertEquals(to_tuple(ast_tree), o)
+                self._assertTrueorder(ast_tree, (0, 0))
+
+    def test_slice(self):
+        slc = ast.parse("x[::]").body[0].value.slice
+        self.assertIsNone(slc.upper)
+        self.assertIsNone(slc.lower)
+        self.assertIsInstance(slc.step, ast.Name)
+        self.assertEqual(slc.step.id, "None")
+
+    def test_from_import(self):
+        im = ast.parse("from . import y").body[0]
+        self.assertIsNone(im.module)
+
+    def test_base_classes(self):
+        self.assertTrue(issubclass(ast.For, ast.stmt))
+        self.assertTrue(issubclass(ast.Name, ast.expr))
+        self.assertTrue(issubclass(ast.stmt, ast.AST))
+        self.assertTrue(issubclass(ast.expr, ast.AST))
+        self.assertTrue(issubclass(ast.comprehension, ast.AST))
+        self.assertTrue(issubclass(ast.Gt, ast.AST))
+
+    def test_field_attr_existence(self):
+        for name, item in ast.__dict__.iteritems():
+            if isinstance(item, type) and name != 'AST' and name[0].isupper(): # XXX: pypy does not allow abstract ast class instanciation
+                x = item()
+                if isinstance(x, ast.AST):
+                    self.assertEquals(type(x._fields), tuple)
+
+    def test_arguments(self):
+        x = ast.arguments()
+        self.assertEquals(x._fields, ('args', 'vararg', 'kwarg', 'defaults'))
+        try:
+            x.vararg
+        except AttributeError, e:
+            self.assertEquals(e.args[0],
+                              "'arguments' object has no attribute 'vararg'")
+        else:
+            self.assert_(False)
+        x = ast.arguments(1, 2, 3, 4)
+        self.assertEquals(x.vararg, 2)
+
+    def test_field_attr_writable(self):
+        x = ast.Num()
+        # We can assign to _fields
+        x._fields = 666
+        self.assertEquals(x._fields, 666)
+
+    def test_classattrs(self):
+        x = ast.Num()
+        self.assertEquals(x._fields, ('n',))
+        try:
+            x.n
+        except AttributeError, e:
+            self.assertEquals(e.args[0],
+                              "'Num' object has no attribute 'n'")
+        else:
+            self.assert_(False)
+
+        x = ast.Num(42)
+        self.assertEquals(x.n, 42)
+        try:
+            x.lineno
+        except AttributeError, e:
+            self.assertEquals(e.args[0],
+                              "'Num' object has no attribute 'lineno'")
+        else:
+            self.assert_(False)
+
+        y = ast.Num()
+        x.lineno = y
+        self.assertEquals(x.lineno, y)
+
+        try:
+            x.foobar
+        except AttributeError, e:
+            self.assertEquals(e.args[0],
+                              "'Num' object has no attribute 'foobar'")
+        else:
+            self.assert_(False)
+
+        x = ast.Num(lineno=2)
+        self.assertEquals(x.lineno, 2)
+
+        x = ast.Num(42, lineno=0)
+        self.assertEquals(x.lineno, 0)
+        self.assertEquals(x._fields, ('n',))
+        self.assertEquals(x.n, 42)
+
+        self.assertRaises(TypeError, ast.Num, 1, 2)
+        self.assertRaises(TypeError, ast.Num, 1, 2, lineno=0)
+
+    def test_module(self):
+        body = [ast.Num(42)]
+        x = ast.Module(body)
+        self.assertEquals(x.body, body)
+
+    def test_nodeclass(self):
+        x = ast.BinOp()
+        self.assertEquals(x._fields, ('left', 'op', 'right'))
+
+        # Zero arguments constructor explicitely allowed
+        x = ast.BinOp()
+        # Random attribute allowed too
+        x.foobarbaz = 5
+        self.assertEquals(x.foobarbaz, 5)
+
+        n1 = ast.Num(1)
+        n3 = ast.Num(3)
+        addop = ast.Add()
+        x = ast.BinOp(n1, addop, n3)
+        self.assertEquals(x.left, n1)
+        self.assertEquals(x.op, addop)
+        self.assertEquals(x.right, n3)
+        
+        x = ast.BinOp(1, 2, 3)
+        self.assertEquals(x.left, 1)
+        self.assertEquals(x.op, 2)
+        self.assertEquals(x.right, 3)
+
+        x = ast.BinOp(1, 2, 3, lineno=0)
+        self.assertEquals(x.lineno, 0)
+
+        # node raises exception when not given enough arguments
+        self.assertRaises(TypeError, ast.BinOp, 1, 2)
+        # node raises exception when given too many arguments
+        self.assertRaises(TypeError, ast.BinOp, 1, 2, 3, 4)
+        # node raises exception when not given enough arguments
+        self.assertRaises(TypeError, ast.BinOp, 1, 2, lineno=0)
+        # node raises exception when given too many arguments
+        self.assertRaises(TypeError, ast.BinOp, 1, 2, 3, 4, lineno=0)
+
+        # can set attributes through kwargs too
+        x = ast.BinOp(left=1, op=2, right=3, lineno=0)
+        self.assertEquals(x.left, 1)
+        self.assertEquals(x.op, 2)
+        self.assertEquals(x.right, 3)
+        self.assertEquals(x.lineno, 0)
+
+        # Random kwargs also allowed
+        x = ast.BinOp(1, 2, 3, foobarbaz=42)
+        self.assertEquals(x.foobarbaz, 42)
+
+    def test_no_fields(self):
+        # this used to fail because Sub._fields was None
+        x = ast.Sub()
+        self.assertEquals(x._fields, ())
+
+    def test_pickling(self):
+        import pickle
+        mods = [pickle]
+        try:
+            import cPickle
+            mods.append(cPickle)
+        except ImportError:
+            pass
+        protocols = [0, 1, 2]
+        for mod in mods:
+            for protocol in protocols:
+                for ast in (compile(i, "?", "exec", 0x400) for i in exec_tests):
+                    ast2 = mod.loads(mod.dumps(ast, protocol))
+                    self.assertEquals(to_tuple(ast2), to_tuple(ast))
+
+
+class ASTHelpers_Test(unittest.TestCase):
+
+    def test_parse(self):
+        a = ast.parse('foo(1 + 1)')
+        b = compile('foo(1 + 1)', '<unknown>', 'exec', ast.PyCF_ONLY_AST)
+        self.assertEqual(ast.dump(a), ast.dump(b))
+
+    def test_dump(self):
+        node = ast.parse('spam(eggs, "and cheese")')
+        self.assertEqual(ast.dump(node),
+            "Module(body=[Expr(value=Call(func=Name(id='spam', ctx=Load()), "
+            "args=[Name(id='eggs', ctx=Load()), Str(s='and cheese')], "
+            "keywords=[], starargs=None, kwargs=None))])"
+        )
+        self.assertEqual(ast.dump(node, annotate_fields=False),
+            "Module([Expr(Call(Name('spam', Load()), [Name('eggs', Load()), "
+            "Str('and cheese')], [], None, None))])"
+        )
+        self.assertEqual(ast.dump(node, include_attributes=True),
+            "Module(body=[Expr(value=Call(func=Name(id='spam', ctx=Load(), "
+            "lineno=1, col_offset=0), args=[Name(id='eggs', ctx=Load(), "
+            "lineno=1, col_offset=5), Str(s='and cheese', lineno=1, "
+            "col_offset=11)], keywords=[], starargs=None, kwargs=None, "
+            "lineno=1, col_offset=0), lineno=1, col_offset=0)])"
+        )
+
+    def test_copy_location(self):
+        src = ast.parse('1 + 1', mode='eval')
+        src.body.right = ast.copy_location(ast.Num(2), src.body.right)
+        self.assertEqual(ast.dump(src, include_attributes=True),
+            'Expression(body=BinOp(left=Num(n=1, lineno=1, col_offset=0), '
+            'op=Add(), right=Num(n=2, lineno=1, col_offset=4), lineno=1, '
+            'col_offset=0))'
+        )
+
+    def test_fix_missing_locations(self):
+        src = ast.parse('write("spam")')
+        src.body.append(ast.Expr(ast.Call(ast.Name('spam', ast.Load()),
+                                          [ast.Str('eggs')], [], None, None)))
+        self.assertEqual(src, ast.fix_missing_locations(src))
+        self.assertEqual(ast.dump(src, include_attributes=True),
+            "Module(body=[Expr(value=Call(func=Name(id='write', ctx=Load(), "
+            "lineno=1, col_offset=0), args=[Str(s='spam', lineno=1, "
+            "col_offset=6)], keywords=[], starargs=None, kwargs=None, "
+            "lineno=1, col_offset=0), lineno=1, col_offset=0), "
+            "Expr(value=Call(func=Name(id='spam', ctx=Load(), lineno=1, "
+            "col_offset=0), args=[Str(s='eggs', lineno=1, col_offset=0)], "
+            "keywords=[], starargs=None, kwargs=None, lineno=1, "
+            "col_offset=0), lineno=1, col_offset=0)])"
+        )
+
+    def test_increment_lineno(self):
+        src = ast.parse('1 + 1', mode='eval')
+        self.assertEqual(ast.increment_lineno(src, n=3), src)
+        self.assertEqual(ast.dump(src, include_attributes=True),
+            'Expression(body=BinOp(left=Num(n=1, lineno=4, col_offset=0), '
+            'op=Add(), right=Num(n=1, lineno=4, col_offset=4), lineno=4, '
+            'col_offset=0))'
+        )
+
+    def test_iter_fields(self):
+        node = ast.parse('foo()', mode='eval')
+        d = dict(ast.iter_fields(node.body))
+        self.assertEqual(d.pop('func').id, 'foo')
+        self.assertEqual(d, {'keywords': [], 'kwargs': None,
+                             'args': [], 'starargs': None})
+
+    def test_iter_child_nodes(self):
+        node = ast.parse("spam(23, 42, eggs='leek')", mode='eval')
+        self.assertEqual(len(list(ast.iter_child_nodes(node.body))), 4)
+        iterator = ast.iter_child_nodes(node.body)
+        self.assertEqual(next(iterator).id, 'spam')
+        self.assertEqual(next(iterator).n, 23)
+        self.assertEqual(next(iterator).n, 42)
+        self.assertEqual(ast.dump(next(iterator)),
+            "keyword(arg='eggs', value=Str(s='leek'))"
+        )
+
+    def test_get_docstring(self):
+        node = ast.parse('def foo():\n  """line one\n  line two"""')
+        self.assertEqual(ast.get_docstring(node.body[0]),
+                         'line one\nline two')
+
+    def test_literal_eval(self):
+        self.assertEqual(ast.literal_eval('[1, 2, 3]'), [1, 2, 3])
+        self.assertEqual(ast.literal_eval('{"foo": 42}'), {"foo": 42})
+        self.assertEqual(ast.literal_eval('(True, False, None)'), (True, False, None))
+        self.assertRaises(ValueError, ast.literal_eval, 'foo()')
+
+    def test_literal_eval_issue4907(self):
+        self.assertEqual(ast.literal_eval('2j'), 2j)
+        self.assertEqual(ast.literal_eval('10 + 2j'), 10 + 2j)
+        self.assertEqual(ast.literal_eval('1.5 - 2j'), 1.5 - 2j)
+        self.assertRaises(ValueError, ast.literal_eval, '2 + (3 + 4j)')
+
+
+def test_main():
+    with test_support.check_py3k_warnings(("backquote not supported",
+                                             SyntaxWarning)):
+        test_support.run_unittest(AST_Tests, ASTHelpers_Test)
+
+def main():
+    if __name__ != '__main__':
+        return
+    if sys.argv[1:] == ['-g']:
+        for statements, kind in ((exec_tests, "exec"), (single_tests, "single"),
+                                 (eval_tests, "eval")):
+            print kind+"_results = ["
+            for s in statements:
+                print repr(to_tuple(compile(s, "?", kind, 0x400)))+","
+            print "]"
+        print "main()"
+        raise SystemExit
+    test_main()
+
+#### EVERYTHING BELOW IS GENERATED #####
+exec_results = [
+('Module', [('Expr', (1, 0), ('Name', (1, 0), 'None', ('Load',)))]),
+('Module', [('FunctionDef', (1, 0), 'f', ('arguments', [], None, None, []), [('Pass', (1, 9))], [])]),
+('Module', [('FunctionDef', (1, 0), 'f', ('arguments', [('Name', (1, 6), 'a', ('Param',))], None, None, []), [('Pass', (1, 10))], [])]),
+('Module', [('FunctionDef', (1, 0), 'f', ('arguments', [('Name', (1, 6), 'a', ('Param',))], None, None, [('Num', (1, 8), 0)]), [('Pass', (1, 12))], [])]),
+('Module', [('FunctionDef', (1, 0), 'f', ('arguments', [], 'args', None, []), [('Pass', (1, 14))], [])]),
+('Module', [('FunctionDef', (1, 0), 'f', ('arguments', [], None, 'kwargs', []), [('Pass', (1, 17))], [])]),
+('Module', [('FunctionDef', (1, 0), 'f', ('arguments', [('Name', (1, 6), 'a', ('Param',)), ('Name', (1, 9), 'b', ('Param',)), ('Name', (1, 14), 'c', ('Param',)), ('Name', (1, 22), 'd', ('Param',)), ('Name', (1, 28), 'e', ('Param',))], 'args', 'kwargs', [('Num', (1, 11), 1), ('Name', (1, 16), 'None', ('Load',)), ('List', (1, 24), [], ('Load',)), ('Dict', (1, 30), [], [])]), [('Pass', (1, 52))], [])]),
+('Module', [('ClassDef', (1, 0), 'C', [], [('Pass', (1, 8))], [])]),
+('Module', [('ClassDef', (1, 0), 'C', [('Name', (1, 8), 'object', ('Load',))], [('Pass', (1, 17))], [])]),
+('Module', [('FunctionDef', (1, 0), 'f', ('arguments', [], None, None, []), [('Return', (1, 8), ('Num', (1, 15), 1))], [])]),
+('Module', [('Delete', (1, 0), [('Name', (1, 4), 'v', ('Del',))])]),
+('Module', [('Assign', (1, 0), [('Name', (1, 0), 'v', ('Store',))], ('Num', (1, 4), 1))]),
+('Module', [('AugAssign', (1, 0), ('Name', (1, 0), 'v', ('Store',)), ('Add',), ('Num', (1, 5), 1))]),
+('Module', [('Print', (1, 0), ('Name', (1, 8), 'f', ('Load',)), [('Num', (1, 11), 1)], False)]),
+('Module', [('For', (1, 0), ('Name', (1, 4), 'v', ('Store',)), ('Name', (1, 9), 'v', ('Load',)), [('Pass', (1, 11))], [])]),
+('Module', [('While', (1, 0), ('Name', (1, 6), 'v', ('Load',)), [('Pass', (1, 8))], [])]),
+('Module', [('If', (1, 0), ('Name', (1, 3), 'v', ('Load',)), [('Pass', (1, 5))], [])]),
+('Module', [('Raise', (1, 0), ('Name', (1, 6), 'Exception', ('Load',)), ('Str', (1, 17), 'string'), None)]),
+('Module', [('TryExcept', (1, 0), [('Pass', (2, 2))], [('ExceptHandler', (3, 0), ('Name', (3, 7), 'Exception', ('Load',)), None, [('Pass', (4, 2))])], [])]),
+('Module', [('TryFinally', (1, 0), [('Pass', (2, 2))], [('Pass', (4, 2))])]),
+('Module', [('Assert', (1, 0), ('Name', (1, 7), 'v', ('Load',)), None)]),
+('Module', [('Import', (1, 0), [('alias', 'sys', None)])]),
+('Module', [('ImportFrom', (1, 0), 'sys', [('alias', 'v', None)], 0)]),
+('Module', [('Exec', (1, 0), ('Str', (1, 5), 'v'), None, None)]),
+('Module', [('Global', (1, 0), ['v'])]),
+('Module', [('Expr', (1, 0), ('Num', (1, 0), 1))]),
+('Module', [('Pass', (1, 0))]),
+('Module', [('Break', (1, 0))]),
+('Module', [('Continue', (1, 0))]),
+('Module', [('For', (1, 0), ('Tuple', (1, 4), [('Name', (1, 4), 'a', ('Store',)), ('Name', (1, 6), 'b', ('Store',))], ('Store',)), ('Name', (1, 11), 'c', ('Load',)), [('Pass', (1, 14))], [])]),
+('Module', [('Expr', (1, 0), ('ListComp', (1, 1), ('Tuple', (1, 2), [('Name', (1, 2), 'a', ('Load',)), ('Name', (1, 4), 'b', ('Load',))], ('Load',)), [('comprehension', ('Tuple', (1, 11), [('Name', (1, 11), 'a', ('Store',)), ('Name', (1, 13), 'b', ('Store',))], ('Store',)), ('Name', (1, 18), 'c', ('Load',)), [])]))]),
+('Module', [('Expr', (1, 0), ('GeneratorExp', (1, 1), ('Tuple', (1, 2), [('Name', (1, 2), 'a', ('Load',)), ('Name', (1, 4), 'b', ('Load',))], ('Load',)), [('comprehension', ('Tuple', (1, 11), [('Name', (1, 11), 'a', ('Store',)), ('Name', (1, 13), 'b', ('Store',))], ('Store',)), ('Name', (1, 18), 'c', ('Load',)), [])]))]),
+('Module', [('Expr', (1, 0), ('GeneratorExp', (1, 1), ('Tuple', (1, 2), [('Name', (1, 2), 'a', ('Load',)), ('Name', (1, 4), 'b', ('Load',))], ('Load',)), [('comprehension', ('Tuple', (1, 12), [('Name', (1, 12), 'a', ('Store',)), ('Name', (1, 14), 'b', ('Store',))], ('Store',)), ('Name', (1, 20), 'c', ('Load',)), [])]))]),
+('Module', [('Expr', (1, 0), ('GeneratorExp', (2, 4), ('Tuple', (3, 4), [('Name', (3, 4), 'Aa', ('Load',)), ('Name', (5, 7), 'Bb', ('Load',))], ('Load',)), [('comprehension', ('Tuple', (8, 4), [('Name', (8, 4), 'Aa', ('Store',)), ('Name', (10, 4), 'Bb', ('Store',))], ('Store',)), ('Name', (10, 10), 'Cc', ('Load',)), [])]))]),
+('Module', [('Expr', (1, 0), ('DictComp', (1, 1), ('Name', (1, 1), 'a', ('Load',)), ('Name', (1, 5), 'b', ('Load',)), [('comprehension', ('Name', (1, 11), 'w', ('Store',)), ('Name', (1, 16), 'x', ('Load',)), []), ('comprehension', ('Name', (1, 22), 'm', ('Store',)), ('Name', (1, 27), 'p', ('Load',)), [('Name', (1, 32), 'g', ('Load',))])]))]),
+('Module', [('Expr', (1, 0), ('DictComp', (1, 1), ('Name', (1, 1), 'a', ('Load',)), ('Name', (1, 5), 'b', ('Load',)), [('comprehension', ('Tuple', (1, 11), [('Name', (1, 11), 'v', ('Store',)), ('Name', (1, 13), 'w', ('Store',))], ('Store',)), ('Name', (1, 18), 'x', ('Load',)), [])]))]),
+('Module', [('Expr', (1, 0), ('SetComp', (1, 1), ('Name', (1, 1), 'r', ('Load',)), [('comprehension', ('Name', (1, 7), 'l', ('Store',)), ('Name', (1, 12), 'x', ('Load',)), [('Name', (1, 17), 'g', ('Load',))])]))]),
+('Module', [('Expr', (1, 0), ('SetComp', (1, 1), ('Name', (1, 1), 'r', ('Load',)), [('comprehension', ('Tuple', (1, 7), [('Name', (1, 7), 'l', ('Store',)), ('Name', (1, 9), 'm', ('Store',))], ('Store',)), ('Name', (1, 14), 'x', ('Load',)), [])]))]),
+]
+single_results = [
+('Interactive', [('Expr', (1, 0), ('BinOp', (1, 0), ('Num', (1, 0), 1), ('Add',), ('Num', (1, 2), 2)))]),
+]
+eval_results = [
+('Expression', ('Name', (1, 0), 'None', ('Load',))),
+('Expression', ('BoolOp', (1, 0), ('And',), [('Name', (1, 0), 'a', ('Load',)), ('Name', (1, 6), 'b', ('Load',))])),
+('Expression', ('BinOp', (1, 0), ('Name', (1, 0), 'a', ('Load',)), ('Add',), ('Name', (1, 4), 'b', ('Load',)))),
+('Expression', ('UnaryOp', (1, 0), ('Not',), ('Name', (1, 4), 'v', ('Load',)))),
+('Expression', ('Lambda', (1, 0), ('arguments', [], None, None, []), ('Name', (1, 7), 'None', ('Load',)))),
+('Expression', ('Dict', (1, 0), [('Num', (1, 2), 1)], [('Num', (1, 4), 2)])),
+('Expression', ('Dict', (1, 0), [], [])),
+('Expression', ('Set', (1, 0), [('Name', (1, 1), 'None', ('Load',))])),
+('Expression', ('Dict', (1, 0), [('Num', (2, 6), 1)], [('Num', (4, 10), 2)])),
+('Expression', ('ListComp', (1, 1), ('Name', (1, 1), 'a', ('Load',)), [('comprehension', ('Name', (1, 7), 'b', ('Store',)), ('Name', (1, 12), 'c', ('Load',)), [('Name', (1, 17), 'd', ('Load',))])])),
+('Expression', ('GeneratorExp', (1, 1), ('Name', (1, 1), 'a', ('Load',)), [('comprehension', ('Name', (1, 7), 'b', ('Store',)), ('Name', (1, 12), 'c', ('Load',)), [('Name', (1, 17), 'd', ('Load',))])])),
+('Expression', ('Compare', (1, 0), ('Num', (1, 0), 1), [('Lt',), ('Lt',)], [('Num', (1, 4), 2), ('Num', (1, 8), 3)])),
+('Expression', ('Call', (1, 0), ('Name', (1, 0), 'f', ('Load',)), [('Num', (1, 2), 1), ('Num', (1, 4), 2)], [('keyword', 'c', ('Num', (1, 8), 3))], ('Name', (1, 11), 'd', ('Load',)), ('Name', (1, 15), 'e', ('Load',)))),
+('Expression', ('Repr', (1, 0), ('Name', (1, 1), 'v', ('Load',)))),
+('Expression', ('Num', (1, 0), 10L)),
+('Expression', ('Str', (1, 0), 'string')),
+('Expression', ('Attribute', (1, 0), ('Name', (1, 0), 'a', ('Load',)), 'b', ('Load',))),
+('Expression', ('Subscript', (1, 0), ('Name', (1, 0), 'a', ('Load',)), ('Slice', ('Name', (1, 2), 'b', ('Load',)), ('Name', (1, 4), 'c', ('Load',)), None), ('Load',))),
+('Expression', ('Name', (1, 0), 'v', ('Load',))),
+('Expression', ('List', (1, 0), [('Num', (1, 1), 1), ('Num', (1, 3), 2), ('Num', (1, 5), 3)], ('Load',))),
+('Expression', ('List', (1, 0), [], ('Load',))),
+('Expression', ('Tuple', (1, 0), [('Num', (1, 0), 1), ('Num', (1, 2), 2), ('Num', (1, 4), 3)], ('Load',))),
+('Expression', ('Tuple', (1, 1), [('Num', (1, 1), 1), ('Num', (1, 3), 2), ('Num', (1, 5), 3)], ('Load',))),
+('Expression', ('Tuple', (1, 0), [], ('Load',))),
+('Expression', ('Call', (1, 0), ('Attribute', (1, 0), ('Attribute', (1, 0), ('Attribute', (1, 0), ('Name', (1, 0), 'a', ('Load',)), 'b', ('Load',)), 'c', ('Load',)), 'd', ('Load',)), [('Subscript', (1, 8), ('Attribute', (1, 8), ('Name', (1, 8), 'a', ('Load',)), 'b', ('Load',)), ('Slice', ('Num', (1, 12), 1), ('Num', (1, 14), 2), None), ('Load',))], [], None, None)),
+]
+main()

diff --git a/pypy/interpreter/astcompiler/astbuilder.py b/pypy/interpreter/astcompiler/astbuilder.py
--- a/pypy/interpreter/astcompiler/astbuilder.py
+++ b/pypy/interpreter/astcompiler/astbuilder.py
@@ -1130,13 +1130,13 @@
         elif first_child_type == tokens.LBRACE:
             maker = atom_node.children[1]
             if maker.type == tokens.RBRACE:
-                return ast.Dict(None, None, maker.lineno, maker.column)
+                return ast.Dict(None, None, atom_node.lineno, atom_node.column)
             n_maker_children = len(maker.children)
             if n_maker_children == 1 or maker.children[1].type == tokens.COMMA:
                 elts = []
                 for i in range(0, n_maker_children, 2):
                     elts.append(self.handle_expr(maker.children[i]))
-                return ast.Set(elts, maker.lineno, maker.column)
+                return ast.Set(elts, atom_node.lineno, atom_node.column)
             if maker.children[1].type == syms.comp_for:
                 return self.handle_setcomp(maker)
             if (n_maker_children > 3 and
@@ -1147,7 +1147,7 @@
             for i in range(0, n_maker_children, 4):
                 keys.append(self.handle_expr(maker.children[i]))
                 values.append(self.handle_expr(maker.children[i + 2]))
-            return ast.Dict(keys, values, maker.lineno, maker.column)
+            return ast.Dict(keys, values, atom_node.lineno, atom_node.column)
         elif first_child_type == tokens.BACKQUOTE:
             expr = self.handle_testlist(atom_node.children[1])
             return ast.Repr(expr, atom_node.lineno, atom_node.column)
@@ -1197,7 +1197,8 @@
     def comprehension_helper(self, comp_node,
                              handle_source_expr_meth="handle_expr",
                              for_type=syms.comp_for, if_type=syms.comp_if,
-                             iter_type=syms.comp_iter):
+                             iter_type=syms.comp_iter,
+                             comp_fix_unamed_tuple_location=False):
         handle_source_expression = getattr(self, handle_source_expr_meth)
         fors_count = self.count_comp_fors(comp_node, for_type, if_type)
         comps = []
@@ -1209,8 +1210,15 @@
             if len(for_node.children) == 1:
                 comp = ast.comprehension(for_targets[0], expr, None)
             else:
-                target = ast.Tuple(for_targets, ast.Store, comp_node.lineno,
-                                   comp_node.column)
+                col = comp_node.column
+                line = comp_node.lineno
+                # Modified in python2.7, see http://bugs.python.org/issue6704
+                if comp_fix_unamed_tuple_location:
+                    expr_node = for_targets[0]
+                    assert isinstance(expr_node, ast.expr)
+                    col = expr_node.col_offset
+                    line = expr_node.lineno
+                target = ast.Tuple(for_targets, ast.Store, line, col)
                 comp = ast.comprehension(target, expr, None)
             if len(comp_node.children) == 5:
                 comp_node = comp_iter = comp_node.children[4]
@@ -1232,7 +1240,8 @@
 
     def handle_genexp(self, genexp_node):
         elt = self.handle_expr(genexp_node.children[0])
-        comps = self.comprehension_helper(genexp_node.children[1])
+        comps = self.comprehension_helper(genexp_node.children[1],
+                                          comp_fix_unamed_tuple_location=True)
         return ast.GeneratorExp(elt, comps, genexp_node.lineno,
                                 genexp_node.column)
 
@@ -1241,19 +1250,22 @@
         comps = self.comprehension_helper(listcomp_node.children[1],
                                           "handle_testlist",
                                           syms.list_for, syms.list_if,
-                                          syms.list_iter)
+                                          syms.list_iter,
+                                          comp_fix_unamed_tuple_location=True)
         return ast.ListComp(elt, comps, listcomp_node.lineno,
                             listcomp_node.column)
 
     def handle_setcomp(self, set_maker):
         elt = self.handle_expr(set_maker.children[0])
-        comps = self.comprehension_helper(set_maker.children[1])
+        comps = self.comprehension_helper(set_maker.children[1],
+                                          comp_fix_unamed_tuple_location=True)
         return ast.SetComp(elt, comps, set_maker.lineno, set_maker.column)
 
     def handle_dictcomp(self, dict_maker):
         key = self.handle_expr(dict_maker.children[0])
         value = self.handle_expr(dict_maker.children[2])
-        comps = self.comprehension_helper(dict_maker.children[3])
+        comps = self.comprehension_helper(dict_maker.children[3],
+                                          comp_fix_unamed_tuple_location=True)
         return ast.DictComp(key, value, comps, dict_maker.lineno,
                             dict_maker.column)
 

From commits-noreply at bitbucket.org  Thu Feb 24 18:25:34 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Thu, 24 Feb 2011 18:25:34 +0100 (CET)
Subject: [pypy-svn] pypy default: add an in-progress test
Message-ID: <20110224172534.07452282BF2@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42260:69b171e9f11b
Date: 2011-02-24 15:21 +0100
http://bitbucket.org/pypy/pypy/changeset/69b171e9f11b/

Log:	add an in-progress test

diff --git a/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py b/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py
--- a/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py
@@ -70,6 +70,38 @@
             jump(p0, p1, p2, p3, i13, p11)
         """)
 
+
+    def test_recursive_call(self):
+        py.test.skip('in-progress')
+        def fn():
+            def rec(n):
+                if n == 0:
+                    return 0
+                return 1 + rec(n-1)
+            #
+            # this loop is traced and then aborted, because the trace is too
+            # long. But then "rec" is marked as "don't inline"
+            i = 0
+            j = 0
+            while i < 20:
+                i += 1
+                j += rec(100)
+            #
+            # next time we try to trace "rec", instead of inlining we compile
+            # it separately and generate a call_assembler
+            i = 0
+            j = 0
+            while i < 20:
+                i += 1
+                j += rec(100) # ID: call_rec
+                a = 0
+            return j
+        #
+        log = self.run(fn, [], threshold=18)
+        loop, = log.loops_by_filename(self.filepath)
+        import pdb;pdb.set_trace()
+
+
     def test_cmp_exc(self):
         def f1(n):
             # So we don't get a LOAD_GLOBAL op

diff --git a/pypy/module/pypyjit/test_pypy_c/test_model.py b/pypy/module/pypyjit/test_pypy_c/test_model.py
--- a/pypy/module/pypyjit/test_pypy_c/test_model.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_model.py
@@ -18,7 +18,7 @@
     def setup_method(self, meth):
         self.filepath = self.tmpdir.join(meth.im_func.func_name + '.py')
 
-    def run(self, func, args=[], threshold=1000):
+    def run(self, func, args=[], **jitopts):
         # write the snippet
         arglist = ', '.join(map(repr, args))
         with self.filepath.open("w") as f:
@@ -27,10 +27,13 @@
         #
         # run a child pypy-c with logging enabled
         logfile = self.filepath.new(ext='.log')
+        #
+        cmdline = [sys.executable, '-S']
+        for key, value in jitopts.iteritems():
+            cmdline += ['--jit', '%s=%s' % (key, value)]
+        cmdline.append(str(self.filepath))
+        #
         env={'PYPYLOG': 'jit-log-opt,jit-summary:' + str(logfile)}
-        cmdline = [sys.executable, '-S',
-                   '--jit', 'threshold=%d' % threshold,
-                   str(self.filepath)]
         pipe = subprocess.Popen(cmdline,
                                 env=env,
                                 stdout=subprocess.PIPE,

From commits-noreply at bitbucket.org  Thu Feb 24 18:25:35 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Thu, 24 Feb 2011 18:25:35 +0100 (CET)
Subject: [pypy-svn] pypy default: start to write unit tests for match()
Message-ID: <20110224172535.CB692282BF2@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42261:ceaa2b31c3c8
Date: 2011-02-24 15:49 +0100
http://bitbucket.org/pypy/pypy/changeset/ceaa2b31c3c8/

Log:	start to write unit tests for match()

diff --git a/pypy/module/pypyjit/test_pypy_c/model.py b/pypy/module/pypyjit/test_pypy_c/model.py
--- a/pypy/module/pypyjit/test_pypy_c/model.py
+++ b/pypy/module/pypyjit/test_pypy_c/model.py
@@ -178,7 +178,8 @@
         args = map(str.strip, args)
         return opname, resvar, args
 
-    def preprocess_expected_src(self, src):
+    @classmethod
+    def preprocess_expected_src(cls, src):
         # all loops decrement the tick-counter at the end. The rpython code is
         # in jump_absolute() in pypyjit/interp.py. The string --TICK-- is
         # replaced with the corresponding operations, so that tests don't have
@@ -206,11 +207,12 @@
             return alpha_map[v1] == v2
         return match_var
 
-    def match_ops(self, ops, expected_src):
-        expected_src = self.preprocess_expected_src(expected_src)
-        match_var = self._get_match_var()
+    @classmethod
+    def match_ops(cls, ops, expected_src):
+        expected_src = cls.preprocess_expected_src(expected_src)
+        match_var = cls._get_match_var()
         #
-        expected_ops = self.parse_ops(expected_src)
+        expected_ops = cls.parse_ops(expected_src)
         assert len(ops) == len(expected_ops), "wrong number of operations"
         for op, (exp_opname, exp_res, exp_args) in zip(ops, expected_ops):
             assert op.name == exp_opname

diff --git a/pypy/module/pypyjit/test_pypy_c/test_model.py b/pypy/module/pypyjit/test_pypy_c/test_model.py
--- a/pypy/module/pypyjit/test_pypy_c/test_model.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_model.py
@@ -79,6 +79,17 @@
         opcodes_names = [opcode.__class__.__name__ for opcode in myline]
         assert opcodes_names == ['LOAD_FAST', 'LOAD_CONST', 'BINARY_ADD', 'STORE_FAST']
 
+class TestMath(object):
+
+    def match(self, src1, src2):
+        """Wrapper around LoopWithIds.match_ops"""
+        from pypy.tool.jitlogparser.parser import parse
+        loop = parse(src1)
+        try:
+            return LoopWithIds.match_ops(loop.operations, src2)
+        except AssertionError:
+            return False
+
     def test_match_var(self):
         match_var = LoopWithIds._get_match_var()
         assert match_var('v0', 'V0')
@@ -95,6 +106,25 @@
         assert not match_var('ConstClass(bar)', 'v1')
         assert not match_var('v2', 'ConstClass(baz)')
 
+    def test_match(self):
+        loop = """
+            [i0]
+            i2 = int_add(i0, 1)
+            jump(i2)
+        """
+        expected = """
+            i5 = int_add(i2, 1)
+            jump(i5)
+        """
+        assert self.match(loop, expected)
+        #
+        expected = """
+            i5 = int_sub(i2, 1)
+            jump(i5)
+        """
+        assert not self.match(loop, expected)
+
+
 class TestRunPyPyC(BaseTestPyPyC):
 
     def test_run_function(self):

From commits-noreply at bitbucket.org  Thu Feb 24 18:25:36 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Thu, 24 Feb 2011 18:25:36 +0100 (CET)
Subject: [pypy-svn] pypy default: move all the matching-related functions to
	theirs own	class, to ease testing
Message-ID: <20110224172536.98A0B282BF2@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42262:45839e0a80b2
Date: 2011-02-24 16:18 +0100
http://bitbucket.org/pypy/pypy/changeset/45839e0a80b2/

Log:	move all the matching-related functions to theirs own class, to ease
	testing

diff --git a/pypy/module/pypyjit/test_pypy_c/model.py b/pypy/module/pypyjit/test_pypy_c/model.py
--- a/pypy/module/pypyjit/test_pypy_c/model.py
+++ b/pypy/module/pypyjit/test_pypy_c/model.py
@@ -150,6 +150,22 @@
                 for op in self._ops_for_chunk(chunk, include_debug_merge_points):
                     yield op
 
+    def match(self, expected_src):
+        ops = list(self.allops())
+        matcher = OpMatcher(ops)
+        return matcher.match(expected_src)
+
+    def match_by_id(self, id, expected_src):
+        ops = list(self.ops_by_id(id))
+        matcher = OpMatcher(ops)
+        return matcher.match(expected_src)
+
+
+class OpMatcher(object):
+
+    def __init__(self, ops):
+        self.ops = ops
+        self.alpha_map = None
 
     @classmethod
     def parse_ops(cls, src):
@@ -207,14 +223,13 @@
             return alpha_map[v1] == v2
         return match_var
 
-    @classmethod
-    def match_ops(cls, ops, expected_src):
-        expected_src = cls.preprocess_expected_src(expected_src)
-        match_var = cls._get_match_var()
+    def match(self, expected_src):
+        expected_src = self.preprocess_expected_src(expected_src)
+        match_var = self._get_match_var()
         #
-        expected_ops = cls.parse_ops(expected_src)
-        assert len(ops) == len(expected_ops), "wrong number of operations"
-        for op, (exp_opname, exp_res, exp_args) in zip(ops, expected_ops):
+        expected_ops = self.parse_ops(expected_src)
+        assert len(self.ops) == len(expected_ops), "wrong number of operations"
+        for op, (exp_opname, exp_res, exp_args) in zip(self.ops, expected_ops):
             assert op.name == exp_opname
             match_var(op.res, exp_res)
             assert len(op.args) == len(exp_args), "wrong number of arguments"
@@ -222,10 +237,3 @@
                 assert match_var(arg, exp_arg), "variable mismatch"
         return True
 
-    def match(self, expected_src):
-        ops = list(self.allops())
-        return self.match_ops(ops, expected_src)
-
-    def match_by_id(self, id, expected_src):
-        ops = list(self.ops_by_id(id))
-        return self.match_ops(ops, expected_src)

diff --git a/pypy/module/pypyjit/test_pypy_c/test_model.py b/pypy/module/pypyjit/test_pypy_c/test_model.py
--- a/pypy/module/pypyjit/test_pypy_c/test_model.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_model.py
@@ -4,7 +4,8 @@
 from lib_pypy import disassembler
 from pypy.tool.udir import udir
 from pypy.tool import logparser
-from pypy.module.pypyjit.test_pypy_c.model import Log, find_ids_range, find_ids, LoopWithIds
+from pypy.module.pypyjit.test_pypy_c.model import Log, find_ids_range, find_ids, \
+    LoopWithIds, OpMatcher
 
 class BaseTestPyPyC(object):
     def setup_class(cls):
@@ -79,19 +80,20 @@
         opcodes_names = [opcode.__class__.__name__ for opcode in myline]
         assert opcodes_names == ['LOAD_FAST', 'LOAD_CONST', 'BINARY_ADD', 'STORE_FAST']
 
-class TestMath(object):
+class TestOpMatcher(object):
 
     def match(self, src1, src2):
         """Wrapper around LoopWithIds.match_ops"""
         from pypy.tool.jitlogparser.parser import parse
         loop = parse(src1)
+        matcher = OpMatcher(loop.operations)
         try:
-            return LoopWithIds.match_ops(loop.operations, src2)
+            return matcher.match(src2)
         except AssertionError:
             return False
 
     def test_match_var(self):
-        match_var = LoopWithIds._get_match_var()
+        match_var = OpMatcher._get_match_var()
         assert match_var('v0', 'V0')
         assert not match_var('v0', 'V1')
         assert match_var('v0', 'V0')
@@ -106,7 +108,13 @@
         assert not match_var('ConstClass(bar)', 'v1')
         assert not match_var('v2', 'ConstClass(baz)')
 
-    def test_match(self):
+    def test_parse_op(self):
+        res = OpMatcher.parse_op("  a =   int_add(  b,  3 ) # foo")
+        assert res == ("int_add", "a", ["b", "3"])
+        res = OpMatcher.parse_op("guard_true(a)")
+        assert res == ("guard_true", None, ["a"])
+
+    def test_exact_match(self):
         loop = """
             [i0]
             i2 = int_add(i0, 1)
@@ -123,6 +131,32 @@
             jump(i5)
         """
         assert not self.match(loop, expected)
+        #
+        expected = """
+            i5 = int_sub(i2, 1)
+            jump(i5)
+            extra_stuff(i5)
+        """
+        assert not self.match(loop, expected)
+
+
+    def test_partial_match(self):
+        py.test.skip('in-progress')
+        loop = """
+            [i0]
+            i1 = int_add(i0, 1)
+            i2 = int_sub(i1, 10)
+            i3 = int_floordiv(i2, 100)
+            i4 = int_mul(i1, 1000)
+            jump(i3)
+        """
+        expected = """
+            i1 = int_add(0, 1)
+            ...
+            i4 = int_mul(i1, 1000)
+        """
+        assert self.match(loop, expected)
+
 
 
 class TestRunPyPyC(BaseTestPyPyC):
@@ -235,12 +269,6 @@
             'jump'
             ]
 
-    def test_parse_op(self):
-        res = LoopWithIds.parse_op("  a =   int_add(  b,  3 ) # foo")
-        assert res == ("int_add", "a", ["b", "3"])
-        res = LoopWithIds.parse_op("guard_true(a)")
-        assert res == ("guard_true", None, ["a"])
-
     def test_match(self):
         def f():
             i = 0

From commits-noreply at bitbucket.org  Thu Feb 24 18:25:39 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Thu, 24 Feb 2011 18:25:39 +0100 (CET)
Subject: [pypy-svn] pypy default: make match_var an instance method,
	instead of a closure over alpha_map
Message-ID: <20110224172539.5242B2A2034@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42263:2a643e0b3484
Date: 2011-02-24 16:24 +0100
http://bitbucket.org/pypy/pypy/changeset/2a643e0b3484/

Log:	make match_var an instance method, instead of a closure over
	alpha_map

diff --git a/pypy/module/pypyjit/test_pypy_c/model.py b/pypy/module/pypyjit/test_pypy_c/model.py
--- a/pypy/module/pypyjit/test_pypy_c/model.py
+++ b/pypy/module/pypyjit/test_pypy_c/model.py
@@ -165,7 +165,7 @@
 
     def __init__(self, ops):
         self.ops = ops
-        self.alpha_map = None
+        self.alpha_map = {}
 
     @classmethod
     def parse_ops(cls, src):
@@ -211,29 +211,26 @@
         return src
 
     @classmethod
-    def _get_match_var(cls):
-        def is_const(v1):
-            return isinstance(v1, str) and v1.startswith('ConstClass(')
-        alpha_map = {}
-        def match_var(v1, v2):
-            if is_const(v1) or is_const(v2):
-                return v1 == v2
-            if v1 not in alpha_map:
-                alpha_map[v1] = v2
-            return alpha_map[v1] == v2
-        return match_var
+    def is_const(cls, v1):
+        return isinstance(v1, str) and v1.startswith('ConstClass(')
+    
+    def match_var(self, v1, v2):
+        if self.is_const(v1) or self.is_const(v2):
+            return v1 == v2
+        if v1 not in self.alpha_map:
+            self.alpha_map[v1] = v2
+        return self.alpha_map[v1] == v2
 
     def match(self, expected_src):
         expected_src = self.preprocess_expected_src(expected_src)
-        match_var = self._get_match_var()
         #
         expected_ops = self.parse_ops(expected_src)
         assert len(self.ops) == len(expected_ops), "wrong number of operations"
         for op, (exp_opname, exp_res, exp_args) in zip(self.ops, expected_ops):
             assert op.name == exp_opname
-            match_var(op.res, exp_res)
+            self.match_var(op.res, exp_res)
             assert len(op.args) == len(exp_args), "wrong number of arguments"
             for arg, exp_arg in zip(op.args, exp_args):
-                assert match_var(arg, exp_arg), "variable mismatch"
+                assert self.match_var(arg, exp_arg), "variable mismatch"
         return True
 

diff --git a/pypy/module/pypyjit/test_pypy_c/test_model.py b/pypy/module/pypyjit/test_pypy_c/test_model.py
--- a/pypy/module/pypyjit/test_pypy_c/test_model.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_model.py
@@ -93,7 +93,7 @@
             return False
 
     def test_match_var(self):
-        match_var = OpMatcher._get_match_var()
+        match_var = OpMatcher([]).match_var
         assert match_var('v0', 'V0')
         assert not match_var('v0', 'V1')
         assert match_var('v0', 'V0')

From commits-noreply at bitbucket.org  Thu Feb 24 18:25:40 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Thu, 24 Feb 2011 18:25:40 +0100 (CET)
Subject: [pypy-svn] pypy default: factor out match_op
Message-ID: <20110224172540.591ED2A2034@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42264:7bcd7ca4395b
Date: 2011-02-24 16:25 +0100
http://bitbucket.org/pypy/pypy/changeset/7bcd7ca4395b/

Log:	factor out match_op

diff --git a/pypy/module/pypyjit/test_pypy_c/model.py b/pypy/module/pypyjit/test_pypy_c/model.py
--- a/pypy/module/pypyjit/test_pypy_c/model.py
+++ b/pypy/module/pypyjit/test_pypy_c/model.py
@@ -221,16 +221,20 @@
             self.alpha_map[v1] = v2
         return self.alpha_map[v1] == v2
 
+    def match_op(self, op, (exp_opname, exp_res, exp_args)):
+        assert op.name == exp_opname
+        self.match_var(op.res, exp_res)
+        assert len(op.args) == len(exp_args), "wrong number of arguments"
+        for arg, exp_arg in zip(op.args, exp_args):
+            assert self.match_var(arg, exp_arg), "variable mismatch"
+        
+
     def match(self, expected_src):
         expected_src = self.preprocess_expected_src(expected_src)
         #
         expected_ops = self.parse_ops(expected_src)
         assert len(self.ops) == len(expected_ops), "wrong number of operations"
-        for op, (exp_opname, exp_res, exp_args) in zip(self.ops, expected_ops):
-            assert op.name == exp_opname
-            self.match_var(op.res, exp_res)
-            assert len(op.args) == len(exp_args), "wrong number of arguments"
-            for arg, exp_arg in zip(op.args, exp_args):
-                assert self.match_var(arg, exp_arg), "variable mismatch"
+        for op, exp_op in zip(self.ops, expected_ops):
+            self.match_op(op, exp_op)
         return True
 

From commits-noreply at bitbucket.org  Thu Feb 24 18:25:41 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Thu, 24 Feb 2011 18:25:41 +0100 (CET)
Subject: [pypy-svn] pypy default: use an Exception to signal match failure
Message-ID: <20110224172541.C11D32A2034@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42265:10fbd717649b
Date: 2011-02-24 16:55 +0100
http://bitbucket.org/pypy/pypy/changeset/10fbd717649b/

Log:	use an Exception to signal match failure

diff --git a/pypy/module/pypyjit/test_pypy_c/model.py b/pypy/module/pypyjit/test_pypy_c/model.py
--- a/pypy/module/pypyjit/test_pypy_c/model.py
+++ b/pypy/module/pypyjit/test_pypy_c/model.py
@@ -160,6 +160,8 @@
         matcher = OpMatcher(ops)
         return matcher.match(expected_src)
 
+class InvalidMatch(Exception):
+    pass
 
 class OpMatcher(object):
 
@@ -221,20 +223,30 @@
             self.alpha_map[v1] = v2
         return self.alpha_map[v1] == v2
 
+    def _assert(self, cond, message):
+        if not cond:
+            raise InvalidMatch(message)
+
     def match_op(self, op, (exp_opname, exp_res, exp_args)):
-        assert op.name == exp_opname
+        self._assert(op.name == exp_opname, "operation mismatch")
         self.match_var(op.res, exp_res)
-        assert len(op.args) == len(exp_args), "wrong number of arguments"
+        self._assert(len(op.args) == len(exp_args), "wrong number of arguments")
         for arg, exp_arg in zip(op.args, exp_args):
-            assert self.match_var(arg, exp_arg), "variable mismatch"
-        
+            self._assert(self.match_var(arg, exp_arg), "variable mismatch")
+
+    def match_loop(self, expected_ops):
+        self._assert(len(self.ops) == len(expected_ops), "wrong number of operations")
+        for op, exp_op in zip(self.ops, expected_ops):
+            self.match_op(op, exp_op)
 
     def match(self, expected_src):
         expected_src = self.preprocess_expected_src(expected_src)
-        #
         expected_ops = self.parse_ops(expected_src)
-        assert len(self.ops) == len(expected_ops), "wrong number of operations"
-        for op, exp_op in zip(self.ops, expected_ops):
-            self.match_op(op, exp_op)
-        return True
+        try:
+            self.match_loop(expected_ops)
+        except InvalidMatch:
+            #raise # uncomment this and use py.test --pdb for better debugging
+            return False
+        else:
+            return True
 

diff --git a/pypy/module/pypyjit/test_pypy_c/test_model.py b/pypy/module/pypyjit/test_pypy_c/test_model.py
--- a/pypy/module/pypyjit/test_pypy_c/test_model.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_model.py
@@ -83,14 +83,10 @@
 class TestOpMatcher(object):
 
     def match(self, src1, src2):
-        """Wrapper around LoopWithIds.match_ops"""
         from pypy.tool.jitlogparser.parser import parse
         loop = parse(src1)
         matcher = OpMatcher(loop.operations)
-        try:
-            return matcher.match(src2)
-        except AssertionError:
-            return False
+        return matcher.match(src2)
 
     def test_match_var(self):
         match_var = OpMatcher([]).match_var
@@ -299,7 +295,7 @@
             jump(p0, p1, p2, p3, i8)
         """)
         #
-        py.test.raises(AssertionError, loop.match, """
+        assert not loop.match("""
             i6 = int_lt(i4, 1003)
             guard_true(i6)
             i8 = int_add(i5, 1) # variable mismatch
@@ -341,7 +337,7 @@
             guard_no_exception()
         """)
         #
-        py.test.raises(AssertionError, loop.match_by_id, 'increment', """
+        assert not loop.match_by_id('increment', """
             p12 = call(ConstClass(rbigint.SUB), p4, ConstPtr(ptr11))
             guard_no_exception()
         """)

From commits-noreply at bitbucket.org  Thu Feb 24 18:25:43 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Thu, 24 Feb 2011 18:25:43 +0100 (CET)
Subject: [pypy-svn] pypy default: refactor to have two independent iterators
	to iterate	over the ops and the expected_ops
Message-ID: <20110224172543.8998D282C19@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42266:d4109a613da9
Date: 2011-02-24 17:19 +0100
http://bitbucket.org/pypy/pypy/changeset/d4109a613da9/

Log:	refactor to have two independent iterators to iterate over the ops
	and the expected_ops

diff --git a/pypy/module/pypyjit/test_pypy_c/model.py b/pypy/module/pypyjit/test_pypy_c/model.py
--- a/pypy/module/pypyjit/test_pypy_c/model.py
+++ b/pypy/module/pypyjit/test_pypy_c/model.py
@@ -188,6 +188,8 @@
         line = line.strip()
         if not line:
             return None
+        if line == '...':
+            return line
         opname, _, args = line.partition('(')
         opname = opname.strip()
         assert args.endswith(')')
@@ -234,10 +236,25 @@
         for arg, exp_arg in zip(op.args, exp_args):
             self._assert(self.match_var(arg, exp_arg), "variable mismatch")
 
+    def _next_op(self, iter_ops, message, assert_raises=False):
+        try:
+            op = iter_ops.next()
+        except StopIteration:
+            self._assert(assert_raises, message)
+            return
+        else:
+            self._assert(not assert_raises, message)
+            return op
+
     def match_loop(self, expected_ops):
-        self._assert(len(self.ops) == len(expected_ops), "wrong number of operations")
-        for op, exp_op in zip(self.ops, expected_ops):
+        iter_exp_ops = iter(expected_ops)
+        iter_ops = iter(self.ops)
+        for exp_op in iter_exp_ops:
+            op = self._next_op(iter_ops, "not enough operations")
             self.match_op(op, exp_op)
+        #
+        # make sure we exhausted iter_ops
+        self._next_op(iter_ops, "operation list too long", assert_raises=True)
 
     def match(self, expected_src):
         expected_src = self.preprocess_expected_src(expected_src)

diff --git a/pypy/module/pypyjit/test_pypy_c/test_model.py b/pypy/module/pypyjit/test_pypy_c/test_model.py
--- a/pypy/module/pypyjit/test_pypy_c/test_model.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_model.py
@@ -129,11 +129,17 @@
         assert not self.match(loop, expected)
         #
         expected = """
-            i5 = int_sub(i2, 1)
+            i5 = int_add(i2, 1)
             jump(i5)
             extra_stuff(i5)
         """
         assert not self.match(loop, expected)
+        #
+        expected = """
+            i5 = int_add(i2, 1)
+            # missing op at the end
+        """
+        assert not self.match(loop, expected)
 
 
     def test_partial_match(self):

From commits-noreply at bitbucket.org  Thu Feb 24 18:25:44 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Thu, 24 Feb 2011 18:25:44 +0100 (CET)
Subject: [pypy-svn] pypy default: allow to use '_' as an always-matching
	variable name
Message-ID: <20110224172544.DD79F282C1A@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42267:fe948fe12dc7
Date: 2011-02-24 17:55 +0100
http://bitbucket.org/pypy/pypy/changeset/fe948fe12dc7/

Log:	allow to use '_' as an always-matching variable name

diff --git a/pypy/module/pypyjit/test_pypy_c/model.py b/pypy/module/pypyjit/test_pypy_c/model.py
--- a/pypy/module/pypyjit/test_pypy_c/model.py
+++ b/pypy/module/pypyjit/test_pypy_c/model.py
@@ -218,12 +218,15 @@
     def is_const(cls, v1):
         return isinstance(v1, str) and v1.startswith('ConstClass(')
     
-    def match_var(self, v1, v2):
-        if self.is_const(v1) or self.is_const(v2):
-            return v1 == v2
+    def match_var(self, v1, exp_v2):
+        assert v1 != '_'
+        if exp_v2 == '_':
+            return True
+        if self.is_const(v1) or self.is_const(exp_v2):
+            return v1 == exp_v2
         if v1 not in self.alpha_map:
-            self.alpha_map[v1] = v2
-        return self.alpha_map[v1] == v2
+            self.alpha_map[v1] = exp_v2
+        return self.alpha_map[v1] == exp_v2
 
     def _assert(self, cond, message):
         if not cond:

diff --git a/pypy/module/pypyjit/test_pypy_c/test_model.py b/pypy/module/pypyjit/test_pypy_c/test_model.py
--- a/pypy/module/pypyjit/test_pypy_c/test_model.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_model.py
@@ -103,6 +103,12 @@
         assert match_var('ConstClass(foo)', 'ConstClass(foo)')
         assert not match_var('ConstClass(bar)', 'v1')
         assert not match_var('v2', 'ConstClass(baz)')
+        #
+        # the var '_' matches everything (but only on the right, of course)
+        assert match_var('v0', '_')
+        assert match_var('v0', 'V0')
+        assert match_var('ConstPtr(ptr0)', '_')
+        py.test.raises(AssertionError, "match_var('_', 'v0')")
 
     def test_parse_op(self):
         res = OpMatcher.parse_op("  a =   int_add(  b,  3 ) # foo")
@@ -160,7 +166,6 @@
         assert self.match(loop, expected)
 
 
-
 class TestRunPyPyC(BaseTestPyPyC):
 
     def test_run_function(self):

From commits-noreply at bitbucket.org  Thu Feb 24 18:25:48 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Thu, 24 Feb 2011 18:25:48 +0100 (CET)
Subject: [pypy-svn] pypy default: implement non-greedy partial matching with
	the "..."	operator
Message-ID: <20110224172548.DB10B2A2045@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42268:41520d1f66c3
Date: 2011-02-24 17:56 +0100
http://bitbucket.org/pypy/pypy/changeset/41520d1f66c3/

Log:	implement non-greedy partial matching with the "..." operator

diff --git a/pypy/module/pypyjit/test_pypy_c/model.py b/pypy/module/pypyjit/test_pypy_c/model.py
--- a/pypy/module/pypyjit/test_pypy_c/model.py
+++ b/pypy/module/pypyjit/test_pypy_c/model.py
@@ -239,25 +239,51 @@
         for arg, exp_arg in zip(op.args, exp_args):
             self._assert(self.match_var(arg, exp_arg), "variable mismatch")
 
-    def _next_op(self, iter_ops, message, assert_raises=False):
+    def _next_op(self, iter_ops, assert_raises=False):
         try:
             op = iter_ops.next()
         except StopIteration:
-            self._assert(assert_raises, message)
+            self._assert(assert_raises, "not enough operations")
             return
         else:
-            self._assert(not assert_raises, message)
+            self._assert(not assert_raises, "operation list too long")
             return op
 
+    def match_until(self, until_op, iter_ops):
+        while True:
+            op = self._next_op(iter_ops)
+            try:
+                # try to match the op, but be sure not to modify the
+                # alpha-renaming map in case the match does not work
+                alpha_map = self.alpha_map.copy()
+                self.match_op(op, until_op)
+            except InvalidMatch:
+                # it did not match: rollback the alpha_map, and just skip this
+                # operation
+                self.alpha_map = alpha_map
+            else:
+                # it matched! The '...' operator ends here
+                return op
+
     def match_loop(self, expected_ops):
+        """
+        A note about partial matching: the '...' operator is non-greedy,
+        i.e. it matches all the operations until it finds one that matches
+        what is after the '...'
+        """
         iter_exp_ops = iter(expected_ops)
         iter_ops = iter(self.ops)
         for exp_op in iter_exp_ops:
-            op = self._next_op(iter_ops, "not enough operations")
+            if exp_op == '...':
+                # loop until we find an operation which matches
+                exp_op = iter_exp_ops.next()
+                op = self.match_until(exp_op, iter_ops)
+            else:
+                op = self._next_op(iter_ops)
             self.match_op(op, exp_op)
         #
         # make sure we exhausted iter_ops
-        self._next_op(iter_ops, "operation list too long", assert_raises=True)
+        self._next_op(iter_ops, assert_raises=True)
 
     def match(self, expected_src):
         expected_src = self.preprocess_expected_src(expected_src)

diff --git a/pypy/module/pypyjit/test_pypy_c/test_model.py b/pypy/module/pypyjit/test_pypy_c/test_model.py
--- a/pypy/module/pypyjit/test_pypy_c/test_model.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_model.py
@@ -149,19 +149,19 @@
 
 
     def test_partial_match(self):
-        py.test.skip('in-progress')
         loop = """
             [i0]
             i1 = int_add(i0, 1)
             i2 = int_sub(i1, 10)
             i3 = int_floordiv(i2, 100)
             i4 = int_mul(i1, 1000)
-            jump(i3)
+            jump(i4)
         """
         expected = """
             i1 = int_add(0, 1)
             ...
             i4 = int_mul(i1, 1000)
+            jump(i4)
         """
         assert self.match(loop, expected)
 

From commits-noreply at bitbucket.org  Thu Feb 24 18:25:49 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Thu, 24 Feb 2011 18:25:49 +0100 (CET)
Subject: [pypy-svn] pypy default: a passing test
Message-ID: <20110224172549.D6AB0282BF2@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42269:06cd583a2697
Date: 2011-02-24 17:58 +0100
http://bitbucket.org/pypy/pypy/changeset/06cd583a2697/

Log:	a passing test

diff --git a/pypy/module/pypyjit/test_pypy_c/test_model.py b/pypy/module/pypyjit/test_pypy_c/test_model.py
--- a/pypy/module/pypyjit/test_pypy_c/test_model.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_model.py
@@ -165,6 +165,25 @@
         """
         assert self.match(loop, expected)
 
+    def test_partial_match_is_non_greedy(self):
+        loop = """
+            [i0]
+            i1 = int_add(i0, 1)
+            i2 = int_sub(i1, 10)
+            i3 = int_mul(i2, 1000)
+            i4 = int_mul(i1, 1000)
+            jump(i4)
+        """
+        expected = """
+            i1 = int_add(0, 1)
+            ...
+            _ = int_mul(_, 1000)
+            jump(i4)
+        """
+        # this does not match, because the ... stops at the first int_mul, and
+        # then the second one does not match
+        assert not self.match(loop, expected)
+
 
 class TestRunPyPyC(BaseTestPyPyC):
 

From commits-noreply at bitbucket.org  Thu Feb 24 18:25:51 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Thu, 24 Feb 2011 18:25:51 +0100 (CET)
Subject: [pypy-svn] pypy default: allow to use "..." at the end of the list
Message-ID: <20110224172551.6BF1A282BF2@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42270:8c5655ae5e43
Date: 2011-02-24 18:00 +0100
http://bitbucket.org/pypy/pypy/changeset/8c5655ae5e43/

Log:	allow to use "..." at the end of the list

diff --git a/pypy/module/pypyjit/test_pypy_c/model.py b/pypy/module/pypyjit/test_pypy_c/model.py
--- a/pypy/module/pypyjit/test_pypy_c/model.py
+++ b/pypy/module/pypyjit/test_pypy_c/model.py
@@ -276,7 +276,12 @@
         for exp_op in iter_exp_ops:
             if exp_op == '...':
                 # loop until we find an operation which matches
-                exp_op = iter_exp_ops.next()
+                try:
+                    exp_op = iter_exp_ops.next()
+                except StopIteration:
+                    # the ... is the last line in the expected_ops, so we just
+                    # return because it matches everything until the end
+                    return
                 op = self.match_until(exp_op, iter_ops)
             else:
                 op = self._next_op(iter_ops)

diff --git a/pypy/module/pypyjit/test_pypy_c/test_model.py b/pypy/module/pypyjit/test_pypy_c/test_model.py
--- a/pypy/module/pypyjit/test_pypy_c/test_model.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_model.py
@@ -184,6 +184,21 @@
         # then the second one does not match
         assert not self.match(loop, expected)
 
+    def test_partial_match_at_the_end(self):
+        loop = """
+            [i0]
+            i1 = int_add(i0, 1)
+            i2 = int_sub(i1, 10)
+            i3 = int_floordiv(i2, 100)
+            i4 = int_mul(i1, 1000)
+            jump(i4)
+        """
+        expected = """
+            i1 = int_add(0, 1)
+            ...
+        """
+        assert self.match(loop, expected)
+
 
 class TestRunPyPyC(BaseTestPyPyC):
 

From commits-noreply at bitbucket.org  Thu Feb 24 18:25:53 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Thu, 24 Feb 2011 18:25:53 +0100 (CET)
Subject: [pypy-svn] pypy default: test and fix
Message-ID: <20110224172553.5EAE1282BF2@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42271:2b5341e8ed15
Date: 2011-02-24 18:06 +0100
http://bitbucket.org/pypy/pypy/changeset/2b5341e8ed15/

Log:	test and fix

diff --git a/pypy/module/pypyjit/test_pypy_c/model.py b/pypy/module/pypyjit/test_pypy_c/model.py
--- a/pypy/module/pypyjit/test_pypy_c/model.py
+++ b/pypy/module/pypyjit/test_pypy_c/model.py
@@ -194,8 +194,11 @@
         opname = opname.strip()
         assert args.endswith(')')
         args = args[:-1]
-        args = args.split(',')
-        args = map(str.strip, args)
+        if args:
+            args = args.split(',')
+            args = map(str.strip, args)
+        else:
+            args = []
         return opname, resvar, args
 
     @classmethod

diff --git a/pypy/module/pypyjit/test_pypy_c/test_model.py b/pypy/module/pypyjit/test_pypy_c/test_model.py
--- a/pypy/module/pypyjit/test_pypy_c/test_model.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_model.py
@@ -115,6 +115,8 @@
         assert res == ("int_add", "a", ["b", "3"])
         res = OpMatcher.parse_op("guard_true(a)")
         assert res == ("guard_true", None, ["a"])
+        res = OpMatcher.parse_op("force_token()")
+        assert res == ("force_token", None, [])
 
     def test_exact_match(self):
         loop = """

From commits-noreply at bitbucket.org  Thu Feb 24 18:25:54 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Thu, 24 Feb 2011 18:25:54 +0100 (CET)
Subject: [pypy-svn] pypy default: a test checking that we actually compile
	and	call_assembler recursive functions
Message-ID: <20110224172554.ACC29282BF2@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42272:7ab9cadaadc6
Date: 2011-02-24 18:20 +0100
http://bitbucket.org/pypy/pypy/changeset/7ab9cadaadc6/

Log:	a test checking that we actually compile and call_assembler
	recursive functions

diff --git a/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py b/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py
--- a/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py
@@ -72,7 +72,6 @@
 
 
     def test_recursive_call(self):
-        py.test.skip('in-progress')
         def fn():
             def rec(n):
                 if n == 0:
@@ -99,8 +98,13 @@
         #
         log = self.run(fn, [], threshold=18)
         loop, = log.loops_by_filename(self.filepath)
-        import pdb;pdb.set_trace()
-
+        assert loop.match_by_id('call_rec', """
+            ...
+            p53 = call_assembler(p35, p7, ConstPtr(ptr21), ConstPtr(ptr49), 0, ConstPtr(ptr51), -1, ConstPtr(ptr52), ConstPtr(ptr52), ConstPtr(ptr52), ConstPtr(ptr52), ConstPtr(ptr48))
+            guard_not_forced()
+            guard_no_exception()
+            ...
+        """)
 
     def test_cmp_exc(self):
         def f1(n):

diff --git a/pypy/tool/jitlogparser/parser.py b/pypy/tool/jitlogparser/parser.py
--- a/pypy/tool/jitlogparser/parser.py
+++ b/pypy/tool/jitlogparser/parser.py
@@ -56,6 +56,8 @@
             if args[-1].startswith('descr='):
                 descr = args[-1][len('descr='):]
                 args = args[:-1]
+            if args == ['']:
+                args = []
             return (args, descr)
 
     def box_for_var(self, res):

From commits-noreply at bitbucket.org  Fri Feb 25 01:28:48 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Fri, 25 Feb 2011 01:28:48 +0100 (CET)
Subject: [pypy-svn] pypy default: Use the thread module for test_ctypes.
Message-ID: <20110225002848.CBC7F282BEB@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r42273:b8bb5d085684
Date: 2011-02-25 01:23 +0100
http://bitbucket.org/pypy/pypy/changeset/b8bb5d085684/

Log:	Use the thread module for test_ctypes.

diff --git a/lib-python/conftest.py b/lib-python/conftest.py
--- a/lib-python/conftest.py
+++ b/lib-python/conftest.py
@@ -498,7 +498,7 @@
     RegrTest('test_coding.py'),
     RegrTest('test_complex_args.py'),
     RegrTest('test_contextlib.py', usemodules="thread"),
-    RegrTest('test_ctypes.py', usemodules="_rawffi"),
+    RegrTest('test_ctypes.py', usemodules="_rawffi thread"),
     RegrTest('test_defaultdict.py', usemodules='_collections'),
     RegrTest('test_email_renamed.py'),
     RegrTest('test_exception_variations.py'),

From commits-noreply at bitbucket.org  Fri Feb 25 04:43:14 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Fri, 25 Feb 2011 04:43:14 +0100 (CET)
Subject: [pypy-svn] pypy default: missing docstring for sum
Message-ID: <20110225034314.E580C282B8B@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r42274:babd65a23ee9
Date: 2011-02-24 22:42 -0500
http://bitbucket.org/pypy/pypy/changeset/babd65a23ee9/

Log:	missing docstring for sum

diff --git a/pypy/module/__builtin__/functional.py b/pypy/module/__builtin__/functional.py
--- a/pypy/module/__builtin__/functional.py
+++ b/pypy/module/__builtin__/functional.py
@@ -330,6 +330,11 @@
     return result_w
 
 def sum(space, w_sequence, w_start=None):
+    """sum(sequence[, start]) -> value
+
+Returns the sum of a sequence of numbers (NOT strings) plus the value
+of parameter 'start' (which defaults to 0).  When the sequence is
+empty, returns start."""
     if space.is_w(w_start, space.w_None):
         w_start = space.wrap(0)
     elif space.is_true(space.isinstance(w_start, space.w_basestring)):

From commits-noreply at bitbucket.org  Fri Feb 25 10:27:34 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Fri, 25 Feb 2011 10:27:34 +0100 (CET)
Subject: [pypy-svn] pypy default: make sure to match also the descr of
	operations
Message-ID: <20110225092734.6AD65282B8B@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42275:c3d44e598019
Date: 2011-02-25 10:26 +0100
http://bitbucket.org/pypy/pypy/changeset/c3d44e598019/

Log:	make sure to match also the descr of operations

diff --git a/pypy/module/pypyjit/test_pypy_c/model.py b/pypy/module/pypyjit/test_pypy_c/model.py
--- a/pypy/module/pypyjit/test_pypy_c/model.py
+++ b/pypy/module/pypyjit/test_pypy_c/model.py
@@ -180,8 +180,8 @@
         if '#' in line:
             line = line[:line.index('#')]
         # find the resvar, if any
-        if '=' in line:
-            resvar, _, line = line.partition('=')
+        if ' = ' in line:
+            resvar, _, line = line.partition(' = ')
             resvar = resvar.strip()
         else:
             resvar = None
@@ -194,12 +194,14 @@
         opname = opname.strip()
         assert args.endswith(')')
         args = args[:-1]
-        if args:
-            args = args.split(',')
-            args = map(str.strip, args)
+        args = args.split(',')
+        args = map(str.strip, args)
+        if args[-1].startswith('descr='):
+            descr = args.pop()
+            descr = descr[len('descr='):]
         else:
-            args = []
-        return opname, resvar, args
+            descr = None
+        return opname, resvar, args, descr
 
     @classmethod
     def preprocess_expected_src(cls, src):
@@ -208,11 +210,11 @@
         # replaced with the corresponding operations, so that tests don't have
         # to repeat it every time
         ticker_check = """
-            ticker0 = getfield_raw(ticker_address)
+            ticker0 = getfield_raw(ticker_address, descr=<SignedFieldDescr pypysig_long_struct.c_value 0>)
             ticker1 = int_sub(ticker0, 1)
-            setfield_raw(ticker_address, ticker1)
+            setfield_raw(ticker_address, ticker1, descr=<SignedFieldDescr pypysig_long_struct.c_value 0>)
             ticker_cond = int_lt(ticker1, 0)
-            guard_false(ticker_cond)
+            guard_false(ticker_cond, descr=...)
         """
         src = src.replace('--TICK--', ticker_check)
         return src
@@ -235,12 +237,13 @@
         if not cond:
             raise InvalidMatch(message)
 
-    def match_op(self, op, (exp_opname, exp_res, exp_args)):
+    def match_op(self, op, (exp_opname, exp_res, exp_args, exp_descr)):
         self._assert(op.name == exp_opname, "operation mismatch")
         self.match_var(op.res, exp_res)
         self._assert(len(op.args) == len(exp_args), "wrong number of arguments")
         for arg, exp_arg in zip(op.args, exp_args):
             self._assert(self.match_var(arg, exp_arg), "variable mismatch")
+        self._assert(op.descr == exp_descr or exp_descr == '...', "descr mismatch")
 
     def _next_op(self, iter_ops, assert_raises=False):
         try:

diff --git a/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py b/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py
--- a/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_pypy_c_new.py
@@ -25,14 +25,14 @@
         loop, = log.loops_by_filename(self.filepath)
         assert loop.match("""
             i9 = int_le(i7, i8)
-            guard_true(i9)
+            guard_true(i9, descr=...)
             i11 = int_add_ovf(i7, 1)
-            guard_no_overflow()
+            guard_no_overflow(descr=...)
             i12 = int_and(i8, i11)
             i13 = int_add_ovf(i6, i12)
-            guard_no_overflow()
+            guard_no_overflow(descr=...)
             --TICK--
-            jump(p0, p1, p2, p3, p4, p5, i13, i11, i8)
+            jump(p0, p1, p2, p3, p4, p5, i13, i11, i8, descr=...)
         """)
 
     def test_factorial(self):
@@ -47,12 +47,12 @@
         loop, = log.loops_by_filename(self.filepath)
         assert loop.match("""
             i7 = int_gt(i4, 1)
-            guard_true(i7)
+            guard_true(i7, descr=...)
             i8 = int_mul_ovf(i5, i4)
-            guard_no_overflow()
+            guard_no_overflow(descr=...)
             i10 = int_sub(i4, 1)
             --TICK--
-            jump(p0, p1, p2, p3, i10, i8)
+            jump(p0, p1, p2, p3, i10, i8, descr=...)
         """)
         #
         log = self.run(fact, [25], threshold=20)
@@ -60,14 +60,14 @@
         loop, = log.loops_by_filename(self.filepath)
         assert loop.match("""
             i7 = int_gt(i4, 1)
-            guard_true(i7)
-            p9 = call(ConstClass(fromint), i4)
-            guard_no_exception()
-            p11 = call(ConstClass(rbigint.mul), p5, p9)
-            guard_no_exception()
+            guard_true(i7, descr=...)
+            p9 = call(ConstClass(fromint), i4, descr=...)
+            guard_no_exception(descr=...)
+            p11 = call(ConstClass(rbigint.mul), p5, p9, descr=...)
+            guard_no_exception(descr=...)
             i13 = int_sub(i4, 1)
             --TICK--
-            jump(p0, p1, p2, p3, i13, p11)
+            jump(p0, p1, p2, p3, i13, p11, descr=...)
         """)
 
 
@@ -100,9 +100,9 @@
         loop, = log.loops_by_filename(self.filepath)
         assert loop.match_by_id('call_rec', """
             ...
-            p53 = call_assembler(p35, p7, ConstPtr(ptr21), ConstPtr(ptr49), 0, ConstPtr(ptr51), -1, ConstPtr(ptr52), ConstPtr(ptr52), ConstPtr(ptr52), ConstPtr(ptr52), ConstPtr(ptr48))
-            guard_not_forced()
-            guard_no_exception()
+            p53 = call_assembler(p35, p7, ConstPtr(ptr21), ConstPtr(ptr49), 0, ConstPtr(ptr51), -1, ConstPtr(ptr52), ConstPtr(ptr52), ConstPtr(ptr52), ConstPtr(ptr52), ConstPtr(ptr48), descr=...)
+            guard_not_forced(descr=...)
+            guard_no_exception(descr=...)
             ...
         """)
 

diff --git a/pypy/tool/jitlogparser/test/test_parser.py b/pypy/tool/jitlogparser/test/test_parser.py
--- a/pypy/tool/jitlogparser/test/test_parser.py
+++ b/pypy/tool/jitlogparser/test/test_parser.py
@@ -21,6 +21,7 @@
     assert ops[0].repr() == 'i9 = int_lt(i7, 1003)'
     assert ops[2].descr is not None
     assert len(ops[2].args) == 1
+    assert ops[-1].repr() == 'i13 = getfield_raw(151937600, descr=<SignedFieldDescr pypysig_long_struct.c_value 0>)'
 
 def test_parse_non_code():
     ops = parse('''

diff --git a/pypy/module/pypyjit/test_pypy_c/test_model.py b/pypy/module/pypyjit/test_pypy_c/test_model.py
--- a/pypy/module/pypyjit/test_pypy_c/test_model.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_model.py
@@ -112,11 +112,13 @@
 
     def test_parse_op(self):
         res = OpMatcher.parse_op("  a =   int_add(  b,  3 ) # foo")
-        assert res == ("int_add", "a", ["b", "3"])
+        assert res == ("int_add", "a", ["b", "3"], None)
         res = OpMatcher.parse_op("guard_true(a)")
-        assert res == ("guard_true", None, ["a"])
-        res = OpMatcher.parse_op("force_token()")
-        assert res == ("force_token", None, [])
+        assert res == ("guard_true", None, ["a"], None)
+        res = OpMatcher.parse_op("setfield_gc(p0, i0, descr=<foobar>)")
+        assert res == ("setfield_gc", None, ["p0", "i0"], "<foobar>")
+        res = OpMatcher.parse_op("i1 = getfield_gc(p0, descr=<foobar>)")
+        assert res == ("getfield_gc", "i1", ["p0"], "<foobar>")
 
     def test_exact_match(self):
         loop = """
@@ -126,19 +128,19 @@
         """
         expected = """
             i5 = int_add(i2, 1)
-            jump(i5)
+            jump(i5, descr=...)
         """
         assert self.match(loop, expected)
         #
         expected = """
             i5 = int_sub(i2, 1)
-            jump(i5)
+            jump(i5, descr=...)
         """
         assert not self.match(loop, expected)
         #
         expected = """
             i5 = int_add(i2, 1)
-            jump(i5)
+            jump(i5, descr=...)
             extra_stuff(i5)
         """
         assert not self.match(loop, expected)
@@ -149,6 +151,16 @@
         """
         assert not self.match(loop, expected)
 
+    def test_match_descr(self):
+        loop = """
+            [p0]
+            setfield_gc(p0, 1, descr=<foobar>)
+        """
+        assert self.match(loop, "setfield_gc(p0, 1, descr=<foobar>)")
+        assert self.match(loop, "setfield_gc(p0, 1, descr=...)")
+        assert not self.match(loop, "setfield_gc(p0, 1)")
+        assert not self.match(loop, "setfield_gc(p0, 1, descr=<zzz>)")
+
 
     def test_partial_match(self):
         loop = """
@@ -163,7 +175,7 @@
             i1 = int_add(0, 1)
             ...
             i4 = int_mul(i1, 1000)
-            jump(i4)
+            jump(i4, descr=...)
         """
         assert self.match(loop, expected)
 
@@ -174,13 +186,13 @@
             i2 = int_sub(i1, 10)
             i3 = int_mul(i2, 1000)
             i4 = int_mul(i1, 1000)
-            jump(i4)
+            jump(i4, descr=...)
         """
         expected = """
             i1 = int_add(0, 1)
             ...
             _ = int_mul(_, 1000)
-            jump(i4)
+            jump(i4, descr=...)
         """
         # this does not match, because the ... stops at the first int_mul, and
         # then the second one does not match
@@ -312,7 +324,7 @@
             'jump'
             ]
 
-    def test_match(self):
+    def test_loop_match(self):
         def f():
             i = 0
             while i < 1003:
@@ -323,23 +335,23 @@
         loop, = log.loops_by_id('increment')
         assert loop.match("""
             i6 = int_lt(i4, 1003)
-            guard_true(i6)
+            guard_true(i6, descr=...)
             i8 = int_add(i4, 1)
             # signal checking stuff
-            i10 = getfield_raw(37212896)
+            i10 = getfield_raw(37212896, descr=<SignedFieldDescr pypysig_long_struct.c_value 0>)
             i12 = int_sub(i10, 1)
-            setfield_raw(37212896, i12)
+            setfield_raw(37212896, i12, descr=<SignedFieldDescr pypysig_long_struct.c_value 0>)
             i14 = int_lt(i12, 0)
-            guard_false(i14)
-            jump(p0, p1, p2, p3, i8)
+            guard_false(i14, descr=...)
+            jump(p0, p1, p2, p3, i8, descr=...)
         """)
         #
         assert loop.match("""
             i6 = int_lt(i4, 1003)
-            guard_true(i6)
+            guard_true(i6, descr=...)
             i8 = int_add(i4, 1)
             --TICK--
-            jump(p0, p1, p2, p3, i8)
+            jump(p0, p1, p2, p3, i8, descr=...)
         """)
         #
         assert not loop.match("""
@@ -347,7 +359,7 @@
             guard_true(i6)
             i8 = int_add(i5, 1) # variable mismatch
             --TICK--
-            jump(p0, p1, p2, p3, i8)
+            jump(p0, p1, p2, p3, i8, descr=...)
         """)
 
     def test_match_by_id(self):
@@ -367,7 +379,7 @@
         """)
         assert loop.match_by_id('product', """
             i4 = int_sub_ovf(i3, 1)
-            guard_no_overflow()
+            guard_no_overflow(descr=...)
         """)
 
     def test_match_constants(self):
@@ -380,12 +392,12 @@
         log = self.run(f)
         loop, = log.loops_by_id('increment')
         assert loop.match_by_id('increment', """
-            p12 = call(ConstClass(rbigint.add), p4, ConstPtr(ptr11))
-            guard_no_exception()
+            p12 = call(ConstClass(rbigint.add), p4, ConstPtr(ptr11), descr=...)
+            guard_no_exception(descr=...)
         """)
         #
         assert not loop.match_by_id('increment', """
-            p12 = call(ConstClass(rbigint.SUB), p4, ConstPtr(ptr11))
-            guard_no_exception()
+            p12 = call(ConstClass(rbigint.SUB), p4, ConstPtr(ptr11), descr=...)
+            guard_no_exception(descr=...)
         """)
         

diff --git a/pypy/tool/jitlogparser/parser.py b/pypy/tool/jitlogparser/parser.py
--- a/pypy/tool/jitlogparser/parser.py
+++ b/pypy/tool/jitlogparser/parser.py
@@ -33,7 +33,10 @@
         return self._is_guard
 
     def repr(self):
-        arglist = ', '.join(self.getargs())
+        args = self.getargs()
+        if self.descr is not None:
+            args.append('descr=%s' % self.descr)
+        arglist = ', '.join(args)
         if self.res is not None:
             return '%s = %s(%s)' % (self.getres(), self.name, arglist)
         else:

From commits-noreply at bitbucket.org  Fri Feb 25 12:24:48 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Fri, 25 Feb 2011 12:24:48 +0100 (CET)
Subject: [pypy-svn] pypy arm-backend-2: fix and extend tests
Message-ID: <20110225112448.BFDD7282B8B@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backend-2
Changeset: r42276:561811f72698
Date: 2011-02-24 11:04 +0100
http://bitbucket.org/pypy/pypy/changeset/561811f72698/

Log:	fix and extend tests

diff --git a/pypy/jit/backend/arm/test/test_regalloc.py b/pypy/jit/backend/arm/test/test_regalloc.py
--- a/pypy/jit/backend/arm/test/test_regalloc.py
+++ b/pypy/jit/backend/arm/test/test_regalloc.py
@@ -431,6 +431,7 @@
     A = lltype.GcArray(lltype.Char)
     I = lltype.GcArray(lltype.Signed)
     arraydescr = cpu.arraydescrof(A)
+    arraydescr_i = cpu.arraydescrof(I)
 
     namespace = locals().copy()
 
@@ -507,7 +508,7 @@
     def test_setarrayitem3_gc(self):
         ops = '''
         [p0, i0, i1]
-        setarrayitem_gc(p0, i1, i0, descr=arraydescr)
+        setarrayitem_gc(p0, i1, i0, descr=arraydescr_i)
         finish()
         '''
         s = lltype.malloc(self.I, 3)
@@ -517,7 +518,7 @@
     def test_setarrayitem4_gc(self):
         ops = '''
         [p0, i0]
-        setarrayitem_gc(p0, 1, i0, descr=arraydescr)
+        setarrayitem_gc(p0, 1, i0, descr=arraydescr_i)
         finish()
         '''
         s = lltype.malloc(self.I, 3)
@@ -670,6 +671,7 @@
         self.run(loop)
         assert self.getint(0) == 29
 
+class TestJumps(BaseTestRegalloc):
     def test_jump_with_consts(self):
         loop = """
         [i0, i1, i2, i3, i4, i5, i6, i7, i8, i9, i10, i11, i12, i13, i14]
@@ -681,7 +683,7 @@
 
     def test_from_loop_to_loop(self):
         def assembler_helper(failindex, virtualizable):
-            return 1
+            return 3
 
         FUNCPTR = lltype.Ptr(lltype.FuncType([lltype.Signed, llmemory.GCREF],
                                              lltype.Signed))
@@ -696,22 +698,23 @@
         loop1 = """
         [i0, i1, i2, i3, i4, i5, i6, i7, i8, i9, i10]
         i11 = int_add(i0, i1)
-        finish(i0, i1, i2, i3, i4, i5, i6, i7, i8, i9, i10, i11)
+        finish(i11, i0, i1, i2, i3, i4, i5, i6, i7, i8, i9, i10)
         """
         large = self.interpret(loop1, range(11), run=False)
         large.token.outermost_jitdriver_sd = FakeJitDriverSD()
         self.namespace['looptoken'] = large.token
         assert self.namespace['looptoken']._arm_bootstrap_code != 0
         loop2 = """
-       [i0]
+        [i0]
         i1 = force_token()
-        call_assembler(1,2,3,4,5,6,7,8,9,10,11, descr=looptoken)
+        i2 = call_assembler(1,2,3,4,5,6,7,8,9,10,11, descr=looptoken)
         guard_not_forced() [i0]
-        finish(i0)
+        finish(i0, i2)
         """
 
         self.interpret(loop2, [110])
-        assert self.getint(0) == 0
+        assert self.getint(0) == 110
+        assert self.getint(1) == 3
 
     def test_far_far_jump(self):
         ops = """
@@ -740,3 +743,43 @@
         """
         self.interpret(ops, range(11))
         assert self.getint(0) == 2 # and not segfault()
+
+class TestStrOps(BaseTestRegalloc):
+    def test_newstr(self):
+        ops = """
+        [i0]
+        p1 = newstr(300)
+        i2 = strlen(p1)
+        finish(i2)
+        """
+        self.interpret(ops, [0])
+        assert self.getint(0) == 300
+        ops = """
+        [i0]
+        p1 = newstr(i0)
+        i2 = strlen(p1)
+        finish(i2)
+        """
+        self.interpret(ops, [300])
+        assert self.getint(0) == 300
+
+    def test_strlen(self):
+        s = rstr.mallocstr(300)
+        ops = """
+        [p0]
+        i1 = strlen(p0)
+        finish(i1)
+        """
+        self.interpret(ops, [s])
+        assert self.getint(0) == 300
+
+    def test_len_of_newstr(self):
+        ops = """
+        []
+        p0 = newstr(300)
+        finish(p0)
+        """
+        self.interpret(ops, [])
+        string = self.getptr(0, lltype.Ptr(rstr.STR))
+        assert len(string.chars) == 300
+

From commits-noreply at bitbucket.org  Fri Feb 25 12:25:01 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Fri, 25 Feb 2011 12:25:01 +0100 (CET)
Subject: [pypy-svn] pypy arm-backend-2: merge default
Message-ID: <20110225112501.91ABB282B90@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backend-2
Changeset: r42277:18ea8abbee36
Date: 2011-02-24 15:32 +0100
http://bitbucket.org/pypy/pypy/changeset/18ea8abbee36/

Log:	merge default

diff --git a/pypy/module/_rawffi/error.py b/pypy/module/_rawffi/error.py
deleted file mode 100644
--- a/pypy/module/_rawffi/error.py
+++ /dev/null
@@ -1,2 +0,0 @@
-class SegfaultException(Exception):
-    pass

diff --git a/pypy/translator/goal/app_main.py b/pypy/translator/goal/app_main.py
--- a/pypy/translator/goal/app_main.py
+++ b/pypy/translator/goal/app_main.py
@@ -507,6 +507,8 @@
             success = run_toplevel(run_it)
         elif run_module:
             # handle the "-m" command
+            # '' on sys.path is required also here
+            sys.path.insert(0, '')
             import runpy
             success = run_toplevel(runpy._run_module_as_main, sys.argv[0])
         elif run_stdin:

diff --git a/pypy/translator/goal/test2/test_app_main.py b/pypy/translator/goal/test2/test_app_main.py
--- a/pypy/translator/goal/test2/test_app_main.py
+++ b/pypy/translator/goal/test2/test_app_main.py
@@ -435,6 +435,8 @@
         child.expect('True')
         child.sendline('"pypy.translator.goal.test2.mymodule" in sys.modules')
         child.expect('False')
+        child.sendline('sys.path[0]')
+        child.expect("''")
 
     def test_option_i_noexit(self):
         child = self.spawn(['-i', '-c', 'import sys; sys.exit(1)'])

diff --git a/pypy/module/_socket/app_socket.py b/pypy/module/_socket/app_socket.py
deleted file mode 100644
--- a/pypy/module/_socket/app_socket.py
+++ /dev/null
@@ -1,15 +0,0 @@
-"""Implementation module for socket operations.
-
-See the socket module for documentation."""
-
-class error(IOError):
-    pass
-
-class herror(error):
-    pass
-
-class gaierror(error):
-    pass
-
-class timeout(error):
-    pass

diff --git a/pypy/jit/codewriter/jtransform.py b/pypy/jit/codewriter/jtransform.py
--- a/pypy/jit/codewriter/jtransform.py
+++ b/pypy/jit/codewriter/jtransform.py
@@ -834,8 +834,10 @@
                 op2 = self._handle_oopspec_call(op1, args,
                                                 EffectInfo.OS_LLONG_%s,
                                                 EffectInfo.EF_PURE)
+                if %r == "TO_INT":
+                    assert op2.result.concretetype == lltype.Signed
                 return op2
-        ''' % (_op, _oopspec.lower(), _oopspec)).compile()
+        ''' % (_op, _oopspec.lower(), _oopspec, _oopspec)).compile()
 
     def _normalize(self, oplist):
         if isinstance(oplist, SpaceOperation):
@@ -873,14 +875,24 @@
             args = op.args
             if fromll:
                 opname = 'truncate_longlong_to_int'
+                RESULT = lltype.Signed
             else:
                 from pypy.rpython.lltypesystem import rffi
                 if rffi.cast(op.args[0].concretetype, -1) < 0:
                     opname = 'cast_int_to_longlong'
                 else:
                     opname = 'cast_uint_to_longlong'
-            op1 = SpaceOperation(opname, args, op.result)
-            return self.rewrite_operation(op1)
+                RESULT = lltype.SignedLongLong
+            v = varoftype(RESULT)
+            op1 = SpaceOperation(opname, args, v)
+            op2 = self.rewrite_operation(op1)
+            #
+            # force a renaming to put the correct result in place, even though
+            # it might be slightly mistyped (e.g. Signed versus Unsigned)
+            assert op2.result is v
+            op2.result = op.result
+            #
+            return op2
 
     # ----------
     # Renames, from the _old opname to the _new one.

diff --git a/pypy/module/thread/app_thread.py b/pypy/module/thread/app_thread.py
deleted file mode 100644
--- a/pypy/module/thread/app_thread.py
+++ /dev/null
@@ -1,7 +0,0 @@
-class error(Exception):
-    pass
-
-def exit():
-    """This is synonymous to ``raise SystemExit''.  It will cause the current
-thread to exit silently unless the exception is caught."""
-    raise SystemExit

diff --git a/pypy/module/mmap/app_mmap.py b/pypy/module/mmap/app_mmap.py
deleted file mode 100644
--- a/pypy/module/mmap/app_mmap.py
+++ /dev/null
@@ -1,6 +0,0 @@
-ACCESS_READ = 1
-ACCESS_WRITE = 2
-ACCESS_COPY = 3
-
-class error(EnvironmentError):
-    pass

diff --git a/pypy/module/pypyjit/test/test_pypy_c_new.py b/pypy/module/pypyjit/test/test_pypy_c_new.py
deleted file mode 100644
--- a/pypy/module/pypyjit/test/test_pypy_c_new.py
+++ /dev/null
@@ -1,99 +0,0 @@
-
-import py, sys, re
-py.test.skip("in-progress?")
-import subprocess
-import disassembler
-from pypy.tool.udir import udir
-from pypy.tool import logparser
-
-class Trace(object):
-    pass
-
-class BaseTestPyPyC(object):
-    def setup_class(cls):
-        cls.tmpdir = udir.join('test-pypy-jit')
-        cls.tmpdir.ensure(dir=True)
-
-    def setup_method(self, meth):
-        self.filepath = self.tmpdir.join(meth.im_func.func_name + '.py')
-
-    def parse_out(self, out):
-        out = out.strip("\n")
-        if out == 'None':
-            return None
-        try:
-            return int(out)
-        except ValueError:
-            return out
-
-    def parse_func(self, func):
-        # find lines such as # LOOP <name> is in a line
-        code = disassembler.dis(func)
-        result = {}
-        for i, line in enumerate(py.code.Source(func)):
-            m = re.search('# LOOP (\w+)', line)
-            if m:
-                name = m.group(1)
-                result[name] = []
-                for opcode in code.opcodes:
-                    no = opcode.lineno - func.func_code.co_firstlineno
-                    if i - 1 <= no <= i + 1:
-                        result[name].append(opcode)
-        return result
-    
-    def run(self, func):
-        with self.filepath.open("w") as f:
-            f.write(str(py.code.Source(func)) + "\n")
-            f.write("print %s()\n" % func.func_name)
-        logfile = self.filepath.new(ext='.log')
-        pipe = subprocess.Popen([sys.executable, str(self.filepath)],
-                                stdout=subprocess.PIPE, stderr=subprocess.PIPE,
-                                env={'PYPYLOG': "jit-log-opt,jit-summary:" + str(logfile)})
-        pipe.wait()
-        stderr = pipe.stderr.read()
-        assert not stderr
-        res = self.parse_out(pipe.stdout.read())
-        bytecodes = self.parse_func(func)
-        assert res == func()
-        log = logparser.parse_log_file(str(logfile))
-        parts = logparser.extract_category(log, 'jit-log-opt-')
-        log.xxx
-        return Trace()
-
-class TestInfrastructure(BaseTestPyPyC):
-    def test_parse_func(self):
-        def f():
-            i = 0
-            x = 0
-            # LOOP name
-            z = x + 3
-            return z
-
-        res = self.parse_func(f)
-        assert len(res) == 1
-        assert len(res['name']) == 6
-
-    def test_full(self):
-        def f():
-            i = 0
-            while i < 1003:
-                # LOOP one
-                i += 1
-
-        trace = self.run(f)
-        loop = trace.get_loops('one')
-        loop.get_bytecode(3, 'LOAD_FAST').match('''
-        int_add
-        guard_true
-        ''')
-        loop.get_bytecode(4, 'LOAD_CONST').match_stats(
-            guard='3', call='1-2', call_may_force='0'
-        )
-        # this would make operations that are "costly" obligatory to pass
-        # like new
-        loo.get_bytecode(5, 'INPLACE_ADD').match_stats(
-            allocs='5-10'
-            )
-
-class TestPyPyCNew(BaseTestPyPyC):
-    pass

diff --git a/pypy/jit/tool/pypytrace.vim b/pypy/jit/tool/pypytrace.vim
--- a/pypy/jit/tool/pypytrace.vim
+++ b/pypy/jit/tool/pypytrace.vim
@@ -1,5 +1,6 @@
 " Language   : PyPy JIT traces
 " Maintainer : Armin Rigo
+" Usage      : set syntax=pypytrace
 
 if exists("b:current_syntax")
  finish

diff --git a/pypy/interpreter/pyopcode.py b/pypy/interpreter/pyopcode.py
--- a/pypy/interpreter/pyopcode.py
+++ b/pypy/interpreter/pyopcode.py
@@ -754,6 +754,7 @@
     def cmp_is_not(self, w_1, w_2):
         return self.space.not_(self.space.is_(w_1, w_2))
 
+    @jit.unroll_safe
     def cmp_exc_match(self, w_1, w_2):
         if self.space.is_true(self.space.isinstance(w_2, self.space.w_tuple)):
             for w_t in self.space.fixedview(w_2):

diff --git a/pypy/module/signal/app_signal.py b/pypy/module/signal/app_signal.py
deleted file mode 100644
--- a/pypy/module/signal/app_signal.py
+++ /dev/null
@@ -1,14 +0,0 @@
-
-
-def default_int_handler(signum, frame):
-    """
-    default_int_handler(...)
-
-    The default handler for SIGINT installed by Python.
-    It raises KeyboardInterrupt.
-    """
-    raise KeyboardInterrupt()
-
-
-class ItimerError(IOError):
-    pass
\ No newline at end of file

diff --git a/pypy/rpython/memory/gc/minimark.py b/pypy/rpython/memory/gc/minimark.py
--- a/pypy/rpython/memory/gc/minimark.py
+++ b/pypy/rpython/memory/gc/minimark.py
@@ -1,7 +1,7 @@
 """ MiniMark GC.
 
 Environment variables can be used to fine-tune the following parameters:
-    
+
  PYPY_GC_NURSERY        The nursery size.  Defaults to half the size of
                         the L2 cache.  Try values like '1.2MB'.  Small values
                         (like 1 or 1KB) are useful for debugging.
@@ -108,6 +108,8 @@
 GCFLAG_HAS_CARDS    = first_gcflag << 5
 GCFLAG_CARDS_SET    = first_gcflag << 6     # <- at least one card bit is set
 
+TID_MASK            = (first_gcflag << 7) - 1
+
 
 FORWARDSTUB = lltype.GcStruct('forwarding_stub',
                               ('forw', llmemory.Address))
@@ -820,9 +822,13 @@
         that can never be set on a young object -- except if tid == -42.
         """
         assert self.is_in_nursery(obj)
-        result = (self.header(obj).tid & GCFLAG_FINALIZATION_ORDERING != 0)
+        tid = self.header(obj).tid
+        result = (tid & GCFLAG_FINALIZATION_ORDERING != 0)
         if result:
-            ll_assert(self.header(obj).tid == -42, "bogus header for young obj")
+            ll_assert(tid == -42, "bogus header for young obj")
+        else:
+            ll_assert(bool(tid), "bogus header (1)")
+            ll_assert(tid & ~TID_MASK == 0, "bogus header (2)")
         return result
 
     def get_forwarding_address(self, obj):

diff --git a/pypy/module/zlib/app_zlib.py b/pypy/module/zlib/app_zlib.py
deleted file mode 100644
--- a/pypy/module/zlib/app_zlib.py
+++ /dev/null
@@ -1,11 +0,0 @@
-
-"""
-Application-level definitions for the zlib module.
-
-NOT_RPYTHON
-"""
-
-class error(Exception):
-    """
-    Raised by zlib operations.
-    """

diff --git a/pypy/translator/platform/__init__.py b/pypy/translator/platform/__init__.py
--- a/pypy/translator/platform/__init__.py
+++ b/pypy/translator/platform/__init__.py
@@ -19,9 +19,11 @@
 
     def __repr__(self):
         if self.err:
-            return "<CompilationError err=%s>" % py.io.saferepr(self.err)
+            attr = 'err'
         else:
-            return "<CompilationError out=%s>" % py.io.saferepr(self.out)
+            attr = 'out'
+        text = getattr(self, attr).replace('\n', '\n\t')
+        return 'CompilationError(%s="""\n\t%s""")' % (attr, text)
 
     __str__ = __repr__
 

diff --git a/pypy/translator/c/gcc/trackgcroot.py b/pypy/translator/c/gcc/trackgcroot.py
--- a/pypy/translator/c/gcc/trackgcroot.py
+++ b/pypy/translator/c/gcc/trackgcroot.py
@@ -456,7 +456,7 @@
         'inc', 'dec', 'not', 'neg', 'or', 'and', 'sbb', 'adc',
         'shl', 'shr', 'sal', 'sar', 'rol', 'ror', 'mul', 'imul', 'div', 'idiv',
         'bswap', 'bt', 'rdtsc',
-        'punpck', 'pshufd', 
+        'punpck', 'pshufd', 'pcmp',
         # zero-extending moves should not produce GC pointers
         'movz', 
         ])

diff --git a/pypy/interpreter/baseobjspace.py b/pypy/interpreter/baseobjspace.py
--- a/pypy/interpreter/baseobjspace.py
+++ b/pypy/interpreter/baseobjspace.py
@@ -3,6 +3,7 @@
 from pypy.interpreter.executioncontext import ExecutionContext, ActionFlag
 from pypy.interpreter.executioncontext import UserDelAction, FrameTraceAction
 from pypy.interpreter.error import OperationError, operationerrfmt
+from pypy.interpreter.error import new_exception_class
 from pypy.interpreter.argument import Arguments
 from pypy.interpreter.miscutils import ThreadLocals
 from pypy.tool.cache import Cache
@@ -958,6 +959,10 @@
     def exception_issubclass_w(self, w_cls1, w_cls2):
         return self.is_true(self.issubtype(w_cls1, w_cls2))
 
+    def new_exception_class(self, *args, **kwargs):
+        "NOT_RPYTHON; convenience method to create excceptions in modules"
+        return new_exception_class(self, *args, **kwargs)
+
     # end of special support code
 
     def eval(self, expression, w_globals, w_locals, hidden_applevel=False):

diff --git a/pypy/module/select/app_select.py b/pypy/module/select/app_select.py
deleted file mode 100644
--- a/pypy/module/select/app_select.py
+++ /dev/null
@@ -1,2 +0,0 @@
-class error(Exception):
-    pass

diff --git a/pypy/module/termios/app_termios.py b/pypy/module/termios/app_termios.py
deleted file mode 100644
--- a/pypy/module/termios/app_termios.py
+++ /dev/null
@@ -1,3 +0,0 @@
-
-class error(Exception):
-    pass

diff --git a/pypy/interpreter/function.py b/pypy/interpreter/function.py
--- a/pypy/interpreter/function.py
+++ b/pypy/interpreter/function.py
@@ -563,7 +563,7 @@
 
 class StaticMethod(Wrappable):
     """The staticmethod objects."""
-    _immutable_ = True
+    _immutable_fields_ = ['w_function']
 
     def __init__(self, w_function):
         self.w_function = w_function
@@ -577,7 +577,7 @@
 
 class ClassMethod(Wrappable):
     """The classmethod objects."""
-    _immutable_ = True
+    _immutable_fields_ = ['w_function']
 
     def __init__(self, w_function):
         self.w_function = w_function

diff --git a/pypy/objspace/std/stringobject.py b/pypy/objspace/std/stringobject.py
--- a/pypy/objspace/std/stringobject.py
+++ b/pypy/objspace/std/stringobject.py
@@ -21,7 +21,7 @@
 
 class W_StringObject(W_Object):
     from pypy.objspace.std.stringtype import str_typedef as typedef
-    _immutable_ = True
+    _immutable_fields_ = ['_value']
 
     def __init__(w_self, str):
         w_self._value = str

diff --git a/pypy/module/_ssl/app_ssl.py b/pypy/module/_ssl/app_ssl.py
deleted file mode 100644
--- a/pypy/module/_ssl/app_ssl.py
+++ /dev/null
@@ -1,7 +0,0 @@
-import _socket
-
-class SSLError(_socket.error):
-    pass
-
-__doc__ = """Implementation module for SSL socket operations. 
-See the socket module for documentation."""

diff --git a/pypy/module/parser/app_helpers.py b/pypy/module/parser/app_helpers.py
deleted file mode 100644
--- a/pypy/module/parser/app_helpers.py
+++ /dev/null
@@ -1,2 +0,0 @@
-class ParserError(Exception):
-    pass

diff --git a/pypy/module/pyexpat/app_pyexpat.py b/pypy/module/pyexpat/app_pyexpat.py
deleted file mode 100644
--- a/pypy/module/pyexpat/app_pyexpat.py
+++ /dev/null
@@ -1,2 +0,0 @@
-class ExpatError(Exception):
-    pass

diff --git a/pypy/module/signal/interp_signal.py b/pypy/module/signal/interp_signal.py
--- a/pypy/module/signal/interp_signal.py
+++ b/pypy/module/signal/interp_signal.py
@@ -208,6 +208,16 @@
         return action.handlers_w[signum]
     return space.wrap(SIG_DFL)
 
+def default_int_handler(space, w_signum, w_frame):
+    """
+    default_int_handler(...)
+
+    The default handler for SIGINT installed by Python.
+    It raises KeyboardInterrupt.
+    """
+    raise OperationError(space.w_KeyboardInterrupt,
+                         space.w_None)
+
 @jit.dont_look_inside
 @unwrap_spec(timeout=int)
 def alarm(space, timeout):
@@ -303,9 +313,13 @@
     w_interval = space.wrap(double_from_timeval(val.c_it_interval))
     return space.newtuple([w_value, w_interval])
 
+class Cache:
+    def __init__(self, space):
+        self.w_itimererror = space.new_exception_class("signal.ItimerError",
+                                                       space.w_IOError)
+
 def get_itimer_error(space):
-    mod = space.getbuiltinmodule("signal")
-    return space.getattr(mod, space.wrap("ItimerError"))
+    return space.fromcache(Cache).w_itimererror
 
 @jit.dont_look_inside
 @unwrap_spec(which=int, first=float, interval=float)

diff --git a/pypy/module/binascii/app_binascii.py b/pypy/module/binascii/app_binascii.py
deleted file mode 100644
--- a/pypy/module/binascii/app_binascii.py
+++ /dev/null
@@ -1,6 +0,0 @@
-
-class Error(Exception):
-    pass
-
-class Incomplete(Exception):
-    pass

diff --git a/pypy/rpython/test/test_rclass.py b/pypy/rpython/test/test_rclass.py
--- a/pypy/rpython/test/test_rclass.py
+++ b/pypy/rpython/test/test_rclass.py
@@ -853,17 +853,17 @@
                accessor.fields == {"ov" : ""} # for ootype
 
     def test_immutable_subclass_1(self):
+        from pypy.rpython.rclass import ImmutableConflictError
         from pypy.jit.metainterp.typesystem import deref
         class A(object):
             _immutable_ = True
         class B(A):
             pass
         def f():
+            A()
             B().v = 123
             return B()
-        t, typer, graph = self.gengraph(f, [])
-        B_TYPE = deref(graph.getreturnvar().concretetype)
-        assert B_TYPE._hints["immutable"]    # inherited from A
+        py.test.raises(ImmutableConflictError, self.gengraph, f, [])
 
     def test_immutable_subclass_2(self):
         from pypy.jit.metainterp.typesystem import deref
@@ -872,6 +872,7 @@
         class B(A):
             _immutable_ = True
         def f():
+            A()
             B().v = 123
             return B()
         t, typer, graph = self.gengraph(f, [])

diff --git a/pypy/module/zipimport/app_zipimport.py b/pypy/module/zipimport/app_zipimport.py
deleted file mode 100644
--- a/pypy/module/zipimport/app_zipimport.py
+++ /dev/null
@@ -1,4 +0,0 @@
-
-class ZipImportError(ImportError):
-    pass
-

From commits-noreply at bitbucket.org  Fri Feb 25 12:25:03 2011
From: commits-noreply at bitbucket.org (bivab)
Date: Fri, 25 Feb 2011 12:25:03 +0100 (CET)
Subject: [pypy-svn] pypy arm-backend-2: Fix frame size calculation for
	bridges and	spilling in large frames
Message-ID: <20110225112503.5BCB12A202D@codespeak.net>

Author: David Schneider <david.schneider at picle.org>
Branch: arm-backend-2
Changeset: r42278:b06fe41076c0
Date: 2011-02-25 12:19 +0100
http://bitbucket.org/pypy/pypy/changeset/b06fe41076c0/

Log:	Fix frame size calculation for bridges and spilling in large frames

diff --git a/pypy/jit/backend/arm/assembler.py b/pypy/jit/backend/arm/assembler.py
--- a/pypy/jit/backend/arm/assembler.py
+++ b/pypy/jit/backend/arm/assembler.py
@@ -354,7 +354,7 @@
                 assert 0, 'invalid location'
         sp_patch_location = self._prepare_sp_patch_position()
         self.mc.B_offs(loop_head)
-        self._patch_sp_offset(sp_patch_location, looptoken)
+        self._patch_sp_offset(sp_patch_location, looptoken._arm_frame_depth)
 
     def _dump(self, ops, type='loop'):
         debug_start('jit-backend-ops')
@@ -386,7 +386,7 @@
         self._walk_operations(operations, regalloc)
 
         looptoken._arm_frame_depth = regalloc.frame_manager.frame_depth
-        self._patch_sp_offset(sp_patch_location, looptoken)
+        self._patch_sp_offset(sp_patch_location, looptoken._arm_frame_depth)
 
         self.align()
 
@@ -414,15 +414,15 @@
         regalloc = ARMRegisterManager(longevity, assembler=self,
                                             frame_manager=ARMFrameManager())
 
+        sp_patch_location = self._prepare_sp_patch_position()
         frame_depth = faildescr._arm_frame_depth
         locs = self.decode_inputargs(enc, inputargs, regalloc)
-        sp_patch_location = self._prepare_sp_patch_position()
         regalloc.update_bindings(locs, frame_depth, inputargs)
 
         self._walk_operations(operations, regalloc)
 
-        original_loop_token._arm_frame_depth = regalloc.frame_manager.frame_depth
-        self._patch_sp_offset(sp_patch_location, original_loop_token)
+        #original_loop_token._arm_frame_depth = regalloc.frame_manager.frame_depth
+        self._patch_sp_offset(sp_patch_location, regalloc.frame_manager.frame_depth)
 
         bridge_start = self.materialize_loop(original_loop_token)
         self.update_descrs_for_bridges(bridge_start)
@@ -466,13 +466,13 @@
             self.mc.MOV_rr(r.r0.value, r.r0.value)
         return l
 
-    def _patch_sp_offset(self, pos, looptoken):
+    def _patch_sp_offset(self, pos, frame_depth):
         cb = OverwritingBuilder(self.mc, pos, OverwritingBuilder.size_of_gen_load_int)
         # Note: the frame_depth is one less than the value stored in the frame
         # manager
-        if looptoken._arm_frame_depth == 1:
+        if frame_depth == 1:
             return
-        n = (looptoken._arm_frame_depth-1)*WORD
+        n = (frame_depth-1)*WORD
         self._adjust_sp(n, cb, base_reg=r.fp)
 
     def _adjust_sp(self, n, cb=None, fcond=c.AL, base_reg=r.sp):
@@ -577,12 +577,29 @@
             prev_loc = new_loc
             if not loc.is_stack():
                 return
-        if loc.is_stack() and prev_loc.is_reg():
-            self.mc.STR_ri(prev_loc.value, r.fp.value, loc.position*-WORD, cond=cond)
-        elif loc.is_reg() and prev_loc.is_stack():
-            self.mc.LDR_ri(loc.value, r.fp.value, prev_loc.position*-WORD, cond=cond)
+
+        if loc.is_stack() or prev_loc.is_stack():
+            temp = r.lr
+            if loc.is_stack() and prev_loc.is_reg():
+                offset = ConstInt(loc.position*-WORD)
+                if not _check_imm_arg(offset):
+                    self.mc.gen_load_int(temp.value, offset.value)
+                    self.mc.STR_rr(prev_loc.value, r.fp.value, temp.value, cond=cond)
+                else:
+                    self.mc.STR_ri(prev_loc.value, r.fp.value, offset.value, cond=cond)
+            elif loc.is_reg() and prev_loc.is_stack():
+                offset = ConstInt(prev_loc.position*-WORD)
+                if not _check_imm_arg(offset):
+                    self.mc.gen_load_int(temp.value, offset.value)
+                    self.mc.LDR_rr(loc.value, r.fp.value, temp.value, cond=cond)
+                else:
+                    self.mc.LDR_ri(loc.value, r.fp.value, offset.value, cond=cond)
+            else:
+                assert 0, 'unsupported case'
         elif loc.is_reg() and prev_loc.is_reg():
             self.mc.MOV_rr(loc.value, prev_loc.value, cond=cond)
+        else:
+            assert 0, 'unsupported case'
     mov_loc_loc = regalloc_mov
 
     def regalloc_push(self, loc):

diff --git a/pypy/jit/backend/arm/opassembler.py b/pypy/jit/backend/arm/opassembler.py
--- a/pypy/jit/backend/arm/opassembler.py
+++ b/pypy/jit/backend/arm/opassembler.py
@@ -699,7 +699,6 @@
         # patch the jump to the fast path
         offset = self.mc.currpos() - fast_jmp_pos
         pmc = OverwritingBuilder(self.mc, fast_jmp_pos, WORD)
-        #pmc = ARMv7InMemoryBuilder(fast_jmp_location, WORD)
         pmc.ADD_ri(r.pc.value, r.pc.value, offset - PC_OFFSET, cond=c.EQ)
 
         # Reset the vable token --- XXX really too much special logic here:-(

diff --git a/pypy/jit/backend/arm/test/test_regalloc.py b/pypy/jit/backend/arm/test/test_regalloc.py
--- a/pypy/jit/backend/arm/test/test_regalloc.py
+++ b/pypy/jit/backend/arm/test/test_regalloc.py
@@ -22,7 +22,7 @@
     get_funcptr_for_newarray = get_funcptr_for_new
     get_funcptr_for_newstr = get_funcptr_for_new
     get_funcptr_for_newunicode = get_funcptr_for_new
- 
+
     def rewrite_assembler(self, cpu, operations):
         pass
 
@@ -348,7 +348,7 @@
         '''
         self.interpret(ops, [0, 0, 3, 0])
         assert self.getints(3) == [1, -3, 10]
-        
+
     def test_compare_memory_result_survives(self):
         ops = '''
         [i0, i1, i2, i3]
@@ -409,7 +409,7 @@
         assert self.getints(9) == range(9)
 
 class TestRegallocCompOps(BaseTestRegalloc):
-    
+
     def test_cmp_op_0(self):
         ops = '''
         [i0, i3]
@@ -612,7 +612,7 @@
         ops = '''
         [i0, i1,  i2, i3, i4, i5, i6, i7, i8, i9]
         i10 = call(ConstClass(f1ptr), i0, descr=f1_calldescr)
-        i11 = call(ConstClass(f2ptr), i10, i1, descr=f2_calldescr)        
+        i11 = call(ConstClass(f2ptr), i10, i1, descr=f2_calldescr)
         finish(i11, i1,  i2, i3, i4, i5, i6, i7, i8, i9)
         '''
         loop = self.interpret(ops, [4, 7, 9, 9 ,9, 9, 9, 9, 9, 9, 9])
@@ -638,14 +638,14 @@
         assert self.getint(0) == 5
         ops = '''
         [i2, i1]
-        i3 = call(ConstClass(f2ptr), i2, i1, descr=f2_calldescr)        
-        finish(i3, descr=fdescr2)        
+        i3 = call(ConstClass(f2ptr), i2, i1, descr=f2_calldescr)
+        finish(i3, descr=fdescr2)
         '''
         bridge = self.attach_bridge(ops, loop, -2)
 
 
         self.cpu.set_future_value_int(0, 4)
-        self.cpu.set_future_value_int(1, 7)        
+        self.cpu.set_future_value_int(1, 7)
         self.run(loop)
         assert self.getint(0) == 5*7
 
@@ -660,8 +660,8 @@
         assert self.getint(0) == 4*7
         ops = '''
         [i2]
-        i3 = call(ConstClass(f1ptr), i2, descr=f1_calldescr)        
-        finish(i3, descr=fdescr2)        
+        i3 = call(ConstClass(f1ptr), i2, descr=f1_calldescr)
+        finish(i3, descr=fdescr2)
         '''
         bridge = self.attach_bridge(ops, loop, -2)
 

diff --git a/pypy/jit/backend/arm/test/test_assembler.py b/pypy/jit/backend/arm/test/test_assembler.py
--- a/pypy/jit/backend/arm/test/test_assembler.py
+++ b/pypy/jit/backend/arm/test/test_assembler.py
@@ -56,6 +56,18 @@
         self.a.gen_func_epilog()
         assert run_asm(self.a) == -3
 
+    def test_load_int1(self):
+        self.a.gen_func_prolog()
+        self.a.mc.gen_load_int(r.r0.value, 440)
+        self.a.gen_func_epilog()
+        assert run_asm(self.a) == 440
+
+    def test_load_int2(self):
+        self.a.gen_func_prolog()
+        self.a.mc.gen_load_int(r.r0.value, 464)
+        self.a.gen_func_epilog()
+        assert run_asm(self.a) == 464
+
 
     def test_or(self):
         self.a.gen_func_prolog()

From commits-noreply at bitbucket.org  Fri Feb 25 14:58:52 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Fri, 25 Feb 2011 14:58:52 +0100 (CET)
Subject: [pypy-svn] pypy default: add a way to override which classes are
	created
Message-ID: <20110225135852.762DD282B8B@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42279:845d4344fc53
Date: 2011-02-25 13:55 +0100
http://bitbucket.org/pypy/pypy/changeset/845d4344fc53/

Log:	add a way to override which classes are created

diff --git a/pypy/tool/jitlogparser/parser.py b/pypy/tool/jitlogparser/parser.py
--- a/pypy/tool/jitlogparser/parser.py
+++ b/pypy/tool/jitlogparser/parser.py
@@ -48,6 +48,10 @@
         ##                                             for a in self.args]))
 
 class SimpleParser(OpParser):
+
+    # factory method
+    Op = Op
+    
     def parse_args(self, opname, argspec):
         if not argspec.strip():
             return [], None
@@ -67,7 +71,7 @@
         return res
 
     def create_op(self, opnum, args, res, descr):
-        return Op(intern(opname[opnum].lower()), args, res, descr)
+        return self.Op(intern(opname[opnum].lower()), args, res, descr)
 
 class NonCodeError(Exception):
     pass
@@ -132,6 +136,9 @@
     _lineset = None
     is_bytecode = False
     inline_level = None
+
+    # factory method
+    TraceForOpcode = TraceForOpcode
     
     def __init__(self, chunks, path, storage):
         self.path = path
@@ -172,13 +179,13 @@
         for op in operations:
             if op.name == 'debug_merge_point':
                 if so_far:
-                    append_to_res(TraceForOpcode(so_far, storage))
+                    append_to_res(cls.TraceForOpcode(so_far, storage))
                     if limit:
                         break
                     so_far = []
             so_far.append(op)
         if so_far:
-            append_to_res(TraceForOpcode(so_far, storage))
+            append_to_res(cls.TraceForOpcode(so_far, storage))
         # wrap stack back up
         if not stack:
             # no ops whatsoever

From commits-noreply at bitbucket.org  Fri Feb 25 14:58:53 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Fri, 25 Feb 2011 14:58:53 +0100 (CET)
Subject: [pypy-svn] pypy default: make parse() a classmethod of
	SimpleParser: this way, it's easier to subclass it
Message-ID: <20110225135853.C6AD5282B8B@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r42280:7231f9b4e4d8
Date: 2011-02-25 14:39 +0100
http://bitbucket.org/pypy/pypy/changeset/7231f9b4e4d8/

Log:	make parse() a classmethod of SimpleParser: this way, it's easier to
	subclass it

diff --git a/pypy/module/pypyjit/test_pypy_c/model.py b/pypy/module/pypyjit/test_pypy_c/model.py
--- a/pypy/module/pypyjit/test_pypy_c/model.py
+++ b/pypy/module/pypyjit/test_pypy_c/model.py
@@ -1,7 +1,7 @@
 import py
 import re
 import os.path
-from pypy.tool.jitlogparser.parser import parse, Function, TraceForOpcode
+from pypy.tool.jitlogparser.parser import SimpleParser, Function, TraceForOpcode
 from pypy.tool.jitlogparser.storage import LoopStorage
 
 
@@ -40,7 +40,7 @@
 class Log(object):
     def __init__(self, func, rawtraces):
         storage = LoopStorage()
-        traces = [parse(rawtrace) for rawtrace in rawtraces]
+        traces = [SimpleParser.parse_from_input(rawtrace) for rawtrace in rawtraces]
         traces = storage.reconnect_loops(traces)
         self.loops = [LoopWithIds.from_trace(trace, storage) for trace in traces]
 

diff --git a/pypy/tool/jitlogparser/parser.py b/pypy/tool/jitlogparser/parser.py
--- a/pypy/tool/jitlogparser/parser.py
+++ b/pypy/tool/jitlogparser/parser.py
@@ -51,6 +51,11 @@
 
     # factory method
     Op = Op
+
+    @classmethod
+    def parse_from_input(cls, input):
+        return cls(input, None, {}, 'lltype', None,
+                   nonstrict=True).parse()
     
     def parse_args(self, opname, argspec):
         if not argspec.strip():
@@ -73,6 +78,8 @@
     def create_op(self, opnum, args, res, descr):
         return self.Op(intern(opname[opnum].lower()), args, res, descr)
 
+
+
 class NonCodeError(Exception):
     pass
 
@@ -246,10 +253,6 @@
                 print >>out, "  ", source
             chunk.pretty_print(out)
 
-def parse(input):
-    return SimpleParser(input, None, {}, 'lltype', None,
-                        nonstrict=True).parse()
-
 
 def adjust_bridges(loop, bridges):
     """ Slice given loop according to given bridges to follow. Returns a plain

diff --git a/pypy/tool/jitlogparser/test/test_parser.py b/pypy/tool/jitlogparser/test/test_parser.py
--- a/pypy/tool/jitlogparser/test/test_parser.py
+++ b/pypy/tool/jitlogparser/test/test_parser.py
@@ -1,10 +1,13 @@
 from pypy.jit.metainterp.resoperation import ResOperation, rop
 from pypy.jit.metainterp.history import ConstInt, Const
-from pypy.tool.jitlogparser.parser import parse, TraceForOpcode, Function,\
+from pypy.tool.jitlogparser.parser import SimpleParser, TraceForOpcode, Function,\
      adjust_bridges
 from pypy.tool.jitlogparser.storage import LoopStorage
 import py
 
+def parse(input):
+    return SimpleParser.parse_from_input(input)
+
 
 def test_parse():
     ops = parse('''

diff --git a/pypy/module/pypyjit/test_pypy_c/test_model.py b/pypy/module/pypyjit/test_pypy_c/test_model.py
--- a/pypy/module/pypyjit/test_pypy_c/test_model.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_model.py
@@ -83,8 +83,8 @@
 class TestOpMatcher(object):
 
     def match(self, src1, src2):
-        from pypy.tool.jitlogparser.parser import parse
-        loop = parse(src1)
+        from pypy.tool.jitlogparser.parser import SimpleParser
+        loop = SimpleParser.parse_from_input(src1)
         matcher = OpMatcher(loop.operations)
         return matcher.match(src2)
 

From commits-noreply at bitbucket.org  Fri Feb 25 14:59:31 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Fri, 25 Feb 2011 14:59:31 +0100 (CET)
Subject: [pypy-svn] jitviewer default: skip this test,
	it's broken and the function is not called from anywhere
Message-ID: <20110225135931.1E110282B8B@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r86:2e686f8bb32d
Date: 2011-02-25 12:14 +0100
http://bitbucket.org/pypy/jitviewer/changeset/2e686f8bb32d/

Log:	skip this test, it's broken and the function is not called from
	anywhere

diff --git a/_jitviewer/test/test_loops.py b/_jitviewer/test/test_loops.py
--- a/_jitviewer/test/test_loops.py
+++ b/_jitviewer/test/test_loops.py
@@ -195,6 +195,7 @@
 '''.split("\n")
 
 def test_parse_log_count():
+    py.test.skip('fixme')
     class Loop(object):
         pass
     

From commits-noreply at bitbucket.org  Fri Feb 25 14:59:32 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Fri, 25 Feb 2011 14:59:32 +0100 (CET)
Subject: [pypy-svn] jitviewer default: rename loops.py to parser.py,
	to math the naming scheme used in pypy.tool.jitlogparser
Message-ID: <20110225135932.0458D282B8B@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r87:6645fdedd4c9
Date: 2011-02-25 12:20 +0100
http://bitbucket.org/pypy/jitviewer/changeset/6645fdedd4c9/

Log:	rename loops.py to parser.py, to math the naming scheme used in
	pypy.tool.jitlogparser

diff --git a/bin/jitviewer.py b/bin/jitviewer.py
--- a/bin/jitviewer.py
+++ b/bin/jitviewer.py
@@ -20,7 +20,7 @@
 import flask
 import inspect
 from pypy.tool.logparser import parse_log_file, extract_category
-from _jitviewer.loops import (parse, slice_debug_merge_points, adjust_bridges,
+from _jitviewer.parser import (parse, slice_debug_merge_points, adjust_bridges,
                    parse_log_counts)
 from _jitviewer.storage import LoopStorage
 from _jitviewer.display import CodeRepr, CodeReprNoFile

diff --git a/_jitviewer/test/test_loops.py b/_jitviewer/test/test_loops.py
deleted file mode 100644
--- a/_jitviewer/test/test_loops.py
+++ /dev/null
@@ -1,217 +0,0 @@
-
-from pypy.jit.metainterp.resoperation import ResOperation, rop
-from pypy.jit.metainterp.history import ConstInt, Const
-from _jitviewer.loops import parse, Bytecode, Function,\
-     slice_debug_merge_points,\
-     adjust_bridges, parse_log_counts, cssclass
-from _jitviewer.storage import LoopStorage
-import py
-
-def test_parse():
-    ops = parse('''
-    [i7]
-    i9 = int_lt(i7, 1003)
-    guard_true(i9, descr=<Guard2>) []
-    i13 = getfield_raw(151937600, descr=<SignedFieldDescr pypysig_long_struct.c_value 0>)
-    ''').operations
-    assert len(ops) == 3
-    assert ops[0].name == 'int_lt'
-    assert ops[1].name == 'guard_true'
-    assert ops[1].descr is not None
-    assert ops[0].res == 'i9'
-    assert ops[0].html_repr().plaintext() == 'i9 = i7 < 1003'
-    assert ops[2].descr is not None
-    assert len(ops[2].args) == 1
-    assert ops[2].html_repr().plaintext() == 'i13 = ((pypysig_long_struct)151937600).value'
-
-def test_parse_non_code():
-    ops = parse('''
-    []
-    debug_merge_point("SomeRandomStuff", 0)
-    ''')
-    res = slice_debug_merge_points(ops.operations, LoopStorage())
-    assert len(res.chunks) == 1
-    assert res.chunks[0].html_repr()
-
-def test_split():
-    ops = parse('''
-    [i0]
-    debug_merge_point("<code object stuff, file '/tmp/x.py', line 200> #10 ADD", 0)
-    debug_merge_point("<code object stuff, file '/tmp/x.py', line 200> #11 SUB", 0)
-    i1 = int_add(i0, 1)
-    debug_merge_point("<code object stuff, file '/tmp/x.py', line 200> #11 SUB", 0)
-    i2 = int_add(i1, 1)
-    ''')
-    res = slice_debug_merge_points(ops.operations, LoopStorage())
-    assert len(res.chunks) == 3
-    assert len(res.chunks[0].operations) == 1
-    assert len(res.chunks[1].operations) == 2
-    assert len(res.chunks[2].operations) == 2
-    assert res.chunks[2].bytecode_no == 11
-
-def test_inlined_call():
-    ops = parse("""
-    []
-    debug_merge_point('<code object inlined_call, file 'source.py', line 12> #28 CALL_FUNCTION', 0)
-    i18 = getfield_gc(p0, descr=<BoolFieldDescr pypy.interpreter.pyframe.PyFrame.inst_is_being_profiled 89>)
-    debug_merge_point('<code object inner, file 'source.py', line 9> #0 LOAD_FAST', 1)
-    debug_merge_point('<code object inner, file 'source.py', line 9> #3 LOAD_CONST', 1)
-    debug_merge_point('<code object inner, file 'source.py', line 9> #7 RETURN_VALUE', 1)
-    debug_merge_point('<code object inlined_call, file 'source.py', line 12> #31 STORE_FAST', 0)
-    """)
-    res = slice_debug_merge_points(ops.operations, LoopStorage())
-    assert len(res.chunks) == 3 # two chunks + inlined call
-    assert isinstance(res.chunks[0], Bytecode)
-    assert isinstance(res.chunks[1], Function)
-    assert isinstance(res.chunks[2], Bytecode)
-    assert res.chunks[1].path == "1"
-    assert len(res.chunks[1].chunks) == 3
-    
-def test_name():
-    ops = parse('''
-    [i0]
-    debug_merge_point("<code object stuff, file '/tmp/x.py', line 200> #10 ADD", 0)
-    debug_merge_point("<code object stuff, file '/tmp/x.py', line 201> #11 SUB", 0)
-    i1 = int_add(i0, 1)
-    debug_merge_point("<code object stuff, file '/tmp/x.py', line 202> #11 SUB", 0)
-    i2 = int_add(i1, 1)
-    ''')
-    res = slice_debug_merge_points(ops.operations, LoopStorage())
-    assert res.repr() == res.chunks[0].repr()
-    assert res.repr() == "stuff, file '/tmp/x.py', line 200"
-    assert res.startlineno == 200
-    assert res.filename == '/tmp/x.py'
-    assert res.name == 'stuff'
-
-def test_name_no_first():
-    ops = parse('''
-    [i0]
-    i3 = int_add(i0, 1)
-    debug_merge_point("<code object stuff, file '/tmp/x.py', line 200> #10 ADD", 0)
-    debug_merge_point("<code object stuff, file '/tmp/x.py', line 201> #11 SUB", 0)
-    i1 = int_add(i0, 1)
-    debug_merge_point("<code object stuff, file '/tmp/x.py', line 202> #11 SUB", 0)
-    i2 = int_add(i1, 1)
-    ''')
-    res = slice_debug_merge_points(ops.operations, LoopStorage())
-    assert res.repr() == res.chunks[1].repr()
-
-def test_lineno():
-    fname = str(py.path.local(__file__).join('..', 'x.py'))
-    ops = parse('''
-    [i0, i1]
-    debug_merge_point("<code object f, file '%(fname)s', line 2> #0 LOAD_FAST", 0)
-    debug_merge_point("<code object f, file '%(fname)s', line 2> #3 LOAD_FAST", 0)
-    debug_merge_point("<code object f, file '%(fname)s', line 2> #6 BINARY_ADD", 0)
-    debug_merge_point("<code object f, file '%(fname)s', line 2> #7 RETURN_VALUE", 0)
-    ''' % locals())
-    res = slice_debug_merge_points(ops.operations, LoopStorage())
-    assert res.chunks[1].lineno == 3
-
-def test_linerange():
-    fname = str(py.path.local(__file__).join('..', 'x.py'))
-    ops = parse('''
-    [i0, i1]
-    debug_merge_point("<code object f, file '%(fname)s', line 5> #9 LOAD_FAST", 0)
-    debug_merge_point("<code object f, file '%(fname)s', line 5> #12 LOAD_CONST", 0)
-    debug_merge_point("<code object f, file '%(fname)s', line 5> #22 LOAD_CONST", 0)
-    debug_merge_point("<code object f, file '%(fname)s', line 5> #28 LOAD_CONST", 0)
-    debug_merge_point("<code object f, file '%(fname)s', line 5> #6 SETUP_LOOP", 0)
-    ''' % locals())
-    res = slice_debug_merge_points(ops.operations, LoopStorage())
-    assert res.linerange == (7, 9)
-    assert res.lineset == set([7, 8, 9])
-
-def test_linerange_notstarts():
-    fname = str(py.path.local(__file__).join('..', 'x.py'))
-    ops = parse("""
-    [p6, p1]
-    debug_merge_point('<code object h, file '%(fname)s', line 11> #17 FOR_ITER', 0)
-    guard_class(p6, 144264192, descr=<Guard2>)
-    p12 = getfield_gc(p6, descr=<GcPtrFieldDescr pypy.objspace.std.iterobject.W_AbstractSeqIterObject.inst_w_seq 12>)
-    """ % locals())
-    res = slice_debug_merge_points(ops.operations, LoopStorage())
-    assert res.lineset
-
-def test_reassign_loops():
-    main = parse('''
-    [v0]
-    guard_false(v0, descr=<Guard18>) []
-    ''')
-    main.count = 10
-    bridge = parse('''
-    # bridge out of Guard 18 with 13 ops
-    [i0, i1]
-    int_add(i0, i1)
-    ''')
-    bridge.count = 3
-    entry_bridge = parse('''
-    # Loop 3 : entry bridge
-    []
-    ''')
-    loops = LoopStorage().reconnect_loops([main, bridge, entry_bridge])
-    assert len(loops) == 2
-    assert len(loops[0].operations[0].bridge.operations) == 1
-    assert loops[0].operations[0].bridge.no == 18
-    assert loops[0].operations[0].percentage == 30
-
-def test_adjust_bridges():
-    main = parse('''
-    [v0]
-    guard_false(v0, descr=<Guard13>)
-    guard_true(v0, descr=<Guard5>)
-    ''')
-    bridge = parse('''
-    # bridge out of Guard 13
-    []
-    int_add(0, 1)
-    ''')
-    loops = LoopStorage().reconnect_loops([main, bridge])
-    assert adjust_bridges(main, {})[1].name == 'guard_true'
-    assert adjust_bridges(main, {'loop-13': True})[1].name == 'int_add'
-
-def test_parsing_strliteral():
-    loop = parse("""
-    debug_merge_point('StrLiteralSearch at 11/51 [17, 8, 3, 1, 1, 1, 1, 51, 0, 19, 51, 1]', 0)
-    """)
-    ops = slice_debug_merge_points(loop.operations, LoopStorage())
-    chunk = ops.chunks[0]
-    assert chunk.bytecode_name == 'StrLiteralSearch'
-
-LINES = '''
-0:3
-1:3
-2:604
-3:396
-4:102
-5:2000
-6:3147
-7:2445
-8:2005
-9:2000
-10:1420
-11:40
-12:0
-'''.split("\n")
-
-def test_parse_log_count():
-    py.test.skip('fixme')
-    class Loop(object):
-        pass
-    
-    loops = [Loop() for i in range(13)]
-    nums = parse_log_counts(LINES, loops)
-    assert nums[5] == 2000
-    assert loops[9].count == 2000
-
-def test_highlight_var():
-    ops = parse('''
-    [p0]
-    guard_class(p0, 144264192, descr=<Guard0>)
-    ''').operations
-    assert len(ops) == 1
-    op = ops[0]
-    assert op.name == 'guard_class'
-    html = op.html_repr()
-    p0 = cssclass('p0', 'p0', onmouseover="highlight_var(this)", onmouseout="disable_var(this)")
-    assert p0 in html

diff --git a/_jitviewer/loops.py b/_jitviewer/parser.py
copy from _jitviewer/loops.py
copy to _jitviewer/parser.py

diff --git a/_jitviewer/loops.py b/_jitviewer/loops.py
deleted file mode 100644
--- a/_jitviewer/loops.py
+++ /dev/null
@@ -1,373 +0,0 @@
-
-import re, sys
-from pypy.jit.metainterp.resoperation import rop, opname
-from disassembler import dis
-from pypy.jit.tool.oparser import OpParser
-
-class Html(str):
-    def __html__(self):
-        return self
-
-    def plaintext(self):
-        # This is not a general way to strip tags, but it's good enough to use
-        # in tests
-        import re
-        s = re.sub('<.*?>', '', self)
-        s = s.replace("&lt;", "<")
-        s = s.replace("&gt;", ">")
-        s = s.replace("&amp;", "&")
-        return s
-
-
-def cssclass(cls, s, **kwds):
-    attrs = ['%s="%s"' % (name, value) for name, value in kwds.iteritems()]
-    return '<span class="%s" %s>%s</span>' % (cls, ' '.join(attrs), s)
-
-
-def _new_binop(name):
-    import cgi
-    name = cgi.escape(name)
-    def f(self):
-        return '%s = %s %s %s' % (self.getres(), self.getarg(0), name, self.getarg(1))
-    return f
-
-class Op(object):
-    bridge = None
-    
-    def __init__(self, name, args, res, descr):
-        self.name = name
-        self.args = args
-        self.res = res
-        self.descr = descr
-        self._is_guard = name.startswith('guard_')
-        if self._is_guard:
-            self.guard_no = int(self.descr[len('<Guard'):-1])
-
-    def setfailargs(self, _):
-        pass
-
-    def html_repr(self):
-        s = getattr(self, 'repr_' + self.name, self.generic_repr)()
-        if self.is_guard():
-            s = '<span class="guard">guard</span>(' + s + ')'
-        return Html(s)
-
-    def getarg(self, i):
-        return self._getvar(self.args[i])
-
-    def getargs(self):
-        return [self._getvar(v) for v in self.args]
-
-    def getres(self):
-        return self._getvar(self.res)
-
-    def _getvar(self, v):
-        return cssclass(v, v, onmouseover='highlight_var(this)', onmouseout='disable_var(this)')
-
-    def is_guard(self):
-        return self._is_guard
-
-    for bin_op, name in [('==', 'int_eq'),
-                         ('!=', 'int_ne'),
-                         ('==', 'float_eq'),
-                         ('!=', 'float_ne'),
-                         ('>', 'int_gt'),
-                         ('<', 'int_lt'),
-                         ('<=', 'int_le'),
-                         ('>=', 'int_ge'),
-                         ('+', 'int_add'),
-                         ('+', 'float_add'),
-                         ('-', 'int_sub'),
-                         ('-', 'float_sub'),
-                         ('*', 'int_mul'),
-                         ('*', 'float_mul'),
-                         ('&', 'int_and')]:
-        locals()['repr_' + name] = _new_binop(bin_op)
-
-    def repr_guard_true(self):
-        return '%s is true' % self.getarg(0)
-
-    def repr_guard_false(self):
-        return '%s is false' % self.getarg(0)
-
-    def repr_guard_value(self):
-        return '%s is %s' % (self.getarg(0), self.getarg(1))
-
-    def repr_guard_isnull(self):
-        return '%s is null' % self.getarg(0)
-
-    def repr_getfield_raw(self):
-        name, field = self.descr.split(' ')[1].rsplit('.', 1)
-        return '%s = ((%s)%s).%s' % (self.getres(), name, self.getarg(0), field[2:])
-
-    def repr_getfield_gc(self):
-        fullname, field = self.descr.split(' ')[1].rsplit('.', 1)
-        names = fullname.rsplit('.', 1)
-        if len(names) == 2:
-            namespace, classname = names
-        else:
-            namespace = ''
-            classname = names[0]
-        namespace = cssclass('namespace', namespace)
-        classname = cssclass('classname', classname)
-        field = cssclass('fieldname', field)
-            
-        obj = self.getarg(0)
-        return '%s = ((%s.%s)%s).%s' % (self.getres(), namespace, classname, obj, field)
-    repr_getfield_gc_pure = repr_getfield_gc
-
-    def repr_setfield_raw(self):
-        name, field = self.descr.split(' ')[1].rsplit('.', 1)
-        return '((%s)%s).%s = %s' % (name, self.getarg(0), field[2:], self.getarg(1))
-
-    def repr_setfield_gc(self):
-        name, field = self.descr.split(' ')[1].rsplit('.', 1)
-        return '((%s)%s).%s = %s' % (name, self.getarg(0), field, self.getarg(1))
-
-    def generic_repr(self):
-        arglist = ', '.join(self.getargs())
-        if self.res is not None:
-            return '%s = %s(%s)' % (self.getres(), self.name, arglist)
-        else:
-            return '%s(%s)' % (self.name, arglist)
-
-    def __repr__(self):
-        return '<%s (%s)>' % (self.name, ', '.join([repr(a)
-                                                    for a in self.args]))
-
-class SimpleParser(OpParser):
-    def parse_args(self, opname, argspec):
-        if not argspec.strip():
-            return [], None
-        if opname == 'debug_merge_point':
-            return argspec.rsplit(", ", 1), None
-        else:
-            args = argspec.split(', ')
-            descr = None
-            if args[-1].startswith('descr='):
-                descr = args[-1][len('descr='):]
-                args = args[:-1]
-            return (args, descr)
-
-    def box_for_var(self, res):
-        return res
-
-    def create_op(self, opnum, args, res, descr):
-        return Op(intern(opname[opnum].lower()), args, res, descr)
-
-class NonCodeError(Exception):
-    pass
-
-class Bytecode(object):
-    filename = None
-    startlineno = 0
-    name = None
-    code = None
-    bytecode_no = 0
-    bytecode_name = None
-    is_bytecode = True
-    inline_level = None
-    
-    def __init__(self, operations, storage):
-        if operations[0].name == 'debug_merge_point':
-            self.inline_level = int(operations[0].args[1])
-            m = re.search('<code object ([<>\w]+), file \'(.+?)\', line (\d+)> #(\d+) (\w+)',
-                         operations[0].getarg(0))
-            if m is None:
-                # a non-code loop, like StrLiteralSearch or something
-                self.bytecode_name = operations[0].args[0].split(" ")[0][1:]
-            else:
-                self.name, self.filename, lineno, bytecode_no, self.bytecode_name = m.groups()
-                self.startlineno = int(lineno)
-                self.bytecode_no = int(bytecode_no)
-        self.operations = operations
-        self.storage = storage
-
-    def repr(self):
-        if self.filename is None:
-            return "Unknown"
-        return "%s, file '%s', line %d" % (self.name, self.filename,
-                                           self.startlineno)
-
-    def getcode(self):
-        if self.code is None:
-            self.code = dis(self.storage.load_code(self.filename)[self.startlineno])
-        return self.code
-
-    def getlineno(self):
-        code = self.getcode()
-        return code.map[self.bytecode_no].lineno
-    lineno = property(getlineno)
-
-    def getline_starts_here(self):
-        code = self.getcode()
-        return code.map[self.bytecode_no].line_starts_here
-    line_starts_here = property(getline_starts_here)
-
-    def __repr__(self):
-        return "[%s]" % ", ".join([repr(op) for op in self.operations])
-
-    def pretty_print(self, out):
-        pass
-
-    def html_repr(self):
-        if self.filename is not None:
-            code = self.getcode()
-            opcode = self.code.map[self.bytecode_no]
-            return '%s %s' % (self.bytecode_name, opcode.argstr)
-        else:
-            return self.bytecode_name
-
-class Function(object):
-    filename = None
-    name = None
-    startlineno = 0
-    _linerange = None
-    _lineset = None
-    is_bytecode = False
-    inline_level = None
-    
-    def __init__(self, chunks, path, storage):
-        self.path = path
-        self.chunks = chunks
-        for chunk in self.chunks:
-            if chunk.filename is not None:
-                self.startlineno = chunk.startlineno
-                self.filename = chunk.filename
-                self.name = chunk.name
-                self.inline_level = chunk.inline_level
-                break
-        self.storage = storage
-
-    def getlinerange(self):
-        if self._linerange is None:
-            self._compute_linerange()
-        return self._linerange
-    linerange = property(getlinerange)
-
-    def getlineset(self):
-        if self._lineset is None:
-            self._compute_linerange()
-        return self._lineset
-    lineset = property(getlineset)
-
-    def _compute_linerange(self):
-        self._lineset = set()
-        minline = sys.maxint
-        maxline = -1
-        for chunk in self.chunks:
-            if chunk.is_bytecode and chunk.filename is not None:
-                lineno = chunk.lineno
-                minline = min(minline, lineno)
-                maxline = max(maxline, lineno)
-                if chunk.line_starts_here or len(chunk.operations) > 1:
-                    self._lineset.add(lineno)
-        if minline == sys.maxint:
-            minline = 0
-            maxline = 0
-        self._linerange = minline, maxline
-
-    def html_repr(self):
-        return "inlined call to %s in %s" % (self.name, self.filename)
-
-    def repr(self):
-        if self.filename is None:
-            return "Unknown"
-        return "%s, file '%s', line %d" % (self.name, self.filename,
-                                           self.startlineno)
-        
-    def __repr__(self):
-        return "[%s]" % ", ".join([repr(chunk) for chunk in self.chunks])
-
-    def pretty_print(self, out):
-        print >>out, "Loop starting at %s in %s at %d" % (self.name,
-                                        self.filename, self.startlineno)
-        lineno = -1
-        for chunk in self.chunks:
-            if chunk.filename is not None and chunk.lineno != lineno:
-                lineno = chunk.lineno
-                source = chunk.getcode().source[chunk.lineno -
-                                                chunk.startlineno]
-                print >>out, "  ", source
-            chunk.pretty_print(out)
-
-def parse_log_counts(input, loops):
-    if not input:
-        return
-    lines = input[-1].splitlines()
-    nums = []
-    i = 0
-    for line in lines:
-        if line:
-            num, count = line.split(':')
-            assert int(num) == i
-            count = int(count)
-            nums.append(count)
-            loops[i].count = count
-            i += 1
-    return nums
-
-def parse(input):
-    return SimpleParser(input, None, {}, 'lltype', None,
-                        nonstrict=True).parse()
-
-def slice_debug_merge_points(operations, storage, limit=None):
-    """ Slice given operation list into a chain of Bytecode chunks.
-    Also detect inlined functions and make them Function
-    """
-    stack = []
-
-    def getpath(stack):
-        return ",".join([str(len(v)) for v in stack])
-
-    def append_to_res(bc):
-        if not stack:
-            stack.append([])
-        else:
-            if bc.inline_level is not None and bc.inline_level + 1 != len(stack):
-                if bc.inline_level < len(stack):
-                    last = stack.pop()
-                    stack[-1].append(Function(last, getpath(stack), storage))
-                else:
-                    stack.append([])
-        stack[-1].append(bc)
-
-    so_far = []
-    stack = []
-    for op in operations:
-        if op.name == 'debug_merge_point':
-            if so_far:
-                append_to_res(Bytecode(so_far, storage))
-                if limit:
-                    break
-                so_far = []
-        so_far.append(op)
-    if so_far:
-        append_to_res(Bytecode(so_far, storage))
-    # wrap stack back up
-    if not stack:
-        # no ops whatsoever
-        return Function([], getpath(stack), storage)
-    while True:
-        next = stack.pop()
-        if not stack:
-            return Function(next, getpath(stack), storage)
-        stack[-1].append(Function(next, getpath(stack), storage))
-
-def adjust_bridges(loop, bridges):
-    """ Slice given loop according to given bridges to follow. Returns a plain
-    list of operations.
-    """
-    ops = loop.operations
-    res = []
-    i = 0
-    while i < len(ops):
-        op = ops[i]
-        if op.is_guard() and bridges.get('loop-' + str(op.guard_no), None):
-            res.append(op)
-            i = 0
-            ops = op.bridge.operations
-        else:
-            res.append(op)
-            i += 1
-    return res

diff --git a/_jitviewer/test/test_loops.py b/_jitviewer/test/test_parser.py
copy from _jitviewer/test/test_loops.py
copy to _jitviewer/test/test_parser.py
--- a/_jitviewer/test/test_loops.py
+++ b/_jitviewer/test/test_parser.py
@@ -1,7 +1,7 @@
 
 from pypy.jit.metainterp.resoperation import ResOperation, rop
 from pypy.jit.metainterp.history import ConstInt, Const
-from _jitviewer.loops import parse, Bytecode, Function,\
+from _jitviewer.parser import parse, Bytecode, Function,\
      slice_debug_merge_points,\
      adjust_bridges, parse_log_counts, cssclass
 from _jitviewer.storage import LoopStorage

From commits-noreply at bitbucket.org  Fri Feb 25 14:59:33 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Fri, 25 Feb 2011 14:59:33 +0100 (CET)
Subject: [pypy-svn] jitviewer default: kill disassembler.py,
	and use the one in lib_pypy instead
Message-ID: <20110225135933.48EBC282BF2@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r88:43c0a992cccb
Date: 2011-02-25 13:58 +0100
http://bitbucket.org/pypy/jitviewer/changeset/43c0a992cccb/

Log:	kill disassembler.py, and use the one in lib_pypy instead

diff --git a/_jitviewer/disassembler.py b/_jitviewer/disassembler.py
deleted file mode 100644
--- a/_jitviewer/disassembler.py
+++ /dev/null
@@ -1,297 +0,0 @@
-"""Disassembler of Python byte code into mnemonics.
-
-Comes from standard library, modified for the purpose of having a structured
-view on things
-"""
-
-import sys
-import types
-import inspect
-
-from opcode import *
-from opcode import __all__ as _opcodes_all
-
-__all__ = ["dis","disassemble","distb","disco"] + _opcodes_all
-del _opcodes_all
-
-class Opcode(object):
-    """ An abstract base class for all opcode implementations
-    """
-    def __init__(self, pos, lineno, arg=None, argstr=''):
-        self.pos = pos
-        self.arg = arg
-        self.argstr = argstr
-        self.lineno = lineno
-        self.line_starts_here = False
-
-    def __repr__(self):
-        if self.arg is None:
-            return "<%s at %d>" % (self.__class__.__name__, self.pos)
-        return "<%s (%s) at %d>" % (self.__class__.__name__, self.arg, self.pos)
-
-class CodeRepresentation(object):
-    """ Representation of opcodes
-    """
-    def __init__(self, opcodes, source):
-        self.opcodes = opcodes
-        self.map = {}
-        current_lineno = None
-        for opcode in opcodes:
-            self.map[opcode.pos] = opcode
-            if opcode.lineno != current_lineno:
-                opcode.line_starts_here = True
-            current_lineno = opcode.lineno
-        self.source = source.split("\n")
-
-def _setup():
-    for opcode in opname:
-        if not opcode.startswith('<'):
-            class O(Opcode):
-                pass
-            opcode = opcode.replace('+', '_')
-            O.__name__ = opcode
-            globals()[opcode] = O
-
-_setup()
-
-def dis(x=None):
-    """Disassemble classes, methods, functions, or code.
-
-    With no argument, disassemble the last traceback.
-
-    """
-    if x is None:
-        distb()
-        return
-    if type(x) is types.InstanceType:
-        x = x.__class__
-    if hasattr(x, 'im_func'):
-        x = x.im_func
-    if hasattr(x, 'func_code'):
-        x = x.func_code
-    if hasattr(x, '__dict__'):
-        xxx
-        items = x.__dict__.items()
-        items.sort()
-        for name, x1 in items:
-            if type(x1) in (types.MethodType,
-                            types.FunctionType,
-                            types.CodeType,
-                            types.ClassType):
-                print "Disassembly of %s:" % name
-                try:
-                    dis(x1)
-                except TypeError, msg:
-                    print "Sorry:", msg
-                print
-    elif hasattr(x, 'co_code'):
-        return disassemble(x)
-    elif isinstance(x, str):
-        return disassemble_string(x)
-    else:
-        raise TypeError, \
-              "don't know how to disassemble %s objects" % \
-              type(x).__name__
-
-def distb(tb=None):
-    """Disassemble a traceback (default: last traceback)."""
-    if tb is None:
-        try:
-            tb = sys.last_traceback
-        except AttributeError:
-            raise RuntimeError, "no last traceback to disassemble"
-        while tb.tb_next: tb = tb.tb_next
-    disassemble(tb.tb_frame.f_code, tb.tb_lasti)
-
-def disassemble(co, lasti=-1):
-    """Disassemble a code object."""
-    source = inspect.getsource(co)
-    code = co.co_code
-    labels = findlabels(code)
-    linestarts = dict(findlinestarts(co))
-    n = len(code)
-    i = 0
-    extended_arg = 0
-    free = None
-    res = []
-    lastline = co.co_firstlineno
-    while i < n:
-        c = code[i]
-        op = ord(c)
-        if i in linestarts:
-            lastline = linestarts[i]
-
-        #if i == lasti:
-        #    xxx
-        #    print '-->',
-        #else:
-        #    xxx
-        #    print '   ',
-        #if i in labels:
-        #    xxx
-        #    print '>>',
-        #else:
-        #    xxx
-        #    print '  ',
-        #xxx
-        pos = i
-        i = i + 1
-        if op >= HAVE_ARGUMENT:
-            oparg = ord(code[i]) + ord(code[i+1])*256 + extended_arg
-            opargstr = str(oparg)
-            extended_arg = 0
-            i = i+2
-            if op == EXTENDED_ARG:
-                extended_arg = oparg*65536L
-            if op in hasconst:
-                opargstr = repr(co.co_consts[oparg])
-            elif op in hasname:
-                opargstr = co.co_names[oparg]
-            elif op in hasjrel:
-                opargstr = 'to ' + repr(i + oparg)
-            elif op in haslocal:
-                opargstr = co.co_varnames[oparg]
-            elif op in hascompare:
-                opargstr = cmp_op[oparg]
-            elif op in hasfree:
-                if free is None:
-                    free = co.co_cellvars + co.co_freevars
-                opargstr = free[oparg]
-        else:
-            oparg = None
-            opargstr = ''
-        opcls = globals()[opname[op].replace('+', '_')]
-        res.append(opcls(pos, lastline, oparg, opargstr))
-    return CodeRepresentation(res, source)
-
-def disassemble_string(code, lasti=-1, varnames=None, names=None,
-                       constants=None):
-    labels = findlabels(code)
-    n = len(code)
-    i = 0
-    while i < n:
-        c = code[i]
-        op = ord(c)
-        if i == lasti:
-            xxx
-            print '-->',
-        else:
-            xxx
-            print '   ',
-        if i in labels:
-            xxx
-            print '>>',
-        else:
-            xxx
-            print '  ',
-        xxxx
-        print repr(i).rjust(4),
-        print opname[op].ljust(15),
-        i = i+1
-        if op >= HAVE_ARGUMENT:
-            oparg = ord(code[i]) + ord(code[i+1])*256
-            i = i+2
-            xxx
-            print repr(oparg).rjust(5),
-            if op in hasconst:
-                if constants:
-                    xxx
-                    print '(' + repr(constants[oparg]) + ')',
-                else:
-                    xxx
-                    print '(%d)'%oparg,
-            elif op in hasname:
-                if names is not None:
-                    xxx
-                    print '(' + names[oparg] + ')',
-                else:
-                    xxx
-                    print '(%d)'%oparg,
-            elif op in hasjrel:
-                xxx
-                print '(to ' + repr(i + oparg) + ')',
-            elif op in haslocal:
-                if varnames:
-                    xxx
-                    print '(' + varnames[oparg] + ')',
-                else:
-                    xxx
-                    print '(%d)' % oparg,
-            elif op in hascompare:
-                xxx
-                print '(' + cmp_op[oparg] + ')',
-        xxx
-        print
-
-disco = disassemble                     # XXX For backwards compatibility
-
-def findlabels(code):
-    """Detect all offsets in a byte code which are jump targets.
-
-    Return the list of offsets.
-
-    """
-    labels = []
-    n = len(code)
-    i = 0
-    while i < n:
-        c = code[i]
-        op = ord(c)
-        i = i+1
-        if op >= HAVE_ARGUMENT:
-            oparg = ord(code[i]) + ord(code[i+1])*256
-            i = i+2
-            label = -1
-            if op in hasjrel:
-                label = i+oparg
-            elif op in hasjabs:
-                label = oparg
-            if label >= 0:
-                if label not in labels:
-                    labels.append(label)
-    return labels
-
-def findlinestarts(code):
-    """Find the offsets in a byte code which are start of lines in the source.
-
-    Generate pairs (offset, lineno) as described in Python/compile.c.
-
-    """
-    byte_increments = [ord(c) for c in code.co_lnotab[0::2]]
-    line_increments = [ord(c) for c in code.co_lnotab[1::2]]
-
-    lastlineno = None
-    lineno = code.co_firstlineno
-    addr = 0
-    for byte_incr, line_incr in zip(byte_increments, line_increments):
-        if byte_incr:
-            if lineno != lastlineno:
-                yield (addr, lineno)
-                lastlineno = lineno
-            addr += byte_incr
-        lineno += line_incr
-    if lineno != lastlineno:
-        yield (addr, lineno)
-
-def _test():
-    """Simple test program to disassemble a file."""
-    if sys.argv[1:]:
-        if sys.argv[2:]:
-            sys.stderr.write("usage: python dis.py [-|file]\n")
-            sys.exit(2)
-        fn = sys.argv[1]
-        if not fn or fn == "-":
-            fn = None
-    else:
-        fn = None
-    if fn is None:
-        f = sys.stdin
-    else:
-        f = open(fn)
-    source = f.read()
-    if fn is not None:
-        f.close()
-    else:
-        fn = "<stdin>"
-    code = compile(source, fn, "exec")
-    dis(code)

diff --git a/_jitviewer/parser.py b/_jitviewer/parser.py
--- a/_jitviewer/parser.py
+++ b/_jitviewer/parser.py
@@ -1,7 +1,6 @@
-
 import re, sys
 from pypy.jit.metainterp.resoperation import rop, opname
-from disassembler import dis
+from lib_pypy.disassembler import dis # imported from the pypy source tree
 from pypy.jit.tool.oparser import OpParser
 
 class Html(str):

diff --git a/_jitviewer/test/test_disassembler.py b/_jitviewer/test/test_disassembler.py
deleted file mode 100644
--- a/_jitviewer/test/test_disassembler.py
+++ /dev/null
@@ -1,34 +0,0 @@
-
-from _jitviewer import disassembler
-import sys
-import py
-
-def f(a, b):
-    return a + b
-
-def g(a, b):
-    c = a+b
-    return c
-
-def test_disassembler():
-    res = disassembler.dis(f)
-    if sys.version_info[:2] != (2, 6):
-        py.test.skip("2.6 only test")
-    assert len(res.opcodes) == 4
-    assert [x.__class__.__name__ for x in res.opcodes] == [
-        'LOAD_FAST', 'LOAD_FAST', 'BINARY_ADD', 'RETURN_VALUE']
-    for i in range(4):
-        assert res.opcodes[i].lineno == f.func_code.co_firstlineno + 1
-    assert res.opcodes[0].argstr == 'a'
-
-def test_line_starting_opcodes():
-    if sys.version_info[:2] != (2, 6):
-        py.test.skip("2.6 only test")
-    res = disassembler.dis(g)
-    assert len(res.opcodes) == 6
-    for i, opcode in enumerate(res.opcodes):
-        if i in (0, 4):
-            assert opcode.__class__.__name__ == 'LOAD_FAST'
-            assert opcode.line_starts_here
-        else:
-            assert not opcode.line_starts_here

From commits-noreply at bitbucket.org  Fri Feb 25 14:59:34 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Fri, 25 Feb 2011 14:59:34 +0100 (CET)
Subject: [pypy-svn] jitviewer default: kill parser.Op and use a subclass
	of	pypy.tool.jitlogparser.parser.Op instead,
	which adds html formatting
Message-ID: <20110225135934.08B98282BF2@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r89:160d38f9d451
Date: 2011-02-25 14:05 +0100
http://bitbucket.org/pypy/jitviewer/changeset/160d38f9d451/

Log:	kill parser.Op and use a subclass of
	pypy.tool.jitlogparser.parser.Op instead, which adds html formatting

diff --git a/_jitviewer/parser.py b/_jitviewer/parser.py
--- a/_jitviewer/parser.py
+++ b/_jitviewer/parser.py
@@ -1,7 +1,8 @@
 import re, sys
+from lib_pypy.disassembler import dis # imported from the pypy source tree
 from pypy.jit.metainterp.resoperation import rop, opname
-from lib_pypy.disassembler import dis # imported from the pypy source tree
 from pypy.jit.tool.oparser import OpParser
+from pypy.tool.jitlogparser.parser import Op
 
 class Html(str):
     def __html__(self):
@@ -30,42 +31,20 @@
         return '%s = %s %s %s' % (self.getres(), self.getarg(0), name, self.getarg(1))
     return f
 
-class Op(object):
-    bridge = None
-    
-    def __init__(self, name, args, res, descr):
-        self.name = name
-        self.args = args
-        self.res = res
-        self.descr = descr
-        self._is_guard = name.startswith('guard_')
-        if self._is_guard:
-            self.guard_no = int(self.descr[len('<Guard'):-1])
-
-    def setfailargs(self, _):
-        pass
+class OpHtml(Op):
+    """
+    Subclass of Op with human-friendly tml representation
+    """
 
     def html_repr(self):
-        s = getattr(self, 'repr_' + self.name, self.generic_repr)()
+        s = getattr(self, 'repr_' + self.name, self.repr)()
         if self.is_guard():
             s = '<span class="guard">guard</span>(' + s + ')'
         return Html(s)
 
-    def getarg(self, i):
-        return self._getvar(self.args[i])
-
-    def getargs(self):
-        return [self._getvar(v) for v in self.args]
-
-    def getres(self):
-        return self._getvar(self.res)
-
     def _getvar(self, v):
         return cssclass(v, v, onmouseover='highlight_var(this)', onmouseout='disable_var(this)')
 
-    def is_guard(self):
-        return self._is_guard
-
     for bin_op, name in [('==', 'int_eq'),
                          ('!=', 'int_ne'),
                          ('==', 'float_eq'),
@@ -123,16 +102,6 @@
         name, field = self.descr.split(' ')[1].rsplit('.', 1)
         return '((%s)%s).%s = %s' % (name, self.getarg(0), field, self.getarg(1))
 
-    def generic_repr(self):
-        arglist = ', '.join(self.getargs())
-        if self.res is not None:
-            return '%s = %s(%s)' % (self.getres(), self.name, arglist)
-        else:
-            return '%s(%s)' % (self.name, arglist)
-
-    def __repr__(self):
-        return '<%s (%s)>' % (self.name, ', '.join([repr(a)
-                                                    for a in self.args]))
 
 class SimpleParser(OpParser):
     def parse_args(self, opname, argspec):
@@ -152,7 +121,7 @@
         return res
 
     def create_op(self, opnum, args, res, descr):
-        return Op(intern(opname[opnum].lower()), args, res, descr)
+        return OpHtml(intern(opname[opnum].lower()), args, res, descr)
 
 class NonCodeError(Exception):
     pass

From commits-noreply at bitbucket.org  Fri Feb 25 14:59:34 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Fri, 25 Feb 2011 14:59:34 +0100 (CET)
Subject: [pypy-svn] jitviewer default: kill SimpleParser,
	replace it with the version found in jitlogparser
Message-ID: <20110225135934.86EB1282BF2@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r90:137d91d598e2
Date: 2011-02-25 14:09 +0100
http://bitbucket.org/pypy/jitviewer/changeset/137d91d598e2/

Log:	kill SimpleParser, replace it with the version found in jitlogparser

diff --git a/_jitviewer/parser.py b/_jitviewer/parser.py
--- a/_jitviewer/parser.py
+++ b/_jitviewer/parser.py
@@ -2,7 +2,7 @@
 from lib_pypy.disassembler import dis # imported from the pypy source tree
 from pypy.jit.metainterp.resoperation import rop, opname
 from pypy.jit.tool.oparser import OpParser
-from pypy.tool.jitlogparser.parser import Op
+from pypy.tool.jitlogparser import parser
 
 class Html(str):
     def __html__(self):
@@ -31,7 +31,7 @@
         return '%s = %s %s %s' % (self.getres(), self.getarg(0), name, self.getarg(1))
     return f
 
-class OpHtml(Op):
+class OpHtml(parser.Op):
     """
     Subclass of Op with human-friendly tml representation
     """
@@ -103,25 +103,9 @@
         return '((%s)%s).%s = %s' % (name, self.getarg(0), field, self.getarg(1))
 
 
-class SimpleParser(OpParser):
-    def parse_args(self, opname, argspec):
-        if not argspec.strip():
-            return [], None
-        if opname == 'debug_merge_point':
-            return argspec.rsplit(", ", 1), None
-        else:
-            args = argspec.split(', ')
-            descr = None
-            if args[-1].startswith('descr='):
-                descr = args[-1][len('descr='):]
-                args = args[:-1]
-            return (args, descr)
+class ParserWithHtmlRepr(parser.SimpleParser):
+    Op = OpHtml
 
-    def box_for_var(self, res):
-        return res
-
-    def create_op(self, opnum, args, res, descr):
-        return OpHtml(intern(opname[opnum].lower()), args, res, descr)
 
 class NonCodeError(Exception):
     pass
@@ -276,8 +260,8 @@
     return nums
 
 def parse(input):
-    return SimpleParser(input, None, {}, 'lltype', None,
-                        nonstrict=True).parse()
+    return ParserWithHtmlRepr(input, None, {}, 'lltype', None,
+                              nonstrict=True).parse()
 
 def slice_debug_merge_points(operations, storage, limit=None):
     """ Slice given operation list into a chain of Bytecode chunks.

From commits-noreply at bitbucket.org  Fri Feb 25 14:59:35 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Fri, 25 Feb 2011 14:59:35 +0100 (CET)
Subject: [pypy-svn] jitviewer default: kill storage.py,
	use the version from the jitlogparser
Message-ID: <20110225135935.89CD9282BF2@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r91:f04731152909
Date: 2011-02-25 14:16 +0100
http://bitbucket.org/pypy/jitviewer/changeset/f04731152909/

Log:	kill storage.py, use the version from the jitlogparser

diff --git a/bin/jitviewer.py b/bin/jitviewer.py
--- a/bin/jitviewer.py
+++ b/bin/jitviewer.py
@@ -22,9 +22,9 @@
 from pypy.tool.logparser import parse_log_file, extract_category
 from _jitviewer.parser import (parse, slice_debug_merge_points, adjust_bridges,
                    parse_log_counts)
-from _jitviewer.storage import LoopStorage
 from _jitviewer.display import CodeRepr, CodeReprNoFile
 import _jitviewer
+from pypy.tool.jitlogparser.storage import LoopStorage
 
 from pygments import highlight
 from pygments.lexers import PythonLexer

diff --git a/_jitviewer/test/test_parser.py b/_jitviewer/test/test_parser.py
--- a/_jitviewer/test/test_parser.py
+++ b/_jitviewer/test/test_parser.py
@@ -1,10 +1,10 @@
 
 from pypy.jit.metainterp.resoperation import ResOperation, rop
 from pypy.jit.metainterp.history import ConstInt, Const
+from pypy.tool.jitlogparser.storage import LoopStorage
 from _jitviewer.parser import parse, Bytecode, Function,\
      slice_debug_merge_points,\
      adjust_bridges, parse_log_counts, cssclass
-from _jitviewer.storage import LoopStorage
 import py
 
 def test_parse():

diff --git a/_jitviewer/storage.py b/_jitviewer/storage.py
deleted file mode 100644
--- a/_jitviewer/storage.py
+++ /dev/null
@@ -1,56 +0,0 @@
-
-""" This file represents a storage mechanism that let us invent unique names
-for all loops and bridges, so http requests can refer to them by name
-"""
-
-import os
-from loops import Function, Bytecode
-from module_finder import gather_all_code_objs
-
-class LoopStorage(object):
-    def __init__(self, extrapath=None):
-        self.loops = None
-        self.functions = {}
-        self.codes = {}
-        self.extrapath = extrapath
-
-    def load_code(self, fname):
-        try:
-            return self.codes[fname]
-        except KeyError:
-            if os.path.isabs(fname):
-                res = gather_all_code_objs(fname)
-            else:
-                if self.extrapath is None:
-                    raise IOError("Cannot find %s" % fname)
-                res = gather_all_code_objs(os.path.join(self.extrapath, fname))
-            self.codes[fname] = res
-            return res
-
-    def reconnect_loops(self, loops):
-        """ Re-connect loops in a way that entry bridges are filtered out
-        and normal bridges are associated with guards. Returning list of
-        normal loops.
-        """
-        res = []
-        guard_dict = {}
-        for loop_no, loop in enumerate(loops):
-            for op in loop.operations:
-                if op.name.startswith('guard_'):
-                    guard_dict[int(op.descr[len('<Guard'):-1])] = (op, loop)
-        for loop in loops:
-            if loop.comment:
-                comment = loop.comment.strip()
-                if 'entry bridge' in comment:
-                    pass
-                elif comment.startswith('# bridge out of'):
-                    no = int(comment[len('# bridge out of Guard '):].split(' ', 1)[0])
-                    op, parent = guard_dict[no]
-                    op.bridge = loop
-                    op.percentage = ((getattr(loop, 'count', 1) * 100) /
-                                     max(getattr(parent, 'count', 1), 1))
-                    loop.no = no
-                    continue
-            res.append(loop)
-        self.loops = res
-        return res

diff --git a/_jitviewer/test/test_storage.py b/_jitviewer/test/test_storage.py
deleted file mode 100644
--- a/_jitviewer/test/test_storage.py
+++ /dev/null
@@ -1,9 +0,0 @@
-
-import py
-from _jitviewer.storage import LoopStorage
-
-def test_load_codes():
-    tmppath = py.test.ensuretemp('load_codes')
-    tmppath.join("x.py").write("def f(): pass") # one code
-    s = LoopStorage(str(tmppath))
-    assert s.load_code(str(tmppath.join('x.py'))) == s.load_code('x.py')

From commits-noreply at bitbucket.org  Fri Feb 25 14:59:36 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Fri, 25 Feb 2011 14:59:36 +0100 (CET)
Subject: [pypy-svn] jitviewer default: kill the class Bytecode,
	use a subclass of jitlogparser.TraceForOpcode instead
Message-ID: <20110225135936.2B9D0282B90@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r92:c4c55bb7e810
Date: 2011-02-25 14:14 +0100
http://bitbucket.org/pypy/jitviewer/changeset/c4c55bb7e810/

Log:	kill the class Bytecode, use a subclass of
	jitlogparser.TraceForOpcode instead

diff --git a/_jitviewer/parser.py b/_jitviewer/parser.py
--- a/_jitviewer/parser.py
+++ b/_jitviewer/parser.py
@@ -107,60 +107,7 @@
     Op = OpHtml
 
 
-class NonCodeError(Exception):
-    pass
-
-class Bytecode(object):
-    filename = None
-    startlineno = 0
-    name = None
-    code = None
-    bytecode_no = 0
-    bytecode_name = None
-    is_bytecode = True
-    inline_level = None
-    
-    def __init__(self, operations, storage):
-        if operations[0].name == 'debug_merge_point':
-            self.inline_level = int(operations[0].args[1])
-            m = re.search('<code object ([<>\w]+), file \'(.+?)\', line (\d+)> #(\d+) (\w+)',
-                         operations[0].getarg(0))
-            if m is None:
-                # a non-code loop, like StrLiteralSearch or something
-                self.bytecode_name = operations[0].args[0].split(" ")[0][1:]
-            else:
-                self.name, self.filename, lineno, bytecode_no, self.bytecode_name = m.groups()
-                self.startlineno = int(lineno)
-                self.bytecode_no = int(bytecode_no)
-        self.operations = operations
-        self.storage = storage
-
-    def repr(self):
-        if self.filename is None:
-            return "Unknown"
-        return "%s, file '%s', line %d" % (self.name, self.filename,
-                                           self.startlineno)
-
-    def getcode(self):
-        if self.code is None:
-            self.code = dis(self.storage.load_code(self.filename)[self.startlineno])
-        return self.code
-
-    def getlineno(self):
-        code = self.getcode()
-        return code.map[self.bytecode_no].lineno
-    lineno = property(getlineno)
-
-    def getline_starts_here(self):
-        code = self.getcode()
-        return code.map[self.bytecode_no].line_starts_here
-    line_starts_here = property(getline_starts_here)
-
-    def __repr__(self):
-        return "[%s]" % ", ".join([repr(op) for op in self.operations])
-
-    def pretty_print(self, out):
-        pass
+class TraceForOpcodeHtml(parser.TraceForOpcode):
 
     def html_repr(self):
         if self.filename is not None:
@@ -289,13 +236,13 @@
     for op in operations:
         if op.name == 'debug_merge_point':
             if so_far:
-                append_to_res(Bytecode(so_far, storage))
+                append_to_res(TraceForOpcodeHtml(so_far, storage))
                 if limit:
                     break
                 so_far = []
         so_far.append(op)
     if so_far:
-        append_to_res(Bytecode(so_far, storage))
+        append_to_res(TraceForOpcodeHtml(so_far, storage))
     # wrap stack back up
     if not stack:
         # no ops whatsoever

diff --git a/_jitviewer/test/test_parser.py b/_jitviewer/test/test_parser.py
--- a/_jitviewer/test/test_parser.py
+++ b/_jitviewer/test/test_parser.py
@@ -1,8 +1,7 @@
-
+from pypy.tool.jitlogparser.storage import LoopStorage
 from pypy.jit.metainterp.resoperation import ResOperation, rop
 from pypy.jit.metainterp.history import ConstInt, Const
-from pypy.tool.jitlogparser.storage import LoopStorage
-from _jitviewer.parser import parse, Bytecode, Function,\
+from _jitviewer.parser import parse, TraceForOpcodeHtml, Function,\
      slice_debug_merge_points,\
      adjust_bridges, parse_log_counts, cssclass
 import py
@@ -36,10 +35,10 @@
 def test_split():
     ops = parse('''
     [i0]
-    debug_merge_point("<code object stuff, file '/tmp/x.py', line 200> #10 ADD", 0)
-    debug_merge_point("<code object stuff, file '/tmp/x.py', line 200> #11 SUB", 0)
+    debug_merge_point("<code object stuff, file '/I/dont/exist.py', line 200> #10 ADD", 0)
+    debug_merge_point("<code object stuff, file '/I/dont/exist.py', line 200> #11 SUB", 0)
     i1 = int_add(i0, 1)
-    debug_merge_point("<code object stuff, file '/tmp/x.py', line 200> #11 SUB", 0)
+    debug_merge_point("<code object stuff, file '/I/dont/exist.py', line 200> #11 SUB", 0)
     i2 = int_add(i1, 1)
     ''')
     res = slice_debug_merge_points(ops.operations, LoopStorage())
@@ -61,36 +60,36 @@
     """)
     res = slice_debug_merge_points(ops.operations, LoopStorage())
     assert len(res.chunks) == 3 # two chunks + inlined call
-    assert isinstance(res.chunks[0], Bytecode)
+    assert isinstance(res.chunks[0], TraceForOpcodeHtml)
     assert isinstance(res.chunks[1], Function)
-    assert isinstance(res.chunks[2], Bytecode)
+    assert isinstance(res.chunks[2], TraceForOpcodeHtml)
     assert res.chunks[1].path == "1"
     assert len(res.chunks[1].chunks) == 3
     
 def test_name():
     ops = parse('''
     [i0]
-    debug_merge_point("<code object stuff, file '/tmp/x.py', line 200> #10 ADD", 0)
-    debug_merge_point("<code object stuff, file '/tmp/x.py', line 201> #11 SUB", 0)
+    debug_merge_point("<code object stuff, file '/I/dont/exist.py', line 200> #10 ADD", 0)
+    debug_merge_point("<code object stuff, file '/I/dont/exist.py', line 201> #11 SUB", 0)
     i1 = int_add(i0, 1)
-    debug_merge_point("<code object stuff, file '/tmp/x.py', line 202> #11 SUB", 0)
+    debug_merge_point("<code object stuff, file '/I/dont/exist.py', line 202> #11 SUB", 0)
     i2 = int_add(i1, 1)
     ''')
     res = slice_debug_merge_points(ops.operations, LoopStorage())
     assert res.repr() == res.chunks[0].repr()
-    assert res.repr() == "stuff, file '/tmp/x.py', line 200"
+    assert res.repr() == "stuff, file '/I/dont/exist.py', line 200"
     assert res.startlineno == 200
-    assert res.filename == '/tmp/x.py'
+    assert res.filename == '/I/dont/exist.py'
     assert res.name == 'stuff'
 
 def test_name_no_first():
     ops = parse('''
     [i0]
     i3 = int_add(i0, 1)
-    debug_merge_point("<code object stuff, file '/tmp/x.py', line 200> #10 ADD", 0)
-    debug_merge_point("<code object stuff, file '/tmp/x.py', line 201> #11 SUB", 0)
+    debug_merge_point("<code object stuff, file '/I/dont/exist.py', line 200> #10 ADD", 0)
+    debug_merge_point("<code object stuff, file '/I/dont/exist.py', line 201> #11 SUB", 0)
     i1 = int_add(i0, 1)
-    debug_merge_point("<code object stuff, file '/tmp/x.py', line 202> #11 SUB", 0)
+    debug_merge_point("<code object stuff, file '/I/dont/exist.py', line 202> #11 SUB", 0)
     i2 = int_add(i1, 1)
     ''')
     res = slice_debug_merge_points(ops.operations, LoopStorage())

From commits-noreply at bitbucket.org  Fri Feb 25 14:59:36 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Fri, 25 Feb 2011 14:59:36 +0100 (CET)
Subject: [pypy-svn] jitviewer default: kill Function and
	slice_debug_merge_points,
	replace them by the versions found in the jitlogparser
Message-ID: <20110225135936.E10DF282B90@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r93:2aaa9b272f33
Date: 2011-02-25 14:26 +0100
http://bitbucket.org/pypy/jitviewer/changeset/2aaa9b272f33/

Log:	kill Function and slice_debug_merge_points, replace them by the
	versions found in the jitlogparser

diff --git a/bin/jitviewer.py b/bin/jitviewer.py
--- a/bin/jitviewer.py
+++ b/bin/jitviewer.py
@@ -20,7 +20,7 @@
 import flask
 import inspect
 from pypy.tool.logparser import parse_log_file, extract_category
-from _jitviewer.parser import (parse, slice_debug_merge_points, adjust_bridges,
+from _jitviewer.parser import (parse, FunctionHtml, adjust_bridges,
                    parse_log_counts)
 from _jitviewer.display import CodeRepr, CodeReprNoFile
 import _jitviewer
@@ -49,8 +49,8 @@
                 is_entry = True
             else:
                 is_entry = False
-            func = slice_debug_merge_points(loop.operations, self.storage,
-                                            limit=1)
+            func = FunctionHtml.from_operations(loop.operations, self.storage,
+                                                limit=1)
             func.count = getattr(loop, 'count', '?')
             loops.append((is_entry, index, func))
         loops.sort(lambda a, b: cmp(b[2].count, a[2].count))
@@ -67,7 +67,7 @@
         no = int(flask.request.args.get('no', '0'))
         orig_loop = self.storage.loops[no]
         ops = adjust_bridges(orig_loop, flask.request.args)
-        loop = slice_debug_merge_points(ops, self.storage)
+        loop = FunctionHtml.from_operations(ops, self.storage)
         path = flask.request.args.get('path', '').split(',')
         if path:
             up = '"' + ','.join(path[:-1]) + '"'

diff --git a/_jitviewer/parser.py b/_jitviewer/parser.py
--- a/_jitviewer/parser.py
+++ b/_jitviewer/parser.py
@@ -117,78 +117,12 @@
         else:
             return self.bytecode_name
 
-class Function(object):
-    filename = None
-    name = None
-    startlineno = 0
-    _linerange = None
-    _lineset = None
-    is_bytecode = False
-    inline_level = None
+class FunctionHtml(parser.Function):
+    TraceForOpcode = TraceForOpcodeHtml
     
-    def __init__(self, chunks, path, storage):
-        self.path = path
-        self.chunks = chunks
-        for chunk in self.chunks:
-            if chunk.filename is not None:
-                self.startlineno = chunk.startlineno
-                self.filename = chunk.filename
-                self.name = chunk.name
-                self.inline_level = chunk.inline_level
-                break
-        self.storage = storage
-
-    def getlinerange(self):
-        if self._linerange is None:
-            self._compute_linerange()
-        return self._linerange
-    linerange = property(getlinerange)
-
-    def getlineset(self):
-        if self._lineset is None:
-            self._compute_linerange()
-        return self._lineset
-    lineset = property(getlineset)
-
-    def _compute_linerange(self):
-        self._lineset = set()
-        minline = sys.maxint
-        maxline = -1
-        for chunk in self.chunks:
-            if chunk.is_bytecode and chunk.filename is not None:
-                lineno = chunk.lineno
-                minline = min(minline, lineno)
-                maxline = max(maxline, lineno)
-                if chunk.line_starts_here or len(chunk.operations) > 1:
-                    self._lineset.add(lineno)
-        if minline == sys.maxint:
-            minline = 0
-            maxline = 0
-        self._linerange = minline, maxline
-
     def html_repr(self):
         return "inlined call to %s in %s" % (self.name, self.filename)
 
-    def repr(self):
-        if self.filename is None:
-            return "Unknown"
-        return "%s, file '%s', line %d" % (self.name, self.filename,
-                                           self.startlineno)
-        
-    def __repr__(self):
-        return "[%s]" % ", ".join([repr(chunk) for chunk in self.chunks])
-
-    def pretty_print(self, out):
-        print >>out, "Loop starting at %s in %s at %d" % (self.name,
-                                        self.filename, self.startlineno)
-        lineno = -1
-        for chunk in self.chunks:
-            if chunk.filename is not None and chunk.lineno != lineno:
-                lineno = chunk.lineno
-                source = chunk.getcode().source[chunk.lineno -
-                                                chunk.startlineno]
-                print >>out, "  ", source
-            chunk.pretty_print(out)
 
 def parse_log_counts(input, loops):
     if not input:
@@ -210,49 +144,6 @@
     return ParserWithHtmlRepr(input, None, {}, 'lltype', None,
                               nonstrict=True).parse()
 
-def slice_debug_merge_points(operations, storage, limit=None):
-    """ Slice given operation list into a chain of Bytecode chunks.
-    Also detect inlined functions and make them Function
-    """
-    stack = []
-
-    def getpath(stack):
-        return ",".join([str(len(v)) for v in stack])
-
-    def append_to_res(bc):
-        if not stack:
-            stack.append([])
-        else:
-            if bc.inline_level is not None and bc.inline_level + 1 != len(stack):
-                if bc.inline_level < len(stack):
-                    last = stack.pop()
-                    stack[-1].append(Function(last, getpath(stack), storage))
-                else:
-                    stack.append([])
-        stack[-1].append(bc)
-
-    so_far = []
-    stack = []
-    for op in operations:
-        if op.name == 'debug_merge_point':
-            if so_far:
-                append_to_res(TraceForOpcodeHtml(so_far, storage))
-                if limit:
-                    break
-                so_far = []
-        so_far.append(op)
-    if so_far:
-        append_to_res(TraceForOpcodeHtml(so_far, storage))
-    # wrap stack back up
-    if not stack:
-        # no ops whatsoever
-        return Function([], getpath(stack), storage)
-    while True:
-        next = stack.pop()
-        if not stack:
-            return Function(next, getpath(stack), storage)
-        stack[-1].append(Function(next, getpath(stack), storage))
-
 def adjust_bridges(loop, bridges):
     """ Slice given loop according to given bridges to follow. Returns a plain
     list of operations.

diff --git a/_jitviewer/test/test_parser.py b/_jitviewer/test/test_parser.py
--- a/_jitviewer/test/test_parser.py
+++ b/_jitviewer/test/test_parser.py
@@ -1,8 +1,7 @@
 from pypy.tool.jitlogparser.storage import LoopStorage
 from pypy.jit.metainterp.resoperation import ResOperation, rop
 from pypy.jit.metainterp.history import ConstInt, Const
-from _jitviewer.parser import parse, TraceForOpcodeHtml, Function,\
-     slice_debug_merge_points,\
+from _jitviewer.parser import parse, TraceForOpcodeHtml, FunctionHtml,\
      adjust_bridges, parse_log_counts, cssclass
 import py
 
@@ -28,7 +27,7 @@
     []
     debug_merge_point("SomeRandomStuff", 0)
     ''')
-    res = slice_debug_merge_points(ops.operations, LoopStorage())
+    res = FunctionHtml.from_operations(ops.operations, LoopStorage())
     assert len(res.chunks) == 1
     assert res.chunks[0].html_repr()
 
@@ -41,7 +40,7 @@
     debug_merge_point("<code object stuff, file '/I/dont/exist.py', line 200> #11 SUB", 0)
     i2 = int_add(i1, 1)
     ''')
-    res = slice_debug_merge_points(ops.operations, LoopStorage())
+    res = FunctionHtml.from_operations(ops.operations, LoopStorage())
     assert len(res.chunks) == 3
     assert len(res.chunks[0].operations) == 1
     assert len(res.chunks[1].operations) == 2
@@ -58,10 +57,10 @@
     debug_merge_point('<code object inner, file 'source.py', line 9> #7 RETURN_VALUE', 1)
     debug_merge_point('<code object inlined_call, file 'source.py', line 12> #31 STORE_FAST', 0)
     """)
-    res = slice_debug_merge_points(ops.operations, LoopStorage())
+    res = FunctionHtml.from_operations(ops.operations, LoopStorage())
     assert len(res.chunks) == 3 # two chunks + inlined call
     assert isinstance(res.chunks[0], TraceForOpcodeHtml)
-    assert isinstance(res.chunks[1], Function)
+    assert isinstance(res.chunks[1], FunctionHtml)
     assert isinstance(res.chunks[2], TraceForOpcodeHtml)
     assert res.chunks[1].path == "1"
     assert len(res.chunks[1].chunks) == 3
@@ -75,7 +74,7 @@
     debug_merge_point("<code object stuff, file '/I/dont/exist.py', line 202> #11 SUB", 0)
     i2 = int_add(i1, 1)
     ''')
-    res = slice_debug_merge_points(ops.operations, LoopStorage())
+    res = FunctionHtml.from_operations(ops.operations, LoopStorage())
     assert res.repr() == res.chunks[0].repr()
     assert res.repr() == "stuff, file '/I/dont/exist.py', line 200"
     assert res.startlineno == 200
@@ -92,7 +91,7 @@
     debug_merge_point("<code object stuff, file '/I/dont/exist.py', line 202> #11 SUB", 0)
     i2 = int_add(i1, 1)
     ''')
-    res = slice_debug_merge_points(ops.operations, LoopStorage())
+    res = FunctionHtml.from_operations(ops.operations, LoopStorage())
     assert res.repr() == res.chunks[1].repr()
 
 def test_lineno():
@@ -104,7 +103,7 @@
     debug_merge_point("<code object f, file '%(fname)s', line 2> #6 BINARY_ADD", 0)
     debug_merge_point("<code object f, file '%(fname)s', line 2> #7 RETURN_VALUE", 0)
     ''' % locals())
-    res = slice_debug_merge_points(ops.operations, LoopStorage())
+    res = FunctionHtml.from_operations(ops.operations, LoopStorage())
     assert res.chunks[1].lineno == 3
 
 def test_linerange():
@@ -117,7 +116,7 @@
     debug_merge_point("<code object f, file '%(fname)s', line 5> #28 LOAD_CONST", 0)
     debug_merge_point("<code object f, file '%(fname)s', line 5> #6 SETUP_LOOP", 0)
     ''' % locals())
-    res = slice_debug_merge_points(ops.operations, LoopStorage())
+    res = FunctionHtml.from_operations(ops.operations, LoopStorage())
     assert res.linerange == (7, 9)
     assert res.lineset == set([7, 8, 9])
 
@@ -129,7 +128,7 @@
     guard_class(p6, 144264192, descr=<Guard2>)
     p12 = getfield_gc(p6, descr=<GcPtrFieldDescr pypy.objspace.std.iterobject.W_AbstractSeqIterObject.inst_w_seq 12>)
     """ % locals())
-    res = slice_debug_merge_points(ops.operations, LoopStorage())
+    res = FunctionHtml.from_operations(ops.operations, LoopStorage())
     assert res.lineset
 
 def test_reassign_loops():
@@ -173,7 +172,7 @@
     loop = parse("""
     debug_merge_point('StrLiteralSearch at 11/51 [17, 8, 3, 1, 1, 1, 1, 51, 0, 19, 51, 1]', 0)
     """)
-    ops = slice_debug_merge_points(loop.operations, LoopStorage())
+    ops = FunctionHtml.from_operations(loop.operations, LoopStorage())
     chunk = ops.chunks[0]
     assert chunk.bytecode_name == 'StrLiteralSearch'
 

From commits-noreply at bitbucket.org  Fri Feb 25 14:59:39 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Fri, 25 Feb 2011 14:59:39 +0100 (CET)
Subject: [pypy-svn] jitviewer default: kill adjust_bridges,
	use the one in jitlogparser
Message-ID: <20110225135939.967FD282C27@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r94:c638d97d3b8c
Date: 2011-02-25 14:34 +0100
http://bitbucket.org/pypy/jitviewer/changeset/c638d97d3b8c/

Log:	kill adjust_bridges, use the one in jitlogparser

diff --git a/bin/jitviewer.py b/bin/jitviewer.py
--- a/bin/jitviewer.py
+++ b/bin/jitviewer.py
@@ -20,11 +20,12 @@
 import flask
 import inspect
 from pypy.tool.logparser import parse_log_file, extract_category
-from _jitviewer.parser import (parse, FunctionHtml, adjust_bridges,
-                   parse_log_counts)
+from pypy.tool.jitlogparser.storage import LoopStorage
+from pypy.tool.jitlogparser.parser import adjust_bridges
+#
+from _jitviewer.parser import parse, FunctionHtml, parse_log_counts
 from _jitviewer.display import CodeRepr, CodeReprNoFile
 import _jitviewer
-from pypy.tool.jitlogparser.storage import LoopStorage
 
 from pygments import highlight
 from pygments.lexers import PythonLexer

diff --git a/_jitviewer/parser.py b/_jitviewer/parser.py
--- a/_jitviewer/parser.py
+++ b/_jitviewer/parser.py
@@ -143,21 +143,3 @@
 def parse(input):
     return ParserWithHtmlRepr(input, None, {}, 'lltype', None,
                               nonstrict=True).parse()
-
-def adjust_bridges(loop, bridges):
-    """ Slice given loop according to given bridges to follow. Returns a plain
-    list of operations.
-    """
-    ops = loop.operations
-    res = []
-    i = 0
-    while i < len(ops):
-        op = ops[i]
-        if op.is_guard() and bridges.get('loop-' + str(op.guard_no), None):
-            res.append(op)
-            i = 0
-            ops = op.bridge.operations
-        else:
-            res.append(op)
-            i += 1
-    return res

diff --git a/_jitviewer/test/test_parser.py b/_jitviewer/test/test_parser.py
--- a/_jitviewer/test/test_parser.py
+++ b/_jitviewer/test/test_parser.py
@@ -2,7 +2,7 @@
 from pypy.jit.metainterp.resoperation import ResOperation, rop
 from pypy.jit.metainterp.history import ConstInt, Const
 from _jitviewer.parser import parse, TraceForOpcodeHtml, FunctionHtml,\
-     adjust_bridges, parse_log_counts, cssclass
+     parse_log_counts, cssclass
 import py
 
 def test_parse():
@@ -153,21 +153,6 @@
     assert loops[0].operations[0].bridge.no == 18
     assert loops[0].operations[0].percentage == 30
 
-def test_adjust_bridges():
-    main = parse('''
-    [v0]
-    guard_false(v0, descr=<Guard13>)
-    guard_true(v0, descr=<Guard5>)
-    ''')
-    bridge = parse('''
-    # bridge out of Guard 13
-    []
-    int_add(0, 1)
-    ''')
-    loops = LoopStorage().reconnect_loops([main, bridge])
-    assert adjust_bridges(main, {})[1].name == 'guard_true'
-    assert adjust_bridges(main, {'loop-13': True})[1].name == 'int_add'
-
 def test_parsing_strliteral():
     loop = parse("""
     debug_merge_point('StrLiteralSearch at 11/51 [17, 8, 3, 1, 1, 1, 1, 51, 0, 19, 51, 1]', 0)

From commits-noreply at bitbucket.org  Fri Feb 25 14:59:41 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Fri, 25 Feb 2011 14:59:41 +0100 (CET)
Subject: [pypy-svn] jitviewer default: kill parse(),
	use the version in the jitlogparser
Message-ID: <20110225135941.E3899282C1A@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r95:de507e82569f
Date: 2011-02-25 14:41 +0100
http://bitbucket.org/pypy/jitviewer/changeset/de507e82569f/

Log:	kill parse(), use the version in the jitlogparser

diff --git a/bin/jitviewer.py b/bin/jitviewer.py
--- a/bin/jitviewer.py
+++ b/bin/jitviewer.py
@@ -23,7 +23,7 @@
 from pypy.tool.jitlogparser.storage import LoopStorage
 from pypy.tool.jitlogparser.parser import adjust_bridges
 #
-from _jitviewer.parser import parse, FunctionHtml, parse_log_counts
+from _jitviewer.parser import ParserWithHtmlRepr, FunctionHtml, parse_log_counts
 from _jitviewer.display import CodeRepr, CodeReprNoFile
 import _jitviewer
 
@@ -137,7 +137,8 @@
     else:
         port = int(sys.argv[2])
     storage = LoopStorage(extra_path)
-    loops = [parse(l) for l in extract_category(log, "jit-log-opt-")]
+    loops = [ParserWithHtmlRepr.parse_from_input(l)
+             for l in extract_category(log, "jit-log-opt-")]
     parse_log_counts(extract_category(log, 'jit-backend-count'), loops)
     storage.reconnect_loops(loops)
     app = OverrideFlask('__name__', root_path=PATH)

diff --git a/_jitviewer/parser.py b/_jitviewer/parser.py
--- a/_jitviewer/parser.py
+++ b/_jitviewer/parser.py
@@ -139,7 +139,3 @@
             loops[i].count = count
             i += 1
     return nums
-
-def parse(input):
-    return ParserWithHtmlRepr(input, None, {}, 'lltype', None,
-                              nonstrict=True).parse()

From commits-noreply at bitbucket.org  Fri Feb 25 14:59:42 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Fri, 25 Feb 2011 14:59:42 +0100 (CET)
Subject: [pypy-svn] jitviewer default: typo
Message-ID: <20110225135942.7CE1F282BF7@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r96:bce30a06ccd2
Date: 2011-02-25 14:42 +0100
http://bitbucket.org/pypy/jitviewer/changeset/bce30a06ccd2/

Log:	typo

diff --git a/_jitviewer/parser.py b/_jitviewer/parser.py
--- a/_jitviewer/parser.py
+++ b/_jitviewer/parser.py
@@ -33,7 +33,7 @@
 
 class OpHtml(parser.Op):
     """
-    Subclass of Op with human-friendly tml representation
+    Subclass of Op with human-friendly html representation
     """
 
     def html_repr(self):

From commits-noreply at bitbucket.org  Fri Feb 25 14:59:43 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Fri, 25 Feb 2011 14:59:43 +0100 (CET)
Subject: [pypy-svn] jitviewer default: kill the (most) part of test_parser
	which is	already covered by the test_parser in the
	jitlogparser, and leave only the html-related stuff
Message-ID: <20110225135943.254B7282BF7@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r97:4883ec3e7559
Date: 2011-02-25 14:49 +0100
http://bitbucket.org/pypy/jitviewer/changeset/4883ec3e7559/

Log:	kill the (most) part of test_parser which is already covered by the
	test_parser in the jitlogparser, and leave only the html-related
	stuff

diff --git a/_jitviewer/test/test_parser.py b/_jitviewer/test/test_parser.py
--- a/_jitviewer/test/test_parser.py
+++ b/_jitviewer/test/test_parser.py
@@ -1,182 +1,19 @@
-from pypy.tool.jitlogparser.storage import LoopStorage
-from pypy.jit.metainterp.resoperation import ResOperation, rop
-from pypy.jit.metainterp.history import ConstInt, Const
-from _jitviewer.parser import parse, TraceForOpcodeHtml, FunctionHtml,\
-     parse_log_counts, cssclass
+from _jitviewer.parser import ParserWithHtmlRepr, parse_log_counts, cssclass
 import py
 
-def test_parse():
+def parse(input):
+    return ParserWithHtmlRepr.parse_from_input(input)
+
+def test_html_repr():
     ops = parse('''
     [i7]
     i9 = int_lt(i7, 1003)
     guard_true(i9, descr=<Guard2>) []
     i13 = getfield_raw(151937600, descr=<SignedFieldDescr pypysig_long_struct.c_value 0>)
     ''').operations
-    assert len(ops) == 3
-    assert ops[0].name == 'int_lt'
-    assert ops[1].name == 'guard_true'
-    assert ops[1].descr is not None
-    assert ops[0].res == 'i9'
     assert ops[0].html_repr().plaintext() == 'i9 = i7 < 1003'
-    assert ops[2].descr is not None
-    assert len(ops[2].args) == 1
     assert ops[2].html_repr().plaintext() == 'i13 = ((pypysig_long_struct)151937600).value'
 
-def test_parse_non_code():
-    ops = parse('''
-    []
-    debug_merge_point("SomeRandomStuff", 0)
-    ''')
-    res = FunctionHtml.from_operations(ops.operations, LoopStorage())
-    assert len(res.chunks) == 1
-    assert res.chunks[0].html_repr()
-
-def test_split():
-    ops = parse('''
-    [i0]
-    debug_merge_point("<code object stuff, file '/I/dont/exist.py', line 200> #10 ADD", 0)
-    debug_merge_point("<code object stuff, file '/I/dont/exist.py', line 200> #11 SUB", 0)
-    i1 = int_add(i0, 1)
-    debug_merge_point("<code object stuff, file '/I/dont/exist.py', line 200> #11 SUB", 0)
-    i2 = int_add(i1, 1)
-    ''')
-    res = FunctionHtml.from_operations(ops.operations, LoopStorage())
-    assert len(res.chunks) == 3
-    assert len(res.chunks[0].operations) == 1
-    assert len(res.chunks[1].operations) == 2
-    assert len(res.chunks[2].operations) == 2
-    assert res.chunks[2].bytecode_no == 11
-
-def test_inlined_call():
-    ops = parse("""
-    []
-    debug_merge_point('<code object inlined_call, file 'source.py', line 12> #28 CALL_FUNCTION', 0)
-    i18 = getfield_gc(p0, descr=<BoolFieldDescr pypy.interpreter.pyframe.PyFrame.inst_is_being_profiled 89>)
-    debug_merge_point('<code object inner, file 'source.py', line 9> #0 LOAD_FAST', 1)
-    debug_merge_point('<code object inner, file 'source.py', line 9> #3 LOAD_CONST', 1)
-    debug_merge_point('<code object inner, file 'source.py', line 9> #7 RETURN_VALUE', 1)
-    debug_merge_point('<code object inlined_call, file 'source.py', line 12> #31 STORE_FAST', 0)
-    """)
-    res = FunctionHtml.from_operations(ops.operations, LoopStorage())
-    assert len(res.chunks) == 3 # two chunks + inlined call
-    assert isinstance(res.chunks[0], TraceForOpcodeHtml)
-    assert isinstance(res.chunks[1], FunctionHtml)
-    assert isinstance(res.chunks[2], TraceForOpcodeHtml)
-    assert res.chunks[1].path == "1"
-    assert len(res.chunks[1].chunks) == 3
-    
-def test_name():
-    ops = parse('''
-    [i0]
-    debug_merge_point("<code object stuff, file '/I/dont/exist.py', line 200> #10 ADD", 0)
-    debug_merge_point("<code object stuff, file '/I/dont/exist.py', line 201> #11 SUB", 0)
-    i1 = int_add(i0, 1)
-    debug_merge_point("<code object stuff, file '/I/dont/exist.py', line 202> #11 SUB", 0)
-    i2 = int_add(i1, 1)
-    ''')
-    res = FunctionHtml.from_operations(ops.operations, LoopStorage())
-    assert res.repr() == res.chunks[0].repr()
-    assert res.repr() == "stuff, file '/I/dont/exist.py', line 200"
-    assert res.startlineno == 200
-    assert res.filename == '/I/dont/exist.py'
-    assert res.name == 'stuff'
-
-def test_name_no_first():
-    ops = parse('''
-    [i0]
-    i3 = int_add(i0, 1)
-    debug_merge_point("<code object stuff, file '/I/dont/exist.py', line 200> #10 ADD", 0)
-    debug_merge_point("<code object stuff, file '/I/dont/exist.py', line 201> #11 SUB", 0)
-    i1 = int_add(i0, 1)
-    debug_merge_point("<code object stuff, file '/I/dont/exist.py', line 202> #11 SUB", 0)
-    i2 = int_add(i1, 1)
-    ''')
-    res = FunctionHtml.from_operations(ops.operations, LoopStorage())
-    assert res.repr() == res.chunks[1].repr()
-
-def test_lineno():
-    fname = str(py.path.local(__file__).join('..', 'x.py'))
-    ops = parse('''
-    [i0, i1]
-    debug_merge_point("<code object f, file '%(fname)s', line 2> #0 LOAD_FAST", 0)
-    debug_merge_point("<code object f, file '%(fname)s', line 2> #3 LOAD_FAST", 0)
-    debug_merge_point("<code object f, file '%(fname)s', line 2> #6 BINARY_ADD", 0)
-    debug_merge_point("<code object f, file '%(fname)s', line 2> #7 RETURN_VALUE", 0)
-    ''' % locals())
-    res = FunctionHtml.from_operations(ops.operations, LoopStorage())
-    assert res.chunks[1].lineno == 3
-
-def test_linerange():
-    fname = str(py.path.local(__file__).join('..', 'x.py'))
-    ops = parse('''
-    [i0, i1]
-    debug_merge_point("<code object f, file '%(fname)s', line 5> #9 LOAD_FAST", 0)
-    debug_merge_point("<code object f, file '%(fname)s', line 5> #12 LOAD_CONST", 0)
-    debug_merge_point("<code object f, file '%(fname)s', line 5> #22 LOAD_CONST", 0)
-    debug_merge_point("<code object f, file '%(fname)s', line 5> #28 LOAD_CONST", 0)
-    debug_merge_point("<code object f, file '%(fname)s', line 5> #6 SETUP_LOOP", 0)
-    ''' % locals())
-    res = FunctionHtml.from_operations(ops.operations, LoopStorage())
-    assert res.linerange == (7, 9)
-    assert res.lineset == set([7, 8, 9])
-
-def test_linerange_notstarts():
-    fname = str(py.path.local(__file__).join('..', 'x.py'))
-    ops = parse("""
-    [p6, p1]
-    debug_merge_point('<code object h, file '%(fname)s', line 11> #17 FOR_ITER', 0)
-    guard_class(p6, 144264192, descr=<Guard2>)
-    p12 = getfield_gc(p6, descr=<GcPtrFieldDescr pypy.objspace.std.iterobject.W_AbstractSeqIterObject.inst_w_seq 12>)
-    """ % locals())
-    res = FunctionHtml.from_operations(ops.operations, LoopStorage())
-    assert res.lineset
-
-def test_reassign_loops():
-    main = parse('''
-    [v0]
-    guard_false(v0, descr=<Guard18>) []
-    ''')
-    main.count = 10
-    bridge = parse('''
-    # bridge out of Guard 18 with 13 ops
-    [i0, i1]
-    int_add(i0, i1)
-    ''')
-    bridge.count = 3
-    entry_bridge = parse('''
-    # Loop 3 : entry bridge
-    []
-    ''')
-    loops = LoopStorage().reconnect_loops([main, bridge, entry_bridge])
-    assert len(loops) == 2
-    assert len(loops[0].operations[0].bridge.operations) == 1
-    assert loops[0].operations[0].bridge.no == 18
-    assert loops[0].operations[0].percentage == 30
-
-def test_parsing_strliteral():
-    loop = parse("""
-    debug_merge_point('StrLiteralSearch at 11/51 [17, 8, 3, 1, 1, 1, 1, 51, 0, 19, 51, 1]', 0)
-    """)
-    ops = FunctionHtml.from_operations(loop.operations, LoopStorage())
-    chunk = ops.chunks[0]
-    assert chunk.bytecode_name == 'StrLiteralSearch'
-
-LINES = '''
-0:3
-1:3
-2:604
-3:396
-4:102
-5:2000
-6:3147
-7:2445
-8:2005
-9:2000
-10:1420
-11:40
-12:0
-'''.split("\n")
-
 def test_parse_log_count():
     py.test.skip('fixme')
     class Loop(object):

From commits-noreply at bitbucket.org  Fri Feb 25 14:59:43 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Fri, 25 Feb 2011 14:59:43 +0100 (CET)
Subject: [pypy-svn] jitviewer default: kill module_finder,
	it is no longer used here (but a copy is now present in
	the	jitlogparser
Message-ID: <20110225135943.D1959282B8B@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r98:bee2d3ff0ba1
Date: 2011-02-25 14:52 +0100
http://bitbucket.org/pypy/jitviewer/changeset/bee2d3ff0ba1/

Log:	kill module_finder, it is no longer used here (but a copy is now
	present in the jitlogparser

diff --git a/_jitviewer/module_finder.py b/_jitviewer/module_finder.py
deleted file mode 100644
--- a/_jitviewer/module_finder.py
+++ /dev/null
@@ -1,25 +0,0 @@
-
-import os, sys, marshal, types, struct, imp
-
-def _all_codes_from(code):
-    res = {}
-    more = [code]
-    while more:
-        next = more.pop()
-        res[next.co_firstlineno] = next
-        more += [co for co in next.co_consts
-                 if isinstance(co, types.CodeType)]
-    return res
-
-def gather_all_code_objs(fname):
-    """ Gathers all code objects from a give fname and sorts them by
-    starting lineno
-    """
-    fname = str(fname)
-    if fname.endswith('.pyc'):
-        code = compile(open(fname[:-1]).read(), fname, 'exec')
-    elif fname.endswith('.py'):
-        code = compile(open(fname).read(), fname, 'exec')
-    else:
-        raise Exception("Unknown file extension: %s" % fname)
-    return _all_codes_from(code)

diff --git a/_jitviewer/test/test_module.py b/_jitviewer/test/test_module.py
deleted file mode 100644
--- a/_jitviewer/test/test_module.py
+++ /dev/null
@@ -1,21 +0,0 @@
-
-import py
-from _jitviewer.module_finder import gather_all_code_objs
-import re, sys
-
-def setup_module(mod):
-    if sys.version_info[:2] != (2.6):
-        py.test.skip("Specific python 2.6 tests")
-
-def test_gather_code_py():
-    fname = re.__file__
-    codes = gather_all_code_objs(fname)
-    assert len(codes) == 21
-    assert sorted(codes.keys()) == [102, 134, 139, 144, 153, 164, 169, 181, 188, 192, 197, 206, 229, 251, 266, 271, 277, 285, 293, 294, 308]
-
-def test_load_code():
-    fname = re.__file__
-    code = gather_all_code_objs(fname)[144]
-    assert code.co_name == 'sub'
-    assert code.co_filename == '/usr/lib/python2.6/re.py'
-    assert code.co_firstlineno == 144

From commits-noreply at bitbucket.org  Fri Feb 25 14:59:44 2011
From: commits-noreply at bitbucket.org (antocuni)
Date: Fri, 25 Feb 2011 14:59:44 +0100 (CET)
Subject: [pypy-svn] jitviewer default: kill unused (and broken) import
Message-ID: <20110225135944.8A322282B8B@codespeak.net>

Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: 
Changeset: r99:c54dd7a9f8ef
Date: 2011-02-25 14:58 +0100
http://bitbucket.org/pypy/jitviewer/changeset/c54dd7a9f8ef/

Log:	kill unused (and broken) import

diff --git a/_jitviewer/display.py b/_jitviewer/display.py
--- a/_jitviewer/display.py
+++ b/_jitviewer/display.py
@@ -1,6 +1,5 @@
+from pypy.jit.metainterp.resoperation import rop
 
-from pypy.jit.metainterp.resoperation import rop
-from loops import Bytecode
 
 class LineRepr(object):
     """ A representation of a single line

From commits-noreply at bitbucket.org  Fri Feb 25 15:42:54 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Fri, 25 Feb 2011 15:42:54 +0100 (CET)
Subject: [pypy-svn] pypy default: Test and fix for an obscure case.
Message-ID: <20110225144254.90CF4282B8B@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r42281:99f9d995f74c
Date: 2011-02-25 14:36 +0100
http://bitbucket.org/pypy/pypy/changeset/99f9d995f74c/

Log:	Test and fix for an obscure case.

diff --git a/pypy/module/__builtin__/test/test_builtin.py b/pypy/module/__builtin__/test/test_builtin.py
--- a/pypy/module/__builtin__/test/test_builtin.py
+++ b/pypy/module/__builtin__/test/test_builtin.py
@@ -166,6 +166,12 @@
         assert sum([1,2,3]) ==6
         assert sum([],5) ==5
         assert sum([1,2,3],4) ==10
+        #
+        class Foo(object):
+            def __radd__(self, other):
+                assert other is None
+                return 42
+        assert sum([Foo()], None) == 42
 
     def test_type_selftest(self):
         assert type(type) is type

diff --git a/pypy/module/__builtin__/functional.py b/pypy/module/__builtin__/functional.py
--- a/pypy/module/__builtin__/functional.py
+++ b/pypy/module/__builtin__/functional.py
@@ -329,15 +329,13 @@
         result_w.append(w_res)
     return result_w
 
-def sum(space, w_sequence, w_start=None):
+def sum(space, w_sequence, w_start=0):
     """sum(sequence[, start]) -> value
 
 Returns the sum of a sequence of numbers (NOT strings) plus the value
 of parameter 'start' (which defaults to 0).  When the sequence is
 empty, returns start."""
-    if space.is_w(w_start, space.w_None):
-        w_start = space.wrap(0)
-    elif space.is_true(space.isinstance(w_start, space.w_basestring)):
+    if space.is_true(space.isinstance(w_start, space.w_basestring)):
         msg = "sum() can't sum strings"
         raise OperationError(space.w_TypeError, space.wrap(msg))
     w_iter = space.iter(w_sequence)

From commits-noreply at bitbucket.org  Fri Feb 25 15:42:55 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Fri, 25 Feb 2011 15:42:55 +0100 (CET)
Subject: [pypy-svn] pypy refactor-rerase: (cfbolz, arigo)  Initial work.
Message-ID: <20110225144255.7ABEB282B8B@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: refactor-rerase
Changeset: r42282:0c75afdcd413
Date: 2011-02-25 14:39 +0100
http://bitbucket.org/pypy/pypy/changeset/0c75afdcd413/

Log:	(cfbolz, arigo) Initial work.

diff --git a/pypy/rlib/rerased.py b/pypy/rlib/rerased.py
--- a/pypy/rlib/rerased.py
+++ b/pypy/rlib/rerased.py
@@ -52,14 +52,75 @@
     return isinstance(e._x, int)
 
 
+class ErasingPairIdentity(object):
+    def __init__(self, name):
+        self.name = name
+    def __repr__(self):
+        return 'ErasingPairIdentity(%r)' % self.name
+
+def new_erasing_pair(name):
+    identity = ErasingPairIdentity(name)
+
+    def erase(x):
+        return Erased(x, identity)
+
+    def unerase(y):
+        assert y._identity is identity
+        return y._x
+
+    def _getdict(bk):
+        try:
+            dict = bk._erasing_pairs_tunnel
+        except AttributeError:
+            dict = bk._erasing_pairs_tunnel = {}
+        return dict
+
+    class Entry(ExtRegistryEntry):
+        _about_ = erase
+
+        def compute_result_annotation(self, s_obj):
+            bk = self.bookkeeper
+            dict = _getdict(bk)
+            s_previousobj, reflowpositions = dict.setdefault(
+                identity, (annmodel.s_ImpossibleValue, {}))
+            s_obj = annmodel.unionof(s_previousobj, s_obj)
+            if s_obj != s_previousobj:
+                dict[identity] = (s_obj, reflowpositions)
+                for position in reflowpositions:
+                    bk.annotator.reflowfromposition(position)
+            return SomeErased()
+
+        def specialize_call(self, hop):
+            return hop.r_result.specialize_call(hop)
+
+    class Entry(ExtRegistryEntry):
+        _about_ = unerase
+
+        def compute_result_annotation(self, s_obj):
+            assert SomeErased().contains(s_obj)
+            bk = self.bookkeeper
+            dict = _getdict(bk)
+            s_obj, reflowpositions = dict.setdefault(
+                identity, (annmodel.s_ImpossibleValue, []))
+            reflowpositions[bk.position_key] = True
+            return s_obj
+
+        def specialize_call(self, hop):
+            v, t = hop.inputargs(hop.args_r[0], lltype.Void)
+            return hop.genop('cast_opaque_ptr', [v], resulttype = hop.r_result)
+
+    return erase, unerase
+
+
 # ---------- implementation-specific ----------
 
 class Erased(object):
     _list_item_type = None
-    def __init__(self, x):
+    def __init__(self, x, identity=None):
         self._x = x
+        self._identity = identity
     def __repr__(self):
-        return "Erased(%r)" % (self._x, )
+        return "Erased(%r, %r)" % (self._x, self._identity)
 
 class Entry(ExtRegistryEntry):
     _about_ = erase

diff --git a/pypy/rlib/test/test_rerased.py b/pypy/rlib/test/test_rerased.py
--- a/pypy/rlib/test/test_rerased.py
+++ b/pypy/rlib/test/test_rerased.py
@@ -159,3 +159,77 @@
     interpret(l, [0])
     interpret(l, [1])
     interpret(l, [2])
+
+# ____________________________________________________________
+
+def test_erasing_pair():
+    erase, unerase = new_erasing_pair("test1")
+    class X:
+        pass
+    x = X()
+    erased = erase(x)
+    assert unerase(erased) is x
+    #
+    assert not is_integer(erased)
+    #
+    erase2, unerase2 = new_erasing_pair("test2")
+    py.test.raises(AssertionError, unerase2, erased)
+
+def test_annotate_erasing_pair():
+    erase, unerase = new_erasing_pair("test1")
+    erase2, unerase2 = new_erasing_pair("test2")
+    class Foo:
+        pass
+    #
+    def make(n):
+        if n > 5:
+            return erase([5, 6, n-6])
+        else:
+            foo = Foo()
+            foo.bar = n+1
+            return erase2(foo)
+
+    def check(x, n):
+        if n > 5:
+            return unerase(x)[2]
+        else:
+            return unerase2(x).bar
+
+    def f(n):
+        x = make(n)
+        return check(x, n)
+    #
+    a = RPythonAnnotator()
+    s = a.build_types(f, [int])
+    assert isinstance(s, annmodel.SomeInteger)
+
+def test_annotate_reflowing():
+    erase, unerase = new_erasing_pair("test1")
+    class A: pass
+    class B(A): pass
+    class C(B): pass
+    class D(C): pass
+
+    def f():
+        x = erase(None)
+        while True:
+            inst = unerase(x)
+            if inst is None:
+                inst = D()
+                x = erase(inst)
+            elif isinstance(inst, D):
+                inst = C()
+                x = erase(inst)
+            elif isinstance(inst, C):
+                inst = B()
+                x = erase(inst)
+            elif isinstance(inst, B):
+                inst = A()
+                x = erase(inst)
+            else:
+                return inst
+    #
+    a = RPythonAnnotator()
+    s = a.build_types(f, [])
+    assert isinstance(s, annmodel.SomeInstance)
+    assert s.classdef == a.bookkeeper.getuniqueclassdef(A)

From commits-noreply at bitbucket.org  Fri Feb 25 15:42:56 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Fri, 25 Feb 2011 15:42:56 +0100 (CET)
Subject: [pypy-svn] pypy refactor-rerase: (cfbolz, arigo)
Message-ID: <20110225144256.CDECE2A202E@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: refactor-rerase
Changeset: r42283:2aae7e82f9d8
Date: 2011-02-25 15:40 +0100
http://bitbucket.org/pypy/pypy/changeset/2aae7e82f9d8/

Log:	(cfbolz, arigo)

	We only support the pair mechanism. Make it rtype.

diff --git a/pypy/rlib/rerased.py b/pypy/rlib/rerased.py
--- a/pypy/rlib/rerased.py
+++ b/pypy/rlib/rerased.py
@@ -1,6 +1,18 @@
 """ Contains a mechanism for turning any class instance and any integer into a
 pointer-like thing. Gives full control over pointer tagging, i.e. there won't
-be tag checks everywhere in the C code. """
+be tag checks everywhere in the C code.
+
+Usage:  erasestuff, unerasestuff = new_erasing_pair('stuff')
+
+An erasestuff(x) object contains a reference to 'x'.  Nothing can be done with
+this object, except calling unerasestuff(), which returns 'x' again.  The point
+is that all erased objects can be mixed together, whether they are instances,
+lists, strings, etc.  As a special case, an erased object can also be an
+integer fitting into 31/63 bits, with erase_int() and unerase_int().
+
+Warning: some care is needed to make sure that you call the unerase function
+corresponding to the original creator's erase function.  Otherwise, segfault.
+"""
 
 import sys
 from pypy.annotation import model as annmodel
@@ -15,49 +27,59 @@
 
 
 
-def erase(x):
-    """Creates an 'erased' object that contains a reference to 'x'. Nothing can
-    be done with this object, except calling unerase(y, <type>) on it.
-    x needs to be either an instance or an integer fitting into 31/63 bits."""
-    if isinstance(x, int):
-        res = 2 * x + 1
-        if res > sys.maxint or res < -sys.maxint - 1:
-            raise OverflowError
-    assert not isinstance(x, list)
-    return Erased(x)
+def erase_int(x):
+    assert isinstance(x, int)
+    res = 2 * x + 1
+    if res > sys.maxint or res < -sys.maxint - 1:
+        raise OverflowError
+    return Erased(x, _identity_for_ints)
 
-def erase_fixedsizelist(l, type):
-    assert isinstance(l, list)
-    result = Erased(l)
-    result._list_item_type = type
-    return result
-
-def unerase(y, type):
-    """Turn an erased object back into an object of type 'type'."""
-    if y._x is None:
-        return None
-    assert isinstance(y._x, type)
+def unerase_int(y):
+    assert y._identity is _identity_for_ints
+    assert isinstance(y._x, int)
     return y._x
 
-def unerase_fixedsizelist(y, type):
-    if y._x is None:
-        return None
-    assert isinstance(y._x, list)
-    if y._x:
-        assert isinstance(y._x[0], type)
-    return y._x
-
-def is_integer(e):
-    """Gives information whether the erased argument is a tagged integer or not."""
-    return isinstance(e._x, int)
-
 
 class ErasingPairIdentity(object):
+
     def __init__(self, name):
         self.name = name
+
     def __repr__(self):
         return 'ErasingPairIdentity(%r)' % self.name
 
+    def _getdict(self, bk):
+        try:
+            dict = bk._erasing_pairs_tunnel
+        except AttributeError:
+            dict = bk._erasing_pairs_tunnel = {}
+        return dict
+
+    def enter_tunnel(self, bookkeeper, s_obj):
+        dict = self._getdict(bookkeeper)
+        s_previousobj, reflowpositions = dict.setdefault(
+            self, (annmodel.s_ImpossibleValue, {}))
+        s_obj = annmodel.unionof(s_previousobj, s_obj)
+        if s_obj != s_previousobj:
+            dict[self] = (s_obj, reflowpositions)
+            for position in reflowpositions:
+                bookkeeper.annotator.reflowfromposition(position)
+
+    def leave_tunnel(self, bookkeeper):
+        dict = self._getdict(bookkeeper)
+        s_obj, reflowpositions = dict.setdefault(
+            self, (annmodel.s_ImpossibleValue, {}))
+        reflowpositions[bookkeeper.position_key] = True
+        return s_obj
+
+    def get_input_annotation(self, bookkeeper):
+        dict = self._getdict(bookkeeper)
+        s_obj, _ = dict[self]
+        return s_obj
+
+_identity_for_ints = ErasingPairIdentity("int")
+
+
 def new_erasing_pair(name):
     identity = ErasingPairIdentity(name)
 
@@ -68,45 +90,29 @@
         assert y._identity is identity
         return y._x
 
-    def _getdict(bk):
-        try:
-            dict = bk._erasing_pairs_tunnel
-        except AttributeError:
-            dict = bk._erasing_pairs_tunnel = {}
-        return dict
-
     class Entry(ExtRegistryEntry):
         _about_ = erase
 
         def compute_result_annotation(self, s_obj):
-            bk = self.bookkeeper
-            dict = _getdict(bk)
-            s_previousobj, reflowpositions = dict.setdefault(
-                identity, (annmodel.s_ImpossibleValue, {}))
-            s_obj = annmodel.unionof(s_previousobj, s_obj)
-            if s_obj != s_previousobj:
-                dict[identity] = (s_obj, reflowpositions)
-                for position in reflowpositions:
-                    bk.annotator.reflowfromposition(position)
+            identity.enter_tunnel(self.bookkeeper, s_obj)
             return SomeErased()
 
         def specialize_call(self, hop):
-            return hop.r_result.specialize_call(hop)
+            bk = hop.rtyper.annotator.bookkeeper
+            s_obj = identity.get_input_annotation(bk)
+            return hop.r_result.rtype_erase(hop, s_obj)
 
     class Entry(ExtRegistryEntry):
         _about_ = unerase
 
         def compute_result_annotation(self, s_obj):
             assert SomeErased().contains(s_obj)
-            bk = self.bookkeeper
-            dict = _getdict(bk)
-            s_obj, reflowpositions = dict.setdefault(
-                identity, (annmodel.s_ImpossibleValue, []))
-            reflowpositions[bk.position_key] = True
-            return s_obj
+            return identity.leave_tunnel(self.bookkeeper)
 
         def specialize_call(self, hop):
-            v, t = hop.inputargs(hop.args_r[0], lltype.Void)
+            if hop.r_result.lowleveltype is lltype.Void:
+                return hop.inputconst(lltype.Void, None)
+            [v] = hop.inputargs(hop.args_r[0])
             return hop.genop('cast_opaque_ptr', [v], resulttype = hop.r_result)
 
     return erase, unerase
@@ -115,103 +121,55 @@
 # ---------- implementation-specific ----------
 
 class Erased(object):
-    _list_item_type = None
-    def __init__(self, x, identity=None):
+    def __init__(self, x, identity):
         self._x = x
         self._identity = identity
     def __repr__(self):
         return "Erased(%r, %r)" % (self._x, self._identity)
 
 class Entry(ExtRegistryEntry):
-    _about_ = erase
+    _about_ = erase_int
 
     def compute_result_annotation(self, s_obj):
+        assert annmodel.SomeInteger().contains(s_obj)
         return SomeErased()
 
     def specialize_call(self, hop):
-        return hop.r_result.specialize_call(hop)
+        return hop.r_result.rtype_erase_int(hop)
 
 class Entry(ExtRegistryEntry):
-    _about_ = erase_fixedsizelist
+    _about_ = unerase_int
 
-    def compute_result_annotation(self, s_arg, s_type):
-        # s_type ignored: only for prebuilt erased lists
-        assert isinstance(s_arg, annmodel.SomeList)
-        s_arg.listdef.never_resize()
-        return SomeErased()
+    def compute_result_annotation(self, s_obj):
+        assert SomeErased().contains(s_obj)
+        return annmodel.SomeInteger()
 
     def specialize_call(self, hop):
-        return hop.r_result.specialize_call(hop)
+        [v] = hop.inputargs(hop.args_r[0])
+        assert isinstance(hop.s_result, annmodel.SomeInteger)
+        return hop.gendirectcall(ll_unerase_int, v)
 
-class Entry(ExtRegistryEntry):
-    _about_ = unerase
-
-    def compute_result_annotation(self, s_obj, s_type):
-        assert s_type.is_constant()
-        if s_type.const is int:
-            return annmodel.SomeInteger()
-        assert isinstance(s_type, annmodel.SomePBC)
-        assert len(s_type.descriptions) == 1
-        clsdef = s_type.any_description().getuniqueclassdef()
-        return annmodel.SomeInstance(clsdef)
-
-    def specialize_call(self, hop):
-        v, t = hop.inputargs(hop.args_r[0], lltype.Void)
-        if isinstance(hop.s_result, annmodel.SomeInteger):
-            c_one = hop.inputconst(lltype.Signed, 1)
-            vi = hop.genop('cast_ptr_to_int', [v], resulttype=lltype.Signed)
-            return hop.genop('int_rshift', [vi, c_one], resulttype=lltype.Signed)
-        return hop.genop('cast_opaque_ptr', [v], resulttype = hop.r_result)
-
-class Entry(ExtRegistryEntry):
-    _about_ = unerase_fixedsizelist
-
-    def compute_result_annotation(self, s_obj, s_type):
-        assert isinstance(s_type, annmodel.SomePBC)
-        assert len(s_type.descriptions) == 1
-        clsdef = s_type.any_description().getuniqueclassdef()
-        s_item = annmodel.SomeInstance(clsdef)
-        return self.bookkeeper.newlist(s_item)
-
-    def specialize_call(self, hop):
-        v, t = hop.inputargs(hop.args_r[0], lltype.Void)
-        return hop.genop('cast_opaque_ptr', [v], resulttype = hop.r_result)
-
-
-class Entry(ExtRegistryEntry):
-    _about_ = is_integer
-
-    def compute_result_annotation(self, s_obj):
-        return annmodel.SomeBool()
-
-    def specialize_call(self, hop):
-        v, = hop.inputargs(hop.args_r[0])
-        c_one = hop.inputconst(lltype.Signed, 1)
-        vi = hop.genop('cast_ptr_to_int', [v], resulttype=lltype.Signed)
-        vb = hop.genop('int_and', [vi, c_one], resulttype=lltype.Signed)
-        return hop.genop('int_is_true', [vb], resulttype=lltype.Bool)
+def ll_unerase_int(gcref):
+    from pypy.rpython.lltypesystem.lloperation import llop
+    from pypy.rlib.debug import ll_assert
+    x = llop.cast_ptr_to_int(lltype.Signed, gcref)
+    ll_assert((x&1) != 0, "unerased_int(): not an integer")
+    return x >> 1
 
 
 class Entry(ExtRegistryEntry):
     _type_ = Erased
 
     def compute_annotation(self):
+        identity = self.instance._identity
         s_obj = self.bookkeeper.immutablevalue(self.instance._x)
-        if self.instance._list_item_type is not None:
-            # only non-resizable lists of instances for now
-            clsdef = self.bookkeeper.getuniqueclassdef(self.instance._list_item_type)
-            s_item = annmodel.SomeInstance(clsdef)
-            s_obj.listdef.generalize(s_item)
-            self.instance._s_list = s_obj
+        identity.enter_tunnel(self.bookkeeper, s_obj)
         return SomeErased()
 
 # annotation and rtyping support
 
 class SomeErased(annmodel.SomeObject):
 
-    def __init__(self, s_obj=None):
-        self.s_obj = s_obj # only non-None for constants
-
     def can_be_none(self):
         return False # cannot be None, but can contain a None
 
@@ -232,48 +190,35 @@
     def __init__(self, rtyper):
         self.rtyper = rtyper
 
-    def specialize_call(self, hop):
-        s_arg = hop.args_s[0]
-        r_generic_object = getinstancerepr(hop.rtyper, None)
-        if (isinstance(s_arg, annmodel.SomeInstance) or
-                (s_arg.is_constant() and s_arg.const is None)):
-            hop.exception_cannot_occur()
-            [v_instance] = hop.inputargs(r_generic_object)   # might generate a cast_pointer
-            v = hop.genop('cast_opaque_ptr', [v_instance],
-                          resulttype=self.lowleveltype)
-            return v
-        elif isinstance(s_arg, annmodel.SomeList):
-            hop.exception_cannot_occur()
-            r_list = self.rtyper.getrepr(s_arg)
-            v_list = hop.inputarg(r_list, 0)
-            v = hop.genop('cast_opaque_ptr', [v_list],
-                          resulttype=self.lowleveltype)
-            return v
-        else:
-            assert isinstance(s_arg, annmodel.SomeInteger)
-            v_value = hop.inputarg(lltype.Signed, arg=0)
-            c_one = hop.inputconst(lltype.Signed, 1)
-            hop.exception_is_here()
-            v2 = hop.genop('int_lshift_ovf', [v_value, c_one],
-                           resulttype = lltype.Signed)
-            v2p1 = hop.genop('int_add', [v2, c_one],
-                             resulttype = lltype.Signed)
-            v_instance = hop.genop('cast_int_to_ptr', [v2p1],
-                                   resulttype=self.lowleveltype)
-            v = hop.genop('cast_opaque_ptr', [v_instance],
-                          resulttype=self.lowleveltype)
-            return v
+    def rtype_erase(self, hop, s_obj):
+        hop.exception_cannot_occur()
+        r_obj = self.rtyper.getrepr(s_obj)
+        if r_obj.lowleveltype is lltype.Void:
+            return hop.inputconst(self.lowleveltype,
+                                  lltype.nullptr(self.lowleveltype.TO))
+        [v_obj] = hop.inputargs(r_obj)
+        return hop.genop('cast_opaque_ptr', [v_obj],
+                         resulttype=self.lowleveltype)
 
+    def rtype_erase_int(self, hop):
+        [v_value] = hop.inputargs(lltype.Signed)
+        c_one = hop.inputconst(lltype.Signed, 1)
+        hop.exception_is_here()
+        v2 = hop.genop('int_lshift_ovf', [v_value, c_one],
+                       resulttype = lltype.Signed)
+        v2p1 = hop.genop('int_add', [v2, c_one],
+                         resulttype = lltype.Signed)
+        v_instance = hop.genop('cast_int_to_ptr', [v2p1],
+                               resulttype=self.lowleveltype)
+        v = hop.genop('cast_opaque_ptr', [v_instance],
+                      resulttype=self.lowleveltype)
+        return v
 
     def convert_const(self, value):
-        if isinstance(value._x, int):
+        if value._identity is _identity_for_ints:
             return lltype.cast_int_to_ptr(self.lowleveltype, value._x * 2 + 1)
-        if isinstance(value._x, list):
-            r_list = self.rtyper.getrepr(value._s_list)
-            v = r_list.convert_const(value._x)
-            return lltype.cast_opaque_ptr(self.lowleveltype, v)
-        else:
-            r_generic_object = getinstancerepr(self.rtyper, None)
-            v = r_generic_object.convert_const(value._x)
-            return lltype.cast_opaque_ptr(self.lowleveltype, v)
-
+        bk = self.rtyper.annotator.bookkeeper
+        s_obj = value._identity.get_input_annotation(bk)
+        r_obj = self.rtyper.getrepr(s_obj)
+        v = r_obj.convert_const(value._x)
+        return lltype.cast_opaque_ptr(self.lowleveltype, v)

diff --git a/pypy/rlib/test/test_rerased.py b/pypy/rlib/test/test_rerased.py
--- a/pypy/rlib/test/test_rerased.py
+++ b/pypy/rlib/test/test_rerased.py
@@ -17,37 +17,42 @@
 class Z(X):
     pass
 
+eraseX, uneraseX = new_erasing_pair("X")
+erase_list_X, unerase_list_X = new_erasing_pair("list of X")
+
 
 def test_simple():
     x1 = X()
-    e = erase(x1)
-    assert is_integer(e) is False
-    assert unerase(e, X) is x1
+    e = eraseX(x1)
+    #assert is_integer(e) is False
+    assert uneraseX(e) is x1
 
 def test_simple_none():
-    e = erase(None)
-    assert unerase(e, X) is None
+    e = eraseX(None)
+    assert uneraseX(e) is None
 
 def test_simple_int():
-    e = erase(15)
-    assert is_integer(e) is True
-    assert unerase(e, int) == 15
+    e = erase_int(15)
+    #assert is_integer(e) is True
+    assert unerase_int(e) == 15
 
 def test_simple_int_overflow():
-    py.test.raises(OverflowError, erase, sys.maxint)
-    py.test.raises(OverflowError, erase, sys.maxint-1)
-    py.test.raises(OverflowError, erase, -sys.maxint)
-    py.test.raises(OverflowError, erase, -sys.maxint-1)
+    erase_int(sys.maxint//2)
+    py.test.raises(OverflowError, erase_int, sys.maxint//2 + 1)
+    py.test.raises(OverflowError, erase_int, sys.maxint)
+    py.test.raises(OverflowError, erase_int, sys.maxint-1)
+    py.test.raises(OverflowError, erase_int, -sys.maxint)
+    py.test.raises(OverflowError, erase_int, -sys.maxint-1)
 
 def test_list():
     l = [X()]
-    e = erase_fixedsizelist(l, X)
-    assert is_integer(e) is False
-    assert unerase_fixedsizelist(e, X) is l
+    e = erase_list_X(l)
+    #assert is_integer(e) is False
+    assert unerase_list_X(e) is l
 
 def test_annotate_1():
     def f():
-        return erase(X())
+        return eraseX(X())
     a = RPythonAnnotator()
     s = a.build_types(f, [])
     assert isinstance(s, SomeErased)
@@ -55,9 +60,9 @@
 def test_annotate_2():
     def f():
         x1 = X()
-        e = erase(x1)
-        assert not is_integer(e)
-        x2 = unerase(e, X)
+        e = eraseX(x1)
+        #assert not is_integer(e)
+        x2 = uneraseX(e)
         return x2
     a = RPythonAnnotator()
     s = a.build_types(f, [])
@@ -66,9 +71,9 @@
 
 def test_annotate_3():
     def f():
-        e = erase(16)
-        assert is_integer(e)
-        x2 = unerase(e, int)
+        e = erase_int(16)
+        #assert is_integer(e)
+        x2 = unerase_int(e)
         return x2
     a = RPythonAnnotator()
     s = a.build_types(f, [])
@@ -76,51 +81,53 @@
 
 def test_rtype_1():
     def f():
-        return erase(X())
+        return eraseX(X())
     x = interpret(f, [])
     assert lltype.typeOf(x) == llmemory.GCREF
 
 def test_rtype_2():
     def f():
         x1 = X()
-        e = erase(x1)
-        assert not is_integer(e)
-        x2 = unerase(e, X)
+        e = eraseX(x1)
+        #assert not is_integer(e)
+        x2 = uneraseX(e)
         return x2
     x = interpret(f, [])
     assert lltype.castable(OBJECTPTR, lltype.typeOf(x)) > 0
 
 def test_rtype_3():
     def f():
-        e = erase(16)
-        assert is_integer(e)
-        x2 = unerase(e, int)
+        e = erase_int(16)
+        #assert is_integer(e)
+        x2 = unerase_int(e)
         return x2
     x = interpret(f, [])
     assert x == 16
 
 
 def test_prebuilt_erased():
-    e1 = erase(16)
+    e1 = erase_int(16)
     x1 = X()
-    e2 = erase(x1)
+    x1.foobar = 42
+    e2 = eraseX(x1)
 
     def f():
-        assert is_integer(e1)
-        assert not is_integer(e2)
-        x2 = unerase(e1, int)
+        #assert is_integer(e1)
+        #assert not is_integer(e2)
+        x1.foobar += 1
+        x2 = unerase_int(e1) + uneraseX(e2).foobar
         return x2
     x = interpret(f, [])
-    assert x == 16
+    assert x == 16 + 42 + 1
 
 def test_overflow():
     def f(i):
         try:
-            e = erase(i)
+            e = erase_int(i)
         except OverflowError:
             return -1
-        assert is_integer(e)
-        return unerase(e, int)
+        #assert is_integer(e)
+        return unerase_int(e)
     x = interpret(f, [16])
     assert x == 16
     x = interpret(f, [sys.maxint])
@@ -128,7 +135,14 @@
 
 def test_none():
     def foo():
-        return unerase(erase(None), X)
+        return uneraseX(eraseX(None))
+    assert foo() is None
+    res = interpret(foo, [])
+    assert not res
+    #
+    def foo():
+        eraseX(X())
+        return uneraseX(eraseX(None))
     assert foo() is None
     res = interpret(foo, [])
     assert not res
@@ -143,19 +157,19 @@
 
 def test_rtype_list():
     prebuilt_l = [X()]
-    prebuilt_e = erase_fixedsizelist(prebuilt_l, X)
+    prebuilt_e = erase_list_X(prebuilt_l)
     def l(flag):
         if flag == 1:
             l = [X()]
-            e = erase_fixedsizelist(l, X)
+            e = erase_list_X(l)
         elif flag == 2:
             l = prebuilt_l
-            e = erase_fixedsizelist(l, X)
+            e = erase_list_X(l)
         else:
             l = prebuilt_l
             e = prebuilt_e
-        assert is_integer(e) is False
-        assert unerase_fixedsizelist(e, X) is l
+        #assert is_integer(e) is False
+        assert unerase_list_X(e) is l
     interpret(l, [0])
     interpret(l, [1])
     interpret(l, [2])
@@ -170,8 +184,6 @@
     erased = erase(x)
     assert unerase(erased) is x
     #
-    assert not is_integer(erased)
-    #
     erase2, unerase2 = new_erasing_pair("test2")
     py.test.raises(AssertionError, unerase2, erased)
 
@@ -233,3 +245,32 @@
     s = a.build_types(f, [])
     assert isinstance(s, annmodel.SomeInstance)
     assert s.classdef == a.bookkeeper.getuniqueclassdef(A)
+
+def test_annotate_prebuilt():
+    erase, unerase = new_erasing_pair("test1")
+    class X(object):
+        pass
+    x1 = X()
+    e1 = erase(x1)
+    e2 = erase(None)
+
+    def f(i):
+        if i:
+            e = e1
+        else:
+            e = e2
+        return unerase(e)
+    #
+    a = RPythonAnnotator()
+    s = a.build_types(f, [int])
+    assert isinstance(s, annmodel.SomeInstance)
+    assert s.classdef == a.bookkeeper.getuniqueclassdef(X)
+    assert s.can_be_none()
+
+def test_annotate_prebuilt_int():
+    e1 = erase_int(42)
+    def f(i):
+        return unerase_int(e1)
+    a = RPythonAnnotator()
+    s = a.build_types(f, [int])
+    assert isinstance(s, annmodel.SomeInteger)

From commits-noreply at bitbucket.org  Fri Feb 25 15:42:57 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Fri, 25 Feb 2011 15:42:57 +0100 (CET)
Subject: [pypy-svn] pypy default: merge heads
Message-ID: <20110225144257.6A20A2A2030@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r42284:aa1abfba10e1
Date: 2011-02-25 15:41 +0100
http://bitbucket.org/pypy/pypy/changeset/aa1abfba10e1/

Log:	merge heads


From commits-noreply at bitbucket.org  Fri Feb 25 15:51:13 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Fri, 25 Feb 2011 15:51:13 +0100 (CET)
Subject: [pypy-svn] pypy refactor-rerase: (cfbolz, arigo)
Message-ID: <20110225145113.E4CC5282B8B@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: refactor-rerase
Changeset: r42285:5e61e66be07a
Date: 2011-02-25 15:49 +0100
http://bitbucket.org/pypy/pypy/changeset/5e61e66be07a/

Log:	(cfbolz, arigo)

	Use the new interface.

diff --git a/pypy/objspace/std/mapdict.py b/pypy/objspace/std/mapdict.py
--- a/pypy/objspace/std/mapdict.py
+++ b/pypy/objspace/std/mapdict.py
@@ -1,6 +1,7 @@
 import weakref
 from pypy.rlib import jit, objectmodel, debug
 from pypy.rlib.rarithmetic import intmask, r_uint
+from pypy.rlib import rerased
 
 from pypy.interpreter.baseobjspace import W_Root
 from pypy.objspace.std.dictmultiobject import W_DictMultiObject
@@ -490,8 +491,11 @@
 memo_get_subclass_of_correct_size._annspecialcase_ = "specialize:memo"
 _subclass_cache = {}
 
+erase_item, unerase_item = rerased.new_erasing_pair("mapdict storage item")
+erase_list, unerase_list = rerased.new_erasing_pair("mapdict storage list")
+
 def _make_subclass_size_n(supercls, n):
-    from pypy.rlib import unroll, rerased
+    from pypy.rlib import unroll
     rangen = unroll.unrolling_iterable(range(n))
     nmin1 = n - 1
     rangenmin1 = unroll.unrolling_iterable(range(nmin1))
@@ -499,7 +503,7 @@
         def _init_empty(self, map):
             from pypy.rlib.debug import make_sure_not_resized
             for i in rangen:
-                setattr(self, "_value%s" % i, rerased.erase(None))
+                setattr(self, "_value%s" % i, erase_item(None))
             self.map = map
 
         def _has_storage_list(self):
@@ -507,7 +511,7 @@
 
         def _mapdict_get_storage_list(self):
             erased = getattr(self, "_value%s" % nmin1)
-            return rerased.unerase_fixedsizelist(erased, W_Root)
+            return unerase_list(erased)
 
         def _mapdict_read_storage(self, index):
             assert index >= 0
@@ -515,14 +519,14 @@
                 for i in rangenmin1:
                     if index == i:
                         erased = getattr(self, "_value%s" % i)
-                        return rerased.unerase(erased, W_Root)
+                        return unerase_item(erased)
             if self._has_storage_list():
                 return self._mapdict_get_storage_list()[index - nmin1]
             erased = getattr(self, "_value%s" % nmin1)
-            return rerased.unerase(erased, W_Root)
+            return unerase_item(erased)
 
         def _mapdict_write_storage(self, index, value):
-            erased = rerased.erase(value)
+            erased = erase_item(value)
             for i in rangenmin1:
                 if index == i:
                     setattr(self, "_value%s" % i, erased)
@@ -542,27 +546,27 @@
             len_storage = len(storage)
             for i in rangenmin1:
                 if i < len_storage:
-                    erased = rerased.erase(storage[i])
+                    erased = erase_item(storage[i])
                 else:
-                    erased = rerased.erase(None)
+                    erased = erase_item(None)
                 setattr(self, "_value%s" % i, erased)
             has_storage_list = self._has_storage_list()
             if len_storage < n:
                 assert not has_storage_list
-                erased = rerased.erase(None)
+                erased = erase_item(None)
             elif len_storage == n:
                 assert not has_storage_list
-                erased = rerased.erase(storage[nmin1])
+                erased = erase_item(storage[nmin1])
             elif not has_storage_list:
                 # storage is longer than self.map.length() only due to
                 # overallocation
-                erased = rerased.erase(storage[nmin1])
+                erased = erase_item(storage[nmin1])
                 # in theory, we should be ultra-paranoid and check all entries,
                 # but checking just one should catch most problems anyway:
                 assert storage[n] is None
             else:
                 storage_list = storage[nmin1:]
-                erased = rerased.erase_fixedsizelist(storage_list, W_Root)
+                erased = erase_list(storage_list)
             setattr(self, "_value%s" % nmin1, erased)
 
     subcls.__name__ = supercls.__name__ + "Size%s" % n

diff --git a/pypy/objspace/std/test/test_mapdict.py b/pypy/objspace/std/test/test_mapdict.py
--- a/pypy/objspace/std/test/test_mapdict.py
+++ b/pypy/objspace/std/test/test_mapdict.py
@@ -366,7 +366,6 @@
 
 def test_specialized_class():
     from pypy.objspace.std.objectobject import W_ObjectObject
-    from pypy.rlib import rerased
     classes = memo_get_subclass_of_correct_size(space, W_ObjectObject)
     w1 = W_Root()
     w2 = W_Root()
@@ -379,12 +378,12 @@
         obj = objectcls()
         obj.user_setup(space, cls)
         obj.setdictvalue(space, "a", w1)
-        assert rerased.unerase(obj._value0, W_Root) is w1
+        assert unerase_item(obj._value0) is w1
         assert obj.getdictvalue(space, "a") is w1
         assert obj.getdictvalue(space, "b") is None
         assert obj.getdictvalue(space, "c") is None
         obj.setdictvalue(space, "a", w2)
-        assert rerased.unerase(obj._value0, W_Root) is w2
+        assert unerase_item(obj._value0) is w2
         assert obj.getdictvalue(space, "a") == w2
         assert obj.getdictvalue(space, "b") is None
         assert obj.getdictvalue(space, "c") is None
@@ -402,7 +401,7 @@
 
         res = obj.deldictvalue(space, "a")
         assert res
-        assert rerased.unerase(obj._value0, W_Root) is w4
+        assert unerase_item(obj._value0) is w4
         assert obj.getdictvalue(space, "a") is None
         assert obj.getdictvalue(space, "b") is w4
         assert obj.getdictvalue(space, "c") is None

From commits-noreply at bitbucket.org  Fri Feb 25 17:01:58 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Fri, 25 Feb 2011 17:01:58 +0100 (CET)
Subject: [pypy-svn] pypy default: Merge branch "refactor-rerase": small
	refactoring of	the API of
Message-ID: <20110225160158.14F3C282B8B@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r42286:4962fb2cf76e
Date: 2011-02-25 17:01 +0100
http://bitbucket.org/pypy/pypy/changeset/4962fb2cf76e/

Log:	Merge branch "refactor-rerase": small refactoring of the API of
	rerased.py --- a simplification and generalization.


From commits-noreply at bitbucket.org  Fri Feb 25 17:01:59 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Fri, 25 Feb 2011 17:01:59 +0100 (CET)
Subject: [pypy-svn] pypy refactor-rerase: Close branch.
Message-ID: <20110225160159.623A9282B90@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: refactor-rerase
Changeset: r42287:2505c4c78d90
Date: 2011-02-25 17:01 +0100
http://bitbucket.org/pypy/pypy/changeset/2505c4c78d90/

Log:	Close branch.


From hpk at codespeak.net  Fri Feb 25 17:10:05 2011
From: hpk at codespeak.net (hpk at codespeak.net)
Date: Fri, 25 Feb 2011 17:10:05 +0100 (CET)
Subject: [pypy-svn] r80392 - pypy/extradoc/pycon-advertisement
Message-ID: <20110225161005.B5148282B8B@codespeak.net>

Author: hpk
Date: Fri Feb 25 17:10:03 2011
New Revision: 80392

Added:
   pypy/extradoc/pycon-advertisement/draft.2
Log:
suggestion for a more higher level pycon ad 


Added: pypy/extradoc/pycon-advertisement/draft.2
==============================================================================
--- (empty file)
+++ pypy/extradoc/pycon-advertisement/draft.2	Fri Feb 25 17:10:03 2011
@@ -0,0 +1,19 @@
+
+Contract the agile Python speed experts for:
+
+- Make your product run seemlessly on top of PyPy
+- speed up your Python based project
+- teaching to make the best out of PyPy
+
+Open end and merlinux along with the PyPy developers stand ready to help,
+contact us at contracts at pypy.org (XXX or the respective OE/merlinux addresses)
+
+Support the general open source project
+
+- contribute code
+- come to sprints
+- make a donation to our non-profit foundation 
+
+see http://pypy.org for more info.
+
+Come to IRC #pypy on irc.freenode.net 


From commits-noreply at bitbucket.org  Fri Feb 25 17:25:40 2011
From: commits-noreply at bitbucket.org (cfbolz)
Date: Fri, 25 Feb 2011 17:25:40 +0100 (CET)
Subject: [pypy-svn] pypy default: explicitly turn bytecode into a list of
	ints, the stdlib seems to produce longs sometimes
Message-ID: <20110225162540.4581F282B8B@codespeak.net>

Author: Carl Friedrich Bolz <cfbolz at gmx.de>
Branch: 
Changeset: r42288:c2f296937d52
Date: 2011-02-25 17:09 +0100
http://bitbucket.org/pypy/pypy/changeset/c2f296937d52/

Log:	explicitly turn bytecode into a list of ints, the stdlib seems to
	produce longs sometimes

diff --git a/pypy/rlib/rsre/test/test_zjit.py b/pypy/rlib/rsre/test/test_zjit.py
--- a/pypy/rlib/rsre/test/test_zjit.py
+++ b/pypy/rlib/rsre/test/test_zjit.py
@@ -30,7 +30,7 @@
 def list2array(lst):
     a = lltype.malloc(lltype.GcArray(lltype.Signed), len(lst))
     for i, x in enumerate(lst):
-        a[i] = x
+        a[i] = int(x)
     return a
 
 def array2list(a):

From commits-noreply at bitbucket.org  Fri Feb 25 17:25:41 2011
From: commits-noreply at bitbucket.org (cfbolz)
Date: Fri, 25 Feb 2011 17:25:41 +0100 (CET)
Subject: [pypy-svn] pypy default: try to not create too many guards when
	matching	character groups, otherwise you
Message-ID: <20110225162541.14B4B282B8B@codespeak.net>

Author: Carl Friedrich Bolz <cfbolz at gmx.de>
Branch: 
Changeset: r42289:9c5aa9785fd0
Date: 2011-02-25 17:11 +0100
http://bitbucket.org/pypy/pypy/changeset/9c5aa9785fd0/

Log:	try to not create too many guards when matching character groups,
	otherwise you get a huge mess of traces. needs more work still.

diff --git a/pypy/rlib/rsre/test/test_zjit.py b/pypy/rlib/rsre/test/test_zjit.py
--- a/pypy/rlib/rsre/test/test_zjit.py
+++ b/pypy/rlib/rsre/test/test_zjit.py
@@ -120,3 +120,31 @@
         res = self.meta_interp_match("(a|b)*a", "a" * 100)
         assert res == 100
         self.check_loops(guard_value=0)
+
+
+    # group guards tests
+
+    def test_group_range(self):
+        res = self.meta_interp_match(r"<[^b-c]+>", "<aeaeaea>")
+        assert res == 9
+        self.check_enter_count(1)
+
+    def test_group_single_chars(self):
+        res = self.meta_interp_match(r"<[ae]+>", "<aeaeaea>")
+        assert res == 9
+        self.check_enter_count(2) # XXX should be 1
+
+    def test_group_digit(self):
+        res = self.meta_interp_match(r"<[^\d]+>", "<..a..aa>")
+        assert res == 9
+        self.check_enter_count(1)
+
+    def test_group_space(self):
+        res = self.meta_interp_match(r"<\S+>", "<..a..aa>")
+        assert res == 9
+        self.check_enter_count(1)
+
+    def test_group_word(self):
+        res = self.meta_interp_match(r"<\w+>", "<ab09_a1>")
+        assert res == 9
+        self.check_enter_count(1)

diff --git a/pypy/rlib/rsre/rsre_char.py b/pypy/rlib/rsre/rsre_char.py
--- a/pypy/rlib/rsre/rsre_char.py
+++ b/pypy/rlib/rsre/rsre_char.py
@@ -71,14 +71,14 @@
 underline = ord("_")
 
 def is_digit(code):
-    return code <= 57 and code >= 48
+    return (code <= 57) & (code >= 48)
 
 def is_uni_digit(code):
     assert unicodedb is not None
     return unicodedb.isdecimal(code)
 
 def is_space(code):
-    return code == 32 or (code <= 13 and code >= 9)
+    return (code == 32) | ((code <= 13) & (code >= 9))
 
 def is_uni_space(code):
     assert unicodedb is not None
@@ -191,7 +191,7 @@
 
 def set_range(pat, index, char_code):
     # <RANGE> <lower> <upper>
-    if pat[index+1] <= char_code <= pat[index+2]:
+    if (pat[index+1] <= char_code) & (char_code <= pat[index+2]):
         return SET_OK
     return index + 3
 

From commits-noreply at bitbucket.org  Fri Feb 25 17:25:42 2011
From: commits-noreply at bitbucket.org (cfbolz)
Date: Fri, 25 Feb 2011 17:25:42 +0100 (CET)
Subject: [pypy-svn] pypy default: write this function in the style of the
	others around it
Message-ID: <20110225162542.1A3382A2002@codespeak.net>

Author: Carl Friedrich Bolz <cfbolz at gmx.de>
Branch: 
Changeset: r42290:a39777796c07
Date: 2011-02-25 17:22 +0100
http://bitbucket.org/pypy/pypy/changeset/a39777796c07/

Log:	write this function in the style of the others around it

diff --git a/pypy/rpython/rstr.py b/pypy/rpython/rstr.py
--- a/pypy/rpython/rstr.py
+++ b/pypy/rpython/rstr.py
@@ -621,8 +621,8 @@
     __metaclass__ = StaticMethods
 
     def ll_char_isspace(ch):
-        c = ord(ch) 
-        return c == 32 or (c <= 13 and c >= 9)   # c in (9, 10, 11, 12, 13, 32)
+        c = ord(ch)
+        return c == 32 or (9 <= c <= 13)   # c in (9, 10, 11, 12, 13, 32)
 
     def ll_char_isdigit(ch):
         c = ord(ch)

From cfbolz at codespeak.net  Fri Feb 25 18:10:20 2011
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Fri, 25 Feb 2011 18:10:20 +0100 (CET)
Subject: [pypy-svn] r80393 - pypy/extradoc/planning
Message-ID: <20110225171020.D6675282B8B@codespeak.net>

Author: cfbolz
Date: Fri Feb 25 18:10:18 2011
New Revision: 80393

Modified:
   pypy/extradoc/planning/jit.txt
Log:
two tasks


Modified: pypy/extradoc/planning/jit.txt
==============================================================================
--- pypy/extradoc/planning/jit.txt	(original)
+++ pypy/extradoc/planning/jit.txt	Fri Feb 25 18:10:18 2011
@@ -48,6 +48,12 @@
 - local imports should be jitted more efficiently, right now they produce a
   long trace and they are rather common (e.g. in translate.py)
 
+- don't use XCHG in the x86 backend, as that implies some sort of locking, that
+  we don't need and might be expensive.
+
+- the integer range analysis cannot deal with int_between, because it is
+  lowered to uint arithmetic too early
+
 OPTIMIZATIONS
 -------------
 


From commits-noreply at bitbucket.org  Fri Feb 25 19:32:41 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Fri, 25 Feb 2011 19:32:41 +0100 (CET)
Subject: [pypy-svn] pypy jit-usable_retrace: intorduce inputvalues on loops
	to pass along	information about inputs
Message-ID: <20110225183241.0DD1536C226@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-usable_retrace
Changeset: r42291:5fe82da5e779
Date: 2011-02-25 18:51 +0100
http://bitbucket.org/pypy/pypy/changeset/5fe82da5e779/

Log:	intorduce inputvalues on loops to pass along information about
	inputs

diff --git a/pypy/jit/metainterp/compile.py b/pypy/jit/metainterp/compile.py
--- a/pypy/jit/metainterp/compile.py
+++ b/pypy/jit/metainterp/compile.py
@@ -81,7 +81,8 @@
 # ____________________________________________________________
 
 def compile_new_loop(metainterp, old_loop_tokens, greenkey, inputargs, h_ops,
-                     start_resumedescr, full_preamble_needed=True):
+                     start_resumedescr, full_preamble_needed=True,
+                     inputvalues=None):
     """Try to compile a new loop by closing the current history back
     to the first operation.
     """
@@ -101,6 +102,7 @@
     loop.preamble.inputargs = loop.inputargs
     loop.preamble.token = make_loop_token(len(loop.inputargs), jitdriver_sd)
     loop.preamble.start_resumedescr = start_resumedescr
+    loop.inputvalues = inputvalues
 
     try:
         old_loop_token = jitdriver_sd.warmstate.optimize_loop(

diff --git a/pypy/jit/metainterp/history.py b/pypy/jit/metainterp/history.py
--- a/pypy/jit/metainterp/history.py
+++ b/pypy/jit/metainterp/history.py
@@ -785,6 +785,7 @@
 
 class TreeLoop(object):
     inputargs = None
+    inputvalues = None
     operations = None
     token = None
 

diff --git a/pypy/jit/metainterp/optimizeopt/optimizer.py b/pypy/jit/metainterp/optimizeopt/optimizer.py
--- a/pypy/jit/metainterp/optimizeopt/optimizer.py
+++ b/pypy/jit/metainterp/optimizeopt/optimizer.py
@@ -259,8 +259,8 @@
         self.posponedop = None
         self.exception_might_have_happened = False
         self.newoperations = []
+        self.set_optimizations(optimizations)
 
-        self.set_optimizations(optimizations)
 
     def set_optimizations(self, optimizations):
         if optimizations:

diff --git a/pypy/jit/metainterp/pyjitpl.py b/pypy/jit/metainterp/pyjitpl.py
--- a/pypy/jit/metainterp/pyjitpl.py
+++ b/pypy/jit/metainterp/pyjitpl.py
@@ -2485,7 +2485,8 @@
         loop_token = compile.compile_new_loop(self.metainterp, [],
                                               self.greenkey,
                                               self.inputargs, self.h_ops,
-                                              self.start_resumedescr, False)
+                                              self.start_resumedescr, False,
+                                              values)
         if loop_token is None:
             return
 

diff --git a/pypy/jit/metainterp/optimizeopt/virtualize.py b/pypy/jit/metainterp/optimizeopt/virtualize.py
--- a/pypy/jit/metainterp/optimizeopt/virtualize.py
+++ b/pypy/jit/metainterp/optimizeopt/virtualize.py
@@ -179,7 +179,6 @@
                            self.source_op)
         new.box = self.box
         return new
-            
 
 class VStructValue(AbstractVirtualStructValue):
 

From commits-noreply at bitbucket.org  Fri Feb 25 19:32:47 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Fri, 25 Feb 2011 19:32:47 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: hg merge default
Message-ID: <20110225183247.45C27282B8B@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r42292:a423fa72477c
Date: 2011-02-25 18:52 +0100
http://bitbucket.org/pypy/pypy/changeset/a423fa72477c/

Log:	hg merge default


From commits-noreply at bitbucket.org  Fri Feb 25 19:32:47 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Fri, 25 Feb 2011 19:32:47 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: hg merge
Message-ID: <20110225183247.B8E02282B90@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r42293:50bfd5b1dc81
Date: 2011-02-25 19:28 +0100
http://bitbucket.org/pypy/pypy/changeset/50bfd5b1dc81/

Log:	hg merge


From commits-noreply at bitbucket.org  Fri Feb 25 19:32:48 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Fri, 25 Feb 2011 19:32:48 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: Loop is now retraced and
	specialized to new	intbounds on second call
Message-ID: <20110225183248.7127036C226@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r42294:1957b6f4b9a8
Date: 2011-02-25 19:16 +0100
http://bitbucket.org/pypy/pypy/changeset/1957b6f4b9a8/

Log:	Loop is now retraced and specialized to new intbounds on second call

diff --git a/pypy/module/pypyjit/test/test_pypy_c.py b/pypy/module/pypyjit/test/test_pypy_c.py
--- a/pypy/module/pypyjit/test/test_pypy_c.py
+++ b/pypy/module/pypyjit/test/test_pypy_c.py
@@ -1488,7 +1488,7 @@
                 sa += a % b
                 i += 1
             return sa
-        ''', 11,  ([10, 20], 200 * (10 % 20)),
+        ''', 22,  ([10, 20], 200 * (10 % 20)),
                  ([-10, -20], 200 * (-10 % -20)),
                         count_debug_merge_point=False)
         assert self.jit_summary.tracing_no == 2

From commits-noreply at bitbucket.org  Fri Feb 25 19:32:49 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Fri, 25 Feb 2011 19:32:49 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: Each call is retraced due to
	different	intbounds
Message-ID: <20110225183249.0FA1136C226@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r42295:216bef1815c8
Date: 2011-02-25 19:23 +0100
http://bitbucket.org/pypy/pypy/changeset/216bef1815c8/

Log:	Each call is retraced due to different intbounds

diff --git a/pypy/module/pypyjit/test/test_pypy_c.py b/pypy/module/pypyjit/test/test_pypy_c.py
--- a/pypy/module/pypyjit/test/test_pypy_c.py
+++ b/pypy/module/pypyjit/test/test_pypy_c.py
@@ -1473,7 +1473,7 @@
 %s
                 i += 1
             return sa
-        ''' % code, 150, ([a1, b1], 2000 * res1),
+        ''' % code, 450, ([a1, b1], 2000 * res1),
                          ([a2, b2], 2000 * res2),
                          ([a3, b3], 2000 * res3),
                          count_debug_merge_point=False)

From commits-noreply at bitbucket.org  Fri Feb 25 20:31:08 2011
From: commits-noreply at bitbucket.org (cfbolz)
Date: Fri, 25 Feb 2011 20:31:08 +0100 (CET)
Subject: [pypy-svn] pypy default: (arigo,
	cfbolz): add an interface for int_between checks
Message-ID: <20110225193108.52B58282B8B@codespeak.net>

Author: Carl Friedrich Bolz <cfbolz at gmx.de>
Branch: 
Changeset: r42296:8afcf372c10d
Date: 2011-02-25 18:19 +0100
http://bitbucket.org/pypy/pypy/changeset/8afcf372c10d/

Log:	(arigo, cfbolz): add an interface for int_between checks

diff --git a/pypy/rlib/rarithmetic.py b/pypy/rlib/rarithmetic.py
--- a/pypy/rlib/rarithmetic.py
+++ b/pypy/rlib/rarithmetic.py
@@ -881,3 +881,13 @@
         # we use cast_primitive to go between Float and SingleFloat.
         return hop.genop('cast_primitive', [v],
                          resulttype = lltype.SingleFloat)
+
+
+def int_between(n, m, p):
+    """ check that n <= m < p. This assumes that n <= p. This is useful because
+    the JIT special-cases it. """
+    from pypy.rpython.lltypesystem import lltype
+    from pypy.rpython.lltypesystem.lloperation import llop
+    if not objectmodel.we_are_translated():
+        assert n <= p
+    return llop.int_between(lltype.Bool, n, m, p)

diff --git a/pypy/rlib/test/test_rarithmetic.py b/pypy/rlib/test/test_rarithmetic.py
--- a/pypy/rlib/test/test_rarithmetic.py
+++ b/pypy/rlib/test/test_rarithmetic.py
@@ -523,3 +523,12 @@
     almost_equal(round_double(-0.5e22, -22), -1e22)
     almost_equal(round_double(0.5e22, -22), 1e22)
     almost_equal(round_double(1.5e22, -22), 2e22)
+
+def test_int_between():
+    assert int_between(1, 1, 3)
+    assert int_between(1, 2, 3)
+    assert not int_between(1, 0, 2)
+    assert not int_between(1, 5, 2)
+    assert not int_between(1, 2, 2)
+    assert not int_between(1, 1, 1)
+

From commits-noreply at bitbucket.org  Fri Feb 25 20:31:08 2011
From: commits-noreply at bitbucket.org (cfbolz)
Date: Fri, 25 Feb 2011 20:31:08 +0100 (CET)
Subject: [pypy-svn] pypy default: (arigo,
	cfbolz): use int_between in the rsre engine
Message-ID: <20110225193108.DE2DA282B8B@codespeak.net>

Author: Carl Friedrich Bolz <cfbolz at gmx.de>
Branch: 
Changeset: r42297:9870431466e8
Date: 2011-02-25 18:21 +0100
http://bitbucket.org/pypy/pypy/changeset/9870431466e8/

Log:	(arigo, cfbolz): use int_between in the rsre engine

diff --git a/pypy/rlib/rsre/rsre_char.py b/pypy/rlib/rsre/rsre_char.py
--- a/pypy/rlib/rsre/rsre_char.py
+++ b/pypy/rlib/rsre/rsre_char.py
@@ -5,6 +5,7 @@
 from pypy.rlib.rlocale import tolower, isalnum
 from pypy.rlib.unroll import unrolling_iterable
 from pypy.rlib import jit
+from pypy.rlib.rarithmetic import int_between
 
 # Note: the unicode parts of this module require you to call
 # rsre_char.set_unicode_db() first, to select one of the modules
@@ -59,11 +60,10 @@
     elif flags & SRE_FLAG_LOCALE:
         return tolower(char_ord)
     else:
-        if ord('A') <= char_ord <= ord('Z'):   # ASCII lower
+        if int_between(ord('A'), char_ord, ord('Z') + 1):   # ASCII lower
             char_ord += ord('a') - ord('A')
     return char_ord
 
-
 #### Category helpers
 
 is_a_word = [(chr(i).isalnum() or chr(i) == '_') for i in range(256)]
@@ -71,14 +71,14 @@
 underline = ord("_")
 
 def is_digit(code):
-    return (code <= 57) & (code >= 48)
+    return int_between(48, code, 58)
 
 def is_uni_digit(code):
     assert unicodedb is not None
     return unicodedb.isdecimal(code)
 
 def is_space(code):
-    return (code == 32) | ((code <= 13) & (code >= 9))
+    return (code == 32) | int_between(9, code, 14)
 
 def is_uni_space(code):
     assert unicodedb is not None
@@ -191,7 +191,7 @@
 
 def set_range(pat, index, char_code):
     # <RANGE> <lower> <upper>
-    if (pat[index+1] <= char_code) & (char_code <= pat[index+2]):
+    if int_between(pat[index+1], char_code, pat[index+2] + 1):
         return SET_OK
     return index + 3
 

From commits-noreply at bitbucket.org  Fri Feb 25 20:31:09 2011
From: commits-noreply at bitbucket.org (cfbolz)
Date: Fri, 25 Feb 2011 20:31:09 +0100 (CET)
Subject: [pypy-svn] pypy default: (arigo,
	cfbolz) rewrite character group matching to only produce one guard
	at
Message-ID: <20110225193109.BEFA7282B8B@codespeak.net>

Author: Carl Friedrich Bolz <cfbolz at gmx.de>
Branch: 
Changeset: r42298:40d6226b9524
Date: 2011-02-25 18:58 +0100
http://bitbucket.org/pypy/pypy/changeset/40d6226b9524/

Log:	(arigo, cfbolz) rewrite character group matching to only produce one
	guard at the end, instead of one guard after every check.

diff --git a/pypy/rlib/rsre/test/test_zjit.py b/pypy/rlib/rsre/test/test_zjit.py
--- a/pypy/rlib/rsre/test/test_zjit.py
+++ b/pypy/rlib/rsre/test/test_zjit.py
@@ -1,3 +1,4 @@
+import py
 from pypy.jit.metainterp.test import test_basic
 from pypy.rlib.nonconst import NonConstant
 from pypy.rlib.rsre.test.test_match import get_code
@@ -132,7 +133,7 @@
     def test_group_single_chars(self):
         res = self.meta_interp_match(r"<[ae]+>", "<aeaeaea>")
         assert res == 9
-        self.check_enter_count(2) # XXX should be 1
+        self.check_enter_count(1)
 
     def test_group_digit(self):
         res = self.meta_interp_match(r"<[^\d]+>", "<..a..aa>")
@@ -148,3 +149,14 @@
         res = self.meta_interp_match(r"<\w+>", "<ab09_a1>")
         assert res == 9
         self.check_enter_count(1)
+
+    def test_group_complex(self):
+        res = self.meta_interp_match(r"<[a at h\d\s]+>", "<a93919a @ a23>")
+        assert res == 15
+        self.check_enter_count(1)
+
+    @py.test.mark.xfail
+    def test_group_space_but_not_space(self):
+        res = self.meta_interp_match(r"<[\S ]+>", "<..a   .. aa>")
+        assert res == 13
+        self.check_enter_count(1)

diff --git a/pypy/rlib/rsre/rsre_char.py b/pypy/rlib/rsre/rsre_char.py
--- a/pypy/rlib/rsre/rsre_char.py
+++ b/pypy/rlib/rsre/rsre_char.py
@@ -114,7 +114,7 @@
         if category_code == i:
             result = function(char_code)
             if negate:
-                return not result
+                return not result # XXX this might lead to a guard
             else:
                 return result
         i = i + 1
@@ -135,69 +135,62 @@
 
 ##### Charset evaluation
 
-SET_OK = -1
-SET_NOT_OK = -2
-
 @jit.unroll_safe
 def check_charset(pattern, ppos, char_code):
     """Checks whether a character matches set of arbitrary length.
     The set starts at pattern[ppos]."""
-    negated = SET_OK
-    while ppos >= 0:
+    negated = False
+    result = False
+    while True:
         opcode = pattern[ppos]
         i = 0
         for function in set_dispatch_unroll:
             if function is not None and opcode == i:
-                ppos = function(pattern, ppos, char_code)
+                newresult, ppos = function(pattern, ppos, char_code)
+                result |= newresult
                 break
             i = i + 1
         else:
-            if opcode == 26:   # NEGATE
-                negated ^= (SET_OK ^ SET_NOT_OK)
+            if opcode == 0: # FAILURE
+                break
+            elif opcode == 26:   # NEGATE
+                negated ^= True
                 ppos += 1
             else:
                 return False
-    return ppos == negated
-
-def set_failure(pat, index, char_code):
-    return SET_NOT_OK
+    if negated:
+        return not result
+    return result
 
 def set_literal(pat, index, char_code):
     # <LITERAL> <code>
-    if pat[index+1] == char_code:
-        return SET_OK
-    else:
-        return index + 2
+    match = pat[index+1] == char_code
+    return match, index + 2
 
 def set_category(pat, index, char_code):
     # <CATEGORY> <code>
-    if category_dispatch(pat[index+1], char_code):
-        return SET_OK
-    else:
-        return index + 2
+    match = category_dispatch(pat[index+1], char_code)
+    return match, index + 2
 
 def set_charset(pat, index, char_code):
     # <CHARSET> <bitmap> (16 bits per code word)
     if CODESIZE == 2:
-        if char_code < 256 and pat[index+1+(char_code >> 4)] \
-                                        & (1 << (char_code & 15)):
-            return SET_OK
-        return index + 17  # skip bitmap
+        match = char_code < 256 and \
+                (pat[index+1+(char_code >> 4)] & (1 << (char_code & 15)))
+        return match, index + 17  # skip bitmap
     else:
-        if char_code < 256 and pat[index+1+(char_code >> 5)] \
-                                        & (1 << (char_code & 31)):
-            return SET_OK
-        return index + 9   # skip bitmap
+        match = char_code < 256 and \
+                (pat[index+1+(char_code >> 5)] & (1 << (char_code & 31)))
+        return match, index + 9   # skip bitmap
 
 def set_range(pat, index, char_code):
     # <RANGE> <lower> <upper>
-    if int_between(pat[index+1], char_code, pat[index+2] + 1):
-        return SET_OK
-    return index + 3
+    match = int_between(pat[index+1], char_code, pat[index+2] + 1)
+    return match, index + 3
 
 def set_bigcharset(pat, index, char_code):
     # <BIGCHARSET> <blockcount> <256 blockindices> <blocks>
-    # XXX this function probably needs a makeover
+    # XXX this function needs a makeover, it's very bad
     count = pat[index+1]
     index += 2
     if char_code < 65536:
@@ -212,12 +205,12 @@
             shift = 5
         block_value = pat[index+(block * (32 / CODESIZE)
                                  + ((char_code & 255) >> shift))]
-        if block_value & (1 << (char_code & ((8 * CODESIZE) - 1))):
-            return SET_OK
+        match = (block_value & (1 << (char_code & ((8 * CODESIZE) - 1)))) != 0
     else:
         index += 256 / CODESIZE  # skip block indices
+        match = False
     index += count * (32 / CODESIZE)  # skip blocks
-    return index
+    return match, index
 
 def to_byte_array(int_value):
     """Creates a list of bytes out of an integer representing data that is
@@ -231,7 +224,8 @@
     return byte_array
 
 set_dispatch_table = [
-    set_failure, None, None, None, None, None, None, None, None,
+    None, # FAILURE
+    None, None, None, None, None, None, None, None,
     set_category, set_charset, set_bigcharset, None, None, None,
     None, None, None, None, set_literal, None, None, None, None,
     None, None,

From commits-noreply at bitbucket.org  Fri Feb 25 20:31:13 2011
From: commits-noreply at bitbucket.org (cfbolz)
Date: Fri, 25 Feb 2011 20:31:13 +0100 (CET)
Subject: [pypy-svn] pypy refactor-not-in-translator: (arigo,
	cfbolz): a branch where we want "not x" to no longer split paths
	in	the
Message-ID: <20110225193113.684D3282C1D@codespeak.net>

Author: Carl Friedrich Bolz <cfbolz at gmx.de>
Branch: refactor-not-in-translator
Changeset: r42299:d23ba11740c2
Date: 2011-02-25 19:37 +0100
http://bitbucket.org/pypy/pypy/changeset/d23ba11740c2/

Log:	(arigo, cfbolz): a branch where we want "not x" to no longer split
	paths in the flow object space. So far, flow space and annotation
	are fixed.

diff --git a/pypy/annotation/test/test_annrpython.py b/pypy/annotation/test/test_annrpython.py
--- a/pypy/annotation/test/test_annrpython.py
+++ b/pypy/annotation/test/test_annrpython.py
@@ -1415,6 +1415,20 @@
         s = a.build_types(snippet.prime, [int])
         assert s.knowntype == bool
 
+    def test_not_bool(self):
+        def f(x):
+            return not x
+        a = self.RPythonAnnotator()
+        s = a.build_types(f, [int])
+        assert s == annmodel.SomeBool()
+
+        def g():
+            return f(True)
+        a = self.RPythonAnnotator()
+        s = a.build_types(g, [])
+        assert isinstance(s, annmodel.SomeBool)
+        assert s.const is False
+
     def test_and_is_true_coalesce(self):
         def f(a,b,c,d,e):
             x = a and b
@@ -2012,6 +2026,7 @@
         assert isinstance(s, annmodel.SomeInstance)
         assert s.classdef == a.bookkeeper.getuniqueclassdef(B)
 
+
     def test_type_is_no_improvement(self):
         class B(object):
             pass
@@ -2027,6 +2042,21 @@
         s = a.build_types(f, [D])
         assert s == annmodel.SomeImpossibleValue()
 
+    def test_type_not_isinstance(self):
+        class B(object):
+            pass
+        class C(B):
+            pass
+        def f(x):
+            c = not isinstance(x, C)
+            if c:
+                return None
+            return x
+        a = self.RPythonAnnotator()
+        s = a.build_types(f, [B])
+        assert isinstance(s, annmodel.SomeInstance)
+        assert s.classdef == a.bookkeeper.getuniqueclassdef(C)
+
     def test_is_constant_instance(self):
         class A(object):
             pass

diff --git a/pypy/annotation/unaryop.py b/pypy/annotation/unaryop.py
--- a/pypy/annotation/unaryop.py
+++ b/pypy/annotation/unaryop.py
@@ -20,7 +20,8 @@
 def immutablevalue(x):
     return getbookkeeper().immutablevalue(x)
 
-UNARY_OPERATIONS = set(['len', 'is_true', 'getattr', 'setattr', 'delattr',
+UNARY_OPERATIONS = set(['len', 'is_true', 'is_false',
+                        'getattr', 'setattr', 'delattr',
                         'simple_call', 'call_args', 'str', 'repr',
                         'iter', 'next', 'invert', 'type', 'issubtype',
                         'pos', 'neg', 'nonzero', 'abs', 'hex', 'oct',
@@ -93,7 +94,18 @@
             s_nonnone_obj = s_obj.nonnoneify()
         add_knowntypedata(knowntypedata, True, [arg], s_nonnone_obj)
         return r
-        
+
+    def is_false(s_obj):
+        s_bool = s_obj.is_true()
+        s_res = SomeBool()
+        if s_bool.is_constant():
+            s_res.const = not s_bool.const
+        if hasattr(s_bool, "knowntypedata"):
+            ktd = {}
+            for (cond, var), s_stuff in s_bool.knowntypedata.iteritems():
+                ktd[not cond, var] = s_stuff
+            s_res.knowntypedata = ktd
+        return s_res
 
     def nonzero(obj):
         return obj.is_true()

diff --git a/pypy/objspace/flow/test/test_objspace.py b/pypy/objspace/flow/test/test_objspace.py
--- a/pypy/objspace/flow/test/test_objspace.py
+++ b/pypy/objspace/flow/test/test_objspace.py
@@ -829,6 +829,13 @@
         simplify_graph(graph)
         assert self.all_operations(graph) == {'getitem': 1}
 
+    def test_not_does_not_split(self):
+        def f(x):
+            return not x
+        graph = self.codetest(f)
+        assert self.all_operations(graph) == {"is_false": 1}
+
+
     def test_context_manager(self):
         def f(c, x):
             with x:

diff --git a/pypy/objspace/flow/objspace.py b/pypy/objspace/flow/objspace.py
--- a/pypy/objspace/flow/objspace.py
+++ b/pypy/objspace/flow/objspace.py
@@ -333,6 +333,17 @@
         context = self.getexecutioncontext()
         return context.guessbool(w_truthvalue)
 
+    def not_(self, w_obj):
+        try:
+            obj = self.unwrap_for_computation(w_obj)
+        except UnwrapException:
+            pass
+        else:
+            return not obj
+        w_falthvalue = self.do_operation('is_false', w_obj)
+        return w_falthvalue
+
+
     def iter(self, w_iterable):
         try:
             iterable = self.unwrap(w_iterable)

From commits-noreply at bitbucket.org  Fri Feb 25 20:31:14 2011
From: commits-noreply at bitbucket.org (cfbolz)
Date: Fri, 25 Feb 2011 20:31:14 +0100 (CET)
Subject: [pypy-svn] pypy refactor-not-in-translator: (arigo,
	cfbolz): what's not tested is broken
Message-ID: <20110225193114.BBE8B282C1B@codespeak.net>

Author: Carl Friedrich Bolz <cfbolz at gmx.de>
Branch: refactor-not-in-translator
Changeset: r42300:fb34c35512b7
Date: 2011-02-25 19:45 +0100
http://bitbucket.org/pypy/pypy/changeset/fb34c35512b7/

Log:	(arigo, cfbolz): what's not tested is broken

diff --git a/pypy/objspace/flow/objspace.py b/pypy/objspace/flow/objspace.py
--- a/pypy/objspace/flow/objspace.py
+++ b/pypy/objspace/flow/objspace.py
@@ -339,7 +339,7 @@
         except UnwrapException:
             pass
         else:
-            return not obj
+            return self.wrap(not obj)
         w_falthvalue = self.do_operation('is_false', w_obj)
         return w_falthvalue
 

diff --git a/pypy/objspace/flow/test/test_objspace.py b/pypy/objspace/flow/test/test_objspace.py
--- a/pypy/objspace/flow/test/test_objspace.py
+++ b/pypy/objspace/flow/test/test_objspace.py
@@ -834,6 +834,11 @@
             return not x
         graph = self.codetest(f)
         assert self.all_operations(graph) == {"is_false": 1}
+        x = True
+        def f():
+            return not x
+        graph = self.codetest(f)
+        assert self.all_operations(graph) == {}
 
 
     def test_context_manager(self):

From commits-noreply at bitbucket.org  Fri Feb 25 20:31:16 2011
From: commits-noreply at bitbucket.org (cfbolz)
Date: Fri, 25 Feb 2011 20:31:16 +0100 (CET)
Subject: [pypy-svn] pypy default: (arigo, cfbolz): a completely obscure bug,
	that is visible on our branch
Message-ID: <20110225193116.5F2342A2002@codespeak.net>

Author: Carl Friedrich Bolz <cfbolz at gmx.de>
Branch: 
Changeset: r42301:014997264748
Date: 2011-02-25 20:09 +0100
http://bitbucket.org/pypy/pypy/changeset/014997264748/

Log:	(arigo, cfbolz): a completely obscure bug, that is visible on our
	branch

diff --git a/pypy/rpython/rpbc.py b/pypy/rpython/rpbc.py
--- a/pypy/rpython/rpbc.py
+++ b/pypy/rpython/rpbc.py
@@ -629,6 +629,8 @@
         return inputconst(Void, None)
     
     def rtype_is_((robj1, rnone2), hop):
+        if hop.s_result.is_constant():
+            return hop.inputconst(Bool, hop.s_result.const)
         return hop.rtyper.type_system.rpbc.rtype_is_None(robj1, rnone2, hop)
 
 class __extend__(pairtype(NoneFrozenPBCRepr, Repr)):
@@ -637,6 +639,8 @@
         return inputconst(r_to, None)
 
     def rtype_is_((rnone1, robj2), hop):
+        if hop.s_result.is_constant():
+            return hop.inputconst(Bool, hop.s_result.const)
         return hop.rtyper.type_system.rpbc.rtype_is_None(
                                                 robj2, rnone1, hop, pos=1)
 

diff --git a/pypy/rpython/test/test_rpbc.py b/pypy/rpython/test/test_rpbc.py
--- a/pypy/rpython/test/test_rpbc.py
+++ b/pypy/rpython/test/test_rpbc.py
@@ -1613,6 +1613,13 @@
         res = self.interpret(fn, [0])
         assert res == -1
 
+    def test_is_none(self):
+        from pypy.rlib.nonconst import NonConstant
+        def g(x):
+            return NonConstant(g) is None
+        res = self.interpret(g, [1])
+        assert not res
+
 class TestLLtype(BaseTestRPBC, LLRtypeMixin):
     pass
 
@@ -1974,4 +1981,3 @@
     assert res == -1
     res = interp.eval_graph(graph, [0, 1])
     assert res == 1
-

From commits-noreply at bitbucket.org  Fri Feb 25 20:31:17 2011
From: commits-noreply at bitbucket.org (cfbolz)
Date: Fri, 25 Feb 2011 20:31:17 +0100 (CET)
Subject: [pypy-svn] pypy refactor-not-in-translator: (arigo,
	cfbolz): fix a lot of tests
Message-ID: <20110225193117.470132A2002@codespeak.net>

Author: Carl Friedrich Bolz <cfbolz at gmx.de>
Branch: refactor-not-in-translator
Changeset: r42302:71a347edcb75
Date: 2011-02-25 20:10 +0100
http://bitbucket.org/pypy/pypy/changeset/71a347edcb75/

Log:	(arigo, cfbolz): fix a lot of tests

diff --git a/pypy/rpython/rmodel.py b/pypy/rpython/rmodel.py
--- a/pypy/rpython/rmodel.py
+++ b/pypy/rpython/rmodel.py
@@ -215,6 +215,10 @@
         else:
             return hop.genop('int_is_true', [vlen], resulttype=Bool)
 
+    def rtype_is_false(self, hop):
+        v = self.rtype_is_true(hop)
+        return hop.genop("bool_not", [v], resulttype=Bool)
+
     def rtype_hash(self, hop):
         ll_hash = self.get_ll_hash_function()
         v, = hop.inputargs(self)

From commits-noreply at bitbucket.org  Fri Feb 25 20:31:17 2011
From: commits-noreply at bitbucket.org (cfbolz)
Date: Fri, 25 Feb 2011 20:31:17 +0100 (CET)
Subject: [pypy-svn] pypy refactor-not-in-translator: merge default
Message-ID: <20110225193117.98D132A202C@codespeak.net>

Author: Carl Friedrich Bolz <cfbolz at gmx.de>
Branch: refactor-not-in-translator
Changeset: r42303:1c18e31f07ee
Date: 2011-02-25 20:11 +0100
http://bitbucket.org/pypy/pypy/changeset/1c18e31f07ee/

Log:	merge default


From commits-noreply at bitbucket.org  Fri Feb 25 20:31:18 2011
From: commits-noreply at bitbucket.org (cfbolz)
Date: Fri, 25 Feb 2011 20:31:18 +0100 (CET)
Subject: [pypy-svn] pypy refactor-not-in-translator: (arigo,
	cfbolz): test and fix
Message-ID: <20110225193118.9B773282B90@codespeak.net>

Author: Carl Friedrich Bolz <cfbolz at gmx.de>
Branch: refactor-not-in-translator
Changeset: r42304:539fbad9fcf7
Date: 2011-02-25 20:18 +0100
http://bitbucket.org/pypy/pypy/changeset/539fbad9fcf7/

Log:	(arigo, cfbolz): test and fix

diff --git a/pypy/annotation/test/test_annrpython.py b/pypy/annotation/test/test_annrpython.py
--- a/pypy/annotation/test/test_annrpython.py
+++ b/pypy/annotation/test/test_annrpython.py
@@ -1680,6 +1680,14 @@
         assert isinstance(s2.dictdef.dictkey.  s_value, annmodel.SomeInteger)
         assert isinstance(s2.dictdef.dictvalue.s_value, annmodel.SomeInteger)
 
+    def test_not_dict(self):
+        def f(x):
+            d = {x: x}
+            return not d
+        a = self.RPythonAnnotator()
+        s = a.build_types(f, [str])
+        assert isinstance(s, annmodel.SomeBool)
+
     def test_pbc_union(self):
         class A:
             def meth(self):

diff --git a/pypy/annotation/unaryop.py b/pypy/annotation/unaryop.py
--- a/pypy/annotation/unaryop.py
+++ b/pypy/annotation/unaryop.py
@@ -86,7 +86,7 @@
         knowntypedata = r.knowntypedata = {}
         fn, block, i = bk.position_key
         op = block.operations[i]
-        assert op.opname == "is_true" or op.opname == "nonzero"
+        assert op.opname in ["is_true", "nonzero", "is_false"]
         assert len(op.args) == 1
         arg = op.args[0]
         s_nonnone_obj = s_obj

From commits-noreply at bitbucket.org  Fri Feb 25 22:35:10 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Fri, 25 Feb 2011 22:35:10 +0100 (CET)
Subject: [pypy-svn] pypy default: hg merge jit-virtual_state
Message-ID: <20110225213510.E8FCB282B8B@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: 
Changeset: r42305:3fe93dd500bd
Date: 2011-02-25 22:34 +0100
http://bitbucket.org/pypy/pypy/changeset/3fe93dd500bd/

Log:	hg merge jit-virtual_state


From commits-noreply at bitbucket.org  Sat Feb 26 05:53:30 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Sat, 26 Feb 2011 05:53:30 +0100 (CET)
Subject: [pypy-svn] pypy default: A little cleanup to epoll,
	consolidate the "is closed" logic and remove self.space,
	I don't like it.
Message-ID: <20110226045330.DAFFE2A202D@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r42306:902b78330a62
Date: 2011-02-25 23:53 -0500
http://bitbucket.org/pypy/pypy/changeset/902b78330a62/

Log:	A little cleanup to epoll, consolidate the "is closed" logic and
	remove self.space, I don't like it.

diff --git a/pypy/module/select/interp_epoll.py b/pypy/module/select/interp_epoll.py
--- a/pypy/module/select/interp_epoll.py
+++ b/pypy/module/select/interp_epoll.py
@@ -55,9 +55,9 @@
     compilation_info=eci,
 )
 
+
 class W_Epoll(Wrappable):
     def __init__(self, space, epfd):
-        self.space = space
         self.epfd = epfd
 
     @unwrap_spec(sizehint=int)
@@ -81,19 +81,22 @@
     def __del__(self):
         self.close()
 
-    def check_closed(self):
-        if self.epfd < 0:
-            raise OperationError(self.space.w_ValueError,
-                self.space.wrap("I/O operation on closed epoll fd")
+    def check_closed(self, space):
+        if self.get_closed():
+            raise OperationError(space.w_ValueError,
+                space.wrap("I/O operation on closed epoll fd")
             )
 
+    def get_closed(self):
+        return self.epfd < 0
+
     def close(self):
-        if not self.epfd < 0:
+        if not self.get_closed():
             socketclose(self.epfd)
             self.epfd = -1
 
-    def epoll_ctl(self, ctl, w_fd, eventmask, ignore_ebadf=False):
-        fd = self.space.c_filedescriptor_w(w_fd)
+    def epoll_ctl(self, space, ctl, w_fd, eventmask, ignore_ebadf=False):
+        fd = space.c_filedescriptor_w(w_fd)
         with lltype.scoped_alloc(epoll_event) as ev:
             ev.c_events = rffi.cast(rffi.UINT, eventmask)
             rffi.setintfield(ev.c_data, 'c_fd', fd)
@@ -102,36 +105,36 @@
             if ignore_ebadf and get_errno() == errno.EBADF:
                 result = 0
             if result < 0:
-                raise exception_from_errno(self.space, self.space.w_IOError)
+                raise exception_from_errno(space, space.w_IOError)
 
     def descr_get_closed(self, space):
-        return space.wrap(self.epfd < 0)
+        return space.wrap(self.get_closed())
 
     def descr_fileno(self, space):
-        self.check_closed()
+        self.check_closed(space)
         return space.wrap(self.epfd)
 
     def descr_close(self, space):
-        self.check_closed()
+        self.check_closed(space)
         self.close()
 
     @unwrap_spec(eventmask=int)
     def descr_register(self, space, w_fd, eventmask=-1):
-        self.check_closed()
-        self.epoll_ctl(EPOLL_CTL_ADD, w_fd, eventmask)
+        self.check_closed(space)
+        self.epoll_ctl(space, EPOLL_CTL_ADD, w_fd, eventmask)
 
     def descr_unregister(self, space, w_fd):
-        self.check_closed()
-        self.epoll_ctl(EPOLL_CTL_DEL, w_fd, 0, ignore_ebadf=True)
+        self.check_closed(space)
+        self.epoll_ctl(space, EPOLL_CTL_DEL, w_fd, 0, ignore_ebadf=True)
 
     @unwrap_spec(eventmask=int)
     def descr_modify(self, space, w_fd, eventmask=-1):
-        self.check_closed()
-        self.epoll_ctl(EPOLL_CTL_MOD, w_fd, eventmask)
+        self.check_closed(space)
+        self.epoll_ctl(space, EPOLL_CTL_MOD, w_fd, eventmask)
 
     @unwrap_spec(timeout=float, maxevents=int)
     def descr_poll(self, space, timeout=-1.0, maxevents=-1):
-        self.check_closed()
+        self.check_closed(space)
         if timeout < 0:
             timeout = -1.0
         else:

From commits-noreply at bitbucket.org  Sat Feb 26 12:21:43 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sat, 26 Feb 2011 12:21:43 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix for test_transformed_gc.py.
Message-ID: <20110226112143.A4C33282BAD@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r42307:6dd8d5c6623f
Date: 2011-02-26 12:21 +0100
http://bitbucket.org/pypy/pypy/changeset/6dd8d5c6623f/

Log:	Fix for test_transformed_gc.py.

diff --git a/pypy/rpython/llinterp.py b/pypy/rpython/llinterp.py
--- a/pypy/rpython/llinterp.py
+++ b/pypy/rpython/llinterp.py
@@ -5,6 +5,7 @@
 from pypy.rpython.lltypesystem import rclass
 from pypy.rpython.ootypesystem import ootype
 from pypy.rlib.objectmodel import ComputedIntSymbolic, CDefinedIntSymbolic
+from pypy.rlib.objectmodel import Symbolic
 from pypy.rlib import rstackovf
 
 import sys, os
@@ -1132,7 +1133,9 @@
         # special case
         if type(x) is CDefinedIntSymbolic:
             x = x.default
-        assert isinstance(x, int)
+        # if type(x) is a subclass of Symbolic, bool(x) will usually raise
+        # a TypeError -- unless __nonzero__ has been explicitly overridden.
+        assert isinstance(x, (int, Symbolic))
         return bool(x)
 
     # read frame var support

From commits-noreply at bitbucket.org  Sat Feb 26 23:04:16 2011
From: commits-noreply at bitbucket.org (arigo)
Date: Sat, 26 Feb 2011 23:04:16 +0100 (CET)
Subject: [pypy-svn] pypy default: Add the flag EF_PURE to some more OS_xxx
	functions.
Message-ID: <20110226220416.D3D972A202B@codespeak.net>

Author: Armin Rigo <arigo at tunes.org>
Branch: 
Changeset: r42309:52e1f41fe018
Date: 2011-02-26 23:03 +0100
http://bitbucket.org/pypy/pypy/changeset/52e1f41fe018/

Log:	Add the flag EF_PURE to some more OS_xxx functions.

diff --git a/pypy/jit/codewriter/jtransform.py b/pypy/jit/codewriter/jtransform.py
--- a/pypy/jit/codewriter/jtransform.py
+++ b/pypy/jit/codewriter/jtransform.py
@@ -1243,7 +1243,7 @@
         return op1
 
     def _register_extra_helper(self, oopspecindex, oopspec_name,
-                               argtypes, resulttype):
+                               argtypes, resulttype, effectinfo):
         # a bit hackish
         if self.callcontrol.callinfocollection.has_oopspec(oopspecindex):
             return
@@ -1253,7 +1253,8 @@
         op = SpaceOperation('pseudo_call_cannot_raise',
                             [c_func] + [varoftype(T) for T in argtypes],
                             varoftype(resulttype))
-        calldescr = self.callcontrol.getcalldescr(op, oopspecindex)
+        calldescr = self.callcontrol.getcalldescr(op, oopspecindex,
+                                                  effectinfo)
         if isinstance(c_func.value, str):    # in tests only
             func = c_func.value
         else:
@@ -1312,11 +1313,15 @@
                 if args[0].concretetype.TO == rstr.UNICODE:
                     otherindex += EffectInfo._OS_offset_uni
                 self._register_extra_helper(otherindex, othername,
-                                            argtypes, resulttype)
+                                            argtypes, resulttype,
+                                            EffectInfo.EF_PURE)
         #
-        return self._handle_oopspec_call(op, args, dict[oopspec_name])
+        return self._handle_oopspec_call(op, args, dict[oopspec_name],
+                                         EffectInfo.EF_PURE)
 
     def _handle_str2unicode_call(self, op, oopspec_name, args):
+        # ll_str2unicode is not EF_PURE, because it can raise
+        # UnicodeDecodeError...
         return self._handle_oopspec_call(op, args, EffectInfo.OS_STR2UNICODE)
 
     # ----------

diff --git a/pypy/jit/codewriter/test/test_jtransform.py b/pypy/jit/codewriter/test/test_jtransform.py
--- a/pypy/jit/codewriter/test/test_jtransform.py
+++ b/pypy/jit/codewriter/test/test_jtransform.py
@@ -116,6 +116,10 @@
             argtypes = argtypes[oopspecindex]
             assert argtypes[0] == [v.concretetype for v in op.args[1:]]
             assert argtypes[1] == op.result.concretetype
+            if oopspecindex == EI.OS_STR2UNICODE:
+                assert extraeffect == None    # not pure, can raise!
+            else:
+                assert extraeffect == EI.EF_PURE
         return 'calldescr-%d' % oopspecindex
     def calldescr_canraise(self, calldescr):
         return False

From commits-noreply at bitbucket.org  Sat Feb 26 23:28:43 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Sat, 26 Feb 2011 23:28:43 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix cpyext tests on Windows
Message-ID: <20110226222843.6507D282BAD@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r42310:9ec0c12251ab
Date: 2011-02-26 23:24 +0100
http://bitbucket.org/pypy/pypy/changeset/9ec0c12251ab/

Log:	Fix cpyext tests on Windows

diff --git a/pypy/translator/platform/windows.py b/pypy/translator/platform/windows.py
--- a/pypy/translator/platform/windows.py
+++ b/pypy/translator/platform/windows.py
@@ -120,7 +120,12 @@
         return ['/I%s' % (idir,) for idir in include_dirs]
 
     def _libs(self, libraries):
-        return ['%s.lib' % (lib,) for lib in libraries]
+        libs = []
+        for lib in libraries:
+            if lib.endswith('.dll'):
+                lib = lib[:-4]
+            libs.append('%s.lib' % (lib,))
+        return libs
 
     def _libdirs(self, library_dirs):
         return ['/LIBPATH:%s' % (ldir,) for ldir in library_dirs]

diff --git a/pypy/module/cpyext/api.py b/pypy/module/cpyext/api.py
--- a/pypy/module/cpyext/api.py
+++ b/pypy/module/cpyext/api.py
@@ -832,6 +832,9 @@
         if sys.platform == "win32":
             # '%s' undefined; assuming extern returning int
             compile_extra.append("/we4013")
+            # Sometimes the library is wrapped into another DLL, ensure that
+            # the correct bootstrap code is installed
+            kwds["link_extra"] = ["msvcrt.lib"]
         elif sys.platform == 'linux2':
             compile_extra.append("-Werror=implicit-function-declaration")
         export_symbols_eci.append('pypyAPI')

From commits-noreply at bitbucket.org  Sat Feb 26 23:28:44 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Sat, 26 Feb 2011 23:28:44 +0100 (CET)
Subject: [pypy-svn] pypy default: (kleptog) Implement PySlice_GetIndices()
Message-ID: <20110226222844.9BE24282BAD@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r42311:443ddfc7ee63
Date: 2011-02-26 23:26 +0100
http://bitbucket.org/pypy/pypy/changeset/443ddfc7ee63/

Log:	(kleptog) Implement PySlice_GetIndices()

diff --git a/pypy/module/cpyext/stubs.py b/pypy/module/cpyext/stubs.py
--- a/pypy/module/cpyext/stubs.py
+++ b/pypy/module/cpyext/stubs.py
@@ -2157,26 +2157,6 @@
     """Empty an existing set of all elements."""
     raise NotImplementedError
 
- at cpython_api([PyObject, Py_ssize_t, Py_ssize_t, Py_ssize_t, Py_ssize_t], rffi.INT_real, error=-1)
-def PySlice_GetIndices(space, slice, length, start, stop, step):
-    """Retrieve the start, stop and step indices from the slice object slice,
-    assuming a sequence of length length. Treats indices greater than
-    length as errors.
-    
-    Returns 0 on success and -1 on error with no exception set (unless one of
-    the indices was not None and failed to be converted to an integer,
-    in which case -1 is returned with an exception set).
-    
-    You probably do not want to use this function.  If you want to use slice
-    objects in versions of Python prior to 2.3, you would probably do well to
-    incorporate the source of PySlice_GetIndicesEx(), suitably renamed,
-    in the source of your extension.
-    
-    This function used an int type for length and an
-    int * type for start, stop, and step. This might require
-    changes in your code for properly supporting 64-bit systems."""
-    raise NotImplementedError
-
 @cpython_api([PyObjectP], lltype.Void)
 def PyString_InternInPlace(space, string):
     """Intern the argument *string in place.  The argument must be the address of a

diff --git a/pypy/module/cpyext/test/test_sliceobject.py b/pypy/module/cpyext/test/test_sliceobject.py
--- a/pypy/module/cpyext/test/test_sliceobject.py
+++ b/pypy/module/cpyext/test/test_sliceobject.py
@@ -26,6 +26,21 @@
             return rv
         assert get_indices(w(10), w(20), w(1), 200) == (10, 20, 1, 10)
 
+    def test_GetIndices(self, space, api):
+        w = space.wrap
+        def get_indices(w_start, w_stop, w_step, length):
+            w_slice = space.newslice(w_start, w_stop, w_step)
+            values = lltype.malloc(Py_ssize_tP.TO, 3, flavor='raw')
+            
+            res = api.PySlice_GetIndices(w_slice, 100, values, 
+                rffi.ptradd(values, 1), 
+                rffi.ptradd(values, 2))
+            assert res == 0
+            rv = values[0], values[1], values[2]
+            lltype.free(values, flavor='raw')
+            return rv
+        assert get_indices(w(10), w(20), w(1), 200) == (10, 20, 1)
+
 class AppTestSliceMembers(AppTestCpythonExtensionBase):
     def test_members(self):
         module = self.import_extension('foo', [

diff --git a/pypy/module/cpyext/sliceobject.py b/pypy/module/cpyext/sliceobject.py
--- a/pypy/module/cpyext/sliceobject.py
+++ b/pypy/module/cpyext/sliceobject.py
@@ -81,3 +81,29 @@
     start_p[0], stop_p[0], step_p[0], slicelength_p[0] = \
             w_slice.indices4(space, length)
     return 0
+
+ at cpython_api([PySliceObject, Py_ssize_t, Py_ssize_tP, Py_ssize_tP, Py_ssize_tP],
+                rffi.INT_real, error=-1)
+def PySlice_GetIndices(space, w_slice, length, start_p, stop_p, step_p):
+    """Retrieve the start, stop and step indices from the slice object slice,
+    assuming a sequence of length length. Treats indices greater than
+    length as errors.
+    
+    Returns 0 on success and -1 on error with no exception set (unless one of
+    the indices was not None and failed to be converted to an integer,
+    in which case -1 is returned with an exception set).
+    
+    You probably do not want to use this function.  If you want to use slice
+    objects in versions of Python prior to 2.3, you would probably do well to
+    incorporate the source of PySlice_GetIndicesEx(), suitably renamed,
+    in the source of your extension.
+    
+    This function used an int type for length and an
+    int * type for start, stop, and step. This might require
+    changes in your code for properly supporting 64-bit systems."""
+    if not PySlice_Check(space, w_slice):
+        PyErr_BadInternalCall(space)
+    assert isinstance(w_slice, W_SliceObject)
+    start_p[0], stop_p[0], step_p[0] = \
+            w_slice.indices3(space, length)
+    return 0

From commits-noreply at bitbucket.org  Sat Feb 26 23:40:29 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Sat, 26 Feb 2011 23:40:29 +0100 (CET)
Subject: [pypy-svn] pypy default: Remove implemented functions from stubs.py
Message-ID: <20110226224029.885EE282BAD@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r42312:392bbf936179
Date: 2011-02-26 23:40 +0100
http://bitbucket.org/pypy/pypy/changeset/392bbf936179/

Log:	Remove implemented functions from stubs.py some cleanup in cpyext
	docstrings

diff --git a/pypy/module/cpyext/stubs.py b/pypy/module/cpyext/stubs.py
--- a/pypy/module/cpyext/stubs.py
+++ b/pypy/module/cpyext/stubs.py
@@ -1492,13 +1492,6 @@
     PyInterpreterState_Clear()."""
     raise NotImplementedError
 
- at cpython_api([], PyThreadState)
-def PyThreadState_Get(space, ):
-    """Return the current thread state.  The global interpreter lock must be held.
-    When the current thread state is NULL, this issues a fatal error (so that
-    the caller needn't check for NULL)."""
-    raise NotImplementedError
-
 @cpython_api([], PyObject)
 def PyThreadState_GetDict(space, ):
     """Return a dictionary in which extensions can store thread-specific state
@@ -1635,19 +1628,6 @@
     defined."""
     raise NotImplementedError
 
- at cpython_api([], PyInterpreterState)
-def PyInterpreterState_Head(space, ):
-    """Return the interpreter state object at the head of the list of all such objects.
-    """
-    raise NotImplementedError
-
- at cpython_api([PyInterpreterState], PyInterpreterState)
-def PyInterpreterState_Next(space, interp):
-    """Return the next interpreter state object after interp from the list of all
-    such objects.
-    """
-    raise NotImplementedError
-
 @cpython_api([PyInterpreterState], PyThreadState)
 def PyInterpreterState_ThreadHead(space, interp):
     """Return the a pointer to the first PyThreadState object in the list of
@@ -1979,25 +1959,6 @@
     changes in your code for properly supporting 64-bit systems."""
     raise NotImplementedError
 
- at cpython_api([PyObject, Py_ssize_t, PyObject], rffi.INT_real, error=-1)
-def PySequence_SetItem(space, o, i, v):
-    """Assign object v to the ith element of o.  Returns -1 on failure.  This
-    is the equivalent of the Python statement o[i] = v.  This function does
-    not steal a reference to v.
-    
-    This function used an int type for i. This might require
-    changes in your code for properly supporting 64-bit systems."""
-    raise NotImplementedError
-
- at cpython_api([PyObject, Py_ssize_t], rffi.INT_real, error=-1)
-def PySequence_DelItem(space, o, i):
-    """Delete the ith element of object o.  Returns -1 on failure.  This is the
-    equivalent of the Python statement del o[i].
-    
-    This function used an int type for i. This might require
-    changes in your code for properly supporting 64-bit systems."""
-    raise NotImplementedError
-
 @cpython_api([PyObject, PyObject], Py_ssize_t, error=-1)
 def PySequence_Count(space, o, value):
     """Return the number of occurrences of value in o, that is, return the number

diff --git a/pypy/module/cpyext/unicodeobject.py b/pypy/module/cpyext/unicodeobject.py
--- a/pypy/module/cpyext/unicodeobject.py
+++ b/pypy/module/cpyext/unicodeobject.py
@@ -361,10 +361,7 @@
 
     If byteorder is NULL, the codec starts in native order mode.
 
-    Return NULL if an exception was raised by the codec.
-
-    This function used an int type for size. This might require
-    changes in your code for properly supporting 64-bit systems."""
+    Return NULL if an exception was raised by the codec."""
 
     string = rffi.charpsize2str(s, size)
 
@@ -405,10 +402,7 @@
 @cpython_api([rffi.CCHARP, Py_ssize_t, rffi.CCHARP], PyObject)
 def PyUnicode_DecodeASCII(space, s, size, errors):
     """Create a Unicode object by decoding size bytes of the ASCII encoded string
-    s.  Return NULL if an exception was raised by the codec.
-    
-    This function used an int type for size. This might require
-    changes in your code for properly supporting 64-bit systems."""
+    s.  Return NULL if an exception was raised by the codec."""
     w_s = space.wrap(rffi.charpsize2str(s, size))
     return space.call_method(w_s, 'decode', space.wrap('ascii'))
 
@@ -416,9 +410,7 @@
 def PyUnicode_EncodeASCII(space, s, size, errors):
     """Encode the Py_UNICODE buffer of the given size using ASCII and return a
     Python string object.  Return NULL if an exception was raised by the codec.
-    
-    This function used an int type for size. This might require
-    changes in your code for properly supporting 64-bit systems."""
+    """
 
     w_s = space.wrap(rffi.wcharpsize2unicode(s, size))
     return space.call_method(w_s, 'encode', space.wrap('ascii'))

diff --git a/pypy/module/cpyext/pystate.py b/pypy/module/cpyext/pystate.py
--- a/pypy/module/cpyext/pystate.py
+++ b/pypy/module/cpyext/pystate.py
@@ -73,12 +73,12 @@
         return ec.cpyext_threadstate
 
 @cpython_api([], PyThreadState, error=CANNOT_FAIL)
-def PyThreadState_Get(space, ):
+def PyThreadState_Get(space):
     state = space.fromcache(InterpreterState)
     return state.get_thread_state(space)
 
 @cpython_api([], PyInterpreterState, error=CANNOT_FAIL)
-def PyInterpreterState_Head(space, ):
+def PyInterpreterState_Head(space):
     """Return the interpreter state object at the head of the list of all such objects.
     """
     return space.fromcache(InterpreterState).interpreter_state

diff --git a/pypy/module/cpyext/sequence.py b/pypy/module/cpyext/sequence.py
--- a/pypy/module/cpyext/sequence.py
+++ b/pypy/module/cpyext/sequence.py
@@ -141,10 +141,7 @@
 def PySequence_SetItem(space, w_o, i, w_v):
     """Assign object v to the ith element of o.  Returns -1 on failure.  This
     is the equivalent of the Python statement o[i] = v.  This function does
-    not steal a reference to v.
-
-    This function used an int type for i. This might require
-    changes in your code for properly supporting 64-bit systems."""
+    not steal a reference to v."""
     if PyDict_Check(space, w_o) or not PySequence_Check(space, w_o):
         raise operationerrfmt(space.w_TypeError, "'%s' object does not support item assignment",
                              space.type(w_o).getname(space))
@@ -154,9 +151,6 @@
 @cpython_api([PyObject, Py_ssize_t], rffi.INT_real, error=-1)
 def PySequence_DelItem(space, w_o, i):
     """Delete the ith element of object o.  Returns -1 on failure.  This is the
-    equivalent of the Python statement del o[i].
-
-    This function used an int type for i. This might require
-    changes in your code for properly supporting 64-bit systems."""
+    equivalent of the Python statement del o[i]."""
     space.delitem(w_o, space.wrap(i))
     return 0

diff --git a/pypy/module/cpyext/sliceobject.py b/pypy/module/cpyext/sliceobject.py
--- a/pypy/module/cpyext/sliceobject.py
+++ b/pypy/module/cpyext/sliceobject.py
@@ -96,11 +96,7 @@
     You probably do not want to use this function.  If you want to use slice
     objects in versions of Python prior to 2.3, you would probably do well to
     incorporate the source of PySlice_GetIndicesEx(), suitably renamed,
-    in the source of your extension.
-    
-    This function used an int type for length and an
-    int * type for start, stop, and step. This might require
-    changes in your code for properly supporting 64-bit systems."""
+    in the source of your extension."""
     if not PySlice_Check(space, w_slice):
         PyErr_BadInternalCall(space)
     assert isinstance(w_slice, W_SliceObject)

From lac at codespeak.net  Sun Feb 27 17:12:32 2011
From: lac at codespeak.net (lac at codespeak.net)
Date: Sun, 27 Feb 2011 17:12:32 +0100 (CET)
Subject: [pypy-svn] r80396 - pypy/extradoc/pycon-advertisement
Message-ID: <20110227161232.7AB33282B8B@codespeak.net>

Author: lac
Date: Sun Feb 27 17:12:29 2011
New Revision: 80396

Added:
   pypy/extradoc/pycon-advertisement/draft.3
Log:
Combined version for the pycon swag bag


Added: pypy/extradoc/pycon-advertisement/draft.3
==============================================================================
--- (empty file)
+++ pypy/extradoc/pycon-advertisement/draft.3	Sun Feb 27 17:12:29 2011
@@ -0,0 +1,64 @@
+High-level languages are faster than low-level ones! - (Armin Rigo circa 1999)
+
+For the past 8 years, the PyPy team has been working on this dream.  We
+are delighted that we can now bring you a very fast, very flexible, and very
+compliant Python, ready for production use.  We are very  proud of this 
+result, and especially of our Just-In-Time compiler generator.
+
+* most Python benchmarks run much faster than with CPython or Psyco
+* tight loops run between 20 and 60 times faster than CPython
+* our favourite large Python program, PyPy itself (200 KLocs) runs 2x as fast
+* smaller total memory usage for large and long-running programs
+* full (and JIT-ed) ctypes support for calling C libraries from Python
+* a new "cpyext" layer which integrates existing CPython C extensions
+* an experimental super-fast JIT-compilation of calls to C++ libraries
+* support for X86 and amd64 -- we are very close to supporting ARM
+* we are reliable, and with over 16,000 tests we can prove it.
+* our generational GC is much faster than CPython's refcounting one.
+* by the end of PyCon we will have full compatibility with  CPython 2.7
+
+To see our speed results visit the PyPy Speed Center: http://speed.pypy.org/
+
+To find out more about PyPy:
+   - visit  http://pypy.org
+   - read our blog at http://morepypy.blogspot.com/ 
+   - or come by the #pypy channel on irc.freenode.net 
+
+
+But we'd like your help. There are many ways you could support PyPy, an 
+MIT-licensed Open Souce project, including:
+   - contributing code  
+   - coming to sprints
+   - running your test suites under PyPy and reporting any errors you find
+   - donating money to us through our non-profit foundation, via Flattr, 
+     paypal, or Google Checkout. 
+
+Making a donation is easy. It's just one click away on our blog site
+http://morepypy.blogspot.com/ And because PyPy is a member of the
+Software Freedom Conservancy, a a 501(c)(3) charity, American donors
+can often deduct the donation on their USA taxes.
+
+Another way that you could help us is through finding us Paid Consulting 
+Work.  We are the proven Python speed experts. And we know that companies and 
+organizations exist out there for whom speed really is money.  They'd be 
+happy to pay us to speed up their code, if only they knew that this service
+was available.  Maybe you work for such a company, or maybe you know somebody
+who does.  We'd like to get in touch.  Things we'd like to do for you include:
+
+- speeding up your Python based project
+- making your product run seemlessly on top of PyPy
+- teaching you how to get the best out of PyPy
+- modifying PyPy to better suit your needs
+- on-site mentoring of developers, company sprints, teaching agile methodology
+- commercial support contracts for PyPy
+- porting your favourite C/C++ extension library to RPython, so that PyPy
+  can jit it.
+- integrating PyPy with Cython or NumPy
+
+We could write a new backend so that PyPy can run on your your favourite
+VM or CPU archtecture, or even write an interpreter for your second favourite
+dynamic programming language, so that it can be sped up as well!
+
+Open End and merlinux along with the PyPy developers stand ready to help,
+contact us at contracts at pypy.org
+


From arigo at codespeak.net  Sun Feb 27 18:07:59 2011
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sun, 27 Feb 2011 18:07:59 +0100 (CET)
Subject: [pypy-svn] r80397 - pypy/extradoc/pycon-advertisement
Message-ID: <20110227170759.5345F282B8B@codespeak.net>

Author: arigo
Date: Sun Feb 27 18:07:57 2011
New Revision: 80397

Modified:
   pypy/extradoc/pycon-advertisement/draft.3
Log:
Remove duplicates.

Modified: pypy/extradoc/pycon-advertisement/draft.3
==============================================================================
--- pypy/extradoc/pycon-advertisement/draft.3	(original)
+++ pypy/extradoc/pycon-advertisement/draft.3	Sun Feb 27 18:07:57 2011
@@ -35,7 +35,7 @@
 
 Making a donation is easy. It's just one click away on our blog site
 http://morepypy.blogspot.com/ And because PyPy is a member of the
-Software Freedom Conservancy, a a 501(c)(3) charity, American donors
+Software Freedom Conservancy, a 501(c)(3) charity, American donors
 can often deduct the donation on their USA taxes.
 
 Another way that you could help us is through finding us Paid Consulting 
@@ -55,7 +55,7 @@
   can jit it.
 - integrating PyPy with Cython or NumPy
 
-We could write a new backend so that PyPy can run on your your favourite
+We could write a new backend so that PyPy can run on your favourite
 VM or CPU archtecture, or even write an interpreter for your second favourite
 dynamic programming language, so that it can be sped up as well!
 


From commits-noreply at bitbucket.org  Sun Feb 27 19:38:43 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Sun, 27 Feb 2011 19:38:43 +0100 (CET)
Subject: [pypy-svn] pypy jit-usable_retrace: hg merge default
Message-ID: <20110227183843.BC5F02A202D@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-usable_retrace
Changeset: r42313:2d707366bab6
Date: 2011-02-27 09:28 +0100
http://bitbucket.org/pypy/pypy/changeset/2d707366bab6/

Log:	hg merge default


From commits-noreply at bitbucket.org  Sun Feb 27 19:38:46 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Sun, 27 Feb 2011 19:38:46 +0100 (CET)
Subject: [pypy-svn] pypy jit-usable_retrace: - Retraces now inherits the
	values from the	bridge producing them.
Message-ID: <20110227183846.9C2192A2033@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-usable_retrace
Changeset: r42314:9b3ef86e6087
Date: 2011-02-27 19:34 +0100
http://bitbucket.org/pypy/pypy/changeset/9b3ef86e6087/

Log:	- Retraces now inherits the values from the bridge producing them.
	- Short preamble now created by enum_forced_boxes and from
	optimizer.producer and there is no need to figgure out when it is
	safe to reorder resops (work in progress, only VirtalValue's
	support).
	- GUARD_NONNULL's emitted for boxes where box.nonnull() is True if
	needed to call a loop instead of retracing.
	- Guards emitted for boxes if box.getint() layes within the required
	intebount if needed to call a loop instead of retracing.

diff --git a/pypy/jit/metainterp/optimizeopt/unroll.py b/pypy/jit/metainterp/optimizeopt/unroll.py
--- a/pypy/jit/metainterp/optimizeopt/unroll.py
+++ b/pypy/jit/metainterp/optimizeopt/unroll.py
@@ -7,7 +7,7 @@
 from pypy.rlib.debug import debug_start, debug_stop, debug_print
 from pypy.jit.metainterp.optimizeutil import InvalidLoop, RetraceLoop
 from pypy.jit.metainterp.jitexc import JitException
-from pypy.jit.metainterp.history import make_hashable_int
+from pypy.jit.metainterp.history import make_hashable_int, BoxPtr
 from pypy.jit.codewriter.effectinfo import EffectInfo
 
 # Assumptions
@@ -136,6 +136,7 @@
         assert len(self.state) == len(other.state)
         for i in range(len(self.state)):
             if not self.state[i].generalization_of(other.state[i]):
+                print 'Fails on element: %d'%i
                 return False
         return True
 
@@ -145,6 +146,9 @@
             self.state[i].generate_guards(other.state[i], args[i],
                                           cpu, extra_guards)
 
+    def __repr__(self):
+        return "VirtualState:\n" + "\n".join(["  "+str(s) for s in self.state])
+
 class VirtualStateAdder(resume.ResumeDataVirtualAdder):
     def __init__(self, optimizer):
         self.fieldboxes = {}
@@ -215,9 +219,10 @@
     def _generate_guards(self, other, box, cpu, extra_guards):
         if not isinstance(other, NotVirtualInfo):
             raise InvalidLoop
+
         if self.level == LEVEL_KNOWNCLASS and \
-           box.nonnull() and \
-           self.known_class.same_constant(cpu.ts.cls_of_box(box)):
+               box.nonnull() and \
+               self.known_class.same_constant(cpu.ts.cls_of_box(box)):
             # Note: This is only a hint on what the class of box was
             # during the trace. There are actually no guarentees that this
             # box realy comes from a trace. The hint is used here to choose
@@ -228,11 +233,52 @@
             op = ResOperation(rop.GUARD_CLASS, [box, self.known_class], None)
             extra_guards.append(op)
             return
+
+        if self.level == LEVEL_NONNULL and \
+               other.level == LEVEL_UNKNOWN and \
+               isinstance(box, BoxPtr) and \
+               box.nonnull():
+            op = ResOperation(rop.GUARD_NONNULL, [box], None)
+            extra_guards.append(op)
+            return
+        
+        if self.level == LEVEL_UNKNOWN and \
+               other.level == LEVEL_UNKNOWN and \
+               isinstance(box, BoxInt) and \
+               self.intbound.contains(box.getint()):
+            if self.intbound.has_lower:
+                bound = self.intbound.lower
+                if not (other.intbound.has_lower and \
+                        other.intbound.lower >= bound):
+                    res = BoxInt()
+                    op = ResOperation(rop.INT_GE, [box, ConstInt(bound)], res)
+                    extra_guards.append(op)
+                    op = ResOperation(rop.GUARD_TRUE, [res], None)
+                    extra_guards.append(op)
+            if self.intbound.has_upper:
+                bound = self.intbound.upper
+                if not (other.intbound.has_upper and \
+                        other.intbound.upper <= bound):
+                    res = BoxInt()
+                    op = ResOperation(rop.INT_LE, [box, ConstInt(bound)], res)
+                    extra_guards.append(op)
+                    op = ResOperation(rop.GUARD_TRUE, [res], None)
+                    extra_guards.append(op)
+            return
+            
         # Remaining cases are probably not interesting
         raise InvalidLoop
         if self.level == LEVEL_CONSTANT:
             import pdb; pdb.set_trace()
             raise NotImplementedError
+
+    def __repr__(self):
+        l = {LEVEL_UNKNOWN: 'Unknonw',
+             LEVEL_NONNULL: 'NonNull',
+             LEVEL_KNOWNCLASS: 'KnownClass',
+             LEVEL_CONSTANT: 'Constant',
+             None: 'None'}[self.level]
+        return 'NotVirtualInfo(' + l + ', ' + str(self.intbound) + ')'
         
 
 class UnrollOptimizer(Optimization):
@@ -266,6 +312,11 @@
             modifier = VirtualStateAdder(self.optimizer)
             virtual_state = modifier.get_virtual_state(jump_args)
 
+            self.short_operations = []
+            self.boxes_seen_in_short = {}
+            for a in loop.preamble.inputargs:
+                self.boxes_seen_in_short[a] = True
+
             loop.preamble.operations = self.optimizer.newoperations
             self.optimizer = self.optimizer.reconstruct_for_next_iteration()
             inputargs = self.inline(self.cloned_operations,
@@ -274,6 +325,8 @@
             jmp = ResOperation(rop.JUMP, loop.inputargs[:], None)
             jmp.setdescr(loop.token)
             loop.preamble.operations.append(jmp)
+            if self.short_operations is not None:
+                self.short_operations.append(jmp)
 
             loop.operations = self.optimizer.newoperations
 
@@ -285,13 +338,15 @@
                 new_snapshot_args = []
                 for a in snapshot_args:
                     if not isinstance(a, Const):
-                        a = loop.preamble.inputargs[jump_args.index(a)]
+                        #a = loop.preamble.inputargs[jump_args.index(a)]
+                        a = self.getvalue(a).get_key_box()
                     new_snapshot_args.append(a)
                 snapshot.boxes = new_snapshot_args
                 snapshot = snapshot.prev
-
-            short = self.create_short_preamble(loop.preamble, loop)
-            if short:
+                
+            #short = self.create_short_preamble(loop.preamble, loop)
+            short = self.short_operations
+            if short is not None:
                 if False:
                     # FIXME: This should save some memory but requires
                     # a lot of tests to be fixed...
@@ -307,7 +362,8 @@
                         short[i] = op
 
                 short_loop = TreeLoop('short preamble')
-                short_loop.inputargs = loop.preamble.inputargs[:]
+                #short_loop.inputargs = loop.preamble.inputargs[:]
+                short_loop.inputargs = [self.getvalue(b).get_key_box() for b in jump_args]
                 short_loop.operations = short
 
                 # Clone ops and boxes to get private versions and 
@@ -317,6 +373,7 @@
                 ops = [inliner.inline_op(op) for op in short_loop.operations]
                 short_loop.operations = ops
                 descr = start_resumedescr.clone_if_mutable()
+                #self.inliner.inline_descr_inplace(descr)
                 inliner.inline_descr_inplace(descr)
                 short_loop.start_resumedescr = descr
 
@@ -333,21 +390,34 @@
                 for op in short_loop.operations:
                     if op.result:
                         op.result.forget_value()
-                
-    def inline(self, loop_operations, loop_args, jump_args):
-        self.inliner = inliner = Inliner(loop_args, jump_args)
-           
-        for v in self.optimizer.values.values():
-            v.last_guard_index = -1 # FIXME: Are there any more indexes stored?
 
+    def enum_forced_boxes(self, jump_args, short):
         inputargs = []
         seen_inputargs = {}
         for arg in jump_args:
             boxes = []
-            self.getvalue(arg).enum_forced_boxes(boxes, seen_inputargs)
+            self.getvalue(arg).enum_forced_boxes(boxes, seen_inputargs, short)
             for a in boxes:
                 if not isinstance(a, Const):
                     inputargs.append(a)
+        return inputargs
+        
+    def inline(self, loop_operations, loop_args, jump_args):
+
+        # FIXME: Move this to recreate_for_next_iteration
+        for v in self.optimizer.values.values():
+            v.last_guard_index = -1 # FIXME: Are there any more indexes stored?
+
+        boxes = {}
+        for i in range(len(loop_args)):
+            if loop_args[i] is jump_args[i]:
+                value = self.getvalue(loop_args[i])
+                value.enum_forced_boxes([], boxes, [])
+        passing = boxes.keys()
+        self.inliner = inliner = Inliner(loop_args + passing,
+                                         jump_args + passing)
+
+        inputargs = self.enum_forced_boxes(jump_args, self.short_operations)
 
         # This loop is equivalent to the main optimization loop in
         # Optimizer.propagate_all_forward
@@ -378,6 +448,7 @@
                 if not isinstance(a, Const) and not a in boxes_created_this_iteration:
                     if a not in inputargs:
                         inputargs.append(a)
+                        self.produce_box_in_short_preamble(a)
                         box = inliner.inline_arg(a)
                         if box in self.optimizer.values:
                             box = self.optimizer.values[box].force_box()
@@ -413,6 +484,32 @@
 
         return True
 
+    def produce_box_in_short_preamble(self, box):
+        if box in self.boxes_seen_in_short or isinstance(box, Const):
+            return
+        if self.short_operations is None:
+            return
+        self.boxes_seen_in_short[box] = True
+        op = self.optimizer.producer[box]
+
+        ok = False
+        if op.is_always_pure() or op.is_ovf():
+            ok = True
+            # FIXME: Allow getitems if they are still in the heap cache
+        elif op.getopnum() == rop.CALL:
+            effectinfo = op.getdescr().get_extra_info()
+            if effectinfo.extraeffect == EffectInfo.EF_LOOPINVARIANT:
+                ok = True
+            if effectinfo.extraeffect == EffectInfo.EF_PURE:
+                ok = True
+        if ok:
+            for arg in op.getarglist():
+                self.produce_box_in_short_preamble(arg)
+            if self.short_operations is not None:
+                self.short_operations.append(op)
+        else:
+            self.short_operations = None
+        
     def create_short_preamble(self, preamble, loop):
         #return None # Dissable
 
@@ -647,9 +744,16 @@
                 args = op.getarglist()
                 modifier = VirtualStateAdder(self.optimizer)
                 virtual_state = modifier.get_virtual_state(args)
+                print
+                print
+                print 'Lookup: ', virtual_state
+                print
+                
                 for sh in short:
                     ok = False
                     extra_guards = []
+                    print sh.virtual_state
+                    print
                     if sh.virtual_state.generalization_of(virtual_state):
                         ok = True
                     else:
@@ -677,9 +781,10 @@
                             jumpop = self.optimizer.newoperations.pop()
                             assert jumpop.getopnum() == rop.JUMP
                             for guard in extra_guards:
-                                descr = sh.start_resumedescr.clone_if_mutable()
-                                self.inliner.inline_descr_inplace(descr)
-                                guard.setdescr(descr)
+                                if guard.is_guard():
+                                    descr = sh.start_resumedescr.clone_if_mutable()
+                                    self.inliner.inline_descr_inplace(descr)
+                                    guard.setdescr(descr)
                                 self.emit_operation(guard)
                             self.optimizer.newoperations.append(jumpop)
                         return
@@ -687,7 +792,7 @@
                 if descr.failed_states:
                     retraced_count += len(descr.failed_states)
                 limit = self.optimizer.metainterp_sd.warmrunnerdesc.memory_manager.retrace_limit
-                if not self.retrace and retraced_count<limit:
+                if not self.retrace and (retraced_count<limit or limit<0):
                     if not descr.failed_states:
                         raise RetraceLoop
                     for failed in descr.failed_states:

diff --git a/pypy/jit/metainterp/test/test_basic.py b/pypy/jit/metainterp/test/test_basic.py
--- a/pypy/jit/metainterp/test/test_basic.py
+++ b/pypy/jit/metainterp/test/test_basic.py
@@ -362,6 +362,23 @@
                           'int_add': 1, 'int_sub': 1, 'int_gt': 1,
                           'jump': 1})
 
+    def test_loop_invariant_mul2(self):
+        myjitdriver = JitDriver(greens = [], reds = ['y', 'res', 'x'])
+        def f(x, y):
+            res = 0
+            while y > 0:
+                myjitdriver.can_enter_jit(x=x, y=y, res=res)
+                myjitdriver.jit_merge_point(x=x, y=y, res=res)
+                res += x * x + 2 * x
+                y -= 1
+            return res
+        res = self.meta_interp(f, [6, 7])
+        assert res == f(6, 7)
+        self.check_loop_count(1)
+        self.check_loops({'guard_true': 1,
+                          'int_add': 1, 'int_sub': 1, 'int_gt': 1,
+                          'jump': 1})
+
     def test_loop_invariant_mul_ovf(self):
         myjitdriver = JitDriver(greens = [], reds = ['y', 'res', 'x'])
         def f(x, y):
@@ -412,10 +429,10 @@
         res = self.meta_interp(f, [6, 32])
         assert res == 1167
         self.check_loop_count(3)
-        self.check_loops({'int_add': 3, 'int_lt': 2,
+        self.check_loops({'int_add': 3, 'int_lt': 1,
                           'int_sub': 2, 'guard_false': 1,
                           'jump': 2,
-                          'int_gt': 1, 'guard_true': 2})
+                          'int_gt': 1, 'guard_true': 1})
 
 
     def test_loop_invariant_mul_bridge_maintaining2(self):
@@ -434,10 +451,10 @@
         res = self.meta_interp(f, [6, 32])
         assert res == 1692
         self.check_loop_count(3)
-        self.check_loops({'int_add': 3, 'int_lt': 2,
+        self.check_loops({'int_add': 3, 'int_lt': 1,
                           'int_sub': 2, 'guard_false': 1,
                           'jump': 2,
-                          'int_gt': 1, 'guard_true': 2})
+                          'int_gt': 1, 'guard_true': 1})
 
     def test_loop_invariant_mul_bridge_maintaining3(self):
         myjitdriver = JitDriver(greens = [], reds = ['y', 'res', 'x', 'm'])
@@ -1504,8 +1521,8 @@
             return x
         res = self.meta_interp(f, [299], listops=True)
         assert res == f(299)
-        self.check_loops(guard_class=0, guard_value=2)        
-        self.check_loops(guard_class=0, guard_value=5, everywhere=True)
+        self.check_loops(guard_class=0, guard_value=3)        
+        self.check_loops(guard_class=0, guard_value=6, everywhere=True)
 
     def test_merge_guardnonnull_guardclass(self):
         from pypy.rlib.objectmodel import instantiate
@@ -1533,11 +1550,10 @@
             return x
         res = self.meta_interp(f, [299], listops=True)
         assert res == f(299)
-        self.check_loops(guard_class=0, guard_nonnull=0,
-                         guard_nonnull_class=2, guard_isnull=0)
-        self.check_loops(guard_class=0, guard_nonnull=0,
-                         guard_nonnull_class=4, guard_isnull=1,
-                         everywhere=True)
+        # There will be 1 guard_nonnull on l at the end of each bridge
+        # as the bridges don't inhert that information from the loop
+        self.check_loops(guard_class=0, guard_nonnull=2,
+                         guard_nonnull_class=2, guard_isnull=1)
 
     def test_merge_guardnonnull_guardvalue(self):
         from pypy.rlib.objectmodel import instantiate
@@ -1564,11 +1580,10 @@
             return x
         res = self.meta_interp(f, [299], listops=True)
         assert res == f(299)
-        self.check_loops(guard_class=0, guard_nonnull=0, guard_value=1,
+        # There will be 1 guard_nonnull on l at the end of each bridge
+        # as the bridges don't inhert that information from the loop
+        self.check_loops(guard_class=0, guard_nonnull=2, guard_value=2,
                          guard_nonnull_class=0, guard_isnull=1)
-        self.check_loops(guard_class=0, guard_nonnull=0, guard_value=3,
-                         guard_nonnull_class=0, guard_isnull=2,
-                         everywhere=True)
 
     def test_merge_guardnonnull_guardvalue_2(self):
         from pypy.rlib.objectmodel import instantiate
@@ -1595,11 +1610,10 @@
             return x
         res = self.meta_interp(f, [299], listops=True)
         assert res == f(299)
-        self.check_loops(guard_class=0, guard_nonnull=0, guard_value=2,
-                         guard_nonnull_class=0, guard_isnull=0)
-        self.check_loops(guard_class=0, guard_nonnull=0, guard_value=4,
-                         guard_nonnull_class=0, guard_isnull=1,
-                         everywhere=True)
+        # There will be 1 guard_nonnull on l at the end of each bridge
+        # as the bridges don't inhert that information from the loop
+        self.check_loops(guard_class=0, guard_nonnull=2, guard_value=2,
+                         guard_nonnull_class=0, guard_isnull=1)
 
     def test_merge_guardnonnull_guardclass_guardvalue(self):
         from pypy.rlib.objectmodel import instantiate
@@ -1629,11 +1643,10 @@
             return x
         res = self.meta_interp(f, [399], listops=True)
         assert res == f(399)
-        self.check_loops(guard_class=0, guard_nonnull=0, guard_value=2,
-                         guard_nonnull_class=0, guard_isnull=0)
-        self.check_loops(guard_class=0, guard_nonnull=0, guard_value=5,
-                         guard_nonnull_class=0, guard_isnull=1,
-                         everywhere=True)
+        # There will be 1 guard_nonnull on l at the end of each bridge
+        # as the bridges don't inhert that information from the loop
+        self.check_loops(guard_class=0, guard_nonnull=3, guard_value=3,
+                         guard_nonnull_class=0, guard_isnull=1)
 
     def test_residual_call_doesnt_lose_info(self):
         myjitdriver = JitDriver(greens = [], reds = ['x', 'y', 'l'])
@@ -1886,7 +1899,7 @@
                           'int_add': 1, 'int_mul': 1, 'int_sub': 2,
                           'int_gt': 2, 'jump': 2})
 
-    def test_multiple_specialied_versions_array(self):
+    def test_multiple_specialied_versions_array1(self):
         myjitdriver = JitDriver(greens = [], reds = ['idx', 'y', 'x', 'res',
                                                      'array'])
         class Base:
@@ -1925,7 +1938,50 @@
         res = self.meta_interp(g, [6, 14])
         assert res == g(6, 14)
         self.check_loop_count(9)
-        self.check_loops(getarrayitem_gc=6, everywhere=True)
+        self.check_loops(getarrayitem_gc=8, everywhere=True)
+        self.check_loops(getarrayitem_gc=4);
+
+    def test_multiple_specialied_versions_array2(self):
+        myjitdriver = JitDriver(greens = [], reds = ['idx', 'n', 'm', 'y', 'x',
+                                                     'res', 'array'])
+        class Base:
+            def __init__(self, val):
+                self.val = val
+        class A(Base):
+            def binop(self, other):
+                return A(self.val + other.val)
+        class B(Base):
+            def binop(self, other):
+                return B(self.val - other.val)
+        def f(x, y, n, m):
+            res = x
+            array = [1, 2, 3]
+            array[1] = 7
+            idx = 0
+            while y > m:
+                myjitdriver.can_enter_jit(idx=idx, y=y, x=x, res=res,
+                                          array=array, n=n, m=m)
+                myjitdriver.jit_merge_point(idx=idx, y=y, x=x, res=res,
+                                            array=array, n=n, m=m)
+                res = res.binop(x)
+                res.val += array[idx] + array[1]
+                if y < n:
+                    idx = 2
+                y -= 1
+            return res
+        def g(x, y, m):
+            a1 = f(A(x), y, y/2, m)
+            a2 = f(A(x), y, y/2, m)
+            b1 = f(B(x), y, y/2, m)
+            b2 = f(B(x), y, y/2, m)
+            assert a1.val == a2.val
+            assert b1.val == b2.val
+            return a1.val + b1.val
+        res = self.meta_interp(g, [6, 28, 0])
+        assert res == g(6, 28, 0)
+        self.check_loop_count(9)
+        self.check_loops(getarrayitem_gc=12, everywhere=True)
+        self.check_loops(getarrayitem_gc=4)
 
     def test_multiple_specialied_versions_bridge(self):
         myjitdriver = JitDriver(greens = [], reds = ['y', 'x', 'z', 'res'])
@@ -2113,7 +2169,7 @@
         res = self.meta_interp(g, [3, 23])
         assert res == 7068153
         self.check_loop_count(6)
-        self.check_loops(guard_true=4, guard_class=0, int_add=2, int_mul=2,
+        self.check_loops(guard_true=5, guard_class=0, int_add=2, int_mul=2,
                          guard_false=2)
 
     def test_dont_trace_every_iteration(self):

diff --git a/pypy/jit/metainterp/resume.py b/pypy/jit/metainterp/resume.py
--- a/pypy/jit/metainterp/resume.py
+++ b/pypy/jit/metainterp/resume.py
@@ -484,7 +484,6 @@
     def _generalization_of(self, other):
         raise NotImplementedError
 
-
 class VirtualInfo(AbstractVirtualStructInfo):
     def __init__(self, known_class, fielddescrs):
         AbstractVirtualStructInfo.__init__(self, fielddescrs)
@@ -506,6 +505,9 @@
         if not self.known_class.same_constant(other.known_class):
             return False
         return True
+
+    def __repr__(self):
+        return 'VirtualInfo(' + str(self.known_class) + ')'
         
 
 class VStructInfo(AbstractVirtualStructInfo):

diff --git a/pypy/jit/metainterp/optimizeopt/optimizer.py b/pypy/jit/metainterp/optimizeopt/optimizer.py
--- a/pypy/jit/metainterp/optimizeopt/optimizer.py
+++ b/pypy/jit/metainterp/optimizeopt/optimizer.py
@@ -34,7 +34,8 @@
 
     def __init__(self, box, level=None, known_class=None, intbound=None):
         self.box = box
-        self.level = level
+        if level is not None:
+            self.level = level
         self.known_class = known_class
         if intbound:
             self.intbound = intbound
@@ -51,7 +52,7 @@
     def get_key_box(self):
         return self.box
 
-    def enum_forced_boxes(self, boxes, already_seen):
+    def enum_forced_boxes(self, boxes, already_seen, shortops):
         key = self.get_key_box()
         if key not in already_seen:
             boxes.append(self.force_box())
@@ -259,9 +260,17 @@
         self.posponedop = None
         self.exception_might_have_happened = False
         self.newoperations = []
+        if self.loop.inputvalues:
+            self.setup_inputstate()
         self.set_optimizations(optimizations)
 
-
+    def setup_inputstate(self):
+        valuemap = {}
+        assert len(self.loop.inputvalues) == len(self.loop.inputargs)
+        for i in range(len(self.loop.inputvalues)):
+            value = self.loop.inputvalues[i].get_cloned(self, valuemap)
+            self.make_equal_to(self.loop.inputargs[i], value)
+        
     def set_optimizations(self, optimizations):
         if optimizations:
             self.first_optimization = optimizations[0]

diff --git a/pypy/jit/metainterp/optimizeopt/virtualize.py b/pypy/jit/metainterp/optimizeopt/virtualize.py
--- a/pypy/jit/metainterp/optimizeopt/virtualize.py
+++ b/pypy/jit/metainterp/optimizeopt/virtualize.py
@@ -132,7 +132,7 @@
                 fieldvalue = self._fields[ofs]
                 fieldvalue.get_args_for_fail(modifier)
 
-    def enum_forced_boxes(self, boxes, already_seen):
+    def enum_forced_boxes(self, boxes, already_seen, shortops):
         key = self.get_key_box()
         if key in already_seen:
             return
@@ -140,7 +140,11 @@
         if self.box is None:
             lst = self._get_field_descr_list()
             for ofs in lst:
-                self._fields[ofs].enum_forced_boxes(boxes, already_seen)
+                op = ResOperation(rop.GETFIELD_GC, [key],
+                                  self._fields[ofs].get_key_box(), ofs)
+                shortops.append(op)
+                self._fields[ofs].enum_forced_boxes(boxes, already_seen,
+                                                    shortops)
         else:
             boxes.append(self.box)
 

From commits-noreply at bitbucket.org  Sun Feb 27 19:38:47 2011
From: commits-noreply at bitbucket.org (hakanardo)
Date: Sun, 27 Feb 2011 19:38:47 +0100 (CET)
Subject: [pypy-svn] pypy jit-virtual_state: hg ci --close-branch
Message-ID: <20110227183847.6260A2A2034@codespeak.net>

Author: Hakan Ardo <hakan at debian.org>
Branch: jit-virtual_state
Changeset: r42315:f32c04c9a7ee
Date: 2011-02-27 19:35 +0100
http://bitbucket.org/pypy/pypy/changeset/f32c04c9a7ee/

Log:	hg ci --close-branch


From commits-noreply at bitbucket.org  Sun Feb 27 21:18:06 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Sun, 27 Feb 2011 21:18:06 +0100 (CET)
Subject: [pypy-svn] pypy kqueue: Added tests and skeleton.
Message-ID: <20110227201806.2B5D8282C5D@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: kqueue
Changeset: r42316:682046107ffe
Date: 2011-02-27 15:16 -0500
http://bitbucket.org/pypy/pypy/changeset/682046107ffe/

Log:	Added tests and skeleton.

diff --git a/pypy/module/select/test/test_kqueue.py b/pypy/module/select/test/test_kqueue.py
new file mode 100644
--- /dev/null
+++ b/pypy/module/select/test/test_kqueue.py
@@ -0,0 +1,181 @@
+from pypy.conftest import gettestobjspace
+
+
+class AppTestKqueue(object):
+    def setup_class(cls):
+        import errno
+        import select
+
+        cls.space = gettestobjspace(usemodules=["select", "_socket", "posix"])
+
+        if not hasattr(select, "kqueue"):
+            py.test.skip("test requires BSD")
+
+    def test_create(self):
+        import select
+
+        kq = select.kqueue()
+        assert kq.fileno() > 0
+        assert not kq.closed
+        kq.close()
+        assert kq.closed
+        raises(ValueError, kq.fileno)
+
+    def test_create_event(self):
+        import select
+        import sys
+
+        fd = sys.stderr.fileno()
+        ev = select.kevent(fd)
+        other = select.kevent(1000)
+        assert ev.ident == fd
+        assert ev.filter == select.KQ_FILTER_READ
+        assert ev.flags == select.KQ_EV_ADD
+        assert ev.fflags == 0
+        assert ev.data == 0
+        assert ev.udata == 0
+        assert ev == ev
+        assert ev != other
+        assert cmp(ev, other) == -1
+        assert ev < other
+        assert other >= ev
+        raises(TypeError, cmp, ev, None)
+        raises(TypeError, cmp, ev, 1)
+        raises(TypeError, cmp, ev, "ev")
+
+        ev = select.kevent(fd, select.KQ_FILTER_WRITE)
+        assert ev.ident == fd
+        assert ev.filter == select.KQ_FILTER_WRITE
+        assert ev.flags == select.KQ_EV_ADD
+        assert ev.fflags == 0
+        assert ev.data == 0
+        assert ev.udata == 0
+        assert ev == ev
+        assert ev != other
+
+        ev = select.kevent(fd, select.KQ_FILTER_WRITE, select.KQ_EV_ONESHOT)
+        assert ev.ident == fd
+        assert ev.filter == select.KQ_FILTER_WRITE
+        assert ev.flags == select.KQ_EV_ONESHOT
+        assert ev.fflags == 0
+        assert ev.data == 0
+        assert ev.udata == 0
+        assert ev == ev
+        assert ev != other
+
+        ev = select.kevent(1, 2, 3, 4, 5, 6)
+        assert ev.ident == 1
+        assert ev.filter == 2
+        assert ev.flags == 3
+        assert ev.fflags == 4
+        assert ev.data == 5
+        assert ev.udata == 6
+        assert ev == ev
+        assert ev != other
+
+        big = sys.maxsize * 2 + 1
+        ev = select.kevent(big, 1, 2, 3, sys.maxsize, big)
+        assert ev.ident == big
+        assert ev.filter == 1
+        assert ev.flags == 2
+        assert ev.fflags == 3
+        assert ev.data == sys.maxsize
+        assert ev.udata == big
+        assert ev == ev
+        assert ev != other
+
+    def test_queue_event(self):
+        import errno
+        import socket
+        import sys
+
+        server_socket = socket.socket()
+        server_socket.bind(("127.0.0.1", 0))
+        server_socket.listen(1)
+        client = socket.socket()
+        client.setblocking(False)
+        exc = raises(client.connect, ("127.0.0.1", server_socket.getsockname()[1]))
+        assert exc.value.args[0] == errno.EINPROGRESS
+        server, addr = server_socket.accept()
+
+        if sys.platform.startswith("darwin"):
+            flags = select.KQ_EV_ADD | select.KQ_EV_ENABLE
+        else:
+            flags = 0
+
+        kq1 = select.kqueue()
+        kq2 = select.kqueue.fromfd(kq.fileno())
+
+        ev = select.kevent(server.fileno(), select.KQ_FILTER_WRITE, select.KQ_EV_ADD | select.KQ_EV_ENABLE)
+        kq1.control([ev], 0)
+        ev = select.kevent(server.fileno(), select.KQ_FILTER_READ, select.KQ_EV_ADD | select.KQ_EV_ENABLE)
+        kq1.control([ev], 0)
+        ev = select.kevent(client.fileno(), select.KQ_FILTER_WRITE, select.KQ_EV_ADD | select.KQ_EV_ENABLE)
+        kq2.control([ev], 0)
+        ev = select.kevent(client.fileno(), select.KQ_FILTER_READ, select.KQ_EV_ADD | select.KQ_EV_ENABLE)
+        kq2.control([ev], 0)
+
+        events = kq1.control(None, 4, 1)
+        events = [(e.ident, e.filter, e.flags) for e in events]
+        events.sort()
+        assert events == [
+            (client.fileno(), select.KQ_FILTER_WRITE, flags),
+            (server.fileno(), select.KQ_FILTER_WRITE, flags),
+        ]
+        client.send("Hello!")
+        server.send("world!!!")
+
+        for i in xrange(10):
+            events = kq1.control(None, 4, 1)
+            if len(events) == 4:
+                break
+            time.sleep(.1)
+        else:
+            assert False, "timeout waiting for event notification"
+
+        events = [(e.ident, e.filter, e.flags) for e in events]
+        events.sort()
+        assert events == [
+            (client.fileno(), select.KQ_FILTER_WRITE, flags),
+            (client.fileno(), select.KQ_FILTER_READ, flags),
+            (server.fileno(), select.KQ_FILTER_WRITE, flags),
+            (server.fileno(), select.KQ_FILTER_READ, flags),
+        ]
+
+        ev = select.kevent(client.fileno(), select.KQ_FILTER_WRITE, select.KQ_EV_DELETE)
+        kq1.control([ev], 0)
+        ev = select.kevent(client.fileno(), select.KQ_FILTER_READ, select.KQ_EV_DELETE)
+        kq1.control([ev], 0)
+        ev = select.kevent(server.fileno(), select.KQ_FILTER_READ, select.KQ_EV_DELETE)
+        kq1.control([ev], 0, 0)
+
+        events = kq.control([], 4, 0.99)
+        events = [(e.ident, e.filter, e.flags) for e in events]
+        events.sort()
+        assert events == [
+            (serve.fileno(), select.KQ_FILTER_WRITE, flags),
+        ]
+
+        client.close()
+        server.close()
+        server_socket.close()
+
+    def test_pair(self):
+        import select
+        import socket
+
+        kq = select.kqueue()
+        a, b = socket.socketpair()
+
+        a.send('foo')
+        event1 = select.kevent(a, select.KQ_FILTER_READ, select.KQ_EV_ADD | select.KQ_EV_ENABLE)
+        event2 = select.kevent(b, select.KQ_FILTER_READ, select.KQ_EV_ADD | select.KQ_EV_ENABLE)
+        r = kq.control([event1, event2], 1, 1)
+        assert r
+        assert r[0].flags & select.KQ_EV_ERROR
+        data = b.recv(r[0].data)
+        assert data == 'foo'
+
+        a.close()
+        b.close()
+        kq.close()
\ No newline at end of file

diff --git a/pypy/module/select/test/test_epoll.py b/pypy/module/select/test/test_epoll.py
--- a/pypy/module/select/test/test_epoll.py
+++ b/pypy/module/select/test/test_epoll.py
@@ -5,11 +5,11 @@
 
 class AppTestEpoll(object):
     def setup_class(cls):
-        cls.space = gettestobjspace(usemodules=["select", "_socket", "posix"])
-
         import errno
         import select
 
+        cls.space = gettestobjspace(usemodules=["select", "_socket", "posix"])
+
         if not hasattr(select, "epoll"):
             py.test.skip("test requires linux 2.6")
         try:

diff --git a/pypy/module/select/__init__.py b/pypy/module/select/__init__.py
--- a/pypy/module/select/__init__.py
+++ b/pypy/module/select/__init__.py
@@ -27,6 +27,9 @@
             if hasattr(select, symbol):
                 interpleveldefs[symbol] = "space.wrap(%s)" % getattr(select, symbol)
 
+    if hasattr(select, "kqueue"):
+        interpleveldefs["kqueue"] = "interp_kqueue.W_Kqueue"
+
 
     def buildloaders(cls):
         from pypy.rlib import rpoll

diff --git a/pypy/module/select/interp_kqueue.py b/pypy/module/select/interp_kqueue.py
new file mode 100644
--- /dev/null
+++ b/pypy/module/select/interp_kqueue.py
@@ -0,0 +1,5 @@
+from pypy.interpreter.baseobjspace import Wrappable
+
+
+class W_Kqueue(Wrappable):
+    pass
\ No newline at end of file

From commits-noreply at bitbucket.org  Sun Feb 27 21:30:14 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Sun, 27 Feb 2011 21:30:14 +0100 (CET)
Subject: [pypy-svn] pypy kqueue: Fix a typo in the tests.
Message-ID: <20110227203014.C30B1282C36@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: kqueue
Changeset: r42317:dae97b70bdfe
Date: 2011-02-27 15:29 -0500
http://bitbucket.org/pypy/pypy/changeset/dae97b70bdfe/

Log:	Fix a typo in the tests.

diff --git a/pypy/module/select/test/test_kqueue.py b/pypy/module/select/test/test_kqueue.py
--- a/pypy/module/select/test/test_kqueue.py
+++ b/pypy/module/select/test/test_kqueue.py
@@ -94,7 +94,7 @@
         server_socket.listen(1)
         client = socket.socket()
         client.setblocking(False)
-        exc = raises(client.connect, ("127.0.0.1", server_socket.getsockname()[1]))
+        exc = raises(socket.error, client.connect, ("127.0.0.1", server_socket.getsockname()[1]))
         assert exc.value.args[0] == errno.EINPROGRESS
         server, addr = server_socket.accept()
 

From commits-noreply at bitbucket.org  Sun Feb 27 21:33:33 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Sun, 27 Feb 2011 21:33:33 +0100 (CET)
Subject: [pypy-svn] pypy kqueue: Remove this test,
	it doesn't seem to pass under CPython.
Message-ID: <20110227203333.A60312A2037@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: kqueue
Changeset: r42318:1ef20b0124c0
Date: 2011-02-27 15:33 -0500
http://bitbucket.org/pypy/pypy/changeset/1ef20b0124c0/

Log:	Remove this test, it doesn't seem to pass under CPython.

diff --git a/pypy/module/select/test/test_kqueue.py b/pypy/module/select/test/test_kqueue.py
--- a/pypy/module/select/test/test_kqueue.py
+++ b/pypy/module/select/test/test_kqueue.py
@@ -73,17 +73,6 @@
         assert ev == ev
         assert ev != other
 
-        big = sys.maxsize * 2 + 1
-        ev = select.kevent(big, 1, 2, 3, sys.maxsize, big)
-        assert ev.ident == big
-        assert ev.filter == 1
-        assert ev.flags == 2
-        assert ev.fflags == 3
-        assert ev.data == sys.maxsize
-        assert ev.udata == big
-        assert ev == ev
-        assert ev != other
-
     def test_queue_event(self):
         import errno
         import socket

From commits-noreply at bitbucket.org  Sun Feb 27 21:34:08 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Sun, 27 Feb 2011 21:34:08 +0100 (CET)
Subject: [pypy-svn] pypy kqueue: Missing import.
Message-ID: <20110227203408.563FF282C36@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: kqueue
Changeset: r42319:0b8353f1a1b4
Date: 2011-02-27 15:33 -0500
http://bitbucket.org/pypy/pypy/changeset/0b8353f1a1b4/

Log:	Missing import.

diff --git a/pypy/module/select/test/test_kqueue.py b/pypy/module/select/test/test_kqueue.py
--- a/pypy/module/select/test/test_kqueue.py
+++ b/pypy/module/select/test/test_kqueue.py
@@ -75,6 +75,7 @@
 
     def test_queue_event(self):
         import errno
+        import select
         import socket
         import sys
 

From commits-noreply at bitbucket.org  Sun Feb 27 21:34:42 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Sun, 27 Feb 2011 21:34:42 +0100 (CET)
Subject: [pypy-svn] pypy kqueue: typo fix.
Message-ID: <20110227203442.376192A2037@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: kqueue
Changeset: r42320:089f3314fd8b
Date: 2011-02-27 15:34 -0500
http://bitbucket.org/pypy/pypy/changeset/089f3314fd8b/

Log:	typo fix.

diff --git a/pypy/module/select/test/test_kqueue.py b/pypy/module/select/test/test_kqueue.py
--- a/pypy/module/select/test/test_kqueue.py
+++ b/pypy/module/select/test/test_kqueue.py
@@ -94,7 +94,7 @@
             flags = 0
 
         kq1 = select.kqueue()
-        kq2 = select.kqueue.fromfd(kq.fileno())
+        kq2 = select.kqueue.fromfd(kq1.fileno())
 
         ev = select.kevent(server.fileno(), select.KQ_FILTER_WRITE, select.KQ_EV_ADD | select.KQ_EV_ENABLE)
         kq1.control([ev], 0)

From commits-noreply at bitbucket.org  Sun Feb 27 21:35:42 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Sun, 27 Feb 2011 21:35:42 +0100 (CET)
Subject: [pypy-svn] pypy kqueue: One more typo fix.
Message-ID: <20110227203542.45484282C36@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: kqueue
Changeset: r42321:4e7c33ca4b88
Date: 2011-02-27 15:35 -0500
http://bitbucket.org/pypy/pypy/changeset/4e7c33ca4b88/

Log:	One more typo fix.

diff --git a/pypy/module/select/test/test_kqueue.py b/pypy/module/select/test/test_kqueue.py
--- a/pypy/module/select/test/test_kqueue.py
+++ b/pypy/module/select/test/test_kqueue.py
@@ -139,7 +139,7 @@
         ev = select.kevent(server.fileno(), select.KQ_FILTER_READ, select.KQ_EV_DELETE)
         kq1.control([ev], 0, 0)
 
-        events = kq.control([], 4, 0.99)
+        events = kq1.control([], 4, 0.99)
         events = [(e.ident, e.filter, e.flags) for e in events]
         events.sort()
         assert events == [

From commits-noreply at bitbucket.org  Sun Feb 27 21:36:13 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Sun, 27 Feb 2011 21:36:13 +0100 (CET)
Subject: [pypy-svn] pypy kqueue: Final typo fix.
Message-ID: <20110227203613.9B0462A2037@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: kqueue
Changeset: r42322:0a34a5b002d1
Date: 2011-02-27 15:36 -0500
http://bitbucket.org/pypy/pypy/changeset/0a34a5b002d1/

Log:	Final typo fix.

diff --git a/pypy/module/select/test/test_kqueue.py b/pypy/module/select/test/test_kqueue.py
--- a/pypy/module/select/test/test_kqueue.py
+++ b/pypy/module/select/test/test_kqueue.py
@@ -143,7 +143,7 @@
         events = [(e.ident, e.filter, e.flags) for e in events]
         events.sort()
         assert events == [
-            (serve.fileno(), select.KQ_FILTER_WRITE, flags),
+            (server.fileno(), select.KQ_FILTER_WRITE, flags),
         ]
 
         client.close()

From commits-noreply at bitbucket.org  Sun Feb 27 22:42:55 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Sun, 27 Feb 2011 22:42:55 +0100 (CET)
Subject: [pypy-svn] pypy default: Remove warning messages at the end of some
	tests, 
Message-ID: <20110227214255.F1B6E282CF8@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r42323:573706bcbe9e
Date: 2011-02-27 21:45 +0100
http://bitbucket.org/pypy/pypy/changeset/573706bcbe9e/

Log:	Remove warning messages at the end of some tests, when
	LowLevelType.__eq__ always raises because some globals are reset to
	None.

diff --git a/pypy/rpython/lltypesystem/lltype.py b/pypy/rpython/lltypesystem/lltype.py
--- a/pypy/rpython/lltypesystem/lltype.py
+++ b/pypy/rpython/lltypesystem/lltype.py
@@ -21,6 +21,10 @@
         weakref.WeakValueDictionary.__init__(self, *args, **kwargs)
         remove_base = self._remove
         def remove(*args):
+            if safe_equal is None:
+                # The interpreter is shutting down, and the comparison
+                # function is already gone.
+                return
             if TLS is None: # Happens when the interpreter is shutting down
                 return remove_base(*args)
             nested_hash_level = TLS.nested_hash_level

From commits-noreply at bitbucket.org  Sun Feb 27 22:42:56 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Sun, 27 Feb 2011 22:42:56 +0100 (CET)
Subject: [pypy-svn] pypy default: Export two cpyext functions that are
	already	implemented in C code
Message-ID: <20110227214256.9536D282CF8@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r42324:fe3ff859a338
Date: 2011-02-27 21:46 +0100
http://bitbucket.org/pypy/pypy/changeset/fe3ff859a338/

Log:	Export two cpyext functions that are already implemented in C code

diff --git a/pypy/module/cpyext/api.py b/pypy/module/cpyext/api.py
--- a/pypy/module/cpyext/api.py
+++ b/pypy/module/cpyext/api.py
@@ -308,7 +308,7 @@
 SYMBOLS_C = [
     'Py_FatalError', 'PyOS_snprintf', 'PyOS_vsnprintf', 'PyArg_Parse',
     'PyArg_ParseTuple', 'PyArg_UnpackTuple', 'PyArg_ParseTupleAndKeywords',
-    '_PyArg_NoKeywords',
+    'PyArg_VaParse', 'PyArg_VaParseTupleAndKeywords', '_PyArg_NoKeywords',
     'PyString_FromFormat', 'PyString_FromFormatV',
     'PyModule_AddObject', 'PyModule_AddIntConstant', 'PyModule_AddStringConstant',
     'Py_BuildValue', 'Py_VaBuildValue', 'PyTuple_Pack',

From commits-noreply at bitbucket.org  Sun Feb 27 22:42:58 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Sun, 27 Feb 2011 22:42:58 +0100 (CET)
Subject: [pypy-svn] pypy default: Regenerate cpyext/stubs.py after CPython
	2.7	documentation, 
Message-ID: <20110227214258.09A9C282D07@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r42325:4f3d3a30782f
Date: 2011-02-27 22:24 +0100
http://bitbucket.org/pypy/pypy/changeset/4f3d3a30782f/

Log:	Regenerate cpyext/stubs.py after CPython 2.7 documentation,
	massively edited so it can be imported in cpyext.

	"only" 40 functions were added. cpyext implements exactly half of
	the 688 functions composing the CPython API!

diff --git a/pypy/module/cpyext/stubs.py b/pypy/module/cpyext/stubs.py
--- a/pypy/module/cpyext/stubs.py
+++ b/pypy/module/cpyext/stubs.py
@@ -1,16 +1,16 @@
 from pypy.module.cpyext.api import (
     cpython_api, PyObject, PyObjectP, CANNOT_FAIL
     )
+from pypy.module.cpyext.complexobject import Py_complex_ptr as Py_complex
 from pypy.rpython.lltypesystem import rffi, lltype
 
 # we don't really care
 PyTypeObjectPtr = rffi.VOIDP
-Py_ssize_t = rffi.VOIDP
+Py_ssize_t = rffi.SSIZE_T
 PyMethodDef = rffi.VOIDP
 PyGetSetDef = rffi.VOIDP
 PyMemberDef = rffi.VOIDP
 Py_buffer = rffi.VOIDP
-Py_complex = rffi.VOIDP
 va_list = rffi.VOIDP
 PyDateTime_Date = rffi.VOIDP
 PyDateTime_DateTime = rffi.VOIDP
@@ -20,19 +20,21 @@
 PyFileObject = rffi.VOIDP
 PyCodeObject = rffi.VOIDP
 PyFrameObject = rffi.VOIDP
+PyFloatObject = rffi.VOIDP
 _inittab = rffi.VOIDP
 PyThreadState = rffi.VOIDP
 PyInterpreterState = rffi.VOIDP
 PyOS_sighandler_t = rffi.VOIDP
-Py_UNICODE = rffi.VOIDP
+Py_UNICODE = lltype.UniChar
 PyCompilerFlags = rffi.VOIDP
 _node = rffi.VOIDP
+Py_tracefunc = rffi.VOIDP
 
 @cpython_api([PyObject], lltype.Void)
 def _PyObject_Del(space, op):
     raise NotImplementedError
 
- at cpython_api([PyObject], rffi.INT_real, error=0)
+ at cpython_api([PyObject], rffi.INT_real, error=CANNOT_FAIL)
 def PyObject_CheckBuffer(space, obj):
     """Return 1 if obj supports the buffer interface otherwise 0."""
     raise NotImplementedError
@@ -59,10 +61,6 @@
     
     The following table gives possible values to the flags arguments.
     
-    
-    
-    
-    
     Flag
     
     Description
@@ -187,21 +185,7 @@
     ~Py_buffer.format."""
     raise NotImplementedError
 
- at cpython_api([PyObject, rffi.VOIDP_real, Py_ssize_t, lltype.Char], rffi.INT_real, error=-1)
-def PyObject_CopyToObject(space, obj, buf, len, fortran):
-    """Copy len bytes of data pointed to by the contiguous chunk of memory
-    pointed to by buf into the buffer exported by obj.  The buffer must of
-    course be writable.  Return 0 on success and return -1 and raise an error
-    on failure.  If the object does not have a writable buffer, then an error
-    is raised.  If fortran is 'F', then if the object is
-    multi-dimensional, then the data will be copied into the array in
-    Fortran-style (first dimension varies the fastest).  If fortran is
-    'C', then the data will be copied into the array in C-style (last
-    dimension varies the fastest).  If fortran is 'A', then it does not
-    matter and the copy will be made in whatever way is more efficient."""
-    raise NotImplementedError
-
- at cpython_api([Py_buffer, lltype.Char], rffi.INT_real, error=-1)
+ at cpython_api([Py_buffer, lltype.Char], rffi.INT_real, error=CANNOT_FAIL)
 def PyBuffer_IsContiguous(space, view, fortran):
     """Return 1 if the memory defined by the view is C-style (fortran is
     'C') or Fortran-style (fortran is 'F') contiguous or either one
@@ -215,13 +199,43 @@
     given shape with the given number of bytes per element."""
     raise NotImplementedError
 
- at cpython_api([Py_buffer, PyObject, rffi.VOIDP_real, Py_ssize_t, rffi.INT_real, rffi.INT_real], rffi.INT_real, error=-1)
+ at cpython_api([Py_buffer, PyObject, rffi.VOIDP, Py_ssize_t, rffi.INT_real, rffi.INT_real], rffi.INT_real, error=-1)
 def PyBuffer_FillInfo(space, view, obj, buf, len, readonly, infoflags):
     """Fill in a buffer-info structure, view, correctly for an exporter that can
     only share a contiguous chunk of memory of "unsigned bytes" of the given
     length.  Return 0 on success and -1 (with raising an error) on error."""
     raise NotImplementedError
 
+ at cpython_api([Py_buffer], PyObject)
+def PyMemoryView_FromBuffer(space, view):
+    """Create a memoryview object wrapping the given buffer-info structure view.
+    The memoryview object then owns the buffer, which means you shouldn't
+    try to release it yourself: it will be released on deallocation of the
+    memoryview object."""
+    raise NotImplementedError
+
+ at cpython_api([PyObject, rffi.INT_real, lltype.Char], PyObject)
+def PyMemoryView_GetContiguous(space, obj, buffertype, order):
+    """Create a memoryview object to a contiguous chunk of memory (in either
+    'C' or 'F'ortran order) from an object that defines the buffer
+    interface. If memory is contiguous, the memoryview object points to the
+    original memory. Otherwise copy is made and the memoryview points to a
+    new bytes object."""
+    raise NotImplementedError
+
+ at cpython_api([PyObject], rffi.INT_real, error=CANNOT_FAIL)
+def PyMemoryView_Check(space, obj):
+    """Return true if the object obj is a memoryview object.  It is not
+    currently allowed to create subclasses of memoryview."""
+    raise NotImplementedError
+
+ at cpython_api([PyObject], Py_buffer)
+def PyMemoryView_GET_BUFFER(space, obj):
+    """Return a pointer to the buffer-info structure wrapped by the given
+    object.  The object must be a memoryview instance; this macro doesn't
+    check its type, you must do it yourself or you will risk crashes."""
+    raise NotImplementedError
+
 @cpython_api([PyObject], rffi.INT_real, error=CANNOT_FAIL)
 def PyByteArray_Check(space, o):
     """Return true if the object o is a bytearray object or an instance of a
@@ -253,12 +267,12 @@
     """Concat bytearrays a and b and return a new bytearray with the result."""
     raise NotImplementedError
 
- at cpython_api([PyObject], Py_ssize_t)
+ at cpython_api([PyObject], Py_ssize_t, error=-1)
 def PyByteArray_Size(space, bytearray):
     """Return the size of bytearray after checking for a NULL pointer."""
     raise NotImplementedError
 
- at cpython_api([PyObject], rffi.CCHARP, error=lltype.nullptr(rffi.CCHARP.TO))
+ at cpython_api([PyObject], rffi.CCHARP)
 def PyByteArray_AsString(space, bytearray):
     """Return the contents of bytearray as a char array after checking for a
     NULL pointer."""
@@ -269,7 +283,7 @@
     """Resize the internal buffer of bytearray to len."""
     raise NotImplementedError
 
- at cpython_api([PyObject], rffi.CCHARP, error=CANNOT_FAIL)
+ at cpython_api([PyObject], rffi.CCHARP)
 def PyByteArray_AS_STRING(space, bytearray):
     """Macro version of PyByteArray_AsString()."""
     raise NotImplementedError
@@ -280,7 +294,7 @@
     raise NotImplementedError
 
 @cpython_api([PyObject], rffi.INT_real, error=CANNOT_FAIL)
-def PyCell_Check(space, cell):
+def PyCell_Check(space, ob):
     """Return true if ob is a cell object; ob must not be NULL."""
     raise NotImplementedError
 
@@ -333,12 +347,12 @@
     """Return true if co is a code object"""
     raise NotImplementedError
 
- at cpython_api([PyObject], rffi.INT_real, error=-1)
+ at cpython_api([PyObject], rffi.INT_real, error=CANNOT_FAIL)
 def PyCode_GetNumFree(space, co):
     """Return the number of free variables in co."""
     raise NotImplementedError
 
- at cpython_api([rffi.INT_real, rffi.INT_real, rffi.INT_real, rffi.INT_real, PyObject, PyObject, PyObject, PyObject, PyObject, PyObject, PyObject, PyObject, rffi.INT_real, PyObject], PyObject)
+ at cpython_api([rffi.INT_real, rffi.INT_real, rffi.INT_real, rffi.INT_real, PyObject, PyObject, PyObject, PyObject, PyObject, PyObject, PyObject, PyObject, rffi.INT_real, PyObject], PyCodeObject)
 def PyCode_New(space, argcount, nlocals, stacksize, flags, code, consts, names, varnames, freevars, cellvars, filename, name, firstlineno, lnotab):
     """Return a new code object.  If you need a dummy code object to
     create a frame, use PyCode_NewEmpty() instead.  Calling
@@ -346,6 +360,107 @@
     version since the definition of the bytecode changes often."""
     raise NotImplementedError
 
+ at cpython_api([PyObject], rffi.INT_real, error=-1)
+def PyCodec_Register(space, search_function):
+    """Register a new codec search function.
+    
+    As side effect, this tries to load the encodings package, if not yet
+    done, to make sure that it is always first in the list of search functions."""
+    raise NotImplementedError
+
+ at cpython_api([PyObject, rffi.CCHARP, rffi.CCHARP], PyObject)
+def PyCodec_Encode(space, object, encoding, errors):
+    """Generic codec based encoding API.
+    
+    object is passed through the encoder function found for the given
+    encoding using the error handling method defined by errors.  errors may
+    be NULL to use the default method defined for the codec.  Raises a
+    LookupError if no encoder can be found."""
+    raise NotImplementedError
+
+ at cpython_api([PyObject, rffi.CCHARP, rffi.CCHARP], PyObject)
+def PyCodec_Decode(space, object, encoding, errors):
+    """Generic codec based decoding API.
+    
+    object is passed through the decoder function found for the given
+    encoding using the error handling method defined by errors.  errors may
+    be NULL to use the default method defined for the codec.  Raises a
+    LookupError if no encoder can be found."""
+    raise NotImplementedError
+
+ at cpython_api([rffi.CCHARP], PyObject)
+def PyCodec_Encoder(space, encoding):
+    """Get an encoder function for the given encoding."""
+    raise NotImplementedError
+
+ at cpython_api([rffi.CCHARP], PyObject)
+def PyCodec_Decoder(space, encoding):
+    """Get a decoder function for the given encoding."""
+    raise NotImplementedError
+
+ at cpython_api([rffi.CCHARP, PyObject, rffi.CCHARP], PyObject)
+def PyCodec_StreamReader(space, encoding, stream, errors):
+    """Get a StreamReader factory function for the given encoding."""
+    raise NotImplementedError
+
+ at cpython_api([rffi.CCHARP, PyObject, rffi.CCHARP], PyObject)
+def PyCodec_StreamWriter(space, encoding, stream, errors):
+    """Get a StreamWriter factory function for the given encoding."""
+    raise NotImplementedError
+
+ at cpython_api([rffi.CCHARP, PyObject], rffi.INT_real, error=-1)
+def PyCodec_RegisterError(space, name, error):
+    """Register the error handling callback function error under the given name.
+    This callback function will be called by a codec when it encounters
+    unencodable characters/undecodable bytes and name is specified as the error
+    parameter in the call to the encode/decode function.
+    
+    The callback gets a single argument, an instance of
+    UnicodeEncodeError, UnicodeDecodeError or
+    UnicodeTranslateError that holds information about the problematic
+    sequence of characters or bytes and their offset in the original string (see
+    unicodeexceptions for functions to extract this information).  The
+    callback must either raise the given exception, or return a two-item tuple
+    containing the replacement for the problematic sequence, and an integer
+    giving the offset in the original string at which encoding/decoding should be
+    resumed.
+    
+    Return 0 on success, -1 on error."""
+    raise NotImplementedError
+
+ at cpython_api([rffi.CCHARP], PyObject)
+def PyCodec_LookupError(space, name):
+    """Lookup the error handling callback function registered under name.  As a
+    special case NULL can be passed, in which case the error handling callback
+    for "strict" will be returned."""
+    raise NotImplementedError
+
+ at cpython_api([PyObject], PyObject)
+def PyCodec_StrictErrors(space, exc):
+    """Raise exc as an exception."""
+    raise NotImplementedError
+
+ at cpython_api([PyObject], PyObject)
+def PyCodec_IgnoreErrors(space, exc):
+    """Ignore the unicode error, skipping the faulty input."""
+    raise NotImplementedError
+
+ at cpython_api([PyObject], PyObject)
+def PyCodec_ReplaceErrors(space, exc):
+    """Replace the unicode encode error with ? or U+FFFD."""
+    raise NotImplementedError
+
+ at cpython_api([PyObject], PyObject)
+def PyCodec_XMLCharRefReplaceErrors(space, exc):
+    """Replace the unicode encode error with XML character references."""
+    raise NotImplementedError
+
+ at cpython_api([PyObject], PyObject)
+def PyCodec_BackslashReplaceErrors(space, exc):
+    r"""Replace the unicode encode error with backslash escapes (\x, \u and
+    \U)."""
+    raise NotImplementedError
+
 @cpython_api([Py_complex, Py_complex], Py_complex)
 def _Py_c_sum(space, left, right):
     """Return the sum of two complex numbers, using the C Py_complex
@@ -431,12 +546,10 @@
     
     See the Unix man page strtod(2) for details.
     
-    
-    
     Use PyOS_string_to_double() instead."""
     raise NotImplementedError
 
- at cpython_api([rffi.CCHARP, rffi.SIZE_T, rffi.CCHARP, rffi.DOUBLE], rffi.CCHARP, error=lltype.nullptr(rffi.CCHARP.TO))
+ at cpython_api([rffi.CCHARP, rffi.SIZE_T, rffi.CCHARP, rffi.DOUBLE], rffi.CCHARP)
 def PyOS_ascii_formatd(space, buffer, buf_len, format, d):
     """Convert a double to a string using the '.' as the decimal
     separator. format is a printf()-style format string specifying the
@@ -446,13 +559,11 @@
     The return value is a pointer to buffer with the converted string or NULL if
     the conversion failed.
     
-    
-    
     This function is removed in Python 2.7 and 3.1.  Use PyOS_double_to_string()
     instead."""
     raise NotImplementedError
 
- at cpython_api([rffi.DOUBLE, lltype.Char, rffi.INT_real, rffi.INT_real, rffi.INTP], rffi.CCHARP, error=lltype.nullptr(rffi.CCHARP.TO))
+ at cpython_api([rffi.DOUBLE, lltype.Char, rffi.INT_real, rffi.INT_real, rffi.INTP], rffi.CCHARP)
 def PyOS_double_to_string(space, val, format_code, precision, flags, ptype):
     """Convert a double val to a string using supplied
     format_code, precision, and flags.
@@ -491,19 +602,17 @@
     
     See the Unix man page atof(2) for details.
     
-    
-    
     Use PyOS_string_to_double() instead."""
     raise NotImplementedError
 
- at cpython_api([rffi.CCHARP, rffi.CCHARP], rffi.CCHARP, error=CANNOT_FAIL)
+ at cpython_api([rffi.CCHARP, rffi.CCHARP], rffi.CCHARP)
 def PyOS_stricmp(space, s1, s2):
     """Case insensitive comparison of strings. The function works almost
     identically to strcmp() except that it ignores the case.
     """
     raise NotImplementedError
 
- at cpython_api([rffi.CCHARP, rffi.CCHARP, Py_ssize_t], rffi.CCHARP, error=CANNOT_FAIL)
+ at cpython_api([rffi.CCHARP, rffi.CCHARP, Py_ssize_t], rffi.CCHARP)
 def PyOS_strnicmp(space, s1, s2, size):
     """Case insensitive comparison of strings. The function works almost
     identically to strncmp() except that it ignores the case.
@@ -536,7 +645,7 @@
 def PyDescr_NewMethod(space, type, meth):
     raise NotImplementedError
 
- at cpython_api([PyTypeObjectPtr, wrapperbase, rffi.VOIDP_real], PyObject)
+ at cpython_api([PyTypeObjectPtr, wrapperbase, rffi.VOIDP], PyObject)
 def PyDescr_NewWrapper(space, type, wrapper, wrapped):
     raise NotImplementedError
 
@@ -553,7 +662,7 @@
     raise NotImplementedError
 
 @cpython_api([PyObject, PyObject], PyObject)
-def PyWrapper_New(space, a, b):
+def PyWrapper_New(space, w_d, w_self):
     raise NotImplementedError
 
 @cpython_api([PyObject], PyObject)
@@ -609,7 +718,6 @@
     """Similar to PyErr_SetFromWindowsErr(), with an additional parameter
     specifying the exception type to be raised. Availability: Windows.
     
-    
     Return value: always NULL."""
     raise NotImplementedError
 
@@ -626,7 +734,6 @@
     """Similar to PyErr_SetFromWindowsErrWithFilename(), with an additional
     parameter specifying the exception type to be raised. Availability: Windows.
     
-    
     Return value: always NULL."""
     raise NotImplementedError
 
@@ -640,18 +747,10 @@
     raise NotImplementedError
 
 @cpython_api([], lltype.Void)
-def PyErr_SetInterrupt(space, ):
-    """
-    
-    
-    
-    This function simulates the effect of a SIGINT signal arriving --- the
-    next time PyErr_CheckSignals() is called,  KeyboardInterrupt will
-    be raised.  It may be called without holding the interpreter lock.
-    
-    % XXX This was described as obsolete, but is used in
-    
-    % thread.interrupt_main() (used from IDLE), so it's still needed."""
+def PyErr_SetInterrupt(space):
+    """This function simulates the effect of a SIGINT signal arriving --- the
+    next time PyErr_CheckSignals() is called, KeyboardInterrupt will be raised.
+    It may be called without holding the interpreter lock."""
     raise NotImplementedError
 
 @cpython_api([rffi.INT_real], rffi.INT_real, error=CANNOT_FAIL)
@@ -676,7 +775,72 @@
     the warning message."""
     raise NotImplementedError
 
- at cpython_api([rffi.CCHARP], rffi.INT_real, error=-1)
+ at cpython_api([rffi.CCHARP, rffi.CCHARP, Py_ssize_t, Py_ssize_t, Py_ssize_t, rffi.CCHARP], PyObject)
+def PyUnicodeDecodeError_Create(space, encoding, object, length, start, end, reason):
+    """Create a UnicodeDecodeError object with the attributes encoding,
+    object, length, start, end and reason."""
+    raise NotImplementedError
+
+ at cpython_api([rffi.CCHARP, rffi.CWCHARP, Py_ssize_t, Py_ssize_t, Py_ssize_t, rffi.CCHARP], PyObject)
+def PyUnicodeEncodeError_Create(space, encoding, object, length, start, end, reason):
+    """Create a UnicodeEncodeError object with the attributes encoding,
+    object, length, start, end and reason."""
+    raise NotImplementedError
+
+ at cpython_api([rffi.CWCHARP, Py_ssize_t, Py_ssize_t, Py_ssize_t, rffi.CCHARP], PyObject)
+def PyUnicodeTranslateError_Create(space, object, length, start, end, reason):
+    """Create a UnicodeTranslateError object with the attributes object,
+    length, start, end and reason."""
+    raise NotImplementedError
+
+ at cpython_api([PyObject], PyObject)
+def PyUnicodeDecodeError_GetEncoding(space, exc):
+    """Return the encoding attribute of the given exception object."""
+    raise NotImplementedError
+
+ at cpython_api([PyObject], PyObject)
+def PyUnicodeDecodeError_GetObject(space, exc):
+    """Return the object attribute of the given exception object."""
+    raise NotImplementedError
+
+ at cpython_api([PyObject, Py_ssize_t], rffi.INT_real, error=-1)
+def PyUnicodeDecodeError_GetStart(space, exc, start):
+    """Get the start attribute of the given exception object and place it into
+    *start.  start must not be NULL.  Return 0 on success, -1 on
+    failure."""
+    raise NotImplementedError
+
+ at cpython_api([PyObject, Py_ssize_t], rffi.INT_real, error=-1)
+def PyUnicodeDecodeError_SetStart(space, exc, start):
+    """Set the start attribute of the given exception object to start.  Return
+    0 on success, -1 on failure."""
+    raise NotImplementedError
+
+ at cpython_api([PyObject, Py_ssize_t], rffi.INT_real, error=-1)
+def PyUnicodeDecodeError_GetEnd(space, exc, end):
+    """Get the end attribute of the given exception object and place it into
+    *end.  end must not be NULL.  Return 0 on success, -1 on
+    failure."""
+    raise NotImplementedError
+
+ at cpython_api([PyObject, Py_ssize_t], rffi.INT_real, error=-1)
+def PyUnicodeDecodeError_SetEnd(space, exc, end):
+    """Set the end attribute of the given exception object to end.  Return 0
+    on success, -1 on failure."""
+    raise NotImplementedError
+
+ at cpython_api([PyObject], PyObject)
+def PyUnicodeDecodeError_GetReason(space, exc):
+    """Return the reason attribute of the given exception object."""
+    raise NotImplementedError
+
+ at cpython_api([PyObject, rffi.CCHARP], rffi.INT_real, error=-1)
+def PyUnicodeDecodeError_SetReason(space, exc, reason):
+    """Set the reason attribute of the given exception object to reason.  Return
+    0 on success, -1 on failure."""
+    raise NotImplementedError
+
+ at cpython_api([rffi.CCHARP], rffi.INT_real, error=1)
 def Py_EnterRecursiveCall(space, where):
     """Marks a point where a recursive C-level call is about to be performed.
     
@@ -694,7 +858,7 @@
     raise NotImplementedError
 
 @cpython_api([], lltype.Void)
-def Py_LeaveRecursiveCall(space, ):
+def Py_LeaveRecursiveCall(space):
     """Ends a Py_EnterRecursiveCall().  Must be called once for each
     successful invocation of Py_EnterRecursiveCall()."""
     raise NotImplementedError
@@ -717,8 +881,17 @@
     
     The GIL must be held while calling this function.
     
-    The suggested use is to call this after PyFile_AsFile() just before
-    you release the GIL.
+    The suggested use is to call this after PyFile_AsFile() and before
+    you release the GIL:
+    
+    FILE *fp = PyFile_AsFile(p);
+    PyFile_IncUseCount(p);
+    /* ... */
+    Py_BEGIN_ALLOW_THREADS
+    do_something(fp);
+    Py_END_ALLOW_THREADS
+    /* ... */
+    PyFile_DecUseCount(p);
     """
     raise NotImplementedError
 
@@ -728,7 +901,8 @@
     indicate that the caller is done with its own use of the FILE*.
     This may only be called to undo a prior call to PyFile_IncUseCount().
     
-    The GIL must be held while calling this function.
+    The GIL must be held while calling this function (see the example
+    above).
     """
     raise NotImplementedError
 
@@ -740,8 +914,7 @@
 
 @cpython_api([PyFileObject, rffi.INT_real], lltype.Void)
 def PyFile_SetBufSize(space, p, n):
-    """
-    Available on systems with setvbuf() only.  This should only be called
+    """Available on systems with setvbuf() only.  This should only be called
     immediately after file object creation."""
     raise NotImplementedError
 
@@ -787,8 +960,8 @@
     failure; the appropriate exception will be set."""
     raise NotImplementedError
 
- at cpython_api([rffi.VOIDP_real], PyObject)
-def PyFloat_GetInfo(space, info):
+ at cpython_api([], PyObject)
+def PyFloat_GetInfo(space):
     """Return a structseq instance which contains information about the
     precision, minimum and maximum values of a float. It's a thin wrapper
     around the header file float.h.
@@ -796,25 +969,25 @@
     raise NotImplementedError
 
 @cpython_api([], rffi.DOUBLE, error=CANNOT_FAIL)
-def PyFloat_GetMax(space, ):
+def PyFloat_GetMax(space):
     """Return the maximum representable finite float DBL_MAX as C double.
     """
     raise NotImplementedError
 
 @cpython_api([], rffi.DOUBLE, error=CANNOT_FAIL)
-def PyFloat_GetMin(space, ):
+def PyFloat_GetMin(space):
     """Return the minimum normalized positive float DBL_MIN as C double.
     """
     raise NotImplementedError
 
 @cpython_api([], rffi.INT_real, error=CANNOT_FAIL)
-def PyFloat_ClearFreeList(space, ):
+def PyFloat_ClearFreeList(space):
     """Clear the float free list. Return the number of items that could not
     be freed.
     """
     raise NotImplementedError
 
- at cpython_api([rffi.CCHARP, PyObject], lltype.Void)
+ at cpython_api([rffi.CCHARP, PyFloatObject], lltype.Void)
 def PyFloat_AsString(space, buf, v):
     """Convert the argument v to a string, using the same rules as
     str(). The length of buf should be at least 100.
@@ -825,7 +998,7 @@
     Use PyObject_Str() or PyOS_double_to_string() instead."""
     raise NotImplementedError
 
- at cpython_api([rffi.CCHARP, PyObject], lltype.Void)
+ at cpython_api([rffi.CCHARP, PyFloatObject], lltype.Void)
 def PyFloat_AsReprString(space, buf, v):
     """Same as PyFloat_AsString, except uses the same rules as
     repr().  The length of buf should be at least 100.
@@ -896,7 +1069,7 @@
     raise NotImplementedError
 
 @cpython_api([PyTypeObjectPtr, Py_ssize_t], PyObject)
-def _PyObject_GC_NewVar(space, type, size):
+def PyObject_GC_NewVar(space, type, size):
     """Analogous to PyObject_NewVar() but for container objects with the
     Py_TPFLAGS_HAVE_GC flag set.
     
@@ -905,7 +1078,7 @@
     raise NotImplementedError
 
 @cpython_api([PyObject, Py_ssize_t], PyObject)
-def _PyObject_GC_Resize(space, op, newsize):
+def PyObject_GC_Resize(space, op, newsize):
     """Resize an object allocated by PyObject_NewVar().  Returns the
     resized object or NULL on failure.
     
@@ -926,12 +1099,12 @@
     raise NotImplementedError
 
 @cpython_api([PyObject], rffi.INT_real, error=CANNOT_FAIL)
-def PyGen_Check(space, gen):
+def PyGen_Check(space, ob):
     """Return true if ob is a generator object; ob must not be NULL."""
     raise NotImplementedError
 
 @cpython_api([PyObject], rffi.INT_real, error=CANNOT_FAIL)
-def PyGen_CheckExact(space, gen):
+def PyGen_CheckExact(space, ob):
     """Return true if ob's type is PyGen_Type is a generator object; ob must not
     be NULL."""
     raise NotImplementedError
@@ -945,12 +1118,8 @@
 
 @cpython_api([rffi.CCHARP, PyObject, PyObject, PyObject], PyObject)
 def PyImport_ImportModuleEx(space, name, globals, locals, fromlist):
-    """
-    
-    
-    
-    Import a module.  This is best described by referring to the built-in Python
-    function __import__(), as the standard __import__() function calls
+    """Import a module.  This is best described by referring to the built-in
+    Python function __import__(), as the standard __import__() function calls
     this function directly.
     
     The return value is a new reference to the imported module or top-level package,
@@ -980,12 +1149,8 @@
 
 @cpython_api([PyObject], PyObject)
 def PyImport_ReloadModule(space, m):
-    """
-    
-    
-    
-    Reload a module.  This is best described by referring to the built-in Python
-    function reload(), as the standard reload() function calls this
+    """Reload a module.  This is best described by referring to the built-in
+    Python function reload(), as the standard reload() function calls this
     function directly.  Return a new reference to the reloaded module, or NULL
     with an exception set on failure (the module still exists in this case)."""
     raise NotImplementedError
@@ -1006,21 +1171,20 @@
 
 @cpython_api([rffi.CCHARP, PyObject], PyObject)
 def PyImport_ExecCodeModule(space, name, co):
-    """
-    
-    
-    
-    Given a module name (possibly of the form package.module) and a code object
-    read from a Python bytecode file or obtained from the built-in function
-    compile(), load the module.  Return a new reference to the module object,
-    or NULL with an exception set if an error occurred.  Before Python 2.4, the
-    module could still be created in error cases.  Starting with Python 2.4, name
-    is removed from sys.modules in error cases, and even if name was already
-    in sys.modules on entry to PyImport_ExecCodeModule().  Leaving
+    """Given a module name (possibly of the form package.module) and a code
+    object read from a Python bytecode file or obtained from the built-in
+    function compile(), load the module.  Return a new reference to the module
+    object, or NULL with an exception set if an error occurred.  Before Python
+    2.4, the module could still be created in error cases.  Starting with Python
+    2.4, name is removed from sys.modules in error cases, and even if name was
+    already in sys.modules on entry to PyImport_ExecCodeModule().  Leaving
     incompletely initialized modules in sys.modules is dangerous, as imports of
     such modules have no way to know that the module object is an unknown (and
     probably damaged with respect to the module author's intents) state.
     
+    The module's __file__ attribute will be set to the code object's
+    co_filename.
+    
     This function will reload the module if it was already imported.  See
     PyImport_ReloadModule() for the intended way to reload a module.
     
@@ -1030,15 +1194,21 @@
     name is removed from sys.modules in error cases."""
     raise NotImplementedError
 
+ at cpython_api([rffi.CCHARP, PyObject, rffi.CCHARP], PyObject)
+def PyImport_ExecCodeModuleEx(space, name, co, pathname):
+    """Like PyImport_ExecCodeModule(), but the __file__ attribute of
+    the module object is set to pathname if it is non-NULL."""
+    raise NotImplementedError
+
 @cpython_api([], lltype.Signed, error=CANNOT_FAIL)
-def PyImport_GetMagicNumber(space, ):
+def PyImport_GetMagicNumber(space):
     """Return the magic number for Python bytecode files (a.k.a. .pyc and
     .pyo files).  The magic number should be present in the first four bytes
     of the bytecode file, in little-endian byte order."""
     raise NotImplementedError
 
 @cpython_api([], PyObject)
-def PyImport_GetModuleDict(space, ):
+def PyImport_GetModuleDict(space):
     """Return the dictionary used for the module administration (a.k.a.
     sys.modules).  Note that this is a per-interpreter variable."""
     borrow_from()
@@ -1057,20 +1227,30 @@
     raise NotImplementedError
 
 @cpython_api([], lltype.Void)
-def _PyImport_Init(space, ):
+def _PyImport_Init(space):
     """Initialize the import mechanism.  For internal use only."""
     raise NotImplementedError
 
 @cpython_api([], lltype.Void)
-def PyImport_Cleanup(space, ):
+def PyImport_Cleanup(space):
     """Empty the module table.  For internal use only."""
     raise NotImplementedError
 
 @cpython_api([], lltype.Void)
-def _PyImport_Fini(space, ):
+def _PyImport_Fini(space):
     """Finalize the import mechanism.  For internal use only."""
     raise NotImplementedError
 
+ at cpython_api([rffi.CCHARP, rffi.CCHARP], PyObject)
+def _PyImport_FindExtension(space, name, filename):
+    """For internal use only."""
+    raise NotImplementedError
+
+ at cpython_api([rffi.CCHARP, rffi.CCHARP], PyObject)
+def _PyImport_FixupExtension(space, name, filename):
+    """For internal use only."""
+    raise NotImplementedError
+
 @cpython_api([rffi.CCHARP], rffi.INT_real, error=-1)
 def PyImport_ImportFrozenModule(space, name):
     """Load a frozen module named name.  Return 1 for success, 0 if the
@@ -1101,21 +1281,17 @@
     raise NotImplementedError
 
 @cpython_api([], lltype.Void)
-def Py_Initialize(space, ):
-    """
-    
-    
-    
-    Initialize the Python interpreter.  In an application embedding  Python, this
-    should be called before using any other Python/C API functions; with the
-    exception of Py_SetProgramName(), PyEval_InitThreads(),
-    PyEval_ReleaseLock(), and PyEval_AcquireLock(). This initializes
-    the table of loaded modules (sys.modules), and creates the fundamental
-    modules __builtin__, __main__ and sys.  It also initializes
-    the module search path (sys.path). It does not set sys.argv; use
-    PySys_SetArgv() for that.  This is a no-op when called for a second time
-    (without calling Py_Finalize() first).  There is no return value; it is a
-    fatal error if the initialization fails."""
+def Py_Initialize(space):
+    """Initialize the Python interpreter.  In an application embedding Python,
+    this should be called before using any other Python/C API functions; with
+    the exception of Py_SetProgramName(), PyEval_InitThreads(),
+    PyEval_ReleaseLock(), and PyEval_AcquireLock(). This initializes the table
+    of loaded modules (sys.modules), and creates the fundamental modules
+    __builtin__, __main__ and sys.  It also initializes the module search path
+    (sys.path). It does not set sys.argv; use PySys_SetArgvEx() for that.  This
+    is a no-op when called for a second time (without calling Py_Finalize()
+    first).  There is no return value; it is a fatal error if the initialization
+    fails."""
     raise NotImplementedError
 
 @cpython_api([rffi.INT_real], lltype.Void)
@@ -1127,7 +1303,7 @@
     raise NotImplementedError
 
 @cpython_api([], lltype.Void)
-def Py_Finalize(space, ):
+def Py_Finalize(space):
     """Undo all initializations made by Py_Initialize() and subsequent use of
     Python/C API functions, and destroy all sub-interpreters (see
     Py_NewInterpreter() below) that were created and not yet destroyed since
@@ -1156,106 +1332,21 @@
     Py_Finalize() more than once."""
     raise NotImplementedError
 
- at cpython_api([], PyThreadState)
-def Py_NewInterpreter(space, ):
-    """
-    
-    
-    
-    Create a new sub-interpreter.  This is an (almost) totally separate environment
-    for the execution of Python code.  In particular, the new interpreter has
-    separate, independent versions of all imported modules, including the
-    fundamental modules __builtin__, __main__ and sys.  The
-    table of loaded modules (sys.modules) and the module search path
-    (sys.path) are also separate.  The new environment has no sys.argv
-    variable.  It has new standard I/O stream file objects sys.stdin,
-    sys.stdout and sys.stderr (however these refer to the same underlying
-    FILE structures in the C library).
-    
-    The return value points to the first thread state created in the new
-    sub-interpreter.  This thread state is made in the current thread state.
-    Note that no actual thread is created; see the discussion of thread states
-    below.  If creation of the new interpreter is unsuccessful, NULL is
-    returned; no exception is set since the exception state is stored in the
-    current thread state and there may not be a current thread state.  (Like all
-    other Python/C API functions, the global interpreter lock must be held before
-    calling this function and is still held when it returns; however, unlike most
-    other Python/C API functions, there needn't be a current thread state on
-    entry.)
-    
-    
-    
-    
-    
-    Extension modules are shared between (sub-)interpreters as follows: the first
-    time a particular extension is imported, it is initialized normally, and a
-    (shallow) copy of its module's dictionary is squirreled away.  When the same
-    extension is imported by another (sub-)interpreter, a new module is initialized
-    and filled with the contents of this copy; the extension's init function is
-    not called.  Note that this is different from what happens when an extension is
-    imported after the interpreter has been completely re-initialized by calling
-    Py_Finalize() and Py_Initialize(); in that case, the extension's
-    initmodule function is called again.
-    
-    
-    
-    
-    
-    Bugs and caveats: Because sub-interpreters (and the main interpreter) are
-    part of the same process, the insulation between them isn't perfect --- for
-    example, using low-level file operations like  os.close() they can
-    (accidentally or maliciously) affect each other's open files.  Because of the
-    way extensions are shared between (sub-)interpreters, some extensions may not
-    work properly; this is especially likely when the extension makes use of
-    (static) global variables, or when the extension manipulates its module's
-    dictionary after its initialization.  It is possible to insert objects created
-    in one sub-interpreter into a namespace of another sub-interpreter; this should
-    be done with great care to avoid sharing user-defined functions, methods,
-    instances or classes between sub-interpreters, since import operations executed
-    by such objects may affect the wrong (sub-)interpreter's dictionary of loaded
-    modules.  (XXX This is a hard-to-fix bug that will be addressed in a future
-    release.)
-    
-    Also note that the use of this functionality is incompatible with extension
-    modules such as PyObjC and ctypes that use the PyGILState_*() APIs (and
-    this is inherent in the way the PyGILState_*() functions work).  Simple
-    things may work, but confusing behavior will always be near."""
-    raise NotImplementedError
-
- at cpython_api([PyThreadState], lltype.Void)
-def Py_EndInterpreter(space, tstate):
-    """
-    
-    
-    
-    Destroy the (sub-)interpreter represented by the given thread state. The given
-    thread state must be the current thread state.  See the discussion of thread
-    states below.  When the call returns, the current thread state is NULL.  All
-    thread states associated with this interpreter are destroyed.  (The global
-    interpreter lock must be held before calling this function and is still held
-    when it returns.)  Py_Finalize() will destroy all sub-interpreters that
-    haven't been explicitly destroyed at that point."""
-    raise NotImplementedError
-
 @cpython_api([rffi.CCHARP], lltype.Void)
 def Py_SetProgramName(space, name):
-    """
-    
-    
-    
-    This function should be called before Py_Initialize() is called for
-    the first time, if it is called at all.  It tells the interpreter the value
-    of the argv[0] argument to the main() function of the program.
-    This is used by Py_GetPath() and some other functions below to find
-    the Python run-time libraries relative to the interpreter executable.  The
-    default value is 'python'.  The argument should point to a
-    zero-terminated character string in static storage whose contents will not
-    change for the duration of the program's execution.  No code in the Python
-    interpreter will change the contents of this storage."""
-    raise NotImplementedError
-
- at cpython_api([], rffi.CCHARP, error=CANNOT_FAIL)
-def Py_GetPrefix(space, ):
+    """This function should be called before Py_Initialize() is called for the
+    first time, if it is called at all.  It tells the interpreter the value of
+    the argv[0] argument to the main() function of the program.  This is used by
+    Py_GetPath() and some other functions below to find the Python run-time
+    libraries relative to the interpreter executable.  The default value is
+    'python'.  The argument should point to a zero-terminated character string
+    in static storage whose contents will not change for the duration of the
+    program's execution.  No code in the Python interpreter will change the
+    contents of this storage."""
+    raise NotImplementedError
+
+ at cpython_api([], rffi.CCHARP)
+def Py_GetPrefix(space):
     """Return the prefix for installed platform-independent files. This is derived
     through a number of complicated rules from the program name set with
     Py_SetProgramName() and some environment variables; for example, if the
@@ -1267,8 +1358,8 @@
     It is only useful on Unix.  See also the next function."""
     raise NotImplementedError
 
- at cpython_api([], rffi.CCHARP, error=CANNOT_FAIL)
-def Py_GetExecPrefix(space, ):
+ at cpython_api([], rffi.CCHARP)
+def Py_GetExecPrefix(space):
     """Return the exec-prefix for installed platform-dependent files.  This is
     derived through a number of complicated rules from the program name set with
     Py_SetProgramName() and some environment variables; for example, if the
@@ -1302,63 +1393,46 @@
     platform."""
     raise NotImplementedError
 
- at cpython_api([], rffi.CCHARP, error=CANNOT_FAIL)
-def Py_GetProgramFullPath(space, ):
-    """
-    
-    
-    
-    Return the full program name of the Python executable; this is  computed as a
-    side-effect of deriving the default module search path  from the program name
-    (set by Py_SetProgramName() above). The returned string points into
-    static storage; the caller should not modify its value.  The value is available
-    to Python code as sys.executable."""
-    raise NotImplementedError
-
- at cpython_api([], rffi.CCHARP, error=CANNOT_FAIL)
-def Py_GetPath(space, ):
-    """
-    
-    
-    
-    Return the default module search path; this is computed from the program name
-    (set by Py_SetProgramName() above) and some environment variables.
-    The returned string consists of a series of directory names separated by a
-    platform dependent delimiter character.  The delimiter character is ':'
-    on Unix and Mac OS X, ';' on Windows.  The returned string points into
-    static storage; the caller should not modify its value.  The list
-    sys.path is initialized with this value on interpreter startup; it
-    can be (and usually is) modified later to change the search path for loading
-    modules.
+ at cpython_api([], rffi.CCHARP)
+def Py_GetProgramFullPath(space):
+    """Return the full program name of the Python executable; this is computed
+    as a side-effect of deriving the default module search path from the program
+    name (set by Py_SetProgramName() above). The returned string points into
+    static storage; the caller should not modify its value.  The value is
+    available to Python code as sys.executable."""
+    raise NotImplementedError
+
+ at cpython_api([], rffi.CCHARP)
+def Py_GetPath(space):
+    """Return the default module search path; this is computed from the program
+    name (set by Py_SetProgramName() above) and some environment variables.  The
+    returned string consists of a series of directory names separated by a
+    platform dependent delimiter character.  The delimiter character is ':' on
+    Unix and Mac OS X, ';' on Windows.  The returned string points into static
+    storage; the caller should not modify its value.  The list sys.path is
+    initialized with this value on interpreter startup; it can be (and usually
+    is) modified later to change the search path for loading modules.
     
     XXX should give the exact rules"""
     raise NotImplementedError
 
- at cpython_api([], rffi.CCHARP, error=CANNOT_FAIL)
-def Py_GetVersion(space, ):
+ at cpython_api([], rffi.CCHARP)
+def Py_GetVersion(space):
     """Return the version of this Python interpreter.  This is a string that looks
     something like
     
     "1.5 (\#67, Dec 31 1997, 22:34:28) [GCC 2.7.2.2]"
     
-    
-    
-    
-    
     The first word (up to the first space character) is the current Python version;
     the first three characters are the major and minor version separated by a
     period.  The returned string points into static storage; the caller should not
     modify its value.  The value is available to Python code as sys.version."""
     raise NotImplementedError
 
- at cpython_api([], rffi.CCHARP, error=CANNOT_FAIL)
-def Py_GetPlatform(space, ):
-    """
-    
-    
-    
-    Return the platform identifier for the current platform.  On Unix, this is
-    formed from the "official" name of the operating system, converted to lower
+ at cpython_api([], rffi.CCHARP)
+def Py_GetPlatform(space):
+    """Return the platform identifier for the current platform.  On Unix, this
+    is formed from the"official" name of the operating system, converted to lower
     case, followed by the major revision number; e.g., for Solaris 2.x, which is
     also known as SunOS 5.x, the value is 'sunos5'.  On Mac OS X, it is
     'darwin'.  On Windows, it is 'win'.  The returned string points into
@@ -1366,114 +1440,111 @@
     to Python code as sys.platform."""
     raise NotImplementedError
 
- at cpython_api([], rffi.CCHARP, error=CANNOT_FAIL)
-def Py_GetCopyright(space, ):
+ at cpython_api([], rffi.CCHARP)
+def Py_GetCopyright(space):
     """Return the official copyright string for the current Python version, for example
     
     'Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam'
     
-    
-    
-    
-    
     The returned string points into static storage; the caller should not modify its
     value.  The value is available to Python code as sys.copyright."""
     raise NotImplementedError
 
- at cpython_api([], rffi.CCHARP, error=CANNOT_FAIL)
-def Py_GetCompiler(space, ):
+ at cpython_api([], rffi.CCHARP)
+def Py_GetCompiler(space):
     """Return an indication of the compiler used to build the current Python version,
     in square brackets, for example:
     
     "[GCC 2.7.2.2]"
     
-    
-    
-    
-    
     The returned string points into static storage; the caller should not modify its
     value.  The value is available to Python code as part of the variable
     sys.version."""
     raise NotImplementedError
 
- at cpython_api([], rffi.CCHARP, error=CANNOT_FAIL)
-def Py_GetBuildInfo(space, ):
+ at cpython_api([], rffi.CCHARP)
+def Py_GetBuildInfo(space):
     """Return information about the sequence number and build date and time  of the
     current Python interpreter instance, for example
     
     "\#67, Aug  1 1997, 22:34:28"
     
-    
-    
-    
-    
     The returned string points into static storage; the caller should not modify its
     value.  The value is available to Python code as part of the variable
     sys.version."""
     raise NotImplementedError
 
- at cpython_api([rffi.INT_real, rffi.CCHARPP], lltype.Void)
-def PySys_SetArgv(space, argc, argv):
-    """
+ at cpython_api([rffi.INT_real, rffi.CCHARPP, rffi.INT_real], lltype.Void)
+def PySys_SetArgvEx(space, argc, argv, updatepath):
+    """Set sys.argv based on argc and argv.  These parameters are similar to
+    those passed to the program's main() function with the difference that the
+    first entry should refer to the script file to be executed rather than the
+    executable hosting the Python interpreter.  If there isn't a script that
+    will be run, the first entry in argv can be an empty string.  If this
+    function fails to initialize sys.argv, a fatal condition is signalled using
+    Py_FatalError().
     
+    If updatepath is zero, this is all the function does.  If updatepath
+    is non-zero, the function also modifies sys.path according to the
+    following algorithm:
     
+    If the name of an existing script is passed in argv[0], the absolute
+    path of the directory where the script is located is prepended to
+    sys.path.
     
-    Set sys.argv based on argc and argv.  These parameters are
-    similar to those passed to the program's main() function with the
-    difference that the first entry should refer to the script file to be
-    executed rather than the executable hosting the Python interpreter.  If there
-    isn't a script that will be run, the first entry in argv can be an empty
-    string.  If this function fails to initialize sys.argv, a fatal
-    condition is signalled using Py_FatalError().
+    Otherwise (that is, if argc is 0 or argv[0] doesn't point
+    to an existing file name), an empty string is prepended to
+    sys.path, which is the same as prepending the current working
+    directory (".").
     
-    This function also prepends the executed script's path to sys.path.
-    If no script is executed (in the case of calling python -c or just the
-    interactive interpreter), the empty string is used instead.
+    It is recommended that applications embedding the Python interpreter
+    for purposes other than executing a single script pass 0 as updatepath,
+    and update sys.path themselves if desired.
+    See CVE-2008-5983.
+    
+    On versions before 2.6.6, you can achieve the same effect by manually
+    popping the first sys.path element after having called
+    PySys_SetArgv(), for example using:
+    
+    PyRun_SimpleString("import sys; sys.path.pop(0)\n");
     
     XXX impl. doesn't seem consistent in allowing 0/NULL for the params;
     check w/ Guido."""
     raise NotImplementedError
 
+ at cpython_api([rffi.INT_real, rffi.CCHARPP], lltype.Void)
+def PySys_SetArgv(space, argc, argv):
+    """This function works like PySys_SetArgvEx() with updatepath set to 1."""
+    raise NotImplementedError
+
 @cpython_api([rffi.CCHARP], lltype.Void)
 def Py_SetPythonHome(space, home):
     """Set the default "home" directory, that is, the location of the standard
-    Python libraries.  The libraries are searched in
-    home/lib/pythonversion and home/lib/pythonversion.
+    Python libraries.  See PYTHONHOME for the meaning of the
+    argument string.
+    
     The argument should point to a zero-terminated character string in static
     storage whose contents will not change for the duration of the program's
     execution.  No code in the Python interpreter will change the contents of
     this storage."""
     raise NotImplementedError
 
- at cpython_api([], rffi.CCHARP, error=CANNOT_FAIL)
-def Py_GetPythonHome(space, ):
+ at cpython_api([], rffi.CCHARP)
+def Py_GetPythonHome(space):
     """Return the default "home", that is, the value set by a previous call to
     Py_SetPythonHome(), or the value of the PYTHONHOME
     environment variable if it is set."""
     raise NotImplementedError
 
 @cpython_api([], lltype.Void)
-def PyEval_AcquireLock(space, ):
-    """Acquire the global interpreter lock.  The lock must have been created earlier.
-    If this thread already has the lock, a deadlock ensues.  This function is not
-    available when thread support is disabled at compile time."""
-    raise NotImplementedError
-
- at cpython_api([], lltype.Void)
-def PyEval_ReleaseLock(space, ):
-    """Release the global interpreter lock.  The lock must have been created earlier.
-    This function is not available when thread support is disabled at compile time."""
-    raise NotImplementedError
-
- at cpython_api([], lltype.Void)
-def PyEval_ReInitThreads(space, ):
+def PyEval_ReInitThreads(space):
     """This function is called from PyOS_AfterFork() to ensure that newly
     created child processes don't hold locks referring to threads which
     are not running in the child process."""
     raise NotImplementedError
 
 @cpython_api([], PyInterpreterState)
-def PyInterpreterState_New(space, ):
+def PyInterpreterState_New(space):
     """Create a new interpreter state object.  The global interpreter lock need not
     be held, but may be held if it is necessary to serialize calls to this
     function."""
@@ -1493,7 +1564,7 @@
     raise NotImplementedError
 
 @cpython_api([], PyObject)
-def PyThreadState_GetDict(space, ):
+def PyThreadState_GetDict(space):
     """Return a dictionary in which extensions can store thread-specific state
     information.  Each extension should use a unique key to use to store state in
     the dictionary.  It is okay to call this function when no current thread state
@@ -1517,14 +1588,73 @@
     """
     raise NotImplementedError
 
- at cpython_api([rffi.VOIDP_real, rffi.VOIDP_real], lltype.Void)
-def Py_AddPendingCall(space, func, arg):
-    """
+ at cpython_api([], lltype.Void)
+def PyEval_AcquireLock(space):
+    """Acquire the global interpreter lock.  The lock must have been created earlier.
+    If this thread already has the lock, a deadlock ensues.
     
+    This function does not change the current thread state.  Please use
+    PyEval_RestoreThread() or PyEval_AcquireThread()
+    instead."""
+    raise NotImplementedError
+
+ at cpython_api([], lltype.Void)
+def PyEval_ReleaseLock(space):
+    """Release the global interpreter lock.  The lock must have been created earlier.
     
+    This function does not change the current thread state.  Please use
+    PyEval_SaveThread() or PyEval_ReleaseThread()
+    instead."""
+    raise NotImplementedError
+
+ at cpython_api([], PyThreadState)
+def Py_NewInterpreter(space):
+    """Create a new sub-interpreter.  This is an (almost) totally separate
+    environment for the execution of Python code.  In particular, the new
+    interpreter has separate, independent versions of all imported modules,
+    including the fundamental modules builtins, __main__ and sys.  The table of
+    loaded modules (sys.modules) and the module search path (sys.path) are also
+    separate.  The new environment has no sys.argv variable.  It has new standard
+    I/O stream file objects sys.stdin, sys.stdout and sys.stderr (however these
+    refer to the same underlying file descriptors).
     
-    Post a notification to the Python main thread.  If successful, func will be
-    called with the argument arg at the earliest convenience.  func will be
+    The return value points to the first thread state created in the new
+    sub-interpreter.  This thread state is made in the current thread state.
+    Note that no actual thread is created; see the discussion of thread states
+    below.  If creation of the new interpreter is unsuccessful, NULL is
+    returned; no exception is set since the exception state is stored in the
+    current thread state and there may not be a current thread state.  (Like all
+    other Python/C API functions, the global interpreter lock must be held before
+    calling this function and is still held when it returns; however, unlike most
+    other Python/C API functions, there needn't be a current thread state on
+    entry.)
+    
+    Extension modules are shared between (sub-)interpreters as follows: the first
+    time a particular extension is imported, it is initialized normally, and a
+    (shallow) copy of its module's dictionary is squirreled away.  When the same
+    extension is imported by another (sub-)interpreter, a new module is initialized
+    and filled with the contents of this copy; the extension's init function is
+    not called.  Note that this is different from what happens when an extension is
+    imported after the interpreter has been completely re-initialized by calling
+    Py_Finalize() and Py_Initialize(); in that case, the extension's
+    initmodule function is called again."""
+    raise NotImplementedError
+
+ at cpython_api([PyThreadState], lltype.Void)
+def Py_EndInterpreter(space, tstate):
+    """Destroy the (sub-)interpreter represented by the given thread state. The
+    given thread state must be the current thread state.  See the discussion of
+    thread states below.  When the call returns, the current thread state is
+    NULL.  All thread states associated with this interpreter are destroyed.
+    (The global interpreter lock must be held before calling this function and is
+    still held when it returns.)  Py_Finalize() will destroy all sub-interpreters
+    that haven't been explicitly destroyed at that point."""
+    raise NotImplementedError
+
+ at cpython_api([rffi.VOIDP], lltype.Void)
+def Py_AddPendingCall(space, func):
+    """Post a notification to the Python main thread.  If successful, func will
+    be called with the argument arg at the earliest convenience.  func will be
     called having the global interpreter lock held and can thus use the full
     Python API and can take any action such as setting object attributes to
     signal IO completion.  It must return 0 on success, or -1 signalling an
@@ -1543,7 +1673,7 @@
     """
     raise NotImplementedError
 
- at cpython_api([PyObject, PyObject], lltype.Void)
+ at cpython_api([Py_tracefunc, PyObject], lltype.Void)
 def PyEval_SetProfile(space, func, obj):
     """Set the profiler function to func.  The obj parameter is passed to the
     function as its first parameter, and may be any Python object, or NULL.  If
@@ -1553,7 +1683,7 @@
     events."""
     raise NotImplementedError
 
- at cpython_api([PyObject, PyObject], lltype.Void)
+ at cpython_api([Py_tracefunc, PyObject], lltype.Void)
 def PyEval_SetTrace(space, func, obj):
     """Set the tracing function to func.  This is similar to
     PyEval_SetProfile(), except the tracing function does receive line-number
@@ -1565,10 +1695,6 @@
     """Return a tuple of function call counts.  There are constants defined for the
     positions within the tuple:
     
-    
-    
-    
-    
     Name
     
     Value
@@ -1649,28 +1775,35 @@
     """
     raise NotImplementedError
 
+ at cpython_api([PyObject], rffi.ULONGLONG, error=-1)
+def PyInt_AsUnsignedLongLongMask(space, io):
+    """Will first attempt to cast the object to a PyIntObject or
+    PyLongObject, if it is not already one, and then return its value as
+    unsigned long long, without checking for overflow.
+    """
+    raise NotImplementedError
+
 @cpython_api([], lltype.Signed, error=CANNOT_FAIL)
-def PyInt_GetMax(space, ):
-    """
-    Return the system's idea of the largest integer it can handle
-    (LONG_MAX, as defined in the system header files)."""
+def PyInt_GetMax(space):
+    """Return the system's idea of the largest integer it can handle (LONG_MAX,
+    as defined in the system header files)."""
     raise NotImplementedError
 
 @cpython_api([], rffi.INT_real, error=CANNOT_FAIL)
-def PyInt_ClearFreeList(space, ):
+def PyInt_ClearFreeList(space):
     """Clear the integer free list. Return the number of items that could not
     be freed.
     """
     raise NotImplementedError
 
 @cpython_api([PyObject], rffi.INT_real, error=CANNOT_FAIL)
-def PySeqIter_Check(space, seq):
+def PySeqIter_Check(space, op):
     """Return true if the type of op is PySeqIter_Type.
     """
     raise NotImplementedError
 
 @cpython_api([PyObject], rffi.INT_real, error=CANNOT_FAIL)
-def PyCallIter_Check(space, iter):
+def PyCallIter_Check(space, op):
     """Return true if the type of op is PyCallIter_Type.
     """
     raise NotImplementedError
@@ -1708,19 +1841,13 @@
     radix must be in the range [2, 36]; if it is out of range, ValueError
     will be raised.
     
-    
-    
     This function used an int for length. This might require
     changes in your code for properly supporting 64-bit systems."""
     raise NotImplementedError
 
- at cpython_api([PyObject], Py_ssize_t)
+ at cpython_api([PyObject], Py_ssize_t, error=-1)
 def PyLong_AsSsize_t(space, pylong):
-    """
-    
-    
-    
-    Return a C Py_ssize_t representation of the contents of pylong.  If
+    """Return a C Py_ssize_t representation of the contents of pylong.  If
     pylong is greater than PY_SSIZE_T_MAX, an OverflowError is raised
     and -1 will be returned.
     """
@@ -1761,14 +1888,14 @@
     version indicates the file format."""
     raise NotImplementedError
 
- at cpython_api([FILE], lltype.Signed, error=-1)
+ at cpython_api([FILE], lltype.Signed, error=CANNOT_FAIL)
 def PyMarshal_ReadLongFromFile(space, file):
     """Return a C long from the data stream in a FILE* opened
     for reading.  Only a 32-bit value can be read in using this function,
     regardless of the native size of long."""
     raise NotImplementedError
 
- at cpython_api([FILE], rffi.INT_real, error=-1)
+ at cpython_api([FILE], rffi.INT_real, error=CANNOT_FAIL)
 def PyMarshal_ReadShortFromFile(space, file):
     """Return a C short from the data stream in a FILE* opened
     for reading.  Only a 16-bit value can be read in using this function,
@@ -1806,7 +1933,7 @@
     raise NotImplementedError
 
 @cpython_api([], rffi.INT_real, error=CANNOT_FAIL)
-def PyMethod_ClearFreeList(space, ):
+def PyMethod_ClearFreeList(space):
     """Clear the free list. Return the total number of freed items.
     """
     raise NotImplementedError
@@ -1820,34 +1947,43 @@
 
 @cpython_api([rffi.CCHARP], PyObject)
 def PyModule_New(space, name):
+    """Return a new module object with the __name__ attribute set to name.  Only
+    the module's __doc__ and __name__ attributes are filled in; the caller is
+    responsible for providing a __file__ attribute."""
+    raise NotImplementedError
+
+ at cpython_api([PyObject], rffi.CCHARP)
+def PyModule_GetFilename(space, module):
+    """Return the name of the file from which module was loaded using module's
+    __file__ attribute.  If this is not defined, or if it is not a string, raise
+    SystemError and return NULL."""
+    raise NotImplementedError
+
+ at cpython_api([PyObject, rffi.INT], rffi.INT_real, error=-1)
+def PyModule_AddIntMacro(space, module, macro):
+    """Add an int constant to module. The name and the value are taken from
+    macro. For example PyModule_AddConstant(module, AF_INET) adds the int
+    constant AF_INET with the value of AF_INET to module.
+    Return -1 on error, 0 on success.
     """
-    
-    
-    
-    Return a new module object with the __name__ attribute set to name.
-    Only the module's __doc__ and __name__ attributes are filled in;
-    the caller is responsible for providing a __file__ attribute."""
-    raise NotImplementedError
-
- at cpython_api([PyObject], rffi.CCHARP, error=lltype.nullptr(rffi.CCHARP.TO))
-def PyModule_GetFilename(space, module):
+    raise NotImplementedError
+
+ at cpython_api([PyObject, rffi.CCHARP], rffi.INT_real, error=-1)
+def PyModule_AddStringMacro(space, module, macro):
+    """Add a string constant to module.
     """
-    Return the name of the file from which module was loaded using module's
-    __file__ attribute.  If this is not defined, or if it is not a string,
-    raise SystemError and return NULL."""
     raise NotImplementedError
 
 @cpython_api([PyObjectP, PyObjectP], rffi.INT_real, error=-1)
 def PyNumber_Coerce(space, p1, p2):
-    """
-    This function takes the addresses of two variables of type PyObject*.
-    If the objects pointed to by *p1 and *p2 have the same type, increment
-    their reference count and return 0 (success). If the objects can be
-    converted to a common numeric type, replace *p1 and *p2 by their
-    converted value (with 'new' reference counts), and return 0. If no
-    conversion is possible, or if some other error occurs, return -1 (failure)
-    and don't increment the reference counts.  The call PyNumber_Coerce(&o1,
-    &o2) is equivalent to the Python statement o1, o2 = coerce(o1, o2)."""
+    """This function takes the addresses of two variables of type PyObject*.  If
+    the objects pointed to by *p1 and *p2 have the same type, increment their
+    reference count and return 0 (success). If the objects can be converted to a
+    common numeric type, replace *p1 and *p2 by their converted value (with
+    'new' reference counts), and return 0. If no conversion is possible, or if
+    some other error occurs, return -1 (failure) and don't increment the
+    reference counts.  The call PyNumber_Coerce(&o1, &o2) is equivalent to the
+    Python statement o1, o2 = coerce(o1, o2)."""
     raise NotImplementedError
 
 @cpython_api([PyObjectP, PyObjectP], rffi.INT_real, error=-1)
@@ -1876,20 +2012,15 @@
 
 @cpython_api([PyObject, PyObject, rffi.INTP], rffi.INT_real, error=-1)
 def PyObject_Cmp(space, o1, o2, result):
-    """
-    Compare the values of o1 and o2 using a routine provided by o1, if one
-    exists, otherwise with a routine provided by o2.  The result of the comparison
-    is returned in result.  Returns -1 on failure.  This is the equivalent of
-    the Python statement result = cmp(o1, o2)."""
+    """Compare the values of o1 and o2 using a routine provided by o1, if one
+    exists, otherwise with a routine provided by o2.  The result of the
+    comparison is returned in result.  Returns -1 on failure.  This is the
+    equivalent of the Python statement result = cmp(o1, o2)."""
     raise NotImplementedError
 
 @cpython_api([PyObject], PyObject)
 def PyObject_Bytes(space, o):
-    """
-    
-    
-    
-    Compute a bytes representation of object o.  In 2.x, this is just a alias
+    """Compute a bytes representation of object o.  In 2.x, this is just a alias
     for PyObject_Str()."""
     raise NotImplementedError
 
@@ -1904,13 +2035,15 @@
 
 @cpython_api([PyObject], PyObject)
 def PyObject_Dir(space, o):
-    """This is equivalent to the Python expression dir(o), returning a (possibly empty) list of strings appropriate for the object argument, or NULL if there was an error.  If the argument is NULL, this is like the Python dir(),
+    """This is equivalent to the Python expression dir(o), returning a (possibly
+    empty) list of strings appropriate for the object argument, or NULL if there
+    was an error.  If the argument is NULL, this is like the Python dir(),
     returning the names of the current locals; in this case, if no execution frame
     is active then NULL is returned but PyErr_Occurred() will return false."""
     raise NotImplementedError
 
 @cpython_api([], PyFrameObject)
-def PyEval_GetFrame(space, ):
+def PyEval_GetFrame(space):
     """Return the current thread state's frame, which is NULL if no frame is
     currently executing."""
     borrow_from()
@@ -1922,18 +2055,18 @@
     raise NotImplementedError
 
 @cpython_api([], rffi.INT_real, error=CANNOT_FAIL)
-def PyEval_GetRestricted(space, ):
+def PyEval_GetRestricted(space):
     """If there is a current frame and it is executing in restricted mode, return true,
     otherwise false."""
     raise NotImplementedError
 
- at cpython_api([PyObject], rffi.CCHARP, error=CANNOT_FAIL)
+ at cpython_api([PyObject], rffi.CCHARP)
 def PyEval_GetFuncName(space, func):
     """Return the name of func if it is a function, class or instance object, else the
     name of funcs type."""
     raise NotImplementedError
 
- at cpython_api([PyObject], rffi.CCHARP, error=CANNOT_FAIL)
+ at cpython_api([PyObject], rffi.CCHARP)
 def PyEval_GetFuncDesc(space, func):
     """Return a description string, depending on the type of func.
     Return values include "()" for functions and methods, " constructor",
@@ -1978,7 +2111,7 @@
     in your code for properly supporting 64-bit systems."""
     raise NotImplementedError
 
- at cpython_api([PyObject], PyObjectP, error=lltype.nullptr(PyObjectP.TO))
+ at cpython_api([PyObject], PyObjectP)
 def PySequence_Fast_ITEMS(space, o):
     """Return the underlying array of PyObject pointers.  Assumes that o was returned
     by PySequence_Fast() and o is not NULL.
@@ -1996,8 +2129,6 @@
     PySequence_Check(o)() is true and without adjustment for negative
     indices.
     
-    
-    
     This function used an int type for i. This might require
     changes in your code for properly supporting 64-bit systems."""
     raise NotImplementedError
@@ -2054,21 +2185,17 @@
     building-up new frozensets with PySet_Add()."""
     raise NotImplementedError
 
- at cpython_api([PyObject], Py_ssize_t)
+ at cpython_api([PyObject], Py_ssize_t, error=-1)
 def PySet_Size(space, anyset):
-    """
-    
-    
-    
-    Return the length of a set or frozenset object. Equivalent to
-    len(anyset).  Raises a PyExc_SystemError if anyset is not a
-    set, frozenset, or an instance of a subtype.
+    """Return the length of a set or frozenset object. Equivalent to
+    len(anyset).  Raises a PyExc_SystemError if anyset is not a set, frozenset,
+    or an instance of a subtype.
     
     This function returned an int. This might require changes in
     your code for properly supporting 64-bit systems."""
     raise NotImplementedError
 
- at cpython_api([PyObject], Py_ssize_t)
+ at cpython_api([PyObject], Py_ssize_t, error=-1)
 def PySet_GET_SIZE(space, anyset):
     """Macro form of PySet_Size() without error checking."""
     raise NotImplementedError
@@ -2193,7 +2320,7 @@
     raise NotImplementedError
 
 @cpython_api([], lltype.Void)
-def PyOS_AfterFork(space, ):
+def PyOS_AfterFork(space):
     """Function to update some internal state after a process fork; this should be
     called in the new process if the Python interpreter will continue to be used.
     If a new executable is loaded into the new process, this function does not need
@@ -2201,7 +2328,7 @@
     raise NotImplementedError
 
 @cpython_api([], rffi.INT_real, error=CANNOT_FAIL)
-def PyOS_CheckStack(space, ):
+def PyOS_CheckStack(space):
     """Return true when the interpreter runs out of stack space.  This is a reliable
     check, but is only available when USE_STACKCHECK is defined (currently
     on Windows using the Microsoft Visual C++ compiler).  USE_STACKCHECK
@@ -2233,7 +2360,7 @@
     raise NotImplementedError
 
 @cpython_api([], lltype.Void)
-def PySys_ResetWarnOptions(space, ):
+def PySys_ResetWarnOptions(space):
     """Reset sys.warnoptions to an empty list."""
     raise NotImplementedError
 
@@ -2250,7 +2377,7 @@
     raise NotImplementedError
 
 @cpython_api([rffi.CCHARP, ], lltype.Void)
-def PySys_WriteStdout(space, format, ):
+def PySys_WriteStdout(space, format):
     """Write the output string described by format to sys.stdout.  No
     exceptions are raised, even if truncation occurs (see below).
     
@@ -2267,33 +2394,26 @@
     raise NotImplementedError
 
 @cpython_api([rffi.CCHARP, ], lltype.Void)
-def PySys_WriteStderr(space, format, ):
+def PySys_WriteStderr(space, format):
     """As above, but write to sys.stderr or stderr instead."""
     raise NotImplementedError
 
 @cpython_api([rffi.INT_real], lltype.Void)
 def Py_Exit(space, status):
-    """
-    
-    
-    
-    Exit the current process.  This calls Py_Finalize() and then calls the
+    """Exit the current process.  This calls Py_Finalize() and then calls the
     standard C library function exit(status)."""
     raise NotImplementedError
 
 @cpython_api([rffi.VOIDP], rffi.INT_real, error=-1)
 def Py_AtExit(space, func):
-    """
-    
-    
-    
-    Register a cleanup function to be called by Py_Finalize().  The cleanup
-    function will be called with no arguments and should return no value.  At most
-    32 cleanup functions can be registered.  When the registration is successful,
-    Py_AtExit() returns 0; on failure, it returns -1.  The cleanup
-    function registered last is called first. Each cleanup function will be called
-    at most once.  Since Python's internal finalization will have completed before
-    the cleanup function, no Python APIs should be called by func."""
+    """Register a cleanup function to be called by Py_Finalize().  The cleanup
+    function will be called with no arguments and should return no value.  At
+    most 32 cleanup functions can be registered.  When the registration is
+    successful, Py_AtExit() returns 0; on failure, it returns -1.  The cleanup
+    function registered last is called first. Each cleanup function will be
+    called at most once.  Since Python's internal finalization will have
+    completed before the cleanup function, no Python APIs should be called by
+    func."""
     raise NotImplementedError
 
 @cpython_api([PyObject, Py_ssize_t, Py_ssize_t], PyObject)
@@ -2306,7 +2426,7 @@
     raise NotImplementedError
 
 @cpython_api([], rffi.INT_real, error=CANNOT_FAIL)
-def PyTuple_ClearFreeList(space, ):
+def PyTuple_ClearFreeList(space):
     """Clear the free list. Return the total number of freed items.
     """
     raise NotImplementedError
@@ -2333,7 +2453,7 @@
     raise NotImplementedError
 
 @cpython_api([], rffi.INT_real, error=CANNOT_FAIL)
-def PyUnicode_ClearFreeList(space, ):
+def PyUnicode_ClearFreeList(space):
     """Clear the free list. Return the total number of freed items.
     """
     raise NotImplementedError
@@ -2381,6 +2501,151 @@
     possible.  This macro does not raise exceptions."""
     raise NotImplementedError
 
+ at cpython_api([rffi.CCHARP], PyObject)
+def PyUnicode_FromFormat(space, format):
+    """Take a C printf()-style format string and a variable number of
+    arguments, calculate the size of the resulting Python unicode string and return
+    a string with the values formatted into it.  The variable arguments must be C
+    types and must correspond exactly to the format characters in the format
+    string.  The following format characters are allowed:
+    
+    Format Characters
+    
+    Type
+    
+    Comment
+    
+    %%
+    
+    n/a
+    
+    The literal % character.
+    
+    %c
+    
+    int
+    
+    A single character,
+    represented as an C int.
+    
+    %d
+    
+    int
+    
+    Exactly equivalent to
+    printf("%d").
+    
+    %u
+    
+    unsigned int
+    
+    Exactly equivalent to
+    printf("%u").
+    
+    %ld
+    
+    long
+    
+    Exactly equivalent to
+    printf("%ld").
+    
+    %lu
+    
+    unsigned long
+    
+    Exactly equivalent to
+    printf("%lu").
+    
+    %zd
+    
+    Py_ssize_t
+    
+    Exactly equivalent to
+    printf("%zd").
+    
+    %zu
+    
+    size_t
+    
+    Exactly equivalent to
+    printf("%zu").
+    
+    %i
+    
+    int
+    
+    Exactly equivalent to
+    printf("%i").
+    
+    %x
+    
+    int
+    
+    Exactly equivalent to
+    printf("%x").
+    
+    %s
+    
+    char*
+    
+    A null-terminated C character
+    array.
+    
+    %p
+    
+    void*
+    
+    The hex representation of a C
+    pointer. Mostly equivalent to
+    printf("%p") except that
+    it is guaranteed to start with
+    the literal 0x regardless
+    of what the platform's
+    printf yields.
+    
+    %U
+    
+    PyObject*
+    
+    A unicode object.
+    
+    %V
+    
+    PyObject*, char *
+    
+    A unicode object (which may be
+    NULL) and a null-terminated
+    C character array as a second
+    parameter (which will be used,
+    if the first parameter is
+    NULL).
+    
+    %S
+    
+    PyObject*
+    
+    The result of calling
+    PyObject_Unicode().
+    
+    %R
+    
+    PyObject*
+    
+    The result of calling
+    PyObject_Repr().
+    
+    An unrecognized format character causes all the rest of the format string to be
+    copied as-is to the result string, and any extra arguments discarded.
+    """
+    raise NotImplementedError
+
+ at cpython_api([rffi.CCHARP, va_list], PyObject)
+def PyUnicode_FromFormatV(space, format, vargs):
+    """Identical to PyUnicode_FromFormat() except that it takes exactly two
+    arguments.
+    """
+    raise NotImplementedError
+
 @cpython_api([rffi.CWCHARP, Py_ssize_t, rffi.CCHARP, rffi.CCHARP], PyObject)
 def PyUnicode_Encode(space, s, size, encoding, errors):
     """Encode the Py_UNICODE buffer of the given size and return a Python
@@ -2400,8 +2665,6 @@
     treated as an error. Those bytes will not be decoded and the number of bytes
     that have been decoded will be stored in consumed.
     
-    
-    
     This function used an int type for size. This might require
     changes in your code for properly supporting 64-bit systems."""
     raise NotImplementedError
@@ -2489,8 +2752,6 @@
     split surrogate pair) as an error. Those bytes will not be decoded and the
     number of bytes that have been decoded will be stored in consumed.
     
-    
-    
     This function used an int type for size and an int *
     type for consumed. This might require changes in your code for
     properly supporting 64-bit systems."""
@@ -2526,6 +2787,32 @@
     raise NotImplementedError
 
 @cpython_api([rffi.CCHARP, Py_ssize_t, rffi.CCHARP], PyObject)
+def PyUnicode_DecodeUTF7(space, s, size, errors):
+    """Create a Unicode object by decoding size bytes of the UTF-7 encoded string
+    s.  Return NULL if an exception was raised by the codec."""
+    raise NotImplementedError
+
+ at cpython_api([rffi.CCHARP, Py_ssize_t, rffi.CCHARP, Py_ssize_t], PyObject)
+def PyUnicode_DecodeUTF7Stateful(space, s, size, errors, consumed):
+    """If consumed is NULL, behave like PyUnicode_DecodeUTF7().  If
+    consumed is not NULL, trailing incomplete UTF-7 base-64 sections will not
+    be treated as an error.  Those bytes will not be decoded and the number of
+    bytes that have been decoded will be stored in consumed."""
+    raise NotImplementedError
+
+ at cpython_api([rffi.CWCHARP, Py_ssize_t, rffi.INT_real, rffi.INT_real, rffi.CCHARP], PyObject)
+def PyUnicode_EncodeUTF7(space, s, size, base64SetO, base64WhiteSpace, errors):
+    """Encode the Py_UNICODE buffer of the given size using UTF-7 and
+    return a Python bytes object.  Return NULL if an exception was raised by
+    the codec.
+    
+    If base64SetO is nonzero, "Set O" (punctuation that has no otherwise
+    special meaning) will be encoded in base-64.  If base64WhiteSpace is
+    nonzero, whitespace will be encoded in base-64.  Both are set to zero for the
+    Python "utf-7" codec."""
+    raise NotImplementedError
+
+ at cpython_api([rffi.CCHARP, Py_ssize_t, rffi.CCHARP], PyObject)
 def PyUnicode_DecodeUnicodeEscape(space, s, size, errors):
     """Create a Unicode object by decoding size bytes of the Unicode-Escape encoded
     string s.  Return NULL if an exception was raised by the codec.
@@ -2729,7 +3016,7 @@
     systems."""
     raise NotImplementedError
 
- at cpython_api([PyObject, PyObject, Py_ssize_t, Py_ssize_t], Py_ssize_t)
+ at cpython_api([PyObject, PyObject, Py_ssize_t, Py_ssize_t], Py_ssize_t, error=-1)
 def PyUnicode_Count(space, str, substr, start, end):
     """Return the number of non-overlapping occurrences of substr in
     str[start:end].  Return -1 if an error occurred.
@@ -2878,13 +3165,13 @@
 
 @cpython_api([FILE, rffi.CCHARP, PyCompilerFlags], rffi.INT_real, error=-1)
 def PyRun_InteractiveOneFlags(space, fp, filename, flags):
-    """Read and execute a single statement from a file associated with an interactive
-    device according to the flags argument.  If filename is NULL, "???" is
-    used instead.  The user will be prompted using sys.ps1 and sys.ps2.
-    Returns 0 when the input was executed successfully, -1 if there was an
-    exception, or an error code from the errcode.h include file distributed
-    as part of Python if there was a parse error.  (Note that errcode.h is
-    not included by Python.h, so must be included specifically if needed.)"""
+    """Read and execute a single statement from a file associated with an
+    interactive device according to the flags argument.  The user will be
+    prompted using sys.ps1 and sys.ps2.  Returns 0 when the input was
+    executed successfully, -1 if there was an exception, or an error code
+    from the errcode.h include file distributed as part of Python if
+    there was a parse error.  (Note that errcode.h is not included by
+    Python.h, so must be included specifically if needed.)"""
     raise NotImplementedError
 
 @cpython_api([FILE, rffi.CCHARP], rffi.INT_real, error=-1)
@@ -2896,8 +3183,8 @@
 @cpython_api([FILE, rffi.CCHARP, PyCompilerFlags], rffi.INT_real, error=-1)
 def PyRun_InteractiveLoopFlags(space, fp, filename, flags):
     """Read and execute statements from a file associated with an interactive device
-    until EOF is reached.  If filename is NULL, "???" is used instead.  The
-    user will be prompted using sys.ps1 and sys.ps2.  Returns 0 at EOF."""
+    until EOF is reached.  The user will be prompted using sys.ps1 and
+    sys.ps2.  Returns 0 at EOF."""
     raise NotImplementedError
 
 @cpython_api([rffi.CCHARP, rffi.INT_real], _node)
@@ -2982,7 +3269,7 @@
     be parsed or compiled."""
     raise NotImplementedError
 
- at cpython_api([PyObject, PyObject, PyObject], PyObject)
+ at cpython_api([PyCodeObject, PyObject, PyObject], PyObject)
 def PyEval_EvalCode(space, co, globals, locals):
     """This is a simplified interface to PyEval_EvalCodeEx(), with just
     the code object, and the dictionaries of global and local variables.
@@ -3013,26 +3300,26 @@
     throw() methods of generator objects."""
     raise NotImplementedError
 
- at cpython_api([PyCompilerFlags], rffi.INT_real, error=0)
+ at cpython_api([PyCompilerFlags], rffi.INT_real, error=CANNOT_FAIL)
 def PyEval_MergeCompilerFlags(space, cf):
     """This function changes the flags of the current evaluation frame, and returns
     true on success, false on failure."""
     raise NotImplementedError
 
 @cpython_api([PyObject], rffi.INT_real, error=CANNOT_FAIL)
-def PyWeakref_Check(space, ref):
+def PyWeakref_Check(space, ob):
     """Return true if ob is either a reference or proxy object.
     """
     raise NotImplementedError
 
 @cpython_api([PyObject], rffi.INT_real, error=CANNOT_FAIL)
-def PyWeakref_CheckRef(space, ref):
+def PyWeakref_CheckRef(space, ob):
     """Return true if ob is a reference object.
     """
     raise NotImplementedError
 
 @cpython_api([PyObject], rffi.INT_real, error=CANNOT_FAIL)
-def PyWeakref_CheckProxy(space, ref):
+def PyWeakref_CheckProxy(space, ob):
     """Return true if ob is a proxy object.
     """
     raise NotImplementedError

diff --git a/pypy/module/cpyext/stubgen.py b/pypy/module/cpyext/stubgen.py
--- a/pypy/module/cpyext/stubgen.py
+++ b/pypy/module/cpyext/stubgen.py
@@ -19,15 +19,18 @@
         "PyTypeObject*": "PyTypeObjectPtr",
         "PyVarObject*": "PyObject",
         "const char*": "rffi.CCHARP",
+        "double": "rffi.DOUBLE",
         "PyObject*": "PyObject",
         "PyObject**": "PyObjectP",
         "char*": "rffi.CCHARP",
         "PyMethodDef*": "PyMethodDef",
         "Py_ssize_t": "Py_ssize_t",
         "Py_ssize_t*": "Py_ssize_t",
+        "size_t": "rffi.SIZE_T",
         "...": "...",
         "char": "lltype.Char",
         "long": "lltype.Signed",
+        "Py_buffer*": "Py_buffer",
         "": "",
         }
 
@@ -56,7 +59,8 @@
         if not par[0].has_key('names') or not par[0]['names']:
             continue
         functionname = par[0]['names'][0]
-        if api.FUNCTIONS.get(functionname) is not None:
+        if (functionname in api.FUNCTIONS or
+            functionname in api.SYMBOLS_C):
             print "Wow, you implemented already", functionname
             continue
         borrows = docstring = ""

From commits-noreply at bitbucket.org  Sun Feb 27 22:43:01 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Sun, 27 Feb 2011 22:43:01 +0100 (CET)
Subject: [pypy-svn] pypy default: (kleptog) Implement PyErr_WriteUnraisable
Message-ID: <20110227214301.1947B282D0E@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r42326:9adf3462fae3
Date: 2011-02-27 22:42 +0100
http://bitbucket.org/pypy/pypy/changeset/9adf3462fae3/

Log:	(kleptog) Implement PyErr_WriteUnraisable

diff --git a/pypy/module/cpyext/stubs.py b/pypy/module/cpyext/stubs.py
--- a/pypy/module/cpyext/stubs.py
+++ b/pypy/module/cpyext/stubs.py
@@ -763,18 +763,6 @@
     only be called from the main thread."""
     raise NotImplementedError
 
- at cpython_api([PyObject], lltype.Void)
-def PyErr_WriteUnraisable(space, obj):
-    """This utility function prints a warning message to sys.stderr when an
-    exception has been set but it is impossible for the interpreter to actually
-    raise the exception.  It is used, for example, when an exception occurs in an
-    __del__() method.
-    
-    The function is called with a single argument obj that identifies the context
-    in which the unraisable exception occurred. The repr of obj will be printed in
-    the warning message."""
-    raise NotImplementedError
-
 @cpython_api([rffi.CCHARP, rffi.CCHARP, Py_ssize_t, Py_ssize_t, Py_ssize_t, rffi.CCHARP], PyObject)
 def PyUnicodeDecodeError_Create(space, encoding, object, length, start, end, reason):
     """Create a UnicodeDecodeError object with the attributes encoding,

diff --git a/pypy/module/cpyext/test/test_pyerrors.py b/pypy/module/cpyext/test/test_pyerrors.py
--- a/pypy/module/cpyext/test/test_pyerrors.py
+++ b/pypy/module/cpyext/test/test_pyerrors.py
@@ -89,6 +89,14 @@
         assert "cpyext is cool" in err
         assert not api.PyErr_Occurred()
 
+    def test_WriteUnraisable(self, space, api, capfd):
+        api.PyErr_SetObject(space.w_ValueError, space.wrap("message"))
+        w_where = space.wrap("location")
+        api.PyErr_WriteUnraisable(w_where)
+        out, err = capfd.readouterr()
+        assert ("Exception ValueError: 'message' "
+                "in W_StringObject('location') ignored") == err.strip()
+
 class AppTestFetch(AppTestCpythonExtensionBase):
     def setup_class(cls):
         AppTestCpythonExtensionBase.setup_class.im_func(cls)

diff --git a/pypy/module/cpyext/pyerrors.py b/pypy/module/cpyext/pyerrors.py
--- a/pypy/module/cpyext/pyerrors.py
+++ b/pypy/module/cpyext/pyerrors.py
@@ -284,3 +284,19 @@
                                     space.wrap("traceback"))
     space.call_method(w_traceback, "print_tb", w_tb, space.w_None, w_file)
     return 0
+
+ at cpython_api([PyObject], lltype.Void)
+def PyErr_WriteUnraisable(space, w_where):
+    """This utility function prints a warning message to sys.stderr when an
+    exception has been set but it is impossible for the interpreter to actually
+    raise the exception.  It is used, for example, when an exception occurs in
+    an __del__() method.
+    
+    The function is called with a single argument obj that identifies the
+    context in which the unraisable exception occurred. The repr of obj will be
+    printed in the warning message."""
+    
+    state = space.fromcache(State)
+    operror = state.clear_exception()
+    if operror:
+        operror.write_unraisable(space, w_where)

From commits-noreply at bitbucket.org  Sun Feb 27 23:39:06 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Sun, 27 Feb 2011 23:39:06 +0100 (CET)
Subject: [pypy-svn] pypy default: (kleptog) Implement
	PyString_AsEncodedObject
Message-ID: <20110227223906.A834E2A2053@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r42327:f8db2ce04d45
Date: 2011-02-27 23:38 +0100
http://bitbucket.org/pypy/pypy/changeset/f8db2ce04d45/

Log:	(kleptog) Implement PyString_AsEncodedObject

diff --git a/pypy/module/cpyext/stringobject.py b/pypy/module/cpyext/stringobject.py
--- a/pypy/module/cpyext/stringobject.py
+++ b/pypy/module/cpyext/stringobject.py
@@ -244,3 +244,22 @@
     object with the same value."""
     s = rffi.charp2str(string)
     return space.new_interned_str(s)
+
+ at cpython_api([PyObject, rffi.CCHARP, rffi.CCHARP], PyObject)
+def PyString_AsEncodedObject(space, w_str, encoding, errors):
+    """Encode a string object using the codec registered for encoding and return
+    the result as Python object. encoding and errors have the same meaning as
+    the parameters of the same name in the string encode() method. The codec to
+    be used is looked up using the Python codec registry. Return NULL if an
+    exception was raised by the codec.
+    
+    This function is not available in 3.x and does not have a PyBytes alias."""
+    if not PyString_Check(space, w_str):
+        PyErr_BadArgument(space)
+
+    w_encoding = w_errors = space.w_None
+    if encoding:
+        w_encoding = space.wrap(rffi.charp2str(encoding))
+    if errors:
+        w_errors = space.wrap(rffi.charp2str(errors))
+    return space.call_method(w_str, 'encode', w_encoding, w_errors)

diff --git a/pypy/module/cpyext/stubs.py b/pypy/module/cpyext/stubs.py
--- a/pypy/module/cpyext/stubs.py
+++ b/pypy/module/cpyext/stubs.py
@@ -2287,17 +2287,6 @@
     changes in your code for properly supporting 64-bit systems."""
     raise NotImplementedError
 
- at cpython_api([PyObject, rffi.CCHARP, rffi.CCHARP], PyObject)
-def PyString_AsEncodedObject(space, str, encoding, errors):
-    """Encode a string object using the codec registered for encoding and return the
-    result as Python object. encoding and errors have the same meaning as the
-    parameters of the same name in the string encode() method. The codec to be
-    used is looked up using the Python codec registry. Return NULL if an exception
-    was raised by the codec.
-    
-    This function is not available in 3.x and does not have a PyBytes alias."""
-    raise NotImplementedError
-
 @cpython_api([FILE, rffi.CCHARP], rffi.INT_real, error=CANNOT_FAIL)
 def Py_FdIsInteractive(space, fp, filename):
     """Return true (nonzero) if the standard I/O file fp with name filename is

diff --git a/pypy/module/cpyext/test/test_stringobject.py b/pypy/module/cpyext/test/test_stringobject.py
--- a/pypy/module/cpyext/test/test_stringobject.py
+++ b/pypy/module/cpyext/test/test_stringobject.py
@@ -253,3 +253,29 @@
         w_s2 = api.PyString_InternFromString(buf)
         rffi.free_charp(buf)
         assert w_s1 is w_s2
+
+    def test_AsEncodedObject(self, space, api):
+        ptr = space.wrap('abc')
+
+        errors = rffi.str2charp("strict")
+
+        encoding = rffi.str2charp("hex")
+        res = api.PyString_AsEncodedObject(
+            ptr, encoding, errors)
+        assert space.unwrap(res) == "616263"
+
+        res = api.PyString_AsEncodedObject(
+            ptr, encoding, lltype.nullptr(rffi.CCHARP.TO))
+        assert space.unwrap(res) == "616263"
+        rffi.free_charp(encoding)
+
+        encoding = rffi.str2charp("unknown_encoding")
+        self.raises(space, api, LookupError, api.PyString_AsEncodedObject,
+                    ptr, encoding, errors)
+        rffi.free_charp(encoding)
+
+        rffi.free_charp(errors)
+
+        res = api.PyString_AsEncodedObject(
+            ptr, lltype.nullptr(rffi.CCHARP.TO), lltype.nullptr(rffi.CCHARP.TO))
+        assert space.unwrap(res) == "abc"

From commits-noreply at bitbucket.org  Sun Feb 27 23:43:00 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Sun, 27 Feb 2011 23:43:00 +0100 (CET)
Subject: [pypy-svn] pypy kqueue: Allow submodules for MixedModules,
	imported from my psycopg2 fork.
Message-ID: <20110227224300.283B82A2053@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: kqueue
Changeset: r42328:1a5f26bde444
Date: 2011-02-27 17:42 -0500
http://bitbucket.org/pypy/pypy/changeset/1a5f26bde444/

Log:	Allow submodules for MixedModules, imported from my psycopg2 fork.

diff --git a/pypy/interpreter/baseobjspace.py b/pypy/interpreter/baseobjspace.py
--- a/pypy/interpreter/baseobjspace.py
+++ b/pypy/interpreter/baseobjspace.py
@@ -374,9 +374,9 @@
         else:
             name = importname
 
-        w_name = self.wrap(name)
-        w_mod = self.wrap(Module(self, w_name))
-        self.builtin_modules[name] = w_mod
+        mod = Module(self, self.wrap(name))
+        mod.install()
+
         return name
 
     def getbuiltinmodule(self, name, force_init=False):
@@ -455,22 +455,23 @@
         from pypy.module.exceptions import Module
         w_name = self.wrap('exceptions')
         self.exceptions_module = Module(self, w_name)
-        self.builtin_modules['exceptions'] = self.wrap(self.exceptions_module)
+        self.exceptions_module.install()
 
         from pypy.module.sys import Module
         w_name = self.wrap('sys')
         self.sys = Module(self, w_name)
-        self.builtin_modules['sys'] = self.wrap(self.sys)
+        self.sys.install()
 
         from pypy.module.imp import Module
         w_name = self.wrap('imp')
-        self.builtin_modules['imp'] = self.wrap(Module(self, w_name))
+        mod = Module(self, w_name)
+        mod.install()
 
         from pypy.module.__builtin__ import Module
         w_name = self.wrap('__builtin__')
         self.builtin = Module(self, w_name)
         w_builtin = self.wrap(self.builtin)
-        self.builtin_modules['__builtin__'] = self.wrap(w_builtin)
+        w_builtin.install()
         self.setitem(self.builtin.w_dict, self.wrap('__builtins__'), w_builtin)
 
         bootstrap_modules = set(('sys', 'imp', '__builtin__', 'exceptions'))

diff --git a/pypy/interpreter/mixedmodule.py b/pypy/interpreter/mixedmodule.py
--- a/pypy/interpreter/mixedmodule.py
+++ b/pypy/interpreter/mixedmodule.py
@@ -1,7 +1,7 @@
 from pypy.interpreter.module import Module
 from pypy.interpreter.function import Function, BuiltinFunction
-from pypy.interpreter import gateway 
-from pypy.interpreter.error import OperationError 
+from pypy.interpreter import gateway
+from pypy.interpreter.error import OperationError
 from pypy.interpreter.baseobjspace import W_Root
 import os, sys
 
@@ -19,12 +19,26 @@
     # after startup().
     w_initialdict = None
 
-    def __init__(self, space, w_name): 
-        """ NOT_RPYTHON """ 
-        Module.__init__(self, space, w_name) 
-        self.lazy = True 
+    def __init__(self, space, w_name):
+        """ NOT_RPYTHON """
+        Module.__init__(self, space, w_name)
+        self.lazy = True
         self.__class__.buildloaders()
         self.loaders = self.loaders.copy()    # copy from the class to the inst
+        self.submodules_w = []
+
+    def install(self):
+        """NOT_RPYTHON: install this module, and it's submodules into
+        space.builtin_modules"""
+        Module.install(self)
+        if hasattr(self, "submodules"):
+            space = self.space
+            name = space.unwrap(self.w_name)
+            for sub_name, module_cls in self.submodules.iteritems():
+                module_name = space.wrap("%s.%s" % (name, sub_name))
+                m = module_cls(space, module_name)
+                m.install()
+                self.submodules_w.append(m)
 
     def init(self, space):
         """This is called each time the module is imported or reloaded
@@ -34,11 +48,18 @@
             # the saved dict, as done with built-in and extension modules
             # on CPython.
             space.call_method(self.w_dict, 'update', self.w_initialdict)
-        else:
+
+        for w_submodule in self.submodules_w:
+            name = space.str_w(w_submodule.w_name)
+            space.setitem(self.w_dict, space.wrap(name.split(".")[-1]), w_submodule)
+            space.getbuiltinmodule(name)
+
+        if self.w_initialdict is None:
             Module.init(self, space)
             if not self.lazy and self.w_initialdict is None:
                 self.w_initialdict = space.call_method(self.w_dict, 'items')
 
+
     def get_applevel_name(cls):
         """ NOT_RPYTHON """
         if cls.applevel_name is not None:
@@ -50,13 +71,13 @@
 
     def get(self, name):
         space = self.space
-        w_value = self.getdictvalue(space, name) 
-        if w_value is None: 
+        w_value = self.getdictvalue(space, name)
+        if w_value is None:
             raise OperationError(space.w_AttributeError, space.wrap(name))
-        return w_value 
+        return w_value
 
-    def call(self, name, *args_w): 
-        w_builtin = self.get(name) 
+    def call(self, name, *args_w):
+        w_builtin = self.get(name)
         return self.space.call_function(w_builtin, *args_w)
 
     def getdictvalue(self, space, name):
@@ -67,12 +88,12 @@
 
     def _load_lazily(self, space, name):
         w_name = space.new_interned_str(name)
-        try: 
+        try:
             loader = self.loaders[name]
-        except KeyError: 
-            return None 
-        else: 
-            w_value = loader(space) 
+        except KeyError:
+            return None
+        else:
+            w_value = loader(space)
             func = space.interpclass_w(w_value)
             # the idea of the following code is that all functions that are
             # directly in a mixed-module are "builtin", e.g. they get a
@@ -90,19 +111,19 @@
                     func._builtinversion_ = bltin
                     bltin.name = name
                 w_value = space.wrap(bltin)
-            space.setitem(self.w_dict, w_name, w_value) 
+            space.setitem(self.w_dict, w_name, w_value)
             return w_value
 
 
-    def getdict(self): 
-        if self.lazy: 
+    def getdict(self):
+        if self.lazy:
             space = self.space
-            for name in self.loaders: 
-                w_value = self.get(name)  
-                space.setitem(self.w_dict, space.new_interned_str(name), w_value) 
+            for name in self.loaders:
+                w_value = self.get(name)
+                space.setitem(self.w_dict, space.new_interned_str(name), w_value)
             self.lazy = False
             self.w_initialdict = space.call_method(self.w_dict, 'items')
-        return self.w_dict 
+        return self.w_dict
 
     def _freeze_(self):
         self.getdict()
@@ -113,19 +134,19 @@
         # not constant
         return False
 
-    def buildloaders(cls): 
-        """ NOT_RPYTHON """ 
-        if not hasattr(cls, 'loaders'): 
+    def buildloaders(cls):
+        """ NOT_RPYTHON """
+        if not hasattr(cls, 'loaders'):
             # build a constant dictionary out of
-            # applevel/interplevel definitions 
+            # applevel/interplevel definitions
             cls.loaders = loaders = {}
             pkgroot = cls.__module__
             appname = cls.get_applevel_name()
-            for name, spec in cls.interpleveldefs.items(): 
-                loaders[name] = getinterpevalloader(pkgroot, spec) 
-            for name, spec in cls.appleveldefs.items(): 
+            for name, spec in cls.interpleveldefs.items():
+                loaders[name] = getinterpevalloader(pkgroot, spec)
+            for name, spec in cls.appleveldefs.items():
                 loaders[name] = getappfileloader(pkgroot, appname, spec)
-            assert '__file__' not in loaders 
+            assert '__file__' not in loaders
             if cls.expose__file__attribute:
                 loaders['__file__'] = cls.get__file__
             if '__doc__' not in loaders:
@@ -141,27 +162,27 @@
         w_obj = loader(space)
         space.setattr(space.wrap(self), space.wrap(name), w_obj)
 
-    def get__file__(cls, space): 
-        """ NOT_RPYTHON. 
-        return the __file__ attribute of a MixedModule 
-        which is the root-directory for the various 
+    def get__file__(cls, space):
+        """ NOT_RPYTHON.
+        return the __file__ attribute of a MixedModule
+        which is the root-directory for the various
         applevel and interplevel snippets that make
-        up the module. 
-        """ 
-        try: 
-            fname = cls._fname 
-        except AttributeError: 
+        up the module.
+        """
+        try:
+            fname = cls._fname
+        except AttributeError:
             pkgroot = cls.__module__
             mod = __import__(pkgroot, None, None, ['__doc__'])
-            fname = mod.__file__ 
+            fname = mod.__file__
             assert os.path.basename(fname).startswith('__init__.py')
             # make it clear that it's not really the interp-level module
             # at this path that we are seeing, but an app-level version of it
             fname = os.path.dirname(fname)
-            cls._fname = fname 
-        return space.wrap(fname) 
+            cls._fname = fname
+        return space.wrap(fname)
 
-    get__file__ = classmethod(get__file__) 
+    get__file__ = classmethod(get__file__)
 
     def get__doc__(cls, space):
         return space.wrap(cls.__doc__)
@@ -169,18 +190,18 @@
 
 
 def getinterpevalloader(pkgroot, spec):
-    """ NOT_RPYTHON """     
-    def ifileloader(space): 
+    """ NOT_RPYTHON """
+    def ifileloader(space):
         d = {'space' : space}
-        # EVIL HACK (but it works, and this is not RPython :-) 
-        while 1: 
-            try: 
-                value = eval(spec, d) 
-            except NameError, ex: 
-                name = ex.args[0].split("'")[1] # super-Evil 
+        # EVIL HACK (but it works, and this is not RPython :-)
+        while 1:
+            try:
+                value = eval(spec, d)
+            except NameError, ex:
+                name = ex.args[0].split("'")[1] # super-Evil
                 if name in d:
                     raise   # propagate the NameError
-                try: 
+                try:
                     d[name] = __import__(pkgroot+'.'+name, None, None, [name])
                 except ImportError:
                     etype, evalue, etb = sys.exc_info()
@@ -190,9 +211,9 @@
                         # didn't help, re-raise the original exception for
                         # clarity
                         raise etype, evalue, etb
-            else: 
+            else:
                 #print spec, "->", value
-                if hasattr(value, 'func_code'):  # semi-evil 
+                if hasattr(value, 'func_code'):  # semi-evil
                     return space.wrap(gateway.interp2app(value))
 
                 try:
@@ -205,13 +226,13 @@
                 assert isinstance(value, W_Root), (
                     "interpleveldef %s.%s must return a wrapped object "
                     "(got %r instead)" % (pkgroot, spec, value))
-                return value 
-    return ifileloader 
-        
+                return value
+    return ifileloader
+
 applevelcache = {}
 def getappfileloader(pkgroot, appname, spec):
-    """ NOT_RPYTHON """ 
-    # hum, it's a bit more involved, because we usually 
+    """ NOT_RPYTHON """
+    # hum, it's a bit more involved, because we usually
     # want the import at applevel
     modname, attrname = spec.split('.')
     impbase = pkgroot + '.' + modname
@@ -229,7 +250,6 @@
         app = gateway.applevel(source, filename=fn, modname=appname)
         applevelcache[impbase] = app
 
-    def afileloader(space): 
+    def afileloader(space):
         return app.wget(space, attrname)
-    return afileloader 
-
+    return afileloader

diff --git a/pypy/interpreter/test/test_mixedmodule.py b/pypy/interpreter/test/test_mixedmodule.py
new file mode 100644
--- /dev/null
+++ b/pypy/interpreter/test/test_mixedmodule.py
@@ -0,0 +1,80 @@
+from pypy.interpreter.mixedmodule import MixedModule
+
+
+class TestMixedModule(object):
+    def test_install(self):
+        class Module(MixedModule):
+            interpleveldefs = {}
+            appleveldefs = {}
+
+        m = Module(self.space, self.space.wrap("test_module"))
+        m.install()
+
+        assert self.space.builtin_modules["test_module"] is m
+
+    def test_submodule(self):
+        class SubModule(MixedModule):
+            interpleveldefs = {}
+            appleveldefs = {}
+
+        class Module(MixedModule):
+            interpleveldefs = {}
+            appleveldefs = {}
+            submodules = {
+                "sub": SubModule
+            }
+
+        m = Module(self.space, self.space.wrap("test_module"))
+        m.install()
+
+        assert self.space.builtin_modules["test_module"] is m
+        assert isinstance(self.space.builtin_modules["test_module.sub"], SubModule)
+
+class AppTestMixedModule(object):
+    def setup_class(cls):
+        space = cls.space
+
+        class SubModule(MixedModule):
+            interpleveldefs = {
+                "value": "space.wrap(14)"
+            }
+            appleveldefs = {}
+
+        class Module(MixedModule):
+            interpleveldefs = {}
+            appleveldefs = {}
+            submodules = {
+                "sub": SubModule
+            }
+
+        m = Module(space, space.wrap("test_module"))
+        m.install()
+
+    def teardown_class(cls):
+        from pypy.module.sys.state import get
+
+        space = cls.space
+        del space.builtin_modules["test_module"]
+        del space.builtin_modules["test_module.sub"]
+        w_modules = get(space).w_modules
+        space.delitem(w_modules, space.wrap("test_module"))
+        space.delitem(w_modules, space.wrap("test_module.sub"))
+
+    def test_attibute(self):
+        import test_module
+
+        assert hasattr(test_module, "sub")
+
+    def test_submodule_import(self):
+        from test_module import sub
+
+    def test_direct_import(self):
+        import test_module.sub
+
+        assert test_module.sub
+        assert test_module.sub.value == 14
+
+    def test_import_from(self):
+        from test_module.sub import value
+
+        assert value == 14

diff --git a/pypy/interpreter/module.py b/pypy/interpreter/module.py
--- a/pypy/interpreter/module.py
+++ b/pypy/interpreter/module.py
@@ -13,10 +13,10 @@
 
     def __init__(self, space, w_name, w_dict=None, add_package=True):
         self.space = space
-        if w_dict is None: 
+        if w_dict is None:
             w_dict = space.newdict(module=True)
-        self.w_dict = w_dict 
-        self.w_name = w_name 
+        self.w_dict = w_dict
+        self.w_name = w_name
         if w_name is not None:
             space.setitem(w_dict, space.new_interned_str('__name__'), w_name)
         if add_package:
@@ -26,6 +26,11 @@
                           space.w_None)
         self.startup_called = False
 
+    def install(self):
+        """NOT_RPYTHON: installs this module into space.builtin_modules"""
+        w_mod = self.space.wrap(self)
+        self.space.builtin_modules[self.space.unwrap(self.w_name)] = w_mod
+
     def setup_after_space_initialization(self):
         """NOT_RPYTHON: to allow built-in modules to do some more setup
         after the space is fully initialized."""
@@ -64,14 +69,14 @@
     def descr_module__init__(self, w_name, w_doc=None):
         space = self.space
         self.w_name = w_name
-        if w_doc is None:  
+        if w_doc is None:
             w_doc = space.w_None
         space.setitem(self.w_dict, space.new_interned_str('__name__'), w_name)
         space.setitem(self.w_dict, space.new_interned_str('__doc__'), w_doc)
 
     def descr__reduce__(self, space):
         w_name = space.finditem(self.w_dict, space.wrap('__name__'))
-        if (w_name is None or 
+        if (w_name is None or
             not space.is_true(space.isinstance(w_name, space.w_str))):
             # maybe raise exception here (XXX this path is untested)
             return space.w_None
@@ -81,9 +86,9 @@
             from pypy.interpreter.mixedmodule import MixedModule
             w_mod    = space.getbuiltinmodule('_pickle_support')
             mod      = space.interp_w(MixedModule, w_mod)
-            new_inst = mod.get('module_new')            
+            new_inst = mod.get('module_new')
             return space.newtuple([new_inst, space.newtuple([w_name,
-                                    self.getdict()]), 
+                                    self.getdict()]),
                                   ])
         #already imported case
         w_import = space.builtin.get('__import__')

From commits-noreply at bitbucket.org  Sun Feb 27 23:45:47 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Sun, 27 Feb 2011 23:45:47 +0100 (CET)
Subject: [pypy-svn] pypy mixed-submodules: Closed branch.
Message-ID: <20110227224547.C70822A2054@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: mixed-submodules
Changeset: r42329:c0c4bdb28289
Date: 2011-02-27 17:44 -0500
http://bitbucket.org/pypy/pypy/changeset/c0c4bdb28289/

Log:	Closed branch.


From commits-noreply at bitbucket.org  Sun Feb 27 23:45:49 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Sun, 27 Feb 2011 23:45:49 +0100 (CET)
Subject: [pypy-svn] pypy default: Bring the mixed submodule stuff into the
	right branch.
Message-ID: <20110227224549.27C9F2A2053@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r42330:430fd2b72ed9
Date: 2011-02-27 17:45 -0500
http://bitbucket.org/pypy/pypy/changeset/430fd2b72ed9/

Log:	Bring the mixed submodule stuff into the right branch.

diff --git a/pypy/interpreter/baseobjspace.py b/pypy/interpreter/baseobjspace.py
--- a/pypy/interpreter/baseobjspace.py
+++ b/pypy/interpreter/baseobjspace.py
@@ -374,9 +374,9 @@
         else:
             name = importname
 
-        w_name = self.wrap(name)
-        w_mod = self.wrap(Module(self, w_name))
-        self.builtin_modules[name] = w_mod
+        mod = Module(self, self.wrap(name))
+        mod.install()
+
         return name
 
     def getbuiltinmodule(self, name, force_init=False):
@@ -455,22 +455,23 @@
         from pypy.module.exceptions import Module
         w_name = self.wrap('exceptions')
         self.exceptions_module = Module(self, w_name)
-        self.builtin_modules['exceptions'] = self.wrap(self.exceptions_module)
+        self.exceptions_module.install()
 
         from pypy.module.sys import Module
         w_name = self.wrap('sys')
         self.sys = Module(self, w_name)
-        self.builtin_modules['sys'] = self.wrap(self.sys)
+        self.sys.install()
 
         from pypy.module.imp import Module
         w_name = self.wrap('imp')
-        self.builtin_modules['imp'] = self.wrap(Module(self, w_name))
+        mod = Module(self, w_name)
+        mod.install()
 
         from pypy.module.__builtin__ import Module
         w_name = self.wrap('__builtin__')
         self.builtin = Module(self, w_name)
         w_builtin = self.wrap(self.builtin)
-        self.builtin_modules['__builtin__'] = self.wrap(w_builtin)
+        w_builtin.install()
         self.setitem(self.builtin.w_dict, self.wrap('__builtins__'), w_builtin)
 
         bootstrap_modules = set(('sys', 'imp', '__builtin__', 'exceptions'))

diff --git a/pypy/interpreter/mixedmodule.py b/pypy/interpreter/mixedmodule.py
--- a/pypy/interpreter/mixedmodule.py
+++ b/pypy/interpreter/mixedmodule.py
@@ -1,7 +1,7 @@
 from pypy.interpreter.module import Module
 from pypy.interpreter.function import Function, BuiltinFunction
-from pypy.interpreter import gateway 
-from pypy.interpreter.error import OperationError 
+from pypy.interpreter import gateway
+from pypy.interpreter.error import OperationError
 from pypy.interpreter.baseobjspace import W_Root
 import os, sys
 
@@ -19,12 +19,26 @@
     # after startup().
     w_initialdict = None
 
-    def __init__(self, space, w_name): 
-        """ NOT_RPYTHON """ 
-        Module.__init__(self, space, w_name) 
-        self.lazy = True 
+    def __init__(self, space, w_name):
+        """ NOT_RPYTHON """
+        Module.__init__(self, space, w_name)
+        self.lazy = True
         self.__class__.buildloaders()
         self.loaders = self.loaders.copy()    # copy from the class to the inst
+        self.submodules_w = []
+
+    def install(self):
+        """NOT_RPYTHON: install this module, and it's submodules into
+        space.builtin_modules"""
+        Module.install(self)
+        if hasattr(self, "submodules"):
+            space = self.space
+            name = space.unwrap(self.w_name)
+            for sub_name, module_cls in self.submodules.iteritems():
+                module_name = space.wrap("%s.%s" % (name, sub_name))
+                m = module_cls(space, module_name)
+                m.install()
+                self.submodules_w.append(m)
 
     def init(self, space):
         """This is called each time the module is imported or reloaded
@@ -34,11 +48,18 @@
             # the saved dict, as done with built-in and extension modules
             # on CPython.
             space.call_method(self.w_dict, 'update', self.w_initialdict)
-        else:
+
+        for w_submodule in self.submodules_w:
+            name = space.str_w(w_submodule.w_name)
+            space.setitem(self.w_dict, space.wrap(name.split(".")[-1]), w_submodule)
+            space.getbuiltinmodule(name)
+
+        if self.w_initialdict is None:
             Module.init(self, space)
             if not self.lazy and self.w_initialdict is None:
                 self.w_initialdict = space.call_method(self.w_dict, 'items')
 
+
     def get_applevel_name(cls):
         """ NOT_RPYTHON """
         if cls.applevel_name is not None:
@@ -50,13 +71,13 @@
 
     def get(self, name):
         space = self.space
-        w_value = self.getdictvalue(space, name) 
-        if w_value is None: 
+        w_value = self.getdictvalue(space, name)
+        if w_value is None:
             raise OperationError(space.w_AttributeError, space.wrap(name))
-        return w_value 
+        return w_value
 
-    def call(self, name, *args_w): 
-        w_builtin = self.get(name) 
+    def call(self, name, *args_w):
+        w_builtin = self.get(name)
         return self.space.call_function(w_builtin, *args_w)
 
     def getdictvalue(self, space, name):
@@ -67,12 +88,12 @@
 
     def _load_lazily(self, space, name):
         w_name = space.new_interned_str(name)
-        try: 
+        try:
             loader = self.loaders[name]
-        except KeyError: 
-            return None 
-        else: 
-            w_value = loader(space) 
+        except KeyError:
+            return None
+        else:
+            w_value = loader(space)
             func = space.interpclass_w(w_value)
             # the idea of the following code is that all functions that are
             # directly in a mixed-module are "builtin", e.g. they get a
@@ -90,19 +111,19 @@
                     func._builtinversion_ = bltin
                     bltin.name = name
                 w_value = space.wrap(bltin)
-            space.setitem(self.w_dict, w_name, w_value) 
+            space.setitem(self.w_dict, w_name, w_value)
             return w_value
 
 
-    def getdict(self): 
-        if self.lazy: 
+    def getdict(self):
+        if self.lazy:
             space = self.space
-            for name in self.loaders: 
-                w_value = self.get(name)  
-                space.setitem(self.w_dict, space.new_interned_str(name), w_value) 
+            for name in self.loaders:
+                w_value = self.get(name)
+                space.setitem(self.w_dict, space.new_interned_str(name), w_value)
             self.lazy = False
             self.w_initialdict = space.call_method(self.w_dict, 'items')
-        return self.w_dict 
+        return self.w_dict
 
     def _freeze_(self):
         self.getdict()
@@ -113,19 +134,19 @@
         # not constant
         return False
 
-    def buildloaders(cls): 
-        """ NOT_RPYTHON """ 
-        if not hasattr(cls, 'loaders'): 
+    def buildloaders(cls):
+        """ NOT_RPYTHON """
+        if not hasattr(cls, 'loaders'):
             # build a constant dictionary out of
-            # applevel/interplevel definitions 
+            # applevel/interplevel definitions
             cls.loaders = loaders = {}
             pkgroot = cls.__module__
             appname = cls.get_applevel_name()
-            for name, spec in cls.interpleveldefs.items(): 
-                loaders[name] = getinterpevalloader(pkgroot, spec) 
-            for name, spec in cls.appleveldefs.items(): 
+            for name, spec in cls.interpleveldefs.items():
+                loaders[name] = getinterpevalloader(pkgroot, spec)
+            for name, spec in cls.appleveldefs.items():
                 loaders[name] = getappfileloader(pkgroot, appname, spec)
-            assert '__file__' not in loaders 
+            assert '__file__' not in loaders
             if cls.expose__file__attribute:
                 loaders['__file__'] = cls.get__file__
             if '__doc__' not in loaders:
@@ -141,27 +162,27 @@
         w_obj = loader(space)
         space.setattr(space.wrap(self), space.wrap(name), w_obj)
 
-    def get__file__(cls, space): 
-        """ NOT_RPYTHON. 
-        return the __file__ attribute of a MixedModule 
-        which is the root-directory for the various 
+    def get__file__(cls, space):
+        """ NOT_RPYTHON.
+        return the __file__ attribute of a MixedModule
+        which is the root-directory for the various
         applevel and interplevel snippets that make
-        up the module. 
-        """ 
-        try: 
-            fname = cls._fname 
-        except AttributeError: 
+        up the module.
+        """
+        try:
+            fname = cls._fname
+        except AttributeError:
             pkgroot = cls.__module__
             mod = __import__(pkgroot, None, None, ['__doc__'])
-            fname = mod.__file__ 
+            fname = mod.__file__
             assert os.path.basename(fname).startswith('__init__.py')
             # make it clear that it's not really the interp-level module
             # at this path that we are seeing, but an app-level version of it
             fname = os.path.dirname(fname)
-            cls._fname = fname 
-        return space.wrap(fname) 
+            cls._fname = fname
+        return space.wrap(fname)
 
-    get__file__ = classmethod(get__file__) 
+    get__file__ = classmethod(get__file__)
 
     def get__doc__(cls, space):
         return space.wrap(cls.__doc__)
@@ -169,18 +190,18 @@
 
 
 def getinterpevalloader(pkgroot, spec):
-    """ NOT_RPYTHON """     
-    def ifileloader(space): 
+    """ NOT_RPYTHON """
+    def ifileloader(space):
         d = {'space' : space}
-        # EVIL HACK (but it works, and this is not RPython :-) 
-        while 1: 
-            try: 
-                value = eval(spec, d) 
-            except NameError, ex: 
-                name = ex.args[0].split("'")[1] # super-Evil 
+        # EVIL HACK (but it works, and this is not RPython :-)
+        while 1:
+            try:
+                value = eval(spec, d)
+            except NameError, ex:
+                name = ex.args[0].split("'")[1] # super-Evil
                 if name in d:
                     raise   # propagate the NameError
-                try: 
+                try:
                     d[name] = __import__(pkgroot+'.'+name, None, None, [name])
                 except ImportError:
                     etype, evalue, etb = sys.exc_info()
@@ -190,9 +211,9 @@
                         # didn't help, re-raise the original exception for
                         # clarity
                         raise etype, evalue, etb
-            else: 
+            else:
                 #print spec, "->", value
-                if hasattr(value, 'func_code'):  # semi-evil 
+                if hasattr(value, 'func_code'):  # semi-evil
                     return space.wrap(gateway.interp2app(value))
 
                 try:
@@ -205,13 +226,13 @@
                 assert isinstance(value, W_Root), (
                     "interpleveldef %s.%s must return a wrapped object "
                     "(got %r instead)" % (pkgroot, spec, value))
-                return value 
-    return ifileloader 
-        
+                return value
+    return ifileloader
+
 applevelcache = {}
 def getappfileloader(pkgroot, appname, spec):
-    """ NOT_RPYTHON """ 
-    # hum, it's a bit more involved, because we usually 
+    """ NOT_RPYTHON """
+    # hum, it's a bit more involved, because we usually
     # want the import at applevel
     modname, attrname = spec.split('.')
     impbase = pkgroot + '.' + modname
@@ -229,7 +250,6 @@
         app = gateway.applevel(source, filename=fn, modname=appname)
         applevelcache[impbase] = app
 
-    def afileloader(space): 
+    def afileloader(space):
         return app.wget(space, attrname)
-    return afileloader 
-
+    return afileloader

diff --git a/pypy/interpreter/test/test_mixedmodule.py b/pypy/interpreter/test/test_mixedmodule.py
new file mode 100644
--- /dev/null
+++ b/pypy/interpreter/test/test_mixedmodule.py
@@ -0,0 +1,80 @@
+from pypy.interpreter.mixedmodule import MixedModule
+
+
+class TestMixedModule(object):
+    def test_install(self):
+        class Module(MixedModule):
+            interpleveldefs = {}
+            appleveldefs = {}
+
+        m = Module(self.space, self.space.wrap("test_module"))
+        m.install()
+
+        assert self.space.builtin_modules["test_module"] is m
+
+    def test_submodule(self):
+        class SubModule(MixedModule):
+            interpleveldefs = {}
+            appleveldefs = {}
+
+        class Module(MixedModule):
+            interpleveldefs = {}
+            appleveldefs = {}
+            submodules = {
+                "sub": SubModule
+            }
+
+        m = Module(self.space, self.space.wrap("test_module"))
+        m.install()
+
+        assert self.space.builtin_modules["test_module"] is m
+        assert isinstance(self.space.builtin_modules["test_module.sub"], SubModule)
+
+class AppTestMixedModule(object):
+    def setup_class(cls):
+        space = cls.space
+
+        class SubModule(MixedModule):
+            interpleveldefs = {
+                "value": "space.wrap(14)"
+            }
+            appleveldefs = {}
+
+        class Module(MixedModule):
+            interpleveldefs = {}
+            appleveldefs = {}
+            submodules = {
+                "sub": SubModule
+            }
+
+        m = Module(space, space.wrap("test_module"))
+        m.install()
+
+    def teardown_class(cls):
+        from pypy.module.sys.state import get
+
+        space = cls.space
+        del space.builtin_modules["test_module"]
+        del space.builtin_modules["test_module.sub"]
+        w_modules = get(space).w_modules
+        space.delitem(w_modules, space.wrap("test_module"))
+        space.delitem(w_modules, space.wrap("test_module.sub"))
+
+    def test_attibute(self):
+        import test_module
+
+        assert hasattr(test_module, "sub")
+
+    def test_submodule_import(self):
+        from test_module import sub
+
+    def test_direct_import(self):
+        import test_module.sub
+
+        assert test_module.sub
+        assert test_module.sub.value == 14
+
+    def test_import_from(self):
+        from test_module.sub import value
+
+        assert value == 14

diff --git a/pypy/interpreter/module.py b/pypy/interpreter/module.py
--- a/pypy/interpreter/module.py
+++ b/pypy/interpreter/module.py
@@ -13,10 +13,10 @@
 
     def __init__(self, space, w_name, w_dict=None, add_package=True):
         self.space = space
-        if w_dict is None: 
+        if w_dict is None:
             w_dict = space.newdict(module=True)
-        self.w_dict = w_dict 
-        self.w_name = w_name 
+        self.w_dict = w_dict
+        self.w_name = w_name
         if w_name is not None:
             space.setitem(w_dict, space.new_interned_str('__name__'), w_name)
         if add_package:
@@ -26,6 +26,11 @@
                           space.w_None)
         self.startup_called = False
 
+    def install(self):
+        """NOT_RPYTHON: installs this module into space.builtin_modules"""
+        w_mod = self.space.wrap(self)
+        self.space.builtin_modules[self.space.unwrap(self.w_name)] = w_mod
+
     def setup_after_space_initialization(self):
         """NOT_RPYTHON: to allow built-in modules to do some more setup
         after the space is fully initialized."""
@@ -64,14 +69,14 @@
     def descr_module__init__(self, w_name, w_doc=None):
         space = self.space
         self.w_name = w_name
-        if w_doc is None:  
+        if w_doc is None:
             w_doc = space.w_None
         space.setitem(self.w_dict, space.new_interned_str('__name__'), w_name)
         space.setitem(self.w_dict, space.new_interned_str('__doc__'), w_doc)
 
     def descr__reduce__(self, space):
         w_name = space.finditem(self.w_dict, space.wrap('__name__'))
-        if (w_name is None or 
+        if (w_name is None or
             not space.is_true(space.isinstance(w_name, space.w_str))):
             # maybe raise exception here (XXX this path is untested)
             return space.w_None
@@ -81,9 +86,9 @@
             from pypy.interpreter.mixedmodule import MixedModule
             w_mod    = space.getbuiltinmodule('_pickle_support')
             mod      = space.interp_w(MixedModule, w_mod)
-            new_inst = mod.get('module_new')            
+            new_inst = mod.get('module_new')
             return space.newtuple([new_inst, space.newtuple([w_name,
-                                    self.getdict()]), 
+                                    self.getdict()]),
                                   ])
         #already imported case
         w_import = space.builtin.get('__import__')

From arigo at codespeak.net  Mon Feb 28 00:34:17 2011
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Mon, 28 Feb 2011 00:34:17 +0100 (CET)
Subject: [pypy-svn] r80399 - in pypy/extradoc/talk/stanford-ee380-2011: . ui
	ui/default
Message-ID: <20110227233417.F11382A2056@codespeak.net>

Author: arigo
Date: Mon Feb 28 00:34:15 2011
New Revision: 80399

Added:
   pypy/extradoc/talk/stanford-ee380-2011/speed.png   (contents, props changed)
   pypy/extradoc/talk/stanford-ee380-2011/talk.txt
   pypy/extradoc/talk/stanford-ee380-2011/ui/
   pypy/extradoc/talk/stanford-ee380-2011/ui/default/
   pypy/extradoc/talk/stanford-ee380-2011/ui/default/blank.gif   (contents, props changed)
   pypy/extradoc/talk/stanford-ee380-2011/ui/default/framing.css
   pypy/extradoc/talk/stanford-ee380-2011/ui/default/iepngfix.htc
   pypy/extradoc/talk/stanford-ee380-2011/ui/default/opera.css
   pypy/extradoc/talk/stanford-ee380-2011/ui/default/outline.css
   pypy/extradoc/talk/stanford-ee380-2011/ui/default/pretty.css
   pypy/extradoc/talk/stanford-ee380-2011/ui/default/print.css
   pypy/extradoc/talk/stanford-ee380-2011/ui/default/s5-core.css
   pypy/extradoc/talk/stanford-ee380-2011/ui/default/slides.css
   pypy/extradoc/talk/stanford-ee380-2011/ui/default/slides.js
   pypy/extradoc/talk/stanford-ee380-2011/ui/py-web.png   (contents, props changed)
   pypy/extradoc/talk/stanford-ee380-2011/ui/py.css
Log:
First part of the talk.  Draft.


Added: pypy/extradoc/talk/stanford-ee380-2011/speed.png
==============================================================================
Binary file. No diff available.

Added: pypy/extradoc/talk/stanford-ee380-2011/talk.txt
==============================================================================
--- (empty file)
+++ pypy/extradoc/talk/stanford-ee380-2011/talk.txt	Mon Feb 28 00:34:15 2011
@@ -0,0 +1,421 @@
+.. include:: <s5defs.txt>
+
+=========================================================
+Python in Python: the PyPy system
+=========================================================
+
+
+.. admonition:: Armin Rigo
+
+    - *Heinrich-Heine Universit?t, Germany*
+    - *Open End AB, Sweden*
+
+    March 2011
+
+
+
+
+What is Python
+---------------------------------------------------------
+
+
+What is Python
+---------------------
+
+::
+
+  class Foo(object):
+
+      def __init__(self, value):
+          self.value = value
+
+      def double(self):
+          return Foo(self.value * 2)
+
+  print Foo(42).double().value
+  print Foo("hello").double().value
+
+
+In two words
+------------
+
+* Strongly, trivially, dynamically typed language
+
+* Ints, floats, longs, string, unicode,
+  lists, tuples, dicts, iterators,
+  functions, classes...
+
+
+Python likes its dictionaries
+-----------------------------
+
+::
+
+  d = {}
+  for i in [1, 2, 3, 4]:
+      d[i] = i*i
+  print d
+
+* in this example, we get ``{1:1, 2:4, 3:9, 4:16}``
+
+
+Python is not Java
+------------------
+
+::
+
+  for name in ["add", "sub", "mul"]:
+      def f(x, y):
+          ...
+      globals()[name] = f
+
+
+Python is complicated
+---------------------
+
+How ``a + b`` works (simplified!):
+
+* look up the method __add__ on the type of a
+
+* if there is one, call it
+
+* if it returns NotImplemented, or if there is none,
+  look up the method __radd__ on the type of b
+
+* if there is one, call it
+
+* if there is none, or we get NotImplemented again,
+  raise an exception TypeError
+
+
+Python is a mess
+----------------
+
+How ``a.attr`` or ``a.method()`` works:
+
+* ...
+
+* no way to write it down in just one slide
+
+
+What this talk is about
+-----------------------
+
+* The PyPy project: a framework in which to write interpreters for
+  this kind of language
+
+* "Python in Python" -- roughly
+
+* From the user's point of view (i.e. the programmer in Python),
+  PyPy is very similar to CPython.
+
+
+
+
+CPython and PyPy
+--------------------------------------------------------------------
+
+
+CPython and PyPy
+----------------
+
+* Two implementations
+
+* Two interpreters
+
+* CPython is written in C, PyPy is written in Python
+
+* PyPy tries to be equivalent to CPython
+
+
+...and Jython and IronPython
+----------------------------
+
+* Jython: Python for the Java VM
+
+* IronPython: Python for .NET
+
+* Both try to integrate well with their VM
+
+
+What is PyPy
+------------
+
+* A project started in 2003
+
+* An Open Source effort of volunteers
+
+* With some funding support: 2 years from the European Union (2005-2007),
+  and now from Germany and Sweden (2010-2011).
+
+
+What is PyPy
+------------
+
+* Test-driven development
+
+* Now contains about 200 KLoC, and 150 KLoc of tests
+
+
+What is the point of PyPy?
+--------------------------
+
+* CPython is older, it's the "official" version
+
+* PyPy is just a replacement, so why?
+
+* Moreover PyPy is not quite complete (e.g. C extension
+  modules are only partially supported)
+
+
+Speed
+-----
+
+* First answer: PyPy is faster, and may use less memory
+
+* ...or at least, it is "often" the case
+
+
+Speed
+-----
+
+.. image:: speed.png
+
+http://speed.pypy.org/
+
+
+And (optionally) extra features
+-------------------------------
+
+* "Stackless"
+
+* Non-Python
+
+* and many smaller experiments
+
+* it is a better experimentation platform than CPython
+
+
+Multi-threading
+---------------
+
+* Bad support on CPython (GIL)
+
+* PyPy has no answer to this question (there is also a GIL)
+
+
+
+
+Architecture
+------------------------------------------------------------------------
+
+
+Architecture
+------------
+
+PyPy has two parts:
+
+* A Python interpreter, written in *RPython*
+
+* A compilation toolchain -- the "translator" -- that translates
+  RPython code into C code (mainly)
+
+
+The translation toolchain
+-------------------------
+
+* Takes a program written in RPython, a custom subset of Python
+
+* Outputs the "same" program written in C
+
+* See demo
+
+
+PyPy's Python interpreter
+-------------------------
+
+* A priori similar to CPython, but written in RPython.
+
+* See demo (py.py)
+
+
+RPython is still mostly Python
+------------------------------
+
+* Completely valid Python (can be tested directly)
+
+* Can use lists, dicts, tuples, classes and instances, and so on
+
+* Contains no garbage collection detail (Py_INCREF/Py_DECREF in CPython)
+
+* Really a subset of Python: roughly "how a Java programmer writes his
+  first Python program"
+
+* ...well, plus tons of tricks ``:-)``
+
+
+RPython meta-programming
+------------------------
+
+* RPython is actually only a restriction on the code after being imported,
+  so we can build up everything in (normal) full Python::
+
+    for name in ["add", "sub", "mul"]:
+        def f(x, y):
+            ...
+        globals()[name] = f
+
+* here, the code in ``f()`` is RPython, but the loop around it is not.
+
+
+A bit of history
+----------------
+
+* Squeak and Scheme48 are also interpreters written in themselves
+
+* Or more precisely, like PyPy, a subset of themselves
+
+* In PyPy, the RPython subset is at a higher level
+
+* General rule: *every aspect that is independent from the high-level
+  description of the interpreter is left out of RPython*
+
+
+
+
+Architecture: the interpreter
+--------------------------------------------------------------------------
+
+
+Overview of the interpreter
+---------------------------
+
+* A compiler that produces a custom bytecode format
+
+* An interpreter for this bytecode
+
+* A large library of object types (the "object space")
+
+* A collection of extension modules
+
+
+The bytecode interpreter
+------------------------
+
+* A straightforward, recursive interpreter
+
+* Stack-based
+
+* Every call to a Python function makes a frame object
+
+* Then the interpreter is written as methods on this frame object
+
+
+The object space
+----------------
+
+* Implements all the built-in types
+
+* Structure more flexible than CPython's family of C functions
+
+* Very open to experimentation
+
+
+Separation of levels
+--------------------
+
+* Important: *all* objects that appear in the interpreted program are,
+  in the interpreter, instances of W_XxxObject.
+
+* Again, similar to CPython: an object in Python is implemented,
+  in the interpreter, as a C structure PyXxxObject.
+
+
+Example: smalllong
+------------------
+
+* Standard Python types: int (32/64-bit) and long
+  (integer of unlimited size)
+
+* In CPython, the type is directly linked to its (single) implementation in C.
+  In PyPy, it is not.
+
+* So we could easily add an implementation W_SmallLongObject for
+  integers that happen to fit in 64 bits
+
+* And there is also W_LongObject for the general case
+
+
+Example: smallint
+-----------------
+
+* *Tagged integers,* common in interpreters (but not in CPython)
+
+* Idea, in C terms: take the integer objects whose value fits in 31/63
+  bits, and encode them as odd-valued pseudo-pointers, instead of
+  pointers to separately-allocated integer objects
+
+* We did it in PyPy, but it's disabled now because it does not give
+  the expected performance gain
+
+
+Example: multidict
+------------------
+
+* Similarly, we have several implementations of dict
+
+* For the different typical usage patterns of dicts in Python
+
+* E.g. module dicts (containing all global names of a module),
+  class dicts, instance dicts, user dicts (typically containing
+  non-string keys)
+
+
+Example: mapdict
+----------------
+
+* An instance in Python uses a dictionary to store attributes::
+
+     >>> x = MyClass()
+     >>> x.a = 5
+     >>> x.__dict__
+     {'a': 5}
+     >>> x.__dict__ = {'b': 6}
+     >>> x.b
+     6
+
+
+Example: mapdict
+----------------
+
+* An instance is thus two objects: a dict and a wrapper around it
+
+* Requires a lot of memory
+
+* This is different than Java, Smalltalk or C++, where the class
+  enforces the exact set of attributes of its instances
+
+* But it is like Self and JavaScript
+
+
+Maps
+----------------
+
+* We can reuse the technique introduced in Self: "maps"
+
+* The JavaScript engine V8 also uses them, calling them "hidden classes"
+
+* Idea: it is likely that a lot of instances of a given class will
+  have the same set of attributes
+
+* So we split the attributes into a per-instance part (just an array of
+  field values) and a shared part (giving the attribute names, and their
+  indices in the arrays of the individual instances).
+
+
+
+
+Architecture: the translation toolchain
+---------------------------------------------------------------------
+
+xxx

Added: pypy/extradoc/talk/stanford-ee380-2011/ui/default/blank.gif
==============================================================================
Binary file. No diff available.

Added: pypy/extradoc/talk/stanford-ee380-2011/ui/default/framing.css
==============================================================================
--- (empty file)
+++ pypy/extradoc/talk/stanford-ee380-2011/ui/default/framing.css	Mon Feb 28 00:34:15 2011
@@ -0,0 +1,25 @@
+/* This file has been placed in the public domain. */
+/* The following styles size, place, and layer the slide components.
+   Edit these if you want to change the overall slide layout.
+   The commented lines can be uncommented (and modified, if necessary) 
+   to help you with the rearrangement process. */
+
+/* target = 1024x768 */
+
+div#header, div#footer, .slide {width: 100%; top: 0; left: 0;}
+div#header {position: fixed; top: 0; height: 3em; z-index: 1;}
+div#footer {top: auto; bottom: 0; height: 2.5em; z-index: 5;}
+.slide {top: 0; width: 92%; padding: 2.5em 4% 4%; z-index: 2;}
+div#controls {left: 50%; bottom: 0; width: 50%; z-index: 100;}
+div#controls form {position: absolute; bottom: 0; right: 0; width: 100%;
+  margin: 0;}
+#currentSlide {position: absolute; width: 10%; left: 45%; bottom: 1em;
+  z-index: 10;}
+html>body #currentSlide {position: fixed;}
+
+/*
+div#header {background: #FCC;}
+div#footer {background: #CCF;}
+div#controls {background: #BBD;}
+div#currentSlide {background: #FFC;}
+*/

Added: pypy/extradoc/talk/stanford-ee380-2011/ui/default/iepngfix.htc
==============================================================================
--- (empty file)
+++ pypy/extradoc/talk/stanford-ee380-2011/ui/default/iepngfix.htc	Mon Feb 28 00:34:15 2011
@@ -0,0 +1,42 @@
+<public:component>
+<public:attach event="onpropertychange" onevent="doFix()" />
+
+<script>
+
+// IE5.5+ PNG Alpha Fix v1.0 by Angus Turnbull http://www.twinhelix.com
+// Free usage permitted as long as this notice remains intact.
+
+// This must be a path to a blank image. That's all the configuration you need here.
+var blankImg = 'ui/default/blank.gif';
+
+var f = 'DXImageTransform.Microsoft.AlphaImageLoader';
+
+function filt(s, m) {
+ if (filters[f]) {
+  filters[f].enabled = s ? true : false;
+  if (s) with (filters[f]) { src = s; sizingMethod = m }
+ } else if (s) style.filter = 'progid:'+f+'(src="'+s+'",sizingMethod="'+m+'")';
+}
+
+function doFix() {
+ if ((parseFloat(navigator.userAgent.match(/MSIE (\S+)/)[1]) < 5.5) ||
+  (event && !/(background|src)/.test(event.propertyName))) return;
+
+ if (tagName == 'IMG') {
+  if ((/\.png$/i).test(src)) {
+   filt(src, 'image');  // was 'scale'
+   src = blankImg;
+  } else if (src.indexOf(blankImg) < 0) filt();
+ } else if (style.backgroundImage) {
+  if (style.backgroundImage.match(/^url[("']+(.*\.png)[)"']+$/i)) {
+   var s = RegExp.$1;
+   style.backgroundImage = '';
+   filt(s, 'crop');
+  } else filt();
+ }
+}
+
+doFix();
+
+</script>
+</public:component>
\ No newline at end of file

Added: pypy/extradoc/talk/stanford-ee380-2011/ui/default/opera.css
==============================================================================
--- (empty file)
+++ pypy/extradoc/talk/stanford-ee380-2011/ui/default/opera.css	Mon Feb 28 00:34:15 2011
@@ -0,0 +1,8 @@
+/* This file has been placed in the public domain. */
+/* DO NOT CHANGE THESE unless you really want to break Opera Show */
+.slide {
+	visibility: visible !important;
+	position: static !important;
+	page-break-before: always;
+}
+#slide0 {page-break-before: avoid;}

Added: pypy/extradoc/talk/stanford-ee380-2011/ui/default/outline.css
==============================================================================
--- (empty file)
+++ pypy/extradoc/talk/stanford-ee380-2011/ui/default/outline.css	Mon Feb 28 00:34:15 2011
@@ -0,0 +1,16 @@
+/* This file has been placed in the public domain. */
+/* Don't change this unless you want the layout stuff to show up in the
+   outline view! */
+
+.layout div, #footer *, #controlForm * {display: none;}
+#footer, #controls, #controlForm, #navLinks, #toggle {
+  display: block; visibility: visible; margin: 0; padding: 0;}
+#toggle {float: right; padding: 0.5em;}
+html>body #toggle {position: fixed; top: 0; right: 0;}
+
+/* making the outline look pretty-ish */
+
+#slide0 h1, #slide0 h2, #slide0 h3, #slide0 h4 {border: none; margin: 0;}
+#toggle {border: 1px solid; border-width: 0 0 1px 1px; background: #FFF;}
+
+.outline {display: inline ! important;}

Added: pypy/extradoc/talk/stanford-ee380-2011/ui/default/pretty.css
==============================================================================
--- (empty file)
+++ pypy/extradoc/talk/stanford-ee380-2011/ui/default/pretty.css	Mon Feb 28 00:34:15 2011
@@ -0,0 +1,121 @@
+/* This file has been placed in the public domain. */
+/* Following are the presentation styles -- edit away! */
+
+html, body {margin: 0; padding: 0;}
+body {background: #fff color: #222; font-size: 2em;}
+/* Replace the background style above with the style below (and again for
+   div#header) for a graphic: */
+/* background: white url(bodybg.gif) -16px 0 no-repeat; */
+:link, :visited {text-decoration: none; color: #00C;}
+#controls :active {color: #88A !important;}
+#controls :focus {outline: 1px dotted #227;}
+h1, h2, h3, h4 {font-size: 100%; margin: 0; padding: 0; font-weight: inherit;}
+
+blockquote {padding: 0 2em 0.5em; margin: 0 1.5em 0.5em;}
+blockquote p {margin: 0;}
+
+kbd {font-weight: bold; font-size: 1em;}
+sup {font-size: smaller; line-height: 1px;}
+
+.slide pre {padding: 0; margin-left: 0; margin-right: 0; font-size: 90%;}
+.slide ul ul li {list-style: square; }
+.slide img.leader {display: block; margin: 0 auto;}
+.slide tt {font-size: 90%;}
+
+div#header, div#footer {background: #005; color: #AAB; font-family: sans-serif;}
+/* background: #005 url(bodybg.gif) -16px 0 no-repeat; */
+div#footer {font-size: 0.5em; font-weight: bold; padding: 1em 0;}
+#footer h1 {display: block; padding: 0 1em;}
+#footer h2 {display: block; padding: 0.8em 1em 0;}
+
+.slide {font-size: 1.5em;}
+.slide li {font-size: 1.0em; padding-bottom: 0.2em;}
+.slide h1 {position: absolute; top: 0.45em; z-index: 1;
+  margin: 0; padding-left: 0.7em; white-space: nowrap;
+  font: bold 110% sans-serif; color: #DDE; background: #005;}
+.slide h2 {font: bold 120%/1em sans-serif; padding-top: 0.5em;}
+.slide h3 {font: bold 100% sans-serif; padding-top: 0.5em;}
+h1 abbr {font-variant: small-caps;}
+
+div#controls {position: absolute; left: 50%; bottom: 0;
+  width: 50%; text-align: right; font: bold 0.9em sans-serif;}
+html>body div#controls {position: fixed; padding: 0 0 1em 0; top: auto;}
+div#controls form {position: absolute; bottom: 0; right: 0; width: 100%;
+  margin: 0; padding: 0;}
+#controls #navLinks a {padding: 0; margin: 0 0.5em; 
+  background: #005; border: none; color: #779; cursor: pointer;}
+#controls #navList {height: 1em;}
+#controls #navList #jumplist {position: absolute; bottom: 0; right: 0;
+  background: #DDD; color: #227;}
+
+#currentSlide {text-align: center; font-size: 0.5em; color: #449;
+  font-family: sans-serif; font-weight: bold;}
+
+#slide0 {padding-top: 1.5em}
+#slide0 h1 {position: static; margin: 1em 0 0; padding: 0; color: #000;
+  font: bold 2em sans-serif; white-space: normal; background: transparent;}
+#slide0 h2 {font: bold italic 1em sans-serif; margin: 0.25em;}
+#slide0 h3 {margin-top: 1.5em; font-size: 1.5em;}
+#slide0 h4 {margin-top: 0; font-size: 1em;}
+
+ul.urls {list-style: none; display: inline; margin: 0;}
+.urls li {display: inline; margin: 0;}
+.external {border-bottom: 1px dotted gray;}
+html>body .external {border-bottom: none;}
+.external:after {content: " \274F"; font-size: smaller; color: #77B;}
+
+.incremental, .incremental *, .incremental *:after {visibility: visible;
+  color: white; border: 0;}
+img.incremental {visibility: hidden;}
+.slide .current {color: green;}
+
+.slide-display {display: inline ! important;}
+
+.huge {font-family: sans-serif; font-weight: bold; font-size: 150%;}
+.big {font-family: sans-serif; font-weight: bold; font-size: 120%;}
+.small {font-size: 75%;}
+.tiny {font-size: 50%;}
+.huge tt, .big tt, .small tt, .tiny tt {font-size: 115%;}
+.huge pre, .big pre, .small pre, .tiny pre {font-size: 115%;}
+
+.maroon {color: maroon;}
+.red {color: red;}
+.magenta {color: magenta;}
+.fuchsia {color: fuchsia;}
+.pink {color: #FAA;}
+.orange {color: orange;}
+.yellow {color: yellow;}
+.lime {color: lime;}
+.green {color: green;}
+.olive {color: olive;}
+.teal {color: teal;}
+.cyan {color: cyan;}
+.aqua {color: aqua;}
+.blue {color: blue;}
+.navy {color: navy;}
+.purple {color: purple;}
+.black {color: black;}
+.gray {color: gray;}
+.silver {color: silver;}
+.white {color: white;}
+
+.left {text-align: left ! important;}
+.center {text-align: center ! important;}
+.right {text-align: right ! important;}
+
+.animation {position: relative; margin: 1em 0; padding: 0;}
+.animation img {position: absolute;}
+
+/* Docutils-specific overrides */
+
+.slide table.docinfo {margin: 1em 0 0.5em 2em;}
+
+pre.literal-block, pre.doctest-block {background-color: white;}
+
+tt.docutils {background-color: white;}
+
+/* diagnostics */
+/*
+li:after {content: " [" attr(class) "]"; color: #F88;}
+div:before {content: "[" attr(class) "]"; color: #F88;}
+*/

Added: pypy/extradoc/talk/stanford-ee380-2011/ui/default/print.css
==============================================================================
--- (empty file)
+++ pypy/extradoc/talk/stanford-ee380-2011/ui/default/print.css	Mon Feb 28 00:34:15 2011
@@ -0,0 +1,24 @@
+/* This file has been placed in the public domain. */
+/* The following rule is necessary to have all slides appear in print!
+   DO NOT REMOVE IT! */
+.slide, ul {page-break-inside: avoid; visibility: visible !important;}
+h1 {page-break-after: avoid;}
+
+body {font-size: 12pt; background: white;}
+* {color: black;}
+
+#slide0 h1 {font-size: 200%; border: none; margin: 0.5em 0 0.25em;}
+#slide0 h3 {margin: 0; padding: 0;}
+#slide0 h4 {margin: 0 0 0.5em; padding: 0;}
+#slide0 {margin-bottom: 3em;}
+
+#header {display: none;}
+#footer h1 {margin: 0; border-bottom: 1px solid; color: gray;
+  font-style: italic;}
+#footer h2, #controls {display: none;}
+
+.print {display: inline ! important;}
+
+/* The following rule keeps the layout stuff out of print.
+   Remove at your own risk! */
+.layout, .layout * {display: none !important;}

Added: pypy/extradoc/talk/stanford-ee380-2011/ui/default/s5-core.css
==============================================================================
--- (empty file)
+++ pypy/extradoc/talk/stanford-ee380-2011/ui/default/s5-core.css	Mon Feb 28 00:34:15 2011
@@ -0,0 +1,11 @@
+/* This file has been placed in the public domain. */
+/* Do not edit or override these styles!
+   The system will likely break if you do. */
+
+div#header, div#footer, div#controls, .slide {position: absolute;}
+html>body div#header, html>body div#footer, 
+  html>body div#controls, html>body .slide {position: fixed;}
+.handout {display: none;}
+.layout {display: block;}
+.slide, .hideme, .incremental {visibility: hidden;}
+#slide0 {visibility: visible;}

Added: pypy/extradoc/talk/stanford-ee380-2011/ui/default/slides.css
==============================================================================
--- (empty file)
+++ pypy/extradoc/talk/stanford-ee380-2011/ui/default/slides.css	Mon Feb 28 00:34:15 2011
@@ -0,0 +1,13 @@
+/* This file has been placed in the public domain. */
+
+/* required to make the slide show run at all */
+ at import url(s5-core.css);
+
+/* sets basic placement and size of slide components */
+ at import url(framing.css);
+
+/* styles that make the slides look good */
+ at import url(pretty.css);
+
+/* pypy override */
+ at import url(../py.css);

Added: pypy/extradoc/talk/stanford-ee380-2011/ui/default/slides.js
==============================================================================
--- (empty file)
+++ pypy/extradoc/talk/stanford-ee380-2011/ui/default/slides.js	Mon Feb 28 00:34:15 2011
@@ -0,0 +1,558 @@
+// S5 v1.1 slides.js -- released into the Public Domain
+// Modified for Docutils (http://docutils.sf.net) by David Goodger
+//
+// Please see http://www.meyerweb.com/eric/tools/s5/credits.html for
+// information about all the wonderful and talented contributors to this code!
+
+var undef;
+var slideCSS = '';
+var snum = 0;
+var smax = 1;
+var slideIDs = new Array();
+var incpos = 0;
+var number = undef;
+var s5mode = true;
+var defaultView = 'slideshow';
+var controlVis = 'visible';
+
+var isIE = navigator.appName == 'Microsoft Internet Explorer' ? 1 : 0;
+var isOp = navigator.userAgent.indexOf('Opera') > -1 ? 1 : 0;
+var isGe = navigator.userAgent.indexOf('Gecko') > -1 && navigator.userAgent.indexOf('Safari') < 1 ? 1 : 0;
+
+function hasClass(object, className) {
+	if (!object.className) return false;
+	return (object.className.search('(^|\\s)' + className + '(\\s|$)') != -1);
+}
+
+function hasValue(object, value) {
+	if (!object) return false;
+	return (object.search('(^|\\s)' + value + '(\\s|$)') != -1);
+}
+
+function removeClass(object,className) {
+	if (!object) return;
+	object.className = object.className.replace(new RegExp('(^|\\s)'+className+'(\\s|$)'), RegExp.$1+RegExp.$2);
+}
+
+function addClass(object,className) {
+	if (!object || hasClass(object, className)) return;
+	if (object.className) {
+		object.className += ' '+className;
+	} else {
+		object.className = className;
+	}
+}
+
+function GetElementsWithClassName(elementName,className) {
+	var allElements = document.getElementsByTagName(elementName);
+	var elemColl = new Array();
+	for (var i = 0; i< allElements.length; i++) {
+		if (hasClass(allElements[i], className)) {
+			elemColl[elemColl.length] = allElements[i];
+		}
+	}
+	return elemColl;
+}
+
+function isParentOrSelf(element, id) {
+	if (element == null || element.nodeName=='BODY') return false;
+	else if (element.id == id) return true;
+	else return isParentOrSelf(element.parentNode, id);
+}
+
+function nodeValue(node) {
+	var result = "";
+	if (node.nodeType == 1) {
+		var children = node.childNodes;
+		for (var i = 0; i < children.length; ++i) {
+			result += nodeValue(children[i]);
+		}		
+	}
+	else if (node.nodeType == 3) {
+		result = node.nodeValue;
+	}
+	return(result);
+}
+
+function slideLabel() {
+	var slideColl = GetElementsWithClassName('*','slide');
+	var list = document.getElementById('jumplist');
+	smax = slideColl.length;
+	for (var n = 0; n < smax; n++) {
+		var obj = slideColl[n];
+
+		var did = 'slide' + n.toString();
+		if (obj.getAttribute('id')) {
+			slideIDs[n] = obj.getAttribute('id');
+		}
+		else {
+			obj.setAttribute('id',did);
+			slideIDs[n] = did;
+		}
+		if (isOp) continue;
+
+		var otext = '';
+		var menu = obj.firstChild;
+		if (!menu) continue; // to cope with empty slides
+		while (menu && menu.nodeType == 3) {
+			menu = menu.nextSibling;
+		}
+	 	if (!menu) continue; // to cope with slides with only text nodes
+
+		var menunodes = menu.childNodes;
+		for (var o = 0; o < menunodes.length; o++) {
+			otext += nodeValue(menunodes[o]);
+		}
+		list.options[list.length] = new Option(n + ' : '  + otext, n);
+	}
+}
+
+function currentSlide() {
+	var cs;
+	var footer_nodes;
+	var vis = 'visible';
+	if (document.getElementById) {
+		cs = document.getElementById('currentSlide');
+		footer_nodes = document.getElementById('footer').childNodes;
+	} else {
+		cs = document.currentSlide;
+		footer = document.footer.childNodes;
+	}
+	cs.innerHTML = '<span id="csHere">' + snum + '<\/span> ' + 
+		'<span id="csSep">\/<\/span> ' + 
+		'<span id="csTotal">' + (smax-1) + '<\/span>';
+	if (snum == 0) {
+		vis = 'hidden';
+	}
+	cs.style.visibility = vis;
+	for (var i = 0; i < footer_nodes.length; i++) {
+		if (footer_nodes[i].nodeType == 1) {
+			footer_nodes[i].style.visibility = vis;
+		}
+	}		
+}
+
+function go(step) {
+	if (document.getElementById('slideProj').disabled || step == 0) return;
+	var jl = document.getElementById('jumplist');
+	var cid = slideIDs[snum];
+	var ce = document.getElementById(cid);
+	if (incrementals[snum].length > 0) {
+		for (var i = 0; i < incrementals[snum].length; i++) {
+			removeClass(incrementals[snum][i], 'current');
+			removeClass(incrementals[snum][i], 'incremental');
+		}
+	}
+	if (step != 'j') {
+		snum += step;
+		lmax = smax - 1;
+		if (snum > lmax) snum = lmax;
+		if (snum < 0) snum = 0;
+	} else
+		snum = parseInt(jl.value);
+	var nid = slideIDs[snum];
+	var ne = document.getElementById(nid);
+	if (!ne) {
+		ne = document.getElementById(slideIDs[0]);
+		snum = 0;
+	}
+	if (step < 0) {incpos = incrementals[snum].length} else {incpos = 0;}
+	if (incrementals[snum].length > 0 && incpos == 0) {
+		for (var i = 0; i < incrementals[snum].length; i++) {
+			if (hasClass(incrementals[snum][i], 'current'))
+				incpos = i + 1;
+			else
+				addClass(incrementals[snum][i], 'incremental');
+		}
+	}
+	if (incrementals[snum].length > 0 && incpos > 0)
+		addClass(incrementals[snum][incpos - 1], 'current');
+	ce.style.visibility = 'hidden';
+	ne.style.visibility = 'visible';
+	jl.selectedIndex = snum;
+	currentSlide();
+	number = 0;
+}
+
+function goTo(target) {
+	if (target >= smax || target == snum) return;
+	go(target - snum);
+}
+
+function subgo(step) {
+	if (step > 0) {
+		removeClass(incrementals[snum][incpos - 1],'current');
+		removeClass(incrementals[snum][incpos], 'incremental');
+		addClass(incrementals[snum][incpos],'current');
+		incpos++;
+	} else {
+		incpos--;
+		removeClass(incrementals[snum][incpos],'current');
+		addClass(incrementals[snum][incpos], 'incremental');
+		addClass(incrementals[snum][incpos - 1],'current');
+	}
+}
+
+function toggle() {
+	var slideColl = GetElementsWithClassName('*','slide');
+	var slides = document.getElementById('slideProj');
+	var outline = document.getElementById('outlineStyle');
+	if (!slides.disabled) {
+		slides.disabled = true;
+		outline.disabled = false;
+		s5mode = false;
+		fontSize('1em');
+		for (var n = 0; n < smax; n++) {
+			var slide = slideColl[n];
+			slide.style.visibility = 'visible';
+		}
+	} else {
+		slides.disabled = false;
+		outline.disabled = true;
+		s5mode = true;
+		fontScale();
+		for (var n = 0; n < smax; n++) {
+			var slide = slideColl[n];
+			slide.style.visibility = 'hidden';
+		}
+		slideColl[snum].style.visibility = 'visible';
+	}
+}
+
+function showHide(action) {
+	var obj = GetElementsWithClassName('*','hideme')[0];
+	switch (action) {
+	case 's': obj.style.visibility = 'visible'; break;
+	case 'h': obj.style.visibility = 'hidden'; break;
+	case 'k':
+		if (obj.style.visibility != 'visible') {
+			obj.style.visibility = 'visible';
+		} else {
+			obj.style.visibility = 'hidden';
+		}
+	break;
+	}
+}
+
+// 'keys' code adapted from MozPoint (http://mozpoint.mozdev.org/)
+function keys(key) {
+	if (!key) {
+		key = event;
+		key.which = key.keyCode;
+	}
+	if (key.which == 84) {
+		toggle();
+		return;
+	}
+	if (s5mode) {
+		switch (key.which) {
+			case 10: // return
+			case 13: // enter
+				if (window.event && isParentOrSelf(window.event.srcElement, 'controls')) return;
+				if (key.target && isParentOrSelf(key.target, 'controls')) return;
+				if(number != undef) {
+					goTo(number);
+					break;
+				}
+			case 32: // spacebar
+			case 34: // page down
+			case 39: // rightkey
+			case 40: // downkey
+				if(number != undef) {
+					go(number);
+				} else if (!incrementals[snum] || incpos >= incrementals[snum].length) {
+					go(1);
+				} else {
+					subgo(1);
+				}
+				break;
+			case 33: // page up
+			case 37: // leftkey
+			case 38: // upkey
+				if(number != undef) {
+					go(-1 * number);
+				} else if (!incrementals[snum] || incpos <= 0) {
+					go(-1);
+				} else {
+					subgo(-1);
+				}
+				break;
+			case 36: // home
+				goTo(0);
+				break;
+			case 35: // end
+				goTo(smax-1);
+				break;
+			case 67: // c
+				showHide('k');
+				break;
+		}
+		if (key.which < 48 || key.which > 57) {
+			number = undef;
+		} else {
+			if (window.event && isParentOrSelf(window.event.srcElement, 'controls')) return;
+			if (key.target && isParentOrSelf(key.target, 'controls')) return;
+			number = (((number != undef) ? number : 0) * 10) + (key.which - 48);
+		}
+	}
+	return false;
+}
+
+function clicker(e) {
+	number = undef;
+	var target;
+	if (window.event) {
+		target = window.event.srcElement;
+		e = window.event;
+	} else target = e.target;
+    if (target.href != null || hasValue(target.rel, 'external') || isParentOrSelf(target, 'controls') || isParentOrSelf(target,'embed') || isParentOrSelf(target, 'object')) return true; 
+	if (!e.which || e.which == 1) {
+		if (!incrementals[snum] || incpos >= incrementals[snum].length) {
+			go(1);
+		} else {
+			subgo(1);
+		}
+	}
+}
+
+function findSlide(hash) {
+	var target = document.getElementById(hash);
+	if (target) {
+		for (var i = 0; i < slideIDs.length; i++) {
+			if (target.id == slideIDs[i]) return i;
+		}
+	}
+	return null;
+}
+
+function slideJump() {
+	if (window.location.hash == null || window.location.hash == '') {
+		currentSlide();
+		return;
+	}
+	if (window.location.hash == null) return;
+	var dest = null;
+	dest = findSlide(window.location.hash.slice(1));
+	if (dest == null) {
+		dest = 0;
+	}
+	go(dest - snum);
+}
+
+function fixLinks() {
+	var thisUri = window.location.href;
+	thisUri = thisUri.slice(0, thisUri.length - window.location.hash.length);
+	var aelements = document.getElementsByTagName('A');
+	for (var i = 0; i < aelements.length; i++) {
+		var a = aelements[i].href;
+		var slideID = a.match('\#.+');
+		if ((slideID) && (slideID[0].slice(0,1) == '#')) {
+			var dest = findSlide(slideID[0].slice(1));
+			if (dest != null) {
+				if (aelements[i].addEventListener) {
+					aelements[i].addEventListener("click", new Function("e",
+						"if (document.getElementById('slideProj').disabled) return;" +
+						"go("+dest+" - snum); " +
+						"if (e.preventDefault) e.preventDefault();"), true);
+				} else if (aelements[i].attachEvent) {
+					aelements[i].attachEvent("onclick", new Function("",
+						"if (document.getElementById('slideProj').disabled) return;" +
+						"go("+dest+" - snum); " +
+						"event.returnValue = false;"));
+				}
+			}
+		}
+	}
+}
+
+function externalLinks() {
+	if (!document.getElementsByTagName) return;
+	var anchors = document.getElementsByTagName('a');
+	for (var i=0; i<anchors.length; i++) {
+		var anchor = anchors[i];
+		if (anchor.getAttribute('href') && hasValue(anchor.rel, 'external')) {
+			anchor.target = '_blank';
+			addClass(anchor,'external');
+		}
+	}
+}
+
+function createControls() {
+	var controlsDiv = document.getElementById("controls");
+	if (!controlsDiv) return;
+	var hider = ' onmouseover="showHide(\'s\');" onmouseout="showHide(\'h\');"';
+	var hideDiv, hideList = '';
+	if (controlVis == 'hidden') {
+		hideDiv = hider;
+	} else {
+		hideList = hider;
+	}
+	controlsDiv.innerHTML = '<form action="#" id="controlForm"' + hideDiv + '>' +
+	'<div id="navLinks">' +
+	'<a accesskey="t" id="toggle" href="javascript:toggle();">&#216;<\/a>' +
+	'<a accesskey="z" id="prev" href="javascript:go(-1);">&laquo;<\/a>' +
+	'<a accesskey="x" id="next" href="javascript:go(1);">&raquo;<\/a>' +
+	'<div id="navList"' + hideList + '><select id="jumplist" onchange="go(\'j\');"><\/select><\/div>' +
+	'<\/div><\/form>';
+	if (controlVis == 'hidden') {
+		var hidden = document.getElementById('navLinks');
+	} else {
+		var hidden = document.getElementById('jumplist');
+	}
+	addClass(hidden,'hideme');
+}
+
+function fontScale() {  // causes layout problems in FireFox that get fixed if browser's Reload is used; same may be true of other Gecko-based browsers
+	if (!s5mode) return false;
+	var vScale = 22;  // both yield 32 (after rounding) at 1024x768
+	var hScale = 32;  // perhaps should auto-calculate based on theme's declared value?
+	if (window.innerHeight) {
+		var vSize = window.innerHeight;
+		var hSize = window.innerWidth;
+	} else if (document.documentElement.clientHeight) {
+		var vSize = document.documentElement.clientHeight;
+		var hSize = document.documentElement.clientWidth;
+	} else if (document.body.clientHeight) {
+		var vSize = document.body.clientHeight;
+		var hSize = document.body.clientWidth;
+	} else {
+		var vSize = 700;  // assuming 1024x768, minus chrome and such
+		var hSize = 1024; // these do not account for kiosk mode or Opera Show
+	}
+	var newSize = Math.min(Math.round(vSize/vScale),Math.round(hSize/hScale));
+	fontSize(newSize + 'px');
+	if (isGe) {  // hack to counter incremental reflow bugs
+		var obj = document.getElementsByTagName('body')[0];
+		obj.style.display = 'none';
+		obj.style.display = 'block';
+	}
+}
+
+function fontSize(value) {
+	if (!(s5ss = document.getElementById('s5ss'))) {
+		if (!isIE) {
+			document.getElementsByTagName('head')[0].appendChild(s5ss = document.createElement('style'));
+			s5ss.setAttribute('media','screen, projection');
+			s5ss.setAttribute('id','s5ss');
+		} else {
+			document.createStyleSheet();
+			document.s5ss = document.styleSheets[document.styleSheets.length - 1];
+		}
+	}
+	if (!isIE) {
+		while (s5ss.lastChild) s5ss.removeChild(s5ss.lastChild);
+		s5ss.appendChild(document.createTextNode('body {font-size: ' + value + ' !important;}'));
+	} else {
+		document.s5ss.addRule('body','font-size: ' + value + ' !important;');
+	}
+}
+
+function notOperaFix() {
+	slideCSS = document.getElementById('slideProj').href;
+	var slides = document.getElementById('slideProj');
+	var outline = document.getElementById('outlineStyle');
+	slides.setAttribute('media','screen');
+	outline.disabled = true;
+	if (isGe) {
+		slides.setAttribute('href','null');   // Gecko fix
+		slides.setAttribute('href',slideCSS); // Gecko fix
+	}
+	if (isIE && document.styleSheets && document.styleSheets[0]) {
+		document.styleSheets[0].addRule('img', 'behavior: url(ui/default/iepngfix.htc)');
+		document.styleSheets[0].addRule('div', 'behavior: url(ui/default/iepngfix.htc)');
+		document.styleSheets[0].addRule('.slide', 'behavior: url(ui/default/iepngfix.htc)');
+	}
+}
+
+function getIncrementals(obj) {
+	var incrementals = new Array();
+	if (!obj) 
+		return incrementals;
+	var children = obj.childNodes;
+	for (var i = 0; i < children.length; i++) {
+		var child = children[i];
+		if (hasClass(child, 'incremental')) {
+			if (child.nodeName == 'OL' || child.nodeName == 'UL') {
+				removeClass(child, 'incremental');
+				for (var j = 0; j < child.childNodes.length; j++) {
+					if (child.childNodes[j].nodeType == 1) {
+						addClass(child.childNodes[j], 'incremental');
+					}
+				}
+			} else {
+				incrementals[incrementals.length] = child;
+				removeClass(child,'incremental');
+			}
+		}
+		if (hasClass(child, 'show-first')) {
+			if (child.nodeName == 'OL' || child.nodeName == 'UL') {
+				removeClass(child, 'show-first');
+				if (child.childNodes[isGe].nodeType == 1) {
+					removeClass(child.childNodes[isGe], 'incremental');
+				}
+			} else {
+				incrementals[incrementals.length] = child;
+			}
+		}
+		incrementals = incrementals.concat(getIncrementals(child));
+	}
+	return incrementals;
+}
+
+function createIncrementals() {
+	var incrementals = new Array();
+	for (var i = 0; i < smax; i++) {
+		incrementals[i] = getIncrementals(document.getElementById(slideIDs[i]));
+	}
+	return incrementals;
+}
+
+function defaultCheck() {
+	var allMetas = document.getElementsByTagName('meta');
+	for (var i = 0; i< allMetas.length; i++) {
+		if (allMetas[i].name == 'defaultView') {
+			defaultView = allMetas[i].content;
+		}
+		if (allMetas[i].name == 'controlVis') {
+			controlVis = allMetas[i].content;
+		}
+	}
+}
+
+// Key trap fix, new function body for trap()
+function trap(e) {
+	if (!e) {
+		e = event;
+		e.which = e.keyCode;
+	}
+	try {
+		modifierKey = e.ctrlKey || e.altKey || e.metaKey;
+	}
+	catch(e) {
+		modifierKey = false;
+	}
+	return modifierKey || e.which == 0;
+}
+
+function startup() {
+	defaultCheck();
+	if (!isOp) createControls();
+	slideLabel();
+	fixLinks();
+	externalLinks();
+	fontScale();
+	if (!isOp) {
+		notOperaFix();
+		incrementals = createIncrementals();
+		slideJump();
+		if (defaultView == 'outline') {
+			toggle();
+		}
+		document.onkeyup = keys;
+		document.onkeypress = trap;
+		document.onclick = clicker;
+	}
+}
+
+window.onload = startup;
+window.onresize = function(){setTimeout('fontScale()', 50);}

Added: pypy/extradoc/talk/stanford-ee380-2011/ui/py-web.png
==============================================================================
Binary file. No diff available.

Added: pypy/extradoc/talk/stanford-ee380-2011/ui/py.css
==============================================================================
--- (empty file)
+++ pypy/extradoc/talk/stanford-ee380-2011/ui/py.css	Mon Feb 28 00:34:15 2011
@@ -0,0 +1,77 @@
+body, h1, h2, h3, h4, td, p, div {
+  font-family: sans-serif;
+/* font-size: 1em; */
+}
+
+div {
+  font-size: 0.7em;
+}
+
+#slide0 h1.title {
+  text-align: center;
+  font-family: sans-serif;
+}
+
+div#header, div#footer, div#controls {
+  background-color: white;
+}
+
+div#header {
+  background-image: url("py-web.png");
+  background-repeat: no-repeat;
+  margin: 3px;
+  height: 120px;
+  border-bottom: 1px solid black;
+}
+
+.slide h1 {
+  background-color: white;
+  margin-left: 180px;
+  padding-left: 0px;
+  color: black;
+  font-size: 1.2em;
+}
+
+div#footer {
+  padding: 3px;
+  height: 4em;
+  border-top: 1px solid black;
+}
+
+div#footer h1, div#footer h2, div#footer h3 {
+  font-family: "Times New Roman";
+  font-style: italic;
+  padding: 0px;
+}
+
+div#footer h1 {
+  font-size: 2em;
+}
+
+#controls {
+  border: 1px solid red;
+  background-color: red;
+  width: 100px;
+  visibility: hidden;
+}
+
+#controls #controlForm {
+  visibility: visible;
+}
+
+#navList, #navLinks {
+  background-color: transparent;
+}
+
+#navLinks a#toggle, #navLinks a#prev, #navLinks a#next {
+  background-color: transparent;
+}
+
+/* some hacks to fix whitespace between list items */
+
+li, li p {
+  line-height: 1.2em;
+/* font-size: 1em; */
+  margin: 0px;
+  margin-left: 30px;
+}


From commits-noreply at bitbucket.org  Mon Feb 28 00:46:04 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Mon, 28 Feb 2011 00:46:04 +0100 (CET)
Subject: [pypy-svn] pypy default: Python2.7 can encode/decode single
	surrogates to utf-8.
Message-ID: <20110227234604.7D93D2A2056@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r42331:28d62f149675
Date: 2011-02-28 00:43 +0100
http://bitbucket.org/pypy/pypy/changeset/28d62f149675/

Log:	Python2.7 can encode/decode single surrogates to utf-8. Add a test
	for this.

diff --git a/pypy/rlib/test/test_runicode.py b/pypy/rlib/test/test_runicode.py
--- a/pypy/rlib/test/test_runicode.py
+++ b/pypy/rlib/test/test_runicode.py
@@ -1,4 +1,4 @@
-# -*- coding: utf8 -*-
+# -*- coding: utf-8 -*-
 
 import py
 import sys, random
@@ -135,6 +135,11 @@
         for s in ["\xd7\x90", "\xd6\x96", "\xeb\x96\x95", "\xf0\x90\x91\x93"]:
             self.checkdecode(s, "utf-8")
 
+    def test_utf8_surrogate(self):
+        # A surrogate should not be valid utf-8, but python 2.x accepts them.
+        # This test will raise an error with python 3.x
+        self.checkdecode(u"\ud800", "utf-8")
+
     def test_utf8_errors(self):
         for s in [# unexpected end of data
                   "\xd7", "\xd6", "\xeb\x96", "\xf0\x90\x91"]:

From commits-noreply at bitbucket.org  Mon Feb 28 05:08:42 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Mon, 28 Feb 2011 05:08:42 +0100 (CET)
Subject: [pypy-svn] pypy default: Missing import.
Message-ID: <20110228040842.5ACD52A2036@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r42332:17df0857a7c8
Date: 2011-02-27 23:08 -0500
http://bitbucket.org/pypy/pypy/changeset/17df0857a7c8/

Log:	Missing import.

diff --git a/pypy/module/cpyext/stringobject.py b/pypy/module/cpyext/stringobject.py
--- a/pypy/module/cpyext/stringobject.py
+++ b/pypy/module/cpyext/stringobject.py
@@ -3,6 +3,7 @@
 from pypy.module.cpyext.api import (
     cpython_api, cpython_struct, bootstrap_function, build_type_checkers,
     PyObjectFields, Py_ssize_t, CONST_STRING)
+from pypy.module.cpyext.pyerrors import PyErr_BadArgument
 from pypy.module.cpyext.pyobject import (
     PyObject, PyObjectP, Py_DecRef, make_ref, from_ref, track_reference,
     make_typedescr, get_typedescr)
@@ -252,7 +253,7 @@
     the parameters of the same name in the string encode() method. The codec to
     be used is looked up using the Python codec registry. Return NULL if an
     exception was raised by the codec.
-    
+
     This function is not available in 3.x and does not have a PyBytes alias."""
     if not PyString_Check(space, w_str):
         PyErr_BadArgument(space)

diff --git a/pypy/module/cpyext/test/test_stringobject.py b/pypy/module/cpyext/test/test_stringobject.py
--- a/pypy/module/cpyext/test/test_stringobject.py
+++ b/pypy/module/cpyext/test/test_stringobject.py
@@ -49,7 +49,7 @@
         assert module.get_hello2() == 'Hello world'
         assert module.test_Size()
         raises(TypeError, module.test_Size_exception)
-    
+
         assert module.test_is_string("")
         assert not module.test_is_string(())
 
@@ -152,7 +152,7 @@
             ''')
         res = module.test_string_format_v(1, "xyz")
         assert res == "bla 1 ble xyz\n"
-        
+
     def test_format(self):
         module = self.import_extension('foo', [
             ("test_string_format", "METH_VARARGS",
@@ -203,7 +203,7 @@
         assert c_buf.c_bf_getreadbuffer(py_obj, 0, ref) == 10
         lltype.free(ref, flavor='raw')
         Py_DecRef(space, py_obj)
-        
+
     def test_Concat(self, space, api):
         ref = make_ref(space, space.wrap('abc'))
         ptr = lltype.malloc(PyObjectP.TO, 1, flavor='raw')
@@ -215,7 +215,7 @@
         ptr[0] = lltype.nullptr(PyObject.TO)
         api.PyString_Concat(ptr, space.wrap('def')) # should not crash
         lltype.free(ptr, flavor='raw')
-    
+
     def test_ConcatAndDel(self, space, api):
         ref1 = make_ref(space, space.wrap('abc'))
         ref2 = make_ref(space, space.wrap('def'))
@@ -279,3 +279,7 @@
         res = api.PyString_AsEncodedObject(
             ptr, lltype.nullptr(rffi.CCHARP.TO), lltype.nullptr(rffi.CCHARP.TO))
         assert space.unwrap(res) == "abc"
+
+        self.raises(space, api, TypeError, api.PyString_AsEncodedObject,
+            space.wrap(2), lltype.nullptr(rffi.CCHARP.TO), lltype.nullptr(rffi.CCHARP.TO)
+        )

From commits-noreply at bitbucket.org  Mon Feb 28 07:13:10 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Mon, 28 Feb 2011 07:13:10 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix translation.
Message-ID: <20110228061310.84DFC2A2036@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r42333:d9f29729ac37
Date: 2011-02-28 01:12 -0500
http://bitbucket.org/pypy/pypy/changeset/d9f29729ac37/

Log:	Fix translation.

diff --git a/pypy/module/cpyext/pyerrors.py b/pypy/module/cpyext/pyerrors.py
--- a/pypy/module/cpyext/pyerrors.py
+++ b/pypy/module/cpyext/pyerrors.py
@@ -45,7 +45,7 @@
     If the error indicator is not set, set all three variables to NULL.  If it is
     set, it will be cleared and you own a reference to each object retrieved.  The
     value and traceback object may be NULL even when the type object is not.
-    
+
     This function is normally only used by code that needs to handle exceptions or
     by code that needs to save and restore the error indicator temporarily."""
     state = space.fromcache(State)
@@ -70,7 +70,7 @@
     reference to each object before the call and after the call you no longer own
     these references.  (If you don't understand this, don't use this function.  I
     warned you.)
-    
+
     This function is normally only used by code that needs to save and restore the
     error indicator temporarily; use PyErr_Fetch() to save the current
     exception state."""
@@ -100,7 +100,7 @@
     """This is a shorthand for PyErr_SetString(PyExc_TypeError, message), where
     message indicates that a built-in operation was invoked with an illegal
     argument.  It is mostly for internal use."""
-    raise OperationError(space.w_TypeError, 
+    raise OperationError(space.w_TypeError,
             space.wrap("bad argument type for built-in operation"))
 
 @cpython_api([], lltype.Void)
@@ -198,7 +198,7 @@
     the  currently executing line of code in that stack frame.  A stacklevel of 1
     is the function calling PyErr_WarnEx(), 2 is  the function above that,
     and so forth.
-    
+
     This function normally prints a warning message to sys.stderr; however, it is
     also possible that the user has specified that warnings are to be turned into
     errors, and in that case this will raise an exception.  It is also possible that
@@ -210,7 +210,7 @@
     intentional.)  If an exception is raised, the caller should do its normal
     exception handling (for example, Py_DECREF() owned references and return
     an error value).
-    
+
     Warning categories must be subclasses of Warning; the default warning
     category is RuntimeWarning.  The standard Python warning categories are
     available as global variables whose names are PyExc_ followed by the Python
@@ -221,7 +221,7 @@
     PyExc_FutureWarning.  PyExc_Warning is a subclass of
     PyExc_Exception; the other warning categories are subclasses of
     PyExc_Warning.
-    
+
     For information about warning control, see the documentation for the
     warnings module and the -W option in the command line
     documentation.  There is no C API for warning control."""
@@ -241,7 +241,7 @@
     below) or NULL; the message argument is a message string.  The warning will
     appear to be issued from the function calling PyErr_Warn(), equivalent to
     calling PyErr_WarnEx() with a stacklevel of 1.
-    
+
     Deprecated; use PyErr_WarnEx() instead."""
     return PyErr_WarnEx(space, w_category, message, 1)
 
@@ -291,12 +291,12 @@
     exception has been set but it is impossible for the interpreter to actually
     raise the exception.  It is used, for example, when an exception occurs in
     an __del__() method.
-    
+
     The function is called with a single argument obj that identifies the
     context in which the unraisable exception occurred. The repr of obj will be
     printed in the warning message."""
-    
+
     state = space.fromcache(State)
     operror = state.clear_exception()
     if operror:
-        operror.write_unraisable(space, w_where)
+        operror.write_unraisable(space, space.str_w(space.repr(w_where)))

diff --git a/pypy/module/cpyext/test/test_pyerrors.py b/pypy/module/cpyext/test/test_pyerrors.py
--- a/pypy/module/cpyext/test/test_pyerrors.py
+++ b/pypy/module/cpyext/test/test_pyerrors.py
@@ -68,7 +68,7 @@
         api.PyErr_NoMemory()
         assert space.eq_w(state.operror.w_type, space.w_MemoryError)
         api.PyErr_Clear()
-        
+
     def test_BadArgument(self, space, api):
         api.PyErr_BadArgument()
         state = space.fromcache(State)
@@ -94,8 +94,7 @@
         w_where = space.wrap("location")
         api.PyErr_WriteUnraisable(w_where)
         out, err = capfd.readouterr()
-        assert ("Exception ValueError: 'message' "
-                "in W_StringObject('location') ignored") == err.strip()
+        assert "Exception ValueError: 'message' in 'location' ignored" == err.strip()
 
 class AppTestFetch(AppTestCpythonExtensionBase):
     def setup_class(cls):

From commits-noreply at bitbucket.org  Mon Feb 28 08:29:24 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Mon, 28 Feb 2011 08:29:24 +0100 (CET)
Subject: [pypy-svn] pypy move-rfloat: Branch to move float operations
	from	rlib/rarithmetic to rlib/rfloat
Message-ID: <20110228072924.8F3072A2036@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: move-rfloat
Changeset: r42334:b91d187fc9cd
Date: 2011-02-28 08:26 +0100
http://bitbucket.org/pypy/pypy/changeset/b91d187fc9cd/

Log:	Branch to move float operations from rlib/rarithmetic to rlib/rfloat


From commits-noreply at bitbucket.org  Mon Feb 28 08:29:37 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Mon, 28 Feb 2011 08:29:37 +0100 (CET)
Subject: [pypy-svn] pypy move-rfloat: Move float/double related code from
	rarithmetic.py	to rfloat.py
Message-ID: <20110228072937.8CF5C2A203C@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: move-rfloat
Changeset: r42335:d7a6c72e96db
Date: 2011-02-28 08:24 +0100
http://bitbucket.org/pypy/pypy/changeset/d7a6c72e96db/

Log:	Move float/double related code from rarithmetic.py to rfloat.py

diff --git a/pypy/objspace/std/formatting.py b/pypy/objspace/std/formatting.py
--- a/pypy/objspace/std/formatting.py
+++ b/pypy/objspace/std/formatting.py
@@ -2,8 +2,8 @@
 String formatting routines.
 """
 from pypy.rlib.unroll import unrolling_iterable
-from pypy.rlib.rarithmetic import (
-    ovfcheck, formatd, DTSF_ALT, isnan, isinf)
+from pypy.rlib.rarithmetic import ovfcheck
+from pypy.rlib.rfloat import formatd, DTSF_ALT, isnan, isinf
 from pypy.interpreter.error import OperationError
 from pypy.tool.sourcetools import func_with_new_name
 from pypy.rlib.rstring import StringBuilder, UnicodeBuilder

diff --git a/pypy/objspace/std/complexobject.py b/pypy/objspace/std/complexobject.py
--- a/pypy/objspace/std/complexobject.py
+++ b/pypy/objspace/std/complexobject.py
@@ -5,7 +5,7 @@
 from pypy.objspace.std.register_all import register_all
 from pypy.objspace.std.floatobject import W_FloatObject, _hash_float
 from pypy.objspace.std.longobject import W_LongObject
-from pypy.rlib.rarithmetic import (
+from pypy.rlib.rfloat import (
     formatd, DTSF_STR_PRECISION, isinf, isnan, copysign)
 
 import math

diff --git a/pypy/rlib/rfloat.py b/pypy/rlib/rfloat.py
--- a/pypy/rlib/rfloat.py
+++ b/pypy/rlib/rfloat.py
@@ -1,8 +1,11 @@
 """Float constants"""
 
+import math
 from pypy.rpython.tool import rffi_platform
 from pypy.translator.tool.cbuild import ExternalCompilationInfo
+from pypy.rlib import objectmodel
 
+USE_SHORT_FLOAT_REPR = True # XXX make it a translation option?
 
 class CConfig:
     _compilation_info_ = ExternalCompilationInfo(includes=["float.h"])
@@ -19,3 +22,368 @@
 del float_constants, int_constants, const
 
 globals().update(rffi_platform.configure(CConfig))
+
+def rstring_to_float(s):
+    if USE_SHORT_FLOAT_REPR:
+        from pypy.rlib.rdtoa import strtod
+        return strtod(s)
+
+    sign, before_point, after_point, exponent = break_up_float(s)
+
+    if not before_point and not after_point:
+        raise ValueError
+
+    return parts_to_float(sign, before_point, after_point, exponent)
+
+# float as string  -> sign, beforept, afterpt, exponent
+def break_up_float(s):
+    i = 0
+
+    sign = ''
+    before_point = ''
+    after_point = ''
+    exponent = ''
+
+    if s[i] in '+-':
+        sign = s[i]
+        i += 1
+
+    while i < len(s) and s[i] in '0123456789':
+        before_point += s[i]
+        i += 1
+
+    if i == len(s):
+        return sign, before_point, after_point, exponent
+
+    if s[i] == '.':
+        i += 1
+        while i < len(s) and s[i] in '0123456789':
+            after_point += s[i]
+            i += 1
+            
+        if i == len(s):
+            return sign, before_point, after_point, exponent
+
+    if s[i] not in  'eE':
+        raise ValueError
+
+    i += 1
+    if i == len(s):
+        raise ValueError
+
+    if s[i] in '-+':
+        exponent += s[i]
+        i += 1
+
+    if i == len(s):
+        raise ValueError
+    
+    while i < len(s) and s[i] in '0123456789':
+        exponent += s[i]
+        i += 1
+
+    if i != len(s):
+        raise ValueError
+
+    return sign, before_point, after_point, exponent
+
+# string -> float helper
+
+def parts_to_float(sign, beforept, afterpt, exponent):
+    "NOT_RPYTHON"
+    if not exponent:
+        exponent = '0'
+    return float("%s%s.%se%s" % (sign, beforept, afterpt, exponent))
+
+# float -> string
+
+DTSF_STR_PRECISION = 12
+
+DTSF_SIGN      = 0x1
+DTSF_ADD_DOT_0 = 0x2
+DTSF_ALT       = 0x4
+
+DIST_FINITE   = 1
+DIST_NAN      = 2
+DIST_INFINITY = 3
+
+# Equivalent to CPython's PyOS_double_to_string
+def _formatd(x, code, precision, flags):
+    "NOT_RPYTHON"
+    if flags & DTSF_ALT:
+        alt = '#'
+    else:
+        alt = ''
+
+    if code == 'r':
+        fmt = "%r"
+    else:
+        fmt = "%%%s.%d%s" % (alt, precision, code)
+    s = fmt % (x,)
+
+    if flags & DTSF_ADD_DOT_0:
+        # We want float numbers to be recognizable as such,
+        # i.e., they should contain a decimal point or an exponent.
+        # However, %g may print the number as an integer;
+        # in such cases, we append ".0" to the string.
+        for c in s:
+            if c in '.eE':
+                break
+        else:
+            s += '.0'
+    elif code == 'r' and s.endswith('.0'):
+        s = s[:-2]
+
+    return s
+
+def formatd(x, code, precision, flags=0):
+    if USE_SHORT_FLOAT_REPR:
+        from pypy.rlib.rdtoa import dtoa_formatd
+        return dtoa_formatd(x, code, precision, flags)
+    else:
+        return _formatd(x, code, precision, flags)
+
+def double_to_string(value, tp, precision, flags):
+    if isnan(value):
+        special = DIST_NAN
+    elif isinf(value):
+        special = DIST_INFINITY
+    else:
+        special = DIST_FINITE
+    result = formatd(value, tp, precision, flags)
+    return result, special
+
+if USE_SHORT_FLOAT_REPR:
+    def round_double(value, ndigits):
+        # The basic idea is very simple: convert and round the double to
+        # a decimal string using _Py_dg_dtoa, then convert that decimal
+        # string back to a double with _Py_dg_strtod.  There's one minor
+        # difficulty: Python 2.x expects round to do
+        # round-half-away-from-zero, while _Py_dg_dtoa does
+        # round-half-to-even.  So we need some way to detect and correct
+        # the halfway cases.
+
+        # a halfway value has the form k * 0.5 * 10**-ndigits for some
+        # odd integer k.  Or in other words, a rational number x is
+        # exactly halfway between two multiples of 10**-ndigits if its
+        # 2-valuation is exactly -ndigits-1 and its 5-valuation is at
+        # least -ndigits.  For ndigits >= 0 the latter condition is
+        # automatically satisfied for a binary float x, since any such
+        # float has nonnegative 5-valuation.  For 0 > ndigits >= -22, x
+        # needs to be an integral multiple of 5**-ndigits; we can check
+        # this using fmod.  For -22 > ndigits, there are no halfway
+        # cases: 5**23 takes 54 bits to represent exactly, so any odd
+        # multiple of 0.5 * 10**n for n >= 23 takes at least 54 bits of
+        # precision to represent exactly.
+
+        sign = copysign(1.0, value)
+        value = abs(value)
+
+        # find 2-valuation value
+        m, expo = math.frexp(value)
+        while m != math.floor(m):
+            m *= 2.0
+            expo -= 1
+
+        # determine whether this is a halfway case.
+        halfway_case = 0
+        if expo == -ndigits - 1:
+            if ndigits >= 0:
+                halfway_case = 1
+            elif ndigits >= -22:
+                # 22 is the largest k such that 5**k is exactly
+                # representable as a double
+                five_pow = 1.0
+                for i in range(-ndigits):
+                    five_pow *= 5.0
+                if math.fmod(value, five_pow) == 0.0:
+                    halfway_case = 1
+
+        # round to a decimal string; use an extra place for halfway case
+        strvalue = formatd(value, 'f', ndigits + halfway_case)
+
+        if halfway_case:
+            buf = [c for c in strvalue]
+            if ndigits >= 0:
+                endpos = len(buf) - 1
+            else:
+                endpos = len(buf) + ndigits
+            # Sanity checks: there should be exactly ndigits+1 places
+            # following the decimal point, and the last digit in the
+            # buffer should be a '5'
+            if not objectmodel.we_are_translated():
+                assert buf[endpos] == '5'
+                if '.' in buf:
+                    assert endpos == len(buf) - 1
+                    assert buf.index('.') == len(buf) - ndigits - 2
+
+            # increment and shift right at the same time
+            i = endpos - 1
+            carry = 1
+            while i >= 0:
+                digit = ord(buf[i])
+                if digit == ord('.'):
+                    buf[i+1] = chr(digit)
+                    i -= 1
+                    digit = ord(buf[i])
+
+                carry += digit - ord('0')
+                buf[i+1] = chr(carry % 10 + ord('0'))
+                carry /= 10
+                i -= 1
+            buf[0] = chr(carry + ord('0'))
+            if ndigits < 0:
+                buf.append('0')
+
+            strvalue = ''.join(buf)
+
+        return sign * rstring_to_float(strvalue)
+
+else:
+    # fallback version, to be used when correctly rounded
+    # binary<->decimal conversions aren't available
+    def round_double(value, ndigits):
+        if ndigits >= 0:
+            if ndigits > 22:
+                # pow1 and pow2 are each safe from overflow, but
+                # pow1*pow2 ~= pow(10.0, ndigits) might overflow
+                pow1 = math.pow(10.0, ndigits - 22)
+                pow2 = 1e22
+            else:
+                pow1 = math.pow(10.0, ndigits)
+                pow2 = 1.0
+
+            y = (value * pow1) * pow2
+            # if y overflows, then rounded value is exactly x
+            if isinf(y):
+                return value
+
+        else:
+            pow1 = math.pow(10.0, -ndigits);
+            pow2 = 1.0 # unused; for translation
+            y = value / pow1
+
+        if y >= 0.0:
+            z = math.floor(y + 0.5)
+        else:
+            z = math.ceil(y - 0.5)
+        if math.fabs(y-z) == 1.0:   # obscure case, see the test
+            z = y
+
+        if ndigits >= 0:
+            z = (z / pow2) / pow1
+        else:
+            z *= pow1
+        return z
+
+INFINITY = 1e200 * 1e200
+NAN = INFINITY / INFINITY
+
+try:
+    # Try to get math functions added in 2.6.
+    from math import isinf, isnan, copysign, acosh, asinh, atanh, log1p
+except ImportError:
+    def isinf(x):
+        "NOT_RPYTHON"
+        return x == INFINITY or x == -INFINITY
+
+    def isnan(v):
+        "NOT_RPYTHON"
+        return v != v
+
+    def copysign(x, y):
+        """NOT_RPYTHON. Return x with the sign of y"""
+        if x < 0.:
+            x = -x
+        if y > 0. or (y == 0. and math.atan2(y, -1.) > 0.):
+            return x
+        else:
+            return -x
+
+    _2_to_m28 = 3.7252902984619141E-09; # 2**-28
+    _2_to_p28 = 268435456.0; # 2**28
+    _ln2 = 6.93147180559945286227E-01
+
+    def acosh(x):
+        "NOT_RPYTHON"
+        if isnan(x):
+            return NAN
+        if x < 1.:
+            raise ValueError("math domain error")
+        if x >= _2_to_p28:
+            if isinf(x):
+                return x
+            else:
+                return math.log(x) + _ln2
+        if x == 1.:
+            return 0.
+        if x >= 2.:
+            t = x * x
+            return math.log(2. * x - 1. / (x + math.sqrt(t - 1.0)))
+        t = x - 1.0
+        return log1p(t + math.sqrt(2. * t + t * t))
+
+    def asinh(x):
+        "NOT_RPYTHON"
+        absx = abs(x)
+        if isnan(x) or isinf(x):
+            return x
+        if absx < _2_to_m28:
+            return x
+        if absx > _2_to_p28:
+            w = math.log(absx) + _ln2
+        elif absx > 2.:
+            w = math.log(2. * absx + 1. / (math.sqrt(x * x + 1.) + absx))
+        else:
+            t = x * x
+            w = log1p(absx + t / (1. + math.sqrt(1. + t)))
+        return copysign(w, x)
+
+    def atanh(x):
+        "NOT_RPYTHON"
+        if isnan(x):
+            return x
+        absx = abs(x)
+        if absx >= 1.:
+            raise ValueError("math domain error")
+        if absx < _2_to_m28:
+            return x
+        if absx < .5:
+            t = absx + absx
+            t = .5 * log1p(t + t * absx / (1. - absx))
+        else:
+            t = .5 * log1p((absx + absx) / (1. - absx))
+        return copysign(t, x)
+
+    def log1p(x):
+        "NOT_RPYTHON"
+        from pypy.rlib import rfloat
+        if abs(x) < rfloat.DBL_EPSILON // 2.:
+            return x
+        elif -.5 <= x <= 1.:
+            y = 1. + x
+            return math.log(y) - ((y - 1.) - x) / y
+        else:
+            return math.log(1. + x)
+
+try:
+    from math import expm1 # Added in Python 2.7.
+except ImportError:
+    def expm1(x):
+        "NOT_RPYTHON"
+        if abs(x) < .7:
+            u = math.exp(x)
+            if u == 1.:
+                return x
+            return (u - 1.) * x / math.log(u)
+        return math.exp(x) - 1.
+
+def round_away(x):
+    # round() from libm, which is not available on all platforms!
+    absx = abs(x)
+    if absx - math.floor(absx) >= .5:
+        r = math.ceil(absx)
+    else:
+        r = math.floor(absx)
+    return copysign(r, x)
+

diff --git a/pypy/objspace/std/floattype.py b/pypy/objspace/std/floattype.py
--- a/pypy/objspace/std/floattype.py
+++ b/pypy/objspace/std/floattype.py
@@ -135,14 +135,14 @@
         i += 1
         if length - i >= 2 and s[i:i + 2].lower() == "nf":
             i += 2
-            value = rarithmetic.INFINITY
+            value = rfloat.INFINITY
             if length - i >= 5 and s[i:i + 5].lower() == "inity":
                 i += 5
     elif s[i] == "n" or s[i] == "N":
         i += 1
         if length - i >= 2 and s[i:i + 2].lower() == "an":
             i += 2
-            value = rarithmetic.NAN
+            value = rfloat.NAN
     else:
         if (s[i] == "0" and length - i > 1 and
             (s[i + 1] == "x" or s[i + 1] == "X")):

diff --git a/pypy/rlib/rarithmetic.py b/pypy/rlib/rarithmetic.py
--- a/pypy/rlib/rarithmetic.py
+++ b/pypy/rlib/rarithmetic.py
@@ -33,12 +33,10 @@
 
 
 """
-import sys, math
+import sys
 from pypy.rpython import extregistry
 from pypy.rlib import objectmodel
 
-USE_SHORT_FLOAT_REPR = True # XXX make it a translation option?
-
 # set up of machine internals
 _bits = 0
 _itest = 1
@@ -60,117 +58,6 @@
     LONG_BIT_SHIFT += 1
     assert LONG_BIT_SHIFT < 99, "LONG_BIT_SHIFT value not found?"
 
-INFINITY = 1e200 * 1e200
-NAN = INFINITY / INFINITY
-
-try:
-    # Try to get math functions added in 2.6.
-    from math import isinf, isnan, copysign, acosh, asinh, atanh, log1p
-except ImportError:
-    def isinf(x):
-        "NOT_RPYTHON"
-        return x == INFINITY or x == -INFINITY
-
-    def isnan(v):
-        "NOT_RPYTHON"
-        return v != v
-
-    def copysign(x, y):
-        """NOT_RPYTHON. Return x with the sign of y"""
-        if x < 0.:
-            x = -x
-        if y > 0. or (y == 0. and math.atan2(y, -1.) > 0.):
-            return x
-        else:
-            return -x
-
-    _2_to_m28 = 3.7252902984619141E-09; # 2**-28
-    _2_to_p28 = 268435456.0; # 2**28
-    _ln2 = 6.93147180559945286227E-01
-
-    def acosh(x):
-        "NOT_RPYTHON"
-        if isnan(x):
-            return NAN
-        if x < 1.:
-            raise ValueError("math domain error")
-        if x >= _2_to_p28:
-            if isinf(x):
-                return x
-            else:
-                return math.log(x) + _ln2
-        if x == 1.:
-            return 0.
-        if x >= 2.:
-            t = x * x
-            return math.log(2. * x - 1. / (x + math.sqrt(t - 1.0)))
-        t = x - 1.0
-        return log1p(t + math.sqrt(2. * t + t * t))
-
-    def asinh(x):
-        "NOT_RPYTHON"
-        absx = abs(x)
-        if isnan(x) or isinf(x):
-            return x
-        if absx < _2_to_m28:
-            return x
-        if absx > _2_to_p28:
-            w = math.log(absx) + _ln2
-        elif absx > 2.:
-            w = math.log(2. * absx + 1. / (math.sqrt(x * x + 1.) + absx))
-        else:
-            t = x * x
-            w = log1p(absx + t / (1. + math.sqrt(1. + t)))
-        return copysign(w, x)
-
-    def atanh(x):
-        "NOT_RPYTHON"
-        if isnan(x):
-            return x
-        absx = abs(x)
-        if absx >= 1.:
-            raise ValueError("math domain error")
-        if absx < _2_to_m28:
-            return x
-        if absx < .5:
-            t = absx + absx
-            t = .5 * log1p(t + t * absx / (1. - absx))
-        else:
-            t = .5 * log1p((absx + absx) / (1. - absx))
-        return copysign(t, x)
-
-    def log1p(x):
-        "NOT_RPYTHON"
-        from pypy.rlib import rfloat
-        if abs(x) < rfloat.DBL_EPSILON // 2.:
-            return x
-        elif -.5 <= x <= 1.:
-            y = 1. + x
-            return math.log(y) - ((y - 1.) - x) / y
-        else:
-            return math.log(1. + x)
-
-try:
-    from math import expm1 # Added in Python 2.7.
-except ImportError:
-    def expm1(x):
-        "NOT_RPYTHON"
-        if abs(x) < .7:
-            u = math.exp(x)
-            if u == 1.:
-                return x
-            return (u - 1.) * x / math.log(u)
-        return math.exp(x) - 1.
-
-def round_away(x):
-    # round() from libm, which is not available on all platforms!
-    absx = abs(x)
-    if absx - math.floor(absx) >= .5:
-        r = math.ceil(absx)
-    else:
-        r = math.floor(absx)
-    return copysign(r, x)
-
 def intmask(n):
     if isinstance(n, int):
         return int(n)   # possibly bool->int
@@ -243,6 +130,7 @@
 # successfully be casted to an int.
 if sys.maxint == 2147483647:
     def ovfcheck_float_to_int(x):
+        from pypy.rlib.rfloat import isnan
         if isnan(x):
             raise OverflowError
         if -2147483649.0 < x < 2147483648.0:
@@ -253,6 +141,7 @@
     # Note the "<= x <" here, as opposed to "< x <" above.
     # This is justified by test_typed in translator/c/test.
     def ovfcheck_float_to_int(x):
+        from pypy.rlib.rfloat import isnan
         if isnan(x):
             raise OverflowError
         if -9223372036854776832.0 <= x < 9223372036854775296.0:
@@ -551,259 +440,6 @@
     r_int64 = int
 
 
-def rstring_to_float(s):
-    if USE_SHORT_FLOAT_REPR:
-        from pypy.rlib.rdtoa import strtod
-        return strtod(s)
-
-    sign, before_point, after_point, exponent = break_up_float(s)
-
-    if not before_point and not after_point:
-        raise ValueError
-
-    return parts_to_float(sign, before_point, after_point, exponent)
-
-# float as string  -> sign, beforept, afterpt, exponent
-def break_up_float(s):
-    i = 0
-
-    sign = ''
-    before_point = ''
-    after_point = ''
-    exponent = ''
-
-    if s[i] in '+-':
-        sign = s[i]
-        i += 1
-
-    while i < len(s) and s[i] in '0123456789':
-        before_point += s[i]
-        i += 1
-
-    if i == len(s):
-        return sign, before_point, after_point, exponent
-
-    if s[i] == '.':
-        i += 1
-        while i < len(s) and s[i] in '0123456789':
-            after_point += s[i]
-            i += 1
-            
-        if i == len(s):
-            return sign, before_point, after_point, exponent
-
-    if s[i] not in  'eE':
-        raise ValueError
-
-    i += 1
-    if i == len(s):
-        raise ValueError
-
-    if s[i] in '-+':
-        exponent += s[i]
-        i += 1
-
-    if i == len(s):
-        raise ValueError
-    
-    while i < len(s) and s[i] in '0123456789':
-        exponent += s[i]
-        i += 1
-
-    if i != len(s):
-        raise ValueError
-
-    return sign, before_point, after_point, exponent
-
-# string -> float helper
-
-def parts_to_float(sign, beforept, afterpt, exponent):
-    "NOT_RPYTHON"
-    if not exponent:
-        exponent = '0'
-    return float("%s%s.%se%s" % (sign, beforept, afterpt, exponent))
-
-# float -> string
-
-DTSF_STR_PRECISION = 12
-
-DTSF_SIGN      = 0x1
-DTSF_ADD_DOT_0 = 0x2
-DTSF_ALT       = 0x4
-
-DIST_FINITE   = 1
-DIST_NAN      = 2
-DIST_INFINITY = 3
-
-# Equivalent to CPython's PyOS_double_to_string
-def _formatd(x, code, precision, flags):
-    "NOT_RPYTHON"
-    if flags & DTSF_ALT:
-        alt = '#'
-    else:
-        alt = ''
-
-    if code == 'r':
-        fmt = "%r"
-    else:
-        fmt = "%%%s.%d%s" % (alt, precision, code)
-    s = fmt % (x,)
-
-    if flags & DTSF_ADD_DOT_0:
-        # We want float numbers to be recognizable as such,
-        # i.e., they should contain a decimal point or an exponent.
-        # However, %g may print the number as an integer;
-        # in such cases, we append ".0" to the string.
-        for c in s:
-            if c in '.eE':
-                break
-        else:
-            s += '.0'
-    elif code == 'r' and s.endswith('.0'):
-        s = s[:-2]
-
-    return s
-
-def formatd(x, code, precision, flags=0):
-    if USE_SHORT_FLOAT_REPR:
-        from pypy.rlib.rdtoa import dtoa_formatd
-        return dtoa_formatd(x, code, precision, flags)
-    else:
-        return _formatd(x, code, precision, flags)
-
-def double_to_string(value, tp, precision, flags):
-    if isnan(value):
-        special = DIST_NAN
-    elif isinf(value):
-        special = DIST_INFINITY
-    else:
-        special = DIST_FINITE
-    result = formatd(value, tp, precision, flags)
-    return result, special
-
-if USE_SHORT_FLOAT_REPR:
-    def round_double(value, ndigits):
-        # The basic idea is very simple: convert and round the double to
-        # a decimal string using _Py_dg_dtoa, then convert that decimal
-        # string back to a double with _Py_dg_strtod.  There's one minor
-        # difficulty: Python 2.x expects round to do
-        # round-half-away-from-zero, while _Py_dg_dtoa does
-        # round-half-to-even.  So we need some way to detect and correct
-        # the halfway cases.
-
-        # a halfway value has the form k * 0.5 * 10**-ndigits for some
-        # odd integer k.  Or in other words, a rational number x is
-        # exactly halfway between two multiples of 10**-ndigits if its
-        # 2-valuation is exactly -ndigits-1 and its 5-valuation is at
-        # least -ndigits.  For ndigits >= 0 the latter condition is
-        # automatically satisfied for a binary float x, since any such
-        # float has nonnegative 5-valuation.  For 0 > ndigits >= -22, x
-        # needs to be an integral multiple of 5**-ndigits; we can check
-        # this using fmod.  For -22 > ndigits, there are no halfway
-        # cases: 5**23 takes 54 bits to represent exactly, so any odd
-        # multiple of 0.5 * 10**n for n >= 23 takes at least 54 bits of
-        # precision to represent exactly.
-
-        sign = copysign(1.0, value)
-        value = abs(value)
-
-        # find 2-valuation value
-        m, expo = math.frexp(value)
-        while m != math.floor(m):
-            m *= 2.0
-            expo -= 1
-
-        # determine whether this is a halfway case.
-        halfway_case = 0
-        if expo == -ndigits - 1:
-            if ndigits >= 0:
-                halfway_case = 1
-            elif ndigits >= -22:
-                # 22 is the largest k such that 5**k is exactly
-                # representable as a double
-                five_pow = 1.0
-                for i in range(-ndigits):
-                    five_pow *= 5.0
-                if math.fmod(value, five_pow) == 0.0:
-                    halfway_case = 1
-
-        # round to a decimal string; use an extra place for halfway case
-        strvalue = formatd(value, 'f', ndigits + halfway_case)
-
-        if halfway_case:
-            buf = [c for c in strvalue]
-            if ndigits >= 0:
-                endpos = len(buf) - 1
-            else:
-                endpos = len(buf) + ndigits
-            # Sanity checks: there should be exactly ndigits+1 places
-            # following the decimal point, and the last digit in the
-            # buffer should be a '5'
-            if not objectmodel.we_are_translated():
-                assert buf[endpos] == '5'
-                if '.' in buf:
-                    assert endpos == len(buf) - 1
-                    assert buf.index('.') == len(buf) - ndigits - 2
-
-            # increment and shift right at the same time
-            i = endpos - 1
-            carry = 1
-            while i >= 0:
-                digit = ord(buf[i])
-                if digit == ord('.'):
-                    buf[i+1] = chr(digit)
-                    i -= 1
-                    digit = ord(buf[i])
-
-                carry += digit - ord('0')
-                buf[i+1] = chr(carry % 10 + ord('0'))
-                carry /= 10
-                i -= 1
-            buf[0] = chr(carry + ord('0'))
-            if ndigits < 0:
-                buf.append('0')
-
-            strvalue = ''.join(buf)
-
-        return sign * rstring_to_float(strvalue)
-
-else:
-    # fallback version, to be used when correctly rounded
-    # binary<->decimal conversions aren't available
-    def round_double(value, ndigits):
-        if ndigits >= 0:
-            if ndigits > 22:
-                # pow1 and pow2 are each safe from overflow, but
-                # pow1*pow2 ~= pow(10.0, ndigits) might overflow
-                pow1 = math.pow(10.0, ndigits - 22)
-                pow2 = 1e22
-            else:
-                pow1 = math.pow(10.0, ndigits)
-                pow2 = 1.0
-
-            y = (value * pow1) * pow2
-            # if y overflows, then rounded value is exactly x
-            if isinf(y):
-                return value
-
-        else:
-            pow1 = math.pow(10.0, -ndigits);
-            pow2 = 1.0 # unused; for translation
-            y = value / pow1
-
-        if y >= 0.0:
-            z = math.floor(y + 0.5)
-        else:
-            z = math.ceil(y - 0.5)
-        if math.fabs(y-z) == 1.0:   # obscure case, see the test
-            z = y
-
-        if ndigits >= 0:
-            z = (z / pow2) / pow1
-        else:
-            z *= pow1
-        return z
-
 # the 'float' C type
 
 class r_singlefloat(object):

diff --git a/pypy/rlib/test/test_rmarshal.py b/pypy/rlib/test/test_rmarshal.py
--- a/pypy/rlib/test/test_rmarshal.py
+++ b/pypy/rlib/test/test_rmarshal.py
@@ -2,7 +2,8 @@
 import marshal
 from pypy.rlib.rmarshal import *
 from pypy.annotation import model as annmodel
-from pypy.rlib.rarithmetic import formatd, LONG_BIT
+from pypy.rlib.rarithmetic import LONG_BIT
+from pypy.rlib.rfloat import formatd
 
 types_that_can_be_none = [
     [int],

diff --git a/pypy/objspace/std/newformat.py b/pypy/objspace/std/newformat.py
--- a/pypy/objspace/std/newformat.py
+++ b/pypy/objspace/std/newformat.py
@@ -3,9 +3,9 @@
 import string
 
 from pypy.interpreter.error import OperationError
-from pypy.rlib import rstring, runicode, rlocale, rarithmetic
+from pypy.rlib import rstring, runicode, rlocale, rarithmetic, rfloat
 from pypy.rlib.objectmodel import specialize
-from pypy.rlib.rarithmetic import copysign, formatd
+from pypy.rlib.rfloat import copysign, formatd
 
 
 @specialize.argtype(1)
@@ -906,7 +906,7 @@
         if tp == "\0":
             tp = "g"
             default_precision = 12
-            flags |= rarithmetic.DTSF_ADD_DOT_0
+            flags |= rfloat.DTSF_ADD_DOT_0
         elif tp == "n":
             tp = "g"
         value = space.float_w(w_float)
@@ -918,8 +918,8 @@
             add_pct = False
         if self._precision == -1:
             self._precision = default_precision
-        result, special = rarithmetic.double_to_string(value, tp,
-                                                       self._precision, flags)
+        result, special = rfloat.double_to_string(value, tp,
+                                                  self._precision, flags)
         if add_pct:
             result += "%"
         n_digits = len(result)

diff --git a/pypy/rlib/objectmodel.py b/pypy/rlib/objectmodel.py
--- a/pypy/rlib/objectmodel.py
+++ b/pypy/rlib/objectmodel.py
@@ -266,7 +266,8 @@
     except the fact that the integer case is not treated specially.
     In RPython, floats cannot be used with ints in dicts, anyway.
     """
-    from pypy.rlib.rarithmetic import intmask, isinf, isnan
+    from pypy.rlib.rarithmetic import intmask
+    from pypy.rlib.rfloat import isinf, isnan
     if isinf(f):
         if f < 0.0:
             return -271828

diff --git a/pypy/module/cmath/special_value.py b/pypy/module/cmath/special_value.py
--- a/pypy/module/cmath/special_value.py
+++ b/pypy/module/cmath/special_value.py
@@ -1,5 +1,5 @@
 import math
-from pypy.rlib.rarithmetic import isnan, isinf, copysign
+from pypy.rlib.rfloat import isnan, isinf, copysign
 
 # code to deal with special values (infinities, NaNs, ...)
 #

diff --git a/pypy/module/cmath/interp_cmath.py b/pypy/module/cmath/interp_cmath.py
--- a/pypy/module/cmath/interp_cmath.py
+++ b/pypy/module/cmath/interp_cmath.py
@@ -1,7 +1,7 @@
 import math
 from math import fabs
 from pypy.rlib.objectmodel import specialize
-from pypy.rlib.rarithmetic import copysign, asinh, log1p, isinf, isnan
+from pypy.rlib.rfloat import copysign, asinh, log1p, isinf, isnan
 from pypy.tool.sourcetools import func_with_new_name
 from pypy.interpreter.error import OperationError
 from pypy.interpreter.gateway import NoneNotWrapped

diff --git a/pypy/rpython/lltypesystem/module/test/test_llinterp_math.py b/pypy/rpython/lltypesystem/module/test/test_llinterp_math.py
--- a/pypy/rpython/lltypesystem/module/test/test_llinterp_math.py
+++ b/pypy/rpython/lltypesystem/module/test/test_llinterp_math.py
@@ -5,7 +5,7 @@
 from pypy.rpython.test.tool import BaseRtypingTest, LLRtypeMixin
 from pypy.rpython.lltypesystem.module import ll_math
 import math
-from pypy.rlib import rarithmetic
+from pypy.rlib import rfloat
 
 # XXX no OORtypeMixin here
 
@@ -15,7 +15,7 @@
             fn = getattr(math, name)
             assert_exact = True
         except AttributeError:
-            fn = getattr(rarithmetic, name)
+            fn = getattr(rfloat, name)
             assert_exact = False
         if name == 'acosh':
             value = 1.3     # acosh(x) is only defined for x >= 1.0

diff --git a/pypy/tool/uid.py b/pypy/tool/uid.py
--- a/pypy/tool/uid.py
+++ b/pypy/tool/uid.py
@@ -48,7 +48,7 @@
         # we also have to avoid confusing 0.0 and -0.0 (needed e.g. for
         # translating the cmath module)
         if key[0] is float and not self.value:
-            from pypy.rlib.rarithmetic import copysign
+            from pypy.rlib.rfloat import copysign
             if copysign(1., self.value) == -1.:    # -0.0
                 key = (float, "-0.0")
         #

diff --git a/pypy/rpython/lltypesystem/module/ll_math.py b/pypy/rpython/lltypesystem/module/ll_math.py
--- a/pypy/rpython/lltypesystem/module/ll_math.py
+++ b/pypy/rpython/lltypesystem/module/ll_math.py
@@ -8,7 +8,7 @@
 from pypy.tool.autopath import pypydir
 from pypy.rlib import rposix
 from pypy.translator.tool.cbuild import ExternalCompilationInfo
-from pypy.rlib.rarithmetic import isinf, isnan, INFINITY, NAN
+from pypy.rlib.rfloat import isinf, isnan, INFINITY, NAN
 
 if sys.platform == "win32":
     eci = ExternalCompilationInfo()

diff --git a/pypy/objspace/std/floatobject.py b/pypy/objspace/std/floatobject.py
--- a/pypy/objspace/std/floatobject.py
+++ b/pypy/objspace/std/floatobject.py
@@ -8,9 +8,10 @@
 from pypy.objspace.std.register_all import register_all
 from pypy.objspace.std.noneobject import W_NoneObject
 from pypy.objspace.std.longobject import W_LongObject
-from pypy.rlib.rarithmetic import ovfcheck_float_to_int, intmask, isinf, isnan
-from pypy.rlib.rarithmetic import (LONG_BIT, INFINITY, copysign,
-    formatd, DTSF_ADD_DOT_0, DTSF_STR_PRECISION, NAN)
+from pypy.rlib.rarithmetic import ovfcheck_float_to_int, intmask, LONG_BIT
+from pypy.rlib.rfloat import (
+    isinf, isnan, INFINITY, NAN, copysign, formatd,
+    DTSF_ADD_DOT_0, DTSF_STR_PRECISION)
 from pypy.rlib.rbigint import rbigint
 from pypy.rlib.objectmodel import we_are_translated
 from pypy.rlib import rfloat

diff --git a/pypy/rpython/module/ll_strtod.py b/pypy/rpython/module/ll_strtod.py
--- a/pypy/rpython/module/ll_strtod.py
+++ b/pypy/rpython/module/ll_strtod.py
@@ -1,7 +1,7 @@
 
 import py
 from pypy.rpython.extfunc import BaseLazyRegistering, extdef, registering
-from pypy.rlib import rarithmetic
+from pypy.rlib import rfloat
 from pypy.rpython.lltypesystem import lltype, rffi
 from pypy.tool.autopath import pypydir
 from pypy.rpython.ootypesystem import ootype
@@ -21,7 +21,7 @@
     def __init__(self):
         self.configure(CConfig)
     
-    @registering(rarithmetic._formatd)
+    @registering(rfloat._formatd)
     def register_formatd(self):
         ll_strtod = self.llexternal('LL_strtod_formatd',
                                     [rffi.DOUBLE, rffi.CHAR, rffi.INT], rffi.CCHARP,
@@ -46,11 +46,11 @@
             res = ll_strtod(x, code, precision)
             s = rffi.charp2str(res)
 
-            if flags & rarithmetic.DTSF_ADD_DOT_0:
+            if flags & rfloat.DTSF_ADD_DOT_0:
                 s = ensure_decimal_point(s, precision)
 
             # Add sign when requested
-            if flags & rarithmetic.DTSF_SIGN and s[0] != '-':
+            if flags & rfloat.DTSF_SIGN and s[0] != '-':
                 s = '+' + s
 
             # Convert to upper case
@@ -60,13 +60,13 @@
             return s
 
         def oofakeimpl(x, code, precision, flags):
-            return ootype.oostring(rarithmetic.formatd(x, code, precision, flags), -1)
+            return ootype.oostring(rfloat.formatd(x, code, precision, flags), -1)
 
         return extdef([float, lltype.Char, int, int], str, 'll_strtod.ll_strtod_formatd',
                       llimpl=llimpl, oofakeimpl=oofakeimpl,
                       sandboxsafe=True)
 
-    @registering(rarithmetic.parts_to_float)
+    @registering(rfloat.parts_to_float)
     def register_parts_to_float(self):
         ll_parts_to_float = self.llexternal('LL_strtod_parts_to_float',
                                             [rffi.CCHARP] * 4, rffi.DOUBLE,
@@ -80,8 +80,8 @@
             return res
 
         def oofakeimpl(sign, beforept, afterpt, exponent):
-            return rarithmetic.parts_to_float(sign._str, beforept._str,
-                                              afterpt._str, exponent._str)
+            return rfloat.parts_to_float(sign._str, beforept._str,
+                                         afterpt._str, exponent._str)
 
         return extdef([str, str, str, str], float,
                       'll_strtod.ll_strtod_parts_to_float', llimpl=llimpl,

diff --git a/pypy/rlib/test/test_rdtoa.py b/pypy/rlib/test/test_rdtoa.py
--- a/pypy/rlib/test/test_rdtoa.py
+++ b/pypy/rlib/test/test_rdtoa.py
@@ -1,5 +1,5 @@
 from pypy.rlib.rdtoa import strtod, dtoa
-from pypy.rlib import rarithmetic
+from pypy.rlib import rfloat
 
 def test_strtod():
     assert strtod("12345") == 12345.0
@@ -16,14 +16,14 @@
     assert dtoa(3.47) == "3.47"
     assert dtoa(1.1) == "1.1"
     assert dtoa(-1.1) == "-1.1"
-    assert dtoa(1.1, flags=rarithmetic.DTSF_SIGN) == "+1.1"
+    assert dtoa(1.1, flags=rfloat.DTSF_SIGN) == "+1.1"
     assert dtoa(12.3577) == "12.3577"
     assert dtoa(10.0) == "10"
     assert dtoa(1.0e100) == "1e+100"
 
-    assert dtoa(rarithmetic.INFINITY) == 'inf'
-    assert dtoa(-rarithmetic.INFINITY) == '-inf'
-    assert dtoa(rarithmetic.NAN) == 'nan'
+    assert dtoa(rfloat.INFINITY) == 'inf'
+    assert dtoa(-rfloat.INFINITY) == '-inf'
+    assert dtoa(rfloat.NAN) == 'nan'
 
 def test_dtoa_precision():
     assert dtoa(1.1, code='f', precision=2) == "1.10"

diff --git a/pypy/module/sys/system.py b/pypy/module/sys/system.py
--- a/pypy/module/sys/system.py
+++ b/pypy/module/sys/system.py
@@ -1,7 +1,6 @@
 """Information about the current system."""
 from pypy.interpreter import gateway
 from pypy.rlib import rfloat, rbigint
-from pypy.rlib.rarithmetic import USE_SHORT_FLOAT_REPR
 from pypy.rpython.lltypesystem import rffi
 
 
@@ -59,7 +58,7 @@
     return space.call_function(w_long_info, space.newtuple(info_w))
 
 def get_float_repr_style(space):
-    if USE_SHORT_FLOAT_REPR:
+    if rfloat.USE_SHORT_FLOAT_REPR:
         return space.wrap("short")
     else:
-        return space.wrap("legacy")
\ No newline at end of file
+        return space.wrap("legacy")

diff --git a/pypy/objspace/std/strutil.py b/pypy/objspace/std/strutil.py
--- a/pypy/objspace/std/strutil.py
+++ b/pypy/objspace/std/strutil.py
@@ -2,7 +2,8 @@
 Pure Python implementation of string utilities.
 """
 
-from pypy.rlib.rarithmetic import ovfcheck, rstring_to_float, INFINITY, NAN
+from pypy.rlib.rarithmetic import ovfcheck
+from pypy.rlib.rfloat import rstring_to_float, INFINITY, NAN
 from pypy.rlib.rbigint import rbigint, parse_digit_string
 from pypy.interpreter.error import OperationError
 import math

diff --git a/pypy/translator/c/primitive.py b/pypy/translator/c/primitive.py
--- a/pypy/translator/c/primitive.py
+++ b/pypy/translator/c/primitive.py
@@ -1,7 +1,8 @@
 import sys
 from pypy.rlib.objectmodel import Symbolic, ComputedIntSymbolic
 from pypy.rlib.objectmodel import CDefinedIntSymbolic
-from pypy.rlib.rarithmetic import r_longlong, isinf, isnan
+from pypy.rlib.rarithmetic import r_longlong
+from pypy.rlib.rfloat import isinf, isnan
 from pypy.rpython.lltypesystem.lltype import *
 from pypy.rpython.lltypesystem import rffi, llgroup
 from pypy.rpython.lltypesystem.llmemory import Address, \

diff --git a/pypy/translator/c/test/test_extfunc.py b/pypy/translator/c/test/test_extfunc.py
--- a/pypy/translator/c/test/test_extfunc.py
+++ b/pypy/translator/c/test/test_extfunc.py
@@ -262,8 +262,8 @@
     assert t0 <= res <= t1
 
 
-def test_rarith_parts_to_float():
-    from pypy.rlib.rarithmetic import parts_to_float
+def test_parts_to_float():
+    from pypy.rlib.rfloat import parts_to_float
     def fn(sign, beforept, afterpt, exponent):
         return parts_to_float(sign, beforept, afterpt, exponent)
 
@@ -281,8 +281,8 @@
     for parts, val in data:
         assert f(*parts) == val
 
-def test_rarith_formatd():
-    from pypy.rlib.rarithmetic import formatd
+def test_formatd():
+    from pypy.rlib.rfloat import formatd
     def fn(x):
         return formatd(x, 'f', 2, 0)
 
@@ -292,7 +292,7 @@
     assert f(1.5) == "1.50"
     assert f(2.0) == "2.00"
 
-def test_rarith_float_to_str():
+def test_float_to_str():
     def fn(f):
         return str(f)
     f = compile(fn, [float])

diff --git a/pypy/jit/backend/x86/test/test_zmath.py b/pypy/jit/backend/x86/test/test_zmath.py
--- a/pypy/jit/backend/x86/test/test_zmath.py
+++ b/pypy/jit/backend/x86/test/test_zmath.py
@@ -5,14 +5,14 @@
 from pypy.module.math.test import test_direct
 from pypy.translator.c.test.test_genc import compile
 from pypy.jit.backend.x86.support import ensure_sse2_floats
-from pypy.rlib import rarithmetic
+from pypy.rlib import rfloat
 
 
 def get_test_case((fnname, args, expected)):
     try:
         fn = getattr(math, fnname)
     except AttributeError:
-        fn = getattr(rarithmetic, fnname)
+        fn = getattr(rfloat, fnname)
     expect_valueerror = (expected == ValueError)
     expect_overflowerror = (expected == OverflowError)
     check = test_direct.get_tester(expected)

diff --git a/pypy/rlib/rdtoa.py b/pypy/rlib/rdtoa.py
--- a/pypy/rlib/rdtoa.py
+++ b/pypy/rlib/rdtoa.py
@@ -1,5 +1,5 @@
 from __future__ import with_statement
-from pypy.rlib import rarithmetic
+from pypy.rlib import rfloat
 from pypy.translator.tool.cbuild import ExternalCompilationInfo
 from pypy.tool.autopath import pypydir
 from pypy.rpython.lltypesystem import lltype, rffi
@@ -61,7 +61,7 @@
     if digits[0] == 'i' or digits[0] == 'I':
         if sign == 1:
             return special_strings[2]
-        elif flags & rarithmetic.DTSF_SIGN:
+        elif flags & rfloat.DTSF_SIGN:
             return special_strings[1]
         else:
             return special_strings[0]
@@ -110,9 +110,9 @@
             use_exp = True
         elif decpt > precision:
             use_exp = True
-        elif flags & rarithmetic.DTSF_ADD_DOT_0 and decpt == precision:
+        elif flags & rfloat.DTSF_ADD_DOT_0 and decpt == precision:
             use_exp = True
-        if flags & rarithmetic.DTSF_ALT:
+        if flags & rfloat.DTSF_ALT:
             vdigits_end = precision
     elif code == 'r':
         #  convert to exponential format at 1e16.  We used to convert
@@ -140,7 +140,7 @@
     else:
         vdigits_start = 0
     if vdigits_end <= decpt:
-        if not use_exp and flags & rarithmetic.DTSF_ADD_DOT_0:
+        if not use_exp and flags & rfloat.DTSF_ADD_DOT_0:
             vdigits_end = decpt + 1
         else:
             vdigits_end = decpt
@@ -153,7 +153,7 @@
 
     if sign == 1:
         builder.append('-')
-    elif flags & rarithmetic.DTSF_SIGN:
+    elif flags & rfloat.DTSF_SIGN:
         builder.append('+')
 
     # note that exactly one of the three 'if' conditions is true, so
@@ -186,7 +186,7 @@
     s = builder.build()
 
     # Delete a trailing decimal pt unless using alternative formatting.
-    if not flags & rarithmetic.DTSF_ALT:
+    if not flags & rfloat.DTSF_ALT:
         last = len(s) - 1
         if last >= 0 and s[last] == '.':
             s = s[:last]

diff --git a/pypy/rpython/lltypesystem/ll_str.py b/pypy/rpython/lltypesystem/ll_str.py
--- a/pypy/rpython/lltypesystem/ll_str.py
+++ b/pypy/rpython/lltypesystem/ll_str.py
@@ -1,6 +1,6 @@
 from pypy.rpython.lltypesystem.lltype import GcArray, Array, Char, malloc
 from pypy.rpython.annlowlevel import llstr
-from pypy.rlib.rarithmetic import r_uint, r_longlong, r_ulonglong, formatd
+from pypy.rlib.rarithmetic import r_uint, r_longlong, r_ulonglong
 
 CHAR_ARRAY = GcArray(Char)
 
@@ -126,5 +126,6 @@
 ll_int2oct._pure_function_ = True
 
 def ll_float_str(repr, f):
+    from pypy.rlib.rfloat import formatd
     return llstr(formatd(f, 'f', 6))
 ll_float_str._pure_function_ = True

diff --git a/pypy/module/math/interp_math.py b/pypy/module/math/interp_math.py
--- a/pypy/module/math/interp_math.py
+++ b/pypy/module/math/interp_math.py
@@ -1,7 +1,7 @@
 import math
 import sys
 
-from pypy.rlib import rarithmetic, unroll
+from pypy.rlib import rfloat, unroll
 from pypy.interpreter.error import OperationError
 from pypy.interpreter.gateway import NoneNotWrapped
 
@@ -67,15 +67,15 @@
     # No exceptions possible.
     x = _get_double(space, w_x)
     y = _get_double(space, w_y)
-    return space.wrap(rarithmetic.copysign(x, y))
+    return space.wrap(rfloat.copysign(x, y))
 
 def isinf(space, w_x):
     """Return True if x is infinity."""
-    return space.wrap(rarithmetic.isinf(_get_double(space, w_x)))
+    return space.wrap(rfloat.isinf(_get_double(space, w_x)))
 
 def isnan(space, w_x):
     """Return True if x is not a number."""
-    return space.wrap(rarithmetic.isnan(_get_double(space, w_x)))
+    return space.wrap(rfloat.isnan(_get_double(space, w_x)))
 
 def pow(space, w_x, w_y):
     """pow(x,y)
@@ -334,19 +334,19 @@
             v = hi
         del partials[added:]
         if v != 0.0:
-            if rarithmetic.isinf(v) or rarithmetic.isnan(v):
-                if (not rarithmetic.isinf(original) and
-                    not rarithmetic.isnan(original)):
+            if rfloat.isinf(v) or rfloat.isnan(v):
+                if (not rfloat.isinf(original) and
+                    not rfloat.isnan(original)):
                     raise OperationError(space.w_OverflowError,
                                          space.wrap("intermediate overflow"))
-                if rarithmetic.isinf(original):
+                if rfloat.isinf(original):
                     inf_sum += original
                 special_sum += original
                 del partials[:]
             else:
                 partials.append(v)
     if special_sum != 0.0:
-        if rarithmetic.isnan(special_sum):
+        if rfloat.isnan(special_sum):
             raise OperationError(space.w_ValueError, space.wrap("-inf + inf"))
         return space.wrap(special_sum)
     hi = 0.0
@@ -390,23 +390,23 @@
 
 def log1p(space, w_x):
     """Find log(x + 1)."""
-    return math1(space, rarithmetic.log1p, w_x)
+    return math1(space, rfloat.log1p, w_x)
 
 def acosh(space, w_x):
     """Inverse hyperbolic cosine"""
-    return math1(space, rarithmetic.acosh, w_x)
+    return math1(space, rfloat.acosh, w_x)
 
 def asinh(space, w_x):
     """Inverse hyperbolic sine"""
-    return math1(space, rarithmetic.asinh, w_x)
+    return math1(space, rfloat.asinh, w_x)
 
 def atanh(space, w_x):
     """Inverse hyperbolic tangent"""
-    return math1(space, rarithmetic.atanh, w_x)
+    return math1(space, rfloat.atanh, w_x)
 
 def expm1(space, w_x):
     """exp(x) - 1"""
-    return math1(space, rarithmetic.expm1, w_x)
+    return math1(space, rfloat.expm1, w_x)
 
 def erf(space, w_x):
     """The error function"""
@@ -462,7 +462,7 @@
     return p / q * x * math.exp(-x2) / _sqrtpi
 
 def _erf(x):
-    if rarithmetic.isnan(x):
+    if rfloat.isnan(x):
         return x
     absx = abs(x)
     if absx < ERF_SERIES_CUTOFF:
@@ -472,7 +472,7 @@
         return 1. - cf if x > 0. else cf - 1.
 
 def _erfc(x):
-    if rarithmetic.isnan(x):
+    if rfloat.isnan(x):
         return x
     absx = abs(x)
     if absx < ERF_SERIES_CUTOFF:
@@ -483,7 +483,7 @@
 
 def _sinpi(x):
     y = math.fmod(abs(x), 2.)
-    n = int(rarithmetic.round_away(2. * y))
+    n = int(rfloat.round_away(2. * y))
     if n == 0:
         r = math.sin(math.pi * y)
     elif n == 1:
@@ -496,7 +496,7 @@
         r = math.sin(math.pi * (y - 2.))
     else:
         raise AssertionError("should not reach")
-    return rarithmetic.copysign(1., x) * r
+    return rfloat.copysign(1., x) * r
 
 _lanczos_g = 6.024680040776729583740234375
 _lanczos_g_minus_half = 5.524680040776729583740234375
@@ -543,9 +543,9 @@
     return num / den
 
 def _gamma(x):
-    if rarithmetic.isnan(x) or (rarithmetic.isinf(x) and x > 0.):
+    if rfloat.isnan(x) or (rfloat.isinf(x) and x > 0.):
         return x
-    if rarithmetic.isinf(x):
+    if rfloat.isinf(x):
         raise ValueError("math domain error")
     if x == 0.:
         raise ValueError("math domain error")
@@ -557,7 +557,7 @@
     absx = abs(x)
     if absx < 1e-20:
         r = 1. / x
-        if rarithmetic.isinf(r):
+        if rfloat.isinf(r):
             raise OverflowError("math range error")
         return r
     if absx > 200.:
@@ -591,15 +591,15 @@
             sqrtpow = math.pow(y, absx / 2. - .25)
             r *= sqrtpow
             r *= sqrtpow
-    if rarithmetic.isinf(r):
+    if rfloat.isinf(r):
         raise OverflowError("math range error")
     return r
 
 def _lgamma(x):
-    if rarithmetic.isnan(x):
+    if rfloat.isnan(x):
         return x
-    if rarithmetic.isinf(x):
-        return rarithmetic.INFINITY
+    if rfloat.isinf(x):
+        return rfloat.INFINITY
     if x == math.floor(x) and x <= 2.:
         if x <= 0.:
             raise ValueError("math range error")
@@ -614,6 +614,6 @@
         r = (math.log(math.pi) - math.log(abs(_sinpi(absx))) - math.log(absx) -
              (math.log(_lanczos_sum(absx)) - _lanczos_g +
               (absx - .5) * (math.log(absx + _lanczos_g - .5) - 1)))
-    if rarithmetic.isinf(r):
+    if rfloat.isinf(r):
         raise OverflowError("math domain error")
     return r

diff --git a/pypy/rlib/rstruct/ieee.py b/pypy/rlib/rstruct/ieee.py
--- a/pypy/rlib/rstruct/ieee.py
+++ b/pypy/rlib/rstruct/ieee.py
@@ -4,7 +4,7 @@
 
 import math
 
-from pypy.rlib import rarithmetic, objectmodel
+from pypy.rlib import rarithmetic, rfloat, objectmodel
 from pypy.rlib.rarithmetic import r_ulonglong
 
 
@@ -58,7 +58,7 @@
 
     if exp == MAX_EXP - MIN_EXP + 2:
         # nan or infinity
-        result = rarithmetic.NAN if mant else rarithmetic.INFINITY
+        result = rfloat.NAN if mant else rfloat.INFINITY
     elif exp == 0:
         # subnormal or zero
         result = math.ldexp(mant, MIN_EXP - MANT_DIG)
@@ -86,11 +86,11 @@
     else:
         raise ValueError("invalid size value")
 
-    sign = rarithmetic.copysign(1.0, x) < 0.0
-    if rarithmetic.isinf(x):
+    sign = rfloat.copysign(1.0, x) < 0.0
+    if rfloat.isinf(x):
         mant = r_ulonglong(0)
         exp = MAX_EXP - MIN_EXP + 2
-    elif rarithmetic.isnan(x):
+    elif rfloat.isnan(x):
         mant = r_ulonglong(1) << (MANT_DIG-2) # other values possible
         exp = MAX_EXP - MIN_EXP + 2
     elif x == 0.0:

diff --git a/pypy/translator/c/node.py b/pypy/translator/c/node.py
--- a/pypy/translator/c/node.py
+++ b/pypy/translator/c/node.py
@@ -11,7 +11,7 @@
 from pypy.translator.c.support import c_char_array_constant, barebonearray
 from pypy.translator.c.primitive import PrimitiveType, name_signed
 from pypy.rlib import exports
-from pypy.rlib.rarithmetic import isinf, isnan
+from pypy.rlib.rfloat import isinf, isnan
 from pypy.rlib.rstackovf import _StackOverflow
 from pypy.translator.c import extfunc
 from pypy.translator.tool.cbuild import ExternalCompilationInfo

diff --git a/pypy/rlib/rmarshal.py b/pypy/rlib/rmarshal.py
--- a/pypy/rlib/rmarshal.py
+++ b/pypy/rlib/rmarshal.py
@@ -6,8 +6,8 @@
 from pypy.annotation.signature import annotation
 from pypy.annotation.listdef import ListDef, TooLateForChange
 from pypy.tool.pairtype import pair, pairtype
-from pypy.rlib.rarithmetic import formatd, r_longlong, intmask, LONG_BIT
-from pypy.rlib.rarithmetic import rstring_to_float
+from pypy.rlib.rarithmetic import r_longlong, intmask, LONG_BIT
+from pypy.rlib.rfloat import formatd, rstring_to_float
 from pypy.rlib.unroll import unrolling_iterable
 
 class CannotMarshal(Exception):

diff --git a/pypy/rlib/rbigint.py b/pypy/rlib/rbigint.py
--- a/pypy/rlib/rbigint.py
+++ b/pypy/rlib/rbigint.py
@@ -1,6 +1,7 @@
 from pypy.rlib.rarithmetic import LONG_BIT, intmask, r_uint, r_ulonglong
-from pypy.rlib.rarithmetic import ovfcheck, r_longlong, widen, isinf, isnan
+from pypy.rlib.rarithmetic import ovfcheck, r_longlong, widen
 from pypy.rlib.rarithmetic import most_neg_value_of_same_type
+from pypy.rlib.rfloat import isinf, isnan
 from pypy.rlib.debug import make_sure_not_resized, check_regular_int
 from pypy.rlib.objectmodel import we_are_translated
 from pypy.rpython.lltypesystem import lltype, rffi

diff --git a/pypy/rlib/test/test_rstruct.py b/pypy/rlib/test/test_rstruct.py
--- a/pypy/rlib/test/test_rstruct.py
+++ b/pypy/rlib/test/test_rstruct.py
@@ -2,7 +2,8 @@
 from pypy.rpython.test.tool import BaseRtypingTest, LLRtypeMixin, OORtypeMixin
 from pypy.rlib.rstruct.runpack import runpack
 from pypy.rlib.rstruct import ieee
-from pypy.rlib.rarithmetic import LONG_BIT, INFINITY, NAN, isnan
+from pypy.rlib.rarithmetic import LONG_BIT
+from pypy.rlib.rfloat import INFINITY, NAN, isnan
 from pypy.translator.c.test.test_genc import compile
 import struct
 

diff --git a/pypy/translator/cli/ilgenerator.py b/pypy/translator/cli/ilgenerator.py
--- a/pypy/translator/cli/ilgenerator.py
+++ b/pypy/translator/cli/ilgenerator.py
@@ -2,7 +2,7 @@
 from pypy.rpython.lltypesystem.lltype import SignedLongLong, UnsignedLongLong
 from pypy.rpython.lltypesystem import rffi
 from pypy.rlib.objectmodel import CDefinedIntSymbolic
-from pypy.rlib.rarithmetic import isnan, isinf
+from pypy.rlib.rfloat import isnan, isinf
 from pypy.rpython.ootypesystem import ootype
 from pypy.translator.oosupport.metavm import Generator
 from pypy.translator.oosupport.constant import push_constant

diff --git a/pypy/rlib/test/test_objectmodel.py b/pypy/rlib/test/test_objectmodel.py
--- a/pypy/rlib/test/test_objectmodel.py
+++ b/pypy/rlib/test/test_objectmodel.py
@@ -159,7 +159,7 @@
     assert compute_hash(None) == 0
 
 def test_compute_hash_float():
-    from pypy.rlib.rarithmetic import INFINITY, NAN
+    from pypy.rlib.rfloat import INFINITY, NAN
     assert compute_hash(INFINITY) == 314159
     assert compute_hash(-INFINITY) == -271828
     assert compute_hash(NAN) == 0
@@ -331,7 +331,7 @@
                     compute_hash(("world", None, 42, 7.5)))
             q = Foo()
             assert compute_hash(q) == compute_identity_hash(q)
-            from pypy.rlib.rarithmetic import INFINITY, NAN
+            from pypy.rlib.rfloat import INFINITY, NAN
             assert compute_hash(INFINITY) == 314159
             assert compute_hash(-INFINITY) == -271828
             assert compute_hash(NAN) == 0

diff --git a/pypy/annotation/model.py b/pypy/annotation/model.py
--- a/pypy/annotation/model.py
+++ b/pypy/annotation/model.py
@@ -34,7 +34,7 @@
 from pypy.tool.pairtype import pair, extendabletype
 from pypy.tool.tls import tlsobject
 from pypy.rlib.rarithmetic import r_uint, r_ulonglong, base_int
-from pypy.rlib.rarithmetic import r_singlefloat, r_longfloat, isnan
+from pypy.rlib.rarithmetic import r_singlefloat, r_longfloat
 import inspect, weakref
 
 DEBUG = False    # set to False to disable recording of debugging information
@@ -165,12 +165,12 @@
     def __eq__(self, other):
         if (type(self) is SomeFloat and type(other) is SomeFloat and
             self.is_constant() and other.is_constant()):
+            from pypy.rlib.rfloat import isnan, copysign
             # NaN unpleasantness.
             if isnan(self.const) and isnan(other.const):
                 return True
             # 0.0 vs -0.0 unpleasantness.
             if not self.const and not other.const:
-                from pypy.rlib.rarithmetic import copysign
                 return copysign(1., self.const) == copysign(1., other.const)
             #
         return super(SomeFloat, self).__eq__(other)

diff --git a/pypy/translator/c/test/test_math.py b/pypy/translator/c/test/test_math.py
--- a/pypy/translator/c/test/test_math.py
+++ b/pypy/translator/c/test/test_math.py
@@ -1,14 +1,14 @@
 import py, math
 from pypy.module.math.test import test_direct
 from pypy.translator.c.test.test_standalone import StandaloneTests
-from pypy.rlib import rarithmetic
+from pypy.rlib import rfloat
 
 
 def get_test_case((fnname, args, expected)):
     try:
         fn = getattr(math, fnname)
     except AttributeError:
-        fn = getattr(rarithmetic, fnname)
+        fn = getattr(rfloat, fnname)
     expect_valueerror = (expected == ValueError)
     expect_overflowerror = (expected == OverflowError)
     check = test_direct.get_tester(expected)

diff --git a/pypy/jit/backend/test/runner_test.py b/pypy/jit/backend/test/runner_test.py
--- a/pypy/jit/backend/test/runner_test.py
+++ b/pypy/jit/backend/test/runner_test.py
@@ -1263,7 +1263,7 @@
         if not self.cpu.supports_floats:
             py.test.skip("requires floats")
 
-        from pypy.rlib.rarithmetic import INFINITY, NAN, isinf, isnan
+        from pypy.rlib.rfloat import INFINITY, NAN, isinf, isnan
         from pypy.jit.metainterp.resoperation import opname
 
         fzer = boxfloat(0.0)

diff --git a/pypy/module/cmath/constant.py b/pypy/module/cmath/constant.py
--- a/pypy/module/cmath/constant.py
+++ b/pypy/module/cmath/constant.py
@@ -1,5 +1,5 @@
 import math
-from pypy.rlib.rarithmetic import isinf
+from pypy.rlib.rfloat import isinf
 from pypy.rpython.tool import rffi_platform
 from pypy.translator.tool.cbuild import ExternalCompilationInfo
 

diff --git a/pypy/interpreter/astcompiler/assemble.py b/pypy/interpreter/astcompiler/assemble.py
--- a/pypy/interpreter/astcompiler/assemble.py
+++ b/pypy/interpreter/astcompiler/assemble.py
@@ -8,7 +8,7 @@
 
 from pypy.interpreter.error import OperationError
 from pypy.rlib.objectmodel import we_are_translated
-from pypy.rlib import rarithmetic
+from pypy.rlib import rfloat
 
 
 class Instruction(object):
@@ -233,7 +233,7 @@
         w_type = space.type(obj)
         if space.is_w(w_type, space.w_float):
             val = space.float_w(obj)
-            if val == 0.0 and rarithmetic.copysign(1., val) < 0:
+            if val == 0.0 and rfloat.copysign(1., val) < 0:
                 w_key = space.newtuple([obj, space.w_float, space.w_None])
             else:
                 w_key = space.newtuple([obj, space.w_float])
@@ -243,9 +243,9 @@
             real = space.float_w(w_real)
             imag = space.float_w(w_imag)
             real_negzero = (real == 0.0 and
-                            rarithmetic.copysign(1., real) < 0)
+                            rfloat.copysign(1., real) < 0)
             imag_negzero = (imag == 0.0 and
-                            rarithmetic.copysign(1., imag) < 0)
+                            rfloat.copysign(1., imag) < 0)
             if real_negzero and imag_negzero:
                 tup = [obj, space.w_complex, space.w_None, space.w_None,
                        space.w_None]

diff --git a/pypy/rlib/test/test_rarithmetic.py b/pypy/rlib/test/test_rarithmetic.py
--- a/pypy/rlib/test/test_rarithmetic.py
+++ b/pypy/rlib/test/test_rarithmetic.py
@@ -299,25 +299,6 @@
     assert type(abs(r_longlong(1))) is r_longlong
 
 
-def test_break_up_float():
-    assert break_up_float('1') == ('', '1', '', '')
-    assert break_up_float('+1') == ('+', '1', '', '')
-    assert break_up_float('-1') == ('-', '1', '', '')
-
-    assert break_up_float('.5') == ('', '', '5', '')
-    
-    assert break_up_float('1.2e3') == ('', '1', '2', '3')
-    assert break_up_float('1.2e+3') == ('', '1', '2', '+3')
-    assert break_up_float('1.2e-3') == ('', '1', '2', '-3')
-
-    # some that will get thrown out on return:
-    assert break_up_float('.') == ('', '', '', '')
-    assert break_up_float('+') == ('+', '', '', '')
-    assert break_up_float('-') == ('-', '', '', '')
-    assert break_up_float('e1') == ('', '', '', '1')
-
-    py.test.raises(ValueError, break_up_float, 'e')
-
 def test_r_singlefloat():
     x = r_singlefloat(2.5)       # exact number
     assert float(x) == 2.5
@@ -335,65 +316,6 @@
     py.test.raises(TypeError, "x>y")
 
 class BaseTestRarithmetic(BaseRtypingTest):
-    def test_formatd(self):
-        from pypy.rlib.rarithmetic import formatd
-        def f(x):
-            return formatd(x, 'f', 2, 0)
-        res = self.ll_to_string(self.interpret(f, [10/3.0]))
-        assert res == '3.33'
-
-    def test_formatd_repr(self):
-        from pypy.rlib.rarithmetic import formatd
-        def f(x):
-            return formatd(x, 'r', 0, 0)
-        res = self.ll_to_string(self.interpret(f, [1.1]))
-        assert res == '1.1'
-
-    def test_formatd_huge(self):
-        def f(x):
-            return formatd(x, 'f', 1234, 0)
-        res = self.ll_to_string(self.interpret(f, [1.0]))
-        assert res == '1.' + 1234 * '0'
-
-    def test_formatd_F(self):
-        from pypy.translator.c.test.test_genc import compile
-        from pypy.rlib.rarithmetic import formatd
-
-        def func(x):
-            # Test the %F format, which is not supported by
-            # the Microsoft's msvcrt library.
-            return formatd(x, 'F', 4)
-
-        f = compile(func, [float])
-        assert f(10/3.0) == '3.3333'
-
-    def test_parts_to_float(self):
-        from pypy.rlib.rarithmetic import parts_to_float, break_up_float
-        def f(x):
-            if x == 0:
-                s = '1.0'
-            else:
-                s = '1e-100'
-            sign, beforept, afterpt, expt = break_up_float(s)   
-            return parts_to_float(sign, beforept, afterpt, expt)
-        res = self.interpret(f, [0])
-        assert res == 1.0
-
-        res = self.interpret(f, [1])
-        assert res == 1e-100
-
-    def test_string_to_float(self):
-        from pypy.rlib.rarithmetic import rstring_to_float
-        def func(x):
-            if x == 0:
-                s = '1e23'
-            else:
-                s = '-1e23'
-            return rstring_to_float(s)
-
-        assert self.interpret(func, [0]) == 1e23
-        assert self.interpret(func, [1]) == -1e23
-
     def test_compare_singlefloat_crashes(self):
         from pypy.rlib.rarithmetic import r_singlefloat
         from pypy.rpython.error import MissingRTypeOperation
@@ -403,28 +325,11 @@
             return a == b
         py.test.raises(MissingRTypeOperation, "self.interpret(f, [42.0])")
 
-
 class TestLLtype(BaseTestRarithmetic, LLRtypeMixin):
     pass
 
 class TestOOtype(BaseTestRarithmetic, OORtypeMixin):
-    def test_formatd(self):
-        skip('formatd is broken on ootype')
-
-    def test_formatd_repr(self):
-        skip('formatd is broken on ootype')
-
-    def test_formatd_huge(self):
-        skip('formatd is broken on ootype')
-
-    def test_string_to_float(self):
-        skip('string_to_float is broken on ootype')
-
-def test_isinf():
-    assert isinf(INFINITY)
-
-def test_isnan():
-    assert isnan(NAN)
+    pass
 
 def test_int_real_union():
     from pypy.rpython.lltypesystem.rffi import r_int_real
@@ -448,82 +353,6 @@
     for i in xrange(31):
         assert highest_bit(2**i) == i
 
-def test_copysign():
-    assert copysign(1, 1) == 1
-    assert copysign(-1, 1) == 1
-    assert copysign(-1, -1) == -1
-    assert copysign(1, -1) == -1
-    assert copysign(1, -0.) == -1
-
-def test_round_away():
-    assert round_away(.1) == 0.
-    assert round_away(.5) == 1.
-    assert round_away(.7) == 1.
-    assert round_away(1.) == 1.
-    assert round_away(-.5) == -1.
-    assert round_away(-.1) == 0.
-    assert round_away(-.7) == -1.
-    assert round_away(0.) == 0.
-
-def test_round_double():
-    def almost_equal(x, y):
-        assert round(abs(x-y), 7) == 0
-
-    almost_equal(round_double(0.125, 2), 0.13)
-    almost_equal(round_double(0.375, 2), 0.38)
-    almost_equal(round_double(0.625, 2), 0.63)
-    almost_equal(round_double(0.875, 2), 0.88)
-    almost_equal(round_double(-0.125, 2), -0.13)
-    almost_equal(round_double(-0.375, 2), -0.38)
-    almost_equal(round_double(-0.625, 2), -0.63)
-    almost_equal(round_double(-0.875, 2), -0.88)
-
-    almost_equal(round_double(0.25, 1), 0.3)
-    almost_equal(round_double(0.75, 1), 0.8)
-    almost_equal(round_double(-0.25, 1), -0.3)
-    almost_equal(round_double(-0.75, 1), -0.8)
-
-    round_double(-6.5, 0) == -7.0
-    round_double(-5.5, 0) == -6.0
-    round_double(-1.5, 0) == -2.0
-    round_double(-0.5, 0) == -1.0
-    round_double(0.5, 0) == 1.0
-    round_double(1.5, 0) == 2.0
-    round_double(2.5, 0) == 3.0
-    round_double(3.5, 0) == 4.0
-    round_double(4.5, 0) == 5.0
-    round_double(5.5, 0) == 6.0
-    round_double(6.5, 0) == 7.0
-
-    round_double(-25.0, -1) == -30.0
-    round_double(-15.0, -1) == -20.0
-    round_double(-5.0, -1) == -10.0
-    round_double(5.0, -1) == 10.0
-    round_double(15.0, -1) == 20.0
-    round_double(25.0, -1) == 30.0
-    round_double(35.0, -1) == 40.0
-    round_double(45.0, -1) == 50.0
-    round_double(55.0, -1) == 60.0
-    round_double(65.0, -1) == 70.0
-    round_double(75.0, -1) == 80.0
-    round_double(85.0, -1) == 90.0
-    round_double(95.0, -1) == 100.0
-    round_double(12325.0, -1) == 12330.0
-
-    round_double(350.0, -2) == 400.0
-    round_double(450.0, -2) == 500.0
-
-    almost_equal(round_double(0.5e21, -21), 1e21)
-    almost_equal(round_double(1.5e21, -21), 2e21)
-    almost_equal(round_double(2.5e21, -21), 3e21)
-    almost_equal(round_double(5.5e21, -21), 6e21)
-    almost_equal(round_double(8.5e21, -21), 9e21)
-
-    almost_equal(round_double(-1.5e22, -22), -2e22)
-    almost_equal(round_double(-0.5e22, -22), -1e22)
-    almost_equal(round_double(0.5e22, -22), 1e22)
-    almost_equal(round_double(1.5e22, -22), 2e22)
-
 def test_int_between():
     assert int_between(1, 1, 3)
     assert int_between(1, 2, 3)

diff --git a/pypy/rpython/extfuncregistry.py b/pypy/rpython/extfuncregistry.py
--- a/pypy/rpython/extfuncregistry.py
+++ b/pypy/rpython/extfuncregistry.py
@@ -10,7 +10,7 @@
 from pypy.rpython.ootypesystem.module import ll_math as oo_math
 from pypy.rpython.module import ll_os
 from pypy.rpython.module import ll_time
-from pypy.rlib import rarithmetic
+from pypy.rlib import rfloat
 try:
     import termios
 except ImportError:
@@ -25,18 +25,18 @@
     try:
         f = getattr(math, name)
     except AttributeError:
-        f = getattr(rarithmetic, name)
+        f = getattr(rfloat, name)
     register_external(f, [float], float,
                       export_name="ll_math.ll_math_%s" % name,
                        sandboxsafe=True, llimpl=llimpl)
 
-register_external(rarithmetic.isinf, [float], bool,
+register_external(rfloat.isinf, [float], bool,
                   export_name="ll_math.ll_math_isinf", sandboxsafe=True,
                   llimpl=ll_math.ll_math_isinf)
-register_external(rarithmetic.isnan, [float], bool,
+register_external(rfloat.isnan, [float], bool,
                   export_name="ll_math.ll_math_isnan", sandboxsafe=True,
                   llimpl=ll_math.ll_math_isnan)
-register_external(rarithmetic.copysign, [float, float], float,
+register_external(rfloat.copysign, [float, float], float,
                   export_name="ll_math.ll_math_copysign", sandboxsafe=True,
                   llimpl=ll_math.ll_math_copysign)
 

diff --git a/pypy/module/cmath/test/test_cmath.py b/pypy/module/cmath/test/test_cmath.py
--- a/pypy/module/cmath/test/test_cmath.py
+++ b/pypy/module/cmath/test/test_cmath.py
@@ -1,6 +1,6 @@
 from __future__ import with_statement
 from pypy.conftest import gettestobjspace
-from pypy.rlib.rarithmetic import copysign, isnan, isinf
+from pypy.rlib.rfloat import copysign, isnan, isinf
 from pypy.module.cmath import interp_cmath
 import os, sys, math
 

diff --git a/pypy/rpython/tool/test/test_rffi_platform.py b/pypy/rpython/tool/test/test_rffi_platform.py
--- a/pypy/rpython/tool/test/test_rffi_platform.py
+++ b/pypy/rpython/tool/test/test_rffi_platform.py
@@ -5,7 +5,8 @@
 from pypy.tool.udir import udir
 from pypy.translator.tool.cbuild import ExternalCompilationInfo
 from pypy.translator.platform import platform
-from pypy.rlib.rarithmetic import r_uint, r_longlong, r_ulonglong, isnan
+from pypy.rlib.rarithmetic import r_uint, r_longlong, r_ulonglong
+from pypy.rlib.rfloat import isnan
 
 def import_ctypes():
     try:

diff --git a/pypy/rpython/rstr.py b/pypy/rpython/rstr.py
--- a/pypy/rpython/rstr.py
+++ b/pypy/rpython/rstr.py
@@ -712,7 +712,7 @@
 
     def ll_float(ll_str):
         from pypy.rpython.annlowlevel import hlstr
-        from pypy.rlib.rarithmetic import rstring_to_float
+        from pypy.rlib.rfloat import rstring_to_float
         s = hlstr(ll_str)
         assert s is not None
 

diff --git a/pypy/translator/jvm/generator.py b/pypy/translator/jvm/generator.py
--- a/pypy/translator/jvm/generator.py
+++ b/pypy/translator/jvm/generator.py
@@ -10,7 +10,7 @@
 from pypy.translator.oosupport.function import render_sub_op
 from pypy.rpython.ootypesystem import ootype
 from pypy.rlib.objectmodel import CDefinedIntSymbolic
-from pypy.rlib.rarithmetic import isnan, isinf
+from pypy.rlib.rfloat import isnan, isinf
 from pypy.translator.oosupport.constant import push_constant
 import pypy.translator.jvm.typesystem as jvm
 

diff --git a/pypy/rpython/module/test/test_ll_strtod.py b/pypy/rpython/module/test/test_ll_strtod.py
--- a/pypy/rpython/module/test/test_ll_strtod.py
+++ b/pypy/rpython/module/test/test_ll_strtod.py
@@ -1,14 +1,14 @@
 import py
 
 from pypy.rpython.test.tool import BaseRtypingTest, LLRtypeMixin, OORtypeMixin
-from pypy.rlib import rarithmetic
+from pypy.rlib import rfloat
 
 class BaseTestStrtod(BaseRtypingTest):    
     def test_formatd(self):
         for flags in [0,
-                      rarithmetic.DTSF_ADD_DOT_0]:
+                      rfloat.DTSF_ADD_DOT_0]:
             def f(y):
-                return rarithmetic.formatd(y, 'g', 2, flags)
+                return rfloat.formatd(y, 'g', 2, flags)
 
             assert self.ll_to_string(self.interpret(f, [3.0])) == f(3.0)
 
@@ -18,7 +18,7 @@
         def f(a, b, c, d):
             a,b,c,d = hlstr(a), hlstr(b), hlstr(c), hlstr(d)
             
-            return rarithmetic.parts_to_float(a, b, c, d)
+            return rfloat.parts_to_float(a, b, c, d)
         
         data = [
         (("","1","","")     , 1.0),

diff --git a/pypy/module/__builtin__/operation.py b/pypy/module/__builtin__/operation.py
--- a/pypy/module/__builtin__/operation.py
+++ b/pypy/module/__builtin__/operation.py
@@ -7,7 +7,7 @@
 from pypy.interpreter.gateway import interp2app, unwrap_spec
 from pypy.interpreter.typedef import TypeDef
 from pypy.rlib.runicode import UNICHR
-from pypy.rlib.rarithmetic import isnan, isinf, round_double
+from pypy.rlib.rfloat import isnan, isinf, round_double
 from pypy.rlib import rfloat
 import math
 import __builtin__

diff --git a/pypy/rlib/rstruct/test/test_ieee.py b/pypy/rlib/rstruct/test/test_ieee.py
--- a/pypy/rlib/rstruct/test/test_ieee.py
+++ b/pypy/rlib/rstruct/test/test_ieee.py
@@ -2,7 +2,7 @@
 import random
 import struct
 
-from pypy.rlib.rarithmetic import isnan
+from pypy.rlib.rfloat import isnan
 from pypy.rlib.rstruct.ieee import float_pack, float_unpack
 
 

diff --git a/pypy/module/math/test/test_direct.py b/pypy/module/math/test/test_direct.py
--- a/pypy/module/math/test/test_direct.py
+++ b/pypy/module/math/test/test_direct.py
@@ -2,8 +2,8 @@
 """
 
 import py, sys, math
-from pypy.rlib import rarithmetic
-from pypy.rlib.rarithmetic import isinf, isnan, INFINITY, NAN
+from pypy.rlib import rfloat
+from pypy.rlib.rfloat import isinf, isnan, INFINITY, NAN
 
 consistent_host = True
 if '__pypy__' not in sys.builtin_module_names:
@@ -229,7 +229,7 @@
         try:
             fn = getattr(math, fnname)
         except AttributeError:
-            fn = getattr(rarithmetic, fnname)
+            fn = getattr(rfloat, fnname)
         do_test(fn, fnname, args, expected)
     #
     dict[fnname] = dict.get(fnname, 0) + 1

From commits-noreply at bitbucket.org  Mon Feb 28 10:11:11 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Mon, 28 Feb 2011 10:11:11 +0100 (CET)
Subject: [pypy-svn] pypy default: ignore pand
Message-ID: <20110228091111.5266C2A203C@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: 
Changeset: r42336:c144397c7482
Date: 2011-02-28 11:01 +0200
http://bitbucket.org/pypy/pypy/changeset/c144397c7482/

Log:	ignore pand

diff --git a/pypy/translator/c/gcc/trackgcroot.py b/pypy/translator/c/gcc/trackgcroot.py
--- a/pypy/translator/c/gcc/trackgcroot.py
+++ b/pypy/translator/c/gcc/trackgcroot.py
@@ -456,7 +456,7 @@
         'inc', 'dec', 'not', 'neg', 'or', 'and', 'sbb', 'adc',
         'shl', 'shr', 'sal', 'sar', 'rol', 'ror', 'mul', 'imul', 'div', 'idiv',
         'bswap', 'bt', 'rdtsc',
-        'punpck', 'pshufd', 'pcmp',
+        'punpck', 'pshufd', 'pcmp', 'pand',
         # zero-extending moves should not produce GC pointers
         'movz', 
         ])

From commits-noreply at bitbucket.org  Mon Feb 28 10:11:11 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Mon, 28 Feb 2011 10:11:11 +0100 (CET)
Subject: [pypy-svn] pypy default: I *think* all of those are sandboxsafe
Message-ID: <20110228091111.DDFD82A203C@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: 
Changeset: r42337:f63baa941c49
Date: 2011-02-28 11:10 +0200
http://bitbucket.org/pypy/pypy/changeset/f63baa941c49/

Log:	I *think* all of those are sandboxsafe

diff --git a/pypy/module/signal/interp_signal.py b/pypy/module/signal/interp_signal.py
--- a/pypy/module/signal/interp_signal.py
+++ b/pypy/module/signal/interp_signal.py
@@ -60,7 +60,8 @@
     globals()[k] = v
 
 def external(name, args, result, **kwds):
-    return rffi.llexternal(name, args, result, compilation_info=eci, **kwds)
+    return rffi.llexternal(name, args, result, compilation_info=eci,
+                           sandboxsafe=True, **kwds)
 
 pypysig_ignore = external('pypysig_ignore', [rffi.INT], lltype.Void)
 pypysig_default = external('pypysig_default', [rffi.INT], lltype.Void)

From commits-noreply at bitbucket.org  Mon Feb 28 11:32:31 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Mon, 28 Feb 2011 11:32:31 +0100 (CET)
Subject: [pypy-svn] pypy default: Fix jit translation on windows
Message-ID: <20110228103231.925692A203C@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r42338:4e6b9e0fa4e8
Date: 2011-02-28 10:49 +0100
http://bitbucket.org/pypy/pypy/changeset/4e6b9e0fa4e8/

Log:	Fix jit translation on windows

diff --git a/pypy/translator/c/gcc/trackgcroot.py b/pypy/translator/c/gcc/trackgcroot.py
--- a/pypy/translator/c/gcc/trackgcroot.py
+++ b/pypy/translator/c/gcc/trackgcroot.py
@@ -799,7 +799,8 @@
                 insns.append(InsnStackAdjust(int(target.rsplit('@', 1)[1])))
             # Some (intrinsic?) functions use the "fastcall" calling convention
             # XXX without any declaration, how can we guess the stack effect?
-            if target in ['__alldiv', '__allrem', '__allmul', '__alldvrm']:
+            if target in ['__alldiv', '__allrem', '__allmul', '__alldvrm',
+                          '__aulldiv', '__aullrem', '__aullmul', '__aulldvrm']:
                 insns.append(InsnStackAdjust(16))
         return insns
 

From commits-noreply at bitbucket.org  Mon Feb 28 11:32:31 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Mon, 28 Feb 2011 11:32:31 +0100 (CET)
Subject: [pypy-svn] pypy real-voidp: A branch for the C backen to render
	rffi.VOIDP as	void*
Message-ID: <20110228103231.E274A2A203D@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: real-voidp
Changeset: r42339:4773c035eb1a
Date: 2011-02-28 10:52 +0100
http://bitbucket.org/pypy/pypy/changeset/4773c035eb1a/

Log:	A branch for the C backen to render rffi.VOIDP as void*


From commits-noreply at bitbucket.org  Mon Feb 28 11:32:32 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Mon, 28 Feb 2011 11:32:32 +0100 (CET)
Subject: [pypy-svn] pypy real-voidp: The basic idea
Message-ID: <20110228103232.A540A2A203C@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: real-voidp
Changeset: r42340:2ec574e20deb
Date: 2011-02-28 10:54 +0100
http://bitbucket.org/pypy/pypy/changeset/2ec574e20deb/

Log:	The basic idea

diff --git a/pypy/rpython/lltypesystem/rffi.py b/pypy/rpython/lltypesystem/rffi.py
--- a/pypy/rpython/lltypesystem/rffi.py
+++ b/pypy/rpython/lltypesystem/rffi.py
@@ -550,8 +550,8 @@
 r_singlefloat = rarithmetic.r_singlefloat
 
 # void *   - for now, represented as char *
-VOIDP = lltype.Ptr(lltype.Array(lltype.Char, hints={'nolength': True}))
 VOIDP_real = lltype.Ptr(lltype.Array(lltype.Char, hints={'nolength': True, 'render_as_void': True}))
+VOIDP = VOIDP_real
 NULL = lltype.nullptr(VOIDP.TO)
 
 # void **

From commits-noreply at bitbucket.org  Mon Feb 28 11:32:33 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Mon, 28 Feb 2011 11:32:33 +0100 (CET)
Subject: [pypy-svn] pypy real-voidp: Fix test_rffi.py: when calling an
	external, any pointer is convertible to VOIDP
Message-ID: <20110228103233.2BA692A203D@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: real-voidp
Changeset: r42341:34bba670c8b5
Date: 2011-02-28 11:08 +0100
http://bitbucket.org/pypy/pypy/changeset/34bba670c8b5/

Log:	Fix test_rffi.py: when calling an external, any pointer is
	convertible to VOIDP

diff --git a/pypy/rpython/lltypesystem/lltype.py b/pypy/rpython/lltypesystem/lltype.py
--- a/pypy/rpython/lltypesystem/lltype.py
+++ b/pypy/rpython/lltypesystem/lltype.py
@@ -1178,6 +1178,7 @@
             return '* %s' % (self._obj0,)
 
     def __call__(self, *args):
+        from pypy.rpython.lltypesystem import rffi
         if isinstance(self._T, FuncType):
             if len(args) != len(self._T.ARGS):
                 raise TypeError,"calling %r with wrong argument number: %r" % (self._T, args)
@@ -1191,11 +1192,17 @@
                             pass
                         else:
                             assert a == value
+                    # Any pointer is convertible to void*
+                    elif (ARG is rffi.VOIDP and
+                          isinstance(typeOf(a), Ptr)):
+                        pass
                     # special case: ARG can be a container type, in which
                     # case a should be a pointer to it.  This must also be
                     # special-cased in the backends.
-                    elif not (isinstance(ARG, ContainerType)
-                            and typeOf(a) == Ptr(ARG)):
+                    elif (isinstance(ARG, ContainerType) and
+                          typeOf(a) == Ptr(ARG)):
+                        pass
+                    else:
                         args_repr = [typeOf(arg) for arg in args]
                         raise TypeError, ("calling %r with wrong argument "
                                           "types: %r" % (self._T, args_repr))

From commits-noreply at bitbucket.org  Mon Feb 28 11:32:34 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Mon, 28 Feb 2011 11:32:34 +0100 (CET)
Subject: [pypy-svn] pypy default: Skip some tests when run with the -A option
Message-ID: <20110228103234.638392A206F@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r42342:0ee3aec5d618
Date: 2011-02-28 11:29 +0100
http://bitbucket.org/pypy/pypy/changeset/0ee3aec5d618/

Log:	Skip some tests when run with the -A option

diff --git a/pypy/interpreter/test/test_module.py b/pypy/interpreter/test/test_module.py
--- a/pypy/interpreter/test/test_module.py
+++ b/pypy/interpreter/test/test_module.py
@@ -51,6 +51,8 @@
 
     def test_repr(self):
         import sys
+        if not hasattr(sys, "pypy_objspaceclass"):
+            skip("need PyPy for _pypy_interact")
         r = repr(sys)
         assert r == "<module 'sys' (built-in)>"
         

diff --git a/pypy/interpreter/test/test_mixedmodule.py b/pypy/interpreter/test/test_mixedmodule.py
--- a/pypy/interpreter/test/test_mixedmodule.py
+++ b/pypy/interpreter/test/test_mixedmodule.py
@@ -1,4 +1,5 @@
 from pypy.interpreter.mixedmodule import MixedModule
+import py.test
 
 
 class TestMixedModule(object):
@@ -31,6 +32,8 @@
         assert isinstance(self.space.builtin_modules["test_module.sub"], SubModule)
 
 class AppTestMixedModule(object):
+    pytestmark = py.test.mark.skipif("config.option.runappdirect")
+
     def setup_class(cls):
         space = cls.space
 

From commits-noreply at bitbucket.org  Mon Feb 28 17:24:14 2011
From: commits-noreply at bitbucket.org (alex_gaynor)
Date: Mon, 28 Feb 2011 17:24:14 +0100 (CET)
Subject: [pypy-svn] pypy default: Allow module kwarg on TinyObjSpace.
Message-ID: <20110228162414.830EA2A2031@codespeak.net>

Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch: 
Changeset: r42343:0823fd8d45b7
Date: 2011-02-28 11:23 -0500
http://bitbucket.org/pypy/pypy/changeset/0823fd8d45b7/

Log:	Allow module kwarg on TinyObjSpace.

diff --git a/pypy/conftest.py b/pypy/conftest.py
--- a/pypy/conftest.py
+++ b/pypy/conftest.py
@@ -150,7 +150,7 @@
     def str_w(self, w_str):
         return w_str
 
-    def newdict(self):
+    def newdict(self, module=None):
         return {}
 
     def newtuple(self, iterable):

From commits-noreply at bitbucket.org  Mon Feb 28 17:49:07 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Mon, 28 Feb 2011 17:49:07 +0100 (CET)
Subject: [pypy-svn] pypy out-of-line-guards: add movhpd to ignored ops
Message-ID: <20110228164907.3116A2A2031@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: out-of-line-guards
Changeset: r42344:badaaaabb867
Date: 2011-02-28 18:48 +0200
http://bitbucket.org/pypy/pypy/changeset/badaaaabb867/

Log:	add movhpd to ignored ops

diff --git a/pypy/translator/c/gcc/trackgcroot.py b/pypy/translator/c/gcc/trackgcroot.py
--- a/pypy/translator/c/gcc/trackgcroot.py
+++ b/pypy/translator/c/gcc/trackgcroot.py
@@ -450,7 +450,7 @@
         # floating-point operations cannot produce GC pointers
         'f',
         'cvt', 'ucomi', 'comi', 'subs', 'subp' , 'adds', 'addp', 'xorp',
-        'movap', 'movd', 'movlp', 'sqrtsd',
+        'movap', 'movd', 'movlp', 'sqrtsd', 'movhpd',
         'mins', 'minp', 'maxs', 'maxp', 'unpck', 'pxor', 'por', # sse2
         # arithmetic operations should not produce GC pointers
         'inc', 'dec', 'not', 'neg', 'or', 'and', 'sbb', 'adc',

From commits-noreply at bitbucket.org  Mon Feb 28 17:50:25 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Mon, 28 Feb 2011 17:50:25 +0100 (CET)
Subject: [pypy-svn] pypy default: Add movhpd to the list of ignored ops
Message-ID: <20110228165025.DD9F52A2031@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: 
Changeset: r42345:6f579c6ec4ec
Date: 2011-02-28 18:50 +0200
http://bitbucket.org/pypy/pypy/changeset/6f579c6ec4ec/

Log:	Add movhpd to the list of ignored ops

diff --git a/pypy/translator/c/gcc/trackgcroot.py b/pypy/translator/c/gcc/trackgcroot.py
--- a/pypy/translator/c/gcc/trackgcroot.py
+++ b/pypy/translator/c/gcc/trackgcroot.py
@@ -450,7 +450,7 @@
         # floating-point operations cannot produce GC pointers
         'f',
         'cvt', 'ucomi', 'comi', 'subs', 'subp' , 'adds', 'addp', 'xorp',
-        'movap', 'movd', 'movlp', 'sqrtsd',
+        'movap', 'movd', 'movlp', 'sqrtsd', 'movhpd',
         'mins', 'minp', 'maxs', 'maxp', 'unpck', 'pxor', 'por', # sse2
         # arithmetic operations should not produce GC pointers
         'inc', 'dec', 'not', 'neg', 'or', 'and', 'sbb', 'adc',

From commits-noreply at bitbucket.org  Mon Feb 28 17:52:28 2011
From: commits-noreply at bitbucket.org (fijal)
Date: Mon, 28 Feb 2011 17:52:28 +0100 (CET)
Subject: [pypy-svn] pypy default: couple more ops
Message-ID: <20110228165228.451322A2031@codespeak.net>

Author: Maciej Fijalkowski <fijall at gmail.com>
Branch: 
Changeset: r42346:d65bf418a793
Date: 2011-02-28 18:52 +0200
http://bitbucket.org/pypy/pypy/changeset/d65bf418a793/

Log:	couple more ops

diff --git a/pypy/translator/c/gcc/trackgcroot.py b/pypy/translator/c/gcc/trackgcroot.py
--- a/pypy/translator/c/gcc/trackgcroot.py
+++ b/pypy/translator/c/gcc/trackgcroot.py
@@ -456,7 +456,7 @@
         'inc', 'dec', 'not', 'neg', 'or', 'and', 'sbb', 'adc',
         'shl', 'shr', 'sal', 'sar', 'rol', 'ror', 'mul', 'imul', 'div', 'idiv',
         'bswap', 'bt', 'rdtsc',
-        'punpck', 'pshufd', 'pcmp', 'pand',
+        'punpck', 'pshufd', 'pcmp', 'pand', 'psslw', 'pssld', 'psslq',
         # zero-extending moves should not produce GC pointers
         'movz', 
         ])

From commits-noreply at bitbucket.org  Mon Feb 28 19:19:59 2011
From: commits-noreply at bitbucket.org (tav)
Date: Mon, 28 Feb 2011 19:19:59 +0100 (CET)
Subject: [pypy-svn] pypy default: Added a .gitignore file for the Git users.
Message-ID: <20110228181959.9963D2A2031@codespeak.net>

Author: tav <tav at espians.com>
Branch: 
Changeset: r42348:3931680de346
Date: 2011-02-28 18:19 +0000
http://bitbucket.org/pypy/pypy/changeset/3931680de346/

Log:	Added a .gitignore file for the Git users.

diff --git a/.gitignore b/.gitignore
new file mode 100644
--- /dev/null
+++ b/.gitignore
@@ -0,0 +1,14 @@
+.hg
+.svn
+
+*.pyc
+*.pyo
+*~
+
+bin/pypy-c
+include/*.h
+lib_pypy/ctypes_config_cache/_[^_]*_*.py
+pypy/_cache
+pypy/translator/goal/pypy-c
+pypy/translator/goal/target*-c
+release/
\ No newline at end of file

From commits-noreply at bitbucket.org  Mon Feb 28 20:24:37 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Mon, 28 Feb 2011 20:24:37 +0100 (CET)
Subject: [pypy-svn] pypy default: sometimes the library is a py.path.local
	object
Message-ID: <20110228192437.2F6FF2A2031@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r42349:f587142fce96
Date: 2011-02-28 13:02 +0100
http://bitbucket.org/pypy/pypy/changeset/f587142fce96/

Log:	sometimes the library is a py.path.local object

diff --git a/pypy/translator/platform/windows.py b/pypy/translator/platform/windows.py
--- a/pypy/translator/platform/windows.py
+++ b/pypy/translator/platform/windows.py
@@ -122,6 +122,7 @@
     def _libs(self, libraries):
         libs = []
         for lib in libraries:
+            lib = str(lib)
             if lib.endswith('.dll'):
                 lib = lib[:-4]
             libs.append('%s.lib' % (lib,))

From commits-noreply at bitbucket.org  Mon Feb 28 20:24:37 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Mon, 28 Feb 2011 20:24:37 +0100 (CET)
Subject: [pypy-svn] pypy real-voidp: hg merge default
Message-ID: <20110228192437.6444C2A206C@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: real-voidp
Changeset: r42350:22ffe3ad59c5
Date: 2011-02-28 13:02 +0100
http://bitbucket.org/pypy/pypy/changeset/22ffe3ad59c5/

Log:	hg merge default


From commits-noreply at bitbucket.org  Mon Feb 28 20:24:42 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Mon, 28 Feb 2011 20:24:42 +0100 (CET)
Subject: [pypy-svn] pypy real-voidp: Various rffi.VOIDP fixes,
	all tests pass in module/cpyext
Message-ID: <20110228192442.498CC2A2071@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: real-voidp
Changeset: r42351:267041210939
Date: 2011-02-28 16:39 +0100
http://bitbucket.org/pypy/pypy/changeset/267041210939/

Log:	Various rffi.VOIDP fixes, all tests pass in module/cpyext

diff --git a/pypy/module/cpyext/api.py b/pypy/module/cpyext/api.py
--- a/pypy/module/cpyext/api.py
+++ b/pypy/module/cpyext/api.py
@@ -1005,7 +1005,7 @@
     # exception checking occurs in call_external_function.
     argnames = ', '.join(['a%d' % i for i in range(len(FT.ARGS))])
     source = py.code.Source("""
-        def call_external_function(funcptr, %(argnames)s):
+        def cpy_call_external(funcptr, %(argnames)s):
             # NB. it is essential that no exception checking occurs here!
             res = funcptr(%(argnames)s)
             return res
@@ -1013,12 +1013,10 @@
     miniglobals = {'__name__':    __name__, # for module name propagation
                    }
     exec source.compile() in miniglobals
-    call_external_function = miniglobals['call_external_function']
+    call_external_function = miniglobals['cpy_call_external']
     call_external_function._dont_inline_ = True
     call_external_function._annspecialcase_ = 'specialize:ll'
     call_external_function._gctransformer_hint_close_stack_ = True
-    call_external_function = func_with_new_name(call_external_function,
-                                                'ccall_' + name)
     # don't inline, as a hack to guarantee that no GC pointer is alive
     # anywhere in call_external_function
 

diff --git a/pypy/rpython/lltypesystem/ll2ctypes.py b/pypy/rpython/lltypesystem/ll2ctypes.py
--- a/pypy/rpython/lltypesystem/ll2ctypes.py
+++ b/pypy/rpython/lltypesystem/ll2ctypes.py
@@ -1026,7 +1026,10 @@
             funcname, place))
 
     # get_ctypes_type() can raise NotImplementedError too
-    cfunc.argtypes = [get_ctypes_type(T) for T in FUNCTYPE.ARGS
+    from pypy.rpython.lltypesystem import rffi
+    cfunc.argtypes = [get_ctypes_type(T) if T is not rffi.VOIDP
+                                         else ctypes.c_void_p
+                      for T in FUNCTYPE.ARGS
                       if not T is lltype.Void]
     if FUNCTYPE.RESULT is lltype.Void:
         cfunc.restype = None

diff --git a/pypy/module/cpyext/typeobject.py b/pypy/module/cpyext/typeobject.py
--- a/pypy/module/cpyext/typeobject.py
+++ b/pypy/module/cpyext/typeobject.py
@@ -372,12 +372,27 @@
     Py_DecRef(space, pyref)
     return space.len_w(w_str)
 
+ at cpython_api([PyObject, lltype.Signed, rffi.CCHARPP], lltype.Signed,
+             external=False, error=-1)
+def str_getcharbuffer(space, w_str, segment, ref):
+    from pypy.module.cpyext.stringobject import PyString_AsString
+    if segment != 0:
+        raise OperationError(space.w_SystemError, space.wrap
+                             ("accessing non-existent string segment"))
+    pyref = make_ref(space, w_str)
+    ref[0] = PyString_AsString(space, pyref)
+    # Stolen reference: the object has better exist somewhere else
+    Py_DecRef(space, pyref)
+    return space.len_w(w_str)
+
 def setup_string_buffer_procs(space, pto):
     c_buf = lltype.malloc(PyBufferProcs, flavor='raw', zero=True)
     c_buf.c_bf_getsegcount = llhelper(str_segcount.api_func.functype,
                                       str_segcount.api_func.get_wrapper(space))
     c_buf.c_bf_getreadbuffer = llhelper(str_getreadbuffer.api_func.functype,
                                  str_getreadbuffer.api_func.get_wrapper(space))
+    c_buf.c_bf_getcharbuffer = llhelper(str_getcharbuffer.api_func.functype,
+                                 str_getcharbuffer.api_func.get_wrapper(space))
     pto.c_tp_as_buffer = c_buf
 
 @cpython_api([PyObject], lltype.Void, external=False)

diff --git a/pypy/module/cpyext/object.py b/pypy/module/cpyext/object.py
--- a/pypy/module/cpyext/object.py
+++ b/pypy/module/cpyext/object.py
@@ -399,7 +399,7 @@
                         obj, lltype.nullptr(rffi.INTP.TO)) != 1:
         raise OperationError(space.w_TypeError, space.wrap(
             "expected a single-segment buffer object"))
-    size = generic_cpy_call(space, pb.c_bf_getreadbuffer,
+    size = generic_cpy_call(space, pb.c_bf_getcharbuffer,
                             obj, 0, bufferp)
     if size < 0:
         return -1

diff --git a/pypy/rlib/rwin32.py b/pypy/rlib/rwin32.py
--- a/pypy/rlib/rwin32.py
+++ b/pypy/rlib/rwin32.py
@@ -172,7 +172,7 @@
 
     def llimpl_FormatError(code):
         "Return a message corresponding to the given Windows error code."
-        buf = lltype.malloc(rffi.VOIDPP.TO, 1, flavor='raw')
+        buf = lltype.malloc(rffi.CCHARPP.TO, 1, flavor='raw')
 
         try:
             msglen = FormatMessage(FORMAT_MESSAGE_ALLOCATE_BUFFER |
@@ -180,7 +180,7 @@
                                    None,
                                    code,
                                    DEFAULT_LANGUAGE,
-                                   rffi.cast(rffi.VOIDP, buf),
+                                   rffi.cast(rffi.CCHARP, buf),
                                    0, None)
 
             if msglen <= 2 or msglen > sys.maxint:

diff --git a/pypy/module/cpyext/test/test_stringobject.py b/pypy/module/cpyext/test/test_stringobject.py
--- a/pypy/module/cpyext/test/test_stringobject.py
+++ b/pypy/module/cpyext/test/test_stringobject.py
@@ -236,7 +236,7 @@
             api.PyString_Format(space.wrap('%s %d'), space.wrap((1, 2))))
 
     def test_asbuffer(self, space, api):
-        bufp = lltype.malloc(rffi.VOIDPP.TO, 1, flavor='raw')
+        bufp = lltype.malloc(rffi.CCHARPP.TO, 1, flavor='raw')
         lenp = lltype.malloc(Py_ssize_tP.TO, 1, flavor='raw')
 
         w_text = space.wrap("text")

diff --git a/pypy/rpython/lltypesystem/lltype.py b/pypy/rpython/lltypesystem/lltype.py
--- a/pypy/rpython/lltypesystem/lltype.py
+++ b/pypy/rpython/lltypesystem/lltype.py
@@ -1139,6 +1139,11 @@
                 raise TypeError("cannot directly assign to container array items")
             T2 = typeOf(val)
             if T2 != T1:
+                from pypy.rpython.lltypesystem import rffi
+                if T1 is rffi.VOIDP and isinstance(T2, Ptr):
+                    # Any pointer is convertible to void*
+                    val = rffi.cast(rffi.VOIDP, val)
+                else:
                     raise TypeError("%r items:\n"
                                     "expect %r\n"
                                     "   got %r" % (self._T, T1, T2))
@@ -1194,7 +1199,7 @@
                             assert a == value
                     # Any pointer is convertible to void*
                     elif (ARG is rffi.VOIDP and
-                          isinstance(typeOf(a), Ptr)):
+                          (a is None or isinstance(typeOf(a), Ptr))):
                         pass
                     # special case: ARG can be a container type, in which
                     # case a should be a pointer to it.  This must also be

From commits-noreply at bitbucket.org  Mon Feb 28 20:24:44 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Mon, 28 Feb 2011 20:24:44 +0100 (CET)
Subject: [pypy-svn] pypy real-voidp: Various translation and compilation
	fixes.
Message-ID: <20110228192444.4A1102A2072@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: real-voidp
Changeset: r42352:edd24dce9065
Date: 2011-02-28 20:21 +0100
http://bitbucket.org/pypy/pypy/changeset/edd24dce9065/

Log:	Various translation and compilation fixes.

diff --git a/pypy/translator/c/node.py b/pypy/translator/c/node.py
--- a/pypy/translator/c/node.py
+++ b/pypy/translator/c/node.py
@@ -363,7 +363,10 @@
         return self.itemindex_access_expr(baseexpr, index)
 
     def itemindex_access_expr(self, baseexpr, indexexpr):
-        return 'RPyBareItem(%s, %s)' % (baseexpr, indexexpr)
+        if self.ARRAY._hints.get("render_as_void"):
+            return 'RPyBareItem((char*)%s, %s)' % (baseexpr, indexexpr)
+        else:
+            return 'RPyBareItem(%s, %s)' % (baseexpr, indexexpr)
 
     def definition(self):
         return []    # no declaration is needed

diff --git a/pypy/rpython/lltypesystem/rffi.py b/pypy/rpython/lltypesystem/rffi.py
--- a/pypy/rpython/lltypesystem/rffi.py
+++ b/pypy/rpython/lltypesystem/rffi.py
@@ -185,6 +185,15 @@
                     # XXX leaks if a unicode2wcharp() fails with MemoryError
                     # and was not the first in this function
                     freeme = arg
+            elif TARGET is VOIDP:
+                if arg is None:
+                    arg = lltype.nullptr(VOIDP.TO)
+                elif isinstance(arg, str):
+                    arg = str2charp(arg)
+                    freeme = arg
+                elif isinstance(arg, unicode):
+                    arg = unicode2wcharp(arg)
+                    freeme = arg
             elif _isfunctype(TARGET) and not _isllptr(arg):
                 # XXX pass additional arguments
                 if invoke_around_handlers:
@@ -552,7 +561,7 @@
 # void *   - for now, represented as char *
 VOIDP_real = lltype.Ptr(lltype.Array(lltype.Char, hints={'nolength': True, 'render_as_void': True}))
 VOIDP = VOIDP_real
-NULL = lltype.nullptr(VOIDP.TO)
+NULL = None
 
 # void **
 VOIDPP = CArrayPtr(VOIDP)

diff --git a/pypy/rlib/rsocket.py b/pypy/rlib/rsocket.py
--- a/pypy/rlib/rsocket.py
+++ b/pypy/rlib/rsocket.py
@@ -1,3 +1,4 @@
+from __future__ import with_statement
 """
 An RPython implementation of sockets based on rffi.
 Note that the interface has to be slightly different - this is not
@@ -1044,21 +1045,21 @@
         self.settimeout(timeout)
 
     def setsockopt(self, level, option, value):
-        res = _c.socketsetsockopt(self.fd, level, option, value, len(value))
-        if res < 0:
-            raise self.error_handler()
+        with rffi.scoped_str2charp(value) as buf:
+            res = _c.socketsetsockopt(self.fd, level, option,
+                                      rffi.cast(rffi.VOIDP, buf),
+                                      len(value))
+            if res < 0:
+                raise self.error_handler()
 
     def setsockopt_int(self, level, option, value):
-        flag_p = lltype.malloc(rffi.INTP.TO, 1, flavor='raw')
-        try:
+        with lltype.scoped_alloc(rffi.INTP.TO, 1) as flag_p:
             flag_p[0] = rffi.cast(rffi.INT, value)
             res = _c.socketsetsockopt(self.fd, level, option,
                                       rffi.cast(rffi.VOIDP, flag_p),
                                       rffi.sizeof(rffi.INT))
-        finally:
-            lltype.free(flag_p, flavor='raw')
-        if res < 0:
-            raise self.error_handler()
+            if res < 0:
+                raise self.error_handler()
 
     def settimeout(self, timeout):
         """Set the timeout of the socket. A timeout < 0 means that

diff --git a/pypy/translator/c/funcgen.py b/pypy/translator/c/funcgen.py
--- a/pypy/translator/c/funcgen.py
+++ b/pypy/translator/c/funcgen.py
@@ -644,9 +644,17 @@
         return '%s = %s;' % (self.expr(op.result), items)
 
     def OP_DIRECT_PTRADD(self, op):
-        return '%s = %s + %s;' % (self.expr(op.result),
-                                  self.expr(op.args[0]),
-                                  self.expr(op.args[1]))
+        ARRAY = self.lltypemap(op.args[0]).TO
+        if ARRAY._hints.get("render_as_void"):
+            return '%s = (char *)%s + %s;' % (
+                self.expr(op.result), 
+                self.expr(op.args[0]),
+                self.expr(op.args[1]))
+        else:
+            return '%s = %s + %s;' % (
+                self.expr(op.result),
+                self.expr(op.args[0]),
+                self.expr(op.args[1]))
 
     def OP_CAST_POINTER(self, op):
         TYPE = self.lltypemap(op.result)

diff --git a/pypy/rpython/lltypesystem/lltype.py b/pypy/rpython/lltypesystem/lltype.py
--- a/pypy/rpython/lltypesystem/lltype.py
+++ b/pypy/rpython/lltypesystem/lltype.py
@@ -1197,9 +1197,11 @@
                             pass
                         else:
                             assert a == value
+                    # None is acceptable for any pointer
+                    elif isinstance(ARG, Ptr) and a is None:
+                        pass
                     # Any pointer is convertible to void*
-                    elif (ARG is rffi.VOIDP and
-                          (a is None or isinstance(typeOf(a), Ptr))):
+                    elif ARG is rffi.VOIDP and isinstance(typeOf(a), Ptr):
                         pass
                     # special case: ARG can be a container type, in which
                     # case a should be a pointer to it.  This must also be

From commits-noreply at bitbucket.org  Mon Feb 28 20:24:44 2011
From: commits-noreply at bitbucket.org (amauryfa)
Date: Mon, 28 Feb 2011 20:24:44 +0100 (CET)
Subject: [pypy-svn] pypy default: Merge heads
Message-ID: <20110228192444.BADD62A2074@codespeak.net>

Author: Amaury Forgeot d'Arc <amauryfa at gmail.com>
Branch: 
Changeset: r42353:28258c92cb25
Date: 2011-02-28 20:23 +0100
http://bitbucket.org/pypy/pypy/changeset/28258c92cb25/

Log:	Merge heads


From ronny at codespeak.net  Mon Feb 28 22:41:18 2011
From: ronny at codespeak.net (ronny at codespeak.net)
Date: Mon, 28 Feb 2011 22:41:18 +0100 (CET)
Subject: [pypy-svn] r80402 - pypy/extradoc/planning/hg-migration
Message-ID: <20110228214118.07B002A206D@codespeak.net>

Author: ronny
Date: Mon Feb 28 22:41:15 2011
New Revision: 80402

Modified:
   pypy/extradoc/planning/hg-migration/usermap.txt
Log:
fix 2 mail addresses

Modified: pypy/extradoc/planning/hg-migration/usermap.txt
==============================================================================
--- pypy/extradoc/planning/hg-migration/usermap.txt	(original)
+++ pypy/extradoc/planning/hg-migration/usermap.txt	Mon Feb 28 22:41:15 2011
@@ -73,12 +73,12 @@
 davide=Davide Ancona <davide at codespeak.net>
 oscar=Oscar Nierstrasz <oscar at codespeak.net>
 goden=Eugene Oden <goden at codespeak.net>
-leuschel=Michael Leuschel <leuschel at codespeak.net>
+leuschel=Michael Leuschel <leuschel at cs.uni-duesseldorf.de>
 docgok=Henry Mason <docgok at codespeak.net>
 guenter=Guenter Jantzen <guenter at codespeak.net>
 bert=Bert Freudenberg <bert at codespeak.net>
 lukas=Lukas Renggli <lukas at codespeak.net>
-lene=Lene Wagner <lene at codespeak.net>
+lene=Lene Wagner <lene at merlinux.eu>
 regmee=Amit Regmi <regmee at codespeak.net>
 adurdin=Andrew Durdin <adurdin at codespeak.net>
 benyoung=Ben Young <benyoung at codespeak.net>