Packaging

@@ -154,7 +154,7 @@
 is 2 GB unless you hack a lot), then you can (for now) tweak some parameters
 via environment variables and command-line options.  The following command
 takes a bit more time, but finishes with only using 3.0 GB of RAM (on
-Linux 64-bit; probably not much more than 1.5 GB on 32-bit).  It should be
+Linux 64-bit; probably not much more than 1.6 GB on 32-bit).  It should be
 noted that it is less than with CPython.
  PYPY_GC_MAX_DELTA=200MB pypy --jit loop_longevity=300 ./translate.py -Ojit

diff --git a/source/download.txt b/source/download.txt
--- a/source/download.txt
+++ b/source/download.txt
@@ -111,7 +111,7 @@
 
      hg clone https://bitbucket.org/pypy/pypy
 
-2. Make sure you installed the dependencies.  See the list here__.
+2. Make sure you **installed the dependencies.**  See the list here__.
 
    .. __: http://pypy.readthedocs.org/en/latest/getting-started-python.html#translating-the-pypy-python-interpreter
 

From noreply at buildbot.pypy.org  Mon Sep  5 18:44:39 2011
From: noreply at buildbot.pypy.org (justinpeel)
Date: Mon,  5 Sep 2011 18:44:39 +0200 (CEST)
Subject: [pypy-commit] pypy gc-trace-faster: some more refactoring. Not
 quite as good as I had wanted it to be,
 but it gets rid of 127/128 unnecessary checks for
 bool(self.young_rawmalloced_objects).
Message-ID: <20110905164439.4085A8203C@wyvern.cs.uni-duesseldorf.de>

Author: Justin Peel 
Branch: gc-trace-faster
Changeset: r47084:f8ddb3402976
Date: 2011-09-05 10:44 -0600
http://bitbucket.org/pypy/pypy/changeset/f8ddb3402976/

Log:	some more refactoring. Not quite as good as I had wanted it to be,
	but it gets rid of 127/128 unnecessary checks for
	bool(self.young_rawmalloced_objects).

diff --git a/pypy/rpython/memory/gc/minimark.py b/pypy/rpython/memory/gc/minimark.py
--- a/pypy/rpython/memory/gc/minimark.py
+++ b/pypy/rpython/memory/gc/minimark.py
@@ -1300,10 +1300,6 @@
     def collect_cardrefs_to_nursery(self):
         size_gc_header = self.gcheaderbuilder.size_gc_header
         oldlist = self.old_objects_with_cards_set
-        if bool(self.young_rawmalloced_objects):
-            callfunc = self.trace_and_drag_out_of_nursery_partial_young_raw
-        else:
-            callfunc = self.trace_and_drag_out_of_nursery_partial
         while oldlist.non_empty():
             obj = oldlist.pop()
             #
@@ -1351,8 +1347,12 @@
                                 ll_assert(cardbyte <= 1 and bytes == 0,
                                           "premature end of object")
                             #self.trace_and_drag_out_of_nursery_partial(
-                            callfunc(
-                                obj, interval_start, interval_stop)
+                            if bool(self.young_rawmalloced_objects):
+                                self.trace_and_drag_out_of_nursery_partial_young_raw(
+                                    obj, interval_start, interval_stop)
+                            else:
+                                self.trace_and_drag_out_of_nursery_partial(
+                                    obj, interval_start, interval_stop)
                         #
                         interval_start = interval_stop
                         cardbyte >>= 1
@@ -1363,10 +1363,6 @@
         # Follow the old_objects_pointing_to_young list and move the
         # young objects they point to out of the nursery.
         oldlist = self.old_objects_pointing_to_young
-        if bool(self.young_rawmalloced_objects):
-            trace_and_drag_out_of_nursery_func = self.trace_and_drag_out_of_nursery_young_raw
-        else:
-            trace_and_drag_out_of_nursery_func = self.trace_and_drag_out_of_nursery
         while oldlist.non_empty():
             obj = oldlist.pop()
             #
@@ -1383,7 +1379,11 @@
             # Trace the 'obj' to replace pointers to nursery with pointers
             # outside the nursery, possibly forcing nursery objects out
             # and adding them to 'old_objects_pointing_to_young' as well.
-            trace_and_drag_out_of_nursery_func(obj)
+            if bool(self.young_rawmalloced_objects):
+                self.trace_and_drag_out_of_nursery_young_raw(obj)
+            else:
+                self.trace_and_drag_out_of_nursery(obj)
+
 
     def trace_and_drag_out_of_nursery(self, obj):
         """obj must not be in the nursery.  This copies all the

From noreply at buildbot.pypy.org  Mon Sep  5 19:03:10 2011
From: noreply at buildbot.pypy.org (alex_gaynor)
Date: Mon,  5 Sep 2011 19:03:10 +0200 (CEST)
Subject: [pypy-commit] pypy default: Bytes aren't free,
	don't waste it on an extra space.
Message-ID: <20110905170310.883838203C@wyvern.cs.uni-duesseldorf.de>

Author: Alex Gaynor 
Branch: 
Changeset: r47085:baa4ae3fbdee
Date: 2011-09-04 16:54 -0700
http://bitbucket.org/pypy/pypy/changeset/baa4ae3fbdee/

Log:	Bytes aren't free, don't waste it on an extra space.

diff --git a/pypy/module/micronumpy/interp_dtype.py b/pypy/module/micronumpy/interp_dtype.py
--- a/pypy/module/micronumpy/interp_dtype.py
+++ b/pypy/module/micronumpy/interp_dtype.py
@@ -224,7 +224,7 @@
         return math.tan(v)
     @unaryop
     def arcsin(self, v):
-        if v < -1.0 or  v > 1.0:
+        if v < -1.0 or v > 1.0:
             return rfloat.NAN
         return math.asin(v)
     @unaryop

From noreply at buildbot.pypy.org  Mon Sep  5 19:03:11 2011
From: noreply at buildbot.pypy.org (alex_gaynor)
Date: Mon,  5 Sep 2011 19:03:11 +0200 (CEST)
Subject: [pypy-commit] pypy default: merged upstream.
Message-ID: <20110905170311.CD5238203C@wyvern.cs.uni-duesseldorf.de>

Author: Alex Gaynor 
Branch: 
Changeset: r47086:4d04034495ce
Date: 2011-09-05 09:12 -0700
http://bitbucket.org/pypy/pypy/changeset/4d04034495ce/

Log:	merged upstream.

diff --git a/lib_pypy/_elementtree.py b/lib_pypy/_elementtree.py
new file mode 100644
--- /dev/null
+++ b/lib_pypy/_elementtree.py
@@ -0,0 +1,6 @@
+# Just use ElementTree.
+
+from xml.etree import ElementTree
+
+globals().update(ElementTree.__dict__)
+del __all__
diff --git a/lib_pypy/stackless.py b/lib_pypy/stackless.py
--- a/lib_pypy/stackless.py
+++ b/lib_pypy/stackless.py
@@ -4,121 +4,124 @@
 Please refer to their documentation.
 """
 
-DEBUG = True
-
-def dprint(*args):
-    for arg in args:
-        print arg,
-    print
 
 import traceback
-import sys
+import _continuation
+from functools import partial
+
+class TaskletExit(Exception):
+    pass
+
+CoroutineExit = TaskletExit
+
+class GWrap(_continuation.continulet):
+    """This is just a wrapper around continulet to allow
+       to stick additional attributes to a continulet.
+       To be more concrete, we need a backreference to
+       the coroutine object"""
+
+
+class coroutine(object):
+    "we can't have continulet as a base, because continulets can't be rebound"
+
+    def __init__(self):
+        self._frame = None
+        self.is_zombie = False
+
+    def __getattr__(self, attr):
+        return getattr(self._frame, attr)
+
+    def __del__(self):
+        self.is_zombie = True
+        del self._frame
+        self._frame = None
+
+    def bind(self, func, *argl, **argd):
+        """coro.bind(f, *argl, **argd) -> None.
+           binds function f to coro. f will be called with
+           arguments *argl, **argd
+        """
+        if self._frame is None or not self._frame.is_pending():
+
+            def _func(c, *args, **kwargs):
+                return func(*args, **kwargs)
+            
+            run = partial(_func, *argl, **argd)
+            self._frame = frame = GWrap(run)
+        else:
+            raise ValueError("cannot bind a bound coroutine")
+
+    def switch(self):
+        """coro.switch() -> returnvalue
+           switches to coroutine coro. If the bound function
+           f finishes, the returnvalue is that of f, otherwise
+           None is returned
+        """
+        current = _getcurrent()
+        current._jump_to(self)
+
+    def _jump_to(self, coroutine):
+        _tls.current_coroutine = coroutine
+        self._frame.switch(to=coroutine._frame)
+
+    def kill(self):
+        """coro.kill() : kill coroutine coro"""
+        _tls.current_coroutine = self
+        self._frame.throw(CoroutineExit)
+
+    def _is_alive(self):
+        if self._frame is None:
+            return False
+        return not self._frame.is_pending()
+    is_alive = property(_is_alive)
+    del _is_alive
+
+    def getcurrent():
+        """coroutine.getcurrent() -> the currently running coroutine"""
+        try:
+            return _getcurrent()
+        except AttributeError:
+            return _maincoro
+    getcurrent = staticmethod(getcurrent)
+
+    def __reduce__(self):
+        raise TypeError, 'pickling is not possible based upon continulets'
+
+
+def _getcurrent():
+    "Returns the current coroutine (i.e. the one which called this function)."
+    try:
+        return _tls.current_coroutine
+    except AttributeError:
+        # first call in this thread: current == main
+        _coroutine_create_main()
+        return _tls.current_coroutine
+
 try:
-    # If _stackless can be imported then TaskletExit and CoroutineExit are 
-    # automatically added to the builtins.
-    from _stackless import coroutine, greenlet
-except ImportError: # we are running from CPython
-    from greenlet import greenlet, GreenletExit
-    TaskletExit = CoroutineExit = GreenletExit
-    del GreenletExit
-    try:
-        from functools import partial
-    except ImportError: # we are not running python 2.5
-        class partial(object):
-            # just enough of 'partial' to be usefull
-            def __init__(self, func, *argl, **argd):
-                self.func = func
-                self.argl = argl
-                self.argd = argd
+    from thread import _local
+except ImportError:
+    class _local(object):    # assume no threads
+        pass
 
-            def __call__(self):
-                return self.func(*self.argl, **self.argd)
+_tls = _local()
 
-    class GWrap(greenlet):
-        """This is just a wrapper around greenlets to allow
-           to stick additional attributes to a greenlet.
-           To be more concrete, we need a backreference to
-           the coroutine object"""
+def _coroutine_create_main():
+    # create the main coroutine for this thread
+    _tls.current_coroutine = None
+    main_coroutine = coroutine()
+    main_coroutine.bind(lambda x:x)
+    _tls.main_coroutine = main_coroutine
+    _tls.current_coroutine = main_coroutine
+    return main_coroutine
 
-    class MWrap(object):
-        def __init__(self,something):
-            self.something = something
 
-        def __getattr__(self, attr):
-            return getattr(self.something, attr)
+_maincoro = _coroutine_create_main()
 
-    class coroutine(object):
-        "we can't have greenlet as a base, because greenlets can't be rebound"
-
-        def __init__(self):
-            self._frame = None
-            self.is_zombie = False
-
-        def __getattr__(self, attr):
-            return getattr(self._frame, attr)
-
-        def __del__(self):
-            self.is_zombie = True
-            del self._frame
-            self._frame = None
-
-        def bind(self, func, *argl, **argd):
-            """coro.bind(f, *argl, **argd) -> None.
-               binds function f to coro. f will be called with
-               arguments *argl, **argd
-            """
-            if self._frame is None or self._frame.dead:
-                self._frame = frame = GWrap()
-                frame.coro = self
-            if hasattr(self._frame, 'run') and self._frame.run:
-                raise ValueError("cannot bind a bound coroutine")
-            self._frame.run = partial(func, *argl, **argd)
-
-        def switch(self):
-            """coro.switch() -> returnvalue
-               switches to coroutine coro. If the bound function
-               f finishes, the returnvalue is that of f, otherwise
-               None is returned
-            """
-            try:
-                return greenlet.switch(self._frame)
-            except TypeError, exp: # self._frame is the main coroutine
-                return greenlet.switch(self._frame.something)
-
-        def kill(self):
-            """coro.kill() : kill coroutine coro"""
-            self._frame.throw()
-
-        def _is_alive(self):
-            if self._frame is None:
-                return False
-            return not self._frame.dead
-        is_alive = property(_is_alive)
-        del _is_alive
-
-        def getcurrent():
-            """coroutine.getcurrent() -> the currently running coroutine"""
-            try:
-                return greenlet.getcurrent().coro
-            except AttributeError:
-                return _maincoro
-        getcurrent = staticmethod(getcurrent)
-
-        def __reduce__(self):
-            raise TypeError, 'pickling is not possible based upon greenlets'
-
-    _maincoro = coroutine()
-    maingreenlet = greenlet.getcurrent()
-    _maincoro._frame = frame = MWrap(maingreenlet)
-    frame.coro = _maincoro
-    del frame
-    del maingreenlet
 
 from collections import deque
 
 import operator
-__all__ = 'run getcurrent getmain schedule tasklet channel coroutine \
-                greenlet'.split()
+__all__ = 'run getcurrent getmain schedule tasklet channel coroutine'.split()
 
 _global_task_id = 0
 _squeue = None
@@ -131,7 +134,8 @@
 def _scheduler_remove(value):
     try:
         del _squeue[operator.indexOf(_squeue, value)]
-    except ValueError:pass
+    except ValueError:
+        pass
 
 def _scheduler_append(value, normal=True):
     if normal:
diff --git a/pypy/jit/backend/llsupport/regalloc.py b/pypy/jit/backend/llsupport/regalloc.py
--- a/pypy/jit/backend/llsupport/regalloc.py
+++ b/pypy/jit/backend/llsupport/regalloc.py
@@ -57,11 +57,13 @@
     all_regs              = []
     no_lower_byte_regs    = []
     save_around_call_regs = []
-    
+    frame_reg             = None
+
     def __init__(self, longevity, frame_manager=None, assembler=None):
         self.free_regs = self.all_regs[:]
         self.longevity = longevity
         self.reg_bindings = {}
+        self.bindings_to_frame_reg = {}
         self.position = -1
         self.frame_manager = frame_manager
         self.assembler = assembler
@@ -218,6 +220,10 @@
         self.reg_bindings[v] = loc
         return loc
 
+    def force_allocate_frame_reg(self, v):
+        """ Allocate the new variable v in the frame register."""
+        self.bindings_to_frame_reg[v] = None
+
     def force_spill_var(self, var):
         self._sync_var(var)
         try:
@@ -236,6 +242,8 @@
         try:
             return self.reg_bindings[box]
         except KeyError:
+            if box in self.bindings_to_frame_reg:
+                return self.frame_reg
             return self.frame_manager.loc(box)
 
     def return_constant(self, v, forbidden_vars=[], selected_reg=None):
@@ -264,8 +272,9 @@
         self._check_type(v)
         if isinstance(v, Const):
             return self.return_constant(v, forbidden_vars, selected_reg)
-        
         prev_loc = self.loc(v)
+        if prev_loc is self.frame_reg and selected_reg is None:
+            return prev_loc
         loc = self.force_allocate_reg(v, forbidden_vars, selected_reg,
                                       need_lower_byte=need_lower_byte)
         if prev_loc is not loc:
diff --git a/pypy/jit/backend/x86/assembler.py b/pypy/jit/backend/x86/assembler.py
--- a/pypy/jit/backend/x86/assembler.py
+++ b/pypy/jit/backend/x86/assembler.py
@@ -957,6 +957,7 @@
         if (isinstance(from_loc, RegLoc) and from_loc.is_xmm) or (isinstance(to_loc, RegLoc) and to_loc.is_xmm):
             self.mc.MOVSD(to_loc, from_loc)
         else:
+            assert to_loc is not ebp
             self.mc.MOV(to_loc, from_loc)
 
     regalloc_mov = mov # legacy interface
@@ -2510,11 +2511,6 @@
 
     genop_discard_cond_call_gc_wb_array = genop_discard_cond_call_gc_wb
 
-    def genop_force_token(self, op, arglocs, resloc):
-        # RegAlloc.consider_force_token ensures this:
-        assert isinstance(resloc, RegLoc)
-        self.mc.LEA_rb(resloc.value, FORCE_INDEX_OFS)
-
     def not_implemented_op_discard(self, op, arglocs):
         not_implemented("not implemented operation: %s" % op.getopname())
 
diff --git a/pypy/jit/backend/x86/regalloc.py b/pypy/jit/backend/x86/regalloc.py
--- a/pypy/jit/backend/x86/regalloc.py
+++ b/pypy/jit/backend/x86/regalloc.py
@@ -29,6 +29,7 @@
     all_regs = [eax, ecx, edx, ebx, esi, edi]
     no_lower_byte_regs = [esi, edi]
     save_around_call_regs = [eax, edx, ecx]
+    frame_reg = ebp
 
     REGLOC_TO_GCROOTMAP_REG_INDEX = {
         ebx: 1,
@@ -312,8 +313,11 @@
                     self.fm.frame_bindings[arg] = loc
             else:
                 if isinstance(loc, RegLoc):
-                    self.rm.reg_bindings[arg] = loc
-                    used[loc] = None
+                    if loc is ebp:
+                        self.rm.bindings_to_frame_reg[arg] = None
+                    else:
+                        self.rm.reg_bindings[arg] = loc
+                        used[loc] = None
                 else:
                     self.fm.frame_bindings[arg] = loc
         self.rm.free_regs = []
@@ -1358,8 +1362,8 @@
                                             self.assembler.datablockwrapper)
 
     def consider_force_token(self, op):
-        loc = self.rm.force_allocate_reg(op.result)
-        self.Perform(op, [], loc)
+        # the FORCE_TOKEN operation returns directly 'ebp'
+        self.rm.force_allocate_frame_reg(op.result)
 
     def not_implemented_op(self, op):
         not_implemented("not implemented operation: %s" % op.getopname())
diff --git a/pypy/jit/backend/x86/runner.py b/pypy/jit/backend/x86/runner.py
--- a/pypy/jit/backend/x86/runner.py
+++ b/pypy/jit/backend/x86/runner.py
@@ -119,7 +119,8 @@
             setitem(index, null)
 
     def get_latest_force_token(self):
-        return self.assembler.fail_ebp + FORCE_INDEX_OFS
+        # the FORCE_TOKEN operation and this helper both return 'ebp'.
+        return self.assembler.fail_ebp
 
     def execute_token(self, executable_token):
         addr = executable_token._x86_bootstrap_code
@@ -153,8 +154,9 @@
                                        flavor='raw', zero=True,
                                        immortal=True)
 
-    def force(self, addr_of_force_index):
+    def force(self, addr_of_force_token):
         TP = rffi.CArrayPtr(lltype.Signed)
+        addr_of_force_index = addr_of_force_token + FORCE_INDEX_OFS
         fail_index = rffi.cast(TP, addr_of_force_index)[0]
         assert fail_index >= 0, "already forced!"
         faildescr = self.get_fail_descr_from_number(fail_index)
@@ -164,7 +166,7 @@
         # start of "no gc operation!" block
         fail_index_2 = self.assembler.grab_frame_values(
             bytecode,
-            addr_of_force_index - FORCE_INDEX_OFS,
+            addr_of_force_token,
             self.all_null_registers)
         self.assembler.leave_jitted_hook()
         # end of "no gc operation!" block
diff --git a/pypy/objspace/std/test/test_methodcache.py b/pypy/objspace/std/test/test_methodcache.py
--- a/pypy/objspace/std/test/test_methodcache.py
+++ b/pypy/objspace/std/test/test_methodcache.py
@@ -88,30 +88,37 @@
   
     def test_many_names(self):
         import __pypy__
-        class A(object):
-            foo = 5
-            bar = 6
-            baz = 7
-            xyz = 8
-            stuff = 9
-            a = 10
-            foobar = 11
+        for j in range(20):
+            class A(object):
+                foo = 5
+                bar = 6
+                baz = 7
+                xyz = 8
+                stuff = 9
+                a = 10
+                foobar = 11
 
-        a = A()
-        names = [name for name in A.__dict__.keys()
-                      if not name.startswith('_')]
-        names.sort()
-        names_repeated = names * 10
-        result = []
-        __pypy__.reset_method_cache_counter()
-        for name in names_repeated:
-            result.append(getattr(a, name))
-        append_counter = __pypy__.method_cache_counter("append")
-        names_counters = [__pypy__.method_cache_counter(name)
-                          for name in names]
-        assert append_counter[0] >= 5 * len(names)
-        for name, count in zip(names, names_counters):
-            assert count[0] >= 5, str((name, count))
+            a = A()
+            names = [name for name in A.__dict__.keys()
+                          if not name.startswith('_')]
+            names.sort()
+            names_repeated = names * 10
+            result = []
+            __pypy__.reset_method_cache_counter()
+            for name in names_repeated:
+                result.append(getattr(a, name))
+            append_counter = __pypy__.method_cache_counter("append")
+            names_counters = [__pypy__.method_cache_counter(name)
+                              for name in names]
+            try:
+                assert append_counter[0] >= 10 * len(names) - 1
+                for name, count in zip(names, names_counters):
+                    assert count == (9, 1), str((name, count))
+                break
+            except AssertionError:
+                pass
+        else:
+            raise
 
     def test_mutating_bases(self):
         class C(object):
diff --git a/pypy/rpython/module/ll_os_stat.py b/pypy/rpython/module/ll_os_stat.py
--- a/pypy/rpython/module/ll_os_stat.py
+++ b/pypy/rpython/module/ll_os_stat.py
@@ -49,19 +49,8 @@
     ]
 N_INDEXABLE_FIELDS = 10
 
-# for now, check the host Python to know which st_xxx fields exist
-STAT_FIELDS = [(_name, _TYPE) for (_name, _TYPE) in ALL_STAT_FIELDS
-                              if hasattr(os.stat_result, _name)]
-
-STAT_FIELD_TYPES = dict(STAT_FIELDS)      # {'st_xxx': TYPE}
-
-STAT_FIELD_NAMES = [_name for (_name, _TYPE) in ALL_STAT_FIELDS
-                          if _name in STAT_FIELD_TYPES]
-
-del _name, _TYPE
-
 # For OO backends, expose only the portable fields (the first 10).
-PORTABLE_STAT_FIELDS = STAT_FIELDS[:N_INDEXABLE_FIELDS]
+PORTABLE_STAT_FIELDS = ALL_STAT_FIELDS[:N_INDEXABLE_FIELDS]
 
 # ____________________________________________________________
 #
@@ -142,17 +131,22 @@
     includes = INCLUDES
 )
 
-if sys.platform != 'win32':
+if TIMESPEC is not None:
+    class CConfig_for_timespec:
+        _compilation_info_ = compilation_info
+        TIMESPEC = TIMESPEC
+    TIMESPEC = lltype.Ptr(
+        platform.configure(CConfig_for_timespec)['TIMESPEC'])
+
+
+def posix_declaration(try_to_add=None):
+    global STAT_STRUCT
 
     LL_STAT_FIELDS = STAT_FIELDS[:]
+    if try_to_add:
+        LL_STAT_FIELDS.append(try_to_add)
 
     if TIMESPEC is not None:
-        class CConfig_for_timespec:
-            _compilation_info_ = compilation_info
-            TIMESPEC = TIMESPEC
-
-        TIMESPEC = lltype.Ptr(
-            platform.configure(CConfig_for_timespec)['TIMESPEC'])
 
         def _expand(lst, originalname, timespecname):
             for i, (_name, _TYPE) in enumerate(lst):
@@ -178,9 +172,34 @@
     class CConfig:
         _compilation_info_ = compilation_info
         STAT_STRUCT = platform.Struct('struct %s' % _name_struct_stat, LL_STAT_FIELDS)
-    config = platform.configure(CConfig)
+    try:
+        config = platform.configure(CConfig)
+    except platform.CompilationError:
+        if try_to_add:
+            return    # failed to add this field, give up
+        raise
 
     STAT_STRUCT = lltype.Ptr(config['STAT_STRUCT'])
+    if try_to_add:
+        STAT_FIELDS.append(try_to_add)
+
+
+# This lists only the fields that have been found on the underlying platform.
+# Initially only the PORTABLE_STAT_FIELDS, but more may be added by the
+# following loop.
+STAT_FIELDS = PORTABLE_STAT_FIELDS[:]
+
+if sys.platform != 'win32':
+    posix_declaration()
+    for _i in range(len(PORTABLE_STAT_FIELDS), len(ALL_STAT_FIELDS)):
+        posix_declaration(ALL_STAT_FIELDS[_i])
+    del _i
+
+# these two global vars only list the fields defined in the underlying platform
+STAT_FIELD_TYPES = dict(STAT_FIELDS)      # {'st_xxx': TYPE}
+STAT_FIELD_NAMES = [_name for (_name, _TYPE) in STAT_FIELDS]
+del _name, _TYPE
+
 
 def build_stat_result(st):
     # only for LL backends
diff --git a/pypy/rpython/module/test/test_ll_os_stat.py b/pypy/rpython/module/test/test_ll_os_stat.py
--- a/pypy/rpython/module/test/test_ll_os_stat.py
+++ b/pypy/rpython/module/test/test_ll_os_stat.py
@@ -2,6 +2,16 @@
 import sys, os
 import py
 
+
+class TestLinuxImplementation:
+    def setup_class(cls):
+        if not sys.platform.startswith('linux'):
+            py.test.skip("linux specific tests")
+
+    def test_has_all_fields(self):
+        assert ll_os_stat.STAT_FIELDS == ll_os_stat.ALL_STAT_FIELDS[:13]
+
+
 class TestWin32Implementation:
     def setup_class(cls):
         if sys.platform != 'win32':

From noreply at buildbot.pypy.org  Mon Sep  5 19:03:13 2011
From: noreply at buildbot.pypy.org (alex_gaynor)
Date: Mon,  5 Sep 2011 19:03:13 +0200 (CEST)
Subject: [pypy-commit] pypy default: (snus,
 alex) Added the comparison functions to micronumpy. This is mostly
 the work from the numpy-comparisons branch, refactored by me.
Message-ID: <20110905170313.140378203C@wyvern.cs.uni-duesseldorf.de>

Author: Alex Gaynor 
Branch: 
Changeset: r47087:618b0bba96a2
Date: 2011-09-05 10:02 -0700
http://bitbucket.org/pypy/pypy/changeset/618b0bba96a2/

Log:	(snus, alex) Added the comparison functions to micronumpy. This is
	mostly the work from the numpy-comparisons branch, refactored by me.

diff --git a/pypy/module/micronumpy/__init__.py b/pypy/module/micronumpy/__init__.py
--- a/pypy/module/micronumpy/__init__.py
+++ b/pypy/module/micronumpy/__init__.py
@@ -26,13 +26,19 @@
         ("copysign", "copysign"),
         ("cos", "cos"),
         ("divide", "divide"),
+        ("equal", "equal"),
         ("exp", "exp"),
         ("fabs", "fabs"),
         ("floor", "floor"),
+        ("greater", "greater"),
+        ("greater_equal", "greater_equal"),
+        ("less", "less"),
+        ("less_equal", "less_equal"),
         ("maximum", "maximum"),
         ("minimum", "minimum"),
         ("multiply", "multiply"),
         ("negative", "negative"),
+        ("not_equal", "not_equal"),
         ("reciprocal", "reciprocal"),
         ("sign", "sign"),
         ("sin", "sin"),
diff --git a/pypy/module/micronumpy/interp_dtype.py b/pypy/module/micronumpy/interp_dtype.py
--- a/pypy/module/micronumpy/interp_dtype.py
+++ b/pypy/module/micronumpy/interp_dtype.py
@@ -129,6 +129,16 @@
         ))
     return impl
 
+def raw_binop(func):
+    # Returns the result unwrapped.
+    @functools.wraps(func)
+    def impl(self, v1, v2):
+        return func(self,
+            self.for_computation(self.unbox(v1)),
+            self.for_computation(self.unbox(v2))
+        )
+    return impl
+
 def unaryop(func):
     @functools.wraps(func)
     def impl(self, v):
@@ -170,8 +180,24 @@
 
     def bool(self, v):
         return bool(self.for_computation(self.unbox(v)))
+    @raw_binop
+    def eq(self, v1, v2):
+        return v1 == v2
+    @raw_binop
     def ne(self, v1, v2):
-        return self.for_computation(self.unbox(v1)) != self.for_computation(self.unbox(v2))
+        return v1 != v2
+    @raw_binop
+    def lt(self, v1, v2):
+        return v1 < v2
+    @raw_binop
+    def le(self, v1, v2):
+        return v1 <= v2
+    @raw_binop
+    def gt(self, v1, v2):
+        return v1 > v2
+    @raw_binop
+    def ge(self, v1, v2):
+        return v1 >= v2
 
 
 class FloatArithmeticDtype(ArithmaticTypeMixin):
diff --git a/pypy/module/micronumpy/interp_numarray.py b/pypy/module/micronumpy/interp_numarray.py
--- a/pypy/module/micronumpy/interp_numarray.py
+++ b/pypy/module/micronumpy/interp_numarray.py
@@ -74,6 +74,13 @@
     descr_pow = _binop_impl("power")
     descr_mod = _binop_impl("mod")
 
+    descr_eq = _binop_impl("equal")
+    descr_ne = _binop_impl("not_equal")
+    descr_lt = _binop_impl("less")
+    descr_le = _binop_impl("less_equal")
+    descr_gt = _binop_impl("greater")
+    descr_ge = _binop_impl("greater_equal")
+
     def _binop_right_impl(ufunc_name):
         def impl(self, space, w_other):
             w_other = scalar_w(space,
@@ -404,10 +411,11 @@
     """
     Intermediate class for performing binary operations.
     """
-    def __init__(self, signature, res_dtype, left, right):
+    def __init__(self, signature, calc_dtype, res_dtype, left, right):
         VirtualArray.__init__(self, signature, res_dtype)
         self.left = left
         self.right = right
+        self.calc_dtype = calc_dtype
 
     def _del_sources(self):
         self.left = None
@@ -421,14 +429,14 @@
         return self.right.find_size()
 
     def _eval(self, i):
-        lhs = self.left.eval(i).convert_to(self.res_dtype)
-        rhs = self.right.eval(i).convert_to(self.res_dtype)
+        lhs = self.left.eval(i).convert_to(self.calc_dtype)
+        rhs = self.right.eval(i).convert_to(self.calc_dtype)
 
         sig = jit.promote(self.signature)
         assert isinstance(sig, signature.Signature)
         call_sig = sig.components[0]
         assert isinstance(call_sig, signature.Call2)
-        return call_sig.func(self.res_dtype, lhs, rhs)
+        return call_sig.func(self.calc_dtype, lhs, rhs)
 
 class ViewArray(BaseArray):
     """
@@ -573,18 +581,28 @@
     __pos__ = interp2app(BaseArray.descr_pos),
     __neg__ = interp2app(BaseArray.descr_neg),
     __abs__ = interp2app(BaseArray.descr_abs),
+
     __add__ = interp2app(BaseArray.descr_add),
     __sub__ = interp2app(BaseArray.descr_sub),
     __mul__ = interp2app(BaseArray.descr_mul),
     __div__ = interp2app(BaseArray.descr_div),
     __pow__ = interp2app(BaseArray.descr_pow),
     __mod__ = interp2app(BaseArray.descr_mod),
+
     __radd__ = interp2app(BaseArray.descr_radd),
     __rsub__ = interp2app(BaseArray.descr_rsub),
     __rmul__ = interp2app(BaseArray.descr_rmul),
     __rdiv__ = interp2app(BaseArray.descr_rdiv),
     __rpow__ = interp2app(BaseArray.descr_rpow),
     __rmod__ = interp2app(BaseArray.descr_rmod),
+
+    __eq__ = interp2app(BaseArray.descr_eq),
+    __ne__ = interp2app(BaseArray.descr_ne),
+    __lt__ = interp2app(BaseArray.descr_lt),
+    __le__ = interp2app(BaseArray.descr_le),
+    __gt__ = interp2app(BaseArray.descr_gt),
+    __ge__ = interp2app(BaseArray.descr_ge),
+
     __repr__ = interp2app(BaseArray.descr_repr),
     __str__ = interp2app(BaseArray.descr_str),
 
diff --git a/pypy/module/micronumpy/interp_ufuncs.py b/pypy/module/micronumpy/interp_ufuncs.py
--- a/pypy/module/micronumpy/interp_ufuncs.py
+++ b/pypy/module/micronumpy/interp_ufuncs.py
@@ -113,10 +113,11 @@
     argcount = 2
 
     def __init__(self, func, name, promote_to_float=False, promote_bools=False,
-        identity=None):
+        identity=None, comparison_func=False):
 
         W_Ufunc.__init__(self, name, promote_to_float, promote_bools, identity)
         self.func = func
+        self.comparison_func = comparison_func
         self.signature = signature.Call2(func)
         self.reduce_signature = signature.BaseSignature()
 
@@ -127,18 +128,25 @@
         [w_lhs, w_rhs] = args_w
         w_lhs = convert_to_array(space, w_lhs)
         w_rhs = convert_to_array(space, w_rhs)
-        res_dtype = find_binop_result_dtype(space,
+        calc_dtype = find_binop_result_dtype(space,
             w_lhs.find_dtype(), w_rhs.find_dtype(),
             promote_to_float=self.promote_to_float,
             promote_bools=self.promote_bools,
         )
+        if self.comparison_func:
+            res_dtype = space.fromcache(interp_dtype.W_BoolDtype)
+        else:
+            res_dtype = calc_dtype
         if isinstance(w_lhs, Scalar) and isinstance(w_rhs, Scalar):
-            return self.func(res_dtype, w_lhs.value, w_rhs.value).wrap(space)
+            return self.func(calc_dtype,
+                w_lhs.value.convert_to(calc_dtype),
+                w_rhs.value.convert_to(calc_dtype)
+            ).wrap(space)
 
         new_sig = signature.Signature.find_sig([
             self.signature, w_lhs.signature, w_rhs.signature
         ])
-        w_res = Call2(new_sig, res_dtype, w_lhs, w_rhs)
+        w_res = Call2(new_sig, calc_dtype, res_dtype, w_lhs, w_rhs)
         w_lhs.add_invalidates(w_res)
         w_rhs.add_invalidates(w_res)
         return w_res
@@ -209,13 +217,16 @@
     return space.fromcache(interp_dtype.W_Float64Dtype)
 
 
-def ufunc_dtype_caller(ufunc_name, op_name, argcount):
+def ufunc_dtype_caller(space, ufunc_name, op_name, argcount, comparison_func):
     if argcount == 1:
         def impl(res_dtype, value):
             return getattr(res_dtype, op_name)(value)
     elif argcount == 2:
         def impl(res_dtype, lvalue, rvalue):
-            return getattr(res_dtype, op_name)(lvalue, rvalue)
+            res = getattr(res_dtype, op_name)(lvalue, rvalue)
+            if comparison_func:
+                res = space.fromcache(interp_dtype.W_BoolDtype).box(res)
+            return res
     return func_with_new_name(impl, ufunc_name)
 
 class UfuncState(object):
@@ -229,6 +240,13 @@
             ("mod", "mod", 2, {"promote_bools": True}),
             ("power", "pow", 2, {"promote_bools": True}),
 
+            ("equal", "eq", 2, {"comparison_func": True}),
+            ("not_equal", "ne", 2, {"comparison_func": True}),
+            ("less", "lt", 2, {"comparison_func": True}),
+            ("less_equal", "le", 2, {"comparison_func": True}),
+            ("greater", "gt", 2, {"comparison_func": True}),
+            ("greater_equal", "ge", 2, {"comparison_func": True}),
+
             ("maximum", "max", 2),
             ("minimum", "min", 2),
 
@@ -262,7 +280,9 @@
             identity = space.fromcache(interp_dtype.W_Int64Dtype).adapt_val(identity)
         extra_kwargs["identity"] = identity
 
-        func = ufunc_dtype_caller(ufunc_name, op_name, argcount)
+        func = ufunc_dtype_caller(space, ufunc_name, op_name, argcount,
+            comparison_func=extra_kwargs.get("comparison_func", False)
+        )
         if argcount == 1:
             ufunc = W_Ufunc1(func, ufunc_name, **extra_kwargs)
         elif argcount == 2:
diff --git a/pypy/module/micronumpy/test/test_numarray.py b/pypy/module/micronumpy/test/test_numarray.py
--- a/pypy/module/micronumpy/test/test_numarray.py
+++ b/pypy/module/micronumpy/test/test_numarray.py
@@ -557,6 +557,26 @@
         assert array([1.2, 5]).dtype is dtype(float)
         assert array([]).dtype is dtype(float)
 
+    def test_comparison(self):
+        import operator
+        from numpy import array, dtype
+
+        a = array(range(5))
+        b = array(range(5), float)
+        for func in [
+            operator.eq, operator.ne, operator.lt, operator.le, operator.gt,
+            operator.ge
+        ]:
+            c = func(a, 3)
+            assert c.dtype is dtype(bool)
+            for i in xrange(5):
+                assert c[i] == func(a[i], 3)
+
+            c = func(b, 3)
+            assert c.dtype is dtype(bool)
+            for i in xrange(5):
+                assert c[i] == func(b[i], 3)
+
 
 class AppTestSupport(object):
     def setup_class(cls):
diff --git a/pypy/module/micronumpy/test/test_ufuncs.py b/pypy/module/micronumpy/test/test_ufuncs.py
--- a/pypy/module/micronumpy/test/test_ufuncs.py
+++ b/pypy/module/micronumpy/test/test_ufuncs.py
@@ -310,4 +310,30 @@
         assert add.reduce([1, 2, 3]) == 6
         assert maximum.reduce([1]) == 1
         assert maximum.reduce([1, 2, 3]) == 3
-        raises(ValueError, maximum.reduce, [])
\ No newline at end of file
+        raises(ValueError, maximum.reduce, [])
+
+    def test_comparisons(self):
+        import operator
+        from numpy import equal, not_equal, less, less_equal, greater, greater_equal
+
+        for ufunc, func in [
+            (equal, operator.eq),
+            (not_equal, operator.ne),
+            (less, operator.lt),
+            (less_equal, operator.le),
+            (greater, operator.gt),
+            (greater_equal, operator.ge),
+        ]:
+            for a, b in [
+                (3, 3),
+                (3, 4),
+                (4, 3),
+                (3.0, 3.0),
+                (3.0, 3.5),
+                (3.5, 3.0),
+                (3.0, 3),
+                (3, 3.0),
+                (3.5, 3),
+                (3, 3.5),
+            ]:
+                assert ufunc(a, b) is func(a, b)
\ No newline at end of file

From noreply at buildbot.pypy.org  Mon Sep  5 19:03:14 2011
From: noreply at buildbot.pypy.org (alex_gaynor)
Date: Mon,  5 Sep 2011 19:03:14 +0200 (CEST)
Subject: [pypy-commit] pypy numpy-comparison: Close branch.
Message-ID: <20110905170314.407768203C@wyvern.cs.uni-duesseldorf.de>

Author: Alex Gaynor 
Branch: numpy-comparison
Changeset: r47088:5a08d2cfb491
Date: 2011-09-05 10:02 -0700
http://bitbucket.org/pypy/pypy/changeset/5a08d2cfb491/

Log:	Close branch.


From noreply at buildbot.pypy.org  Mon Sep  5 21:09:25 2011
From: noreply at buildbot.pypy.org (alex_gaynor)
Date: Mon,  5 Sep 2011 21:09:25 +0200 (CEST)
Subject: [pypy-commit] pypy default: a skipped, failing test.
Message-ID: <20110905190925.1073F8203C@wyvern.cs.uni-duesseldorf.de>

Author: Alex Gaynor 
Branch: 
Changeset: r47089:386189328fb9
Date: 2011-09-05 12:09 -0700
http://bitbucket.org/pypy/pypy/changeset/386189328fb9/

Log:	a skipped, failing test.

diff --git a/pypy/jit/metainterp/optimizeopt/heap.py b/pypy/jit/metainterp/optimizeopt/heap.py
--- a/pypy/jit/metainterp/optimizeopt/heap.py
+++ b/pypy/jit/metainterp/optimizeopt/heap.py
@@ -25,7 +25,7 @@
         #      'cached_fields'.
         #
         self._cached_fields = {}
-        self._cached_fields_getfield_op = {}        
+        self._cached_fields_getfield_op = {}
         self._lazy_setfield = None
         self._lazy_setfield_registered = False
 
@@ -75,7 +75,7 @@
     def remember_field_value(self, structvalue, fieldvalue, getfield_op=None):
         assert self._lazy_setfield is None
         self._cached_fields[structvalue] = fieldvalue
-        self._cached_fields_getfield_op[structvalue] = getfield_op        
+        self._cached_fields_getfield_op[structvalue] = getfield_op
 
     def force_lazy_setfield(self, optheap, can_cache=True):
         op = self._lazy_setfield
@@ -163,7 +163,7 @@
 
     def new(self):
         return OptHeap()
-        
+
     def produce_potential_short_preamble_ops(self, sb):
         descrkeys = self.cached_fields.keys()
         if not we_are_translated():
diff --git a/pypy/jit/metainterp/optimizeopt/test/test_optimizebasic.py b/pypy/jit/metainterp/optimizeopt/test/test_optimizebasic.py
--- a/pypy/jit/metainterp/optimizeopt/test/test_optimizebasic.py
+++ b/pypy/jit/metainterp/optimizeopt/test/test_optimizebasic.py
@@ -4711,6 +4711,33 @@
         """
         self.optimize_loop(ops, expected)
 
+    def test_forced_virtuals_aliasing(self):
+        ops = """
+        [i0, i1]
+        p0 = new(descr=ssize)
+        p1 = new(descr=ssize)
+        escape(p0)
+        escape(p1)
+        setfield_gc(p0, i0, descr=adescr)
+        setfield_gc(p1, i1, descr=adescr)
+        i2 = getfield_gc(p0, descr=adescr)
+        jump(i2, i2)
+        """
+        expected = """
+        [i0, i1]
+        p0 = new(descr=ssize)
+        escape(p0)
+        p1 = new(descr=ssize)
+        escape(p1)
+        setfield_gc(p0, i0, descr=adescr)
+        setfield_gc(p1, i1, descr=adescr)
+        jump(i0, i0)
+        """
+        py.test.skip("not implemented")
+        # setfields on things that used to be virtual still can't alias each
+        # other
+        self.optimize_loop(ops, expected)
+
 
 class TestLLtype(BaseTestOptimizeBasic, LLtypeMixin):
     pass

From noreply at buildbot.pypy.org  Mon Sep  5 23:20:08 2011
From: noreply at buildbot.pypy.org (amauryfa)
Date: Mon,  5 Sep 2011 23:20:08 +0200 (CEST)
Subject: [pypy-commit] pypy compile-from-stream: Fix recently merged test
Message-ID: <20110905212008.8C0BB8203C@wyvern.cs.uni-duesseldorf.de>

Author: Amaury Forgeot d'Arc 
Branch: compile-from-stream
Changeset: r47090:6b67dd77810b
Date: 2011-08-31 01:14 +0200
http://bitbucket.org/pypy/pypy/changeset/6b67dd77810b/

Log:	Fix recently merged test

diff --git a/pypy/module/imp/test/test_import.py b/pypy/module/imp/test/test_import.py
--- a/pypy/module/imp/test/test_import.py
+++ b/pypy/module/imp/test/test_import.py
@@ -782,7 +782,7 @@
                                                  w_modulename,
                                                  w_mod,
                                                  pathname,
-                                                 stream.readall())
+                                                 stream)
         finally:
             space.setattr(space.sys, space.wrap('dont_write_bytecode'),
                           space.w_False)

From noreply at buildbot.pypy.org  Mon Sep  5 23:20:09 2011
From: noreply at buildbot.pypy.org (amauryfa)
Date: Mon,  5 Sep 2011 23:20:09 +0200 (CEST)
Subject: [pypy-commit] pypy compile-from-stream: Simplify some tests
Message-ID: <20110905212009.CF6628203C@wyvern.cs.uni-duesseldorf.de>

Author: Amaury Forgeot d'Arc 
Branch: compile-from-stream
Changeset: r47091:47e802416bf2
Date: 2011-08-31 01:34 +0200
http://bitbucket.org/pypy/pypy/changeset/47e802416bf2/

Log:	Simplify some tests

diff --git a/pypy/module/imp/test/test_import.py b/pypy/module/imp/test/test_import.py
--- a/pypy/module/imp/test/test_import.py
+++ b/pypy/module/imp/test/test_import.py
@@ -695,9 +695,8 @@
         pathname = _testfilesource()
         stream = streamio.open_file_as_stream(pathname, "r")
         try:
-            w_ret = importing.parse_source_module(space,
-                                                  pathname,
-                                                  stream.readall())
+            w_ret = importing.parse_source_file_module(
+                space, pathname, stream)
         finally:
             stream.close()
         pycode = space.interpclass_w(w_ret)
@@ -839,9 +838,8 @@
         os.chmod(pathname, 0777)
         stream = streamio.open_file_as_stream(pathname, "r")
         try:
-            w_ret = importing.parse_source_module(space,
-                                                  pathname,
-                                                  stream.readall())
+            w_ret = importing.parse_source_file_module(
+                space, pathname, stream)
         finally:
             stream.close()
         pycode = space.interpclass_w(w_ret)

From noreply at buildbot.pypy.org  Mon Sep  5 23:20:11 2011
From: noreply at buildbot.pypy.org (amauryfa)
Date: Mon,  5 Sep 2011 23:20:11 +0200 (CEST)
Subject: [pypy-commit] pypy default: implement the pwd module at
	interp-level, 
Message-ID: <20110905212011.162368203C@wyvern.cs.uni-duesseldorf.de>

Author: Amaury Forgeot d'Arc 
Branch: 
Changeset: r47092:21f8faf13c20
Date: 2011-09-05 23:14 +0200
http://bitbucket.org/pypy/pypy/changeset/21f8faf13c20/

Log:	implement the pwd module at interp-level, should help the cyclic
	imports on some platforms (pwd imported ctypes which needs
	pwd.pw_dir to find libraries)

diff --git a/lib-python/conftest.py b/lib-python/conftest.py
--- a/lib-python/conftest.py
+++ b/lib-python/conftest.py
@@ -359,7 +359,7 @@
     RegrTest('test_property.py', core=True),
     RegrTest('test_pstats.py'),
     RegrTest('test_pty.py', skip="unsupported extension module"),
-    RegrTest('test_pwd.py', skip=skip_win32),
+    RegrTest('test_pwd.py', usemodules="pwd", skip=skip_win32),
     RegrTest('test_py3kwarn.py'),
     RegrTest('test_pyclbr.py'),
     RegrTest('test_pydoc.py'),
diff --git a/pypy/config/pypyoption.py b/pypy/config/pypyoption.py
--- a/pypy/config/pypyoption.py
+++ b/pypy/config/pypyoption.py
@@ -27,7 +27,7 @@
 # --allworkingmodules
 working_modules = default_modules.copy()
 working_modules.update(dict.fromkeys(
-    ["_socket", "unicodedata", "mmap", "fcntl", "_locale",
+    ["_socket", "unicodedata", "mmap", "fcntl", "_locale", "pwd",
      "rctime" , "select", "zipimport", "_lsprof",
      "crypt", "signal", "_rawffi", "termios", "zlib", "bz2",
      "struct", "_hashlib", "_md5", "_sha", "_minimal_curses", "cStringIO",
@@ -58,6 +58,7 @@
     # unix only modules
     del working_modules["crypt"]
     del working_modules["fcntl"]
+    del working_modules["pwd"]
     del working_modules["termios"]
     del working_modules["_minimal_curses"]
 
diff --git a/pypy/doc/config/objspace.usemodules.pwd.txt b/pypy/doc/config/objspace.usemodules.pwd.txt
new file mode 100644
--- /dev/null
+++ b/pypy/doc/config/objspace.usemodules.pwd.txt
@@ -0,0 +1,2 @@
+Use the 'pwd' module. 
+This module is expected to be fully working.
diff --git a/pypy/module/pwd/__init__.py b/pypy/module/pwd/__init__.py
new file mode 100644
--- /dev/null
+++ b/pypy/module/pwd/__init__.py
@@ -0,0 +1,25 @@
+from pypy.interpreter.mixedmodule import MixedModule
+
+class Module(MixedModule):
+    """
+    This module provides access to the Unix password database.
+    It is available on all Unix versions.
+
+    Password database entries are reported as 7-tuples containing the following
+    items from the password database (see `'), in order:
+    pw_name, pw_passwd, pw_uid, pw_gid, pw_gecos, pw_dir, pw_shell.
+    The uid and gid items are integers, all others are strings. An
+    exception is raised if the entry asked for cannot be found.
+    """
+
+    interpleveldefs = {
+        'getpwuid': 'interp_pwd.getpwuid',
+        'getpwnam': 'interp_pwd.getpwnam',
+        'getpwall': 'interp_pwd.getpwall',
+    }
+
+    appleveldefs = {
+        'struct_passwd': 'app_pwd.struct_passwd',
+        'struct_pwent': 'app_pwd.struct_passwd',
+    }
+
diff --git a/pypy/module/pwd/app_pwd.py b/pypy/module/pwd/app_pwd.py
new file mode 100644
--- /dev/null
+++ b/pypy/module/pwd/app_pwd.py
@@ -0,0 +1,20 @@
+from _structseq import structseqtype, structseqfield
+
+class struct_passwd:
+    """
+    pwd.struct_passwd: Results from getpw*() routines.
+
+    This object may be accessed either as a tuple of
+      (pw_name,pw_passwd,pw_uid,pw_gid,pw_gecos,pw_dir,pw_shell)
+    or via the object attributes as named in the above tuple.
+    """
+    __metaclass__ = structseqtype
+    name = "pwd.struct_passwd"
+
+    pw_name   = structseqfield(0, "user name")
+    pw_passwd = structseqfield(1, "password")
+    pw_uid    = structseqfield(2, "user id")
+    pw_gid    = structseqfield(3, "group id")
+    pw_gecos  = structseqfield(4, "real name")
+    pw_dir    = structseqfield(5, "home directory")
+    pw_shell  = structseqfield(6, "shell program")
diff --git a/pypy/module/pwd/interp_pwd.py b/pypy/module/pwd/interp_pwd.py
new file mode 100644
--- /dev/null
+++ b/pypy/module/pwd/interp_pwd.py
@@ -0,0 +1,89 @@
+from pypy.translator.tool.cbuild import ExternalCompilationInfo
+from pypy.rpython.tool import rffi_platform
+from pypy.rpython.lltypesystem import rffi, lltype
+from pypy.interpreter.gateway import interp2app, unwrap_spec
+from pypy.interpreter.error import OperationError, operationerrfmt
+
+class CConfig:
+    _compilation_info_ = ExternalCompilationInfo(
+        includes=['pwd.h']
+        )
+
+    uid_t = rffi_platform.SimpleType("uid_t")
+
+    passwd = rffi_platform.Struct(
+        'struct passwd',
+        [('pw_name', rffi.CCHARP),
+         ('pw_passwd', rffi.CCHARP),
+         ('pw_uid', rffi.INT),
+         ('pw_gid', rffi.INT),
+         ('pw_gecos', rffi.CCHARP),
+         ('pw_dir', rffi.CCHARP),
+         ('pw_shell', rffi.CCHARP),
+         ])
+
+config = rffi_platform.configure(CConfig)
+passwd_p = lltype.Ptr(config['passwd'])
+uid_t = config['uid_t']
+
+c_getpwuid = rffi.llexternal("getpwuid", [uid_t], passwd_p)
+c_getpwnam = rffi.llexternal("getpwnam", [rffi.CCHARP], passwd_p)
+c_setpwent = rffi.llexternal("setpwent", [], lltype.Void)
+c_getpwent = rffi.llexternal("getpwent", [], passwd_p)
+c_endpwent = rffi.llexternal("endpwent", [], lltype.Void)
+
+def make_struct_passwd(space, pw):
+    w_passwd_struct = space.getattr(space.getbuiltinmodule('pwd'),
+                                    space.wrap('struct_passwd'))
+    w_tuple = space.newtuple([
+        space.wrap(rffi.charp2str(pw.c_pw_name)),
+        space.wrap(rffi.charp2str(pw.c_pw_passwd)),
+        space.wrap(pw.c_pw_uid),
+        space.wrap(pw.c_pw_gid),
+        space.wrap(rffi.charp2str(pw.c_pw_gecos)),
+        space.wrap(rffi.charp2str(pw.c_pw_dir)),
+        space.wrap(rffi.charp2str(pw.c_pw_shell)),
+        ])
+    return space.call_function(w_passwd_struct, w_tuple)
+
+ at unwrap_spec(uid=int)
+def getpwuid(space, uid):
+    """
+    getpwuid(uid) -> (pw_name,pw_passwd,pw_uid,
+                      pw_gid,pw_gecos,pw_dir,pw_shell)
+    Return the password database entry for the given numeric user ID.
+    See pwd.__doc__ for more on password database entries.
+    """
+    pw = c_getpwuid(uid)
+    if not pw:
+        raise operationerrfmt(space.w_KeyError,
+            "getpwuid(): uid not found: %d", uid)
+    return make_struct_passwd(space, pw)
+
+ at unwrap_spec(name=str)
+def getpwnam(space, name):
+    """
+    getpwnam(name) -> (pw_name,pw_passwd,pw_uid,
+                        pw_gid,pw_gecos,pw_dir,pw_shell)
+    Return the password database entry for the given user name.
+    See pwd.__doc__ for more on password database entries.
+    """
+    pw = c_getpwnam(name)
+    if not pw:
+        raise operationerrfmt(space.w_KeyError,
+            "getpwnam(): name not found: %s", name)
+    return make_struct_passwd(space, pw)
+
+def getpwall(space):
+    users_w = []
+    c_setpwent()
+    try:
+        while True:
+            pw = c_getpwent()
+            if not pw:
+                break
+            users_w.append(make_struct_passwd(space, pw))
+    finally:
+        c_endpwent()
+    return space.newlist(users_w)
+    
diff --git a/pypy/module/pwd/test/test_pwd.py b/pypy/module/pwd/test/test_pwd.py
new file mode 100644
--- /dev/null
+++ b/pypy/module/pwd/test/test_pwd.py
@@ -0,0 +1,25 @@
+from pypy.conftest import gettestobjspace
+
+class AppTestPwd:
+    def setup_class(cls):
+        cls.space = gettestobjspace(usemodules=['pwd'])
+
+    def test_getpwuid(self):
+        import pwd
+        raises(KeyError, pwd.getpwuid, -1)
+        pw = pwd.getpwuid(0)
+        assert pw.pw_name == 'root'
+        assert isinstance(pw.pw_passwd, str)
+        assert pw.pw_uid == 0
+        assert pw.pw_gid == 0
+        assert pw.pw_dir == '/root'
+        assert pw.pw_shell.startswith('/')
+
+    def test_getpwnam(self):
+        import pwd
+        raises(KeyError, pwd.getpwnam, '~invalid~')
+        assert pwd.getpwnam('root').pw_name == 'root'
+
+    def test_getpwall(self):
+        import pwd
+        assert pwd.getpwnam('root') in pwd.getpwall()

From noreply at buildbot.pypy.org  Mon Sep  5 23:20:12 2011
From: noreply at buildbot.pypy.org (amauryfa)
Date: Mon,  5 Sep 2011 23:20:12 +0200 (CEST)
Subject: [pypy-commit] pypy default: Small cleanup in marshall functions
Message-ID: <20110905212012.4C2718203C@wyvern.cs.uni-duesseldorf.de>

Author: Amaury Forgeot d'Arc 
Branch: 
Changeset: r47093:66a579857a84
Date: 2011-08-31 01:25 +0200
http://bitbucket.org/pypy/pypy/changeset/66a579857a84/

Log:	Small cleanup in marshall functions

diff --git a/pypy/module/marshal/interp_marshal.py b/pypy/module/marshal/interp_marshal.py
--- a/pypy/module/marshal/interp_marshal.py
+++ b/pypy/module/marshal/interp_marshal.py
@@ -40,7 +40,7 @@
         reader = FileReader(space, w_f)
     try:
         u = Unmarshaller(space, reader)
-        return u.load_w_obj(False)
+        return u.load_w_obj()
     finally:
         reader.finished()
 
@@ -49,7 +49,7 @@
 ignored."""
     space.timer.start("marshal loads")
     u = StringUnmarshaller(space, w_str)
-    obj = u.load_w_obj(False)
+    obj = u.load_w_obj()
     space.timer.stop("marshal loads")
     return obj
 
@@ -424,7 +424,7 @@
         lng = self.get_lng()
         return self.get(lng)
 
-    def get_w_obj(self, allow_null):
+    def get_w_obj(self, allow_null=False):
         space = self.space
         w_ret = space.w_None # something not None
         tc = self.get1()
@@ -434,9 +434,9 @@
                 'NULL object in marshal data'))
         return w_ret
 
-    def load_w_obj(self, allow_null):
+    def load_w_obj(self):
         try:
-            return self.get_w_obj(allow_null)
+            return self.get_w_obj()
         except rstackovf.StackOverflow:
             rstackovf.check_stack_overflow()
             self._overflow()
diff --git a/pypy/objspace/std/marshal_impl.py b/pypy/objspace/std/marshal_impl.py
--- a/pypy/objspace/std/marshal_impl.py
+++ b/pypy/objspace/std/marshal_impl.py
@@ -325,10 +325,10 @@
     # of building a list of tuples.
     w_dic = space.newdict()
     while 1:
-        w_key = u.get_w_obj(True)
+        w_key = u.get_w_obj(allow_null=True)
         if w_key is None:
             break
-        w_value = u.get_w_obj(False)
+        w_value = u.get_w_obj()
         space.setitem(w_dic, w_key, w_value)
     return w_dic
 register(TYPE_DICT, unmarshal_DictMulti)
@@ -364,7 +364,7 @@
 # so we no longer can handle it in interp_marshal.atom_strlist
 
 def unmarshal_str(u):
-    w_obj = u.get_w_obj(False)
+    w_obj = u.get_w_obj()
     try:
         return u.space.str_w(w_obj)
     except OperationError, e:

From noreply at buildbot.pypy.org  Tue Sep  6 01:30:33 2011
From: noreply at buildbot.pypy.org (amauryfa)
Date: Tue,  6 Sep 2011 01:30:33 +0200 (CEST)
Subject: [pypy-commit] pypy default: Issue863 Implement sys._current_frames()
Message-ID: <20110905233033.6AF298203C@wyvern.cs.uni-duesseldorf.de>

Author: Amaury Forgeot d'Arc 
Branch: 
Changeset: r47094:411a4e22b5bc
Date: 2011-09-06 01:29 +0200
http://bitbucket.org/pypy/pypy/changeset/411a4e22b5bc/

Log:	Issue863 Implement sys._current_frames() "for debugging purposes"

diff --git a/pypy/interpreter/miscutils.py b/pypy/interpreter/miscutils.py
--- a/pypy/interpreter/miscutils.py
+++ b/pypy/interpreter/miscutils.py
@@ -167,3 +167,7 @@
 
     def getmainthreadvalue(self):
         return self._value
+
+    def getallvalues(self):
+        return {0: self._value}
+
diff --git a/pypy/module/sys/__init__.py b/pypy/module/sys/__init__.py
--- a/pypy/module/sys/__init__.py
+++ b/pypy/module/sys/__init__.py
@@ -47,6 +47,7 @@
         'pypy_initial_path'     : 'state.pypy_initial_path',
 
         '_getframe'             : 'vm._getframe', 
+        '_current_frames'       : 'vm._current_frames', 
         'setrecursionlimit'     : 'vm.setrecursionlimit', 
         'getrecursionlimit'     : 'vm.getrecursionlimit', 
         'setcheckinterval'      : 'vm.setcheckinterval', 
diff --git a/pypy/module/sys/test/test_sysmodule.py b/pypy/module/sys/test/test_sysmodule.py
--- a/pypy/module/sys/test/test_sysmodule.py
+++ b/pypy/module/sys/test/test_sysmodule.py
@@ -1,6 +1,6 @@
 # -*- coding: iso-8859-1 -*-
 import autopath
-from pypy.conftest import option
+from pypy.conftest import option, gettestobjspace
 from py.test import raises
 from pypy.interpreter.gateway import app2interp_temp
 import sys
@@ -524,3 +524,51 @@
         # If this ever actually becomes a compilation option this test should
         # be changed.
         assert sys.float_repr_style == "short"
+
+class AppTestCurrentFrames:
+
+    def test_current_frames(self):
+        try:
+            import thread
+        except ImportError:
+            pass
+        else:
+            skip('This test requires an intepreter without threads')
+        import sys
+
+        def f():
+            return sys._current_frames()
+        frames = f()
+        assert frames.keys() == [0]
+        assert frames[0].f_code.co_name == 'f'
+
+class AppTestCurrentFramesWithThread(AppTestCurrentFrames):
+    def setup_class(cls):
+        cls.space = gettestobjspace(usemodules=('thread',))
+
+    def test_current_frames(self):
+        import sys
+        import time
+        import thread
+
+        thread_id = thread.get_ident()
+        self.ready = False
+        def other_thread():
+            self.ready = True
+            print "thread started"
+            time.sleep(5)
+        thread.start_new_thread(other_thread, ())
+
+        def f():
+            for i in range(100):
+                if self.ready: break
+                time.sleep(0.1)
+            return sys._current_frames()
+        
+        frames = f()
+        thisframe = frames.pop(thread_id)
+        assert thisframe.f_code.co_name == 'f'
+
+        assert len(frames) == 1
+        _, other_frame = frames.popitem()
+        assert other_frame.f_code.co_name == 'other_thread'
diff --git a/pypy/module/sys/vm.py b/pypy/module/sys/vm.py
--- a/pypy/module/sys/vm.py
+++ b/pypy/module/sys/vm.py
@@ -43,6 +43,23 @@
     f.mark_as_escaped()
     return space.wrap(f)
 
+def _current_frames(space):
+    """_current_frames() -> dictionary
+
+    Return a dictionary mapping each current thread T's thread id to T's
+    current stack frame.
+
+    This function should be used for specialized purposes only."""
+    w_result = space.newdict()
+    ecs = space.threadlocals.getallvalues()
+    for thread_ident, ec in ecs.items():
+        f = ec.gettopframe_nohidden()
+        f.mark_as_escaped()
+        space.setitem(w_result,
+                      space.wrap(thread_ident),
+                      space.wrap(f))
+    return w_result                      
+
 def setrecursionlimit(space, w_new_limit):
     """setrecursionlimit() sets the maximum number of nested calls that
 can occur before a RuntimeError is raised.  On PyPy the limit is
diff --git a/pypy/module/thread/threadlocals.py b/pypy/module/thread/threadlocals.py
--- a/pypy/module/thread/threadlocals.py
+++ b/pypy/module/thread/threadlocals.py
@@ -43,6 +43,9 @@
         ident = self._mainthreadident
         return self._valuedict.get(ident, None)
 
+    def getallvalues(self):
+        return self._valuedict
+
     def enter_thread(self, space):
         "Notification that the current thread is just starting."
         ec = space.getexecutioncontext()

From noreply at buildbot.pypy.org  Tue Sep  6 02:03:14 2011
From: noreply at buildbot.pypy.org (amauryfa)
Date: Tue,  6 Sep 2011 02:03:14 +0200 (CEST)
Subject: [pypy-commit] pypy default: Fix translation
Message-ID: <20110906000314.7D5168203C@wyvern.cs.uni-duesseldorf.de>

Author: Amaury Forgeot d'Arc 
Branch: 
Changeset: r47095:517b88c3b7d5
Date: 2011-09-06 02:02 +0200
http://bitbucket.org/pypy/pypy/changeset/517b88c3b7d5/

Log:	Fix translation

diff --git a/pypy/module/pwd/interp_pwd.py b/pypy/module/pwd/interp_pwd.py
--- a/pypy/module/pwd/interp_pwd.py
+++ b/pypy/module/pwd/interp_pwd.py
@@ -4,10 +4,12 @@
 from pypy.interpreter.gateway import interp2app, unwrap_spec
 from pypy.interpreter.error import OperationError, operationerrfmt
 
+eci = ExternalCompilationInfo(
+    includes=['pwd.h']
+    )
+
 class CConfig:
-    _compilation_info_ = ExternalCompilationInfo(
-        includes=['pwd.h']
-        )
+    _compilation_info_ = eci
 
     uid_t = rffi_platform.SimpleType("uid_t")
 
@@ -26,11 +28,14 @@
 passwd_p = lltype.Ptr(config['passwd'])
 uid_t = config['uid_t']
 
-c_getpwuid = rffi.llexternal("getpwuid", [uid_t], passwd_p)
-c_getpwnam = rffi.llexternal("getpwnam", [rffi.CCHARP], passwd_p)
-c_setpwent = rffi.llexternal("setpwent", [], lltype.Void)
-c_getpwent = rffi.llexternal("getpwent", [], passwd_p)
-c_endpwent = rffi.llexternal("endpwent", [], lltype.Void)
+def external(name, args, result, **kwargs):
+    return rffi.llexternal(name, args, result, compilation_info=eci, **kwargs)
+
+c_getpwuid = external("getpwuid", [uid_t], passwd_p)
+c_getpwnam = external("getpwnam", [rffi.CCHARP], passwd_p)
+c_setpwent = external("setpwent", [], lltype.Void)
+c_getpwent = external("getpwent", [], passwd_p)
+c_endpwent = external("endpwent", [], lltype.Void)
 
 def make_struct_passwd(space, pw):
     w_passwd_struct = space.getattr(space.getbuiltinmodule('pwd'),

From noreply at buildbot.pypy.org  Tue Sep  6 06:27:19 2011
From: noreply at buildbot.pypy.org (arigo)
Date: Tue,  6 Sep 2011 06:27:19 +0200 (CEST)
Subject: [pypy-commit] pyrepl default: Port 659f7a0b3256 from pypy.
Message-ID: <20110906042719.A2C1E8203C@wyvern.cs.uni-duesseldorf.de>

Author: Armin Rigo 
Branch: 
Changeset: r127:183fb78bf113
Date: 2011-09-06 06:27 +0200
http://bitbucket.org/pypy/pyrepl/changeset/183fb78bf113/

Log:	Port 659f7a0b3256 from pypy.

diff --git a/pyrepl/reader.py b/pyrepl/reader.py
--- a/pyrepl/reader.py
+++ b/pyrepl/reader.py
@@ -576,7 +576,7 @@
         self.console.push_char(char)
         self.handle1(0)
     
-    def readline(self):
+    def readline(self, returns_unicode=False):
         """Read a line.  The implementation of this method also shows
         how to drive Reader if you want more control over the event
         loop."""
@@ -585,6 +585,8 @@
             self.refresh()
             while not self.finished:
                 self.handle1()
+            if returns_unicode:
+                return self.get_unicode()
             return self.get_buffer()
         finally:
             self.restore()
diff --git a/pyrepl/readline.py b/pyrepl/readline.py
--- a/pyrepl/readline.py
+++ b/pyrepl/readline.py
@@ -198,7 +198,7 @@
         reader.ps1 = prompt
         return reader.readline()
 
-    def multiline_input(self, more_lines, ps1, ps2):
+    def multiline_input(self, more_lines, ps1, ps2, returns_unicode=False):
         """Read an input on possibly multiple lines, asking for more
         lines as long as 'more_lines(unicodetext)' returns an object whose
         boolean value is true.
@@ -209,7 +209,7 @@
             reader.more_lines = more_lines
             reader.ps1 = reader.ps2 = ps1
             reader.ps3 = reader.ps4 = ps2
-            return reader.readline()
+            return reader.readline(returns_unicode=returns_unicode)
         finally:
             reader.more_lines = saved
 
diff --git a/pyrepl/simple_interact.py b/pyrepl/simple_interact.py
--- a/pyrepl/simple_interact.py
+++ b/pyrepl/simple_interact.py
@@ -54,7 +54,8 @@
             ps1 = getattr(sys, 'ps1', '>>> ')
             ps2 = getattr(sys, 'ps2', '... ')
             try:
-                statement = multiline_input(more_lines, ps1, ps2)
+                statement = multiline_input(more_lines, ps1, ps2,
+                                            returns_unicode=True)
             except EOFError:
                 break
             more = console.push(statement)

From noreply at buildbot.pypy.org  Tue Sep  6 06:27:27 2011
From: noreply at buildbot.pypy.org (arigo)
Date: Tue,  6 Sep 2011 06:27:27 +0200 (CEST)
Subject: [pypy-commit] pypy default: A failing test.
Message-ID: <20110906042727.1049C8203C@wyvern.cs.uni-duesseldorf.de>

Author: Armin Rigo 
Branch: 
Changeset: r47096:950b50221e93
Date: 2011-09-06 06:16 +0200
http://bitbucket.org/pypy/pypy/changeset/950b50221e93/

Log:	A failing test.

diff --git a/pypy/interpreter/test/test_exec.py b/pypy/interpreter/test/test_exec.py
--- a/pypy/interpreter/test/test_exec.py
+++ b/pypy/interpreter/test/test_exec.py
@@ -219,3 +219,17 @@
             raise e
 
         assert res == 1
+
+    def test_exec_unicode(self):
+        # 's' is a string
+        s = "x = u'\xd0\xb9\xd1\x86\xd1\x83\xd0\xba\xd0\xb5\xd0\xbd'"
+        # 'u' is a unicode
+        u = s.decode('utf-8')
+        exec u
+        assert len(x) == 6
+        assert ord(x[0]) == 0x0439
+        assert ord(x[1]) == 0x0446
+        assert ord(x[2]) == 0x0443
+        assert ord(x[3]) == 0x043a
+        assert ord(x[4]) == 0x0435
+        assert ord(x[5]) == 0x043d

From noreply at buildbot.pypy.org  Tue Sep  6 06:27:28 2011
From: noreply at buildbot.pypy.org (arigo)
Date: Tue,  6 Sep 2011 06:27:28 +0200 (CEST)
Subject: [pypy-commit] pypy default: Import into _pypy_interact.py the
 change from code.py to decode
Message-ID: <20110906042728.4B21D8203C@wyvern.cs.uni-duesseldorf.de>

Author: Armin Rigo 
Branch: 
Changeset: r47097:659f7a0b3256
Date: 2011-09-06 06:25 +0200
http://bitbucket.org/pypy/pypy/changeset/659f7a0b3256/

Log:	Import into _pypy_interact.py the change from code.py to decode the
	raw string into a unicode string. Fix simple_interact.py to just do
	the right thing and not encode the unicode string in the first
	place, by passing an extra keyword argument.

diff --git a/lib_pypy/_pypy_interact.py b/lib_pypy/_pypy_interact.py
--- a/lib_pypy/_pypy_interact.py
+++ b/lib_pypy/_pypy_interact.py
@@ -56,6 +56,10 @@
                 prompt = getattr(sys, 'ps1', '>>> ')
             try:
                 line = raw_input(prompt)
+                # Can be None if sys.stdin was redefined
+                encoding = getattr(sys.stdin, 'encoding', None)
+                if encoding and not isinstance(line, unicode):
+                    line = line.decode(encoding)
             except EOFError:
                 console.write("\n")
                 break
diff --git a/lib_pypy/pyrepl/reader.py b/lib_pypy/pyrepl/reader.py
--- a/lib_pypy/pyrepl/reader.py
+++ b/lib_pypy/pyrepl/reader.py
@@ -576,7 +576,7 @@
         self.console.push_char(char)
         self.handle1(0)
     
-    def readline(self):
+    def readline(self, returns_unicode=False):
         """Read a line.  The implementation of this method also shows
         how to drive Reader if you want more control over the event
         loop."""
@@ -585,6 +585,8 @@
             self.refresh()
             while not self.finished:
                 self.handle1()
+            if returns_unicode:
+                return self.get_unicode()
             return self.get_buffer()
         finally:
             self.restore()
diff --git a/lib_pypy/pyrepl/readline.py b/lib_pypy/pyrepl/readline.py
--- a/lib_pypy/pyrepl/readline.py
+++ b/lib_pypy/pyrepl/readline.py
@@ -198,7 +198,7 @@
         reader.ps1 = prompt
         return reader.readline()
 
-    def multiline_input(self, more_lines, ps1, ps2):
+    def multiline_input(self, more_lines, ps1, ps2, returns_unicode=False):
         """Read an input on possibly multiple lines, asking for more
         lines as long as 'more_lines(unicodetext)' returns an object whose
         boolean value is true.
@@ -209,7 +209,7 @@
             reader.more_lines = more_lines
             reader.ps1 = reader.ps2 = ps1
             reader.ps3 = reader.ps4 = ps2
-            return reader.readline()
+            return reader.readline(returns_unicode=returns_unicode)
         finally:
             reader.more_lines = saved
 
diff --git a/lib_pypy/pyrepl/simple_interact.py b/lib_pypy/pyrepl/simple_interact.py
--- a/lib_pypy/pyrepl/simple_interact.py
+++ b/lib_pypy/pyrepl/simple_interact.py
@@ -54,7 +54,8 @@
             ps1 = getattr(sys, 'ps1', '>>> ')
             ps2 = getattr(sys, 'ps2', '... ')
             try:
-                statement = multiline_input(more_lines, ps1, ps2)
+                statement = multiline_input(more_lines, ps1, ps2,
+                                            returns_unicode=True)
             except EOFError:
                 break
             more = console.push(statement)

From noreply at buildbot.pypy.org  Tue Sep  6 06:37:13 2011
From: noreply at buildbot.pypy.org (arigo)
Date: Tue,  6 Sep 2011 06:37:13 +0200 (CEST)
Subject: [pypy-commit] pypy default: More tests.
Message-ID: <20110906043713.4BBE08203C@wyvern.cs.uni-duesseldorf.de>

Author: Armin Rigo 
Branch: 
Changeset: r47098:5d1b417d7c94
Date: 2011-09-06 06:35 +0200
http://bitbucket.org/pypy/pypy/changeset/5d1b417d7c94/

Log:	More tests.

diff --git a/pypy/interpreter/test/test_exec.py b/pypy/interpreter/test/test_exec.py
--- a/pypy/interpreter/test/test_exec.py
+++ b/pypy/interpreter/test/test_exec.py
@@ -233,3 +233,16 @@
         assert ord(x[3]) == 0x043a
         assert ord(x[4]) == 0x0435
         assert ord(x[5]) == 0x043d
+
+    def test_eval_unicode(self):
+        u = "u'%s'" % unichr(0x1234)
+        v = eval(u)
+        assert v == unichr(0x1234)
+
+    def test_compile_unicode(self):
+        s = "x = u'\xd0\xb9\xd1\x86\xd1\x83\xd0\xba\xd0\xb5\xd0\xbd'"
+        u = s.decode('utf-8')
+        c = compile(u, '', 'exec')
+        exec c
+        assert len(x) == 6
+        assert ord(x[0]) == 0x0439

From noreply at buildbot.pypy.org  Tue Sep  6 06:37:14 2011
From: noreply at buildbot.pypy.org (arigo)
Date: Tue,  6 Sep 2011 06:37:14 +0200 (CEST)
Subject: [pypy-commit] pypy default: Fix: only 'exec' was broken.
Message-ID: <20110906043714.92B2B8203C@wyvern.cs.uni-duesseldorf.de>

Author: Armin Rigo 
Branch: 
Changeset: r47099:2c2f3d3849d5
Date: 2011-09-06 06:35 +0200
http://bitbucket.org/pypy/pypy/changeset/2c2f3d3849d5/

Log:	Fix: only 'exec' was broken.

diff --git a/pypy/interpreter/pyopcode.py b/pypy/interpreter/pyopcode.py
--- a/pypy/interpreter/pyopcode.py
+++ b/pypy/interpreter/pyopcode.py
@@ -1523,10 +1523,8 @@
 
         if not isinstance(prog, codetype):
             filename = ''
-            if not isinstance(prog, str):
-                if isinstance(prog, basestring):
-                    prog = str(prog)
-                elif isinstance(prog, file):
+            if not isinstance(prog, basestring):
+                if isinstance(prog, file):
                     filename = prog.name
                     prog = prog.read()
                 else:

From noreply at buildbot.pypy.org  Tue Sep  6 08:19:09 2011
From: noreply at buildbot.pypy.org (alex_gaynor)
Date: Tue,  6 Sep 2011 08:19:09 +0200 (CEST)
Subject: [pypy-commit] pypy default: fix indentation error.
Message-ID: <20110906061909.EE14C8203C@wyvern.cs.uni-duesseldorf.de>

Author: Alex Gaynor 
Branch: 
Changeset: r47100:21dd54a85247
Date: 2011-09-06 06:18 +0000
http://bitbucket.org/pypy/pypy/changeset/21dd54a85247/

Log:	fix indentation error.

diff --git a/pypy/module/micronumpy/test/test_ufuncs.py b/pypy/module/micronumpy/test/test_ufuncs.py
--- a/pypy/module/micronumpy/test/test_ufuncs.py
+++ b/pypy/module/micronumpy/test/test_ufuncs.py
@@ -336,4 +336,4 @@
                 (3.5, 3),
                 (3, 3.5),
             ]:
-                assert ufunc(a, b) is func(a, b)
\ No newline at end of file
+                assert ufunc(a, b) is func(a, b)

From noreply at buildbot.pypy.org  Tue Sep  6 13:02:14 2011
From: noreply at buildbot.pypy.org (arigo)
Date: Tue,  6 Sep 2011 13:02:14 +0200 (CEST)
Subject: [pypy-commit] pypy default: Cast these fields to a Python-level
 'int', i.e. a C 'long'.
Message-ID: <20110906110214.3CFF18203C@wyvern.cs.uni-duesseldorf.de>

Author: Armin Rigo 
Branch: 
Changeset: r47101:9efba345739c
Date: 2011-09-06 13:01 +0200
http://bitbucket.org/pypy/pypy/changeset/9efba345739c/

Log:	Cast these fields to a Python-level 'int', i.e. a C 'long'. This is
	the same as CPython does.

diff --git a/pypy/module/pwd/interp_pwd.py b/pypy/module/pwd/interp_pwd.py
--- a/pypy/module/pwd/interp_pwd.py
+++ b/pypy/module/pwd/interp_pwd.py
@@ -3,6 +3,7 @@
 from pypy.rpython.lltypesystem import rffi, lltype
 from pypy.interpreter.gateway import interp2app, unwrap_spec
 from pypy.interpreter.error import OperationError, operationerrfmt
+from pypy.rlib.rarithmetic import intmask
 
 eci = ExternalCompilationInfo(
     includes=['pwd.h']
@@ -43,8 +44,8 @@
     w_tuple = space.newtuple([
         space.wrap(rffi.charp2str(pw.c_pw_name)),
         space.wrap(rffi.charp2str(pw.c_pw_passwd)),
-        space.wrap(pw.c_pw_uid),
-        space.wrap(pw.c_pw_gid),
+        space.wrap(intmask(pw.c_pw_uid)),
+        space.wrap(intmask(pw.c_pw_gid)),
         space.wrap(rffi.charp2str(pw.c_pw_gecos)),
         space.wrap(rffi.charp2str(pw.c_pw_dir)),
         space.wrap(rffi.charp2str(pw.c_pw_shell)),
diff --git a/pypy/module/pwd/test/test_pwd.py b/pypy/module/pwd/test/test_pwd.py
--- a/pypy/module/pwd/test/test_pwd.py
+++ b/pypy/module/pwd/test/test_pwd.py
@@ -14,6 +14,9 @@
         assert pw.pw_gid == 0
         assert pw.pw_dir == '/root'
         assert pw.pw_shell.startswith('/')
+        #
+        assert type(pw.pw_uid) is int
+        assert type(pw.pw_gid) is int
 
     def test_getpwnam(self):
         import pwd

From noreply at buildbot.pypy.org  Tue Sep  6 13:37:07 2011
From: noreply at buildbot.pypy.org (arigo)
Date: Tue,  6 Sep 2011 13:37:07 +0200 (CEST)
Subject: [pypy-commit] pypy default: Fix this test on 64-bit: avoids that
 random unrelated operations
Message-ID: <20110906113707.5983E8203C@wyvern.cs.uni-duesseldorf.de>

Author: Armin Rigo 
Branch: 
Changeset: r47102:a41ea5a18e1c
Date: 2011-09-06 13:36 +0200
http://bitbucket.org/pypy/pypy/changeset/a41ea5a18e1c/

Log:	Fix this test on 64-bit: avoids that random unrelated operations
	show up here, by adding a dummy getattr previously in the loop.

diff --git a/pypy/module/pypyjit/test_pypy_c/test_instance.py b/pypy/module/pypyjit/test_pypy_c/test_instance.py
--- a/pypy/module/pypyjit/test_pypy_c/test_instance.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_instance.py
@@ -142,6 +142,7 @@
             i = 0
             b = B(1)
             while i < 100:
+                b.x
                 v = b.x # ID: loadattr
                 i += v
             return i
@@ -150,8 +151,6 @@
         loop, = log.loops_by_filename(self.filepath)
         assert loop.match_by_id('loadattr',
         '''
-        guard_not_invalidated(descr=...)
-        i16 = arraylen_gc(p10, descr=)
         i19 = call(ConstClass(ll_dict_lookup), _, _, _, descr=...)
         guard_no_exception(descr=...)
         i21 = int_and(i19, _)

From noreply at buildbot.pypy.org  Tue Sep  6 17:27:20 2011
From: noreply at buildbot.pypy.org (arigo)
Date: Tue,  6 Sep 2011 17:27:20 +0200 (CEST)
Subject: [pypy-commit] pypy default: Store the failargs too. Not used right
	now.
Message-ID: <20110906152720.B811F8203C@wyvern.cs.uni-duesseldorf.de>

Author: Armin Rigo 
Branch: 
Changeset: r47103:862346e1db81
Date: 2011-09-06 17:26 +0200
http://bitbucket.org/pypy/pypy/changeset/862346e1db81/

Log:	Store the failargs too. Not used right now.

diff --git a/pypy/tool/jitlogparser/parser.py b/pypy/tool/jitlogparser/parser.py
--- a/pypy/tool/jitlogparser/parser.py
+++ b/pypy/tool/jitlogparser/parser.py
@@ -8,6 +8,7 @@
     bridge = None
     offset = None
     asm = None
+    failargs = ()
 
     def __init__(self, name, args, res, descr):
         self.name = name
@@ -18,8 +19,8 @@
         if self._is_guard:
             self.guard_no = int(self.descr[len('

Author: Armin Rigo 
Branch: 
Changeset: r47104:572d0576c166
Date: 2011-09-06 17:26 +0200
http://bitbucket.org/pypy/pypy/changeset/572d0576c166/

Log:	Python 2.5 compatibility.

diff --git a/pypy/module/pypyjit/test_pypy_c/model.py b/pypy/module/pypyjit/test_pypy_c/model.py
--- a/pypy/module/pypyjit/test_pypy_c/model.py
+++ b/pypy/module/pypyjit/test_pypy_c/model.py
@@ -2,7 +2,10 @@
 import sys
 import re
 import os.path
-from _pytest.assertion import newinterpret
+try:
+    from _pytest.assertion import newinterpret
+except ImportError:   # e.g. Python 2.5
+    newinterpret = None
 from pypy.tool.jitlogparser.parser import SimpleParser, Function, TraceForOpcode
 from pypy.tool.jitlogparser.storage import LoopStorage
 
@@ -196,7 +199,7 @@
                     source = str(source.deindent()).strip()
         except py.error.ENOENT:
             source = None
-        if source and source.startswith('self._assert('):
+        if source and source.startswith('self._assert(') and newinterpret:
             # transform self._assert(x, 'foo') into assert x, 'foo'
             source = source.replace('self._assert(', 'assert ')
             source = source[:-1] # remove the trailing ')'
diff --git a/pypy/module/pypyjit/test_pypy_c/test_00_model.py b/pypy/module/pypyjit/test_pypy_c/test_00_model.py
--- a/pypy/module/pypyjit/test_pypy_c/test_00_model.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_00_model.py
@@ -1,3 +1,4 @@
+from __future__ import with_statement
 import sys
 import types
 import subprocess

From noreply at buildbot.pypy.org  Tue Sep  6 17:39:40 2011
From: noreply at buildbot.pypy.org (justinpeel)
Date: Tue,  6 Sep 2011 17:39:40 +0200 (CEST)
Subject: [pypy-commit] pypy gc-trace-faster: Attempt to reduce calls to
 trace_and_drag_out_of_nursery_partial. Not as effective as hoped,
 but still gives some improvement.
Message-ID: <20110906153940.37A418203C@wyvern.cs.uni-duesseldorf.de>

Author: Justin Peel 
Branch: gc-trace-faster
Changeset: r47105:2f505084ea21
Date: 2011-09-06 09:38 -0600
http://bitbucket.org/pypy/pypy/changeset/2f505084ea21/

Log:	Attempt to reduce calls to trace_and_drag_out_of_nursery_partial.
	Not as effective as hoped, but still gives some improvement.

diff --git a/pypy/rpython/memory/gc/minimark.py b/pypy/rpython/memory/gc/minimark.py
--- a/pypy/rpython/memory/gc/minimark.py
+++ b/pypy/rpython/memory/gc/minimark.py
@@ -1334,8 +1334,47 @@
                 while bytes > 0:
                     p -= 1
                     cardbyte = ord(p.char[0])
+                    bytes -= 1
+                    if cardbyte == 0:
+                        # keep moving along while there are unmarked bytes
+                        if bytes == 0:
+                            break
+                        p -= 1
+                        cardbyte = ord(p.char[0])
+                        bytes -= 1
+                        counter = 1
+                        while bytes > 0 and cardbyte == 0:
+                            p -= 1
+                            cardbyte = ord(p.char[0])
+                            bytes -= 1
+                            counter += 1
+                        interval_start = interval_start + counter*8*self.card_page_indices
+                    if cardbyte == 255 and bytes > 0:
+                        # keep moving until we find a byte that isn't fully marked
+                        p.char[0] = '\x00'
+                        counter = 1
+                        p -= 1
+                        cardbyte = ord(p.char[0])
+                        bytes -= 1
+                        while bytes > 0 and cardbyte == 255:
+                            p.char[0] = '\x00'
+                            p -= 1
+                            cardbyte = ord(p.char[0])
+                            bytes -= 1
+                            counter += 1
+                        interval_stop = interval_start + counter*8*self.card_page_indices
+                        if interval_stop > length:
+                            interval_stop = length
+                            ll_assert(bytes == 0,
+                                "premature end of object")
+                        if bool(self.young_rawmalloced_objects):
+                            self.trace_and_drag_out_of_nursery_partial_young_raw(
+                                obj, interval_start, interval_stop)
+                        else:
+                            self.trace_and_drag_out_of_nursery_partial(
+                                obj, interval_start, interval_stop)
+                        interval_start = interval_stop
                     p.char[0] = '\x00'           # reset the bits
-                    bytes -= 1
                     next_byte_start = interval_start + 8*self.card_page_indices
                     #
                     while cardbyte != 0:

From noreply at buildbot.pypy.org  Tue Sep  6 18:04:34 2011
From: noreply at buildbot.pypy.org (antocuni)
Date: Tue,  6 Sep 2011 18:04:34 +0200 (CEST)
Subject: [pypy-commit] pypy ffistruct: a branch to implement a JIT friendly
 struct type in _ffi
Message-ID: <20110906160434.88F008203C@wyvern.cs.uni-duesseldorf.de>

Author: Antonio Cuni 
Branch: ffistruct
Changeset: r47106:8b5b5a7627cf
Date: 2011-09-05 15:43 +0200
http://bitbucket.org/pypy/pypy/changeset/8b5b5a7627cf/

Log:	a branch to implement a JIT friendly struct type in _ffi


From noreply at buildbot.pypy.org  Tue Sep  6 18:04:36 2011
From: noreply at buildbot.pypy.org (antocuni)
Date: Tue,  6 Sep 2011 18:04:36 +0200 (CEST)
Subject: [pypy-commit] pypy ffistruct: implement FFIType.sizeof();
 put some test logic into a base class
Message-ID: <20110906160436.053D48203C@wyvern.cs.uni-duesseldorf.de>

Author: Antonio Cuni 
Branch: ffistruct
Changeset: r47107:318937fd8e2e
Date: 2011-09-05 16:24 +0200
http://bitbucket.org/pypy/pypy/changeset/318937fd8e2e/

Log:	implement FFIType.sizeof(); put some test logic into a base class

diff --git a/pypy/module/_ffi/__init__.py b/pypy/module/_ffi/__init__.py
--- a/pypy/module/_ffi/__init__.py
+++ b/pypy/module/_ffi/__init__.py
@@ -10,4 +10,7 @@
         'get_libc':'interp_ffi.get_libc',
     }
 
-    appleveldefs = {}
+    appleveldefs = {
+        'Structure': 'app_struct.Structure',
+        'Field':     'app_struct.Field',
+        }
diff --git a/pypy/module/_ffi/interp_ffi.py b/pypy/module/_ffi/interp_ffi.py
--- a/pypy/module/_ffi/interp_ffi.py
+++ b/pypy/module/_ffi/interp_ffi.py
@@ -30,6 +30,12 @@
             return space.w_None
         return self.w_pointer_to
 
+    def descr_sizeof(self, space):
+        return space.wrap(self.sizeof())
+
+    def sizeof(self):
+        return intmask(self.ffitype.c_size)
+
     def repr(self, space):
         return space.wrap(self.__repr__())
 
@@ -86,6 +92,7 @@
     'FFIType',
     __repr__ = interp2app(W_FFIType.repr),
     deref_pointer = interp2app(W_FFIType.descr_deref_pointer),
+    sizeof = interp2app(W_FFIType.descr_sizeof),
     )
 
 
diff --git a/pypy/module/_ffi/test/test__ffi.py b/pypy/module/_ffi/test/test__ffi.py
--- a/pypy/module/_ffi/test/test__ffi.py
+++ b/pypy/module/_ffi/test/test__ffi.py
@@ -7,7 +7,7 @@
 
 import os, sys, py
 
-class AppTestFfi:
+class BaseAppTestFFI(object):
 
     @classmethod
     def prepare_c_example(cls):
@@ -36,7 +36,6 @@
         eci = ExternalCompilationInfo(export_symbols=[])
         return str(platform.compile([c_file], eci, 'x', standalone=False))
 
-    
     def setup_class(cls):
         from pypy.rpython.lltypesystem import rffi
         from pypy.rlib.libffi import get_libc_name, CDLL, types
@@ -52,7 +51,12 @@
         pow = libm.getpointer('pow', [], types.void)
         pow_addr = rffi.cast(rffi.LONG, pow.funcsym)
         cls.w_pow_addr = space.wrap(pow_addr)
-        #
+
+class AppTestFFI(BaseAppTestFFI):
+
+    def setup_class(cls):
+        BaseAppTestFFI.setup_class.im_func(cls)
+        space = cls.space
         # these are needed for test_single_float_args
         from ctypes import c_float
         f_12_34 = c_float(12.34).value
@@ -82,7 +86,12 @@
         from _ffi import types
         assert str(types.sint) == ""
         assert str(types.uint) == ""
-        
+
+    def test_sizeof(self):
+        from _ffi import types
+        assert types.sbyte.sizeof() == 1
+        assert types.sint.sizeof() == 4
+    
     def test_callfunc(self):
         from _ffi import CDLL, types
         libm = CDLL(self.libm_name)

From noreply at buildbot.pypy.org  Tue Sep  6 18:04:37 2011
From: noreply at buildbot.pypy.org (antocuni)
Date: Tue,  6 Sep 2011 18:04:37 +0200 (CEST)
Subject: [pypy-commit] pypy ffistruct: start to implement _ffi.Structure
Message-ID: <20110906160437.8D2EA8203C@wyvern.cs.uni-duesseldorf.de>

Author: Antonio Cuni 
Branch: ffistruct
Changeset: r47108:6fe3c422d545
Date: 2011-09-05 16:37 +0200
http://bitbucket.org/pypy/pypy/changeset/6fe3c422d545/

Log:	start to implement _ffi.Structure

diff --git a/pypy/module/_ffi/app_struct.py b/pypy/module/_ffi/app_struct.py
new file mode 100644
--- /dev/null
+++ b/pypy/module/_ffi/app_struct.py
@@ -0,0 +1,29 @@
+class Field(object):
+
+    def __init__(self, name, ffitype):
+        self.name = name
+        self.ffitype = ffitype
+        self.offset = -1
+
+class MetaStructure(type):
+
+    def __new__(cls, name, bases, dic):
+        cls._compute_shape(dic)
+        return type.__new__(cls, name, bases, dic)
+
+    @classmethod
+    def _compute_shape(cls, dic):
+        fields = dic.get('_fields_')
+        if fields is None:
+            return
+        size = 0
+        for field in fields:
+            field.offset = size # XXX: alignment!
+            size += field.ffitype.sizeof()
+            dic[field.name] = field
+        dic['_size_'] = size
+
+
+class Structure(object):
+
+    __metaclass__ = MetaStructure
diff --git a/pypy/module/_ffi/test/test_struct.py b/pypy/module/_ffi/test/test_struct.py
new file mode 100644
--- /dev/null
+++ b/pypy/module/_ffi/test/test_struct.py
@@ -0,0 +1,19 @@
+from pypy.module._ffi.test.test__ffi import BaseAppTestFFI
+
+class AppTestStruct(BaseAppTestFFI):
+
+    def test_compute_shape(self):
+        from _ffi import Structure, Field, types
+        class Point(Structure):
+            _fields_ = [
+                Field('x', types.slong),
+                Field('y', types.slong),
+                ]
+
+        longsize = types.slong.sizeof()
+        assert isinstance(Point.x, Field)
+        assert isinstance(Point.y, Field)
+        assert Point.x.offset == 0
+        assert Point.y.offset == longsize
+        assert Point._size_ == longsize*2
+        

From noreply at buildbot.pypy.org  Tue Sep  6 18:04:39 2011
From: noreply at buildbot.pypy.org (antocuni)
Date: Tue,  6 Sep 2011 18:04:39 +0200 (CEST)
Subject: [pypy-commit] pypy ffistruct: split the implementation/tests of
 W_FFIType and W_FuncPtr into two separate files
Message-ID: <20110906160439.2CAEC8203C@wyvern.cs.uni-duesseldorf.de>

Author: Antonio Cuni 
Branch: ffistruct
Changeset: r47109:edee74efbb9c
Date: 2011-09-05 16:47 +0200
http://bitbucket.org/pypy/pypy/changeset/edee74efbb9c/

Log:	split the implementation/tests of W_FFIType and W_FuncPtr into two
	separate files

diff --git a/pypy/module/_ffi/__init__.py b/pypy/module/_ffi/__init__.py
--- a/pypy/module/_ffi/__init__.py
+++ b/pypy/module/_ffi/__init__.py
@@ -1,13 +1,12 @@
 from pypy.interpreter.mixedmodule import MixedModule
-from pypy.module._ffi import interp_ffi
 
 class Module(MixedModule):
 
     interpleveldefs = {
-        'CDLL':    'interp_ffi.W_CDLL',
-        'types':   'interp_ffi.W_types',
-        'FuncPtr': 'interp_ffi.W_FuncPtr',
-        'get_libc':'interp_ffi.get_libc',
+        'types':   'interp_ffitype.W_types',
+        'CDLL':    'interp_funcptr.W_CDLL',
+        'FuncPtr': 'interp_funcptr.W_FuncPtr',
+        'get_libc':'interp_funcptr.get_libc',
     }
 
     appleveldefs = {
diff --git a/pypy/module/_ffi/app_struct.py b/pypy/module/_ffi/app_struct.py
--- a/pypy/module/_ffi/app_struct.py
+++ b/pypy/module/_ffi/app_struct.py
@@ -5,6 +5,15 @@
         self.ffitype = ffitype
         self.offset = -1
 
+    ## def __get__(self, obj, cls=None):
+    ##     if obj is None:
+    ##         return self
+    ##     return getfield(obj._buffer, self.ffitype, self.offset)
+
+    ## def __set__(self, obj, value):
+    ##     setfield(obj._buffer, self.ffitype, self.offset, value)
+
+
 class MetaStructure(type):
 
     def __new__(cls, name, bases, dic):
diff --git a/pypy/module/_ffi/interp_ffitype.py b/pypy/module/_ffi/interp_ffitype.py
new file mode 100644
--- /dev/null
+++ b/pypy/module/_ffi/interp_ffitype.py
@@ -0,0 +1,156 @@
+from pypy.rlib import libffi
+from pypy.rlib.rarithmetic import intmask
+from pypy.interpreter.baseobjspace import Wrappable
+from pypy.interpreter.typedef import TypeDef
+from pypy.interpreter.gateway import interp2app
+
+class W_FFIType(Wrappable):
+
+    _immutable_fields_ = ['name', 'ffitype', 'w_datashape', 'w_pointer_to']
+
+    def __init__(self, name, ffitype, w_datashape=None, w_pointer_to=None):
+        self.name = name
+        self.ffitype = ffitype
+        self.w_datashape = w_datashape
+        self.w_pointer_to = w_pointer_to
+        if self.is_struct():
+            assert w_datashape is not None
+
+    def descr_deref_pointer(self, space):
+        if self.w_pointer_to is None:
+            return space.w_None
+        return self.w_pointer_to
+
+    def descr_sizeof(self, space):
+        return space.wrap(self.sizeof())
+
+    def sizeof(self):
+        return intmask(self.ffitype.c_size)
+
+    def repr(self, space):
+        return space.wrap(self.__repr__())
+
+    def __repr__(self):
+        return "" % self.name
+
+    def is_signed(self):
+        return (self is app_types.slong or
+                self is app_types.sint or
+                self is app_types.sshort or
+                self is app_types.sbyte or
+                self is app_types.slonglong)
+
+    def is_unsigned(self):
+        return (self is app_types.ulong or
+                self is app_types.uint or
+                self is app_types.ushort or
+                self is app_types.ubyte or
+                self is app_types.ulonglong)
+
+    def is_pointer(self):
+        return self.ffitype is libffi.types.pointer
+
+    def is_char(self):
+        return self is app_types.char
+
+    def is_unichar(self):
+        return self is app_types.unichar
+
+    def is_longlong(self):
+        return libffi.IS_32_BIT and (self is app_types.slonglong or
+                                     self is app_types.ulonglong)
+
+    def is_double(self):
+        return self is app_types.double
+
+    def is_singlefloat(self):
+        return self is app_types.float
+
+    def is_void(self):
+        return self is app_types.void
+
+    def is_struct(self):
+        return libffi.types.is_struct(self.ffitype)
+
+    def is_char_p(self):
+        return self is app_types.char_p
+
+    def is_unichar_p(self):
+        return self is app_types.unichar_p
+
+
+W_FFIType.typedef = TypeDef(
+    'FFIType',
+    __repr__ = interp2app(W_FFIType.repr),
+    deref_pointer = interp2app(W_FFIType.descr_deref_pointer),
+    sizeof = interp2app(W_FFIType.descr_sizeof),
+    )
+
+
+def build_ffi_types():
+    types = [
+        # note: most of the type name directly come from the C equivalent,
+        # with the exception of bytes: in C, ubyte and char are equivalent,
+        # but for _ffi the first expects a number while the second a 1-length
+        # string
+        W_FFIType('slong',     libffi.types.slong),
+        W_FFIType('sint',      libffi.types.sint),
+        W_FFIType('sshort',    libffi.types.sshort),
+        W_FFIType('sbyte',     libffi.types.schar),
+        W_FFIType('slonglong', libffi.types.slonglong),
+        #
+        W_FFIType('ulong',     libffi.types.ulong),
+        W_FFIType('uint',      libffi.types.uint),
+        W_FFIType('ushort',    libffi.types.ushort),
+        W_FFIType('ubyte',     libffi.types.uchar),
+        W_FFIType('ulonglong', libffi.types.ulonglong),
+        #
+        W_FFIType('char',      libffi.types.uchar),
+        W_FFIType('unichar',   libffi.types.wchar_t),
+        #
+        W_FFIType('double',    libffi.types.double),
+        W_FFIType('float',     libffi.types.float),
+        W_FFIType('void',      libffi.types.void),
+        W_FFIType('void_p',    libffi.types.pointer),
+        #
+        # missing types:
+
+        ## 's' : ffi_type_pointer,
+        ## 'z' : ffi_type_pointer,
+        ## 'O' : ffi_type_pointer,
+        ## 'Z' : ffi_type_pointer,
+
+        ]
+    d = dict([(t.name, t) for t in types])
+    w_char = d['char']
+    w_unichar = d['unichar']
+    d['char_p'] = W_FFIType('char_p', libffi.types.pointer, w_pointer_to = w_char)
+    d['unichar_p'] = W_FFIType('unichar_p', libffi.types.pointer, w_pointer_to = w_unichar)
+    return d
+
+class app_types:
+    pass
+app_types.__dict__ = build_ffi_types()
+
+def descr_new_pointer(space, w_cls, w_pointer_to):
+    try:
+        return descr_new_pointer.cache[w_pointer_to]
+    except KeyError:
+        if w_pointer_to is app_types.char:
+            w_result = app_types.char_p
+        elif w_pointer_to is app_types.unichar:
+            w_result = app_types.unichar_p
+        else:
+            w_pointer_to = space.interp_w(W_FFIType, w_pointer_to)
+            name = '(pointer to %s)' % w_pointer_to.name
+            w_result = W_FFIType(name, libffi.types.pointer, w_pointer_to = w_pointer_to)
+        descr_new_pointer.cache[w_pointer_to] = w_result
+        return w_result
+descr_new_pointer.cache = {}
+
+class W_types(Wrappable):
+    pass
+W_types.typedef = TypeDef(
+    'types',
+    Pointer = interp2app(descr_new_pointer, as_classmethod=True),
+    **app_types.__dict__)
diff --git a/pypy/module/_ffi/interp_ffi.py b/pypy/module/_ffi/interp_funcptr.py
rename from pypy/module/_ffi/interp_ffi.py
rename to pypy/module/_ffi/interp_funcptr.py
--- a/pypy/module/_ffi/interp_ffi.py
+++ b/pypy/module/_ffi/interp_funcptr.py
@@ -4,6 +4,7 @@
 from pypy.interpreter.gateway import interp2app, unwrap_spec
 from pypy.interpreter.typedef import TypeDef
 from pypy.module._rawffi.structure import W_StructureInstance, W_Structure
+from pypy.module._ffi.interp_ffitype import W_FFIType
 #
 from pypy.rpython.lltypesystem import lltype, rffi
 #
@@ -13,157 +14,6 @@
 from pypy.rlib.rarithmetic import intmask, r_uint
 from pypy.rlib.objectmodel import we_are_translated
 
-class W_FFIType(Wrappable):
-
-    _immutable_fields_ = ['name', 'ffitype', 'w_datashape', 'w_pointer_to']
-
-    def __init__(self, name, ffitype, w_datashape=None, w_pointer_to=None):
-        self.name = name
-        self.ffitype = ffitype
-        self.w_datashape = w_datashape
-        self.w_pointer_to = w_pointer_to
-        if self.is_struct():
-            assert w_datashape is not None
-
-    def descr_deref_pointer(self, space):
-        if self.w_pointer_to is None:
-            return space.w_None
-        return self.w_pointer_to
-
-    def descr_sizeof(self, space):
-        return space.wrap(self.sizeof())
-
-    def sizeof(self):
-        return intmask(self.ffitype.c_size)
-
-    def repr(self, space):
-        return space.wrap(self.__repr__())
-
-    def __repr__(self):
-        return "" % self.name
-
-    def is_signed(self):
-        return (self is app_types.slong or
-                self is app_types.sint or
-                self is app_types.sshort or
-                self is app_types.sbyte or
-                self is app_types.slonglong)
-
-    def is_unsigned(self):
-        return (self is app_types.ulong or
-                self is app_types.uint or
-                self is app_types.ushort or
-                self is app_types.ubyte or
-                self is app_types.ulonglong)
-
-    def is_pointer(self):
-        return self.ffitype is libffi.types.pointer
-
-    def is_char(self):
-        return self is app_types.char
-
-    def is_unichar(self):
-        return self is app_types.unichar
-
-    def is_longlong(self):
-        return libffi.IS_32_BIT and (self is app_types.slonglong or
-                                     self is app_types.ulonglong)
-
-    def is_double(self):
-        return self is app_types.double
-
-    def is_singlefloat(self):
-        return self is app_types.float
-
-    def is_void(self):
-        return self is app_types.void
-
-    def is_struct(self):
-        return libffi.types.is_struct(self.ffitype)
-
-    def is_char_p(self):
-        return self is app_types.char_p
-
-    def is_unichar_p(self):
-        return self is app_types.unichar_p
-
-
-W_FFIType.typedef = TypeDef(
-    'FFIType',
-    __repr__ = interp2app(W_FFIType.repr),
-    deref_pointer = interp2app(W_FFIType.descr_deref_pointer),
-    sizeof = interp2app(W_FFIType.descr_sizeof),
-    )
-
-
-def build_ffi_types():
-    types = [
-        # note: most of the type name directly come from the C equivalent,
-        # with the exception of bytes: in C, ubyte and char are equivalent,
-        # but for _ffi the first expects a number while the second a 1-length
-        # string
-        W_FFIType('slong',     libffi.types.slong),
-        W_FFIType('sint',      libffi.types.sint),
-        W_FFIType('sshort',    libffi.types.sshort),
-        W_FFIType('sbyte',     libffi.types.schar),
-        W_FFIType('slonglong', libffi.types.slonglong),
-        #
-        W_FFIType('ulong',     libffi.types.ulong),
-        W_FFIType('uint',      libffi.types.uint),
-        W_FFIType('ushort',    libffi.types.ushort),
-        W_FFIType('ubyte',     libffi.types.uchar),
-        W_FFIType('ulonglong', libffi.types.ulonglong),
-        #
-        W_FFIType('char',      libffi.types.uchar),
-        W_FFIType('unichar',   libffi.types.wchar_t),
-        #
-        W_FFIType('double',    libffi.types.double),
-        W_FFIType('float',     libffi.types.float),
-        W_FFIType('void',      libffi.types.void),
-        W_FFIType('void_p',    libffi.types.pointer),
-        #
-        # missing types:
-
-        ## 's' : ffi_type_pointer,
-        ## 'z' : ffi_type_pointer,
-        ## 'O' : ffi_type_pointer,
-        ## 'Z' : ffi_type_pointer,
-
-        ]
-    d = dict([(t.name, t) for t in types])
-    w_char = d['char']
-    w_unichar = d['unichar']
-    d['char_p'] = W_FFIType('char_p', libffi.types.pointer, w_pointer_to = w_char)
-    d['unichar_p'] = W_FFIType('unichar_p', libffi.types.pointer, w_pointer_to = w_unichar)
-    return d
-
-class app_types:
-    pass
-app_types.__dict__ = build_ffi_types()
-
-def descr_new_pointer(space, w_cls, w_pointer_to):
-    try:
-        return descr_new_pointer.cache[w_pointer_to]
-    except KeyError:
-        if w_pointer_to is app_types.char:
-            w_result = app_types.char_p
-        elif w_pointer_to is app_types.unichar:
-            w_result = app_types.unichar_p
-        else:
-            w_pointer_to = space.interp_w(W_FFIType, w_pointer_to)
-            name = '(pointer to %s)' % w_pointer_to.name
-            w_result = W_FFIType(name, libffi.types.pointer, w_pointer_to = w_pointer_to)
-        descr_new_pointer.cache[w_pointer_to] = w_result
-        return w_result
-descr_new_pointer.cache = {}
-
-class W_types(Wrappable):
-    pass
-W_types.typedef = TypeDef(
-    'types',
-    Pointer = interp2app(descr_new_pointer, as_classmethod=True),
-    **app_types.__dict__)
-
 
 def unwrap_ffitype(space, w_argtype, allow_void=False):
     res = w_argtype.ffitype
diff --git a/pypy/module/_ffi/test/test_ffitype.py b/pypy/module/_ffi/test/test_ffitype.py
new file mode 100644
--- /dev/null
+++ b/pypy/module/_ffi/test/test_ffitype.py
@@ -0,0 +1,37 @@
+from pypy.module._ffi.test.test_funcptr import BaseAppTestFFI
+
+class AppTestFFIType(BaseAppTestFFI):
+
+    def test_simple_types(self):
+        from _ffi import types
+        assert str(types.sint) == ""
+        assert str(types.uint) == ""
+
+    def test_sizeof(self):
+        from _ffi import types
+        assert types.sbyte.sizeof() == 1
+        assert types.sint.sizeof() == 4
+
+    def test_typed_pointer(self):
+        from _ffi import types
+        intptr = types.Pointer(types.sint) # create a typed pointer to sint
+        assert intptr.deref_pointer() is types.sint
+        assert str(intptr) == ''
+        assert types.sint.deref_pointer() is None
+        raises(TypeError, "types.Pointer(42)")
+
+    def test_pointer_identity(self):
+        from _ffi import types
+        x = types.Pointer(types.slong)
+        y = types.Pointer(types.slong)
+        z = types.Pointer(types.char)
+        assert x is y
+        assert x is not z
+
+    def test_char_p_cached(self):
+        from _ffi import types
+        x = types.Pointer(types.char)
+        assert x is types.char_p
+        x = types.Pointer(types.unichar)
+        assert x is types.unichar_p
+
diff --git a/pypy/module/_ffi/test/test__ffi.py b/pypy/module/_ffi/test/test_funcptr.py
rename from pypy/module/_ffi/test/test__ffi.py
rename to pypy/module/_ffi/test/test_funcptr.py
--- a/pypy/module/_ffi/test/test__ffi.py
+++ b/pypy/module/_ffi/test/test_funcptr.py
@@ -82,16 +82,6 @@
         res = dll.getfunc('Py_IsInitialized', [], types.slong)()
         assert res == 1
 
-    def test_simple_types(self):
-        from _ffi import types
-        assert str(types.sint) == ""
-        assert str(types.uint) == ""
-
-    def test_sizeof(self):
-        from _ffi import types
-        assert types.sbyte.sizeof() == 1
-        assert types.sint.sizeof() == 4
-    
     def test_callfunc(self):
         from _ffi import CDLL, types
         libm = CDLL(self.libm_name)
@@ -266,29 +256,6 @@
         assert list(array) == list('foobar\00')
         do_nothing.free_temp_buffers()
 
-    def test_typed_pointer(self):
-        from _ffi import types
-        intptr = types.Pointer(types.sint) # create a typed pointer to sint
-        assert intptr.deref_pointer() is types.sint
-        assert str(intptr) == ''
-        assert types.sint.deref_pointer() is None
-        raises(TypeError, "types.Pointer(42)")
-
-    def test_pointer_identity(self):
-        from _ffi import types
-        x = types.Pointer(types.slong)
-        y = types.Pointer(types.slong)
-        z = types.Pointer(types.char)
-        assert x is y
-        assert x is not z
-
-    def test_char_p_cached(self):
-        from _ffi import types
-        x = types.Pointer(types.char)
-        assert x is types.char_p
-        x = types.Pointer(types.unichar)
-        assert x is types.unichar_p
-
     def test_typed_pointer_args(self):
         """
             extern int dummy; // defined in test_void_result 
diff --git a/pypy/module/_ffi/test/test_struct.py b/pypy/module/_ffi/test/test_struct.py
--- a/pypy/module/_ffi/test/test_struct.py
+++ b/pypy/module/_ffi/test/test_struct.py
@@ -1,4 +1,4 @@
-from pypy.module._ffi.test.test__ffi import BaseAppTestFFI
+from pypy.module._ffi.test.test_funcptr import BaseAppTestFFI
 
 class AppTestStruct(BaseAppTestFFI):
 

From noreply at buildbot.pypy.org  Tue Sep  6 18:04:40 2011
From: noreply at buildbot.pypy.org (antocuni)
Date: Tue,  6 Sep 2011 18:04:40 +0200 (CEST)
Subject: [pypy-commit] pypy ffistruct: introduce the concept of _StructDescr,
 which describes the layout and ffitype of a structure
Message-ID: <20110906160440.C55018203C@wyvern.cs.uni-duesseldorf.de>

Author: Antonio Cuni 
Branch: ffistruct
Changeset: r47110:b8cb7ac0d45a
Date: 2011-09-06 16:33 +0200
http://bitbucket.org/pypy/pypy/changeset/b8cb7ac0d45a/

Log:	introduce the concept of _StructDescr, which describes the layout
	and ffitype of a structure

diff --git a/pypy/module/_ffi/__init__.py b/pypy/module/_ffi/__init__.py
--- a/pypy/module/_ffi/__init__.py
+++ b/pypy/module/_ffi/__init__.py
@@ -7,6 +7,7 @@
         'CDLL':    'interp_funcptr.W_CDLL',
         'FuncPtr': 'interp_funcptr.W_FuncPtr',
         'get_libc':'interp_funcptr.get_libc',
+        '_StructDescr': 'interp_struct.W__StructDescr',
     }
 
     appleveldefs = {
diff --git a/pypy/module/_ffi/app_struct.py b/pypy/module/_ffi/app_struct.py
--- a/pypy/module/_ffi/app_struct.py
+++ b/pypy/module/_ffi/app_struct.py
@@ -1,3 +1,5 @@
+import _ffi
+
 class Field(object):
 
     def __init__(self, name, ffitype):
@@ -13,7 +15,6 @@
     ## def __set__(self, obj, value):
     ##     setfield(obj._buffer, self.ffitype, self.offset, value)
 
-
 class MetaStructure(type):
 
     def __new__(cls, name, bases, dic):
@@ -26,13 +27,16 @@
         if fields is None:
             return
         size = 0
+        ffitypes = []
         for field in fields:
             field.offset = size # XXX: alignment!
             size += field.ffitype.sizeof()
+            ffitypes.append(field.ffitype)
             dic[field.name] = field
-        dic['_size_'] = size
+        alignment = 0 # XXX
+        struct_descr = _ffi._StructDescr(size, alignment, ffitypes)
+        dic['_struct_'] = struct_descr
 
 
 class Structure(object):
-
     __metaclass__ = MetaStructure
diff --git a/pypy/module/_ffi/test/test_struct.py b/pypy/module/_ffi/test/test_struct.py
--- a/pypy/module/_ffi/test/test_struct.py
+++ b/pypy/module/_ffi/test/test_struct.py
@@ -2,6 +2,13 @@
 
 class AppTestStruct(BaseAppTestFFI):
 
+    def test__StructDescr(self):
+        from _ffi import _StructDescr, types
+        longsize = types.slong.sizeof()
+        descr = _StructDescr(longsize*2, 0, [types.slong, types.slong])
+        assert descr.ffitype.sizeof() == longsize*2
+        assert repr(descr.ffitype) == '>'
+
     def test_compute_shape(self):
         from _ffi import Structure, Field, types
         class Point(Structure):
@@ -15,5 +22,4 @@
         assert isinstance(Point.y, Field)
         assert Point.x.offset == 0
         assert Point.y.offset == longsize
-        assert Point._size_ == longsize*2
-        
+        assert Point._struct_.ffitype.sizeof() == longsize*2

From noreply at buildbot.pypy.org  Tue Sep  6 18:04:42 2011
From: noreply at buildbot.pypy.org (antocuni)
Date: Tue,  6 Sep 2011 18:04:42 +0200 (CEST)
Subject: [pypy-commit] pypy ffistruct: store also the struct name in the
	descr
Message-ID: <20110906160442.883BB8203C@wyvern.cs.uni-duesseldorf.de>

Author: Antonio Cuni 
Branch: ffistruct
Changeset: r47111:69a6b100e900
Date: 2011-09-06 16:36 +0200
http://bitbucket.org/pypy/pypy/changeset/69a6b100e900/

Log:	store also the struct name in the descr

diff --git a/pypy/module/_ffi/app_struct.py b/pypy/module/_ffi/app_struct.py
--- a/pypy/module/_ffi/app_struct.py
+++ b/pypy/module/_ffi/app_struct.py
@@ -18,11 +18,11 @@
 class MetaStructure(type):
 
     def __new__(cls, name, bases, dic):
-        cls._compute_shape(dic)
+        cls._compute_shape(name, dic)
         return type.__new__(cls, name, bases, dic)
 
     @classmethod
-    def _compute_shape(cls, dic):
+    def _compute_shape(cls, name, dic):
         fields = dic.get('_fields_')
         if fields is None:
             return
@@ -34,7 +34,7 @@
             ffitypes.append(field.ffitype)
             dic[field.name] = field
         alignment = 0 # XXX
-        struct_descr = _ffi._StructDescr(size, alignment, ffitypes)
+        struct_descr = _ffi._StructDescr(name, size, alignment, ffitypes)
         dic['_struct_'] = struct_descr
 
 
diff --git a/pypy/module/_ffi/test/test_struct.py b/pypy/module/_ffi/test/test_struct.py
--- a/pypy/module/_ffi/test/test_struct.py
+++ b/pypy/module/_ffi/test/test_struct.py
@@ -5,9 +5,10 @@
     def test__StructDescr(self):
         from _ffi import _StructDescr, types
         longsize = types.slong.sizeof()
-        descr = _StructDescr(longsize*2, 0, [types.slong, types.slong])
+        descr = _StructDescr('foo', longsize*2, 0, [types.slong, types.slong])
+        assert descr.name == 'foo'
         assert descr.ffitype.sizeof() == longsize*2
-        assert repr(descr.ffitype) == '>'
+        assert repr(descr.ffitype) == ''
 
     def test_compute_shape(self):
         from _ffi import Structure, Field, types
@@ -22,4 +23,5 @@
         assert isinstance(Point.y, Field)
         assert Point.x.offset == 0
         assert Point.y.offset == longsize
+        assert Point._struct_.name == 'Point'
         assert Point._struct_.ffitype.sizeof() == longsize*2

From noreply at buildbot.pypy.org  Tue Sep  6 18:04:44 2011
From: noreply at buildbot.pypy.org (antocuni)
Date: Tue,  6 Sep 2011 18:04:44 +0200 (CEST)
Subject: [pypy-commit] pypy ffistruct: don't store the name on the struct
 descr, but make it accessible from the ffi type
Message-ID: <20110906160444.122DB8203C@wyvern.cs.uni-duesseldorf.de>

Author: Antonio Cuni 
Branch: ffistruct
Changeset: r47112:ced67009fa52
Date: 2011-09-06 16:39 +0200
http://bitbucket.org/pypy/pypy/changeset/ced67009fa52/

Log:	don't store the name on the struct descr, but make it accessible
	from the ffi type

diff --git a/pypy/module/_ffi/interp_ffitype.py b/pypy/module/_ffi/interp_ffitype.py
--- a/pypy/module/_ffi/interp_ffitype.py
+++ b/pypy/module/_ffi/interp_ffitype.py
@@ -1,7 +1,7 @@
 from pypy.rlib import libffi
 from pypy.rlib.rarithmetic import intmask
 from pypy.interpreter.baseobjspace import Wrappable
-from pypy.interpreter.typedef import TypeDef
+from pypy.interpreter.typedef import TypeDef, interp_attrproperty
 from pypy.interpreter.gateway import interp2app
 
 class W_FFIType(Wrappable):
@@ -81,6 +81,7 @@
 
 W_FFIType.typedef = TypeDef(
     'FFIType',
+    name = interp_attrproperty('name', W_FFIType),
     __repr__ = interp2app(W_FFIType.repr),
     deref_pointer = interp2app(W_FFIType.descr_deref_pointer),
     sizeof = interp2app(W_FFIType.descr_sizeof),
diff --git a/pypy/module/_ffi/test/test_ffitype.py b/pypy/module/_ffi/test/test_ffitype.py
--- a/pypy/module/_ffi/test/test_ffitype.py
+++ b/pypy/module/_ffi/test/test_ffitype.py
@@ -6,7 +6,9 @@
         from _ffi import types
         assert str(types.sint) == ""
         assert str(types.uint) == ""
-
+        assert types.sint.name == 'sint'
+        assert types.uint.name == 'uint'
+        
     def test_sizeof(self):
         from _ffi import types
         assert types.sbyte.sizeof() == 1
diff --git a/pypy/module/_ffi/test/test_struct.py b/pypy/module/_ffi/test/test_struct.py
--- a/pypy/module/_ffi/test/test_struct.py
+++ b/pypy/module/_ffi/test/test_struct.py
@@ -6,9 +6,8 @@
         from _ffi import _StructDescr, types
         longsize = types.slong.sizeof()
         descr = _StructDescr('foo', longsize*2, 0, [types.slong, types.slong])
-        assert descr.name == 'foo'
         assert descr.ffitype.sizeof() == longsize*2
-        assert repr(descr.ffitype) == ''
+        assert descr.ffitype.name == 'struct foo'
 
     def test_compute_shape(self):
         from _ffi import Structure, Field, types
@@ -23,5 +22,6 @@
         assert isinstance(Point.y, Field)
         assert Point.x.offset == 0
         assert Point.y.offset == longsize
-        assert Point._struct_.name == 'Point'
         assert Point._struct_.ffitype.sizeof() == longsize*2
+        assert Point._struct_.ffitype.name == 'struct Point'
+        

From noreply at buildbot.pypy.org  Tue Sep  6 19:06:32 2011
From: noreply at buildbot.pypy.org (snus_mumrik)
Date: Tue,  6 Sep 2011 19:06:32 +0200 (CEST)
Subject: [pypy-commit] pypy numpy-indexing-by-arrays: Branch for adding
 "index arrays" to numpy
Message-ID: <20110906170632.981E38203C@wyvern.cs.uni-duesseldorf.de>

Author: Ilya Osadchiy 
Branch: numpy-indexing-by-arrays
Changeset: r47113:41bb9c2e7f3a
Date: 2011-09-03 14:25 +0300
http://bitbucket.org/pypy/pypy/changeset/41bb9c2e7f3a/

Log:	Branch for adding "index arrays" to numpy


From noreply at buildbot.pypy.org  Tue Sep  6 19:06:33 2011
From: noreply at buildbot.pypy.org (snus_mumrik)
Date: Tue,  6 Sep 2011 19:06:33 +0200 (CEST)
Subject: [pypy-commit] pypy numpy-indexing-by-arrays: Initial implementation
Message-ID: <20110906170633.D19AF8203C@wyvern.cs.uni-duesseldorf.de>

Author: Ilya Osadchiy 
Branch: numpy-indexing-by-arrays
Changeset: r47114:7e7b4f1c2c5c
Date: 2011-09-05 22:24 +0300
http://bitbucket.org/pypy/pypy/changeset/7e7b4f1c2c5c/

Log:	Initial implementation

diff --git a/pypy/module/micronumpy/interp_numarray.py b/pypy/module/micronumpy/interp_numarray.py
--- a/pypy/module/micronumpy/interp_numarray.py
+++ b/pypy/module/micronumpy/interp_numarray.py
@@ -217,7 +217,6 @@
         return space.wrap("[" + " ".join(concrete._getnums(True)) + "]")
 
     def descr_getitem(self, space, w_idx):
-        # TODO: indexing by arrays and lists
         if space.isinstance_w(w_idx, space.w_tuple):
             length = space.len_w(w_idx)
             if length == 0:
@@ -226,6 +225,24 @@
                 raise OperationError(space.w_IndexError,
                                      space.wrap("invalid index"))
             w_idx = space.getitem(w_idx, space.wrap(0))
+        elif space.issequence_w(w_idx):
+            w_idx = convert_to_array(space, w_idx)
+            bool_dtype = space.fromcache(interp_dtype.W_BoolDtype)
+            int_dtype = space.fromcache(interp_dtype.W_Int64Dtype)
+            if w_idx.find_dtype() is bool_dtype:
+                # TODO: indexing by bool array
+                raise NotImplementedError("sorry, not yet implemented")
+            else:
+                # Indexing by array
+
+                # FIXME: should raise exception if any index in
+                # array is out od bound, but this kills lazy execution
+                new_sig = signature.Signature.find_sig([
+                    IndexedByArray.signature, self.signature
+                ])                
+                res = IndexedByArray(new_sig, int_dtype, self, w_idx)
+                return space.wrap(res)
+
         start, stop, step, slice_length = space.decode_index4(w_idx, self.find_size())
         if step == 0:
             # Single index
@@ -430,6 +447,29 @@
         assert isinstance(call_sig, signature.Call2)
         return call_sig.func(self.res_dtype, lhs, rhs)
 
+class IndexedByArray(VirtualArray):
+    """
+    Intermediate class for performing indexing of array by another array
+    """
+    signature = signature.BaseSignature()
+    def __init__(self, signature, int_dtype, source, index):
+        VirtualArray.__init__(self, signature, source.find_dtype())
+        self.source = source
+        self.index = index
+        self.int_dtype = int_dtype
+
+    def _del_sources(self):
+        self.source = None
+        self.index = None
+
+    def _find_size(self):
+        return self.index.find_size()
+
+    def _eval(self, i):
+        idx = self.int_dtype.unbox(self.index.eval(i).convert_to(self.int_dtype))
+        val = self.source.eval(idx).convert_to(self.res_dtype)
+        return val
+
 class ViewArray(BaseArray):
     """
     Class for representing views of arrays, they will reflect changes of parent
diff --git a/pypy/module/micronumpy/interp_ufuncs.py b/pypy/module/micronumpy/interp_ufuncs.py
--- a/pypy/module/micronumpy/interp_ufuncs.py
+++ b/pypy/module/micronumpy/interp_ufuncs.py
@@ -200,7 +200,7 @@
     int64_dtype = space.fromcache(interp_dtype.W_Int64Dtype)
 
     if space.is_w(w_type, space.w_bool):
-        if current_guess is None:
+        if current_guess is None or current_guess is bool_dtype:
             return bool_dtype
     elif space.is_w(w_type, space.w_int):
         if (current_guess is None or current_guess is bool_dtype or
@@ -270,4 +270,4 @@
         setattr(self, ufunc_name, ufunc)
 
 def get(space):
-    return space.fromcache(UfuncState)
\ No newline at end of file
+    return space.fromcache(UfuncState)
diff --git a/pypy/module/micronumpy/test/test_numarray.py b/pypy/module/micronumpy/test/test_numarray.py
--- a/pypy/module/micronumpy/test/test_numarray.py
+++ b/pypy/module/micronumpy/test/test_numarray.py
@@ -119,6 +119,20 @@
         for i in xrange(5):
             assert a[i] == b[i]
 
+    def test_index_by_array(self):
+        from numpy import array
+        a = array(range(5))
+        idx_list = [3, 1, 3, 2, 0, 4]
+        idx_arr = array(idx_list)
+        a_by_arr = a[idx_arr]
+        assert len(a_by_arr) == 6
+        for i in xrange(6):
+            assert a_by_arr[i] == range(5)[idx_list[i]]
+        a_by_list = a[idx_list]
+        assert len(a_by_list) == 6
+        for i in xrange(6):
+            assert a_by_list[i] == range(5)[idx_list[i]]
+
     def test_setitem(self):
         from numpy import array
         a = array(range(5))

From noreply at buildbot.pypy.org  Tue Sep  6 19:14:18 2011
From: noreply at buildbot.pypy.org (fijal)
Date: Tue,  6 Sep 2011 19:14:18 +0200 (CEST)
Subject: [pypy-commit] pypy inline-dict-ops: merge default
Message-ID: <20110906171418.A48F38203C@wyvern.cs.uni-duesseldorf.de>

Author: Maciej Fijalkowski 
Branch: inline-dict-ops
Changeset: r47115:36e972ae4251
Date: 2011-09-05 09:00 +0200
http://bitbucket.org/pypy/pypy/changeset/36e972ae4251/

Log:	merge default

diff too long, truncating to 10000 out of 41373 lines

diff --git a/LICENSE b/LICENSE
--- a/LICENSE
+++ b/LICENSE
@@ -37,22 +37,22 @@
     Armin Rigo
     Maciej Fijalkowski
     Carl Friedrich Bolz
+    Antonio Cuni
     Amaury Forgeot d'Arc
-    Antonio Cuni
     Samuele Pedroni
     Michael Hudson
     Holger Krekel
+    Benjamin Peterson
     Christian Tismer
-    Benjamin Peterson
+    Hakan Ardo
+    Alex Gaynor
     Eric van Riet Paap
-    Anders Chrigström
-    Håkan Ardö
+    Anders Chrigstrom
+    David Schneider
     Richard Emslie
     Dan Villiom Podlaski Christiansen
     Alexander Schremmer
-    Alex Gaynor
-    David Schneider
-    Aurelién Campeas
+    Aurelien Campeas
     Anders Lehmann
     Camillo Bruni
     Niklaus Haldimann
@@ -63,16 +63,17 @@
     Bartosz Skowron
     Jakub Gustak
     Guido Wesdorp
+    Daniel Roberts
     Adrien Di Mascio
     Laura Creighton
     Ludovic Aubry
     Niko Matsakis
-    Daniel Roberts
     Jason Creighton
-    Jacob Hallén
+    Jacob Hallen
     Alex Martelli
     Anders Hammarquist
     Jan de Mooij
+    Wim Lavrijsen
     Stephan Diehl
     Michael Foord
     Stefan Schwarzer
@@ -83,9 +84,13 @@
     Alexandre Fayolle
     Marius Gedminas
     Simon Burton
+    Justin Peel
     Jean-Paul Calderone
     John Witulski
+    Lukas Diekmann
+    holger krekel
     Wim Lavrijsen
+    Dario Bertini
     Andreas Stührk
     Jean-Philippe St. Pierre
     Guido van Rossum
@@ -97,15 +102,16 @@
     Georg Brandl
     Gerald Klix
     Wanja Saatkamp
+    Ronny Pfannschmidt
     Boris Feigin
     Oscar Nierstrasz
-    Dario Bertini
     David Malcolm
     Eugene Oden
     Henry Mason
+    Sven Hager
     Lukas Renggli
+    Ilya Osadchiy
     Guenter Jantzen
-    Ronny Pfannschmidt
     Bert Freudenberg
     Amit Regmi
     Ben Young
@@ -122,8 +128,8 @@
     Jared Grubb
     Karl Bartel
     Gabriel Lavoie
+    Victor Stinner
     Brian Dorsey
-    Victor Stinner
     Stuart Williams
     Toby Watson
     Antoine Pitrou
@@ -134,19 +140,23 @@
     Jonathan David Riehl
     Elmo Mäntynen
     Anders Qvist
-    Beatrice Düring
+    Beatrice During
     Alexander Sedov
+    Timo Paulssen
+    Corbin Simpson
     Vincent Legoll
+    Romain Guillebert
     Alan McIntyre
-    Romain Guillebert
     Alex Perry
     Jens-Uwe Mager
+    Simon Cross
     Dan Stromberg
-    Lukas Diekmann
+    Guillebert Romain
     Carl Meyer
     Pieter Zieschang
     Alejandro J. Cura
     Sylvain Thenault
+    Christoph Gerum
     Travis Francis Athougies
     Henrik Vendelbo
     Lutz Paelike
@@ -157,6 +167,7 @@
     Miguel de Val Borro
     Ignas Mikalajunas
     Artur Lisiecki
+    Philip Jenvey
     Joshua Gilbert
     Godefroid Chappelle
     Yusei Tahara
@@ -165,27 +176,31 @@
     Gustavo Niemeyer
     William Leslie
     Akira Li
-    Kristján Valur Jónsson
+    Kristjan Valur Jonsson
     Bobby Impollonia
+    Michael Hudson-Doyle
     Andrew Thompson
     Anders Sigfridsson
+    Floris Bruynooghe
     Jacek Generowicz
     Dan Colish
-    Sven Hager
     Zooko Wilcox-O Hearn
+    Dan Villiom Podlaski Christiansen
     Anders Hammarquist
+    Chris Lambacher
     Dinu Gherman
     Dan Colish
+    Brett Cannon
     Daniel Neuhäuser
     Michael Chermside
     Konrad Delong
     Anna Ravencroft
     Greg Price
     Armin Ronacher
+    Christian Muirhead
     Jim Baker
-    Philip Jenvey
     Rodrigo Araújo
-    Brett Cannon
+    Romain Guillebert
 
     Heinrich-Heine University, Germany 
     Open End AB (formerly AB Strakt), Sweden
diff --git a/ctypes_configure/configure.py b/ctypes_configure/configure.py
--- a/ctypes_configure/configure.py
+++ b/ctypes_configure/configure.py
@@ -559,7 +559,9 @@
 C_HEADER = """
 #include 
 #include    /* for offsetof() */
-#include    /* FreeBSD: for uint64_t */
+#ifndef _WIN32
+#  include    /* FreeBSD: for uint64_t */
+#endif
 
 void dump(char* key, int value) {
     printf("%s: %d\\n", key, value);
diff --git a/ctypes_configure/stdoutcapture.py b/ctypes_configure/stdoutcapture.py
--- a/ctypes_configure/stdoutcapture.py
+++ b/ctypes_configure/stdoutcapture.py
@@ -15,6 +15,15 @@
             not hasattr(os, 'fdopen')):
             self.dummy = 1
         else:
+            try:
+                self.tmpout = os.tmpfile()
+                if mixed_out_err:
+                    self.tmperr = self.tmpout
+                else:
+                    self.tmperr = os.tmpfile()
+            except OSError:     # bah?  on at least one Windows box
+                self.dummy = 1
+                return
             self.dummy = 0
             # make new stdout/stderr files if needed
             self.localoutfd = os.dup(1)
@@ -29,11 +38,6 @@
                 sys.stderr = os.fdopen(self.localerrfd, 'w', 0)
             else:
                 self.saved_stderr = None
-            self.tmpout = os.tmpfile()
-            if mixed_out_err:
-                self.tmperr = self.tmpout
-            else:
-                self.tmperr = os.tmpfile()
             os.dup2(self.tmpout.fileno(), 1)
             os.dup2(self.tmperr.fileno(), 2)
 
diff --git a/lib-python/conftest.py b/lib-python/conftest.py
--- a/lib-python/conftest.py
+++ b/lib-python/conftest.py
@@ -154,18 +154,18 @@
     RegrTest('test_cmd.py'),
     RegrTest('test_cmd_line_script.py'),
     RegrTest('test_codeccallbacks.py', core=True),
-    RegrTest('test_codecencodings_cn.py'),
-    RegrTest('test_codecencodings_hk.py'),
-    RegrTest('test_codecencodings_jp.py'),
-    RegrTest('test_codecencodings_kr.py'),
-    RegrTest('test_codecencodings_tw.py'),
+    RegrTest('test_codecencodings_cn.py', usemodules='_multibytecodec'),
+    RegrTest('test_codecencodings_hk.py', usemodules='_multibytecodec'),
+    RegrTest('test_codecencodings_jp.py', usemodules='_multibytecodec'),
+    RegrTest('test_codecencodings_kr.py', usemodules='_multibytecodec'),
+    RegrTest('test_codecencodings_tw.py', usemodules='_multibytecodec'),
 
-    RegrTest('test_codecmaps_cn.py'),
-    RegrTest('test_codecmaps_hk.py'),
-    RegrTest('test_codecmaps_jp.py'),
-    RegrTest('test_codecmaps_kr.py'),
-    RegrTest('test_codecmaps_tw.py'),
-    RegrTest('test_codecs.py', core=True),
+    RegrTest('test_codecmaps_cn.py', usemodules='_multibytecodec'),
+    RegrTest('test_codecmaps_hk.py', usemodules='_multibytecodec'),
+    RegrTest('test_codecmaps_jp.py', usemodules='_multibytecodec'),
+    RegrTest('test_codecmaps_kr.py', usemodules='_multibytecodec'),
+    RegrTest('test_codecmaps_tw.py', usemodules='_multibytecodec'),
+    RegrTest('test_codecs.py', core=True, usemodules='_multibytecodec'),
     RegrTest('test_codeop.py', core=True),
     RegrTest('test_coercion.py', core=True),
     RegrTest('test_collections.py'),
@@ -314,7 +314,7 @@
     RegrTest('test_mmap.py'),
     RegrTest('test_module.py', core=True),
     RegrTest('test_modulefinder.py'),
-    RegrTest('test_multibytecodec.py'),
+    RegrTest('test_multibytecodec.py', usemodules='_multibytecodec'),
     RegrTest('test_multibytecodec_support.py', skip="not a test"),
     RegrTest('test_multifile.py'),
     RegrTest('test_multiprocessing.py', skip='FIXME leaves subprocesses'),
diff --git a/lib-python/modified-2.7/ctypes/util.py b/lib-python/modified-2.7/ctypes/util.py
--- a/lib-python/modified-2.7/ctypes/util.py
+++ b/lib-python/modified-2.7/ctypes/util.py
@@ -72,8 +72,8 @@
         return name
 
 if os.name == "posix" and sys.platform == "darwin":
-    from ctypes.macholib.dyld import dyld_find as _dyld_find
     def find_library(name):
+        from ctypes.macholib.dyld import dyld_find as _dyld_find
         possible = ['lib%s.dylib' % name,
                     '%s.dylib' % name,
                     '%s.framework/%s' % (name, name)]
diff --git a/lib-python/modified-2.7/distutils/unixccompiler.py b/lib-python/modified-2.7/distutils/unixccompiler.py
--- a/lib-python/modified-2.7/distutils/unixccompiler.py
+++ b/lib-python/modified-2.7/distutils/unixccompiler.py
@@ -324,7 +324,7 @@
             # On OSX users can specify an alternate SDK using
             # '-isysroot', calculate the SDK root if it is specified
             # (and use it further on)
-            cflags = sysconfig.get_config_var('CFLAGS')
+            cflags = sysconfig.get_config_var('CFLAGS') or ''
             m = re.search(r'-isysroot\s+(\S+)', cflags)
             if m is None:
                 sysroot = '/'
diff --git a/lib-python/modified-2.7/gzip.py b/lib-python/modified-2.7/gzip.py
new file mode 100644
--- /dev/null
+++ b/lib-python/modified-2.7/gzip.py
@@ -0,0 +1,514 @@
+"""Functions that read and write gzipped files.
+
+The user of the file doesn't have to worry about the compression,
+but random access is not allowed."""
+
+# based on Andrew Kuchling's minigzip.py distributed with the zlib module
+
+import struct, sys, time, os
+import zlib
+import io
+import __builtin__
+
+__all__ = ["GzipFile","open"]
+
+FTEXT, FHCRC, FEXTRA, FNAME, FCOMMENT = 1, 2, 4, 8, 16
+
+READ, WRITE = 1, 2
+
+def write32u(output, value):
+    # The L format writes the bit pattern correctly whether signed
+    # or unsigned.
+    output.write(struct.pack("'
+
+    def _check_closed(self):
+        """Raises a ValueError if the underlying file object has been closed.
+
+        """
+        if self.closed:
+            raise ValueError('I/O operation on closed file.')
+
+    def _init_write(self, filename):
+        self.name = filename
+        self.crc = zlib.crc32("") & 0xffffffffL
+        self.size = 0
+        self.writebuf = []
+        self.bufsize = 0
+
+    def _write_gzip_header(self):
+        self.fileobj.write('\037\213')             # magic header
+        self.fileobj.write('\010')                 # compression method
+        fname = os.path.basename(self.name)
+        if fname.endswith(".gz"):
+            fname = fname[:-3]
+        flags = 0
+        if fname:
+            flags = FNAME
+        self.fileobj.write(chr(flags))
+        mtime = self.mtime
+        if mtime is None:
+            mtime = time.time()
+        write32u(self.fileobj, long(mtime))
+        self.fileobj.write('\002')
+        self.fileobj.write('\377')
+        if fname:
+            self.fileobj.write(fname + '\000')
+
+    def _init_read(self):
+        self.crc = zlib.crc32("") & 0xffffffffL
+        self.size = 0
+
+    def _read_gzip_header(self):
+        magic = self.fileobj.read(2)
+        if magic != '\037\213':
+            raise IOError, 'Not a gzipped file'
+        method = ord( self.fileobj.read(1) )
+        if method != 8:
+            raise IOError, 'Unknown compression method'
+        flag = ord( self.fileobj.read(1) )
+        self.mtime = read32(self.fileobj)
+        # extraflag = self.fileobj.read(1)
+        # os = self.fileobj.read(1)
+        self.fileobj.read(2)
+
+        if flag & FEXTRA:
+            # Read & discard the extra field, if present
+            xlen = ord(self.fileobj.read(1))
+            xlen = xlen + 256*ord(self.fileobj.read(1))
+            self.fileobj.read(xlen)
+        if flag & FNAME:
+            # Read and discard a null-terminated string containing the filename
+            while True:
+                s = self.fileobj.read(1)
+                if not s or s=='\000':
+                    break
+        if flag & FCOMMENT:
+            # Read and discard a null-terminated string containing a comment
+            while True:
+                s = self.fileobj.read(1)
+                if not s or s=='\000':
+                    break
+        if flag & FHCRC:
+            self.fileobj.read(2)     # Read & discard the 16-bit header CRC
+
+    def write(self,data):
+        self._check_closed()
+        if self.mode != WRITE:
+            import errno
+            raise IOError(errno.EBADF, "write() on read-only GzipFile object")
+
+        if self.fileobj is None:
+            raise ValueError, "write() on closed GzipFile object"
+
+        # Convert data type if called by io.BufferedWriter.
+        if isinstance(data, memoryview):
+            data = data.tobytes()
+
+        if len(data) > 0:
+            self.size = self.size + len(data)
+            self.crc = zlib.crc32(data, self.crc) & 0xffffffffL
+            self.fileobj.write( self.compress.compress(data) )
+            self.offset += len(data)
+
+        return len(data)
+
+    def read(self, size=-1):
+        self._check_closed()
+        if self.mode != READ:
+            import errno
+            raise IOError(errno.EBADF, "read() on write-only GzipFile object")
+
+        if self.extrasize <= 0 and self.fileobj is None:
+            return ''
+
+        readsize = 1024
+        if size < 0:        # get the whole thing
+            try:
+                while True:
+                    self._read(readsize)
+                    readsize = min(self.max_read_chunk, readsize * 2)
+            except EOFError:
+                size = self.extrasize
+        elif size == 0:
+            return ""
+        else:               # just get some more of it
+            try:
+                while size > self.extrasize:
+                    self._read(readsize)
+                    readsize = min(self.max_read_chunk, readsize * 2)
+            except EOFError:
+                if size > self.extrasize:
+                    size = self.extrasize
+
+        offset = self.offset - self.extrastart
+        chunk = self.extrabuf[offset: offset + size]
+        self.extrasize = self.extrasize - size
+
+        self.offset += size
+        return chunk
+
+    def _unread(self, buf):
+        self.extrasize = len(buf) + self.extrasize
+        self.offset -= len(buf)
+
+    def _read(self, size=1024):
+        if self.fileobj is None:
+            raise EOFError, "Reached EOF"
+
+        if self._new_member:
+            # If the _new_member flag is set, we have to
+            # jump to the next member, if there is one.
+            #
+            # First, check if we're at the end of the file;
+            # if so, it's time to stop; no more members to read.
+            pos = self.fileobj.tell()   # Save current position
+            self.fileobj.seek(0, 2)     # Seek to end of file
+            if pos == self.fileobj.tell():
+                raise EOFError, "Reached EOF"
+            else:
+                self.fileobj.seek( pos ) # Return to original position
+
+            self._init_read()
+            self._read_gzip_header()
+            self.decompress = zlib.decompressobj(-zlib.MAX_WBITS)
+            self._new_member = False
+
+        # Read a chunk of data from the file
+        buf = self.fileobj.read(size)
+
+        # If the EOF has been reached, flush the decompression object
+        # and mark this object as finished.
+
+        if buf == "":
+            uncompress = self.decompress.flush()
+            self._read_eof()
+            self._add_read_data( uncompress )
+            raise EOFError, 'Reached EOF'
+
+        uncompress = self.decompress.decompress(buf)
+        self._add_read_data( uncompress )
+
+        if self.decompress.unused_data != "":
+            # Ending case: we've come to the end of a member in the file,
+            # so seek back to the start of the unused data, finish up
+            # this member, and read a new gzip header.
+            # (The number of bytes to seek back is the length of the unused
+            # data, minus 8 because _read_eof() will rewind a further 8 bytes)
+            self.fileobj.seek( -len(self.decompress.unused_data)+8, 1)
+
+            # Check the CRC and file size, and set the flag so we read
+            # a new member on the next call
+            self._read_eof()
+            self._new_member = True
+
+    def _add_read_data(self, data):
+        self.crc = zlib.crc32(data, self.crc) & 0xffffffffL
+        offset = self.offset - self.extrastart
+        self.extrabuf = self.extrabuf[offset:] + data
+        self.extrasize = self.extrasize + len(data)
+        self.extrastart = self.offset
+        self.size = self.size + len(data)
+
+    def _read_eof(self):
+        # We've read to the end of the file, so we have to rewind in order
+        # to reread the 8 bytes containing the CRC and the file size.
+        # We check the that the computed CRC and size of the
+        # uncompressed data matches the stored values.  Note that the size
+        # stored is the true file size mod 2**32.
+        self.fileobj.seek(-8, 1)
+        crc32 = read32(self.fileobj)
+        isize = read32(self.fileobj)  # may exceed 2GB
+        if crc32 != self.crc:
+            raise IOError("CRC check failed %s != %s" % (hex(crc32),
+                                                         hex(self.crc)))
+        elif isize != (self.size & 0xffffffffL):
+            raise IOError, "Incorrect length of data produced"
+
+        # Gzip files can be padded with zeroes and still have archives.
+        # Consume all zero bytes and set the file position to the first
+        # non-zero byte. See http://www.gzip.org/#faq8
+        c = "\x00"
+        while c == "\x00":
+            c = self.fileobj.read(1)
+        if c:
+            self.fileobj.seek(-1, 1)
+
+    @property
+    def closed(self):
+        return self.fileobj is None
+
+    def close(self):
+        if self.fileobj is None:
+            return
+        if self.mode == WRITE:
+            self.fileobj.write(self.compress.flush())
+            write32u(self.fileobj, self.crc)
+            # self.size may exceed 2GB, or even 4GB
+            write32u(self.fileobj, self.size & 0xffffffffL)
+            self.fileobj = None
+        elif self.mode == READ:
+            self.fileobj = None
+        if self.myfileobj:
+            self.myfileobj.close()
+            self.myfileobj = None
+
+    def flush(self,zlib_mode=zlib.Z_SYNC_FLUSH):
+        self._check_closed()
+        if self.mode == WRITE:
+            # Ensure the compressor's buffer is flushed
+            self.fileobj.write(self.compress.flush(zlib_mode))
+            self.fileobj.flush()
+
+    def fileno(self):
+        """Invoke the underlying file object's fileno() method.
+
+        This will raise AttributeError if the underlying file object
+        doesn't support fileno().
+        """
+        return self.fileobj.fileno()
+
+    def rewind(self):
+        '''Return the uncompressed stream file position indicator to the
+        beginning of the file'''
+        if self.mode != READ:
+            raise IOError("Can't rewind in write mode")
+        self.fileobj.seek(0)
+        self._new_member = True
+        self.extrabuf = ""
+        self.extrasize = 0
+        self.extrastart = 0
+        self.offset = 0
+
+    def readable(self):
+        return self.mode == READ
+
+    def writable(self):
+        return self.mode == WRITE
+
+    def seekable(self):
+        return True
+
+    def seek(self, offset, whence=0):
+        if whence:
+            if whence == 1:
+                offset = self.offset + offset
+            else:
+                raise ValueError('Seek from end not supported')
+        if self.mode == WRITE:
+            if offset < self.offset:
+                raise IOError('Negative seek in write mode')
+            count = offset - self.offset
+            for i in range(count // 1024):
+                self.write(1024 * '\0')
+            self.write((count % 1024) * '\0')
+        elif self.mode == READ:
+            if offset == self.offset:
+                self.read(0) # to make sure that this file is open
+                return self.offset
+            if offset < self.offset:
+                # for negative seek, rewind and do positive seek
+                self.rewind()
+            count = offset - self.offset
+            for i in range(count // 1024):
+                self.read(1024)
+            self.read(count % 1024)
+
+        return self.offset
+
+    def readline(self, size=-1):
+        if size < 0:
+            # Shortcut common case - newline found in buffer.
+            offset = self.offset - self.extrastart
+            i = self.extrabuf.find('\n', offset) + 1
+            if i > 0:
+                self.extrasize -= i - offset
+                self.offset += i - offset
+                return self.extrabuf[offset: i]
+
+            size = sys.maxint
+            readsize = self.min_readsize
+        else:
+            readsize = size
+        bufs = []
+        while size != 0:
+            c = self.read(readsize)
+            i = c.find('\n')
+
+            # We set i=size to break out of the loop under two
+            # conditions: 1) there's no newline, and the chunk is
+            # larger than size, or 2) there is a newline, but the
+            # resulting line would be longer than 'size'.
+            if (size <= i) or (i == -1 and len(c) > size):
+                i = size - 1
+
+            if i >= 0 or c == '':
+                bufs.append(c[:i + 1])    # Add portion of last chunk
+                self._unread(c[i + 1:])   # Push back rest of chunk
+                break
+
+            # Append chunk to list, decrease 'size',
+            bufs.append(c)
+            size = size - len(c)
+            readsize = min(size, readsize * 2)
+        if readsize > self.min_readsize:
+            self.min_readsize = min(readsize, self.min_readsize * 2, 512)
+        return ''.join(bufs) # Return resulting line
+
+
+def _test():
+    # Act like gzip; with -d, act like gunzip.
+    # The input file is not deleted, however, nor are any other gzip
+    # options or features supported.
+    args = sys.argv[1:]
+    decompress = args and args[0] == "-d"
+    if decompress:
+        args = args[1:]
+    if not args:
+        args = ["-"]
+    for arg in args:
+        if decompress:
+            if arg == "-":
+                f = GzipFile(filename="", mode="rb", fileobj=sys.stdin)
+                g = sys.stdout
+            else:
+                if arg[-3:] != ".gz":
+                    print "filename doesn't end in .gz:", repr(arg)
+                    continue
+                f = open(arg, "rb")
+                g = __builtin__.open(arg[:-3], "wb")
+        else:
+            if arg == "-":
+                f = sys.stdin
+                g = GzipFile(filename="", mode="wb", fileobj=sys.stdout)
+            else:
+                f = __builtin__.open(arg, "rb")
+                g = open(arg + ".gz", "wb")
+        while True:
+            chunk = f.read(1024)
+            if not chunk:
+                break
+            g.write(chunk)
+        if g is not sys.stdout:
+            g.close()
+        if f is not sys.stdin:
+            f.close()
+
+if __name__ == '__main__':
+    _test()
diff --git a/lib-python/modified-2.7/sqlite3/test/regression.py b/lib-python/modified-2.7/sqlite3/test/regression.py
--- a/lib-python/modified-2.7/sqlite3/test/regression.py
+++ b/lib-python/modified-2.7/sqlite3/test/regression.py
@@ -274,6 +274,18 @@
         cur.execute("UPDATE foo SET id = 3 WHERE id = 1")
         self.assertEqual(cur.description, None)
 
+    def CheckStatementCache(self):
+        cur = self.con.cursor()
+        cur.execute("CREATE TABLE foo (id INTEGER)")
+        values = [(i,) for i in xrange(5)]
+        cur.executemany("INSERT INTO foo (id) VALUES (?)", values)
+
+        cur.execute("SELECT id FROM foo")
+        self.assertEqual(list(cur), values)
+        self.con.commit()
+        cur.execute("SELECT id FROM foo")
+        self.assertEqual(list(cur), values)
+
 def suite():
     regression_suite = unittest.makeSuite(RegressionTests, "Check")
     return unittest.TestSuite((regression_suite,))
diff --git a/lib-python/modified-2.7/tarfile.py b/lib-python/modified-2.7/tarfile.py
--- a/lib-python/modified-2.7/tarfile.py
+++ b/lib-python/modified-2.7/tarfile.py
@@ -252,8 +252,8 @@
        the high bit set. So we calculate two checksums, unsigned and
        signed.
     """
-    unsigned_chksum = 256 + sum(struct.unpack("148B", buf[:148]) + struct.unpack("356B", buf[156:512]))
-    signed_chksum = 256 + sum(struct.unpack("148b", buf[:148]) + struct.unpack("356b", buf[156:512]))
+    unsigned_chksum = 256 + sum(struct.unpack("148B8x356B", buf[:512]))
+    signed_chksum = 256 + sum(struct.unpack("148b8x356b", buf[:512]))
     return unsigned_chksum, signed_chksum
 
 def copyfileobj(src, dst, length=None):
@@ -265,7 +265,6 @@
     if length is None:
         shutil.copyfileobj(src, dst)
         return
-
     BUFSIZE = 16 * 1024
     blocks, remainder = divmod(length, BUFSIZE)
     for b in xrange(blocks):
@@ -802,19 +801,19 @@
         if self.closed:
             raise ValueError("I/O operation on closed file")
 
-        buf = ""
         if self.buffer:
             if size is None:
-                buf = self.buffer
+                buf = self.buffer + self.fileobj.read()
                 self.buffer = ""
             else:
                 buf = self.buffer[:size]
                 self.buffer = self.buffer[size:]
-
-        if size is None:
-            buf += self.fileobj.read()
+                buf += self.fileobj.read(size - len(buf))
         else:
-            buf += self.fileobj.read(size - len(buf))
+            if size is None:
+                buf = self.fileobj.read()
+            else:
+                buf = self.fileobj.read(size)
 
         self.position += len(buf)
         return buf
diff --git a/lib-python/modified-2.7/test/regrtest.py b/lib-python/modified-2.7/test/regrtest.py
--- a/lib-python/modified-2.7/test/regrtest.py
+++ b/lib-python/modified-2.7/test/regrtest.py
@@ -1403,7 +1403,26 @@
         test_zipimport
         test_zlib
         """,
-    'openbsd3':
+    'openbsd4':
+        """
+        test_ascii_formatd
+        test_bsddb
+        test_bsddb3
+        test_ctypes
+        test_dl
+        test_epoll
+        test_gdbm
+        test_locale
+        test_normalization
+        test_ossaudiodev
+        test_pep277
+        test_tcl
+        test_tk
+        test_ttk_guionly
+        test_ttk_textonly
+        test_multiprocessing
+        """,
+    'openbsd5':
         """
         test_ascii_formatd
         test_bsddb
diff --git a/lib-python/modified-2.7/test/test_bz2.py b/lib-python/modified-2.7/test/test_bz2.py
--- a/lib-python/modified-2.7/test/test_bz2.py
+++ b/lib-python/modified-2.7/test/test_bz2.py
@@ -50,6 +50,7 @@
         self.filename = TESTFN
 
     def tearDown(self):
+        test_support.gc_collect()
         if os.path.isfile(self.filename):
             os.unlink(self.filename)
 
diff --git a/lib-python/modified-2.7/test/test_fcntl.py b/lib-python/modified-2.7/test/test_fcntl.py
new file mode 100644
--- /dev/null
+++ b/lib-python/modified-2.7/test/test_fcntl.py
@@ -0,0 +1,108 @@
+"""Test program for the fcntl C module.
+
+OS/2+EMX doesn't support the file locking operations.
+
+"""
+import os
+import struct
+import sys
+import unittest
+from test.test_support import (verbose, TESTFN, unlink, run_unittest,
+    import_module)
+
+# Skip test if no fnctl module.
+fcntl = import_module('fcntl')
+
+
+# TODO - Write tests for flock() and lockf().
+
+def get_lockdata():
+    if sys.platform.startswith('atheos'):
+        start_len = "qq"
+    else:
+        try:
+            os.O_LARGEFILE
+        except AttributeError:
+            start_len = "ll"
+        else:
+            start_len = "qq"
+
+    if sys.platform in ('netbsd1', 'netbsd2', 'netbsd3',
+                        'Darwin1.2', 'darwin',
+                        'freebsd2', 'freebsd3', 'freebsd4', 'freebsd5',
+                        'freebsd6', 'freebsd7', 'freebsd8',
+                        'bsdos2', 'bsdos3', 'bsdos4',
+                        'openbsd', 'openbsd2', 'openbsd3', 'openbsd4', 'openbsd5'):
+        if struct.calcsize('l') == 8:
+            off_t = 'l'
+            pid_t = 'i'
+        else:
+            off_t = 'lxxxx'
+            pid_t = 'l'
+        lockdata = struct.pack(off_t + off_t + pid_t + 'hh', 0, 0, 0,
+                               fcntl.F_WRLCK, 0)
+    elif sys.platform in ['aix3', 'aix4', 'hp-uxB', 'unixware7']:
+        lockdata = struct.pack('hhlllii', fcntl.F_WRLCK, 0, 0, 0, 0, 0, 0)
+    elif sys.platform in ['os2emx']:
+        lockdata = None
+    else:
+        lockdata = struct.pack('hh'+start_len+'hh', fcntl.F_WRLCK, 0, 0, 0, 0, 0)
+    if lockdata:
+        if verbose:
+            print 'struct.pack: ', repr(lockdata)
+    return lockdata
+
+lockdata = get_lockdata()
+
+
+class TestFcntl(unittest.TestCase):
+
+    def setUp(self):
+        self.f = None
+
+    def tearDown(self):
+        if self.f and not self.f.closed:
+            self.f.close()
+        unlink(TESTFN)
+
+    def test_fcntl_fileno(self):
+        # the example from the library docs
+        self.f = open(TESTFN, 'w')
+        rv = fcntl.fcntl(self.f.fileno(), fcntl.F_SETFL, os.O_NONBLOCK)
+        if verbose:
+            print 'Status from fcntl with O_NONBLOCK: ', rv
+        if sys.platform not in ['os2emx']:
+            rv = fcntl.fcntl(self.f.fileno(), fcntl.F_SETLKW, lockdata)
+            if verbose:
+                print 'String from fcntl with F_SETLKW: ', repr(rv)
+        self.f.close()
+
+    def test_fcntl_file_descriptor(self):
+        # again, but pass the file rather than numeric descriptor
+        self.f = open(TESTFN, 'w')
+        rv = fcntl.fcntl(self.f, fcntl.F_SETFL, os.O_NONBLOCK)
+        if sys.platform not in ['os2emx']:
+            rv = fcntl.fcntl(self.f, fcntl.F_SETLKW, lockdata)
+        self.f.close()
+
+    def test_fcntl_64_bit(self):
+        # Issue #1309352: fcntl shouldn't fail when the third arg fits in a
+        # C 'long' but not in a C 'int'.
+        try:
+            cmd = fcntl.F_NOTIFY
+            # This flag is larger than 2**31 in 64-bit builds
+            flags = fcntl.DN_MULTISHOT
+        except AttributeError:
+            self.skipTest("F_NOTIFY or DN_MULTISHOT unavailable")
+        fd = os.open(os.path.dirname(os.path.abspath(TESTFN)), os.O_RDONLY)
+        try:
+            fcntl.fcntl(fd, cmd, flags)
+        finally:
+            os.close(fd)
+
+
+def test_main():
+    run_unittest(TestFcntl)
+
+if __name__ == '__main__':
+    test_main()
diff --git a/lib-python/modified-2.7/test/test_multibytecodec.py b/lib-python/modified-2.7/test/test_multibytecodec.py
--- a/lib-python/modified-2.7/test/test_multibytecodec.py
+++ b/lib-python/modified-2.7/test/test_multibytecodec.py
@@ -148,7 +148,8 @@
 class Test_StreamReader(unittest.TestCase):
     def test_bug1728403(self):
         try:
-            open(TESTFN, 'w').write('\xa1')
+            with open(TESTFN, 'w') as f:
+                f.write('\xa1')
             f = codecs.open(TESTFN, encoding='cp949')
             self.assertRaises(UnicodeDecodeError, f.read, 2)
         finally:
diff --git a/lib-python/modified-2.7/test/test_tempfile.py b/lib-python/modified-2.7/test/test_tempfile.py
--- a/lib-python/modified-2.7/test/test_tempfile.py
+++ b/lib-python/modified-2.7/test/test_tempfile.py
@@ -23,8 +23,8 @@
 
 # TEST_FILES may need to be tweaked for systems depending on the maximum
 # number of files that can be opened at one time (see ulimit -n)
-if sys.platform in ('openbsd3', 'openbsd4'):
-    TEST_FILES = 48
+if sys.platform.startswith("openbsd"):
+    TEST_FILES = 64 # ulimit -n defaults to 128 for normal users
 else:
     TEST_FILES = 100
 
diff --git a/lib_pypy/_ctypes/basics.py b/lib_pypy/_ctypes/basics.py
--- a/lib_pypy/_ctypes/basics.py
+++ b/lib_pypy/_ctypes/basics.py
@@ -54,7 +54,8 @@
     def get_ffi_argtype(self):
         if self._ffiargtype:
             return self._ffiargtype
-        return _shape_to_ffi_type(self._ffiargshape)
+        self._ffiargtype = _shape_to_ffi_type(self._ffiargshape)
+        return self._ffiargtype
 
     def _CData_output(self, resbuffer, base=None, index=-1):
         #assert isinstance(resbuffer, _rawffi.ArrayInstance)
@@ -166,7 +167,8 @@
     return tp._alignmentofinstances()
 
 def byref(cdata):
-    from ctypes import pointer
+    # "pointer" is imported at the end of this module to avoid circular
+    # imports
     return pointer(cdata)
 
 def cdata_from_address(self, address):
@@ -224,5 +226,9 @@
     'Z' : _ffi.types.void_p,
     'X' : _ffi.types.void_p,
     'v' : _ffi.types.sshort,
+    '?' : _ffi.types.ubyte,
     }
 
+
+# used by "byref"
+from _ctypes.pointer import pointer
diff --git a/lib_pypy/_ctypes/function.py b/lib_pypy/_ctypes/function.py
--- a/lib_pypy/_ctypes/function.py
+++ b/lib_pypy/_ctypes/function.py
@@ -91,13 +91,15 @@
                     raise TypeError(
                         "item %d in _argtypes_ has no from_param method" % (
                             i + 1,))
-            #
-            if all([hasattr(argtype, '_ffiargshape') for argtype in argtypes]):
-                fastpath_cls = make_fastpath_subclass(self.__class__)
-                fastpath_cls.enable_fastpath_maybe(self)
             self._argtypes_ = list(argtypes)
+            self._check_argtypes_for_fastpath()
     argtypes = property(_getargtypes, _setargtypes)
 
+    def _check_argtypes_for_fastpath(self):
+        if all([hasattr(argtype, '_ffiargshape') for argtype in self._argtypes_]):
+            fastpath_cls = make_fastpath_subclass(self.__class__)
+            fastpath_cls.enable_fastpath_maybe(self)
+
     def _getparamflags(self):
         return self._paramflags
 
@@ -216,6 +218,7 @@
                 import ctypes
                 restype = ctypes.c_int
             self._ptr = self._getfuncptr_fromaddress(self._argtypes_, restype)
+            self._check_argtypes_for_fastpath()
             return
 
         
@@ -466,7 +469,8 @@
         newargs = []
         for argtype, arg in zip(argtypes, args):
             param = argtype.from_param(arg)
-            if argtype._type_ == 'P': # special-case for c_void_p
+            _type_ = getattr(argtype, '_type_', None)
+            if _type_ == 'P': # special-case for c_void_p
                 param = param._get_buffer_value()
             elif self._is_primitive(argtype):
                 param = param.value
diff --git a/lib_pypy/_ctypes/structure.py b/lib_pypy/_ctypes/structure.py
--- a/lib_pypy/_ctypes/structure.py
+++ b/lib_pypy/_ctypes/structure.py
@@ -14,6 +14,15 @@
             raise TypeError("Expected CData subclass, got %s" % (tp,))
         if isinstance(tp, StructOrUnionMeta):
             tp._make_final()
+        if len(f) == 3:
+            if (not hasattr(tp, '_type_')
+                or not isinstance(tp._type_, str)
+                or tp._type_ not in "iIhHbBlL"):
+                #XXX: are those all types?
+                #     we just dont get the type name
+                #     in the interp levle thrown TypeError
+                #     from rawffi if there are more
+                raise TypeError('bit fields not allowed for type ' + tp.__name__)
 
     all_fields = []
     for cls in reversed(inspect.getmro(superclass)):
@@ -34,34 +43,37 @@
     for i, field in enumerate(all_fields):
         name = field[0]
         value = field[1]
+        is_bitfield = (len(field) == 3)
         fields[name] = Field(name,
                              self._ffistruct.fieldoffset(name),
                              self._ffistruct.fieldsize(name),
-                             value, i)
+                             value, i, is_bitfield)
 
     if anonymous_fields:
         resnames = []
         for i, field in enumerate(all_fields):
             name = field[0]
             value = field[1]
+            is_bitfield = (len(field) == 3)
             startpos = self._ffistruct.fieldoffset(name)
             if name in anonymous_fields:
                 for subname in value._names:
                     resnames.append(subname)
-                    relpos = startpos + value._fieldtypes[subname].offset
-                    subvalue = value._fieldtypes[subname].ctype
+                    subfield = getattr(value, subname)
+                    relpos = startpos + subfield.offset
+                    subvalue = subfield.ctype
                     fields[subname] = Field(subname,
                                             relpos, subvalue._sizeofinstances(),
-                                            subvalue, i)
+                                            subvalue, i, is_bitfield)
             else:
                 resnames.append(name)
         names = resnames
     self._names = names
-    self._fieldtypes = fields
+    self.__dict__.update(fields)
 
 class Field(object):
-    def __init__(self, name, offset, size, ctype, num):
-        for k in ('name', 'offset', 'size', 'ctype', 'num'):
+    def __init__(self, name, offset, size, ctype, num, is_bitfield):
+        for k in ('name', 'offset', 'size', 'ctype', 'num', 'is_bitfield'):
             self.__dict__[k] = locals()[k]
 
     def __setattr__(self, name, value):
@@ -71,6 +83,35 @@
         return "" % (self.name, self.offset,
                                                    self.size)
 
+    def __get__(self, obj, cls=None):
+        if obj is None:
+            return self
+        if self.is_bitfield:
+            # bitfield member, use direct access
+            return obj._buffer.__getattr__(self.name)
+        else:
+            fieldtype = self.ctype
+            offset = self.num
+            suba = obj._subarray(fieldtype, self.name)
+            return fieldtype._CData_output(suba, obj, offset)
+
+
+    def __set__(self, obj, value):
+        fieldtype = self.ctype
+        cobj = fieldtype.from_param(value)
+        if ensure_objects(cobj) is not None:
+            key = keepalive_key(self.num)
+            store_reference(obj, key, cobj._objects)
+        arg = cobj._get_buffer_value()
+        if fieldtype._fficompositesize is not None:
+            from ctypes import memmove
+            dest = obj._buffer.fieldaddress(self.name)
+            memmove(dest, arg, fieldtype._fficompositesize)
+        else:
+            obj._buffer.__setattr__(self.name, arg)
+
+
+
 # ________________________________________________________________
 
 def _set_shape(tp, rawfields, is_union=False):
@@ -79,17 +120,12 @@
     tp._ffiargshape = tp._ffishape = (tp._ffistruct, 1)
     tp._fficompositesize = tp._ffistruct.size
 
-def struct_getattr(self, name):
-    if name not in ('_fields_', '_fieldtypes'):
-        if hasattr(self, '_fieldtypes') and name in self._fieldtypes:
-            return self._fieldtypes[name]
-    return _CDataMeta.__getattribute__(self, name)
 
 def struct_setattr(self, name, value):
     if name == '_fields_':
         if self.__dict__.get('_fields_', None) is not None:
             raise AttributeError("_fields_ is final")
-        if self in [v for k, v in value]:
+        if self in [f[1] for f in value]:
             raise AttributeError("Structure or union cannot contain itself")
         names_and_fields(
             self,
@@ -127,14 +163,14 @@
         if '_fields_' not in self.__dict__:
             self._fields_ = []
             self._names = []
-            self._fieldtypes = {}
             _set_shape(self, [], self._is_union)
 
-    __getattr__ = struct_getattr
     __setattr__ = struct_setattr
 
     def from_address(self, address):
         instance = StructOrUnion.__new__(self)
+        if isinstance(address, _rawffi.StructureInstance):
+            address = address.buffer
         instance.__dict__['_buffer'] = self._ffistruct.fromaddress(address)
         return instance
 
@@ -200,40 +236,6 @@
         A = _rawffi.Array(fieldtype._ffishape)
         return A.fromaddress(address, 1)
 
-    def __setattr__(self, name, value):
-        try:
-            field = self._fieldtypes[name]
-        except KeyError:
-            return _CData.__setattr__(self, name, value)
-        fieldtype = field.ctype
-        cobj = fieldtype.from_param(value)
-        if ensure_objects(cobj) is not None:
-            key = keepalive_key(field.num)
-            store_reference(self, key, cobj._objects)
-        arg = cobj._get_buffer_value()
-        if fieldtype._fficompositesize is not None:
-            from ctypes import memmove
-            dest = self._buffer.fieldaddress(name)
-            memmove(dest, arg, fieldtype._fficompositesize)
-        else:
-            self._buffer.__setattr__(name, arg)
-
-    def __getattribute__(self, name):
-        if name == '_fieldtypes':
-            return _CData.__getattribute__(self, '_fieldtypes')
-        try:
-            field = self._fieldtypes[name]
-        except KeyError:
-            return _CData.__getattribute__(self, name)
-        if field.size >> 16:
-            # bitfield member, use direct access
-            return self._buffer.__getattr__(name)
-        else:
-            fieldtype = field.ctype
-            offset = field.num
-            suba = self._subarray(fieldtype, name)
-            return fieldtype._CData_output(suba, self, offset)
-
     def _get_buffer_for_param(self):
         return self
 
diff --git a/lib_pypy/_sqlite3.py b/lib_pypy/_sqlite3.py
--- a/lib_pypy/_sqlite3.py
+++ b/lib_pypy/_sqlite3.py
@@ -24,6 +24,7 @@
 from ctypes import c_void_p, c_int, c_double, c_int64, c_char_p, cdll
 from ctypes import POINTER, byref, string_at, CFUNCTYPE, cast
 from ctypes import sizeof, c_ssize_t
+from collections import OrderedDict
 import datetime
 import sys
 import time
@@ -274,6 +275,28 @@
 def unicode_text_factory(x):
     return unicode(x, 'utf-8')
 
+
+class StatementCache(object):
+    def __init__(self, connection, maxcount):
+        self.connection = connection
+        self.maxcount = maxcount
+        self.cache = OrderedDict()
+
+    def get(self, sql, cursor, row_factory):
+        try:
+            stat = self.cache[sql]
+        except KeyError:
+            stat = Statement(self.connection, sql)
+            self.cache[sql] = stat
+            if len(self.cache) > self.maxcount:
+                self.cache.popitem(0)
+        #
+        if stat.in_use:
+            stat = Statement(self.connection, sql)
+        stat.set_row_factory(row_factory)
+        return stat
+
+
 class Connection(object):
     def __init__(self, database, timeout=5.0, detect_types=0, isolation_level="",
                  check_same_thread=True, factory=None, cached_statements=100):
@@ -291,6 +314,7 @@
         self.row_factory = None
         self._isolation_level = isolation_level
         self.detect_types = detect_types
+        self.statement_cache = StatementCache(self, cached_statements)
 
         self.cursors = []
 
@@ -399,7 +423,7 @@
         cur = Cursor(self)
         if not isinstance(sql, (str, unicode)):
             raise Warning("SQL is of wrong type. Must be string or unicode.")
-        statement = Statement(cur, sql, self.row_factory)
+        statement = self.statement_cache.get(sql, cur, self.row_factory)
         return statement
 
     def _get_isolation_level(self):
@@ -681,6 +705,8 @@
         from sqlite3.dump import _iterdump
         return _iterdump(self)
 
+DML, DQL, DDL = range(3)
+
 class Cursor(object):
     def __init__(self, con):
         if not isinstance(con, Connection):
@@ -708,12 +734,12 @@
         if type(sql) is unicode:
             sql = sql.encode("utf-8")
         self._check_closed()
-        self.statement = Statement(self, sql, self.row_factory)
+        self.statement = self.connection.statement_cache.get(sql, self, self.row_factory)
 
         if self.connection._isolation_level is not None:
-            if self.statement.kind == "DDL":
+            if self.statement.kind == DDL:
                 self.connection.commit()
-            elif self.statement.kind == "DML":
+            elif self.statement.kind == DML:
                 self.connection._begin()
 
         self.statement.set_params(params)
@@ -724,19 +750,18 @@
             self.statement.reset()
             raise self.connection._get_exception(ret)
 
-        if self.statement.kind == "DQL":
-            if ret == SQLITE_ROW:
-                self.statement._build_row_cast_map()
-                self.statement._readahead()
-            else:
-                self.statement.item = None
-                self.statement.exhausted = True
+        if self.statement.kind == DQL and ret == SQLITE_ROW:
+            self.statement._build_row_cast_map()
+            self.statement._readahead(self)
+        else:
+            self.statement.item = None
+            self.statement.exhausted = True
 
-        if self.statement.kind in ("DML", "DDL"):
+        if self.statement.kind == DML or self.statement.kind == DDL:
             self.statement.reset()
 
         self.rowcount = -1
-        if self.statement.kind == "DML":
+        if self.statement.kind == DML:
             self.rowcount = sqlite.sqlite3_changes(self.connection.db)
 
         return self
@@ -747,8 +772,9 @@
         if type(sql) is unicode:
             sql = sql.encode("utf-8")
         self._check_closed()
-        self.statement = Statement(self, sql, self.row_factory)
-        if self.statement.kind == "DML":
+        self.statement = self.connection.statement_cache.get(sql, self, self.row_factory)
+
+        if self.statement.kind == DML:
             self.connection._begin()
         else:
             raise ProgrammingError, "executemany is only for DML statements"
@@ -800,7 +826,7 @@
         return self
 
     def __iter__(self):
-        return self.statement
+        return iter(self.fetchone, None)
 
     def _check_reset(self):
         if self.reset:
@@ -817,7 +843,7 @@
             return None
 
         try:
-            return self.statement.next()
+            return self.statement.next(self)
         except StopIteration:
             return None
 
@@ -831,7 +857,7 @@
         if size is None:
             size = self.arraysize
         lst = []
-        for row in self.statement:
+        for row in self:
             lst.append(row)
             if len(lst) == size:
                 break
@@ -842,7 +868,7 @@
         self._check_reset()
         if self.statement is None:
             return []
-        return list(self.statement)
+        return list(self)
 
     def _getdescription(self):
         if self._description is None:
@@ -872,39 +898,47 @@
     lastrowid = property(_getlastrowid)
 
 class Statement(object):
-    def __init__(self, cur, sql, row_factory):
+    def __init__(self, connection, sql):
         self.statement = None
         if not isinstance(sql, str):
             raise ValueError, "sql must be a string"
-        self.con = cur.connection
-        self.cur = weakref.ref(cur)
+        self.con = connection
         self.sql = sql # DEBUG ONLY
-        self.row_factory = row_factory
         first_word = self._statement_kind = sql.lstrip().split(" ")[0].upper()
         if first_word in ("INSERT", "UPDATE", "DELETE", "REPLACE"):
-            self.kind = "DML"
+            self.kind = DML
         elif first_word in ("SELECT", "PRAGMA"):
-            self.kind = "DQL"
+            self.kind = DQL
         else:
-            self.kind = "DDL"
+            self.kind = DDL
         self.exhausted = False
+        self.in_use = False
+        #
+        # set by set_row_factory
+        self.row_factory = None
 
         self.statement = c_void_p()
         next_char = c_char_p()
-        ret = sqlite.sqlite3_prepare_v2(self.con.db, sql, -1, byref(self.statement), byref(next_char))
+        sql_char = c_char_p(sql)
+        ret = sqlite.sqlite3_prepare_v2(self.con.db, sql_char, -1, byref(self.statement), byref(next_char))
         if ret == SQLITE_OK and self.statement.value is None:
             # an empty statement, we work around that, as it's the least trouble
             ret = sqlite.sqlite3_prepare_v2(self.con.db, "select 42", -1, byref(self.statement), byref(next_char))
-            self.kind = "DQL"
+            self.kind = DQL
 
         if ret != SQLITE_OK:
             raise self.con._get_exception(ret)
         self.con._remember_statement(self)
         if _check_remaining_sql(next_char.value):
-            raise Warning, "One and only one statement required"
+            raise Warning, "One and only one statement required: %r" % (
+                next_char.value,)
+        # sql_char should remain alive until here
 
         self._build_row_cast_map()
 
+    def set_row_factory(self, row_factory):
+        self.row_factory = row_factory
+
     def _build_row_cast_map(self):
         self.row_cast_map = []
         for i in xrange(sqlite.sqlite3_column_count(self.statement)):
@@ -974,6 +1008,7 @@
         ret = sqlite.sqlite3_reset(self.statement)
         if ret != SQLITE_OK:
             raise self.con._get_exception(ret)
+        self.mark_dirty()
 
         if params is None:
             if sqlite.sqlite3_bind_parameter_count(self.statement) != 0:
@@ -1004,10 +1039,7 @@
                     raise ProgrammingError("missing parameter '%s'" %param)
                 self.set_param(idx, param)
 
-    def __iter__(self):
-        return self
-
-    def next(self):
+    def next(self, cursor):
         self.con._check_closed()
         self.con._check_thread()
         if self.exhausted:
@@ -1023,10 +1055,10 @@
             sqlite.sqlite3_reset(self.statement)
             raise exc
 
-        self._readahead()
+        self._readahead(cursor)
         return item
 
-    def _readahead(self):
+    def _readahead(self, cursor):
         self.column_count = sqlite.sqlite3_column_count(self.statement)
         row = []
         for i in xrange(self.column_count):
@@ -1061,23 +1093,30 @@
 
         row = tuple(row)
         if self.row_factory is not None:
-            row = self.row_factory(self.cur(), row)
+            row = self.row_factory(cursor, row)
         self.item = row
 
     def reset(self):
         self.row_cast_map = None
-        return sqlite.sqlite3_reset(self.statement)
+        ret = sqlite.sqlite3_reset(self.statement)
+        self.in_use = False
+        self.exhausted = False
+        return ret
 
     def finalize(self):
         sqlite.sqlite3_finalize(self.statement)
         self.statement = None
+        self.in_use = False
+
+    def mark_dirty(self):
+        self.in_use = True
 
     def __del__(self):
         sqlite.sqlite3_finalize(self.statement)
         self.statement = None
 
     def _get_description(self):
-        if self.kind == "DML":
+        if self.kind == DML:
             return None
         desc = []
         for i in xrange(sqlite.sqlite3_column_count(self.statement)):
diff --git a/lib_pypy/_subprocess.py b/lib_pypy/_subprocess.py
--- a/lib_pypy/_subprocess.py
+++ b/lib_pypy/_subprocess.py
@@ -35,7 +35,7 @@
 _DuplicateHandle.restype = ctypes.c_int
     
 _WaitForSingleObject = _kernel32.WaitForSingleObject
-_WaitForSingleObject.argtypes = [ctypes.c_int, ctypes.c_int]
+_WaitForSingleObject.argtypes = [ctypes.c_int, ctypes.c_uint]
 _WaitForSingleObject.restype = ctypes.c_int
 
 _GetExitCodeProcess = _kernel32.GetExitCodeProcess
diff --git a/lib_pypy/distributed/test/test_distributed.py b/lib_pypy/distributed/test/test_distributed.py
--- a/lib_pypy/distributed/test/test_distributed.py
+++ b/lib_pypy/distributed/test/test_distributed.py
@@ -9,7 +9,7 @@
 class AppTestDistributed(object):
     def setup_class(cls):
         cls.space = gettestobjspace(**{"objspace.std.withtproxy": True,
-            "usemodules":("_stackless",)})
+            "usemodules":("_continuation",)})
 
     def test_init(self):
         import distributed
@@ -91,10 +91,8 @@
 
 class AppTestDistributedTasklets(object):
     spaceconfig = {"objspace.std.withtproxy": True,
-                   "objspace.usemodules._stackless": True}
+                   "objspace.usemodules._continuation": True}
     def setup_class(cls):
-        #cls.space = gettestobjspace(**{"objspace.std.withtproxy": True,
-        #    "usemodules":("_stackless",)})
         cls.w_test_env = cls.space.appexec([], """():
         from distributed import test_env
         return test_env
diff --git a/lib_pypy/distributed/test/test_greensock.py b/lib_pypy/distributed/test/test_greensock.py
--- a/lib_pypy/distributed/test/test_greensock.py
+++ b/lib_pypy/distributed/test/test_greensock.py
@@ -10,7 +10,7 @@
         if not option.runappdirect:
             py.test.skip("Cannot run this on top of py.py because of PopenGateway")
         cls.space = gettestobjspace(**{"objspace.std.withtproxy": True,
-                                       "usemodules":("_stackless",)})
+                                       "usemodules":("_continuation",)})
         cls.w_remote_side_code = cls.space.appexec([], """():
         import sys
         sys.path.insert(0, '%s')
diff --git a/lib_pypy/distributed/test/test_socklayer.py b/lib_pypy/distributed/test/test_socklayer.py
--- a/lib_pypy/distributed/test/test_socklayer.py
+++ b/lib_pypy/distributed/test/test_socklayer.py
@@ -9,7 +9,8 @@
 class AppTestSocklayer:
     def setup_class(cls):
         cls.space = gettestobjspace(**{"objspace.std.withtproxy": True,
-                                       "usemodules":("_stackless","_socket", "select")})
+                                       "usemodules":("_continuation",
+                                                     "_socket", "select")})
     
     def test_socklayer(self):
         class X(object):
diff --git a/lib_pypy/greenlet.py b/lib_pypy/greenlet.py
--- a/lib_pypy/greenlet.py
+++ b/lib_pypy/greenlet.py
@@ -1,1 +1,151 @@
-from _stackless import greenlet
+import _continuation, sys
+
+
+# ____________________________________________________________
+# Exceptions
+
+class GreenletExit(Exception):
+    """This special exception does not propagate to the parent greenlet; it
+can be used to kill a single greenlet."""
+
+error = _continuation.error
+
+# ____________________________________________________________
+# Helper function
+
+def getcurrent():
+    "Returns the current greenlet (i.e. the one which called this function)."
+    try:
+        return _tls.current
+    except AttributeError:
+        # first call in this thread: current == main
+        _green_create_main()
+        return _tls.current
+
+# ____________________________________________________________
+# The 'greenlet' class
+
+_continulet = _continuation.continulet
+
+class greenlet(_continulet):
+    getcurrent = staticmethod(getcurrent)
+    error = error
+    GreenletExit = GreenletExit
+    __main = False
+    __started = False
+
+    def __new__(cls, *args, **kwds):
+        self = _continulet.__new__(cls)
+        self.parent = getcurrent()
+        return self
+
+    def __init__(self, run=None, parent=None):
+        if run is not None:
+            self.run = run
+        if parent is not None:
+            self.parent = parent
+
+    def switch(self, *args):
+        "Switch execution to this greenlet, optionally passing the values "
+        "given as argument(s).  Returns the value passed when switching back."
+        return self.__switch(_continulet.switch, args)
+
+    def throw(self, typ=GreenletExit, val=None, tb=None):
+        "raise exception in greenlet, return value passed when switching back"
+        return self.__switch(_continulet.throw, typ, val, tb)
+
+    def __switch(target, unbound_method, *args):
+        current = getcurrent()
+        #
+        while not target:
+            if not target.__started:
+                if unbound_method != _continulet.throw:
+                    greenlet_func = _greenlet_start
+                else:
+                    greenlet_func = _greenlet_throw
+                _continulet.__init__(target, greenlet_func, *args)
+                unbound_method = _continulet.switch
+                args = ()
+                target.__started = True
+                break
+            # already done, go to the parent instead
+            # (NB. infinite loop possible, but unlikely, unless you mess
+            # up the 'parent' explicitly.  Good enough, because a Ctrl-C
+            # will show that the program is caught in this loop here.)
+            target = target.parent
+        #
+        try:
+            if current.__main:
+                if target.__main:
+                    # switch from main to main
+                    if unbound_method == _continulet.throw:
+                        raise args[0], args[1], args[2]
+                    (args,) = args
+                else:
+                    # enter from main to target
+                    args = unbound_method(target, *args)
+            else:
+                if target.__main:
+                    # leave to go to target=main
+                    args = unbound_method(current, *args)
+                else:
+                    # switch from non-main to non-main
+                    args = unbound_method(current, *args, to=target)
+        except GreenletExit, e:
+            args = (e,)
+        finally:
+            _tls.current = current
+        #
+        if len(args) == 1:
+            return args[0]
+        else:
+            return args
+
+    def __nonzero__(self):
+        return self.__main or _continulet.is_pending(self)
+
+    @property
+    def dead(self):
+        return self.__started and not self
+
+    @property
+    def gr_frame(self):
+        raise NotImplementedError("attribute 'gr_frame' of greenlet objects")
+
+# ____________________________________________________________
+# Internal stuff
+
+try:
+    from thread import _local
+except ImportError:
+    class _local(object):    # assume no threads
+        pass
+
+_tls = _local()
+
+def _green_create_main():
+    # create the main greenlet for this thread
+    _tls.current = None
+    gmain = greenlet.__new__(greenlet)
+    gmain._greenlet__main = True
+    gmain._greenlet__started = True
+    assert gmain.parent is None
+    _tls.main = gmain
+    _tls.current = gmain
+
+def _greenlet_start(greenlet, args):
+    _tls.current = greenlet
+    try:
+        res = greenlet.run(*args)
+    finally:
+        if greenlet.parent is not _tls.main:
+            _continuation.permute(greenlet, greenlet.parent)
+    return (res,)
+
+def _greenlet_throw(greenlet, exc, value, tb):
+    _tls.current = greenlet
+    try:
+        raise exc, value, tb
+    finally:
+        if greenlet.parent is not _tls.main:
+            _continuation.permute(greenlet, greenlet.parent)
diff --git a/lib_pypy/pypy_test/test_coroutine.py b/lib_pypy/pypy_test/test_coroutine.py
--- a/lib_pypy/pypy_test/test_coroutine.py
+++ b/lib_pypy/pypy_test/test_coroutine.py
@@ -2,7 +2,7 @@
 from py.test import skip, raises
 
 try:
-    from lib_pypy.stackless import coroutine, CoroutineExit
+    from stackless import coroutine, CoroutineExit
 except ImportError, e:
     skip('cannot import stackless: %s' % (e,))
 
@@ -20,10 +20,6 @@
         assert not co.is_zombie
 
     def test_is_zombie_del_without_frame(self):
-        try:
-            import _stackless # are we on pypy with a stackless build?
-        except ImportError:
-            skip("only works on pypy-c-stackless")
         import gc
         res = []
         class MyCoroutine(coroutine):
@@ -45,10 +41,6 @@
         assert res[0], "is_zombie was False in __del__"
 
     def test_is_zombie_del_with_frame(self):
-        try:
-            import _stackless # are we on pypy with a stackless build?
-        except ImportError:
-            skip("only works on pypy-c-stackless")
         import gc
         res = []
         class MyCoroutine(coroutine):
diff --git a/lib_pypy/pyrepl/reader.py b/lib_pypy/pyrepl/reader.py
--- a/lib_pypy/pyrepl/reader.py
+++ b/lib_pypy/pyrepl/reader.py
@@ -401,13 +401,19 @@
             return "(arg: %s) "%self.arg
         if "\n" in self.buffer:
             if lineno == 0:
-                return self._ps2
+                res = self.ps2
             elif lineno == self.buffer.count("\n"):
-                return self._ps4
+                res = self.ps4
             else:
-                return self._ps3
+                res = self.ps3
         else:
-            return self._ps1
+            res = self.ps1
+        # Lazily call str() on self.psN, and cache the results using as key
+        # the object on which str() was called.  This ensures that even if the
+        # same object is used e.g. for ps1 and ps2, str() is called only once.
+        if res not in self._pscache:
+            self._pscache[res] = str(res)
+        return self._pscache[res]
 
     def push_input_trans(self, itrans):
         self.input_trans_stack.append(self.input_trans)
@@ -473,8 +479,7 @@
             self.pos = 0
             self.dirty = 1
             self.last_command = None
-            self._ps1, self._ps2, self._ps3, self._ps4 = \
-                           map(str, [self.ps1, self.ps2, self.ps3, self.ps4])
+            self._pscache = {}
         except:
             self.restore()
             raise
diff --git a/lib_pypy/pyrepl/readline.py b/lib_pypy/pyrepl/readline.py
--- a/lib_pypy/pyrepl/readline.py
+++ b/lib_pypy/pyrepl/readline.py
@@ -33,7 +33,7 @@
 from pyrepl.unix_console import UnixConsole, _error
 
 
-ENCODING = 'latin1'     # XXX hard-coded
+ENCODING = sys.getfilesystemencoding() or 'latin1'     # XXX review
 
 __all__ = ['add_history',
            'clear_history',
diff --git a/pypy/annotation/builtin.py b/pypy/annotation/builtin.py
--- a/pypy/annotation/builtin.py
+++ b/pypy/annotation/builtin.py
@@ -308,9 +308,6 @@
             clsdef = clsdef.commonbase(cdef)
     return SomeInstance(clsdef)
 
-def robjmodel_we_are_translated():
-    return immutablevalue(True)
-
 def robjmodel_r_dict(s_eqfn, s_hashfn, s_force_non_null=None):
     if s_force_non_null is None:
         force_non_null = False
@@ -376,8 +373,6 @@
 
 BUILTIN_ANALYZERS[pypy.rlib.rarithmetic.intmask] = rarith_intmask
 BUILTIN_ANALYZERS[pypy.rlib.objectmodel.instantiate] = robjmodel_instantiate
-BUILTIN_ANALYZERS[pypy.rlib.objectmodel.we_are_translated] = (
-    robjmodel_we_are_translated)
 BUILTIN_ANALYZERS[pypy.rlib.objectmodel.r_dict] = robjmodel_r_dict
 BUILTIN_ANALYZERS[pypy.rlib.objectmodel.hlinvoke] = robjmodel_hlinvoke
 BUILTIN_ANALYZERS[pypy.rlib.objectmodel.keepalive_until_here] = robjmodel_keepalive_until_here
@@ -416,7 +411,8 @@
 from pypy.annotation.model import SomePtr
 from pypy.rpython.lltypesystem import lltype
 
-def malloc(s_T, s_n=None, s_flavor=None, s_zero=None, s_track_allocation=None):
+def malloc(s_T, s_n=None, s_flavor=None, s_zero=None, s_track_allocation=None,
+           s_add_memory_pressure=None):
     assert (s_n is None or s_n.knowntype == int
             or issubclass(s_n.knowntype, pypy.rlib.rarithmetic.base_int))
     assert s_T.is_constant()
@@ -432,6 +428,8 @@
     else:
         assert s_flavor.is_constant()
         assert s_track_allocation is None or s_track_allocation.is_constant()
+        assert (s_add_memory_pressure is None or
+                s_add_memory_pressure.is_constant())
         # not sure how to call malloc() for the example 'p' in the
         # presence of s_extraargs
         r = SomePtr(lltype.Ptr(s_T.const))
diff --git a/pypy/annotation/description.py b/pypy/annotation/description.py
--- a/pypy/annotation/description.py
+++ b/pypy/annotation/description.py
@@ -399,9 +399,7 @@
                 if b1 is object:
                     continue
                 if b1.__dict__.get('_mixin_', False):
-                    assert b1.__bases__ == () or b1.__bases__ == (object,), (
-                        "mixin class %r should have no base" % (b1,))
-                    self.add_sources_for_class(b1, mixin=True)
+                    self.add_mixin(b1)
                 else:
                     assert base is object, ("multiple inheritance only supported "
                                             "with _mixin_: %r" % (cls,))
@@ -469,6 +467,15 @@
                 return
         self.classdict[name] = Constant(value)
 
+    def add_mixin(self, base):
+        for subbase in base.__bases__:
+            if subbase is object:
+                continue
+            assert subbase.__dict__.get("_mixin_", False), ("Mixin class %r has non"
+                "mixin base class %r" % (base, subbase))
+            self.add_mixin(subbase)
+        self.add_sources_for_class(base, mixin=True)
+
     def add_sources_for_class(self, cls, mixin=False):
         for name, value in cls.__dict__.items():
             self.add_source_attribute(name, value, mixin)
diff --git a/pypy/config/makerestdoc.py b/pypy/config/makerestdoc.py
--- a/pypy/config/makerestdoc.py
+++ b/pypy/config/makerestdoc.py
@@ -134,7 +134,7 @@
         for child in self._children:
             subpath = fullpath + "." + child._name
             toctree.append(subpath)
-        content.add(Directive("toctree", *toctree, maxdepth=4))
+        content.add(Directive("toctree", *toctree, **{'maxdepth': 4}))
         content.join(
             ListItem(Strong("name:"), self._name),
             ListItem(Strong("description:"), self.doc))
diff --git a/pypy/config/pypyoption.py b/pypy/config/pypyoption.py
--- a/pypy/config/pypyoption.py
+++ b/pypy/config/pypyoption.py
@@ -33,7 +33,8 @@
      "struct", "_hashlib", "_md5", "_sha", "_minimal_curses", "cStringIO",
      "thread", "itertools", "pyexpat", "_ssl", "cpyext", "array",
      "_bisect", "binascii", "_multiprocessing", '_warnings',
-     "_collections", "_multibytecodec", "micronumpy", "_ffi"]
+     "_collections", "_multibytecodec", "micronumpy", "_ffi",
+     "_continuation"]
 ))
 
 translation_modules = default_modules.copy()
@@ -99,6 +100,7 @@
     "_ssl"      : ["pypy.module._ssl.interp_ssl"],
     "_hashlib"  : ["pypy.module._ssl.interp_ssl"],
     "_minimal_curses": ["pypy.module._minimal_curses.fficurses"],
+    "_continuation": ["pypy.rlib.rstacklet"],
     }
 
 def get_module_validator(modname):
diff --git a/pypy/config/test/test_config.py b/pypy/config/test/test_config.py
--- a/pypy/config/test/test_config.py
+++ b/pypy/config/test/test_config.py
@@ -1,5 +1,5 @@
 from pypy.config.config import *
-import py
+import py, sys
 
 def make_description():
     gcoption = ChoiceOption('name', 'GC name', ['ref', 'framework'], 'ref')
@@ -69,13 +69,15 @@
     attrs = dir(config)
     assert '__repr__' in attrs        # from the type
     assert '_cfgimpl_values' in attrs # from self
-    assert 'gc' in attrs              # custom attribute
-    assert 'objspace' in attrs        # custom attribute
+    if sys.version_info >= (2, 6):
+        assert 'gc' in attrs              # custom attribute
+        assert 'objspace' in attrs        # custom attribute
     #
     attrs = dir(config.gc)
-    assert 'name' in attrs
-    assert 'dummy' in attrs
-    assert 'float' in attrs
+    if sys.version_info >= (2, 6):
+        assert 'name' in attrs
+        assert 'dummy' in attrs
+        assert 'float' in attrs
 
 def test_arbitrary_option():
     descr = OptionDescription("top", "", [
@@ -279,11 +281,11 @@
 
 def test_underscore_in_option_name():
     descr = OptionDescription("opt", "", [
-        BoolOption("_stackless", "", default=False),
+        BoolOption("_foobar", "", default=False),
     ])
     config = Config(descr)
     parser = to_optparse(config)
-    assert parser.has_option("--_stackless")
+    assert parser.has_option("--_foobar")
 
 def test_none():
     dummy1 = BoolOption('dummy1', 'doc dummy', default=False, cmdline=None)
diff --git a/pypy/config/translationoption.py b/pypy/config/translationoption.py
--- a/pypy/config/translationoption.py
+++ b/pypy/config/translationoption.py
@@ -13,6 +13,10 @@
 DEFL_LOW_INLINE_THRESHOLD = DEFL_INLINE_THRESHOLD / 2.0
 
 DEFL_GC = "minimark"
+if sys.platform.startswith("linux"):
+    DEFL_ROOTFINDER_WITHJIT = "asmgcc"
+else:
+    DEFL_ROOTFINDER_WITHJIT = "shadowstack"
 
 IS_64_BITS = sys.maxint > 2147483647
 
@@ -24,10 +28,9 @@
 
 translation_optiondescription = OptionDescription(
         "translation", "Translation Options", [
-    BoolOption("stackless", "enable stackless features during compilation",
-               default=False, cmdline="--stackless",
-               requires=[("translation.type_system", "lltype"),
-                         ("translation.gcremovetypeptr", False)]),  # XXX?
+    BoolOption("continuation", "enable single-shot continuations",
+               default=False, cmdline="--continuation",
+               requires=[("translation.type_system", "lltype")]),
     ChoiceOption("type_system", "Type system to use when RTyping",
                  ["lltype", "ootype"], cmdline=None, default="lltype",
                  requires={
@@ -66,7 +69,8 @@
                      "statistics": [("translation.gctransformer", "framework")],
                      "generation": [("translation.gctransformer", "framework")],
                      "hybrid": [("translation.gctransformer", "framework")],
-                     "boehm": [("translation.gctransformer", "boehm")],
+                     "boehm": [("translation.gctransformer", "boehm"),
+                               ("translation.continuation", False)],  # breaks
                      "markcompact": [("translation.gctransformer", "framework")],
                      "minimark": [("translation.gctransformer", "framework")],
                      },
@@ -109,7 +113,7 @@
     BoolOption("jit", "generate a JIT",
                default=False,
                suggests=[("translation.gc", DEFL_GC),
-                         ("translation.gcrootfinder", "asmgcc"),
+                         ("translation.gcrootfinder", DEFL_ROOTFINDER_WITHJIT),
                          ("translation.list_comprehension_operations", True)]),
     ChoiceOption("jit_backend", "choose the backend for the JIT",
                  ["auto", "x86", "x86-without-sse2", "llvm"],
@@ -385,8 +389,6 @@
             config.translation.suggest(withsmallfuncsets=5)
         elif word == 'jit':
             config.translation.suggest(jit=True)
-            if config.translation.stackless:
-                raise NotImplementedError("JIT conflicts with stackless for now")
         elif word == 'removetypeptr':
             config.translation.suggest(gcremovetypeptr=True)
         else:
diff --git a/pypy/doc/_ref.txt b/pypy/doc/_ref.txt
--- a/pypy/doc/_ref.txt
+++ b/pypy/doc/_ref.txt
@@ -1,11 +1,10 @@
 .. _`ctypes_configure/doc/sample.py`: https://bitbucket.org/pypy/pypy/src/default/ctypes_configure/doc/sample.py
 .. _`demo/`: https://bitbucket.org/pypy/pypy/src/default/demo/
-.. _`demo/pickle_coroutine.py`: https://bitbucket.org/pypy/pypy/src/default/demo/pickle_coroutine.py
 .. _`lib-python/`: https://bitbucket.org/pypy/pypy/src/default/lib-python/
 .. _`lib-python/2.7/dis.py`: https://bitbucket.org/pypy/pypy/src/default/lib-python/2.7/dis.py
 .. _`lib_pypy/`: https://bitbucket.org/pypy/pypy/src/default/lib_pypy/
+.. _`lib_pypy/greenlet.py`: https://bitbucket.org/pypy/pypy/src/default/lib_pypy/greenlet.py
 .. _`lib_pypy/pypy_test/`: https://bitbucket.org/pypy/pypy/src/default/lib_pypy/pypy_test/
-.. _`lib_pypy/stackless.py`: https://bitbucket.org/pypy/pypy/src/default/lib_pypy/stackless.py
 .. _`lib_pypy/tputil.py`: https://bitbucket.org/pypy/pypy/src/default/lib_pypy/tputil.py
 .. _`pypy/annotation`:
 .. _`pypy/annotation/`: https://bitbucket.org/pypy/pypy/src/default/pypy/annotation/
@@ -55,7 +54,6 @@
 .. _`pypy/module`:
 .. _`pypy/module/`: https://bitbucket.org/pypy/pypy/src/default/pypy/module/
 .. _`pypy/module/__builtin__/__init__.py`: https://bitbucket.org/pypy/pypy/src/default/pypy/module/__builtin__/__init__.py
-.. _`pypy/module/_stackless/test/test_composable_coroutine.py`: https://bitbucket.org/pypy/pypy/src/default/pypy/module/_stackless/test/test_composable_coroutine.py
 .. _`pypy/objspace`:
 .. _`pypy/objspace/`: https://bitbucket.org/pypy/pypy/src/default/pypy/objspace/
 .. _`pypy/objspace/dump.py`: https://bitbucket.org/pypy/pypy/src/default/pypy/objspace/dump.py
@@ -117,6 +115,7 @@
 .. _`pypy/translator/`: https://bitbucket.org/pypy/pypy/src/default/pypy/translator/
 .. _`pypy/translator/backendopt/`: https://bitbucket.org/pypy/pypy/src/default/pypy/translator/backendopt/
 .. _`pypy/translator/c/`: https://bitbucket.org/pypy/pypy/src/default/pypy/translator/c/
+.. _`pypy/translator/c/src/stacklet/`: https://bitbucket.org/pypy/pypy/src/default/pypy/translator/c/src/stacklet/
 .. _`pypy/translator/cli/`: https://bitbucket.org/pypy/pypy/src/default/pypy/translator/cli/
 .. _`pypy/translator/goal/`: https://bitbucket.org/pypy/pypy/src/default/pypy/translator/goal/
 .. _`pypy/translator/jvm/`: https://bitbucket.org/pypy/pypy/src/default/pypy/translator/jvm/
diff --git a/pypy/doc/architecture.rst b/pypy/doc/architecture.rst
--- a/pypy/doc/architecture.rst
+++ b/pypy/doc/architecture.rst
@@ -153,7 +153,7 @@
 
 * Optionally, `various transformations`_ can then be applied which, for
   example, perform optimizations such as inlining, add capabilities
-  such as stackless_-style concurrency, or insert code for the
+  such as stackless-style concurrency (deprecated), or insert code for the
   `garbage collector`_.
 
 * Then, the graphs are converted to source code for the target platform
@@ -255,7 +255,6 @@
 
 .. _Python: http://docs.python.org/reference/
 .. _Psyco: http://psyco.sourceforge.net
-.. _stackless: stackless.html
 .. _`generate Just-In-Time Compilers`: jit/index.html
 .. _`JIT Generation in PyPy`: jit/index.html
 .. _`implement your own interpreter`: http://morepypy.blogspot.com/2011/04/tutorial-writing-interpreter-with-pypy.html
diff --git a/pypy/doc/conf.py b/pypy/doc/conf.py
--- a/pypy/doc/conf.py
+++ b/pypy/doc/conf.py
@@ -45,9 +45,9 @@
 # built documents.
 #
 # The short X.Y version.
-version = '1.5'
+version = '1.6'
 # The full version, including alpha/beta/rc tags.
-release = '1.5'
+release = '1.6'
 
 # The language for content autogenerated by Sphinx. Refer to documentation
 # for a list of supported languages.
diff --git a/pypy/doc/config/objspace.usemodules._stackless.txt b/pypy/doc/config/objspace.usemodules._continuation.txt
rename from pypy/doc/config/objspace.usemodules._stackless.txt
rename to pypy/doc/config/objspace.usemodules._continuation.txt
--- a/pypy/doc/config/objspace.usemodules._stackless.txt
+++ b/pypy/doc/config/objspace.usemodules._continuation.txt
@@ -1,6 +1,4 @@
-Use the '_stackless' module. 
+Use the '_continuation' module. 
 
-Exposes the `stackless` primitives, and also implies a stackless build. 
-See also :config:`translation.stackless`.
-
-.. _`stackless`: ../stackless.html
+Exposes the `continulet` app-level primitives.
+See also :config:`translation.continuation`.
diff --git a/pypy/doc/config/translation.stackless.txt b/pypy/doc/config/translation.continuation.txt
rename from pypy/doc/config/translation.stackless.txt
rename to pypy/doc/config/translation.continuation.txt
--- a/pypy/doc/config/translation.stackless.txt
+++ b/pypy/doc/config/translation.continuation.txt
@@ -1,5 +1,2 @@
-Run the `stackless transform`_ on each generated graph, which enables the use
-of coroutines at RPython level and the "stackless" module when translating
-PyPy.
-
-.. _`stackless transform`: ../stackless.html
+Enable the use of a stackless-like primitive called "stacklet".
+In PyPy, this is exposed at app-level by the "_continuation" module.
diff --git a/pypy/doc/contributor.rst b/pypy/doc/contributor.rst
--- a/pypy/doc/contributor.rst
+++ b/pypy/doc/contributor.rst
@@ -9,22 +9,22 @@
     Armin Rigo
     Maciej Fijalkowski
     Carl Friedrich Bolz
+    Antonio Cuni
     Amaury Forgeot d'Arc
-    Antonio Cuni
     Samuele Pedroni
     Michael Hudson
     Holger Krekel
+    Benjamin Peterson
     Christian Tismer
-    Benjamin Peterson
+    Hakan Ardo
+    Alex Gaynor
     Eric van Riet Paap
-    Anders Chrigström
-    Håkan Ardö
+    Anders Chrigstrom
+    David Schneider
     Richard Emslie
     Dan Villiom Podlaski Christiansen
     Alexander Schremmer
-    Alex Gaynor
-    David Schneider
-    Aurelién Campeas
+    Aurelien Campeas
     Anders Lehmann
     Camillo Bruni
     Niklaus Haldimann
@@ -35,16 +35,17 @@
     Bartosz Skowron
     Jakub Gustak
     Guido Wesdorp
+    Daniel Roberts
     Adrien Di Mascio
     Laura Creighton
     Ludovic Aubry
     Niko Matsakis
-    Daniel Roberts
     Jason Creighton
-    Jacob Hallén
+    Jacob Hallen
     Alex Martelli
     Anders Hammarquist
     Jan de Mooij
+    Wim Lavrijsen
     Stephan Diehl
     Michael Foord
     Stefan Schwarzer
@@ -55,9 +56,13 @@
     Alexandre Fayolle
     Marius Gedminas
     Simon Burton
+    Justin Peel
     Jean-Paul Calderone
     John Witulski
+    Lukas Diekmann
+    holger krekel
     Wim Lavrijsen
+    Dario Bertini
     Andreas Stührk
     Jean-Philippe St. Pierre
     Guido van Rossum
@@ -69,15 +74,16 @@
     Georg Brandl
     Gerald Klix
     Wanja Saatkamp
+    Ronny Pfannschmidt
     Boris Feigin
     Oscar Nierstrasz
-    Dario Bertini
     David Malcolm
     Eugene Oden
     Henry Mason
+    Sven Hager
     Lukas Renggli
+    Ilya Osadchiy
     Guenter Jantzen
-    Ronny Pfannschmidt
     Bert Freudenberg
     Amit Regmi
     Ben Young
@@ -94,8 +100,8 @@
     Jared Grubb
     Karl Bartel
     Gabriel Lavoie
+    Victor Stinner
     Brian Dorsey
-    Victor Stinner
     Stuart Williams
     Toby Watson
     Antoine Pitrou
@@ -106,19 +112,23 @@
     Jonathan David Riehl
     Elmo Mäntynen
     Anders Qvist
-    Beatrice Düring
+    Beatrice During
     Alexander Sedov
+    Timo Paulssen
+    Corbin Simpson
     Vincent Legoll
+    Romain Guillebert
     Alan McIntyre
-    Romain Guillebert
     Alex Perry
     Jens-Uwe Mager
+    Simon Cross
     Dan Stromberg
-    Lukas Diekmann
+    Guillebert Romain
     Carl Meyer
     Pieter Zieschang
     Alejandro J. Cura
     Sylvain Thenault
+    Christoph Gerum
     Travis Francis Athougies
     Henrik Vendelbo
     Lutz Paelike
@@ -129,6 +139,7 @@
     Miguel de Val Borro
     Ignas Mikalajunas
     Artur Lisiecki
+    Philip Jenvey
     Joshua Gilbert
     Godefroid Chappelle
     Yusei Tahara
@@ -137,24 +148,29 @@
     Gustavo Niemeyer
     William Leslie
     Akira Li
-    Kristján Valur Jónsson
+    Kristjan Valur Jonsson
     Bobby Impollonia
+    Michael Hudson-Doyle
     Andrew Thompson
     Anders Sigfridsson
+    Floris Bruynooghe
     Jacek Generowicz
     Dan Colish
-    Sven Hager
     Zooko Wilcox-O Hearn
+    Dan Villiom Podlaski Christiansen
     Anders Hammarquist
+    Chris Lambacher
     Dinu Gherman
     Dan Colish
+    Brett Cannon
     Daniel Neuhäuser
     Michael Chermside
     Konrad Delong
     Anna Ravencroft
     Greg Price
     Armin Ronacher
+    Christian Muirhead
     Jim Baker
-    Philip Jenvey
     Rodrigo Araújo
+    Romain Guillebert
 
diff --git a/pypy/doc/cpython_differences.rst b/pypy/doc/cpython_differences.rst
--- a/pypy/doc/cpython_differences.rst
+++ b/pypy/doc/cpython_differences.rst
@@ -24,6 +24,7 @@
     _bisect
     _codecs
     _collections
+    `_continuation`_
     `_ffi`_
     _hashlib
     _io
@@ -84,9 +85,12 @@
 
     _winreg
 
-  Extra module with Stackless_ only:
-
-    _stackless
+  Note that only some of these modules are built-in in a typical
+  CPython installation, and the rest is from non built-in extension
+  modules.  This means that e.g. ``import parser`` will, on CPython,
+  find a local file ``parser.py``, while ``import sys`` will not find a
+  local file ``sys.py``.  In PyPy the difference does not exist: all
+  these modules are built-in.
 
 * Supported by being rewritten in pure Python (possibly using ``ctypes``):
   see the `lib_pypy/`_ directory.  Examples of modules that we
@@ -101,11 +105,11 @@
 
 .. the nonstandard modules are listed below...
 .. _`__pypy__`: __pypy__-module.html
+.. _`_continuation`: stackless.html
 .. _`_ffi`: ctypes-implementation.html
 .. _`_rawffi`: ctypes-implementation.html
 .. _`_minimal_curses`: config/objspace.usemodules._minimal_curses.html
 .. _`cpyext`: http://morepypy.blogspot.com/2010/04/using-cpython-extension-modules-with.html
-.. _Stackless: stackless.html
 
 
 Differences related to garbage collection strategies
@@ -280,7 +284,14 @@
   never a dictionary as it sometimes is in CPython. Assigning to
   ``__builtins__`` has no effect.
 
-* object identity of immutable keys in dictionaries is not necessarily preserved.
-  Never compare immutable objects with ``is``.
+* Do not compare immutable objects with ``is``.  For example on CPython
+  it is true that ``x is 0`` works, i.e. does the same as ``type(x) is
+  int and x == 0``, but it is so by accident.  If you do instead
+  ``x is 1000``, then it stops working, because 1000 is too large and
+  doesn't come from the internal cache.  In PyPy it fails to work in
+  both cases, because we have no need for a cache at all.
+
+* Also, object identity of immutable keys in dictionaries is not necessarily
+  preserved.
 
 .. include:: _ref.txt
diff --git a/pypy/doc/faq.rst b/pypy/doc/faq.rst
--- a/pypy/doc/faq.rst
+++ b/pypy/doc/faq.rst
@@ -315,6 +315,28 @@
 
 .. _`Andrew Brown's tutorial`: http://morepypy.blogspot.com/2011/04/tutorial-writing-interpreter-with-pypy.html
 
+---------------------------------------------------------
+Can RPython modules for PyPy be translated independently?
+---------------------------------------------------------
+
+No, you have to rebuild the entire interpreter.  This means two things:
+
+* It is imperative to use test-driven development.  You have to test
+  exhaustively your module in pure Python, before even attempting to
+  translate it.  Once you translate it, you should have only a few typing
+  issues left to fix, but otherwise the result should work out of the box.
+
+* Second, and perhaps most important: do you have a really good reason
+  for writing the module in RPython in the first place?  Nowadays you
+  should really look at alternatives, like writing it in pure Python,
+  using ctypes if it needs to call C code.  Other alternatives are being
+  developed too (as of summer 2011), like a Cython binding.
+
+In this context it is not that important to be able to translate
+RPython modules independently of translating the complete interpreter.
+(It could be done given enough efforts, but it's a really serious
+undertaking.  Consider it as quite unlikely for now.)
+
 ----------------------------------------------------------
 Why does PyPy draw a Mandelbrot fractal while translating?
 ----------------------------------------------------------
diff --git a/pypy/doc/garbage_collection.rst b/pypy/doc/garbage_collection.rst
--- a/pypy/doc/garbage_collection.rst
+++ b/pypy/doc/garbage_collection.rst
@@ -147,7 +147,7 @@
 You can read more about them at the start of
 `pypy/rpython/memory/gc/minimark.py`_.
 
-In more details:
+In more detail:
 
 - The small newly malloced objects are allocated in the nursery (case 1).
   All objects living in the nursery are "young".
diff --git a/pypy/doc/getting-started-python.rst b/pypy/doc/getting-started-python.rst
--- a/pypy/doc/getting-started-python.rst
+++ b/pypy/doc/getting-started-python.rst
@@ -32,7 +32,10 @@
 .. _`windows document`: windows.html
 
 You can translate the whole of PyPy's Python interpreter to low level C code,
-or `CLI code`_.
+or `CLI code`_.  If you intend to build using gcc, check to make sure that
+the version you have is not 4.2 or you will run into `this bug`_.
+
+.. _`this bug`: https://bugs.launchpad.net/ubuntu/+source/gcc-4.2/+bug/187391
 
 1. First `download a pre-built PyPy`_ for your architecture which you will
    use to translate your Python interpreter.  It is, of course, possible to
@@ -64,7 +67,6 @@
    * ``libssl-dev`` (for the optional ``_ssl`` module)
    * ``libgc-dev`` (for the Boehm garbage collector: only needed when translating with `--opt=0, 1` or `size`)
    * ``python-sphinx`` (for the optional documentation build.  You need version 1.0.7 or later)
-   * ``python-greenlet`` (for the optional stackless support in interpreted mode/testing)
 
 
 3. Translation is time-consuming -- 45 minutes on a very fast machine --
@@ -102,7 +104,7 @@
 
     $ ./pypy-c
     Python 2.7.0 (61ef2a11b56a, Mar 02 2011, 03:00:11)
-    [PyPy 1.5.0-alpha0 with GCC 4.4.3] on linux2
+    [PyPy 1.6.0 with GCC 4.4.3] on linux2
     Type "help", "copyright", "credits" or "license" for more information.
     And now for something completely different: ``this sentence is false''
     >>>> 46 - 4
@@ -117,19 +119,8 @@
 Installation_ below.
 
 The ``translate.py`` script takes a very large number of options controlling
-what to translate and how.  See ``translate.py -h``. Some of the more
-interesting options (but for now incompatible with the JIT) are:
-
-   * ``--stackless``: this produces a pypy-c that includes features
-     inspired by `Stackless Python `__.
-
-   * ``--gc=boehm|ref|marknsweep|semispace|generation|hybrid|minimark``:
-     choose between using
-     the `Boehm-Demers-Weiser garbage collector`_, our reference
-     counting implementation or one of own collector implementations
-     (the default depends on the optimization level but is usually
-     ``minimark``).
-
+what to translate and how.  See ``translate.py -h``. The default options
+should be suitable for mostly everybody by now.
 Find a more detailed description of the various options in our `configuration
 sections`_.
 
@@ -162,7 +153,7 @@
 
     $ ./pypy-cli
     Python 2.7.0 (61ef2a11b56a, Mar 02 2011, 03:00:11)
-    [PyPy 1.5.0-alpha0] on linux2
+    [PyPy 1.6.0] on linux2
     Type "help", "copyright", "credits" or "license" for more information.
     And now for something completely different: ``distopian and utopian chairs''
     >>>> 
@@ -199,7 +190,7 @@
 
         $ ./pypy-jvm 
         Python 2.7.0 (61ef2a11b56a, Mar 02 2011, 03:00:11)
-        [PyPy 1.5.0-alpha0] on linux2
+        [PyPy 1.6.0] on linux2
         Type "help", "copyright", "credits" or "license" for more information.
         And now for something completely different: ``# assert did not crash''
         >>>> 
@@ -238,7 +229,7 @@
 the ``bin/pypy`` executable.
 
 To install PyPy system wide on unix-like systems, it is recommended to put the
-whole hierarchy alone (e.g. in ``/opt/pypy1.5``) and put a symlink to the
+whole hierarchy alone (e.g. in ``/opt/pypy1.6``) and put a symlink to the
 ``pypy`` executable into ``/usr/bin`` or ``/usr/local/bin``
 
 If the executable fails to find suitable libraries, it will report
diff --git a/pypy/doc/getting-started.rst b/pypy/doc/getting-started.rst
--- a/pypy/doc/getting-started.rst
+++ b/pypy/doc/getting-started.rst
@@ -53,11 +53,11 @@
 PyPy is ready to be executed as soon as you unpack the tarball or the zip
 file, with no need to install it in any specific location::
 
-    $ tar xf pypy-1.5-linux.tar.bz2
+    $ tar xf pypy-1.6-linux.tar.bz2
 
-    $ ./pypy-1.5-linux/bin/pypy
+    $ ./pypy-1.6/bin/pypy
     Python 2.7.1 (?, Apr 27 2011, 12:44:21)
-    [PyPy 1.5.0-alpha0 with GCC 4.4.3] on linux2
+    [PyPy 1.6.0 with GCC 4.4.3] on linux2
     Type "help", "copyright", "credits" or "license" for more information.
     And now for something completely different: ``implementing LOGO in LOGO:
     "turtles all the way down"''
@@ -73,16 +73,16 @@
 
     $ curl -O http://python-distribute.org/distribute_setup.py
 
-    $ curl -O https://github.com/pypa/pip/raw/master/contrib/get-pip.py
+    $ curl -O https://raw.github.com/pypa/pip/master/contrib/get-pip.py
 
-    $ ./pypy-1.5-linux/bin/pypy distribute_setup.py
+    $ ./pypy-1.6/bin/pypy distribute_setup.py
 
-    $ ./pypy-1.5-linux/bin/pypy get-pip.py
+    $ ./pypy-1.6/bin/pypy get-pip.py
 
-    $ ./pypy-1.5-linux/bin/pip install pygments  # for example
+    $ ./pypy-1.6/bin/pip install pygments  # for example
 
-3rd party libraries will be installed in ``pypy-1.5-linux/site-packages``, and
-the scripts in ``pypy-1.5-linux/bin``.
+3rd party libraries will be installed in ``pypy-1.6/site-packages``, and
+the scripts in ``pypy-1.6/bin``.
 
 Installing using virtualenv
 ---------------------------
diff --git a/pypy/doc/how-to-release.rst b/pypy/doc/how-to-release.rst
--- a/pypy/doc/how-to-release.rst
+++ b/pypy/doc/how-to-release.rst
@@ -21,8 +21,8 @@
 Release Steps
 ----------------
 
-* at code freeze make a release branch under
-  http://codepeak.net/svn/pypy/release/x.y(.z). IMPORTANT: bump the
+* at code freeze make a release branch using release-x.x.x in mercurial
+  IMPORTANT: bump the
   pypy version number in module/sys/version.py and in
   module/cpyext/include/patchlevel.h, notice that the branch
   will capture the revision number of this change for the release;
@@ -42,18 +42,11 @@
     JIT: windows, linux, os/x
     no JIT: windows, linux, os/x
     sandbox: linux, os/x
-    stackless: windows, linux, os/x
 
 * write release announcement pypy/doc/release-x.y(.z).txt
   the release announcement should contain a direct link to the download page
 * update pypy.org (under extradoc/pypy.org), rebuild and commit
 
-* update http://codespeak.net/pypy/trunk:
-   code0> + chmod -R yourname:users /www/codespeak.net/htdocs/pypy/trunk
-   local> cd ..../pypy/doc && py.test
-   local> cd ..../pypy
-   local> rsync -az doc codespeak.net:/www/codespeak.net/htdocs/pypy/trunk/pypy/
-
 * post announcement on morepypy.blogspot.com
 * send announcements to pypy-dev, python-list,
   python-announce, python-dev ...
diff --git a/pypy/doc/index-of-release-notes.rst b/pypy/doc/index-of-release-notes.rst
--- a/pypy/doc/index-of-release-notes.rst
+++ b/pypy/doc/index-of-release-notes.rst
@@ -16,3 +16,4 @@
    release-1.4.0beta.rst
    release-1.4.1.rst
    release-1.5.0.rst
+   release-1.6.0.rst
diff --git a/pypy/doc/index.rst b/pypy/doc/index.rst
--- a/pypy/doc/index.rst
+++ b/pypy/doc/index.rst
@@ -15,7 +15,7 @@
 
 * `FAQ`_: some frequently asked questions.
 
-* `Release 1.5`_: the latest official release
+* `Release 1.6`_: the latest official release
 
 * `PyPy Blog`_: news and status info about PyPy 
 
@@ -35,7 +35,7 @@
 
   * `Differences between PyPy and CPython`_
   * `What PyPy can do for your objects`_
-  * `Stackless and coroutines`_
+  * `Continulets and greenlets`_
   * `JIT Generation in PyPy`_ 
   * `Sandboxing Python code`_
 
@@ -77,7 +77,7 @@
 .. _`Getting Started`: getting-started.html
 .. _`Papers`: extradoc.html
 .. _`Videos`: video-index.html
-.. _`Release 1.5`: http://pypy.org/download.html
+.. _`Release 1.6`: http://pypy.org/download.html
 .. _`speed.pypy.org`: http://speed.pypy.org
 .. _`RPython toolchain`: translation.html
 .. _`potential project ideas`: project-ideas.html
@@ -122,9 +122,9 @@
 Windows, on top of .NET, and on top of Java.
 To dig into PyPy it is recommended to try out the current
 Mercurial default branch, which is always working or mostly working,
-instead of the latest release, which is `1.5`__.
+instead of the latest release, which is `1.6`__.
 
-.. __: release-1.5.0.html
+.. __: release-1.6.0.html
 
 PyPy is mainly developed on Linux and Mac OS X.  Windows is supported,
 but platform-specific bugs tend to take longer before we notice and fix
@@ -292,8 +292,6 @@
 
 `pypy/translator/jvm/`_            the Java backend
 
-`pypy/translator/stackless/`_      the `Stackless Transform`_
-
 `pypy/translator/tool/`_           helper tools for translation, including the Pygame
                                    `graph viewer`_
 
@@ -318,7 +316,7 @@
 .. _`transparent proxies`: objspace-proxies.html#tproxy
 .. _`Differences between PyPy and CPython`: cpython_differences.html
 .. _`What PyPy can do for your objects`: objspace-proxies.html
-.. _`Stackless and coroutines`: stackless.html
+.. _`Continulets and greenlets`: stackless.html
 .. _StdObjSpace: objspace.html#the-standard-object-space 
 .. _`abstract interpretation`: http://en.wikipedia.org/wiki/Abstract_interpretation
 .. _`rpython`: coding-guide.html#rpython 
@@ -337,7 +335,6 @@
 .. _`low-level type system`: rtyper.html#low-level-type
 .. _`object-oriented type system`: rtyper.html#oo-type
 .. _`garbage collector`: garbage_collection.html
-.. _`Stackless Transform`: translation.html#the-stackless-transform
 .. _`main PyPy-translation scripts`: getting-started-python.html#translating-the-pypy-python-interpreter
 .. _`.NET`: http://www.microsoft.com/net/
 .. _Mono: http://www.mono-project.com/
diff --git a/pypy/doc/jit/pyjitpl5.rst b/pypy/doc/jit/pyjitpl5.rst
--- a/pypy/doc/jit/pyjitpl5.rst
+++ b/pypy/doc/jit/pyjitpl5.rst
@@ -103,7 +103,7 @@
 
 The meta-interpreter starts interpreting the JIT bytecode.  Each operation is
 executed and then recorded in a list of operations, called the trace.
-Operations can have a list of boxes that operate on, arguments.  Some operations
+Operations can have a list of boxes they operate on, arguments.  Some operations
 (like GETFIELD and GETARRAYITEM) also have special objects that describe how
 their arguments are laid out in memory.  All possible operations generated by
 tracing are listed in metainterp/resoperation.py.  When a (interpreter-level)
diff --git a/pypy/doc/project-ideas.rst b/pypy/doc/project-ideas.rst
--- a/pypy/doc/project-ideas.rst
+++ b/pypy/doc/project-ideas.rst
@@ -48,12 +48,6 @@
 
 .. image:: image/jitviewer.png
 
-We would like to add one level to this hierarchy, by showing the generated
-machine code for each jit operation.  The necessary information is already in
-the log file produced by the JIT, so it is "only" a matter of teaching the
-jitviewer to display it.  Ideally, the machine code should be hidden by
-default and viewable on request.
-
 The jitviewer is a web application based on flask and jinja2 (and jQuery on
 the client): if you have great web developing skills and want to help PyPy,
 this is an ideal task to get started, because it does not require any deep
diff --git a/pypy/doc/release-1.6.0.rst b/pypy/doc/release-1.6.0.rst
new file mode 100644
--- /dev/null
+++ b/pypy/doc/release-1.6.0.rst
@@ -0,0 +1,95 @@
+========================
+PyPy 1.6 - kickass panda
+========================
+
+We're pleased to announce the 1.6 release of PyPy. This release brings a lot
+of bugfixes and performance improvements over 1.5, and improves support for
+Windows 32bit and OS X 64bit. This version fully implements Python 2.7.1 and
+has beta level support for loading CPython C extensions.  You can download it
+here:
+
+    http://pypy.org/download.html
+
+What is PyPy?
+=============
+
+PyPy is a very compliant Python interpreter, almost a drop-in replacement for
+CPython 2.7.1. It's fast (`pypy 1.5 and cpython 2.6.2`_ performance comparison)
+due to its integrated tracing JIT compiler.
+
+This release supports x86 machines running Linux 32/64 or Mac OS X.  Windows 32
+is beta (it roughly works but a lot of small issues have not been fixed so
+far).  Windows 64 is not yet supported.
+
+The main topics of this release are speed and stability: on average on
+our benchmark suite, PyPy 1.6 is between **20% and 30%** faster than PyPy 1.5,
+which was already much faster than CPython on our set of benchmarks.
+
+The speed improvements have been made possible by optimizing many of the
+layers which compose PyPy.  In particular, we improved: the Garbage Collector,
+the JIT warmup time, the optimizations performed by the JIT, the quality of
+the generated machine code and the implementation of our Python interpreter.
+
+.. _`pypy 1.5 and cpython 2.6.2`: http://speed.pypy.org
+
+
+Highlights
+==========
+
+* Numerous performance improvements, overall giving considerable speedups:
+
+  - better GC behavior when dealing with very large objects and arrays
+
+  - **fast ctypes:** now calls to ctypes functions are seen and optimized
+    by the JIT, and they are up to 60 times faster than PyPy 1.5 and 10 times
+    faster than CPython
+
+  - improved generators(1): simple generators now are inlined into the caller
+    loop, making performance up to 3.5 times faster than PyPy 1.5.
+
+  - improved generators(2): thanks to other optimizations, even generators
+    that are not inlined are between 10% and 20% faster than PyPy 1.5.
+
+  - faster warmup time for the JIT
+
+  - JIT support for single floats (e.g., for ``array('f')``)
+
+  - optimized dictionaries: the internal representation of dictionaries is now
+    dynamically selected depending on the type of stored objects, resulting in
+    faster code and smaller memory footprint.  For example, dictionaries whose
+    keys are all strings, or all integers. Other dictionaries are also smaller
+    due to bugfixes.
+
+* JitViewer: this is the first official release which includes the JitViewer,
+  a web-based tool which helps you to see which parts of your Python code have
+  been compiled by the JIT, down until the assembler. The `jitviewer`_ 0.1 has
+  already been release and works well with PyPy 1.6.
+
+* The CPython extension module API has been improved and now supports many
+  more extensions. For information on which one are supported, please refer to
+  our `compatibility wiki`_.
+
+* Multibyte encoding support: this was of of the last areas in which we were
+  still behind CPython, but now we fully support them.
+
+* Preliminary support for NumPy: this release includes a preview of a very
+  fast NumPy module integrated with the PyPy JIT.  Unfortunately, this does
+  not mean that you can expect to take an existing NumPy program and run it on
+  PyPy, because the module is still unfinished and supports only some of the
+  numpy API. However, barring some details, what works should be
+  blazingly fast :-)
+
+* Bugfixes: since the 1.5 release we fixed 53 bugs in our `bug tracker`_, not
+  counting the numerous bugs that were found and reported through other
+  channels than the bug tracker.
+
+Cheers,
+
+Hakan Ardo, Carl Friedrich Bolz, Laura Creighton, Antonio Cuni,
+Maciej Fijalkowski, Amaury Forgeot d'Arc, Alex Gaynor,
+Armin Rigo and the PyPy team
+
+.. _`jitviewer`: http://morepypy.blogspot.com/2011/08/visualization-of-jitted-code.html
+.. _`bug tracker`: https://bugs.pypy.org
+.. _`compatibility wiki`: https://bitbucket.org/pypy/compatibility/wiki/Home
+
diff --git a/pypy/doc/rlib.rst b/pypy/doc/rlib.rst
--- a/pypy/doc/rlib.rst
+++ b/pypy/doc/rlib.rst
@@ -134,69 +134,6 @@
 a hierarchy of Address classes, in a typical static-OO-programming style.
 
 
-``rstack``
-==========
-
-The `pypy/rlib/rstack.py`_ module allows an RPython program to control its own execution stack.
-This is only useful if the program is translated using stackless. An old
-description of the exposed functions is below.
-
-We introduce an RPython type ``frame_stack_top`` and a built-in function
-``yield_current_frame_to_caller()`` that work as follows (see example below):
-
-* The built-in function ``yield_current_frame_to_caller()`` causes the current
-  function's state to be captured in a new ``frame_stack_top`` object that is
-  returned to the parent.  Only one frame, the current one, is captured this
-  way.  The current frame is suspended and the caller continues to run.  Note
-  that the caller is only resumed once: when
-  ``yield_current_frame_to_caller()`` is called.  See below.
-
-* A ``frame_stack_top`` object can be jumped to by calling its ``switch()``
-  method with no argument.
-
-* ``yield_current_frame_to_caller()`` and ``switch()`` themselves return a new
-  ``frame_stack_top`` object: the freshly captured state of the caller of the
-  source ``switch()`` that was just executed, or None in the case described
-  below.
-
-* the function that called ``yield_current_frame_to_caller()`` also has a
-  normal return statement, like all functions.  This statement must return
-  another ``frame_stack_top`` object.  The latter is *not* returned to the
-  original caller; there is no way to return several times to the caller.
-  Instead, it designates the place to which the execution must jump, as if by
-  a ``switch()``.  The place to which we jump this way will see a None as the
-  source frame stack top.
-
-* every frame stack top must be resumed once and only once.  Not resuming
-  it at all causes a leak.  Resuming it several times causes a crash.
-
-* a function that called ``yield_current_frame_to_caller()`` should not raise.
-  It would have no implicit parent frame to propagate the exception to.  That
-  would be a crashingly bad idea.
-
-The following example would print the numbers from 1 to 7 in order::
-
-    def g():
-        print 2
-        frametop_before_5 = yield_current_frame_to_caller()
-        print 4
-        frametop_before_7 = frametop_before_5.switch()
-        print 6
-        return frametop_before_7
-
-    def f():
-        print 1
-        frametop_before_4 = g()
-        print 3
-        frametop_before_6 = frametop_before_4.switch()
-        print 5
-        frametop_after_return = frametop_before_6.switch()
-        print 7
-        assert frametop_after_return is None
-
-    f()
-
-
 ``streamio``
 ============
 
diff --git a/pypy/doc/stackless.rst b/pypy/doc/stackless.rst
--- a/pypy/doc/stackless.rst
+++ b/pypy/doc/stackless.rst
@@ -8,446 +8,299 @@
 ================
 
 PyPy can expose to its user language features similar to the ones
-present in `Stackless Python`_: **no recursion depth limit**, and the
-ability to write code in a **massively concurrent style**.  It actually
-exposes three different paradigms to choose from:
+present in `Stackless Python`_: the ability to write code in a
+**massively concurrent style**.  (It does not (any more) offer the
+ability to run with no `recursion depth limit`_, but the same effect
+can be achieved indirectly.)
 
-* `Tasklets and channels`_;
+This feature is based on a custom primitive called a continulet_.
+Continulets can be directly used by application code, or it is possible
+to write (entirely at app-level) more user-friendly interfaces.
 
-* Greenlets_;
+Currently PyPy implements greenlets_ on top of continulets.  It would be
+easy to implement tasklets and channels as well, emulating the model
+of `Stackless Python`_.
 
-* Plain coroutines_.
+Continulets are extremely light-weight, which means that PyPy should be
+able to handle programs containing large amounts of them.  However, due
+to an implementation restriction, a PyPy compiled with
+``--gcrootfinder=shadowstack`` consumes at least one page of physical
+memory (4KB) per live continulet, and half a megabyte of virtual memory
+on 32-bit or a complete megabyte on 64-bit.  Moreover, the feature is
+only available (so far) on x86 and x86-64 CPUs; for other CPUs you need
+to add a short page of custom assembler to
+`pypy/translator/c/src/stacklet/`_.
 
-All of them are extremely light-weight, which means that PyPy should be
-able to handle programs containing large amounts of coroutines, tasklets
-and greenlets.
 
+Theory
+======
 
-Requirements
-++++++++++++++++
+The fundamental idea is that, at any point in time, the program happens
+to run one stack of frames (or one per thread, in case of
+multi-threading).  To see the stack, start at the top frame and follow
+the chain of ``f_back`` until you reach the bottom frame.  From the
+point of view of one of these frames, it has a ``f_back`` pointing to
+another frame (unless it is the bottom frame), and it is itself being
+pointed to by another frame (unless it is the top frame).
 
-If you are running py.py on top of CPython, then you need to enable
-the _stackless module by running it as follows::
+The theory behind continulets is to literally take the previous sentence
+as definition of "an O.K. situation".  The trick is that there are
+O.K. situations that are more complex than just one stack: you will
+always have one stack, but you can also have in addition one or more
+detached *cycles* of frames, such that by following the ``f_back`` chain
+you run in a circle.  But note that these cycles are indeed completely
+detached: the top frame (the currently running one) is always the one
+which is not the ``f_back`` of anybody else, and it is always the top of
+a stack that ends with the bottom frame, never a part of these extra
+cycles.
 
-    py.py --withmod-_stackless
+How do you create such cycles?  The fundamental operation to do so is to
+take two frames and *permute* their ``f_back`` --- i.e. exchange them.
+You can permute any two ``f_back`` without breaking the rule of "an O.K.
+situation".  Say for example that ``f`` is some frame halfway down the
+stack, and you permute its ``f_back`` with the ``f_back`` of the top
+frame.  Then you have removed from the normal stack all intermediate
+frames, and turned them into one stand-alone cycle.  By doing the same
+permutation again you restore the original situation.
 
-This is implemented internally using greenlets, so it only works on a
-platform where `greenlets`_ are supported.  A few features do
-not work this way, though, and really require a translated
-``pypy-c``.
+In practice, in PyPy, you cannot change the ``f_back`` of an abitrary
+frame, but only of frames stored in ``continulets``.
 
-To obtain a translated version of ``pypy-c`` that includes Stackless
-support, run translate.py as follows::
-
-    cd pypy/translator/goal
-    python translate.py --stackless
+Continulets are internally implemented using stacklets.  Stacklets are a
+bit more primitive (they are really one-shot continuations), but that
+idea only works in C, not in Python.  The basic idea of continulets is
+to have at any point in time a complete valid stack; this is important
+e.g. to correctly propagate exceptions (and it seems to give meaningful
+tracebacks too).
 
 
 Application level interface
 =============================
 
-A stackless PyPy contains a module called ``stackless``.  The interface
-exposed by this module have not been refined much, so it should be
-considered in-flux (as of 2007).
 
-So far, PyPy does not provide support for ``stackless`` in a threaded
-environment.  This limitation is not fundamental, as previous experience
-has shown, so supporting this would probably be reasonably easy.
+.. _continulet:
 
-An interesting point is that the same ``stackless`` module can provide
-a number of different concurrency paradigms at the same time.  From a
-theoretical point of view, none of above-mentioned existing three
-paradigms considered on its own is new: two of them are from previous
-Python work, and the third one is a variant of the classical coroutine.
-The new part is that the PyPy implementation manages to provide all of
-them and let the user implement more.  Moreover - and this might be an
-important theoretical contribution of this work - we manage to provide
-these concurrency concepts in a "composable" way.  In other words, it
-is possible to naturally mix in a single application multiple
-concurrency paradigms, and multiple unrelated usages of the same
-paradigm.  This is discussed in the Composability_ section below.
+Continulets
++++++++++++
 
+A translated PyPy contains by default a module called ``_continuation``
+exporting the type ``continulet``.  A ``continulet`` object from this
+module is a container that stores a "one-shot continuation".  It plays
+the role of an extra frame you can insert in the stack, and whose
+``f_back`` can be changed.
 
-Infinite recursion
-++++++++++++++++++
+To make a continulet object, call ``continulet()`` with a callable and
+optional extra arguments.
 
-Any stackless PyPy executable natively supports recursion that is only
-limited by the available memory.  As in normal Python, though, there is
-an initial recursion limit (which is 5000 in all pypy-c's, and 1000 in
-CPython).  It can be changed with ``sys.setrecursionlimit()``.  With a
-stackless PyPy, any value is acceptable - use ``sys.maxint`` for
-unlimited.
+Later, the first time you ``switch()`` to the continulet, the callable
+is invoked with the same continulet object as the extra first argument.
+At that point, the one-shot continuation stored in the continulet points
+to the caller of ``switch()``.  In other words you have a perfectly
+normal-looking stack of frames.  But when ``switch()`` is called again,
+this stored one-shot continuation is exchanged with the current one; it
+means that the caller of ``switch()`` is suspended with its continuation
+stored in the container, and the old continuation from the continulet
+object is resumed.
 
-In some cases, you can write Python code that causes interpreter-level
-infinite recursion -- i.e. infinite recursion without going via
-application-level function calls.  It is possible to limit that too,
-with ``_stackless.set_stack_depth_limit()``, or to unlimit it completely
-by setting it to ``sys.maxint``.
+The most primitive API is actually 'permute()', which just permutes the
+one-shot continuation stored in two (or more) continulets.
 
+In more details:
 
-Coroutines
-++++++++++
+* ``continulet(callable, *args, **kwds)``: make a new continulet.
+  Like a generator, this only creates it; the ``callable`` is only
+  actually called the first time it is switched to.  It will be
+  called as follows::
 
-A Coroutine is similar to a very small thread, with no preemptive scheduling.
-Within a family of coroutines, the flow of execution is explicitly
-transferred from one to another by the programmer.  When execution is
-transferred to a coroutine, it begins to execute some Python code.  When
-it transfers execution away from itself it is temporarily suspended, and
-when execution returns to it it resumes its execution from the
-point where it was suspended.  Conceptually, only one coroutine is
-actively running at any given time (but see Composability_ below).
+      callable(cont, *args, **kwds)
 
-The ``stackless.coroutine`` class is instantiated with no argument.
-It provides the following methods and attributes:
+  where ``cont`` is the same continulet object.
 
-* ``stackless.coroutine.getcurrent()``
+  Note that it is actually ``cont.__init__()`` that binds
+  the continulet.  It is also possible to create a not-bound-yet
+  continulet by calling explicitly ``continulet.__new__()``, and
+  only bind it later by calling explicitly ``cont.__init__()``.
 
-    Static method returning the currently running coroutine.  There is a
-    so-called "main" coroutine object that represents the "outer"
-    execution context, where your main program started and where it runs
-    as long as it does not switch to another coroutine.
+* ``cont.switch(value=None, to=None)``: start the continulet if
+  it was not started yet.  Otherwise, store the current continuation
+  in ``cont``, and activate the target continuation, which is the
+  one that was previously stored in ``cont``.  Note that the target
+  continuation was itself previously suspended by another call to
+  ``switch()``; this older ``switch()`` will now appear to return.
+  The ``value`` argument is any object that is carried to the target
+  and returned by the target's ``switch()``.
 
-* ``coro.bind(callable, *args, **kwds)``
+  If ``to`` is given, it must be another continulet object.  In
+  that case, performs a "double switch": it switches as described
+  above to ``cont``, and then immediately switches again to ``to``.
+  This is different from switching directly to ``to``: the current
+  continuation gets stored in ``cont``, the old continuation from
+  ``cont`` gets stored in ``to``, and only then we resume the
+  execution from the old continuation out of ``to``.
 
-    Bind the coroutine so that it will execute ``callable(*args,
-    **kwds)``.  The call is not performed immediately, but only the
-    first time we call the ``coro.switch()`` method.  A coroutine must
-    be bound before it is switched to.  When the coroutine finishes
-    (because the call to the callable returns), the coroutine exits and
-    implicitly switches back to another coroutine (its "parent"); after
-    this point, it is possible to bind it again and switch to it again.
-    (Which coroutine is the parent of which is not documented, as it is
-    likely to change when the interface is refined.)
+* ``cont.throw(type, value=None, tb=None, to=None)``: similar to
+  ``switch()``, except that immediately after the switch is done, raise
+  the given exception in the target.
 
-* ``coro.switch()``
+* ``cont.is_pending()``: return True if the continulet is pending.
+  This is False when it is not initialized (because we called
+  ``__new__`` and not ``__init__``) or when it is finished (because
+  the ``callable()`` returned).  When it is False, the continulet
+  object is empty and cannot be ``switch()``-ed to.
 
-    Suspend the current (caller) coroutine, and resume execution in the
-    target coroutine ``coro``.
+* ``permute(*continulets)``: a global function that permutes the
+  continuations stored in the given continulets arguments.  Mostly
+  theoretical.  In practice, using ``cont.switch()`` is easier and
+  more efficient than using ``permute()``; the latter does not on
+  its own change the currently running frame.
 
-* ``coro.kill()``
 
-    Kill ``coro`` by sending a CoroutineExit exception and switching
-    execution immediately to it. This exception can be caught in the 
-    coroutine itself and can be raised from any call to ``coro.switch()``. 
-    This exception isn't propagated to the parent coroutine.
+Genlets
++++++++
 
-* ``coro.throw(type, value)``
+The ``_continuation`` module also exposes the ``generator`` decorator::
 
-    Insert an exception in ``coro`` an resume switches execution
-    immediately to it. In the coroutine itself, this exception
-    will come from any call to ``coro.switch()`` and can be caught. If the
-    exception isn't caught, it will be propagated to the parent coroutine.
+    @generator
+    def f(cont, a, b):
+        cont.switch(a + b)
+        cont.switch(a + b + 1)
 
-When a coroutine is garbage-collected, it gets the ``.kill()`` method sent to
-it. This happens at the point the next ``.switch`` method is called, so the
-target coroutine of this call will be executed only after the ``.kill`` has
-finished.
+    for i in f(10, 20):
+        print i
 
-Example
-~~~~~~~
+This example prints 30 and 31.  The only advantage over using regular
+generators is that the generator itself is not limited to ``yield``
+statements that must all occur syntactically in the same function.
+Instead, we can pass around ``cont``, e.g. to nested sub-functions, and
+call ``cont.switch(x)`` from there.
 
-Here is a classical producer/consumer example: an algorithm computes a
-sequence of values, while another consumes them.  For our purposes we
-assume that the producer can generate several values at once, and the
-consumer can process up to 3 values in a batch - it can also process
-batches with fewer than 3 values without waiting for the producer (which
-would be messy to express with a classical Python generator). ::
+The ``generator`` decorator can also be applied to methods::
 
-    def producer(lst):
-        while True:
-            ...compute some more values...
-            lst.extend(new_values)
-            coro_consumer.switch()
-
-    def consumer(lst):
-        while True:
-            # First ask the producer for more values if needed
-            while len(lst) == 0:
-                coro_producer.switch()
-            # Process the available values in a batch, but at most 3
-            batch = lst[:3]
-            del lst[:3]
-            ...process batch...
-
-    # Initialize two coroutines with a shared list as argument
-    exchangelst = []
-    coro_producer = coroutine()
-    coro_producer.bind(producer, exchangelst)
-    coro_consumer = coroutine()
-    coro_consumer.bind(consumer, exchangelst)
-
-    # Start running the consumer coroutine
-    coro_consumer.switch()
-
-
-Tasklets and channels
-+++++++++++++++++++++
-
-The ``stackless`` module also provides an interface that is roughly
-compatible with the interface of the ``stackless`` module in `Stackless
-Python`_: it contains ``stackless.tasklet`` and ``stackless.channel``
-classes.  Tasklets are also similar to microthreads, but (like coroutines)
-they don't actually run in parallel with other microthreads; instead,
-they synchronize and exchange data with each other over Channels, and
-these exchanges determine which Tasklet runs next.
-
-For usage reference, see the documentation on the `Stackless Python`_
-website.
-
-Note that Tasklets and Channels are implemented at application-level in
-`lib_pypy/stackless.py`_ on top of coroutines_.  You can refer to this
-module for more details and API documentation.
-
-The stackless.py code tries to resemble the stackless C code as much
-as possible. This makes the code somewhat unpythonic.
-
-Bird's eye view of tasklets and channels
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Tasklets are a bit like threads: they encapsulate a function in such a way that
-they can be suspended/restarted any time. Unlike threads, they won't
-run concurrently, but must be cooperative. When using stackless
-features, it is vitally important that no action is performed that blocks
-everything else.  In particular, blocking input/output should be centralized
-to a single tasklet.
-
-Communication between tasklets is done via channels. 
-There are three ways for a tasklet to give up control:
-
-1. call ``stackless.schedule()``
-2. send something over a channel
-3. receive something from a channel
-
-A (live) tasklet can either be running, waiting to get scheduled, or be
-blocked by a channel.
-
-Scheduling is done in strictly round-robin manner. A blocked tasklet
-is removed from the scheduling queue and will be reinserted when it
-becomes unblocked.
-
-Example
-~~~~~~~
-
-Here is a many-producers many-consumers example, where any consumer can
-process the result of any producer.  For this situation we set up a
-single channel where all producer send, and on which all consumers
-wait::
-
-    def producer(chan):
-        while True:
-            chan.send(...next value...)
-
-    def consumer(chan):
-        while True:
-            x = chan.receive()
-            ...do something with x...
-
-    # Set up the N producer and M consumer tasklets
-    common_channel = stackless.channel()
-    for i in range(N):
-        stackless.tasklet(producer, common_channel)()
-    for i in range(M):
-        stackless.tasklet(consumer, common_channel)()
-
-    # Run it all
-    stackless.run()
-
-Each item sent over the channel is received by one of the waiting
-consumers; which one is not specified.  The producers block until their
-item is consumed: the channel is not a queue, but rather a meeting point
-which causes tasklets to block until both a consumer and a producer are
-ready.  In practice, the reason for having several consumers receiving
-on a single channel is that some of the consumers can be busy in other
-ways part of the time.  For example, each consumer might receive a
-database request, process it, and send the result to a further channel
-before it asks for the next request.  In this situation, further
-requests can still be received by other consumers.
+    class X:
+        @generator
+        def f(self, cont, a, b):
+            ...
 
 
 Greenlets
 +++++++++
 
-A Greenlet is a kind of primitive Tasklet with a lower-level interface
-and with exact control over the execution order.  Greenlets are similar
-to Coroutines, with a slightly different interface: greenlets put more
-emphasis on a tree structure.  The various greenlets of a program form a
-precise tree, which fully determines their order of execution.
+Greenlets are implemented on top of continulets in `lib_pypy/greenlet.py`_.
+See the official `documentation of the greenlets`_.
 
-For usage reference, see the `documentation of the greenlets`_.
-The PyPy interface is identical.  You should use ``greenlet.greenlet``
-instead of ``stackless.greenlet`` directly, because the greenlet library
-can give you the latter when you ask for the former on top of PyPy.
+Note that unlike the CPython greenlets, this version does not suffer
+from GC issues: if the program "forgets" an unfinished greenlet, it will
+always be collected at the next garbage collection.
 
-PyPy's greenlets do not suffer from the cyclic GC limitation that the
-CPython greenlets have: greenlets referencing each other via local
-variables tend to leak on top of CPython (where it is mostly impossible
-to do the right thing).  It works correctly on top of PyPy.
 
+Unimplemented features
+++++++++++++++++++++++
 
-Coroutine Pickling
-++++++++++++++++++
+The following features (present in some past Stackless version of PyPy)
+are for the time being not supported any more:
 
-Coroutines and tasklets can be pickled and unpickled, i.e. serialized to
-a string of bytes for the purpose of storage or transmission.  This
-allows "live" coroutines or tasklets to be made persistent, moved to
-other machines, or cloned in any way.  The standard ``pickle`` module
-works with coroutines and tasklets (at least in a translated ``pypy-c``;
-unpickling live coroutines or tasklets cannot be easily implemented on
-top of CPython).
+* Tasklets and channels (currently ``stackless.py`` seems to import,
+  but you have tasklets on top of coroutines on top of greenlets on
+  top of continulets on top of stacklets, and it's probably not too
+  hard to cut two of these levels by adapting ``stackless.py`` to
+  use directly continulets)
 
-To be able to achieve this result, we have to consider many objects that
-are not normally pickleable in CPython.  Here again, the `Stackless
-Python`_ implementation has paved the way, and we follow the same
-general design decisions: simple internal objects like bound method
-objects and various kinds of iterators are supported; frame objects can
-be fully pickled and unpickled
-(by serializing a reference to the bytecode they are
-running in addition to all the local variables).  References to globals
-and modules are pickled by name, similarly to references to functions
-and classes in the traditional CPython ``pickle``.
+* Coroutines (could be rewritten at app-level)
 
-The "magic" part of this process is the implementation of the unpickling
-of a chain of frames.  The Python interpreter of PyPy uses
-interpreter-level recursion to represent application-level calls.  The
-reason for this is that it tremendously simplifies the implementation of
-the interpreter itself.  Indeed, in Python, almost any operation can
-potentially result in a non-tail-recursive call to another Python
-function.  This makes writing a non-recursive interpreter extremely
-tedious; instead, we rely on lower-level transformations during the
-translation process to control this recursion.  This is the `Stackless
-Transform`_, which is at the heart of PyPy's support for stackless-style
-concurrency.
+* Pickling and unpickling continulets (*)
 
-At any point in time, a chain of Python-level frames corresponds to a
-chain of interpreter-level frames (e.g. C frames in pypy-c), where each
-single Python-level frame corresponds to one or a few interpreter-level
-frames - depending on the length of the interpreter-level call chain
-from one bytecode evaluation loop to the next (recursively invoked) one.
+* Continuing execution of a continulet in a different thread (*)
 
-This means that it is not sufficient to simply create a chain of Python
-frame objects in the heap of a process before we can resume execution of
-these newly built frames.  We must recreate a corresponding chain of
-interpreter-level frames.  To this end, we have inserted a few *named
-resume points* (see 3.2.4, in `D07.1 Massive Parallelism and Translation Aspects`_) in the Python interpreter of PyPy.  This is the
-motivation for implementing the interpreter-level primitives
-``resume_state_create()`` and ``resume_state_invoke()``, the powerful
-interface that allows an RPython program to artificially rebuild a chain
-of calls in a reflective way, completely from scratch, and jump to it.
+* Automatic unlimited stack (must be emulated__ so far)
 
-.. _`D07.1 Massive Parallelism and Translation Aspects`: http://codespeak.net/pypy/extradoc/eu-report/D07.1_Massive_Parallelism_and_Translation_Aspects-2007-02-28.pdf
+* Support for other CPUs than x86 and x86-64
 
-Example
-~~~~~~~
+* The app-level ``f_back`` field of frames crossing continulet boundaries
+  is None for now, unlike what I explain in the theoretical overview
+  above.  It mostly means that in a ``pdb.set_trace()`` you cannot go
+  ``up`` past countinulet boundaries.  This could be fixed.
 
-(See `demo/pickle_coroutine.py`_ for the complete source of this demo.)
+.. __: `recursion depth limit`_
 
-Consider a program which contains a part performing a long-running
-computation::
+(*) Pickling, as well as changing threads, could be implemented by using
+a "soft" stack switching mode again.  We would get either "hard" or
+"soft" switches, similarly to Stackless Python 3rd version: you get a
+"hard" switch (like now) when the C stack contains non-trivial C frames
+to save, and a "soft" switch (like previously) when it contains only
+simple calls from Python to Python.  Soft-switched continulets would
+also consume a bit less RAM, and the switch might be a bit faster too
+(unsure about that; what is the Stackless Python experience?).
 
-    def ackermann(x, y):
-        if x == 0:
-            return y + 1
-        if y == 0:
-            return ackermann(x - 1, 1)
-        return ackermann(x - 1, ackermann(x, y - 1))
 
-By using pickling, we can save the state of the computation while it is
-running, for the purpose of restoring it later and continuing the
-computation at another time or on a different machine.  However,
-pickling does not produce a whole-program dump: it can only pickle
-individual coroutines.  This means that the computation should be
-started in its own coroutine::
+Recursion depth limit
++++++++++++++++++++++
 
-    # Make a coroutine that will run 'ackermann(3, 8)'
-    coro = coroutine()
-    coro.bind(ackermann, 3, 8)
+You can use continulets to emulate the infinite recursion depth present
+in Stackless Python and in stackless-enabled older versions of PyPy.
 
-    # Now start running the coroutine
-    result = coro.switch()
+The trick is to start a continulet "early", i.e. when the recursion
+depth is very low, and switch to it "later", i.e. when the recursion
+depth is high.  Example::
 
-The coroutine itself must switch back to the main program when it needs
-to be interrupted (we can only pickle suspended coroutines).  Due to
-current limitations this requires an explicit check in the
-``ackermann()`` function::
+    from _continuation import continulet
 
-    def ackermann(x, y):
-        if interrupt_flag:      # test a global flag
-            main.switch()       # and switch back to 'main' if it is set
-        if x == 0:
-            return y + 1
-        if y == 0:
-            return ackermann(x - 1, 1)
-        return ackermann(x - 1, ackermann(x, y - 1))
+    def invoke(_, callable, arg):
+        return callable(arg)
 
-The global ``interrupt_flag`` would be set for example by a timeout, or
-by a signal handler reacting to Ctrl-C, etc.  It causes the coroutine to
-transfer control back to the main program.  The execution comes back
-just after the line ``coro.switch()``, where we can pickle the coroutine
-if necessary::
+    def bootstrap(c):
+        # this loop runs forever, at a very low recursion depth
+        callable, arg = c.switch()
+        while True:
+            # start a new continulet from here, and switch to
+            # it using an "exchange", i.e. a switch with to=.
+            to = continulet(invoke, callable, arg)
+            callable, arg = c.switch(to=to)
 
-    if not coro.is_alive:
-        print "finished; the result is:", result
-    else:
-        # save the state of the suspended coroutine
-        f = open('demo.pickle', 'w')
-        pickle.dump(coro, f)
-        f.close()
+    c = continulet(bootstrap)
+    c.switch()
 
-The process can then stop.  At any later time, or on another machine,
-we can reload the file and restart the coroutine with::
 
-    f = open('demo.pickle', 'r')
-    coro = pickle.load(f)
-    f.close()
-    result = coro.switch()
+    def recursive(n):
+        if n == 0:
+            return ("ok", n)
+        if n % 200 == 0:
+            prev = c.switch((recursive, n - 1))
+        else:
+            prev = recursive(n - 1)
+        return (prev[0], prev[1] + 1)
 
-Limitations
-~~~~~~~~~~~
+    print recursive(999999)     # prints ('ok', 999999)
 
-Coroutine pickling is subject to some limitations.  First of all, it is
-not a whole-program "memory dump".  It means that only the "local" state
-of a coroutine is saved.  The local state is defined to include the
-chain of calls and the local variables, but not for example the value of
-any global variable.
+Note that if you press Ctrl-C while running this example, the traceback
+will be built with *all* recursive() calls so far, even if this is more
+than the number that can possibly fit in the C stack.  These frames are
+"overlapping" each other in the sense of the C stack; more precisely,
+they are copied out of and into the C stack as needed.
 
-As in normal Python, the pickle will not include any function object's
-code, any class definition, etc., but only references to functions and
-classes.  Unlike normal Python, the pickle contains frames.  A pickled
-frame stores a bytecode index, representing the current execution
-position.  This means that the user program cannot be modified *at all*
-between pickling and unpickling!
+(The example above also makes use of the following general "guideline"
+to help newcomers write continulets: in ``bootstrap(c)``, only call
+methods on ``c``, not on another continulet object.  That's why we wrote
+``c.switch(to=to)`` and not ``to.switch()``, which would mess up the
+state.  This is however just a guideline; in general we would recommend
+to use other interfaces like genlets and greenlets.)
 
-On the other hand, the pickled data is fairly independent from the
-platform and from the PyPy version.
 
-Pickling/unpickling fails if the coroutine is suspended in a state that
-involves Python frames which were *indirectly* called.  To define this
-more precisely, a Python function can issue a regular function or method
-call to invoke another Python function - this is a *direct* call and can
-be pickled and unpickled.  But there are many ways to invoke a Python
-function indirectly.  For example, most operators can invoke a special
-method ``__xyz__()`` on a class, various built-in functions can call
-back Python functions, signals can invoke signal handlers, and so on.
-These cases are not supported yet.
-
-
-Composability
-+++++++++++++
+Theory of composability
++++++++++++++++++++++++
 
 Although the concept of coroutines is far from new, they have not been
 generally integrated into mainstream languages, or only in limited form
 (like generators in Python and iterators in C#).  We can argue that a
 possible reason for that is that they do not scale well when a program's
 complexity increases: they look attractive in small examples, but the
-models that require explicit switching, by naming the target coroutine,
-do not compose naturally.  This means that a program that uses
-coroutines for two unrelated purposes may run into conflicts caused by
-unexpected interactions.
+models that require explicit switching, for example by naming the target
+coroutine, do not compose naturally.  This means that a program that
+uses coroutines for two unrelated purposes may run into conflicts caused
+by unexpected interactions.
 
 To illustrate the problem, consider the following example (simplified
-code; see the full source in
-`pypy/module/_stackless/test/test_composable_coroutine.py`_).  First, a
-simple usage of coroutine::
+code using a theorical ``coroutine`` class).  First, a simple usage of
+coroutine::
 
     main_coro = coroutine.getcurrent()    # the main (outer) coroutine
     data = []
@@ -530,74 +383,35 @@
 main coroutine, which confuses the ``generator_iterator.next()`` method
 (it gets resumed, but not as a result of a call to ``Yield()``).
 
-As part of trying to combine multiple different paradigms into a single
-application-level module, we have built a way to solve this problem.
-The idea is to avoid the notion of a single, global "main" coroutine (or
-a single main greenlet, or a single main tasklet).  Instead, each
-conceptually separated user of one of these concurrency interfaces can
-create its own "view" on what the main coroutine/greenlet/tasklet is,
-which other coroutine/greenlet/tasklets there are, and which of these is
-the currently running one.  Each "view" is orthogonal to the others.  In
-particular, each view has one (and exactly one) "current"
-coroutine/greenlet/tasklet at any point in time.  When the user switches
-to a coroutine/greenlet/tasklet, it implicitly means that he wants to
-switch away from the current coroutine/greenlet/tasklet *that belongs to
-the same view as the target*.
+Thus the notion of coroutine is *not composable*.  By opposition, the
+primitive notion of continulets is composable: if you build two
+different interfaces on top of it, or have a program that uses twice the
+same interface in two parts, then assuming that both parts independently
+work, the composition of the two parts still works.
 
-The precise application-level interface has not been fixed yet; so far,
-"views" in the above sense are objects of the type
-``stackless.usercostate``.  The above two examples can be rewritten in
-the following way::
+A full proof of that claim would require careful definitions, but let us
+just claim that this fact is true because of the following observation:
+the API of continulets is such that, when doing a ``switch()``, it
+requires the program to have some continulet to explicitly operate on.
+It shuffles the current continuation with the continuation stored in
+that continulet, but has no effect outside.  So if a part of a program
+has a continulet object, and does not expose it as a global, then the
+rest of the program cannot accidentally influence the continuation
+stored in that continulet object.
 
-    producer_view = stackless.usercostate()   # a local view
-    main_coro = producer_view.getcurrent()    # the main (outer) coroutine
-    ...
-    producer_coro = producer_view.newcoroutine()
-    ...
-
-and::
-
-    generators_view = stackless.usercostate()
-
-    def generator(f):
-        def wrappedfunc(*args, **kwds):
-            g = generators_view.newcoroutine(generator_iterator)
-            ...
-
-            ...generators_view.getcurrent()...
-
-Then the composition ``grab_values()`` works as expected, because the
-two views are independent.  The coroutine captured as ``self.caller`` in
-the ``generator_iterator.next()`` method is the main coroutine of the
-``generators_view``.  It is no longer the same object as the main
-coroutine of the ``producer_view``, so when ``data_producer()`` issues
-the following command::
-
-    main_coro.switch()
-
-the control flow cannot accidentally jump back to
-``generator_iterator.next()``.  In other words, from the point of view
-of ``producer_view``, the function ``grab_next_value()`` always runs in
-its main coroutine ``main_coro`` and the function ``data_producer`` in
-its coroutine ``producer_coro``.  This is the case independently of
-which ``generators_view``-based coroutine is the current one when
-``grab_next_value()`` is called.
-
-Only code that has explicit access to the ``producer_view`` or its
-coroutine objects can perform switches that are relevant for the
-generator code.  If the view object and the coroutine objects that share
-this view are all properly encapsulated inside the generator logic, no
-external code can accidentally temper with the expected control flow any
-longer.
-
-In conclusion: we will probably change the app-level interface of PyPy's
-stackless module in the future to not expose coroutines and greenlets at
-all, but only views.  They are not much more difficult to use, and they
-scale automatically to larger programs.
+In other words, if we regard the continulet object as being essentially
+a modifiable ``f_back``, then it is just a link between the frame of
+``callable()`` and the parent frame --- and it cannot be arbitrarily
+changed by unrelated code, as long as they don't explicitly manipulate
+the continulet object.  Typically, both the frame of ``callable()``
+(commonly a local function) and its parent frame (which is the frame
+that switched to it) belong to the same class or module; so from that
+point of view the continulet is a purely local link between two local
+frames.  It doesn't make sense to have a concept that allows this link
+to be manipulated from outside.
 
 
 .. _`Stackless Python`: http://www.stackless.com
 .. _`documentation of the greenlets`: http://packages.python.org/greenlet/
-.. _`Stackless Transform`: translation.html#the-stackless-transform
 
 .. include:: _ref.txt
diff --git a/pypy/doc/translation.rst b/pypy/doc/translation.rst
--- a/pypy/doc/translation.rst
+++ b/pypy/doc/translation.rst
@@ -552,14 +552,15 @@
 
 The stackless transform converts functions into a form that knows how
 to save the execution point and active variables into a heap structure
-and resume execution at that point.  This is used to implement
+and resume execution at that point.  This was used to implement
 coroutines as an RPython-level feature, which in turn are used to
-implement `coroutines, greenlets and tasklets`_ as an application
+implement coroutines, greenlets and tasklets as an application
 level feature for the Standard Interpreter.
 
-Enable the stackless transformation with :config:`translation.stackless`.
+The stackless transformation has been deprecated and is no longer
+available in trunk.  It has been replaced with continulets_.
 
-.. _`coroutines, greenlets and tasklets`: stackless.html
+.. _continulets: stackless.html
 
 .. _`preparing the graphs for source generation`:
 
diff --git a/pypy/interpreter/baseobjspace.py b/pypy/interpreter/baseobjspace.py
--- a/pypy/interpreter/baseobjspace.py
+++ b/pypy/interpreter/baseobjspace.py
@@ -111,6 +111,9 @@
     def setslotvalue(self, index, w_val):
         raise NotImplementedError
 
+    def delslotvalue(self, index):
+        raise NotImplementedError
+
     def descr_call_mismatch(self, space, opname, RequiredClass, args):
         if RequiredClass is None:
             classname = '?'
@@ -623,9 +626,9 @@
             self.default_compiler = compiler
             return compiler
 
-    def createframe(self, code, w_globals, closure=None):
+    def createframe(self, code, w_globals, outer_func=None):
         "Create an empty PyFrame suitable for this code object."
-        return self.FrameClass(self, code, w_globals, closure)
+        return self.FrameClass(self, code, w_globals, outer_func)
 
     def allocate_lock(self):
         """Return an interp-level Lock object if threads are enabled,
diff --git a/pypy/interpreter/function.py b/pypy/interpreter/function.py
--- a/pypy/interpreter/function.py
+++ b/pypy/interpreter/function.py
@@ -30,7 +30,7 @@
     can_change_code = True
     _immutable_fields_ = ['code?',
                           'w_func_globals?',
-                          'closure?',
+                          'closure?[*]',
                           'defs_w?[*]',
                           'name?']
 
@@ -96,7 +96,7 @@
             assert isinstance(code, PyCode)
             if nargs < 5:
                 new_frame = self.space.createframe(code, self.w_func_globals,
-                                                   self.closure)
+                                                   self)
                 for i in funccallunrolling:
                     if i < nargs:
                         new_frame.locals_stack_w[i] = args_w[i]
@@ -156,7 +156,7 @@
     def _flat_pycall(self, code, nargs, frame):
         # code is a PyCode
         new_frame = self.space.createframe(code, self.w_func_globals,
-                                                   self.closure)
+                                                   self)
         for i in xrange(nargs):
             w_arg = frame.peekvalue(nargs-1-i)
             new_frame.locals_stack_w[i] = w_arg
@@ -167,7 +167,7 @@
     def _flat_pycall_defaults(self, code, nargs, frame, defs_to_load):
         # code is a PyCode
         new_frame = self.space.createframe(code, self.w_func_globals,
-                                                   self.closure)
+                                                   self)
         for i in xrange(nargs):
             w_arg = frame.peekvalue(nargs-1-i)
             new_frame.locals_stack_w[i] = w_arg
diff --git a/pypy/interpreter/gateway.py b/pypy/interpreter/gateway.py
--- a/pypy/interpreter/gateway.py
+++ b/pypy/interpreter/gateway.py
@@ -64,7 +64,7 @@
                 self.visit_self(el[1], *args)
             else:
                 self.visit_function(el, *args)
-        else:
+        elif isinstance(el, type):
             for typ in self.bases_order:
                 if issubclass(el, typ):
                     visit = getattr(self, "visit__%s" % (typ.__name__,))
@@ -73,6 +73,8 @@
             else:
                 raise Exception("%s: no match for unwrap_spec element %s" % (
                     self.__class__.__name__, el))
+        else:
+            raise Exception("unable to dispatch, %s, perhaps your parameter should have started with w_?" % el)
 
     def apply_over(self, unwrap_spec, *extra):
         dispatch = self.dispatch
diff --git a/pypy/interpreter/nestedscope.py b/pypy/interpreter/nestedscope.py
--- a/pypy/interpreter/nestedscope.py
+++ b/pypy/interpreter/nestedscope.py
@@ -8,7 +8,7 @@
 
 class Cell(Wrappable):
     "A simple container for a wrapped value."
-    
+
     def __init__(self, w_value=None):
         self.w_value = w_value
 
@@ -90,32 +90,33 @@
     #     variables coming from a parent function in which i'm nested
     # 'closure' is a list of Cell instances: the received free vars.
 
-    cells = None
-
     @jit.unroll_safe
-    def initialize_frame_scopes(self, closure, code):
-        super_initialize_frame_scopes(self, closure, code)
+    def initialize_frame_scopes(self, outer_func, code):
+        super_initialize_frame_scopes(self, outer_func, code)
         ncellvars = len(code.co_cellvars)
         nfreevars = len(code.co_freevars)
         if not nfreevars:
             if not ncellvars:
+                self.cells = []
                 return            # no self.cells needed - fast path
-            if closure is None:
-                closure = []
-        elif closure is None:
+        elif outer_func is None:
             space = self.space
             raise OperationError(space.w_TypeError,
                                  space.wrap("directly executed code object "
                                             "may not contain free variables"))
-        if len(closure) != nfreevars:
+        if outer_func and outer_func.closure:
+            closure_size = len(outer_func.closure)
+        else:
+            closure_size = 0
+        if closure_size != nfreevars:
             raise ValueError("code object received a closure with "
                                  "an unexpected number of free variables")
         self.cells = [None] * (ncellvars + nfreevars)
         for i in range(ncellvars):
             self.cells[i] = Cell()
         for i in range(nfreevars):
-            self.cells[i + ncellvars] = closure[i]
-    
+            self.cells[i + ncellvars] = outer_func.closure[i]
+
     def _getcells(self):
         return self.cells
 
diff --git a/pypy/interpreter/pycode.py b/pypy/interpreter/pycode.py
--- a/pypy/interpreter/pycode.py
+++ b/pypy/interpreter/pycode.py
@@ -198,7 +198,7 @@
 
     def funcrun(self, func, args):
         frame = self.space.createframe(self, func.w_func_globals,
-                                  func.closure)
+                                  func)
         sig = self._signature
         # speed hack
         fresh_frame = jit.hint(frame, access_directly=True,
@@ -211,7 +211,7 @@
 
     def funcrun_obj(self, func, w_obj, args):
         frame = self.space.createframe(self, func.w_func_globals,
-                                  func.closure)
+                                  func)
         sig = self._signature
         # speed hack
         fresh_frame = jit.hint(frame, access_directly=True,
diff --git a/pypy/interpreter/pyframe.py b/pypy/interpreter/pyframe.py
--- a/pypy/interpreter/pyframe.py
+++ b/pypy/interpreter/pyframe.py
@@ -51,7 +51,10 @@
     is_being_profiled        = False
     escaped                  = False  # see mark_as_escaped()
 
-    def __init__(self, space, code, w_globals, closure):
+    def __init__(self, space, code, w_globals, outer_func):
+        if not we_are_translated():
+            assert type(self) in (space.FrameClass, CPythonFrame), (
+                "use space.FrameClass(), not directly PyFrame()")
         self = hint(self, access_directly=True, fresh_virtualizable=True)
         assert isinstance(code, pycode.PyCode)
         self.pycode = code
@@ -67,7 +70,7 @@
             self.builtin = space.builtin.pick_builtin(w_globals)
         # regular functions always have CO_OPTIMIZED and CO_NEWLOCALS.
         # class bodies only have CO_NEWLOCALS.
-        self.initialize_frame_scopes(closure, code)
+        self.initialize_frame_scopes(outer_func, code)
         self.f_lineno = code.co_firstlineno
 
     def mark_as_escaped(self):
@@ -80,7 +83,7 @@
         self.escaped = True
 
     def append_block(self, block):
-        block.previous = self.lastblock
+        assert block.previous is self.lastblock
         self.lastblock = block
 
     def pop_block(self):
@@ -106,15 +109,16 @@
         while i >= 0:
             block = lst[i]
             i -= 1
-            self.append_block(block)
+            block.previous = self.lastblock
+            self.lastblock = block
 
     def get_builtin(self):
         if self.space.config.objspace.honor__builtins__:
             return self.builtin
         else:
             return self.space.builtin
-        
-    def initialize_frame_scopes(self, closure, code): 
+
+    def initialize_frame_scopes(self, outer_func, code):
         # regular functions always have CO_OPTIMIZED and CO_NEWLOCALS.
         # class bodies only have CO_NEWLOCALS.
         # CO_NEWLOCALS: make a locals dict unless optimized is also set
@@ -381,7 +385,11 @@
         
         # do not use the instance's __init__ but the base's, because we set
         # everything like cells from here
-        PyFrame.__init__(self, space, pycode, w_globals, closure)
+        # XXX hack
+        from pypy.interpreter.function import Function
+        outer_func = Function(space, None, closure=closure,
+                             forcename="fake")
+        PyFrame.__init__(self, space, pycode, w_globals, outer_func)
         f_back = space.interp_w(PyFrame, w_f_back, can_be_None=True)
         new_frame.f_backref = jit.non_virtual_ref(f_back)
 
diff --git a/pypy/interpreter/pyopcode.py b/pypy/interpreter/pyopcode.py
--- a/pypy/interpreter/pyopcode.py
+++ b/pypy/interpreter/pyopcode.py
@@ -892,16 +892,16 @@
         raise BytecodeCorruption, "old opcode, no longer in use"
 
     def SETUP_LOOP(self, offsettoend, next_instr):
-        block = LoopBlock(self, next_instr + offsettoend)
-        self.append_block(block)
+        block = LoopBlock(self, next_instr + offsettoend, self.lastblock)
+        self.lastblock = block
 
     def SETUP_EXCEPT(self, offsettoend, next_instr):
-        block = ExceptBlock(self, next_instr + offsettoend)
-        self.append_block(block)
+        block = ExceptBlock(self, next_instr + offsettoend, self.lastblock)
+        self.lastblock = block
 
     def SETUP_FINALLY(self, offsettoend, next_instr):
-        block = FinallyBlock(self, next_instr + offsettoend)
-        self.append_block(block)
+        block = FinallyBlock(self, next_instr + offsettoend, self.lastblock)
+        self.lastblock = block
 
     def SETUP_WITH(self, offsettoend, next_instr):
         w_manager = self.peekvalue()
@@ -915,8 +915,8 @@
         w_exit = self.space.get(w_descr, w_manager)
         self.settopvalue(w_exit)
         w_result = self.space.get_and_call_function(w_enter, w_manager)
-        block = WithBlock(self, next_instr + offsettoend)
-        self.append_block(block)
+        block = WithBlock(self, next_instr + offsettoend, self.lastblock)
+        self.lastblock = block
         self.pushvalue(w_result)
 
     def WITH_CLEANUP(self, oparg, next_instr):
@@ -1247,10 +1247,10 @@
 
     _immutable_ = True
 
-    def __init__(self, frame, handlerposition):
+    def __init__(self, frame, handlerposition, previous):
         self.handlerposition = handlerposition
         self.valuestackdepth = frame.valuestackdepth
-        self.previous = None # this makes a linked list of blocks
+        self.previous = previous   # this makes a linked list of blocks
 
     def __eq__(self, other):
         return (self.__class__ is other.__class__ and
diff --git a/pypy/interpreter/pyparser/future.py b/pypy/interpreter/pyparser/future.py
--- a/pypy/interpreter/pyparser/future.py
+++ b/pypy/interpreter/pyparser/future.py
@@ -109,25 +109,19 @@
             self.getc() == self.getc(+2)):
             self.pos += 3
             while 1: # Deal with a triple quoted docstring
-                if self.getc() == '\\':
-                    self.pos += 2
+                c = self.getc()
+                if c == '\\':
+                    self.pos += 1
+                    self._skip_next_char_from_docstring()
+                elif c != endchar:
+                    self._skip_next_char_from_docstring()
                 else:
-                    c = self.getc()
-                    if c != endchar:
-                        self.pos += 1
-                        if c == '\n':
-                            self.atbol()
-                        elif c == '\r':
-                            if self.getc() == '\n':
-                                self.pos += 1
-                                self.atbol()
-                    else:
-                        self.pos += 1
-                        if (self.getc() == endchar and
-                            self.getc(+1) == endchar):
-                            self.pos += 2
-                            self.consume_empty_line()
-                            break
+                    self.pos += 1
+                    if (self.getc() == endchar and
+                        self.getc(+1) == endchar):
+                        self.pos += 2
+                        self.consume_empty_line()
+                        break
 
         else: # Deal with a single quoted docstring
             self.pos += 1
@@ -138,17 +132,21 @@
                     self.consume_empty_line()
                     return
                 elif c == '\\':
-                    # Deal with linefeeds
-                    if self.getc() != '\r':
-                        self.pos += 1
-                    else:
-                        self.pos += 1
-                        if self.getc() == '\n':
-                            self.pos += 1
+                    self._skip_next_char_from_docstring()
                 elif c in '\r\n':
                     # Syntax error
                     return
 
+    def _skip_next_char_from_docstring(self):
+        c = self.getc()
+        self.pos += 1
+        if c == '\n':
+            self.atbol()
+        elif c == '\r':
+            if self.getc() == '\n':
+                self.pos += 1
+            self.atbol()
+
     def consume_continuation(self):
         c = self.getc()
         if c in '\n\r':
diff --git a/pypy/interpreter/pyparser/test/test_futureautomaton.py b/pypy/interpreter/pyparser/test/test_futureautomaton.py
--- a/pypy/interpreter/pyparser/test/test_futureautomaton.py
+++ b/pypy/interpreter/pyparser/test/test_futureautomaton.py
@@ -221,6 +221,14 @@
     assert f.lineno == 3
     assert f.col_offset == 0
 
+def test_lots_of_continuation_lines():
+    s = "\\\n\\\n\\\n\\\n\\\n\\\n\nfrom __future__ import with_statement\n"
+    f = run(s)
+    assert f.pos == len(s)
+    assert f.flags == fut.CO_FUTURE_WITH_STATEMENT
+    assert f.lineno == 8
+    assert f.col_offset == 0
+
 # This looks like a bug in cpython parser
 # and would require extensive modifications
 # to future.py in order to emulate the same behaviour
@@ -239,3 +247,19 @@
         raise AssertionError('IndentationError not raised')
     assert f.lineno == 2
     assert f.col_offset == 0
+
+def test_continuation_lines_in_docstring_single_quoted():
+    s = '"\\\n\\\n\\\n\\\n\\\n\\\n"\nfrom  __future__ import division\n'
+    f = run(s)
+    assert f.pos == len(s)
+    assert f.flags == fut.CO_FUTURE_DIVISION
+    assert f.lineno == 8
+    assert f.col_offset == 0
+
+def test_continuation_lines_in_docstring_triple_quoted():
+    s = '"""\\\n\\\n\\\n\\\n\\\n\\\n"""\nfrom  __future__ import division\n'
+    f = run(s)
+    assert f.pos == len(s)
+    assert f.flags == fut.CO_FUTURE_DIVISION
+    assert f.lineno == 8
+    assert f.col_offset == 0
diff --git a/pypy/interpreter/test/test_gateway.py b/pypy/interpreter/test/test_gateway.py
--- a/pypy/interpreter/test/test_gateway.py
+++ b/pypy/interpreter/test/test_gateway.py
@@ -704,7 +704,7 @@
 class TestPassThroughArguments_CALL_METHOD(TestPassThroughArguments):
 
     def setup_class(cls):
-        space = gettestobjspace(usemodules=('_stackless',), **{
+        space = gettestobjspace(usemodules=('itertools',), **{
             "objspace.opcodes.CALL_METHOD": True
             })
         cls.space = space
diff --git a/pypy/interpreter/typedef.py b/pypy/interpreter/typedef.py
--- a/pypy/interpreter/typedef.py
+++ b/pypy/interpreter/typedef.py
@@ -258,6 +258,11 @@
                     self.slots_w = [None] * nslots
             def setslotvalue(self, index, w_value):
                 self.slots_w[index] = w_value
+            def delslotvalue(self, index):
+                if self.slots_w[index] is None:
+                    return False
+                self.slots_w[index] = None
+                return True
             def getslotvalue(self, index):
                 return self.slots_w[index]
         add(Proto)
@@ -530,11 +535,10 @@
         """member.__delete__(obj)
         Delete the value of the slot 'member' from the given 'obj'."""
         self.typecheck(space, w_obj)
-        w_oldresult = w_obj.getslotvalue(self.index)
-        if w_oldresult is None:
+        success = w_obj.delslotvalue(self.index)
+        if not success:
             raise OperationError(space.w_AttributeError,
                                  space.wrap(self.name)) # XXX better message
-        w_obj.setslotvalue(self.index, None)
 
 Member.typedef = TypeDef(
     "member_descriptor",
diff --git a/pypy/jit/backend/llgraph/llimpl.py b/pypy/jit/backend/llgraph/llimpl.py
--- a/pypy/jit/backend/llgraph/llimpl.py
+++ b/pypy/jit/backend/llgraph/llimpl.py
@@ -55,6 +55,12 @@
     else:
         return LLSupport.from_rstr(s)
 
+FLOAT_ARRAY_TP = lltype.Ptr(lltype.Array(lltype.Float, hints={"nolength": True}))
+def maybe_uncast(TP, array):
+    if array._TYPE.TO._hints.get("uncast_on_llgraph"):
+        array = rffi.cast(TP, array)
+    return array
+
 # a list of argtypes of all operations - couldn't find any and it's
 # very useful.  Note however that the table is half-broken here and
 # there, in ways that are sometimes a bit hard to fix; that's why
@@ -1102,13 +1108,15 @@
         return heaptracker.adr2int(llmemory.cast_ptr_to_adr(x))
     if TP == llmemory.Address:
         return heaptracker.adr2int(x)
+    if TP is lltype.SingleFloat:
+        return longlong.singlefloat2int(x)
     return lltype.cast_primitive(lltype.Signed, x)
 
 def cast_from_int(TYPE, x):
     if isinstance(TYPE, lltype.Ptr):
         if isinstance(x, (int, long, llmemory.AddressAsInt)):
             x = llmemory.cast_int_to_adr(x)
-        if TYPE is rffi.VOIDP:
+        if TYPE is rffi.VOIDP or TYPE.TO._hints.get("uncast_on_llgraph"):
             # assume that we want a "C-style" cast, without typechecking the value
             return rffi.cast(TYPE, x)
         return llmemory.cast_adr_to_ptr(x, TYPE)
@@ -1117,6 +1125,9 @@
             x = llmemory.cast_int_to_adr(x)
         assert lltype.typeOf(x) == llmemory.Address
         return x
+    elif TYPE is lltype.SingleFloat:
+        assert lltype.typeOf(x) is lltype.Signed
+        return longlong.int2singlefloat(x)
     else:
         if lltype.typeOf(x) == llmemory.Address:
             x = heaptracker.adr2int(x)
@@ -1171,6 +1182,7 @@
     del _future_values[:]
 
 def set_future_value_int(index, value):
+    assert lltype.typeOf(value) is lltype.Signed
     set_future_value_ref(index, value)
 
 def set_future_value_float(index, value):
@@ -1354,8 +1366,8 @@
     return cast_to_floatstorage(array.getitem(index))
 
 def do_getarrayitem_raw_float(array, index):
-    array = array.adr.ptr._obj
-    return cast_to_floatstorage(array.getitem(index))
+    array = maybe_uncast(FLOAT_ARRAY_TP, array.adr.ptr)
+    return cast_to_floatstorage(array._obj.getitem(index))
 
 def do_getarrayitem_gc_ptr(array, index):
     array = array._obj.container
@@ -1433,8 +1445,9 @@
     newvalue = cast_from_floatstorage(ITEMTYPE, newvalue)
     array.setitem(index, newvalue)
 
+
 def do_setarrayitem_raw_float(array, index, newvalue):
-    array = array.adr.ptr
+    array = maybe_uncast(FLOAT_ARRAY_TP, array.adr.ptr)
     ITEMTYPE = lltype.typeOf(array).TO.OF
     newvalue = cast_from_floatstorage(ITEMTYPE, newvalue)
     array._obj.setitem(index, newvalue)
@@ -1537,6 +1550,7 @@
     'i': lltype.Signed,
     'f': lltype.Float,
     'L': lltype.SignedLongLong,
+    'S': lltype.SingleFloat,
     'v': lltype.Void,
     }
 
diff --git a/pypy/jit/backend/llgraph/runner.py b/pypy/jit/backend/llgraph/runner.py
--- a/pypy/jit/backend/llgraph/runner.py
+++ b/pypy/jit/backend/llgraph/runner.py
@@ -23,13 +23,14 @@
 class Descr(history.AbstractDescr):
 
     def __init__(self, ofs, typeinfo, extrainfo=None, name=None,
-                 arg_types=None, count_fields_if_immut=-1):
+                 arg_types=None, count_fields_if_immut=-1, ffi_flags=0):
         self.ofs = ofs
         self.typeinfo = typeinfo
         self.extrainfo = extrainfo
         self.name = name
         self.arg_types = arg_types
         self.count_fields_if_immut = count_fields_if_immut
+        self.ffi_flags = ffi_flags
 
     def get_arg_types(self):
         return self.arg_types
@@ -65,6 +66,9 @@
     def count_fields_if_immutable(self):
         return self.count_fields_if_immut
 
+    def get_ffi_flags(self):
+        return self.ffi_flags
+
     def __lt__(self, other):
         raise TypeError("cannot use comparison on Descrs")
     def __le__(self, other):
@@ -89,6 +93,7 @@
 class BaseCPU(model.AbstractCPU):
     supports_floats = True
     supports_longlong = llimpl.IS_32_BIT
+    supports_singlefloats = True
 
     def __init__(self, rtyper, stats=None, opts=None,
                  translate_support_code=False,
@@ -111,14 +116,14 @@
         return False
 
     def getdescr(self, ofs, typeinfo='?', extrainfo=None, name=None,
-                 arg_types=None, count_fields_if_immut=-1):
+                 arg_types=None, count_fields_if_immut=-1, ffi_flags=0):
         key = (ofs, typeinfo, extrainfo, name, arg_types,
-               count_fields_if_immut)
+               count_fields_if_immut, ffi_flags)
         try:
             return self._descrs[key]
         except KeyError:
             descr = Descr(ofs, typeinfo, extrainfo, name, arg_types,
-                          count_fields_if_immut)
+                          count_fields_if_immut, ffi_flags)
             self._descrs[key] = descr
             return descr
 
@@ -321,7 +326,7 @@
         token = history.getkind(getattr(S, fieldname))
         return self.getdescr(ofs, token[0], name=fieldname, extrainfo=ofs2)
 
-    def calldescrof(self, FUNC, ARGS, RESULT, extrainfo=None):
+    def calldescrof(self, FUNC, ARGS, RESULT, extrainfo):
         arg_types = []
         for ARG in ARGS:
             token = history.getkind(ARG)
@@ -335,16 +340,21 @@
         return self.getdescr(0, token[0], extrainfo=extrainfo,
                              arg_types=''.join(arg_types))
 
-    def calldescrof_dynamic(self, ffi_args, ffi_result, extrainfo=None):
+    def calldescrof_dynamic(self, ffi_args, ffi_result, extrainfo, ffi_flags):
         from pypy.jit.backend.llsupport.ffisupport import get_ffi_type_kind
+        from pypy.jit.backend.llsupport.ffisupport import UnsupportedKind
         arg_types = []
-        for arg in ffi_args:
-            kind = get_ffi_type_kind(arg)
-            if kind != history.VOID:
-                arg_types.append(kind)
-        reskind = get_ffi_type_kind(ffi_result)
+        try:
+            for arg in ffi_args:
+                kind = get_ffi_type_kind(self, arg)
+                if kind != history.VOID:
+                    arg_types.append(kind)
+            reskind = get_ffi_type_kind(self, ffi_result)
+        except UnsupportedKind:
+            return None
         return self.getdescr(0, reskind, extrainfo=extrainfo,
-                             arg_types=''.join(arg_types))
+                             arg_types=''.join(arg_types),
+                             ffi_flags=ffi_flags)
 
 
     def grab_exc_value(self):
@@ -552,7 +562,7 @@
         return FieldDescr.new(T1, fieldname)
 
     @staticmethod
-    def calldescrof(FUNC, ARGS, RESULT, extrainfo=None):
+    def calldescrof(FUNC, ARGS, RESULT, extrainfo):
         return StaticMethDescr.new(FUNC, ARGS, RESULT, extrainfo)
 
     @staticmethod
diff --git a/pypy/jit/backend/llgraph/test/test_llgraph.py b/pypy/jit/backend/llgraph/test/test_llgraph.py
--- a/pypy/jit/backend/llgraph/test/test_llgraph.py
+++ b/pypy/jit/backend/llgraph/test/test_llgraph.py
@@ -19,6 +19,9 @@
     def setup_method(self, _):
         self.cpu = self.cpu_type(None)
 
+    def test_memoryerror(self):
+        py.test.skip("does not make much sense on the llgraph backend")
+
 
 def test_cast_adr_to_int_and_back():
     X = lltype.Struct('X', ('foo', lltype.Signed))
diff --git a/pypy/jit/backend/llsupport/descr.py b/pypy/jit/backend/llsupport/descr.py
--- a/pypy/jit/backend/llsupport/descr.py
+++ b/pypy/jit/backend/llsupport/descr.py
@@ -270,10 +270,12 @@
     _clsname = ''
     loop_token = None
     arg_classes = ''     # <-- annotation hack
+    ffi_flags = 0
 
-    def __init__(self, arg_classes, extrainfo=None):
+    def __init__(self, arg_classes, extrainfo=None, ffi_flags=0):
         self.arg_classes = arg_classes    # string of "r" and "i" (ref/int)
         self.extrainfo = extrainfo
+        self.ffi_flags = ffi_flags
 
     def __repr__(self):
         res = '%s(%s)' % (self.__class__.__name__, self.arg_classes)
@@ -294,6 +296,13 @@
     def get_extra_info(self):
         return self.extrainfo
 
+    def get_ffi_flags(self):
+        return self.ffi_flags
+
+    def get_call_conv(self):
+        from pypy.rlib.clibffi import get_call_conv
+        return get_call_conv(self.ffi_flags, True)
+
     def get_arg_types(self):
         return self.arg_classes
 
@@ -313,6 +322,8 @@
                 c = 'f'
             elif c == 'f' and longlong.supports_longlong:
                 return 'longlong.getrealfloat(%s)' % (process('L'),)
+            elif c == 'S':
+                return 'longlong.int2singlefloat(%s)' % (process('i'),)
             arg = 'args_%s[%d]' % (c, seen[c])
             seen[c] += 1
             return arg
@@ -328,6 +339,8 @@
                 return lltype.Void
             elif arg == 'L':
                 return lltype.SignedLongLong
+            elif arg == 'S':
+                return lltype.SingleFloat
             else:
                 raise AssertionError(arg)
 
@@ -344,6 +357,8 @@
             result = 'rffi.cast(lltype.SignedLongLong, res)'
         elif self.get_return_type() == history.VOID:
             result = 'None'
+        elif self.get_return_type() == 'S':
+            result = 'longlong.singlefloat2int(res)'
         else:
             assert 0
         source = py.code.Source("""
@@ -354,14 +369,15 @@
         """ % locals())
         ARGS = [TYPE(arg) for arg in self.arg_classes]
         FUNC = lltype.FuncType(ARGS, RESULT)
-        d = locals().copy()
-        d.update(globals())
+        d = globals().copy()
+        d.update(locals())
         exec source.compile() in d
         self.call_stub = d['call_stub']
 
     def verify_types(self, args_i, args_r, args_f, return_type):
         assert self._return_type in return_type
-        assert self.arg_classes.count('i') == len(args_i or ())
+        assert (self.arg_classes.count('i') +
+                self.arg_classes.count('S')) == len(args_i or ())
         assert self.arg_classes.count('r') == len(args_r or ())
         assert (self.arg_classes.count('f') +
                 self.arg_classes.count('L')) == len(args_f or ())
@@ -394,8 +410,8 @@
     """
     _clsname = 'DynamicIntCallDescr'
 
-    def __init__(self, arg_classes, result_size, result_sign, extrainfo=None):
-        BaseIntCallDescr.__init__(self, arg_classes, extrainfo)
+    def __init__(self, arg_classes, result_size, result_sign, extrainfo=None, ffi_flags=0):
+        BaseIntCallDescr.__init__(self, arg_classes, extrainfo, ffi_flags)
         assert isinstance(result_sign, bool)
         self._result_size = chr(result_size)
         self._result_sign = result_sign
@@ -438,23 +454,39 @@
     def get_result_size(self, translate_support_code):
         return 0
 
+_SingleFloatCallDescr = None   # built lazily
+
 def getCallDescrClass(RESULT):
     if RESULT is lltype.Void:
         return VoidCallDescr
     if RESULT is lltype.Float:
         return FloatCallDescr
+    if RESULT is lltype.SingleFloat:
+        global _SingleFloatCallDescr
+        if _SingleFloatCallDescr is None:
+            assert rffi.sizeof(rffi.UINT) == rffi.sizeof(RESULT)
+            class SingleFloatCallDescr(getCallDescrClass(rffi.UINT)):
+                _clsname = 'SingleFloatCallDescr'
+                _return_type = 'S'
+            _SingleFloatCallDescr = SingleFloatCallDescr
+        return _SingleFloatCallDescr
     if is_longlong(RESULT):
         return LongLongCallDescr
     return getDescrClass(RESULT, BaseIntCallDescr, GcPtrCallDescr,
                          NonGcPtrCallDescr, 'Call', 'get_result_size',
                          Ellipsis,  # <= floatattrname should not be used here
                          '_is_result_signed')
+getCallDescrClass._annspecialcase_ = 'specialize:memo'
 
 def get_call_descr(gccache, ARGS, RESULT, extrainfo=None):
     arg_classes = []
     for ARG in ARGS:
         kind = getkind(ARG)
-        if   kind == 'int': arg_classes.append('i')
+        if   kind == 'int':
+            if ARG is lltype.SingleFloat:
+                arg_classes.append('S')
+            else:
+                arg_classes.append('i')
         elif kind == 'ref': arg_classes.append('r')
         elif kind == 'float':
             if is_longlong(ARG):
@@ -486,6 +518,9 @@
             return GcPtrDescr
         else:
             return NonGcPtrDescr
+    if TYPE is lltype.SingleFloat:
+        assert rffi.sizeof(rffi.UINT) == rffi.sizeof(TYPE)
+        TYPE = rffi.UINT
     try:
         return _cache[nameprefix, TYPE]
     except KeyError:
diff --git a/pypy/jit/backend/llsupport/ffisupport.py b/pypy/jit/backend/llsupport/ffisupport.py
--- a/pypy/jit/backend/llsupport/ffisupport.py
+++ b/pypy/jit/backend/llsupport/ffisupport.py
@@ -1,41 +1,58 @@
 from pypy.rlib.rarithmetic import intmask
 from pypy.jit.metainterp import history
-from pypy.jit.backend.llsupport.descr import DynamicIntCallDescr, NonGcPtrCallDescr,\
-    FloatCallDescr, VoidCallDescr
+from pypy.rpython.lltypesystem import rffi
+from pypy.jit.backend.llsupport.descr import (
+    DynamicIntCallDescr, NonGcPtrCallDescr, FloatCallDescr, VoidCallDescr,
+    LongLongCallDescr, getCallDescrClass)
 
 class UnsupportedKind(Exception):
     pass
 
-def get_call_descr_dynamic(ffi_args, ffi_result, extrainfo=None):
+def get_call_descr_dynamic(cpu, ffi_args, ffi_result, extrainfo=None, ffi_flags=0):
     """Get a call descr: the types of result and args are represented by
     rlib.libffi.types.*"""
     try:
-        reskind = get_ffi_type_kind(ffi_result)
-        argkinds = [get_ffi_type_kind(arg) for arg in ffi_args]
+        reskind = get_ffi_type_kind(cpu, ffi_result)
+        argkinds = [get_ffi_type_kind(cpu, arg) for arg in ffi_args]
     except UnsupportedKind:
-        return None # ??
+        return None
     arg_classes = ''.join(argkinds)
     if reskind == history.INT:
         size = intmask(ffi_result.c_size)
         signed = is_ffi_type_signed(ffi_result)
-        return DynamicIntCallDescr(arg_classes, size, signed, extrainfo)
+        return DynamicIntCallDescr(arg_classes, size, signed, extrainfo,
+                                   ffi_flags=ffi_flags)
     elif reskind == history.REF:
-        return  NonGcPtrCallDescr(arg_classes, extrainfo)
+        return  NonGcPtrCallDescr(arg_classes, extrainfo,
+                                  ffi_flags=ffi_flags)
     elif reskind == history.FLOAT:
-        return FloatCallDescr(arg_classes, extrainfo)
+        return FloatCallDescr(arg_classes, extrainfo,
+                              ffi_flags=ffi_flags)
     elif reskind == history.VOID:
-        return VoidCallDescr(arg_classes, extrainfo)
+        return VoidCallDescr(arg_classes, extrainfo,
+                             ffi_flags=ffi_flags)
+    elif reskind == 'L':
+        return LongLongCallDescr(arg_classes, extrainfo,
+                                 ffi_flags=ffi_flags)
+    elif reskind == 'S':
+        SingleFloatCallDescr = getCallDescrClass(rffi.FLOAT)
+        return SingleFloatCallDescr(arg_classes, extrainfo,
+                                    ffi_flags=ffi_flags)
     assert False
 
-def get_ffi_type_kind(ffi_type):
+def get_ffi_type_kind(cpu, ffi_type):
     from pypy.rlib.libffi import types
     kind = types.getkind(ffi_type)
     if kind == 'i' or kind == 'u':
         return history.INT
-    elif kind == 'f':
+    elif cpu.supports_floats and kind == 'f':
         return history.FLOAT
     elif kind == 'v':
         return history.VOID
+    elif cpu.supports_longlong and (kind == 'I' or kind == 'U'):     # longlong
+        return 'L'
+    elif cpu.supports_singlefloats and kind == 's':    # singlefloat
+        return 'S'
     raise UnsupportedKind("Unsupported kind '%s'" % kind)
 
 def is_ffi_type_signed(ffi_type):
diff --git a/pypy/jit/backend/llsupport/gc.py b/pypy/jit/backend/llsupport/gc.py
--- a/pypy/jit/backend/llsupport/gc.py
+++ b/pypy/jit/backend/llsupport/gc.py
@@ -366,36 +366,92 @@
 
     def add_jit2gc_hooks(self, jit2gc):
         #
-        def collect_jit_stack_root(callback, gc, addr):
-            if addr.signed[0] != GcRootMap_shadowstack.MARKER:
-                # common case
-                if gc.points_to_valid_gc_object(addr):
-                    callback(gc, addr)
-                return WORD
-            else:
-                # case of a MARKER followed by an assembler stack frame
-                follow_stack_frame_of_assembler(callback, gc, addr)
-                return 2 * WORD
+        # ---------------
+        # This is used to enumerate the shadowstack in the presence
+        # of the JIT.  It is also used by the stacklet support in
+        # rlib/_stacklet_shadowstack.  That's why it is written as
+        # an iterator that can also be used with a custom_trace.
         #
-        def follow_stack_frame_of_assembler(callback, gc, addr):
-            frame_addr = addr.signed[1]
-            addr = llmemory.cast_int_to_adr(frame_addr + self.force_index_ofs)
-            force_index = addr.signed[0]
-            if force_index < 0:
-                force_index = ~force_index
-            callshape = self._callshapes[force_index]
-            n = 0
-            while True:
-                offset = rffi.cast(lltype.Signed, callshape[n])
-                if offset == 0:
-                    break
-                addr = llmemory.cast_int_to_adr(frame_addr + offset)
-                if gc.points_to_valid_gc_object(addr):
-                    callback(gc, addr)
-                n += 1
+        class RootIterator:
+            _alloc_flavor_ = "raw"
+
+            def next(iself, gc, next, range_highest):
+                # Return the "next" valid GC object' address.  This usually
+                # means just returning "next", until we reach "range_highest",
+                # except that we are skipping NULLs.  If "next" contains a
+                # MARKER instead, then we go into JIT-frame-lookup mode.
+                #
+                while True:
+                    #
+                    # If we are not iterating right now in a JIT frame
+                    if iself.frame_addr == 0:
+                        #
+                        # Look for the next shadowstack address that
+                        # contains a valid pointer
+                        while next != range_highest:
+                            if next.signed[0] == self.MARKER:
+                                break
+                            if gc.points_to_valid_gc_object(next):
+                                return next
+                            next += llmemory.sizeof(llmemory.Address)
+                        else:
+                            return llmemory.NULL     # done
+                        #
+                        # It's a JIT frame.  Save away 'next' for later, and
+                        # go into JIT-frame-exploring mode.
+                        next += llmemory.sizeof(llmemory.Address)
+                        frame_addr = next.signed[0]
+                        iself.saved_next = next
+                        iself.frame_addr = frame_addr
+                        addr = llmemory.cast_int_to_adr(frame_addr +
+                                                        self.force_index_ofs)
+                        addr = iself.translateptr(iself.context, addr)
+                        force_index = addr.signed[0]
+                        if force_index < 0:
+                            force_index = ~force_index
+                        # NB: the next line reads a still-alive _callshapes,
+                        # because we ensure that just before we called this
+                        # piece of assembler, we put on the (same) stack a
+                        # pointer to a loop_token that keeps the force_index
+                        # alive.
+                        callshape = self._callshapes[force_index]
+                    else:
+                        # Continuing to explore this JIT frame
+                        callshape = iself.callshape
+                    #
+                    # 'callshape' points to the next INT of the callshape.
+                    # If it's zero we are done with the JIT frame.
+                    while rffi.cast(lltype.Signed, callshape[0]) != 0:
+                        #
+                        # Non-zero: it's an offset inside the JIT frame.
+                        # Read it and increment 'callshape'.
+                        offset = rffi.cast(lltype.Signed, callshape[0])
+                        callshape = lltype.direct_ptradd(callshape, 1)
+                        addr = llmemory.cast_int_to_adr(iself.frame_addr +
+                                                        offset)
+                        addr = iself.translateptr(iself.context, addr)
+                        if gc.points_to_valid_gc_object(addr):
+                            #
+                            # The JIT frame contains a valid GC pointer at
+                            # this address (as opposed to NULL).  Save
+                            # 'callshape' for the next call, and return the
+                            # address.
+                            iself.callshape = callshape
+                            return addr
+                    #
+                    # Restore 'prev' and loop back to the start.
+                    iself.frame_addr = 0
+                    next = iself.saved_next
+                    next += llmemory.sizeof(llmemory.Address)
+
+        # ---------------
         #
+        root_iterator = RootIterator()
+        root_iterator.frame_addr = 0
+        root_iterator.context = llmemory.NULL
+        root_iterator.translateptr = lambda context, addr: addr
         jit2gc.update({
-            'rootstackhook': collect_jit_stack_root,
+            'root_iterator': root_iterator,
             })
 
     def initialize(self):
@@ -544,18 +600,19 @@
         assert self.GCClass.inline_simple_malloc
         assert self.GCClass.inline_simple_malloc_varsize
 
-        # make a malloc function, with three arguments
+        # make a malloc function, with two arguments
         def malloc_basic(size, tid):
             type_id = llop.extract_ushort(llgroup.HALFWORD, tid)
             has_finalizer = bool(tid & (1<", res)
             return res
@@ -571,14 +628,10 @@
         def malloc_array(itemsize, tid, num_elem):
             type_id = llop.extract_ushort(llgroup.HALFWORD, tid)
             check_typeid(type_id)
-            try:
-                return llop1.do_malloc_varsize_clear(
-                    llmemory.GCREF,
-                    type_id, num_elem, self.array_basesize, itemsize,
-                    self.array_length_ofs, True)
-            except MemoryError:
-                fatalerror("out of memory (from JITted code)")
-                return lltype.nullptr(llmemory.GCREF.TO)
+            return llop1.do_malloc_varsize_clear(
+                llmemory.GCREF,
+                type_id, num_elem, self.array_basesize, itemsize,
+                self.array_length_ofs)
         self.malloc_array = malloc_array
         self.GC_MALLOC_ARRAY = lltype.Ptr(lltype.FuncType(
             [lltype.Signed] * 3, llmemory.GCREF))
@@ -591,23 +644,15 @@
         unicode_type_id = self.layoutbuilder.get_type_id(rstr.UNICODE)
         #
         def malloc_str(length):
-            try:
-                return llop1.do_malloc_varsize_clear(
-                    llmemory.GCREF,
-                    str_type_id, length, str_basesize, str_itemsize,
-                    str_ofs_length, True)
-            except MemoryError:
-                fatalerror("out of memory (from JITted code)")
-                return lltype.nullptr(llmemory.GCREF.TO)
+            return llop1.do_malloc_varsize_clear(
+                llmemory.GCREF,
+                str_type_id, length, str_basesize, str_itemsize,
+                str_ofs_length)
         def malloc_unicode(length):
-            try:
-                return llop1.do_malloc_varsize_clear(
-                    llmemory.GCREF,
-                    unicode_type_id, length, unicode_basesize,unicode_itemsize,
-                    unicode_ofs_length, True)
-            except MemoryError:
-                fatalerror("out of memory (from JITted code)")
-                return lltype.nullptr(llmemory.GCREF.TO)
+            return llop1.do_malloc_varsize_clear(
+                llmemory.GCREF,
+                unicode_type_id, length, unicode_basesize,unicode_itemsize,
+                unicode_ofs_length)
         self.malloc_str = malloc_str
         self.malloc_unicode = malloc_unicode
         self.GC_MALLOC_STR_UNICODE = lltype.Ptr(lltype.FuncType(
@@ -628,16 +673,12 @@
             if self.DEBUG:
                 random_usage_of_xmm_registers()
             assert size >= self.minimal_size_in_nursery
-            try:
-                # NB. although we call do_malloc_fixedsize_clear() here,
-                # it's a bit of a hack because we set tid to 0 and may
-                # also use it to allocate varsized objects.  The tid
-                # and possibly the length are both set afterward.
-                gcref = llop1.do_malloc_fixedsize_clear(llmemory.GCREF,
-                                            0, size, True, False, False)
-            except MemoryError:
-                fatalerror("out of memory (from JITted code)")
-                return 0
+            # NB. although we call do_malloc_fixedsize_clear() here,
+            # it's a bit of a hack because we set tid to 0 and may
+            # also use it to allocate varsized objects.  The tid
+            # and possibly the length are both set afterward.
+            gcref = llop1.do_malloc_fixedsize_clear(llmemory.GCREF,
+                                        0, size, False, False)
             return rffi.cast(lltype.Signed, gcref)
         self.malloc_slowpath = malloc_slowpath
         self.MALLOC_SLOWPATH = lltype.FuncType([lltype.Signed], lltype.Signed)
diff --git a/pypy/jit/backend/llsupport/llmodel.py b/pypy/jit/backend/llsupport/llmodel.py
--- a/pypy/jit/backend/llsupport/llmodel.py
+++ b/pypy/jit/backend/llsupport/llmodel.py
@@ -253,13 +253,13 @@
         return ofs, size, sign
     unpack_arraydescr_size._always_inline_ = True
 
-    def calldescrof(self, FUNC, ARGS, RESULT, extrainfo=None):
+    def calldescrof(self, FUNC, ARGS, RESULT, extrainfo):
         return get_call_descr(self.gc_ll_descr, ARGS, RESULT, extrainfo)
 
-    def calldescrof_dynamic(self, ffi_args, ffi_result, extrainfo=None):
+    def calldescrof_dynamic(self, ffi_args, ffi_result, extrainfo, ffi_flags):
         from pypy.jit.backend.llsupport import ffisupport
-        return ffisupport.get_call_descr_dynamic(ffi_args, ffi_result,
-                                                 extrainfo)
+        return ffisupport.get_call_descr_dynamic(self, ffi_args, ffi_result,
+                                                 extrainfo, ffi_flags)
 
     def get_overflow_error(self):
         ovf_vtable = self.cast_adr_to_int(self._ovf_error_vtable)
@@ -586,7 +586,7 @@
     def bh_call_i(self, func, calldescr, args_i, args_r, args_f):
         assert isinstance(calldescr, BaseIntCallDescr)
         if not we_are_translated():
-            calldescr.verify_types(args_i, args_r, args_f, history.INT)
+            calldescr.verify_types(args_i, args_r, args_f, history.INT + 'S')
         return calldescr.call_stub(func, args_i, args_r, args_f)
 
     def bh_call_r(self, func, calldescr, args_i, args_r, args_f):
diff --git a/pypy/jit/backend/llsupport/test/test_descr.py b/pypy/jit/backend/llsupport/test/test_descr.py
--- a/pypy/jit/backend/llsupport/test/test_descr.py
+++ b/pypy/jit/backend/llsupport/test/test_descr.py
@@ -51,7 +51,8 @@
     S = lltype.GcStruct('S', ('x', lltype.Char),
                              ('y', lltype.Ptr(T)),
                              ('z', lltype.Ptr(U)),
-                             ('f', lltype.Float))
+                             ('f', lltype.Float),
+                             ('s', lltype.SingleFloat))
     assert getFieldDescrClass(lltype.Ptr(T)) is GcPtrFieldDescr
     assert getFieldDescrClass(lltype.Ptr(U)) is NonGcPtrFieldDescr
     cls = getFieldDescrClass(lltype.Char)
@@ -60,6 +61,10 @@
     clsf = getFieldDescrClass(lltype.Float)
     assert clsf != cls
     assert clsf == getFieldDescrClass(lltype.Float)
+    clss = getFieldDescrClass(lltype.SingleFloat)
+    assert clss not in (cls, clsf)
+    assert clss == getFieldDescrClass(lltype.SingleFloat)
+    assert clss == getFieldDescrClass(rffi.UINT)    # for now
     #
     c0 = GcCache(False)
     c1 = GcCache(True)
@@ -71,14 +76,17 @@
         descr_y = get_field_descr(c2, S, 'y')
         descr_z = get_field_descr(c2, S, 'z')
         descr_f = get_field_descr(c2, S, 'f')
+        descr_s = get_field_descr(c2, S, 's')
         assert descr_x.__class__ is cls
         assert descr_y.__class__ is GcPtrFieldDescr
         assert descr_z.__class__ is NonGcPtrFieldDescr
         assert descr_f.__class__ is clsf
+        assert descr_s.__class__ is clss
         assert descr_x.name == 'S.x'
         assert descr_y.name == 'S.y'
         assert descr_z.name == 'S.z'
         assert descr_f.name == 'S.f'
+        assert descr_s.name == 'S.s'
         if not tsc:
             assert descr_x.offset < descr_y.offset < descr_z.offset
             assert descr_x.sort_key() < descr_y.sort_key() < descr_z.sort_key()
@@ -86,23 +94,29 @@
             assert descr_y.get_field_size(False) == rffi.sizeof(lltype.Ptr(T))
             assert descr_z.get_field_size(False) == rffi.sizeof(lltype.Ptr(U))
             assert descr_f.get_field_size(False) == rffi.sizeof(lltype.Float)
+            assert descr_s.get_field_size(False) == rffi.sizeof(
+                                                            lltype.SingleFloat)
         else:
             assert isinstance(descr_x.offset, Symbolic)
             assert isinstance(descr_y.offset, Symbolic)
             assert isinstance(descr_z.offset, Symbolic)
             assert isinstance(descr_f.offset, Symbolic)
+            assert isinstance(descr_s.offset, Symbolic)
             assert isinstance(descr_x.get_field_size(True), Symbolic)
             assert isinstance(descr_y.get_field_size(True), Symbolic)
             assert isinstance(descr_z.get_field_size(True), Symbolic)
             assert isinstance(descr_f.get_field_size(True), Symbolic)
+            assert isinstance(descr_s.get_field_size(True), Symbolic)
         assert not descr_x.is_pointer_field()
         assert     descr_y.is_pointer_field()
         assert not descr_z.is_pointer_field()
         assert not descr_f.is_pointer_field()
+        assert not descr_s.is_pointer_field()
         assert not descr_x.is_float_field()
         assert not descr_y.is_float_field()
         assert not descr_z.is_float_field()
         assert     descr_f.is_float_field()
+        assert not descr_s.is_float_field()
 
 
 def test_get_field_descr_sign():
@@ -136,6 +150,7 @@
     A4 = lltype.GcArray(lltype.Float)
     A5 = lltype.GcArray(lltype.Struct('x', ('v', lltype.Signed),
                                       ('k', lltype.Signed)))
+    A6 = lltype.GcArray(lltype.SingleFloat)
     assert getArrayDescrClass(A2) is GcPtrArrayDescr
     assert getArrayDescrClass(A3) is NonGcPtrArrayDescr
     cls = getArrayDescrClass(A1)
@@ -144,6 +159,9 @@
     clsf = getArrayDescrClass(A4)
     assert clsf != cls
     assert clsf == getArrayDescrClass(lltype.GcArray(lltype.Float))
+    clss = getArrayDescrClass(A5)
+    assert clss not in (clsf, cls)
+    assert clss == getArrayDescrClass(lltype.GcArray(rffi.UINT))
     #
     c0 = GcCache(False)
     descr1 = get_array_descr(c0, A1)
@@ -151,10 +169,12 @@
     descr3 = get_array_descr(c0, A3)
     descr4 = get_array_descr(c0, A4)
     descr5 = get_array_descr(c0, A5)
+    descr6 = get_array_descr(c0, A6)
     assert descr1.__class__ is cls
     assert descr2.__class__ is GcPtrArrayDescr
     assert descr3.__class__ is NonGcPtrArrayDescr
     assert descr4.__class__ is clsf
+    assert descr5.__class__ is clss
     assert descr1 == get_array_descr(c0, lltype.GcArray(lltype.Char))
     assert not descr1.is_array_of_pointers()
     assert     descr2.is_array_of_pointers()
@@ -174,24 +194,29 @@
     assert descr2.get_base_size(False) == get_alignment('p')
     assert descr3.get_base_size(False) == get_alignment('p')
     assert descr4.get_base_size(False) == get_alignment('d')
+    assert descr5.get_base_size(False) == get_alignment('f')
     assert descr1.get_ofs_length(False) == 0
     assert descr2.get_ofs_length(False) == 0
     assert descr3.get_ofs_length(False) == 0
     assert descr4.get_ofs_length(False) == 0
+    assert descr5.get_ofs_length(False) == 0
     assert descr1.get_item_size(False) == rffi.sizeof(lltype.Char)
     assert descr2.get_item_size(False) == rffi.sizeof(lltype.Ptr(T))
     assert descr3.get_item_size(False) == rffi.sizeof(lltype.Ptr(U))
     assert descr4.get_item_size(False) == rffi.sizeof(lltype.Float)
     assert descr5.get_item_size(False) == rffi.sizeof(lltype.Signed) * 2
+    assert descr6.get_item_size(False) == rffi.sizeof(lltype.SingleFloat)
     #
     assert isinstance(descr1.get_base_size(True), Symbolic)
     assert isinstance(descr2.get_base_size(True), Symbolic)
     assert isinstance(descr3.get_base_size(True), Symbolic)
     assert isinstance(descr4.get_base_size(True), Symbolic)
+    assert isinstance(descr5.get_base_size(True), Symbolic)
     assert isinstance(descr1.get_ofs_length(True), Symbolic)
     assert isinstance(descr2.get_ofs_length(True), Symbolic)
     assert isinstance(descr3.get_ofs_length(True), Symbolic)
     assert isinstance(descr4.get_ofs_length(True), Symbolic)
+    assert isinstance(descr5.get_ofs_length(True), Symbolic)
     assert isinstance(descr1.get_item_size(True), Symbolic)
     assert isinstance(descr2.get_item_size(True), Symbolic)
     assert isinstance(descr3.get_item_size(True), Symbolic)
@@ -216,6 +241,11 @@
     assert descr.is_array_of_floats()
     assert descr.get_base_size(False) == 0
     assert descr.get_ofs_length(False) == -1
+    CA = rffi.CArray(rffi.FLOAT)
+    descr = get_array_descr(c0, CA)
+    assert not descr.is_array_of_floats()
+    assert descr.get_base_size(False) == 0
+    assert descr.get_ofs_length(False) == -1
 
 
 def test_get_array_descr_sign():
@@ -263,6 +293,11 @@
     assert descr4.get_result_size(False) == rffi.sizeof(lltype.Float)
     assert descr4.get_return_type() == history.FLOAT
     assert descr4.arg_classes == "ff"
+    #
+    descr5 = get_call_descr(c0, [lltype.SingleFloat], lltype.SingleFloat)
+    assert descr5.get_result_size(False) == rffi.sizeof(lltype.SingleFloat)
+    assert descr5.get_return_type() == "S"
+    assert descr5.arg_classes == "S"
 
 def test_get_call_descr_not_translated_longlong():
     if sys.maxint > 2147483647:
@@ -292,6 +327,11 @@
     assert isinstance(descr4.get_result_size(True), Symbolic)
     assert descr4.get_return_type() == history.FLOAT
     assert descr4.arg_classes == "ff"
+    #
+    descr5 = get_call_descr(c1, [lltype.SingleFloat], lltype.SingleFloat)
+    assert isinstance(descr5.get_result_size(True), Symbolic)
+    assert descr5.get_return_type() == "S"
+    assert descr5.arg_classes == "S"
 
 def test_call_descr_extra_info():
     c1 = GcCache(True)
@@ -349,8 +389,11 @@
     #
     descr4f = get_call_descr(c0, [lltype.Char, lltype.Ptr(S)], lltype.Float)
     assert 'FloatCallDescr' in descr4f.repr_of_descr()
+    #
+    descr5f = get_call_descr(c0, [lltype.Char], lltype.SingleFloat)
+    assert 'SingleFloatCallDescr' in descr5f.repr_of_descr()
 
-def test_call_stubs():
+def test_call_stubs_1():
     c0 = GcCache(False)
     ARGS = [lltype.Char, lltype.Signed]
     RES = lltype.Char
@@ -364,6 +407,8 @@
     res = call_stub(rffi.cast(lltype.Signed, fnptr), [1, 2], None, None)
     assert res == ord('c')
 
+def test_call_stubs_2():
+    c0 = GcCache(False)
     ARRAY = lltype.GcArray(lltype.Signed)
     ARGS = [lltype.Float, lltype.Ptr(ARRAY)]
     RES = lltype.Float
@@ -379,3 +424,27 @@
     res = descr2.call_stub(rffi.cast(lltype.Signed, fnptr),
                            [], [opaquea], [longlong.getfloatstorage(3.5)])
     assert longlong.getrealfloat(res) == 4.5
+
+def test_call_stubs_single_float():
+    from pypy.rlib.longlong2float import uint2singlefloat, singlefloat2uint
+    from pypy.rlib.rarithmetic import r_singlefloat, intmask
+    #
+    c0 = GcCache(False)
+    ARGS = [lltype.SingleFloat, lltype.SingleFloat, lltype.SingleFloat]
+    RES = lltype.SingleFloat
+
+    def f(a, b, c):
+        a = float(a)
+        b = float(b)
+        c = float(c)
+        x = a - (b / c)
+        return r_singlefloat(x)
+
+    fnptr = llhelper(lltype.Ptr(lltype.FuncType(ARGS, RES)), f)
+    descr2 = get_call_descr(c0, ARGS, RES)
+    a = intmask(singlefloat2uint(r_singlefloat(-10.0)))
+    b = intmask(singlefloat2uint(r_singlefloat(3.0)))
+    c = intmask(singlefloat2uint(r_singlefloat(2.0)))
+    res = descr2.call_stub(rffi.cast(lltype.Signed, fnptr),
+                           [a, b, c], [], [])
+    assert float(uint2singlefloat(rffi.r_uint(res))) == -11.5
diff --git a/pypy/jit/backend/llsupport/test/test_ffisupport.py b/pypy/jit/backend/llsupport/test/test_ffisupport.py
--- a/pypy/jit/backend/llsupport/test/test_ffisupport.py
+++ b/pypy/jit/backend/llsupport/test/test_ffisupport.py
@@ -1,24 +1,56 @@
 from pypy.rlib.libffi import types
-from pypy.jit.backend.llsupport.ffisupport import get_call_descr_dynamic, \
-    VoidCallDescr, DynamicIntCallDescr
-    
+from pypy.jit.codewriter.longlong import is_64_bit
+from pypy.jit.backend.llsupport.ffisupport import *
+
+
+class FakeCPU:
+    def __init__(self, supports_floats=False, supports_longlong=False,
+                 supports_singlefloats=False):
+        self.supports_floats = supports_floats
+        self.supports_longlong = supports_longlong
+        self.supports_singlefloats = supports_singlefloats
+
+
 def test_call_descr_dynamic():
+    args = [types.sint, types.pointer]
+    descr = get_call_descr_dynamic(FakeCPU(), args, types.sint, ffi_flags=42)
+    assert isinstance(descr, DynamicIntCallDescr)
+    assert descr.arg_classes == 'ii'
+    assert descr.get_ffi_flags() == 42
 
     args = [types.sint, types.double, types.pointer]
-    descr = get_call_descr_dynamic(args, types.void)
+    descr = get_call_descr_dynamic(FakeCPU(), args, types.void)
+    assert descr is None    # missing floats
+    descr = get_call_descr_dynamic(FakeCPU(supports_floats=True),
+                                   args, types.void, ffi_flags=43)
     assert isinstance(descr, VoidCallDescr)
     assert descr.arg_classes == 'ifi'
+    assert descr.get_ffi_flags() == 43
 
-    descr = get_call_descr_dynamic([], types.sint8)
+    descr = get_call_descr_dynamic(FakeCPU(), [], types.sint8)
     assert isinstance(descr, DynamicIntCallDescr)
     assert descr.get_result_size(False) == 1
     assert descr.is_result_signed() == True
 
-    descr = get_call_descr_dynamic([], types.uint8)
+    descr = get_call_descr_dynamic(FakeCPU(), [], types.uint8)
     assert isinstance(descr, DynamicIntCallDescr)
     assert descr.get_result_size(False) == 1
     assert descr.is_result_signed() == False
 
-    descr = get_call_descr_dynamic([], types.float)
-    assert descr is None # single floats are not supported so far
-    
+    if not is_64_bit:
+        descr = get_call_descr_dynamic(FakeCPU(), [], types.slonglong)
+        assert descr is None   # missing longlongs
+        descr = get_call_descr_dynamic(FakeCPU(supports_longlong=True),
+                                       [], types.slonglong, ffi_flags=43)
+        assert isinstance(descr, LongLongCallDescr)
+        assert descr.get_ffi_flags() == 43
+    else:
+        assert types.slonglong is types.slong
+
+    descr = get_call_descr_dynamic(FakeCPU(), [], types.float)
+    assert descr is None   # missing singlefloats
+    descr = get_call_descr_dynamic(FakeCPU(supports_singlefloats=True),
+                                   [], types.float, ffi_flags=44)
+    SingleFloatCallDescr = getCallDescrClass(rffi.FLOAT)
+    assert isinstance(descr, SingleFloatCallDescr)
+    assert descr.get_ffi_flags() == 44
diff --git a/pypy/jit/backend/llsupport/test/test_gc.py b/pypy/jit/backend/llsupport/test/test_gc.py
--- a/pypy/jit/backend/llsupport/test/test_gc.py
+++ b/pypy/jit/backend/llsupport/test/test_gc.py
@@ -246,9 +246,8 @@
     def __init__(self):
         self.record = []
 
-    def do_malloc_fixedsize_clear(self, RESTYPE, type_id, size, can_collect,
+    def do_malloc_fixedsize_clear(self, RESTYPE, type_id, size,
                                   has_finalizer, contains_weakptr):
-        assert can_collect
         assert not contains_weakptr
         p = llmemory.raw_malloc(size)
         p = llmemory.cast_adr_to_ptr(p, RESTYPE)
@@ -258,8 +257,7 @@
         return p
 
     def do_malloc_varsize_clear(self, RESTYPE, type_id, length, size,
-                                itemsize, offset_to_length, can_collect):
-        assert can_collect
+                                itemsize, offset_to_length):
         p = llmemory.raw_malloc(size + itemsize * length)
         (p + offset_to_length).signed[0] = length
         p = llmemory.cast_adr_to_ptr(p, RESTYPE)
diff --git a/pypy/jit/backend/model.py b/pypy/jit/backend/model.py
--- a/pypy/jit/backend/model.py
+++ b/pypy/jit/backend/model.py
@@ -8,12 +8,13 @@
     # ^^^ This is only useful on 32-bit platforms.  If True,
     # longlongs are supported by the JIT, but stored as doubles.
     # Boxes and Consts are BoxFloats and ConstFloats.
+    supports_singlefloats = False
 
     done_with_this_frame_void_v = -1
     done_with_this_frame_int_v = -1
     done_with_this_frame_ref_v = -1
     done_with_this_frame_float_v = -1
-    exit_frame_with_exception_v = -1
+    propagate_exception_v = -1
     total_compiled_loops = 0
     total_compiled_bridges = 0
     total_freed_loops = 0
diff --git a/pypy/jit/backend/test/calling_convention_test.py b/pypy/jit/backend/test/calling_convention_test.py
--- a/pypy/jit/backend/test/calling_convention_test.py
+++ b/pypy/jit/backend/test/calling_convention_test.py
@@ -8,6 +8,7 @@
                                          ConstObj, BoxFloat, ConstFloat)
 from pypy.jit.metainterp.resoperation import ResOperation, rop
 from pypy.jit.metainterp.typesystem import deref
+from pypy.jit.codewriter.effectinfo import EffectInfo
 from pypy.jit.tool.oparser import parse
 from pypy.rpython.lltypesystem import lltype, llmemory, rstr, rffi, rclass
 from pypy.rpython.ootypesystem import ootype
@@ -96,7 +97,8 @@
             FUNC = self.FuncType(funcargs, F)
             FPTR = self.Ptr(FUNC)
             func_ptr = llhelper(FPTR, func)
-            calldescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
+            calldescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
+                                        EffectInfo.MOST_GENERAL)
             funcbox = self.get_funcbox(cpu, func_ptr)
 
             ops = '[%s]\n' % arguments
@@ -148,7 +150,8 @@
             FUNC = self.FuncType(args, F)
             FPTR = self.Ptr(FUNC)
             func_ptr = llhelper(FPTR, func)
-            calldescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
+            calldescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
+                                        EffectInfo.MOST_GENERAL)
             funcbox = self.get_funcbox(cpu, func_ptr)
 
             res = self.execute_operation(rop.CALL,
@@ -190,7 +193,8 @@
             FUNC = self.FuncType(args, F)
             FPTR = self.Ptr(FUNC)
             func_ptr = llhelper(FPTR, func)
-            calldescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
+            calldescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
+                                        EffectInfo.MOST_GENERAL)
             funcbox = self.get_funcbox(cpu, func_ptr)
 
             res = self.execute_operation(rop.CALL,
@@ -268,7 +272,8 @@
                 else:
                     ARGS.append(lltype.Signed)
             FakeJitDriverSD.portal_calldescr = self.cpu.calldescrof(
-                lltype.Ptr(lltype.FuncType(ARGS, RES)), ARGS, RES)
+                lltype.Ptr(lltype.FuncType(ARGS, RES)), ARGS, RES,
+                EffectInfo.MOST_GENERAL)
             ops = '''
             [%s]
             f99 = call_assembler(%s, descr=called_looptoken)
@@ -290,3 +295,59 @@
                 assert abs(x - expected_result) < 0.0001
             finally:
                 del self.cpu.done_with_this_frame_float_v
+
+    def test_call_with_singlefloats(self):
+        cpu = self.cpu
+        if not cpu.supports_floats or not cpu.supports_singlefloats:
+            py.test.skip('requires floats and singlefloats')
+
+        import random
+        from pypy.rlib.libffi import types
+        from pypy.rlib.rarithmetic import r_singlefloat
+
+        def func(*args):
+            res = 0.0
+            for i, x in enumerate(args):
+                res += (i + 1.1) * float(x)
+            return res
+
+        F = lltype.Float
+        S = lltype.SingleFloat
+        I = lltype.Signed
+        floats = [random.random() - 0.5 for i in range(8)]
+        singlefloats = [r_singlefloat(random.random() - 0.5) for i in range(8)]
+        ints = [random.randrange(-99, 99) for i in range(8)]
+        for repeat in range(100):
+            args = []
+            argvalues = []
+            argslist = []
+            local_floats = list(floats)
+            local_singlefloats = list(singlefloats)
+            local_ints = list(ints)
+            for i in range(8):
+                case = random.randrange(0, 3)
+                if case == 0:
+                    args.append(F)
+                    arg = local_floats.pop()
+                    argslist.append(boxfloat(arg))
+                elif case == 1:
+                    args.append(S)
+                    arg = local_singlefloats.pop()
+                    argslist.append(BoxInt(longlong.singlefloat2int(arg)))
+                else:
+                    args.append(I)
+                    arg = local_ints.pop()
+                    argslist.append(BoxInt(arg))
+                argvalues.append(arg)
+            FUNC = self.FuncType(args, F)
+            FPTR = self.Ptr(FUNC)
+            func_ptr = llhelper(FPTR, func)
+            calldescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
+                                        EffectInfo.MOST_GENERAL)
+            funcbox = self.get_funcbox(cpu, func_ptr)
+
+            res = self.execute_operation(rop.CALL,
+                                         [funcbox] + argslist,
+                                         'float', descr=calldescr)
+            expected = func(*argvalues)
+            assert abs(res.getfloat() - expected) < 0.0001
diff --git a/pypy/jit/backend/test/runner_test.py b/pypy/jit/backend/test/runner_test.py
--- a/pypy/jit/backend/test/runner_test.py
+++ b/pypy/jit/backend/test/runner_test.py
@@ -9,6 +9,7 @@
                                          ConstObj, BoxFloat, ConstFloat)
 from pypy.jit.metainterp.resoperation import ResOperation, rop
 from pypy.jit.metainterp.typesystem import deref
+from pypy.jit.codewriter.effectinfo import EffectInfo
 from pypy.jit.tool.oparser import parse
 from pypy.rpython.lltypesystem import lltype, llmemory, rstr, rffi, rclass
 from pypy.rpython.ootypesystem import ootype
@@ -445,7 +446,8 @@
             return chr(ord(c) + 1)
         FPTR = self.Ptr(self.FuncType([lltype.Char], lltype.Char))
         func_ptr = llhelper(FPTR, func)
-        calldescr = cpu.calldescrof(deref(FPTR), (lltype.Char,), lltype.Char)
+        calldescr = cpu.calldescrof(deref(FPTR), (lltype.Char,), lltype.Char,
+                                    EffectInfo.MOST_GENERAL)
         x = cpu.bh_call_i(self.get_funcbox(cpu, func_ptr).value,
                           calldescr, [ord('A')], None, None)
         assert x == ord('B')
@@ -458,14 +460,15 @@
                                           lltype.Float))
             func_ptr = llhelper(FPTR, func)
             FTP = deref(FPTR)
-            calldescr = cpu.calldescrof(FTP, FTP.ARGS, FTP.RESULT)
+            calldescr = cpu.calldescrof(FTP, FTP.ARGS, FTP.RESULT,
+                                        EffectInfo.MOST_GENERAL)
             x = cpu.bh_call_f(self.get_funcbox(cpu, func_ptr).value,
                               calldescr,
                               [42], None, [longlong.getfloatstorage(3.5)])
             assert longlong.getrealfloat(x) == 3.5 - 42
 
     def test_call(self):
-        from pypy.rlib.libffi import types
+        from pypy.rlib.libffi import types, FUNCFLAG_CDECL
 
         def func_int(a, b):
             return a + b
@@ -486,13 +489,16 @@
             FUNC = deref(FPTR)
             funcbox = self.get_funcbox(cpu, func_ptr)
             # first, try it with the "normal" calldescr
-            calldescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
+            calldescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
+                                        EffectInfo.MOST_GENERAL)
             res = self.execute_operation(rop.CALL,
                                          [funcbox, BoxInt(num), BoxInt(num)],
                                          'int', descr=calldescr)
             assert res.value == 2 * num
             # then, try it with the dynamic calldescr
-            dyn_calldescr = cpu.calldescrof_dynamic([ffi_type, ffi_type], ffi_type)
+            dyn_calldescr = cpu.calldescrof_dynamic([ffi_type, ffi_type], ffi_type,
+                                                    EffectInfo.MOST_GENERAL,
+                                                    ffi_flags=FUNCFLAG_CDECL)
             res = self.execute_operation(rop.CALL,
                                          [funcbox, BoxInt(num), BoxInt(num)],
                                          'int', descr=dyn_calldescr)
@@ -507,7 +513,8 @@
             FUNC = self.FuncType([F] * 7 + [I] * 2 + [F] * 3, F)
             FPTR = self.Ptr(FUNC)
             func_ptr = llhelper(FPTR, func)
-            calldescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
+            calldescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
+                                        EffectInfo.MOST_GENERAL)
             funcbox = self.get_funcbox(cpu, func_ptr)
             args = ([boxfloat(.1) for i in range(7)] +
                     [BoxInt(1), BoxInt(2), boxfloat(.2), boxfloat(.3),
@@ -529,7 +536,8 @@
 
         FUNC = self.FuncType([lltype.Signed]*16, lltype.Signed)
         FPTR = self.Ptr(FUNC)
-        calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
+        calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
+                                         EffectInfo.MOST_GENERAL)
         func_ptr = llhelper(FPTR, func)
         args = range(16)
         funcbox = self.get_funcbox(self.cpu, func_ptr)
@@ -552,7 +560,8 @@
             FPTR = self.Ptr(self.FuncType([TP] * nb_args, TP))
             func_ptr = llhelper(FPTR, func_ints)
             FUNC = deref(FPTR)
-            calldescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
+            calldescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
+                                        EffectInfo.MOST_GENERAL)
             funcbox = self.get_funcbox(cpu, func_ptr)
             args = [280-24*i for i in range(nb_args)]
             res = self.execute_operation(rop.CALL,
@@ -566,7 +575,8 @@
 
         FUNC = self.FuncType([lltype.Float, lltype.Float], lltype.Float)
         FPTR = self.Ptr(FUNC)
-        calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
+        calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
+                                         EffectInfo.MOST_GENERAL)
         func_ptr = llhelper(FPTR, func)
         funcbox = self.get_funcbox(self.cpu, func_ptr)
         res = self.execute_operation(rop.CALL, [funcbox, constfloat(1.5),
@@ -1627,7 +1637,8 @@
         '''
         FPTR = lltype.Ptr(lltype.FuncType([lltype.Signed], lltype.Void))
         fptr = llhelper(FPTR, func)
-        calldescr = self.cpu.calldescrof(FPTR.TO, FPTR.TO.ARGS, FPTR.TO.RESULT)
+        calldescr = self.cpu.calldescrof(FPTR.TO, FPTR.TO.ARGS, FPTR.TO.RESULT,
+                                         EffectInfo.MOST_GENERAL)
 
         xtp = lltype.malloc(rclass.OBJECT_VTABLE, immortal=True)
         xtp.subclassrange_min = 1
@@ -1845,7 +1856,8 @@
         FUNC = self.FuncType([lltype.Signed, lltype.Signed], lltype.Void)
         func_ptr = llhelper(lltype.Ptr(FUNC), maybe_force)
         funcbox = self.get_funcbox(self.cpu, func_ptr).constbox()
-        calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
+        calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
+                                         EffectInfo.MOST_GENERAL)
         cpu = self.cpu
         i0 = BoxInt()
         i1 = BoxInt()
@@ -1888,7 +1900,8 @@
         FUNC = self.FuncType([lltype.Signed, lltype.Signed], lltype.Signed)
         func_ptr = llhelper(lltype.Ptr(FUNC), maybe_force)
         funcbox = self.get_funcbox(self.cpu, func_ptr).constbox()
-        calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
+        calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
+                                         EffectInfo.MOST_GENERAL)
         cpu = self.cpu
         i0 = BoxInt()
         i1 = BoxInt()
@@ -1933,7 +1946,8 @@
         FUNC = self.FuncType([lltype.Signed, lltype.Signed], lltype.Float)
         func_ptr = llhelper(lltype.Ptr(FUNC), maybe_force)
         funcbox = self.get_funcbox(self.cpu, func_ptr).constbox()
-        calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
+        calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
+                                         EffectInfo.MOST_GENERAL)
         cpu = self.cpu
         i0 = BoxInt()
         i1 = BoxInt()
@@ -1969,7 +1983,7 @@
         assert values == [1, 10]
 
     def test_call_to_c_function(self):
-        from pypy.rlib.libffi import CDLL, types, ArgChain
+        from pypy.rlib.libffi import CDLL, types, ArgChain, FUNCFLAG_CDECL
         from pypy.rpython.lltypesystem.ll2ctypes import libc_name
         libc = CDLL(libc_name)
         c_tolower = libc.getpointer('tolower', [types.uchar], types.sint)
@@ -1979,7 +1993,9 @@
         cpu = self.cpu
         func_adr = llmemory.cast_ptr_to_adr(c_tolower.funcsym)
         funcbox = ConstInt(heaptracker.adr2int(func_adr))
-        calldescr = cpu.calldescrof_dynamic([types.uchar], types.sint)
+        calldescr = cpu.calldescrof_dynamic([types.uchar], types.sint,
+                                            EffectInfo.MOST_GENERAL,
+                                            ffi_flags=FUNCFLAG_CDECL)
         i1 = BoxInt()
         i2 = BoxInt()
         tok = BoxInt()
@@ -2035,7 +2051,9 @@
         funcbox = ConstInt(heaptracker.adr2int(func_adr))
         calldescr = cpu.calldescrof_dynamic([types.pointer, types_size_t,
                                              types_size_t, types.pointer],
-                                            types.void)
+                                            types.void,
+                                            EffectInfo.MOST_GENERAL,
+                                            ffi_flags=clibffi.FUNCFLAG_CDECL)
         i0 = BoxInt()
         i1 = BoxInt()
         i2 = BoxInt()
@@ -2061,6 +2079,62 @@
         assert len(glob.lst) > 0
         lltype.free(raw, flavor='raw')
 
+    def test_call_to_winapi_function(self):
+        from pypy.rlib.clibffi import _WIN32, FUNCFLAG_STDCALL
+        if not _WIN32:
+            py.test.skip("Windows test only")
+        from pypy.rlib.libffi import CDLL, types, ArgChain
+        from pypy.rlib.rwin32 import DWORD
+        libc = CDLL('KERNEL32')
+        c_GetCurrentDir = libc.getpointer('GetCurrentDirectoryA',
+                                          [types.ulong, types.pointer],
+                                          types.ulong)
+
+        cwd = os.getcwd()
+        buflen = len(cwd) + 10
+        buffer = lltype.malloc(rffi.CCHARP.TO, buflen, flavor='raw')
+        argchain = ArgChain().arg(rffi.cast(DWORD, buflen)).arg(buffer)
+        res = c_GetCurrentDir.call(argchain, DWORD)
+        assert rffi.cast(lltype.Signed, res) == len(cwd)
+        assert rffi.charp2strn(buffer, buflen) == cwd
+        lltype.free(buffer, flavor='raw')
+
+        cpu = self.cpu
+        func_adr = llmemory.cast_ptr_to_adr(c_GetCurrentDir.funcsym)
+        funcbox = ConstInt(heaptracker.adr2int(func_adr))
+        calldescr = cpu.calldescrof_dynamic([types.ulong, types.pointer],
+                                            types.ulong,
+                                            EffectInfo.MOST_GENERAL,
+                                            ffi_flags=FUNCFLAG_STDCALL)
+        i1 = BoxInt()
+        i2 = BoxInt()
+        faildescr = BasicFailDescr(1)
+        # if the stdcall convention is ignored, then ESP is wrong after the
+        # call: 8 bytes too much.  If we repeat the call often enough, crash.
+        ops = []
+        for i in range(50):
+            i3 = BoxInt()
+            ops += [
+                ResOperation(rop.CALL_RELEASE_GIL, [funcbox, i1, i2], i3,
+                             descr=calldescr),
+                ResOperation(rop.GUARD_NOT_FORCED, [], None, descr=faildescr),
+                ]
+            ops[-1].setfailargs([])
+        ops += [
+            ResOperation(rop.FINISH, [i3], None, descr=BasicFailDescr(0))
+        ]
+        looptoken = LoopToken()
+        self.cpu.compile_loop([i1, i2], ops, looptoken)
+
+        buffer = lltype.malloc(rffi.CCHARP.TO, buflen, flavor='raw')
+        self.cpu.set_future_value_int(0, buflen)
+        self.cpu.set_future_value_int(1, rffi.cast(lltype.Signed, buffer))
+        fail = self.cpu.execute_token(looptoken)
+        assert fail.identifier == 0
+        assert self.cpu.get_latest_value_int(0) == len(cwd)
+        assert rffi.charp2strn(buffer, buflen) == cwd
+        lltype.free(buffer, flavor='raw')
+
     def test_guard_not_invalidated(self):
         cpu = self.cpu
         i0 = BoxInt()
@@ -2330,7 +2404,8 @@
         ARGS = [lltype.Signed] * 10
         RES = lltype.Signed
         FakeJitDriverSD.portal_calldescr = self.cpu.calldescrof(
-            lltype.Ptr(lltype.FuncType(ARGS, RES)), ARGS, RES)
+            lltype.Ptr(lltype.FuncType(ARGS, RES)), ARGS, RES,
+            EffectInfo.MOST_GENERAL)
         for i in range(10):
             self.cpu.set_future_value_int(i, i+1)
         res = self.cpu.execute_token(looptoken)
@@ -2370,7 +2445,8 @@
         ARGS = [lltype.Float, lltype.Float]
         RES = lltype.Float
         FakeJitDriverSD.portal_calldescr = self.cpu.calldescrof(
-            lltype.Ptr(lltype.FuncType(ARGS, RES)), ARGS, RES)
+            lltype.Ptr(lltype.FuncType(ARGS, RES)), ARGS, RES,
+            EffectInfo.MOST_GENERAL)
         
         ops = '''
         [f0, f1]
@@ -2460,7 +2536,8 @@
         ARGS = [lltype.Float, lltype.Float]
         RES = lltype.Float
         FakeJitDriverSD.portal_calldescr = self.cpu.calldescrof(
-            lltype.Ptr(lltype.FuncType(ARGS, RES)), ARGS, RES)
+            lltype.Ptr(lltype.FuncType(ARGS, RES)), ARGS, RES,
+            EffectInfo.MOST_GENERAL)
         
         ops = '''
         [f0, f1]
@@ -2672,7 +2749,8 @@
             #
             FUNC = self.FuncType([lltype.Signed], RESTYPE)
             FPTR = self.Ptr(FUNC)
-            calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
+            calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
+                                             EffectInfo.MOST_GENERAL)
             x = self.cpu.bh_call_i(self.get_funcbox(self.cpu, f).value,
                                    calldescr, [value], None, None)
             assert x == expected, (
@@ -2705,7 +2783,8 @@
             #
             FUNC = self.FuncType([lltype.Signed], RESTYPE)
             FPTR = self.Ptr(FUNC)
-            calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
+            calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
+                                             EffectInfo.MOST_GENERAL)
             funcbox = self.get_funcbox(self.cpu, f)
             res = self.execute_operation(rop.CALL, [funcbox, BoxInt(value)],
                                          'int', descr=calldescr)
@@ -2739,7 +2818,8 @@
         #
         FUNC = self.FuncType([lltype.SignedLongLong], lltype.SignedLongLong)
         FPTR = self.Ptr(FUNC)
-        calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
+        calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
+                                         EffectInfo.MOST_GENERAL)
         x = self.cpu.bh_call_f(self.get_funcbox(self.cpu, f).value,
                                calldescr, None, None, [value])
         assert x == expected
@@ -2766,12 +2846,74 @@
         #
         FUNC = self.FuncType([lltype.SignedLongLong], lltype.SignedLongLong)
         FPTR = self.Ptr(FUNC)
-        calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
+        calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
+                                         EffectInfo.MOST_GENERAL)
         funcbox = self.get_funcbox(self.cpu, f)
         res = self.execute_operation(rop.CALL, [funcbox, BoxFloat(value)],
                                      'float', descr=calldescr)
         assert res.getfloatstorage() == expected
 
+    def test_singlefloat_result_of_call_direct(self):
+        if not self.cpu.supports_singlefloats:
+            py.test.skip("singlefloat test")
+        from pypy.translator.tool.cbuild import ExternalCompilationInfo
+        from pypy.rlib.rarithmetic import r_singlefloat
+        eci = ExternalCompilationInfo(
+            separate_module_sources=["""
+            float fn_test_result_of_call(float x)
+            {
+                return x / 2.0f;
+            }
+            """],
+            export_symbols=['fn_test_result_of_call'])
+        f = rffi.llexternal('fn_test_result_of_call', [lltype.SingleFloat],
+                            lltype.SingleFloat,
+                            compilation_info=eci, _nowrapper=True)
+        value = r_singlefloat(-42.5)
+        expected = r_singlefloat(-21.25)
+        assert f(value) == expected
+        #
+        FUNC = self.FuncType([lltype.SingleFloat], lltype.SingleFloat)
+        FPTR = self.Ptr(FUNC)
+        calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
+                                         EffectInfo.MOST_GENERAL)
+        ivalue = longlong.singlefloat2int(value)
+        iexpected = longlong.singlefloat2int(expected)
+        x = self.cpu.bh_call_i(self.get_funcbox(self.cpu, f).value,
+                               calldescr, [ivalue], None, None)
+        assert x == iexpected
+
+    def test_singlefloat_result_of_call_compiled(self):
+        if not self.cpu.supports_singlefloats:
+            py.test.skip("test of singlefloat result")
+        from pypy.translator.tool.cbuild import ExternalCompilationInfo
+        from pypy.rlib.rarithmetic import r_singlefloat
+        eci = ExternalCompilationInfo(
+            separate_module_sources=["""
+            float fn_test_result_of_call(float x)
+            {
+                return x / 2.0f;
+            }
+            """],
+            export_symbols=['fn_test_result_of_call'])
+        f = rffi.llexternal('fn_test_result_of_call', [lltype.SingleFloat],
+                            lltype.SingleFloat,
+                            compilation_info=eci, _nowrapper=True)
+        value = r_singlefloat(-42.5)
+        expected = r_singlefloat(-21.25)
+        assert f(value) == expected
+        #
+        FUNC = self.FuncType([lltype.SingleFloat], lltype.SingleFloat)
+        FPTR = self.Ptr(FUNC)
+        calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
+                                         EffectInfo.MOST_GENERAL)
+        funcbox = self.get_funcbox(self.cpu, f)
+        ivalue = longlong.singlefloat2int(value)
+        iexpected = longlong.singlefloat2int(expected)
+        res = self.execute_operation(rop.CALL, [funcbox, BoxInt(ivalue)],
+                                     'int', descr=calldescr)
+        assert res.value == iexpected
+
     def test_free_loop_and_bridges(self):
         from pypy.jit.backend.llsupport.llmodel import AbstractLLCPU
         if not isinstance(self.cpu, AbstractLLCPU):
@@ -2786,6 +2928,26 @@
         assert mem2 < mem1
         assert mem2 == mem0
 
+    def test_memoryerror(self):
+        excdescr = BasicFailDescr(666)
+        self.cpu.propagate_exception_v = self.cpu.get_fail_descr_number(
+            excdescr)
+        self.cpu.setup_once()    # xxx redo it, because we added
+                                 # propagate_exception_v
+        i0 = BoxInt()
+        p0 = BoxPtr()
+        operations = [
+            ResOperation(rop.NEWUNICODE, [i0], p0),
+            ResOperation(rop.FINISH, [p0], None, descr=BasicFailDescr(1))
+            ]
+        inputargs = [i0]
+        looptoken = LoopToken()
+        self.cpu.compile_loop(inputargs, operations, looptoken)
+        # overflowing value:
+        self.cpu.set_future_value_int(0, sys.maxint // 4 + 1)
+        fail = self.cpu.execute_token(looptoken)
+        assert fail.identifier == excdescr.identifier
+
 
 class OOtypeBackendTest(BaseBackendTest):
 
diff --git a/pypy/jit/backend/test/test_ll_random.py b/pypy/jit/backend/test/test_ll_random.py
--- a/pypy/jit/backend/test/test_ll_random.py
+++ b/pypy/jit/backend/test/test_ll_random.py
@@ -6,6 +6,7 @@
 from pypy.jit.metainterp.history import BoxPtr, BoxInt
 from pypy.jit.metainterp.history import BasicFailDescr
 from pypy.jit.codewriter import heaptracker
+from pypy.jit.codewriter.effectinfo import EffectInfo
 from pypy.rpython.annlowlevel import llhelper
 from pypy.rlib.rarithmetic import intmask
 from pypy.rpython.llinterp import LLException
@@ -468,6 +469,10 @@
         exec code in d
         return subset, d['f'], vtableptr
 
+    def getcalldescr(self, builder, TP):
+        ef = EffectInfo.MOST_GENERAL
+        return builder.cpu.calldescrof(TP, TP.ARGS, TP.RESULT, ef)
+
 # 1. non raising call and guard_no_exception
 class CallOperation(BaseCallOperation):
     def produce_into(self, builder, r):
@@ -481,7 +486,7 @@
         ptr = llhelper(lltype.Ptr(TP), f)
         c_addr = ConstAddr(llmemory.cast_ptr_to_adr(ptr), builder.cpu)
         args = [c_addr] + subset
-        descr = builder.cpu.calldescrof(TP, TP.ARGS, TP.RESULT)
+        descr = self.getcalldescr(builder, TP)
         self.put(builder, args, descr)
         op = ResOperation(rop.GUARD_NO_EXCEPTION, [], None,
                           descr=BasicFailDescr())
@@ -501,7 +506,7 @@
         ptr = llhelper(lltype.Ptr(TP), f)
         c_addr = ConstAddr(llmemory.cast_ptr_to_adr(ptr), builder.cpu)
         args = [c_addr] + subset
-        descr = builder.cpu.calldescrof(TP, TP.ARGS, TP.RESULT)
+        descr = self.getcalldescr(builder, TP)
         self.put(builder, args, descr)
         _, vtableptr = builder.get_random_structure_type_and_vtable(r)
         exc_box = ConstAddr(llmemory.cast_ptr_to_adr(vtableptr), builder.cpu)
@@ -523,7 +528,7 @@
         ptr = llhelper(lltype.Ptr(TP), f)
         c_addr = ConstAddr(llmemory.cast_ptr_to_adr(ptr), builder.cpu)
         args = [c_addr] + subset
-        descr = builder.cpu.calldescrof(TP, TP.ARGS, TP.RESULT)
+        descr = self.getcalldescr(builder, TP)
         self.put(builder, args, descr)
         exc_box = ConstAddr(llmemory.cast_ptr_to_adr(exc), builder.cpu)
         op = ResOperation(rop.GUARD_EXCEPTION, [exc_box], BoxPtr(),
@@ -540,7 +545,7 @@
         ptr = llhelper(lltype.Ptr(TP), f)
         c_addr = ConstAddr(llmemory.cast_ptr_to_adr(ptr), builder.cpu)
         args = [c_addr] + subset
-        descr = builder.cpu.calldescrof(TP, TP.ARGS, TP.RESULT)
+        descr = self.getcalldescr(builder, TP)
         self.put(builder, args, descr)
         op = ResOperation(rop.GUARD_NO_EXCEPTION, [], BoxPtr(),
                           descr=BasicFailDescr())
@@ -559,7 +564,7 @@
         ptr = llhelper(lltype.Ptr(TP), f)
         c_addr = ConstAddr(llmemory.cast_ptr_to_adr(ptr), builder.cpu)
         args = [c_addr] + subset
-        descr = builder.cpu.calldescrof(TP, TP.ARGS, TP.RESULT)
+        descr = self.getcalldescr(builder, TP)
         self.put(builder, args, descr)
         while True:
             _, vtableptr = builder.get_random_structure_type_and_vtable(r)
diff --git a/pypy/jit/backend/x86/assembler.py b/pypy/jit/backend/x86/assembler.py
--- a/pypy/jit/backend/x86/assembler.py
+++ b/pypy/jit/backend/x86/assembler.py
@@ -34,6 +34,7 @@
 from pypy.rlib.debug import (debug_print, debug_start, debug_stop,
                              have_debug_prints)
 from pypy.rlib import rgc
+from pypy.rlib.clibffi import FFI_DEFAULT_ABI
 from pypy.jit.backend.x86.jump import remap_frame_layout
 from pypy.jit.codewriter.effectinfo import EffectInfo
 from pypy.jit.codewriter import longlong
@@ -55,7 +56,9 @@
         self.exc = exc
         self.is_guard_not_invalidated = is_guard_not_invalidated
 
-DEBUG_COUNTER = lltype.Struct('DEBUG_COUNTER', ('i', lltype.Signed))
+DEBUG_COUNTER = lltype.Struct('DEBUG_COUNTER', ('i', lltype.Signed),
+                              ('bridge', lltype.Signed), # 0 or 1
+                              ('number', lltype.Signed))
 
 class Assembler386(object):
     _regalloc = None
@@ -88,6 +91,8 @@
         self._current_depths_cache = (0, 0)
         self.datablockwrapper = None
         self.stack_check_slowpath = 0
+        self.propagate_exception_path = 0
+        self.gcrootmap_retaddr_forced = 0
         self.teardown()
 
     def leave_jitted_hook(self):
@@ -124,6 +129,7 @@
             self._build_failure_recovery(True, withfloats=True)
             support.ensure_sse2_floats()
             self._build_float_constants()
+        self._build_propagate_exception_path()
         if gc_ll_descr.get_malloc_slowpath_addr is not None:
             self._build_malloc_slowpath()
         self._build_stack_check_slowpath()
@@ -137,6 +143,9 @@
         assert self.memcpy_addr != 0, "setup_once() not called?"
         self.current_clt = looptoken.compiled_loop_token
         self.pending_guard_tokens = []
+        if WORD == 8:
+            self.pending_memoryerror_trampoline_from = []
+            self.error_trampoline_64 = 0
         self.mc = codebuf.MachineCodeBlockWrapper()
         #assert self.datablockwrapper is None --- but obscure case
         # possible, e.g. getting MemoryError and continuing
@@ -146,6 +155,8 @@
 
     def teardown(self):
         self.pending_guard_tokens = None
+        if WORD == 8:
+            self.pending_memoryerror_trampoline_from = None
         self.mc = None
         self.looppos = -1
         self.currently_compiling_loop = None
@@ -154,9 +165,12 @@
     def finish_once(self):
         if self._debug:
             debug_start('jit-backend-counts')
-            for i in range(len(self.loop_run_counters)):
-                struct = self.loop_run_counters[i]
-                debug_print(str(i) + ':' + str(struct.i))
+            for struct in self.loop_run_counters:
+                if struct.bridge:
+                    prefix = 'bridge '
+                else:
+                    prefix = 'loop '
+                debug_print(prefix + str(struct.number) + ':' + str(struct.i))
             debug_stop('jit-backend-counts')
 
     def _build_float_constants(self):
@@ -231,15 +245,47 @@
         if self.cpu.supports_floats:          # restore the XMM registers
             for i in range(self.cpu.NUM_REGS):# from where they were saved
                 mc.MOVSD_xs(i, (WORD*2)+8*i)
+        #
+        # Note: we check this after the code above, just because the code
+        # above is more than 127 bytes on 64-bits...
+        mc.TEST_rr(eax.value, eax.value)
+        mc.J_il8(rx86.Conditions['Z'], 0) # patched later
+        jz_location = mc.get_relative_pos()
+        #
         nursery_free_adr = self.cpu.gc_ll_descr.get_nursery_free_addr()
         mc.MOV(edx, heap(nursery_free_adr))   # load this in EDX
         mc.RET()
+        #
+        # If the slowpath malloc failed, we raise a MemoryError that
+        # always interrupts the current loop, as a "good enough"
+        # approximation.  Also note that we didn't RET from this helper;
+        # but the code we jump to will actually restore the stack
+        # position based on EBP, which will get us out of here for free.
+        offset = mc.get_relative_pos() - jz_location
+        assert 0 < offset <= 127
+        mc.overwrite(jz_location-1, chr(offset))
+        mc.JMP(imm(self.propagate_exception_path))
+        #
         rawstart = mc.materialize(self.cpu.asmmemmgr, [])
         self.malloc_slowpath2 = rawstart
 
+    def _build_propagate_exception_path(self):
+        if self.cpu.propagate_exception_v < 0:
+            return      # not supported (for tests, or non-translated)
+        #
+        self.mc = codebuf.MachineCodeBlockWrapper()
+        # call on_leave_jitted_save_exc()
+        addr = self.cpu.get_on_leave_jitted_int(save_exception=True)
+        self.mc.CALL(imm(addr))
+        self.mc.MOV_ri(eax.value, self.cpu.propagate_exception_v)
+        self._call_footer()
+        rawstart = self.mc.materialize(self.cpu.asmmemmgr, [])
+        self.propagate_exception_path = rawstart
+        self.mc = None
+
     def _build_stack_check_slowpath(self):
         _, _, slowpathaddr = self.cpu.insert_stack_check()
-        if slowpathaddr == 0 or self.cpu.exit_frame_with_exception_v < 0:
+        if slowpathaddr == 0 or self.cpu.propagate_exception_v < 0:
             return      # no stack check (for tests, or non-translated)
         #
         # make a "function" that is called immediately at the start of
@@ -295,19 +341,11 @@
         offset = mc.get_relative_pos() - jnz_location
         assert 0 < offset <= 127
         mc.overwrite(jnz_location-1, chr(offset))
-        # clear the exception from the global position
-        mc.MOV(eax, heap(self.cpu.pos_exc_value()))
-        mc.MOV(heap(self.cpu.pos_exception()), imm0)
-        mc.MOV(heap(self.cpu.pos_exc_value()), imm0)
-        # save the current exception instance into fail_boxes_ptr[0]
-        adr = self.fail_boxes_ptr.get_addr_for_num(0)
-        mc.MOV(heap(adr), eax)
-        # call the helper function to set the GC flag on the fail_boxes_ptr
-        # array (note that there is no exception any more here)
-        addr = self.cpu.get_on_leave_jitted_int(save_exception=False)
+        # call on_leave_jitted_save_exc()
+        addr = self.cpu.get_on_leave_jitted_int(save_exception=True)
         mc.CALL(imm(addr))
         #
-        mc.MOV_ri(eax.value, self.cpu.exit_frame_with_exception_v)
+        mc.MOV_ri(eax.value, self.cpu.propagate_exception_v)
         #
         # footer -- note the ADD, which skips the return address of this
         # function, and will instead return to the caller's caller.  Note
@@ -320,6 +358,7 @@
         self.stack_check_slowpath = rawstart
 
     @staticmethod
+    @rgc.no_collect
     def _release_gil_asmgcc(css):
         # similar to trackgcroot.py:pypy_asm_stackwalk, first part
         from pypy.rpython.memory.gctransform import asmgcroot
@@ -335,6 +374,7 @@
             before()
 
     @staticmethod
+    @rgc.no_collect
     def _reacquire_gil_asmgcc(css):
         # first reacquire the GIL
         after = rffi.aroundstate.after
@@ -349,12 +389,14 @@
         next.prev = prev
 
     @staticmethod
+    @rgc.no_collect
     def _release_gil_shadowstack():
         before = rffi.aroundstate.before
         if before:
             before()
 
     @staticmethod
+    @rgc.no_collect
     def _reacquire_gil_shadowstack():
         after = rffi.aroundstate.after
         if after:
@@ -403,7 +445,7 @@
         self.setup(looptoken)
         self.currently_compiling_loop = looptoken
         if log:
-            self._register_counter()
+            self._register_counter(False, looptoken.number)
             operations = self._inject_debugging_code(looptoken, operations)
 
         regalloc = RegAlloc(self, self.cpu.translate_support_code)
@@ -472,7 +514,7 @@
 
         self.setup(original_loop_token)
         if log:
-            self._register_counter()
+            self._register_counter(True, descr_number)
             operations = self._inject_debugging_code(faildescr, operations)
 
         arglocs = self.rebuild_faillocs_from_descr(failure_recovery)
@@ -519,6 +561,8 @@
         # at the end of self.mc.
         for tok in self.pending_guard_tokens:
             tok.pos_recovery_stub = self.generate_quick_failure(tok)
+        if WORD == 8 and len(self.pending_memoryerror_trampoline_from) > 0:
+            self.error_trampoline_64 = self.generate_propagate_error_64()
 
     def patch_pending_failure_recoveries(self, rawstart):
         # after we wrote the assembler to raw memory, set up
@@ -555,6 +599,12 @@
                 # less, we would run into the issue that overwriting the
                 # 5 bytes here might get a few nonsense bytes at the
                 # return address of the following CALL.
+        if WORD == 8:
+            for pos_after_jz in self.pending_memoryerror_trampoline_from:
+                assert self.error_trampoline_64 != 0     # only if non-empty
+                mc = codebuf.MachineCodeBlockWrapper()
+                mc.writeimm32(self.error_trampoline_64 - pos_after_jz)
+                mc.copy_to_raw_memory(rawstart + pos_after_jz - 4)
 
     def get_asmmemmgr_blocks(self, looptoken):
         clt = looptoken.compiled_loop_token
@@ -569,7 +619,7 @@
         return self.mc.materialize(self.cpu.asmmemmgr, allblocks,
                                    self.cpu.gc_ll_descr.gcrootmap)
 
-    def _register_counter(self):
+    def _register_counter(self, bridge, number):
         if self._debug:
             # YYY very minor leak -- we need the counters to stay alive
             # forever, just because we want to report them at the end
@@ -577,6 +627,8 @@
             struct = lltype.malloc(DEBUG_COUNTER, flavor='raw',
                                    track_allocation=False)
             struct.i = 0
+            struct.bridge = int(bridge)
+            struct.number = number
             self.loop_run_counters.append(struct)
 
     def _find_failure_recovery_bytecode(self, faildescr):
@@ -1067,9 +1119,10 @@
                     self.implement_guard(guard_token, checkfalsecond)
         return genop_cmp_guard_float
 
-    def _emit_call(self, force_index, x, arglocs, start=0, tmp=eax):
+    def _emit_call(self, force_index, x, arglocs, start=0, tmp=eax,
+                   argtypes=None, callconv=FFI_DEFAULT_ABI):
         if IS_X86_64:
-            return self._emit_call_64(force_index, x, arglocs, start)
+            return self._emit_call_64(force_index, x, arglocs, start, argtypes)
 
         p = 0
         n = len(arglocs)
@@ -1096,13 +1149,24 @@
         # x is a location
         self.mc.CALL(x)
         self.mark_gc_roots(force_index)
+        #
+        if callconv != FFI_DEFAULT_ABI:
+            self._fix_stdcall(callconv, p)
 
-    def _emit_call_64(self, force_index, x, arglocs, start):
+    def _fix_stdcall(self, callconv, p):
+        from pypy.rlib.clibffi import FFI_STDCALL
+        assert callconv == FFI_STDCALL
+        # it's a bit stupid, but we're just going to cancel the fact that
+        # the called function just added 'p' to ESP, by subtracting it again.
+        self.mc.SUB_ri(esp.value, p)
+
+    def _emit_call_64(self, force_index, x, arglocs, start, argtypes):
         src_locs = []
         dst_locs = []
         xmm_src_locs = []
         xmm_dst_locs = []
         pass_on_stack = []
+        singlefloats = None
 
         # In reverse order for use with pop()
         unused_gpr = [r9, r8, ecx, edx, esi, edi]
@@ -1122,6 +1186,11 @@
                     xmm_dst_locs.append(unused_xmm.pop())
                 else:
                     pass_on_stack.append(loc)
+            elif (argtypes is not None and argtypes[i-start] == 'S' and
+                  len(unused_xmm) > 0):
+                # Singlefloat argument
+                if singlefloats is None: singlefloats = []
+                singlefloats.append((loc, unused_xmm.pop()))
             else:
                 if len(unused_gpr) > 0:
                     src_locs.append(loc)
@@ -1149,9 +1218,15 @@
                 else:
                     self.mc.MOV_sr(i*WORD, loc.value)
 
-        # Handle register arguments
+        # Handle register arguments: first remap the xmm arguments
+        remap_frame_layout(self, xmm_src_locs, xmm_dst_locs,
+                           X86_64_XMM_SCRATCH_REG)
+        # Load the singlefloat arguments from main regs or stack to xmm regs
+        if singlefloats is not None:
+            for src, dst in singlefloats:
+                self.mc.MOVD(dst, src)
+        # Finally remap the arguments in the main regs
         remap_frame_layout(self, src_locs, dst_locs, X86_64_SCRATCH_REG)
-        remap_frame_layout(self, xmm_src_locs, xmm_dst_locs, X86_64_XMM_SCRATCH_REG)
 
         self._regalloc.reserve_param(len(pass_on_stack))
         self.mc.CALL(x)
@@ -1266,6 +1341,20 @@
     def genop_cast_int_to_float(self, op, arglocs, resloc):
         self.mc.CVTSI2SD(resloc, arglocs[0])
 
+    def genop_cast_float_to_singlefloat(self, op, arglocs, resloc):
+        loc0, loctmp = arglocs
+        self.mc.CVTSD2SS(loctmp, loc0)
+        assert isinstance(resloc, RegLoc)
+        assert isinstance(loctmp, RegLoc)
+        self.mc.MOVD_rx(resloc.value, loctmp.value)
+
+    def genop_cast_singlefloat_to_float(self, op, arglocs, resloc):
+        loc0, = arglocs
+        assert isinstance(resloc, RegLoc)
+        assert isinstance(loc0, RegLoc)
+        self.mc.MOVD_xr(resloc.value, loc0.value)
+        self.mc.CVTSS2SD_xx(resloc.value, resloc.value)
+
     def genop_guard_int_is_true(self, op, guard_op, guard_token, arglocs, resloc):
         guard_opnum = guard_op.getopnum()
         self.mc.CMP(arglocs[0], imm0)
@@ -1387,7 +1476,7 @@
         assert isinstance(loc_vtable, ImmedLoc)
         arglocs = arglocs[:-1]
         self.call(self.malloc_func_addr, arglocs, eax)
-        # xxx ignore NULL returns for now
+        self.propagate_memoryerror_if_eax_is_null()
         self.set_vtable(eax, loc_vtable)
 
     def set_vtable(self, loc, loc_vtable):
@@ -1406,18 +1495,35 @@
     def genop_new(self, op, arglocs, result_loc):
         assert result_loc is eax
         self.call(self.malloc_func_addr, arglocs, eax)
+        self.propagate_memoryerror_if_eax_is_null()
 
     def genop_new_array(self, op, arglocs, result_loc):
         assert result_loc is eax
         self.call(self.malloc_array_func_addr, arglocs, eax)
+        self.propagate_memoryerror_if_eax_is_null()
 
     def genop_newstr(self, op, arglocs, result_loc):
         assert result_loc is eax
         self.call(self.malloc_str_func_addr, arglocs, eax)
+        self.propagate_memoryerror_if_eax_is_null()
 
     def genop_newunicode(self, op, arglocs, result_loc):
         assert result_loc is eax
         self.call(self.malloc_unicode_func_addr, arglocs, eax)
+        self.propagate_memoryerror_if_eax_is_null()
+
+    def propagate_memoryerror_if_eax_is_null(self):
+        # if self.propagate_exception_path == 0 (tests), this may jump to 0
+        # and segfaults.  too bad.  the alternative is to continue anyway
+        # with eax==0, but that will segfault too.
+        self.mc.TEST_rr(eax.value, eax.value)
+        if WORD == 4:
+            self.mc.J_il(rx86.Conditions['Z'], self.propagate_exception_path)
+            self.mc.add_pending_relocation()
+        elif WORD == 8:
+            self.mc.J_il(rx86.Conditions['Z'], 0)
+            pos = self.mc.get_relative_pos()
+            self.pending_memoryerror_trampoline_from.append(pos)
 
     # ----------
 
@@ -1702,6 +1808,12 @@
         return GuardToken(faildescr, failargs, fail_locs, exc,
                           is_guard_not_invalidated)
 
+    def generate_propagate_error_64(self):
+        assert WORD == 8
+        startpos = self.mc.get_relative_pos()
+        self.mc.JMP(imm(self.propagate_exception_path))
+        return startpos
+
     def generate_quick_failure(self, guardtok):
         """Generate the initial code for handling a failure.  We try to
         keep it as compact as possible.
@@ -2037,7 +2149,9 @@
         else:
             tmp = eax
 
-        self._emit_call(force_index, x, arglocs, 3, tmp=tmp)
+        self._emit_call(force_index, x, arglocs, 3, tmp=tmp,
+                        argtypes=op.getdescr().get_arg_types(),
+                        callconv=op.getdescr().get_call_conv())
 
         if IS_X86_32 and isinstance(resloc, StackLoc) and resloc.width == 8:
             # a float or a long long return
@@ -2049,7 +2163,19 @@
                 #     and this way is simpler also because the result loc
                 #     can just be always a stack location
             else:
-                self.mc.FSTP_b(resloc.value)   # float return
+                self.mc.FSTPL_b(resloc.value)   # float return
+        elif op.getdescr().get_return_type() == 'S':
+            # singlefloat return
+            assert resloc is eax
+            if IS_X86_32:
+                # must convert ST(0) to a 32-bit singlefloat and load it into EAX
+                # mess mess mess
+                self.mc.SUB_ri(esp.value, 4)
+                self.mc.FSTPS_s(0)
+                self.mc.POP_r(eax.value)
+            elif IS_X86_64:
+                # must copy from the lower 32 bits of XMM0 into eax
+                self.mc.MOVD_rx(eax.value, xmm0.value)
         elif size == WORD:
             assert resloc is eax or resloc is xmm0    # a full word
         elif size == 0:
@@ -2121,13 +2247,27 @@
                 css = get_ebp_ofs(pos + use_words - 1)
                 self._regalloc.close_stack_struct = css
             # The location where the future CALL will put its return address
-            # will be [ESP-WORD], so save that as the next frame's top address
-            self.mc.LEA_rs(eax.value, -WORD)        # LEA EAX, [ESP-4]
+            # will be [ESP-WORD].  But we can't use that as the next frame's
+            # top address!  As the code after releasegil() runs without the
+            # GIL, it might not be set yet by the time we need it (very
+            # unlikely), or it might be overwritten by the following call
+            # to reaquiregil() (much more likely).  So we hack even more
+            # and use a dummy location containing a dummy value (a pointer
+            # to itself) which we pretend is the return address :-/ :-/ :-/
+            # It prevents us to store any %esp-based stack locations but we
+            # don't so far.
+            adr = self.datablockwrapper.malloc_aligned(WORD, WORD)
+            rffi.cast(rffi.CArrayPtr(lltype.Signed), adr)[0] = adr
+            self.gcrootmap_retaddr_forced = adr
             frame_ptr = css + WORD * (2+asmgcroot.FRAME_PTR)
-            self.mc.MOV_br(frame_ptr, eax.value)    # MOV [css.frame], EAX
+            if rx86.fits_in_32bits(adr):
+                self.mc.MOV_bi(frame_ptr, adr)          # MOV [css.frame], adr
+            else:
+                self.mc.MOV_ri(eax.value, adr)          # MOV EAX, adr
+                self.mc.MOV_br(frame_ptr, eax.value)    # MOV [css.frame], EAX
             # Save ebp
             index_of_ebp = css + WORD * (2+asmgcroot.INDEX_OF_EBP)
-            self.mc.MOV_br(index_of_ebp, ebp.value) # MOV [css.ebp], EBP
+            self.mc.MOV_br(index_of_ebp, ebp.value)     # MOV [css.ebp], EBP
             # Call the closestack() function (also releasing the GIL)
             if IS_X86_32:
                 reg = eax
@@ -2155,6 +2295,9 @@
         if gcrootmap.is_shadow_stack:
             args = []
         else:
+            assert self.gcrootmap_retaddr_forced == -1, (
+                      "missing mark_gc_roots() in CALL_RELEASE_GIL")
+            self.gcrootmap_retaddr_forced = 0
             css = self._regalloc.close_stack_struct
             assert css != 0
             if IS_X86_32:
@@ -2207,7 +2350,7 @@
         self._emit_call(fail_index, imm(asm_helper_adr), [eax, arglocs[1]], 0,
                         tmp=ecx)
         if IS_X86_32 and isinstance(result_loc, StackLoc) and result_loc.type == FLOAT:
-            self.mc.FSTP_b(result_loc.value)
+            self.mc.FSTPL_b(result_loc.value)
         #else: result_loc is already either eax or None, checked below
         self.mc.JMP_l8(0) # jump to done, patched later
         jmp_location = self.mc.get_relative_pos()
@@ -2405,7 +2548,13 @@
             if gcrootmap.is_shadow_stack:
                 gcrootmap.write_callshape(mark, force_index)
             else:
-                self.mc.insert_gcroot_marker(mark)
+                if self.gcrootmap_retaddr_forced == 0:
+                    self.mc.insert_gcroot_marker(mark)   # common case
+                else:
+                    assert self.gcrootmap_retaddr_forced != -1, (
+                              "two mark_gc_roots() in a CALL_RELEASE_GIL")
+                    gcrootmap.put(self.gcrootmap_retaddr_forced, mark)
+                    self.gcrootmap_retaddr_forced = -1
 
     def target_arglocs(self, loop_token):
         return loop_token._x86_arglocs
diff --git a/pypy/jit/backend/x86/codebuf.py b/pypy/jit/backend/x86/codebuf.py
--- a/pypy/jit/backend/x86/codebuf.py
+++ b/pypy/jit/backend/x86/codebuf.py
@@ -25,8 +25,11 @@
         self.init_block_builder()
         # a list of relative positions; for each position p, the bytes
         # at [p-4:p] encode an absolute address that will need to be
-        # made relative.
-        self.relocations = []
+        # made relative.  Only works on 32-bit!
+        if WORD == 4:
+            self.relocations = []
+        else:
+            self.relocations = None
         #
         # ResOperation --> offset in the assembly.
         # ops_offset[None] represents the beginning of the code after the last op
@@ -42,9 +45,10 @@
 
     def copy_to_raw_memory(self, addr):
         self._copy_to_raw_memory(addr)
-        for reloc in self.relocations:
-            p = addr + reloc
-            adr = rffi.cast(rffi.LONGP, p - WORD)
-            adr[0] = intmask(adr[0] - p)
+        if self.relocations is not None:
+            for reloc in self.relocations:
+                p = addr + reloc
+                adr = rffi.cast(rffi.LONGP, p - WORD)
+                adr[0] = intmask(adr[0] - p)
         valgrind.discard_translations(addr, self.get_relative_pos())
         self._dump(addr, "jit-backend-dump", backend_name)
diff --git a/pypy/jit/backend/x86/regalloc.py b/pypy/jit/backend/x86/regalloc.py
--- a/pypy/jit/backend/x86/regalloc.py
+++ b/pypy/jit/backend/x86/regalloc.py
@@ -706,6 +706,17 @@
         self.Perform(op, [loc0], loc1)
         self.rm.possibly_free_var(op.getarg(0))
 
+    def consider_cast_float_to_singlefloat(self, op):
+        loc0 = self.xrm.make_sure_var_in_reg(op.getarg(0))
+        loc1 = self.rm.force_allocate_reg(op.result)
+        self.xrm.possibly_free_var(op.getarg(0))
+        tmpxvar = TempBox()
+        loctmp = self.xrm.force_allocate_reg(tmpxvar)   # may be equal to loc0
+        self.xrm.possibly_free_var(tmpxvar)
+        self.Perform(op, [loc0, loctmp], loc1)
+
+    consider_cast_singlefloat_to_float = consider_cast_int_to_float
+
     def _consider_llong_binop_xx(self, op):
         # must force both arguments into xmm registers, because we don't
         # know if they will be suitably aligned.  Exception: if the second
@@ -833,8 +844,8 @@
 
     def consider_call(self, op):
         effectinfo = op.getdescr().get_extra_info()
-        if effectinfo is not None:
-            oopspecindex = effectinfo.oopspecindex
+        oopspecindex = effectinfo.oopspecindex
+        if oopspecindex != EffectInfo.OS_NONE:
             if IS_X86_32:
                 # support for some of the llong operations,
                 # which only exist on x86-32
diff --git a/pypy/jit/backend/x86/regloc.py b/pypy/jit/backend/x86/regloc.py
--- a/pypy/jit/backend/x86/regloc.py
+++ b/pypy/jit/backend/x86/regloc.py
@@ -521,6 +521,8 @@
     UCOMISD = _binaryop('UCOMISD')
     CVTSI2SD = _binaryop('CVTSI2SD')
     CVTTSD2SI = _binaryop('CVTTSD2SI')
+    CVTSD2SS = _binaryop('CVTSD2SS')
+    CVTSS2SD = _binaryop('CVTSS2SD')
     
     SQRTSD = _binaryop('SQRTSD')
 
@@ -534,6 +536,8 @@
     PXOR  = _binaryop('PXOR')
     PCMPEQD = _binaryop('PCMPEQD')
 
+    MOVD = _binaryop('MOVD')
+
     CALL = _relative_unaryop('CALL')
     JMP = _relative_unaryop('JMP')
 
diff --git a/pypy/jit/backend/x86/runner.py b/pypy/jit/backend/x86/runner.py
--- a/pypy/jit/backend/x86/runner.py
+++ b/pypy/jit/backend/x86/runner.py
@@ -19,6 +19,7 @@
 class AbstractX86CPU(AbstractLLCPU):
     debug = True
     supports_floats = True
+    supports_singlefloats = True
 
     BOOTSTRAP_TP = lltype.FuncType([], lltype.Signed)
     dont_keepalive_stuff = False # for tests
diff --git a/pypy/jit/backend/x86/rx86.py b/pypy/jit/backend/x86/rx86.py
--- a/pypy/jit/backend/x86/rx86.py
+++ b/pypy/jit/backend/x86/rx86.py
@@ -527,6 +527,7 @@
 
     NOP = insn('\x90')
     RET = insn('\xC3')
+    RET16_i = insn('\xC2', immediate(1, 'h'))
 
     PUSH_r = insn(rex_nw, register(1), '\x50')
     PUSH_b = insn(rex_nw, '\xFF', orbyte(6<<3), stack_bp(1))
@@ -573,7 +574,8 @@
     BTS_jr = insn(rex_w, '\x0F\xAB', register(2,8), abs_, immediate(1))
 
     # x87 instructions
-    FSTP_b = insn('\xDD', orbyte(3<<3), stack_bp(1))
+    FSTPL_b = insn('\xDD', orbyte(3<<3), stack_bp(1)) # rffi.DOUBLE ('as' wants L??)
+    FSTPS_s = insn('\xD9', orbyte(3<<3), stack_sp(1)) # lltype.SingleFloat
 
     # ------------------------------ Random mess -----------------------
     RDTSC = insn('\x0F\x31')
@@ -590,8 +592,18 @@
     CVTTSD2SI_rx = xmminsn('\xF2', rex_w, '\x0F\x2C', register(1, 8), register(2), '\xC0')
     CVTTSD2SI_rb = xmminsn('\xF2', rex_w, '\x0F\x2C', register(1, 8), stack_bp(2))
 
-    MOVD_rx = xmminsn('\x66', rex_w, '\x0F\x7E', register(2, 8), register(1), '\xC0')
-    MOVD_xr = xmminsn('\x66', rex_w, '\x0F\x6E', register(1, 8), register(2), '\xC0')
+    CVTSD2SS_xx = xmminsn('\xF2', rex_nw, '\x0F\x5A',
+                          register(1, 8), register(2), '\xC0')
+    CVTSD2SS_xb = xmminsn('\xF2', rex_nw, '\x0F\x5A',
+                          register(1, 8), stack_bp(2))
+    CVTSS2SD_xx = xmminsn('\xF3', rex_nw, '\x0F\x5A',
+                          register(1, 8), register(2), '\xC0')
+    CVTSS2SD_xb = xmminsn('\xF3', rex_nw, '\x0F\x5A',
+                          register(1, 8), stack_bp(2))
+
+    MOVD_rx = xmminsn('\x66', rex_nw, '\x0F\x7E', register(2, 8), register(1), '\xC0')
+    MOVD_xr = xmminsn('\x66', rex_nw, '\x0F\x6E', register(1, 8), register(2), '\xC0')
+    MOVD_xb = xmminsn('\x66', rex_nw, '\x0F\x6E', register(1, 8), stack_bp(2))
 
     PSRAD_xi = xmminsn('\x66', rex_nw, '\x0F\x72', register(1), '\xE0', immediate(2, 'b'))
 
diff --git a/pypy/jit/backend/x86/test/test_gc_integration.py b/pypy/jit/backend/x86/test/test_gc_integration.py
--- a/pypy/jit/backend/x86/test/test_gc_integration.py
+++ b/pypy/jit/backend/x86/test/test_gc_integration.py
@@ -7,6 +7,7 @@
      BoxPtr, ConstPtr, TreeLoop
 from pypy.jit.metainterp.resoperation import rop, ResOperation
 from pypy.jit.codewriter import heaptracker
+from pypy.jit.codewriter.effectinfo import EffectInfo
 from pypy.jit.backend.llsupport.descr import GcCache
 from pypy.jit.backend.llsupport.gc import GcLLDescription
 from pypy.jit.backend.detect_cpu import getcpuclass
@@ -76,7 +77,8 @@
         for box in boxes:
             regalloc.rm.try_allocate_reg(box)
         TP = lltype.FuncType([], lltype.Signed)
-        calldescr = cpu.calldescrof(TP, TP.ARGS, TP.RESULT)
+        calldescr = cpu.calldescrof(TP, TP.ARGS, TP.RESULT,
+                                    EffectInfo.MOST_GENERAL)
         regalloc.rm._check_invariants()
         box = boxes[0]
         regalloc.position = 0
diff --git a/pypy/jit/backend/x86/test/test_regalloc.py b/pypy/jit/backend/x86/test/test_regalloc.py
--- a/pypy/jit/backend/x86/test/test_regalloc.py
+++ b/pypy/jit/backend/x86/test/test_regalloc.py
@@ -16,6 +16,7 @@
 from pypy.rpython.annlowlevel import llhelper
 from pypy.rpython.lltypesystem import rclass, rstr
 from pypy.jit.codewriter import longlong
+from pypy.jit.codewriter.effectinfo import EffectInfo
 from pypy.jit.backend.x86.rx86 import *
 
 def test_is_comparison_or_ovf_op():
@@ -92,7 +93,8 @@
     zd_addr = cpu.cast_int_to_adr(zero_division_tp)
     zero_division_error = llmemory.cast_adr_to_ptr(zd_addr,
                                             lltype.Ptr(rclass.OBJECT_VTABLE))
-    raising_calldescr = cpu.calldescrof(FPTR.TO, FPTR.TO.ARGS, FPTR.TO.RESULT)
+    raising_calldescr = cpu.calldescrof(FPTR.TO, FPTR.TO.ARGS, FPTR.TO.RESULT,
+                                        EffectInfo.MOST_GENERAL)
 
     fdescr1 = BasicFailDescr(1)
     fdescr2 = BasicFailDescr(2)
@@ -115,9 +117,12 @@
     f2ptr = llhelper(F2PTR, f2)
     f10ptr = llhelper(F10PTR, f10)
 
-    f1_calldescr = cpu.calldescrof(F1PTR.TO, F1PTR.TO.ARGS, F1PTR.TO.RESULT)
-    f2_calldescr = cpu.calldescrof(F2PTR.TO, F2PTR.TO.ARGS, F2PTR.TO.RESULT)
-    f10_calldescr = cpu.calldescrof(F10PTR.TO, F10PTR.TO.ARGS, F10PTR.TO.RESULT)
+    f1_calldescr = cpu.calldescrof(F1PTR.TO, F1PTR.TO.ARGS, F1PTR.TO.RESULT,
+                                   EffectInfo.MOST_GENERAL)
+    f2_calldescr = cpu.calldescrof(F2PTR.TO, F2PTR.TO.ARGS, F2PTR.TO.RESULT,
+                                   EffectInfo.MOST_GENERAL)
+    f10_calldescr= cpu.calldescrof(F10PTR.TO, F10PTR.TO.ARGS, F10PTR.TO.RESULT,
+                                   EffectInfo.MOST_GENERAL)
 
     namespace = locals().copy()
     type_system = 'lltype'
diff --git a/pypy/jit/backend/x86/test/test_regloc.py b/pypy/jit/backend/x86/test/test_regloc.py
--- a/pypy/jit/backend/x86/test/test_regloc.py
+++ b/pypy/jit/backend/x86/test/test_regloc.py
@@ -62,7 +62,7 @@
             assert mc.relocations == [5]
             expected = "\xE8" + struct.pack(' movl $xxx, %eax
                 suffix = 'l'
-                if ops[1][2:].isdigit():
-                    ops[1] += 'd'
-                else:
-                    ops[1] = '%e' + ops[1][2:]
+                ops[1] = reduce_to_32bit(ops[1])
+            if instrname.lower() == 'movd':
+                ops[0] = reduce_to_32bit(ops[0])
+                ops[1] = reduce_to_32bit(ops[1])
             #
             op = '\t%s%s %s%s' % (instrname.lower(), suffix,
                                   ', '.join(ops), following)
diff --git a/pypy/jit/codewriter/assembler.py b/pypy/jit/codewriter/assembler.py
--- a/pypy/jit/codewriter/assembler.py
+++ b/pypy/jit/codewriter/assembler.py
@@ -76,6 +76,8 @@
                 TYPE = llmemory.Address
             if TYPE == llmemory.Address:
                 value = heaptracker.adr2int(value)
+            if TYPE is lltype.SingleFloat:
+                value = longlong.singlefloat2int(value)
             if not isinstance(value, (llmemory.AddressAsInt,
                                       ComputedIntSymbolic)):
                 value = lltype.cast_primitive(lltype.Signed, value)
diff --git a/pypy/jit/codewriter/call.py b/pypy/jit/codewriter/call.py
--- a/pypy/jit/codewriter/call.py
+++ b/pypy/jit/codewriter/call.py
@@ -6,7 +6,7 @@
 from pypy.jit.codewriter import support
 from pypy.jit.codewriter.jitcode import JitCode
 from pypy.jit.codewriter.effectinfo import (VirtualizableAnalyzer,
-    QuasiImmutAnalyzer, CanReleaseGILAnalyzer, effectinfo_from_writeanalyze,
+    QuasiImmutAnalyzer, RandomEffectsAnalyzer, effectinfo_from_writeanalyze,
     EffectInfo, CallInfoCollection)
 from pypy.translator.simplify import get_funcobj, get_functype
 from pypy.rpython.lltypesystem import lltype, llmemory
@@ -31,7 +31,7 @@
             self.readwrite_analyzer = ReadWriteAnalyzer(translator)
             self.virtualizable_analyzer = VirtualizableAnalyzer(translator)
             self.quasiimmut_analyzer = QuasiImmutAnalyzer(translator)
-            self.canreleasegil_analyzer = CanReleaseGILAnalyzer(translator)
+            self.randomeffects_analyzer = RandomEffectsAnalyzer(translator)
         #
         for index, jd in enumerate(jitdrivers_sd):
             jd.index = index
@@ -187,7 +187,7 @@
             fnaddr = llmemory.cast_ptr_to_adr(fnptr)
         NON_VOID_ARGS = [ARG for ARG in FUNC.ARGS if ARG is not lltype.Void]
         calldescr = self.cpu.calldescrof(FUNC, tuple(NON_VOID_ARGS),
-                                         FUNC.RESULT)
+                                         FUNC.RESULT, EffectInfo.MOST_GENERAL)
         return (fnaddr, calldescr)
 
     def getcalldescr(self, op, oopspecindex=EffectInfo.OS_NONE,
@@ -219,9 +219,11 @@
                 assert not NON_VOID_ARGS, ("arguments not supported for "
                                            "loop-invariant function!")
         # build the extraeffect
-        can_release_gil = self.canreleasegil_analyzer.analyze(op)
-        # can_release_gil implies can_invalidate
-        can_invalidate = can_release_gil or self.quasiimmut_analyzer.analyze(op)
+        random_effects = self.randomeffects_analyzer.analyze(op)
+        if random_effects:
+            extraeffect = EffectInfo.EF_RANDOM_EFFECTS
+        # random_effects implies can_invalidate
+        can_invalidate = random_effects or self.quasiimmut_analyzer.analyze(op)
         if extraeffect is None:
             if self.virtualizable_analyzer.analyze(op):
                 extraeffect = EffectInfo.EF_FORCES_VIRTUAL_OR_VIRTUALIZABLE
@@ -239,12 +241,10 @@
         #
         effectinfo = effectinfo_from_writeanalyze(
             self.readwrite_analyzer.analyze(op), self.cpu, extraeffect,
-            oopspecindex, can_invalidate, can_release_gil)
+            oopspecindex, can_invalidate)
         #
-        if oopspecindex != EffectInfo.OS_NONE:
-            assert effectinfo is not None
+        assert effectinfo is not None
         if elidable or loopinvariant:
-            assert effectinfo is not None
             assert extraeffect != EffectInfo.EF_FORCES_VIRTUAL_OR_VIRTUALIZABLE
             # XXX this should also say assert not can_invalidate, but
             #     it can't because our analyzer is not good enough for now
@@ -264,8 +264,7 @@
 
     def calldescr_canraise(self, calldescr):
         effectinfo = calldescr.get_extra_info()
-        return (effectinfo is None or
-                effectinfo.extraeffect > EffectInfo.EF_CANNOT_RAISE)
+        return effectinfo.check_can_raise()
 
     def jitdriver_sd_from_portal_graph(self, graph):
         for jd in self.jitdrivers_sd:
diff --git a/pypy/jit/codewriter/effectinfo.py b/pypy/jit/codewriter/effectinfo.py
--- a/pypy/jit/codewriter/effectinfo.py
+++ b/pypy/jit/codewriter/effectinfo.py
@@ -15,6 +15,7 @@
     EF_ELIDABLE_CAN_RAISE              = 3 #elidable function (but can raise)
     EF_CAN_RAISE                       = 4 #normal function (can raise)
     EF_FORCES_VIRTUAL_OR_VIRTUALIZABLE = 5 #can raise and force virtualizables
+    EF_RANDOM_EFFECTS                  = 6 #can do whatever
 
     # the 'oopspecindex' field is one of the following values:
     OS_NONE                     = 0    # normal case, no oopspec
@@ -80,17 +81,26 @@
                 write_descrs_fields, write_descrs_arrays,
                 extraeffect=EF_CAN_RAISE,
                 oopspecindex=OS_NONE,
-                can_invalidate=False, can_release_gil=False):
-        key = (frozenset(readonly_descrs_fields),
-               frozenset(readonly_descrs_arrays),
-               frozenset(write_descrs_fields),
-               frozenset(write_descrs_arrays),
+                can_invalidate=False):
+        key = (frozenset_or_none(readonly_descrs_fields),
+               frozenset_or_none(readonly_descrs_arrays),
+               frozenset_or_none(write_descrs_fields),
+               frozenset_or_none(write_descrs_arrays),
                extraeffect,
                oopspecindex,
-               can_invalidate,
-               can_release_gil)
+               can_invalidate)
         if key in cls._cache:
             return cls._cache[key]
+        if extraeffect == EffectInfo.EF_RANDOM_EFFECTS:
+            assert readonly_descrs_fields is None
+            assert readonly_descrs_arrays is None
+            assert write_descrs_fields is None
+            assert write_descrs_arrays is None
+        else:
+            assert readonly_descrs_fields is not None
+            assert readonly_descrs_arrays is not None
+            assert write_descrs_fields is not None
+            assert write_descrs_arrays is not None
         result = object.__new__(cls)
         result.readonly_descrs_fields = readonly_descrs_fields
         result.readonly_descrs_arrays = readonly_descrs_arrays
@@ -104,11 +114,13 @@
             result.write_descrs_arrays = write_descrs_arrays
         result.extraeffect = extraeffect
         result.can_invalidate = can_invalidate
-        result.can_release_gil = can_release_gil
         result.oopspecindex = oopspecindex
         cls._cache[key] = result
         return result
 
+    def check_can_raise(self):
+        return self.extraeffect > self.EF_CANNOT_RAISE
+
     def check_can_invalidate(self):
         return self.can_invalidate
 
@@ -116,56 +128,71 @@
         return self.extraeffect >= self.EF_FORCES_VIRTUAL_OR_VIRTUALIZABLE
 
     def has_random_effects(self):
-        return self.oopspecindex == self.OS_LIBFFI_CALL or self.can_release_gil
+        return self.extraeffect >= self.EF_RANDOM_EFFECTS
+
+
+def frozenset_or_none(x):
+    if x is None:
+        return None
+    return frozenset(x)
+
+EffectInfo.MOST_GENERAL = EffectInfo(None, None, None, None,
+                                     EffectInfo.EF_RANDOM_EFFECTS,
+                                     can_invalidate=True)
+
 
 def effectinfo_from_writeanalyze(effects, cpu,
                                  extraeffect=EffectInfo.EF_CAN_RAISE,
                                  oopspecindex=EffectInfo.OS_NONE,
-                                 can_invalidate=False,
-                                 can_release_gil=False):
+                                 can_invalidate=False):
     from pypy.translator.backendopt.writeanalyze import top_set
-    if effects is top_set:
-        return None
-    readonly_descrs_fields = []
-    readonly_descrs_arrays = []
-    write_descrs_fields = []
-    write_descrs_arrays = []
+    if effects is top_set or extraeffect == EffectInfo.EF_RANDOM_EFFECTS:
+        readonly_descrs_fields = None
+        readonly_descrs_arrays = None
+        write_descrs_fields = None
+        write_descrs_arrays = None
+        extraeffect = EffectInfo.EF_RANDOM_EFFECTS
+    else:
+        readonly_descrs_fields = []
+        readonly_descrs_arrays = []
+        write_descrs_fields = []
+        write_descrs_arrays = []
 
-    def add_struct(descrs_fields, (_, T, fieldname)):
-        T = deref(T)
-        if consider_struct(T, fieldname):
-            descr = cpu.fielddescrof(T, fieldname)
-            descrs_fields.append(descr)
+        def add_struct(descrs_fields, (_, T, fieldname)):
+            T = deref(T)
+            if consider_struct(T, fieldname):
+                descr = cpu.fielddescrof(T, fieldname)
+                descrs_fields.append(descr)
 
-    def add_array(descrs_arrays, (_, T)):
-        ARRAY = deref(T)
-        if consider_array(ARRAY):
-            descr = cpu.arraydescrof(ARRAY)
-            descrs_arrays.append(descr)
+        def add_array(descrs_arrays, (_, T)):
+            ARRAY = deref(T)
+            if consider_array(ARRAY):
+                descr = cpu.arraydescrof(ARRAY)
+                descrs_arrays.append(descr)
 
-    for tup in effects:
-        if tup[0] == "struct":
-            add_struct(write_descrs_fields, tup)
-        elif tup[0] == "readstruct":
-            tupw = ("struct",) + tup[1:]
-            if tupw not in effects:
-                add_struct(readonly_descrs_fields, tup)
-        elif tup[0] == "array":
-            add_array(write_descrs_arrays, tup)
-        elif tup[0] == "readarray":
-            tupw = ("array",) + tup[1:]
-            if tupw not in effects:
-                add_array(readonly_descrs_arrays, tup)
-        else:
-            assert 0
+        for tup in effects:
+            if tup[0] == "struct":
+                add_struct(write_descrs_fields, tup)
+            elif tup[0] == "readstruct":
+                tupw = ("struct",) + tup[1:]
+                if tupw not in effects:
+                    add_struct(readonly_descrs_fields, tup)
+            elif tup[0] == "array":
+                add_array(write_descrs_arrays, tup)
+            elif tup[0] == "readarray":
+                tupw = ("array",) + tup[1:]
+                if tupw not in effects:
+                    add_array(readonly_descrs_arrays, tup)
+            else:
+                assert 0
+    #
     return EffectInfo(readonly_descrs_fields,
                       readonly_descrs_arrays,
                       write_descrs_fields,
                       write_descrs_arrays,
                       extraeffect,
                       oopspecindex,
-                      can_invalidate,
-                      can_release_gil)
+                      can_invalidate)
 
 def consider_struct(TYPE, fieldname):
     if fieldType(TYPE, fieldname) is lltype.Void:
@@ -201,12 +228,13 @@
     def analyze_simple_operation(self, op, graphinfo):
         return op.opname == 'jit_force_quasi_immutable'
 
-class CanReleaseGILAnalyzer(BoolGraphAnalyzer):
+class RandomEffectsAnalyzer(BoolGraphAnalyzer):
     def analyze_direct_call(self, graph, seen=None):
-        releases_gil = False
         if hasattr(graph, "func") and hasattr(graph.func, "_ptr"):
-            releases_gil = graph.func._ptr._obj.releases_gil
-        return releases_gil or super(CanReleaseGILAnalyzer, self).analyze_direct_call(graph, seen)
+            if graph.func._ptr._obj.random_effects_on_gcobjs:
+                return True
+        return super(RandomEffectsAnalyzer, self).analyze_direct_call(graph,
+                                                                      seen)
 
     def analyze_simple_operation(self, op, graphinfo):
         return False
diff --git a/pypy/jit/codewriter/jtransform.py b/pypy/jit/codewriter/jtransform.py
--- a/pypy/jit/codewriter/jtransform.py
+++ b/pypy/jit/codewriter/jtransform.py
@@ -1,4 +1,5 @@
 import py
+
 from pypy.jit.codewriter import support, heaptracker, longlong
 from pypy.jit.codewriter.effectinfo import EffectInfo
 from pypy.jit.codewriter.flatten import ListOfKind, IndirectCallTargets
@@ -9,7 +10,7 @@
 from pypy.objspace.flow.model import SpaceOperation, Variable, Constant, c_last_exception
 from pypy.rlib import objectmodel
 from pypy.rlib.jit import _we_are_jitted
-from pypy.rpython.lltypesystem import lltype, llmemory, rstr, rclass
+from pypy.rpython.lltypesystem import lltype, llmemory, rstr, rclass, rffi
 from pypy.rpython.rclass import IR_QUASIIMMUTABLE, IR_QUASIIMMUTABLE_ARRAY
 from pypy.translator.simplify import get_funcobj
 from pypy.translator.unsimplify import varoftype
@@ -22,6 +23,11 @@
     t = Transformer(cpu, callcontrol, portal_jd)
     t.transform(graph)
 
+def integer_bounds(size, unsigned):
+    if unsigned:
+        return 0, 1 << (8 * size)
+    else:
+        return -(1 << (8 * size - 1)), 1 << (8 * size - 1)
 
 class Transformer(object):
     vable_array_vars = None
@@ -200,7 +206,6 @@
             self.vable_array_vars[op.result]= self.vable_array_vars[op.args[0]]
 
     rewrite_op_cast_pointer = rewrite_op_same_as
-    rewrite_op_cast_opaque_ptr = rewrite_op_same_as   # rlib.rerased
     def rewrite_op_cast_bool_to_int(self, op): pass
     def rewrite_op_cast_bool_to_uint(self, op): pass
     def rewrite_op_cast_char_to_int(self, op): pass
@@ -574,6 +579,7 @@
                 pure = '_pure'
         else:
             pure = ''
+        self.check_field_access(v_inst.concretetype.TO)
         argname = getattr(v_inst.concretetype.TO, '_gckind', 'gc')
         descr = self.cpu.fielddescrof(v_inst.concretetype.TO,
                                       c_fieldname.value)
@@ -607,6 +613,7 @@
             return [SpaceOperation('-live-', [], None),
                     SpaceOperation('setfield_vable_%s' % kind,
                                    [v_inst, descr, v_value], None)]
+        self.check_field_access(v_inst.concretetype.TO)
         argname = getattr(v_inst.concretetype.TO, '_gckind', 'gc')
         descr = self.cpu.fielddescrof(v_inst.concretetype.TO,
                                       c_fieldname.value)
@@ -619,6 +626,22 @@
         return (op.args[1].value == 'typeptr' and
                 op.args[0].concretetype.TO._hints.get('typeptr'))
 
+    def check_field_access(self, STRUCT):
+        # check against a GcStruct with a nested GcStruct as a first argument
+        # but which is not an object at all; see metainterp/test/test_loop,
+        # test_regular_pointers_in_short_preamble.
+        if not isinstance(STRUCT, lltype.GcStruct):
+            return
+        if STRUCT._first_struct() == (None, None):
+            return
+        PARENT = STRUCT
+        while not PARENT._hints.get('typeptr'):
+            _, PARENT = PARENT._first_struct()
+            if PARENT is None:
+                raise NotImplementedError("%r is a GcStruct using nesting but "
+                                          "not inheriting from object" %
+                                          (STRUCT,))
+
     def get_vinfo(self, v_virtualizable):
         if self.callcontrol is None:      # for tests
             return None
@@ -791,75 +814,127 @@
             raise NotImplementedError("cast_ptr_to_int")
 
     def rewrite_op_force_cast(self, op):
-        assert not self._is_gc(op.args[0])
-        fromll = longlong.is_longlong(op.args[0].concretetype)
-        toll   = longlong.is_longlong(op.result.concretetype)
-        if fromll and toll:
+        v_arg = op.args[0]
+        v_result = op.result
+        assert not self._is_gc(v_arg)
+
+        if v_arg.concretetype == v_result.concretetype:
             return
-        if fromll:
-            args = op.args
-            opname = 'truncate_longlong_to_int'
-            RESULT = lltype.Signed
-            v = varoftype(RESULT)
-            op1 = SpaceOperation(opname, args, v)
-            op2 = self.rewrite_operation(op1)
-            oplist = self.force_cast_without_longlong(op2.result, op.result)
+
+        float_arg = v_arg.concretetype in [lltype.Float, lltype.SingleFloat]
+        float_res = v_result.concretetype in [lltype.Float, lltype.SingleFloat]
+        if not float_arg and not float_res:
+            # some int -> some int cast
+            return self._int_to_int_cast(v_arg, v_result)
+        elif float_arg and float_res:
+            # some float -> some float cast
+            return self._float_to_float_cast(v_arg, v_result)
+        elif not float_arg and float_res:
+            # some int -> some float
+            ops = []
+            v1 = varoftype(lltype.Signed)
+            oplist = self.rewrite_operation(
+                SpaceOperation('force_cast', [v_arg], v1)
+            )
             if oplist:
-                return [op2] + oplist
-            #
-            # force a renaming to put the correct result in place, even though
-            # it might be slightly mistyped (e.g. Signed versus Unsigned)
-            assert op2.result is v
-            op2.result = op.result
-            return op2
-        elif toll:
-            from pypy.rpython.lltypesystem import rffi
-            size, unsigned = rffi.size_and_sign(op.args[0].concretetype)
-            if unsigned:
+                ops.extend(oplist)
+            else:
+                v1 = v_arg
+            v2 = varoftype(lltype.Float)
+            op = self.rewrite_operation(
+                SpaceOperation('cast_int_to_float', [v1], v2)
+            )
+            ops.append(op)
+            op2 = self.rewrite_operation(
+                SpaceOperation('force_cast', [v2], v_result)
+            )
+            if op2:
+                ops.append(op2)
+            else:
+                op.result = v_result
+            return ops
+        elif float_arg and not float_res:
+            # some float -> some int
+            ops = []
+            v1 = varoftype(lltype.Float)
+            op1 = self.rewrite_operation(
+                SpaceOperation('force_cast', [v_arg], v1)
+            )
+            if op1:
+                ops.append(op1)
+            else:
+                v1 = v_arg
+            v2 = varoftype(lltype.Signed)
+            op = self.rewrite_operation(
+                SpaceOperation('cast_float_to_int', [v1], v2)
+            )
+            ops.append(op)
+            oplist = self.rewrite_operation(
+                SpaceOperation('force_cast', [v2], v_result)
+            )
+            if oplist:
+                ops.extend(oplist)
+            else:
+                op.result = v_result
+            return ops
+        else:
+            assert False
+
+    def _int_to_int_cast(self, v_arg, v_result):
+        longlong_arg = longlong.is_longlong(v_arg.concretetype)
+        longlong_res = longlong.is_longlong(v_result.concretetype)
+        size1, unsigned1 = rffi.size_and_sign(v_arg.concretetype)
+        size2, unsigned2 = rffi.size_and_sign(v_result.concretetype)
+
+        if longlong_arg and longlong_res:
+            return
+        elif longlong_arg:
+            v = varoftype(lltype.Signed)
+            op1 = self.rewrite_operation(
+                SpaceOperation('truncate_longlong_to_int', [v_arg], v)
+            )
+            op2 = SpaceOperation('force_cast', [v], v_result)
+            oplist = self.rewrite_operation(op2)
+            if not oplist:
+                op1.result = v_result
+                oplist = []
+            return [op1] + oplist
+        elif longlong_res:
+            if unsigned1:
                 INTERMEDIATE = lltype.Unsigned
             else:
                 INTERMEDIATE = lltype.Signed
             v = varoftype(INTERMEDIATE)
-            oplist = self.force_cast_without_longlong(op.args[0], v)
+            op1 = SpaceOperation('force_cast', [v_arg], v)
+            oplist = self.rewrite_operation(op1)
             if not oplist:
-                v = op.args[0]
+                v = v_arg
                 oplist = []
-            if unsigned:
+            if unsigned1:
                 opname = 'cast_uint_to_longlong'
             else:
                 opname = 'cast_int_to_longlong'
-            op1 = SpaceOperation(opname, [v], op.result)
-            op2 = self.rewrite_operation(op1)
+            op2 = self.rewrite_operation(
+                SpaceOperation(opname, [v], v_result)
+            )
             return oplist + [op2]
-        else:
-            return self.force_cast_without_longlong(op.args[0], op.result)
 
-    def force_cast_without_longlong(self, v_arg, v_result):
-        from pypy.rpython.lltypesystem.rffi import size_and_sign, sizeof, FLOAT
-        #
-        if (v_result.concretetype in (FLOAT, lltype.Float) or
-            v_arg.concretetype in (FLOAT, lltype.Float)):
-            assert (v_result.concretetype == lltype.Float and
-                    v_arg.concretetype == lltype.Float), "xxx unsupported cast"
+        # We've now, ostensibly, dealt with the longlongs, everything should be
+        # a Signed or smaller
+        assert size1 <= rffi.sizeof(lltype.Signed)
+        assert size2 <= rffi.sizeof(lltype.Signed)
+
+        # the target type is LONG or ULONG
+        if size2 == rffi.sizeof(lltype.Signed):
             return
-        #
-        size2, unsigned2 = size_and_sign(v_result.concretetype)
-        assert size2 <= sizeof(lltype.Signed)
-        if size2 == sizeof(lltype.Signed):
-            return     # the target type is LONG or ULONG
-        size1, unsigned1 = size_and_sign(v_arg.concretetype)
-        assert size1 <= sizeof(lltype.Signed)
-        #
-        def bounds(size, unsigned):
-            if unsigned:
-                return 0, 1<<(8*size)
-            else:
-                return -(1<<(8*size-1)), 1<<(8*size-1)
-        min1, max1 = bounds(size1, unsigned1)
-        min2, max2 = bounds(size2, unsigned2)
+
+        min1, max1 = integer_bounds(size1, unsigned1)
+        min2, max2 = integer_bounds(size2, unsigned2)
+
+        # the target type includes the source range
         if min2 <= min1 <= max1 <= max2:
-            return     # the target type includes the source range
-        #
+            return
+
         result = []
         if min2:
             c_min2 = Constant(min2, lltype.Signed)
@@ -867,17 +942,29 @@
             result.append(SpaceOperation('int_sub', [v_arg, c_min2], v2))
         else:
             v2 = v_arg
-        c_mask = Constant(int((1<<(8*size2))-1), lltype.Signed)
-        v3 = varoftype(lltype.Signed)
+        c_mask = Constant(int((1 << (8 * size2)) - 1), lltype.Signed)
+        if min2:
+            v3 = varoftype(lltype.Signed)
+        else:
+            v3 = v_result
         result.append(SpaceOperation('int_and', [v2, c_mask], v3))
         if min2:
             result.append(SpaceOperation('int_add', [v3, c_min2], v_result))
-        else:
-            result[-1].result = v_result
         return result
 
+    def _float_to_float_cast(self, v_arg, v_result):
+        if v_arg.concretetype == lltype.SingleFloat:
+            assert v_result.concretetype == lltype.Float, "cast %s -> %s" % (
+                v_arg.concretetype, v_result.concretetype)
+            return SpaceOperation('cast_singlefloat_to_float', [v_arg],
+                                  v_result)
+        if v_result.concretetype == lltype.SingleFloat:
+            assert v_arg.concretetype == lltype.Float, "cast %s -> %s" % (
+                v_arg.concretetype, v_result.concretetype)
+            return SpaceOperation('cast_float_to_singlefloat', [v_arg],
+                                  v_result)
+
     def rewrite_op_direct_ptradd(self, op):
-        from pypy.rpython.lltypesystem import rffi
         # xxx otherwise, not implemented:
         assert op.args[0].concretetype == rffi.CCHARP
         #
@@ -1423,7 +1510,7 @@
             extraeffect = EffectInfo.EF_CANNOT_RAISE
         elif oopspec_name.startswith('libffi_call_'):
             oopspecindex = EffectInfo.OS_LIBFFI_CALL
-            extraeffect = EffectInfo.EF_FORCES_VIRTUAL_OR_VIRTUALIZABLE
+            extraeffect = EffectInfo.EF_RANDOM_EFFECTS
         else:
             assert False, 'unsupported oopspec: %s' % oopspec_name
         return self._handle_oopspec_call(op, args, oopspecindex, extraeffect)
diff --git a/pypy/jit/codewriter/longlong.py b/pypy/jit/codewriter/longlong.py
--- a/pypy/jit/codewriter/longlong.py
+++ b/pypy/jit/codewriter/longlong.py
@@ -7,7 +7,8 @@
 """
 
 import sys
-from pypy.rpython.lltypesystem import lltype
+from pypy.rpython.lltypesystem import lltype, rffi
+from pypy.rlib import rarithmetic, longlong2float
 
 
 if sys.maxint > 2147483647:
@@ -31,8 +32,6 @@
     # ---------- 32-bit platform ----------
     # the type FloatStorage is r_longlong, and conversion is needed
 
-    from pypy.rlib import rarithmetic, longlong2float
-
     is_64_bit = False
     supports_longlong = True
     r_float_storage = rarithmetic.r_longlong
@@ -41,9 +40,19 @@
     getfloatstorage = longlong2float.float2longlong
     getrealfloat    = longlong2float.longlong2float
     gethash         = lambda xll: rarithmetic.intmask(xll - (xll >> 32))
-    is_longlong     = lambda TYPE: (TYPE == lltype.SignedLongLong or
-                                    TYPE == lltype.UnsignedLongLong)
+    is_longlong     = lambda TYPE: (TYPE is lltype.SignedLongLong or
+                                    TYPE is lltype.UnsignedLongLong)
 
     # -------------------------------------
 
 ZEROF = getfloatstorage(0.0)
+
+# ____________________________________________________________
+
+def int2singlefloat(x):
+    x = rffi.r_uint(x)
+    return longlong2float.uint2singlefloat(x)
+
+def singlefloat2int(x):
+    x = longlong2float.singlefloat2uint(x)
+    return rffi.cast(lltype.Signed, x)
diff --git a/pypy/jit/codewriter/policy.py b/pypy/jit/codewriter/policy.py
--- a/pypy/jit/codewriter/policy.py
+++ b/pypy/jit/codewriter/policy.py
@@ -12,6 +12,7 @@
         self.unsafe_loopy_graphs = set()
         self.supports_floats = False
         self.supports_longlong = False
+        self.supports_singlefloats = False
 
     def set_supports_floats(self, flag):
         self.supports_floats = flag
@@ -19,6 +20,9 @@
     def set_supports_longlong(self, flag):
         self.supports_longlong = flag
 
+    def set_supports_singlefloats(self, flag):
+        self.supports_singlefloats = flag
+
     def dump_unsafe_loops(self):
         f = udir.join("unsafe-loops.txt").open('w')
         strs = [str(graph) for graph in self.unsafe_loopy_graphs]
@@ -58,8 +62,9 @@
                     func, '_jit_unroll_safe_', False)
 
         unsupported = contains_unsupported_variable_type(graph,
-                                                         self.supports_floats,
-                                                         self.supports_longlong)
+                            self.supports_floats,
+                            self.supports_longlong,
+                            self.supports_singlefloats)
         res = see_function and not unsupported
         if res and contains_loop:
             self.unsafe_loopy_graphs.add(graph)
@@ -80,17 +85,24 @@
         return res
 
 def contains_unsupported_variable_type(graph, supports_floats,
-                                       supports_longlong):
+                                              supports_longlong,
+                                              supports_singlefloats):
     getkind = history.getkind
     try:
         for block in graph.iterblocks():
             for v in block.inputargs:
-                getkind(v.concretetype, supports_floats, supports_longlong)
+                getkind(v.concretetype, supports_floats,
+                                        supports_longlong,
+                                        supports_singlefloats)
             for op in block.operations:
                 for v in op.args:
-                    getkind(v.concretetype, supports_floats, supports_longlong)
+                    getkind(v.concretetype, supports_floats,
+                                            supports_longlong,
+                                            supports_singlefloats)
                 v = op.result
-                getkind(v.concretetype, supports_floats, supports_longlong)
+                getkind(v.concretetype, supports_floats,
+                                        supports_longlong,
+                                        supports_singlefloats)
     except NotImplementedError, e:
         log.WARNING('%s, ignoring graph' % (e,))
         log.WARNING('  %s' % (graph,))
diff --git a/pypy/jit/codewriter/support.py b/pypy/jit/codewriter/support.py
--- a/pypy/jit/codewriter/support.py
+++ b/pypy/jit/codewriter/support.py
@@ -86,9 +86,12 @@
     reds_v = op.args[2+numgreens:]
     assert len(reds_v) == numreds
     #
-    def _sort(args_v):
+    def _sort(args_v, is_green):
         from pypy.jit.metainterp.history import getkind
         lst = [v for v in args_v if v.concretetype is not lltype.Void]
+        if is_green:
+            assert len(lst) == len(args_v), (
+                "not supported so far: 'greens' variables contain Void")
         _kind2count = {'int': 1, 'ref': 2, 'float': 3}
         lst2 = sorted(lst, key=lambda v: _kind2count[getkind(v.concretetype)])
         # a crash here means that you have to reorder the variable named in
@@ -97,7 +100,7 @@
         assert lst == lst2
         return lst
     #
-    return (_sort(greens_v), _sort(reds_v))
+    return (_sort(greens_v, True), _sort(reds_v, False))
 
 def maybe_on_top_of_llinterp(rtyper, fnptr):
     # Run a generated graph on top of the llinterp for testing.
diff --git a/pypy/jit/codewriter/test/test_call.py b/pypy/jit/codewriter/test/test_call.py
--- a/pypy/jit/codewriter/test/test_call.py
+++ b/pypy/jit/codewriter/test/test_call.py
@@ -191,4 +191,4 @@
     [block, _] = list(f_graph.iterblocks())
     [op] = block.operations
     call_descr = cc.getcalldescr(op)
-    assert call_descr.extrainfo.can_release_gil
\ No newline at end of file
+    assert call_descr.extrainfo.has_random_effects()
diff --git a/pypy/jit/codewriter/test/test_codewriter.py b/pypy/jit/codewriter/test/test_codewriter.py
--- a/pypy/jit/codewriter/test/test_codewriter.py
+++ b/pypy/jit/codewriter/test/test_codewriter.py
@@ -5,7 +5,7 @@
 from pypy.rpython.lltypesystem import lltype, llmemory, rffi
 
 class FakeCallDescr(AbstractDescr):
-    def __init__(self, FUNC, ARGS, RESULT, effectinfo=None):
+    def __init__(self, FUNC, ARGS, RESULT, effectinfo):
         self.FUNC = FUNC
         self.ARGS = ARGS
         self.RESULT = RESULT
diff --git a/pypy/jit/codewriter/test/test_flatten.py b/pypy/jit/codewriter/test/test_flatten.py
--- a/pypy/jit/codewriter/test/test_flatten.py
+++ b/pypy/jit/codewriter/test/test_flatten.py
@@ -50,7 +50,7 @@
     def __init__(self, rtyper):
         rtyper._builtin_func_for_spec_cache = FakeDict()
         self.rtyper = rtyper
-    def calldescrof(self, FUNC, ARGS, RESULT):
+    def calldescrof(self, FUNC, ARGS, RESULT, effectinfo):
         return FakeDescr()
     def fielddescrof(self, STRUCT, name):
         return FakeDescr()
@@ -324,7 +324,7 @@
     def test_exc_exitswitch(self):
         def g(i):
             pass
-        
+
         def f(i):
             try:
                 g(i)
@@ -854,13 +854,51 @@
             int_return %i0
         """, transform=True)
 
-    def test_force_cast_float(self):
+    def test_force_cast_floats(self):
         from pypy.rpython.lltypesystem import rffi
+        # Caststs to lltype.Float
         def f(n):
             return rffi.cast(lltype.Float, n)
         self.encoding_test(f, [12.456], """
             float_return %f0
         """, transform=True)
+        self.encoding_test(f, [rffi.cast(rffi.SIGNEDCHAR, 42)], """
+            cast_int_to_float %i0 -> %f0
+            float_return %f0
+        """, transform=True)
+
+        # Casts to lltype.SingleFloat
+        def g(n):
+            return rffi.cast(lltype.SingleFloat, n)
+        self.encoding_test(g, [12.456], """
+            cast_float_to_singlefloat %f0 -> %i0
+            int_return %i0
+        """, transform=True)
+        self.encoding_test(g, [rffi.cast(rffi.SIGNEDCHAR, 42)], """
+            cast_int_to_float %i0 -> %f0
+            cast_float_to_singlefloat %f0 -> %i1
+            int_return %i1
+        """, transform=True)
+
+        # Casts from floats
+        def f(n):
+            return rffi.cast(rffi.SIGNEDCHAR, n)
+        self.encoding_test(f, [12.456], """
+            cast_float_to_int %f0 -> %i0
+            int_sub %i0, $-128 -> %i1
+            int_and %i1, $255 -> %i2
+            int_add %i2, $-128 -> %i3
+            int_return %i3
+        """, transform=True)
+        self.encoding_test(f, [rffi.cast(lltype.SingleFloat, 12.456)], """
+            cast_singlefloat_to_float %i0 -> %f0
+            cast_float_to_int %f0 -> %i1
+            int_sub %i1, $-128 -> %i2
+            int_and %i2, $255 -> %i3
+            int_add %i3, $-128 -> %i4
+            int_return %i4
+        """, transform=True)
+
 
     def test_direct_ptradd(self):
         from pypy.rpython.lltypesystem import rffi
diff --git a/pypy/jit/codewriter/test/test_jtransform.py b/pypy/jit/codewriter/test/test_jtransform.py
--- a/pypy/jit/codewriter/test/test_jtransform.py
+++ b/pypy/jit/codewriter/test/test_jtransform.py
@@ -1048,3 +1048,13 @@
         assert op1.opname == 'jit_force_quasi_immutable'
         assert op1.args[0] == v_x
         assert op1.args[1] == ('fielddescr', STRUCT, 'mutate_x')
+
+def test_no_gcstruct_nesting_outside_of_OBJECT():
+    PARENT = lltype.GcStruct('parent')
+    STRUCT = lltype.GcStruct('struct', ('parent', PARENT),
+                                       ('x', lltype.Signed))
+    v_x = varoftype(lltype.Ptr(STRUCT))
+    op = SpaceOperation('getfield', [v_x, Constant('x', lltype.Void)],
+                        varoftype(lltype.Signed))
+    tr = Transformer(None, None)
+    raises(NotImplementedError, tr.rewrite_operation, op)
diff --git a/pypy/jit/codewriter/test/test_longlong.py b/pypy/jit/codewriter/test/test_longlong.py
--- a/pypy/jit/codewriter/test/test_longlong.py
+++ b/pypy/jit/codewriter/test/test_longlong.py
@@ -230,3 +230,18 @@
             assert list(op1.args[3]) == []
             assert list(op1.args[4]) == vlist
             assert op1.result == v_result
+
+
+##def test_singlefloat_constants():
+##    v_x = varoftype(TYPE)
+##    vlist = [v_x, const(rffi.cast(TYPE, 7))]
+##    v_result = varoftype(TYPE)
+##    op = SpaceOperation('llong_add', vlist, v_result)
+##    tr = Transformer(FakeCPU(), FakeBuiltinCallControl())
+##    op1 = tr.rewrite_operation(op)
+##    #
+##    assert op1.opname == 'residual_call_irf_f'
+##    assert list(op1.args[2]) == []
+##    assert list(op1.args[3]) == []
+##    assert list(op1.args[4]) == vlist
+##    assert op1.result == v_result
diff --git a/pypy/jit/codewriter/test/test_policy.py b/pypy/jit/codewriter/test/test_policy.py
--- a/pypy/jit/codewriter/test/test_policy.py
+++ b/pypy/jit/codewriter/test/test_policy.py
@@ -12,24 +12,30 @@
     graph = support.getgraph(f, [5])
     for sf in [False, True]:
         for sll in [False, True]:
-            assert not contains_unsupported_variable_type(graph, sf, sll)
+            for ssf in [False, True]:
+                assert not contains_unsupported_variable_type(graph, sf,
+                                                              sll, ssf)
     #
     graph = support.getgraph(f, [5.5])
     for sf in [False, True]:
         for sll in [False, True]:
-            res = contains_unsupported_variable_type(graph, sf, sll)
-            assert res is not sf
+            for ssf in [False, True]:
+                res = contains_unsupported_variable_type(graph, sf, sll, ssf)
+                assert res is not sf
     #
     graph = support.getgraph(f, [r_singlefloat(5.5)])
     for sf in [False, True]:
         for sll in [False, True]:
-            assert contains_unsupported_variable_type(graph, sf, sll)
+            for ssf in [False, True]:
+                res = contains_unsupported_variable_type(graph, sf, sll, ssf)
+                assert res == (not ssf)
     #
     graph = support.getgraph(f, [r_longlong(5)])
     for sf in [False, True]:
         for sll in [False, True]:
-            res = contains_unsupported_variable_type(graph, sf, sll)
-            assert res == (sys.maxint == 2147483647 and not sll)
+            for ssf in [False, True]:
+                res = contains_unsupported_variable_type(graph, sf, sll, ssf)
+                assert res == (sys.maxint == 2147483647 and not sll)
 
 
 def test_regular_function():
diff --git a/pypy/jit/metainterp/blackhole.py b/pypy/jit/metainterp/blackhole.py
--- a/pypy/jit/metainterp/blackhole.py
+++ b/pypy/jit/metainterp/blackhole.py
@@ -499,6 +499,9 @@
     @arguments("r", returns="i")
     def bhimpl_ptr_nonzero(a):
         return bool(a)
+    @arguments("r", returns="r")
+    def bhimpl_cast_opaque_ptr(a):
+        return a
 
     @arguments("i", returns="i")
     def bhimpl_int_copy(a):
@@ -622,6 +625,19 @@
         x = float(a)
         return longlong.getfloatstorage(x)
 
+    @arguments("f", returns="i")
+    def bhimpl_cast_float_to_singlefloat(a):
+        from pypy.rlib.rarithmetic import r_singlefloat
+        a = longlong.getrealfloat(a)
+        a = r_singlefloat(a)
+        return longlong.singlefloat2int(a)
+
+    @arguments("i", returns="f")
+    def bhimpl_cast_singlefloat_to_float(a):
+        a = longlong.int2singlefloat(a)
+        a = float(a)
+        return longlong.getfloatstorage(a)
+
     # ----------
     # control flow operations
 
diff --git a/pypy/jit/metainterp/compile.py b/pypy/jit/metainterp/compile.py
--- a/pypy/jit/metainterp/compile.py
+++ b/pypy/jit/metainterp/compile.py
@@ -137,6 +137,10 @@
             jitdriver_sd.warmstate.attach_unoptimized_bridge_from_interp(
                 greenkey, loop.preamble.token)
             record_loop_or_bridge(metainterp_sd, loop.preamble)
+        elif token.short_preamble:
+            short = token.short_preamble[-1]
+            metainterp_sd.logger_ops.log_short_preamble(short.inputargs,
+                                                        short.operations)
         return token
     else:
         send_loop_to_backend(greenkey, jitdriver_sd, metainterp_sd, loop,
@@ -637,6 +641,7 @@
         debug_print("compile_new_bridge: got an InvalidLoop")
         # XXX I am fairly convinced that optimize_bridge cannot actually raise
         # InvalidLoop
+        debug_print('InvalidLoop in compile_new_bridge')
         return None
     # Did it work?
     if target_loop_token is not None:
@@ -668,10 +673,9 @@
     def handle_fail(self, metainterp_sd, jitdriver_sd):
         cpu = metainterp_sd.cpu
         exception = cpu.grab_exc_value()
+        assert exception, "PropagateExceptionDescr: no exception??"
         raise metainterp_sd.ExitFrameWithExceptionRef(cpu, exception)
 
-propagate_exception_descr = PropagateExceptionDescr()
-
 def compile_tmp_callback(cpu, jitdriver_sd, greenboxes, redboxes,
                          memory_manager=None):
     """Make a LoopToken that corresponds to assembler code that just
@@ -705,7 +709,7 @@
         finishargs = []
     #
     jd = jitdriver_sd
-    faildescr = propagate_exception_descr
+    faildescr = PropagateExceptionDescr()
     operations = [
         ResOperation(rop.CALL, callargs, result, descr=jd.portal_calldescr),
         ResOperation(rop.GUARD_NO_EXCEPTION, [], None, descr=faildescr),
diff --git a/pypy/jit/metainterp/executor.py b/pypy/jit/metainterp/executor.py
--- a/pypy/jit/metainterp/executor.py
+++ b/pypy/jit/metainterp/executor.py
@@ -47,7 +47,7 @@
     func = argboxes[0].getint()
     # do the call using the correct function from the cpu
     rettype = descr.get_return_type()
-    if rettype == INT:
+    if rettype == INT or rettype == 'S':       # *S*ingle float
         try:
             result = cpu.bh_call_i(func, descr, args_i, args_r, args_f)
         except Exception, e:
@@ -61,7 +61,7 @@
             metainterp.execute_raised(e)
             result = NULL
         return BoxPtr(result)
-    if rettype == FLOAT or rettype == 'L':
+    if rettype == FLOAT or rettype == 'L':     # *L*ong long
         try:
             result = cpu.bh_call_f(func, descr, args_i, args_r, args_f)
         except Exception, e:
diff --git a/pypy/jit/metainterp/history.py b/pypy/jit/metainterp/history.py
--- a/pypy/jit/metainterp/history.py
+++ b/pypy/jit/metainterp/history.py
@@ -20,12 +20,16 @@
 
 FAILARGS_LIMIT = 1000
 
-def getkind(TYPE, supports_floats=True, supports_longlong=True):
+def getkind(TYPE, supports_floats=True,
+                  supports_longlong=True,
+                  supports_singlefloats=True):
     if TYPE is lltype.Void:
         return "void"
     elif isinstance(TYPE, lltype.Primitive):
         if TYPE is lltype.Float and supports_floats:
             return 'float'
+        if TYPE is lltype.SingleFloat and supports_singlefloats:
+            return 'int'     # singlefloats are stored in an int
         if TYPE in (lltype.Float, lltype.SingleFloat):
             raise NotImplementedError("type %s not supported" % TYPE)
         # XXX fix this for oo...
@@ -145,6 +149,7 @@
         """ Implement in call descr.
         Must return INT, REF, FLOAT, or 'v' for void.
         On 32-bit (hack) it can also be 'L' for longlongs.
+        Additionally it can be 'S' for singlefloats.
         """
         raise NotImplementedError
 
diff --git a/pypy/jit/metainterp/optimize.py b/pypy/jit/metainterp/optimize.py
--- a/pypy/jit/metainterp/optimize.py
+++ b/pypy/jit/metainterp/optimize.py
@@ -1,4 +1,4 @@
-from pypy.rlib.debug import debug_start, debug_stop
+from pypy.rlib.debug import debug_start, debug_stop, debug_print
 from pypy.jit.metainterp.jitexc import JitException
 
 class InvalidLoop(JitException):
diff --git a/pypy/jit/metainterp/optimizeopt/__init__.py b/pypy/jit/metainterp/optimizeopt/__init__.py
--- a/pypy/jit/metainterp/optimizeopt/__init__.py
+++ b/pypy/jit/metainterp/optimizeopt/__init__.py
@@ -33,10 +33,6 @@
         if name in enable_opts:
             if opt is not None:
                 o = opt()
-                if unroll and name == 'string':
-                    o.enabled = False
-                # FIXME: Workaround to disable string optimisation
-                # during preamble but to keep it during the loop
                 optimizations.append(o)
             elif name == 'ffi' and config.translation.jit_ffi:
                 # we cannot put the class directly in the unrolling_iterable,
diff --git a/pypy/jit/metainterp/optimizeopt/fficall.py b/pypy/jit/metainterp/optimizeopt/fficall.py
--- a/pypy/jit/metainterp/optimizeopt/fficall.py
+++ b/pypy/jit/metainterp/optimizeopt/fficall.py
@@ -1,12 +1,11 @@
 from pypy.rpython.annlowlevel import cast_base_ptr_to_instance
 from pypy.rlib.objectmodel import we_are_translated
 from pypy.rlib.libffi import Func
-from pypy.rlib.debug import debug_start, debug_stop, debug_print
+from pypy.rlib.debug import debug_print
 from pypy.jit.codewriter.effectinfo import EffectInfo
 from pypy.jit.metainterp.resoperation import rop, ResOperation
 from pypy.jit.metainterp.optimizeopt.util import make_dispatcher_method
 from pypy.jit.metainterp.optimizeopt.optimizer import Optimization
-from pypy.jit.backend.llsupport.ffisupport import UnsupportedKind
 
 
 class FuncInfo(object):
@@ -19,28 +18,27 @@
     def __init__(self, funcval, cpu, prepare_op):
         self.funcval = funcval
         self.opargs = []
-        argtypes, restype = self._get_signature(funcval)
-        try:
-            self.descr = cpu.calldescrof_dynamic(argtypes, restype)
-        except UnsupportedKind:
-            # e.g., I or U for long longs
-            self.descr = None
+        argtypes, restype, flags = self._get_signature(funcval)
+        self.descr = cpu.calldescrof_dynamic(argtypes, restype,
+                                             EffectInfo.MOST_GENERAL,
+                                             ffi_flags=flags)
+        # ^^^ may be None if unsupported
         self.prepare_op = prepare_op
         self.delayed_ops = []
 
     def _get_signature(self, funcval):
         """
-        given the funcval, return a tuple (argtypes, restype), where the
-        actuall types are libffi.types.*
+        given the funcval, return a tuple (argtypes, restype, flags), where
+        the actuall types are libffi.types.*
 
         The implementation is tricky because we have three possible cases:
 
         - translated: the easiest case, we can just cast back the pointer to
-          the original Func instance and read .argtypes and .restype
+          the original Func instance and read .argtypes, .restype and .flags
 
         - completely untranslated: this is what we get from test_optimizeopt
           tests. funcval contains a FakeLLObject whose _fake_class is Func,
-          and we can just get .argtypes and .restype
+          and we can just get .argtypes, .restype and .flags
 
         - partially translated: this happens when running metainterp tests:
           funcval contains the low-level equivalent of a Func, and thus we
@@ -52,10 +50,10 @@
         llfunc = funcval.box.getref_base()
         if we_are_translated():
             func = cast_base_ptr_to_instance(Func, llfunc)
-            return func.argtypes, func.restype
+            return func.argtypes, func.restype, func.flags
         elif getattr(llfunc, '_fake_class', None) is Func:
             # untranslated
-            return llfunc.argtypes, llfunc.restype
+            return llfunc.argtypes, llfunc.restype, llfunc.flags
         else:
             # partially translated
             # llfunc contains an opaque pointer to something like the following:
@@ -66,7 +64,7 @@
             # because we don't have the exact TYPE to cast to.  Instead, we
             # just fish it manually :-(
             f = llfunc._obj.container
-            return f.inst_argtypes, f.inst_restype
+            return f.inst_argtypes, f.inst_restype, f.inst_flags
 
 
 class OptFfiCall(Optimization):
@@ -78,18 +76,9 @@
         else:
             self.logops = None
 
-    def propagate_begin_forward(self):
-        debug_start('jit-log-ffiopt')
-        Optimization.propagate_begin_forward(self)
-
-    def propagate_end_forward(self):
-        debug_stop('jit-log-ffiopt')
-        Optimization.propagate_end_forward(self)
-
-    def reconstruct_for_next_iteration(self, optimizer, valuemap):
+    def new(self):
         return OptFfiCall()
-        # FIXME: Should any status be saved for next iteration?
-
+    
     def begin_optimization(self, funcval, op):
         self.rollback_maybe('begin_optimization', op)
         self.funcinfo = FuncInfo(funcval, self.optimizer.cpu, op)
@@ -184,7 +173,8 @@
     def do_call(self, op):
         funcval = self._get_funcval(op)
         funcinfo = self.funcinfo
-        if not funcinfo or funcinfo.funcval is not funcval:
+        if (not funcinfo or funcinfo.funcval is not funcval or
+            funcinfo.descr is None):
             return [op] # cannot optimize
         funcsymval = self.getvalue(op.getarg(2))
         arglist = [funcsymval.force_box()]
@@ -207,9 +197,7 @@
 
     def _get_oopspec(self, op):
         effectinfo = op.getdescr().get_extra_info()
-        if effectinfo is not None:
-            return effectinfo.oopspecindex
-        return EffectInfo.OS_NONE
+        return effectinfo.oopspecindex
 
     def _get_funcval(self, op):
         return self.getvalue(op.getarg(1))
diff --git a/pypy/jit/metainterp/optimizeopt/generalize.py b/pypy/jit/metainterp/optimizeopt/generalize.py
new file mode 100644
--- /dev/null
+++ b/pypy/jit/metainterp/optimizeopt/generalize.py
@@ -0,0 +1,19 @@
+from pypy.jit.metainterp.optimizeopt.optimizer import MININT, MAXINT
+
+class GeneralizationStrategy(object):
+    def __init__(self, optimizer):
+        self.optimizer = optimizer
+
+    def apply(self):
+        raise NotImplementedError
+
+class KillHugeIntBounds(GeneralizationStrategy):
+    def apply(self):
+        for v in self.optimizer.values.values():
+            if v.is_constant():
+                continue
+            if v.intbound.lower < MININT/2:
+                v.intbound.lower = MININT
+            if v.intbound.upper > MAXINT/2:
+                v.intbound.upper = MAXINT
+          
diff --git a/pypy/jit/metainterp/optimizeopt/heap.py b/pypy/jit/metainterp/optimizeopt/heap.py
--- a/pypy/jit/metainterp/optimizeopt/heap.py
+++ b/pypy/jit/metainterp/optimizeopt/heap.py
@@ -1,9 +1,10 @@
 import os
 
 from pypy.jit.metainterp.jitexc import JitException
-from pypy.jit.metainterp.optimizeopt.optimizer import Optimization
+from pypy.jit.metainterp.optimizeopt.optimizer import Optimization, MODE_ARRAY
+from pypy.jit.metainterp.history import ConstInt, Const
 from pypy.jit.metainterp.optimizeopt.util import make_dispatcher_method
-from pypy.jit.metainterp.resoperation import rop
+from pypy.jit.metainterp.resoperation import rop, ResOperation
 from pypy.rlib.objectmodel import we_are_translated
 
 
@@ -24,6 +25,7 @@
         #      'cached_fields'.
         #
         self._cached_fields = {}
+        self._cached_fields_getfield_op = {}        
         self._lazy_setfield = None
         self._lazy_setfield_registered = False
 
@@ -70,9 +72,10 @@
         else:
             return self._cached_fields.get(structvalue, None)
 
-    def remember_field_value(self, structvalue, fieldvalue):
+    def remember_field_value(self, structvalue, fieldvalue, getfield_op=None):
         assert self._lazy_setfield is None
         self._cached_fields[structvalue] = fieldvalue
+        self._cached_fields_getfield_op[structvalue] = getfield_op        
 
     def force_lazy_setfield(self, optheap, can_cache=True):
         op = self._lazy_setfield
@@ -81,7 +84,7 @@
             # Now we clear _cached_fields, because actually doing the
             # setfield might impact any of the stored result (because of
             # possible aliasing).
-            self._cached_fields.clear()
+            self.clear()
             self._lazy_setfield = None
             optheap.next_optimization.propagate_forward(op)
             if not can_cache:
@@ -91,19 +94,49 @@
             # field.
             structvalue = optheap.getvalue(op.getarg(0))
             fieldvalue  = optheap.getvalue(op.getarglist()[-1])
-            self.remember_field_value(structvalue, fieldvalue)
+            self.remember_field_value(structvalue, fieldvalue, op)
         elif not can_cache:
-            self._cached_fields.clear()
+            self.clear()
 
-    def get_reconstructed(self, optimizer, valuemap):
-        assert self._lazy_setfield is None
-        cf = CachedField()
-        for structvalue, fieldvalue in self._cached_fields.iteritems():
-            structvalue2 = structvalue.get_reconstructed(optimizer, valuemap)
-            fieldvalue2  = fieldvalue .get_reconstructed(optimizer, valuemap)
-            cf._cached_fields[structvalue2] = fieldvalue2
-        return cf
+    def clear(self):
+        self._cached_fields.clear()
+        self._cached_fields_getfield_op.clear()
 
+    def turned_constant(self, newvalue, value):
+        if newvalue not in self._cached_fields and value in self._cached_fields:
+            self._cached_fields[newvalue] = self._cached_fields[value]
+            op = self._cached_fields_getfield_op[value].clone()
+            constbox = value.box
+            assert isinstance(constbox, Const)
+            op.setarg(0, constbox)
+            self._cached_fields_getfield_op[newvalue] = op
+        for structvalue in self._cached_fields.keys():
+            if self._cached_fields[structvalue] is value:
+                self._cached_fields[structvalue] = newvalue
+
+    def produce_potential_short_preamble_ops(self, optimizer, shortboxes, descr):
+        if self._lazy_setfield is not None:
+            return
+        for structvalue in self._cached_fields_getfield_op.keys():
+            op = self._cached_fields_getfield_op[structvalue]
+            if not op:
+                continue
+            if optimizer.getvalue(op.getarg(0)) in optimizer.opaque_pointers:
+                continue
+            if structvalue in self._cached_fields:
+                if op.getopnum() == rop.SETFIELD_GC:
+                    result = op.getarg(1)
+                    if isinstance(result, Const):
+                        newresult = result.clonebox()
+                        optimizer.make_constant(newresult, result)
+                        result = newresult
+                    getop = ResOperation(rop.GETFIELD_GC, [op.getarg(0)],
+                                         result, op.getdescr())
+                    getop = shortboxes.add_potential(getop)
+                    self._cached_fields_getfield_op[structvalue] = getop
+                    self._cached_fields[structvalue] = optimizer.getvalue(result)
+                elif op.result is not None:
+                    shortboxes.add_potential(op)
 
 class BogusPureField(JitException):
     pass
@@ -122,24 +155,32 @@
         self._remove_guard_not_invalidated = False
         self._seen_guard_not_invalidated = False
 
-    def reconstruct_for_next_iteration(self, optimizer, valuemap):
-        new = OptHeap()
+    def force_at_end_of_preamble(self):
+        self.force_all_lazy_setfields_and_arrayitems()
 
-        if True:
-            self.force_all_lazy_setfields_and_arrayitems()
-        else:
-            assert 0   # was: new.lazy_setfields = self.lazy_setfields
+    def flush(self):
+        self.force_all_lazy_setfields_and_arrayitems()
 
-        for descr, d in self.cached_fields.items():
-            new.cached_fields[descr] = d.get_reconstructed(optimizer, valuemap)
+    def new(self):
+        return OptHeap()
+        
+    def produce_potential_short_preamble_ops(self, sb):
+        descrkeys = self.cached_fields.keys()
+        if not we_are_translated():
+            # XXX Pure operation of boxes that are cached in several places will
+            #     only be removed from the peeled loop when red from the first
+            #     place discovered here. This is far from ideal, as it makes
+            #     the effectiveness of our optimization a bit random. It should
+            #     howevere always generate correct results. For tests we dont
+            #     want this randomness.
+            descrkeys.sort(key=str, reverse=True)
+        for descr in descrkeys:
+            d = self.cached_fields[descr]
+            d.produce_potential_short_preamble_ops(self.optimizer, sb, descr)
 
         for descr, submap in self.cached_arrayitems.items():
-            newdict = {}
             for index, d in submap.items():
-                newdict[index] = d.get_reconstructed(optimizer, valuemap)
-            new.cached_arrayitems[descr] = newdict
-
-        return new
+                d.produce_potential_short_preamble_ops(self.optimizer, sb, descr)
 
     def clean_caches(self):
         del self._lazy_setfields_and_arrayitems[:]
@@ -194,43 +235,43 @@
             opnum == rop.CALL_RELEASE_GIL or
             opnum == rop.CALL_ASSEMBLER):
             if opnum == rop.CALL_ASSEMBLER:
-                effectinfo = None
+                self._seen_guard_not_invalidated = False
             else:
                 effectinfo = op.getdescr().get_extra_info()
-            if effectinfo is None or effectinfo.check_can_invalidate():
-                self._seen_guard_not_invalidated = False
-            if effectinfo is not None and not effectinfo.has_random_effects():
-                # XXX we can get the wrong complexity here, if the lists
-                # XXX stored on effectinfo are large
-                for fielddescr in effectinfo.readonly_descrs_fields:
-                    self.force_lazy_setfield(fielddescr)
-                for arraydescr in effectinfo.readonly_descrs_arrays:
-                    self.force_lazy_setarrayitem(arraydescr)
-                for fielddescr in effectinfo.write_descrs_fields:
-                    self.force_lazy_setfield(fielddescr, can_cache=False)
-                for arraydescr in effectinfo.write_descrs_arrays:
-                    self.force_lazy_setarrayitem(arraydescr, can_cache=False)
-                if effectinfo.check_forces_virtual_or_virtualizable():
-                    vrefinfo = self.optimizer.metainterp_sd.virtualref_info
-                    self.force_lazy_setfield(vrefinfo.descr_forced)
-                    # ^^^ we only need to force this field; the other fields
-                    # of virtualref_info and virtualizable_info are not gcptrs.
-                return
+                if effectinfo.check_can_invalidate():
+                    self._seen_guard_not_invalidated = False
+                if not effectinfo.has_random_effects():
+                    self.force_from_effectinfo(effectinfo)
+                    return
         self.force_all_lazy_setfields_and_arrayitems()
         self.clean_caches()
 
+    def force_from_effectinfo(self, effectinfo):
+        # XXX we can get the wrong complexity here, if the lists
+        # XXX stored on effectinfo are large
+        for fielddescr in effectinfo.readonly_descrs_fields:
+            self.force_lazy_setfield(fielddescr)
+        for arraydescr in effectinfo.readonly_descrs_arrays:
+            self.force_lazy_setarrayitem(arraydescr)
+        for fielddescr in effectinfo.write_descrs_fields:
+            self.force_lazy_setfield(fielddescr, can_cache=False)
+        for arraydescr in effectinfo.write_descrs_arrays:
+            self.force_lazy_setarrayitem(arraydescr, can_cache=False)
+        if effectinfo.check_forces_virtual_or_virtualizable():
+            vrefinfo = self.optimizer.metainterp_sd.virtualref_info
+            self.force_lazy_setfield(vrefinfo.descr_forced)
+            # ^^^ we only need to force this field; the other fields
+            # of virtualref_info and virtualizable_info are not gcptrs.
 
     def turned_constant(self, value):
         assert value.is_constant()
         newvalue = self.getvalue(value.box)
         if value is not newvalue:
             for cf in self.cached_fields.itervalues():
-                if value in cf._cached_fields:
-                    cf._cached_fields[newvalue] = cf._cached_fields[value]
+                cf.turned_constant(newvalue, value)
             for submap in self.cached_arrayitems.itervalues():
                 for cf in submap.itervalues():
-                    if value in cf._cached_fields:
-                        cf._cached_fields[newvalue] = cf._cached_fields[value]
+                    cf.turned_constant(newvalue, value)
 
     def force_lazy_setfield(self, descr, can_cache=True):
         try:
@@ -239,13 +280,14 @@
             return
         cf.force_lazy_setfield(self, can_cache)
 
-    def force_lazy_setarrayitem(self, arraydescr, can_cache=True):
+    def force_lazy_setarrayitem(self, arraydescr, indexvalue=None, can_cache=True):
         try:
             submap = self.cached_arrayitems[arraydescr]
         except KeyError:
             return
-        for cf in submap.values():
-            cf.force_lazy_setfield(self, can_cache)
+        for idx, cf in submap.iteritems():
+            if indexvalue is None or indexvalue.intbound.contains(idx):
+                cf.force_lazy_setfield(self, can_cache)
 
     def fixup_guard_situation(self):
         # hackish: reverse the order of the last two operations if it makes
@@ -332,7 +374,7 @@
         self.emit_operation(op)
         # then remember the result of reading the field
         fieldvalue = self.getvalue(op.result)
-        cf.remember_field_value(structvalue, fieldvalue)
+        cf.remember_field_value(structvalue, fieldvalue, op)
 
     def optimize_SETFIELD_GC(self, op):
         if self.has_pure_result(rop.GETFIELD_GC_PURE, [op.getarg(0)],
@@ -349,6 +391,7 @@
         indexvalue = self.getvalue(op.getarg(1))
         cf = None
         if indexvalue.is_constant():
+            arrayvalue.make_len_gt(MODE_ARRAY, op.getdescr(), indexvalue.box.getint())
             # use the cache on (arraydescr, index), which is a constant
             cf = self.arrayitem_cache(op.getdescr(), indexvalue.box.getint())
             fieldvalue = cf.getfield_from_cache(self, arrayvalue)
@@ -357,14 +400,14 @@
                 return
         else:
             # variable index, so make sure the lazy setarrayitems are done
-            self.force_lazy_setarrayitem(op.getdescr())
+            self.force_lazy_setarrayitem(op.getdescr(), indexvalue=indexvalue)
         # default case: produce the operation
         arrayvalue.ensure_nonnull()
         self.emit_operation(op)
         # the remember the result of reading the array item
         if cf is not None:
             fieldvalue = self.getvalue(op.result)
-            cf.remember_field_value(arrayvalue, fieldvalue)
+            cf.remember_field_value(arrayvalue, fieldvalue, op)
 
     def optimize_SETARRAYITEM_GC(self, op):
         if self.has_pure_result(rop.GETARRAYITEM_GC_PURE, [op.getarg(0),
@@ -376,12 +419,14 @@
         #
         indexvalue = self.getvalue(op.getarg(1))
         if indexvalue.is_constant():
+            arrayvalue = self.getvalue(op.getarg(0))
+            arrayvalue.make_len_gt(MODE_ARRAY, op.getdescr(), indexvalue.box.getint())
             # use the cache on (arraydescr, index), which is a constant
             cf = self.arrayitem_cache(op.getdescr(), indexvalue.box.getint())
             cf.do_setfield(self, op)
         else:
             # variable index, so make sure the lazy setarrayitems are done
-            self.force_lazy_setarrayitem(op.getdescr(), can_cache=False)
+            self.force_lazy_setarrayitem(op.getdescr(), indexvalue=indexvalue, can_cache=False)
             # and then emit the operation
             self.emit_operation(op)
 
diff --git a/pypy/jit/metainterp/optimizeopt/intbounds.py b/pypy/jit/metainterp/optimizeopt/intbounds.py
--- a/pypy/jit/metainterp/optimizeopt/intbounds.py
+++ b/pypy/jit/metainterp/optimizeopt/intbounds.py
@@ -1,7 +1,8 @@
+from pypy.jit.metainterp.optimizeopt.optimizer import Optimization, CONST_1, CONST_0, \
+                                                  MODE_ARRAY, MODE_STR, MODE_UNICODE
 from pypy.jit.metainterp.history import ConstInt
 from pypy.jit.metainterp.optimizeopt.intutils import (IntBound, IntLowerBound,
     IntUpperBound)
-from pypy.jit.metainterp.optimizeopt.optimizer import Optimization, CONST_1, CONST_0
 from pypy.jit.metainterp.optimizeopt.util import make_dispatcher_method
 from pypy.jit.metainterp.resoperation import rop
 
@@ -14,18 +15,17 @@
         self.posponedop = None
         self.nextop = None
 
-    def reconstruct_for_next_iteration(self, optimizer, valuemap):
+    def new(self):
         assert self.posponedop is None
-        return self
+        return OptIntBounds()
+        
+    def flush(self):
+        assert self.posponedop is None
 
     def setup(self):
         self.posponedop = None
         self.nextop = None
 
-    def reconstruct_for_next_iteration(self, optimizer, valuemap):
-        assert self.posponedop is None
-        return self
-
     def propagate_forward(self, op):
         if op.is_ovf():
             self.posponedop = op
@@ -125,6 +125,17 @@
         r = self.getvalue(op.result)
         r.intbound.intersect(v1.intbound.div_bound(v2.intbound))
 
+    def optimize_INT_MOD(self, op):
+        self.emit_operation(op)
+        v2 = self.getvalue(op.getarg(1))
+        if v2.is_constant():
+            val = v2.box.getint()
+            r = self.getvalue(op.result)
+            if val < 0:
+                val = -val
+            r.intbound.make_gt(IntBound(-val, -val))
+            r.intbound.make_lt(IntBound(val, val))
+
     def optimize_INT_LSHIFT(self, op):
         v1 = self.getvalue(op.getarg(0))
         v2 = self.getvalue(op.getarg(1))
@@ -275,10 +286,27 @@
 
     def optimize_ARRAYLEN_GC(self, op):
         self.emit_operation(op)
-        v1 = self.getvalue(op.result)
-        v1.intbound.make_ge(IntLowerBound(0))
+        array  = self.getvalue(op.getarg(0))
+        result = self.getvalue(op.result)
+        array.make_len_gt(MODE_ARRAY, op.getdescr(), -1)
+        array.lenbound.bound.intersect(result.intbound)
+        result.intbound = array.lenbound.bound
 
-    optimize_STRLEN = optimize_UNICODELEN = optimize_ARRAYLEN_GC
+    def optimize_STRLEN(self, op):
+        self.emit_operation(op)
+        array  = self.getvalue(op.getarg(0))
+        result = self.getvalue(op.result)
+        array.make_len_gt(MODE_STR, op.getdescr(), -1)
+        array.lenbound.bound.intersect(result.intbound)
+        result.intbound = array.lenbound.bound
+
+    def optimize_UNICODELEN(self, op):
+        self.emit_operation(op)
+        array  = self.getvalue(op.getarg(0))
+        result = self.getvalue(op.result)
+        array.make_len_gt(MODE_UNICODE, op.getdescr(), -1)
+        array.lenbound.bound.intersect(result.intbound)
+        result.intbound = array.lenbound.bound
 
     def optimize_STRGETITEM(self, op):
         self.emit_operation(op)
diff --git a/pypy/jit/metainterp/optimizeopt/intutils.py b/pypy/jit/metainterp/optimizeopt/intutils.py
--- a/pypy/jit/metainterp/optimizeopt/intutils.py
+++ b/pypy/jit/metainterp/optimizeopt/intutils.py
@@ -1,4 +1,9 @@
 from pypy.rlib.rarithmetic import ovfcheck, ovfcheck_lshift, LONG_BIT
+from pypy.jit.metainterp.resoperation import rop, ResOperation
+from pypy.jit.metainterp.history import BoxInt, ConstInt
+import sys
+MAXINT = sys.maxint
+MININT = -sys.maxint - 1
 
 class IntBound(object):
     _attrs_ = ('has_upper', 'has_lower', 'upper', 'lower')
@@ -210,11 +215,11 @@
         
     def __repr__(self):
         if self.has_lower:
-            l = '%4d' % self.lower
+            l = '%d' % self.lower
         else:
             l = '-Inf'
         if self.has_upper:
-            u = '%3d' % self.upper
+            u = '%d' % self.upper
         else:
             u = 'Inf'
         return '%s <= x <= %s' % (l, u)
@@ -224,7 +229,24 @@
         res.has_lower = self.has_lower
         res.has_upper = self.has_upper
         return res
+
+    def make_guards(self, box, guards):
+        if self.has_lower and self.lower > MININT:
+            bound = self.lower
+            res = BoxInt()
+            op = ResOperation(rop.INT_GE, [box, ConstInt(bound)], res)
+            guards.append(op)
+            op = ResOperation(rop.GUARD_TRUE, [res], None)
+            guards.append(op)
+        if self.has_upper and self.upper < MAXINT:
+            bound = self.upper
+            res = BoxInt()
+            op = ResOperation(rop.INT_LE, [box, ConstInt(bound)], res)
+            guards.append(op)
+            op = ResOperation(rop.GUARD_TRUE, [res], None)
+            guards.append(op)
     
+
 class IntUpperBound(IntBound):
     def __init__(self, upper):
         self.has_upper = True
@@ -244,7 +266,23 @@
         self.has_upper = False
         self.has_lower = False
         self.upper = 0
-        self.lower = 0        
+        self.lower = 0
+
+class ImmutableIntUnbounded(IntUnbounded):
+    def _raise(self):
+        raise TypeError('ImmutableIntUnbounded is immutable')
+    def make_le(self, other):
+        self._raise()
+    def make_lt(self, other):
+        self._raise()
+    def make_ge(self, other):
+        self._raise()
+    def make_gt(self, other):
+        self._raise()
+    def make_constant(self, value):
+        self._raise()
+    def intersect(self, other):        
+        self._raise()
 
 def min4(t):
     return min(min(t[0], t[1]), min(t[2], t[3]))
diff --git a/pypy/jit/metainterp/optimizeopt/optimizer.py b/pypy/jit/metainterp/optimizeopt/optimizer.py
--- a/pypy/jit/metainterp/optimizeopt/optimizer.py
+++ b/pypy/jit/metainterp/optimizeopt/optimizer.py
@@ -1,64 +1,105 @@
 from pypy.jit.metainterp import jitprof, resume, compile
 from pypy.jit.metainterp.executor import execute_nonspec
 from pypy.jit.metainterp.history import BoxInt, BoxFloat, Const, ConstInt, REF
-from pypy.jit.metainterp.optimizeopt.intutils import IntBound, IntUnbounded
+from pypy.jit.metainterp.optimizeopt.intutils import IntBound, IntUnbounded, \
+                                                     ImmutableIntUnbounded, \
+                                                     IntLowerBound, MININT, MAXINT
 from pypy.jit.metainterp.optimizeopt.util import (make_dispatcher_method,
     args_dict)
 from pypy.jit.metainterp.resoperation import rop, ResOperation
 from pypy.jit.metainterp.typesystem import llhelper, oohelper
 from pypy.tool.pairtype import extendabletype
+from pypy.rlib.debug import debug_start, debug_stop, debug_print
 
 LEVEL_UNKNOWN    = '\x00'
 LEVEL_NONNULL    = '\x01'
 LEVEL_KNOWNCLASS = '\x02'     # might also mean KNOWNARRAYDESCR, for arrays
 LEVEL_CONSTANT   = '\x03'
 
-import sys
-MAXINT = sys.maxint
-MININT = -sys.maxint - 1
+MODE_ARRAY   = '\x00'
+MODE_STR     = '\x01'
+MODE_UNICODE = '\x02'
+class LenBound(object):
+    def __init__(self, mode, descr, bound):
+        self.mode = mode
+        self.descr = descr
+        self.bound = bound
 
 class OptValue(object):
     __metaclass__ = extendabletype
-    _attrs_ = ('box', 'known_class', 'last_guard_index', 'level', 'intbound')
+    _attrs_ = ('box', 'known_class', 'last_guard_index', 'level', 'intbound', 'lenbound')
     last_guard_index = -1
 
     level = LEVEL_UNKNOWN
     known_class = None
-    intbound = None
+    intbound = ImmutableIntUnbounded()
+    lenbound = None
 
-    def __init__(self, box):
+    def __init__(self, box, level=None, known_class=None, intbound=None):
         self.box = box
-        self.intbound = IntBound(MININT, MAXINT) #IntUnbounded()
+        if level is not None:
+            self.level = level
+        self.known_class = known_class
+        if intbound:
+            self.intbound = intbound
+        else:
+            if isinstance(box, BoxInt):
+                self.intbound = IntBound(MININT, MAXINT)
+            else:
+                self.intbound = IntUnbounded()
+
         if isinstance(box, Const):
             self.make_constant(box)
         # invariant: box is a Const if and only if level == LEVEL_CONSTANT
 
+    def make_len_gt(self, mode, descr, val):
+        if self.lenbound:
+            assert self.lenbound.mode == mode
+            assert self.lenbound.descr == descr
+            self.lenbound.bound.make_gt(IntBound(val, val))
+        else:
+            self.lenbound = LenBound(mode, descr, IntLowerBound(val + 1))
+
+    def make_guards(self, box):
+        guards = []
+        if self.level == LEVEL_CONSTANT:
+            op = ResOperation(rop.GUARD_VALUE, [box, self.box], None)
+            guards.append(op)
+        elif self.level == LEVEL_KNOWNCLASS:
+            op = ResOperation(rop.GUARD_NONNULL, [box], None)
+            guards.append(op)            
+            op = ResOperation(rop.GUARD_CLASS, [box, self.known_class], None)
+            guards.append(op)
+        else:
+            if self.level == LEVEL_NONNULL:
+                op = ResOperation(rop.GUARD_NONNULL, [box], None)
+                guards.append(op)
+            self.intbound.make_guards(box, guards)
+            if self.lenbound:
+                lenbox = BoxInt()
+                if self.lenbound.mode == MODE_ARRAY:
+                    op = ResOperation(rop.ARRAYLEN_GC, [box], lenbox, self.lenbound.descr)
+                elif self.lenbound.mode == MODE_STR:
+                    op = ResOperation(rop.STRLEN, [box], lenbox, self.lenbound.descr)
+                elif self.lenbound.mode == MODE_UNICODE:
+                    op = ResOperation(rop.UNICODELEN, [box], lenbox, self.lenbound.descr)
+                else:
+                    debug_print("Unknown lenbound mode")
+                    assert False
+                guards.append(op)
+                self.lenbound.bound.make_guards(lenbox, guards)
+
+        return guards
+
     def force_box(self):
         return self.box
 
     def get_key_box(self):
         return self.box
 
-    def enum_forced_boxes(self, boxes, already_seen):
-        key = self.get_key_box()
-        if key not in already_seen:
-            boxes.append(self.force_box())
-            already_seen[self.get_key_box()] = None
-
-    def get_reconstructed(self, optimizer, valuemap):
-        if self in valuemap:
-            return valuemap[self]
-        new = self.reconstruct_for_next_iteration(optimizer)
-        valuemap[self] = new
-        self.reconstruct_childs(new, valuemap)
-        return new
-
-    def reconstruct_for_next_iteration(self, optimizer):
+    def force_at_end_of_preamble(self, already_forced):
         return self
 
-    def reconstruct_childs(self, new, valuemap):
-        pass
-
     def get_args_for_fail(self, modifier):
         pass
 
@@ -82,6 +123,7 @@
         assert isinstance(constbox, Const)
         self.box = constbox
         self.level = LEVEL_CONSTANT
+        
         if isinstance(constbox, ConstInt):
             val = constbox.getint()
             self.intbound = IntBound(val, val)
@@ -222,7 +264,9 @@
 
     def pure(self, opnum, args, result):
         op = ResOperation(opnum, args, result)
-        self.optimizer.pure_operations[self.optimizer.make_args_key(op)] = op
+        key = self.optimizer.make_args_key(op)
+        if key not in self.optimizer.pure_operations:
+            self.optimizer.pure_operations[key] = op
 
     def has_pure_result(self, opnum, args, descr):
         op = ResOperation(opnum, args, None, descr)
@@ -235,16 +279,22 @@
     def setup(self):
         pass
 
+    def turned_constant(self, value):
+        pass
+
     def force_at_end_of_preamble(self):
         pass
 
-    def turned_constant(self, value):
+    # It is too late to force stuff here, it must be done in force_at_end_of_preamble
+    def new(self):
+        raise NotImplementedError
+
+    # Called after last operation has been propagated to flush out any posponed ops
+    def flush(self):
         pass
 
-    def reconstruct_for_next_iteration(self, optimizer=None, valuemap=None):
-        #return self.__class__()
-        raise NotImplementedError
-
+    def produce_potential_short_preamble_ops(self, potential_ops):
+        pass
 
 class Optimizer(Optimization):
 
@@ -257,14 +307,17 @@
         self.interned_refs = self.cpu.ts.new_ref_dict()
         self.resumedata_memo = resume.ResumeDataLoopMemo(metainterp_sd)
         self.bool_boxes = {}
-        self.loop_invariant_results = {}
         self.pure_operations = args_dict()
+        self.emitted_pure_operations = {}
         self.producer = {}
         self.pendingfields = []
         self.posponedop = None
         self.exception_might_have_happened = False
         self.quasi_immutable_deps = None
+        self.opaque_pointers = {}
         self.newoperations = []
+        self.emitting_dissabled = False
+        self.emitted_guards = 0        
         if loop is not None:
             self.call_pure_results = loop.call_pure_results
 
@@ -286,39 +339,32 @@
         self.optimizations  = optimizations
 
     def force_at_end_of_preamble(self):
-        self.resumedata_memo = resume.ResumeDataLoopMemo(self.metainterp_sd)
         for o in self.optimizations:
             o.force_at_end_of_preamble()
 
-    def reconstruct_for_next_iteration(self, optimizer=None, valuemap=None):
-        assert optimizer is None
-        assert valuemap is None
-        valuemap = {}
+    def flush(self):
+        for o in self.optimizations:
+            o.flush()
+        assert self.posponedop is None
+
+    def new(self):
+        assert self.posponedop is None
         new = Optimizer(self.metainterp_sd, self.loop)
-        optimizations = [o.reconstruct_for_next_iteration(new, valuemap) for o in
-                         self.optimizations]
+        optimizations = [o.new() for o in self.optimizations]
         new.set_optimizations(optimizations)
-
-        new.values = {}
-        for box, value in self.values.items():
-            new.values[box] = value.get_reconstructed(new, valuemap)
-        new.interned_refs = self.interned_refs
-        new.bool_boxes = {}
-        for value in new.bool_boxes.keys():
-            new.bool_boxes[value.get_reconstructed(new, valuemap)] = None
-
-        # FIXME: Move to rewrite.py
-        new.loop_invariant_results = {}
-        for key, value in self.loop_invariant_results.items():
-            new.loop_invariant_results[key] = \
-                                 value.get_reconstructed(new, valuemap)
-
-        new.pure_operations = self.pure_operations
-        new.producer = self.producer
-        assert self.posponedop is None
         new.quasi_immutable_deps = self.quasi_immutable_deps
-
         return new
+        
+    def produce_potential_short_preamble_ops(self, sb):
+        for op in self.emitted_pure_operations:
+            if op.getopnum() == rop.GETARRAYITEM_GC_PURE or \
+               op.getopnum() == rop.STRGETITEM or \
+               op.getopnum() == rop.UNICODEGETITEM:
+                if not self.getvalue(op.getarg(1)).is_constant():
+                    continue
+            sb.add_potential(op)
+        for opt in self.optimizations:
+            opt.produce_potential_short_preamble_ops(sb)
 
     def turned_constant(self, value):
         for o in self.optimizations:
@@ -433,10 +479,11 @@
         return True
 
     def emit_operation(self, op):
-        ###self.heap_op_optimizer.emitting_operation(op)
-        self._emit_operation(op)
-
-    def _emit_operation(self, op):
+        if op.returns_bool_result():
+            self.bool_boxes[self.getvalue(op.result)] = None
+        if self.emitting_dissabled:
+            return
+        
         for i in range(op.numargs()):
             arg = op.getarg(i)
             if arg in self.values:
@@ -445,11 +492,10 @@
         self.metainterp_sd.profiler.count(jitprof.OPT_OPS)
         if op.is_guard():
             self.metainterp_sd.profiler.count(jitprof.OPT_GUARDS)
+            self.emitted_guards += 1 # FIXME: can we reuse above counter?
             op = self.store_final_boxes_in_guard(op)
         elif op.can_raise():
             self.exception_might_have_happened = True
-        elif op.returns_bool_result():
-            self.bool_boxes[self.getvalue(op.result)] = None
         self.newoperations.append(op)
 
     def store_final_boxes_in_guard(self, op):
@@ -533,6 +579,7 @@
                 return
             else:
                 self.pure_operations[args] = op
+                self.emitted_pure_operations[op] = True
 
         # otherwise, the operation remains
         self.emit_operation(op)
@@ -555,6 +602,35 @@
     def optimize_DEBUG_MERGE_POINT(self, op):
         self.emit_operation(op)
 
+    def optimize_CAST_OPAQUE_PTR(self, op):
+        value = self.getvalue(op.getarg(0))
+        self.opaque_pointers[value] = True
+        self.make_equal_to(op.result, value)
+
+    def optimize_GETARRAYITEM_GC_PURE(self, op):
+        indexvalue = self.getvalue(op.getarg(1))
+        if indexvalue.is_constant():
+            arrayvalue = self.getvalue(op.getarg(0))
+            arrayvalue.make_len_gt(MODE_ARRAY, op.getdescr(), indexvalue.box.getint())
+        self.optimize_default(op)
+
+    def optimize_STRGETITEM(self, op):
+        indexvalue = self.getvalue(op.getarg(1))
+        if indexvalue.is_constant():
+            arrayvalue = self.getvalue(op.getarg(0))
+            arrayvalue.make_len_gt(MODE_STR, op.getdescr(), indexvalue.box.getint())
+        self.optimize_default(op)
+
+    def optimize_UNICODEGETITEM(self, op):
+        indexvalue = self.getvalue(op.getarg(1))
+        if indexvalue.is_constant():
+            arrayvalue = self.getvalue(op.getarg(0))
+            arrayvalue.make_len_gt(MODE_UNICODE, op.getdescr(), indexvalue.box.getint())
+        self.optimize_default(op)
+        
+
+    
+
 dispatch_opt = make_dispatcher_method(Optimizer, 'optimize_',
         default=Optimizer.optimize_default)
 
diff --git a/pypy/jit/metainterp/optimizeopt/rewrite.py b/pypy/jit/metainterp/optimizeopt/rewrite.py
--- a/pypy/jit/metainterp/optimizeopt/rewrite.py
+++ b/pypy/jit/metainterp/optimizeopt/rewrite.py
@@ -13,9 +13,16 @@
     """Rewrite operations into equivalent, cheaper operations.
        This includes already executed operations and constants.
     """
+    def __init__(self):
+        self.loop_invariant_results = {}
+        self.loop_invariant_producer = {}
 
-    def reconstruct_for_next_iteration(self, optimizer, valuemap):
-        return self
+    def new(self):
+        return OptRewrite()
+        
+    def produce_potential_short_preamble_ops(self, sb):
+        for op in self.loop_invariant_producer.values():
+            sb.add_potential(op)
 
     def propagate_forward(self, op):
         args = self.optimizer.make_args_key(op)
@@ -344,16 +351,18 @@
         # expects a compile-time constant
         assert isinstance(arg, Const)
         key = make_hashable_int(arg.getint())
-        resvalue = self.optimizer.loop_invariant_results.get(key, None)
+        
+        resvalue = self.loop_invariant_results.get(key, None)
         if resvalue is not None:
             self.make_equal_to(op.result, resvalue)
             return
         # change the op to be a normal call, from the backend's point of view
         # there is no reason to have a separate operation for this
+        self.loop_invariant_producer[key] = op
         op = op.copy_and_change(rop.CALL)
         self.emit_operation(op)
         resvalue = self.getvalue(op.result)
-        self.optimizer.loop_invariant_results[key] = resvalue
+        self.loop_invariant_results[key] = resvalue
 
     def _optimize_nullness(self, op, box, expect_nonnull):
         value = self.getvalue(box)
@@ -424,11 +433,10 @@
         # specifically the given oopspec call.  For non-oopspec calls,
         # oopspecindex is just zero.
         effectinfo = op.getdescr().get_extra_info()
-        if effectinfo is not None:
-            oopspecindex = effectinfo.oopspecindex
-            if oopspecindex == EffectInfo.OS_ARRAYCOPY:
-                if self._optimize_CALL_ARRAYCOPY(op):
-                    return
+        oopspecindex = effectinfo.oopspecindex
+        if oopspecindex == EffectInfo.OS_ARRAYCOPY:
+            if self._optimize_CALL_ARRAYCOPY(op):
+                return
         self.emit_operation(op)
 
     def _optimize_CALL_ARRAYCOPY(self, op):
diff --git a/pypy/jit/metainterp/optimizeopt/simplify.py b/pypy/jit/metainterp/optimizeopt/simplify.py
--- a/pypy/jit/metainterp/optimizeopt/simplify.py
+++ b/pypy/jit/metainterp/optimizeopt/simplify.py
@@ -25,6 +25,8 @@
         #     but it's a bit hard to implement robustly if heap.py is also run
         pass
 
+    optimize_CAST_OPAQUE_PTR = optimize_VIRTUAL_REF
+
 
 dispatch_opt = make_dispatcher_method(OptSimplify, 'optimize_',
         default=OptSimplify.emit_operation)
diff --git a/pypy/jit/metainterp/optimizeopt/test/test_optimizebasic.py b/pypy/jit/metainterp/optimizeopt/test/test_optimizebasic.py
--- a/pypy/jit/metainterp/optimizeopt/test/test_optimizebasic.py
+++ b/pypy/jit/metainterp/optimizeopt/test/test_optimizebasic.py
@@ -2742,11 +2742,11 @@
     def test_residual_call_invalidate_some_arrays(self):
         ops = """
         [p1, p2, i1]
-        p3 = getarrayitem_gc(p1, 0, descr=arraydescr2)
+        p3 = getarrayitem_gc(p2, 0, descr=arraydescr2)
         p4 = getarrayitem_gc(p2, 1, descr=arraydescr2)
         i2 = getarrayitem_gc(p1, 1, descr=arraydescr)
         i3 = call(i1, descr=writearraydescr)
-        p5 = getarrayitem_gc(p1, 0, descr=arraydescr2)
+        p5 = getarrayitem_gc(p2, 0, descr=arraydescr2)
         p6 = getarrayitem_gc(p2, 1, descr=arraydescr2)
         i4 = getarrayitem_gc(p1, 1, descr=arraydescr)
         escape(p3)
@@ -2759,7 +2759,7 @@
         """
         expected = """
         [p1, p2, i1]
-        p3 = getarrayitem_gc(p1, 0, descr=arraydescr2)
+        p3 = getarrayitem_gc(p2, 0, descr=arraydescr2)
         p4 = getarrayitem_gc(p2, 1, descr=arraydescr2)
         i2 = getarrayitem_gc(p1, 1, descr=arraydescr)
         i3 = call(i1, descr=writearraydescr)
@@ -4621,6 +4621,96 @@
         """
         self.optimize_strunicode_loop(ops, expected)
 
+    def test_intmod_bounds(self):
+        ops = """
+        [i0, i1]
+        i2 = int_mod(i0, 12)
+        i3 = int_gt(i2, 12)
+        guard_false(i3) []
+        i4 = int_lt(i2, -12)
+        guard_false(i4) []
+        i5 = int_mod(i1, -12)
+        i6 = int_lt(i5, -12)
+        guard_false(i6) []
+        i7 = int_gt(i5, 12)
+        guard_false(i7) []
+        jump(i2, i5)
+        """
+        expected = """
+        [i0, i1]
+        i2 = int_mod(i0, 12)
+        i5 = int_mod(i1, -12)
+        jump(i2, i5)
+        """
+        self.optimize_loop(ops, expected)
+
+        # This the sequence of resoperations that is generated for a Python
+        # app-level int % int.  When the modulus is constant and when i0
+        # is known non-negative it should be optimized to a single int_mod.
+        ops = """
+        [i0]
+        i5 = int_ge(i0, 0)
+        guard_true(i5) []
+        i1 = int_mod(i0, 42)
+        i2 = int_rshift(i1, 63)
+        i3 = int_and(42, i2)
+        i4 = int_add(i1, i3)
+        finish(i4)
+        """
+        expected = """
+        [i0]
+        i5 = int_ge(i0, 0)
+        guard_true(i5) []
+        i1 = int_mod(i0, 42)
+        finish(i1)
+        """
+        py.test.skip("in-progress")
+        self.optimize_loop(ops, expected)
+
+        # Also, 'n % power-of-two' can be turned into int_and(),
+        # but that's a bit harder to detect here because it turns into
+        # several operations, and of course it is wrong to just turn
+        # int_mod(i0, 16) into int_and(i0, 15).
+        ops = """
+        [i0]
+        i1 = int_mod(i0, 16)
+        i2 = int_rshift(i1, 63)
+        i3 = int_and(16, i2)
+        i4 = int_add(i1, i3)
+        finish(i4)
+        """
+        expected = """
+        [i0]
+        i4 = int_and(i0, 15)
+        finish(i4)
+        """
+        py.test.skip("harder")
+        self.optimize_loop(ops, expected)
+
+    def test_bounded_lazy_setfield(self):
+        ops = """
+        [p0, i0]
+        i1 = int_gt(i0, 2)
+        guard_true(i1) []
+        setarrayitem_gc(p0, 0, 3)
+        setarrayitem_gc(p0, 2, 4)
+        setarrayitem_gc(p0, i0, 15)
+        i2 = getarrayitem_gc(p0, 2)
+        jump(p0, i2)
+        """
+        # Remove the getarrayitem_gc, because we know that p[i0] does not alias
+        # p0[2]
+        expected = """
+        [p0, i0]
+        i1 = int_gt(i0, 2)
+        guard_true(i1) []
+        setarrayitem_gc(p0, i0, 15)
+        setarrayitem_gc(p0, 0, 3)
+        setarrayitem_gc(p0, 2, 4)
+        jump(p0, 4)
+        """
+        self.optimize_loop(ops, expected)
+
 
 class TestLLtype(BaseTestOptimizeBasic, LLtypeMixin):
     pass
diff --git a/pypy/jit/metainterp/optimizeopt/test/test_optimizefficall.py b/pypy/jit/metainterp/optimizeopt/test/test_optimizefficall.py
--- a/pypy/jit/metainterp/optimizeopt/test/test_optimizefficall.py
+++ b/pypy/jit/metainterp/optimizeopt/test/test_optimizefficall.py
@@ -14,12 +14,15 @@
     can check that the signature of a call is really what you want.
     """
 
-    def __init__(self, arg_types, typeinfo):
+    def __init__(self, arg_types, typeinfo, flags):
         self.arg_types = arg_types
         self.typeinfo = typeinfo   # return type
+        self.flags = flags
 
     def __eq__(self, other):
-        return self.arg_types == other.arg_types and self.typeinfo == other.typeinfo
+        return (self.arg_types == other.arg_types and
+                self.typeinfo == other.typeinfo and
+                self.flags == other.get_ffi_flags())
 
 class FakeLLObject(object):
 
@@ -41,24 +44,31 @@
         vable_token_descr = LLtypeMixin.valuedescr
         valuedescr = LLtypeMixin.valuedescr
 
-        int_float__int = MyCallDescr('if', 'i')
+        int_float__int_42 = MyCallDescr('if', 'i', 42)
+        int_float__int_43 = MyCallDescr('if', 'i', 43)
         funcptr = FakeLLObject()
         func = FakeLLObject(_fake_class=Func,
                             argtypes=[types.sint, types.double],
-                            restype=types.sint)
+                            restype=types.sint,
+                            flags=42)
         func2 = FakeLLObject(_fake_class=Func,
                              argtypes=[types.sint, types.double],
-                             restype=types.sint)
+                             restype=types.sint,
+                             flags=43)
         #
         def calldescr(cpu, FUNC, oopspecindex, extraeffect=None):
-            einfo = EffectInfo([], [], [], [], oopspecindex=oopspecindex,
+            if extraeffect == EffectInfo.EF_RANDOM_EFFECTS:
+                f = None   # means "can force all" really
+            else:
+                f = []
+            einfo = EffectInfo(f, f, f, f, oopspecindex=oopspecindex,
                                extraeffect=extraeffect)
             return cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT, einfo)
         #
         libffi_prepare =  calldescr(cpu, FUNC, EffectInfo.OS_LIBFFI_PREPARE)
         libffi_push_arg = calldescr(cpu, FUNC, EffectInfo.OS_LIBFFI_PUSH_ARG)
         libffi_call =     calldescr(cpu, FUNC, EffectInfo.OS_LIBFFI_CALL,
-                                 EffectInfo.EF_FORCES_VIRTUAL_OR_VIRTUALIZABLE)
+                                    EffectInfo.EF_RANDOM_EFFECTS)
     
     namespace = namespace.__dict__
 
@@ -79,7 +89,7 @@
         """
         expected = """
         [i0, f1]
-        i3 = call_release_gil(12345, i0, f1, descr=int_float__int)
+        i3 = call_release_gil(12345, i0, f1, descr=int_float__int_42)
         guard_not_forced() []
         guard_no_exception() []
         jump(i3, f1)
@@ -119,7 +129,7 @@
         [i0, f1, p2]
         i4 = force_token()
         setfield_gc(p2, i4, descr=vable_token_descr)
-        i3 = call_release_gil(12345, i0, f1, descr=int_float__int)
+        i3 = call_release_gil(12345, i0, f1, descr=int_float__int_42)
         guard_not_forced() [p2]
         guard_no_exception() [p2]
         jump(i3, f1, p2)
@@ -216,7 +226,7 @@
         call(0, ConstPtr(func),                        descr=libffi_prepare)
         #
         # this "nested" call is nicely optimized
-        i4 = call_release_gil(67890, i0, f1, descr=int_float__int)
+        i4 = call_release_gil(67890, i0, f1, descr=int_float__int_43)
         guard_not_forced() []
         guard_no_exception() []
         #
@@ -261,7 +271,7 @@
         expected = """
         [i0, f1, p2]
         setfield_gc(p2, i0, descr=valuedescr)
-        i3 = call_release_gil(12345, i0, f1, descr=int_float__int)
+        i3 = call_release_gil(12345, i0, f1, descr=int_float__int_42)
         guard_not_forced() []
         guard_no_exception() []
         jump(i3, f1, p2)
diff --git a/pypy/jit/metainterp/optimizeopt/test/test_optimizeopt.py b/pypy/jit/metainterp/optimizeopt/test/test_optimizeopt.py
--- a/pypy/jit/metainterp/optimizeopt/test/test_optimizeopt.py
+++ b/pypy/jit/metainterp/optimizeopt/test/test_optimizeopt.py
@@ -61,7 +61,9 @@
             boxes = []
         boxes = []
     def clone_if_mutable(self):
-        return self
+        return FakeDescr()
+    def __eq__(self, other):
+        return isinstance(other, Storage) or isinstance(other, FakeDescr)
 
 
 class BaseTestWithUnroll(BaseTest):
@@ -69,13 +71,14 @@
     enable_opts = "intbounds:rewrite:virtualize:string:heap:unroll"
 
     def optimize_loop(self, ops, expected, expected_preamble=None,
-                      call_pure_results=None):
+                      call_pure_results=None, expected_short=None):
         loop = self.parse(ops)
         if expected != "crash!":
             expected = self.parse(expected)
         if expected_preamble:
             expected_preamble = self.parse(expected_preamble)
-
+        if expected_short:
+            expected_short = self.parse(expected_short)
         loop.preamble = TreeLoop('preamble')
         loop.preamble.inputargs = loop.inputargs
         loop.preamble.token = LoopToken()
@@ -84,17 +87,33 @@
         self._do_optimize_loop(loop, call_pure_results)
         #
         print
+        print "Preamble:"
         print loop.preamble.inputargs
-        print '\n'.join([str(o) for o in loop.preamble.operations])
+        if loop.preamble.operations:
+            print '\n'.join([str(o) for o in loop.preamble.operations])
+        else:
+            print 'Failed!'
         print
+        print "Loop:"
         print loop.inputargs
         print '\n'.join([str(o) for o in loop.operations])
         print
+        if expected_short:
+            print "Short Preamble:"
+            short = loop.preamble.token.short_preamble[0]
+            print short.inputargs
+            print '\n'.join([str(o) for o in short.operations])        
+            print
+        
         assert expected != "crash!", "should have raised an exception"
         self.assert_equal(loop, expected)
         if expected_preamble:
             self.assert_equal(loop.preamble, expected_preamble,
                               text_right='expected preamble')
+        if expected_short:
+            self.assert_equal(short, expected_short,
+                              text_right='expected short preamble')
+            
         return loop
 
 class OptimizeOptTest(BaseTestWithUnroll):
@@ -840,7 +859,13 @@
         p3sub = new_with_vtable(ConstClass(node_vtable2))
         setfield_gc(p3sub, i1, descr=valuedescr)
         setfield_gc(p1, p3sub, descr=nextdescr)
-        jump(i1, p1, p3sub)
+        # XXX: We get two extra operations here because the setfield
+        #      above is the result of forcing p1 and thus not 
+        #      registered with the heap optimizer. I've makred tests
+        #      below with VIRTUALHEAP if they suffer from this issue
+        p3sub2 = getfield_gc(p1, descr=nextdescr) 
+        guard_nonnull_class(p3sub2, ConstClass(node_vtable2)) []
+        jump(i1, p1, p3sub2)
         """
         self.optimize_loop(ops, expected, preamble)
 
@@ -871,7 +896,9 @@
         guard_true(i2b) []
         p3 = new_with_vtable(ConstClass(node_vtable))
         setfield_gc(p3, i2, descr=nextdescr)
-        jump(p3, i2)
+        # XXX: VIRTUALHEAP (see above)
+        i3 = getfield_gc(p3, descr=nextdescr)
+        jump(p3, i3)
         """
         self.optimize_loop(ops, expected, preamble)
 
@@ -1166,6 +1193,29 @@
         """
         self.optimize_loop(ops, expected)
 
+    def test_virtual_field_forced_by_lazy_setfield(self):
+        ops = """
+        [i0, p1, p3]
+        i28 = int_add(i0, 1)
+        p30 = new_with_vtable(ConstClass(node_vtable))
+        setfield_gc(p30, i28, descr=nextdescr)
+        setfield_gc(p3, p30, descr=valuedescr)
+        p45 = getfield_gc(p3, descr=valuedescr)
+        i29 = int_add(i28, 1)
+        jump(i29, p45, p3)
+        """
+        preamble = """
+        [i0, p1, p3]
+        i28 = int_add(i0, 1)
+        i29 = int_add(i28, 1)
+        p30 = new_with_vtable(ConstClass(node_vtable))
+        setfield_gc(p30, i28, descr=nextdescr)
+        setfield_gc(p3, p30, descr=valuedescr)
+        jump(i29, p30, p3)
+        """
+        expected = preamble
+        self.optimize_loop(ops, expected, preamble)
+
     def test_nonvirtual_1(self):
         ops = """
         [i]
@@ -1308,15 +1358,78 @@
         ops = """
         [i]
         i1 = getfield_gc(ConstPtr(myptr), descr=valuedescr)
-        jump(i1)
-        """
-        preamble = ops
-        expected = """
+        call(i1, descr=nonwritedescr)
+        jump(i)
+        """
+        preamble = """
         [i]
-        jump(i)
+        i1 = getfield_gc(ConstPtr(myptr), descr=valuedescr)
+        call(i1, descr=nonwritedescr)
+        jump(i, i1)
+        """
+        expected = """
+        [i, i1]
+        call(i1, descr=nonwritedescr)
+        jump(i, i1)
         """
         self.optimize_loop(ops, expected, preamble)
 
+    def test_varray_boxed1(self):
+        ops = """
+        [p0, p8]
+        p11 = getfield_gc(p0, descr=otherdescr)
+        guard_nonnull(p11) [p0, p8]
+        guard_class(p11, ConstClass(node_vtable2)) [p0, p8]
+        p14 = getfield_gc(p11, descr=otherdescr)
+        guard_isnull(p14) [p0, p8]
+        p18 = getfield_gc(ConstPtr(myptr), descr=otherdescr)
+        guard_isnull(p18) [p0, p8]
+        p31 = new(descr=ssize)
+        setfield_gc(p31, 0, descr=adescr)
+        p33 = new_array(0, descr=arraydescr)
+        setfield_gc(p31, p33, descr=bdescr)
+        p35 = new_with_vtable(ConstClass(node_vtable))
+        setfield_gc(p35, p31, descr=valuedescr)
+        jump(p0, p35)
+        """
+        expected = """
+        [p0]
+        jump(p0)
+        """
+        self.optimize_loop(ops, expected)
+
+    def test_varray_boxed_simplified(self):
+        ops = """
+        [p0, p8]
+        p18 = getfield_gc(ConstPtr(myptr), descr=otherdescr)
+        guard_isnull(p18) [p0, p8]
+        p31 = new(descr=ssize)
+        p35 = new_with_vtable(ConstClass(node_vtable))
+        setfield_gc(p35, p31, descr=valuedescr)        
+        jump(p0, p35)
+        """
+        expected = """
+        [p0]
+        jump(p0)
+        """
+        self.optimize_loop(ops, expected)
+
+    def test_varray_boxed_noconst(self):
+        ops = """
+        [p0, p8, p18, p19]
+        guard_isnull(p18) [p0, p8]
+        p31 = new(descr=ssize)
+        p35 = new_with_vtable(ConstClass(node_vtable))
+        setfield_gc(p35, p31, descr=valuedescr)        
+        jump(p0, p35, p19, p18)
+        """
+        expected = """
+        [p0, p19]
+        guard_isnull(p19) [p0]
+        jump(p0, NULL)
+        """
+        self.optimize_loop(ops, expected)
+        
     def test_varray_1(self):
         ops = """
         [i1]
@@ -1552,6 +1665,24 @@
         """
         self.optimize_loop(ops, expected)
 
+    def test_duplicate_getfield_2(self):
+        ops = """
+        [p1, p2, i0]
+        i1 = getfield_gc(p1, descr=valuedescr)
+        i2 = getfield_gc(p2, descr=valuedescr)
+        i3 = getfield_gc(p1, descr=valuedescr)
+        i4 = getfield_gc(p2, descr=valuedescr)
+        i5 = int_add(i3, i4)
+        i6 = int_add(i0, i5)
+        jump(p1, p2, i6)
+        """
+        expected = """
+        [p1, p2, i0, i5]
+        i6 = int_add(i0, i5)
+        jump(p1, p2, i6, i5)
+        """
+        self.optimize_loop(ops, expected)
+
     def test_getfield_after_setfield(self):
         ops = """
         [p1, i1]
@@ -1728,6 +1859,7 @@
         """
         expected = """
         [p1, i1, i2]
+        setfield_gc(p1, i2, descr=valuedescr)
         jump(p1, i1, i2)
         """
         # in this case, all setfields are removed, because we can prove
@@ -1872,14 +2004,14 @@
         guard_true(i3) []
         i4 = int_neg(i2)
         setfield_gc(p1, i2, descr=valuedescr)
-        jump(p1, i1, i2, i4)
-        """
-        expected = """
-        [p1, i1, i2, i4]
+        jump(p1, i1, i2, i4, i4)
+        """
+        expected = """
+        [p1, i1, i2, i4, i5]
         setfield_gc(p1, i1, descr=valuedescr)
         guard_true(i4) []
         setfield_gc(p1, i2, descr=valuedescr)
-        jump(p1, i1, i2, 1)
+        jump(p1, i1, i2, i5, i5)
         """
         self.optimize_loop(ops, expected, preamble)
 
@@ -1902,14 +2034,14 @@
         i4 = int_neg(i2)
         setfield_gc(p1, NULL, descr=nextdescr)
         escape()
-        jump(p1, i2, i4)
-        """
-        expected = """
-        [p1, i2, i4]
+        jump(p1, i2, i4, i4)
+        """
+        expected = """
+        [p1, i2, i4, i5]
         guard_true(i4) [p1]
         setfield_gc(p1, NULL, descr=nextdescr)
         escape()
-        jump(p1, i2, 1)
+        jump(p1, i2, i5, i5)
         """
         self.optimize_loop(ops, expected, preamble)
 
@@ -1931,14 +2063,14 @@
         i4 = int_neg(i2)
         setfield_gc(p1, NULL, descr=nextdescr)
         escape()
-        jump(p1, i2, i4)
-        """
-        expected = """
-        [p1, i2, i4]
+        jump(p1, i2, i4, i4)
+        """
+        expected = """
+        [p1, i2, i4, i5]
         guard_true(i4) [i2, p1]
         setfield_gc(p1, NULL, descr=nextdescr)
         escape()
-        jump(p1, i2, 1)
+        jump(p1, i2, i5, i5)
         """
         self.optimize_loop(ops, expected)
 
@@ -1954,14 +2086,22 @@
         setfield_gc(p1, i2, descr=valuedescr)
         jump(p1, i1, i2, i4)
         """
-        preamble = ops
-        expected = """
-        [p1, i1, i2, i4]
+        preamble = """
+        [p1, i1, i2, i3]
+        setfield_gc(p1, i1, descr=valuedescr)
+        i5 = int_eq(i3, 5)
+        guard_true(i5) []
+        i4 = int_neg(i2)
+        setfield_gc(p1, i2, descr=valuedescr)
+        jump(p1, i1, i2, i4, i4)
+        """
+        expected = """
+        [p1, i1, i2, i4, i7]
         setfield_gc(p1, i1, descr=valuedescr)
         i5 = int_eq(i4, 5)
         guard_true(i5) []
         setfield_gc(p1, i2, descr=valuedescr)
-        jump(p1, i1, i2, 5)
+        jump(p1, i1, i2, i7, i7)
         """
         self.optimize_loop(ops, expected, preamble)
 
@@ -2035,7 +2175,25 @@
         jump(p1)
         """
         self.optimize_loop(ops, expected)
-
+        
+    def test_duplicate_getarrayitem_2(self):
+        ops = """
+        [p1, i0]
+        i2 = getarrayitem_gc(p1, 0, descr=arraydescr2)
+        i3 = getarrayitem_gc(p1, 1, descr=arraydescr2)
+        i4 = getarrayitem_gc(p1, 0, descr=arraydescr2)
+        i5 = getarrayitem_gc(p1, 1, descr=arraydescr2)
+        i6 = int_add(i3, i4)
+        i7 = int_add(i0, i6)
+        jump(p1, i7)
+        """
+        expected = """
+        [p1, i0, i6]
+        i7 = int_add(i0, i6)
+        jump(p1, i7, i6)
+        """
+        self.optimize_loop(ops, expected)
+        
     def test_duplicate_getarrayitem_after_setarrayitem_1(self):
         ops = """
         [p1, p2]
@@ -2161,15 +2319,15 @@
         p2 = new_with_vtable(ConstClass(node_vtable))
         setfield_gc(p2, p4, descr=nextdescr)
         setfield_gc(p1, p2, descr=nextdescr)
-        jump(p1, i2, i4, p4)
-        """
-        expected = """
-        [p1, i2, i4, p4]
+        jump(p1, i2, i4, p4, i4)
+        """
+        expected = """
+        [p1, i2, i4, p4, i5]
         guard_true(i4) [p1, p4]
         p2 = new_with_vtable(ConstClass(node_vtable))
         setfield_gc(p2, p4, descr=nextdescr)
         setfield_gc(p1, p2, descr=nextdescr)
-        jump(p1, i2, 1, p4)
+        jump(p1, i2, i5, p4, i5)
         """
         self.optimize_loop(ops, expected, preamble)
 
@@ -2625,6 +2783,101 @@
         """
         self.optimize_loop(ops, expected, preamble)
 
+    def test_remove_duplicate_pure_op_ovf_with_lazy_setfield(self):
+        py.test.skip('this optimization is not yet supprted')
+        ops = """
+        [i1, p1]
+        i3 = int_add_ovf(i1, 1)
+        guard_no_overflow() []
+        i3b = int_is_true(i3)
+        guard_true(i3b) []
+        setfield_gc(p1, i1, descr=valuedescr)
+        i4 = int_add_ovf(i1, 1)
+        guard_no_overflow() []
+        i4b = int_is_true(i4)
+        guard_true(i4b) []
+        escape(i3)
+        escape(i4)
+        jump(i1, p1)
+        """
+        preamble = """
+        [i1, p1]
+        i3 = int_add_ovf(i1, 1)
+        guard_no_overflow() []
+        i3b = int_is_true(i3)
+        guard_true(i3b) []
+        setfield_gc(p1, i1, descr=valuedescr)        
+        escape(i3)
+        escape(i3)
+        jump(i1, p1, i3)
+        """
+        expected = """
+        [i1, p1, i3]
+        setfield_gc(p1, i1, descr=valuedescr)        
+        escape(i3)
+        escape(i3)
+        jump(i1, p1, i3)
+        """
+        self.optimize_loop(ops, expected, preamble)
+
+    def test_ovf_guard_in_short_preamble1(self):
+        ops = """
+        [p8, p11, i24]
+        p26 = new_with_vtable(ConstClass(node_vtable))
+        setfield_gc(p26, i24, descr=adescr)        
+        i34 = getfield_gc_pure(p11, descr=valuedescr)
+        i35 = getfield_gc_pure(p26, descr=adescr)
+        i36 = int_add_ovf(i34, i35)
+        guard_no_overflow() []
+        jump(p8, p11, i35)
+        """
+        expected = """
+        [p8, p11, i26]
+        jump(p8, p11, i26)        
+        """
+        self.optimize_loop(ops, expected)
+        
+    def test_ovf_guard_in_short_preamble2(self):
+        ops = """
+        [p8, p11, p12]
+        p16 = getfield_gc(p8, descr=valuedescr)
+        i17 = getfield_gc(p8, descr=nextdescr)
+        i19 = getfield_gc(p16, descr=valuedescr)
+        i20 = int_ge(i17, i19)
+        guard_false(i20) []
+        i21 = getfield_gc(p16, descr=otherdescr)
+        i22 = getfield_gc(p16, descr=nextdescr)
+        i23 = int_mul(i17, i22)
+        i24 = int_add(i21, i23)
+        p26 = new_with_vtable(ConstClass(node_vtable))
+        setfield_gc(p26, i24, descr=adescr)
+        i28 = int_add(i17, 1)
+        setfield_gc(p8, i28, descr=nextdescr)
+        i34 = getfield_gc_pure(p11, descr=valuedescr)
+        i35 = getfield_gc_pure(p26, descr=adescr)
+        guard_nonnull(p12) []
+        i36 = int_add_ovf(i34, i35)
+        guard_no_overflow() []
+        p38 = new_with_vtable(ConstClass(node_vtable))
+        setfield_gc(p38, i36, descr=adescr)
+        jump(p8, p11, p26)
+        """
+        expected = """
+        [p8, p11, i24, i39, i19, p16, i21, i34]
+        i40 = int_ge(i39, i19)
+        guard_false(i40) []
+        i41 = getfield_gc(p16, descr=nextdescr)
+        i42 = int_mul(i39, i41)
+        i43 = int_add(i21, i42)
+        i44 = int_add(i39, 1)
+        setfield_gc(p8, i44, descr=nextdescr)
+        i45 = int_add_ovf(i34, i43)
+        guard_no_overflow() []
+        jump(p8, p11, i43, i44, i19, p16, i21, i34)
+        """
+        self.optimize_loop(ops, expected)
+
+
     def test_int_and_or_with_zero(self):
         ops = """
         [i0, i1]
@@ -3899,11 +4152,13 @@
         jump(p4364)
         """
         expected = """
-        [i0, i1]
+        [i0]
+        i1 = int_sub_ovf(i0, 1)
+        guard_no_overflow() []
         escape(i1)
         i2 = int_add_ovf(i0, 1)
         guard_no_overflow() []
-        jump(i2, i0)
+        jump(i2)
         """
         self.optimize_loop(ops, expected)
 
@@ -4867,32 +5122,38 @@
 
     def test_invariant_ovf(self):
         ops = """
-        [i0, i1, i10, i11, i12]
+        [i0, i1, i10, i11, i20, i21]
         i2 = int_add_ovf(i0, i1)
         guard_no_overflow() []
         i3 = int_sub_ovf(i0, i1)
         guard_no_overflow() []
         i4 = int_mul_ovf(i0, i1)
         guard_no_overflow() []
+        escape(i2)
+        escape(i3)
+        escape(i4)
         i24 = int_mul_ovf(i10, i11)
         guard_no_overflow() []
         i23 = int_sub_ovf(i10, i11)
         guard_no_overflow() []
         i22 = int_add_ovf(i10, i11)
         guard_no_overflow() []
-        jump(i0, i1, i2, i3, i4)
-        """
-        expected = """
-        [i0, i1, i10, i11, i12]
+        jump(i0, i1, i20, i21, i20, i21)
+        """
+        expected = """
+        [i0, i1, i10, i11, i2, i3, i4]
+        escape(i2)
+        escape(i3)
+        escape(i4)        
         i24 = int_mul_ovf(i10, i11)
         guard_no_overflow() []
         i23 = int_sub_ovf(i10, i11)
         guard_no_overflow() []
         i22 = int_add_ovf(i10, i11)
         guard_no_overflow() []
-        jump(i0, i1, i10, i11, i12)
-        """
-        self.optimize_loop(ops, expected, ops)
+        jump(i0, i1, i10, i11, i2, i3, i4) 
+        """
+        self.optimize_loop(ops, expected)
 
     def test_value_proven_to_be_constant_after_two_iterations(self):
         class FakeDescr(AbstractDescr):
@@ -4908,8 +5169,8 @@
         ops = """
         [p0, p1, p2, p3, i4, p5, i6, p7, p8, p9, p14]
         guard_value(i4, 3) []
-        guard_class(p9, 17278984) []
-        guard_class(p9, 17278984) []
+        guard_class(p9, ConstClass(node_vtable)) []
+        guard_class(p9, ConstClass(node_vtable)) []
         p22 = getfield_gc(p9, descr=inst_w_seq)
         guard_nonnull(p22) []
         i23 = getfield_gc(p9, descr=inst_index)
@@ -4924,11 +5185,11 @@
         guard_class(p14, 17273920) []
         guard_class(p14, 17273920) []
 
-        p75 = new_with_vtable(17278984)
+        p75 = new_with_vtable(ConstClass(node_vtable))
         setfield_gc(p75, p14, descr=inst_w_seq)
         setfield_gc(p75, 0, descr=inst_index)
-        guard_class(p75, 17278984) []
-        guard_class(p75, 17278984) []
+        guard_class(p75, ConstClass(node_vtable)) []
+        guard_class(p75, ConstClass(node_vtable)) []
         p79 = getfield_gc(p75, descr=inst_w_seq)
         guard_nonnull(p79) []
         i80 = getfield_gc(p75, descr=inst_index)
@@ -4974,6 +5235,7 @@
         """
         expected = """
         [p0]
+        setfield_gc(p0, p0, descr=valuedescr)
         jump(p0)
         """
         self.optimize_loop(ops, expected, preamble)
@@ -5060,9 +5322,7 @@
         self.optimize_loop(ops, expected)
 
     # ----------
-    def optimize_strunicode_loop(self, ops, optops, preamble=None):
-        if not preamble:
-            preamble = ops # FIXME: Force proper testing of preamble
+    def optimize_strunicode_loop(self, ops, optops, preamble):
         # check with the arguments passed in
         self.optimize_loop(ops, optops, preamble)
         # check with replacing 'str' with 'unicode' everywhere
@@ -5082,7 +5342,7 @@
         [i0]
         jump(i0)
         """
-        self.optimize_strunicode_loop(ops, expected)
+        self.optimize_strunicode_loop(ops, expected, expected)
 
     def test_newstr_2(self):
         ops = """
@@ -5098,7 +5358,7 @@
         [i0, i1]
         jump(i1, i0)
         """
-        self.optimize_strunicode_loop(ops, expected)
+        self.optimize_strunicode_loop(ops, expected, expected)
 
     def test_str_concat_1(self):
         ops = """
@@ -5106,7 +5366,7 @@
         p3 = call(0, p1, p2, descr=strconcatdescr)
         jump(p2, p3)
         """
-        expected = """
+        preamble = """
         [p1, p2]
         i1 = strlen(p1)
         i2 = strlen(p2)
@@ -5114,9 +5374,18 @@
         p3 = newstr(i3)
         copystrcontent(p1, p3, 0, 0, i1)
         copystrcontent(p2, p3, 0, i1, i2)
-        jump(p2, p3)
-        """
-        self.optimize_strunicode_loop(ops, expected)
+        jump(p2, p3, i2)
+        """
+        expected = """
+        [p1, p2, i1]
+        i2 = strlen(p2)
+        i3 = int_add(i1, i2)
+        p3 = newstr(i3)
+        copystrcontent(p1, p3, 0, 0, i1)
+        copystrcontent(p2, p3, 0, i1, i2)
+        jump(p2, p3, i2)
+        """
+        self.optimize_strunicode_loop(ops, expected, preamble)
 
     def test_str_concat_vstr2_str(self):
         ops = """
@@ -5137,7 +5406,7 @@
         copystrcontent(p2, p3, 0, 2, i2)
         jump(i1, i0, p3)
         """
-        self.optimize_strunicode_loop(ops, expected)
+        self.optimize_strunicode_loop(ops, expected, expected)
 
     def test_str_concat_str_vstr2(self):
         ops = """
@@ -5160,7 +5429,7 @@
         i6 = int_add(i5, 1)      # will be killed by the backend
         jump(i1, i0, p3)
         """
-        self.optimize_strunicode_loop(ops, expected)
+        self.optimize_strunicode_loop(ops, expected, expected)
 
     def test_str_concat_str_str_str(self):
         ops = """
@@ -5169,7 +5438,7 @@
         p5 = call(0, p4, p3, descr=strconcatdescr)
         jump(p2, p3, p5)
         """
-        expected = """
+        preamble = """
         [p1, p2, p3]
         i1 = strlen(p1)
         i2 = strlen(p2)
@@ -5180,9 +5449,20 @@
         copystrcontent(p1, p5, 0, 0, i1)
         copystrcontent(p2, p5, 0, i1, i2)
         copystrcontent(p3, p5, 0, i12, i3)
-        jump(p2, p3, p5)
-        """
-        self.optimize_strunicode_loop(ops, expected)
+        jump(p2, p3, p5, i2, i3)
+        """
+        expected = """
+        [p1, p2, p3, i1, i2]
+        i12 = int_add(i1, i2)
+        i3 = strlen(p3)
+        i123 = int_add(i12, i3)
+        p5 = newstr(i123)
+        copystrcontent(p1, p5, 0, 0, i1)
+        copystrcontent(p2, p5, 0, i1, i2)
+        copystrcontent(p3, p5, 0, i12, i3)
+        jump(p2, p3, p5, i2, i3)
+        """
+        self.optimize_strunicode_loop(ops, expected, preamble)
 
     def test_str_concat_str_cstr1(self):
         ops = """
@@ -5199,7 +5479,7 @@
         strsetitem(p3, i2, 120)     # == ord('x')
         jump(p3)
         """
-        self.optimize_strunicode_loop(ops, expected)
+        self.optimize_strunicode_loop(ops, expected, expected)
 
     def test_str_concat_consts(self):
         ops = """
@@ -5210,17 +5490,58 @@
         escape(p3)
         jump()
         """
-        preamble = """
-        []
-        p3 = call(0, s"ab", s"cde", descr=strconcatdescr)
-        escape(p3)
-        jump()
-        """
         expected = """
         []
         escape(s"abcde")
         jump()
         """
+        self.optimize_strunicode_loop(ops, expected, expected)
+
+    def test_str_slice_len_surviving1(self):
+        ops = """
+        [p1, i1, i2, i3]
+        escape(i3)
+        p2 = call(0, p1, i1, i2, descr=strslicedescr)
+        i4 = strlen(p2)
+        jump(p1, i1, i2, i4)
+        """
+        preamble = """
+        [p1, i1, i2, i3]
+        escape(i3)
+        i4 = int_sub(i2, i1)
+        jump(p1, i1, i2, i4, i4)
+        """
+        expected = """
+        [p1, i1, i2, i3, i4]
+        escape(i3)
+        jump(p1, i1, i2, i4, i4)
+        """
+        self.optimize_strunicode_loop(ops, expected, preamble)
+
+    def test_str_slice_len_surviving2(self):
+        ops = """
+        [p1, i1, i2, p2]
+        i5 = getfield_gc(p2, descr=valuedescr)
+        escape(i5)
+        p3 = call(0, p1, i1, i2, descr=strslicedescr)
+        i4 = strlen(p3)
+        setfield_gc(p2, i4, descr=valuedescr)
+        jump(p1, i1, i2, p2)
+        """
+        preamble = """
+        [p1, i1, i2, p2]
+        i5 = getfield_gc(p2, descr=valuedescr)
+        escape(i5)
+        i4 = int_sub(i2, i1)
+        setfield_gc(p2, i4, descr=valuedescr)
+        jump(p1, i1, i2, p2, i4, i4)
+        """
+        expected = """
+        [p1, i1, i2, p2, i5, i6]
+        escape(i5)
+        setfield_gc(p2, i6, descr=valuedescr)
+        jump(p1, i1, i2, p2, i6, i6)
+        """
         self.optimize_strunicode_loop(ops, expected, preamble)
 
     def test_str_slice_1(self):
@@ -5229,14 +5550,20 @@
         p2 = call(0, p1, i1, i2, descr=strslicedescr)
         jump(p2, i1, i2)
         """
-        expected = """
+        preamble = """
         [p1, i1, i2]
         i3 = int_sub(i2, i1)
         p2 = newstr(i3)
         copystrcontent(p1, p2, i1, 0, i3)
-        jump(p2, i1, i2)
-        """
-        self.optimize_strunicode_loop(ops, expected)
+        jump(p2, i1, i2, i3)
+        """
+        expected = """
+        [p1, i1, i2, i3]
+        p2 = newstr(i3)
+        copystrcontent(p1, p2, i1, 0, i3)
+        jump(p2, i1, i2, i3)
+        """
+        self.optimize_strunicode_loop(ops, expected, preamble)
 
     def test_str_slice_2(self):
         ops = """
@@ -5250,7 +5577,7 @@
         copystrcontent(p1, p2, 0, 0, i2)
         jump(p2, i2)
         """
-        self.optimize_strunicode_loop(ops, expected)
+        self.optimize_strunicode_loop(ops, expected, expected)
 
     def test_str_slice_3(self):
         ops = """
@@ -5259,16 +5586,22 @@
         p3 = call(0, p2, i3, i4, descr=strslicedescr)
         jump(p3, i1, i2, i3, i4)
         """
-        expected = """
+        preamble = """
         [p1, i1, i2, i3, i4]
         i0 = int_sub(i2, i1)     # killed by the backend
         i5 = int_sub(i4, i3)
         i6 = int_add(i1, i3)
         p3 = newstr(i5)
         copystrcontent(p1, p3, i6, 0, i5)
-        jump(p3, i1, i2, i3, i4)
-        """
-        self.optimize_strunicode_loop(ops, expected)
+        jump(p3, i1, i2, i3, i4, i5, i6)
+        """
+        expected = """
+        [p1, i1, i2, i3, i4, i5, i6]
+        p3 = newstr(i5)
+        copystrcontent(p1, p3, i6, 0, i5)
+        jump(p3, i1, i2, i3, i4, i5, i6)
+        """
+        self.optimize_strunicode_loop(ops, expected, preamble)
 
     def test_str_slice_getitem1(self):
         ops = """
@@ -5278,15 +5611,21 @@
         escape(i4)
         jump(p1, i1, i2, i3)
         """
-        expected = """
+        preamble = """
         [p1, i1, i2, i3]
         i6 = int_sub(i2, i1)      # killed by the backend
         i5 = int_add(i1, i3)
         i4 = strgetitem(p1, i5)
         escape(i4)
-        jump(p1, i1, i2, i3)
-        """
-        self.optimize_strunicode_loop(ops, expected)
+        jump(p1, i1, i2, i3, i5)
+        """
+        expected = """
+        [p1, i1, i2, i3, i5]
+        i4 = strgetitem(p1, i5)
+        escape(i4)
+        jump(p1, i1, i2, i3, i5)
+        """
+        self.optimize_strunicode_loop(ops, expected, preamble)
 
     def test_str_slice_plain(self):
         ops = """
@@ -5304,7 +5643,7 @@
         escape(i4)
         jump(i3, i4)
         """
-        self.optimize_strunicode_loop(ops, expected)
+        self.optimize_strunicode_loop(ops, expected, expected)
 
     def test_str_slice_concat(self):
         ops = """
@@ -5313,7 +5652,7 @@
         p4 = call(0, p3, p2, descr=strconcatdescr)
         jump(p4, i1, i2, p2)
         """
-        expected = """
+        preamble = """
         [p1, i1, i2, p2]
         i3 = int_sub(i2, i1)     # length of p3
         i4 = strlen(p2)
@@ -5321,14 +5660,22 @@
         p4 = newstr(i5)
         copystrcontent(p1, p4, i1, 0, i3)
         copystrcontent(p2, p4, 0, i3, i4)
-        jump(p4, i1, i2, p2)
-        """
-        self.optimize_strunicode_loop(ops, expected)
+        jump(p4, i1, i2, p2, i5, i3, i4)
+        """
+        expected = """
+        [p1, i1, i2, p2, i5, i3, i4]
+        p4 = newstr(i5)
+        copystrcontent(p1, p4, i1, 0, i3)
+        copystrcontent(p2, p4, 0, i3, i4)
+        jump(p4, i1, i2, p2, i5, i3, i4)
+        """
+        self.optimize_strunicode_loop(ops, expected, preamble)
 
     def test_strgetitem_bounds(self):
         ops = """
         [p0, i0]
         i1 = strgetitem(p0, i0)
+        i10 = strgetitem(p0, i0)
         i2 = int_lt(i1, 256)
         guard_true(i2) []
         i3 = int_ge(i1, 0)
@@ -5337,6 +5684,7 @@
         """
         expected = """
         [p0, i0]
+        i1 = strgetitem(p0, i0)
         jump(p0, i0)
         """
         self.optimize_loop(ops, expected)
@@ -5345,12 +5693,14 @@
         ops = """
         [p0, i0]
         i1 = unicodegetitem(p0, i0)
+        i10 = unicodegetitem(p0, i0)        
         i2 = int_lt(i1, 0)
         guard_false(i2) []
         jump(p0, i0)
         """
         expected = """
         [p0, i0]
+        i1 = unicodegetitem(p0, i0)        
         jump(p0, i0)
         """
         self.optimize_loop(ops, expected)
@@ -5387,7 +5737,7 @@
         self.optimize_loop(ops, expected)
 
     # ----------
-    def optimize_strunicode_loop_extradescrs(self, ops, optops, preamble=None):
+    def optimize_strunicode_loop_extradescrs(self, ops, optops, preamble):
         class FakeCallInfoCollection:
             def callinfo_for_oopspec(self, oopspecindex):
                 calldescrtype = type(LLtypeMixin.strequaldescr)
@@ -5410,7 +5760,7 @@
         escape(i0)
         jump(p1, p2)
         """
-        self.optimize_strunicode_loop_extradescrs(ops, ops)
+        self.optimize_strunicode_loop_extradescrs(ops, ops, ops)
 
     def test_str_equal_noop2(self):
         ops = """
@@ -5420,7 +5770,7 @@
         escape(i0)
         jump(p1, p2, p3)
         """
-        expected = """
+        preamble = """
         [p1, p2, p3]
         i1 = strlen(p1)
         i2 = strlen(p2)
@@ -5430,10 +5780,19 @@
         copystrcontent(p2, p4, 0, i1, i2)
         i0 = call(0, p3, p4, descr=strequaldescr)
         escape(i0)
-        jump(p1, p2, p3)
-        """
-        self.optimize_strunicode_loop_extradescrs(ops,
-                                                  expected)
+        jump(p1, p2, p3, i3, i1, i2)
+        """
+        expected = """
+        [p1, p2, p3, i3, i1, i2]
+        p4 = newstr(i3)
+        copystrcontent(p1, p4, 0, 0, i1)
+        copystrcontent(p2, p4, 0, i1, i2)
+        i0 = call(0, p3, p4, descr=strequaldescr)
+        escape(i0)
+        jump(p1, p2, p3, i3, i1, i2)
+        """
+        self.optimize_strunicode_loop_extradescrs(ops, expected,
+                                                  preamble)
 
     def test_str_equal_slice1(self):
         ops = """
@@ -5443,15 +5802,21 @@
         escape(i0)
         jump(p1, i1, i2, p3)
         """
-        expected = """
+        preamble = """
         [p1, i1, i2, p3]
         i3 = int_sub(i2, i1)
         i0 = call(0, p1, i1, i3, p3, descr=streq_slice_checknull_descr)
         escape(i0)
-        jump(p1, i1, i2, p3)
-        """
-        self.optimize_strunicode_loop_extradescrs(ops,
-                                                  expected)
+        jump(p1, i1, i2, p3, i3)
+        """
+        expected = """
+        [p1, i1, i2, p3, i3]
+        i0 = call(0, p1, i1, i3, p3, descr=streq_slice_checknull_descr)
+        escape(i0)
+        jump(p1, i1, i2, p3, i3)
+        """
+        self.optimize_strunicode_loop_extradescrs(ops, expected,
+                                                  preamble)
 
     def test_str_equal_slice2(self):
         ops = """
@@ -5461,15 +5826,21 @@
         escape(i0)
         jump(p1, i1, i2, p3)
         """
-        expected = """
+        preamble = """
         [p1, i1, i2, p3]
         i4 = int_sub(i2, i1)
         i0 = call(0, p1, i1, i4, p3, descr=streq_slice_checknull_descr)
         escape(i0)
-        jump(p1, i1, i2, p3)
-        """
-        self.optimize_strunicode_loop_extradescrs(ops,
-                                                  expected)
+        jump(p1, i1, i2, p3, i4)
+        """
+        expected = """
+        [p1, i1, i2, p3, i4]
+        i0 = call(0, p1, i1, i4, p3, descr=streq_slice_checknull_descr)
+        escape(i0)
+        jump(p1, i1, i2, p3, i4)
+        """
+        self.optimize_strunicode_loop_extradescrs(ops, expected,
+                                                  preamble)
 
     def test_str_equal_slice3(self):
         ops = """
@@ -5481,14 +5852,21 @@
         jump(p1, i1, i2, p3)
         """
         expected = """
+        [p1, i1, i2, p3, i4]
+        i0 = call(0, p1, i1, i4, p3, descr=streq_slice_nonnull_descr)
+        escape(i0)
+        jump(p1, i1, i2, p3, i4)
+        """
+        preamble = """
         [p1, i1, i2, p3]
+        guard_nonnull(p3) []        
         i4 = int_sub(i2, i1)
         i0 = call(0, p1, i1, i4, p3, descr=streq_slice_nonnull_descr)
         escape(i0)
-        jump(p1, i1, i2, p3)
+        jump(p1, i1, i2, p3, i4)
         """
         self.optimize_strunicode_loop_extradescrs(ops,
-                                                  expected, ops)
+                                                  expected, preamble)
 
     def test_str_equal_slice4(self):
         ops = """
@@ -5498,15 +5876,21 @@
         escape(i0)
         jump(p1, i1, i2)
         """
-        expected = """
+        preamble = """
         [p1, i1, i2]
         i3 = int_sub(i2, i1)
         i0 = call(0, p1, i1, i3, 120, descr=streq_slice_char_descr)
         escape(i0)
-        jump(p1, i1, i2)
-        """
-        self.optimize_strunicode_loop_extradescrs(ops,
-                                                  expected)
+        jump(p1, i1, i2, i3)
+        """
+        expected = """
+        [p1, i1, i2, i3]
+        i0 = call(0, p1, i1, i3, 120, descr=streq_slice_char_descr)
+        escape(i0)
+        jump(p1, i1, i2, i3)
+        """
+        self.optimize_strunicode_loop_extradescrs(ops, expected,
+                                                  preamble)
 
     def test_str_equal_slice5(self):
         ops = """
@@ -5518,15 +5902,21 @@
         escape(i0)
         jump(p1, i1, i2, i3)
         """
-        expected = """
+        preamble = """
         [p1, i1, i2, i3]
         i4 = int_sub(i2, i1)
         i0 = call(0, p1, i1, i4, i3, descr=streq_slice_char_descr)
         escape(i0)
-        jump(p1, i1, i2, i3)
-        """
-        self.optimize_strunicode_loop_extradescrs(ops,
-                                                  expected)
+        jump(p1, i1, i2, i3, i4)
+        """
+        expected = """
+        [p1, i1, i2, i3, i4]
+        i0 = call(0, p1, i1, i4, i3, descr=streq_slice_char_descr)
+        escape(i0)
+        jump(p1, i1, i2, i3, i4)
+        """
+        self.optimize_strunicode_loop_extradescrs(ops, expected,
+                                                  preamble)
 
     def test_str_equal_none1(self):
         ops = """
@@ -5541,7 +5931,7 @@
         escape(i0)
         jump(p1)
         """
-        self.optimize_strunicode_loop_extradescrs(ops, expected)
+        self.optimize_strunicode_loop_extradescrs(ops, expected, expected)
 
     def test_str_equal_none2(self):
         ops = """
@@ -5556,7 +5946,7 @@
         escape(i0)
         jump(p1)
         """
-        self.optimize_strunicode_loop_extradescrs(ops, expected)
+        self.optimize_strunicode_loop_extradescrs(ops, expected, expected)
 
     def test_str_equal_nonnull1(self):
         ops = """
@@ -5572,7 +5962,14 @@
         escape(i0)
         jump(p1)
         """
-        self.optimize_strunicode_loop_extradescrs(ops, expected)
+        preamble = """
+        [p1]
+        guard_nonnull(p1) []
+        i0 = call(0, p1, s"hello world", descr=streq_nonnull_descr)
+        escape(i0)
+        jump(p1)
+        """
+        self.optimize_strunicode_loop_extradescrs(ops, expected, preamble)
 
     def test_str_equal_nonnull2(self):
         ops = """
@@ -5583,13 +5980,19 @@
         jump(p1)
         """
         expected = """
+        [p1, i0]
+        escape(i0)
+        jump(p1, i0)
+        """
+        preamble = """
         [p1]
+        guard_nonnull(p1) []
         i1 = strlen(p1)
         i0 = int_eq(i1, 0)
         escape(i0)
-        jump(p1)
-        """
-        self.optimize_strunicode_loop_extradescrs(ops, expected)
+        jump(p1, i0)
+        """
+        self.optimize_strunicode_loop_extradescrs(ops, expected, preamble)
 
     def test_str_equal_nonnull3(self):
         ops = """
@@ -5605,7 +6008,14 @@
         escape(i0)
         jump(p1)
         """
-        self.optimize_strunicode_loop_extradescrs(ops, expected)
+        preamble = """
+        [p1]
+        guard_nonnull(p1) []
+        i0 = call(0, p1, 120, descr=streq_nonnull_char_descr)
+        escape(i0)
+        jump(p1)
+        """
+        self.optimize_strunicode_loop_extradescrs(ops, expected, preamble)
 
     def test_str_equal_nonnull4(self):
         ops = """
@@ -5615,7 +6025,7 @@
         escape(i0)
         jump(p1, p2)
         """
-        expected = """
+        preamble = """
         [p1, p2]
         i1 = strlen(p1)
         i2 = strlen(p2)
@@ -5625,9 +6035,18 @@
         copystrcontent(p2, p4, 0, i1, i2)
         i0 = call(0, s"hello world", p4, descr=streq_nonnull_descr)
         escape(i0)
-        jump(p1, p2)
-        """
-        self.optimize_strunicode_loop_extradescrs(ops, expected)
+        jump(p1, p2, i3, i1, i2)
+        """
+        expected = """
+        [p1, p2, i3, i1, i2]
+        p4 = newstr(i3)
+        copystrcontent(p1, p4, 0, 0, i1)
+        copystrcontent(p2, p4, 0, i1, i2)
+        i0 = call(0, s"hello world", p4, descr=streq_nonnull_descr)
+        escape(i0)
+        jump(p1, p2, i3, i1, i2)
+        """
+        self.optimize_strunicode_loop_extradescrs(ops, expected, preamble)
 
     def test_str_equal_chars0(self):
         ops = """
@@ -5642,7 +6061,7 @@
         escape(1)
         jump(i1)
         """
-        self.optimize_strunicode_loop_extradescrs(ops, expected)
+        self.optimize_strunicode_loop_extradescrs(ops, expected, expected)
 
     def test_str_equal_chars1(self):
         ops = """
@@ -5653,13 +6072,42 @@
         escape(i0)
         jump(i1)
         """
-        expected = """
+        preamble = """
         [i1]
         i0 = int_eq(i1, 120)     # ord('x')
         escape(i0)
-        jump(i1)
-        """
-        self.optimize_strunicode_loop_extradescrs(ops, expected)
+        jump(i1, i0)
+        """
+        expected = """
+        [i1, i0]
+        escape(i0)
+        jump(i1, i0)
+        """
+        self.optimize_strunicode_loop_extradescrs(ops, expected, preamble)
+
+    def test_str_equal_nonconst(self):
+        ops = """
+        [i1, i2]
+        p1 = newstr(1)
+        strsetitem(p1, 0, i1)
+        p2 = newstr(1)
+        strsetitem(p2, 0, i2)
+        i0 = call(0, p1, p2, descr=strequaldescr)
+        escape(i0)
+        jump(i1, i2)
+        """
+        preamble = """
+        [i1, i2]
+        i0 = int_eq(i1, i2)
+        escape(i0)
+        jump(i1, i2, i0)
+        """
+        expected = """
+        [i1, i2, i0]
+        escape(i0)
+        jump(i1, i2, i0)
+        """
+        self.optimize_strunicode_loop_extradescrs(ops, expected, preamble)
 
     def test_str_equal_chars2(self):
         ops = """
@@ -5680,7 +6128,7 @@
         escape(i0)
         jump(i1, i2)
         """

From noreply at buildbot.pypy.org  Tue Sep  6 19:33:23 2011
From: noreply at buildbot.pypy.org (edelsohn)
Date: Tue,  6 Sep 2011 19:33:23 +0200 (CEST)
Subject: [pypy-commit] pypy ppc-jit-backend: PPC64 updates
Message-ID: <20110906173323.A0F088203C@wyvern.cs.uni-duesseldorf.de>

Author: edelsohn
Branch: ppc-jit-backend
Changeset: r47116:c0df6acc9e45
Date: 2011-09-06 13:26 -0400
http://bitbucket.org/pypy/pypy/changeset/c0df6acc9e45/

Log:	PPC64 updates

diff --git a/pypy/jit/backend/ppc/runner.py b/pypy/jit/backend/ppc/runner.py
--- a/pypy/jit/backend/ppc/runner.py
+++ b/pypy/jit/backend/ppc/runner.py
@@ -12,7 +12,7 @@
 from pypy.jit.backend.x86 import regloc
 from pypy.jit.backend.x86.support import values_array
 from pypy.jit.backend.ppc.ppcgen.ppc_assembler import PPCBuilder
-from pypy.jit.backend.ppc.ppcgen.arch import NONVOLATILES
+from pypy.jit.backend.ppc.ppcgen.arch import IS_PPC_32, NONVOLATILES
 import sys
 
 from pypy.tool.ansi_print import ansi_log
@@ -111,10 +111,15 @@
         return reg
 
     def _make_prologue(self, codebuilder):
-        framesize = 64 + 80
-        codebuilder.stwu(1, 1, -framesize)
-        codebuilder.mflr(0)
-        codebuilder.stw(0, 1, framesize + 4)
+        framesize = 16 * WORD + 20 * WORD
+        if IS_PPC_32:
+            codebuilder.stwu(1, 1, -framesize)
+            codebuilder.mflr(0)
+            codebuilder.stw(0, 1, framesize + WORD)
+        else:
+            codebuilder.stdu(1, 1, -framesize)
+            codebuilder.mflr(0)
+            codebuilder.std(0, 1, framesize + WORD)
         codebuilder.save_nonvolatiles(framesize)
 
     def _make_epilogue(self, codebuilder):
@@ -142,10 +147,13 @@
             descr.patch_pos = patch_pos
             descr.used_mem_indices = used_mem_indices
 
-            framesize = 64 + 80
+            framesize = 16 * WORD + 20 * WORD
             codebuilder.restore_nonvolatiles(framesize)
 
-            codebuilder.lwz(0, 1, framesize + 4) # 36
+            if IS_PPC_32:
+                codebuilder.lwz(0, 1, framesize + WORD) # 36
+            else:
+                codebuilder.ld(0, 1, framesize + WORD) # 36
             codebuilder.mtlr(0)
             codebuilder.addi(1, 1, framesize)
 

From noreply at buildbot.pypy.org  Tue Sep  6 19:33:24 2011
From: noreply at buildbot.pypy.org (edelsohn)
Date: Tue,  6 Sep 2011 19:33:24 +0200 (CEST)
Subject: [pypy-commit] pypy ppc-jit-backend: PPC64 updates
Message-ID: <20110906173324.D6BF48203C@wyvern.cs.uni-duesseldorf.de>

Author: edelsohn
Branch: ppc-jit-backend
Changeset: r47117:6e91461f4d96
Date: 2011-09-06 13:26 -0400
http://bitbucket.org/pypy/pypy/changeset/6e91461f4d96/

Log:	PPC64 updates

diff --git a/pypy/jit/backend/ppc/ppcgen/ppc_assembler.py b/pypy/jit/backend/ppc/ppcgen/ppc_assembler.py
--- a/pypy/jit/backend/ppc/ppcgen/ppc_assembler.py
+++ b/pypy/jit/backend/ppc/ppcgen/ppc_assembler.py
@@ -936,16 +936,21 @@
         if IS_PPC_32:
             self.stwx(source_reg, 0, 0)
         else:
-            # ? 
-            self.std(source_reg, 0, 10)
+            self.stdx(source_reg, 0, 0)
 
     def save_nonvolatiles(self, framesize):
         for i, reg in enumerate(NONVOLATILES):
-            self.stw(reg, 1, framesize - 4 * i)
+            if IS_PPC_32:
+                self.stw(reg, 1, framesize - WORD * i)
+            else:
+                self.std(reg, 1, framesize - WORD * i)
 
     def restore_nonvolatiles(self, framesize):
         for i, reg in enumerate(NONVOLATILES):
-            self.lwz(reg, 1, framesize - i * 4)
+            if IS_PPC_32:
+                self.lwz(reg, 1, framesize - WORD * i)
+            else:
+                self.ld(reg, 1, framesize - WORD * i)
         
 
     # translate a trace operation to corresponding machine code
@@ -1430,10 +1435,16 @@
             for i, arg in enumerate(remaining_args):
                 if isinstance(arg, Box):
                     #self.mr(0, cpu.reg_map[arg])
-                    self.stw(cpu.reg_map[arg], 1, 8 + WORD * i)
+                    if IS_PPC_32:
+                        self.stw(cpu.reg_map[arg], 1, 8 + WORD * i)
+                    else:
+                        self.std(cpu.reg_map[arg], 1, 8 + WORD * i)
                 elif isinstance(arg, Const):
                     self.load_word(0, arg.value)
-                    self.stw(0, 1, 8 + WORD * i)
+                    if IS_PPC_32:
+                        self.stw(0, 1, 8 + WORD * i)
+                    else:
+                        self.std(0, 1, 8 + WORD * i)
                 else:
                     assert 0, "%s not supported yet" % arg
 
@@ -1590,11 +1601,14 @@
             else:
                 assert 0, "arg type not suported"
 
-        framesize = 64 + 80
+        framesize = 16 * WORD + 20 * WORD
 
         self.restore_nonvolatiles(framesize)
 
-        self.lwz(0, 1, framesize + 4) # 36
+        if IS_PPC_32:
+            self.lwz(0, 1, framesize + WORD) # 36
+        else:
+            self.ld(0, 1, framesize + WORD) # 36
         self.mtlr(0)
         self.addi(1, 1, framesize)
         self.load_word(3, identifier)

From noreply at buildbot.pypy.org  Tue Sep  6 19:35:00 2011
From: noreply at buildbot.pypy.org (arigo)
Date: Tue,  6 Sep 2011 19:35:00 +0200 (CEST)
Subject: [pypy-commit] pypy default: 'Python 2.7 -A' raises TypeError in
	this corner case.
Message-ID: <20110906173500.82BAB8203C@wyvern.cs.uni-duesseldorf.de>

Author: Armin Rigo 
Branch: 
Changeset: r47118:ebe7ce81d5a1
Date: 2011-09-06 19:08 +0200
http://bitbucket.org/pypy/pypy/changeset/ebe7ce81d5a1/

Log:	'Python 2.7 -A' raises TypeError in this corner case.

diff --git a/pypy/objspace/std/test/test_stringformat.py b/pypy/objspace/std/test/test_stringformat.py
--- a/pypy/objspace/std/test/test_stringformat.py
+++ b/pypy/objspace/std/test/test_stringformat.py
@@ -168,7 +168,7 @@
 
     def test_incomplete_format(self):
         raises(ValueError, '%'.__mod__, ((23,),))
-        raises(ValueError, '%('.__mod__, ({},))
+        raises((ValueError, TypeError), '%('.__mod__, ({},))
 
     def test_format_char(self):
         import sys

From noreply at buildbot.pypy.org  Tue Sep  6 19:35:01 2011
From: noreply at buildbot.pypy.org (arigo)
Date: Tue,  6 Sep 2011 19:35:01 +0200 (CEST)
Subject: [pypy-commit] pypy default: Tests and fix:
Message-ID: <20110906173501.B99AB8203C@wyvern.cs.uni-duesseldorf.de>

Author: Armin Rigo 
Branch: 
Changeset: r47119:09816858a87a
Date: 2011-09-06 19:33 +0200
http://bitbucket.org/pypy/pypy/changeset/09816858a87a/

Log:	Tests and fix:

	object.__str__() was implemented by calling space.repr(), instead of
	directly calling the __repr__() method. The difference is that the
	first does type checking on the result, while the latter does not.
	It's not the job of object.__str__() to do type checking, but of its
	callers.

diff --git a/pypy/objspace/std/objecttype.py b/pypy/objspace/std/objecttype.py
--- a/pypy/objspace/std/objecttype.py
+++ b/pypy/objspace/std/objecttype.py
@@ -24,7 +24,12 @@
     return w_obj.getrepr(space, '%s object' % (classname,))
 
 def descr__str__(space, w_obj):
-    return space.repr(w_obj)
+    w_type = space.type(w_obj)
+    w_impl = w_type.lookup("__repr__")
+    if w_impl is None:
+        raise OperationError(space.w_TypeError,      # can it really occur?
+                             space.wrap("operand does not support unary str"))
+    return space.get_and_call_function(w_impl, w_obj)
 
 def descr__class__(space, w_obj):
     return space.type(w_obj)
diff --git a/pypy/objspace/std/test/test_obj.py b/pypy/objspace/std/test/test_obj.py
--- a/pypy/objspace/std/test/test_obj.py
+++ b/pypy/objspace/std/test/test_obj.py
@@ -94,3 +94,10 @@
         #assert len(log) == 1
         #assert log[0].message.args == ("object.__init__() takes no parameters",)
         #assert type(log[0].message) is DeprecationWarning
+
+    def test_object_str(self):
+        # obscure case
+        class A(object):
+            def __repr__(self):
+                return 123456
+        assert A().__str__() == 123456
diff --git a/pypy/objspace/std/test/test_unicodeobject.py b/pypy/objspace/std/test/test_unicodeobject.py
--- a/pypy/objspace/std/test/test_unicodeobject.py
+++ b/pypy/objspace/std/test/test_unicodeobject.py
@@ -780,8 +780,22 @@
         assert type(s) is unicode
         assert s == u'\u1234'
 
+        # now the same with a new-style class...
+        class A(object):
+            def __init__(self, num):
+                self.num = num
+            def __str__(self):
+                return unichr(self.num)
+
+        s = '%s' % A(111)    # this is ASCII
+        assert type(s) is unicode
+        assert s == chr(111)
+
+        s = '%s' % A(0x1234)    # this is not ASCII
+        assert type(s) is unicode
+        assert s == u'\u1234'
+
     def test_formatting_unicode__str__2(self):
-        skip("this is completely insane")
         class A:
             def __str__(self):
                 return u'baz'
@@ -798,9 +812,22 @@
         s = '%s %s' % (a, b)
         assert s == u'baz bar'
 
+        skip("but this case here is completely insane")
         s = '%s %s' % (b, a)
         assert s == u'foo baz'
 
+    def test_formatting_unicode__str__3(self):
+        # "bah" is all I can say
+        class X(object):
+            def __repr__(self):
+                return u'\u1234'
+        '%s' % X()
+        #
+        class X(object):
+            def __str__(self):
+                return u'\u1234'
+        '%s' % X()
+
     def test_str_subclass(self):
         class Foo9(str):
             def __unicode__(self):

From noreply at buildbot.pypy.org  Tue Sep  6 19:35:03 2011
From: noreply at buildbot.pypy.org (arigo)
Date: Tue,  6 Sep 2011 19:35:03 +0200 (CEST)
Subject: [pypy-commit] pypy default: Add comment.
Message-ID: <20110906173503.2DAA98203C@wyvern.cs.uni-duesseldorf.de>

Author: Armin Rigo 
Branch: 
Changeset: r47120:ddff981df9d5
Date: 2011-09-06 19:34 +0200
http://bitbucket.org/pypy/pypy/changeset/ddff981df9d5/

Log:	Add comment.

diff --git a/pypy/objspace/std/test/test_obj.py b/pypy/objspace/std/test/test_obj.py
--- a/pypy/objspace/std/test/test_obj.py
+++ b/pypy/objspace/std/test/test_obj.py
@@ -96,7 +96,8 @@
         #assert type(log[0].message) is DeprecationWarning
 
     def test_object_str(self):
-        # obscure case
+        # obscure case: __str__() must delegate to __repr__() without adding
+        # type checking on its own
         class A(object):
             def __repr__(self):
                 return 123456

From noreply at buildbot.pypy.org  Tue Sep  6 22:46:53 2011
From: noreply at buildbot.pypy.org (fijal)
Date: Tue,  6 Sep 2011 22:46:53 +0200 (CEST)
Subject: [pypy-commit] pypy inline-dict-ops: fix tests
Message-ID: <20110906204653.E4E738203C@wyvern.cs.uni-duesseldorf.de>

Author: Maciej Fijalkowski 
Branch: inline-dict-ops
Changeset: r47121:2227e3fb86ed
Date: 2011-09-06 22:46 +0200
http://bitbucket.org/pypy/pypy/changeset/2227e3fb86ed/

Log:	fix tests

diff --git a/pypy/jit/backend/llsupport/test/test_asmmemmgr.py b/pypy/jit/backend/llsupport/test/test_asmmemmgr.py
--- a/pypy/jit/backend/llsupport/test/test_asmmemmgr.py
+++ b/pypy/jit/backend/llsupport/test/test_asmmemmgr.py
@@ -211,14 +211,14 @@
     debug._log = debug.DebugLog()
     try:
         mc._dump(addr, 'test-logname-section')
-        log = list(debug._log)
+        log = list(debug._log) 
     finally:
         debug._log = None
     encoded = ''.join(writtencode).encode('hex').upper()
     ataddr = '@%x' % addr
     assert log == [('test-logname-section',
                     [('debug_print', 'CODE_DUMP', ataddr, '+0 ', encoded)])]
-    #
+    
     lltype.free(p, flavor='raw')
 
 def test_blockbuildermixin2():
diff --git a/pypy/jit/backend/llsupport/test/test_descr.py b/pypy/jit/backend/llsupport/test/test_descr.py
--- a/pypy/jit/backend/llsupport/test/test_descr.py
+++ b/pypy/jit/backend/llsupport/test/test_descr.py
@@ -159,7 +159,7 @@
     clsf = getArrayDescrClass(A4)
     assert clsf != cls
     assert clsf == getArrayDescrClass(lltype.GcArray(lltype.Float))
-    clss = getArrayDescrClass(A5)
+    clss = getArrayDescrClass(A6)
     assert clss not in (clsf, cls)
     assert clss == getArrayDescrClass(lltype.GcArray(rffi.UINT))
     #
@@ -174,7 +174,7 @@
     assert descr2.__class__ is GcPtrArrayDescr
     assert descr3.__class__ is NonGcPtrArrayDescr
     assert descr4.__class__ is clsf
-    assert descr5.__class__ is clss
+    assert descr6.__class__ is clss
     assert descr1 == get_array_descr(c0, lltype.GcArray(lltype.Char))
     assert not descr1.is_array_of_pointers()
     assert     descr2.is_array_of_pointers()

From noreply at buildbot.pypy.org  Tue Sep  6 22:59:00 2011
From: noreply at buildbot.pypy.org (boemmels)
Date: Tue,  6 Sep 2011 22:59:00 +0200 (CEST)
Subject: [pypy-commit] lang-scheme default: Split tests for symbols &
 strings out of test_simple
Message-ID: <20110906205900.C57DF8203C@wyvern.cs.uni-duesseldorf.de>

Author: Juergen Boemmels 
Branch: 
Changeset: r5:324223d56732
Date: 2011-09-05 23:02 +0200
http://bitbucket.org/pypy/lang-scheme/changeset/324223d56732/

Log:	Split tests for symbols & strings out of test_simple

diff --git a/scheme/test/test_parser.py b/scheme/test/test_parser.py
--- a/scheme/test/test_parser.py
+++ b/scheme/test/test_parser.py
@@ -39,8 +39,13 @@
     w_fixnum = parse_sexpr('1123')
     assert unwrap(w_fixnum) == 1123
     assert isinstance(w_fixnum, W_Integer)
-    w_fixnum = parse_sexpr('abfa__')
-    assert isinstance(w_fixnum, W_Symbol)
+
+def test_symbol():
+    w_sym = parse_sexpr('abfa__')
+    assert isinstance(w_sym, W_Symbol)
+    assert w_sym.to_string() == 'abfa__'
+
+def test_string():
     t = parse_sexpr(r'''"don't believe \"them\""''')
     assert isinstance(t, W_String)
     assert unwrap(t) == 'don\'t believe "them"'

From noreply at buildbot.pypy.org  Tue Sep  6 22:59:01 2011
From: noreply at buildbot.pypy.org (boemmels)
Date: Tue,  6 Sep 2011 22:59:01 +0200 (CEST)
Subject: [pypy-commit] lang-scheme default: More symbol test
Message-ID: <20110906205901.D885D8203C@wyvern.cs.uni-duesseldorf.de>

Author: Juergen Boemmels 
Branch: 
Changeset: r6:7702fffadbc2
Date: 2011-09-05 23:19 +0200
http://bitbucket.org/pypy/lang-scheme/changeset/7702fffadbc2/

Log:	More symbol test

diff --git a/scheme/test/test_parser.py b/scheme/test/test_parser.py
--- a/scheme/test/test_parser.py
+++ b/scheme/test/test_parser.py
@@ -44,6 +44,18 @@
     w_sym = parse_sexpr('abfa__')
     assert isinstance(w_sym, W_Symbol)
     assert w_sym.to_string() == 'abfa__'
+    
+    more_syms = ['abc',
+                 'call/cc',
+                 '+',
+                 '-',
+                 'set!',
+                 'eqv?',
+                ]
+    for s in more_syms:
+        w_sym = parse_sexpr(s)
+        assert isinstance(w_sym, W_Symbol)
+        assert w_sym.to_string() == s
 
 def test_string():
     t = parse_sexpr(r'''"don't believe \"them\""''')

From noreply at buildbot.pypy.org  Tue Sep  6 22:59:02 2011
From: noreply at buildbot.pypy.org (boemmels)
Date: Tue,  6 Sep 2011 22:59:02 +0200 (CEST)
Subject: [pypy-commit] lang-scheme default: Fix strings: Allow escaped
	backslash
Message-ID: <20110906205902.EC3D58203C@wyvern.cs.uni-duesseldorf.de>

Author: Juergen Boemmels 
Branch: 
Changeset: r7:0e79d2ada637
Date: 2011-09-06 22:58 +0200
http://bitbucket.org/pypy/lang-scheme/changeset/0e79d2ada637/

Log:	Fix strings: Allow escaped backslash

diff --git a/scheme/ssparser.py b/scheme/ssparser.py
--- a/scheme/ssparser.py
+++ b/scheme/ssparser.py
@@ -2,18 +2,25 @@
 from pypy.rlib.parsing.makepackrat import BacktrackException, Status
 from scheme.object import W_Pair, W_Integer, W_String, symbol, \
         w_nil, W_Boolean, W_Real, quote, qq, unquote, unquote_splicing, \
-        w_ellipsis
+        w_ellipsis, SchemeSyntaxError
 
 def str_unquote(s):
     str_lst = []
-    last_ch = ''
-    for c in s[1:]:
-        if last_ch == '\\' and c == '"':
-            pass
+    pos = 1
+    last = len(s)-1
+    while pos < last:
+        ch = s[pos]
+        if ch == '\\':
+            pos += 1
+            ch = s[pos]
+            if ch == '\\' or ch == '\"':
+                str_lst.append(ch)
+            else:
+                raise SchemeSyntaxError
         else:
-            str_lst.append(last_ch)
+            str_lst.append(ch)
 
-        last_ch = c
+        pos += 1
 
     return ''.join(str_lst)
 
diff --git a/scheme/test/test_parser.py b/scheme/test/test_parser.py
--- a/scheme/test/test_parser.py
+++ b/scheme/test/test_parser.py
@@ -62,6 +62,16 @@
     assert isinstance(t, W_String)
     assert unwrap(t) == 'don\'t believe "them"'
 
+    more_strings = [(r'''"simple string"''', r'''simple string'''),
+                    (r'''"\\ backslash"''', r'''\ backslash'''),
+                    (r'''"\\\\"''',r'''\\'''),
+                    (r'''"with \"quotes\""''', r'''with "quotes"'''),
+                   ]
+    for code, contents in more_strings:
+        w_string = parse_sexpr(code)
+        assert isinstance(w_string, W_String)
+        assert unwrap(w_string) == contents
+
 def test_objects():
     w_fixnum = parse_sexpr('-12345')
     assert isinstance(w_fixnum, W_Integer)

From noreply at buildbot.pypy.org  Wed Sep  7 11:02:02 2011
From: noreply at buildbot.pypy.org (cfbolz)
Date: Wed,  7 Sep 2011 11:02:02 +0200 (CEST)
Subject: [pypy-commit] pypy improve-heap-caching-tracing: cache the length
	of arrays
Message-ID: <20110907090202.292B182213@wyvern.cs.uni-duesseldorf.de>

Author: Carl Friedrich Bolz 
Branch: improve-heap-caching-tracing
Changeset: r47122:e5b582b1252b
Date: 2011-09-06 14:03 +0200
http://bitbucket.org/pypy/pypy/changeset/e5b582b1252b/

Log:	cache the length of arrays

diff --git a/pypy/jit/metainterp/heapcache.py b/pypy/jit/metainterp/heapcache.py
--- a/pypy/jit/metainterp/heapcache.py
+++ b/pypy/jit/metainterp/heapcache.py
@@ -18,6 +18,8 @@
         # heap array cache
         # maps descrs to {index: {from_box: to_box}} dicts
         self.heap_array_cache = {}
+        # cache the length of arrays
+        self.length_cache = {}
 
     def invalidate_caches(self, opnum, descr):
         if opnum == rop.SETFIELD_GC:
@@ -57,6 +59,10 @@
     def new(self, box):
         self.new_boxes[box] = None
 
+    def new_array(self, box, lengthbox):
+        self.new(box)
+        self.arraylen_now_known(box, lengthbox)
+
     def getfield(self, box, descr):
         d = self.heap_cache.get(descr, None)
         if d:
@@ -112,23 +118,26 @@
         indexcache = cache.get(index, None)
         cache[index] = self._do_write_with_aliasing(indexcache, box, valuebox)
 
+    def arraylen(self, box):
+        return self.length_cache.get(box, None)
+
+    def arraylen_now_known(self, box, lengthbox):
+        self.length_cache[box] = lengthbox
+
+    def _replace_box(self, d, oldbox, newbox):
+        new_d = {}
+        for frombox, tobox in d.iteritems():
+            if frombox is oldbox:
+                frombox = newbox
+            if tobox is oldbox:
+                tobox = newbox
+            new_d[frombox] = tobox
+        return new_d
+
     def replace_box(self, oldbox, newbox):
         for descr, d in self.heap_cache.iteritems():
-            new_d = {}
-            for frombox, tobox in d.iteritems():
-                if frombox is oldbox:
-                    frombox = newbox
-                if tobox is oldbox:
-                    tobox = newbox
-                new_d[frombox] = tobox
-            self.heap_cache[descr] = new_d
+            self.heap_cache[descr] = self._replace_box(d, oldbox, newbox)
         for descr, d in self.heap_array_cache.iteritems():
             for index, cache in d.iteritems():
-                new_cache = {}
-                for frombox, tobox in cache.iteritems():
-                    if frombox is oldbox:
-                        frombox = newbox
-                    if tobox is oldbox:
-                        tobox = newbox
-                    new_cache[frombox] = tobox
-                d[index] = new_cache
+                d[index] = self._replace_box(cache, oldbox, newbox)
+        self.length_cache = self._replace_box(self.length_cache, oldbox, newbox)
diff --git a/pypy/jit/metainterp/pyjitpl.py b/pypy/jit/metainterp/pyjitpl.py
--- a/pypy/jit/metainterp/pyjitpl.py
+++ b/pypy/jit/metainterp/pyjitpl.py
@@ -394,9 +394,9 @@
 ##        self.execute(rop.SUBCLASSOF, box1, box2)
 
     @arguments("descr", "box")
-    def opimpl_new_array(self, itemsizedescr, countbox):
-        resbox = self.execute_with_descr(rop.NEW_ARRAY, itemsizedescr, countbox)
-        self.metainterp.heapcache.new(resbox)
+    def opimpl_new_array(self, itemsizedescr, lengthbox):
+        resbox = self.execute_with_descr(rop.NEW_ARRAY, itemsizedescr, lengthbox)
+        self.metainterp.heapcache.new_array(resbox, lengthbox)
         return resbox
 
     @arguments("box", "descr", "box")
@@ -456,7 +456,12 @@
 
     @arguments("box", "descr")
     def opimpl_arraylen_gc(self, arraybox, arraydescr):
-        return self.execute_with_descr(rop.ARRAYLEN_GC, arraydescr, arraybox)
+        lengthbox = self.metainterp.heapcache.arraylen(arraybox)
+        if lengthbox is None:
+            lengthbox = self.execute_with_descr(
+                    rop.ARRAYLEN_GC, arraydescr, arraybox)
+            self.metainterp.heapcache.arraylen_now_known(arraybox, lengthbox)
+        return lengthbox
 
     @arguments("orgpc", "box", "descr", "box")
     def opimpl_check_neg_index(self, orgpc, arraybox, arraydescr, indexbox):
@@ -465,10 +470,9 @@
         negbox = self.implement_guard_value(orgpc, negbox)
         if negbox.getint():
             # the index is < 0; add the array length to it
-            lenbox = self.metainterp.execute_and_record(
-                rop.ARRAYLEN_GC, arraydescr, arraybox)
+            lengthbox = self.opimpl_arraylen_gc(arraybox, arraydescr)
             indexbox = self.metainterp.execute_and_record(
-                rop.INT_ADD, None, indexbox, lenbox)
+                rop.INT_ADD, None, indexbox, lengthbox)
         return indexbox
 
     @arguments("descr", "descr", "descr", "descr", "box")
@@ -721,7 +725,7 @@
     def opimpl_arraylen_vable(self, pc, box, fdescr, adescr):
         if self._nonstandard_virtualizable(pc, box):
             arraybox = self._opimpl_getfield_gc_any(box, fdescr)
-            return self.execute_with_descr(rop.ARRAYLEN_GC, adescr, arraybox)
+            return self.opimpl_arraylen_gc(arraybox, adescr)
         vinfo = self.metainterp.jitdriver_sd.virtualizable_info
         virtualizable_box = self.metainterp.virtualizable_boxes[-1]
         virtualizable = vinfo.unwrap_virtualizable_box(virtualizable_box)
diff --git a/pypy/jit/metainterp/test/test_heapcache.py b/pypy/jit/metainterp/test/test_heapcache.py
--- a/pypy/jit/metainterp/test/test_heapcache.py
+++ b/pypy/jit/metainterp/test/test_heapcache.py
@@ -6,6 +6,8 @@
 box2 = object()
 box3 = object()
 box4 = object()
+lengthbox1 = object()
+lengthbox2 = object()
 descr1 = object()
 descr2 = object()
 descr3 = object()
@@ -206,6 +208,15 @@
         assert h.getarrayitem(box3, descr1, index1) is box4
         assert h.getarrayitem(box1, descr1, index1) is box3 # box1 and box3 cannot alias
 
+    def test_length_cache(self):
+        h = HeapCache()
+        h.new_array(box1, lengthbox1)
+        assert h.arraylen(box1) is lengthbox1
+
+        assert h.arraylen(box2) is None
+        h.arraylen_now_known(box2, lengthbox2)
+        assert h.arraylen(box2) is lengthbox2
+
 
     def test_invalidate_cache(self):
         h = HeapCache()
@@ -252,14 +263,20 @@
         h = HeapCache()
         h.setarrayitem(box1, descr1, index1, box2)
         h.setarrayitem(box1, descr2, index1, box3)
+        h.arraylen_now_known(box1, lengthbox1)
         h.setarrayitem(box2, descr1, index2, box1)
         h.setarrayitem(box3, descr2, index2, box1)
         h.setarrayitem(box2, descr3, index2, box3)
         h.replace_box(box1, box4)
         assert h.getarrayitem(box1, descr1, index1) is None
         assert h.getarrayitem(box1, descr2, index1) is None
+        assert h.arraylen(box1) is None
+        assert h.arraylen(box4) is lengthbox1
         assert h.getarrayitem(box4, descr1, index1) is box2
         assert h.getarrayitem(box4, descr2, index1) is box3
         assert h.getarrayitem(box2, descr1, index2) is box4
         assert h.getarrayitem(box3, descr2, index2) is box4
         assert h.getarrayitem(box2, descr3, index2) is box3
+
+        h.replace_box(lengthbox1, lengthbox2)
+        assert h.arraylen(box4) is lengthbox2
diff --git a/pypy/jit/metainterp/test/test_tracingopts.py b/pypy/jit/metainterp/test/test_tracingopts.py
--- a/pypy/jit/metainterp/test/test_tracingopts.py
+++ b/pypy/jit/metainterp/test/test_tracingopts.py
@@ -469,3 +469,17 @@
         assert res == 2 * -7 + 2 * -8
         self.check_operations_history(getarrayitem_gc=0)
 
+    def test_length_caching(self):
+        class Gbl(object):
+            pass
+        g = Gbl()
+        g.a = [0] * 7
+        def fn(n):
+            a = g.a
+            res = len(a) + len(a)
+            a1 = [0] * n
+            g.a = a1
+            return len(a1) + res
+        res = self.interp_operations(fn, [7])
+        assert res == 7 * 3
+        self.check_operations_history(arraylen_gc=1)

From noreply at buildbot.pypy.org  Wed Sep  7 11:02:03 2011
From: noreply at buildbot.pypy.org (cfbolz)
Date: Wed,  7 Sep 2011 11:02:03 +0200 (CEST)
Subject: [pypy-commit] pypy improve-heap-caching-tracing: typo
Message-ID: <20110907090203.6A01B822AB@wyvern.cs.uni-duesseldorf.de>

Author: Carl Friedrich Bolz 
Branch: improve-heap-caching-tracing
Changeset: r47123:055211ce9263
Date: 2011-09-07 10:38 +0200
http://bitbucket.org/pypy/pypy/changeset/055211ce9263/

Log:	typo

diff --git a/pypy/jit/metainterp/heapcache.py b/pypy/jit/metainterp/heapcache.py
--- a/pypy/jit/metainterp/heapcache.py
+++ b/pypy/jit/metainterp/heapcache.py
@@ -47,7 +47,7 @@
     def is_class_known(self, box):
         return box in self.known_class_boxes
 
-    def class_now_know(self, box):
+    def class_now_known(self, box):
         self.known_class_boxes[box] = None
 
     def is_nonstandard_virtualizable(self, box):
diff --git a/pypy/jit/metainterp/pyjitpl.py b/pypy/jit/metainterp/pyjitpl.py
--- a/pypy/jit/metainterp/pyjitpl.py
+++ b/pypy/jit/metainterp/pyjitpl.py
@@ -375,7 +375,7 @@
         cls = heaptracker.descr2vtable(cpu, sizedescr)
         resbox = self.execute(rop.NEW_WITH_VTABLE, ConstInt(cls))
         self.metainterp.heapcache.new(resbox)
-        self.metainterp.heapcache.class_now_know(resbox)
+        self.metainterp.heapcache.class_now_known(resbox)
         return resbox
 
 ##    @FixME  #arguments("box")
@@ -884,7 +884,7 @@
         clsbox = self.cls_of_box(box)
         if not self.metainterp.heapcache.is_class_known(box):
             self.generate_guard(rop.GUARD_CLASS, box, [clsbox], resumepc=orgpc)
-            self.metainterp.heapcache.class_now_know(box)
+            self.metainterp.heapcache.class_now_known(box)
         return clsbox
 
     @arguments("int", "orgpc")
diff --git a/pypy/jit/metainterp/test/test_heapcache.py b/pypy/jit/metainterp/test/test_heapcache.py
--- a/pypy/jit/metainterp/test/test_heapcache.py
+++ b/pypy/jit/metainterp/test/test_heapcache.py
@@ -40,7 +40,7 @@
         h = HeapCache()
         assert not h.is_class_known(1)
         assert not h.is_class_known(2)
-        h.class_now_know(1)
+        h.class_now_known(1)
         assert h.is_class_known(1)
         assert not h.is_class_known(2)
 

From noreply at buildbot.pypy.org  Wed Sep  7 11:02:04 2011
From: noreply at buildbot.pypy.org (cfbolz)
Date: Wed,  7 Sep 2011 11:02:04 +0200 (CEST)
Subject: [pypy-commit] pypy improve-heap-caching-tracing: some XXXs
Message-ID: <20110907090204.A47FF82213@wyvern.cs.uni-duesseldorf.de>

Author: Carl Friedrich Bolz 
Branch: improve-heap-caching-tracing
Changeset: r47124:c6cae1ac723f
Date: 2011-09-07 10:39 +0200
http://bitbucket.org/pypy/pypy/changeset/c6cae1ac723f/

Log:	some XXXs

diff --git a/pypy/jit/metainterp/pyjitpl.py b/pypy/jit/metainterp/pyjitpl.py
--- a/pypy/jit/metainterp/pyjitpl.py
+++ b/pypy/jit/metainterp/pyjitpl.py
@@ -367,6 +367,7 @@
 
     @arguments("descr")
     def opimpl_new(self, sizedescr):
+        # XXX heapcache.new
         return self.execute_with_descr(rop.NEW, sizedescr)
 
     @arguments("descr")
@@ -478,6 +479,7 @@
     @arguments("descr", "descr", "descr", "descr", "box")
     def opimpl_newlist(self, structdescr, lengthdescr, itemsdescr, arraydescr,
                        sizebox):
+        # XXX use heapcache
         sbox = self.metainterp.execute_and_record(rop.NEW, structdescr)
         self._opimpl_setfield_gc_any(sbox, lengthdescr, sizebox)
         abox = self.metainterp.execute_and_record(rop.NEW_ARRAY, arraydescr,

From noreply at buildbot.pypy.org  Wed Sep  7 11:02:05 2011
From: noreply at buildbot.pypy.org (cfbolz)
Date: Wed,  7 Sep 2011 11:02:05 +0200 (CEST)
Subject: [pypy-commit] pypy improve-heap-caching-tracing: be more precise
	about getarrayitem
Message-ID: <20110907090205.E21A682213@wyvern.cs.uni-duesseldorf.de>

Author: Carl Friedrich Bolz 
Branch: improve-heap-caching-tracing
Changeset: r47125:77217d39842f
Date: 2011-09-07 11:01 +0200
http://bitbucket.org/pypy/pypy/changeset/77217d39842f/

Log:	be more precise about getarrayitem

diff --git a/pypy/jit/metainterp/heapcache.py b/pypy/jit/metainterp/heapcache.py
--- a/pypy/jit/metainterp/heapcache.py
+++ b/pypy/jit/metainterp/heapcache.py
@@ -107,6 +107,17 @@
             if indexcache is not None:
                 return indexcache.get(box, None)
 
+    def getarrayitem_now_known(self, box, descr, indexbox, valuebox):
+        if not isinstance(indexbox, ConstInt):
+            return
+        index = indexbox.getint()
+        cache = self.heap_array_cache.setdefault(descr, {})
+        indexcache = cache.get(index, None)
+        if indexcache is not None:
+            indexcache[box] = valuebox
+        else:
+            cache[index] = {box: valuebox}
+
     def setarrayitem(self, box, descr, indexbox, valuebox):
         if not isinstance(indexbox, ConstInt):
             cache = self.heap_array_cache.get(descr, None)
@@ -118,6 +129,7 @@
         indexcache = cache.get(index, None)
         cache[index] = self._do_write_with_aliasing(indexcache, box, valuebox)
 
+
     def arraylen(self, box):
         return self.length_cache.get(box, None)
 
diff --git a/pypy/jit/metainterp/pyjitpl.py b/pypy/jit/metainterp/pyjitpl.py
--- a/pypy/jit/metainterp/pyjitpl.py
+++ b/pypy/jit/metainterp/pyjitpl.py
@@ -408,7 +408,7 @@
             return tobox
         resbox = self.execute_with_descr(rop.GETARRAYITEM_GC,
                                          arraydescr, arraybox, indexbox)
-        self.metainterp.heapcache.setarrayitem(
+        self.metainterp.heapcache.getarrayitem_now_known(
                 arraybox, arraydescr, indexbox, resbox)
         return resbox
 
diff --git a/pypy/jit/metainterp/test/test_heapcache.py b/pypy/jit/metainterp/test/test_heapcache.py
--- a/pypy/jit/metainterp/test/test_heapcache.py
+++ b/pypy/jit/metainterp/test/test_heapcache.py
@@ -179,6 +179,20 @@
         assert h.getarrayitem(box1, descr1, index1) is None
         assert h.getarrayitem(box1, descr1, index2) is None
 
+    def test_heapcache_read_fields_multiple_array(self):
+        h = HeapCache()
+        h.getarrayitem_now_known(box1, descr1, index1, box2)
+        h.getarrayitem_now_known(box3, descr1, index1, box4)
+        assert h.getarrayitem(box1, descr1, index1) is box2
+        assert h.getarrayitem(box1, descr2, index1) is None
+        assert h.getarrayitem(box3, descr1, index1) is box4
+        assert h.getarrayitem(box3, descr2, index1) is None
+
+        h.reset()
+        assert h.getarrayitem(box1, descr1, index1) is None
+        assert h.getarrayitem(box1, descr2, index1) is None
+        assert h.getarrayitem(box3, descr1, index1) is None
+        assert h.getarrayitem(box3, descr2, index1) is None
 
     def test_heapcache_write_fields_multiple_array(self):
         h = HeapCache()
diff --git a/pypy/jit/metainterp/test/test_tracingopts.py b/pypy/jit/metainterp/test/test_tracingopts.py
--- a/pypy/jit/metainterp/test/test_tracingopts.py
+++ b/pypy/jit/metainterp/test/test_tracingopts.py
@@ -469,6 +469,27 @@
         assert res == 2 * -7 + 2 * -8
         self.check_operations_history(getarrayitem_gc=0)
 
+    def test_heap_caching_multiple_arrays_getarrayitem(self):
+        class Gbl(object):
+            pass
+        g = Gbl()
+        g.a1 = [7, 8, 9]
+        g.a2 = [8, 9, 10, 11]
+
+        def fn(i):
+            if i < 0:
+                g.a1 = [7, 8, 9]
+                g.a2 = [7, 8, 9, 10]
+            jit.promote(i)
+            a1 = g.a1
+            a1[i + 1] = 15 # make lists mutable
+            a2 = g.a2
+            a2[i + 1] = 19
+            return a1[i] + a2[i] + a1[i] + a2[i]
+        res = self.interp_operations(fn, [0])
+        assert res == 2 * 7 + 2 * 8
+        self.check_operations_history(getarrayitem_gc=2)
+
     def test_length_caching(self):
         class Gbl(object):
             pass

From noreply at buildbot.pypy.org  Wed Sep  7 11:04:04 2011
From: noreply at buildbot.pypy.org (fijal)
Date: Wed,  7 Sep 2011 11:04:04 +0200 (CEST)
Subject: [pypy-commit] pypy space-iterator-improvements: kill dead old
	commented out code
Message-ID: <20110907090404.7057382213@wyvern.cs.uni-duesseldorf.de>

Author: Maciej Fijalkowski 
Branch: space-iterator-improvements
Changeset: r47126:dc2ecf85cd66
Date: 2011-09-07 11:02 +0200
http://bitbucket.org/pypy/pypy/changeset/dc2ecf85cd66/

Log:	kill dead old commented out code

diff --git a/pypy/objspace/std/iterobject.py b/pypy/objspace/std/iterobject.py
--- a/pypy/objspace/std/iterobject.py
+++ b/pypy/objspace/std/iterobject.py
@@ -72,10 +72,6 @@
     w_seqiter.index += 1 
     return w_item
 
-# XXX __length_hint__()
-##def len__SeqIter(space,  w_seqiter):
-##    return w_seqiter.getlength(space)
-
 
 def iter__FastTupleIter(space, w_seqiter):
     return w_seqiter
@@ -93,10 +89,6 @@
     w_seqiter.index = index + 1
     return w_item
 
-# XXX __length_hint__()
-##def len__FastTupleIter(space, w_seqiter):
-##    return w_seqiter.getlength(space)
-
 
 def iter__FastListIter(space, w_seqiter):
     return w_seqiter
@@ -114,10 +106,6 @@
     w_seqiter.index = index + 1
     return w_item
 
-# XXX __length_hint__()
-##def len__FastListIter(space, w_seqiter):
-##    return w_seqiter.getlength(space)
-
 
 def iter__ReverseSeqIter(space, w_seqiter):
     return w_seqiter
@@ -135,20 +123,5 @@
         raise OperationError(space.w_StopIteration, space.w_None) 
     return w_item
 
-# XXX __length_hint__()
-##def len__ReverseSeqIter(space, w_seqiter):
-##    if w_seqiter.w_seq is None:
-##        return space.wrap(0)
-##    index = w_seqiter.index+1
-##    w_length = space.len(w_seqiter.w_seq)
-##    # if length of sequence is less than index :exhaust iterator
-##    if space.is_true(space.gt(space.wrap(w_seqiter.index), w_length)):
-##        w_len = space.wrap(0)
-##        w_seqiter.w_seq = None
-##    else:
-##        w_len =space.wrap(index)
-##    if space.is_true(space.lt(w_len,space.wrap(0))):
-##        w_len = space.wrap(0)
-##    return w_len
 
 register_all(vars())

From noreply at buildbot.pypy.org  Wed Sep  7 12:18:13 2011
From: noreply at buildbot.pypy.org (hakanardo)
Date: Wed,  7 Sep 2011 12:18:13 +0200 (CEST)
Subject: [pypy-commit] pypy jit-duplicated_short_boxes: no point in
 importing extra info from preamble about values at these points
Message-ID: <20110907101813.9643182213@wyvern.cs.uni-duesseldorf.de>

Author: Hakan Ardo 
Branch: jit-duplicated_short_boxes
Changeset: r47127:88fb2b9f8a28
Date: 2011-09-07 09:57 +0200
http://bitbucket.org/pypy/pypy/changeset/88fb2b9f8a28/

Log:	no point in importing extra info from preamble about values at these
	points

diff --git a/pypy/jit/metainterp/optimizeopt/optimizer.py b/pypy/jit/metainterp/optimizeopt/optimizer.py
--- a/pypy/jit/metainterp/optimizeopt/optimizer.py
+++ b/pypy/jit/metainterp/optimizeopt/optimizer.py
@@ -561,7 +561,7 @@
         for i in range(n):
             arg = op.getarg(i)
             try:
-                value = self.getvalue(arg)
+                value = self.values[arg]
             except KeyError:
                 pass
             else:
diff --git a/pypy/jit/metainterp/optimizeopt/unroll.py b/pypy/jit/metainterp/optimizeopt/unroll.py
--- a/pypy/jit/metainterp/optimizeopt/unroll.py
+++ b/pypy/jit/metainterp/optimizeopt/unroll.py
@@ -467,7 +467,7 @@
         inputargs.append(box)
         box = newresult
         if box in self.optimizer.values:
-            box = self.optimizer.getvalue(box).force_box()
+            box = self.optimizer.values[box].force_box()
         jumpargs.append(box)
         
 

From noreply at buildbot.pypy.org  Wed Sep  7 12:18:16 2011
From: noreply at buildbot.pypy.org (hakanardo)
Date: Wed,  7 Sep 2011 12:18:16 +0200 (CEST)
Subject: [pypy-commit] pypy jit-duplicated_short_boxes: hg merge default
Message-ID: <20110907101816.550DA82213@wyvern.cs.uni-duesseldorf.de>

Author: Hakan Ardo 
Branch: jit-duplicated_short_boxes
Changeset: r47128:95355f75d469
Date: 2011-09-07 10:00 +0200
http://bitbucket.org/pypy/pypy/changeset/95355f75d469/

Log:	hg merge default

diff --git a/lib-python/conftest.py b/lib-python/conftest.py
--- a/lib-python/conftest.py
+++ b/lib-python/conftest.py
@@ -359,7 +359,7 @@
     RegrTest('test_property.py', core=True),
     RegrTest('test_pstats.py'),
     RegrTest('test_pty.py', skip="unsupported extension module"),
-    RegrTest('test_pwd.py', skip=skip_win32),
+    RegrTest('test_pwd.py', usemodules="pwd", skip=skip_win32),
     RegrTest('test_py3kwarn.py'),
     RegrTest('test_pyclbr.py'),
     RegrTest('test_pydoc.py'),
diff --git a/lib_pypy/_elementtree.py b/lib_pypy/_elementtree.py
new file mode 100644
--- /dev/null
+++ b/lib_pypy/_elementtree.py
@@ -0,0 +1,6 @@
+# Just use ElementTree.
+
+from xml.etree import ElementTree
+
+globals().update(ElementTree.__dict__)
+del __all__
diff --git a/lib_pypy/_pypy_interact.py b/lib_pypy/_pypy_interact.py
--- a/lib_pypy/_pypy_interact.py
+++ b/lib_pypy/_pypy_interact.py
@@ -56,6 +56,10 @@
                 prompt = getattr(sys, 'ps1', '>>> ')
             try:
                 line = raw_input(prompt)
+                # Can be None if sys.stdin was redefined
+                encoding = getattr(sys.stdin, 'encoding', None)
+                if encoding and not isinstance(line, unicode):
+                    line = line.decode(encoding)
             except EOFError:
                 console.write("\n")
                 break
diff --git a/lib_pypy/distributed/test/test_distributed.py b/lib_pypy/distributed/test/test_distributed.py
--- a/lib_pypy/distributed/test/test_distributed.py
+++ b/lib_pypy/distributed/test/test_distributed.py
@@ -9,7 +9,7 @@
 class AppTestDistributed(object):
     def setup_class(cls):
         cls.space = gettestobjspace(**{"objspace.std.withtproxy": True,
-            "usemodules":("_stackless",)})
+            "usemodules":("_continuation",)})
 
     def test_init(self):
         import distributed
@@ -91,10 +91,8 @@
 
 class AppTestDistributedTasklets(object):
     spaceconfig = {"objspace.std.withtproxy": True,
-                   "objspace.usemodules._stackless": True}
+                   "objspace.usemodules._continuation": True}
     def setup_class(cls):
-        #cls.space = gettestobjspace(**{"objspace.std.withtproxy": True,
-        #    "usemodules":("_stackless",)})
         cls.w_test_env = cls.space.appexec([], """():
         from distributed import test_env
         return test_env
diff --git a/lib_pypy/distributed/test/test_greensock.py b/lib_pypy/distributed/test/test_greensock.py
--- a/lib_pypy/distributed/test/test_greensock.py
+++ b/lib_pypy/distributed/test/test_greensock.py
@@ -10,7 +10,7 @@
         if not option.runappdirect:
             py.test.skip("Cannot run this on top of py.py because of PopenGateway")
         cls.space = gettestobjspace(**{"objspace.std.withtproxy": True,
-                                       "usemodules":("_stackless",)})
+                                       "usemodules":("_continuation",)})
         cls.w_remote_side_code = cls.space.appexec([], """():
         import sys
         sys.path.insert(0, '%s')
diff --git a/lib_pypy/distributed/test/test_socklayer.py b/lib_pypy/distributed/test/test_socklayer.py
--- a/lib_pypy/distributed/test/test_socklayer.py
+++ b/lib_pypy/distributed/test/test_socklayer.py
@@ -9,7 +9,8 @@
 class AppTestSocklayer:
     def setup_class(cls):
         cls.space = gettestobjspace(**{"objspace.std.withtproxy": True,
-                                       "usemodules":("_stackless","_socket", "select")})
+                                       "usemodules":("_continuation",
+                                                     "_socket", "select")})
     
     def test_socklayer(self):
         class X(object):
diff --git a/lib_pypy/pypy_test/test_coroutine.py b/lib_pypy/pypy_test/test_coroutine.py
--- a/lib_pypy/pypy_test/test_coroutine.py
+++ b/lib_pypy/pypy_test/test_coroutine.py
@@ -2,7 +2,7 @@
 from py.test import skip, raises
 
 try:
-    from lib_pypy.stackless import coroutine, CoroutineExit
+    from stackless import coroutine, CoroutineExit
 except ImportError, e:
     skip('cannot import stackless: %s' % (e,))
 
@@ -20,10 +20,6 @@
         assert not co.is_zombie
 
     def test_is_zombie_del_without_frame(self):
-        try:
-            import _stackless # are we on pypy with a stackless build?
-        except ImportError:
-            skip("only works on pypy-c-stackless")
         import gc
         res = []
         class MyCoroutine(coroutine):
@@ -45,10 +41,6 @@
         assert res[0], "is_zombie was False in __del__"
 
     def test_is_zombie_del_with_frame(self):
-        try:
-            import _stackless # are we on pypy with a stackless build?
-        except ImportError:
-            skip("only works on pypy-c-stackless")
         import gc
         res = []
         class MyCoroutine(coroutine):
diff --git a/lib_pypy/pyrepl/reader.py b/lib_pypy/pyrepl/reader.py
--- a/lib_pypy/pyrepl/reader.py
+++ b/lib_pypy/pyrepl/reader.py
@@ -576,7 +576,7 @@
         self.console.push_char(char)
         self.handle1(0)
     
-    def readline(self):
+    def readline(self, returns_unicode=False):
         """Read a line.  The implementation of this method also shows
         how to drive Reader if you want more control over the event
         loop."""
@@ -585,6 +585,8 @@
             self.refresh()
             while not self.finished:
                 self.handle1()
+            if returns_unicode:
+                return self.get_unicode()
             return self.get_buffer()
         finally:
             self.restore()
diff --git a/lib_pypy/pyrepl/readline.py b/lib_pypy/pyrepl/readline.py
--- a/lib_pypy/pyrepl/readline.py
+++ b/lib_pypy/pyrepl/readline.py
@@ -198,7 +198,7 @@
         reader.ps1 = prompt
         return reader.readline()
 
-    def multiline_input(self, more_lines, ps1, ps2):
+    def multiline_input(self, more_lines, ps1, ps2, returns_unicode=False):
         """Read an input on possibly multiple lines, asking for more
         lines as long as 'more_lines(unicodetext)' returns an object whose
         boolean value is true.
@@ -209,7 +209,7 @@
             reader.more_lines = more_lines
             reader.ps1 = reader.ps2 = ps1
             reader.ps3 = reader.ps4 = ps2
-            return reader.readline()
+            return reader.readline(returns_unicode=returns_unicode)
         finally:
             reader.more_lines = saved
 
diff --git a/lib_pypy/pyrepl/simple_interact.py b/lib_pypy/pyrepl/simple_interact.py
--- a/lib_pypy/pyrepl/simple_interact.py
+++ b/lib_pypy/pyrepl/simple_interact.py
@@ -54,7 +54,8 @@
             ps1 = getattr(sys, 'ps1', '>>> ')
             ps2 = getattr(sys, 'ps2', '... ')
             try:
-                statement = multiline_input(more_lines, ps1, ps2)
+                statement = multiline_input(more_lines, ps1, ps2,
+                                            returns_unicode=True)
             except EOFError:
                 break
             more = console.push(statement)
diff --git a/lib_pypy/stackless.py b/lib_pypy/stackless.py
--- a/lib_pypy/stackless.py
+++ b/lib_pypy/stackless.py
@@ -4,121 +4,124 @@
 Please refer to their documentation.
 """
 
-DEBUG = True
-
-def dprint(*args):
-    for arg in args:
-        print arg,
-    print
 
 import traceback
-import sys
+import _continuation
+from functools import partial
+
+class TaskletExit(Exception):
+    pass
+
+CoroutineExit = TaskletExit
+
+class GWrap(_continuation.continulet):
+    """This is just a wrapper around continulet to allow
+       to stick additional attributes to a continulet.
+       To be more concrete, we need a backreference to
+       the coroutine object"""
+
+
+class coroutine(object):
+    "we can't have continulet as a base, because continulets can't be rebound"
+
+    def __init__(self):
+        self._frame = None
+        self.is_zombie = False
+
+    def __getattr__(self, attr):
+        return getattr(self._frame, attr)
+
+    def __del__(self):
+        self.is_zombie = True
+        del self._frame
+        self._frame = None
+
+    def bind(self, func, *argl, **argd):
+        """coro.bind(f, *argl, **argd) -> None.
+           binds function f to coro. f will be called with
+           arguments *argl, **argd
+        """
+        if self._frame is None or not self._frame.is_pending():
+
+            def _func(c, *args, **kwargs):
+                return func(*args, **kwargs)
+            
+            run = partial(_func, *argl, **argd)
+            self._frame = frame = GWrap(run)
+        else:
+            raise ValueError("cannot bind a bound coroutine")
+
+    def switch(self):
+        """coro.switch() -> returnvalue
+           switches to coroutine coro. If the bound function
+           f finishes, the returnvalue is that of f, otherwise
+           None is returned
+        """
+        current = _getcurrent()
+        current._jump_to(self)
+
+    def _jump_to(self, coroutine):
+        _tls.current_coroutine = coroutine
+        self._frame.switch(to=coroutine._frame)
+
+    def kill(self):
+        """coro.kill() : kill coroutine coro"""
+        _tls.current_coroutine = self
+        self._frame.throw(CoroutineExit)
+
+    def _is_alive(self):
+        if self._frame is None:
+            return False
+        return not self._frame.is_pending()
+    is_alive = property(_is_alive)
+    del _is_alive
+
+    def getcurrent():
+        """coroutine.getcurrent() -> the currently running coroutine"""
+        try:
+            return _getcurrent()
+        except AttributeError:
+            return _maincoro
+    getcurrent = staticmethod(getcurrent)
+
+    def __reduce__(self):
+        raise TypeError, 'pickling is not possible based upon continulets'
+
+
+def _getcurrent():
+    "Returns the current coroutine (i.e. the one which called this function)."
+    try:
+        return _tls.current_coroutine
+    except AttributeError:
+        # first call in this thread: current == main
+        _coroutine_create_main()
+        return _tls.current_coroutine
+
 try:
-    # If _stackless can be imported then TaskletExit and CoroutineExit are 
-    # automatically added to the builtins.
-    from _stackless import coroutine, greenlet
-except ImportError: # we are running from CPython
-    from greenlet import greenlet, GreenletExit
-    TaskletExit = CoroutineExit = GreenletExit
-    del GreenletExit
-    try:
-        from functools import partial
-    except ImportError: # we are not running python 2.5
-        class partial(object):
-            # just enough of 'partial' to be usefull
-            def __init__(self, func, *argl, **argd):
-                self.func = func
-                self.argl = argl
-                self.argd = argd
+    from thread import _local
+except ImportError:
+    class _local(object):    # assume no threads
+        pass
 
-            def __call__(self):
-                return self.func(*self.argl, **self.argd)
+_tls = _local()
 
-    class GWrap(greenlet):
-        """This is just a wrapper around greenlets to allow
-           to stick additional attributes to a greenlet.
-           To be more concrete, we need a backreference to
-           the coroutine object"""
+def _coroutine_create_main():
+    # create the main coroutine for this thread
+    _tls.current_coroutine = None
+    main_coroutine = coroutine()
+    main_coroutine.bind(lambda x:x)
+    _tls.main_coroutine = main_coroutine
+    _tls.current_coroutine = main_coroutine
+    return main_coroutine
 
-    class MWrap(object):
-        def __init__(self,something):
-            self.something = something
 
-        def __getattr__(self, attr):
-            return getattr(self.something, attr)
+_maincoro = _coroutine_create_main()
 
-    class coroutine(object):
-        "we can't have greenlet as a base, because greenlets can't be rebound"
-
-        def __init__(self):
-            self._frame = None
-            self.is_zombie = False
-
-        def __getattr__(self, attr):
-            return getattr(self._frame, attr)
-
-        def __del__(self):
-            self.is_zombie = True
-            del self._frame
-            self._frame = None
-
-        def bind(self, func, *argl, **argd):
-            """coro.bind(f, *argl, **argd) -> None.
-               binds function f to coro. f will be called with
-               arguments *argl, **argd
-            """
-            if self._frame is None or self._frame.dead:
-                self._frame = frame = GWrap()
-                frame.coro = self
-            if hasattr(self._frame, 'run') and self._frame.run:
-                raise ValueError("cannot bind a bound coroutine")
-            self._frame.run = partial(func, *argl, **argd)
-
-        def switch(self):
-            """coro.switch() -> returnvalue
-               switches to coroutine coro. If the bound function
-               f finishes, the returnvalue is that of f, otherwise
-               None is returned
-            """
-            try:
-                return greenlet.switch(self._frame)
-            except TypeError, exp: # self._frame is the main coroutine
-                return greenlet.switch(self._frame.something)
-
-        def kill(self):
-            """coro.kill() : kill coroutine coro"""
-            self._frame.throw()
-
-        def _is_alive(self):
-            if self._frame is None:
-                return False
-            return not self._frame.dead
-        is_alive = property(_is_alive)
-        del _is_alive
-
-        def getcurrent():
-            """coroutine.getcurrent() -> the currently running coroutine"""
-            try:
-                return greenlet.getcurrent().coro
-            except AttributeError:
-                return _maincoro
-        getcurrent = staticmethod(getcurrent)
-
-        def __reduce__(self):
-            raise TypeError, 'pickling is not possible based upon greenlets'
-
-    _maincoro = coroutine()
-    maingreenlet = greenlet.getcurrent()
-    _maincoro._frame = frame = MWrap(maingreenlet)
-    frame.coro = _maincoro
-    del frame
-    del maingreenlet
 
 from collections import deque
 
 import operator
-__all__ = 'run getcurrent getmain schedule tasklet channel coroutine \
-                greenlet'.split()
+__all__ = 'run getcurrent getmain schedule tasklet channel coroutine'.split()
 
 _global_task_id = 0
 _squeue = None
@@ -131,7 +134,8 @@
 def _scheduler_remove(value):
     try:
         del _squeue[operator.indexOf(_squeue, value)]
-    except ValueError:pass
+    except ValueError:
+        pass
 
 def _scheduler_append(value, normal=True):
     if normal:
diff --git a/pypy/config/pypyoption.py b/pypy/config/pypyoption.py
--- a/pypy/config/pypyoption.py
+++ b/pypy/config/pypyoption.py
@@ -27,7 +27,7 @@
 # --allworkingmodules
 working_modules = default_modules.copy()
 working_modules.update(dict.fromkeys(
-    ["_socket", "unicodedata", "mmap", "fcntl", "_locale",
+    ["_socket", "unicodedata", "mmap", "fcntl", "_locale", "pwd",
      "rctime" , "select", "zipimport", "_lsprof",
      "crypt", "signal", "_rawffi", "termios", "zlib", "bz2",
      "struct", "_hashlib", "_md5", "_sha", "_minimal_curses", "cStringIO",
@@ -58,6 +58,7 @@
     # unix only modules
     del working_modules["crypt"]
     del working_modules["fcntl"]
+    del working_modules["pwd"]
     del working_modules["termios"]
     del working_modules["_minimal_curses"]
 
diff --git a/pypy/config/test/test_config.py b/pypy/config/test/test_config.py
--- a/pypy/config/test/test_config.py
+++ b/pypy/config/test/test_config.py
@@ -281,11 +281,11 @@
 
 def test_underscore_in_option_name():
     descr = OptionDescription("opt", "", [
-        BoolOption("_stackless", "", default=False),
+        BoolOption("_foobar", "", default=False),
     ])
     config = Config(descr)
     parser = to_optparse(config)
-    assert parser.has_option("--_stackless")
+    assert parser.has_option("--_foobar")
 
 def test_none():
     dummy1 = BoolOption('dummy1', 'doc dummy', default=False, cmdline=None)
diff --git a/pypy/doc/config/objspace.usemodules._stackless.txt b/pypy/doc/config/objspace.usemodules._stackless.txt
deleted file mode 100644
--- a/pypy/doc/config/objspace.usemodules._stackless.txt
+++ /dev/null
@@ -1,1 +0,0 @@
-Deprecated.
diff --git a/pypy/doc/config/objspace.usemodules.pwd.txt b/pypy/doc/config/objspace.usemodules.pwd.txt
new file mode 100644
--- /dev/null
+++ b/pypy/doc/config/objspace.usemodules.pwd.txt
@@ -0,0 +1,2 @@
+Use the 'pwd' module. 
+This module is expected to be fully working.
diff --git a/pypy/interpreter/miscutils.py b/pypy/interpreter/miscutils.py
--- a/pypy/interpreter/miscutils.py
+++ b/pypy/interpreter/miscutils.py
@@ -167,3 +167,7 @@
 
     def getmainthreadvalue(self):
         return self._value
+
+    def getallvalues(self):
+        return {0: self._value}
+
diff --git a/pypy/interpreter/pyopcode.py b/pypy/interpreter/pyopcode.py
--- a/pypy/interpreter/pyopcode.py
+++ b/pypy/interpreter/pyopcode.py
@@ -1523,10 +1523,8 @@
 
         if not isinstance(prog, codetype):
             filename = ''
-            if not isinstance(prog, str):
-                if isinstance(prog, basestring):
-                    prog = str(prog)
-                elif isinstance(prog, file):
+            if not isinstance(prog, basestring):
+                if isinstance(prog, file):
                     filename = prog.name
                     prog = prog.read()
                 else:
diff --git a/pypy/interpreter/test/test_exec.py b/pypy/interpreter/test/test_exec.py
--- a/pypy/interpreter/test/test_exec.py
+++ b/pypy/interpreter/test/test_exec.py
@@ -219,3 +219,30 @@
             raise e
 
         assert res == 1
+
+    def test_exec_unicode(self):
+        # 's' is a string
+        s = "x = u'\xd0\xb9\xd1\x86\xd1\x83\xd0\xba\xd0\xb5\xd0\xbd'"
+        # 'u' is a unicode
+        u = s.decode('utf-8')
+        exec u
+        assert len(x) == 6
+        assert ord(x[0]) == 0x0439
+        assert ord(x[1]) == 0x0446
+        assert ord(x[2]) == 0x0443
+        assert ord(x[3]) == 0x043a
+        assert ord(x[4]) == 0x0435
+        assert ord(x[5]) == 0x043d
+
+    def test_eval_unicode(self):
+        u = "u'%s'" % unichr(0x1234)
+        v = eval(u)
+        assert v == unichr(0x1234)
+
+    def test_compile_unicode(self):
+        s = "x = u'\xd0\xb9\xd1\x86\xd1\x83\xd0\xba\xd0\xb5\xd0\xbd'"
+        u = s.decode('utf-8')
+        c = compile(u, '', 'exec')
+        exec c
+        assert len(x) == 6
+        assert ord(x[0]) == 0x0439
diff --git a/pypy/jit/backend/llsupport/regalloc.py b/pypy/jit/backend/llsupport/regalloc.py
--- a/pypy/jit/backend/llsupport/regalloc.py
+++ b/pypy/jit/backend/llsupport/regalloc.py
@@ -57,11 +57,13 @@
     all_regs              = []
     no_lower_byte_regs    = []
     save_around_call_regs = []
-    
+    frame_reg             = None
+
     def __init__(self, longevity, frame_manager=None, assembler=None):
         self.free_regs = self.all_regs[:]
         self.longevity = longevity
         self.reg_bindings = {}
+        self.bindings_to_frame_reg = {}
         self.position = -1
         self.frame_manager = frame_manager
         self.assembler = assembler
@@ -218,6 +220,10 @@
         self.reg_bindings[v] = loc
         return loc
 
+    def force_allocate_frame_reg(self, v):
+        """ Allocate the new variable v in the frame register."""
+        self.bindings_to_frame_reg[v] = None
+
     def force_spill_var(self, var):
         self._sync_var(var)
         try:
@@ -236,6 +242,8 @@
         try:
             return self.reg_bindings[box]
         except KeyError:
+            if box in self.bindings_to_frame_reg:
+                return self.frame_reg
             return self.frame_manager.loc(box)
 
     def return_constant(self, v, forbidden_vars=[], selected_reg=None):
@@ -264,8 +272,9 @@
         self._check_type(v)
         if isinstance(v, Const):
             return self.return_constant(v, forbidden_vars, selected_reg)
-        
         prev_loc = self.loc(v)
+        if prev_loc is self.frame_reg and selected_reg is None:
+            return prev_loc
         loc = self.force_allocate_reg(v, forbidden_vars, selected_reg,
                                       need_lower_byte=need_lower_byte)
         if prev_loc is not loc:
diff --git a/pypy/jit/backend/x86/assembler.py b/pypy/jit/backend/x86/assembler.py
--- a/pypy/jit/backend/x86/assembler.py
+++ b/pypy/jit/backend/x86/assembler.py
@@ -957,6 +957,7 @@
         if (isinstance(from_loc, RegLoc) and from_loc.is_xmm) or (isinstance(to_loc, RegLoc) and to_loc.is_xmm):
             self.mc.MOVSD(to_loc, from_loc)
         else:
+            assert to_loc is not ebp
             self.mc.MOV(to_loc, from_loc)
 
     regalloc_mov = mov # legacy interface
@@ -2510,11 +2511,6 @@
 
     genop_discard_cond_call_gc_wb_array = genop_discard_cond_call_gc_wb
 
-    def genop_force_token(self, op, arglocs, resloc):
-        # RegAlloc.consider_force_token ensures this:
-        assert isinstance(resloc, RegLoc)
-        self.mc.LEA_rb(resloc.value, FORCE_INDEX_OFS)
-
     def not_implemented_op_discard(self, op, arglocs):
         not_implemented("not implemented operation: %s" % op.getopname())
 
diff --git a/pypy/jit/backend/x86/regalloc.py b/pypy/jit/backend/x86/regalloc.py
--- a/pypy/jit/backend/x86/regalloc.py
+++ b/pypy/jit/backend/x86/regalloc.py
@@ -29,6 +29,7 @@
     all_regs = [eax, ecx, edx, ebx, esi, edi]
     no_lower_byte_regs = [esi, edi]
     save_around_call_regs = [eax, edx, ecx]
+    frame_reg = ebp
 
     REGLOC_TO_GCROOTMAP_REG_INDEX = {
         ebx: 1,
@@ -312,8 +313,11 @@
                     self.fm.frame_bindings[arg] = loc
             else:
                 if isinstance(loc, RegLoc):
-                    self.rm.reg_bindings[arg] = loc
-                    used[loc] = None
+                    if loc is ebp:
+                        self.rm.bindings_to_frame_reg[arg] = None
+                    else:
+                        self.rm.reg_bindings[arg] = loc
+                        used[loc] = None
                 else:
                     self.fm.frame_bindings[arg] = loc
         self.rm.free_regs = []
@@ -1358,8 +1362,8 @@
                                             self.assembler.datablockwrapper)
 
     def consider_force_token(self, op):
-        loc = self.rm.force_allocate_reg(op.result)
-        self.Perform(op, [], loc)
+        # the FORCE_TOKEN operation returns directly 'ebp'
+        self.rm.force_allocate_frame_reg(op.result)
 
     def not_implemented_op(self, op):
         not_implemented("not implemented operation: %s" % op.getopname())
diff --git a/pypy/jit/backend/x86/runner.py b/pypy/jit/backend/x86/runner.py
--- a/pypy/jit/backend/x86/runner.py
+++ b/pypy/jit/backend/x86/runner.py
@@ -119,7 +119,8 @@
             setitem(index, null)
 
     def get_latest_force_token(self):
-        return self.assembler.fail_ebp + FORCE_INDEX_OFS
+        # the FORCE_TOKEN operation and this helper both return 'ebp'.
+        return self.assembler.fail_ebp
 
     def execute_token(self, executable_token):
         addr = executable_token._x86_bootstrap_code
@@ -153,8 +154,9 @@
                                        flavor='raw', zero=True,
                                        immortal=True)
 
-    def force(self, addr_of_force_index):
+    def force(self, addr_of_force_token):
         TP = rffi.CArrayPtr(lltype.Signed)
+        addr_of_force_index = addr_of_force_token + FORCE_INDEX_OFS
         fail_index = rffi.cast(TP, addr_of_force_index)[0]
         assert fail_index >= 0, "already forced!"
         faildescr = self.get_fail_descr_from_number(fail_index)
@@ -164,7 +166,7 @@
         # start of "no gc operation!" block
         fail_index_2 = self.assembler.grab_frame_values(
             bytecode,
-            addr_of_force_index - FORCE_INDEX_OFS,
+            addr_of_force_token,
             self.all_null_registers)
         self.assembler.leave_jitted_hook()
         # end of "no gc operation!" block
diff --git a/pypy/jit/metainterp/optimizeopt/heap.py b/pypy/jit/metainterp/optimizeopt/heap.py
--- a/pypy/jit/metainterp/optimizeopt/heap.py
+++ b/pypy/jit/metainterp/optimizeopt/heap.py
@@ -25,7 +25,7 @@
         #      'cached_fields'.
         #
         self._cached_fields = {}
-        self._cached_fields_getfield_op = {}        
+        self._cached_fields_getfield_op = {}
         self._lazy_setfield = None
         self._lazy_setfield_registered = False
 
@@ -81,7 +81,7 @@
     def remember_field_value(self, structvalue, fieldvalue, getfield_op=None):
         assert self._lazy_setfield is None
         self._cached_fields[structvalue] = fieldvalue
-        self._cached_fields_getfield_op[structvalue] = getfield_op        
+        self._cached_fields_getfield_op[structvalue] = getfield_op
 
     def force_lazy_setfield(self, optheap, can_cache=True):
         op = self._lazy_setfield
@@ -167,7 +167,7 @@
 
     def new(self):
         return OptHeap()
-        
+
     def produce_potential_short_preamble_ops(self, sb):
         descrkeys = self.cached_fields.keys()
         if not we_are_translated():
diff --git a/pypy/jit/metainterp/optimizeopt/test/test_optimizebasic.py b/pypy/jit/metainterp/optimizeopt/test/test_optimizebasic.py
--- a/pypy/jit/metainterp/optimizeopt/test/test_optimizebasic.py
+++ b/pypy/jit/metainterp/optimizeopt/test/test_optimizebasic.py
@@ -4711,6 +4711,33 @@
         """
         self.optimize_loop(ops, expected)
 
+    def test_forced_virtuals_aliasing(self):
+        ops = """
+        [i0, i1]
+        p0 = new(descr=ssize)
+        p1 = new(descr=ssize)
+        escape(p0)
+        escape(p1)
+        setfield_gc(p0, i0, descr=adescr)
+        setfield_gc(p1, i1, descr=adescr)
+        i2 = getfield_gc(p0, descr=adescr)
+        jump(i2, i2)
+        """
+        expected = """
+        [i0, i1]
+        p0 = new(descr=ssize)
+        escape(p0)
+        p1 = new(descr=ssize)
+        escape(p1)
+        setfield_gc(p0, i0, descr=adescr)
+        setfield_gc(p1, i1, descr=adescr)
+        jump(i0, i0)
+        """
+        py.test.skip("not implemented")
+        # setfields on things that used to be virtual still can't alias each
+        # other
+        self.optimize_loop(ops, expected)
+
 
 class TestLLtype(BaseTestOptimizeBasic, LLtypeMixin):
     pass
diff --git a/pypy/module/_rawffi/test/test__rawffi.py b/pypy/module/_rawffi/test/test__rawffi.py
--- a/pypy/module/_rawffi/test/test__rawffi.py
+++ b/pypy/module/_rawffi/test/test__rawffi.py
@@ -639,33 +639,6 @@
         a1.free()
         cb.free()
 
-    def test_another_callback_in_stackless(self):
-        try:
-            import _stackless
-        except ImportError:
-            skip("only valid in a stackless pypy-c")
-
-        import _rawffi
-        lib = _rawffi.CDLL(self.lib_name)
-        runcallback = lib.ptr('runcallback', ['P'], 'q')
-        def callback():
-            co = _stackless.coroutine()
-            def f():
-                pass
-            try:
-                co.bind(f)
-                co.switch()
-            except RuntimeError:
-                return 1<<42
-            return -5
-
-        cb = _rawffi.CallbackPtr(callback, [], 'q')
-        a1 = cb.byptr()
-        res = runcallback(a1)
-        assert res[0] == 1<<42
-        a1.free()
-        cb.free()
-
     def test_raising_callback(self):
         import _rawffi, sys
         import StringIO
diff --git a/pypy/module/_stackless/__init__.py b/pypy/module/_stackless/__init__.py
deleted file mode 100644
--- a/pypy/module/_stackless/__init__.py
+++ /dev/null
@@ -1,36 +0,0 @@
-# Package initialisation
-from pypy.interpreter.mixedmodule import MixedModule
-
-class Module(MixedModule):
-    """
-    This module implements Stackless for applications.
-    """
-
-    appleveldefs = {
-        'GreenletExit' : 'app_greenlet.GreenletExit',
-        'GreenletError' : 'app_greenlet.GreenletError',
-    }
-
-    interpleveldefs = {
-        'tasklet'    : 'interp_stackless.tasklet',
-        'coroutine'  : 'interp_coroutine.AppCoroutine',
-        'greenlet'   : 'interp_greenlet.AppGreenlet',
-        'usercostate': 'interp_composable_coroutine.W_UserCoState',
-        '_return_main' : 'interp_coroutine.return_main',
-        'get_stack_depth_limit': 'interp_coroutine.get_stack_depth_limit',
-        'set_stack_depth_limit': 'interp_coroutine.set_stack_depth_limit',
-    }
-
-    def setup_after_space_initialization(self):
-        # post-installing classmethods/staticmethods which
-        # are not yet directly supported
-        from pypy.module._stackless.interp_coroutine import post_install as post_install_coro
-        post_install_coro(self)
-        from pypy.module._stackless.interp_greenlet import post_install as post_install_greenlet
-        post_install_greenlet(self)
-
-        if self.space.config.translation.gc == 'marksweep':
-            from pypy.module._stackless.interp_clonable import post_install as post_install_clonable
-            self.extra_interpdef('clonable', 'interp_clonable.AppClonableCoroutine')
-            self.extra_interpdef('fork',     'interp_clonable.fork')
-            post_install_clonable(self)
diff --git a/pypy/module/_stackless/app_greenlet.py b/pypy/module/_stackless/app_greenlet.py
deleted file mode 100644
--- a/pypy/module/_stackless/app_greenlet.py
+++ /dev/null
@@ -1,5 +0,0 @@
-class GreenletExit(Exception):
-    pass
-
-class GreenletError(Exception):
-    pass
diff --git a/pypy/module/_stackless/interp_clonable.py b/pypy/module/_stackless/interp_clonable.py
deleted file mode 100644
--- a/pypy/module/_stackless/interp_clonable.py
+++ /dev/null
@@ -1,106 +0,0 @@
-from pypy.interpreter.error import OperationError
-from pypy.interpreter.typedef import TypeDef
-from pypy.interpreter.gateway import interp2app
-from pypy.module._stackless.interp_coroutine import AppCoroutine, AppCoState
-from pypy.module._stackless.interp_coroutine import makeStaticMethod
-from pypy.module._stackless.rcoroutine import AbstractThunk
-from pypy.module._stackless.rclonable import InterpClonableMixin
-
-
-class AppClonableCoroutine(AppCoroutine, InterpClonableMixin):
-
-    def newsubctx(self):
-        self.hello_local_pool()
-        AppCoroutine.newsubctx(self)
-        self.goodbye_local_pool()
-
-    def hello(self):
-        self.hello_local_pool()
-        AppCoroutine.hello(self)
-
-    def goodbye(self):
-        AppCoroutine.goodbye(self)
-        self.goodbye_local_pool()
-
-    def descr_method__new__(space, w_subtype):
-        co = space.allocate_instance(AppClonableCoroutine, w_subtype)
-        costate = AppClonableCoroutine._get_state(space)
-        AppClonableCoroutine.__init__(co, space, state=costate)
-        return space.wrap(co)
-
-    def _get_state(space):
-        return space.fromcache(AppClonableCoState)
-    _get_state = staticmethod(_get_state)
-
-    def w_getcurrent(space):
-        return space.wrap(AppClonableCoroutine._get_state(space).current)
-    w_getcurrent = staticmethod(w_getcurrent)
-
-    def w_clone(self):
-        space = self.space
-        costate = self.costate
-        if costate.current is self:
-            raise OperationError(space.w_RuntimeError,
-                                 space.wrap("clone() cannot clone the "
-                                            "current coroutine"
-                                            "; use fork() instead"))
-        copy = AppClonableCoroutine(space, state=costate)
-        copy.subctx = self.clone_into(copy, self.subctx)
-        return space.wrap(copy)
-
-    def descr__reduce__(self, space):
-        raise OperationError(space.w_TypeError,
-                             space.wrap("_stackless.clonable instances are "
-                                        "not picklable"))
-
-
-AppClonableCoroutine.typedef = TypeDef("clonable", AppCoroutine.typedef,
-    __new__    = interp2app(AppClonableCoroutine.descr_method__new__.im_func),
-    getcurrent = interp2app(AppClonableCoroutine.w_getcurrent),
-    clone      = interp2app(AppClonableCoroutine.w_clone),
-    __reduce__ = interp2app(AppClonableCoroutine.descr__reduce__),
-)
-
-class AppClonableCoState(AppCoState):
-    def post_install(self):
-        self.current = self.main = AppClonableCoroutine(self.space, state=self)
-        self.main.subctx.clear_framestack()      # wack
-
-def post_install(module):
-    makeStaticMethod(module, 'clonable', 'getcurrent')
-    space = module.space
-    AppClonableCoroutine._get_state(space).post_install()
-
-# ____________________________________________________________
-
-class ForkThunk(AbstractThunk):
-    def __init__(self, coroutine):
-        self.coroutine = coroutine
-        self.newcoroutine = None
-    def call(self):
-        oldcoro = self.coroutine
-        self.coroutine = None
-        newcoro = AppClonableCoroutine(oldcoro.space, state=oldcoro.costate)
-        newcoro.subctx = oldcoro.clone_into(newcoro, oldcoro.subctx)
-        newcoro.parent = oldcoro
-        self.newcoroutine = newcoro
-
-def fork(space):
-    """Fork, as in the Unix fork(): the call returns twice, and the return
-    value of the call is either the new 'child' coroutine object (if returning
-    into the parent), or None (if returning into the child).  This returns
-    into the parent first, which can switch to the child later.
-    """
-    costate = AppClonableCoroutine._get_state(space)
-    current = costate.current
-    if current is costate.main:
-        raise OperationError(space.w_RuntimeError,
-                             space.wrap("cannot fork() in the main "
-                                        "clonable coroutine"))
-    thunk = ForkThunk(current)
-    coro_fork = AppClonableCoroutine(space, state=costate)
-    coro_fork.bind(thunk)
-    coro_fork.switch()
-    # we resume here twice.  The following would need explanations about
-    # why it returns the correct thing in both the parent and the child...
-    return space.wrap(thunk.newcoroutine)
diff --git a/pypy/module/_stackless/interp_composable_coroutine b/pypy/module/_stackless/interp_composable_coroutine
deleted file mode 100644
--- a/pypy/module/_stackless/interp_composable_coroutine
+++ /dev/null
@@ -1,33 +0,0 @@
-from pypy.interpreter.baseobjspace import Wrappable
-from pypy.interpreter.typedef import TypeDef, interp2app
-from pypy.module._stackless.coroutine import AppCoState, AppCoroutine
-
-
-class W_UserCoState(Wrappable):
-    def __init__(self, space):
-        self.costate = AppCoState(space)
-        self.costate.post_install()
-
-    def descr_method__new__(space, w_subtype):
-        costate = space.allocate_instance(W_UserCoState, w_subtype)
-        W_UserCoState.__init__(costate, space)
-        return space.wrap(costate)
-
-    def w_getcurrent(self):
-        space = self.costate.space
-        return space.wrap(self.costate.current)
-
-    def w_spawn(self, w_subtype=None):
-        space = self.costate.space
-        if space.is_w(w_subtype, space.w_None):
-            w_subtype = space.gettypeobject(AppCoroutine.typedef)
-        co = space.allocate_instance(AppCoroutine, w_subtype)
-        AppCoroutine.__init__(co, space, state=self.costate)
-        return space.wrap(co)
-
-W_UserCoState.typedef = TypeDef("usercostate",
-    __new__ = interp2app(W_UserCoState.descr_method__new__.im_func),
-    __module__ = '_stackless',
-    getcurrent = interp2app(W_UserCoState.w_getcurrent),
-    spawn      = interp2app(W_UserCoState.w_spawn),
-)
diff --git a/pypy/module/_stackless/interp_composable_coroutine.py b/pypy/module/_stackless/interp_composable_coroutine.py
deleted file mode 100644
--- a/pypy/module/_stackless/interp_composable_coroutine.py
+++ /dev/null
@@ -1,34 +0,0 @@
-from pypy.interpreter.baseobjspace import Wrappable
-from pypy.interpreter.typedef import TypeDef, interp2app
-from pypy.module._stackless.interp_coroutine import AppCoState, AppCoroutine
-
-
-class W_UserCoState(Wrappable):
-    def __init__(self, space):
-        self.costate = AppCoState(space)
-        self.costate.post_install()
-
-    def descr_method__new__(space, w_subtype):
-        costate = space.allocate_instance(W_UserCoState, w_subtype)
-        W_UserCoState.__init__(costate, space)
-        return space.wrap(costate)
-
-    def w_getcurrent(self):
-        space = self.costate.space
-        return space.wrap(self.costate.current)
-
-    def w_spawn(self, w_subtype=None):
-        space = self.costate.space
-        if space.is_w(w_subtype, space.w_None):
-            w_subtype = space.gettypeobject(AppCoroutine.typedef)
-        co = space.allocate_instance(AppCoroutine, w_subtype)
-        AppCoroutine.__init__(co, space, state=self.costate)
-        return space.wrap(co)
-
-W_UserCoState.typedef = TypeDef("usercostate",
-    __new__ = interp2app(W_UserCoState.descr_method__new__.im_func),
-    __module__ = '_stackless',
-    getcurrent = interp2app(W_UserCoState.w_getcurrent),
-    spawn      = interp2app(W_UserCoState.w_spawn),
-)
-W_UserCoState.acceptable_as_base_class = False
diff --git a/pypy/module/_stackless/interp_coroutine.py b/pypy/module/_stackless/interp_coroutine.py
deleted file mode 100644
--- a/pypy/module/_stackless/interp_coroutine.py
+++ /dev/null
@@ -1,403 +0,0 @@
-"""
-Coroutine implementation for application level on top
-of the internal coroutines.
-This is an extensible concept. Multiple implementations
-of concurrency can exist together, if they follow the
-basic concept of maintaining their own costate.
-
-There is also some diversification possible by using
-multiple costates for the same type. This leads to
-disjoint switchable sets within the same type.
-
-I'm not so sure to what extent the opposite is possible, too.
-I.e., merging the costate of tasklets and greenlets would
-allow them to be parents of each other. Needs a bit more
-experience to decide where to set the limits.
-"""
-
-from pypy.interpreter.argument import Arguments
-from pypy.interpreter.typedef import GetSetProperty, TypeDef
-from pypy.interpreter.gateway import interp2app, unwrap_spec
-from pypy.interpreter.error import OperationError, operationerrfmt
-
-from pypy.module._stackless.stackless_flags import StacklessFlags
-from pypy.module._stackless.rcoroutine import Coroutine, BaseCoState, AbstractThunk, CoroutineExit
-
-from pypy.module.exceptions.interp_exceptions import W_SystemExit, _new_exception
-
-from pypy.rlib import rstack, jit # for resume points
-from pypy.tool import stdlib_opcode as pythonopcode
-
-class _AppThunk(AbstractThunk):
-
-    def __init__(self, space, costate, w_obj, args):
-        self.space = space
-        self.costate = costate
-        if not space.is_true(space.callable(w_obj)):
-            raise operationerrfmt(
-                space.w_TypeError,
-                "'%s' object is not callable",
-                space.type(w_obj).getname(space))
-        self.w_func = w_obj
-        self.args = args
-
-    def call(self):
-        costate = self.costate
-        w_result = self.space.call_args(self.w_func, self.args)
-        costate.w_tempval = w_result
-
-class _ResumeThunk(AbstractThunk):
-    def __init__(self, space, costate, w_frame):
-        self.space = space
-        self.costate = costate
-        self.w_frame = w_frame
-
-    def call(self):
-        w_result = resume_frame(self.space, self.w_frame)
-        # costate.w_tempval = w_result #XXX?
-
-
-W_CoroutineExit = _new_exception('CoroutineExit', W_SystemExit,
-                        """Coroutine killed manually.""")
-
-# Should be moved to interp_stackless.py if it's ever implemented... Currently
-# used by pypy/lib/stackless.py.
-W_TaskletExit = _new_exception('TaskletExit', W_SystemExit,
-            """Tasklet killed manually.""")
-
-class AppCoroutine(Coroutine): # XXX, StacklessFlags):
-
-    def __init__(self, space, state=None):
-        self.space = space
-        if state is None:
-            state = AppCoroutine._get_state(space)
-        Coroutine.__init__(self, state)
-        self.flags = 0
-        self.newsubctx()
-
-    def newsubctx(self):
-        ec = self.space.getexecutioncontext()
-        self.subctx = ec.Subcontext()
-
-    def descr_method__new__(space, w_subtype):
-        co = space.allocate_instance(AppCoroutine, w_subtype)
-        AppCoroutine.__init__(co, space)
-        return space.wrap(co)
-
-    def _get_state(space):
-        return space.fromcache(AppCoState)
-    _get_state = staticmethod(_get_state)
-
-    def w_bind(self, w_func, __args__):
-        space = self.space
-        if self.frame is not None:
-            raise OperationError(space.w_ValueError, space.wrap(
-                "cannot bind a bound Coroutine"))
-        state = self.costate
-        thunk = _AppThunk(space, state, w_func, __args__)
-        self.bind(thunk)
-
-    def w_switch(self):
-        space = self.space
-        if self.frame is None:
-            raise OperationError(space.w_ValueError, space.wrap(
-                "cannot switch to an unbound Coroutine"))
-        state = self.costate
-        self.switch()
-        w_ret, state.w_tempval = state.w_tempval, space.w_None
-        return w_ret
-
-    def switch(self):
-        space = self.space
-        try:
-            Coroutine.switch(self)
-        except CoroutineExit:
-            raise OperationError(self.costate.w_CoroutineExit, space.w_None)
-
-    def w_finished(self, w_excinfo):
-        pass
-
-    def finish(self, operror=None):
-        space = self.space
-        if isinstance(operror, OperationError):
-            w_exctype = operror.w_type
-            w_excvalue = operror.get_w_value(space)
-            w_exctraceback = operror.get_traceback()
-            w_excinfo = space.newtuple([w_exctype, w_excvalue, w_exctraceback])
-
-            if w_exctype is self.costate.w_CoroutineExit:
-                self.coroutine_exit = True
-        else:
-            w_N = space.w_None
-            w_excinfo = space.newtuple([w_N, w_N, w_N])
-
-        return space.call_method(space.wrap(self),'finished', w_excinfo)
-
-    def hello(self):
-        ec = self.space.getexecutioncontext()
-        self.subctx.enter(ec)
-
-    def goodbye(self):
-        ec = self.space.getexecutioncontext()
-        self.subctx.leave(ec)
-
-    def w_kill(self):
-        self.kill()
-
-    def w_throw(self, w_type, w_value=None, w_traceback=None):
-        space = self.space
-
-        operror = OperationError(w_type, w_value)
-        operror.normalize_exception(space)
-
-        if not space.is_w(w_traceback, space.w_None):
-            from pypy.interpreter import pytraceback
-            tb = space.interpclass_w(w_traceback)
-            if tb is None or not space.is_true(space.isinstance(tb,
-                space.gettypeobject(pytraceback.PyTraceback.typedef))):
-                raise OperationError(space.w_TypeError,
-                      space.wrap("throw: arg 3 must be a traceback or None"))
-            operror.set_traceback(tb)
-
-        self._kill(operror)
-
-    def _userdel(self):
-        if self.get_is_zombie():
-            return
-        self.set_is_zombie(True)
-        self.space.userdel(self.space.wrap(self))
-
-    def w_getcurrent(space):
-        return space.wrap(AppCoroutine._get_state(space).current)
-    w_getcurrent = staticmethod(w_getcurrent)
-
-    def w_getmain(space):
-        return space.wrap(AppCoroutine._get_state(space).main)
-    w_getmain = staticmethod(w_getmain)
-
-    # pickling interface
-    def descr__reduce__(self, space):
-        # this is trying to be simplistic at the moment.
-        # we neither allow to pickle main (which can become a mess
-        # since it has some deep anchestor frames)
-        # nor we allow to pickle the current coroutine.
-        # rule: switch before pickling.
-        # you cannot construct the tree that you are climbing.
-        from pypy.interpreter.mixedmodule import MixedModule
-        w_mod    = space.getbuiltinmodule('_stackless')
-        mod      = space.interp_w(MixedModule, w_mod)
-        w_mod2    = space.getbuiltinmodule('_pickle_support')
-        mod2      = space.interp_w(MixedModule, w_mod2)
-        w_new_inst = mod.get('coroutine')
-        w        = space.wrap
-        nt = space.newtuple
-        ec = self.space.getexecutioncontext()
-
-        if self is self.costate.main:
-            return nt([mod.get('_return_main'), nt([])])
-
-        thunk = self.thunk
-        if isinstance(thunk, _AppThunk):
-            w_args, w_kwds = thunk.args.topacked()
-            w_thunk = nt([thunk.w_func, w_args, w_kwds])
-        else:
-            w_thunk = space.w_None
-
-        tup_base = [
-            ]
-        tup_state = [
-            w(self.flags),
-            self.subctx.getstate(space),
-            w_thunk,
-            w(self.parent),
-            ]
-
-        return nt([w_new_inst, nt(tup_base), nt(tup_state)])
-
-    def descr__setstate__(self, space, w_args):
-        w_flags, w_state, w_thunk, w_parent = space.unpackiterable(w_args,
-                                                        expected_length=4)
-        self.flags = space.int_w(w_flags)
-        if space.is_w(w_parent, space.w_None):
-            w_parent = self.w_getmain(space)
-        self.parent = space.interp_w(AppCoroutine, w_parent)
-        ec = self.space.getexecutioncontext()
-        self.subctx.setstate(space, w_state)
-        if space.is_w(w_thunk, space.w_None):
-            if space.is_w(w_state, space.w_None):
-                self.thunk = None
-            else:
-                self.bind(_ResumeThunk(space, self.costate, self.subctx.topframe))
-        else:
-            w_func, w_args, w_kwds = space.unpackiterable(w_thunk,
-                                                          expected_length=3)
-            args = Arguments.frompacked(space, w_args, w_kwds)
-            self.bind(_AppThunk(space, self.costate, w_func, args))
-
-
-# _mixin_ did not work
-for methname in StacklessFlags.__dict__:
-    meth = getattr(StacklessFlags, methname)
-    if hasattr(meth, 'im_func'):
-        setattr(AppCoroutine, meth.__name__, meth.im_func)
-del meth, methname
-
-def w_get_is_zombie(self, space):
-    return space.wrap(self.get_is_zombie())
-AppCoroutine.w_get_is_zombie = w_get_is_zombie
-
-def w_get_is_alive(self, space):
-    return space.wrap(self.is_alive())
-AppCoroutine.w_get_is_alive = w_get_is_alive
-
-def w_descr__framestack(self, space):
-    assert isinstance(self, AppCoroutine)
-    counter = 0
-    f = self.subctx.topframe
-    while f is not None:
-        counter += 1
-        f = f.f_backref()
-    items = [None] * counter
-    f = self.subctx.topframe
-    while f is not None:
-        counter -= 1
-        assert counter >= 0
-        items[counter] = space.wrap(f)
-        f = f.f_backref()
-    assert counter == 0
-    return space.newtuple(items)
-
-def makeStaticMethod(module, classname, funcname):
-    "NOT_RPYTHON"
-    space = module.space
-    w_klass = space.getattr(space.wrap(module), space.wrap(classname))
-    # HACK HACK HACK
-    # make the typeobject mutable for a while
-    from pypy.objspace.std.typeobject import W_TypeObject
-    assert isinstance(w_klass, W_TypeObject)
-    old_flag = w_klass.flag_heaptype
-    w_klass.flag_heaptype = True
-
-    space.appexec([w_klass, space.wrap(funcname)], """
-        (klass, funcname):
-            func = getattr(klass, funcname)
-            setattr(klass, funcname, staticmethod(func.im_func))
-    """)
-    w_klass.flag_heaptype = old_flag
-
-def post_install(module):
-    makeStaticMethod(module, 'coroutine', 'getcurrent')
-    makeStaticMethod(module, 'coroutine', 'getmain')
-    space = module.space
-    AppCoroutine._get_state(space).post_install()
-
-# space.appexec("""() :
-
-# maybe use __spacebind__ for postprocessing
-
-AppCoroutine.typedef = TypeDef("coroutine",
-    __new__ = interp2app(AppCoroutine.descr_method__new__.im_func),
-    bind = interp2app(AppCoroutine.w_bind),
-    switch = interp2app(AppCoroutine.w_switch),
-    kill = interp2app(AppCoroutine.w_kill),
-    throw = interp2app(AppCoroutine.w_throw),
-    finished = interp2app(AppCoroutine.w_finished),
-    is_alive = GetSetProperty(AppCoroutine.w_get_is_alive),
-    is_zombie = GetSetProperty(AppCoroutine.w_get_is_zombie,
-      doc=AppCoroutine.get_is_zombie.__doc__), #--- this flag is a bit obscure
-      # and not useful (it's totally different from Coroutine.is_zombie(), too)
-      # but lib/stackless.py uses it
-    _framestack = GetSetProperty(w_descr__framestack),
-    getcurrent = interp2app(AppCoroutine.w_getcurrent),
-    getmain = interp2app(AppCoroutine.w_getmain),
-    __reduce__   = interp2app(AppCoroutine.descr__reduce__),
-    __setstate__ = interp2app(AppCoroutine.descr__setstate__),
-    __module__ = '_stackless',
-)
-
-class AppCoState(BaseCoState):
-    def __init__(self, space):
-        BaseCoState.__init__(self)
-        self.w_tempval = space.w_None
-        self.space = space
-
-        # XXX Workaround: for now we need to instantiate these classes
-        # explicitly for translation to work
-        W_CoroutineExit(space)
-        W_TaskletExit(space)
-
-        # Exporting new exception to space
-        self.w_CoroutineExit = space.gettypefor(W_CoroutineExit)
-        space.setitem(
-                      space.exceptions_module.w_dict,
-                      space.new_interned_str('CoroutineExit'),
-                      self.w_CoroutineExit)
-        space.setitem(space.builtin.w_dict,
-                      space.new_interned_str('CoroutineExit'),
-                      self.w_CoroutineExit)
-
-        # Should be moved to interp_stackless.py if it's ever implemented...
-        self.w_TaskletExit = space.gettypefor(W_TaskletExit)
-        space.setitem(
-                      space.exceptions_module.w_dict,
-                      space.new_interned_str('TaskletExit'),
-                      self.w_TaskletExit)
-        space.setitem(space.builtin.w_dict,
-                      space.new_interned_str('TaskletExit'),
-                      self.w_TaskletExit)
-
-    def post_install(self):
-        self.current = self.main = AppCoroutine(self.space, state=self)
-        self.main.subctx.clear_framestack()      # wack
-
-def return_main(space):
-    return AppCoroutine._get_state(space).main
-
-def get_stack_depth_limit(space):
-    return space.wrap(rstack.get_stack_depth_limit())
-
- at unwrap_spec(limit=int)
-def set_stack_depth_limit(space, limit):
-    rstack.set_stack_depth_limit(limit)
-
-
-# ___________________________________________________________________
-# unpickling trampoline
-
-def resume_frame(space, w_frame):
-    from pypy.interpreter.pyframe import PyFrame
-    frame = space.interp_w(PyFrame, w_frame, can_be_None=True)
-    w_result = space.w_None
-    operr = None
-    executioncontext = frame.space.getexecutioncontext()
-    while frame is not None:
-        code = frame.pycode.co_code
-        instr = frame.last_instr
-        opcode = ord(code[instr])
-        map = pythonopcode.opmap
-        call_ops = [map['CALL_FUNCTION'], map['CALL_FUNCTION_KW'], map['CALL_FUNCTION_VAR'],
-                    map['CALL_FUNCTION_VAR_KW'], map['CALL_METHOD']]
-        assert opcode in call_ops
-        instr += 1
-        oparg = ord(code[instr]) | ord(code[instr + 1]) << 8
-        nargs = oparg & 0xff
-        nkwds = (oparg >> 8) & 0xff
-        if nkwds == 0:     # only positional arguments
-            # fast paths leaves things on the stack, pop them
-            if space.config.objspace.opcodes.CALL_METHOD and opcode == map['CALL_METHOD']:
-                frame.dropvalues(nargs + 2)
-            elif opcode == map['CALL_FUNCTION']:
-                frame.dropvalues(nargs + 1)
-
-        # small hack: unlink frame out of the execution context, because
-        # execute_frame will add it there again
-        executioncontext.topframeref = jit.non_virtual_ref(frame.f_backref())
-        frame.last_instr = instr + 1 # continue after the call
-        try:
-            w_result = frame.execute_frame(w_result, operr)
-        except OperationError, operr:
-            pass
-        frame = frame.f_backref()
-    if operr:
-        raise operr
-    return w_result
diff --git a/pypy/module/_stackless/interp_greenlet.py b/pypy/module/_stackless/interp_greenlet.py
deleted file mode 100644
--- a/pypy/module/_stackless/interp_greenlet.py
+++ /dev/null
@@ -1,238 +0,0 @@
-from pypy.interpreter.argument import Arguments
-from pypy.interpreter.typedef import GetSetProperty, TypeDef
-from pypy.interpreter.gateway import interp2app
-from pypy.interpreter.gateway import NoneNotWrapped
-from pypy.interpreter.error import OperationError
-
-from pypy.module._stackless.rcoroutine import Coroutine, BaseCoState
-from pypy.module._stackless.rcoroutine import AbstractThunk, syncstate
-from pypy.module._stackless.interp_coroutine import makeStaticMethod
-
-
-class GreenletThunk(AbstractThunk):
-
-    def __init__(self, greenlet):
-        self.greenlet = greenlet
-
-    def call(self):
-        greenlet = self.greenlet
-        greenlet.active = True
-        try:
-            space = greenlet.space
-            args_w = greenlet.costate.args_w
-            __args__ = Arguments(space, args_w)
-            try:
-                w_run = space.getattr(space.wrap(greenlet), space.wrap('run'))
-                greenlet.w_callable = None
-                w_result = space.call_args(w_run, __args__)
-            except OperationError, operror:
-                if not operror.match(space, greenlet.costate.w_GreenletExit):
-                    raise
-                w_result = operror.get_w_value(space)
-        finally:
-            greenlet.active = False
-        greenlet.costate.args_w = [w_result]
-
-class AppGreenletCoState(BaseCoState):
-    def __init__(self, space):
-        BaseCoState.__init__(self)
-        self.args_w = None
-        self.space = space
-        self.w_GreenletExit  = get(space, "GreenletExit")
-        self.w_GreenletError = get(space, "GreenletError")
-
-    def post_install(self):
-        self.current = self.main = AppGreenlet(self.space, is_main=True)
-
-class AppGreenlet(Coroutine):
-    def __init__(self, space, w_callable=None, is_main=False):
-        Coroutine.__init__(self, self._get_state(space))
-        self.space = space
-        self.w_callable = w_callable
-        self.active = is_main
-        self.subctx = space.getexecutioncontext().Subcontext()
-        if is_main:
-            self.subctx.clear_framestack()      # wack
-        else:
-            self.bind(GreenletThunk(self))
-
-    def descr_method__new__(space, w_subtype, __args__):
-        co = space.allocate_instance(AppGreenlet, w_subtype)
-        AppGreenlet.__init__(co, space)
-        return space.wrap(co)
-
-    def descr_method__init__(self, w_run=NoneNotWrapped,
-                                   w_parent=NoneNotWrapped):
-        if w_run is not None:
-            self.set_run(w_run)
-        if w_parent is not None:
-            self.set_parent(w_parent)
-
-    def _get_state(space):
-        return space.fromcache(AppGreenletCoState)
-    _get_state = staticmethod(_get_state)
-
-    def hello(self):
-        ec = self.space.getexecutioncontext()
-        self.subctx.enter(ec)
-
-    def goodbye(self):
-        ec = self.space.getexecutioncontext()
-        self.subctx.leave(ec)
-
-    def w_getcurrent(space):
-        return space.wrap(AppGreenlet._get_state(space).current)
-    w_getcurrent = staticmethod(w_getcurrent)
-
-    def w_switch(self, args_w):
-        # Find the switch target - it might be a parent greenlet
-        space = self.space
-        costate = self.costate
-        target = self
-        while target.isdead():
-            target = target.parent
-            assert isinstance(target, AppGreenlet)
-        # Switch to it
-        costate.args_w = args_w
-        if target is not costate.current:
-            target.switch()
-        else:
-            # case not handled in Coroutine.switch()
-            syncstate._do_things_to_do()
-        result_w = costate.args_w
-        costate.args_w = None
-        # costate.args_w can be set to None above for throw(), but then
-        # switch() should have raised.  At this point cosstate.args_w != None.
-        assert result_w is not None
-        # Return the result of a switch, packaging it in a tuple if
-        # there is more than one value.
-        if len(result_w) == 1:
-            return result_w[0]
-        return space.newtuple(result_w)
-
-    def w_throw(self, w_type=None, w_value=None, w_traceback=None):
-        space = self.space
-        if space.is_w(w_type, space.w_None):
-            w_type = self.costate.w_GreenletExit
-        # Code copied from RAISE_VARARGS but slightly modified.  Not too nice.
-        operror = OperationError(w_type, w_value)
-        operror.normalize_exception(space)
-        if not space.is_w(w_traceback, space.w_None):
-            from pypy.interpreter import pytraceback
-            tb = space.interpclass_w(w_traceback)
-            if tb is None or not space.is_true(space.isinstance(tb, 
-                space.gettypeobject(pytraceback.PyTraceback.typedef))):
-                raise OperationError(space.w_TypeError,
-                      space.wrap("throw: arg 3 must be a traceback or None"))
-            operror.set_traceback(tb)
-        # Dead greenlet: turn GreenletExit into a regular return
-        if self.isdead() and operror.match(space, self.costate.w_GreenletExit):
-            args_w = [operror.get_w_value(space)]
-        else:
-            syncstate.push_exception(operror)
-            args_w = None
-        return self.w_switch(args_w)
-
-    def _userdel(self):
-        self.space.userdel(self.space.wrap(self))
-
-    def isdead(self):
-        return self.thunk is None and not self.active
-
-    def w_get_is_dead(self, space):
-        return space.newbool(self.isdead())
-
-    def descr__nonzero__(self):
-        return self.space.newbool(self.active)
-
-    def w_get_run(self, space):
-        w_run = self.w_callable
-        if w_run is None:
-            raise OperationError(space.w_AttributeError, space.wrap("run"))
-        return w_run
-
-    def set_run(self, w_run):
-        space = self.space
-        if self.thunk is None:
-            raise OperationError(space.w_AttributeError,
-                                 space.wrap("run cannot be set "
-                                            "after the start of the greenlet"))
-        self.w_callable = w_run
-
-    def w_set_run(self, space, w_run):
-        self.set_run(w_run)
-
-    def w_del_run(self, space):
-        if self.w_callable is None:
-            raise OperationError(space.w_AttributeError, space.wrap("run"))
-        self.w_callable = None
-
-    def w_get_parent(self, space):
-        return space.wrap(self.parent)
-
-    def set_parent(self, w_parent):
-        space = self.space
-        newparent = space.interp_w(AppGreenlet, w_parent)
-        if newparent.costate is not self.costate:
-            raise OperationError(self.costate.w_GreenletError,
-                                 space.wrap("invalid foreign parent"))
-        curr = newparent
-        while curr:
-            if curr is self:
-                raise OperationError(space.w_ValueError,
-                                     space.wrap("cyclic parent chain"))
-            curr = curr.parent
-        self.parent = newparent
-
-    def w_set_parent(self, space, w_parent):
-        self.set_parent(w_parent)
-
-    def w_get_frame(self, space):
-        if not self.active or self.costate.current is self:
-            f = None
-        else:
-            f = self.subctx.topframe
-        return space.wrap(f)
-
-def get(space, name):
-    w_module = space.getbuiltinmodule('_stackless')
-    return space.getattr(w_module, space.wrap(name))
-
-def post_install(module):
-    "NOT_RPYTHON"
-    makeStaticMethod(module, 'greenlet', 'getcurrent')
-    space = module.space
-    state = AppGreenlet._get_state(space)
-    state.post_install()
-    w_greenlet = get(space, 'greenlet')
-    # HACK HACK HACK
-    # make the typeobject mutable for a while
-    from pypy.objspace.std.typeobject import W_TypeObject
-    assert isinstance(w_greenlet, W_TypeObject)
-    old_flag = w_greenlet.flag_heaptype
-    w_greenlet.flag_heaptype = True
-    space.appexec([w_greenlet,
-                   state.w_GreenletExit,
-                   state.w_GreenletError], """
-    (greenlet, exit, error):
-        greenlet.GreenletExit = exit
-        greenlet.error = error
-    """)
-    w_greenlet.flag_heaptype = old_flag
-
-AppGreenlet.typedef = TypeDef("greenlet",
-    __new__ = interp2app(AppGreenlet.descr_method__new__.im_func),
-    __init__ = interp2app(AppGreenlet.descr_method__init__),
-    switch = interp2app(AppGreenlet.w_switch),
-    dead = GetSetProperty(AppGreenlet.w_get_is_dead),
-    run = GetSetProperty(AppGreenlet.w_get_run,
-                         AppGreenlet.w_set_run,
-                         AppGreenlet.w_del_run),
-    parent = GetSetProperty(AppGreenlet.w_get_parent,
-                            AppGreenlet.w_set_parent),
-    getcurrent = interp2app(AppGreenlet.w_getcurrent),
-    throw = interp2app(AppGreenlet.w_throw),
-    gr_frame = GetSetProperty(AppGreenlet.w_get_frame),
-    __nonzero__ = interp2app(AppGreenlet.descr__nonzero__),
-    __module__ = '_stackless',
-)
diff --git a/pypy/module/_stackless/interp_stackless.py b/pypy/module/_stackless/interp_stackless.py
deleted file mode 100644
--- a/pypy/module/_stackless/interp_stackless.py
+++ /dev/null
@@ -1,28 +0,0 @@
-from pypy.interpreter.baseobjspace import Wrappable
-from pypy.interpreter.typedef import TypeDef
-from pypy.interpreter.gateway import interp2app
-import os
-
-
-class tasklet(Wrappable):
-
-    def __init__(self, space):
-        self.space = space
-        self.flags = 0
-        self.state = None
-
-    def descr_method__new__(space, w_subtype):
-        t = space.allocate_instance(tasklet, w_subtype)
-        tasklet.__init__(t, space)
-        return space.wrap(t)
-
-    def w_demo(self):
-        output("42")
-
-tasklet.typedef = TypeDef("tasklet",
-    __new__ = interp2app(tasklet.descr_method__new__.im_func),
-    demo = interp2app(tasklet.w_demo),
-)
-
-def output(stuff):
-    os.write(2, stuff + '\n')
diff --git a/pypy/module/_stackless/rclonable.py b/pypy/module/_stackless/rclonable.py
deleted file mode 100644
--- a/pypy/module/_stackless/rclonable.py
+++ /dev/null
@@ -1,87 +0,0 @@
-from pypy.module._stackless.interp_coroutine import AbstractThunk, Coroutine
-from pypy.rlib.rgc import gc_swap_pool, gc_clone
-from pypy.rlib.objectmodel import we_are_translated
-
-
-class InterpClonableMixin:
-    local_pool = None
-    _mixin_ = True
-
-    def hello_local_pool(self):
-        if we_are_translated():
-            self.saved_pool = gc_swap_pool(self.local_pool)
-
-    def goodbye_local_pool(self):
-        if we_are_translated():
-            self.local_pool = gc_swap_pool(self.saved_pool)
-            self.saved_pool = None
-
-    def clone_into(self, copy, extradata=None):
-        if not we_are_translated():
-            raise NotImplementedError
-        # cannot gc_clone() directly self, because it is not in its own
-        # local_pool.  Moreover, it has a __del__, which cloning doesn't
-        # support properly at the moment.
-        copy.parent = self.parent
-        # the hello/goodbye pair has two purposes: it forces
-        # self.local_pool to be computed even if it was None up to now,
-        # and it puts the 'data' tuple in the correct pool to be cloned.
-        self.hello_local_pool()
-        data = (self.frame, extradata)
-        self.goodbye_local_pool()
-        # clone!
-        data, copy.local_pool = gc_clone(data, self.local_pool)
-        copy.frame, extradata = data
-        copy.thunk = self.thunk # in case we haven't switched to self yet
-        return extradata
-
-
-class InterpClonableCoroutine(Coroutine, InterpClonableMixin):
-
-    def hello(self):
-        self.hello_local_pool()
-
-    def goodbye(self):
-        self.goodbye_local_pool()
-
-    def clone(self):
-        # hack, this is overridden in AppClonableCoroutine
-        if self.getcurrent() is self:
-            raise RuntimeError("clone() cannot clone the current coroutine; "
-                               "use fork() instead")
-        copy = InterpClonableCoroutine(self.costate)
-        self.clone_into(copy)
-        return copy
-
-
-class ForkThunk(AbstractThunk):
-    def __init__(self, coroutine):
-        self.coroutine = coroutine
-        self.newcoroutine = None
-    def call(self):
-        oldcoro = self.coroutine
-        self.coroutine = None
-        newcoro = oldcoro.clone()
-        newcoro.parent = oldcoro
-        self.newcoroutine = newcoro
-
-def fork():
-    """Fork, as in the Unix fork(): the call returns twice, and the return
-    value of the call is either the new 'child' coroutine object (if returning
-    into the parent), or None (if returning into the child).  This returns
-    into the parent first, which can switch to the child later.
-    """
-    current = InterpClonableCoroutine.getcurrent()
-    if not isinstance(current, InterpClonableCoroutine):
-        raise RuntimeError("fork() in a non-clonable coroutine")
-    thunk = ForkThunk(current)
-    coro_fork = InterpClonableCoroutine()
-    coro_fork.bind(thunk)
-    coro_fork.switch()
-    # we resume here twice.  The following would need explanations about
-    # why it returns the correct thing in both the parent and the child...
-    return thunk.newcoroutine
-
-##    from pypy.rpython.lltypesystem import lltype, lloperation
-##    lloperation.llop.debug_view(lltype.Void, current, thunk,
-##        lloperation.llop.gc_x_size_header(lltype.Signed))
diff --git a/pypy/module/_stackless/rcoroutine.py b/pypy/module/_stackless/rcoroutine.py
deleted file mode 100644
--- a/pypy/module/_stackless/rcoroutine.py
+++ /dev/null
@@ -1,10 +0,0 @@
-from pypy.rlib.rcoroutine import make_coroutine_classes
-from pypy.interpreter.baseobjspace import Wrappable
-
-d = make_coroutine_classes(Wrappable)
-
-Coroutine = d['Coroutine']
-BaseCoState = d['BaseCoState']
-AbstractThunk = d['AbstractThunk']
-syncstate = d['syncstate']
-CoroutineExit = d['CoroutineExit']
diff --git a/pypy/module/_stackless/stackless_flags.py b/pypy/module/_stackless/stackless_flags.py
deleted file mode 100644
--- a/pypy/module/_stackless/stackless_flags.py
+++ /dev/null
@@ -1,201 +0,0 @@
-"""
-basic definitions for tasklet flags.
-For simplicity and compatibility,
-they are defined the same for coroutines,
-even if they are not used.
-
-taken from tasklet_structs.h
-----------------------------
-
-/***************************************************************************
-
-    Tasklet Flag Definition
-    -----------------------
-
-    blocked:        The tasklet is either waiting in a channel for
-                    writing (1) or reading (-1) or not blocked (0).
-                    Maintained by the channel logic. Do not change.
-
-    atomic:         If true, schedulers will never switch. Driven by
-                    the code object or dynamically, see below.
-
-    ignore_nesting: Allows auto-scheduling, even if nesting_level
-                    is not zero.
-
-    autoschedule:   The tasklet likes to be auto-scheduled. User driven.
-
-    block_trap:     Debugging aid. Whenever the tasklet would be
-                    blocked by a channel, an exception is raised.
-
-    is_zombie:      This tasklet is almost dead, its deallocation has
-                    started. The tasklet *must* die at some time, or the
-                    process can never end.
-
-    pending_irq:    If set, an interrupt was issued during an atomic
-                    operation, and should be handled when possible.
-
-
-    Policy for atomic/autoschedule and switching:
-    ---------------------------------------------
-    A tasklet switch can always be done explicitly by calling schedule().
-    Atomic and schedule are concerned with automatic features.
-
-    atomic  autoschedule
-
-        1       any     Neither a scheduler nor a watchdog will
-                        try to switch this tasklet.
-
-        0       0       The tasklet can be stopped on desire, or it
-                        can be killed by an exception.
-
-        0       1       Like above, plus auto-scheduling is enabled.
-
-    Default settings:
-    -----------------
-    All flags are zero by default.
-
- ***************************************************************************/
-
-typedef struct _tasklet_flags {
-        int blocked: 2;
-        unsigned int atomic: 1;
-        unsigned int ignore_nesting: 1;
-        unsigned int autoschedule: 1;
-        unsigned int block_trap: 1;
-        unsigned int is_zombie: 1;
-        unsigned int pending_irq: 1;
-} PyTaskletFlagStruc;
-"""
-
-from pypy.rlib.rarithmetic import LONG_BIT, intmask
-
-class BitSetDef(object):
-    __slots__ = "_names __dict__ _attrname".split()
-
-    def __init__(self, _attrname):
-        self._names = []
-        self._attrname = _attrname
-        
-    def __setattr__(self, key, value):
-        if key not in self.__slots__:
-            assert key not in self.__dict__
-            self._names.append(key)
-        object.__setattr__(self, key, value)
-
-    def __iter__(self):
-        return self._enum_objects()
-    
-    def _enum_objects(self):
-        for name in self._names:
-            yield name, getattr(self, name)
-
-# negative values are user-writable
-flags = BitSetDef("flags")
-flags.blocked           =   2, """writing (1) or reading (-1) or not blocked (0)"""
-flags.atomic            =  -1, """If true, schedulers will never switch"""
-flags.ignore_nesting    =  -1, """allow auto-scheduling in nested interpreters"""
-flags.autoschedule      =  -1, """enable auto-scheduling"""
-flags.block_trap        =  -1, """raise an exception instead of blocking"""
-flags.is_zombie         =   1, """__del__ is in progress"""
-flags.pending_irq       =   1, """an interrupt occured while being atomic"""
-
-def make_get_bits(name, bits, shift):
-    """ return a bool for single bits, signed int otherwise """
-    signmask = 1 << (bits - 1 + shift)
-    lshift = bits + shift
-    rshift = bits
-    if bits == 1:
-        return "bool(%s & 0x%x)" % (name, signmask)
-    else:
-        return "intmask(%s << (LONG_BIT-%d)) >> (LONG_BIT-%d)" % (name, lshift, rshift)
-
-def make_set_bits(name, bits, shift):
-    datamask = int('1' * bits, 2)
-    clearmask = datamask << shift
-    return "%s & ~0x%x | (value & 0x%x) << %d" % (name, clearmask, datamask, shift)
-
-def gen_code():
-    from cStringIO import StringIO
-    f = StringIO()
-    print >> f, "class StacklessFlags(object):"
-    print >> f, "    _mixin_ = True"
-    shift = 0
-    field = "self.%s" % flags._attrname
-    for name, (bits, doc) in flags:
-        write, bits = bits < 0, abs(bits)
-        print >> f
-        print >> f, '    def get_%s(self):' % name
-        print >> f, '        """%s"""' % doc
-        print >> f, '        return %s' % make_get_bits(field, bits, shift)
-        print >> f, '    def set_%s(self, value):' % name
-        print >> f, '        """%s"""' % doc
-        print >> f, '        %s = %s' % (field, make_set_bits(field, bits, shift))
-        print >> f, '    set_%s._public = %s' % (name, write)
-        shift += bits
-    return f.getvalue()
-
-# BEGIN generated code
-class StacklessFlags(object):
-    _mixin_ = True
-
-    def get_blocked(self):
-        """writing (1) or reading (-1) or not blocked (0)"""
-        return intmask(self.flags << (LONG_BIT-2)) >> (LONG_BIT-2)
-    def set_blocked(self, value):
-        """writing (1) or reading (-1) or not blocked (0)"""
-        self.flags = self.flags & ~0x3 | (value & 0x3) << 0
-    set_blocked._public = False
-
-    def get_atomic(self):
-        """If true, schedulers will never switch"""
-        return bool(self.flags & 0x4)
-    def set_atomic(self, value):
-        """If true, schedulers will never switch"""
-        self.flags = self.flags & ~0x4 | (value & 0x1) << 2
-    set_atomic._public = True
-
-    def get_ignore_nesting(self):
-        """allow auto-scheduling in nested interpreters"""
-        return bool(self.flags & 0x8)
-    def set_ignore_nesting(self, value):
-        """allow auto-scheduling in nested interpreters"""
-        self.flags = self.flags & ~0x8 | (value & 0x1) << 3
-    set_ignore_nesting._public = True
-
-    def get_autoschedule(self):
-        """enable auto-scheduling"""
-        return bool(self.flags & 0x10)
-    def set_autoschedule(self, value):
-        """enable auto-scheduling"""
-        self.flags = self.flags & ~0x10 | (value & 0x1) << 4
-    set_autoschedule._public = True
-
-    def get_block_trap(self):
-        """raise an exception instead of blocking"""
-        return bool(self.flags & 0x20)
-    def set_block_trap(self, value):
-        """raise an exception instead of blocking"""
-        self.flags = self.flags & ~0x20 | (value & 0x1) << 5
-    set_block_trap._public = True
-
-    def get_is_zombie(self):
-        """__del__ is in progress"""
-        return bool(self.flags & 0x40)
-    def set_is_zombie(self, value):
-        """__del__ is in progress"""
-        self.flags = self.flags & ~0x40 | (value & 0x1) << 6
-    set_is_zombie._public = False
-
-    def get_pending_irq(self):
-        """an interrupt occured while being atomic"""
-        return bool(self.flags & 0x80)
-    def set_pending_irq(self, value):
-        """an interrupt occured while being atomic"""
-        self.flags = self.flags & ~0x80 | (value & 0x1) << 7
-    set_pending_irq._public = False
-
-# END generated code
-
-if __name__ == '__main__':
-    # paste this into the file
-    print gen_code()
diff --git a/pypy/module/_stackless/test/__init__.py b/pypy/module/_stackless/test/__init__.py
deleted file mode 100644
--- a/pypy/module/_stackless/test/__init__.py
+++ /dev/null
@@ -1,1 +0,0 @@
-#
\ No newline at end of file
diff --git a/pypy/module/_stackless/test/conftest.py b/pypy/module/_stackless/test/conftest.py
deleted file mode 100644
--- a/pypy/module/_stackless/test/conftest.py
+++ /dev/null
@@ -1,8 +0,0 @@
-import sys
-import py.test
-
-def pytest_runtest_setup(item):
-    py.test.importorskip('greenlet')
-    if sys.platform == 'win32':
-        py.test.skip("stackless tests segfault on Windows")
-
diff --git a/pypy/module/_stackless/test/slp_test_pickle.py b/pypy/module/_stackless/test/slp_test_pickle.py
deleted file mode 100644
--- a/pypy/module/_stackless/test/slp_test_pickle.py
+++ /dev/null
@@ -1,35 +0,0 @@
-from pypy.conftest import gettestobjspace
-
-# app-level testing of coroutine pickling
-
-class AppTest_Pickle:
-
-    def setup_class(cls):
-        space = gettestobjspace(usemodules=('_stackless',))
-        cls.space = space
-
-    def test_simple_ish(self):
-
-        output = []
-        import _stackless
-        def f(coro, n, x):
-            if n == 0:
-                coro.switch()
-                return
-            f(coro, n-1, 2*x)
-            output.append(x)
-
-        def example():
-            main_coro = _stackless.coroutine.getcurrent()
-            sub_coro = _stackless.coroutine()
-            sub_coro.bind(f, main_coro, 5, 1)
-            sub_coro.switch()
-
-            import pickle
-            pckl = pickle.dumps(sub_coro)
-            new_coro = pickle.loads(pckl)
-
-            new_coro.switch()
-
-        example()
-        assert output == [16, 8, 4, 2, 1]
diff --git a/pypy/module/_stackless/test/test_choicepoint.py b/pypy/module/_stackless/test/test_choicepoint.py
deleted file mode 100644
--- a/pypy/module/_stackless/test/test_choicepoint.py
+++ /dev/null
@@ -1,85 +0,0 @@
-import py; py.test.skip("clonable coroutines not really maintained any more")
-
-from pypy.rlib.rcoroutine import AbstractThunk
-from pypy.module._stackless.rclonable import InterpClonableCoroutine as ClonableCoroutine
-
-class ChoicePointHolder(object):
-    def __init__(self):
-        self.choicepoints = []
-        self.clone_me = False
-        self.answer = 0
-        self.solutions_count = 0
-
-    def next_choice(self):
-        return self.choicepoints.pop()
-
-    def add(self, choice, answer=0):
-        self.choicepoints.append((choice, answer))
-
-    def more_choices(self):
-        return bool(self.choicepoints)
-
-    def choice(self):
-        #os.write(1, "choice\n")
-        self.clone_me = True
-        self.g_main.switch()
-        #os.write(1, "answer: %d\n" % (self.answer,))
-        return self.answer
-
-    def fail(self):
-        self.g_main.switch()
-        assert False
-
-choicepoints = ChoicePointHolder()
-
-# ____________________________________________________________
-
-class SearchTask(AbstractThunk):
-    def call(self):
-        path = []
-        for i in range(10):
-            res = choicepoints.choice()
-            assert len(path) == i
-            path.append(res)
-            #os.write(1, "{%x} trying: %s\n" % (id(path), path))
-            if i == 3:
-                import gc; gc.collect()
-        #os.write(1, "{%x} found a solution: %s\n" % (id(path), path))
-        choicepoints.solutions_count += 1
-
-# ____________________________________________________________
-
-
-class SearchAllTask(AbstractThunk):
-    def call(self):
-        search_coro = ClonableCoroutine()
-        search_coro.bind(SearchTask())
-        choicepoints.add(search_coro)
-
-        #os.write(1, "starting\n")
-        while choicepoints.more_choices():
-            searcher, nextvalue = choicepoints.next_choice()
-            choicepoints.clone_me = False
-            choicepoints.answer = nextvalue
-            #os.write(1, '<<< {%x} %d\n' % (id(searcher), nextvalue))
-            searcher.switch()
-            #os.write(1, '>>> %d\n' % (choicepoints.clone_me,))
-            if choicepoints.clone_me:
-                searcher2 = searcher.clone()
-                #os.write(1, 'searcher = {%x}, searcher2 = {%x}\n' % (
-                #    id(searcher), id(searcher2)))
-                choicepoints.add(searcher, 5)
-                choicepoints.add(searcher2, 4)
-
-def entry_point():
-    choicepoints.g_main = ClonableCoroutine()
-    choicepoints.g_main.bind(SearchAllTask())
-    choicepoints.g_main.switch()
-    return choicepoints.solutions_count
-
-def test_choicepoint():
-    from pypy.translator.c.test import test_newgc
-    tester = test_newgc.TestUsingStacklessFramework()
-    fn = tester.getcompiled(entry_point)
-    res = fn()
-    assert res == 2 ** 10
diff --git a/pypy/module/_stackless/test/test_clonable.py b/pypy/module/_stackless/test/test_clonable.py
deleted file mode 100644
--- a/pypy/module/_stackless/test/test_clonable.py
+++ /dev/null
@@ -1,187 +0,0 @@
-import py; py.test.skip("clonable coroutines not really maintained any more")
-
-from pypy.conftest import gettestobjspace, option
-import py, sys
-
-# app-level testing of coroutine cloning
-
-class AppTestClonable:
-
-    def setup_class(cls):
-        if not option.runappdirect:
-            py.test.skip('pure appdirect test (run with -A)')
-        cls.space = space = gettestobjspace(usemodules=('_stackless',))
-        if not space.is_true(space.appexec([], """():
-            import _stackless
-            return hasattr(_stackless, 'clonable')
-        """)):
-            py.test.skip('no _stackless.clonable')
-
-
-    def test_solver(self):
-        import _stackless
-
-        class Fail(Exception):
-            pass
-
-        class Success(Exception):
-            pass
-
-        def first_solution(func):
-            global next_answer
-            co = _stackless.clonable()
-            co.bind(func)
-            pending = [(co, None)]
-            while pending:
-                co, next_answer = pending.pop()
-                try:
-                    co.switch()
-                except Fail:
-                    pass
-                except Success, e:
-                    return e.args[0]
-                else:
-                    # zero_or_one() called, clone the coroutine
-                    co2 = co.clone()
-                    pending.append((co2, 1))
-                    pending.append((co, 0))
-            raise Fail("no solution")
-
-        pending = []
-        main = _stackless.clonable.getcurrent()
-
-        def zero_or_one():
-            main.switch()
-            return next_answer
-
-        # ____________________________________________________________
-
-        invalid_prefixes = {
-            (0, 0): True,
-            (0, 1, 0): True,
-            (0, 1, 1): True,
-            (1, 0): True,
-            (1, 1, 0, 0): True,
-            }
-
-        def example():
-            test = []
-            for n in range(5):
-                test.append(zero_or_one())
-                if tuple(test) in invalid_prefixes:
-                    raise Fail
-            raise Success(test)
-
-        res = first_solution(example)
-        assert res == [1, 1, 0, 1, 0]
-
-
-    def test_myself_may_not_be_me_any_more(self):
-        import gc
-        from _stackless import clonable
-
-        counter = [0]
-
-        def runner():
-            while 1:
-                assert clonable.getcurrent() is coro
-                counter[0] += 1
-                main.switch()
-
-        main = clonable.getcurrent()
-        coro = clonable()
-        coro.bind(runner)
-
-        coro.switch()
-        assert counter == [1]
-
-        assert clonable.getcurrent() is main
-        coro1 = coro.clone()
-        assert counter == [1]
-        assert clonable.getcurrent() is main
-        coro.switch()
-        assert counter == [2]
-        coro.switch()
-        assert counter == [3]
-        assert clonable.getcurrent() is main
-        del coro1
-        gc.collect()
-        #print "collected!"
-        assert clonable.getcurrent() is main
-        assert counter == [3]
-        coro.switch()
-        assert clonable.getcurrent() is main
-        assert counter == [4]
-
-
-    def test_fork(self):
-        import _stackless
-
-        class Fail(Exception):
-            pass
-
-        class Success(Exception):
-            pass
-
-        def first_solution(func):
-            global next_answer
-            co = _stackless.clonable()
-            co.bind(func)
-            try:
-                co.switch()
-            except Success, e:
-                return e.args[0]
-
-        def zero_or_one():
-            sub = _stackless.fork()
-            if sub is not None:
-                # in the parent: run the child first
-                try:
-                    sub.switch()
-                except Fail:
-                    pass
-                # then proceed with answer '1'
-                return 1
-            else:
-                # in the child: answer '0'
-                return 0
-
-        # ____________________________________________________________
-
-        invalid_prefixes = {
-            (0, 0): True,
-            (0, 1, 0): True,
-            (0, 1, 1): True,
-            (1, 0): True,
-            (1, 1, 0, 0): True,
-            }
-
-        def example():
-            test = []
-            for n in range(5):
-                test.append(zero_or_one())
-                if tuple(test) in invalid_prefixes:
-                    raise Fail
-            raise Success(test)
-
-        res = first_solution(example)
-        assert res == [1, 1, 0, 1, 0]
-
-    def test_clone_before_start(self):
-        """Tests that a clonable coroutine can be
-        cloned before it is started
-        (this used to fail with a segmentation fault)
-        """
-        import _stackless
-
-        counter = [0]
-        def simple_coro():
-            print "hello"
-            counter[0] += 1
-
-        s = _stackless.clonable()
-        s.bind(simple_coro)
-        t = s.clone()
-        s.switch()
-        t.switch()
-        assert counter[0] == 2
diff --git a/pypy/module/_stackless/test/test_composable_coroutine.py b/pypy/module/_stackless/test/test_composable_coroutine.py
deleted file mode 100644
--- a/pypy/module/_stackless/test/test_composable_coroutine.py
+++ /dev/null
@@ -1,133 +0,0 @@
-""" a faith is the connection between past and future that divides the
-    application into switch-compatible chunks.
-    -- stakkars
-"""
-from pypy.conftest import gettestobjspace
-from py.test import skip
-
-class AppTest_ComposableCoroutine:
-
-    def setup_class(cls):
-        space = gettestobjspace(usemodules=('_stackless',))
-        cls.space = space
-
-        cls.w_generator_ = space.appexec([], """():
-            import _stackless
-
-            generators_costate = _stackless.usercostate()
-            main = generators_costate.getcurrent()
-
-            class generator_iterator(_stackless.coroutine):
-
-                def __iter__(self):
-                    return self
-
-                def next(self):
-                    if self.gi_answer is not None:
-                        raise ValueError('stackless-generator'
-                                         ' already executing')
-                    self.gi_answer = []
-                    self.gi_caller = generators_costate.getcurrent()
-                    self.switch()
-                    answer = self.gi_answer
-                    self.gi_answer = None
-                    if answer:
-                        return answer[0]
-                    else:
-                        raise StopIteration
-
-            def generator(f):
-                def myfunc(*args, **kwds):
-                    g = generators_costate.spawn(generator_iterator)
-                    g.gi_answer = None
-                    g.bind(f, *args, **kwds)
-                    return g
-                return myfunc
-
-            def Yield(value):
-                g = generators_costate.getcurrent()
-                if g is main:
-                    raise ValueError('Yield() outside any stackless-generator')
-                assert isinstance(g, generator_iterator)
-                assert g.gi_answer == []
-                g.gi_answer.append(value)
-                g.gi_caller.switch()
-
-            generator.Yield = Yield
-            generator._costate = generators_costate
-            return (generator,)
-        """)
-
-    def test_simple_costate(self):
-        import _stackless
-        costate = _stackless.usercostate()
-        main = costate.getcurrent()
-
-        result = []
-        def f():
-            result.append(costate.getcurrent())
-        co = costate.spawn()
-        co.bind(f)
-        co.switch()
-        assert result == [co]
-
-    def test_generator(self):
-        generator, = self.generator_
-
-        def squares(n):
-            for i in range(n):
-                generator.Yield(i*i)
-        squares = generator(squares)
-
-        lst1 = [i*i for i in range(10)]
-        for got in squares(10):
-            expected = lst1.pop(0)
-            assert got == expected
-        assert lst1 == []
-
-    def test_multiple_costates(self):
-        """Test that two independent costates mix transparently:
-
-        - compute_costate, used for a coroutine that fills a list with
-                           some more items each time it is switched to
-
-        - generators_costate, used interally by self.generator (see above)
-        """
-
-        import _stackless
-        generator, = self.generator_
-
-        # you can see how it fails if we don't have two different costates
-        # by setting compute_costate to generator._costate instead
-        compute_costate = _stackless.usercostate()
-        compute_main = compute_costate.getcurrent()
-        lst = []
-
-        def filler():     # -> 0, 1, 2, 100, 101, 102, 200, 201, 202, 300 ...
-            for k in range(5):
-                for j in range(3):
-                    lst.append(100 * k + j)
-                compute_main.switch()
-
-        filler_co = compute_costate.spawn()
-        filler_co.bind(filler)
-
-        def grab_next_value():
-            while not lst:
-                #print 'filling more...'
-                filler_co.switch()
-                #print 'now lst =', lst
-            #print 'grabbing', lst[0]
-            return lst.pop(0)
-
-        def squares(n):
-            for i in range(n):
-                #print 'square:', i
-                generator.Yield(i*grab_next_value())
-        squares = generator(squares)
-
-        lst1 = [0, 1, 4,  300, 404, 510,  1200, 1407, 1616,  2700]
-        for got in squares(10):
-            expected = lst1.pop(0)
-            assert got == expected
-        assert lst1 == []
diff --git a/pypy/module/_stackless/test/test_coroutine.py b/pypy/module/_stackless/test/test_coroutine.py
deleted file mode 100644
--- a/pypy/module/_stackless/test/test_coroutine.py
+++ /dev/null
@@ -1,168 +0,0 @@
-from pypy.conftest import gettestobjspace, option
-from py.test import skip
-
-
-class AppTest_Coroutine:
-
-    def setup_class(cls):
-        space = gettestobjspace(usemodules=('_stackless',))
-        cls.space = space
-
-    def test_raise_propagate(self):
-        import _stackless as stackless
-        co = stackless.coroutine()
-        def f():
-            return 1/0
-        co.bind(f)
-        try:
-            co.switch()
-        except ZeroDivisionError:
-            pass
-        else:
-            raise AssertionError("exception not propagated")
-
-    def test_strange_test(self):
-        from _stackless import coroutine
-        def f():
-            print "in new coro"
-            return 42
-        def create():
-            b = coroutine()
-            b.bind(f)
-            print "bound"
-            b.switch()
-            print "switched"
-            return b
-        a = coroutine()
-        a.bind(create)
-        b = a.switch()
-        # now b.parent = a
-        def nothing():
-            pass
-        a.bind(nothing)
-        def kill():
-            # this sets a.parent = b
-            a.kill()
-        b.bind(kill)
-        b.switch()
-
-    def test_kill(self):
-        import _stackless as stackless
-        co = stackless.coroutine()
-        def f():
-            pass
-        co.bind(f)
-        assert co.is_alive
-        co.kill()
-        assert not co.is_alive
-
-    def test_kill_running(self):
-        coroutineexit = []
-        import _stackless as stackless
-        main = stackless.coroutine.getcurrent()
-        result = []
-        co = stackless.coroutine()
-        def f():
-            x = 2
-            try:
-                result.append(1)
-                main.switch()
-                x = 3
-            except CoroutineExit:
-                coroutineexit.append(True)
-                raise
-            finally:
-                result.append(x)
-            result.append(4)
-        co.bind(f)
-        assert co.is_alive
-        co.switch()
-        assert co.is_alive
-        assert result == [1]
-        co.kill()
-        assert not co.is_alive
-        assert result == [1, 2]
-        assert coroutineexit == [True]
-
-    def test_bogus_bind(self):
-        import _stackless as stackless
-        co = stackless.coroutine()
-        def f():
-            pass
-        co.bind(f)
-        raises(ValueError, co.bind, f)
-
-    def test__framestack(self):
-        import _stackless as stackless
-        main = stackless.coroutine.getmain()
-        co = stackless.coroutine()
-        def g():
-            return co._framestack
-        def f():
-            return g()
-
-        co.bind(f)
-        stack = co.switch()
-        assert stack == () # running corountine, _framestack is empty
-
-        co = stackless.coroutine()
-        def g():
-            return main.switch()
-        def f():
-            return g()
-
-        co.bind(f)
-        co.switch()
-        stack = co._framestack
-        assert len(stack) == 2
-        assert stack[0].f_code is f.func_code
-        assert stack[1].f_code is g.func_code
-
-        co = stackless.coroutine()
-
-
-
-class AppTestDirect:
-    def setup_class(cls):
-        if not option.runappdirect:
-            skip('pure appdirect test (run with -A)')
-        cls.space = gettestobjspace(usemodules=('_stackless',))
-
-    def test_stack_depth_limit(self):
-        import sys
-        import _stackless as stackless
-        st = stackless.get_stack_depth_limit()
-        try:
-            stackless.set_stack_depth_limit(1)
-            assert stackless.get_stack_depth_limit() == 1
-            try:
-                co = stackless.coroutine()
-                def f():
-                    pass
-                co.bind(f)
-                co.switch()
-            except RuntimeError:
-                pass
-        finally:
-            stackless.set_stack_depth_limit(st)
-
-class TestRandomThings:
-    def setup_class(cls):
-        cls.space = gettestobjspace(usemodules=('_stackless',))
-
-    def test___del___handling(self):
-        space = self.space
-        w_l = space.newlist([])
-        coro = space.appexec([w_l], """(l):
-            from _stackless import coroutine
-            class MyCoroutine(coroutine):
-                def __del__(self):
-                    l.append(self.is_zombie)
-            return MyCoroutine()
-        """)
-        coro.__del__()
-        space.user_del_action.perform(space.getexecutioncontext(), None)
-        coro._kill_finally()
-        assert space.len_w(w_l) == 1
-        res = space.is_true(space.getitem(w_l, space.wrap(0)))
-        assert res
diff --git a/pypy/module/_stackless/test/test_greenlet.py b/pypy/module/_stackless/test/test_greenlet.py
deleted file mode 100644
--- a/pypy/module/_stackless/test/test_greenlet.py
+++ /dev/null
@@ -1,643 +0,0 @@
-from pypy.conftest import gettestobjspace, skip_on_missing_buildoption
-
-class AppTest_Greenlet:
-
-    def setup_class(cls):
-        space = gettestobjspace(usemodules=('_stackless',))
-        cls.space = space
-
-    def test_very_simple(self):
-        from _stackless import greenlet
-        lst = []
-        def f(x):
-            lst.append(x)
-            return x + 10
-        g = greenlet(f)
-        assert not g
-        res = g.switch(20)
-        assert res == 30
-        assert lst == [20]
-        assert g.dead
-        assert not g
-
-    def test_switch_back_to_main(self):
-        from _stackless import greenlet
-        lst = []
-        main = greenlet.getcurrent()
-        def f(x):
-            lst.append(x)
-            x = main.switch(x + 10)
-            return 40 + x 
-        g = greenlet(f)
-        res = g.switch(20)
-        assert res == 30
-        assert lst == [20]
-        assert not g.dead
-        res = g.switch(2)
-        assert res == 42
-        assert g.dead
-
-    def test_simple(self):
-        from _stackless import greenlet
-        lst = []
-        gs = []
-        def f():
-            lst.append(1)
-            greenlet.getcurrent().parent.switch()
-            lst.append(3)
-        g = greenlet(f)
-        lst.append(0)
-        g.switch()
-        lst.append(2)
-        g.switch()
-        lst.append(4)
-        assert lst == range(5)
-
-    def test_exception_simple(self):
-        from _stackless import greenlet
-        def f():
-            raise ValueError
-        g1 = greenlet(f)
-        raises(ValueError, g1.switch)
-
-    def test_exception_propagate(self):
-        from _stackless import greenlet
-        def f():
-            raise ValueError
-        def g():
-            return g1.switch()
-        g1 = greenlet(f)
-        g2 = greenlet(g)
-        raises(ValueError, g1.switch)
-        g1 = greenlet(f)
-        raises(ValueError, g2.switch)
-
-
-    def test_exc_info_save_restore(self):
-        from _stackless import greenlet
-        import sys
-        def f():
-            try:
-                raise ValueError('fun')
-            except:
-                exc_info = sys.exc_info()
-                greenlet(h).switch()
-                assert exc_info == sys.exc_info()
-
-        def h():
-            assert sys.exc_info() == (None, None, None)
-
-        greenlet(f).switch()
-
-    def test_exception(self):
-        from _stackless import greenlet
-        import sys
-        def fmain(seen):
-            try:
-                greenlet.getcurrent().parent.switch()
-            except:
-                seen.append(sys.exc_info()[0])
-                raise
-            raise ValueError
-        seen = []
-        g1 = greenlet(fmain)
-        g2 = greenlet(fmain)
-        g1.switch(seen)
-        g2.switch(seen)
-        raises(TypeError, "g2.parent = 1")
-        g2.parent = g1
-        assert seen == []
-        raises(ValueError, g2.switch)
-        assert seen == [ValueError]
-        g2.switch()
-        assert seen == [ValueError]
-
-    def test_send_exception(self):
-        from _stackless import greenlet
-        import sys
-        def send_exception(g, exc):
-            # note: send_exception(g, exc)  can be now done with  g.throw(exc).
-            # the purpose of this test is to explicitely check the propagation rules.
-            def crasher(exc):
-                raise exc
-            g1 = greenlet(crasher)
-            g1.parent = g
-            g1.switch(exc)
-        def fmain(seen):
-            try:
-                greenlet.getcurrent().parent.switch()
-            except:
-                seen.append(sys.exc_info()[0])
-                raise
-            raise ValueError
-
-        seen = []
-        g1 = greenlet(fmain)
-        g1.switch(seen)
-        raises(KeyError, "send_exception(g1, KeyError)")
-        assert seen == [KeyError]
-        seen = []
-        g1 = greenlet(fmain)
-        g1.switch(seen)
-        raises(KeyError, "g1.throw(KeyError)")
-        assert seen == [KeyError]
-        assert g1.dead
-
-    def test_frame(self):
-        from _stackless import greenlet
-        import sys
-        def f1():
-            f = sys._getframe(0)
-            assert f.f_back is None
-            greenlet.getcurrent().parent.switch(f)
-            return "meaning of life"
-        g = greenlet(f1)
-        frame = g.switch()
-        assert frame is g.gr_frame
-        assert g
-        next = g.switch()
-        assert not g
-        assert next == "meaning of life"
-        assert g.gr_frame is None
-
-    def test_mixing_greenlet_coroutine(self):
-        from _stackless import greenlet, coroutine
-        lst = []
-        def f():
-            lst.append(1)
-            greenlet.getcurrent().parent.switch()
-            lst.append(3)
-        def make_h(c):
-            def h():
-                g = greenlet(f)
-                lst.append(0)
-                g.switch()
-                c.switch()
-                lst.append(2)
-                g.switch()
-                c.switch()
-                lst.append(4)
-                c.switch()
-            return h
-        c1 = coroutine.getcurrent()
-        c2 = coroutine()
-        c3 = coroutine()
-        c2.bind(make_h(c3))
-        c3.bind(make_h(c2))
-        c2.switch()
-        assert lst == [0, 1, 0, 1, 2, 3, 2, 3, 4, 4]
-
-    def test_dealloc(self):
-        skip("not working yet")
-        from _stackless import greenlet
-        import sys
-        def fmain(seen):
-            try:
-                greenlet.getcurrent().parent.switch()
-            except:
-                seen.append(sys.exc_info()[0])
-                raise
-            raise ValueError
-        seen = []
-        seen = []
-        g1 = greenlet(fmain)
-        g2 = greenlet(fmain)
-        g1.switch(seen)
-        g2.switch(seen)
-        assert seen == []
-        del g1
-        assert seen == [greenlet.GreenletExit]
-        del g2
-        assert seen == [greenlet.GreenletExit, greenlet.GreenletExit]
-
-
-# ____________________________________________________________
-#
-# The tests from greenlets.
-# For now, without the ones that involve threads
-#
-class AppTest_PyMagicTestGreenlet:
-
-    def setup_class(cls):
-        space = gettestobjspace(usemodules=('_stackless',))
-        cls.space = space
-        cls.w_glob = space.appexec([], """():
-            import sys
-            from _stackless import greenlet
-
-            class SomeError(Exception):
-                pass
-
-            def fmain(seen):
-                try:
-                    greenlet.getcurrent().parent.switch()
-                except:
-                    seen.append(sys.exc_info()[0])
-                    raise
-                raise SomeError
-
-            class Glob: pass
-            glob = Glob()
-            glob.__dict__.update(locals())
-            return glob
-        """)
-
-    def test_simple(self):
-        greenlet = self.glob.greenlet
-        lst = []
-        def f():
-            lst.append(1)
-            greenlet.getcurrent().parent.switch()
-            lst.append(3)
-        g = greenlet(f)
-        lst.append(0)
-        g.switch()
-        lst.append(2)
-        g.switch()
-        lst.append(4)
-        assert lst == range(5)
-
-    def test_exception(self):
-        greenlet  = self.glob.greenlet
-        fmain     = self.glob.fmain
-        SomeError = self.glob.SomeError
-        seen = []
-        g1 = greenlet(fmain)
-        g2 = greenlet(fmain)
-        g1.switch(seen)
-        g2.switch(seen)
-        g2.parent = g1
-        assert seen == []
-        raises(SomeError, g2.switch)
-        assert seen == [SomeError]
-        g2.switch()
-        assert seen == [SomeError]
-
-    def test_send_exception(self):
-        greenlet  = self.glob.greenlet
-        fmain     = self.glob.fmain
-        def send_exception(g, exc):
-            # note: send_exception(g, exc)  can be now done with  g.throw(exc).
-            # the purpose of this test is to explicitely check the
-            # propagation rules.
-            def crasher(exc):
-                raise exc
-            g1 = greenlet(crasher, parent=g)
-            g1.switch(exc)
-
-        seen = []
-        g1 = greenlet(fmain)
-        g1.switch(seen)
-        raises(KeyError, "send_exception(g1, KeyError)")
-        assert seen == [KeyError]
-
-    def test_dealloc(self):
-        skip("XXX in-progress: GC handling of greenlets")
-        import gc
-        greenlet = self.glob.greenlet
-        fmain    = self.glob.fmain
-        seen = []
-        g1 = greenlet(fmain)
-        g2 = greenlet(fmain)
-        g1.switch(seen)
-        g2.switch(seen)
-        assert seen == []
-        del g1
-        gc.collect()
-        assert seen == [greenlet.GreenletExit]
-        del g2
-        gc.collect()
-        assert seen == [greenlet.GreenletExit, greenlet.GreenletExit]
-
-    def test_frame(self):
-        import sys
-        greenlet = self.glob.greenlet
-        def f1():
-            f = sys._getframe(0)
-            assert f.f_back is None
-            greenlet.getcurrent().parent.switch(f)
-            return "meaning of life"
-        g = greenlet(f1)
-        frame = g.switch()
-        assert frame is g.gr_frame
-        assert g
-        next = g.switch()
-        assert not g
-        assert next == "meaning of life"
-        assert g.gr_frame is None
-
-
-class AppTest_PyMagicTestThrow:
-
-    def setup_class(cls):
-        space = gettestobjspace(usemodules=('_stackless',))
-        cls.space = space
-
-    def test_class(self):
-        from _stackless import greenlet
-        def switch(*args):
-            return greenlet.getcurrent().parent.switch(*args)
-
-        def f():
-            try:
-                switch("ok")
-            except RuntimeError:
-                switch("ok")
-                return
-            switch("fail")
-
-        g = greenlet(f)
-        res = g.switch()
-        assert res == "ok"
-        res = g.throw(RuntimeError)
-        assert res == "ok"
-
-    def test_val(self):
-        from _stackless import greenlet
-        def switch(*args):
-            return greenlet.getcurrent().parent.switch(*args)
-
-        def f():
-            try:
-                switch("ok")
-            except RuntimeError, val:
-                if str(val) == "ciao":
-                    switch("ok")
-                    return
-            switch("fail")
-
-        g = greenlet(f)
-        res = g.switch()
-        assert res == "ok"
-        res = g.throw(RuntimeError("ciao"))
-        assert res == "ok"
-
-        g = greenlet(f)
-        res = g.switch()
-        assert res == "ok"
-        res = g.throw(RuntimeError, "ciao")
-        assert res == "ok"
-
-    def test_kill(self):
-        from _stackless import greenlet
-        def switch(*args):
-            return greenlet.getcurrent().parent.switch(*args)
-
-        def f():
-            switch("ok")
-            switch("fail")
-
-        g = greenlet(f)
-        res = g.switch()
-        assert res == "ok"
-        res = g.throw()
-        assert isinstance(res, greenlet.GreenletExit)
-        assert g.dead
-        res = g.throw()    # immediately eaten by the already-dead greenlet
-        assert isinstance(res, greenlet.GreenletExit)
-
-    def test_throw_goes_to_original_parent(self):
-        from _stackless import greenlet
-        main = greenlet.getcurrent()
-        def f1():
-            try:
-                main.switch("f1 ready to catch")
-            except IndexError:
-                return "caught"
-            else:
-                return "normal exit"
-        def f2():
-            main.switch("from f2")
-
-        g1 = greenlet(f1)
-        g2 = greenlet(f2, parent=g1)
-        raises(IndexError, g2.throw, IndexError)
-        assert g2.dead
-        assert g1.dead
-
-        g1 = greenlet(f1)
-        g2 = greenlet(f2, parent=g1)
-        res = g1.switch()
-        assert res == "f1 ready to catch"
-        res = g2.throw(IndexError)
-        assert res == "caught"
-        assert g2.dead
-        assert g1.dead
-
-        g1 = greenlet(f1)
-        g2 = greenlet(f2, parent=g1)
-        res = g1.switch()
-        assert res == "f1 ready to catch"
-        res = g2.switch()
-        assert res == "from f2"
-        res = g2.throw(IndexError)
-        assert res == "caught"
-        assert g2.dead
-        assert g1.dead
-            
-
-class AppTest_PyMagicTestGenerator:
-
-    def setup_class(cls):
-        space = gettestobjspace(usemodules=('_stackless',))
-        cls.space = space
-
-    def test_generator(self):
-        from _stackless import greenlet
-
-        class genlet(greenlet):
-
-            def __init__(self, *args, **kwds):
-                self.args = args
-                self.kwds = kwds
-
-            def run(self):
-                fn, = self.fn
-                fn(*self.args, **self.kwds)
-
-            def __iter__(self):
-                return self
-
-            def next(self):
-                self.parent = greenlet.getcurrent()
-                result = self.switch()
-                if self:
-                    return result
-                else:
-                    raise StopIteration
-
-        def Yield(value):
-            g = greenlet.getcurrent()
-            while not isinstance(g, genlet):
-                if g is None:
-                    raise RuntimeError, 'yield outside a genlet'
-                g = g.parent
-            g.parent.switch(value)
-
-        def generator(func):
-            class generator(genlet):
-                fn = (func,)
-            return generator
-
-        # ___ test starts here ___
-        seen = []
-        def g(n):
-            for i in range(n):
-                seen.append(i)
-                Yield(i)
-        g = generator(g)
-        for k in range(3):
-            for j in g(5):
-                seen.append(j)
-        assert seen == 3 * [0, 0, 1, 1, 2, 2, 3, 3, 4, 4]
-
-
-class AppTest_PyMagicTestGeneratorNested:
-
-    def setup_class(cls):
-        space = gettestobjspace(usemodules=('_stackless',))
-        cls.space = space
-        cls.w_glob = space.appexec([], """():
-            from _stackless import greenlet
-
-            class genlet(greenlet):
-
-                def __init__(self, *args, **kwds):
-                    self.args = args
-                    self.kwds = kwds
-                    self.child = None
-
-                def run(self):
-                    fn, = self.fn
-                    fn(*self.args, **self.kwds)
-
-                def __iter__(self):
-                    return self
-
-                def set_child(self, child):
-                    self.child = child
-
-                def next(self):
-                    if self.child:
-                        child = self.child
-                        while child.child:
-                            tmp = child
-                            child = child.child
-                            tmp.child = None
-
-                        result = child.switch()
-                    else:
-                        self.parent = greenlet.getcurrent()            
-                        result = self.switch()
-
-                    if self:
-                        return result
-                    else:
-                        raise StopIteration
-
-            def Yield(value, level = 1):
-                g = greenlet.getcurrent()
-
-                while level != 0:
-                    if not isinstance(g, genlet):
-                        raise RuntimeError, 'yield outside a genlet'
-                    if level > 1:
-                        g.parent.set_child(g)
-                    g = g.parent
-                    level -= 1
-
-                g.switch(value)
-
-            def Genlet(func):
-                class Genlet(genlet):
-                    fn = (func,)
-                return Genlet
-
-            class Glob: pass
-            glob = Glob()
-            glob.__dict__.update(locals())
-            return glob
-        """)
-
-    def test_genlet_1(self):
-        Genlet = self.glob.Genlet
-        Yield  = self.glob.Yield
-
-        def g1(n, seen):
-            for i in range(n):
-                seen.append(i+1)
-                yield i
-
-        def g2(n, seen):
-            for i in range(n):
-                seen.append(i+1)
-                Yield(i)
-
-        g2 = Genlet(g2)
-
-        def nested(i):
-            Yield(i)
-
-        def g3(n, seen):
-            for i in range(n):
-                seen.append(i+1)
-                nested(i)
-        g3 = Genlet(g3)
-
-        raises(RuntimeError, Yield, 10)
-        for g in [g1, g2, g3]:
-            seen = []
-            for k in range(3):
-                for j in g(5, seen):
-                    seen.append(j)
-            assert seen == 3 * [1, 0, 2, 1, 3, 2, 4, 3, 5, 4]
-        raises(RuntimeError, Yield, 10)
-
-    def test_nested_genlets(self):
-        Genlet = self.glob.Genlet
-        Yield  = self.glob.Yield
-        def a(n):
-            if n == 0:
-                return
-            for ii in ax(n-1):
-                Yield(ii)
-            Yield(n)
-        ax = Genlet(a)
-        seen = []
-        for ii in ax(5):
-            seen.append(ii)
-        assert seen == [1, 2, 3, 4, 5]
-
-    def test_perms(self):
-        Genlet = self.glob.Genlet
-        Yield  = self.glob.Yield
-        def perms(l):
-            if len(l) > 1:
-                for e in l:
-                    # No syntactical sugar for generator expressions
-                    [Yield([e] + p) for p in perms([x for x in l if x!=e])]
-            else:
-                Yield(l)
-        perms = Genlet(perms)
-        gen_perms = perms(range(4))
-        permutations = list(gen_perms)
-        assert len(permutations) == 4*3*2*1
-        assert [0,1,2,3] in permutations
-        assert [3,2,1,0] in permutations
-
-    def test_layered_genlets(self):
-        Genlet = self.glob.Genlet
-        Yield  = self.glob.Yield
-        def gr1(n):
-            for ii in range(1, n):
-                Yield(ii)
-                Yield(ii * ii, 2)
-        gr1 = Genlet(gr1)
-        def gr2(n, seen):
-            for ii in gr1(n):
-                seen.append(ii)
-        gr2 = Genlet(gr2)
-        seen = []
-        for ii in gr2(5, seen):
-            seen.append(ii)
-        assert seen == [1, 1, 2, 4, 3, 9, 4, 16]
diff --git a/pypy/module/_stackless/test/test_interp_clonable.py b/pypy/module/_stackless/test/test_interp_clonable.py
deleted file mode 100644
--- a/pypy/module/_stackless/test/test_interp_clonable.py
+++ /dev/null
@@ -1,118 +0,0 @@
-"""
-testing cloning
-"""
-import py; py.test.skip("clonable coroutines not really maintained any more")
-
-from pypy import conftest; conftest.translation_test_so_skip_if_appdirect()
-from pypy.translator.c import gc
-from pypy.rpython.memory.gctransform import stacklessframework
-from pypy.rpython.memory.test import test_transformed_gc
-from pypy.module._stackless.rclonable import InterpClonableCoroutine as ClonableCoroutine
-from pypy.module._stackless.rclonable import AbstractThunk, fork
-
-class TestClonableCoroutine(test_transformed_gc.GCTest):
-
-    gcname = "marksweep"
-    stacklessgc = True
-    class gcpolicy(gc.StacklessFrameworkGcPolicy):
-        class transformerclass(stacklessframework.StacklessFrameworkGCTransformer):
-            GC_PARAMS = {'start_heap_size': 4096 }
-
-    def test_clone(self):
-        class T(AbstractThunk):
-            def __init__(self, result):
-                self.result = result
-            def call(self):
-                self.result.append(2)
-                ClonableCoroutine.getmain().switch()
-                self.result.append(4)
-        def f():
-            result = []
-            coro = ClonableCoroutine()
-            coro.bind(T(result))
-            result.append(1)
-            coro.switch()
-            coro2 = coro.clone()
-            result.append(3)
-            coro2.switch()
-            result.append(5)
-            coro.switch()
-            result.append(6)
-            n = 0
-            for i in result:
-                n = n*10 + i
-            return n
-
-        run = self.runner(f)
-        res = run([])
-        assert res == 1234546
-
-    def test_clone_local_state(self):
-        class T(AbstractThunk):
-            def __init__(self, result):
-                self.result = result
-            def call(self):
-                localstate = []
-                localstate.append(10)
-                self.result.append(2)
-                ClonableCoroutine.getmain().switch()
-                localstate.append(20)
-                if localstate == [10, 20]:
-                    self.result.append(4)
-                else:
-                    self.result.append(0)
-        def f():
-            result = []
-            coro = ClonableCoroutine()
-            coro.bind(T(result))
-            result.append(1)
-            coro.switch()
-            coro2 = coro.clone()
-            result.append(3)
-            coro2.switch()
-            result.append(5)
-            coro.switch()
-            result.append(6)
-            n = 0
-            for i in result:
-                n = n*10 + i
-            return n
-
-        run = self.runner(f)
-        res = run([])
-        assert res == 1234546
-
-    def test_fork(self):
-        class T(AbstractThunk):
-            def __init__(self, result):
-                self.result = result
-            def call(self):
-                localdata = [10]
-                self.result.append(2)
-                newcoro = fork()
-                localdata.append(20)
-                if newcoro is not None:
-                    # in the parent
-                    self.result.append(3)
-                    newcoro.switch()
-                    self.result.append(5)
-                else:
-                    # in the child
-                    self.result.append(4)
-                localdata.append(30)
-                self.result.append(localdata != [10, 20, 30])
-        def f():
-            result = []
-            coro = ClonableCoroutine()
-            coro.bind(T(result))
-            result.append(1)
-            coro.switch()
-            result.append(6)
-            n = 0
-            for i in result:
-                n = n*10 + i
-            return n
-
-        run = self.runner(f)
-        res = run([])
-        assert res == 12340506
diff --git a/pypy/module/_stackless/test/test_pickle.py b/pypy/module/_stackless/test/test_pickle.py
deleted file mode 100644
--- a/pypy/module/_stackless/test/test_pickle.py
+++ /dev/null
@@ -1,487 +0,0 @@
-from pypy.conftest import gettestobjspace, option
-import py
-
-# app-level testing of coroutine pickling
-
-
-class AppTestBasic:
-    def setup_class(cls):
-        cls.space = gettestobjspace(usemodules=('_stackless',))
-
-    def test_pickle_main(self):
-        import _stackless, pickle
-        main = _stackless.coroutine.getcurrent()
-        s = pickle.dumps(main)
-        c = pickle.loads(s)
-        assert c is main
-
-
-class AppTestPickle:
-
-    def setup_class(cls):
-        cls.space = gettestobjspace(usemodules=('_stackless',), CALL_METHOD=True)
-
-    def test_pickle_coroutine_empty(self):
-        # this test is limited to basic pickling.
-        # real stacks can only tested with a stackless pypy build.
-        import _stackless as stackless
-        co = stackless.coroutine()
-        import pickle
-        pckl = pickle.dumps(co)
-        co2 = pickle.loads(pckl)
-        # the empty unpickled coroutine can still be used:
-        result = []
-        co2.bind(result.append, 42)
-        co2.switch()
-        assert result == [42]
-
-    def test_pickle_coroutine_bound(self):
-        import pickle
-        import _stackless
-        lst = [4]
-        co = _stackless.coroutine()
-        co.bind(lst.append, 2)
-        pckl = pickle.dumps((co, lst))
-
-        (co2, lst2) = pickle.loads(pckl)
-        assert lst2 == [4]
-        co2.switch()
-        assert lst2 == [4, 2]
-
-
-    def test_simple_ish(self):
-
-        import new, sys
-
-        mod = new.module('mod')
-        sys.modules['mod'] = mod
-        try:
-            exec '''
-output = []
-import _stackless
-def f(coro, n, x):
-    if n == 0:
-        coro.switch()
-        return
-    f(coro, n-1, 2*x)
-    output.append(x)
-
-def example():
-    main_coro = _stackless.coroutine.getcurrent()
-    sub_coro = _stackless.coroutine()
-    sub_coro.bind(f, main_coro, 5, 1)
-    sub_coro.switch()
-
-    import pickle
-    pckl = pickle.dumps(sub_coro)
-    new_coro = pickle.loads(pckl)
-
-    new_coro.switch()
-
-example()
-assert output == [16, 8, 4, 2, 1]
-''' in mod.__dict__
-        finally:
-            del sys.modules['mod']
-
-    def test_pickle_again(self):
-
-        import new, sys
-
-        mod = new.module('mod')
-        sys.modules['mod'] = mod
-        try:
-            exec '''
-output = []
-import _stackless
-def f(coro, n, x):
-    if n == 0:
-        coro.switch()
-        return
-    f(coro, n-1, 2*x)
-    output.append(x)
-
-def example():
-    main_coro = _stackless.coroutine.getcurrent()
-    sub_coro = _stackless.coroutine()
-    sub_coro.bind(f, main_coro, 5, 1)
-    sub_coro.switch()
-
-    import pickle
-    pckl = pickle.dumps(sub_coro)
-    new_coro = pickle.loads(pckl)
-    pckl = pickle.dumps(new_coro)
-    newer_coro = pickle.loads(pckl)
-
-    newer_coro.switch()
-
-example()
-assert output == [16, 8, 4, 2, 1]
-''' in mod.__dict__
-        finally:
-            del sys.modules['mod']
-
-    def test_kwargs(self):
-
-        import new, sys
-
-        mod = new.module('mod')
-        sys.modules['mod'] = mod
-        try:
-            exec '''
-output = []
-import _stackless
-def f(coro, n, x, step=4):
-    if n == 0:
-        coro.switch()
-        return
-    f(coro, n-1, 2*x, step=1)
-    output.append(x)
-
-def example():
-    main_coro = _stackless.coroutine.getcurrent()
-    sub_coro = _stackless.coroutine()
-    sub_coro.bind(f, main_coro, 5, 1, 1)
-    sub_coro.switch()
-
-    import pickle
-    pckl = pickle.dumps(sub_coro)
-    new_coro = pickle.loads(pckl)
-
-    new_coro.switch()
-
-example()
-assert output == [16, 8, 4, 2, 1]
-''' in mod.__dict__
-        finally:
-            del sys.modules['mod']
-
-    def test_starstarargs(self):
-
-        import new, sys
-
-        mod = new.module('mod')
-        sys.modules['mod'] = mod
-        try:
-            exec '''
-output = []
-import _stackless
-def f(coro, n, x, step=4):
-    if n == 0:
-        coro.switch()
-        return
-    f(coro, n-1, 2*x, **{'step': 1})
-    output.append(x)
-
-def example():
-    main_coro = _stackless.coroutine.getcurrent()
-    sub_coro = _stackless.coroutine()
-    sub_coro.bind(f, main_coro, 5, 1, 1)
-    sub_coro.switch()
-
-    import pickle
-    pckl = pickle.dumps(sub_coro)
-    new_coro = pickle.loads(pckl)
-
-    new_coro.switch()
-
-example()
-assert output == [16, 8, 4, 2, 1]
-''' in mod.__dict__
-        finally:
-            del sys.modules['mod']
-
-    def test_closure(self):
-        import new, sys
-
-        mod = new.module('mod')
-        sys.modules['mod'] = mod
-        try:
-            exec '''
-output = []
-import _stackless
-def example():
-    main_coro = _stackless.coroutine.getcurrent()
-    sub_coro = _stackless.coroutine()
-    y = 3
-    def f(coro, n, x):
-        if n == 0:
-            coro.switch()
-            return
-        f(coro, n-1, 2*x)
-        output.append(x+y)
-
-    sub_coro.bind(f, main_coro, 5, 1)
-    sub_coro.switch()
-
-    import pickle
-    pckl = pickle.dumps(sub_coro)
-    new_coro = pickle.loads(pckl)
-
-    new_coro.switch()
-
-example()
-assert output == [19, 11, 7, 5, 4]
-''' in mod.__dict__
-        finally:
-            del sys.modules['mod']
-
-    def test_exception(self):
-        import new, sys
-
-        mod = new.module('mod')
-        sys.modules['mod'] = mod
-        try:
-            exec '''
-output = []
-import _stackless
-def f(coro):
-    try:
-        raise ValueError
-    except:
-        coro.switch()
-        import sys
-        t, v, tb = sys.exc_info()
-        output.append(t)
-
-def example():
-    main_coro = _stackless.coroutine.getcurrent()
-    sub_coro = _stackless.coroutine()
-
-    sub_coro.bind(f, main_coro)
-    sub_coro.switch()
-
-    import pickle
-    pckl = pickle.dumps(sub_coro)
-    new_coro = pickle.loads(pckl)
-
-    new_coro.switch()
-
-example()
-assert output == [ValueError]
-''' in mod.__dict__
-        finally:
-            del sys.modules['mod']
-
-    def test_exception_after_unpickling(self):
-
-        import new, sys
-
-        mod = new.module('mod')
-        sys.modules['mod'] = mod
-        try:
-            exec '''
-output = []
-import _stackless
-def f(coro, n, x):
-    if n == 0:
-        coro.switch()
-        raise ValueError
-    try:
-        f(coro, n-1, 2*x)
-    finally:
-        output.append(x)
-
-def example():
-    main_coro = _stackless.coroutine.getcurrent()
-    sub_coro = _stackless.coroutine()
-    sub_coro.bind(f, main_coro, 5, 1)
-    sub_coro.switch()
-
-    import pickle
-    pckl = pickle.dumps(sub_coro)
-    new_coro = pickle.loads(pckl)
-
-    try:
-        sub_coro.switch()
-    except ValueError:
-        pass
-    else:
-        assert 0
-    try:
-        new_coro.switch()
-    except ValueError:
-        pass
-    else:
-        assert 0
-
-example()
-assert output == [16, 8, 4, 2, 1] * 2
-''' in mod.__dict__
-        finally:
-            del sys.modules['mod']
-
-    def test_loop(self):
-        import new, sys
-
-        mod = new.module('mod')
-        sys.modules['mod'] = mod
-        try:
-            exec '''
-output = []
-import _stackless
-def f(coro):
-    for x in (1,2,3):
-        coro.switch()
-        output.append(x)
-
-def example():
-    main_coro = _stackless.coroutine.getcurrent()
-    sub_coro = _stackless.coroutine()
-
-    sub_coro.bind(f, main_coro)
-    sub_coro.switch()
-
-    import pickle
-    pckl = pickle.dumps(sub_coro)
-    new_coro = pickle.loads(pckl)
-
-    new_coro.switch()
-    new_coro.switch()
-    new_coro.switch()
-
-example()
-assert output == [1, 2, 3]
-''' in mod.__dict__
-        finally:
-            del sys.modules['mod']
-
-    def test_valstack(self):
-        import new, sys
-
-        mod = new.module('mod')
-        sys.modules['mod'] = mod
-        try:
-            exec '''
-output = []
-import _stackless
-def f(coro):
-    r = 1+g(coro)+3
-    output.append(r)
-
-def g(coro):
-    coro.switch()
-    return 2
-
-def example():
-    main_coro = _stackless.coroutine.getcurrent()
-    sub_coro = _stackless.coroutine()
-
-    sub_coro.bind(f, main_coro)
-    sub_coro.switch()
-
-    import pickle
-    pckl = pickle.dumps(sub_coro)
-    new_coro = pickle.loads(pckl)
-
-    new_coro.switch()
-
-
-example()
-assert output == [6]
-''' in mod.__dict__
-        finally:
-            del sys.modules['mod']
-
-
-    def test_exec_and_locals(self):
-        import new, sys
-
-        mod = new.module('mod')
-        sys.modules['mod'] = mod
-        try:
-            exec '''
-output = []
-import _stackless
-
-def f(coro):
-    x = None
-    exec "x = 9"
-    coro.switch()
-    output.append(x)
-
-def example():
-    main_coro = _stackless.coroutine.getcurrent()
-    sub_coro = _stackless.coroutine()
-    sub_coro.bind(f, main_coro)
-    sub_coro.switch()
-
-    import pickle
-    pckl = pickle.dumps(sub_coro)
-    new_coro = pickle.loads(pckl)
-
-    new_coro.switch()
-
-example()
-assert output == [9]
-''' in mod.__dict__
-        finally:
-            del sys.modules['mod']
-
-
-    def test_solver(self):
-        import new, sys
-
-        mod = new.module('mod')
-        sys.modules['mod'] = mod
-        try:
-            exec '''
-import _stackless, pickle
-
-class Fail(Exception):
-    pass
-
-class Success(Exception):
-    pass
-
-def first_solution(func):
-    global next_answer
-    co = _stackless.coroutine()
-    co.bind(func)
-    pending = [(co, None)]
-    while pending:
-        co, next_answer = pending.pop()
-        try:
-            co.switch()
-        except Fail:
-            pass
-        except Success, e:
-            return e.args[0]
-        else:
-            # zero_or_one() called, clone the coroutine
-            # NB. this seems to be quite slow
-            co2 = pickle.loads(pickle.dumps(co))
-            pending.append((co2, 1))
-            pending.append((co, 0))
-    raise Fail("no solution")
-
-pending = []
-main = _stackless.coroutine.getcurrent()
-
-def zero_or_one():
-    main.switch()
-    return next_answer
-
-# ____________________________________________________________
-
-invalid_prefixes = {
-    (0, 0): True,
-    (0, 1, 0): True,
-    (0, 1, 1): True,
-    (1, 0): True,
-    (1, 1, 0, 0): True,
-    }
-
-def example():
-    test = []
-    for n in range(5):
-        test.append(zero_or_one())
-        if tuple(test) in invalid_prefixes:
-            raise Fail
-    raise Success(test)
-
-res = first_solution(example)
-assert res == [1, 1, 0, 1, 0]
-''' in mod.__dict__
-        finally:
-            del sys.modules['mod']
diff --git a/pypy/module/marshal/interp_marshal.py b/pypy/module/marshal/interp_marshal.py
--- a/pypy/module/marshal/interp_marshal.py
+++ b/pypy/module/marshal/interp_marshal.py
@@ -40,7 +40,7 @@
         reader = FileReader(space, w_f)
     try:
         u = Unmarshaller(space, reader)
-        return u.load_w_obj(False)
+        return u.load_w_obj()
     finally:
         reader.finished()
 
@@ -49,7 +49,7 @@
 ignored."""
     space.timer.start("marshal loads")
     u = StringUnmarshaller(space, w_str)
-    obj = u.load_w_obj(False)
+    obj = u.load_w_obj()
     space.timer.stop("marshal loads")
     return obj
 
@@ -424,7 +424,7 @@
         lng = self.get_lng()
         return self.get(lng)
 
-    def get_w_obj(self, allow_null):
+    def get_w_obj(self, allow_null=False):
         space = self.space
         w_ret = space.w_None # something not None
         tc = self.get1()
@@ -434,9 +434,9 @@
                 'NULL object in marshal data'))
         return w_ret
 
-    def load_w_obj(self, allow_null):
+    def load_w_obj(self):
         try:
-            return self.get_w_obj(allow_null)
+            return self.get_w_obj()
         except rstackovf.StackOverflow:
             rstackovf.check_stack_overflow()
             self._overflow()
diff --git a/pypy/module/micronumpy/__init__.py b/pypy/module/micronumpy/__init__.py
--- a/pypy/module/micronumpy/__init__.py
+++ b/pypy/module/micronumpy/__init__.py
@@ -26,13 +26,19 @@
         ("copysign", "copysign"),
         ("cos", "cos"),
         ("divide", "divide"),
+        ("equal", "equal"),
         ("exp", "exp"),
         ("fabs", "fabs"),
         ("floor", "floor"),
+        ("greater", "greater"),
+        ("greater_equal", "greater_equal"),
+        ("less", "less"),
+        ("less_equal", "less_equal"),
         ("maximum", "maximum"),
         ("minimum", "minimum"),
         ("multiply", "multiply"),
         ("negative", "negative"),
+        ("not_equal", "not_equal"),
         ("reciprocal", "reciprocal"),
         ("sign", "sign"),
         ("sin", "sin"),
diff --git a/pypy/module/micronumpy/interp_dtype.py b/pypy/module/micronumpy/interp_dtype.py
--- a/pypy/module/micronumpy/interp_dtype.py
+++ b/pypy/module/micronumpy/interp_dtype.py
@@ -129,6 +129,16 @@
         ))
     return impl
 
+def raw_binop(func):
+    # Returns the result unwrapped.
+    @functools.wraps(func)
+    def impl(self, v1, v2):
+        return func(self,
+            self.for_computation(self.unbox(v1)),
+            self.for_computation(self.unbox(v2))
+        )
+    return impl
+
 def unaryop(func):
     @functools.wraps(func)
     def impl(self, v):
@@ -170,8 +180,24 @@
 
     def bool(self, v):
         return bool(self.for_computation(self.unbox(v)))
+    @raw_binop
+    def eq(self, v1, v2):
+        return v1 == v2
+    @raw_binop
     def ne(self, v1, v2):
-        return self.for_computation(self.unbox(v1)) != self.for_computation(self.unbox(v2))
+        return v1 != v2
+    @raw_binop
+    def lt(self, v1, v2):
+        return v1 < v2
+    @raw_binop
+    def le(self, v1, v2):
+        return v1 <= v2
+    @raw_binop
+    def gt(self, v1, v2):
+        return v1 > v2
+    @raw_binop
+    def ge(self, v1, v2):
+        return v1 >= v2
 
 
 class FloatArithmeticDtype(ArithmaticTypeMixin):
@@ -224,7 +250,7 @@
         return math.tan(v)
     @unaryop
     def arcsin(self, v):
-        if v < -1.0 or  v > 1.0:
+        if v < -1.0 or v > 1.0:
             return rfloat.NAN
         return math.asin(v)
     @unaryop
diff --git a/pypy/module/micronumpy/interp_numarray.py b/pypy/module/micronumpy/interp_numarray.py
--- a/pypy/module/micronumpy/interp_numarray.py
+++ b/pypy/module/micronumpy/interp_numarray.py
@@ -74,6 +74,13 @@
     descr_pow = _binop_impl("power")
     descr_mod = _binop_impl("mod")
 
+    descr_eq = _binop_impl("equal")
+    descr_ne = _binop_impl("not_equal")
+    descr_lt = _binop_impl("less")
+    descr_le = _binop_impl("less_equal")
+    descr_gt = _binop_impl("greater")
+    descr_ge = _binop_impl("greater_equal")
+
     def _binop_right_impl(ufunc_name):
         def impl(self, space, w_other):
             w_other = scalar_w(space,
@@ -404,10 +411,11 @@
     """
     Intermediate class for performing binary operations.
     """
-    def __init__(self, signature, res_dtype, left, right):
+    def __init__(self, signature, calc_dtype, res_dtype, left, right):
         VirtualArray.__init__(self, signature, res_dtype)
         self.left = left
         self.right = right
+        self.calc_dtype = calc_dtype
 
     def _del_sources(self):
         self.left = None
@@ -421,14 +429,14 @@
         return self.right.find_size()
 
     def _eval(self, i):
-        lhs = self.left.eval(i).convert_to(self.res_dtype)
-        rhs = self.right.eval(i).convert_to(self.res_dtype)
+        lhs = self.left.eval(i).convert_to(self.calc_dtype)
+        rhs = self.right.eval(i).convert_to(self.calc_dtype)
 
         sig = jit.promote(self.signature)
         assert isinstance(sig, signature.Signature)
         call_sig = sig.components[0]
         assert isinstance(call_sig, signature.Call2)
-        return call_sig.func(self.res_dtype, lhs, rhs)
+        return call_sig.func(self.calc_dtype, lhs, rhs)
 
 class ViewArray(BaseArray):
     """
@@ -573,18 +581,28 @@
     __pos__ = interp2app(BaseArray.descr_pos),
     __neg__ = interp2app(BaseArray.descr_neg),
     __abs__ = interp2app(BaseArray.descr_abs),
+
     __add__ = interp2app(BaseArray.descr_add),
     __sub__ = interp2app(BaseArray.descr_sub),
     __mul__ = interp2app(BaseArray.descr_mul),
     __div__ = interp2app(BaseArray.descr_div),
     __pow__ = interp2app(BaseArray.descr_pow),
     __mod__ = interp2app(BaseArray.descr_mod),
+
     __radd__ = interp2app(BaseArray.descr_radd),
     __rsub__ = interp2app(BaseArray.descr_rsub),
     __rmul__ = interp2app(BaseArray.descr_rmul),
     __rdiv__ = interp2app(BaseArray.descr_rdiv),
     __rpow__ = interp2app(BaseArray.descr_rpow),
     __rmod__ = interp2app(BaseArray.descr_rmod),
+
+    __eq__ = interp2app(BaseArray.descr_eq),
+    __ne__ = interp2app(BaseArray.descr_ne),
+    __lt__ = interp2app(BaseArray.descr_lt),
+    __le__ = interp2app(BaseArray.descr_le),
+    __gt__ = interp2app(BaseArray.descr_gt),
+    __ge__ = interp2app(BaseArray.descr_ge),
+
     __repr__ = interp2app(BaseArray.descr_repr),
     __str__ = interp2app(BaseArray.descr_str),
 
diff --git a/pypy/module/micronumpy/interp_ufuncs.py b/pypy/module/micronumpy/interp_ufuncs.py
--- a/pypy/module/micronumpy/interp_ufuncs.py
+++ b/pypy/module/micronumpy/interp_ufuncs.py
@@ -113,10 +113,11 @@
     argcount = 2
 
     def __init__(self, func, name, promote_to_float=False, promote_bools=False,
-        identity=None):
+        identity=None, comparison_func=False):
 
         W_Ufunc.__init__(self, name, promote_to_float, promote_bools, identity)
         self.func = func
+        self.comparison_func = comparison_func
         self.signature = signature.Call2(func)
         self.reduce_signature = signature.BaseSignature()
 
@@ -127,18 +128,25 @@
         [w_lhs, w_rhs] = args_w
         w_lhs = convert_to_array(space, w_lhs)
         w_rhs = convert_to_array(space, w_rhs)
-        res_dtype = find_binop_result_dtype(space,
+        calc_dtype = find_binop_result_dtype(space,
             w_lhs.find_dtype(), w_rhs.find_dtype(),
             promote_to_float=self.promote_to_float,
             promote_bools=self.promote_bools,
         )
+        if self.comparison_func:
+            res_dtype = space.fromcache(interp_dtype.W_BoolDtype)
+        else:
+            res_dtype = calc_dtype
         if isinstance(w_lhs, Scalar) and isinstance(w_rhs, Scalar):
-            return self.func(res_dtype, w_lhs.value, w_rhs.value).wrap(space)
+            return self.func(calc_dtype,
+                w_lhs.value.convert_to(calc_dtype),
+                w_rhs.value.convert_to(calc_dtype)
+            ).wrap(space)
 
         new_sig = signature.Signature.find_sig([
             self.signature, w_lhs.signature, w_rhs.signature
         ])
-        w_res = Call2(new_sig, res_dtype, w_lhs, w_rhs)
+        w_res = Call2(new_sig, calc_dtype, res_dtype, w_lhs, w_rhs)
         w_lhs.add_invalidates(w_res)
         w_rhs.add_invalidates(w_res)
         return w_res
@@ -209,13 +217,16 @@
     return space.fromcache(interp_dtype.W_Float64Dtype)
 
 
-def ufunc_dtype_caller(ufunc_name, op_name, argcount):
+def ufunc_dtype_caller(space, ufunc_name, op_name, argcount, comparison_func):
     if argcount == 1:
         def impl(res_dtype, value):
             return getattr(res_dtype, op_name)(value)
     elif argcount == 2:
         def impl(res_dtype, lvalue, rvalue):
-            return getattr(res_dtype, op_name)(lvalue, rvalue)
+            res = getattr(res_dtype, op_name)(lvalue, rvalue)
+            if comparison_func:
+                res = space.fromcache(interp_dtype.W_BoolDtype).box(res)
+            return res
     return func_with_new_name(impl, ufunc_name)
 
 class UfuncState(object):
@@ -229,6 +240,13 @@
             ("mod", "mod", 2, {"promote_bools": True}),
             ("power", "pow", 2, {"promote_bools": True}),
 
+            ("equal", "eq", 2, {"comparison_func": True}),
+            ("not_equal", "ne", 2, {"comparison_func": True}),
+            ("less", "lt", 2, {"comparison_func": True}),
+            ("less_equal", "le", 2, {"comparison_func": True}),
+            ("greater", "gt", 2, {"comparison_func": True}),
+            ("greater_equal", "ge", 2, {"comparison_func": True}),
+
             ("maximum", "max", 2),
             ("minimum", "min", 2),
 
@@ -262,7 +280,9 @@
             identity = space.fromcache(interp_dtype.W_Int64Dtype).adapt_val(identity)
         extra_kwargs["identity"] = identity
 
-        func = ufunc_dtype_caller(ufunc_name, op_name, argcount)
+        func = ufunc_dtype_caller(space, ufunc_name, op_name, argcount,
+            comparison_func=extra_kwargs.get("comparison_func", False)
+        )
         if argcount == 1:
             ufunc = W_Ufunc1(func, ufunc_name, **extra_kwargs)
         elif argcount == 2:
diff --git a/pypy/module/micronumpy/test/test_numarray.py b/pypy/module/micronumpy/test/test_numarray.py
--- a/pypy/module/micronumpy/test/test_numarray.py
+++ b/pypy/module/micronumpy/test/test_numarray.py
@@ -557,6 +557,26 @@
         assert array([1.2, 5]).dtype is dtype(float)
         assert array([]).dtype is dtype(float)
 
+    def test_comparison(self):
+        import operator
+        from numpy import array, dtype
+
+        a = array(range(5))
+        b = array(range(5), float)
+        for func in [
+            operator.eq, operator.ne, operator.lt, operator.le, operator.gt,
+            operator.ge
+        ]:
+            c = func(a, 3)
+            assert c.dtype is dtype(bool)
+            for i in xrange(5):
+                assert c[i] == func(a[i], 3)
+
+            c = func(b, 3)
+            assert c.dtype is dtype(bool)
+            for i in xrange(5):
+                assert c[i] == func(b[i], 3)
+
 
 class AppTestSupport(object):
     def setup_class(cls):
diff --git a/pypy/module/micronumpy/test/test_ufuncs.py b/pypy/module/micronumpy/test/test_ufuncs.py
--- a/pypy/module/micronumpy/test/test_ufuncs.py
+++ b/pypy/module/micronumpy/test/test_ufuncs.py
@@ -310,4 +310,30 @@
         assert add.reduce([1, 2, 3]) == 6
         assert maximum.reduce([1]) == 1
         assert maximum.reduce([1, 2, 3]) == 3
-        raises(ValueError, maximum.reduce, [])
\ No newline at end of file
+        raises(ValueError, maximum.reduce, [])
+
+    def test_comparisons(self):
+        import operator
+        from numpy import equal, not_equal, less, less_equal, greater, greater_equal
+
+        for ufunc, func in [
+            (equal, operator.eq),
+            (not_equal, operator.ne),
+            (less, operator.lt),
+            (less_equal, operator.le),
+            (greater, operator.gt),
+            (greater_equal, operator.ge),
+        ]:
+            for a, b in [
+                (3, 3),
+                (3, 4),
+                (4, 3),
+                (3.0, 3.0),
+                (3.0, 3.5),
+                (3.5, 3.0),
+                (3.0, 3),
+                (3, 3.0),
+                (3.5, 3),
+                (3, 3.5),
+            ]:
+                assert ufunc(a, b) is func(a, b)
diff --git a/pypy/module/pwd/__init__.py b/pypy/module/pwd/__init__.py
new file mode 100644
--- /dev/null
+++ b/pypy/module/pwd/__init__.py
@@ -0,0 +1,25 @@
+from pypy.interpreter.mixedmodule import MixedModule
+
+class Module(MixedModule):
+    """
+    This module provides access to the Unix password database.
+    It is available on all Unix versions.
+
+    Password database entries are reported as 7-tuples containing the following
+    items from the password database (see `'), in order:
+    pw_name, pw_passwd, pw_uid, pw_gid, pw_gecos, pw_dir, pw_shell.
+    The uid and gid items are integers, all others are strings. An
+    exception is raised if the entry asked for cannot be found.
+    """
+
+    interpleveldefs = {
+        'getpwuid': 'interp_pwd.getpwuid',
+        'getpwnam': 'interp_pwd.getpwnam',
+        'getpwall': 'interp_pwd.getpwall',
+    }
+
+    appleveldefs = {
+        'struct_passwd': 'app_pwd.struct_passwd',
+        'struct_pwent': 'app_pwd.struct_passwd',
+    }
+
diff --git a/pypy/module/pwd/app_pwd.py b/pypy/module/pwd/app_pwd.py
new file mode 100644
--- /dev/null
+++ b/pypy/module/pwd/app_pwd.py
@@ -0,0 +1,20 @@
+from _structseq import structseqtype, structseqfield
+
+class struct_passwd:
+    """
+    pwd.struct_passwd: Results from getpw*() routines.
+
+    This object may be accessed either as a tuple of
+      (pw_name,pw_passwd,pw_uid,pw_gid,pw_gecos,pw_dir,pw_shell)
+    or via the object attributes as named in the above tuple.
+    """
+    __metaclass__ = structseqtype
+    name = "pwd.struct_passwd"
+
+    pw_name   = structseqfield(0, "user name")
+    pw_passwd = structseqfield(1, "password")
+    pw_uid    = structseqfield(2, "user id")
+    pw_gid    = structseqfield(3, "group id")
+    pw_gecos  = structseqfield(4, "real name")
+    pw_dir    = structseqfield(5, "home directory")
+    pw_shell  = structseqfield(6, "shell program")
diff --git a/pypy/module/pwd/interp_pwd.py b/pypy/module/pwd/interp_pwd.py
new file mode 100644
--- /dev/null
+++ b/pypy/module/pwd/interp_pwd.py
@@ -0,0 +1,95 @@
+from pypy.translator.tool.cbuild import ExternalCompilationInfo
+from pypy.rpython.tool import rffi_platform
+from pypy.rpython.lltypesystem import rffi, lltype
+from pypy.interpreter.gateway import interp2app, unwrap_spec
+from pypy.interpreter.error import OperationError, operationerrfmt
+from pypy.rlib.rarithmetic import intmask
+
+eci = ExternalCompilationInfo(
+    includes=['pwd.h']
+    )
+
+class CConfig:
+    _compilation_info_ = eci
+
+    uid_t = rffi_platform.SimpleType("uid_t")
+
+    passwd = rffi_platform.Struct(
+        'struct passwd',
+        [('pw_name', rffi.CCHARP),
+         ('pw_passwd', rffi.CCHARP),
+         ('pw_uid', rffi.INT),
+         ('pw_gid', rffi.INT),
+         ('pw_gecos', rffi.CCHARP),
+         ('pw_dir', rffi.CCHARP),
+         ('pw_shell', rffi.CCHARP),
+         ])
+
+config = rffi_platform.configure(CConfig)
+passwd_p = lltype.Ptr(config['passwd'])
+uid_t = config['uid_t']
+
+def external(name, args, result, **kwargs):
+    return rffi.llexternal(name, args, result, compilation_info=eci, **kwargs)
+
+c_getpwuid = external("getpwuid", [uid_t], passwd_p)
+c_getpwnam = external("getpwnam", [rffi.CCHARP], passwd_p)
+c_setpwent = external("setpwent", [], lltype.Void)
+c_getpwent = external("getpwent", [], passwd_p)
+c_endpwent = external("endpwent", [], lltype.Void)
+
+def make_struct_passwd(space, pw):
+    w_passwd_struct = space.getattr(space.getbuiltinmodule('pwd'),
+                                    space.wrap('struct_passwd'))
+    w_tuple = space.newtuple([
+        space.wrap(rffi.charp2str(pw.c_pw_name)),
+        space.wrap(rffi.charp2str(pw.c_pw_passwd)),
+        space.wrap(intmask(pw.c_pw_uid)),
+        space.wrap(intmask(pw.c_pw_gid)),
+        space.wrap(rffi.charp2str(pw.c_pw_gecos)),
+        space.wrap(rffi.charp2str(pw.c_pw_dir)),
+        space.wrap(rffi.charp2str(pw.c_pw_shell)),
+        ])
+    return space.call_function(w_passwd_struct, w_tuple)
+
+ at unwrap_spec(uid=int)
+def getpwuid(space, uid):
+    """
+    getpwuid(uid) -> (pw_name,pw_passwd,pw_uid,
+                      pw_gid,pw_gecos,pw_dir,pw_shell)
+    Return the password database entry for the given numeric user ID.
+    See pwd.__doc__ for more on password database entries.
+    """
+    pw = c_getpwuid(uid)
+    if not pw:
+        raise operationerrfmt(space.w_KeyError,
+            "getpwuid(): uid not found: %d", uid)
+    return make_struct_passwd(space, pw)
+
+ at unwrap_spec(name=str)
+def getpwnam(space, name):
+    """
+    getpwnam(name) -> (pw_name,pw_passwd,pw_uid,
+                        pw_gid,pw_gecos,pw_dir,pw_shell)
+    Return the password database entry for the given user name.
+    See pwd.__doc__ for more on password database entries.
+    """
+    pw = c_getpwnam(name)
+    if not pw:
+        raise operationerrfmt(space.w_KeyError,
+            "getpwnam(): name not found: %s", name)
+    return make_struct_passwd(space, pw)
+
+def getpwall(space):
+    users_w = []
+    c_setpwent()
+    try:
+        while True:
+            pw = c_getpwent()
+            if not pw:
+                break
+            users_w.append(make_struct_passwd(space, pw))
+    finally:
+        c_endpwent()
+    return space.newlist(users_w)
+    
diff --git a/pypy/module/pwd/test/test_pwd.py b/pypy/module/pwd/test/test_pwd.py
new file mode 100644
--- /dev/null
+++ b/pypy/module/pwd/test/test_pwd.py
@@ -0,0 +1,28 @@
+from pypy.conftest import gettestobjspace
+
+class AppTestPwd:
+    def setup_class(cls):
+        cls.space = gettestobjspace(usemodules=['pwd'])
+
+    def test_getpwuid(self):
+        import pwd
+        raises(KeyError, pwd.getpwuid, -1)
+        pw = pwd.getpwuid(0)
+        assert pw.pw_name == 'root'
+        assert isinstance(pw.pw_passwd, str)
+        assert pw.pw_uid == 0
+        assert pw.pw_gid == 0
+        assert pw.pw_dir == '/root'
+        assert pw.pw_shell.startswith('/')
+        #
+        assert type(pw.pw_uid) is int
+        assert type(pw.pw_gid) is int
+
+    def test_getpwnam(self):
+        import pwd
+        raises(KeyError, pwd.getpwnam, '~invalid~')
+        assert pwd.getpwnam('root').pw_name == 'root'
+
+    def test_getpwall(self):
+        import pwd
+        assert pwd.getpwnam('root') in pwd.getpwall()
diff --git a/pypy/module/pypyjit/test_pypy_c/model.py b/pypy/module/pypyjit/test_pypy_c/model.py
--- a/pypy/module/pypyjit/test_pypy_c/model.py
+++ b/pypy/module/pypyjit/test_pypy_c/model.py
@@ -2,7 +2,10 @@
 import sys
 import re
 import os.path
-from _pytest.assertion import newinterpret
+try:
+    from _pytest.assertion import newinterpret
+except ImportError:   # e.g. Python 2.5
+    newinterpret = None
 from pypy.tool.jitlogparser.parser import SimpleParser, Function, TraceForOpcode
 from pypy.tool.jitlogparser.storage import LoopStorage
 
@@ -196,7 +199,7 @@
                     source = str(source.deindent()).strip()
         except py.error.ENOENT:
             source = None
-        if source and source.startswith('self._assert('):
+        if source and source.startswith('self._assert(') and newinterpret:
             # transform self._assert(x, 'foo') into assert x, 'foo'
             source = source.replace('self._assert(', 'assert ')
             source = source[:-1] # remove the trailing ')'
diff --git a/pypy/module/pypyjit/test_pypy_c/test_00_model.py b/pypy/module/pypyjit/test_pypy_c/test_00_model.py
--- a/pypy/module/pypyjit/test_pypy_c/test_00_model.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_00_model.py
@@ -1,3 +1,4 @@
+from __future__ import with_statement
 import sys, os
 import types
 import subprocess
diff --git a/pypy/module/pypyjit/test_pypy_c/test_instance.py b/pypy/module/pypyjit/test_pypy_c/test_instance.py
--- a/pypy/module/pypyjit/test_pypy_c/test_instance.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_instance.py
@@ -142,6 +142,7 @@
             i = 0
             b = B(1)
             while i < 100:
+                b.x
                 v = b.x # ID: loadattr
                 i += v
             return i
@@ -150,8 +151,6 @@
         loop, = log.loops_by_filename(self.filepath)
         assert loop.match_by_id('loadattr',
         '''
-        guard_not_invalidated(descr=...)
-        i16 = arraylen_gc(p10, descr=)
         i19 = call(ConstClass(ll_dict_lookup), _, _, _, descr=...)
         guard_no_exception(descr=...)
         i21 = int_and(i19, _)
diff --git a/pypy/module/sys/__init__.py b/pypy/module/sys/__init__.py
--- a/pypy/module/sys/__init__.py
+++ b/pypy/module/sys/__init__.py
@@ -47,6 +47,7 @@
         'pypy_initial_path'     : 'state.pypy_initial_path',
 
         '_getframe'             : 'vm._getframe', 
+        '_current_frames'       : 'vm._current_frames', 
         'setrecursionlimit'     : 'vm.setrecursionlimit', 
         'getrecursionlimit'     : 'vm.getrecursionlimit', 
         'setcheckinterval'      : 'vm.setcheckinterval', 
diff --git a/pypy/module/sys/test/test_sysmodule.py b/pypy/module/sys/test/test_sysmodule.py
--- a/pypy/module/sys/test/test_sysmodule.py
+++ b/pypy/module/sys/test/test_sysmodule.py
@@ -1,6 +1,6 @@
 # -*- coding: iso-8859-1 -*-
 import autopath
-from pypy.conftest import option
+from pypy.conftest import option, gettestobjspace
 from py.test import raises
 from pypy.interpreter.gateway import app2interp_temp
 import sys
@@ -524,3 +524,51 @@
         # If this ever actually becomes a compilation option this test should
         # be changed.
         assert sys.float_repr_style == "short"
+
+class AppTestCurrentFrames:
+
+    def test_current_frames(self):
+        try:
+            import thread
+        except ImportError:
+            pass
+        else:
+            skip('This test requires an intepreter without threads')
+        import sys
+
+        def f():
+            return sys._current_frames()
+        frames = f()
+        assert frames.keys() == [0]
+        assert frames[0].f_code.co_name == 'f'
+
+class AppTestCurrentFramesWithThread(AppTestCurrentFrames):
+    def setup_class(cls):
+        cls.space = gettestobjspace(usemodules=('thread',))
+
+    def test_current_frames(self):
+        import sys
+        import time
+        import thread
+
+        thread_id = thread.get_ident()
+        self.ready = False
+        def other_thread():
+            self.ready = True
+            print "thread started"
+            time.sleep(5)
+        thread.start_new_thread(other_thread, ())
+
+        def f():
+            for i in range(100):
+                if self.ready: break
+                time.sleep(0.1)
+            return sys._current_frames()
+        
+        frames = f()
+        thisframe = frames.pop(thread_id)
+        assert thisframe.f_code.co_name == 'f'
+
+        assert len(frames) == 1
+        _, other_frame = frames.popitem()
+        assert other_frame.f_code.co_name == 'other_thread'
diff --git a/pypy/module/sys/vm.py b/pypy/module/sys/vm.py
--- a/pypy/module/sys/vm.py
+++ b/pypy/module/sys/vm.py
@@ -43,6 +43,23 @@
     f.mark_as_escaped()
     return space.wrap(f)
 
+def _current_frames(space):
+    """_current_frames() -> dictionary
+
+    Return a dictionary mapping each current thread T's thread id to T's
+    current stack frame.
+
+    This function should be used for specialized purposes only."""
+    w_result = space.newdict()
+    ecs = space.threadlocals.getallvalues()
+    for thread_ident, ec in ecs.items():
+        f = ec.gettopframe_nohidden()
+        f.mark_as_escaped()
+        space.setitem(w_result,
+                      space.wrap(thread_ident),
+                      space.wrap(f))
+    return w_result                      
+
 def setrecursionlimit(space, w_new_limit):
     """setrecursionlimit() sets the maximum number of nested calls that
 can occur before a RuntimeError is raised.  On PyPy the limit is
diff --git a/pypy/module/test_lib_pypy/test_distributed/test_distributed.py b/pypy/module/test_lib_pypy/test_distributed/test_distributed.py
--- a/pypy/module/test_lib_pypy/test_distributed/test_distributed.py
+++ b/pypy/module/test_lib_pypy/test_distributed/test_distributed.py
@@ -1,3 +1,4 @@
+import py; py.test.skip("xxx remove")
 
 """ Controllers tests
 """
@@ -8,7 +9,7 @@
 class AppTestDistributed(object):
     def setup_class(cls):
         cls.space = gettestobjspace(**{"objspace.std.withtproxy": True,
-            "usemodules":("_stackless",)})
+            "usemodules":("_continuation",)})
 
     def test_init(self):
         import distributed
@@ -90,14 +91,12 @@
 
 class AppTestDistributedTasklets(object):
     spaceconfig = {"objspace.std.withtproxy": True,
-                   "objspace.usemodules._stackless": True}
+                   "objspace.usemodules._continuation": True}
     reclimit = sys.getrecursionlimit()
 
     def setup_class(cls):
         import py.test
         py.test.importorskip('greenlet')
-        #cls.space = gettestobjspace(**{"objspace.std.withtproxy": True,
-        #    "usemodules":("_stackless",)})
         cls.w_test_env_ = cls.space.appexec([], """():
         from distributed import test_env
         return (test_env,)
diff --git a/pypy/module/test_lib_pypy/test_distributed/test_greensock.py b/pypy/module/test_lib_pypy/test_distributed/test_greensock.py
--- a/pypy/module/test_lib_pypy/test_distributed/test_greensock.py
+++ b/pypy/module/test_lib_pypy/test_distributed/test_greensock.py
@@ -1,5 +1,4 @@
-
-import py
+import py; py.test.skip("xxx remove")
 from pypy.conftest import gettestobjspace, option
 
 def setup_module(mod):
@@ -10,7 +9,7 @@
         if not option.runappdirect:
             py.test.skip("Cannot run this on top of py.py because of PopenGateway")
         cls.space = gettestobjspace(**{"objspace.std.withtproxy": True,
-                                       "usemodules":("_stackless",)})
+                                       "usemodules":("_continuation",)})
         cls.w_remote_side_code = cls.space.appexec([], """():
         import sys
         sys.path.insert(0, '%s')
diff --git a/pypy/module/test_lib_pypy/test_distributed/test_socklayer.py b/pypy/module/test_lib_pypy/test_distributed/test_socklayer.py
--- a/pypy/module/test_lib_pypy/test_distributed/test_socklayer.py
+++ b/pypy/module/test_lib_pypy/test_distributed/test_socklayer.py
@@ -1,4 +1,4 @@
-import py
+import py; py.test.skip("xxx remove")
 from pypy.conftest import gettestobjspace
 
 def setup_module(mod):
@@ -9,7 +9,8 @@
 class AppTestSocklayer:
     def setup_class(cls):
         cls.space = gettestobjspace(**{"objspace.std.withtproxy": True,
-                                       "usemodules":("_stackless","_socket", "select")})
+                                       "usemodules":("_continuation",
+                                                     "_socket", "select")})
     
     def test_socklayer(self):
         class X(object):
diff --git a/pypy/module/test_lib_pypy/test_stackless_pickle.py b/pypy/module/test_lib_pypy/test_stackless_pickle.py
--- a/pypy/module/test_lib_pypy/test_stackless_pickle.py
+++ b/pypy/module/test_lib_pypy/test_stackless_pickle.py
@@ -1,3 +1,4 @@
+import py; py.test.skip("XXX port me")
 from pypy.conftest import gettestobjspace, option
 
 class AppTest_Stackless:
diff --git a/pypy/module/thread/threadlocals.py b/pypy/module/thread/threadlocals.py
--- a/pypy/module/thread/threadlocals.py
+++ b/pypy/module/thread/threadlocals.py
@@ -43,6 +43,9 @@
         ident = self._mainthreadident
         return self._valuedict.get(ident, None)
 
+    def getallvalues(self):
+        return self._valuedict
+
     def enter_thread(self, space):
         "Notification that the current thread is just starting."
         ec = space.getexecutioncontext()
diff --git a/pypy/objspace/std/marshal_impl.py b/pypy/objspace/std/marshal_impl.py
--- a/pypy/objspace/std/marshal_impl.py
+++ b/pypy/objspace/std/marshal_impl.py
@@ -325,10 +325,10 @@
     # of building a list of tuples.
     w_dic = space.newdict()
     while 1:
-        w_key = u.get_w_obj(True)
+        w_key = u.get_w_obj(allow_null=True)
         if w_key is None:
             break
-        w_value = u.get_w_obj(False)
+        w_value = u.get_w_obj()
         space.setitem(w_dic, w_key, w_value)
     return w_dic
 register(TYPE_DICT, unmarshal_DictMulti)
@@ -364,7 +364,7 @@
 # so we no longer can handle it in interp_marshal.atom_strlist
 
 def unmarshal_str(u):
-    w_obj = u.get_w_obj(False)
+    w_obj = u.get_w_obj()
     try:
         return u.space.str_w(w_obj)
     except OperationError, e:
diff --git a/pypy/objspace/std/objecttype.py b/pypy/objspace/std/objecttype.py
--- a/pypy/objspace/std/objecttype.py
+++ b/pypy/objspace/std/objecttype.py
@@ -24,7 +24,12 @@
     return w_obj.getrepr(space, '%s object' % (classname,))
 
 def descr__str__(space, w_obj):
-    return space.repr(w_obj)
+    w_type = space.type(w_obj)
+    w_impl = w_type.lookup("__repr__")
+    if w_impl is None:
+        raise OperationError(space.w_TypeError,      # can it really occur?
+                             space.wrap("operand does not support unary str"))
+    return space.get_and_call_function(w_impl, w_obj)
 
 def descr__class__(space, w_obj):
     return space.type(w_obj)
diff --git a/pypy/objspace/std/test/test_methodcache.py b/pypy/objspace/std/test/test_methodcache.py
--- a/pypy/objspace/std/test/test_methodcache.py
+++ b/pypy/objspace/std/test/test_methodcache.py
@@ -88,30 +88,37 @@
   
     def test_many_names(self):
         import __pypy__
-        class A(object):
-            foo = 5
-            bar = 6
-            baz = 7
-            xyz = 8
-            stuff = 9
-            a = 10
-            foobar = 11
+        for j in range(20):
+            class A(object):
+                foo = 5
+                bar = 6
+                baz = 7
+                xyz = 8
+                stuff = 9
+                a = 10
+                foobar = 11
 
-        a = A()
-        names = [name for name in A.__dict__.keys()
-                      if not name.startswith('_')]
-        names.sort()
-        names_repeated = names * 10
-        result = []
-        __pypy__.reset_method_cache_counter()
-        for name in names_repeated:
-            result.append(getattr(a, name))
-        append_counter = __pypy__.method_cache_counter("append")
-        names_counters = [__pypy__.method_cache_counter(name)
-                          for name in names]
-        assert append_counter[0] >= 5 * len(names)
-        for name, count in zip(names, names_counters):
-            assert count[0] >= 5, str((name, count))
+            a = A()
+            names = [name for name in A.__dict__.keys()
+                          if not name.startswith('_')]
+            names.sort()
+            names_repeated = names * 10
+            result = []
+            __pypy__.reset_method_cache_counter()
+            for name in names_repeated:
+                result.append(getattr(a, name))
+            append_counter = __pypy__.method_cache_counter("append")
+            names_counters = [__pypy__.method_cache_counter(name)
+                              for name in names]
+            try:
+                assert append_counter[0] >= 10 * len(names) - 1
+                for name, count in zip(names, names_counters):
+                    assert count == (9, 1), str((name, count))
+                break
+            except AssertionError:
+                pass
+        else:
+            raise
 
     def test_mutating_bases(self):
         class C(object):
diff --git a/pypy/objspace/std/test/test_obj.py b/pypy/objspace/std/test/test_obj.py
--- a/pypy/objspace/std/test/test_obj.py
+++ b/pypy/objspace/std/test/test_obj.py
@@ -94,3 +94,11 @@
         #assert len(log) == 1
         #assert log[0].message.args == ("object.__init__() takes no parameters",)
         #assert type(log[0].message) is DeprecationWarning
+
+    def test_object_str(self):
+        # obscure case: __str__() must delegate to __repr__() without adding
+        # type checking on its own
+        class A(object):
+            def __repr__(self):
+                return 123456
+        assert A().__str__() == 123456
diff --git a/pypy/objspace/std/test/test_stringformat.py b/pypy/objspace/std/test/test_stringformat.py
--- a/pypy/objspace/std/test/test_stringformat.py
+++ b/pypy/objspace/std/test/test_stringformat.py
@@ -168,7 +168,7 @@
 
     def test_incomplete_format(self):
         raises(ValueError, '%'.__mod__, ((23,),))
-        raises(ValueError, '%('.__mod__, ({},))
+        raises((ValueError, TypeError), '%('.__mod__, ({},))
 
     def test_format_char(self):
         import sys
diff --git a/pypy/objspace/std/test/test_unicodeobject.py b/pypy/objspace/std/test/test_unicodeobject.py
--- a/pypy/objspace/std/test/test_unicodeobject.py
+++ b/pypy/objspace/std/test/test_unicodeobject.py
@@ -780,8 +780,22 @@
         assert type(s) is unicode
         assert s == u'\u1234'
 
+        # now the same with a new-style class...
+        class A(object):
+            def __init__(self, num):
+                self.num = num
+            def __str__(self):
+                return unichr(self.num)
+
+        s = '%s' % A(111)    # this is ASCII
+        assert type(s) is unicode
+        assert s == chr(111)
+
+        s = '%s' % A(0x1234)    # this is not ASCII
+        assert type(s) is unicode
+        assert s == u'\u1234'
+
     def test_formatting_unicode__str__2(self):
-        skip("this is completely insane")
         class A:
             def __str__(self):
                 return u'baz'
@@ -798,9 +812,22 @@
         s = '%s %s' % (a, b)
         assert s == u'baz bar'
 
+        skip("but this case here is completely insane")
         s = '%s %s' % (b, a)
         assert s == u'foo baz'
 
+    def test_formatting_unicode__str__3(self):
+        # "bah" is all I can say
+        class X(object):
+            def __repr__(self):
+                return u'\u1234'
+        '%s' % X()
+        #
+        class X(object):
+            def __str__(self):
+                return u'\u1234'
+        '%s' % X()
+
     def test_str_subclass(self):
         class Foo9(str):
             def __unicode__(self):
diff --git a/pypy/rlib/rcoroutine.py b/pypy/rlib/rcoroutine.py
deleted file mode 100644
--- a/pypy/rlib/rcoroutine.py
+++ /dev/null
@@ -1,357 +0,0 @@
-"""
-Basic Concept:
---------------
-
-All concurrency is expressed by some means of coroutines.
-This is the lowest possible exposable interface.
-
-A coroutine is a structure that controls a sequence
-of continuations in time. It contains a frame object
-that is a restartable stack chain. This frame object
-is updated on every switch.
-
-The frame can be None. Either the coroutine is not yet
-bound, or it is the current coroutine of some costate.
-See below. XXX rewrite a definition of these terms.
-
-There is always a notation of a "current" and a "last"
-coroutine. Current has no frame and represents the
-running program. last is needed to keep track of the
-coroutine that receives a new frame chain after a switch.
-
-A costate object holds last and current.
-There are different coroutine concepts existing in
-parallel, like plain interp-level coroutines and
-app-level structures like coroutines, greenlets and
-tasklets.
-Every concept is associated with its own costate object.
-This allows for peaceful co-existence of many concepts.
-The type of a switch is determined by the target's costate.
-"""
-
-import py; py.test.skip("fixme: rewrite using rlib.rstacklet")
-# XXX ^^^ the reason it is not done is that pypy.rlib.rcoroutine
-# plus pypy/module/_stackless look like faaaaaar too much code
-# to me :-(
-
-from pypy.rlib.rstack import yield_current_frame_to_caller
-from pypy.rlib.objectmodel import we_are_translated
-
-from pypy.interpreter.error import OperationError
-
-try:
-    from greenlet import greenlet
-    main_greenlet = greenlet.getcurrent()
-except (ImportError, ValueError):
-    def greenlet(*args, **kwargs):
-        raise NotImplementedError("need either greenlets or a translated version of pypy")
-
-class FrameChain(object):
-    """Greenlet-based emulation of the primitive rstack 'frames' of RPython"""
-
-    def __init__(self, thunk=None):
-        if thunk:
-            self.greenlet = greenlet(thunk)
-        else:
-            self.greenlet = greenlet.getcurrent()
-
-    def switch(self):
-        last = FrameChain()
-        return self.greenlet.switch(last)
-
-import sys, os
-
-def make_coroutine_classes(baseclass):
-    class BaseCoState(object):
-        def __init__(self):
-            self.current = self.main = None
-
-        def __repr__(self):
-            "NOT_RPYTHON"
-            # for debugging only
-            return '<%s current=%r>' % (self.__class__.__name__, self.current)
-
-        def update(self, new):
-            syncstate.leaving = self.current
-            syncstate.entering = new
-            self.current = new
-            frame, new.frame = new.frame, None
-            return frame
-
-
-    class CoState(BaseCoState):
-        def __init__(self):
-            BaseCoState.__init__(self)
-            self.current = self.main = Coroutine(self)
-
-    class CoroutineDamage(SystemError):
-        pass
-
-
-    class SyncState(object):
-        def __init__(self):
-            self.reset()
-
-        def reset(self):
-            self.default_costate = None
-            self.leaving = None
-            self.entering = None
-            self.things_to_do = False
-            self.temp_exc = None
-            self.to_delete = []
-
-        def switched(self, incoming_frame):
-            left = syncstate.leaving
-            entered = syncstate.entering
-            syncstate.leaving = syncstate.entering = None
-            if left is not None:   # mostly to work around an annotation problem;
-                                   # should not really be None
-                left.frame = incoming_frame
-                left.goodbye()
-            if entered is not None:
-                entered.hello()
-            if self.things_to_do:
-                self._do_things_to_do()
-
-        def push_exception(self, exc):
-            self.things_to_do = True
-            self.temp_exc = exc
-
-        def check_for_zombie(self, obj):
-            return obj in self.to_delete
-
-        def postpone_deletion(self, obj):
-            self.to_delete.append(obj)
-            self.things_to_do = True
-
-        def _do_things_to_do(self):
-            if self.temp_exc is not None:
-                # somebody left an unhandled exception and switched to us.
-                # this both provides default exception handling and the
-                # way to inject an exception, like CoroutineExit.
-                e, self.temp_exc = self.temp_exc, None
-                self.things_to_do = bool(self.to_delete)
-                raise e
-            while self.to_delete:
-                delete, self.to_delete = self.to_delete, []
-                for obj in delete:
-                    obj.parent = obj.costate.current
-                    obj._kill_finally()
-            else:
-                self.things_to_do = False
-
-        def _freeze_(self):
-            self.reset()
-            return False
-
-    syncstate = SyncState()
-
-
-    class CoroutineExit(SystemExit):
-        # XXX SystemExit's __init__ creates problems in bookkeeper.
-        def __init__(self):
-            pass
-
-    class AbstractThunk(object):
-        def call(self):
-            raise NotImplementedError("abstract base class")
-
-
-    class Coroutine(baseclass):
-        def __init__(self, state=None):
-            self.frame = None
-            if state is None:
-                state = self._get_default_costate()
-            self.costate = state
-            self.parent = None
-            self.thunk = None
-            self.coroutine_exit = False
-
-        def __repr__(self):
-            'NOT_RPYTHON'
-            # just for debugging
-            if hasattr(self, '__name__'):
-                return '' % (self.__name__, self.frame, self.thunk is not None)
-            else:
-                return '' % (self.frame, self.thunk is not None)
-
-        def _get_default_costate():
-            state = syncstate.default_costate
-            if state is None:
-                state = syncstate.default_costate = CoState()
-            return state
-        _get_default_costate = staticmethod(_get_default_costate)
-
-        def _get_default_parent(self):
-            return self.costate.current
-
-        def bind(self, thunk):
-            assert isinstance(thunk, AbstractThunk)
-            if self.frame is not None:
-                raise CoroutineDamage
-            if self.parent is None:
-                self.parent = self._get_default_parent()
-            assert self.parent is not None
-            self.thunk = thunk
-            if we_are_translated():
-                self.frame = self._bind()
-            else:
-                self.frame = self._greenlet_bind()
-
-        def _greenlet_bind(self):
-            weak = [self]
-            def _greenlet_execute(incoming_frame):
-                try:
-                    chain2go2next = weak[0]._execute(incoming_frame)
-                except:
-                    # no exception is supposed to get out of _execute()
-                    # better report it directly into the main greenlet then,
-                    # and hidden to prevent catching
-                    main_greenlet.throw(AssertionError(
-                        "unexpected exception out of Coroutine._execute()",
-                        *sys.exc_info()))
-                    assert 0
-                del weak[0]
-                greenlet.getcurrent().parent = chain2go2next.greenlet
-                return None   # as the result of the FrameChain.switch()
-            chain = FrameChain(_greenlet_execute)
-            return chain
-
-        def _bind(self):
-            state = self.costate
-            incoming_frame = yield_current_frame_to_caller()
-            self = state.current
-            return self._execute(incoming_frame)
-
-        def _execute(self, incoming_frame):
-            state = self.costate
-            try:
-                try:
-                    try:
-                        exc = None
-                        thunk = self.thunk
-                        self.thunk = None
-                        syncstate.switched(incoming_frame)
-                        thunk.call()
-                    except Exception, e:
-                        exc = e
-                        raise
-                finally:
-                    # warning! we must reload the 'self' from the costate,
-                    # because after a clone() the 'self' of both copies
-                    # point to the original!
-                    self = state.current
-                    self.finish(exc)
-            except CoroutineExit:
-                pass
-            except Exception, e:
-                if self.coroutine_exit is False:
-                    # redirect all unhandled exceptions to the parent
-                    syncstate.push_exception(e)
-
-            while self.parent is not None and self.parent.frame is None:
-                # greenlet behavior is fine
-                self.parent = self.parent.parent
-            return state.update(self.parent)
-
-        def switch(self):
-            if self.frame is None:
-                # considered a programming error.
-                # greenlets and tasklets have different ideas about this.
-                raise CoroutineDamage
-            state = self.costate
-            incoming_frame = state.update(self).switch()
-            syncstate.switched(incoming_frame)
-
-        def kill(self):
-            self._kill(CoroutineExit())
-
-        def _kill(self, exc):
-            if self.frame is None:
-                return
-            state = self.costate
-            syncstate.push_exception(exc)
-            # careful here - if setting self.parent to state.current would
-            # create a loop, break it.  The assumption is that 'self'
-            # will die, so that state.current's chain of parents can be
-            # modified to skip 'self' without too many people noticing.
-            p = state.current
-            if p is self or self.parent is None:
-                pass  # killing the current of the main - don't change any parent
-            else:
-                while p.parent is not None:
-                    if p.parent is self:
-                        p.parent = self.parent
-                        break
-                    p = p.parent
-                self.parent = state.current
-            self.switch()
-
-        def _kill_finally(self):
-            try:
-                self._userdel()
-            except Exception:
-                pass # maybe print a warning?
-            self.kill()
-
-        __already_postponed = False
-    
-        def __del__(self):
-            # provide the necessary clean-up
-            # note that AppCoroutine has to take care about this
-            # as well, including a check for user-supplied __del__.
-            # Additionally note that in the context of __del__, we are
-            # not in the position to issue a switch.
-            # we defer it completely.
-            
-            # it is necessary to check whether syncstate is None because CPython
-            # sets it to None when it cleans up the modules, which will lead to
-            # very strange effects
-
-            if not we_are_translated():
-                # we need to make sure that we postpone each coroutine only once on
-                # top of CPython, because this resurrects the coroutine and CPython
-                # calls __del__ again, thus postponing and resurrecting the
-                # coroutine once more :-(
-                if self.__already_postponed:
-                    return
-                self.__already_postponed = True
-            if syncstate is not None:
-                syncstate.postpone_deletion(self)
-
-        # coroutines need complete control over their __del__ behaviour. In
-        # particular they need to care about calling space.userdel themselves
-        handle_del_manually = True
-
-        def _userdel(self):
-            # override this for exposed coros
-            pass
-
-        def is_alive(self):
-            return self.frame is not None or self is self.costate.current
-
-        def is_zombie(self):
-            return self.frame is not None and syncstate.check_for_zombie(self)
-
-        def getcurrent():
-            costate = Coroutine._get_default_costate()
-            return costate.current
-        getcurrent = staticmethod(getcurrent)
-
-        def getmain():
-            costate = Coroutine._get_default_costate()
-            return costate.main
-        getmain = staticmethod(getmain)
-
-        def hello(self):
-            "Called when execution is transferred into this coroutine."
-
-        def goodbye(self):
-            "Called just after execution is transferred away from this coroutine."
-
-        def finish(self, exc=None):
-            "stephan forgot me"
-
-    return locals()
-
-# _________________________________________________
diff --git a/pypy/rlib/test/test_rcoroutine.py b/pypy/rlib/test/test_rcoroutine.py
deleted file mode 100644
--- a/pypy/rlib/test/test_rcoroutine.py
+++ /dev/null
@@ -1,348 +0,0 @@
-"""
-testing coroutines at interprepter level
-"""
-import py
-import os
-from pypy import conftest; conftest.translation_test_so_skip_if_appdirect()
-from pypy.rlib.rcoroutine import make_coroutine_classes
-from pypy.translator.c.test.test_stackless import StacklessTest
-from pypy.translator.c import gc
-
-def setup_module(mod):
-    py.test.importorskip('greenlet')
-
-d = make_coroutine_classes(object)
-syncstate = d['syncstate']
-Coroutine = d['Coroutine']
-AbstractThunk = d['AbstractThunk']
-
-def output(stuff):
-    os.write(2, stuff + '\n')
-
-class _TestCoroutine(StacklessTest):
-    backendopt = True
-    Coroutine = Coroutine
-
-    def setup_method(self, method):
-        syncstate.reset()
-
-    def _freeze_(self):    # for 'self.Coroutine'
-        return True
-
-    def test_coroutine1(self):
-
-        def g(lst, coros):
-            coro_f, coro_g, coro_h = coros
-            lst.append(2)
-            output('g appended 2')
-            coro_h.switch()
-            lst.append(5)
-            output('g appended 5')
-
-        def h(lst, coros):
-            coro_f, coro_g, coro_h = coros
-            lst.append(3)
-            output('h appended 3')
-            coro_f.switch()
-            lst.append(7)
-            output('h appended 7')
-
-        class T(AbstractThunk):
-            def __init__(self, func, arg1, arg2):
-                self.func = func
-                self.arg1 = arg1
-                self.arg2 = arg2
-            def call(self):
-                self.func(self.arg1, self.arg2)
-
-        def f():
-            lst = [1]
-            coro_f = Coroutine.getcurrent()
-            coro_g = self.Coroutine()
-            coro_h = self.Coroutine()
-            coros = [coro_f, coro_g, coro_h]
-            thunk_g = T(g, lst, coros)
-            output('binding g after f set 1')
-            coro_g.bind(thunk_g)
-            thunk_h = T(h, lst, coros)
-            output('binding h after f set 1')
-            coro_h.bind(thunk_h)
-            output('switching to g')
-            coro_g.switch()
-            lst.append(4)
-            output('f appended 4')
-            coro_g.switch()
-            lst.append(6)
-            output('f appended 6')
-            coro_h.switch()
-            lst.append(8)
-            output('f appended 8')
-            n = 0
-            for i in lst:
-                n = n*10 + i
-            return n
-
-        data = self.wrap_stackless_function(f)
-        assert data == 12345678
-
-    def test_coroutine2(self):
-
-        class TBase(AbstractThunk):
-            def call(self):
-                pass
-
-        class T(TBase):
-            def __init__(self, func, arg1, arg2):
-                self.func = func
-                self.arg1 = arg1
-                self.arg2 = arg2
-            def call(self):
-                self.res = self.func(self.arg1, self.arg2)
-
-        class T1(TBase):
-            def __init__(self, func, arg1):
-                self.func = func
-                self.arg1 = arg1
-            def call(self):
-                self.res = self.func(self.arg1)
-
-        def g(lst, coros):
-            coro_f1, coro_g, coro_h = coros
-            lst.append(2)
-            output('g appended 2')
-            coro_h.switch()
-            lst.append(5)
-            output('g appended 5')
-            output('exiting g')
-
-        def h(lst, coros):
-            coro_f1, coro_g, coro_h = coros
-            lst.append(3)
-            output('h appended 3')
-            coro_f1.switch()
-            lst.append(7)
-            output('h appended 7')
-            output('exiting h')
-
-        def f1(coro_f1):
-            lst = [1]
-            coro_g = self.Coroutine()
-            coro_g.__name__ = 'coro_g'
-            coro_h = self.Coroutine()
-            coro_h.__name__ = 'coro_h'
-            coros = [coro_f1, coro_g, coro_h]
-            thunk_g = T(g, lst, coros)
-            output('binding g after f1 set 1')
-            coro_g.bind(thunk_g)
-            thunk_h = T(h, lst, coros)
-            output('binding h after f1 set 1')
-            coro_h.bind(thunk_h)
-            output('switching to g')
-            coro_g.switch()
-            lst.append(4)
-            output('f1 appended 4')
-            coro_g.switch()
-            lst.append(6)
-            output('f1 appended 6')
-            coro_h.switch()
-            lst.append(8)
-            output('f1 appended 8')
-            n = 0
-            for i in lst:
-                n = n*10 + i
-            output('exiting f1')
-            return n     
-
-        def f():
-            coro_f = Coroutine.getcurrent()
-            coro_f.__name__ = 'coro_f'
-            coro_f1 = self.Coroutine()
-            coro_f1.__name__ = 'coro_f1'
-            thunk_f1 = T1(f1, coro_f1)
-            output('binding f1 after f set 1')
-            coro_f1.bind(thunk_f1)
-            coro_f1.switch()
-            output('return to main :-(')
-            return thunk_f1.res
-
-        data = self.wrap_stackless_function(f)
-        assert data == 12345678
-
-    def test_kill_raise_del_coro(self):
-        class T(AbstractThunk):
-            def __init__(self, func, arg):
-                self.func = func
-                self.arg = arg
-            def call(self):
-                self.func(self.arg, self)
-
-        def g(nrec, t, count=0):
-            t.count = count
-            if nrec < 0:
-                raise ValueError
-            if nrec:
-                g(nrec-1, t, count+1)
-            Coroutine.getmain().switch()
-
-        def f():
-            assert Coroutine.getmain().frame is None
-            coro_g = self.Coroutine()
-            coro_g.__name__ = 'coro_g'
-            thunk_g = T(g, 42)
-            coro_g.bind(thunk_g)
-            coro_g.switch()
-            res = thunk_g.count
-            res *= 10
-            res |= coro_g.frame is not None
-            # testing kill
-            coro_g.kill()
-            res *= 10
-            res |= coro_g.frame is None
-            coro_g = self.Coroutine()
-            # see what happens if we __del__
-            thunk_g = T(g, -42)
-            coro_g.bind(thunk_g)
-            try:
-                coro_g.switch()
-            except ValueError:
-                res += 500
-            return res
-
-        data = self.wrap_stackless_function(f)
-        assert data == 4711
-
-    def test_tree_compare(self):
-        class Node:
-            def __init__(self, value, left=None, right=None):
-                self.value = value
-                self.left = left
-                self.right = right
-            def __repr__(self):
-                return 'Node(%r, %r, %r)'%(self.value, self.left, self.right)
-
-        tree1 = Node(1, Node(2, Node(3)))
-        tree2 = Node(1, Node(3, Node(2)))
-        tree3 = Node(1, Node(2), Node(3))
-
-        class Producer(AbstractThunk):
-            def __init__(self, tree, objects, consumer):
-                self.tree = tree
-                self.objects = objects
-                self.consumer = consumer
-            def produce(self, t):
-                if t is None:
-                    return
-                self.objects.append(t.value)
-                self.consumer.switch()
-                self.produce(t.left)
-                self.produce(t.right)
-            def call(self):
-                self.produce(self.tree)
-                while 1:
-                    self.consumer.switch()
-        class Consumer(AbstractThunk):
-            def __init__(self, tree, objects, producer):
-                self.tree = tree
-                self.objects = objects
-                self.producer = producer
-            def consume(self, t):
-                if t is None:
-                    return True
-                self.producer.switch()
-                if not self.objects:
-                    return False
-                if self.objects.pop(0) != t.value:
-                    return False
-                if not self.consume(t.left):
-                    return False
-                return self.consume(t.right)
-
-            def call(self):
-                self.result = self.consume(self.tree)
-                Coroutine.getmain().switch()
-
-        def pre_order_eq(t1, t2):
-            objects = []
-            producer = self.Coroutine()
-            consumer = self.Coroutine()
-
-            producer.bind(Producer(t1, objects, consumer))
-            cons = Consumer(t2, objects, producer)
-            consumer.bind(cons)
-
-            consumer.switch()
-
-            return cons.result
-
-        def ep():
-            return int("%d%d%d%d"%(pre_order_eq(tree1, tree2),
-                                   pre_order_eq(tree1, tree1),
-                                   pre_order_eq(tree1, tree3),
-                                   pre_order_eq(tree2, tree1),
-                                   ))
-
-        output = self.wrap_stackless_function(ep)
-        assert output == int('0110')
-
-    def test_hello_goodbye(self):
-
-        class C(Coroutine):
-            n = 2
-            def __init__(self, n):
-                Coroutine.__init__(self)
-                self.n = n
-            def hello(self):
-                costate.hello_goodbye *= 10
-                costate.hello_goodbye += self.n
-            def goodbye(self):
-                costate.hello_goodbye *= 10
-                costate.hello_goodbye += self.n + 1
-
-        class T(AbstractThunk):
-            def call(self):
-                pass
-
-        costate = Coroutine._get_default_costate()
-        costate.current.__class__ = C
-        costate.hello_goodbye = 0
-
-        def ep():
-            syncstate.default_costate = costate
-            costate.hello_goodbye = 0
-            c1 = C(4)
-            c1.bind(T())
-            c1.switch()
-            return costate.hello_goodbye
-
-        output = self.wrap_stackless_function(ep)
-        # expected result:
-        #   goodbye main   3
-        #   hello   c1     4
-        #   goodbye c1     5
-        #   hello   main   2
-        assert output == 3452
-
-    def test_raise_propagate(self):
-        class T(AbstractThunk):
-            def call(self):
-                raise ValueError
-
-        def ep():
-            c = self.Coroutine()
-            c.bind(T())
-            try:
-                c.switch()
-            except ValueError:
-                return 100
-            else:
-                return -5
-
-        output = self.wrap_stackless_function(ep)
-        assert output == 100
-
-
-TestCoroutine = _TestCoroutine # to activate
-class TestCoroutineOnCPython(_TestCoroutine):
-    def wrap_stackless_function(self, func):
-        return func()
-
diff --git a/pypy/rpython/module/ll_os_stat.py b/pypy/rpython/module/ll_os_stat.py
--- a/pypy/rpython/module/ll_os_stat.py
+++ b/pypy/rpython/module/ll_os_stat.py
@@ -49,19 +49,8 @@
     ]
 N_INDEXABLE_FIELDS = 10
 
-# for now, check the host Python to know which st_xxx fields exist
-STAT_FIELDS = [(_name, _TYPE) for (_name, _TYPE) in ALL_STAT_FIELDS
-                              if hasattr(os.stat_result, _name)]
-
-STAT_FIELD_TYPES = dict(STAT_FIELDS)      # {'st_xxx': TYPE}
-
-STAT_FIELD_NAMES = [_name for (_name, _TYPE) in ALL_STAT_FIELDS
-                          if _name in STAT_FIELD_TYPES]
-
-del _name, _TYPE
-
 # For OO backends, expose only the portable fields (the first 10).
-PORTABLE_STAT_FIELDS = STAT_FIELDS[:N_INDEXABLE_FIELDS]
+PORTABLE_STAT_FIELDS = ALL_STAT_FIELDS[:N_INDEXABLE_FIELDS]
 
 # ____________________________________________________________
 #
@@ -142,17 +131,22 @@
     includes = INCLUDES
 )
 
-if sys.platform != 'win32':
+if TIMESPEC is not None:
+    class CConfig_for_timespec:
+        _compilation_info_ = compilation_info
+        TIMESPEC = TIMESPEC
+    TIMESPEC = lltype.Ptr(
+        platform.configure(CConfig_for_timespec)['TIMESPEC'])
+
+
+def posix_declaration(try_to_add=None):
+    global STAT_STRUCT
 
     LL_STAT_FIELDS = STAT_FIELDS[:]
+    if try_to_add:
+        LL_STAT_FIELDS.append(try_to_add)
 
     if TIMESPEC is not None:
-        class CConfig_for_timespec:
-            _compilation_info_ = compilation_info
-            TIMESPEC = TIMESPEC
-
-        TIMESPEC = lltype.Ptr(
-            platform.configure(CConfig_for_timespec)['TIMESPEC'])
 
         def _expand(lst, originalname, timespecname):
             for i, (_name, _TYPE) in enumerate(lst):
@@ -178,9 +172,34 @@
     class CConfig:
         _compilation_info_ = compilation_info
         STAT_STRUCT = platform.Struct('struct %s' % _name_struct_stat, LL_STAT_FIELDS)
-    config = platform.configure(CConfig)
+    try:
+        config = platform.configure(CConfig)
+    except platform.CompilationError:
+        if try_to_add:
+            return    # failed to add this field, give up
+        raise
 
     STAT_STRUCT = lltype.Ptr(config['STAT_STRUCT'])
+    if try_to_add:
+        STAT_FIELDS.append(try_to_add)
+
+
+# This lists only the fields that have been found on the underlying platform.
+# Initially only the PORTABLE_STAT_FIELDS, but more may be added by the
+# following loop.
+STAT_FIELDS = PORTABLE_STAT_FIELDS[:]
+
+if sys.platform != 'win32':
+    posix_declaration()
+    for _i in range(len(PORTABLE_STAT_FIELDS), len(ALL_STAT_FIELDS)):
+        posix_declaration(ALL_STAT_FIELDS[_i])
+    del _i
+
+# these two global vars only list the fields defined in the underlying platform
+STAT_FIELD_TYPES = dict(STAT_FIELDS)      # {'st_xxx': TYPE}
+STAT_FIELD_NAMES = [_name for (_name, _TYPE) in STAT_FIELDS]
+del _name, _TYPE
+
 
 def build_stat_result(st):
     # only for LL backends
diff --git a/pypy/rpython/module/test/test_ll_os_stat.py b/pypy/rpython/module/test/test_ll_os_stat.py
--- a/pypy/rpython/module/test/test_ll_os_stat.py
+++ b/pypy/rpython/module/test/test_ll_os_stat.py
@@ -2,6 +2,16 @@
 import sys, os
 import py
 
+
+class TestLinuxImplementation:
+    def setup_class(cls):
+        if not sys.platform.startswith('linux'):
+            py.test.skip("linux specific tests")
+
+    def test_has_all_fields(self):
+        assert ll_os_stat.STAT_FIELDS == ll_os_stat.ALL_STAT_FIELDS[:13]
+
+
 class TestWin32Implementation:
     def setup_class(cls):
         if sys.platform != 'win32':
diff --git a/pypy/tool/jitlogparser/parser.py b/pypy/tool/jitlogparser/parser.py
--- a/pypy/tool/jitlogparser/parser.py
+++ b/pypy/tool/jitlogparser/parser.py
@@ -8,6 +8,7 @@
     bridge = None
     offset = None
     asm = None
+    failargs = ()
 
     def __init__(self, name, args, res, descr):
         self.name = name
@@ -18,8 +19,8 @@
         if self._is_guard:
             self.guard_no = int(self.descr[len(' " + arg)
-        if len(argv) > 2 and argv[1] == '--heapsize':
-            # Undocumented option, handled at interp-level.
-            # It has silently no effect with some GCs.
-            # It works in Boehm and in the semispace or generational GCs
-            # (but see comments in semispace.py:set_max_heap_size()).
-            # At the moment this option exists mainly to support sandboxing.
-            from pypy.rlib import rgc
-            rgc.set_max_heap_size(int(argv[2]))
-            argv = argv[:1] + argv[3:]
-        try:
-            try:
-                space.timer.start("space.startup")
-                space.call_function(w_run_toplevel, w_call_startup_gateway)
-                space.timer.stop("space.startup")
-                w_executable = space.wrap(argv[0])
-                w_argv = space.newlist([space.wrap(s) for s in argv[1:]])
-                space.timer.start("w_entry_point")
-                w_exitcode = space.call_function(w_entry_point, w_executable, w_argv, w_os)
-                space.timer.stop("w_entry_point")
-                exitcode = space.int_w(w_exitcode)
-                # try to pull it all in
-            ##    from pypy.interpreter import main, interactive, error
-            ##    con = interactive.PyPyConsole(space)
-            ##    con.interact()
-            except OperationError, e:
-                debug("OperationError:")
-                debug(" operror-type: " + e.w_type.getname(space))
-                debug(" operror-value: " + space.str_w(space.str(e.get_w_value(space))))
-                return 1
-        finally:
-            try:
-                space.timer.start("space.finish")
-                space.call_function(w_run_toplevel, w_call_finish_gateway)
-                space.timer.stop("space.finish")
-            except OperationError, e:
-                debug("OperationError:")
-                debug(" operror-type: " + e.w_type.getname(space))
-                debug(" operror-value: " + space.str_w(space.str(e.get_w_value(space))))
-                return 1
-        space.timer.stop("Entrypoint")
-        space.timer.dump()
-        return exitcode
-    return entry_point
-
-def call_finish(space):
-    space.finish()
-
-def call_startup(space):
-    space.startup()
-
-# _____ Define and setup target ___
-
-# for now this will do for option handling
-
-class PyPyTarget(object):
-
-    usage = SUPPRESS_USAGE
-
-    take_options = True
-
-    def opt_parser(self, config):
-        parser = to_optparse(config, useoptions=["objspace.*"],
-                             parserkwargs={'usage': self.usage})
-        return parser
-
-    def handle_config(self, config, translateconfig):
-        self.translateconfig = translateconfig
-        # set up the objspace optimizations based on the --opt argument
-        from pypy.config.pypyoption import set_pypy_opt_level
-        set_pypy_opt_level(config, translateconfig.opt)
-
-        # as of revision 27081, multimethod.py uses the InstallerVersion1 by default
-        # because it is much faster both to initialize and run on top of CPython.
-        # The InstallerVersion2 is optimized for making a translator-friendly
-        # structure for low level backends. However, InstallerVersion1 is still
-        # preferable for high level backends, so we patch here.
-
-        from pypy.objspace.std import multimethod
-        if config.objspace.std.multimethods == 'mrd':
-            assert multimethod.InstallerVersion1.instance_counter == 0,\
-                   'The wrong Installer version has already been instatiated'
-            multimethod.Installer = multimethod.InstallerVersion2
-        elif config.objspace.std.multimethods == 'doubledispatch':
-            # don't rely on the default, set again here
-            assert multimethod.InstallerVersion2.instance_counter == 0,\
-                   'The wrong Installer version has already been instatiated'
-            multimethod.Installer = multimethod.InstallerVersion1
-
-    def print_help(self, config):
-        self.opt_parser(config).print_help()
-
-    def get_additional_config_options(self):
-        from pypy.config.pypyoption import pypy_optiondescription
-        return pypy_optiondescription
-
-    def target(self, driver, args):
-        driver.exe_name = 'pypy-%(backend)s'
-
-        config = driver.config
-        parser = self.opt_parser(config)
-
-        parser.parse_args(args)
-
-        # expose the following variables to ease debugging
-        global space, entry_point
-
-        if config.objspace.allworkingmodules:
-            from pypy.config.pypyoption import enable_allworkingmodules
-            enable_allworkingmodules(config)
-
-        if config.translation.thread:
-            config.objspace.usemodules.thread = True
-        elif config.objspace.usemodules.thread:
-            try:
-                config.translation.thread = True
-            except ConflictConfigError:
-                # If --allworkingmodules is given, we reach this point
-                # if threads cannot be enabled (e.g. they conflict with
-                # something else).  In this case, we can try setting the
-                # usemodules.thread option to False again.  It will
-                # cleanly fail if that option was set to True by the
-                # command-line directly instead of via --allworkingmodules.
-                config.objspace.usemodules.thread = False
-
-        if config.translation.stackless:
-            config.objspace.usemodules._stackless = True
-        elif config.objspace.usemodules._stackless:
-            try:
-                config.translation.stackless = True
-            except ConflictConfigError:
-                raise ConflictConfigError("please use the --stackless option "
-                                          "to translate.py instead of "
-                                          "--withmod-_stackless directly")
-
-        if not config.translation.rweakref:
-            config.objspace.usemodules._weakref = False
-
-        if self.translateconfig.goal_options.jit:
-            config.objspace.usemodules.pypyjit = True
-        elif config.objspace.usemodules.pypyjit:
-            self.translateconfig.goal_options.jit = True
-
-        if config.translation.backend == "cli":
-            config.objspace.usemodules.clr = True
-        # XXX did it ever work?
-        #elif config.objspace.usemodules.clr:
-        #    config.translation.backend == "cli"
-
-        config.objspace.nofaking = True
-        config.objspace.compiler = "ast"
-        config.translating = True
-
-        import translate
-        translate.log_config(config.objspace, "PyPy config object")
- 
-        # obscure hack to stuff the translation options into the translated PyPy
-        import pypy.module.sys
-        options = make_dict(config)
-        wrapstr = 'space.wrap(%r)' % (options)
-        pypy.module.sys.Module.interpleveldefs['pypy_translation_info'] = wrapstr
-
-        return self.get_entry_point(config)
-
-    def portal(self, driver):
-        from pypy.module.pypyjit.portal import get_portal
-        return get_portal(driver)
-    
-    def get_entry_point(self, config):
-        space = make_objspace(config)
-
-        # manually imports app_main.py
-        filename = os.path.join(this_dir, 'app_main.py')
-        w_dict = space.newdict()
-        space.exec_(open(filename).read(), w_dict, w_dict)
-        for modulename in EXTRA_MODULES:
-            print 'pre-importing', modulename
-            space.exec_("import " + modulename, w_dict, w_dict)
-        print 'phew, ready'
-        entry_point = create_entry_point(space, w_dict)
-
-        return entry_point, None, PyPyAnnotatorPolicy(single_space = space)
-
-    def interface(self, ns):
-        for name in ['take_options', 'handle_config', 'print_help', 'target',
-                     'portal',
-                     'get_additional_config_options']:
-            ns[name] = getattr(self, name)
-
-
-PyPyTarget().interface(globals())
-

From noreply at buildbot.pypy.org  Wed Sep  7 12:58:58 2011
From: noreply at buildbot.pypy.org (arigo)
Date: Wed,  7 Sep 2011 12:58:58 +0200 (CEST)
Subject: [pypy-commit] pypy default: OS/X fix.
Message-ID: <20110907105858.4BE1982213@wyvern.cs.uni-duesseldorf.de>

Author: Armin Rigo 
Branch: 
Changeset: r47129:e556f833d5d4
Date: 2011-09-07 12:58 +0200
http://bitbucket.org/pypy/pypy/changeset/e556f833d5d4/

Log:	OS/X fix.

diff --git a/pypy/module/pwd/test/test_pwd.py b/pypy/module/pwd/test/test_pwd.py
--- a/pypy/module/pwd/test/test_pwd.py
+++ b/pypy/module/pwd/test/test_pwd.py
@@ -5,14 +5,17 @@
         cls.space = gettestobjspace(usemodules=['pwd'])
 
     def test_getpwuid(self):
-        import pwd
+        import pwd, sys
         raises(KeyError, pwd.getpwuid, -1)
         pw = pwd.getpwuid(0)
         assert pw.pw_name == 'root'
         assert isinstance(pw.pw_passwd, str)
         assert pw.pw_uid == 0
         assert pw.pw_gid == 0
-        assert pw.pw_dir == '/root'
+        if sys.platform.startswith('linux'):
+            assert pw.pw_dir == '/root'
+        else:
+            assert pw.pw_dir.startswith('/')
         assert pw.pw_shell.startswith('/')
         #
         assert type(pw.pw_uid) is int

From noreply at buildbot.pypy.org  Wed Sep  7 13:05:47 2011
From: noreply at buildbot.pypy.org (arigo)
Date: Wed,  7 Sep 2011 13:05:47 +0200 (CEST)
Subject: [pypy-commit] pypy default: Fix tests on OS/X (hopefully).
Message-ID: <20110907110547.8A1B082213@wyvern.cs.uni-duesseldorf.de>

Author: Armin Rigo 
Branch: 
Changeset: r47130:5190d760b26c
Date: 2011-09-07 13:05 +0200
http://bitbucket.org/pypy/pypy/changeset/5190d760b26c/

Log:	Fix tests on OS/X (hopefully).

diff --git a/pypy/module/pypyjit/test_pypy_c/test__ffi.py b/pypy/module/pypyjit/test_pypy_c/test__ffi.py
--- a/pypy/module/pypyjit/test_pypy_c/test__ffi.py
+++ b/pypy/module/pypyjit/test_pypy_c/test__ffi.py
@@ -29,11 +29,13 @@
         pow_addr, res = log.result
         assert res == 8.0 * 300
         loop, = log.loops_by_filename(self.filepath)
+        if 'ConstClass(pow)' in repr(loop):   # e.g. OS/X
+            pow_addr = 'ConstClass(pow)'
         assert loop.match_by_id('fficall', """
             guard_not_invalidated(descr=...)
             i17 = force_token()
             setfield_gc(p0, i17, descr=<.* .*PyFrame.vable_token .*>)
-            f21 = call_release_gil(%d, 2.000000, 3.000000, descr=)
+            f21 = call_release_gil(%s, 2.000000, 3.000000, descr=)
             guard_not_forced(descr=...)
             guard_no_exception(descr=...)
         """ % pow_addr)
@@ -129,4 +131,5 @@
         assert opnames.count('call_release_gil') == 1
         idx = opnames.index('call_release_gil')
         call = ops[idx]
-        assert int(call.args[0]) == fabs_addr
+        assert (call.args[0] == 'ConstClass(fabs)' or    # e.g. OS/X
+                int(call.args[0]) == fabs_addr)

From noreply at buildbot.pypy.org  Wed Sep  7 14:55:50 2011
From: noreply at buildbot.pypy.org (antocuni)
Date: Wed,  7 Sep 2011 14:55:50 +0200 (CEST)
Subject: [pypy-commit] pypy ffistruct: move _ffi.Field and most of
 compute_shape to interp-level
Message-ID: <20110907125550.76FC282213@wyvern.cs.uni-duesseldorf.de>

Author: Antonio Cuni 
Branch: ffistruct
Changeset: r47131:f348de8d9bd3
Date: 2011-09-07 09:43 +0200
http://bitbucket.org/pypy/pypy/changeset/f348de8d9bd3/

Log:	move _ffi.Field and most of compute_shape to interp-level

diff --git a/pypy/module/_ffi/__init__.py b/pypy/module/_ffi/__init__.py
--- a/pypy/module/_ffi/__init__.py
+++ b/pypy/module/_ffi/__init__.py
@@ -8,9 +8,9 @@
         'FuncPtr': 'interp_funcptr.W_FuncPtr',
         'get_libc':'interp_funcptr.get_libc',
         '_StructDescr': 'interp_struct.W__StructDescr',
+        'Field':     'interp_struct.W_Field',
     }
 
     appleveldefs = {
         'Structure': 'app_struct.Structure',
-        'Field':     'app_struct.Field',
         }
diff --git a/pypy/module/_ffi/app_struct.py b/pypy/module/_ffi/app_struct.py
--- a/pypy/module/_ffi/app_struct.py
+++ b/pypy/module/_ffi/app_struct.py
@@ -1,20 +1,5 @@
 import _ffi
 
-class Field(object):
-
-    def __init__(self, name, ffitype):
-        self.name = name
-        self.ffitype = ffitype
-        self.offset = -1
-
-    ## def __get__(self, obj, cls=None):
-    ##     if obj is None:
-    ##         return self
-    ##     return getfield(obj._buffer, self.ffitype, self.offset)
-
-    ## def __set__(self, obj, value):
-    ##     setfield(obj._buffer, self.ffitype, self.offset, value)
-
 class MetaStructure(type):
 
     def __new__(cls, name, bases, dic):
@@ -26,15 +11,9 @@
         fields = dic.get('_fields_')
         if fields is None:
             return
-        size = 0
-        ffitypes = []
+        struct_descr = _ffi._StructDescr(name, fields)
         for field in fields:
-            field.offset = size # XXX: alignment!
-            size += field.ffitype.sizeof()
-            ffitypes.append(field.ffitype)
             dic[field.name] = field
-        alignment = 0 # XXX
-        struct_descr = _ffi._StructDescr(name, size, alignment, ffitypes)
         dic['_struct_'] = struct_descr
 
 
diff --git a/pypy/module/_ffi/test/test_struct.py b/pypy/module/_ffi/test/test_struct.py
--- a/pypy/module/_ffi/test/test_struct.py
+++ b/pypy/module/_ffi/test/test_struct.py
@@ -3,9 +3,13 @@
 class AppTestStruct(BaseAppTestFFI):
 
     def test__StructDescr(self):
-        from _ffi import _StructDescr, types
+        from _ffi import _StructDescr, Field, types
         longsize = types.slong.sizeof()
-        descr = _StructDescr('foo', longsize*2, 0, [types.slong, types.slong])
+        fields = [
+            Field('x', types.slong),
+            Field('y', types.slong),
+            ]
+        descr = _StructDescr('foo', fields)
         assert descr.ffitype.sizeof() == longsize*2
         assert descr.ffitype.name == 'struct foo'
 
@@ -24,4 +28,3 @@
         assert Point.y.offset == longsize
         assert Point._struct_.ffitype.sizeof() == longsize*2
         assert Point._struct_.ffitype.name == 'struct Point'
-        

From noreply at buildbot.pypy.org  Wed Sep  7 14:55:51 2011
From: noreply at buildbot.pypy.org (antocuni)
Date: Wed,  7 Sep 2011 14:55:51 +0200 (CEST)
Subject: [pypy-commit] pypy ffistruct: bah,
	forgot to add this file in the previous checkin
Message-ID: <20110907125551.AC51E82213@wyvern.cs.uni-duesseldorf.de>

Author: Antonio Cuni 
Branch: ffistruct
Changeset: r47132:bbd9a00c7369
Date: 2011-09-07 09:46 +0200
http://bitbucket.org/pypy/pypy/changeset/bbd9a00c7369/

Log:	bah, forgot to add this file in the previous checkin

diff --git a/pypy/module/_ffi/interp_struct.py b/pypy/module/_ffi/interp_struct.py
new file mode 100644
--- /dev/null
+++ b/pypy/module/_ffi/interp_struct.py
@@ -0,0 +1,66 @@
+from pypy.rpython.lltypesystem import lltype
+from pypy.rlib import clibffi
+from pypy.interpreter.baseobjspace import Wrappable
+from pypy.interpreter.typedef import TypeDef, interp_attrproperty
+from pypy.interpreter.gateway import interp2app, unwrap_spec
+from pypy.objspace.std.typetype import type_typedef
+from pypy.module._ffi.interp_ffitype import W_FFIType
+
+class W_Field(Wrappable):
+
+    def __init__(self, name, w_ffitype):
+        self.name = name
+        self.w_ffitype = w_ffitype
+        self.offset = -1
+
+    @staticmethod
+    @unwrap_spec(name=str)
+    def descr_new(space, w_type, name, w_ffitype):
+        w_ffitype = space.interp_w(W_FFIType, w_ffitype)
+        return W_Field(name, w_ffitype)
+
+W_Field.typedef = TypeDef(
+    'Field',
+    __new__ = interp2app(W_Field.descr_new),
+    name = interp_attrproperty('name', W_Field),
+    ffitype = interp_attrproperty('w_ffitype', W_Field),
+    offset = interp_attrproperty('offset', W_Field),
+    )
+
+
+# ==============================================================================
+
+
+class W__StructDescr(Wrappable):
+
+    def __init__(self, name, ffistruct):
+        self.ffistruct = ffistruct
+        self.ffitype = W_FFIType('struct %s' % name, ffistruct.ffistruct, 'fixme')
+
+    @staticmethod
+    @unwrap_spec(name=str)
+    def descr_new(space, w_type, name, w_fields):
+        size = 0
+        alignment = 0 # XXX
+        fields_w = space.fixedview(w_fields)
+        field_types = []
+        for w_field in fields_w:
+            w_field = space.interp_w(W_Field, w_field)
+            w_field.offset = size # XXX: alignment!
+            size += w_field.w_ffitype.sizeof()
+            field_types.append(w_field.w_ffitype.ffitype)
+        #
+        ffistruct = clibffi.make_struct_ffitype_e(size, alignment, field_types)
+        return W__StructDescr(name, ffistruct)
+
+    def __del__(self):
+        if self.ffistruct:
+            lltype.free(self.ffistruct, flavor='raw')
+
+
+W__StructDescr.typedef = TypeDef(
+    '_StructDescr',
+    __new__ = interp2app(W__StructDescr.descr_new),
+    ffitype = interp_attrproperty('ffitype', W__StructDescr),
+    )
+

From noreply at buildbot.pypy.org  Wed Sep  7 14:55:52 2011
From: noreply at buildbot.pypy.org (antocuni)
Date: Wed,  7 Sep 2011 14:55:52 +0200 (CEST)
Subject: [pypy-commit] pypy ffistruct: make it possible to allocate a struct,
 and set/get fields on it.  The only supported type is 'long' so far
Message-ID: <20110907125552.E3BF582213@wyvern.cs.uni-duesseldorf.de>

Author: Antonio Cuni 
Branch: ffistruct
Changeset: r47133:63d9d0f04f6a
Date: 2011-09-07 11:48 +0200
http://bitbucket.org/pypy/pypy/changeset/63d9d0f04f6a/

Log:	make it possible to allocate a struct, and set/get fields on it.
	The only supported type is 'long' so far

diff --git a/pypy/module/_ffi/interp_struct.py b/pypy/module/_ffi/interp_struct.py
--- a/pypy/module/_ffi/interp_struct.py
+++ b/pypy/module/_ffi/interp_struct.py
@@ -1,10 +1,10 @@
-from pypy.rpython.lltypesystem import lltype
+from pypy.rpython.lltypesystem import lltype, rffi
 from pypy.rlib import clibffi
 from pypy.interpreter.baseobjspace import Wrappable
 from pypy.interpreter.typedef import TypeDef, interp_attrproperty
 from pypy.interpreter.gateway import interp2app, unwrap_spec
 from pypy.objspace.std.typetype import type_typedef
-from pypy.module._ffi.interp_ffitype import W_FFIType
+from pypy.module._ffi.interp_ffitype import W_FFIType, app_types
 
 class W_Field(Wrappable):
 
@@ -30,12 +30,15 @@
 
 # ==============================================================================
 
-
 class W__StructDescr(Wrappable):
 
-    def __init__(self, name, ffistruct):
+    def __init__(self, name, fields_w, ffistruct):
         self.ffistruct = ffistruct
-        self.ffitype = W_FFIType('struct %s' % name, ffistruct.ffistruct, 'fixme')
+        self.w_ffitype = W_FFIType('struct %s' % name, ffistruct.ffistruct, 'fixme')
+        self.fields_w = fields_w
+        self.name2w_field = {}
+        for w_field in fields_w:
+            self.name2w_field[w_field.name] = w_field
 
     @staticmethod
     @unwrap_spec(name=str)
@@ -51,7 +54,15 @@
             field_types.append(w_field.w_ffitype.ffitype)
         #
         ffistruct = clibffi.make_struct_ffitype_e(size, alignment, field_types)
-        return W__StructDescr(name, ffistruct)
+        return W__StructDescr(name, fields_w, ffistruct)
+
+    def allocate(self, space):
+        return W__StructInstance(self)
+
+    #@jit.elidable...
+    def get_type_and_offset_for_field(self, name):
+        w_field = self.name2w_field[name]
+        return w_field.w_ffitype, w_field.offset
 
     def __del__(self):
         if self.ffistruct:
@@ -61,6 +72,60 @@
 W__StructDescr.typedef = TypeDef(
     '_StructDescr',
     __new__ = interp2app(W__StructDescr.descr_new),
-    ffitype = interp_attrproperty('ffitype', W__StructDescr),
+    ffitype = interp_attrproperty('w_ffitype', W__StructDescr),
+    allocate = interp2app(W__StructDescr.allocate),
     )
 
+
+# ==============================================================================
+
+class W__StructInstance(Wrappable):
+
+    _immutable_fields_ = ['structdescr', 'rawmem']
+
+    def __init__(self, structdescr):
+        self.structdescr = structdescr
+        size = structdescr.w_ffitype.sizeof()
+        self.rawmem = lltype.malloc(rffi.VOIDP.TO, size, flavor='raw',
+                                    zero=True, add_memory_pressure=True)
+
+    def __del__(self):
+        if self.rawmem:
+            lltype.free(self.rawmem, flavor='raw')
+            self.rawmem = lltype.nullptr(rffi.VOIDP.TO)
+
+    def getaddr(self, space):
+        addr = rffi.cast(rffi.ULONG, self.rawmem)
+        return space.wrap(addr)
+
+    @unwrap_spec(name=str)
+    def getfield(self, space, name):
+        w_ffitype, offset = self.structdescr.get_type_and_offset_for_field(name)
+        assert w_ffitype is app_types.slong # XXX: handle all cases
+        FIELD_TYPE  = rffi.LONG
+        #
+        addr = rffi.ptradd(self.rawmem, offset)
+        PTR_FIELD = lltype.Ptr(rffi.CArray(FIELD_TYPE))
+        value = rffi.cast(PTR_FIELD, addr)[0]
+        #
+        return space.wrap(value)
+
+    @unwrap_spec(name=str)
+    def setfield(self, space, name, w_value):
+        w_ffitype, offset = self.structdescr.get_type_and_offset_for_field(name)
+        assert w_ffitype is app_types.slong # XXX: handle all cases
+        FIELD_TYPE  = rffi.LONG
+        value = space.int_w(w_value)
+        #
+        addr = rffi.ptradd(self.rawmem, offset)
+        PTR_FIELD = lltype.Ptr(rffi.CArray(FIELD_TYPE))
+        rffi.cast(PTR_FIELD, addr)[0] = value
+
+
+
+W__StructInstance.typedef = TypeDef(
+    '_StructInstance',
+    getaddr  = interp2app(W__StructInstance.getaddr),
+    getfield = interp2app(W__StructInstance.getfield),
+    setfield = interp2app(W__StructInstance.setfield),
+    )
diff --git a/pypy/module/_ffi/test/test_struct.py b/pypy/module/_ffi/test/test_struct.py
--- a/pypy/module/_ffi/test/test_struct.py
+++ b/pypy/module/_ffi/test/test_struct.py
@@ -2,6 +2,20 @@
 
 class AppTestStruct(BaseAppTestFFI):
 
+    def setup_class(cls):
+        BaseAppTestFFI.setup_class.im_func(cls)
+        #
+        def read_raw_mem(self, addr, typename, length):
+            import ctypes
+            addr = ctypes.cast(addr, ctypes.c_void_p)
+            c_type = getattr(ctypes, typename)
+            array_type = ctypes.POINTER(c_type * length)
+            ptr_array = ctypes.cast(addr, array_type)
+            array = ptr_array[0]
+            lst = [array[i] for i in range(length)]
+            return lst
+        cls.w_read_raw_mem = cls.space.wrap(read_raw_mem)
+        
     def test__StructDescr(self):
         from _ffi import _StructDescr, Field, types
         longsize = types.slong.sizeof()
@@ -28,3 +42,19 @@
         assert Point.y.offset == longsize
         assert Point._struct_.ffitype.sizeof() == longsize*2
         assert Point._struct_.ffitype.name == 'struct Point'
+
+    def test_getfield_setfield(self):
+        from _ffi import _StructDescr, Field, types
+        longsize = types.slong.sizeof()
+        fields = [
+            Field('x', types.slong),
+            Field('y', types.slong),
+            ]
+        descr = _StructDescr('foo', fields)
+        struct = descr.allocate()
+        struct.setfield('x', 42)
+        struct.setfield('y', 43)
+        assert struct.getfield('x') == 42
+        assert struct.getfield('y') == 43
+        mem = self.read_raw_mem(struct.getaddr(), 'c_long', 2)
+        assert mem == [42, 43]

From noreply at buildbot.pypy.org  Wed Sep  7 14:55:54 2011
From: noreply at buildbot.pypy.org (antocuni)
Date: Wed,  7 Sep 2011 14:55:54 +0200 (CEST)
Subject: [pypy-commit] pypy ffistruct: move the code to get/set arbitrary
 struct fields to rlib.libffi. Eventually,
 these two functions will be recognized and optimized by the JIT
Message-ID: <20110907125554.24BF482213@wyvern.cs.uni-duesseldorf.de>

Author: Antonio Cuni 
Branch: ffistruct
Changeset: r47134:750b3e0a7cf1
Date: 2011-09-07 14:53 +0200
http://bitbucket.org/pypy/pypy/changeset/750b3e0a7cf1/

Log:	move the code to get/set arbitrary struct fields to rlib.libffi.
	Eventually, these two functions will be recognized and optimized by
	the JIT

diff --git a/pypy/module/_ffi/interp_struct.py b/pypy/module/_ffi/interp_struct.py
--- a/pypy/module/_ffi/interp_struct.py
+++ b/pypy/module/_ffi/interp_struct.py
@@ -1,5 +1,6 @@
 from pypy.rpython.lltypesystem import lltype, rffi
 from pypy.rlib import clibffi
+from pypy.rlib import libffi
 from pypy.interpreter.baseobjspace import Wrappable
 from pypy.interpreter.typedef import TypeDef, interp_attrproperty
 from pypy.interpreter.gateway import interp2app, unwrap_spec
@@ -104,10 +105,7 @@
         assert w_ffitype is app_types.slong # XXX: handle all cases
         FIELD_TYPE  = rffi.LONG
         #
-        addr = rffi.ptradd(self.rawmem, offset)
-        PTR_FIELD = lltype.Ptr(rffi.CArray(FIELD_TYPE))
-        value = rffi.cast(PTR_FIELD, addr)[0]
-        #
+        value = libffi.struct_getfield(FIELD_TYPE, self.rawmem, offset)
         return space.wrap(value)
 
     @unwrap_spec(name=str)
@@ -117,9 +115,7 @@
         FIELD_TYPE  = rffi.LONG
         value = space.int_w(w_value)
         #
-        addr = rffi.ptradd(self.rawmem, offset)
-        PTR_FIELD = lltype.Ptr(rffi.CArray(FIELD_TYPE))
-        rffi.cast(PTR_FIELD, addr)[0] = value
+        libffi.struct_setfield(FIELD_TYPE, self.rawmem, offset, value)
 
 
 
diff --git a/pypy/rlib/libffi.py b/pypy/rlib/libffi.py
--- a/pypy/rlib/libffi.py
+++ b/pypy/rlib/libffi.py
@@ -410,3 +410,29 @@
 
     def getaddressindll(self, name):
         return dlsym(self.lib, name)
+
+
+# ======================================================================
+
+ at jit.dont_look_inside
+ at specialize.arg(0)
+def struct_getfield(TYPE, addr, offset):
+    """
+    Read the field of type TYPE at addr+offset.
+    addr is of type rffi.VOIDP, offset is an int.
+    """
+    addr = rffi.ptradd(addr, offset)
+    PTR_FIELD = lltype.Ptr(rffi.CArray(TYPE))
+    return rffi.cast(PTR_FIELD, addr)[0]
+
+
+ at jit.dont_look_inside
+ at specialize.arg(0)
+def struct_setfield(TYPE, addr, offset, value):
+    """
+    Read the field of type TYPE at addr+offset.
+    addr is of type rffi.VOIDP, offset is an int.
+    """
+    addr = rffi.ptradd(addr, offset)
+    PTR_FIELD = lltype.Ptr(rffi.CArray(TYPE))
+    rffi.cast(PTR_FIELD, addr)[0] = value
diff --git a/pypy/rlib/test/test_libffi.py b/pypy/rlib/test/test_libffi.py
--- a/pypy/rlib/test/test_libffi.py
+++ b/pypy/rlib/test/test_libffi.py
@@ -5,7 +5,7 @@
 from pypy.rlib.rarithmetic import r_singlefloat, r_longlong, r_ulonglong
 from pypy.rlib.test.test_clibffi import BaseFfiTest, get_libm_name, make_struct_ffitype_e
 from pypy.rlib.libffi import CDLL, Func, get_libc_name, ArgChain, types
-from pypy.rlib.libffi import IS_32_BIT
+from pypy.rlib.libffi import IS_32_BIT, struct_getfield, struct_setfield
 
 class TestLibffiMisc(BaseFfiTest):
 
@@ -52,6 +52,24 @@
         del lib
         assert not ALLOCATED
 
+    def test_struct_fields(self):
+        longsize = 4 if IS_32_BIT else 8
+        POINT = lltype.Struct('POINT', ('x', rffi.LONG), ('y', rffi.LONG))
+        p = lltype.malloc(POINT, flavor='raw')
+        p.x = 42
+        p.y = 43
+        addr = rffi.cast(rffi.VOIDP, p)
+        assert struct_getfield(rffi.LONG, addr, 0) == 42
+        assert struct_getfield(rffi.LONG, addr, longsize) == 43
+        #
+        struct_setfield(rffi.LONG, addr, 0, 123)
+        struct_setfield(rffi.LONG, addr, longsize, 321)
+        assert p.x == 123
+        assert p.y == 321
+        #
+        lltype.free(p, flavor='raw')
+        
+
 class TestLibffiCall(BaseFfiTest):
     """
     Test various kind of calls through libffi.

From noreply at buildbot.pypy.org  Wed Sep  7 14:55:55 2011
From: noreply at buildbot.pypy.org (antocuni)
Date: Wed,  7 Sep 2011 14:55:55 +0200 (CEST)
Subject: [pypy-commit] pypy ffistruct: disable this check for now,
	or else it won't translate
Message-ID: <20110907125555.5969982213@wyvern.cs.uni-duesseldorf.de>

Author: Antonio Cuni 
Branch: ffistruct
Changeset: r47135:55a06bcd9f06
Date: 2011-09-07 14:55 +0200
http://bitbucket.org/pypy/pypy/changeset/55a06bcd9f06/

Log:	disable this check for now, or else it won't translate

diff --git a/pypy/module/_ffi/interp_ffitype.py b/pypy/module/_ffi/interp_ffitype.py
--- a/pypy/module/_ffi/interp_ffitype.py
+++ b/pypy/module/_ffi/interp_ffitype.py
@@ -13,8 +13,9 @@
         self.ffitype = ffitype
         self.w_datashape = w_datashape
         self.w_pointer_to = w_pointer_to
-        if self.is_struct():
-            assert w_datashape is not None
+        ## XXX: re-enable this check when the ffistruct branch is done
+        ## if self.is_struct():
+        ##     assert w_datashape is not None
 
     def descr_deref_pointer(self, space):
         if self.w_pointer_to is None:
diff --git a/pypy/module/_ffi/interp_struct.py b/pypy/module/_ffi/interp_struct.py
--- a/pypy/module/_ffi/interp_struct.py
+++ b/pypy/module/_ffi/interp_struct.py
@@ -35,7 +35,7 @@
 
     def __init__(self, name, fields_w, ffistruct):
         self.ffistruct = ffistruct
-        self.w_ffitype = W_FFIType('struct %s' % name, ffistruct.ffistruct, 'fixme')
+        self.w_ffitype = W_FFIType('struct %s' % name, ffistruct.ffistruct, None)
         self.fields_w = fields_w
         self.name2w_field = {}
         for w_field in fields_w:

From noreply at buildbot.pypy.org  Wed Sep  7 15:13:14 2011
From: noreply at buildbot.pypy.org (cfbolz)
Date: Wed,  7 Sep 2011 15:13:14 +0200 (CEST)
Subject: [pypy-commit] pypy improve-heap-caching-tracing: fix XXX: also
 record newness of non-vtable memory
Message-ID: <20110907131314.B6E1382213@wyvern.cs.uni-duesseldorf.de>

Author: Carl Friedrich Bolz 
Branch: improve-heap-caching-tracing
Changeset: r47136:f7596aea7542
Date: 2011-09-07 11:35 +0200
http://bitbucket.org/pypy/pypy/changeset/f7596aea7542/

Log:	fix XXX: also record newness of non-vtable memory add a new one XXX

diff --git a/pypy/jit/metainterp/pyjitpl.py b/pypy/jit/metainterp/pyjitpl.py
--- a/pypy/jit/metainterp/pyjitpl.py
+++ b/pypy/jit/metainterp/pyjitpl.py
@@ -367,8 +367,9 @@
 
     @arguments("descr")
     def opimpl_new(self, sizedescr):
-        # XXX heapcache.new
-        return self.execute_with_descr(rop.NEW, sizedescr)
+        resbox = self.execute_with_descr(rop.NEW, sizedescr)
+        self.metainterp.heapcache.new(resbox)
+        return resbox
 
     @arguments("descr")
     def opimpl_new_with_vtable(self, sizedescr):
@@ -544,6 +545,7 @@
         if tobox is not None:
             return tobox
         resbox = self.execute_with_descr(opnum, fielddescr, box)
+        # XXX getfield_now_known
         self.metainterp.heapcache.setfield(box, fielddescr, resbox)
         return resbox
 
diff --git a/pypy/jit/metainterp/test/test_tracingopts.py b/pypy/jit/metainterp/test/test_tracingopts.py
--- a/pypy/jit/metainterp/test/test_tracingopts.py
+++ b/pypy/jit/metainterp/test/test_tracingopts.py
@@ -450,6 +450,26 @@
         assert res == 2 * -7 + 2 * -8
         self.check_operations_history(getfield_gc=0)
 
+    def test_heap_caching_multiple_tuples(self):
+        class Gbl(object):
+            pass
+        g = Gbl()
+        def gn(a1, a2):
+            return a1[0] + a2[0]
+        def fn(n):
+            a1 = (n, )
+            g.a = a1
+            a2 = (n - 1, )
+            g.a = a2
+            jit.promote(n)
+            return a1[0] + a2[0] + gn(a1, a2)
+        res = self.interp_operations(fn, [7])
+        assert res == 2 * 7 + 2 * 6
+        self.check_operations_history(getfield_gc_pure=0)
+        res = self.interp_operations(fn, [-7])
+        assert res == 2 * -7 + 2 * -8
+        self.check_operations_history(getfield_gc_pure=0)
+
     def test_heap_caching_multiple_arrays(self):
         class Gbl(object):
             pass

From noreply at buildbot.pypy.org  Wed Sep  7 15:13:15 2011
From: noreply at buildbot.pypy.org (cfbolz)
Date: Wed,  7 Sep 2011 15:13:15 +0200 (CEST)
Subject: [pypy-commit] pypy improve-heap-caching-tracing: test and fix: make
 getfield use getfield_now_known
Message-ID: <20110907131315.F245E82213@wyvern.cs.uni-duesseldorf.de>

Author: Carl Friedrich Bolz 
Branch: improve-heap-caching-tracing
Changeset: r47137:7213ef516d50
Date: 2011-09-07 14:02 +0200
http://bitbucket.org/pypy/pypy/changeset/7213ef516d50/

Log:	test and fix: make getfield use getfield_now_known

diff --git a/pypy/jit/metainterp/pyjitpl.py b/pypy/jit/metainterp/pyjitpl.py
--- a/pypy/jit/metainterp/pyjitpl.py
+++ b/pypy/jit/metainterp/pyjitpl.py
@@ -545,8 +545,7 @@
         if tobox is not None:
             return tobox
         resbox = self.execute_with_descr(opnum, fielddescr, box)
-        # XXX getfield_now_known
-        self.metainterp.heapcache.setfield(box, fielddescr, resbox)
+        self.metainterp.heapcache.getfield_now_known(box, fielddescr, resbox)
         return resbox
 
     @arguments("orgpc", "box", "descr")
diff --git a/pypy/jit/metainterp/test/test_tracingopts.py b/pypy/jit/metainterp/test/test_tracingopts.py
--- a/pypy/jit/metainterp/test/test_tracingopts.py
+++ b/pypy/jit/metainterp/test/test_tracingopts.py
@@ -428,27 +428,38 @@
         self.check_operations_history(getfield_gc=0)
         return
 
-
     def test_heap_caching_multiple_objects(self):
         class Gbl(object):
             pass
         g = Gbl()
         class A(object):
             pass
+        a1 = A()
+        g.a1 = a1
+        a1.x = 7
+        a2 = A()
+        g.a2 = a2
+        a2.x = 7
+        def gn(a1, a2):
+            return a1.x + a2.x
         def fn(n):
-            a1 = A()
-            g.a = a1
-            a1.x = n
-            a2 = A()
-            g.a = a2
-            a2.x = n - 1
-            return a1.x + a2.x + a1.x + a2.x
-        res = self.interp_operations(fn, [7])
-        assert res == 2 * 7 + 2 * 6
-        self.check_operations_history(getfield_gc=0)
+            if n < 0:
+                a1 = A()
+                g.a1 = a1
+                a1.x = n
+                a2 = A()
+                g.a2 = a2
+                a2.x = n - 1
+            else:
+                a1 = g.a1
+                a2 = g.a2
+            return a1.x + a2.x + gn(a1, a2)
         res = self.interp_operations(fn, [-7])
         assert res == 2 * -7 + 2 * -8
-        self.check_operations_history(getfield_gc=0)
+        self.check_operations_history(setfield_gc=4, getfield_gc=0)
+        res = self.interp_operations(fn, [7])
+        assert res == 4 * 7
+        self.check_operations_history(getfield_gc=4)
 
     def test_heap_caching_multiple_tuples(self):
         class Gbl(object):

From noreply at buildbot.pypy.org  Wed Sep  7 15:13:17 2011
From: noreply at buildbot.pypy.org (cfbolz)
Date: Wed,  7 Sep 2011 15:13:17 +0200 (CEST)
Subject: [pypy-commit] pypy improve-heap-caching-tracing: make list
 allocation go through the heap cache
Message-ID: <20110907131317.38C2C82213@wyvern.cs.uni-duesseldorf.de>

Author: Carl Friedrich Bolz 
Branch: improve-heap-caching-tracing
Changeset: r47138:60e23189885e
Date: 2011-09-07 14:13 +0200
http://bitbucket.org/pypy/pypy/changeset/60e23189885e/

Log:	make list allocation go through the heap cache

diff --git a/pypy/jit/metainterp/pyjitpl.py b/pypy/jit/metainterp/pyjitpl.py
--- a/pypy/jit/metainterp/pyjitpl.py
+++ b/pypy/jit/metainterp/pyjitpl.py
@@ -480,11 +480,9 @@
     @arguments("descr", "descr", "descr", "descr", "box")
     def opimpl_newlist(self, structdescr, lengthdescr, itemsdescr, arraydescr,
                        sizebox):
-        # XXX use heapcache
-        sbox = self.metainterp.execute_and_record(rop.NEW, structdescr)
+        sbox = self.opimpl_new(structdescr)
         self._opimpl_setfield_gc_any(sbox, lengthdescr, sizebox)
-        abox = self.metainterp.execute_and_record(rop.NEW_ARRAY, arraydescr,
-                                                  sizebox)
+        abox = self.opimpl_new_array(arraydescr, sizebox)
         self._opimpl_setfield_gc_any(sbox, itemsdescr, abox)
         return sbox
 
diff --git a/pypy/jit/metainterp/test/test_tracingopts.py b/pypy/jit/metainterp/test/test_tracingopts.py
--- a/pypy/jit/metainterp/test/test_tracingopts.py
+++ b/pypy/jit/metainterp/test/test_tracingopts.py
@@ -521,6 +521,29 @@
         assert res == 2 * 7 + 2 * 8
         self.check_operations_history(getarrayitem_gc=2)
 
+
+    def test_heap_caching_multiple_lists(self):
+        class Gbl(object):
+            pass
+        g = Gbl()
+        g.l = []
+        def fn(n):
+            if n < -100:
+                g.l.append(1)
+            a1 = [n, n, n]
+            g.l = a1
+            a1[0] = n
+            a2 = [n, n, n]
+            g.l = a2
+            a2[0] = n - 1
+            return a1[0] + a2[0] + a1[0] + a2[0]
+        res = self.interp_operations(fn, [7])
+        assert res == 2 * 7 + 2 * 6
+        self.check_operations_history(getarrayitem_gc=0, getfield_gc=0)
+        res = self.interp_operations(fn, [-7])
+        assert res == 2 * -7 + 2 * -8
+        self.check_operations_history(getarrayitem_gc=0, getfield_gc=0)
+
     def test_length_caching(self):
         class Gbl(object):
             pass

From noreply at buildbot.pypy.org  Wed Sep  7 15:13:18 2011
From: noreply at buildbot.pypy.org (cfbolz)
Date: Wed,  7 Sep 2011 15:13:18 +0200 (CEST)
Subject: [pypy-commit] pypy improve-heap-caching-tracing: add a sanity
 check: whenever a get operation is not generated, because the heap
Message-ID: <20110907131318.7349782213@wyvern.cs.uni-duesseldorf.de>

Author: Carl Friedrich Bolz 
Branch: improve-heap-caching-tracing
Changeset: r47139:4a4c1b008238
Date: 2011-09-07 15:12 +0200
http://bitbucket.org/pypy/pypy/changeset/4a4c1b008238/

Log:	add a sanity check: whenever a get operation is not generated,
	because the heap cache finds it, look at the concrete objects and
	see whether their contents corresponds to what the heapcache says it
	should

diff --git a/pypy/jit/metainterp/pyjitpl.py b/pypy/jit/metainterp/pyjitpl.py
--- a/pypy/jit/metainterp/pyjitpl.py
+++ b/pypy/jit/metainterp/pyjitpl.py
@@ -406,6 +406,11 @@
         tobox = self.metainterp.heapcache.getarrayitem(
                 arraybox, arraydescr, indexbox)
         if tobox:
+            # sanity check: see whether the current array value
+            # corresponds to what the cache thinks the value is
+            resbox = executor.execute(self.metainterp.cpu, self.metainterp,
+                                      rop.GETARRAYITEM_GC, arraydescr, arraybox, indexbox)
+            assert resbox.constbox().same_constant(tobox.constbox())
             return tobox
         resbox = self.execute_with_descr(rop.GETARRAYITEM_GC,
                                          arraydescr, arraybox, indexbox)
@@ -541,6 +546,10 @@
     def _opimpl_getfield_gc_any_pureornot(self, opnum, box, fielddescr):
         tobox = self.metainterp.heapcache.getfield(box, fielddescr)
         if tobox is not None:
+            # sanity check: see whether the current struct value
+            # corresponds to what the cache thinks the value is
+            resbox = executor.execute(self.metainterp.cpu, self.metainterp,
+                                      rop.GETFIELD_GC, fielddescr, box)
             return tobox
         resbox = self.execute_with_descr(opnum, fielddescr, box)
         self.metainterp.heapcache.getfield_now_known(box, fielddescr, resbox)

From noreply at buildbot.pypy.org  Wed Sep  7 15:50:06 2011
From: noreply at buildbot.pypy.org (antocuni)
Date: Wed,  7 Sep 2011 15:50:06 +0200 (CEST)
Subject: [pypy-commit] pypy ffistruct: fix import
Message-ID: <20110907135006.2333A82213@wyvern.cs.uni-duesseldorf.de>

Author: Antonio Cuni 
Branch: ffistruct
Changeset: r47140:f307fe1ce048
Date: 2011-09-07 15:09 +0200
http://bitbucket.org/pypy/pypy/changeset/f307fe1ce048/

Log:	fix import

diff --git a/pypy/module/_rawffi/interp_rawffi.py b/pypy/module/_rawffi/interp_rawffi.py
--- a/pypy/module/_rawffi/interp_rawffi.py
+++ b/pypy/module/_rawffi/interp_rawffi.py
@@ -253,7 +253,7 @@
         # XXX: this assumes that you have the _ffi module enabled. In the long
         # term, probably we will move the code for build structures and arrays
         # from _rawffi to _ffi
-        from pypy.module._ffi.interp_ffi import W_FFIType
+        from pypy.module._ffi.interp_ffitype import W_FFIType
         return W_FFIType('', self.get_basic_ffi_type(), self)
 
     @unwrap_spec(n=int)

From noreply at buildbot.pypy.org  Wed Sep  7 15:50:07 2011
From: noreply at buildbot.pypy.org (antocuni)
Date: Wed,  7 Sep 2011 15:50:07 +0200 (CEST)
Subject: [pypy-commit] pypy ffistruct: bah,
 apparently interp2app functions cannot be class methods
Message-ID: <20110907135007.552EA82213@wyvern.cs.uni-duesseldorf.de>

Author: Antonio Cuni 
Branch: ffistruct
Changeset: r47141:839a7545459d
Date: 2011-09-07 15:23 +0200
http://bitbucket.org/pypy/pypy/changeset/839a7545459d/

Log:	bah, apparently interp2app functions cannot be class methods

diff --git a/pypy/module/_ffi/interp_struct.py b/pypy/module/_ffi/interp_struct.py
--- a/pypy/module/_ffi/interp_struct.py
+++ b/pypy/module/_ffi/interp_struct.py
@@ -14,15 +14,14 @@
         self.w_ffitype = w_ffitype
         self.offset = -1
 
-    @staticmethod
-    @unwrap_spec(name=str)
-    def descr_new(space, w_type, name, w_ffitype):
-        w_ffitype = space.interp_w(W_FFIType, w_ffitype)
-        return W_Field(name, w_ffitype)
+ at unwrap_spec(name=str)
+def descr_new_field(space, w_type, name, w_ffitype):
+    w_ffitype = space.interp_w(W_FFIType, w_ffitype)
+    return W_Field(name, w_ffitype)
 
 W_Field.typedef = TypeDef(
     'Field',
-    __new__ = interp2app(W_Field.descr_new),
+    __new__ = interp2app(descr_new_field),
     name = interp_attrproperty('name', W_Field),
     ffitype = interp_attrproperty('w_ffitype', W_Field),
     offset = interp_attrproperty('offset', W_Field),
@@ -41,22 +40,6 @@
         for w_field in fields_w:
             self.name2w_field[w_field.name] = w_field
 
-    @staticmethod
-    @unwrap_spec(name=str)
-    def descr_new(space, w_type, name, w_fields):
-        size = 0
-        alignment = 0 # XXX
-        fields_w = space.fixedview(w_fields)
-        field_types = []
-        for w_field in fields_w:
-            w_field = space.interp_w(W_Field, w_field)
-            w_field.offset = size # XXX: alignment!
-            size += w_field.w_ffitype.sizeof()
-            field_types.append(w_field.w_ffitype.ffitype)
-        #
-        ffistruct = clibffi.make_struct_ffitype_e(size, alignment, field_types)
-        return W__StructDescr(name, fields_w, ffistruct)
-
     def allocate(self, space):
         return W__StructInstance(self)
 
@@ -70,9 +53,26 @@
             lltype.free(self.ffistruct, flavor='raw')
 
 
+ at unwrap_spec(name=str)
+def descr_new_structdescr(space, w_type, name, w_fields):
+    size = 0
+    alignment = 0 # XXX
+    fields_w = space.fixedview(w_fields)
+    field_types = []
+    for w_field in fields_w:
+        w_field = space.interp_w(W_Field, w_field)
+        w_field.offset = size # XXX: alignment!
+        size += w_field.w_ffitype.sizeof()
+        field_types.append(w_field.w_ffitype.ffitype)
+    #
+    ffistruct = clibffi.make_struct_ffitype_e(size, alignment, field_types)
+    return W__StructDescr(name, fields_w, ffistruct)
+
+
+
 W__StructDescr.typedef = TypeDef(
     '_StructDescr',
-    __new__ = interp2app(W__StructDescr.descr_new),
+    __new__ = interp2app(descr_new_structdescr),
     ffitype = interp_attrproperty('w_ffitype', W__StructDescr),
     allocate = interp2app(W__StructDescr.allocate),
     )

From noreply at buildbot.pypy.org  Wed Sep  7 16:11:48 2011
From: noreply at buildbot.pypy.org (arigo)
Date: Wed,  7 Sep 2011 16:11:48 +0200 (CEST)
Subject: [pypy-commit] benchmarks default: Reduce the delay and number of
	retries.
Message-ID: <20110907141148.EB81A82213@wyvern.cs.uni-duesseldorf.de>

Author: Armin Rigo 
Branch: 
Changeset: r143:867c2dc2b16d
Date: 2011-09-07 16:11 +0200
http://bitbucket.org/pypy/benchmarks/changeset/867c2dc2b16d/

Log:	Reduce the delay and number of retries.

diff --git a/saveresults.py b/saveresults.py
--- a/saveresults.py
+++ b/saveresults.py
@@ -93,7 +93,7 @@
     info += str(data['commitid']) + ", benchmark " + data['benchmark']
     print(info)
     try:
-        retries = [10, 20, 30, 60, 150, 300]
+        retries = [1, 2, 3, 6]
         while True:
             try:
                 f = urllib2.urlopen(SPEEDURL + 'result/add/', params)

From noreply at buildbot.pypy.org  Wed Sep  7 16:12:05 2011
From: noreply at buildbot.pypy.org (edelsohn)
Date: Wed,  7 Sep 2011 16:12:05 +0200 (CEST)
Subject: [pypy-commit] pypy ppc-jit-backend: Import WORD from arch.py
Message-ID: <20110907141205.3FEB682213@wyvern.cs.uni-duesseldorf.de>

Author: edelsohn
Branch: ppc-jit-backend
Changeset: r47142:ec6bced5ed4f
Date: 2011-09-07 10:05 -0400
http://bitbucket.org/pypy/pypy/changeset/ec6bced5ed4f/

Log:	Import WORD from arch.py

diff --git a/pypy/jit/backend/ppc/runner.py b/pypy/jit/backend/ppc/runner.py
--- a/pypy/jit/backend/ppc/runner.py
+++ b/pypy/jit/backend/ppc/runner.py
@@ -12,7 +12,7 @@
 from pypy.jit.backend.x86 import regloc
 from pypy.jit.backend.x86.support import values_array
 from pypy.jit.backend.ppc.ppcgen.ppc_assembler import PPCBuilder
-from pypy.jit.backend.ppc.ppcgen.arch import IS_PPC_32, NONVOLATILES
+from pypy.jit.backend.ppc.ppcgen.arch import IS_PPC_32, WORD, NONVOLATILES
 import sys
 
 from pypy.tool.ansi_print import ansi_log

From noreply at buildbot.pypy.org  Wed Sep  7 16:53:03 2011
From: noreply at buildbot.pypy.org (antocuni)
Date: Wed,  7 Sep 2011 16:53:03 +0200 (CEST)
Subject: [pypy-commit] pypy ffistruct: turn the interplevel KeyError into an
 applevel AttributeError
Message-ID: <20110907145303.5A21F82213@wyvern.cs.uni-duesseldorf.de>

Author: Antonio Cuni 
Branch: ffistruct
Changeset: r47143:845ee1dd654a
Date: 2011-09-07 16:48 +0200
http://bitbucket.org/pypy/pypy/changeset/845ee1dd654a/

Log:	turn the interplevel KeyError into an applevel AttributeError

diff --git a/pypy/module/_ffi/interp_struct.py b/pypy/module/_ffi/interp_struct.py
--- a/pypy/module/_ffi/interp_struct.py
+++ b/pypy/module/_ffi/interp_struct.py
@@ -4,6 +4,7 @@
 from pypy.interpreter.baseobjspace import Wrappable
 from pypy.interpreter.typedef import TypeDef, interp_attrproperty
 from pypy.interpreter.gateway import interp2app, unwrap_spec
+from pypy.interpreter.error import operationerrfmt
 from pypy.objspace.std.typetype import type_typedef
 from pypy.module._ffi.interp_ffitype import W_FFIType, app_types
 
@@ -32,7 +33,8 @@
 
 class W__StructDescr(Wrappable):
 
-    def __init__(self, name, fields_w, ffistruct):
+    def __init__(self, space, name, fields_w, ffistruct):
+        self.space = space
         self.ffistruct = ffistruct
         self.w_ffitype = W_FFIType('struct %s' % name, ffistruct.ffistruct, None)
         self.fields_w = fields_w
@@ -45,7 +47,11 @@
 
     #@jit.elidable...
     def get_type_and_offset_for_field(self, name):
-        w_field = self.name2w_field[name]
+        try:
+            w_field = self.name2w_field[name]
+        except KeyError:
+            raise operationerrfmt(self.space.w_AttributeError, '%s', name)
+
         return w_field.w_ffitype, w_field.offset
 
     def __del__(self):
diff --git a/pypy/module/_ffi/test/test_struct.py b/pypy/module/_ffi/test/test_struct.py
--- a/pypy/module/_ffi/test/test_struct.py
+++ b/pypy/module/_ffi/test/test_struct.py
@@ -58,3 +58,15 @@
         assert struct.getfield('y') == 43
         mem = self.read_raw_mem(struct.getaddr(), 'c_long', 2)
         assert mem == [42, 43]
+
+    def test_missing_field(self):
+        from _ffi import _StructDescr, Field, types
+        longsize = types.slong.sizeof()
+        fields = [
+            Field('x', types.slong),
+            Field('y', types.slong),
+            ]
+        descr = _StructDescr('foo', fields)
+        struct = descr.allocate()
+        raises(AttributeError, "struct.getfield('missing')")
+        raises(AttributeError, "struct.setfield('missing', 42)")

From noreply at buildbot.pypy.org  Wed Sep  7 16:53:04 2011
From: noreply at buildbot.pypy.org (antocuni)
Date: Wed,  7 Sep 2011 16:53:04 +0200 (CEST)
Subject: [pypy-commit] pypy ffistruct: this method is elidable
Message-ID: <20110907145304.8FA9182213@wyvern.cs.uni-duesseldorf.de>

Author: Antonio Cuni 
Branch: ffistruct
Changeset: r47144:303389ce8fb7
Date: 2011-09-07 16:50 +0200
http://bitbucket.org/pypy/pypy/changeset/303389ce8fb7/

Log:	this method is elidable

diff --git a/pypy/module/_ffi/interp_struct.py b/pypy/module/_ffi/interp_struct.py
--- a/pypy/module/_ffi/interp_struct.py
+++ b/pypy/module/_ffi/interp_struct.py
@@ -1,6 +1,7 @@
 from pypy.rpython.lltypesystem import lltype, rffi
 from pypy.rlib import clibffi
 from pypy.rlib import libffi
+from pypy.rlib import jit
 from pypy.interpreter.baseobjspace import Wrappable
 from pypy.interpreter.typedef import TypeDef, interp_attrproperty
 from pypy.interpreter.gateway import interp2app, unwrap_spec
@@ -45,7 +46,7 @@
     def allocate(self, space):
         return W__StructInstance(self)
 
-    #@jit.elidable...
+    @jit.elidable
     def get_type_and_offset_for_field(self, name):
         try:
             w_field = self.name2w_field[name]

From noreply at buildbot.pypy.org  Wed Sep  7 16:53:05 2011
From: noreply at buildbot.pypy.org (antocuni)
Date: Wed,  7 Sep 2011 16:53:05 +0200 (CEST)
Subject: [pypy-commit] pypy ffistruct: fix the result of a bad rebase
Message-ID: <20110907145305.C215282213@wyvern.cs.uni-duesseldorf.de>

Author: Antonio Cuni 
Branch: ffistruct
Changeset: r47145:4906bb0ec80b
Date: 2011-09-07 16:52 +0200
http://bitbucket.org/pypy/pypy/changeset/4906bb0ec80b/

Log:	fix the result of a bad rebase

diff --git a/pypy/module/_ffi/interp_struct.py b/pypy/module/_ffi/interp_struct.py
--- a/pypy/module/_ffi/interp_struct.py
+++ b/pypy/module/_ffi/interp_struct.py
@@ -73,7 +73,7 @@
         field_types.append(w_field.w_ffitype.ffitype)
     #
     ffistruct = clibffi.make_struct_ffitype_e(size, alignment, field_types)
-    return W__StructDescr(name, fields_w, ffistruct)
+    return W__StructDescr(space, name, fields_w, ffistruct)
 
 
 

From noreply at buildbot.pypy.org  Wed Sep  7 16:55:28 2011
From: noreply at buildbot.pypy.org (antocuni)
Date: Wed,  7 Sep 2011 16:55:28 +0200 (CEST)
Subject: [pypy-commit] pypy ffistruct: rpython fix
Message-ID: <20110907145528.6D8AE82213@wyvern.cs.uni-duesseldorf.de>

Author: Antonio Cuni 
Branch: ffistruct
Changeset: r47146:660db963a4f4
Date: 2011-09-07 16:54 +0200
http://bitbucket.org/pypy/pypy/changeset/660db963a4f4/

Log:	rpython fix

diff --git a/pypy/module/_ffi/interp_struct.py b/pypy/module/_ffi/interp_struct.py
--- a/pypy/module/_ffi/interp_struct.py
+++ b/pypy/module/_ffi/interp_struct.py
@@ -65,15 +65,17 @@
     size = 0
     alignment = 0 # XXX
     fields_w = space.fixedview(w_fields)
+    fields_w2 = [] # its items are annotated as W_Field
     field_types = []
     for w_field in fields_w:
         w_field = space.interp_w(W_Field, w_field)
         w_field.offset = size # XXX: alignment!
         size += w_field.w_ffitype.sizeof()
+        fields_w2.append(w_field)
         field_types.append(w_field.w_ffitype.ffitype)
     #
     ffistruct = clibffi.make_struct_ffitype_e(size, alignment, field_types)
-    return W__StructDescr(space, name, fields_w, ffistruct)
+    return W__StructDescr(space, name, fields_w2, ffistruct)
 
 
 

From noreply at buildbot.pypy.org  Wed Sep  7 17:38:47 2011
From: noreply at buildbot.pypy.org (antocuni)
Date: Wed,  7 Sep 2011 17:38:47 +0200 (CEST)
Subject: [pypy-commit] pypy ffistruct: make it elidable_promote,
 it should completely optimize away the offset/type lookup
Message-ID: <20110907153847.EE2EF82213@wyvern.cs.uni-duesseldorf.de>

Author: Antonio Cuni 
Branch: ffistruct
Changeset: r47147:f407a607b971
Date: 2011-09-07 17:35 +0200
http://bitbucket.org/pypy/pypy/changeset/f407a607b971/

Log:	make it elidable_promote, it should completely optimize away the
	offset/type lookup

diff --git a/pypy/module/_ffi/interp_struct.py b/pypy/module/_ffi/interp_struct.py
--- a/pypy/module/_ffi/interp_struct.py
+++ b/pypy/module/_ffi/interp_struct.py
@@ -46,7 +46,7 @@
     def allocate(self, space):
         return W__StructInstance(self)
 
-    @jit.elidable
+    @jit.elidable_promote()
     def get_type_and_offset_for_field(self, name):
         try:
             w_field = self.name2w_field[name]

From noreply at buildbot.pypy.org  Wed Sep  7 17:38:49 2011
From: noreply at buildbot.pypy.org (antocuni)
Date: Wed,  7 Sep 2011 17:38:49 +0200 (CEST)
Subject: [pypy-commit] pypy ffistruct: merge heads
Message-ID: <20110907153849.33E5782213@wyvern.cs.uni-duesseldorf.de>

Author: Antonio Cuni 
Branch: ffistruct
Changeset: r47148:65841158be6f
Date: 2011-09-07 17:38 +0200
http://bitbucket.org/pypy/pypy/changeset/65841158be6f/

Log:	merge heads

diff --git a/pypy/module/_ffi/interp_struct.py b/pypy/module/_ffi/interp_struct.py
--- a/pypy/module/_ffi/interp_struct.py
+++ b/pypy/module/_ffi/interp_struct.py
@@ -65,15 +65,17 @@
     size = 0
     alignment = 0 # XXX
     fields_w = space.fixedview(w_fields)
+    fields_w2 = [] # its items are annotated as W_Field
     field_types = []
     for w_field in fields_w:
         w_field = space.interp_w(W_Field, w_field)
         w_field.offset = size # XXX: alignment!
         size += w_field.w_ffitype.sizeof()
+        fields_w2.append(w_field)
         field_types.append(w_field.w_ffitype.ffitype)
     #
     ffistruct = clibffi.make_struct_ffitype_e(size, alignment, field_types)
-    return W__StructDescr(space, name, fields_w, ffistruct)
+    return W__StructDescr(space, name, fields_w2, ffistruct)
 
 
 

From noreply at buildbot.pypy.org  Wed Sep  7 18:54:03 2011
From: noreply at buildbot.pypy.org (hakanardo)
Date: Wed,  7 Sep 2011 18:54:03 +0200 (CEST)
Subject: [pypy-commit] pypy jit-str_in_preamble: closing barnch.
 jit-short_from_state gave us string optimization in the preamble without
 the
Message-ID: <20110907165403.D0D0E82213@wyvern.cs.uni-duesseldorf.de>

Author: Hakan Ardo 
Branch: jit-str_in_preamble
Changeset: r47149:4723df6a9296
Date: 2011-09-07 18:53 +0200
http://bitbucket.org/pypy/pypy/changeset/4723df6a9296/

Log:	closing barnch. jit-short_from_state gave us string optimization in
	the preamble without the drawbacks of this branch.


From noreply at buildbot.pypy.org  Wed Sep  7 19:14:56 2011
From: noreply at buildbot.pypy.org (antocuni)
Date: Wed,  7 Sep 2011 19:14:56 +0200 (CEST)
Subject: [pypy-commit] pypy ffistruct: we can't promote strings
Message-ID: <20110907171456.4FCEC82213@wyvern.cs.uni-duesseldorf.de>

Author: Antonio Cuni 
Branch: ffistruct
Changeset: r47150:ba6f286d77a8
Date: 2011-09-07 17:55 +0200
http://bitbucket.org/pypy/pypy/changeset/ba6f286d77a8/

Log:	we can't promote strings

diff --git a/pypy/module/_ffi/interp_struct.py b/pypy/module/_ffi/interp_struct.py
--- a/pypy/module/_ffi/interp_struct.py
+++ b/pypy/module/_ffi/interp_struct.py
@@ -46,7 +46,7 @@
     def allocate(self, space):
         return W__StructInstance(self)
 
-    @jit.elidable_promote()
+    @jit.elidable_promote('0')
     def get_type_and_offset_for_field(self, name):
         try:
             w_field = self.name2w_field[name]

From noreply at buildbot.pypy.org  Wed Sep  7 21:13:54 2011
From: noreply at buildbot.pypy.org (hakanardo)
Date: Wed,  7 Sep 2011 21:13:54 +0200 (CEST)
Subject: [pypy-commit] pypy jit-duplicated_short_boxes: close merged branch
Message-ID: <20110907191354.3306D82213@wyvern.cs.uni-duesseldorf.de>

Author: Hakan Ardo 
Branch: jit-duplicated_short_boxes
Changeset: r47151:ccb8b7b10a93
Date: 2011-09-07 20:55 +0200
http://bitbucket.org/pypy/pypy/changeset/ccb8b7b10a93/

Log:	close merged branch


From noreply at buildbot.pypy.org  Wed Sep  7 21:13:55 2011
From: noreply at buildbot.pypy.org (hakanardo)
Date: Wed,  7 Sep 2011 21:13:55 +0200 (CEST)
Subject: [pypy-commit] pypy default: Merge jit-duplicated_short_boxes,
 which introduces 3 things:
Message-ID: <20110907191355.916E7822AB@wyvern.cs.uni-duesseldorf.de>

Author: Hakan Ardo 
Branch: 
Changeset: r47152:9f54c3c0bb32
Date: 2011-09-07 21:12 +0200
http://bitbucket.org/pypy/pypy/changeset/9f54c3c0bb32/

Log:	Merge jit-duplicated_short_boxes, which introduces 3 things:

	1: ValueImporter. Instead of placing guards for the state of every
	Value that survives from the preamble into the peeled loop in the
	short preamble, only place guards for the Values that are actually
	used during the optimization. This reduces the size of the short
	preambles significanlty.

	2: UnrollableOptimizer. The optimizer has become more complex to
	support unrolling, which has made it slower. Some of this complexity
	is not needed when compiling bridges and has been split out into a
	UnrollableOptimizer subclass which is only used for compiling loops
	when unroll is enabled.

	3: A strategy for solving conflicts among the short boxes. Conflicts
	araises in situations such as:

	 [i1, i2] i3 = int_add(i1, 1) i4 = int_mul(i3, 2) escape(i4)
	jump(i1, i3)

	Specializing the peeled loop to the state at the end of the preamble
	would specialize it to situations when it's inputargs are of the
	form [i1, i1+1]. We dont want that. Instead a new box, i5, is
	introduced, represeting the second argument of the jump as well as
	the second inputargument of the peeled loop. Prior to this branch it
	was the other way around, a new box was introduced to represent
	i1+1. That resulted in the loop invariant int_mul would not be moved
	out of the loop. Instead a potential int_mul(i2, 2) would have been
	removed.

	The same kind of situation can occure when a setfield_is cahced. In
	that case, the set of pure ops that's optimized out would be quite
	random. This branch introduces a strict priority order:
	  - ops found in the original trace
	  - synthetic ops (setfields converted to getfields)
	  - inputargs
	  - potential ops that was never promoted to short_boxes This makes the
	effect of the optimizations less random and should always remove
	loop invariant ops. Non loop invariant cases can still benefit from
	unrolling but in exactly what situations has become more
	complicated.

diff --git a/pypy/jit/metainterp/optimizeopt/heap.py b/pypy/jit/metainterp/optimizeopt/heap.py
--- a/pypy/jit/metainterp/optimizeopt/heap.py
+++ b/pypy/jit/metainterp/optimizeopt/heap.py
@@ -37,6 +37,12 @@
             self.force_lazy_setfield(optheap)
             assert not self.possible_aliasing(optheap, structvalue)
         cached_fieldvalue = self._cached_fields.get(structvalue, None)
+
+        # Hack to ensure constants are imported from the preamble
+        if cached_fieldvalue and fieldvalue.is_constant(): 
+            optheap.optimizer.ensure_imported(cached_fieldvalue)
+            cached_fieldvalue = self._cached_fields.get(structvalue, None)
+
         if cached_fieldvalue is not fieldvalue:
             # common case: store the 'op' as lazy_setfield, and register
             # myself in the optheap's _lazy_setfields_and_arrayitems list
@@ -132,9 +138,7 @@
                         result = newresult
                     getop = ResOperation(rop.GETFIELD_GC, [op.getarg(0)],
                                          result, op.getdescr())
-                    getop = shortboxes.add_potential(getop)
-                    self._cached_fields_getfield_op[structvalue] = getop
-                    self._cached_fields[structvalue] = optimizer.getvalue(result)
+                    shortboxes.add_potential(getop, synthetic=True)
                 elif op.result is not None:
                     shortboxes.add_potential(op)
 
diff --git a/pypy/jit/metainterp/optimizeopt/optimizer.py b/pypy/jit/metainterp/optimizeopt/optimizer.py
--- a/pypy/jit/metainterp/optimizeopt/optimizer.py
+++ b/pypy/jit/metainterp/optimizeopt/optimizer.py
@@ -10,6 +10,7 @@
 from pypy.jit.metainterp.typesystem import llhelper, oohelper
 from pypy.tool.pairtype import extendabletype
 from pypy.rlib.debug import debug_start, debug_stop, debug_print
+from pypy.rlib.objectmodel import specialize
 
 LEVEL_UNKNOWN    = '\x00'
 LEVEL_NONNULL    = '\x01'
@@ -25,6 +26,9 @@
         self.descr = descr
         self.bound = bound
 
+    def clone(self):
+        return LenBound(self.mode, self.descr, self.bound.clone())
+
 class OptValue(object):
     __metaclass__ = extendabletype
     _attrs_ = ('box', 'known_class', 'last_guard_index', 'level', 'intbound', 'lenbound')
@@ -88,8 +92,27 @@
                     assert False
                 guards.append(op)
                 self.lenbound.bound.make_guards(lenbox, guards)
+        return guards
 
-        return guards
+    def import_from(self, other, optimizer):
+        assert self.level <= LEVEL_NONNULL
+        if other.level == LEVEL_CONSTANT:
+            self.make_constant(other.get_key_box())
+            optimizer.turned_constant(self)
+        elif other.level == LEVEL_KNOWNCLASS:
+            self.make_constant_class(other.known_class, -1)
+        else:
+            if other.level == LEVEL_NONNULL:
+                self.ensure_nonnull()
+            self.intbound.intersect(other.intbound)
+            if other.lenbound:
+                if self.lenbound:
+                    assert other.lenbound.mode == self.lenbound.mode
+                    assert other.lenbound.descr == self.lenbound.descr
+                    self.lenbound.bound.intersect(other.lenbound.bound)
+                else:
+                    self.lenbound = other.lenbound.clone()
+                    
 
     def force_box(self):
         return self.box
@@ -308,7 +331,6 @@
         self.resumedata_memo = resume.ResumeDataLoopMemo(metainterp_sd)
         self.bool_boxes = {}
         self.pure_operations = args_dict()
-        self.emitted_pure_operations = {}
         self.producer = {}
         self.pendingfields = []
         self.posponedop = None
@@ -316,12 +338,11 @@
         self.quasi_immutable_deps = None
         self.opaque_pointers = {}
         self.newoperations = []
-        self.emitting_dissabled = False
-        self.emitted_guards = 0        
         if loop is not None:
             self.call_pure_results = loop.call_pure_results
 
         self.set_optimizations(optimizations)
+        self.setup()
 
     def set_optimizations(self, optimizations):
         if optimizations:
@@ -348,23 +369,18 @@
         assert self.posponedop is None
 
     def new(self):
+        new = Optimizer(self.metainterp_sd, self.loop)
+        return self._new(new)
+
+    def _new(self, new):
         assert self.posponedop is None
-        new = Optimizer(self.metainterp_sd, self.loop)
         optimizations = [o.new() for o in self.optimizations]
         new.set_optimizations(optimizations)
         new.quasi_immutable_deps = self.quasi_immutable_deps
         return new
         
     def produce_potential_short_preamble_ops(self, sb):
-        for op in self.emitted_pure_operations:
-            if op.getopnum() == rop.GETARRAYITEM_GC_PURE or \
-               op.getopnum() == rop.STRGETITEM or \
-               op.getopnum() == rop.UNICODEGETITEM:
-                if not self.getvalue(op.getarg(1)).is_constant():
-                    continue
-            sb.add_potential(op)
-        for opt in self.optimizations:
-            opt.produce_potential_short_preamble_ops(sb)
+        raise NotImplementedError('This is implemented in unroll.UnrollableOptimizer')
 
     def turned_constant(self, value):
         for o in self.optimizations:
@@ -386,19 +402,26 @@
         else:
             return box
 
+    @specialize.argtype(0)
     def getvalue(self, box):
         box = self.getinterned(box)
         try:
             value = self.values[box]
         except KeyError:
             value = self.values[box] = OptValue(box)
+        self.ensure_imported(value)
         return value
 
+    def ensure_imported(self, value):
+        pass
+
+    @specialize.argtype(0)
     def get_constant_box(self, box):
         if isinstance(box, Const):
             return box
         try:
             value = self.values[box]
+            self.ensure_imported(value)
         except KeyError:
             return None
         if value.is_constant():
@@ -481,18 +504,22 @@
     def emit_operation(self, op):
         if op.returns_bool_result():
             self.bool_boxes[self.getvalue(op.result)] = None
-        if self.emitting_dissabled:
-            return
+        self._emit_operation(op)
         
+    @specialize.argtype(0)
+    def _emit_operation(self, op):        
         for i in range(op.numargs()):
             arg = op.getarg(i)
-            if arg in self.values:
-                box = self.values[arg].force_box()
-                op.setarg(i, box)
+            try:
+                value = self.values[arg]
+            except KeyError:
+                pass
+            else:
+                self.ensure_imported(value)
+                op.setarg(i, value.force_box())
         self.metainterp_sd.profiler.count(jitprof.OPT_OPS)
         if op.is_guard():
             self.metainterp_sd.profiler.count(jitprof.OPT_GUARDS)
-            self.emitted_guards += 1 # FIXME: can we reuse above counter?
             op = self.store_final_boxes_in_guard(op)
         elif op.can_raise():
             self.exception_might_have_happened = True
@@ -544,6 +571,7 @@
         args[n+1] = op.getdescr()
         return args
 
+    @specialize.argtype(0)
     def optimize_default(self, op):
         canfold = op.is_always_pure()
         if op.is_ovf():
@@ -579,13 +607,16 @@
                 return
             else:
                 self.pure_operations[args] = op
-                self.emitted_pure_operations[op] = True
+                self.remember_emitting_pure(op)
 
         # otherwise, the operation remains
         self.emit_operation(op)
         if nextop:
             self.emit_operation(nextop)
 
+    def remember_emitting_pure(self, op):
+        pass
+    
     def constant_fold(self, op):
         argboxes = [self.get_constant_box(op.getarg(i))
                     for i in range(op.numargs())]
diff --git a/pypy/jit/metainterp/optimizeopt/test/test_optimizeopt.py b/pypy/jit/metainterp/optimizeopt/test/test_optimizeopt.py
--- a/pypy/jit/metainterp/optimizeopt/test/test_optimizeopt.py
+++ b/pypy/jit/metainterp/optimizeopt/test/test_optimizeopt.py
@@ -472,7 +472,13 @@
         [i0]
         jump(i0)
         """
-        self.optimize_loop(ops, expected, preamble)
+        short = """
+        [i0]
+        i1 = int_is_true(i0)
+        guard_value(i1, 1) []
+        jump(i0)
+        """
+        self.optimize_loop(ops, expected, preamble, expected_short=short)
 
     def test_bound_int_is_true(self):
         ops = """
@@ -6997,6 +7003,26 @@
         """
         self.optimize_loop(ops, expected)
         
+    def test_cached_pure_func_of_equal_fields(self):
+        ops = """
+        [p5, p6]
+        i10 = getfield_gc(p5, descr=valuedescr)
+        i11 = getfield_gc(p6, descr=nextdescr)
+        i12 = int_add(i10, 7)
+        i13 = int_add(i11, 7)
+        call(i12, i13, descr=nonwritedescr)
+        setfield_gc(p6, i10, descr=nextdescr)        
+        jump(p5, p6)
+        """
+        expected = """
+        [p5, p6, i14, i12, i10]
+        i13 = int_add(i14, 7)
+        call(i12, i13, descr=nonwritedescr)
+        setfield_gc(p6, i10, descr=nextdescr)        
+        jump(p5, p6, i10, i12, i10)
+        """
+        self.optimize_loop(ops, expected)
+        
     def test_forced_counter(self):
         # XXX: VIRTUALHEAP (see above)
         py.test.skip("would be fixed by make heap optimizer aware of virtual setfields")
@@ -7086,8 +7112,84 @@
         """
         self.optimize_loop(ops, expected)
 
+    def test_import_constants_when_folding_pure_operations(self):
+        ops = """
+        [p0]
+        f1 = getfield_gc(p0, descr=valuedescr)
+        f2 = float_abs(f1)
+        call(7.0, descr=nonwritedescr)
+        setfield_gc(p0, -7.0, descr=valuedescr)
+        jump(p0)
+        """
+        expected = """
+        [p0]
+        call(7.0, descr=nonwritedescr)
+        jump(p0)
+        """
+        self.optimize_loop(ops, expected)
+
+    def test_exploding_duplicatipon(self):
+        ops = """
+        [i1, i2]
+        i3 = int_add(i1, i1)
+        i4 = int_add(i3, i3)
+        i5 = int_add(i4, i4)
+        i6 = int_add(i5, i5)
+        call(i6, descr=nonwritedescr)
+        jump(i1, i3)
+        """
+        expected = """
+        [i1, i2, i6, i3]
+        call(i6, descr=nonwritedescr)
+        jump(i1, i3, i6, i3)
+        """
+        short = """
+        [i1, i2]
+        i3 = int_add(i1, i1)
+        i4 = int_add(i3, i3)
+        i5 = int_add(i4, i4)
+        i6 = int_add(i5, i5)
+        jump(i1, i2, i6, i3)
+        """
+        self.optimize_loop(ops, expected, expected_short=short)
+
+    def test_prioritize_getfield1(self):
+        ops = """
+        [p1, p2]
+        i1 = getfield_gc(p1, descr=valuedescr)
+        setfield_gc(p2, i1, descr=nextdescr)
+        i2 = int_neg(i1)
+        call(i2, descr=nonwritedescr)
+        jump(p1, p2)
+        """
+        expected = """
+        [p1, p2, i2, i1]
+        call(i2, descr=nonwritedescr)
+        setfield_gc(p2, i1, descr=nextdescr)        
+        jump(p1, p2, i2, i1)
+        """
+        self.optimize_loop(ops, expected)
+
+    def test_prioritize_getfield2(self):
+        # Same as previous, but with descrs intercahnged which means
+        # that the getfield is discovered first when looking for
+        # potential short boxes during tests
+        ops = """
+        [p1, p2]
+        i1 = getfield_gc(p1, descr=nextdescr)
+        setfield_gc(p2, i1, descr=valuedescr)
+        i2 = int_neg(i1)
+        call(i2, descr=nonwritedescr)
+        jump(p1, p2)
+        """
+        expected = """
+        [p1, p2, i2, i1]
+        call(i2, descr=nonwritedescr)
+        setfield_gc(p2, i1, descr=valuedescr)        
+        jump(p1, p2, i2, i1)
+        """
+        self.optimize_loop(ops, expected)
         
-
 class TestLLtype(OptimizeOptTest, LLtypeMixin):
     pass
         
diff --git a/pypy/jit/metainterp/optimizeopt/unroll.py b/pypy/jit/metainterp/optimizeopt/unroll.py
--- a/pypy/jit/metainterp/optimizeopt/unroll.py
+++ b/pypy/jit/metainterp/optimizeopt/unroll.py
@@ -70,6 +70,47 @@
         self.snapshot_map[snapshot] = new_snapshot
         return new_snapshot
 
+class UnrollableOptimizer(Optimizer):
+    def setup(self):
+        self.importable_values = {}
+        self.emitting_dissabled = False
+        self.emitted_guards = 0
+        self.emitted_pure_operations = {}
+
+    def ensure_imported(self, value):
+        if not self.emitting_dissabled and value in self.importable_values:
+            imp = self.importable_values[value]
+            del self.importable_values[value]
+            imp.import_value(value)
+
+    def emit_operation(self, op):
+        if op.returns_bool_result():
+            self.bool_boxes[self.getvalue(op.result)] = None
+        if self.emitting_dissabled:
+            return
+        if op.is_guard():
+            self.emitted_guards += 1 # FIXME: can we use counter in self._emit_operation?
+        self._emit_operation(op)
+
+    def new(self):
+        new = UnrollableOptimizer(self.metainterp_sd, self.loop)
+        return self._new(new)
+
+    def remember_emitting_pure(self, op):
+        self.emitted_pure_operations[op] = True
+
+    def produce_potential_short_preamble_ops(self, sb):
+        for op in self.emitted_pure_operations:
+            if op.getopnum() == rop.GETARRAYITEM_GC_PURE or \
+               op.getopnum() == rop.STRGETITEM or \
+               op.getopnum() == rop.UNICODEGETITEM:
+                if not self.getvalue(op.getarg(1)).is_constant():
+                    continue
+            sb.add_potential(op)
+        for opt in self.optimizations:
+            opt.produce_potential_short_preamble_ops(sb)
+
+
 
 class UnrollOptimizer(Optimization):
     """Unroll the loop into two iterations. The first one will
@@ -77,7 +118,7 @@
     distinction anymore)"""
 
     def __init__(self, metainterp_sd, loop, optimizations):
-        self.optimizer = Optimizer(metainterp_sd, loop, optimizations)
+        self.optimizer = UnrollableOptimizer(metainterp_sd, loop, optimizations)
         self.cloned_operations = []
         for op in self.optimizer.loop.operations:
             newop = op.clone()
@@ -150,6 +191,7 @@
                 args = ", ".join([logops.repr_of_arg(arg) for arg in short_inputargs])
                 debug_print('short inputargs: ' + args)
                 self.short_boxes.debug_print(logops)
+                
 
             # Force virtuals amoung the jump_args of the preamble to get the
             # operations needed to setup the proper state of those virtuals
@@ -161,8 +203,9 @@
                 if box in seen:
                     continue
                 seen[box] = True
-                value = preamble_optimizer.getvalue(box)
-                inputarg_setup_ops.extend(value.make_guards(box))
+                preamble_value = preamble_optimizer.getvalue(box)
+                value = self.optimizer.getvalue(box)
+                value.import_from(preamble_value, self.optimizer)
             for box in short_inputargs:
                 if box in seen:
                     continue
@@ -181,23 +224,16 @@
             for op in self.short_boxes.operations():
                 self.ensure_short_op_emitted(op, self.optimizer, seen)
                 if op and op.result:
-                    # The order of these guards is not important as 
-                    # self.optimizer.emitting_dissabled is False
-                    value = preamble_optimizer.getvalue(op.result)
-                    for guard in value.make_guards(op.result):
-                        self.optimizer.send_extra_operation(guard)
+                    preamble_value = preamble_optimizer.getvalue(op.result)
+                    value = self.optimizer.getvalue(op.result)
+                    imp = ValueImporter(self, preamble_value, op)
+                    self.optimizer.importable_values[value] = imp
                     newresult = self.optimizer.getvalue(op.result).get_key_box()
                     if newresult is not op.result:
                         self.short_boxes.alias(newresult, op.result)
             self.optimizer.flush()
             self.optimizer.emitting_dissabled = False
 
-            # XXX Hack to prevent the arraylen/strlen/unicodelen ops generated
-            #     by value.make_guards() from ending up in pure_operations
-            for key, op in self.optimizer.pure_operations.items():
-                if not self.short_boxes.has_producer(op.result):
-                    del self.optimizer.pure_operations[key]
-
             initial_inputargs_len = len(inputargs)
             self.inliner = Inliner(loop.inputargs, jump_args)
 
@@ -276,16 +312,11 @@
 
         short_jumpargs = inputargs[:]
 
-        short = []
-        short_seen = {}
+        short = self.short = []
+        short_seen = self.short_seen = {}
         for box, const in self.constant_inputargs.items():
             short_seen[box] = True
 
-        for op in self.short_boxes.operations():
-            if op is not None:
-                if len(self.getvalue(op.result).make_guards(op.result)) > 0:
-                    self.add_op_to_short(op, short, short_seen, False, True)
-
         # This loop is equivalent to the main optimization loop in
         # Optimizer.propagate_all_forward
         jumpop = None
@@ -380,7 +411,7 @@
         if op.is_ovf():
             guard = ResOperation(rop.GUARD_NO_OVERFLOW, [], None)
             optimizer.send_extra_operation(guard)
-        
+
     def add_op_to_short(self, op, short, short_seen, emit=True, guards_needed=False):
         if op is None:
             return None
@@ -536,6 +567,13 @@
                         loop_token.failed_states.append(virtual_state)
         self.emit_operation(op)
 
+class ValueImporter(object):
+    def __init__(self, unroll, value, op):
+        self.unroll = unroll
+        self.preamble_value = value
+        self.op = op
 
-
-
+    def import_value(self, value):
+        value.import_from(self.preamble_value, self.unroll.optimizer)
+        self.unroll.add_op_to_short(self.op, self.unroll.short, self.unroll.short_seen, False, True)        
+        
diff --git a/pypy/jit/metainterp/optimizeopt/virtualize.py b/pypy/jit/metainterp/optimizeopt/virtualize.py
--- a/pypy/jit/metainterp/optimizeopt/virtualize.py
+++ b/pypy/jit/metainterp/optimizeopt/virtualize.py
@@ -58,6 +58,9 @@
     def _really_force(self):
         raise NotImplementedError("abstract base")
 
+    def import_from(self, other, optimizer):
+        raise NotImplementedError("should not be called at this level")
+    
 def get_fielddescrlist_cache(cpu):
     if not hasattr(cpu, '_optimizeopt_fielddescrlist_cache'):
         result = descrlist_dict()
diff --git a/pypy/jit/metainterp/optimizeopt/virtualstate.py b/pypy/jit/metainterp/optimizeopt/virtualstate.py
--- a/pypy/jit/metainterp/optimizeopt/virtualstate.py
+++ b/pypy/jit/metainterp/optimizeopt/virtualstate.py
@@ -12,6 +12,7 @@
 from pypy.rlib.objectmodel import we_are_translated
 from pypy.rlib.debug import debug_start, debug_stop, debug_print
 from pypy.rlib.objectmodel import we_are_translated
+import os
 
 class AbstractVirtualStateInfo(resume.AbstractVirtualInfo):
     position = -1
@@ -461,8 +462,10 @@
 class ShortBoxes(object):
     def __init__(self, optimizer, surviving_boxes):
         self.potential_ops = {}
-        self.duplicates = {}
+        self.alternatives = {}
+        self.synthetic = {}
         self.aliases = {}
+        self.rename = {}
         self.optimizer = optimizer
         for box in surviving_boxes:
             self.potential_ops[box] = None
@@ -476,33 +479,81 @@
             except BoxNotProducable:
                 pass
 
+    def prioritized_alternatives(self, box):
+        if box not in self.alternatives:
+            return [self.potential_ops[box]]
+        alts = self.alternatives[box]
+        hi, lo = 0, len(alts) - 1
+        while hi < lo:
+            if alts[lo] is None: # Inputarg, lowest priority
+                alts[lo], alts[-1] = alts[-1], alts[lo]
+                lo -= 1
+            elif alts[lo] not in self.synthetic: # Hi priority
+                alts[hi], alts[lo] = alts[lo], alts[hi]
+                hi += 1
+            else: # Low priority
+                lo -= 1
+        return alts
+            
+    def renamed(self, box):
+        if box in self.rename:
+            return self.rename[box]
+        return box
+    
+    def add_to_short(self, box, op):
+        if op:
+            op = op.clone()
+            for i in range(op.numargs()):
+                op.setarg(i, self.renamed(op.getarg(i)))
+        if box in self.short_boxes:
+            if op is None:
+                oldop = self.short_boxes[box].clone()
+                oldres = oldop.result
+                newbox = oldop.result = oldres.clonebox()
+                self.rename[box] = newbox
+                self.short_boxes[box] = None
+                self.short_boxes[newbox] = oldop
+            else:
+                newop = op.clone()
+                newbox = newop.result = op.result.clonebox()
+                self.short_boxes[newop.result] = newop
+            value = self.optimizer.getvalue(box)
+            self.optimizer.make_equal_to(newbox, value)
+        else:
+            self.short_boxes[box] = op
+        
     def produce_short_preamble_box(self, box):
         if box in self.short_boxes:
             return 
         if isinstance(box, Const):
             return 
         if box in self.potential_ops:
-            op = self.potential_ops[box]
-            if op:
-                for arg in op.getarglist():
-                    self.produce_short_preamble_box(arg)
-            self.short_boxes[box] = op
+            ops = self.prioritized_alternatives(box)
+            produced_one = False
+            for op in ops:
+                try:
+                    if op:
+                        for arg in op.getarglist():
+                            self.produce_short_preamble_box(arg)
+                except BoxNotProducable:
+                    pass
+                else:
+                    produced_one = True
+                    self.add_to_short(box, op)
+            if not produced_one:
+                raise BoxNotProducable
         else:
             raise BoxNotProducable
 
-    def add_potential(self, op):
+    def add_potential(self, op, synthetic=False):
         if op.result not in self.potential_ops:
             self.potential_ops[op.result] = op
-            return op
-        newop = op.clone()
-        newop.result = op.result.clonebox()
-        self.potential_ops[newop.result] = newop
-        if op.result in self.duplicates:
-            self.duplicates[op.result].append(newop.result)
         else:
-            self.duplicates[op.result] = [newop.result]
-        self.optimizer.make_equal_to(newop.result, self.optimizer.getvalue(op.result))
-        return newop
+            if op.result not in self.alternatives:
+                self.alternatives[op.result] = [self.potential_ops[op.result]]
+            self.alternatives[op.result].append(op)
+        if synthetic:
+            self.synthetic[op] = True
 
     def debug_print(self, logops):
         debug_start('jit-short-boxes')
diff --git a/pypy/jit/metainterp/test/test_virtualstate.py b/pypy/jit/metainterp/test/test_virtualstate.py
--- a/pypy/jit/metainterp/test/test_virtualstate.py
+++ b/pypy/jit/metainterp/test/test_virtualstate.py
@@ -2,7 +2,7 @@
 import py
 from pypy.jit.metainterp.optimize import InvalidLoop
 from pypy.jit.metainterp.optimizeopt.virtualstate import VirtualStateInfo, VStructStateInfo, \
-     VArrayStateInfo, NotVirtualStateInfo, VirtualState
+     VArrayStateInfo, NotVirtualStateInfo, VirtualState, ShortBoxes
 from pypy.jit.metainterp.optimizeopt.optimizer import OptValue
 from pypy.jit.metainterp.history import BoxInt, BoxFloat, BoxPtr, ConstInt, ConstPtr
 from pypy.rpython.lltypesystem import lltype
@@ -11,6 +11,7 @@
 from pypy.jit.metainterp.history import TreeLoop, LoopToken
 from pypy.jit.metainterp.optimizeopt.test.test_optimizeopt import FakeDescr, FakeMetaInterpStaticData
 from pypy.jit.metainterp.optimize import RetraceLoop
+from pypy.jit.metainterp.resoperation import ResOperation, rop
 
 class TestBasic:
     someptr1 = LLtypeMixin.myptr
@@ -129,6 +130,7 @@
             info.fieldstate = [info]
             assert info.generalization_of(info, {}, {})
 
+
 class BaseTestGenerateGuards(BaseTest):
     def guards(self, info1, info2, box, expected):
         info1.position = info2.position = 0
@@ -910,3 +912,111 @@
 class TestLLtypeBridges(BaseTestBridges, LLtypeMixin):
     pass
 
+class FakeOptimizer:
+    def make_equal_to(*args):
+        pass
+    def getvalue(*args):
+        pass
+
+class TestShortBoxes:
+    p1 = BoxPtr()
+    p2 = BoxPtr()
+    p3 = BoxPtr()
+    p4 = BoxPtr()
+    i1 = BoxInt()
+    i2 = BoxInt()
+    i3 = BoxInt()
+    i4 = BoxInt()
+    
+    def test_short_box_duplication_direct(self):
+        class Optimizer(FakeOptimizer):
+            def produce_potential_short_preamble_ops(_self, sb):
+                sb.add_potential(ResOperation(rop.GETFIELD_GC, [self.p1], self.i1))
+                sb.add_potential(ResOperation(rop.GETFIELD_GC, [self.p2], self.i1))
+        sb = ShortBoxes(Optimizer(), [self.p1, self.p2])
+        assert len(sb.short_boxes) == 4
+        assert self.i1 in sb.short_boxes
+        assert sum([op.result is self.i1 for op in sb.short_boxes.values() if op]) == 1
+
+    def test_dont_duplicate_potential_boxes(self):
+        class Optimizer(FakeOptimizer):
+            def produce_potential_short_preamble_ops(_self, sb):
+                sb.add_potential(ResOperation(rop.GETFIELD_GC, [self.p1], self.i1))
+                sb.add_potential(ResOperation(rop.GETFIELD_GC, [BoxPtr()], self.i1))
+                sb.add_potential(ResOperation(rop.INT_NEG, [self.i1], self.i2))
+                sb.add_potential(ResOperation(rop.INT_ADD, [ConstInt(7), self.i2],
+                                              self.i3))
+        sb = ShortBoxes(Optimizer(), [self.p1, self.p2])
+        assert len(sb.short_boxes) == 5
+
+    def test_prioritize1(self):
+        class Optimizer(FakeOptimizer):
+            def produce_potential_short_preamble_ops(_self, sb):
+                sb.add_potential(ResOperation(rop.GETFIELD_GC, [self.p1], self.i1))
+                sb.add_potential(ResOperation(rop.GETFIELD_GC, [self.p2], self.i1))
+                sb.add_potential(ResOperation(rop.INT_NEG, [self.i1], self.i2))
+        sb = ShortBoxes(Optimizer(), [self.p1, self.p2])
+        assert len(sb.short_boxes.values()) == 5
+        int_neg = [op for op in sb.short_boxes.values()
+                   if op and op.getopnum() == rop.INT_NEG]
+        assert len(int_neg) == 1
+        int_neg = int_neg[0]
+        getfield = [op for op in sb.short_boxes.values()
+                    if op and op.result == int_neg.getarg(0)]
+        assert len(getfield) == 1
+        assert getfield[0].getarg(0) in [self.p1, self.p2]
+
+    def test_prioritize1bis(self):
+        class Optimizer(FakeOptimizer):
+            def produce_potential_short_preamble_ops(_self, sb):
+                sb.add_potential(ResOperation(rop.GETFIELD_GC, [self.p1], self.i1),
+                                 synthetic=True)
+                sb.add_potential(ResOperation(rop.GETFIELD_GC, [self.p2], self.i1),
+                                 synthetic=True)
+                sb.add_potential(ResOperation(rop.INT_NEG, [self.i1], self.i2))
+        sb = ShortBoxes(Optimizer(), [self.p1, self.p2])
+        assert len(sb.short_boxes.values()) == 5
+        int_neg = [op for op in sb.short_boxes.values()
+                   if op and op.getopnum() == rop.INT_NEG]
+        assert len(int_neg) == 1
+        int_neg = int_neg[0]
+        getfield = [op for op in sb.short_boxes.values()
+                    if op and op.result == int_neg.getarg(0)]
+        assert len(getfield) == 1
+        assert getfield[0].getarg(0) in [self.p1, self.p2]
+        
+    def test_prioritize2(self):
+        class Optimizer(FakeOptimizer):
+            def produce_potential_short_preamble_ops(_self, sb):
+                sb.add_potential(ResOperation(rop.GETFIELD_GC, [self.p1], self.i1),
+                                 synthetic=True)
+                sb.add_potential(ResOperation(rop.GETFIELD_GC, [self.p2], self.i1))
+                sb.add_potential(ResOperation(rop.INT_NEG, [self.i1], self.i2))
+        sb = ShortBoxes(Optimizer(), [self.p1, self.p2])
+        assert len(sb.short_boxes.values()) == 5
+        int_neg = [op for op in sb.short_boxes.values()
+                   if op and op.getopnum() == rop.INT_NEG]
+        assert len(int_neg) == 1
+        int_neg = int_neg[0]
+        getfield = [op for op in sb.short_boxes.values()
+                    if op and op.result == int_neg.getarg(0)]
+        assert len(getfield) == 1
+        assert getfield[0].getarg(0) == self.p2
+        
+    def test_prioritize3(self):
+        class Optimizer(FakeOptimizer):
+            def produce_potential_short_preamble_ops(_self, sb):
+                sb.add_potential(ResOperation(rop.GETFIELD_GC, [self.p1], self.i1))
+                sb.add_potential(ResOperation(rop.GETFIELD_GC, [self.p2], self.i1),
+                                 synthetic=True)
+                sb.add_potential(ResOperation(rop.INT_NEG, [self.i1], self.i2))
+        sb = ShortBoxes(Optimizer(), [self.p1, self.p2])
+        assert len(sb.short_boxes.values()) == 5
+        int_neg = [op for op in sb.short_boxes.values()
+                   if op and op.getopnum() == rop.INT_NEG]
+        assert len(int_neg) == 1
+        int_neg = int_neg[0]
+        getfield = [op for op in sb.short_boxes.values()
+                    if op and op.result == int_neg.getarg(0)]
+        assert len(getfield) == 1
+        assert getfield[0].getarg(0) == self.p1
diff --git a/pypy/jit/tl/pypyjit_demo.py b/pypy/jit/tl/pypyjit_demo.py
--- a/pypy/jit/tl/pypyjit_demo.py
+++ b/pypy/jit/tl/pypyjit_demo.py
@@ -2,22 +2,16 @@
 pypyjit.set_param(threshold=200)
 
 
-def main(a, b):
-    i = sa = 0
-    while i < 300:
-        if a > 0: # Specialises the loop
-            pass
-        if b < 2 and b > 0:
-            pass
-        if (a >> b) >= 0:
-            sa += 1
-        if (a << b) > 2:
-            sa += 10000
-        i += 1
-    return sa
+def f(n):
+    pairs = [(0.0, 1.0), (2.0, 3.0)] * n
+    mag = 0
+    for (x1, x2) in pairs:
+        dx = x1 - x2
+        mag += ((dx * dx ) ** (-1.5))            
+    return n
 
 try:
-    print main(2, 1)
+    print f(301)
 
 except Exception, e:
     print "Exception: ", type(e)
diff --git a/pypy/module/pypyjit/test_pypy_c/test_00_model.py b/pypy/module/pypyjit/test_pypy_c/test_00_model.py
--- a/pypy/module/pypyjit/test_pypy_c/test_00_model.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_00_model.py
@@ -1,5 +1,5 @@
 from __future__ import with_statement
-import sys
+import sys, os
 import types
 import subprocess
 import py
diff --git a/pypy/module/pypyjit/test_pypy_c/test_misc.py b/pypy/module/pypyjit/test_pypy_c/test_misc.py
--- a/pypy/module/pypyjit/test_pypy_c/test_misc.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_misc.py
@@ -92,6 +92,43 @@
         """)
 
 
+    def test_cached_pure_func_of_equal_fields(self):            
+        def main(n):
+            class A(object):
+                def __init__(self, val):
+                    self.val1 = self.val2 = val
+            a = A(1)
+            b = A(1)
+            sa = 0
+            while n:
+                sa += 2*a.val1
+                sa += 2*b.val2
+                b.val2 = a.val1
+                n -= 1
+            return sa
+        #
+        log = self.run(main, [1000])
+        assert log.result == 4000
+        loop, = log.loops_by_filename(self.filepath)
+        assert loop.match("""
+            i12 = int_is_true(i4)
+            guard_true(i12, descr=...)
+            guard_not_invalidated(descr=...)
+            i13 = int_add_ovf(i8, i9)
+            guard_no_overflow(descr=...)
+            i10p = getfield_gc_pure(p10, descr=...)
+            i10 = int_mul_ovf(2, i10p)
+            guard_no_overflow(descr=...)
+            i14 = int_add_ovf(i13, i10)
+            guard_no_overflow(descr=...)
+            setfield_gc(p7, p11, descr=...)
+            i17 = int_sub_ovf(i4, 1)
+            guard_no_overflow(descr=...)
+            --TICK--
+            jump(..., descr=...)
+            """)
+
+
     def test_range_iter(self):
         def main(n):
             def g(n):
@@ -115,7 +152,6 @@
             i21 = force_token()
             setfield_gc(p4, i20, descr=<.* .*W_AbstractSeqIterObject.inst_index .*>)
             guard_not_invalidated(descr=...)
-            i26 = int_sub(i9, 1)
             i23 = int_lt(i18, 0)
             guard_false(i23, descr=...)
             i25 = int_ge(i18, i9)

From noreply at buildbot.pypy.org  Wed Sep  7 23:02:29 2011
From: noreply at buildbot.pypy.org (alex_gaynor)
Date: Wed,  7 Sep 2011 23:02:29 +0200 (CEST)
Subject: [pypy-commit] pypy improve-heap-caching-tracing: Add a special case
 for ll_arraycopy in the tracing heap cache.
Message-ID: <20110907210229.94C6F82213@wyvern.cs.uni-duesseldorf.de>

Author: Alex Gaynor 
Branch: improve-heap-caching-tracing
Changeset: r47153:70d999525e7c
Date: 2011-09-07 14:02 -0700
http://bitbucket.org/pypy/pypy/changeset/70d999525e7c/

Log:	Add a special case for ll_arraycopy in the tracing heap cache.

diff --git a/pypy/jit/metainterp/heapcache.py b/pypy/jit/metainterp/heapcache.py
--- a/pypy/jit/metainterp/heapcache.py
+++ b/pypy/jit/metainterp/heapcache.py
@@ -1,5 +1,7 @@
+from pypy.jit.codewriter.effectinfo import EffectInfo
+from pypy.jit.metainterp.history import ConstInt
 from pypy.jit.metainterp.resoperation import rop
-from pypy.jit.metainterp.history import ConstInt
+
 
 class HeapCache(object):
     def __init__(self):
@@ -21,7 +23,7 @@
         # cache the length of arrays
         self.length_cache = {}
 
-    def invalidate_caches(self, opnum, descr):
+    def invalidate_caches(self, opnum, descr, argboxes):
         if opnum == rop.SETFIELD_GC:
             return
         if opnum == rop.SETARRAYITEM_GC:
@@ -41,6 +43,20 @@
                ef == effectinfo.EF_ELIDABLE_CANNOT_RAISE or \
                ef == effectinfo.EF_ELIDABLE_CAN_RAISE:
                 return
+            # A special case for ll_arraycopy, because it is so common, and its
+            # effects are so well defined.
+            elif effectinfo.oopspecindex == EffectInfo.OS_ARRAYCOPY:
+                # The destination box
+                if argboxes[2] in self.new_boxes:
+                    # XXX: no descr here so we invalidate any of them, not just
+                    # of the correct type
+                    for descr, cache in self.heap_array_cache.iteritems():
+                        for idx, cache in cache.iteritems():
+                            for frombox in list(cache):
+                                if frombox not in self.new_boxes:
+                                    del cache[frombox]
+                    return
+
         self.heap_cache.clear()
         self.heap_array_cache.clear()
 
diff --git a/pypy/jit/metainterp/pyjitpl.py b/pypy/jit/metainterp/pyjitpl.py
--- a/pypy/jit/metainterp/pyjitpl.py
+++ b/pypy/jit/metainterp/pyjitpl.py
@@ -1676,7 +1676,7 @@
         # record the operation
         profiler = self.staticdata.profiler
         profiler.count_ops(opnum, RECORDED_OPS)
-        self.heapcache.invalidate_caches(opnum, descr)
+        self.heapcache.invalidate_caches(opnum, descr, argboxes)
         op = self.history.record(opnum, argboxes, resbox, descr)
         self.attach_debug_info(op)
         return resbox
diff --git a/pypy/jit/metainterp/test/test_heapcache.py b/pypy/jit/metainterp/test/test_heapcache.py
--- a/pypy/jit/metainterp/test/test_heapcache.py
+++ b/pypy/jit/metainterp/test/test_heapcache.py
@@ -25,15 +25,17 @@
     EF_FORCES_VIRTUAL_OR_VIRTUALIZABLE = 5 #can raise and force virtualizables
     EF_RANDOM_EFFECTS                  = 6 #can do whatever
 
-    def __init__(self, extraeffect):
+    def __init__(self, extraeffect, oopspecindex):
         self.extraeffect = extraeffect
+        self.oopspecindex = oopspecindex
 
 class FakeCallDescr(object):
-    def __init__(self, extraeffect):
+    def __init__(self, extraeffect, oopspecindex=None):
         self.extraeffect = extraeffect
+        self.oopspecindex = oopspecindex
 
     def get_extra_info(self):
-        return FakeEffektinfo(self.extraeffect)
+        return FakeEffektinfo(self.extraeffect, self.oopspecindex)
 
 class TestHeapCache(object):
     def test_known_class_box(self):
@@ -237,25 +239,25 @@
         h.setfield(box1, descr1, box2)
         h.setarrayitem(box1, descr1, index1, box2)
         h.setarrayitem(box1, descr1, index2, box4)
-        h.invalidate_caches(rop.INT_ADD, None)
-        h.invalidate_caches(rop.INT_ADD_OVF, None)
-        h.invalidate_caches(rop.SETFIELD_RAW, None)
-        h.invalidate_caches(rop.SETARRAYITEM_RAW, None)
+        h.invalidate_caches(rop.INT_ADD, None, [])
+        h.invalidate_caches(rop.INT_ADD_OVF, None, [])
+        h.invalidate_caches(rop.SETFIELD_RAW, None, [])
+        h.invalidate_caches(rop.SETARRAYITEM_RAW, None, [])
         assert h.getfield(box1, descr1) is box2
         assert h.getarrayitem(box1, descr1, index1) is box2
         assert h.getarrayitem(box1, descr1, index2) is box4
 
         h.invalidate_caches(
-            rop.CALL, FakeCallDescr(FakeEffektinfo.EF_ELIDABLE_CANNOT_RAISE))
+            rop.CALL, FakeCallDescr(FakeEffektinfo.EF_ELIDABLE_CANNOT_RAISE), [])
         assert h.getfield(box1, descr1) is box2
         assert h.getarrayitem(box1, descr1, index1) is box2
         assert h.getarrayitem(box1, descr1, index2) is box4
 
         h.invalidate_caches(
-            rop.CALL_LOOPINVARIANT, FakeCallDescr(FakeEffektinfo.EF_LOOPINVARIANT))
+            rop.CALL_LOOPINVARIANT, FakeCallDescr(FakeEffektinfo.EF_LOOPINVARIANT), [])
 
         h.invalidate_caches(
-            rop.CALL, FakeCallDescr(FakeEffektinfo.EF_RANDOM_EFFECTS))
+            rop.CALL, FakeCallDescr(FakeEffektinfo.EF_RANDOM_EFFECTS), [])
         assert h.getfield(box1, descr1) is None
         assert h.getarrayitem(box1, descr1, index1) is None
         assert h.getarrayitem(box1, descr1, index2) is None
@@ -294,3 +296,33 @@
 
         h.replace_box(lengthbox1, lengthbox2)
         assert h.arraylen(box4) is lengthbox2
+
+    def test_ll_arraycopy(self):
+        h = HeapCache()
+        h.new_array(box1, lengthbox1)
+        h.setarrayitem(box1, descr1, index1, box2)
+        h.new_array(box2, lengthbox1)
+        # Just need the destination box for this call
+        h.invalidate_caches(
+            rop.CALL,
+            # XXX: hardcoded oopspecindex
+            FakeCallDescr(FakeEffektinfo.EF_CANNOT_RAISE, 1),
+            [None, None, box2, None, None]
+        )
+        assert h.getarrayitem(box1, descr1, index1) is box2
+        h.invalidate_caches(
+            rop.CALL,
+            FakeCallDescr(FakeEffektinfo.EF_CANNOT_RAISE, 1),
+            [None, None, box3, None, None]
+        )
+        assert h.getarrayitem(box1, descr1, index1) is None
+
+        h.setarrayitem(box4, descr1, index1, box2)
+        assert h.getarrayitem(box4, descr1, index1) is box2
+        h.invalidate_caches(
+            rop.CALL,
+            # XXX: hardcoded oopspecindex
+            FakeCallDescr(FakeEffektinfo.EF_CANNOT_RAISE, 1),
+            [None, None, box2, None, None]
+        )
+        assert h.getarrayitem(box4, descr1, index1) is None
diff --git a/pypy/jit/metainterp/test/test_tracingopts.py b/pypy/jit/metainterp/test/test_tracingopts.py
--- a/pypy/jit/metainterp/test/test_tracingopts.py
+++ b/pypy/jit/metainterp/test/test_tracingopts.py
@@ -558,3 +558,18 @@
         res = self.interp_operations(fn, [7])
         assert res == 7 * 3
         self.check_operations_history(arraylen_gc=1)
+
+    def test_arraycopy(self):
+        class Gbl(object):
+            pass
+        g = Gbl()
+        g.a = [0] * 7
+        def fn(n):
+            assert n >= 0
+            a = g.a
+            x = [0] * n
+            x[2] = 21
+            return len(a[:n]) + x[2]
+        res = self.interp_operations(fn, [3])
+        assert res == 24
+        self.check_operations_history(getarrayitem_gc=0)

From noreply at buildbot.pypy.org  Thu Sep  8 05:37:32 2011
From: noreply at buildbot.pypy.org (wlav)
Date: Thu,  8 Sep 2011 05:37:32 +0200 (CEST)
Subject: [pypy-commit] pypy reflex-support: merge default into branch
Message-ID: <20110908033732.32C2582213@wyvern.cs.uni-duesseldorf.de>

Author: Wim Lavrijsen 
Branch: reflex-support
Changeset: r47154:06400362eacb
Date: 2011-09-06 11:39 -0700
http://bitbucket.org/pypy/pypy/changeset/06400362eacb/

Log:	merge default into branch

diff too long, truncating to 10000 out of 10169 lines

diff --git a/lib-python/conftest.py b/lib-python/conftest.py
--- a/lib-python/conftest.py
+++ b/lib-python/conftest.py
@@ -359,7 +359,7 @@
     RegrTest('test_property.py', core=True),
     RegrTest('test_pstats.py'),
     RegrTest('test_pty.py', skip="unsupported extension module"),
-    RegrTest('test_pwd.py', skip=skip_win32),
+    RegrTest('test_pwd.py', usemodules="pwd", skip=skip_win32),
     RegrTest('test_py3kwarn.py'),
     RegrTest('test_pyclbr.py'),
     RegrTest('test_pydoc.py'),
diff --git a/lib-python/modified-2.7/ctypes/util.py b/lib-python/modified-2.7/ctypes/util.py
--- a/lib-python/modified-2.7/ctypes/util.py
+++ b/lib-python/modified-2.7/ctypes/util.py
@@ -72,8 +72,8 @@
         return name
 
 if os.name == "posix" and sys.platform == "darwin":
-    from ctypes.macholib.dyld import dyld_find as _dyld_find
     def find_library(name):
+        from ctypes.macholib.dyld import dyld_find as _dyld_find
         possible = ['lib%s.dylib' % name,
                     '%s.dylib' % name,
                     '%s.framework/%s' % (name, name)]
diff --git a/lib-python/modified-2.7/gzip.py b/lib-python/modified-2.7/gzip.py
new file mode 100644
--- /dev/null
+++ b/lib-python/modified-2.7/gzip.py
@@ -0,0 +1,514 @@
+"""Functions that read and write gzipped files.
+
+The user of the file doesn't have to worry about the compression,
+but random access is not allowed."""
+
+# based on Andrew Kuchling's minigzip.py distributed with the zlib module
+
+import struct, sys, time, os
+import zlib
+import io
+import __builtin__
+
+__all__ = ["GzipFile","open"]
+
+FTEXT, FHCRC, FEXTRA, FNAME, FCOMMENT = 1, 2, 4, 8, 16
+
+READ, WRITE = 1, 2
+
+def write32u(output, value):
+    # The L format writes the bit pattern correctly whether signed
+    # or unsigned.
+    output.write(struct.pack("'
+
+    def _check_closed(self):
+        """Raises a ValueError if the underlying file object has been closed.
+
+        """
+        if self.closed:
+            raise ValueError('I/O operation on closed file.')
+
+    def _init_write(self, filename):
+        self.name = filename
+        self.crc = zlib.crc32("") & 0xffffffffL
+        self.size = 0
+        self.writebuf = []
+        self.bufsize = 0
+
+    def _write_gzip_header(self):
+        self.fileobj.write('\037\213')             # magic header
+        self.fileobj.write('\010')                 # compression method
+        fname = os.path.basename(self.name)
+        if fname.endswith(".gz"):
+            fname = fname[:-3]
+        flags = 0
+        if fname:
+            flags = FNAME
+        self.fileobj.write(chr(flags))
+        mtime = self.mtime
+        if mtime is None:
+            mtime = time.time()
+        write32u(self.fileobj, long(mtime))
+        self.fileobj.write('\002')
+        self.fileobj.write('\377')
+        if fname:
+            self.fileobj.write(fname + '\000')
+
+    def _init_read(self):
+        self.crc = zlib.crc32("") & 0xffffffffL
+        self.size = 0
+
+    def _read_gzip_header(self):
+        magic = self.fileobj.read(2)
+        if magic != '\037\213':
+            raise IOError, 'Not a gzipped file'
+        method = ord( self.fileobj.read(1) )
+        if method != 8:
+            raise IOError, 'Unknown compression method'
+        flag = ord( self.fileobj.read(1) )
+        self.mtime = read32(self.fileobj)
+        # extraflag = self.fileobj.read(1)
+        # os = self.fileobj.read(1)
+        self.fileobj.read(2)
+
+        if flag & FEXTRA:
+            # Read & discard the extra field, if present
+            xlen = ord(self.fileobj.read(1))
+            xlen = xlen + 256*ord(self.fileobj.read(1))
+            self.fileobj.read(xlen)
+        if flag & FNAME:
+            # Read and discard a null-terminated string containing the filename
+            while True:
+                s = self.fileobj.read(1)
+                if not s or s=='\000':
+                    break
+        if flag & FCOMMENT:
+            # Read and discard a null-terminated string containing a comment
+            while True:
+                s = self.fileobj.read(1)
+                if not s or s=='\000':
+                    break
+        if flag & FHCRC:
+            self.fileobj.read(2)     # Read & discard the 16-bit header CRC
+
+    def write(self,data):
+        self._check_closed()
+        if self.mode != WRITE:
+            import errno
+            raise IOError(errno.EBADF, "write() on read-only GzipFile object")
+
+        if self.fileobj is None:
+            raise ValueError, "write() on closed GzipFile object"
+
+        # Convert data type if called by io.BufferedWriter.
+        if isinstance(data, memoryview):
+            data = data.tobytes()
+
+        if len(data) > 0:
+            self.size = self.size + len(data)
+            self.crc = zlib.crc32(data, self.crc) & 0xffffffffL
+            self.fileobj.write( self.compress.compress(data) )
+            self.offset += len(data)
+
+        return len(data)
+
+    def read(self, size=-1):
+        self._check_closed()
+        if self.mode != READ:
+            import errno
+            raise IOError(errno.EBADF, "read() on write-only GzipFile object")
+
+        if self.extrasize <= 0 and self.fileobj is None:
+            return ''
+
+        readsize = 1024
+        if size < 0:        # get the whole thing
+            try:
+                while True:
+                    self._read(readsize)
+                    readsize = min(self.max_read_chunk, readsize * 2)
+            except EOFError:
+                size = self.extrasize
+        elif size == 0:
+            return ""
+        else:               # just get some more of it
+            try:
+                while size > self.extrasize:
+                    self._read(readsize)
+                    readsize = min(self.max_read_chunk, readsize * 2)
+            except EOFError:
+                if size > self.extrasize:
+                    size = self.extrasize
+
+        offset = self.offset - self.extrastart
+        chunk = self.extrabuf[offset: offset + size]
+        self.extrasize = self.extrasize - size
+
+        self.offset += size
+        return chunk
+
+    def _unread(self, buf):
+        self.extrasize = len(buf) + self.extrasize
+        self.offset -= len(buf)
+
+    def _read(self, size=1024):
+        if self.fileobj is None:
+            raise EOFError, "Reached EOF"
+
+        if self._new_member:
+            # If the _new_member flag is set, we have to
+            # jump to the next member, if there is one.
+            #
+            # First, check if we're at the end of the file;
+            # if so, it's time to stop; no more members to read.
+            pos = self.fileobj.tell()   # Save current position
+            self.fileobj.seek(0, 2)     # Seek to end of file
+            if pos == self.fileobj.tell():
+                raise EOFError, "Reached EOF"
+            else:
+                self.fileobj.seek( pos ) # Return to original position
+
+            self._init_read()
+            self._read_gzip_header()
+            self.decompress = zlib.decompressobj(-zlib.MAX_WBITS)
+            self._new_member = False
+
+        # Read a chunk of data from the file
+        buf = self.fileobj.read(size)
+
+        # If the EOF has been reached, flush the decompression object
+        # and mark this object as finished.
+
+        if buf == "":
+            uncompress = self.decompress.flush()
+            self._read_eof()
+            self._add_read_data( uncompress )
+            raise EOFError, 'Reached EOF'
+
+        uncompress = self.decompress.decompress(buf)
+        self._add_read_data( uncompress )
+
+        if self.decompress.unused_data != "":
+            # Ending case: we've come to the end of a member in the file,
+            # so seek back to the start of the unused data, finish up
+            # this member, and read a new gzip header.
+            # (The number of bytes to seek back is the length of the unused
+            # data, minus 8 because _read_eof() will rewind a further 8 bytes)
+            self.fileobj.seek( -len(self.decompress.unused_data)+8, 1)
+
+            # Check the CRC and file size, and set the flag so we read
+            # a new member on the next call
+            self._read_eof()
+            self._new_member = True
+
+    def _add_read_data(self, data):
+        self.crc = zlib.crc32(data, self.crc) & 0xffffffffL
+        offset = self.offset - self.extrastart
+        self.extrabuf = self.extrabuf[offset:] + data
+        self.extrasize = self.extrasize + len(data)
+        self.extrastart = self.offset
+        self.size = self.size + len(data)
+
+    def _read_eof(self):
+        # We've read to the end of the file, so we have to rewind in order
+        # to reread the 8 bytes containing the CRC and the file size.
+        # We check the that the computed CRC and size of the
+        # uncompressed data matches the stored values.  Note that the size
+        # stored is the true file size mod 2**32.
+        self.fileobj.seek(-8, 1)
+        crc32 = read32(self.fileobj)
+        isize = read32(self.fileobj)  # may exceed 2GB
+        if crc32 != self.crc:
+            raise IOError("CRC check failed %s != %s" % (hex(crc32),
+                                                         hex(self.crc)))
+        elif isize != (self.size & 0xffffffffL):
+            raise IOError, "Incorrect length of data produced"
+
+        # Gzip files can be padded with zeroes and still have archives.
+        # Consume all zero bytes and set the file position to the first
+        # non-zero byte. See http://www.gzip.org/#faq8
+        c = "\x00"
+        while c == "\x00":
+            c = self.fileobj.read(1)
+        if c:
+            self.fileobj.seek(-1, 1)
+
+    @property
+    def closed(self):
+        return self.fileobj is None
+
+    def close(self):
+        if self.fileobj is None:
+            return
+        if self.mode == WRITE:
+            self.fileobj.write(self.compress.flush())
+            write32u(self.fileobj, self.crc)
+            # self.size may exceed 2GB, or even 4GB
+            write32u(self.fileobj, self.size & 0xffffffffL)
+            self.fileobj = None
+        elif self.mode == READ:
+            self.fileobj = None
+        if self.myfileobj:
+            self.myfileobj.close()
+            self.myfileobj = None
+
+    def flush(self,zlib_mode=zlib.Z_SYNC_FLUSH):
+        self._check_closed()
+        if self.mode == WRITE:
+            # Ensure the compressor's buffer is flushed
+            self.fileobj.write(self.compress.flush(zlib_mode))
+            self.fileobj.flush()
+
+    def fileno(self):
+        """Invoke the underlying file object's fileno() method.
+
+        This will raise AttributeError if the underlying file object
+        doesn't support fileno().
+        """
+        return self.fileobj.fileno()
+
+    def rewind(self):
+        '''Return the uncompressed stream file position indicator to the
+        beginning of the file'''
+        if self.mode != READ:
+            raise IOError("Can't rewind in write mode")
+        self.fileobj.seek(0)
+        self._new_member = True
+        self.extrabuf = ""
+        self.extrasize = 0
+        self.extrastart = 0
+        self.offset = 0
+
+    def readable(self):
+        return self.mode == READ
+
+    def writable(self):
+        return self.mode == WRITE
+
+    def seekable(self):
+        return True
+
+    def seek(self, offset, whence=0):
+        if whence:
+            if whence == 1:
+                offset = self.offset + offset
+            else:
+                raise ValueError('Seek from end not supported')
+        if self.mode == WRITE:
+            if offset < self.offset:
+                raise IOError('Negative seek in write mode')
+            count = offset - self.offset
+            for i in range(count // 1024):
+                self.write(1024 * '\0')
+            self.write((count % 1024) * '\0')
+        elif self.mode == READ:
+            if offset == self.offset:
+                self.read(0) # to make sure that this file is open
+                return self.offset
+            if offset < self.offset:
+                # for negative seek, rewind and do positive seek
+                self.rewind()
+            count = offset - self.offset
+            for i in range(count // 1024):
+                self.read(1024)
+            self.read(count % 1024)
+
+        return self.offset
+
+    def readline(self, size=-1):
+        if size < 0:
+            # Shortcut common case - newline found in buffer.
+            offset = self.offset - self.extrastart
+            i = self.extrabuf.find('\n', offset) + 1
+            if i > 0:
+                self.extrasize -= i - offset
+                self.offset += i - offset
+                return self.extrabuf[offset: i]
+
+            size = sys.maxint
+            readsize = self.min_readsize
+        else:
+            readsize = size
+        bufs = []
+        while size != 0:
+            c = self.read(readsize)
+            i = c.find('\n')
+
+            # We set i=size to break out of the loop under two
+            # conditions: 1) there's no newline, and the chunk is
+            # larger than size, or 2) there is a newline, but the
+            # resulting line would be longer than 'size'.
+            if (size <= i) or (i == -1 and len(c) > size):
+                i = size - 1
+
+            if i >= 0 or c == '':
+                bufs.append(c[:i + 1])    # Add portion of last chunk
+                self._unread(c[i + 1:])   # Push back rest of chunk
+                break
+
+            # Append chunk to list, decrease 'size',
+            bufs.append(c)
+            size = size - len(c)
+            readsize = min(size, readsize * 2)
+        if readsize > self.min_readsize:
+            self.min_readsize = min(readsize, self.min_readsize * 2, 512)
+        return ''.join(bufs) # Return resulting line
+
+
+def _test():
+    # Act like gzip; with -d, act like gunzip.
+    # The input file is not deleted, however, nor are any other gzip
+    # options or features supported.
+    args = sys.argv[1:]
+    decompress = args and args[0] == "-d"
+    if decompress:
+        args = args[1:]
+    if not args:
+        args = ["-"]
+    for arg in args:
+        if decompress:
+            if arg == "-":
+                f = GzipFile(filename="", mode="rb", fileobj=sys.stdin)
+                g = sys.stdout
+            else:
+                if arg[-3:] != ".gz":
+                    print "filename doesn't end in .gz:", repr(arg)
+                    continue
+                f = open(arg, "rb")
+                g = __builtin__.open(arg[:-3], "wb")
+        else:
+            if arg == "-":
+                f = sys.stdin
+                g = GzipFile(filename="", mode="wb", fileobj=sys.stdout)
+            else:
+                f = __builtin__.open(arg, "rb")
+                g = open(arg + ".gz", "wb")
+        while True:
+            chunk = f.read(1024)
+            if not chunk:
+                break
+            g.write(chunk)
+        if g is not sys.stdout:
+            g.close()
+        if f is not sys.stdin:
+            f.close()
+
+if __name__ == '__main__':
+    _test()
diff --git a/lib-python/modified-2.7/sqlite3/test/regression.py b/lib-python/modified-2.7/sqlite3/test/regression.py
--- a/lib-python/modified-2.7/sqlite3/test/regression.py
+++ b/lib-python/modified-2.7/sqlite3/test/regression.py
@@ -274,6 +274,18 @@
         cur.execute("UPDATE foo SET id = 3 WHERE id = 1")
         self.assertEqual(cur.description, None)
 
+    def CheckStatementCache(self):
+        cur = self.con.cursor()
+        cur.execute("CREATE TABLE foo (id INTEGER)")
+        values = [(i,) for i in xrange(5)]
+        cur.executemany("INSERT INTO foo (id) VALUES (?)", values)
+
+        cur.execute("SELECT id FROM foo")
+        self.assertEqual(list(cur), values)
+        self.con.commit()
+        cur.execute("SELECT id FROM foo")
+        self.assertEqual(list(cur), values)
+
 def suite():
     regression_suite = unittest.makeSuite(RegressionTests, "Check")
     return unittest.TestSuite((regression_suite,))
diff --git a/lib-python/modified-2.7/tarfile.py b/lib-python/modified-2.7/tarfile.py
--- a/lib-python/modified-2.7/tarfile.py
+++ b/lib-python/modified-2.7/tarfile.py
@@ -252,8 +252,8 @@
        the high bit set. So we calculate two checksums, unsigned and
        signed.
     """
-    unsigned_chksum = 256 + sum(struct.unpack("148B", buf[:148]) + struct.unpack("356B", buf[156:512]))
-    signed_chksum = 256 + sum(struct.unpack("148b", buf[:148]) + struct.unpack("356b", buf[156:512]))
+    unsigned_chksum = 256 + sum(struct.unpack("148B8x356B", buf[:512]))
+    signed_chksum = 256 + sum(struct.unpack("148b8x356b", buf[:512]))
     return unsigned_chksum, signed_chksum
 
 def copyfileobj(src, dst, length=None):
@@ -265,7 +265,6 @@
     if length is None:
         shutil.copyfileobj(src, dst)
         return
-
     BUFSIZE = 16 * 1024
     blocks, remainder = divmod(length, BUFSIZE)
     for b in xrange(blocks):
@@ -802,19 +801,19 @@
         if self.closed:
             raise ValueError("I/O operation on closed file")
 
-        buf = ""
         if self.buffer:
             if size is None:
-                buf = self.buffer
+                buf = self.buffer + self.fileobj.read()
                 self.buffer = ""
             else:
                 buf = self.buffer[:size]
                 self.buffer = self.buffer[size:]
-
-        if size is None:
-            buf += self.fileobj.read()
+                buf += self.fileobj.read(size - len(buf))
         else:
-            buf += self.fileobj.read(size - len(buf))
+            if size is None:
+                buf = self.fileobj.read()
+            else:
+                buf = self.fileobj.read(size)
 
         self.position += len(buf)
         return buf
diff --git a/lib_pypy/_ctypes/basics.py b/lib_pypy/_ctypes/basics.py
--- a/lib_pypy/_ctypes/basics.py
+++ b/lib_pypy/_ctypes/basics.py
@@ -54,7 +54,8 @@
     def get_ffi_argtype(self):
         if self._ffiargtype:
             return self._ffiargtype
-        return _shape_to_ffi_type(self._ffiargshape)
+        self._ffiargtype = _shape_to_ffi_type(self._ffiargshape)
+        return self._ffiargtype
 
     def _CData_output(self, resbuffer, base=None, index=-1):
         #assert isinstance(resbuffer, _rawffi.ArrayInstance)
@@ -166,7 +167,8 @@
     return tp._alignmentofinstances()
 
 def byref(cdata):
-    from ctypes import pointer
+    # "pointer" is imported at the end of this module to avoid circular
+    # imports
     return pointer(cdata)
 
 def cdata_from_address(self, address):
@@ -224,5 +226,9 @@
     'Z' : _ffi.types.void_p,
     'X' : _ffi.types.void_p,
     'v' : _ffi.types.sshort,
+    '?' : _ffi.types.ubyte,
     }
 
+
+# used by "byref"
+from _ctypes.pointer import pointer
diff --git a/lib_pypy/_elementtree.py b/lib_pypy/_elementtree.py
new file mode 100644
--- /dev/null
+++ b/lib_pypy/_elementtree.py
@@ -0,0 +1,6 @@
+# Just use ElementTree.
+
+from xml.etree import ElementTree
+
+globals().update(ElementTree.__dict__)
+del __all__
diff --git a/lib_pypy/_pypy_interact.py b/lib_pypy/_pypy_interact.py
--- a/lib_pypy/_pypy_interact.py
+++ b/lib_pypy/_pypy_interact.py
@@ -56,6 +56,10 @@
                 prompt = getattr(sys, 'ps1', '>>> ')
             try:
                 line = raw_input(prompt)
+                # Can be None if sys.stdin was redefined
+                encoding = getattr(sys.stdin, 'encoding', None)
+                if encoding and not isinstance(line, unicode):
+                    line = line.decode(encoding)
             except EOFError:
                 console.write("\n")
                 break
diff --git a/lib_pypy/_sqlite3.py b/lib_pypy/_sqlite3.py
--- a/lib_pypy/_sqlite3.py
+++ b/lib_pypy/_sqlite3.py
@@ -24,6 +24,7 @@
 from ctypes import c_void_p, c_int, c_double, c_int64, c_char_p, cdll
 from ctypes import POINTER, byref, string_at, CFUNCTYPE, cast
 from ctypes import sizeof, c_ssize_t
+from collections import OrderedDict
 import datetime
 import sys
 import time
@@ -274,6 +275,28 @@
 def unicode_text_factory(x):
     return unicode(x, 'utf-8')
 
+
+class StatementCache(object):
+    def __init__(self, connection, maxcount):
+        self.connection = connection
+        self.maxcount = maxcount
+        self.cache = OrderedDict()
+
+    def get(self, sql, cursor, row_factory):
+        try:
+            stat = self.cache[sql]
+        except KeyError:
+            stat = Statement(self.connection, sql)
+            self.cache[sql] = stat
+            if len(self.cache) > self.maxcount:
+                self.cache.popitem(0)
+        #
+        if stat.in_use:
+            stat = Statement(self.connection, sql)
+        stat.set_row_factory(row_factory)
+        return stat
+
+
 class Connection(object):
     def __init__(self, database, timeout=5.0, detect_types=0, isolation_level="",
                  check_same_thread=True, factory=None, cached_statements=100):
@@ -291,6 +314,7 @@
         self.row_factory = None
         self._isolation_level = isolation_level
         self.detect_types = detect_types
+        self.statement_cache = StatementCache(self, cached_statements)
 
         self.cursors = []
 
@@ -399,7 +423,7 @@
         cur = Cursor(self)
         if not isinstance(sql, (str, unicode)):
             raise Warning("SQL is of wrong type. Must be string or unicode.")
-        statement = Statement(cur, sql, self.row_factory)
+        statement = self.statement_cache.get(sql, cur, self.row_factory)
         return statement
 
     def _get_isolation_level(self):
@@ -681,6 +705,8 @@
         from sqlite3.dump import _iterdump
         return _iterdump(self)
 
+DML, DQL, DDL = range(3)
+
 class Cursor(object):
     def __init__(self, con):
         if not isinstance(con, Connection):
@@ -708,12 +734,12 @@
         if type(sql) is unicode:
             sql = sql.encode("utf-8")
         self._check_closed()
-        self.statement = Statement(self, sql, self.row_factory)
+        self.statement = self.connection.statement_cache.get(sql, self, self.row_factory)
 
         if self.connection._isolation_level is not None:
-            if self.statement.kind == "DDL":
+            if self.statement.kind == DDL:
                 self.connection.commit()
-            elif self.statement.kind == "DML":
+            elif self.statement.kind == DML:
                 self.connection._begin()
 
         self.statement.set_params(params)
@@ -724,18 +750,18 @@
             self.statement.reset()
             raise self.connection._get_exception(ret)
 
-        if self.statement.kind == "DQL"and ret == SQLITE_ROW:
+        if self.statement.kind == DQL and ret == SQLITE_ROW:
             self.statement._build_row_cast_map()
-            self.statement._readahead()
+            self.statement._readahead(self)
         else:
             self.statement.item = None
             self.statement.exhausted = True
 
-        if self.statement.kind in ("DML", "DDL"):
+        if self.statement.kind == DML or self.statement.kind == DDL:
             self.statement.reset()
 
         self.rowcount = -1
-        if self.statement.kind == "DML":
+        if self.statement.kind == DML:
             self.rowcount = sqlite.sqlite3_changes(self.connection.db)
 
         return self
@@ -746,8 +772,9 @@
         if type(sql) is unicode:
             sql = sql.encode("utf-8")
         self._check_closed()
-        self.statement = Statement(self, sql, self.row_factory)
-        if self.statement.kind == "DML":
+        self.statement = self.connection.statement_cache.get(sql, self, self.row_factory)
+
+        if self.statement.kind == DML:
             self.connection._begin()
         else:
             raise ProgrammingError, "executemany is only for DML statements"
@@ -799,7 +826,7 @@
         return self
 
     def __iter__(self):
-        return self.statement
+        return iter(self.fetchone, None)
 
     def _check_reset(self):
         if self.reset:
@@ -816,7 +843,7 @@
             return None
 
         try:
-            return self.statement.next()
+            return self.statement.next(self)
         except StopIteration:
             return None
 
@@ -830,7 +857,7 @@
         if size is None:
             size = self.arraysize
         lst = []
-        for row in self.statement:
+        for row in self:
             lst.append(row)
             if len(lst) == size:
                 break
@@ -841,7 +868,7 @@
         self._check_reset()
         if self.statement is None:
             return []
-        return list(self.statement)
+        return list(self)
 
     def _getdescription(self):
         if self._description is None:
@@ -871,22 +898,24 @@
     lastrowid = property(_getlastrowid)
 
 class Statement(object):
-    def __init__(self, cur, sql, row_factory):
+    def __init__(self, connection, sql):
         self.statement = None
         if not isinstance(sql, str):
             raise ValueError, "sql must be a string"
-        self.con = cur.connection
-        self.cur = weakref.ref(cur)
+        self.con = connection
         self.sql = sql # DEBUG ONLY
-        self.row_factory = row_factory
         first_word = self._statement_kind = sql.lstrip().split(" ")[0].upper()
         if first_word in ("INSERT", "UPDATE", "DELETE", "REPLACE"):
-            self.kind = "DML"
+            self.kind = DML
         elif first_word in ("SELECT", "PRAGMA"):
-            self.kind = "DQL"
+            self.kind = DQL
         else:
-            self.kind = "DDL"
+            self.kind = DDL
         self.exhausted = False
+        self.in_use = False
+        #
+        # set by set_row_factory
+        self.row_factory = None
 
         self.statement = c_void_p()
         next_char = c_char_p()
@@ -895,7 +924,7 @@
         if ret == SQLITE_OK and self.statement.value is None:
             # an empty statement, we work around that, as it's the least trouble
             ret = sqlite.sqlite3_prepare_v2(self.con.db, "select 42", -1, byref(self.statement), byref(next_char))
-            self.kind = "DQL"
+            self.kind = DQL
 
         if ret != SQLITE_OK:
             raise self.con._get_exception(ret)
@@ -907,6 +936,9 @@
 
         self._build_row_cast_map()
 
+    def set_row_factory(self, row_factory):
+        self.row_factory = row_factory
+
     def _build_row_cast_map(self):
         self.row_cast_map = []
         for i in xrange(sqlite.sqlite3_column_count(self.statement)):
@@ -976,6 +1008,7 @@
         ret = sqlite.sqlite3_reset(self.statement)
         if ret != SQLITE_OK:
             raise self.con._get_exception(ret)
+        self.mark_dirty()
 
         if params is None:
             if sqlite.sqlite3_bind_parameter_count(self.statement) != 0:
@@ -1006,10 +1039,7 @@
                     raise ProgrammingError("missing parameter '%s'" %param)
                 self.set_param(idx, param)
 
-    def __iter__(self):
-        return self
-
-    def next(self):
+    def next(self, cursor):
         self.con._check_closed()
         self.con._check_thread()
         if self.exhausted:
@@ -1025,10 +1055,10 @@
             sqlite.sqlite3_reset(self.statement)
             raise exc
 
-        self._readahead()
+        self._readahead(cursor)
         return item
 
-    def _readahead(self):
+    def _readahead(self, cursor):
         self.column_count = sqlite.sqlite3_column_count(self.statement)
         row = []
         for i in xrange(self.column_count):
@@ -1063,23 +1093,30 @@
 
         row = tuple(row)
         if self.row_factory is not None:
-            row = self.row_factory(self.cur(), row)
+            row = self.row_factory(cursor, row)
         self.item = row
 
     def reset(self):
         self.row_cast_map = None
-        return sqlite.sqlite3_reset(self.statement)
+        ret = sqlite.sqlite3_reset(self.statement)
+        self.in_use = False
+        self.exhausted = False
+        return ret
 
     def finalize(self):
         sqlite.sqlite3_finalize(self.statement)
         self.statement = None
+        self.in_use = False
+
+    def mark_dirty(self):
+        self.in_use = True
 
     def __del__(self):
         sqlite.sqlite3_finalize(self.statement)
         self.statement = None
 
     def _get_description(self):
-        if self.kind == "DML":
+        if self.kind == DML:
             return None
         desc = []
         for i in xrange(sqlite.sqlite3_column_count(self.statement)):
diff --git a/lib_pypy/distributed/test/test_distributed.py b/lib_pypy/distributed/test/test_distributed.py
--- a/lib_pypy/distributed/test/test_distributed.py
+++ b/lib_pypy/distributed/test/test_distributed.py
@@ -9,7 +9,7 @@
 class AppTestDistributed(object):
     def setup_class(cls):
         cls.space = gettestobjspace(**{"objspace.std.withtproxy": True,
-            "usemodules":("_stackless",)})
+            "usemodules":("_continuation",)})
 
     def test_init(self):
         import distributed
@@ -91,10 +91,8 @@
 
 class AppTestDistributedTasklets(object):
     spaceconfig = {"objspace.std.withtproxy": True,
-                   "objspace.usemodules._stackless": True}
+                   "objspace.usemodules._continuation": True}
     def setup_class(cls):
-        #cls.space = gettestobjspace(**{"objspace.std.withtproxy": True,
-        #    "usemodules":("_stackless",)})
         cls.w_test_env = cls.space.appexec([], """():
         from distributed import test_env
         return test_env
diff --git a/lib_pypy/distributed/test/test_greensock.py b/lib_pypy/distributed/test/test_greensock.py
--- a/lib_pypy/distributed/test/test_greensock.py
+++ b/lib_pypy/distributed/test/test_greensock.py
@@ -10,7 +10,7 @@
         if not option.runappdirect:
             py.test.skip("Cannot run this on top of py.py because of PopenGateway")
         cls.space = gettestobjspace(**{"objspace.std.withtproxy": True,
-                                       "usemodules":("_stackless",)})
+                                       "usemodules":("_continuation",)})
         cls.w_remote_side_code = cls.space.appexec([], """():
         import sys
         sys.path.insert(0, '%s')
diff --git a/lib_pypy/distributed/test/test_socklayer.py b/lib_pypy/distributed/test/test_socklayer.py
--- a/lib_pypy/distributed/test/test_socklayer.py
+++ b/lib_pypy/distributed/test/test_socklayer.py
@@ -9,7 +9,8 @@
 class AppTestSocklayer:
     def setup_class(cls):
         cls.space = gettestobjspace(**{"objspace.std.withtproxy": True,
-                                       "usemodules":("_stackless","_socket", "select")})
+                                       "usemodules":("_continuation",
+                                                     "_socket", "select")})
     
     def test_socklayer(self):
         class X(object):
diff --git a/lib_pypy/greenlet.py b/lib_pypy/greenlet.py
--- a/lib_pypy/greenlet.py
+++ b/lib_pypy/greenlet.py
@@ -59,7 +59,12 @@
         #
         while not target:
             if not target.__started:
-                _continulet.__init__(target, _greenlet_start, *args)
+                if unbound_method != _continulet.throw:
+                    greenlet_func = _greenlet_start
+                else:
+                    greenlet_func = _greenlet_throw
+                _continulet.__init__(target, greenlet_func, *args)
+                unbound_method = _continulet.switch
                 args = ()
                 target.__started = True
                 break
@@ -136,3 +141,11 @@
         if greenlet.parent is not _tls.main:
             _continuation.permute(greenlet, greenlet.parent)
     return (res,)
+
+def _greenlet_throw(greenlet, exc, value, tb):
+    _tls.current = greenlet
+    try:
+        raise exc, value, tb
+    finally:
+        if greenlet.parent is not _tls.main:
+            _continuation.permute(greenlet, greenlet.parent)
diff --git a/lib_pypy/pypy_test/test_coroutine.py b/lib_pypy/pypy_test/test_coroutine.py
--- a/lib_pypy/pypy_test/test_coroutine.py
+++ b/lib_pypy/pypy_test/test_coroutine.py
@@ -2,7 +2,7 @@
 from py.test import skip, raises
 
 try:
-    from lib_pypy.stackless import coroutine, CoroutineExit
+    from stackless import coroutine, CoroutineExit
 except ImportError, e:
     skip('cannot import stackless: %s' % (e,))
 
@@ -20,10 +20,6 @@
         assert not co.is_zombie
 
     def test_is_zombie_del_without_frame(self):
-        try:
-            import _stackless # are we on pypy with a stackless build?
-        except ImportError:
-            skip("only works on pypy-c-stackless")
         import gc
         res = []
         class MyCoroutine(coroutine):
@@ -45,10 +41,6 @@
         assert res[0], "is_zombie was False in __del__"
 
     def test_is_zombie_del_with_frame(self):
-        try:
-            import _stackless # are we on pypy with a stackless build?
-        except ImportError:
-            skip("only works on pypy-c-stackless")
         import gc
         res = []
         class MyCoroutine(coroutine):
diff --git a/lib_pypy/pyrepl/reader.py b/lib_pypy/pyrepl/reader.py
--- a/lib_pypy/pyrepl/reader.py
+++ b/lib_pypy/pyrepl/reader.py
@@ -401,13 +401,19 @@
             return "(arg: %s) "%self.arg
         if "\n" in self.buffer:
             if lineno == 0:
-                return self._ps2
+                res = self.ps2
             elif lineno == self.buffer.count("\n"):
-                return self._ps4
+                res = self.ps4
             else:
-                return self._ps3
+                res = self.ps3
         else:
-            return self._ps1
+            res = self.ps1
+        # Lazily call str() on self.psN, and cache the results using as key
+        # the object on which str() was called.  This ensures that even if the
+        # same object is used e.g. for ps1 and ps2, str() is called only once.
+        if res not in self._pscache:
+            self._pscache[res] = str(res)
+        return self._pscache[res]
 
     def push_input_trans(self, itrans):
         self.input_trans_stack.append(self.input_trans)
@@ -473,8 +479,7 @@
             self.pos = 0
             self.dirty = 1
             self.last_command = None
-            self._ps1, self._ps2, self._ps3, self._ps4 = \
-                           map(str, [self.ps1, self.ps2, self.ps3, self.ps4])
+            self._pscache = {}
         except:
             self.restore()
             raise
@@ -571,7 +576,7 @@
         self.console.push_char(char)
         self.handle1(0)
     
-    def readline(self):
+    def readline(self, returns_unicode=False):
         """Read a line.  The implementation of this method also shows
         how to drive Reader if you want more control over the event
         loop."""
@@ -580,6 +585,8 @@
             self.refresh()
             while not self.finished:
                 self.handle1()
+            if returns_unicode:
+                return self.get_unicode()
             return self.get_buffer()
         finally:
             self.restore()
diff --git a/lib_pypy/pyrepl/readline.py b/lib_pypy/pyrepl/readline.py
--- a/lib_pypy/pyrepl/readline.py
+++ b/lib_pypy/pyrepl/readline.py
@@ -198,7 +198,7 @@
         reader.ps1 = prompt
         return reader.readline()
 
-    def multiline_input(self, more_lines, ps1, ps2):
+    def multiline_input(self, more_lines, ps1, ps2, returns_unicode=False):
         """Read an input on possibly multiple lines, asking for more
         lines as long as 'more_lines(unicodetext)' returns an object whose
         boolean value is true.
@@ -209,7 +209,7 @@
             reader.more_lines = more_lines
             reader.ps1 = reader.ps2 = ps1
             reader.ps3 = reader.ps4 = ps2
-            return reader.readline()
+            return reader.readline(returns_unicode=returns_unicode)
         finally:
             reader.more_lines = saved
 
diff --git a/lib_pypy/pyrepl/simple_interact.py b/lib_pypy/pyrepl/simple_interact.py
--- a/lib_pypy/pyrepl/simple_interact.py
+++ b/lib_pypy/pyrepl/simple_interact.py
@@ -54,7 +54,8 @@
             ps1 = getattr(sys, 'ps1', '>>> ')
             ps2 = getattr(sys, 'ps2', '... ')
             try:
-                statement = multiline_input(more_lines, ps1, ps2)
+                statement = multiline_input(more_lines, ps1, ps2,
+                                            returns_unicode=True)
             except EOFError:
                 break
             more = console.push(statement)
diff --git a/lib_pypy/stackless.py b/lib_pypy/stackless.py
--- a/lib_pypy/stackless.py
+++ b/lib_pypy/stackless.py
@@ -4,121 +4,124 @@
 Please refer to their documentation.
 """
 
-DEBUG = True
-
-def dprint(*args):
-    for arg in args:
-        print arg,
-    print
 
 import traceback
-import sys
+import _continuation
+from functools import partial
+
+class TaskletExit(Exception):
+    pass
+
+CoroutineExit = TaskletExit
+
+class GWrap(_continuation.continulet):
+    """This is just a wrapper around continulet to allow
+       to stick additional attributes to a continulet.
+       To be more concrete, we need a backreference to
+       the coroutine object"""
+
+
+class coroutine(object):
+    "we can't have continulet as a base, because continulets can't be rebound"
+
+    def __init__(self):
+        self._frame = None
+        self.is_zombie = False
+
+    def __getattr__(self, attr):
+        return getattr(self._frame, attr)
+
+    def __del__(self):
+        self.is_zombie = True
+        del self._frame
+        self._frame = None
+
+    def bind(self, func, *argl, **argd):
+        """coro.bind(f, *argl, **argd) -> None.
+           binds function f to coro. f will be called with
+           arguments *argl, **argd
+        """
+        if self._frame is None or not self._frame.is_pending():
+
+            def _func(c, *args, **kwargs):
+                return func(*args, **kwargs)
+            
+            run = partial(_func, *argl, **argd)
+            self._frame = frame = GWrap(run)
+        else:
+            raise ValueError("cannot bind a bound coroutine")
+
+    def switch(self):
+        """coro.switch() -> returnvalue
+           switches to coroutine coro. If the bound function
+           f finishes, the returnvalue is that of f, otherwise
+           None is returned
+        """
+        current = _getcurrent()
+        current._jump_to(self)
+
+    def _jump_to(self, coroutine):
+        _tls.current_coroutine = coroutine
+        self._frame.switch(to=coroutine._frame)
+
+    def kill(self):
+        """coro.kill() : kill coroutine coro"""
+        _tls.current_coroutine = self
+        self._frame.throw(CoroutineExit)
+
+    def _is_alive(self):
+        if self._frame is None:
+            return False
+        return not self._frame.is_pending()
+    is_alive = property(_is_alive)
+    del _is_alive
+
+    def getcurrent():
+        """coroutine.getcurrent() -> the currently running coroutine"""
+        try:
+            return _getcurrent()
+        except AttributeError:
+            return _maincoro
+    getcurrent = staticmethod(getcurrent)
+
+    def __reduce__(self):
+        raise TypeError, 'pickling is not possible based upon continulets'
+
+
+def _getcurrent():
+    "Returns the current coroutine (i.e. the one which called this function)."
+    try:
+        return _tls.current_coroutine
+    except AttributeError:
+        # first call in this thread: current == main
+        _coroutine_create_main()
+        return _tls.current_coroutine
+
 try:
-    # If _stackless can be imported then TaskletExit and CoroutineExit are 
-    # automatically added to the builtins.
-    from _stackless import coroutine, greenlet
-except ImportError: # we are running from CPython
-    from greenlet import greenlet, GreenletExit
-    TaskletExit = CoroutineExit = GreenletExit
-    del GreenletExit
-    try:
-        from functools import partial
-    except ImportError: # we are not running python 2.5
-        class partial(object):
-            # just enough of 'partial' to be usefull
-            def __init__(self, func, *argl, **argd):
-                self.func = func
-                self.argl = argl
-                self.argd = argd
+    from thread import _local
+except ImportError:
+    class _local(object):    # assume no threads
+        pass
 
-            def __call__(self):
-                return self.func(*self.argl, **self.argd)
+_tls = _local()
 
-    class GWrap(greenlet):
-        """This is just a wrapper around greenlets to allow
-           to stick additional attributes to a greenlet.
-           To be more concrete, we need a backreference to
-           the coroutine object"""
+def _coroutine_create_main():
+    # create the main coroutine for this thread
+    _tls.current_coroutine = None
+    main_coroutine = coroutine()
+    main_coroutine.bind(lambda x:x)
+    _tls.main_coroutine = main_coroutine
+    _tls.current_coroutine = main_coroutine
+    return main_coroutine
 
-    class MWrap(object):
-        def __init__(self,something):
-            self.something = something
 
-        def __getattr__(self, attr):
-            return getattr(self.something, attr)
+_maincoro = _coroutine_create_main()
 
-    class coroutine(object):
-        "we can't have greenlet as a base, because greenlets can't be rebound"
-
-        def __init__(self):
-            self._frame = None
-            self.is_zombie = False
-
-        def __getattr__(self, attr):
-            return getattr(self._frame, attr)
-
-        def __del__(self):
-            self.is_zombie = True
-            del self._frame
-            self._frame = None
-
-        def bind(self, func, *argl, **argd):
-            """coro.bind(f, *argl, **argd) -> None.
-               binds function f to coro. f will be called with
-               arguments *argl, **argd
-            """
-            if self._frame is None or self._frame.dead:
-                self._frame = frame = GWrap()
-                frame.coro = self
-            if hasattr(self._frame, 'run') and self._frame.run:
-                raise ValueError("cannot bind a bound coroutine")
-            self._frame.run = partial(func, *argl, **argd)
-
-        def switch(self):
-            """coro.switch() -> returnvalue
-               switches to coroutine coro. If the bound function
-               f finishes, the returnvalue is that of f, otherwise
-               None is returned
-            """
-            try:
-                return greenlet.switch(self._frame)
-            except TypeError, exp: # self._frame is the main coroutine
-                return greenlet.switch(self._frame.something)
-
-        def kill(self):
-            """coro.kill() : kill coroutine coro"""
-            self._frame.throw()
-
-        def _is_alive(self):
-            if self._frame is None:
-                return False
-            return not self._frame.dead
-        is_alive = property(_is_alive)
-        del _is_alive
-
-        def getcurrent():
-            """coroutine.getcurrent() -> the currently running coroutine"""
-            try:
-                return greenlet.getcurrent().coro
-            except AttributeError:
-                return _maincoro
-        getcurrent = staticmethod(getcurrent)
-
-        def __reduce__(self):
-            raise TypeError, 'pickling is not possible based upon greenlets'
-
-    _maincoro = coroutine()
-    maingreenlet = greenlet.getcurrent()
-    _maincoro._frame = frame = MWrap(maingreenlet)
-    frame.coro = _maincoro
-    del frame
-    del maingreenlet
 
 from collections import deque
 
 import operator
-__all__ = 'run getcurrent getmain schedule tasklet channel coroutine \
-                greenlet'.split()
+__all__ = 'run getcurrent getmain schedule tasklet channel coroutine'.split()
 
 _global_task_id = 0
 _squeue = None
@@ -131,7 +134,8 @@
 def _scheduler_remove(value):
     try:
         del _squeue[operator.indexOf(_squeue, value)]
-    except ValueError:pass
+    except ValueError:
+        pass
 
 def _scheduler_append(value, normal=True):
     if normal:
diff --git a/pypy/config/pypyoption.py b/pypy/config/pypyoption.py
--- a/pypy/config/pypyoption.py
+++ b/pypy/config/pypyoption.py
@@ -27,7 +27,7 @@
 # --allworkingmodules
 working_modules = default_modules.copy()
 working_modules.update(dict.fromkeys(
-    ["_socket", "unicodedata", "mmap", "fcntl", "_locale",
+    ["_socket", "unicodedata", "mmap", "fcntl", "_locale", "pwd",
      "rctime" , "select", "zipimport", "_lsprof",
      "crypt", "signal", "_rawffi", "termios", "zlib", "bz2",
      "struct", "_hashlib", "_md5", "_sha", "_minimal_curses", "cStringIO",
@@ -58,6 +58,7 @@
     # unix only modules
     del working_modules["crypt"]
     del working_modules["fcntl"]
+    del working_modules["pwd"]
     del working_modules["termios"]
     del working_modules["_minimal_curses"]
 
diff --git a/pypy/config/test/test_config.py b/pypy/config/test/test_config.py
--- a/pypy/config/test/test_config.py
+++ b/pypy/config/test/test_config.py
@@ -281,11 +281,11 @@
 
 def test_underscore_in_option_name():
     descr = OptionDescription("opt", "", [
-        BoolOption("_stackless", "", default=False),
+        BoolOption("_foobar", "", default=False),
     ])
     config = Config(descr)
     parser = to_optparse(config)
-    assert parser.has_option("--_stackless")
+    assert parser.has_option("--_foobar")
 
 def test_none():
     dummy1 = BoolOption('dummy1', 'doc dummy', default=False, cmdline=None)
diff --git a/pypy/doc/config/objspace.usemodules._stackless.txt b/pypy/doc/config/objspace.usemodules._stackless.txt
deleted file mode 100644
--- a/pypy/doc/config/objspace.usemodules._stackless.txt
+++ /dev/null
@@ -1,1 +0,0 @@
-Deprecated.
diff --git a/pypy/doc/config/objspace.usemodules.pwd.txt b/pypy/doc/config/objspace.usemodules.pwd.txt
new file mode 100644
--- /dev/null
+++ b/pypy/doc/config/objspace.usemodules.pwd.txt
@@ -0,0 +1,2 @@
+Use the 'pwd' module. 
+This module is expected to be fully working.
diff --git a/pypy/doc/faq.rst b/pypy/doc/faq.rst
--- a/pypy/doc/faq.rst
+++ b/pypy/doc/faq.rst
@@ -315,6 +315,28 @@
 
 .. _`Andrew Brown's tutorial`: http://morepypy.blogspot.com/2011/04/tutorial-writing-interpreter-with-pypy.html
 
+---------------------------------------------------------
+Can RPython modules for PyPy be translated independently?
+---------------------------------------------------------
+
+No, you have to rebuild the entire interpreter.  This means two things:
+
+* It is imperative to use test-driven development.  You have to test
+  exhaustively your module in pure Python, before even attempting to
+  translate it.  Once you translate it, you should have only a few typing
+  issues left to fix, but otherwise the result should work out of the box.
+
+* Second, and perhaps most important: do you have a really good reason
+  for writing the module in RPython in the first place?  Nowadays you
+  should really look at alternatives, like writing it in pure Python,
+  using ctypes if it needs to call C code.  Other alternatives are being
+  developed too (as of summer 2011), like a Cython binding.
+
+In this context it is not that important to be able to translate
+RPython modules independently of translating the complete interpreter.
+(It could be done given enough efforts, but it's a really serious
+undertaking.  Consider it as quite unlikely for now.)
+
 ----------------------------------------------------------
 Why does PyPy draw a Mandelbrot fractal while translating?
 ----------------------------------------------------------
diff --git a/pypy/doc/jit/pyjitpl5.rst b/pypy/doc/jit/pyjitpl5.rst
--- a/pypy/doc/jit/pyjitpl5.rst
+++ b/pypy/doc/jit/pyjitpl5.rst
@@ -103,7 +103,7 @@
 
 The meta-interpreter starts interpreting the JIT bytecode.  Each operation is
 executed and then recorded in a list of operations, called the trace.
-Operations can have a list of boxes that operate on, arguments.  Some operations
+Operations can have a list of boxes they operate on, arguments.  Some operations
 (like GETFIELD and GETARRAYITEM) also have special objects that describe how
 their arguments are laid out in memory.  All possible operations generated by
 tracing are listed in metainterp/resoperation.py.  When a (interpreter-level)
diff --git a/pypy/doc/stackless.rst b/pypy/doc/stackless.rst
--- a/pypy/doc/stackless.rst
+++ b/pypy/doc/stackless.rst
@@ -199,7 +199,11 @@
 The following features (present in some past Stackless version of PyPy)
 are for the time being not supported any more:
 
-* Tasklets and channels (needs to be rewritten at app-level)
+* Tasklets and channels (currently ``stackless.py`` seems to import,
+  but you have tasklets on top of coroutines on top of greenlets on
+  top of continulets on top of stacklets, and it's probably not too
+  hard to cut two of these levels by adapting ``stackless.py`` to
+  use directly continulets)
 
 * Coroutines (could be rewritten at app-level)
 
@@ -209,6 +213,13 @@
 
 * Automatic unlimited stack (must be emulated__ so far)
 
+* Support for other CPUs than x86 and x86-64
+
+* The app-level ``f_back`` field of frames crossing continulet boundaries
+  is None for now, unlike what I explain in the theoretical overview
+  above.  It mostly means that in a ``pdb.set_trace()`` you cannot go
+  ``up`` past countinulet boundaries.  This could be fixed.
+
 .. __: `recursion depth limit`_
 
 (*) Pickling, as well as changing threads, could be implemented by using
@@ -217,9 +228,8 @@
 "hard" switch (like now) when the C stack contains non-trivial C frames
 to save, and a "soft" switch (like previously) when it contains only
 simple calls from Python to Python.  Soft-switched continulets would
-also consume a bit less RAM, at the possible expense of making the
-switch a bit slower (unsure about that; what is the Stackless Python
-experience?).
+also consume a bit less RAM, and the switch might be a bit faster too
+(unsure about that; what is the Stackless Python experience?).
 
 
 Recursion depth limit
diff --git a/pypy/interpreter/baseobjspace.py b/pypy/interpreter/baseobjspace.py
--- a/pypy/interpreter/baseobjspace.py
+++ b/pypy/interpreter/baseobjspace.py
@@ -626,9 +626,9 @@
             self.default_compiler = compiler
             return compiler
 
-    def createframe(self, code, w_globals, closure=None):
+    def createframe(self, code, w_globals, outer_func=None):
         "Create an empty PyFrame suitable for this code object."
-        return self.FrameClass(self, code, w_globals, closure)
+        return self.FrameClass(self, code, w_globals, outer_func)
 
     def allocate_lock(self):
         """Return an interp-level Lock object if threads are enabled,
diff --git a/pypy/interpreter/function.py b/pypy/interpreter/function.py
--- a/pypy/interpreter/function.py
+++ b/pypy/interpreter/function.py
@@ -30,7 +30,7 @@
     can_change_code = True
     _immutable_fields_ = ['code?',
                           'w_func_globals?',
-                          'closure?',
+                          'closure?[*]',
                           'defs_w?[*]',
                           'name?']
 
@@ -96,7 +96,7 @@
             assert isinstance(code, PyCode)
             if nargs < 5:
                 new_frame = self.space.createframe(code, self.w_func_globals,
-                                                   self.closure)
+                                                   self)
                 for i in funccallunrolling:
                     if i < nargs:
                         new_frame.locals_stack_w[i] = args_w[i]
@@ -156,7 +156,7 @@
     def _flat_pycall(self, code, nargs, frame):
         # code is a PyCode
         new_frame = self.space.createframe(code, self.w_func_globals,
-                                                   self.closure)
+                                                   self)
         for i in xrange(nargs):
             w_arg = frame.peekvalue(nargs-1-i)
             new_frame.locals_stack_w[i] = w_arg
@@ -167,7 +167,7 @@
     def _flat_pycall_defaults(self, code, nargs, frame, defs_to_load):
         # code is a PyCode
         new_frame = self.space.createframe(code, self.w_func_globals,
-                                                   self.closure)
+                                                   self)
         for i in xrange(nargs):
             w_arg = frame.peekvalue(nargs-1-i)
             new_frame.locals_stack_w[i] = w_arg
diff --git a/pypy/interpreter/miscutils.py b/pypy/interpreter/miscutils.py
--- a/pypy/interpreter/miscutils.py
+++ b/pypy/interpreter/miscutils.py
@@ -167,3 +167,7 @@
 
     def getmainthreadvalue(self):
         return self._value
+
+    def getallvalues(self):
+        return {0: self._value}
+
diff --git a/pypy/interpreter/nestedscope.py b/pypy/interpreter/nestedscope.py
--- a/pypy/interpreter/nestedscope.py
+++ b/pypy/interpreter/nestedscope.py
@@ -8,7 +8,7 @@
 
 class Cell(Wrappable):
     "A simple container for a wrapped value."
-    
+
     def __init__(self, w_value=None):
         self.w_value = w_value
 
@@ -90,32 +90,33 @@
     #     variables coming from a parent function in which i'm nested
     # 'closure' is a list of Cell instances: the received free vars.
 
-    cells = None
-
     @jit.unroll_safe
-    def initialize_frame_scopes(self, closure, code):
-        super_initialize_frame_scopes(self, closure, code)
+    def initialize_frame_scopes(self, outer_func, code):
+        super_initialize_frame_scopes(self, outer_func, code)
         ncellvars = len(code.co_cellvars)
         nfreevars = len(code.co_freevars)
         if not nfreevars:
             if not ncellvars:
+                self.cells = []
                 return            # no self.cells needed - fast path
-            if closure is None:
-                closure = []
-        elif closure is None:
+        elif outer_func is None:
             space = self.space
             raise OperationError(space.w_TypeError,
                                  space.wrap("directly executed code object "
                                             "may not contain free variables"))
-        if len(closure) != nfreevars:
+        if outer_func and outer_func.closure:
+            closure_size = len(outer_func.closure)
+        else:
+            closure_size = 0
+        if closure_size != nfreevars:
             raise ValueError("code object received a closure with "
                                  "an unexpected number of free variables")
         self.cells = [None] * (ncellvars + nfreevars)
         for i in range(ncellvars):
             self.cells[i] = Cell()
         for i in range(nfreevars):
-            self.cells[i + ncellvars] = closure[i]
-    
+            self.cells[i + ncellvars] = outer_func.closure[i]
+
     def _getcells(self):
         return self.cells
 
diff --git a/pypy/interpreter/pycode.py b/pypy/interpreter/pycode.py
--- a/pypy/interpreter/pycode.py
+++ b/pypy/interpreter/pycode.py
@@ -198,7 +198,7 @@
 
     def funcrun(self, func, args):
         frame = self.space.createframe(self, func.w_func_globals,
-                                  func.closure)
+                                  func)
         sig = self._signature
         # speed hack
         fresh_frame = jit.hint(frame, access_directly=True,
@@ -211,7 +211,7 @@
 
     def funcrun_obj(self, func, w_obj, args):
         frame = self.space.createframe(self, func.w_func_globals,
-                                  func.closure)
+                                  func)
         sig = self._signature
         # speed hack
         fresh_frame = jit.hint(frame, access_directly=True,
diff --git a/pypy/interpreter/pyframe.py b/pypy/interpreter/pyframe.py
--- a/pypy/interpreter/pyframe.py
+++ b/pypy/interpreter/pyframe.py
@@ -51,7 +51,7 @@
     is_being_profiled        = False
     escaped                  = False  # see mark_as_escaped()
 
-    def __init__(self, space, code, w_globals, closure):
+    def __init__(self, space, code, w_globals, outer_func):
         if not we_are_translated():
             assert type(self) in (space.FrameClass, CPythonFrame), (
                 "use space.FrameClass(), not directly PyFrame()")
@@ -70,7 +70,7 @@
             self.builtin = space.builtin.pick_builtin(w_globals)
         # regular functions always have CO_OPTIMIZED and CO_NEWLOCALS.
         # class bodies only have CO_NEWLOCALS.
-        self.initialize_frame_scopes(closure, code)
+        self.initialize_frame_scopes(outer_func, code)
         self.f_lineno = code.co_firstlineno
 
     def mark_as_escaped(self):
@@ -117,8 +117,8 @@
             return self.builtin
         else:
             return self.space.builtin
-        
-    def initialize_frame_scopes(self, closure, code): 
+
+    def initialize_frame_scopes(self, outer_func, code):
         # regular functions always have CO_OPTIMIZED and CO_NEWLOCALS.
         # class bodies only have CO_NEWLOCALS.
         # CO_NEWLOCALS: make a locals dict unless optimized is also set
@@ -385,7 +385,11 @@
         
         # do not use the instance's __init__ but the base's, because we set
         # everything like cells from here
-        PyFrame.__init__(self, space, pycode, w_globals, closure)
+        # XXX hack
+        from pypy.interpreter.function import Function
+        outer_func = Function(space, None, closure=closure,
+                             forcename="fake")
+        PyFrame.__init__(self, space, pycode, w_globals, outer_func)
         f_back = space.interp_w(PyFrame, w_f_back, can_be_None=True)
         new_frame.f_backref = jit.non_virtual_ref(f_back)
 
diff --git a/pypy/interpreter/pyopcode.py b/pypy/interpreter/pyopcode.py
--- a/pypy/interpreter/pyopcode.py
+++ b/pypy/interpreter/pyopcode.py
@@ -1523,10 +1523,8 @@
 
         if not isinstance(prog, codetype):
             filename = ''
-            if not isinstance(prog, str):
-                if isinstance(prog, basestring):
-                    prog = str(prog)
-                elif isinstance(prog, file):
+            if not isinstance(prog, basestring):
+                if isinstance(prog, file):
                     filename = prog.name
                     prog = prog.read()
                 else:
diff --git a/pypy/interpreter/pyparser/future.py b/pypy/interpreter/pyparser/future.py
--- a/pypy/interpreter/pyparser/future.py
+++ b/pypy/interpreter/pyparser/future.py
@@ -109,25 +109,19 @@
             self.getc() == self.getc(+2)):
             self.pos += 3
             while 1: # Deal with a triple quoted docstring
-                if self.getc() == '\\':
-                    self.pos += 2
+                c = self.getc()
+                if c == '\\':
+                    self.pos += 1
+                    self._skip_next_char_from_docstring()
+                elif c != endchar:
+                    self._skip_next_char_from_docstring()
                 else:
-                    c = self.getc()
-                    if c != endchar:
-                        self.pos += 1
-                        if c == '\n':
-                            self.atbol()
-                        elif c == '\r':
-                            if self.getc() == '\n':
-                                self.pos += 1
-                                self.atbol()
-                    else:
-                        self.pos += 1
-                        if (self.getc() == endchar and
-                            self.getc(+1) == endchar):
-                            self.pos += 2
-                            self.consume_empty_line()
-                            break
+                    self.pos += 1
+                    if (self.getc() == endchar and
+                        self.getc(+1) == endchar):
+                        self.pos += 2
+                        self.consume_empty_line()
+                        break
 
         else: # Deal with a single quoted docstring
             self.pos += 1
@@ -138,17 +132,21 @@
                     self.consume_empty_line()
                     return
                 elif c == '\\':
-                    # Deal with linefeeds
-                    if self.getc() != '\r':
-                        self.pos += 1
-                    else:
-                        self.pos += 1
-                        if self.getc() == '\n':
-                            self.pos += 1
+                    self._skip_next_char_from_docstring()
                 elif c in '\r\n':
                     # Syntax error
                     return
 
+    def _skip_next_char_from_docstring(self):
+        c = self.getc()
+        self.pos += 1
+        if c == '\n':
+            self.atbol()
+        elif c == '\r':
+            if self.getc() == '\n':
+                self.pos += 1
+            self.atbol()
+
     def consume_continuation(self):
         c = self.getc()
         if c in '\n\r':
diff --git a/pypy/interpreter/pyparser/test/test_futureautomaton.py b/pypy/interpreter/pyparser/test/test_futureautomaton.py
--- a/pypy/interpreter/pyparser/test/test_futureautomaton.py
+++ b/pypy/interpreter/pyparser/test/test_futureautomaton.py
@@ -221,6 +221,14 @@
     assert f.lineno == 3
     assert f.col_offset == 0
 
+def test_lots_of_continuation_lines():
+    s = "\\\n\\\n\\\n\\\n\\\n\\\n\nfrom __future__ import with_statement\n"
+    f = run(s)
+    assert f.pos == len(s)
+    assert f.flags == fut.CO_FUTURE_WITH_STATEMENT
+    assert f.lineno == 8
+    assert f.col_offset == 0
+
 # This looks like a bug in cpython parser
 # and would require extensive modifications
 # to future.py in order to emulate the same behaviour
@@ -239,3 +247,19 @@
         raise AssertionError('IndentationError not raised')
     assert f.lineno == 2
     assert f.col_offset == 0
+
+def test_continuation_lines_in_docstring_single_quoted():
+    s = '"\\\n\\\n\\\n\\\n\\\n\\\n"\nfrom  __future__ import division\n'
+    f = run(s)
+    assert f.pos == len(s)
+    assert f.flags == fut.CO_FUTURE_DIVISION
+    assert f.lineno == 8
+    assert f.col_offset == 0
+
+def test_continuation_lines_in_docstring_triple_quoted():
+    s = '"""\\\n\\\n\\\n\\\n\\\n\\\n"""\nfrom  __future__ import division\n'
+    f = run(s)
+    assert f.pos == len(s)
+    assert f.flags == fut.CO_FUTURE_DIVISION
+    assert f.lineno == 8
+    assert f.col_offset == 0
diff --git a/pypy/interpreter/test/test_exec.py b/pypy/interpreter/test/test_exec.py
--- a/pypy/interpreter/test/test_exec.py
+++ b/pypy/interpreter/test/test_exec.py
@@ -219,3 +219,30 @@
             raise e
 
         assert res == 1
+
+    def test_exec_unicode(self):
+        # 's' is a string
+        s = "x = u'\xd0\xb9\xd1\x86\xd1\x83\xd0\xba\xd0\xb5\xd0\xbd'"
+        # 'u' is a unicode
+        u = s.decode('utf-8')
+        exec u
+        assert len(x) == 6
+        assert ord(x[0]) == 0x0439
+        assert ord(x[1]) == 0x0446
+        assert ord(x[2]) == 0x0443
+        assert ord(x[3]) == 0x043a
+        assert ord(x[4]) == 0x0435
+        assert ord(x[5]) == 0x043d
+
+    def test_eval_unicode(self):
+        u = "u'%s'" % unichr(0x1234)
+        v = eval(u)
+        assert v == unichr(0x1234)
+
+    def test_compile_unicode(self):
+        s = "x = u'\xd0\xb9\xd1\x86\xd1\x83\xd0\xba\xd0\xb5\xd0\xbd'"
+        u = s.decode('utf-8')
+        c = compile(u, '', 'exec')
+        exec c
+        assert len(x) == 6
+        assert ord(x[0]) == 0x0439
diff --git a/pypy/jit/backend/llgraph/runner.py b/pypy/jit/backend/llgraph/runner.py
--- a/pypy/jit/backend/llgraph/runner.py
+++ b/pypy/jit/backend/llgraph/runner.py
@@ -25,13 +25,14 @@
 class Descr(history.AbstractDescr):
 
     def __init__(self, ofs, typeinfo, extrainfo=None, name=None,
-                 arg_types=None, count_fields_if_immut=-1):
+                 arg_types=None, count_fields_if_immut=-1, ffi_flags=0):
         self.ofs = ofs
         self.typeinfo = typeinfo
         self.extrainfo = extrainfo
         self.name = name
         self.arg_types = arg_types
         self.count_fields_if_immut = count_fields_if_immut
+        self.ffi_flags = ffi_flags
 
     def get_arg_types(self):
         return self.arg_types
@@ -67,6 +68,9 @@
     def count_fields_if_immutable(self):
         return self.count_fields_if_immut
 
+    def get_ffi_flags(self):
+        return self.ffi_flags
+
     def __lt__(self, other):
         raise TypeError("cannot use comparison on Descrs")
     def __le__(self, other):
@@ -114,14 +118,14 @@
         return False
 
     def getdescr(self, ofs, typeinfo='?', extrainfo=None, name=None,
-                 arg_types=None, count_fields_if_immut=-1):
+                 arg_types=None, count_fields_if_immut=-1, ffi_flags=0):
         key = (ofs, typeinfo, extrainfo, name, arg_types,
-               count_fields_if_immut)
+               count_fields_if_immut, ffi_flags)
         try:
             return self._descrs[key]
         except KeyError:
             descr = Descr(ofs, typeinfo, extrainfo, name, arg_types,
-                          count_fields_if_immut)
+                          count_fields_if_immut, ffi_flags)
             self._descrs[key] = descr
             return descr
 
@@ -326,7 +330,7 @@
         return self.getdescr(0, token[0], extrainfo=extrainfo,
                              arg_types=''.join(arg_types))
 
-    def calldescrof_dynamic(self, ffi_args, ffi_result, extrainfo):
+    def calldescrof_dynamic(self, ffi_args, ffi_result, extrainfo, ffi_flags):
         from pypy.jit.backend.llsupport.ffisupport import get_ffi_type_kind
         from pypy.jit.backend.llsupport.ffisupport import UnsupportedKind
         arg_types = []
@@ -339,7 +343,8 @@
         except UnsupportedKind:
             return None
         return self.getdescr(0, reskind, extrainfo=extrainfo,
-                             arg_types=''.join(arg_types))
+                             arg_types=''.join(arg_types),
+                             ffi_flags=ffi_flags)
 
 
     def grab_exc_value(self):
diff --git a/pypy/jit/backend/llsupport/descr.py b/pypy/jit/backend/llsupport/descr.py
--- a/pypy/jit/backend/llsupport/descr.py
+++ b/pypy/jit/backend/llsupport/descr.py
@@ -260,10 +260,12 @@
     _clsname = ''
     loop_token = None
     arg_classes = ''     # <-- annotation hack
+    ffi_flags = 0
 
-    def __init__(self, arg_classes, extrainfo=None):
+    def __init__(self, arg_classes, extrainfo=None, ffi_flags=0):
         self.arg_classes = arg_classes    # string of "r" and "i" (ref/int)
         self.extrainfo = extrainfo
+        self.ffi_flags = ffi_flags
 
     def __repr__(self):
         res = '%s(%s)' % (self.__class__.__name__, self.arg_classes)
@@ -284,6 +286,13 @@
     def get_extra_info(self):
         return self.extrainfo
 
+    def get_ffi_flags(self):
+        return self.ffi_flags
+
+    def get_call_conv(self):
+        from pypy.rlib.clibffi import get_call_conv
+        return get_call_conv(self.ffi_flags, True)
+
     def get_arg_types(self):
         return self.arg_classes
 
@@ -391,8 +400,8 @@
     """
     _clsname = 'DynamicIntCallDescr'
 
-    def __init__(self, arg_classes, result_size, result_sign, extrainfo=None):
-        BaseIntCallDescr.__init__(self, arg_classes, extrainfo)
+    def __init__(self, arg_classes, result_size, result_sign, extrainfo=None, ffi_flags=0):
+        BaseIntCallDescr.__init__(self, arg_classes, extrainfo, ffi_flags)
         assert isinstance(result_sign, bool)
         self._result_size = chr(result_size)
         self._result_sign = result_sign
diff --git a/pypy/jit/backend/llsupport/ffisupport.py b/pypy/jit/backend/llsupport/ffisupport.py
--- a/pypy/jit/backend/llsupport/ffisupport.py
+++ b/pypy/jit/backend/llsupport/ffisupport.py
@@ -8,7 +8,7 @@
 class UnsupportedKind(Exception):
     pass
 
-def get_call_descr_dynamic(cpu, ffi_args, ffi_result, extrainfo=None):
+def get_call_descr_dynamic(cpu, ffi_args, ffi_result, extrainfo=None, ffi_flags=0):
     """Get a call descr: the types of result and args are represented by
     rlib.libffi.types.*"""
     try:
@@ -20,18 +20,24 @@
     if reskind == history.INT:
         size = intmask(ffi_result.c_size)
         signed = is_ffi_type_signed(ffi_result)
-        return DynamicIntCallDescr(arg_classes, size, signed, extrainfo)
+        return DynamicIntCallDescr(arg_classes, size, signed, extrainfo,
+                                   ffi_flags=ffi_flags)
     elif reskind == history.REF:
-        return  NonGcPtrCallDescr(arg_classes, extrainfo)
+        return  NonGcPtrCallDescr(arg_classes, extrainfo,
+                                  ffi_flags=ffi_flags)
     elif reskind == history.FLOAT:
-        return FloatCallDescr(arg_classes, extrainfo)
+        return FloatCallDescr(arg_classes, extrainfo,
+                              ffi_flags=ffi_flags)
     elif reskind == history.VOID:
-        return VoidCallDescr(arg_classes, extrainfo)
+        return VoidCallDescr(arg_classes, extrainfo,
+                             ffi_flags=ffi_flags)
     elif reskind == 'L':
-        return LongLongCallDescr(arg_classes, extrainfo)
+        return LongLongCallDescr(arg_classes, extrainfo,
+                                 ffi_flags=ffi_flags)
     elif reskind == 'S':
         SingleFloatCallDescr = getCallDescrClass(rffi.FLOAT)
-        return SingleFloatCallDescr(arg_classes, extrainfo)
+        return SingleFloatCallDescr(arg_classes, extrainfo,
+                                    ffi_flags=ffi_flags)
     assert False
 
 def get_ffi_type_kind(cpu, ffi_type):
diff --git a/pypy/jit/backend/llsupport/llmodel.py b/pypy/jit/backend/llsupport/llmodel.py
--- a/pypy/jit/backend/llsupport/llmodel.py
+++ b/pypy/jit/backend/llsupport/llmodel.py
@@ -257,10 +257,10 @@
     def calldescrof(self, FUNC, ARGS, RESULT, extrainfo):
         return get_call_descr(self.gc_ll_descr, ARGS, RESULT, extrainfo)
 
-    def calldescrof_dynamic(self, ffi_args, ffi_result, extrainfo):
+    def calldescrof_dynamic(self, ffi_args, ffi_result, extrainfo, ffi_flags):
         from pypy.jit.backend.llsupport import ffisupport
         return ffisupport.get_call_descr_dynamic(self, ffi_args, ffi_result,
-                                                 extrainfo)
+                                                 extrainfo, ffi_flags)
 
     def get_overflow_error(self):
         ovf_vtable = self.cast_adr_to_int(self._ovf_error_vtable)
diff --git a/pypy/jit/backend/llsupport/regalloc.py b/pypy/jit/backend/llsupport/regalloc.py
--- a/pypy/jit/backend/llsupport/regalloc.py
+++ b/pypy/jit/backend/llsupport/regalloc.py
@@ -57,11 +57,13 @@
     all_regs              = []
     no_lower_byte_regs    = []
     save_around_call_regs = []
-    
+    frame_reg             = None
+
     def __init__(self, longevity, frame_manager=None, assembler=None):
         self.free_regs = self.all_regs[:]
         self.longevity = longevity
         self.reg_bindings = {}
+        self.bindings_to_frame_reg = {}
         self.position = -1
         self.frame_manager = frame_manager
         self.assembler = assembler
@@ -218,6 +220,10 @@
         self.reg_bindings[v] = loc
         return loc
 
+    def force_allocate_frame_reg(self, v):
+        """ Allocate the new variable v in the frame register."""
+        self.bindings_to_frame_reg[v] = None
+
     def force_spill_var(self, var):
         self._sync_var(var)
         try:
@@ -236,6 +242,8 @@
         try:
             return self.reg_bindings[box]
         except KeyError:
+            if box in self.bindings_to_frame_reg:
+                return self.frame_reg
             return self.frame_manager.loc(box)
 
     def return_constant(self, v, forbidden_vars=[], selected_reg=None):
@@ -264,8 +272,9 @@
         self._check_type(v)
         if isinstance(v, Const):
             return self.return_constant(v, forbidden_vars, selected_reg)
-        
         prev_loc = self.loc(v)
+        if prev_loc is self.frame_reg and selected_reg is None:
+            return prev_loc
         loc = self.force_allocate_reg(v, forbidden_vars, selected_reg,
                                       need_lower_byte=need_lower_byte)
         if prev_loc is not loc:
diff --git a/pypy/jit/backend/llsupport/test/test_ffisupport.py b/pypy/jit/backend/llsupport/test/test_ffisupport.py
--- a/pypy/jit/backend/llsupport/test/test_ffisupport.py
+++ b/pypy/jit/backend/llsupport/test/test_ffisupport.py
@@ -13,17 +13,19 @@
 
 def test_call_descr_dynamic():
     args = [types.sint, types.pointer]
-    descr = get_call_descr_dynamic(FakeCPU(), args, types.sint)
+    descr = get_call_descr_dynamic(FakeCPU(), args, types.sint, ffi_flags=42)
     assert isinstance(descr, DynamicIntCallDescr)
     assert descr.arg_classes == 'ii'
+    assert descr.get_ffi_flags() == 42
 
     args = [types.sint, types.double, types.pointer]
     descr = get_call_descr_dynamic(FakeCPU(), args, types.void)
     assert descr is None    # missing floats
     descr = get_call_descr_dynamic(FakeCPU(supports_floats=True),
-                                   args, types.void)
+                                   args, types.void, ffi_flags=43)
     assert isinstance(descr, VoidCallDescr)
     assert descr.arg_classes == 'ifi'
+    assert descr.get_ffi_flags() == 43
 
     descr = get_call_descr_dynamic(FakeCPU(), [], types.sint8)
     assert isinstance(descr, DynamicIntCallDescr)
@@ -39,14 +41,16 @@
         descr = get_call_descr_dynamic(FakeCPU(), [], types.slonglong)
         assert descr is None   # missing longlongs
         descr = get_call_descr_dynamic(FakeCPU(supports_longlong=True),
-                                       [], types.slonglong)
+                                       [], types.slonglong, ffi_flags=43)
         assert isinstance(descr, LongLongCallDescr)
+        assert descr.get_ffi_flags() == 43
     else:
         assert types.slonglong is types.slong
 
     descr = get_call_descr_dynamic(FakeCPU(), [], types.float)
     assert descr is None   # missing singlefloats
     descr = get_call_descr_dynamic(FakeCPU(supports_singlefloats=True),
-                                   [], types.float)
+                                   [], types.float, ffi_flags=44)
     SingleFloatCallDescr = getCallDescrClass(rffi.FLOAT)
     assert isinstance(descr, SingleFloatCallDescr)
+    assert descr.get_ffi_flags() == 44
diff --git a/pypy/jit/backend/test/runner_test.py b/pypy/jit/backend/test/runner_test.py
--- a/pypy/jit/backend/test/runner_test.py
+++ b/pypy/jit/backend/test/runner_test.py
@@ -468,7 +468,7 @@
             assert longlong.getrealfloat(x) == 3.5 - 42
 
     def test_call(self):
-        from pypy.rlib.libffi import types
+        from pypy.rlib.libffi import types, FUNCFLAG_CDECL
 
         def func_int(a, b):
             return a + b
@@ -497,7 +497,8 @@
             assert res.value == 2 * num
             # then, try it with the dynamic calldescr
             dyn_calldescr = cpu.calldescrof_dynamic([ffi_type, ffi_type], ffi_type,
-                                                    EffectInfo.MOST_GENERAL)
+                                                    EffectInfo.MOST_GENERAL,
+                                                    ffi_flags=FUNCFLAG_CDECL)
             res = self.execute_operation(rop.CALL,
                                          [funcbox, BoxInt(num), BoxInt(num)],
                                          'int', descr=dyn_calldescr)
@@ -1944,7 +1945,7 @@
         assert values == [1, 10]
 
     def test_call_to_c_function(self):
-        from pypy.rlib.libffi import CDLL, types, ArgChain
+        from pypy.rlib.libffi import CDLL, types, ArgChain, FUNCFLAG_CDECL
         from pypy.rpython.lltypesystem.ll2ctypes import libc_name
         libc = CDLL(libc_name)
         c_tolower = libc.getpointer('tolower', [types.uchar], types.sint)
@@ -1955,7 +1956,8 @@
         func_adr = llmemory.cast_ptr_to_adr(c_tolower.funcsym)
         funcbox = ConstInt(heaptracker.adr2int(func_adr))
         calldescr = cpu.calldescrof_dynamic([types.uchar], types.sint,
-                                            EffectInfo.MOST_GENERAL)
+                                            EffectInfo.MOST_GENERAL,
+                                            ffi_flags=FUNCFLAG_CDECL)
         i1 = BoxInt()
         i2 = BoxInt()
         tok = BoxInt()
@@ -2012,7 +2014,8 @@
         calldescr = cpu.calldescrof_dynamic([types.pointer, types_size_t,
                                              types_size_t, types.pointer],
                                             types.void,
-                                            EffectInfo.MOST_GENERAL)
+                                            EffectInfo.MOST_GENERAL,
+                                            ffi_flags=clibffi.FUNCFLAG_CDECL)
         i0 = BoxInt()
         i1 = BoxInt()
         i2 = BoxInt()
@@ -2038,6 +2041,62 @@
         assert len(glob.lst) > 0
         lltype.free(raw, flavor='raw')
 
+    def test_call_to_winapi_function(self):
+        from pypy.rlib.clibffi import _WIN32, FUNCFLAG_STDCALL
+        if not _WIN32:
+            py.test.skip("Windows test only")
+        from pypy.rlib.libffi import CDLL, types, ArgChain
+        from pypy.rlib.rwin32 import DWORD
+        libc = CDLL('KERNEL32')
+        c_GetCurrentDir = libc.getpointer('GetCurrentDirectoryA',
+                                          [types.ulong, types.pointer],
+                                          types.ulong)
+
+        cwd = os.getcwd()
+        buflen = len(cwd) + 10
+        buffer = lltype.malloc(rffi.CCHARP.TO, buflen, flavor='raw')
+        argchain = ArgChain().arg(rffi.cast(DWORD, buflen)).arg(buffer)
+        res = c_GetCurrentDir.call(argchain, DWORD)
+        assert rffi.cast(lltype.Signed, res) == len(cwd)
+        assert rffi.charp2strn(buffer, buflen) == cwd
+        lltype.free(buffer, flavor='raw')
+
+        cpu = self.cpu
+        func_adr = llmemory.cast_ptr_to_adr(c_GetCurrentDir.funcsym)
+        funcbox = ConstInt(heaptracker.adr2int(func_adr))
+        calldescr = cpu.calldescrof_dynamic([types.ulong, types.pointer],
+                                            types.ulong,
+                                            EffectInfo.MOST_GENERAL,
+                                            ffi_flags=FUNCFLAG_STDCALL)
+        i1 = BoxInt()
+        i2 = BoxInt()
+        faildescr = BasicFailDescr(1)
+        # if the stdcall convention is ignored, then ESP is wrong after the
+        # call: 8 bytes too much.  If we repeat the call often enough, crash.
+        ops = []
+        for i in range(50):
+            i3 = BoxInt()
+            ops += [
+                ResOperation(rop.CALL_RELEASE_GIL, [funcbox, i1, i2], i3,
+                             descr=calldescr),
+                ResOperation(rop.GUARD_NOT_FORCED, [], None, descr=faildescr),
+                ]
+            ops[-1].setfailargs([])
+        ops += [
+            ResOperation(rop.FINISH, [i3], None, descr=BasicFailDescr(0))
+        ]
+        looptoken = LoopToken()
+        self.cpu.compile_loop([i1, i2], ops, looptoken)
+
+        buffer = lltype.malloc(rffi.CCHARP.TO, buflen, flavor='raw')
+        self.cpu.set_future_value_int(0, buflen)
+        self.cpu.set_future_value_int(1, rffi.cast(lltype.Signed, buffer))
+        fail = self.cpu.execute_token(looptoken)
+        assert fail.identifier == 0
+        assert self.cpu.get_latest_value_int(0) == len(cwd)
+        assert rffi.charp2strn(buffer, buflen) == cwd
+        lltype.free(buffer, flavor='raw')
+
     def test_guard_not_invalidated(self):
         cpu = self.cpu
         i0 = BoxInt()
diff --git a/pypy/jit/backend/x86/assembler.py b/pypy/jit/backend/x86/assembler.py
--- a/pypy/jit/backend/x86/assembler.py
+++ b/pypy/jit/backend/x86/assembler.py
@@ -34,6 +34,7 @@
 from pypy.rlib.debug import (debug_print, debug_start, debug_stop,
                              have_debug_prints)
 from pypy.rlib import rgc
+from pypy.rlib.clibffi import FFI_DEFAULT_ABI
 from pypy.jit.backend.x86.jump import remap_frame_layout
 from pypy.jit.metainterp.history import ConstInt, BoxInt
 from pypy.jit.codewriter.effectinfo import EffectInfo
@@ -956,6 +957,7 @@
         if (isinstance(from_loc, RegLoc) and from_loc.is_xmm) or (isinstance(to_loc, RegLoc) and to_loc.is_xmm):
             self.mc.MOVSD(to_loc, from_loc)
         else:
+            assert to_loc is not ebp
             self.mc.MOV(to_loc, from_loc)
 
     regalloc_mov = mov # legacy interface
@@ -1120,7 +1122,7 @@
         return genop_cmp_guard_float
 
     def _emit_call(self, force_index, x, arglocs, start=0, tmp=eax,
-                   argtypes=None):
+                   argtypes=None, callconv=FFI_DEFAULT_ABI):
         if IS_X86_64:
             return self._emit_call_64(force_index, x, arglocs, start, argtypes)
 
@@ -1149,6 +1151,16 @@
         # x is a location
         self.mc.CALL(x)
         self.mark_gc_roots(force_index)
+        #
+        if callconv != FFI_DEFAULT_ABI:
+            self._fix_stdcall(callconv, p)
+
+    def _fix_stdcall(self, callconv, p):
+        from pypy.rlib.clibffi import FFI_STDCALL
+        assert callconv == FFI_STDCALL
+        # it's a bit stupid, but we're just going to cancel the fact that
+        # the called function just added 'p' to ESP, by subtracting it again.
+        self.mc.SUB_ri(esp.value, p)
 
     def _emit_call_64(self, force_index, x, arglocs, start, argtypes):
         src_locs = []
@@ -2127,7 +2139,8 @@
             tmp = eax
 
         self._emit_call(force_index, x, arglocs, 3, tmp=tmp,
-                        argtypes=op.getdescr().get_arg_types())
+                        argtypes=op.getdescr().get_arg_types(),
+                        callconv=op.getdescr().get_call_conv())
 
         if IS_X86_32 and isinstance(resloc, StackLoc) and resloc.width == 8:
             # a float or a long long return
@@ -2498,11 +2511,6 @@
 
     genop_discard_cond_call_gc_wb_array = genop_discard_cond_call_gc_wb
 
-    def genop_force_token(self, op, arglocs, resloc):
-        # RegAlloc.consider_force_token ensures this:
-        assert isinstance(resloc, RegLoc)
-        self.mc.LEA_rb(resloc.value, FORCE_INDEX_OFS)
-
     def not_implemented_op_discard(self, op, arglocs):
         not_implemented("not implemented operation: %s" % op.getopname())
 
diff --git a/pypy/jit/backend/x86/regalloc.py b/pypy/jit/backend/x86/regalloc.py
--- a/pypy/jit/backend/x86/regalloc.py
+++ b/pypy/jit/backend/x86/regalloc.py
@@ -29,6 +29,7 @@
     all_regs = [eax, ecx, edx, ebx, esi, edi]
     no_lower_byte_regs = [esi, edi]
     save_around_call_regs = [eax, edx, ecx]
+    frame_reg = ebp
 
     REGLOC_TO_GCROOTMAP_REG_INDEX = {
         ebx: 1,
@@ -312,8 +313,11 @@
                     self.fm.frame_bindings[arg] = loc
             else:
                 if isinstance(loc, RegLoc):
-                    self.rm.reg_bindings[arg] = loc
-                    used[loc] = None
+                    if loc is ebp:
+                        self.rm.bindings_to_frame_reg[arg] = None
+                    else:
+                        self.rm.reg_bindings[arg] = loc
+                        used[loc] = None
                 else:
                     self.fm.frame_bindings[arg] = loc
         self.rm.free_regs = []
@@ -1358,8 +1362,8 @@
                                             self.assembler.datablockwrapper)
 
     def consider_force_token(self, op):
-        loc = self.rm.force_allocate_reg(op.result)
-        self.Perform(op, [], loc)
+        # the FORCE_TOKEN operation returns directly 'ebp'
+        self.rm.force_allocate_frame_reg(op.result)
 
     def not_implemented_op(self, op):
         not_implemented("not implemented operation: %s" % op.getopname())
diff --git a/pypy/jit/backend/x86/runner.py b/pypy/jit/backend/x86/runner.py
--- a/pypy/jit/backend/x86/runner.py
+++ b/pypy/jit/backend/x86/runner.py
@@ -119,7 +119,8 @@
             setitem(index, null)
 
     def get_latest_force_token(self):
-        return self.assembler.fail_ebp + FORCE_INDEX_OFS
+        # the FORCE_TOKEN operation and this helper both return 'ebp'.
+        return self.assembler.fail_ebp
 
     def execute_token(self, executable_token):
         addr = executable_token._x86_bootstrap_code
@@ -153,8 +154,9 @@
                                        flavor='raw', zero=True,
                                        immortal=True)
 
-    def force(self, addr_of_force_index):
+    def force(self, addr_of_force_token):
         TP = rffi.CArrayPtr(lltype.Signed)
+        addr_of_force_index = addr_of_force_token + FORCE_INDEX_OFS
         fail_index = rffi.cast(TP, addr_of_force_index)[0]
         assert fail_index >= 0, "already forced!"
         faildescr = self.get_fail_descr_from_number(fail_index)
@@ -164,7 +166,7 @@
         # start of "no gc operation!" block
         fail_index_2 = self.assembler.grab_frame_values(
             bytecode,
-            addr_of_force_index - FORCE_INDEX_OFS,
+            addr_of_force_token,
             self.all_null_registers)
         self.assembler.leave_jitted_hook()
         # end of "no gc operation!" block
diff --git a/pypy/jit/backend/x86/rx86.py b/pypy/jit/backend/x86/rx86.py
--- a/pypy/jit/backend/x86/rx86.py
+++ b/pypy/jit/backend/x86/rx86.py
@@ -527,6 +527,7 @@
 
     NOP = insn('\x90')
     RET = insn('\xC3')
+    RET16_i = insn('\xC2', immediate(1, 'h'))
 
     PUSH_r = insn(rex_nw, register(1), '\x50')
     PUSH_b = insn(rex_nw, '\xFF', orbyte(6<<3), stack_bp(1))
diff --git a/pypy/jit/backend/x86/test/test_runner.py b/pypy/jit/backend/x86/test/test_runner.py
--- a/pypy/jit/backend/x86/test/test_runner.py
+++ b/pypy/jit/backend/x86/test/test_runner.py
@@ -433,6 +433,88 @@
                 ops_offset[operations[2]] <=
                 ops_offset[None])
 
+    def test_calling_convention(self, monkeypatch):
+        if WORD != 4:
+            py.test.skip("32-bit only test")
+        from pypy.jit.backend.x86.regloc import eax, edx
+        from pypy.jit.backend.x86 import codebuf
+        from pypy.jit.codewriter.effectinfo import EffectInfo
+        from pypy.rlib.libffi import types, clibffi
+        had_stdcall = hasattr(clibffi, 'FFI_STDCALL')
+        if not had_stdcall:    # not running on Windows, but we can still test
+            monkeypatch.setattr(clibffi, 'FFI_STDCALL', 12345, raising=False)
+        #
+        for ffi in [clibffi.FFI_DEFAULT_ABI, clibffi.FFI_STDCALL]:
+            cpu = self.cpu
+            mc = codebuf.MachineCodeBlockWrapper()
+            mc.MOV_rs(eax.value, 4)      # argument 1
+            mc.MOV_rs(edx.value, 40)     # argument 10
+            mc.SUB_rr(eax.value, edx.value)     # return arg1 - arg10
+            if ffi == clibffi.FFI_DEFAULT_ABI:
+                mc.RET()
+            else:
+                mc.RET16_i(40)
+            rawstart = mc.materialize(cpu.asmmemmgr, [])
+            #
+            calldescr = cpu.calldescrof_dynamic([types.slong] * 10,
+                                                types.slong,
+                                                EffectInfo.MOST_GENERAL,
+                                                ffi_flags=-1)
+            calldescr.get_call_conv = lambda: ffi      # <==== hack
+            funcbox = ConstInt(rawstart)
+            i1 = BoxInt()
+            i2 = BoxInt()
+            i3 = BoxInt()
+            i4 = BoxInt()
+            i5 = BoxInt()
+            i6 = BoxInt()
+            c = ConstInt(-1)
+            faildescr = BasicFailDescr(1)
+            # we must call it repeatedly: if the stack pointer gets increased
+            # by 40 bytes by the STDCALL call, and if we don't expect it,
+            # then we are going to get our stack emptied unexpectedly by
+            # several repeated calls
+            ops = [
+            ResOperation(rop.CALL_RELEASE_GIL,
+                         [funcbox, i1, c, c, c, c, c, c, c, c, i2],
+                         i3, descr=calldescr),
+            ResOperation(rop.GUARD_NOT_FORCED, [], None, descr=faildescr),
+
+            ResOperation(rop.CALL_RELEASE_GIL,
+                         [funcbox, i1, c, c, c, c, c, c, c, c, i2],
+                         i4, descr=calldescr),
+            ResOperation(rop.GUARD_NOT_FORCED, [], None, descr=faildescr),
+
+            ResOperation(rop.CALL_RELEASE_GIL,
+                         [funcbox, i1, c, c, c, c, c, c, c, c, i2],
+                         i5, descr=calldescr),
+            ResOperation(rop.GUARD_NOT_FORCED, [], None, descr=faildescr),
+
+            ResOperation(rop.CALL_RELEASE_GIL,
+                         [funcbox, i1, c, c, c, c, c, c, c, c, i2],
+                         i6, descr=calldescr),
+            ResOperation(rop.GUARD_NOT_FORCED, [], None, descr=faildescr),
+
+            ResOperation(rop.FINISH, [i3, i4, i5, i6], None,
+                         descr=BasicFailDescr(0))
+            ]
+            ops[1].setfailargs([])
+            ops[3].setfailargs([])
+            ops[5].setfailargs([])
+            ops[7].setfailargs([])
+            looptoken = LoopToken()
+            self.cpu.compile_loop([i1, i2], ops, looptoken)
+
+            self.cpu.set_future_value_int(0, 123450)
+            self.cpu.set_future_value_int(1, 123408)
+            fail = self.cpu.execute_token(looptoken)
+            assert fail.identifier == 0
+            assert self.cpu.get_latest_value_int(0) == 42
+            assert self.cpu.get_latest_value_int(1) == 42
+            assert self.cpu.get_latest_value_int(2) == 42
+            assert self.cpu.get_latest_value_int(3) == 42
+
+
 class TestDebuggingAssembler(object):
     def setup_method(self, meth):
         self.cpu = CPU(rtyper=None, stats=FakeStats())
diff --git a/pypy/jit/codewriter/support.py b/pypy/jit/codewriter/support.py
--- a/pypy/jit/codewriter/support.py
+++ b/pypy/jit/codewriter/support.py
@@ -91,9 +91,12 @@
     reds_v = op.args[2+numgreens:]
     assert len(reds_v) == numreds
     #
-    def _sort(args_v):
+    def _sort(args_v, is_green):
         from pypy.jit.metainterp.history import getkind
         lst = [v for v in args_v if v.concretetype is not lltype.Void]
+        if is_green:
+            assert len(lst) == len(args_v), (
+                "not supported so far: 'greens' variables contain Void")
         _kind2count = {'int': 1, 'ref': 2, 'float': 3}
         lst2 = sorted(lst, key=lambda v: _kind2count[getkind(v.concretetype)])
         # a crash here means that you have to reorder the variable named in
@@ -102,7 +105,7 @@
         assert lst == lst2
         return lst
     #
-    return (_sort(greens_v), _sort(reds_v))
+    return (_sort(greens_v, True), _sort(reds_v, False))
 
 def maybe_on_top_of_llinterp(rtyper, fnptr):
     # Run a generated graph on top of the llinterp for testing.
diff --git a/pypy/jit/metainterp/optimizeopt/fficall.py b/pypy/jit/metainterp/optimizeopt/fficall.py
--- a/pypy/jit/metainterp/optimizeopt/fficall.py
+++ b/pypy/jit/metainterp/optimizeopt/fficall.py
@@ -18,26 +18,27 @@
     def __init__(self, funcval, cpu, prepare_op):
         self.funcval = funcval
         self.opargs = []
-        argtypes, restype = self._get_signature(funcval)
+        argtypes, restype, flags = self._get_signature(funcval)
         self.descr = cpu.calldescrof_dynamic(argtypes, restype,
-                                             EffectInfo.MOST_GENERAL)
+                                             EffectInfo.MOST_GENERAL,
+                                             ffi_flags=flags)
         # ^^^ may be None if unsupported
         self.prepare_op = prepare_op
         self.delayed_ops = []
 
     def _get_signature(self, funcval):
         """
-        given the funcval, return a tuple (argtypes, restype), where the
-        actuall types are libffi.types.*
+        given the funcval, return a tuple (argtypes, restype, flags), where
+        the actuall types are libffi.types.*
 
         The implementation is tricky because we have three possible cases:
 
         - translated: the easiest case, we can just cast back the pointer to
-          the original Func instance and read .argtypes and .restype
+          the original Func instance and read .argtypes, .restype and .flags
 
         - completely untranslated: this is what we get from test_optimizeopt
           tests. funcval contains a FakeLLObject whose _fake_class is Func,
-          and we can just get .argtypes and .restype
+          and we can just get .argtypes, .restype and .flags
 
         - partially translated: this happens when running metainterp tests:
           funcval contains the low-level equivalent of a Func, and thus we
@@ -49,10 +50,10 @@
         llfunc = funcval.box.getref_base()
         if we_are_translated():
             func = cast_base_ptr_to_instance(Func, llfunc)
-            return func.argtypes, func.restype
+            return func.argtypes, func.restype, func.flags
         elif getattr(llfunc, '_fake_class', None) is Func:
             # untranslated
-            return llfunc.argtypes, llfunc.restype
+            return llfunc.argtypes, llfunc.restype, llfunc.flags
         else:
             # partially translated
             # llfunc contains an opaque pointer to something like the following:
@@ -63,7 +64,7 @@
             # because we don't have the exact TYPE to cast to.  Instead, we
             # just fish it manually :-(
             f = llfunc._obj.container
-            return f.inst_argtypes, f.inst_restype
+            return f.inst_argtypes, f.inst_restype, f.inst_flags
 
 
 class OptFfiCall(Optimization):
diff --git a/pypy/jit/metainterp/optimizeopt/heap.py b/pypy/jit/metainterp/optimizeopt/heap.py
--- a/pypy/jit/metainterp/optimizeopt/heap.py
+++ b/pypy/jit/metainterp/optimizeopt/heap.py
@@ -25,7 +25,7 @@
         #      'cached_fields'.
         #
         self._cached_fields = {}
-        self._cached_fields_getfield_op = {}        
+        self._cached_fields_getfield_op = {}
         self._lazy_setfield = None
         self._lazy_setfield_registered = False
 
@@ -75,7 +75,7 @@
     def remember_field_value(self, structvalue, fieldvalue, getfield_op=None):
         assert self._lazy_setfield is None
         self._cached_fields[structvalue] = fieldvalue
-        self._cached_fields_getfield_op[structvalue] = getfield_op        
+        self._cached_fields_getfield_op[structvalue] = getfield_op
 
     def force_lazy_setfield(self, optheap, can_cache=True):
         op = self._lazy_setfield
@@ -163,7 +163,7 @@
 
     def new(self):
         return OptHeap()
-        
+
     def produce_potential_short_preamble_ops(self, sb):
         descrkeys = self.cached_fields.keys()
         if not we_are_translated():
diff --git a/pypy/jit/metainterp/optimizeopt/test/test_optimizebasic.py b/pypy/jit/metainterp/optimizeopt/test/test_optimizebasic.py
--- a/pypy/jit/metainterp/optimizeopt/test/test_optimizebasic.py
+++ b/pypy/jit/metainterp/optimizeopt/test/test_optimizebasic.py
@@ -4711,6 +4711,33 @@
         """
         self.optimize_loop(ops, expected)
 
+    def test_forced_virtuals_aliasing(self):
+        ops = """
+        [i0, i1]
+        p0 = new(descr=ssize)
+        p1 = new(descr=ssize)
+        escape(p0)
+        escape(p1)
+        setfield_gc(p0, i0, descr=adescr)
+        setfield_gc(p1, i1, descr=adescr)
+        i2 = getfield_gc(p0, descr=adescr)
+        jump(i2, i2)
+        """
+        expected = """
+        [i0, i1]
+        p0 = new(descr=ssize)
+        escape(p0)
+        p1 = new(descr=ssize)
+        escape(p1)
+        setfield_gc(p0, i0, descr=adescr)
+        setfield_gc(p1, i1, descr=adescr)
+        jump(i0, i0)
+        """
+        py.test.skip("not implemented")
+        # setfields on things that used to be virtual still can't alias each
+        # other
+        self.optimize_loop(ops, expected)
+
 
 class TestLLtype(BaseTestOptimizeBasic, LLtypeMixin):
     pass
diff --git a/pypy/jit/metainterp/optimizeopt/test/test_optimizefficall.py b/pypy/jit/metainterp/optimizeopt/test/test_optimizefficall.py
--- a/pypy/jit/metainterp/optimizeopt/test/test_optimizefficall.py
+++ b/pypy/jit/metainterp/optimizeopt/test/test_optimizefficall.py
@@ -14,12 +14,15 @@
     can check that the signature of a call is really what you want.
     """
 
-    def __init__(self, arg_types, typeinfo):
+    def __init__(self, arg_types, typeinfo, flags):
         self.arg_types = arg_types
         self.typeinfo = typeinfo   # return type
+        self.flags = flags
 
     def __eq__(self, other):
-        return self.arg_types == other.arg_types and self.typeinfo == other.typeinfo
+        return (self.arg_types == other.arg_types and
+                self.typeinfo == other.typeinfo and
+                self.flags == other.get_ffi_flags())
 
 class FakeLLObject(object):
 
@@ -41,14 +44,17 @@
         vable_token_descr = LLtypeMixin.valuedescr
         valuedescr = LLtypeMixin.valuedescr
 
-        int_float__int = MyCallDescr('if', 'i')
+        int_float__int_42 = MyCallDescr('if', 'i', 42)
+        int_float__int_43 = MyCallDescr('if', 'i', 43)
         funcptr = FakeLLObject()
         func = FakeLLObject(_fake_class=Func,
                             argtypes=[types.sint, types.double],
-                            restype=types.sint)
+                            restype=types.sint,
+                            flags=42)
         func2 = FakeLLObject(_fake_class=Func,
                              argtypes=[types.sint, types.double],
-                             restype=types.sint)
+                             restype=types.sint,
+                             flags=43)
         #
         def calldescr(cpu, FUNC, oopspecindex, extraeffect=None):
             if extraeffect == EffectInfo.EF_RANDOM_EFFECTS:
@@ -83,7 +89,7 @@
         """
         expected = """
         [i0, f1]
-        i3 = call_release_gil(12345, i0, f1, descr=int_float__int)
+        i3 = call_release_gil(12345, i0, f1, descr=int_float__int_42)
         guard_not_forced() []
         guard_no_exception() []
         jump(i3, f1)
@@ -123,7 +129,7 @@
         [i0, f1, p2]
         i4 = force_token()
         setfield_gc(p2, i4, descr=vable_token_descr)
-        i3 = call_release_gil(12345, i0, f1, descr=int_float__int)
+        i3 = call_release_gil(12345, i0, f1, descr=int_float__int_42)
         guard_not_forced() [p2]
         guard_no_exception() [p2]
         jump(i3, f1, p2)
@@ -220,7 +226,7 @@
         call(0, ConstPtr(func),                        descr=libffi_prepare)
         #
         # this "nested" call is nicely optimized
-        i4 = call_release_gil(67890, i0, f1, descr=int_float__int)
+        i4 = call_release_gil(67890, i0, f1, descr=int_float__int_43)
         guard_not_forced() []
         guard_no_exception() []
         #
@@ -265,7 +271,7 @@
         expected = """
         [i0, f1, p2]
         setfield_gc(p2, i0, descr=valuedescr)
-        i3 = call_release_gil(12345, i0, f1, descr=int_float__int)
+        i3 = call_release_gil(12345, i0, f1, descr=int_float__int_42)
         guard_not_forced() []
         guard_no_exception() []
         jump(i3, f1, p2)
diff --git a/pypy/jit/metainterp/test/test_warmspot.py b/pypy/jit/metainterp/test/test_warmspot.py
--- a/pypy/jit/metainterp/test/test_warmspot.py
+++ b/pypy/jit/metainterp/test/test_warmspot.py
@@ -252,6 +252,41 @@
         self.check_loops({'int_sub': 1, 'int_gt': 1, 'guard_true': 1,
                           'jump': 1})
 
+    def test_void_red_variable(self):
+        mydriver = JitDriver(greens=[], reds=['a', 'm'])
+        def f1(m):
+            a = None
+            while m > 0:
+                mydriver.jit_merge_point(a=a, m=m)
+                m = m - 1
+                if m == 10:
+                    pass   # other case
+        self.meta_interp(f1, [18])
+
+    def test_bug_constant_rawptrs(self):
+        py.test.skip("crashes because a is a constant")
+        from pypy.rpython.lltypesystem import lltype, rffi
+        mydriver = JitDriver(greens=['a'], reds=['m'])
+        def f1(m):
+            a = lltype.nullptr(rffi.VOIDP.TO)
+            while m > 0:
+                mydriver.jit_merge_point(a=a, m=m)
+                m = m - 1
+        self.meta_interp(f1, [18])
+
+    def test_bug_rawptrs(self):
+        from pypy.rpython.lltypesystem import lltype, rffi
+        mydriver = JitDriver(greens=['a'], reds=['m'])
+        def f1(m):
+            a = lltype.malloc(rffi.VOIDP.TO, 5, flavor='raw')
+            while m > 0:
+                mydriver.jit_merge_point(a=a, m=m)
+                m = m - 1
+                if m == 10:
+                    pass
+            lltype.free(a, flavor='raw')
+        self.meta_interp(f1, [18])
+
 
 class TestLLWarmspot(WarmspotTests, LLJitMixin):
     CPUClass = runner.LLtypeCPU
diff --git a/pypy/jit/metainterp/warmspot.py b/pypy/jit/metainterp/warmspot.py
--- a/pypy/jit/metainterp/warmspot.py
+++ b/pypy/jit/metainterp/warmspot.py
@@ -130,8 +130,15 @@
     results = _find_jit_marker(graphs, 'jit_merge_point')
     if not results:
         raise Exception("no jit_merge_point found!")
+    seen = set([graph for graph, block, pos in results])
+    assert len(seen) == len(results), (
+        "found several jit_merge_points in the same graph")
     return results
 
+def locate_jit_merge_point(graph):
+    [(graph, block, pos)] = find_jit_merge_points([graph])
+    return block, pos, block.operations[pos]
+
 def find_set_param(graphs):
     return _find_jit_marker(graphs, 'set_param')
 
@@ -235,7 +242,7 @@
     def split_graph_and_record_jitdriver(self, graph, block, pos):
         op = block.operations[pos]
         jd = JitDriverStaticData()
-        jd._jit_merge_point_pos = (graph, op)
+        jd._jit_merge_point_in = graph
         args = op.args[2:]
         s_binding = self.translator.annotator.binding
         jd._portal_args_s = [s_binding(v) for v in args]
@@ -245,7 +252,8 @@
         graph.startblock = support.split_before_jit_merge_point(*jmpp)
         graph.startblock.isstartblock = True
         # a crash in the following checkgraph() means that you forgot
-        # to list some variable in greens=[] or reds=[] in JitDriver.
+        # to list some variable in greens=[] or reds=[] in JitDriver,
+        # or that a jit_merge_point() takes a constant as an argument.
         checkgraph(graph)
         for v in graph.getargs():
             assert isinstance(v, Variable)
@@ -503,7 +511,8 @@
             self.make_args_specification(jd)
 
     def make_args_specification(self, jd):
-        graph, op = jd._jit_merge_point_pos
+        graph = jd._jit_merge_point_in
+        _, _, op = locate_jit_merge_point(graph)
         greens_v, reds_v = support.decode_hp_hint_args(op)
         ALLARGS = [v.concretetype for v in (greens_v + reds_v)]
         jd._green_args_spec = [v.concretetype for v in greens_v]
@@ -551,7 +560,7 @@
             assert jitdriver in sublists, \
                    "can_enter_jit with no matching jit_merge_point"
             jd, sublist = sublists[jitdriver]
-            origportalgraph = jd._jit_merge_point_pos[0]
+            origportalgraph = jd._jit_merge_point_in
             if graph is not origportalgraph:
                 sublist.append((graph, block, index))
                 jd.no_loop_header = False
@@ -581,7 +590,7 @@
             can_enter_jits = [(jd.portal_graph, jd.portal_graph.startblock, 0)]
 
         for graph, block, index in can_enter_jits:
-            if graph is jd._jit_merge_point_pos[0]:
+            if graph is jd._jit_merge_point_in:
                 continue
 
             op = block.operations[index]
@@ -639,7 +648,7 @@
         #           while 1:
         #               more stuff
         #
-        origportalgraph = jd._jit_merge_point_pos[0]
+        origportalgraph = jd._jit_merge_point_in
         portalgraph = jd.portal_graph
         PORTALFUNC = jd._PORTAL_FUNCTYPE
 
@@ -655,11 +664,13 @@
         portalfunc_ARGS = []
         nums = {}
         for i, ARG in enumerate(PORTALFUNC.ARGS):
+            kind = history.getkind(ARG)
+            assert kind != 'void'
             if i < len(jd.jitdriver.greens):
                 color = 'green'
             else:
                 color = 'red'
-            attrname = '%s_%s' % (color, history.getkind(ARG))
+            attrname = '%s_%s' % (color, kind)
             count = nums.get(attrname, 0)
             nums[attrname] = count + 1
             portalfunc_ARGS.append((ARG, attrname, count))
@@ -791,14 +802,7 @@
         # ____________________________________________________________
         # Now mutate origportalgraph to end with a call to portal_runner_ptr
         #
-        _, op = jd._jit_merge_point_pos
-        for origblock in origportalgraph.iterblocks():
-            if op in origblock.operations:
-                break
-        else:
-            assert False, "lost the operation %r in the graph %r" % (
-                op, origportalgraph)
-        origindex = origblock.operations.index(op)
+        origblock, origindex, op = locate_jit_merge_point(origportalgraph)
         assert op.opname == 'jit_marker'
         assert op.args[0].value == 'jit_merge_point'
         greens_v, reds_v = support.decode_hp_hint_args(op)
diff --git a/pypy/jit/metainterp/warmstate.py b/pypy/jit/metainterp/warmstate.py
--- a/pypy/jit/metainterp/warmstate.py
+++ b/pypy/jit/metainterp/warmstate.py
@@ -124,7 +124,7 @@
     # Hash of lltype or ootype object.
     # Only supports strings, unicodes and regular instances,
     # as well as primitives that can meaningfully be cast to Signed.
-    if isinstance(TYPE, lltype.Ptr):
+    if isinstance(TYPE, lltype.Ptr) and TYPE.TO._gckind == 'gc':
         if TYPE.TO is rstr.STR or TYPE.TO is rstr.UNICODE:
             return rstr.LLHelpers.ll_strhash(x)    # assumed not null
         else:
@@ -140,7 +140,7 @@
         else:
             return 0
     else:
-        return lltype.cast_primitive(lltype.Signed, x)
+        return rffi.cast(lltype.Signed, x)
 
 @specialize.ll_and_arg(3)
 def set_future_value(cpu, j, value, typecode):
diff --git a/pypy/module/__builtin__/functional.py b/pypy/module/__builtin__/functional.py
--- a/pypy/module/__builtin__/functional.py
+++ b/pypy/module/__builtin__/functional.py
@@ -292,7 +292,7 @@
                 raise
             break
         new_frame = space.createframe(code, w_func.w_func_globals,
-                                      w_func.closure)
+                                      w_func)
         new_frame.locals_stack_w[0] = w_item
         w_res = new_frame.run()
         result_w.append(w_res)
diff --git a/pypy/module/_codecs/interp_codecs.py b/pypy/module/_codecs/interp_codecs.py
--- a/pypy/module/_codecs/interp_codecs.py
+++ b/pypy/module/_codecs/interp_codecs.py
@@ -687,11 +687,15 @@
 # support for the "string escape" codec
 # This is a bytes-to bytes transformation
 
- at unwrap_spec(errors='str_or_None')
-def escape_encode(space, w_string, errors='strict'):
-    w_repr = space.repr(w_string)
-    w_result = space.getslice(w_repr, space.wrap(1), space.wrap(-1))
-    return space.newtuple([w_result, space.len(w_string)])
+ at unwrap_spec(data=str, errors='str_or_None')
+def escape_encode(space, data, errors='strict'):
+    from pypy.objspace.std.stringobject import string_escape_encode
+    result = string_escape_encode(data, quote="'")
+    start = 1
+    end = len(result) - 1
+    assert end >= 0
+    w_result = space.wrap(result[start:end])
+    return space.newtuple([w_result, space.wrap(len(data))])
 
 @unwrap_spec(data=str, errors='str_or_None')
 def escape_decode(space, data, errors='strict'):
diff --git a/pypy/module/_codecs/test/test_codecs.py b/pypy/module/_codecs/test/test_codecs.py
--- a/pypy/module/_codecs/test/test_codecs.py
+++ b/pypy/module/_codecs/test/test_codecs.py
@@ -102,7 +102,6 @@
     
     def test_indexerror(self):
         test =   "\\"     # trailing backslash
-             
         raises (ValueError, test.decode,'string-escape')
 
     def test_charmap_decode(self):
@@ -292,6 +291,10 @@
         assert '\\0f'.decode('string_escape') == chr(0) + 'f'
         assert '\\08'.decode('string_escape') == chr(0) + '8'
 
+    def test_escape_encode(self):
+        assert '"'.encode('string_escape') == '"'
+        assert "'".encode('string_escape') == "\\'"
+
     def test_decode_utf8_different_case(self):
         constant = u"a"
         assert constant.encode("utf-8") == constant.encode("UTF-8")
diff --git a/pypy/module/_continuation/interp_continuation.py b/pypy/module/_continuation/interp_continuation.py
--- a/pypy/module/_continuation/interp_continuation.py
+++ b/pypy/module/_continuation/interp_continuation.py
@@ -43,11 +43,11 @@
     def switch(self, w_to):
         to = self.space.interp_w(W_Continulet, w_to, can_be_None=True)
         if to is not None:
-            if self is to:    # double-switch to myself: no-op
-                return get_result()
             if to.sthread is None:
                 start_state.clear()
                 raise geterror(self.space, "continulet not initialized yet")
+            if self is to:    # double-switch to myself: no-op
+                return get_result()
         if self.sthread is None:
             start_state.clear()
             raise geterror(self.space, "continulet not initialized yet")
diff --git a/pypy/module/_rawffi/test/test__rawffi.py b/pypy/module/_rawffi/test/test__rawffi.py
--- a/pypy/module/_rawffi/test/test__rawffi.py
+++ b/pypy/module/_rawffi/test/test__rawffi.py
@@ -639,33 +639,6 @@
         a1.free()
         cb.free()
 
-    def test_another_callback_in_stackless(self):
-        try:
-            import _stackless
-        except ImportError:
-            skip("only valid in a stackless pypy-c")
-
-        import _rawffi
-        lib = _rawffi.CDLL(self.lib_name)
-        runcallback = lib.ptr('runcallback', ['P'], 'q')
-        def callback():
-            co = _stackless.coroutine()
-            def f():
-                pass
-            try:
-                co.bind(f)
-                co.switch()
-            except RuntimeError:
-                return 1<<42
-            return -5
-
-        cb = _rawffi.CallbackPtr(callback, [], 'q')
-        a1 = cb.byptr()
-        res = runcallback(a1)
-        assert res[0] == 1<<42
-        a1.free()
-        cb.free()
-
     def test_raising_callback(self):
         import _rawffi, sys
         import StringIO
diff --git a/pypy/module/_stackless/__init__.py b/pypy/module/_stackless/__init__.py
deleted file mode 100644
--- a/pypy/module/_stackless/__init__.py
+++ /dev/null
@@ -1,36 +0,0 @@
-# Package initialisation
-from pypy.interpreter.mixedmodule import MixedModule
-
-class Module(MixedModule):
-    """
-    This module implements Stackless for applications.
-    """
-
-    appleveldefs = {
-        'GreenletExit' : 'app_greenlet.GreenletExit',
-        'GreenletError' : 'app_greenlet.GreenletError',
-    }
-
-    interpleveldefs = {
-        'tasklet'    : 'interp_stackless.tasklet',
-        'coroutine'  : 'interp_coroutine.AppCoroutine',
-        'greenlet'   : 'interp_greenlet.AppGreenlet',
-        'usercostate': 'interp_composable_coroutine.W_UserCoState',
-        '_return_main' : 'interp_coroutine.return_main',
-        'get_stack_depth_limit': 'interp_coroutine.get_stack_depth_limit',
-        'set_stack_depth_limit': 'interp_coroutine.set_stack_depth_limit',
-    }
-
-    def setup_after_space_initialization(self):
-        # post-installing classmethods/staticmethods which
-        # are not yet directly supported
-        from pypy.module._stackless.interp_coroutine import post_install as post_install_coro
-        post_install_coro(self)
-        from pypy.module._stackless.interp_greenlet import post_install as post_install_greenlet
-        post_install_greenlet(self)
-
-        if self.space.config.translation.gc == 'marksweep':
-            from pypy.module._stackless.interp_clonable import post_install as post_install_clonable
-            self.extra_interpdef('clonable', 'interp_clonable.AppClonableCoroutine')
-            self.extra_interpdef('fork',     'interp_clonable.fork')
-            post_install_clonable(self)
diff --git a/pypy/module/_stackless/app_greenlet.py b/pypy/module/_stackless/app_greenlet.py
deleted file mode 100644
--- a/pypy/module/_stackless/app_greenlet.py
+++ /dev/null
@@ -1,5 +0,0 @@
-class GreenletExit(Exception):
-    pass
-
-class GreenletError(Exception):
-    pass
diff --git a/pypy/module/_stackless/interp_clonable.py b/pypy/module/_stackless/interp_clonable.py
deleted file mode 100644
--- a/pypy/module/_stackless/interp_clonable.py
+++ /dev/null
@@ -1,106 +0,0 @@
-from pypy.interpreter.error import OperationError
-from pypy.interpreter.typedef import TypeDef
-from pypy.interpreter.gateway import interp2app
-from pypy.module._stackless.interp_coroutine import AppCoroutine, AppCoState
-from pypy.module._stackless.interp_coroutine import makeStaticMethod
-from pypy.module._stackless.rcoroutine import AbstractThunk
-from pypy.module._stackless.rclonable import InterpClonableMixin
-
-
-class AppClonableCoroutine(AppCoroutine, InterpClonableMixin):
-
-    def newsubctx(self):
-        self.hello_local_pool()
-        AppCoroutine.newsubctx(self)
-        self.goodbye_local_pool()
-
-    def hello(self):
-        self.hello_local_pool()
-        AppCoroutine.hello(self)
-
-    def goodbye(self):
-        AppCoroutine.goodbye(self)
-        self.goodbye_local_pool()
-
-    def descr_method__new__(space, w_subtype):
-        co = space.allocate_instance(AppClonableCoroutine, w_subtype)
-        costate = AppClonableCoroutine._get_state(space)
-        AppClonableCoroutine.__init__(co, space, state=costate)
-        return space.wrap(co)
-
-    def _get_state(space):
-        return space.fromcache(AppClonableCoState)
-    _get_state = staticmethod(_get_state)
-
-    def w_getcurrent(space):
-        return space.wrap(AppClonableCoroutine._get_state(space).current)
-    w_getcurrent = staticmethod(w_getcurrent)
-
-    def w_clone(self):
-        space = self.space
-        costate = self.costate
-        if costate.current is self:
-            raise OperationError(space.w_RuntimeError,
-                                 space.wrap("clone() cannot clone the "
-                                            "current coroutine"
-                                            "; use fork() instead"))
-        copy = AppClonableCoroutine(space, state=costate)
-        copy.subctx = self.clone_into(copy, self.subctx)
-        return space.wrap(copy)
-
-    def descr__reduce__(self, space):
-        raise OperationError(space.w_TypeError,
-                             space.wrap("_stackless.clonable instances are "
-                                        "not picklable"))
-
-
-AppClonableCoroutine.typedef = TypeDef("clonable", AppCoroutine.typedef,
-    __new__    = interp2app(AppClonableCoroutine.descr_method__new__.im_func),
-    getcurrent = interp2app(AppClonableCoroutine.w_getcurrent),
-    clone      = interp2app(AppClonableCoroutine.w_clone),
-    __reduce__ = interp2app(AppClonableCoroutine.descr__reduce__),
-)
-
-class AppClonableCoState(AppCoState):
-    def post_install(self):
-        self.current = self.main = AppClonableCoroutine(self.space, state=self)
-        self.main.subctx.clear_framestack()      # wack
-
-def post_install(module):
-    makeStaticMethod(module, 'clonable', 'getcurrent')
-    space = module.space
-    AppClonableCoroutine._get_state(space).post_install()
-
-# ____________________________________________________________
-
-class ForkThunk(AbstractThunk):
-    def __init__(self, coroutine):
-        self.coroutine = coroutine
-        self.newcoroutine = None
-    def call(self):
-        oldcoro = self.coroutine
-        self.coroutine = None
-        newcoro = AppClonableCoroutine(oldcoro.space, state=oldcoro.costate)
-        newcoro.subctx = oldcoro.clone_into(newcoro, oldcoro.subctx)
-        newcoro.parent = oldcoro
-        self.newcoroutine = newcoro
-
-def fork(space):
-    """Fork, as in the Unix fork(): the call returns twice, and the return
-    value of the call is either the new 'child' coroutine object (if returning
-    into the parent), or None (if returning into the child).  This returns
-    into the parent first, which can switch to the child later.
-    """
-    costate = AppClonableCoroutine._get_state(space)
-    current = costate.current
-    if current is costate.main:
-        raise OperationError(space.w_RuntimeError,
-                             space.wrap("cannot fork() in the main "
-                                        "clonable coroutine"))
-    thunk = ForkThunk(current)
-    coro_fork = AppClonableCoroutine(space, state=costate)
-    coro_fork.bind(thunk)
-    coro_fork.switch()
-    # we resume here twice.  The following would need explanations about
-    # why it returns the correct thing in both the parent and the child...
-    return space.wrap(thunk.newcoroutine)
diff --git a/pypy/module/_stackless/interp_composable_coroutine b/pypy/module/_stackless/interp_composable_coroutine
deleted file mode 100644
--- a/pypy/module/_stackless/interp_composable_coroutine
+++ /dev/null
@@ -1,33 +0,0 @@
-from pypy.interpreter.baseobjspace import Wrappable
-from pypy.interpreter.typedef import TypeDef, interp2app
-from pypy.module._stackless.coroutine import AppCoState, AppCoroutine
-
-
-class W_UserCoState(Wrappable):
-    def __init__(self, space):
-        self.costate = AppCoState(space)
-        self.costate.post_install()
-
-    def descr_method__new__(space, w_subtype):
-        costate = space.allocate_instance(W_UserCoState, w_subtype)
-        W_UserCoState.__init__(costate, space)
-        return space.wrap(costate)
-
-    def w_getcurrent(self):
-        space = self.costate.space
-        return space.wrap(self.costate.current)
-
-    def w_spawn(self, w_subtype=None):
-        space = self.costate.space
-        if space.is_w(w_subtype, space.w_None):
-            w_subtype = space.gettypeobject(AppCoroutine.typedef)
-        co = space.allocate_instance(AppCoroutine, w_subtype)
-        AppCoroutine.__init__(co, space, state=self.costate)
-        return space.wrap(co)
-
-W_UserCoState.typedef = TypeDef("usercostate",
-    __new__ = interp2app(W_UserCoState.descr_method__new__.im_func),
-    __module__ = '_stackless',
-    getcurrent = interp2app(W_UserCoState.w_getcurrent),
-    spawn      = interp2app(W_UserCoState.w_spawn),
-)
diff --git a/pypy/module/_stackless/interp_composable_coroutine.py b/pypy/module/_stackless/interp_composable_coroutine.py
deleted file mode 100644
--- a/pypy/module/_stackless/interp_composable_coroutine.py
+++ /dev/null
@@ -1,34 +0,0 @@
-from pypy.interpreter.baseobjspace import Wrappable
-from pypy.interpreter.typedef import TypeDef, interp2app
-from pypy.module._stackless.interp_coroutine import AppCoState, AppCoroutine
-
-
-class W_UserCoState(Wrappable):
-    def __init__(self, space):
-        self.costate = AppCoState(space)
-        self.costate.post_install()
-
-    def descr_method__new__(space, w_subtype):
-        costate = space.allocate_instance(W_UserCoState, w_subtype)
-        W_UserCoState.__init__(costate, space)
-        return space.wrap(costate)
-
-    def w_getcurrent(self):
-        space = self.costate.space
-        return space.wrap(self.costate.current)
-
-    def w_spawn(self, w_subtype=None):
-        space = self.costate.space
-        if space.is_w(w_subtype, space.w_None):
-            w_subtype = space.gettypeobject(AppCoroutine.typedef)
-        co = space.allocate_instance(AppCoroutine, w_subtype)
-        AppCoroutine.__init__(co, space, state=self.costate)
-        return space.wrap(co)
-
-W_UserCoState.typedef = TypeDef("usercostate",
-    __new__ = interp2app(W_UserCoState.descr_method__new__.im_func),
-    __module__ = '_stackless',
-    getcurrent = interp2app(W_UserCoState.w_getcurrent),
-    spawn      = interp2app(W_UserCoState.w_spawn),
-)
-W_UserCoState.acceptable_as_base_class = False
diff --git a/pypy/module/_stackless/interp_coroutine.py b/pypy/module/_stackless/interp_coroutine.py
deleted file mode 100644
--- a/pypy/module/_stackless/interp_coroutine.py
+++ /dev/null
@@ -1,403 +0,0 @@
-"""
-Coroutine implementation for application level on top
-of the internal coroutines.
-This is an extensible concept. Multiple implementations
-of concurrency can exist together, if they follow the
-basic concept of maintaining their own costate.
-
-There is also some diversification possible by using
-multiple costates for the same type. This leads to
-disjoint switchable sets within the same type.
-
-I'm not so sure to what extent the opposite is possible, too.
-I.e., merging the costate of tasklets and greenlets would
-allow them to be parents of each other. Needs a bit more
-experience to decide where to set the limits.
-"""
-
-from pypy.interpreter.argument import Arguments
-from pypy.interpreter.typedef import GetSetProperty, TypeDef
-from pypy.interpreter.gateway import interp2app, unwrap_spec
-from pypy.interpreter.error import OperationError, operationerrfmt
-
-from pypy.module._stackless.stackless_flags import StacklessFlags
-from pypy.module._stackless.rcoroutine import Coroutine, BaseCoState, AbstractThunk, CoroutineExit
-
-from pypy.module.exceptions.interp_exceptions import W_SystemExit, _new_exception
-
-from pypy.rlib import rstack, jit # for resume points
-from pypy.tool import stdlib_opcode as pythonopcode
-
-class _AppThunk(AbstractThunk):
-
-    def __init__(self, space, costate, w_obj, args):
-        self.space = space
-        self.costate = costate
-        if not space.is_true(space.callable(w_obj)):
-            raise operationerrfmt(
-                space.w_TypeError,
-                "'%s' object is not callable",
-                space.type(w_obj).getname(space))
-        self.w_func = w_obj
-        self.args = args
-
-    def call(self):
-        costate = self.costate
-        w_result = self.space.call_args(self.w_func, self.args)
-        costate.w_tempval = w_result
-
-class _ResumeThunk(AbstractThunk):
-    def __init__(self, space, costate, w_frame):
-        self.space = space
-        self.costate = costate
-        self.w_frame = w_frame
-
-    def call(self):
-        w_result = resume_frame(self.space, self.w_frame)
-        # costate.w_tempval = w_result #XXX?
-
-
-W_CoroutineExit = _new_exception('CoroutineExit', W_SystemExit,
-                        """Coroutine killed manually.""")
-
-# Should be moved to interp_stackless.py if it's ever implemented... Currently
-# used by pypy/lib/stackless.py.
-W_TaskletExit = _new_exception('TaskletExit', W_SystemExit,
-            """Tasklet killed manually.""")
-
-class AppCoroutine(Coroutine): # XXX, StacklessFlags):
-
-    def __init__(self, space, state=None):
-        self.space = space
-        if state is None:
-            state = AppCoroutine._get_state(space)
-        Coroutine.__init__(self, state)
-        self.flags = 0
-        self.newsubctx()
-
-    def newsubctx(self):
-        ec = self.space.getexecutioncontext()
-        self.subctx = ec.Subcontext()
-
-    def descr_method__new__(space, w_subtype):
-        co = space.allocate_instance(AppCoroutine, w_subtype)
-        AppCoroutine.__init__(co, space)
-        return space.wrap(co)
-
-    def _get_state(space):
-        return space.fromcache(AppCoState)
-    _get_state = staticmethod(_get_state)
-
-    def w_bind(self, w_func, __args__):
-        space = self.space
-        if self.frame is not None:
-            raise OperationError(space.w_ValueError, space.wrap(
-                "cannot bind a bound Coroutine"))
-        state = self.costate
-        thunk = _AppThunk(space, state, w_func, __args__)
-        self.bind(thunk)
-
-    def w_switch(self):
-        space = self.space
-        if self.frame is None:
-            raise OperationError(space.w_ValueError, space.wrap(
-                "cannot switch to an unbound Coroutine"))
-        state = self.costate
-        self.switch()
-        w_ret, state.w_tempval = state.w_tempval, space.w_None
-        return w_ret
-
-    def switch(self):
-        space = self.space
-        try:
-            Coroutine.switch(self)
-        except CoroutineExit:
-            raise OperationError(self.costate.w_CoroutineExit, space.w_None)
-
-    def w_finished(self, w_excinfo):
-        pass
-
-    def finish(self, operror=None):
-        space = self.space
-        if isinstance(operror, OperationError):
-            w_exctype = operror.w_type
-            w_excvalue = operror.get_w_value(space)
-            w_exctraceback = operror.get_traceback()
-            w_excinfo = space.newtuple([w_exctype, w_excvalue, w_exctraceback])
-
-            if w_exctype is self.costate.w_CoroutineExit:
-                self.coroutine_exit = True
-        else:
-            w_N = space.w_None
-            w_excinfo = space.newtuple([w_N, w_N, w_N])
-
-        return space.call_method(space.wrap(self),'finished', w_excinfo)
-
-    def hello(self):
-        ec = self.space.getexecutioncontext()
-        self.subctx.enter(ec)
-
-    def goodbye(self):
-        ec = self.space.getexecutioncontext()
-        self.subctx.leave(ec)
-
-    def w_kill(self):
-        self.kill()
-
-    def w_throw(self, w_type, w_value=None, w_traceback=None):
-        space = self.space
-
-        operror = OperationError(w_type, w_value)
-        operror.normalize_exception(space)
-
-        if not space.is_w(w_traceback, space.w_None):
-            from pypy.interpreter import pytraceback
-            tb = space.interpclass_w(w_traceback)
-            if tb is None or not space.is_true(space.isinstance(tb,
-                space.gettypeobject(pytraceback.PyTraceback.typedef))):
-                raise OperationError(space.w_TypeError,
-                      space.wrap("throw: arg 3 must be a traceback or None"))
-            operror.set_traceback(tb)
-
-        self._kill(operror)
-
-    def _userdel(self):
-        if self.get_is_zombie():
-            return
-        self.set_is_zombie(True)
-        self.space.userdel(self.space.wrap(self))
-
-    def w_getcurrent(space):
-        return space.wrap(AppCoroutine._get_state(space).current)
-    w_getcurrent = staticmethod(w_getcurrent)
-
-    def w_getmain(space):
-        return space.wrap(AppCoroutine._get_state(space).main)
-    w_getmain = staticmethod(w_getmain)
-
-    # pickling interface
-    def descr__reduce__(self, space):
-        # this is trying to be simplistic at the moment.
-        # we neither allow to pickle main (which can become a mess
-        # since it has some deep anchestor frames)
-        # nor we allow to pickle the current coroutine.
-        # rule: switch before pickling.
-        # you cannot construct the tree that you are climbing.
-        from pypy.interpreter.mixedmodule import MixedModule
-        w_mod    = space.getbuiltinmodule('_stackless')
-        mod      = space.interp_w(MixedModule, w_mod)
-        w_mod2    = space.getbuiltinmodule('_pickle_support')
-        mod2      = space.interp_w(MixedModule, w_mod2)
-        w_new_inst = mod.get('coroutine')
-        w        = space.wrap
-        nt = space.newtuple
-        ec = self.space.getexecutioncontext()
-
-        if self is self.costate.main:
-            return nt([mod.get('_return_main'), nt([])])
-
-        thunk = self.thunk
-        if isinstance(thunk, _AppThunk):
-            w_args, w_kwds = thunk.args.topacked()
-            w_thunk = nt([thunk.w_func, w_args, w_kwds])
-        else:
-            w_thunk = space.w_None
-
-        tup_base = [
-            ]
-        tup_state = [
-            w(self.flags),
-            self.subctx.getstate(space),
-            w_thunk,
-            w(self.parent),
-            ]
-
-        return nt([w_new_inst, nt(tup_base), nt(tup_state)])
-
-    def descr__setstate__(self, space, w_args):
-        w_flags, w_state, w_thunk, w_parent = space.unpackiterable(w_args,
-                                                        expected_length=4)
-        self.flags = space.int_w(w_flags)
-        if space.is_w(w_parent, space.w_None):
-            w_parent = self.w_getmain(space)
-        self.parent = space.interp_w(AppCoroutine, w_parent)
-        ec = self.space.getexecutioncontext()
-        self.subctx.setstate(space, w_state)
-        if space.is_w(w_thunk, space.w_None):
-            if space.is_w(w_state, space.w_None):
-                self.thunk = None
-            else:
-                self.bind(_ResumeThunk(space, self.costate, self.subctx.topframe))
-        else:
-            w_func, w_args, w_kwds = space.unpackiterable(w_thunk,
-                                                          expected_length=3)
-            args = Arguments.frompacked(space, w_args, w_kwds)
-            self.bind(_AppThunk(space, self.costate, w_func, args))
-
-
-# _mixin_ did not work
-for methname in StacklessFlags.__dict__:
-    meth = getattr(StacklessFlags, methname)
-    if hasattr(meth, 'im_func'):
-        setattr(AppCoroutine, meth.__name__, meth.im_func)
-del meth, methname
-
-def w_get_is_zombie(self, space):
-    return space.wrap(self.get_is_zombie())
-AppCoroutine.w_get_is_zombie = w_get_is_zombie
-
-def w_get_is_alive(self, space):
-    return space.wrap(self.is_alive())
-AppCoroutine.w_get_is_alive = w_get_is_alive
-
-def w_descr__framestack(self, space):
-    assert isinstance(self, AppCoroutine)
-    counter = 0
-    f = self.subctx.topframe
-    while f is not None:
-        counter += 1
-        f = f.f_backref()
-    items = [None] * counter
-    f = self.subctx.topframe
-    while f is not None:
-        counter -= 1
-        assert counter >= 0
-        items[counter] = space.wrap(f)
-        f = f.f_backref()
-    assert counter == 0
-    return space.newtuple(items)
-
-def makeStaticMethod(module, classname, funcname):
-    "NOT_RPYTHON"
-    space = module.space
-    w_klass = space.getattr(space.wrap(module), space.wrap(classname))
-    # HACK HACK HACK
-    # make the typeobject mutable for a while
-    from pypy.objspace.std.typeobject import W_TypeObject
-    assert isinstance(w_klass, W_TypeObject)
-    old_flag = w_klass.flag_heaptype
-    w_klass.flag_heaptype = True
-
-    space.appexec([w_klass, space.wrap(funcname)], """
-        (klass, funcname):
-            func = getattr(klass, funcname)
-            setattr(klass, funcname, staticmethod(func.im_func))
-    """)
-    w_klass.flag_heaptype = old_flag
-
-def post_install(module):
-    makeStaticMethod(module, 'coroutine', 'getcurrent')
-    makeStaticMethod(module, 'coroutine', 'getmain')
-    space = module.space
-    AppCoroutine._get_state(space).post_install()
-
-# space.appexec("""() :
-
-# maybe use __spacebind__ for postprocessing
-
-AppCoroutine.typedef = TypeDef("coroutine",
-    __new__ = interp2app(AppCoroutine.descr_method__new__.im_func),
-    bind = interp2app(AppCoroutine.w_bind),
-    switch = interp2app(AppCoroutine.w_switch),
-    kill = interp2app(AppCoroutine.w_kill),
-    throw = interp2app(AppCoroutine.w_throw),
-    finished = interp2app(AppCoroutine.w_finished),
-    is_alive = GetSetProperty(AppCoroutine.w_get_is_alive),
-    is_zombie = GetSetProperty(AppCoroutine.w_get_is_zombie,
-      doc=AppCoroutine.get_is_zombie.__doc__), #--- this flag is a bit obscure
-      # and not useful (it's totally different from Coroutine.is_zombie(), too)
-      # but lib/stackless.py uses it
-    _framestack = GetSetProperty(w_descr__framestack),
-    getcurrent = interp2app(AppCoroutine.w_getcurrent),
-    getmain = interp2app(AppCoroutine.w_getmain),
-    __reduce__   = interp2app(AppCoroutine.descr__reduce__),
-    __setstate__ = interp2app(AppCoroutine.descr__setstate__),
-    __module__ = '_stackless',
-)
-
-class AppCoState(BaseCoState):
-    def __init__(self, space):
-        BaseCoState.__init__(self)
-        self.w_tempval = space.w_None
-        self.space = space
-
-        # XXX Workaround: for now we need to instantiate these classes
-        # explicitly for translation to work
-        W_CoroutineExit(space)
-        W_TaskletExit(space)
-
-        # Exporting new exception to space
-        self.w_CoroutineExit = space.gettypefor(W_CoroutineExit)
-        space.setitem(
-                      space.exceptions_module.w_dict,
-                      space.new_interned_str('CoroutineExit'),
-                      self.w_CoroutineExit)
-        space.setitem(space.builtin.w_dict,
-                      space.new_interned_str('CoroutineExit'),
-                      self.w_CoroutineExit)
-
-        # Should be moved to interp_stackless.py if it's ever implemented...
-        self.w_TaskletExit = space.gettypefor(W_TaskletExit)
-        space.setitem(
-                      space.exceptions_module.w_dict,
-                      space.new_interned_str('TaskletExit'),
-                      self.w_TaskletExit)
-        space.setitem(space.builtin.w_dict,
-                      space.new_interned_str('TaskletExit'),
-                      self.w_TaskletExit)
-
-    def post_install(self):
-        self.current = self.main = AppCoroutine(self.space, state=self)
-        self.main.subctx.clear_framestack()      # wack
-
-def return_main(space):
-    return AppCoroutine._get_state(space).main
-
-def get_stack_depth_limit(space):
-    return space.wrap(rstack.get_stack_depth_limit())
-
- at unwrap_spec(limit=int)
-def set_stack_depth_limit(space, limit):
-    rstack.set_stack_depth_limit(limit)
-
-
-# ___________________________________________________________________
-# unpickling trampoline
-
-def resume_frame(space, w_frame):
-    from pypy.interpreter.pyframe import PyFrame
-    frame = space.interp_w(PyFrame, w_frame, can_be_None=True)
-    w_result = space.w_None
-    operr = None
-    executioncontext = frame.space.getexecutioncontext()
-    while frame is not None:
-        code = frame.pycode.co_code
-        instr = frame.last_instr
-        opcode = ord(code[instr])
-        map = pythonopcode.opmap
-        call_ops = [map['CALL_FUNCTION'], map['CALL_FUNCTION_KW'], map['CALL_FUNCTION_VAR'],
-                    map['CALL_FUNCTION_VAR_KW'], map['CALL_METHOD']]
-        assert opcode in call_ops
-        instr += 1
-        oparg = ord(code[instr]) | ord(code[instr + 1]) << 8
-        nargs = oparg & 0xff
-        nkwds = (oparg >> 8) & 0xff
-        if nkwds == 0:     # only positional arguments
-            # fast paths leaves things on the stack, pop them
-            if space.config.objspace.opcodes.CALL_METHOD and opcode == map['CALL_METHOD']:
-                frame.dropvalues(nargs + 2)
-            elif opcode == map['CALL_FUNCTION']:
-                frame.dropvalues(nargs + 1)
-
-        # small hack: unlink frame out of the execution context, because
-        # execute_frame will add it there again
-        executioncontext.topframeref = jit.non_virtual_ref(frame.f_backref())
-        frame.last_instr = instr + 1 # continue after the call
-        try:
-            w_result = frame.execute_frame(w_result, operr)
-        except OperationError, operr:
-            pass
-        frame = frame.f_backref()
-    if operr:
-        raise operr
-    return w_result
diff --git a/pypy/module/_stackless/interp_greenlet.py b/pypy/module/_stackless/interp_greenlet.py
deleted file mode 100644
--- a/pypy/module/_stackless/interp_greenlet.py
+++ /dev/null
@@ -1,238 +0,0 @@
-from pypy.interpreter.argument import Arguments
-from pypy.interpreter.typedef import GetSetProperty, TypeDef
-from pypy.interpreter.gateway import interp2app
-from pypy.interpreter.gateway import NoneNotWrapped
-from pypy.interpreter.error import OperationError
-
-from pypy.module._stackless.rcoroutine import Coroutine, BaseCoState
-from pypy.module._stackless.rcoroutine import AbstractThunk, syncstate
-from pypy.module._stackless.interp_coroutine import makeStaticMethod
-
-
-class GreenletThunk(AbstractThunk):
-
-    def __init__(self, greenlet):
-        self.greenlet = greenlet
-
-    def call(self):
-        greenlet = self.greenlet
-        greenlet.active = True
-        try:
-            space = greenlet.space
-            args_w = greenlet.costate.args_w
-            __args__ = Arguments(space, args_w)
-            try:
-                w_run = space.getattr(space.wrap(greenlet), space.wrap('run'))
-                greenlet.w_callable = None
-                w_result = space.call_args(w_run, __args__)
-            except OperationError, operror:
-                if not operror.match(space, greenlet.costate.w_GreenletExit):
-                    raise
-                w_result = operror.get_w_value(space)
-        finally:
-            greenlet.active = False
-        greenlet.costate.args_w = [w_result]
-
-class AppGreenletCoState(BaseCoState):
-    def __init__(self, space):
-        BaseCoState.__init__(self)
-        self.args_w = None
-        self.space = space
-        self.w_GreenletExit  = get(space, "GreenletExit")
-        self.w_GreenletError = get(space, "GreenletError")
-
-    def post_install(self):
-        self.current = self.main = AppGreenlet(self.space, is_main=True)
-
-class AppGreenlet(Coroutine):
-    def __init__(self, space, w_callable=None, is_main=False):
-        Coroutine.__init__(self, self._get_state(space))
-        self.space = space
-        self.w_callable = w_callable
-        self.active = is_main
-        self.subctx = space.getexecutioncontext().Subcontext()
-        if is_main:
-            self.subctx.clear_framestack()      # wack
-        else:
-            self.bind(GreenletThunk(self))
-
-    def descr_method__new__(space, w_subtype, __args__):
-        co = space.allocate_instance(AppGreenlet, w_subtype)
-        AppGreenlet.__init__(co, space)
-        return space.wrap(co)
-
-    def descr_method__init__(self, w_run=NoneNotWrapped,
-                                   w_parent=NoneNotWrapped):
-        if w_run is not None:
-            self.set_run(w_run)
-        if w_parent is not None:
-            self.set_parent(w_parent)
-
-    def _get_state(space):
-        return space.fromcache(AppGreenletCoState)
-    _get_state = staticmethod(_get_state)
-
-    def hello(self):
-        ec = self.space.getexecutioncontext()
-        self.subctx.enter(ec)
-
-    def goodbye(self):
-        ec = self.space.getexecutioncontext()
-        self.subctx.leave(ec)
-
-    def w_getcurrent(space):
-        return space.wrap(AppGreenlet._get_state(space).current)
-    w_getcurrent = staticmethod(w_getcurrent)
-
-    def w_switch(self, args_w):
-        # Find the switch target - it might be a parent greenlet
-        space = self.space
-        costate = self.costate
-        target = self
-        while target.isdead():
-            target = target.parent
-            assert isinstance(target, AppGreenlet)
-        # Switch to it
-        costate.args_w = args_w
-        if target is not costate.current:
-            target.switch()
-        else:
-            # case not handled in Coroutine.switch()
-            syncstate._do_things_to_do()
-        result_w = costate.args_w
-        costate.args_w = None
-        # costate.args_w can be set to None above for throw(), but then
-        # switch() should have raised.  At this point cosstate.args_w != None.
-        assert result_w is not None
-        # Return the result of a switch, packaging it in a tuple if
-        # there is more than one value.
-        if len(result_w) == 1:
-            return result_w[0]
-        return space.newtuple(result_w)
-
-    def w_throw(self, w_type=None, w_value=None, w_traceback=None):
-        space = self.space
-        if space.is_w(w_type, space.w_None):
-            w_type = self.costate.w_GreenletExit
-        # Code copied from RAISE_VARARGS but slightly modified.  Not too nice.
-        operror = OperationError(w_type, w_value)
-        operror.normalize_exception(space)
-        if not space.is_w(w_traceback, space.w_None):
-            from pypy.interpreter import pytraceback
-            tb = space.interpclass_w(w_traceback)
-            if tb is None or not space.is_true(space.isinstance(tb, 
-                space.gettypeobject(pytraceback.PyTraceback.typedef))):
-                raise OperationError(space.w_TypeError,
-                      space.wrap("throw: arg 3 must be a traceback or None"))
-            operror.set_traceback(tb)
-        # Dead greenlet: turn GreenletExit into a regular return
-        if self.isdead() and operror.match(space, self.costate.w_GreenletExit):
-            args_w = [operror.get_w_value(space)]
-        else:
-            syncstate.push_exception(operror)
-            args_w = None
-        return self.w_switch(args_w)
-
-    def _userdel(self):
-        self.space.userdel(self.space.wrap(self))
-
-    def isdead(self):
-        return self.thunk is None and not self.active
-
-    def w_get_is_dead(self, space):
-        return space.newbool(self.isdead())
-
-    def descr__nonzero__(self):
-        return self.space.newbool(self.active)
-
-    def w_get_run(self, space):
-        w_run = self.w_callable
-        if w_run is None:
-            raise OperationError(space.w_AttributeError, space.wrap("run"))
-        return w_run
-
-    def set_run(self, w_run):
-        space = self.space
-        if self.thunk is None:
-            raise OperationError(space.w_AttributeError,
-                                 space.wrap("run cannot be set "
-                                            "after the start of the greenlet"))
-        self.w_callable = w_run
-
-    def w_set_run(self, space, w_run):
-        self.set_run(w_run)
-
-    def w_del_run(self, space):
-        if self.w_callable is None:
-            raise OperationError(space.w_AttributeError, space.wrap("run"))
-        self.w_callable = None
-
-    def w_get_parent(self, space):
-        return space.wrap(self.parent)
-
-    def set_parent(self, w_parent):
-        space = self.space
-        newparent = space.interp_w(AppGreenlet, w_parent)
-        if newparent.costate is not self.costate:
-            raise OperationError(self.costate.w_GreenletError,
-                                 space.wrap("invalid foreign parent"))
-        curr = newparent
-        while curr:
-            if curr is self:
-                raise OperationError(space.w_ValueError,
-                                     space.wrap("cyclic parent chain"))
-            curr = curr.parent
-        self.parent = newparent
-
-    def w_set_parent(self, space, w_parent):
-        self.set_parent(w_parent)
-
-    def w_get_frame(self, space):
-        if not self.active or self.costate.current is self:
-            f = None
-        else:
-            f = self.subctx.topframe
-        return space.wrap(f)
-
-def get(space, name):
-    w_module = space.getbuiltinmodule('_stackless')
-    return space.getattr(w_module, space.wrap(name))
-
-def post_install(module):
-    "NOT_RPYTHON"
-    makeStaticMethod(module, 'greenlet', 'getcurrent')
-    space = module.space
-    state = AppGreenlet._get_state(space)
-    state.post_install()
-    w_greenlet = get(space, 'greenlet')
-    # HACK HACK HACK
-    # make the typeobject mutable for a while
-    from pypy.objspace.std.typeobject import W_TypeObject
-    assert isinstance(w_greenlet, W_TypeObject)
-    old_flag = w_greenlet.flag_heaptype
-    w_greenlet.flag_heaptype = True
-    space.appexec([w_greenlet,
-                   state.w_GreenletExit,
-                   state.w_GreenletError], """
-    (greenlet, exit, error):
-        greenlet.GreenletExit = exit
-        greenlet.error = error
-    """)
-    w_greenlet.flag_heaptype = old_flag
-
-AppGreenlet.typedef = TypeDef("greenlet",
-    __new__ = interp2app(AppGreenlet.descr_method__new__.im_func),
-    __init__ = interp2app(AppGreenlet.descr_method__init__),
-    switch = interp2app(AppGreenlet.w_switch),
-    dead = GetSetProperty(AppGreenlet.w_get_is_dead),
-    run = GetSetProperty(AppGreenlet.w_get_run,
-                         AppGreenlet.w_set_run,
-                         AppGreenlet.w_del_run),
-    parent = GetSetProperty(AppGreenlet.w_get_parent,
-                            AppGreenlet.w_set_parent),
-    getcurrent = interp2app(AppGreenlet.w_getcurrent),
-    throw = interp2app(AppGreenlet.w_throw),
-    gr_frame = GetSetProperty(AppGreenlet.w_get_frame),
-    __nonzero__ = interp2app(AppGreenlet.descr__nonzero__),
-    __module__ = '_stackless',
-)
diff --git a/pypy/module/_stackless/interp_stackless.py b/pypy/module/_stackless/interp_stackless.py
deleted file mode 100644
--- a/pypy/module/_stackless/interp_stackless.py
+++ /dev/null
@@ -1,28 +0,0 @@
-from pypy.interpreter.baseobjspace import Wrappable
-from pypy.interpreter.typedef import TypeDef
-from pypy.interpreter.gateway import interp2app
-import os
-
-
-class tasklet(Wrappable):
-
-    def __init__(self, space):
-        self.space = space
-        self.flags = 0
-        self.state = None
-
-    def descr_method__new__(space, w_subtype):
-        t = space.allocate_instance(tasklet, w_subtype)
-        tasklet.__init__(t, space)
-        return space.wrap(t)
-
-    def w_demo(self):
-        output("42")
-
-tasklet.typedef = TypeDef("tasklet",
-    __new__ = interp2app(tasklet.descr_method__new__.im_func),
-    demo = interp2app(tasklet.w_demo),
-)
-
-def output(stuff):
-    os.write(2, stuff + '\n')
diff --git a/pypy/module/_stackless/rclonable.py b/pypy/module/_stackless/rclonable.py
deleted file mode 100644
--- a/pypy/module/_stackless/rclonable.py
+++ /dev/null
@@ -1,87 +0,0 @@
-from pypy.module._stackless.interp_coroutine import AbstractThunk, Coroutine
-from pypy.rlib.rgc import gc_swap_pool, gc_clone
-from pypy.rlib.objectmodel import we_are_translated
-
-
-class InterpClonableMixin:
-    local_pool = None
-    _mixin_ = True
-
-    def hello_local_pool(self):
-        if we_are_translated():
-            self.saved_pool = gc_swap_pool(self.local_pool)
-
-    def goodbye_local_pool(self):
-        if we_are_translated():
-            self.local_pool = gc_swap_pool(self.saved_pool)
-            self.saved_pool = None
-
-    def clone_into(self, copy, extradata=None):
-        if not we_are_translated():
-            raise NotImplementedError
-        # cannot gc_clone() directly self, because it is not in its own
-        # local_pool.  Moreover, it has a __del__, which cloning doesn't
-        # support properly at the moment.
-        copy.parent = self.parent
-        # the hello/goodbye pair has two purposes: it forces
-        # self.local_pool to be computed even if it was None up to now,
-        # and it puts the 'data' tuple in the correct pool to be cloned.
-        self.hello_local_pool()
-        data = (self.frame, extradata)
-        self.goodbye_local_pool()
-        # clone!
-        data, copy.local_pool = gc_clone(data, self.local_pool)
-        copy.frame, extradata = data
-        copy.thunk = self.thunk # in case we haven't switched to self yet
-        return extradata
-
-
-class InterpClonableCoroutine(Coroutine, InterpClonableMixin):
-
-    def hello(self):
-        self.hello_local_pool()
-
-    def goodbye(self):
-        self.goodbye_local_pool()
-
-    def clone(self):
-        # hack, this is overridden in AppClonableCoroutine
-        if self.getcurrent() is self:
-            raise RuntimeError("clone() cannot clone the current coroutine; "
-                               "use fork() instead")
-        copy = InterpClonableCoroutine(self.costate)
-        self.clone_into(copy)
-        return copy
-
-
-class ForkThunk(AbstractThunk):
-    def __init__(self, coroutine):
-        self.coroutine = coroutine
-        self.newcoroutine = None
-    def call(self):
-        oldcoro = self.coroutine
-        self.coroutine = None
-        newcoro = oldcoro.clone()
-        newcoro.parent = oldcoro
-        self.newcoroutine = newcoro
-
-def fork():
-    """Fork, as in the Unix fork(): the call returns twice, and the return
-    value of the call is either the new 'child' coroutine object (if returning
-    into the parent), or None (if returning into the child).  This returns
-    into the parent first, which can switch to the child later.
-    """
-    current = InterpClonableCoroutine.getcurrent()
-    if not isinstance(current, InterpClonableCoroutine):
-        raise RuntimeError("fork() in a non-clonable coroutine")
-    thunk = ForkThunk(current)
-    coro_fork = InterpClonableCoroutine()
-    coro_fork.bind(thunk)
-    coro_fork.switch()
-    # we resume here twice.  The following would need explanations about
-    # why it returns the correct thing in both the parent and the child...
-    return thunk.newcoroutine
-
-##    from pypy.rpython.lltypesystem import lltype, lloperation
-##    lloperation.llop.debug_view(lltype.Void, current, thunk,
-##        lloperation.llop.gc_x_size_header(lltype.Signed))
diff --git a/pypy/module/_stackless/rcoroutine.py b/pypy/module/_stackless/rcoroutine.py
deleted file mode 100644
--- a/pypy/module/_stackless/rcoroutine.py
+++ /dev/null
@@ -1,10 +0,0 @@
-from pypy.rlib.rcoroutine import make_coroutine_classes
-from pypy.interpreter.baseobjspace import Wrappable
-
-d = make_coroutine_classes(Wrappable)
-
-Coroutine = d['Coroutine']
-BaseCoState = d['BaseCoState']
-AbstractThunk = d['AbstractThunk']
-syncstate = d['syncstate']
-CoroutineExit = d['CoroutineExit']
diff --git a/pypy/module/_stackless/stackless_flags.py b/pypy/module/_stackless/stackless_flags.py
deleted file mode 100644
--- a/pypy/module/_stackless/stackless_flags.py
+++ /dev/null
@@ -1,201 +0,0 @@
-"""
-basic definitions for tasklet flags.
-For simplicity and compatibility,
-they are defined the same for coroutines,
-even if they are not used.
-
-taken from tasklet_structs.h
-----------------------------
-
-/***************************************************************************
-
-    Tasklet Flag Definition
-    -----------------------
-
-    blocked:        The tasklet is either waiting in a channel for
-                    writing (1) or reading (-1) or not blocked (0).
-                    Maintained by the channel logic. Do not change.
-
-    atomic:         If true, schedulers will never switch. Driven by
-                    the code object or dynamically, see below.
-
-    ignore_nesting: Allows auto-scheduling, even if nesting_level
-                    is not zero.
-
-    autoschedule:   The tasklet likes to be auto-scheduled. User driven.
-
-    block_trap:     Debugging aid. Whenever the tasklet would be
-                    blocked by a channel, an exception is raised.
-
-    is_zombie:      This tasklet is almost dead, its deallocation has
-                    started. The tasklet *must* die at some time, or the
-                    process can never end.
-
-    pending_irq:    If set, an interrupt was issued during an atomic
-                    operation, and should be handled when possible.
-
-
-    Policy for atomic/autoschedule and switching:
-    ---------------------------------------------
-    A tasklet switch can always be done explicitly by calling schedule().
-    Atomic and schedule are concerned with automatic features.
-
-    atomic  autoschedule
-
-        1       any     Neither a scheduler nor a watchdog will
-                        try to switch this tasklet.
-
-        0       0       The tasklet can be stopped on desire, or it
-                        can be killed by an exception.
-
-        0       1       Like above, plus auto-scheduling is enabled.
-
-    Default settings:
-    -----------------
-    All flags are zero by default.
-
- ***************************************************************************/
-
-typedef struct _tasklet_flags {
-        int blocked: 2;
-        unsigned int atomic: 1;
-        unsigned int ignore_nesting: 1;
-        unsigned int autoschedule: 1;
-        unsigned int block_trap: 1;
-        unsigned int is_zombie: 1;
-        unsigned int pending_irq: 1;
-} PyTaskletFlagStruc;
-"""
-
-from pypy.rlib.rarithmetic import LONG_BIT, intmask
-
-class BitSetDef(object):
-    __slots__ = "_names __dict__ _attrname".split()
-
-    def __init__(self, _attrname):
-        self._names = []
-        self._attrname = _attrname
-        
-    def __setattr__(self, key, value):
-        if key not in self.__slots__:
-            assert key not in self.__dict__
-            self._names.append(key)
-        object.__setattr__(self, key, value)
-
-    def __iter__(self):
-        return self._enum_objects()
-    
-    def _enum_objects(self):
-        for name in self._names:
-            yield name, getattr(self, name)
-
-# negative values are user-writable
-flags = BitSetDef("flags")
-flags.blocked           =   2, """writing (1) or reading (-1) or not blocked (0)"""
-flags.atomic            =  -1, """If true, schedulers will never switch"""
-flags.ignore_nesting    =  -1, """allow auto-scheduling in nested interpreters"""
-flags.autoschedule      =  -1, """enable auto-scheduling"""
-flags.block_trap        =  -1, """raise an exception instead of blocking"""
-flags.is_zombie         =   1, """__del__ is in progress"""
-flags.pending_irq       =   1, """an interrupt occured while being atomic"""
-
-def make_get_bits(name, bits, shift):
-    """ return a bool for single bits, signed int otherwise """
-    signmask = 1 << (bits - 1 + shift)
-    lshift = bits + shift
-    rshift = bits
-    if bits == 1:
-        return "bool(%s & 0x%x)" % (name, signmask)
-    else:
-        return "intmask(%s << (LONG_BIT-%d)) >> (LONG_BIT-%d)" % (name, lshift, rshift)
-
-def make_set_bits(name, bits, shift):
-    datamask = int('1' * bits, 2)
-    clearmask = datamask << shift
-    return "%s & ~0x%x | (value & 0x%x) << %d" % (name, clearmask, datamask, shift)
-
-def gen_code():
-    from cStringIO import StringIO
-    f = StringIO()
-    print >> f, "class StacklessFlags(object):"
-    print >> f, "    _mixin_ = True"
-    shift = 0
-    field = "self.%s" % flags._attrname
-    for name, (bits, doc) in flags:
-        write, bits = bits < 0, abs(bits)
-        print >> f
-        print >> f, '    def get_%s(self):' % name
-        print >> f, '        """%s"""' % doc
-        print >> f, '        return %s' % make_get_bits(field, bits, shift)
-        print >> f, '    def set_%s(self, value):' % name
-        print >> f, '        """%s"""' % doc
-        print >> f, '        %s = %s' % (field, make_set_bits(field, bits, shift))
-        print >> f, '    set_%s._public = %s' % (name, write)
-        shift += bits
-    return f.getvalue()
-
-# BEGIN generated code
-class StacklessFlags(object):
-    _mixin_ = True
-
-    def get_blocked(self):
-        """writing (1) or reading (-1) or not blocked (0)"""
-        return intmask(self.flags << (LONG_BIT-2)) >> (LONG_BIT-2)
-    def set_blocked(self, value):
-        """writing (1) or reading (-1) or not blocked (0)"""
-        self.flags = self.flags & ~0x3 | (value & 0x3) << 0
-    set_blocked._public = False
-
-    def get_atomic(self):
-        """If true, schedulers will never switch"""
-        return bool(self.flags & 0x4)
-    def set_atomic(self, value):
-        """If true, schedulers will never switch"""
-        self.flags = self.flags & ~0x4 | (value & 0x1) << 2
-    set_atomic._public = True
-
-    def get_ignore_nesting(self):
-        """allow auto-scheduling in nested interpreters"""
-        return bool(self.flags & 0x8)
-    def set_ignore_nesting(self, value):
-        """allow auto-scheduling in nested interpreters"""
-        self.flags = self.flags & ~0x8 | (value & 0x1) << 3
-    set_ignore_nesting._public = True
-
-    def get_autoschedule(self):
-        """enable auto-scheduling"""
-        return bool(self.flags & 0x10)
-    def set_autoschedule(self, value):
-        """enable auto-scheduling"""
-        self.flags = self.flags & ~0x10 | (value & 0x1) << 4
-    set_autoschedule._public = True
-
-    def get_block_trap(self):
-        """raise an exception instead of blocking"""
-        return bool(self.flags & 0x20)
-    def set_block_trap(self, value):
-        """raise an exception instead of blocking"""
-        self.flags = self.flags & ~0x20 | (value & 0x1) << 5
-    set_block_trap._public = True
-
-    def get_is_zombie(self):
-        """__del__ is in progress"""
-        return bool(self.flags & 0x40)
-    def set_is_zombie(self, value):
-        """__del__ is in progress"""
-        self.flags = self.flags & ~0x40 | (value & 0x1) << 6
-    set_is_zombie._public = False
-
-    def get_pending_irq(self):
-        """an interrupt occured while being atomic"""
-        return bool(self.flags & 0x80)
-    def set_pending_irq(self, value):
-        """an interrupt occured while being atomic"""
-        self.flags = self.flags & ~0x80 | (value & 0x1) << 7
-    set_pending_irq._public = False
-
-# END generated code
-
-if __name__ == '__main__':
-    # paste this into the file
-    print gen_code()
diff --git a/pypy/module/_stackless/test/__init__.py b/pypy/module/_stackless/test/__init__.py
deleted file mode 100644
--- a/pypy/module/_stackless/test/__init__.py
+++ /dev/null
@@ -1,1 +0,0 @@
-#
\ No newline at end of file
diff --git a/pypy/module/_stackless/test/conftest.py b/pypy/module/_stackless/test/conftest.py
deleted file mode 100644
--- a/pypy/module/_stackless/test/conftest.py
+++ /dev/null
@@ -1,8 +0,0 @@
-import sys
-import py.test
-
-def pytest_runtest_setup(item):
-    py.test.importorskip('greenlet')
-    if sys.platform == 'win32':
-        py.test.skip("stackless tests segfault on Windows")
-
diff --git a/pypy/module/_stackless/test/slp_test_pickle.py b/pypy/module/_stackless/test/slp_test_pickle.py
deleted file mode 100644
--- a/pypy/module/_stackless/test/slp_test_pickle.py
+++ /dev/null
@@ -1,35 +0,0 @@
-from pypy.conftest import gettestobjspace
-
-# app-level testing of coroutine pickling
-
-class AppTest_Pickle:
-
-    def setup_class(cls):
-        space = gettestobjspace(usemodules=('_stackless',))
-        cls.space = space
-
-    def test_simple_ish(self):
-
-        output = []
-        import _stackless
-        def f(coro, n, x):
-            if n == 0:
-                coro.switch()
-                return
-            f(coro, n-1, 2*x)
-            output.append(x)
-
-        def example():
-            main_coro = _stackless.coroutine.getcurrent()
-            sub_coro = _stackless.coroutine()
-            sub_coro.bind(f, main_coro, 5, 1)
-            sub_coro.switch()
-
-            import pickle
-            pckl = pickle.dumps(sub_coro)
-            new_coro = pickle.loads(pckl)
-
-            new_coro.switch()
-
-        example()
-        assert output == [16, 8, 4, 2, 1]
diff --git a/pypy/module/_stackless/test/test_choicepoint.py b/pypy/module/_stackless/test/test_choicepoint.py
deleted file mode 100644
--- a/pypy/module/_stackless/test/test_choicepoint.py
+++ /dev/null
@@ -1,85 +0,0 @@
-import py; py.test.skip("clonable coroutines not really maintained any more")
-
-from pypy.rlib.rcoroutine import AbstractThunk
-from pypy.module._stackless.rclonable import InterpClonableCoroutine as ClonableCoroutine
-
-class ChoicePointHolder(object):
-    def __init__(self):
-        self.choicepoints = []
-        self.clone_me = False
-        self.answer = 0
-        self.solutions_count = 0
-
-    def next_choice(self):
-        return self.choicepoints.pop()
-
-    def add(self, choice, answer=0):
-        self.choicepoints.append((choice, answer))
-
-    def more_choices(self):
-        return bool(self.choicepoints)
-
-    def choice(self):
-        #os.write(1, "choice\n")
-        self.clone_me = True
-        self.g_main.switch()
-        #os.write(1, "answer: %d\n" % (self.answer,))
-        return self.answer
-
-    def fail(self):
-        self.g_main.switch()
-        assert False
-
-choicepoints = ChoicePointHolder()
-
-# ____________________________________________________________
-
-class SearchTask(AbstractThunk):
-    def call(self):
-        path = []
-        for i in range(10):
-            res = choicepoints.choice()
-            assert len(path) == i
-            path.append(res)
-            #os.write(1, "{%x} trying: %s\n" % (id(path), path))
-            if i == 3:
-                import gc; gc.collect()
-        #os.write(1, "{%x} found a solution: %s\n" % (id(path), path))
-        choicepoints.solutions_count += 1
-
-# ____________________________________________________________
-
-
-class SearchAllTask(AbstractThunk):
-    def call(self):
-        search_coro = ClonableCoroutine()
-        search_coro.bind(SearchTask())
-        choicepoints.add(search_coro)
-
-        #os.write(1, "starting\n")
-        while choicepoints.more_choices():
-            searcher, nextvalue = choicepoints.next_choice()
-            choicepoints.clone_me = False
-            choicepoints.answer = nextvalue
-            #os.write(1, '<<< {%x} %d\n' % (id(searcher), nextvalue))
-            searcher.switch()
-            #os.write(1, '>>> %d\n' % (choicepoints.clone_me,))
-            if choicepoints.clone_me:
-                searcher2 = searcher.clone()
-                #os.write(1, 'searcher = {%x}, searcher2 = {%x}\n' % (
-                #    id(searcher), id(searcher2)))
-                choicepoints.add(searcher, 5)
-                choicepoints.add(searcher2, 4)
-
-def entry_point():
-    choicepoints.g_main = ClonableCoroutine()
-    choicepoints.g_main.bind(SearchAllTask())
-    choicepoints.g_main.switch()
-    return choicepoints.solutions_count
-
-def test_choicepoint():
-    from pypy.translator.c.test import test_newgc
-    tester = test_newgc.TestUsingStacklessFramework()
-    fn = tester.getcompiled(entry_point)
-    res = fn()
-    assert res == 2 ** 10
diff --git a/pypy/module/_stackless/test/test_clonable.py b/pypy/module/_stackless/test/test_clonable.py
deleted file mode 100644
--- a/pypy/module/_stackless/test/test_clonable.py
+++ /dev/null
@@ -1,187 +0,0 @@
-import py; py.test.skip("clonable coroutines not really maintained any more")
-
-from pypy.conftest import gettestobjspace, option
-import py, sys
-
-# app-level testing of coroutine cloning
-
-class AppTestClonable:
-
-    def setup_class(cls):
-        if not option.runappdirect:
-            py.test.skip('pure appdirect test (run with -A)')
-        cls.space = space = gettestobjspace(usemodules=('_stackless',))
-        if not space.is_true(space.appexec([], """():
-            import _stackless
-            return hasattr(_stackless, 'clonable')
-        """)):
-            py.test.skip('no _stackless.clonable')
-
-
-    def test_solver(self):
-        import _stackless
-
-        class Fail(Exception):
-            pass
-
-        class Success(Exception):
-            pass
-
-        def first_solution(func):
-            global next_answer
-            co = _stackless.clonable()
-            co.bind(func)
-            pending = [(co, None)]
-            while pending:
-                co, next_answer = pending.pop()
-                try:
-                    co.switch()
-                except Fail:
-                    pass
-                except Success, e:
-                    return e.args[0]
-                else:
-                    # zero_or_one() called, clone the coroutine
-                    co2 = co.clone()
-                    pending.append((co2, 1))
-                    pending.append((co, 0))
-            raise Fail("no solution")
-
-        pending = []
-        main = _stackless.clonable.getcurrent()
-
-        def zero_or_one():
-            main.switch()
-            return next_answer
-
-        # ____________________________________________________________
-
-        invalid_prefixes = {
-            (0, 0): True,
-            (0, 1, 0): True,
-            (0, 1, 1): True,
-            (1, 0): True,
-            (1, 1, 0, 0): True,
-            }
-
-        def example():
-            test = []
-            for n in range(5):
-                test.append(zero_or_one())
-                if tuple(test) in invalid_prefixes:
-                    raise Fail
-            raise Success(test)
-
-        res = first_solution(example)
-        assert res == [1, 1, 0, 1, 0]
-
-
-    def test_myself_may_not_be_me_any_more(self):
-        import gc
-        from _stackless import clonable
-
-        counter = [0]
-
-        def runner():
-            while 1:
-                assert clonable.getcurrent() is coro
-                counter[0] += 1
-                main.switch()
-
-        main = clonable.getcurrent()
-        coro = clonable()
-        coro.bind(runner)
-
-        coro.switch()
-        assert counter == [1]
-
-        assert clonable.getcurrent() is main
-        coro1 = coro.clone()
-        assert counter == [1]
-        assert clonable.getcurrent() is main
-        coro.switch()
-        assert counter == [2]
-        coro.switch()
-        assert counter == [3]
-        assert clonable.getcurrent() is main
-        del coro1
-        gc.collect()
-        #print "collected!"
-        assert clonable.getcurrent() is main
-        assert counter == [3]
-        coro.switch()
-        assert clonable.getcurrent() is main
-        assert counter == [4]
-
-
-    def test_fork(self):
-        import _stackless
-
-        class Fail(Exception):
-            pass
-
-        class Success(Exception):
-            pass
-
-        def first_solution(func):
-            global next_answer
-            co = _stackless.clonable()
-            co.bind(func)
-            try:
-                co.switch()
-            except Success, e:
-                return e.args[0]
-
-        def zero_or_one():
-            sub = _stackless.fork()
-            if sub is not None:
-                # in the parent: run the child first
-                try:
-                    sub.switch()
-                except Fail:
-                    pass
-                # then proceed with answer '1'
-                return 1
-            else:
-                # in the child: answer '0'
-                return 0
-
-        # ____________________________________________________________
-
-        invalid_prefixes = {
-            (0, 0): True,
-            (0, 1, 0): True,
-            (0, 1, 1): True,
-            (1, 0): True,
-            (1, 1, 0, 0): True,
-            }
-
-        def example():
-            test = []
-            for n in range(5):
-                test.append(zero_or_one())
-                if tuple(test) in invalid_prefixes:
-                    raise Fail
-            raise Success(test)
-
-        res = first_solution(example)
-        assert res == [1, 1, 0, 1, 0]
-
-    def test_clone_before_start(self):
-        """Tests that a clonable coroutine can be
-        cloned before it is started
-        (this used to fail with a segmentation fault)
-        """
-        import _stackless
-
-        counter = [0]
-        def simple_coro():
-            print "hello"
-            counter[0] += 1
-
-        s = _stackless.clonable()
-        s.bind(simple_coro)
-        t = s.clone()
-        s.switch()
-        t.switch()
-        assert counter[0] == 2
diff --git a/pypy/module/_stackless/test/test_composable_coroutine.py b/pypy/module/_stackless/test/test_composable_coroutine.py
deleted file mode 100644
--- a/pypy/module/_stackless/test/test_composable_coroutine.py
+++ /dev/null
@@ -1,133 +0,0 @@
-""" a faith is the connection between past and future that divides the
-    application into switch-compatible chunks.
-    -- stakkars
-"""
-from pypy.conftest import gettestobjspace
-from py.test import skip
-
-class AppTest_ComposableCoroutine:
-
-    def setup_class(cls):
-        space = gettestobjspace(usemodules=('_stackless',))
-        cls.space = space
-
-        cls.w_generator_ = space.appexec([], """():
-            import _stackless
-
-            generators_costate = _stackless.usercostate()
-            main = generators_costate.getcurrent()
-
-            class generator_iterator(_stackless.coroutine):
-
-                def __iter__(self):
-                    return self
-
-                def next(self):
-                    if self.gi_answer is not None:
-                        raise ValueError('stackless-generator'
-                                         ' already executing')
-                    self.gi_answer = []
-                    self.gi_caller = generators_costate.getcurrent()
-                    self.switch()
-                    answer = self.gi_answer
-                    self.gi_answer = None
-                    if answer:
-                        return answer[0]
-                    else:
-                        raise StopIteration
-
-            def generator(f):
-                def myfunc(*args, **kwds):
-                    g = generators_costate.spawn(generator_iterator)
-                    g.gi_answer = None
-                    g.bind(f, *args, **kwds)
-                    return g
-                return myfunc
-
-            def Yield(value):
-                g = generators_costate.getcurrent()
-                if g is main:
-                    raise ValueError('Yield() outside any stackless-generator')
-                assert isinstance(g, generator_iterator)
-                assert g.gi_answer == []
-                g.gi_answer.append(value)
-                g.gi_caller.switch()
-
-            generator.Yield = Yield
-            generator._costate = generators_costate
-            return (generator,)
-        """)
-
-    def test_simple_costate(self):
-        import _stackless
-        costate = _stackless.usercostate()
-        main = costate.getcurrent()
-
-        result = []
-        def f():
-            result.append(costate.getcurrent())
-        co = costate.spawn()
-        co.bind(f)
-        co.switch()
-        assert result == [co]
-
-    def test_generator(self):
-        generator, = self.generator_
-
-        def squares(n):
-            for i in range(n):
-                generator.Yield(i*i)
-        squares = generator(squares)
-
-        lst1 = [i*i for i in range(10)]
-        for got in squares(10):
-            expected = lst1.pop(0)
-            assert got == expected
-        assert lst1 == []
-
-    def test_multiple_costates(self):
-        """Test that two independent costates mix transparently:
-
-        - compute_costate, used for a coroutine that fills a list with
-                           some more items each time it is switched to
-
-        - generators_costate, used interally by self.generator (see above)
-        """
-
-        import _stackless
-        generator, = self.generator_
-
-        # you can see how it fails if we don't have two different costates
-        # by setting compute_costate to generator._costate instead
-        compute_costate = _stackless.usercostate()
-        compute_main = compute_costate.getcurrent()
-        lst = []
-
-        def filler():     # -> 0, 1, 2, 100, 101, 102, 200, 201, 202, 300 ...
-            for k in range(5):
-                for j in range(3):
-                    lst.append(100 * k + j)
-                compute_main.switch()
-
-        filler_co = compute_costate.spawn()
-        filler_co.bind(filler)
-
-        def grab_next_value():
-            while not lst:
-                #print 'filling more...'
-                filler_co.switch()
-                #print 'now lst =', lst
-            #print 'grabbing', lst[0]
-            return lst.pop(0)
-
-        def squares(n):
-            for i in range(n):
-                #print 'square:', i
-                generator.Yield(i*grab_next_value())
-        squares = generator(squares)
-
-        lst1 = [0, 1, 4,  300, 404, 510,  1200, 1407, 1616,  2700]
-        for got in squares(10):
-            expected = lst1.pop(0)
-            assert got == expected
-        assert lst1 == []
diff --git a/pypy/module/_stackless/test/test_coroutine.py b/pypy/module/_stackless/test/test_coroutine.py
deleted file mode 100644
--- a/pypy/module/_stackless/test/test_coroutine.py
+++ /dev/null
@@ -1,168 +0,0 @@
-from pypy.conftest import gettestobjspace, option
-from py.test import skip
-
-
-class AppTest_Coroutine:
-
-    def setup_class(cls):
-        space = gettestobjspace(usemodules=('_stackless',))
-        cls.space = space
-
-    def test_raise_propagate(self):
-        import _stackless as stackless
-        co = stackless.coroutine()
-        def f():
-            return 1/0
-        co.bind(f)
-        try:
-            co.switch()
-        except ZeroDivisionError:
-            pass
-        else:
-            raise AssertionError("exception not propagated")
-
-    def test_strange_test(self):
-        from _stackless import coroutine
-        def f():
-            print "in new coro"
-            return 42
-        def create():
-            b = coroutine()
-            b.bind(f)
-            print "bound"
-            b.switch()
-            print "switched"
-            return b
-        a = coroutine()
-        a.bind(create)
-        b = a.switch()
-        # now b.parent = a
-        def nothing():
-            pass
-        a.bind(nothing)
-        def kill():
-            # this sets a.parent = b
-            a.kill()
-        b.bind(kill)
-        b.switch()
-
-    def test_kill(self):
-        import _stackless as stackless
-        co = stackless.coroutine()
-        def f():
-            pass
-        co.bind(f)
-        assert co.is_alive
-        co.kill()
-        assert not co.is_alive
-
-    def test_kill_running(self):
-        coroutineexit = []
-        import _stackless as stackless
-        main = stackless.coroutine.getcurrent()
-        result = []
-        co = stackless.coroutine()
-        def f():
-            x = 2
-            try:
-                result.append(1)
-                main.switch()
-                x = 3
-            except CoroutineExit:
-                coroutineexit.append(True)
-                raise
-            finally:
-                result.append(x)
-            result.append(4)
-        co.bind(f)
-        assert co.is_alive
-        co.switch()
-        assert co.is_alive
-        assert result == [1]
-        co.kill()
-        assert not co.is_alive
-        assert result == [1, 2]
-        assert coroutineexit == [True]
-
-    def test_bogus_bind(self):
-        import _stackless as stackless
-        co = stackless.coroutine()
-        def f():
-            pass
-        co.bind(f)
-        raises(ValueError, co.bind, f)
-
-    def test__framestack(self):
-        import _stackless as stackless
-        main = stackless.coroutine.getmain()
-        co = stackless.coroutine()
-        def g():
-            return co._framestack
-        def f():
-            return g()
-
-        co.bind(f)
-        stack = co.switch()
-        assert stack == () # running corountine, _framestack is empty
-
-        co = stackless.coroutine()
-        def g():
-            return main.switch()
-        def f():
-            return g()
-
-        co.bind(f)
-        co.switch()
-        stack = co._framestack
-        assert len(stack) == 2
-        assert stack[0].f_code is f.func_code
-        assert stack[1].f_code is g.func_code
-
-        co = stackless.coroutine()
-
-
-
-class AppTestDirect:
-    def setup_class(cls):
-        if not option.runappdirect:
-            skip('pure appdirect test (run with -A)')
-        cls.space = gettestobjspace(usemodules=('_stackless',))
-
-    def test_stack_depth_limit(self):
-        import sys
-        import _stackless as stackless
-        st = stackless.get_stack_depth_limit()
-        try:
-            stackless.set_stack_depth_limit(1)
-            assert stackless.get_stack_depth_limit() == 1
-            try:
-                co = stackless.coroutine()
-                def f():
-                    pass
-                co.bind(f)
-                co.switch()
-            except RuntimeError:
-                pass
-        finally:
-            stackless.set_stack_depth_limit(st)
-
-class TestRandomThings:
-    def setup_class(cls):
-        cls.space = gettestobjspace(usemodules=('_stackless',))
-
-    def test___del___handling(self):
-        space = self.space
-        w_l = space.newlist([])
-        coro = space.appexec([w_l], """(l):
-            from _stackless import coroutine
-            class MyCoroutine(coroutine):
-                def __del__(self):
-                    l.append(self.is_zombie)
-            return MyCoroutine()
-        """)
-        coro.__del__()
-        space.user_del_action.perform(space.getexecutioncontext(), None)
-        coro._kill_finally()
-        assert space.len_w(w_l) == 1
-        res = space.is_true(space.getitem(w_l, space.wrap(0)))
-        assert res
diff --git a/pypy/module/_stackless/test/test_greenlet.py b/pypy/module/_stackless/test/test_greenlet.py
deleted file mode 100644
--- a/pypy/module/_stackless/test/test_greenlet.py
+++ /dev/null
@@ -1,643 +0,0 @@
-from pypy.conftest import gettestobjspace, skip_on_missing_buildoption
-
-class AppTest_Greenlet:
-
-    def setup_class(cls):
-        space = gettestobjspace(usemodules=('_stackless',))
-        cls.space = space
-
-    def test_very_simple(self):
-        from _stackless import greenlet
-        lst = []
-        def f(x):
-            lst.append(x)
-            return x + 10
-        g = greenlet(f)
-        assert not g
-        res = g.switch(20)
-        assert res == 30
-        assert lst == [20]
-        assert g.dead
-        assert not g
-
-    def test_switch_back_to_main(self):
-        from _stackless import greenlet
-        lst = []
-        main = greenlet.getcurrent()
-        def f(x):
-            lst.append(x)
-            x = main.switch(x + 10)
-            return 40 + x 
-        g = greenlet(f)
-        res = g.switch(20)
-        assert res == 30
-        assert lst == [20]
-        assert not g.dead
-        res = g.switch(2)
-        assert res == 42
-        assert g.dead
-
-    def test_simple(self):
-        from _stackless import greenlet
-        lst = []
-        gs = []
-        def f():
-            lst.append(1)
-            greenlet.getcurrent().parent.switch()
-            lst.append(3)
-        g = greenlet(f)
-        lst.append(0)
-        g.switch()
-        lst.append(2)
-        g.switch()
-        lst.append(4)
-        assert lst == range(5)
-
-    def test_exception_simple(self):
-        from _stackless import greenlet
-        def f():
-            raise ValueError
-        g1 = greenlet(f)
-        raises(ValueError, g1.switch)
-
-    def test_exception_propagate(self):
-        from _stackless import greenlet
-        def f():
-            raise ValueError
-        def g():
-            return g1.switch()
-        g1 = greenlet(f)
-        g2 = greenlet(g)
-        raises(ValueError, g1.switch)
-        g1 = greenlet(f)
-        raises(ValueError, g2.switch)
-
-
-    def test_exc_info_save_restore(self):
-        from _stackless import greenlet
-        import sys
-        def f():
-            try:
-                raise ValueError('fun')
-            except:
-                exc_info = sys.exc_info()
-                greenlet(h).switch()
-                assert exc_info == sys.exc_info()
-
-        def h():
-            assert sys.exc_info() == (None, None, None)
-
-        greenlet(f).switch()
-
-    def test_exception(self):
-        from _stackless import greenlet
-        import sys
-        def fmain(seen):
-            try:
-                greenlet.getcurrent().parent.switch()
-            except:
-                seen.append(sys.exc_info()[0])
-                raise
-            raise ValueError
-        seen = []
-        g1 = greenlet(fmain)
-        g2 = greenlet(fmain)
-        g1.switch(seen)
-        g2.switch(seen)
-        raises(TypeError, "g2.parent = 1")
-        g2.parent = g1
-        assert seen == []
-        raises(ValueError, g2.switch)
-        assert seen == [ValueError]
-        g2.switch()
-        assert seen == [ValueError]
-
-    def test_send_exception(self):
-        from _stackless import greenlet
-        import sys
-        def send_exception(g, exc):
-            # note: send_exception(g, exc)  can be now done with  g.throw(exc).
-            # the purpose of this test is to explicitely check the propagation rules.
-            def crasher(exc):
-                raise exc
-            g1 = greenlet(crasher)
-            g1.parent = g
-            g1.switch(exc)
-        def fmain(seen):
-            try:
-                greenlet.getcurrent().parent.switch()
-            except:
-                seen.append(sys.exc_info()[0])
-                raise
-            raise ValueError
-
-        seen = []
-        g1 = greenlet(fmain)
-        g1.switch(seen)
-        raises(KeyError, "send_exception(g1, KeyError)")
-        assert seen == [KeyError]
-        seen = []
-        g1 = greenlet(fmain)
-        g1.switch(seen)
-        raises(KeyError, "g1.throw(KeyError)")
-        assert seen == [KeyError]
-        assert g1.dead
-
-    def test_frame(self):
-        from _stackless import greenlet
-        import sys
-        def f1():
-            f = sys._getframe(0)
-            assert f.f_back is None
-            greenlet.getcurrent().parent.switch(f)
-            return "meaning of life"
-        g = greenlet(f1)
-        frame = g.switch()
-        assert frame is g.gr_frame
-        assert g
-        next = g.switch()
-        assert not g
-        assert next == "meaning of life"
-        assert g.gr_frame is None
-
-    def test_mixing_greenlet_coroutine(self):
-        from _stackless import greenlet, coroutine
-        lst = []
-        def f():
-            lst.append(1)
-            greenlet.getcurrent().parent.switch()
-            lst.append(3)
-        def make_h(c):
-            def h():
-                g = greenlet(f)
-                lst.append(0)
-                g.switch()
-                c.switch()
-                lst.append(2)
-                g.switch()
-                c.switch()
-                lst.append(4)
-                c.switch()
-            return h
-        c1 = coroutine.getcurrent()
-        c2 = coroutine()
-        c3 = coroutine()
-        c2.bind(make_h(c3))
-        c3.bind(make_h(c2))
-        c2.switch()
-        assert lst == [0, 1, 0, 1, 2, 3, 2, 3, 4, 4]
-
-    def test_dealloc(self):
-        skip("not working yet")
-        from _stackless import greenlet
-        import sys
-        def fmain(seen):
-            try:
-                greenlet.getcurrent().parent.switch()
-            except:
-                seen.append(sys.exc_info()[0])
-                raise
-            raise ValueError
-        seen = []
-        seen = []
-        g1 = greenlet(fmain)
-        g2 = greenlet(fmain)
-        g1.switch(seen)
-        g2.switch(seen)
-        assert seen == []
-        del g1
-        assert seen == [greenlet.GreenletExit]
-        del g2
-        assert seen == [greenlet.GreenletExit, greenlet.GreenletExit]
-
-
-# ____________________________________________________________
-#
-# The tests from greenlets.
-# For now, without the ones that involve threads
-#
-class AppTest_PyMagicTestGreenlet:
-
-    def setup_class(cls):
-        space = gettestobjspace(usemodules=('_stackless',))
-        cls.space = space
-        cls.w_glob = space.appexec([], """():
-            import sys
-            from _stackless import greenlet
-
-            class SomeError(Exception):
-                pass
-
-            def fmain(seen):
-                try:
-                    greenlet.getcurrent().parent.switch()
-                except:
-                    seen.append(sys.exc_info()[0])
-                    raise
-                raise SomeError
-
-            class Glob: pass
-            glob = Glob()
-            glob.__dict__.update(locals())
-            return glob
-        """)
-
-    def test_simple(self):
-        greenlet = self.glob.greenlet
-        lst = []
-        def f():
-            lst.append(1)
-            greenlet.getcurrent().parent.switch()
-            lst.append(3)
-        g = greenlet(f)
-        lst.append(0)
-        g.switch()
-        lst.append(2)
-        g.switch()
-        lst.append(4)
-        assert lst == range(5)
-
-    def test_exception(self):
-        greenlet  = self.glob.greenlet
-        fmain     = self.glob.fmain
-        SomeError = self.glob.SomeError
-        seen = []
-        g1 = greenlet(fmain)
-        g2 = greenlet(fmain)
-        g1.switch(seen)
-        g2.switch(seen)
-        g2.parent = g1
-        assert seen == []
-        raises(SomeError, g2.switch)
-        assert seen == [SomeError]
-        g2.switch()
-        assert seen == [SomeError]
-
-    def test_send_exception(self):
-        greenlet  = self.glob.greenlet
-        fmain     = self.glob.fmain
-        def send_exception(g, exc):
-            # note: send_exception(g, exc)  can be now done with  g.throw(exc).
-            # the purpose of this test is to explicitely check the
-            # propagation rules.
-            def crasher(exc):
-                raise exc
-            g1 = greenlet(crasher, parent=g)
-            g1.switch(exc)
-
-        seen = []
-        g1 = greenlet(fmain)
-        g1.switch(seen)
-        raises(KeyError, "send_exception(g1, KeyError)")
-        assert seen == [KeyError]
-
-    def test_dealloc(self):
-        skip("XXX in-progress: GC handling of greenlets")
-        import gc
-        greenlet = self.glob.greenlet
-        fmain    = self.glob.fmain
-        seen = []
-        g1 = greenlet(fmain)
-        g2 = greenlet(fmain)
-        g1.switch(seen)
-        g2.switch(seen)
-        assert seen == []
-        del g1
-        gc.collect()
-        assert seen == [greenlet.GreenletExit]
-        del g2
-        gc.collect()
-        assert seen == [greenlet.GreenletExit, greenlet.GreenletExit]
-
-    def test_frame(self):
-        import sys
-        greenlet = self.glob.greenlet
-        def f1():
-            f = sys._getframe(0)
-            assert f.f_back is None
-            greenlet.getcurrent().parent.switch(f)
-            return "meaning of life"
-        g = greenlet(f1)
-        frame = g.switch()
-        assert frame is g.gr_frame
-        assert g
-        next = g.switch()
-        assert not g
-        assert next == "meaning of life"
-        assert g.gr_frame is None
-
-
-class AppTest_PyMagicTestThrow:
-
-    def setup_class(cls):
-        space = gettestobjspace(usemodules=('_stackless',))
-        cls.space = space
-
-    def test_class(self):
-        from _stackless import greenlet
-        def switch(*args):
-            return greenlet.getcurrent().parent.switch(*args)
-
-        def f():
-            try:
-                switch("ok")
-            except RuntimeError:
-                switch("ok")
-                return
-            switch("fail")
-
-        g = greenlet(f)
-        res = g.switch()
-        assert res == "ok"
-        res = g.throw(RuntimeError)
-        assert res == "ok"
-
-    def test_val(self):
-        from _stackless import greenlet
-        def switch(*args):
-            return greenlet.getcurrent().parent.switch(*args)
-
-        def f():
-            try:
-                switch("ok")
-            except RuntimeError, val:
-                if str(val) == "ciao":
-                    switch("ok")
-                    return
-            switch("fail")
-
-        g = greenlet(f)
-        res = g.switch()
-        assert res == "ok"
-        res = g.throw(RuntimeError("ciao"))
-        assert res == "ok"
-
-        g = greenlet(f)
-        res = g.switch()
-        assert res == "ok"
-        res = g.throw(RuntimeError, "ciao")
-        assert res == "ok"
-
-    def test_kill(self):
-        from _stackless import greenlet
-        def switch(*args):
-            return greenlet.getcurrent().parent.switch(*args)
-
-        def f():
-            switch("ok")
-            switch("fail")
-
-        g = greenlet(f)
-        res = g.switch()
-        assert res == "ok"
-        res = g.throw()
-        assert isinstance(res, greenlet.GreenletExit)
-        assert g.dead
-        res = g.throw()    # immediately eaten by the already-dead greenlet
-        assert isinstance(res, greenlet.GreenletExit)
-
-    def test_throw_goes_to_original_parent(self):
-        from _stackless import greenlet
-        main = greenlet.getcurrent()
-        def f1():
-            try:
-                main.switch("f1 ready to catch")
-            except IndexError:
-                return "caught"
-            else:
-                return "normal exit"
-        def f2():
-            main.switch("from f2")
-
-        g1 = greenlet(f1)
-        g2 = greenlet(f2, parent=g1)
-        raises(IndexError, g2.throw, IndexError)
-        assert g2.dead
-        assert g1.dead
-
-        g1 = greenlet(f1)
-        g2 = greenlet(f2, parent=g1)
-        res = g1.switch()
-        assert res == "f1 ready to catch"
-        res = g2.throw(IndexError)
-        assert res == "caught"
-        assert g2.dead
-        assert g1.dead
-
-        g1 = greenlet(f1)
-        g2 = greenlet(f2, parent=g1)
-        res = g1.switch()
-        assert res == "f1 ready to catch"
-        res = g2.switch()
-        assert res == "from f2"
-        res = g2.throw(IndexError)
-        assert res == "caught"
-        assert g2.dead
-        assert g1.dead
-            
-
-class AppTest_PyMagicTestGenerator:
-
-    def setup_class(cls):
-        space = gettestobjspace(usemodules=('_stackless',))
-        cls.space = space
-
-    def test_generator(self):
-        from _stackless import greenlet
-
-        class genlet(greenlet):
-
-            def __init__(self, *args, **kwds):
-                self.args = args
-                self.kwds = kwds
-
-            def run(self):
-                fn, = self.fn
-                fn(*self.args, **self.kwds)
-
-            def __iter__(self):
-                return self
-
-            def next(self):
-                self.parent = greenlet.getcurrent()
-                result = self.switch()
-                if self:
-                    return result
-                else:
-                    raise StopIteration
-
-        def Yield(value):
-            g = greenlet.getcurrent()
-            while not isinstance(g, genlet):
-                if g is None:
-                    raise RuntimeError, 'yield outside a genlet'
-                g = g.parent
-            g.parent.switch(value)
-
-        def generator(func):
-            class generator(genlet):
-                fn = (func,)
-            return generator
-
-        # ___ test starts here ___
-        seen = []
-        def g(n):
-            for i in range(n):
-                seen.append(i)
-                Yield(i)
-        g = generator(g)
-        for k in range(3):
-            for j in g(5):
-                seen.append(j)
-        assert seen == 3 * [0, 0, 1, 1, 2, 2, 3, 3, 4, 4]
-
-
-class AppTest_PyMagicTestGeneratorNested:
-
-    def setup_class(cls):
-        space = gettestobjspace(usemodules=('_stackless',))
-        cls.space = space
-        cls.w_glob = space.appexec([], """():
-            from _stackless import greenlet
-
-            class genlet(greenlet):
-
-                def __init__(self, *args, **kwds):
-                    self.args = args
-                    self.kwds = kwds
-                    self.child = None
-
-                def run(self):
-                    fn, = self.fn
-                    fn(*self.args, **self.kwds)
-
-                def __iter__(self):
-                    return self
-
-                def set_child(self, child):
-                    self.child = child
-
-                def next(self):
-                    if self.child:
-                        child = self.child
-                        while child.child:
-                            tmp = child
-                            child = child.child
-                            tmp.child = None
-
-                        result = child.switch()
-                    else:
-                        self.parent = greenlet.getcurrent()            
-                        result = self.switch()
-
-                    if self:
-                        return result
-                    else:
-                        raise StopIteration
-
-            def Yield(value, level = 1):
-                g = greenlet.getcurrent()
-
-                while level != 0:
-                    if not isinstance(g, genlet):
-                        raise RuntimeError, 'yield outside a genlet'
-                    if level > 1:
-                        g.parent.set_child(g)
-                    g = g.parent
-                    level -= 1
-
-                g.switch(value)
-
-            def Genlet(func):
-                class Genlet(genlet):
-                    fn = (func,)
-                return Genlet
-
-            class Glob: pass
-            glob = Glob()
-            glob.__dict__.update(locals())
-            return glob
-        """)
-
-    def test_genlet_1(self):
-        Genlet = self.glob.Genlet
-        Yield  = self.glob.Yield
-
-        def g1(n, seen):
-            for i in range(n):
-                seen.append(i+1)
-                yield i
-
-        def g2(n, seen):
-            for i in range(n):
-                seen.append(i+1)
-                Yield(i)
-
-        g2 = Genlet(g2)
-
-        def nested(i):
-            Yield(i)
-
-        def g3(n, seen):
-            for i in range(n):
-                seen.append(i+1)
-                nested(i)
-        g3 = Genlet(g3)
-
-        raises(RuntimeError, Yield, 10)
-        for g in [g1, g2, g3]:
-            seen = []
-            for k in range(3):
-                for j in g(5, seen):
-                    seen.append(j)
-            assert seen == 3 * [1, 0, 2, 1, 3, 2, 4, 3, 5, 4]
-        raises(RuntimeError, Yield, 10)
-
-    def test_nested_genlets(self):
-        Genlet = self.glob.Genlet
-        Yield  = self.glob.Yield
-        def a(n):
-            if n == 0:
-                return
-            for ii in ax(n-1):
-                Yield(ii)
-            Yield(n)
-        ax = Genlet(a)
-        seen = []
-        for ii in ax(5):
-            seen.append(ii)
-        assert seen == [1, 2, 3, 4, 5]
-
-    def test_perms(self):
-        Genlet = self.glob.Genlet
-        Yield  = self.glob.Yield
-        def perms(l):
-            if len(l) > 1:
-                for e in l:
-                    # No syntactical sugar for generator expressions
-                    [Yield([e] + p) for p in perms([x for x in l if x!=e])]
-            else:
-                Yield(l)
-        perms = Genlet(perms)
-        gen_perms = perms(range(4))
-        permutations = list(gen_perms)
-        assert len(permutations) == 4*3*2*1
-        assert [0,1,2,3] in permutations
-        assert [3,2,1,0] in permutations
-
-    def test_layered_genlets(self):
-        Genlet = self.glob.Genlet
-        Yield  = self.glob.Yield
-        def gr1(n):
-            for ii in range(1, n):
-                Yield(ii)
-                Yield(ii * ii, 2)
-        gr1 = Genlet(gr1)
-        def gr2(n, seen):
-            for ii in gr1(n):
-                seen.append(ii)
-        gr2 = Genlet(gr2)
-        seen = []
-        for ii in gr2(5, seen):
-            seen.append(ii)
-        assert seen == [1, 1, 2, 4, 3, 9, 4, 16]
diff --git a/pypy/module/_stackless/test/test_interp_clonable.py b/pypy/module/_stackless/test/test_interp_clonable.py
deleted file mode 100644
--- a/pypy/module/_stackless/test/test_interp_clonable.py
+++ /dev/null
@@ -1,118 +0,0 @@
-"""
-testing cloning
-"""
-import py; py.test.skip("clonable coroutines not really maintained any more")
-
-from pypy import conftest; conftest.translation_test_so_skip_if_appdirect()
-from pypy.translator.c import gc
-from pypy.rpython.memory.gctransform import stacklessframework
-from pypy.rpython.memory.test import test_transformed_gc
-from pypy.module._stackless.rclonable import InterpClonableCoroutine as ClonableCoroutine
-from pypy.module._stackless.rclonable import AbstractThunk, fork
-
-class TestClonableCoroutine(test_transformed_gc.GCTest):
-
-    gcname = "marksweep"
-    stacklessgc = True
-    class gcpolicy(gc.StacklessFrameworkGcPolicy):
-        class transformerclass(stacklessframework.StacklessFrameworkGCTransformer):
-            GC_PARAMS = {'start_heap_size': 4096 }
-
-    def test_clone(self):
-        class T(AbstractThunk):
-            def __init__(self, result):
-                self.result = result
-            def call(self):
-                self.result.append(2)
-                ClonableCoroutine.getmain().switch()
-                self.result.append(4)
-        def f():
-            result = []
-            coro = ClonableCoroutine()
-            coro.bind(T(result))
-            result.append(1)
-            coro.switch()
-            coro2 = coro.clone()
-            result.append(3)
-            coro2.switch()
-            result.append(5)
-            coro.switch()
-            result.append(6)
-            n = 0
-            for i in result:
-                n = n*10 + i
-            return n
-
-        run = self.runner(f)
-        res = run([])
-        assert res == 1234546
-
-    def test_clone_local_state(self):
-        class T(AbstractThunk):
-            def __init__(self, result):
-                self.result = result
-            def call(self):
-                localstate = []
-                localstate.append(10)
-                self.result.append(2)
-                ClonableCoroutine.getmain().switch()
-                localstate.append(20)
-                if localstate == [10, 20]:
-                    self.result.append(4)
-                else:
-                    self.result.append(0)
-        def f():
-            result = []
-            coro = ClonableCoroutine()
-            coro.bind(T(result))
-            result.append(1)
-            coro.switch()
-            coro2 = coro.clone()
-            result.append(3)
-            coro2.switch()
-            result.append(5)
-            coro.switch()
-            result.append(6)
-            n = 0
-            for i in result:
-                n = n*10 + i
-            return n
-
-        run = self.runner(f)
-        res = run([])
-        assert res == 1234546
-
-    def test_fork(self):
-        class T(AbstractThunk):
-            def __init__(self, result):
-                self.result = result
-            def call(self):
-                localdata = [10]
-                self.result.append(2)
-                newcoro = fork()
-                localdata.append(20)
-                if newcoro is not None:
-                    # in the parent
-                    self.result.append(3)
-                    newcoro.switch()
-                    self.result.append(5)
-                else:
-                    # in the child
-                    self.result.append(4)
-                localdata.append(30)
-                self.result.append(localdata != [10, 20, 30])
-        def f():
-            result = []
-            coro = ClonableCoroutine()
-            coro.bind(T(result))
-            result.append(1)
-            coro.switch()
-            result.append(6)
-            n = 0
-            for i in result:
-                n = n*10 + i
-            return n
-
-        run = self.runner(f)
-        res = run([])
-        assert res == 12340506
diff --git a/pypy/module/_stackless/test/test_pickle.py b/pypy/module/_stackless/test/test_pickle.py
deleted file mode 100644
--- a/pypy/module/_stackless/test/test_pickle.py
+++ /dev/null
@@ -1,487 +0,0 @@
-from pypy.conftest import gettestobjspace, option
-import py
-
-# app-level testing of coroutine pickling
-
-
-class AppTestBasic:
-    def setup_class(cls):
-        cls.space = gettestobjspace(usemodules=('_stackless',))
-
-    def test_pickle_main(self):
-        import _stackless, pickle
-        main = _stackless.coroutine.getcurrent()
-        s = pickle.dumps(main)
-        c = pickle.loads(s)
-        assert c is main
-
-
-class AppTestPickle:
-
-    def setup_class(cls):
-        cls.space = gettestobjspace(usemodules=('_stackless',), CALL_METHOD=True)
-
-    def test_pickle_coroutine_empty(self):
-        # this test is limited to basic pickling.
-        # real stacks can only tested with a stackless pypy build.
-        import _stackless as stackless
-        co = stackless.coroutine()
-        import pickle
-        pckl = pickle.dumps(co)
-        co2 = pickle.loads(pckl)
-        # the empty unpickled coroutine can still be used:
-        result = []
-        co2.bind(result.append, 42)
-        co2.switch()
-        assert result == [42]
-
-    def test_pickle_coroutine_bound(self):
-        import pickle
-        import _stackless
-        lst = [4]
-        co = _stackless.coroutine()
-        co.bind(lst.append, 2)
-        pckl = pickle.dumps((co, lst))
-
-        (co2, lst2) = pickle.loads(pckl)
-        assert lst2 == [4]
-        co2.switch()
-        assert lst2 == [4, 2]
-
-
-    def test_simple_ish(self):
-
-        import new, sys
-
-        mod = new.module('mod')
-        sys.modules['mod'] = mod
-        try:
-            exec '''
-output = []
-import _stackless
-def f(coro, n, x):
-    if n == 0:
-        coro.switch()
-        return
-    f(coro, n-1, 2*x)
-    output.append(x)
-
-def example():
-    main_coro = _stackless.coroutine.getcurrent()
-    sub_coro = _stackless.coroutine()
-    sub_coro.bind(f, main_coro, 5, 1)
-    sub_coro.switch()
-
-    import pickle
-    pckl = pickle.dumps(sub_coro)
-    new_coro = pickle.loads(pckl)
-
-    new_coro.switch()
-
-example()
-assert output == [16, 8, 4, 2, 1]
-''' in mod.__dict__
-        finally:
-            del sys.modules['mod']
-
-    def test_pickle_again(self):
-
-        import new, sys
-
-        mod = new.module('mod')
-        sys.modules['mod'] = mod
-        try:
-            exec '''
-output = []
-import _stackless
-def f(coro, n, x):
-    if n == 0:
-        coro.switch()
-        return
-    f(coro, n-1, 2*x)
-    output.append(x)
-
-def example():
-    main_coro = _stackless.coroutine.getcurrent()
-    sub_coro = _stackless.coroutine()
-    sub_coro.bind(f, main_coro, 5, 1)
-    sub_coro.switch()
-
-    import pickle
-    pckl = pickle.dumps(sub_coro)
-    new_coro = pickle.loads(pckl)
-    pckl = pickle.dumps(new_coro)
-    newer_coro = pickle.loads(pckl)
-
-    newer_coro.switch()
-
-example()
-assert output == [16, 8, 4, 2, 1]
-''' in mod.__dict__
-        finally:
-            del sys.modules['mod']
-
-    def test_kwargs(self):
-
-        import new, sys
-
-        mod = new.module('mod')
-        sys.modules['mod'] = mod
-        try:
-            exec '''
-output = []
-import _stackless
-def f(coro, n, x, step=4):
-    if n == 0:
-        coro.switch()
-        return
-    f(coro, n-1, 2*x, step=1)
-    output.append(x)
-
-def example():
-    main_coro = _stackless.coroutine.getcurrent()
-    sub_coro = _stackless.coroutine()
-    sub_coro.bind(f, main_coro, 5, 1, 1)
-    sub_coro.switch()
-
-    import pickle
-    pckl = pickle.dumps(sub_coro)
-    new_coro = pickle.loads(pckl)
-
-    new_coro.switch()
-
-example()
-assert output == [16, 8, 4, 2, 1]
-''' in mod.__dict__
-        finally:
-            del sys.modules['mod']
-
-    def test_starstarargs(self):
-
-        import new, sys
-
-        mod = new.module('mod')
-        sys.modules['mod'] = mod
-        try:
-            exec '''
-output = []
-import _stackless
-def f(coro, n, x, step=4):
-    if n == 0:
-        coro.switch()
-        return
-    f(coro, n-1, 2*x, **{'step': 1})
-    output.append(x)
-
-def example():
-    main_coro = _stackless.coroutine.getcurrent()
-    sub_coro = _stackless.coroutine()
-    sub_coro.bind(f, main_coro, 5, 1, 1)
-    sub_coro.switch()
-
-    import pickle
-    pckl = pickle.dumps(sub_coro)
-    new_coro = pickle.loads(pckl)
-
-    new_coro.switch()
-
-example()
-assert output == [16, 8, 4, 2, 1]
-''' in mod.__dict__
-        finally:
-            del sys.modules['mod']
-
-    def test_closure(self):
-        import new, sys
-
-        mod = new.module('mod')
-        sys.modules['mod'] = mod
-        try:
-            exec '''
-output = []
-import _stackless
-def example():
-    main_coro = _stackless.coroutine.getcurrent()
-    sub_coro = _stackless.coroutine()
-    y = 3
-    def f(coro, n, x):
-        if n == 0:
-            coro.switch()
-            return
-        f(coro, n-1, 2*x)
-        output.append(x+y)
-
-    sub_coro.bind(f, main_coro, 5, 1)
-    sub_coro.switch()
-
-    import pickle
-    pckl = pickle.dumps(sub_coro)
-    new_coro = pickle.loads(pckl)
-
-    new_coro.switch()
-
-example()
-assert output == [19, 11, 7, 5, 4]
-''' in mod.__dict__
-        finally:
-            del sys.modules['mod']
-
-    def test_exception(self):
-        import new, sys
-
-        mod = new.module('mod')
-        sys.modules['mod'] = mod
-        try:
-            exec '''
-output = []
-import _stackless
-def f(coro):
-    try:
-        raise ValueError
-    except:
-        coro.switch()
-        import sys
-        t, v, tb = sys.exc_info()
-        output.append(t)
-
-def example():
-    main_coro = _stackless.coroutine.getcurrent()
-    sub_coro = _stackless.coroutine()
-
-    sub_coro.bind(f, main_coro)
-    sub_coro.switch()
-
-    import pickle
-    pckl = pickle.dumps(sub_coro)
-    new_coro = pickle.loads(pckl)
-
-    new_coro.switch()
-
-example()
-assert output == [ValueError]
-''' in mod.__dict__
-        finally:
-            del sys.modules['mod']
-
-    def test_exception_after_unpickling(self):
-
-        import new, sys
-
-        mod = new.module('mod')
-        sys.modules['mod'] = mod
-        try:
-            exec '''
-output = []
-import _stackless
-def f(coro, n, x):
-    if n == 0:
-        coro.switch()
-        raise ValueError
-    try:
-        f(coro, n-1, 2*x)
-    finally:
-        output.append(x)
-
-def example():
-    main_coro = _stackless.coroutine.getcurrent()
-    sub_coro = _stackless.coroutine()
-    sub_coro.bind(f, main_coro, 5, 1)
-    sub_coro.switch()
-
-    import pickle
-    pckl = pickle.dumps(sub_coro)
-    new_coro = pickle.loads(pckl)
-
-    try:
-        sub_coro.switch()
-    except ValueError:
-        pass
-    else:
-        assert 0
-    try:
-        new_coro.switch()
-    except ValueError:
-        pass
-    else:
-        assert 0
-
-example()
-assert output == [16, 8, 4, 2, 1] * 2
-''' in mod.__dict__
-        finally:
-            del sys.modules['mod']
-
-    def test_loop(self):
-        import new, sys
-
-        mod = new.module('mod')
-        sys.modules['mod'] = mod
-        try:
-            exec '''
-output = []
-import _stackless
-def f(coro):
-    for x in (1,2,3):
-        coro.switch()
-        output.append(x)
-
-def example():
-    main_coro = _stackless.coroutine.getcurrent()
-    sub_coro = _stackless.coroutine()
-
-    sub_coro.bind(f, main_coro)
-    sub_coro.switch()
-
-    import pickle
-    pckl = pickle.dumps(sub_coro)
-    new_coro = pickle.loads(pckl)
-
-    new_coro.switch()
-    new_coro.switch()
-    new_coro.switch()
-
-example()
-assert output == [1, 2, 3]
-''' in mod.__dict__
-        finally:
-            del sys.modules['mod']
-
-    def test_valstack(self):
-        import new, sys
-
-        mod = new.module('mod')
-        sys.modules['mod'] = mod
-        try:
-            exec '''
-output = []
-import _stackless
-def f(coro):
-    r = 1+g(coro)+3
-    output.append(r)
-
-def g(coro):
-    coro.switch()
-    return 2
-
-def example():
-    main_coro = _stackless.coroutine.getcurrent()
-    sub_coro = _stackless.coroutine()
-
-    sub_coro.bind(f, main_coro)
-    sub_coro.switch()
-
-    import pickle
-    pckl = pickle.dumps(sub_coro)
-    new_coro = pickle.loads(pckl)
-
-    new_coro.switch()
-
-
-example()
-assert output == [6]
-''' in mod.__dict__
-        finally:
-            del sys.modules['mod']
-
-
-    def test_exec_and_locals(self):
-        import new, sys
-
-        mod = new.module('mod')
-        sys.modules['mod'] = mod
-        try:
-            exec '''
-output = []
-import _stackless
-
-def f(coro):
-    x = None
-    exec "x = 9"
-    coro.switch()
-    output.append(x)
-
-def example():
-    main_coro = _stackless.coroutine.getcurrent()
-    sub_coro = _stackless.coroutine()
-    sub_coro.bind(f, main_coro)
-    sub_coro.switch()
-
-    import pickle
-    pckl = pickle.dumps(sub_coro)
-    new_coro = pickle.loads(pckl)
-
-    new_coro.switch()
-
-example()
-assert output == [9]
-''' in mod.__dict__
-        finally:
-            del sys.modules['mod']
-
-
-    def test_solver(self):
-        import new, sys
-
-        mod = new.module('mod')
-        sys.modules['mod'] = mod
-        try:
-            exec '''
-import _stackless, pickle
-
-class Fail(Exception):
-    pass
-
-class Success(Exception):
-    pass
-
-def first_solution(func):
-    global next_answer
-    co = _stackless.coroutine()
-    co.bind(func)
-    pending = [(co, None)]
-    while pending:
-        co, next_answer = pending.pop()
-        try:
-            co.switch()
-        except Fail:
-            pass
-        except Success, e:
-            return e.args[0]
-        else:
-            # zero_or_one() called, clone the coroutine
-            # NB. this seems to be quite slow
-            co2 = pickle.loads(pickle.dumps(co))
-            pending.append((co2, 1))
-            pending.append((co, 0))
-    raise Fail("no solution")
-
-pending = []
-main = _stackless.coroutine.getcurrent()
-
-def zero_or_one():
-    main.switch()
-    return next_answer
-
-# ____________________________________________________________
-
-invalid_prefixes = {
-    (0, 0): True,
-    (0, 1, 0): True,
-    (0, 1, 1): True,
-    (1, 0): True,
-    (1, 1, 0, 0): True,
-    }
-
-def example():
-    test = []
-    for n in range(5):
-        test.append(zero_or_one())
-        if tuple(test) in invalid_prefixes:
-            raise Fail
-    raise Success(test)
-
-res = first_solution(example)
-assert res == [1, 1, 0, 1, 0]
-''' in mod.__dict__
-        finally:
-            del sys.modules['mod']
diff --git a/pypy/module/_weakref/interp__weakref.py b/pypy/module/_weakref/interp__weakref.py
--- a/pypy/module/_weakref/interp__weakref.py
+++ b/pypy/module/_weakref/interp__weakref.py
@@ -8,24 +8,12 @@
 
 
 class WeakrefLifeline(W_Root):
+    cached_weakref_index = -1
+    cached_proxy_index = -1
+
     def __init__(self, space):
         self.space = space
         self.refs_weak = []
-        self.cached_weakref_index = -1
-        self.cached_proxy_index = -1
-
-    def __del__(self):
-        """This runs when the interp-level object goes away, and allows
-        its lifeline to go away.  The purpose of this is to activate the
-        callbacks even if there is no __del__ method on the interp-level
-        W_Root subclass implementing the object.
-        """
-        for i in range(len(self.refs_weak) - 1, -1, -1):
-            w_ref = self.refs_weak[i]()
-            if w_ref is not None and w_ref.w_callable is not None:
-                w_ref.enqueue_for_destruction(self.space,
-                                              W_WeakrefBase.activate_callback,
-                                              'weakref callback of ')
 
     def clear_all_weakrefs(self):
         """Clear all weakrefs.  This is called when an app-level object has
@@ -39,12 +27,11 @@
         # weakref callbacks are not invoked eagerly here.  They are
         # invoked by self.__del__() anyway.
 
-    @jit.dont_look_inside
-    def get_or_make_weakref(self, space, w_subtype, w_obj, w_callable):
+    def get_or_make_weakref(self, w_subtype, w_obj):
+        space = self.space
         w_weakreftype = space.gettypeobject(W_Weakref.typedef)
         is_weakreftype = space.is_w(w_weakreftype, w_subtype)
-        can_reuse = space.is_w(w_callable, space.w_None)
-        if is_weakreftype and can_reuse and self.cached_weakref_index >= 0:
+        if is_weakreftype and self.cached_weakref_index >= 0:
             w_cached = self.refs_weak[self.cached_weakref_index]()
             if w_cached is not None:
                 return w_cached
@@ -52,16 +39,15 @@
                 self.cached_weakref_index = -1
         w_ref = space.allocate_instance(W_Weakref, w_subtype)
         index = len(self.refs_weak)
-        W_Weakref.__init__(w_ref, space, w_obj, w_callable)
+        W_Weakref.__init__(w_ref, space, w_obj, None)
         self.refs_weak.append(weakref.ref(w_ref))
-        if is_weakreftype and can_reuse:
+        if is_weakreftype:
             self.cached_weakref_index = index
         return w_ref
 
-    @jit.dont_look_inside
-    def get_or_make_proxy(self, space, w_obj, w_callable):
-        can_reuse = space.is_w(w_callable, space.w_None)
-        if can_reuse and self.cached_proxy_index >= 0:
+    def get_or_make_proxy(self, w_obj):
+        space = self.space
+        if self.cached_proxy_index >= 0:
             w_cached = self.refs_weak[self.cached_proxy_index]()
             if w_cached is not None:
                 return w_cached
@@ -69,12 +55,11 @@
                 self.cached_proxy_index = -1
         index = len(self.refs_weak)
         if space.is_true(space.callable(w_obj)):
-            w_proxy = W_CallableProxy(space, w_obj, w_callable)
+            w_proxy = W_CallableProxy(space, w_obj, None)
         else:
-            w_proxy = W_Proxy(space, w_obj, w_callable)
+            w_proxy = W_Proxy(space, w_obj, None)
         self.refs_weak.append(weakref.ref(w_proxy))
-        if can_reuse:
-            self.cached_proxy_index = index
+        self.cached_proxy_index = index
         return w_proxy
 
     def get_any_weakref(self, space):
@@ -90,6 +75,45 @@
                 return w_ref
         return space.w_None
 
+
+class WeakrefLifelineWithCallbacks(WeakrefLifeline):
+
+    def __init__(self, space, oldlifeline=None):
+        self.space = space
+        if oldlifeline is None:
+            self.refs_weak = []
+        else:
+            self.refs_weak = oldlifeline.refs_weak
+
+    def __del__(self):
+        """This runs when the interp-level object goes away, and allows
+        its lifeline to go away.  The purpose of this is to activate the
+        callbacks even if there is no __del__ method on the interp-level
+        W_Root subclass implementing the object.
+        """
+        for i in range(len(self.refs_weak) - 1, -1, -1):
+            w_ref = self.refs_weak[i]()
+            if w_ref is not None and w_ref.w_callable is not None:
+                w_ref.enqueue_for_destruction(self.space,
+                                              W_WeakrefBase.activate_callback,
+                                              'weakref callback of ')
+
+    def make_weakref_with_callback(self, w_subtype, w_obj, w_callable):
+        space = self.space
+        w_ref = space.allocate_instance(W_Weakref, w_subtype)
+        W_Weakref.__init__(w_ref, space, w_obj, w_callable)
+        self.refs_weak.append(weakref.ref(w_ref))
+        return w_ref
+
+    def make_proxy_with_callback(self, w_obj, w_callable):
+        space = self.space
+        if space.is_true(space.callable(w_obj)):
+            w_proxy = W_CallableProxy(space, w_obj, w_callable)
+        else:
+            w_proxy = W_Proxy(space, w_obj, w_callable)
+        self.refs_weak.append(weakref.ref(w_proxy))
+        return w_proxy
+
 # ____________________________________________________________
 
 class Dummy:
@@ -103,8 +127,7 @@
 
 class W_WeakrefBase(Wrappable):
     def __init__(w_self, space, w_obj, w_callable):
-        if space.is_w(w_callable, space.w_None):
-            w_callable = None
+        assert w_callable is not space.w_None    # should be really None
         w_self.space = space
         assert w_obj is not None
         w_self.w_obj_weak = weakref.ref(w_obj)
@@ -177,16 +200,39 @@
     def descr__ne__(self, space, w_ref2):
         return space.not_(space.eq(self, w_ref2))
 
+def getlifeline(space, w_obj):
+    lifeline = w_obj.getweakref()
+    if lifeline is None:
+        lifeline = WeakrefLifeline(space)
+        w_obj.setweakref(space, lifeline)
+    return lifeline
+
+def getlifelinewithcallbacks(space, w_obj):
+    lifeline = w_obj.getweakref()
+    if not isinstance(lifeline, WeakrefLifelineWithCallbacks):  # or None
+        oldlifeline = lifeline
+        lifeline = WeakrefLifelineWithCallbacks(space, oldlifeline)
+        w_obj.setweakref(space, lifeline)
+    return lifeline
+
+ at jit.dont_look_inside
+def get_or_make_weakref(space, w_subtype, w_obj):
+    return getlifeline(space, w_obj).get_or_make_weakref(w_subtype, w_obj)
+
+ at jit.dont_look_inside
+def make_weakref_with_callback(space, w_subtype, w_obj, w_callable):
+    lifeline = getlifelinewithcallbacks(space, w_obj)
+    return lifeline.make_weakref_with_callback(w_subtype, w_obj, w_callable)
+
 def descr__new__weakref(space, w_subtype, w_obj, w_callable=None,
                         __args__=None):
     if __args__.arguments_w:
         raise OperationError(space.w_TypeError, space.wrap(
             "__new__ expected at most 2 arguments"))
-    lifeline = w_obj.getweakref()
-    if lifeline is None:
-        lifeline = WeakrefLifeline(space)
-        w_obj.setweakref(space, lifeline)
-    return lifeline.get_or_make_weakref(space, w_subtype, w_obj, w_callable)
+    if space.is_w(w_callable, space.w_None):
+        return get_or_make_weakref(space, w_subtype, w_obj)
+    else:
+        return make_weakref_with_callback(space, w_subtype, w_obj, w_callable)
 
 W_Weakref.typedef = TypeDef("weakref",
     __doc__ = """A weak reference to an object 'obj'.  A 'callback' can be given,
@@ -239,15 +285,23 @@
         w_obj = force(space, self)
         return space.call_args(w_obj, __args__)
 
+ at jit.dont_look_inside
+def get_or_make_proxy(space, w_obj):
+    return getlifeline(space, w_obj).get_or_make_proxy(w_obj)
+
+ at jit.dont_look_inside
+def make_proxy_with_callback(space, w_obj, w_callable):
+    lifeline = getlifelinewithcallbacks(space, w_obj)
+    return lifeline.make_proxy_with_callback(w_obj, w_callable)
+
 def proxy(space, w_obj, w_callable=None):
     """Create a proxy object that weakly references 'obj'.
 'callback', if given, is called with the proxy as an argument when 'obj'
 is about to be finalized."""
-    lifeline = w_obj.getweakref()
-    if lifeline is None:
-        lifeline = WeakrefLifeline(space)
-        w_obj.setweakref(space, lifeline)
-    return lifeline.get_or_make_proxy(space, w_obj, w_callable)
+    if space.is_w(w_callable, space.w_None):
+        return get_or_make_proxy(space, w_obj)
+    else:
+        return make_proxy_with_callback(space, w_obj, w_callable)
 
 def descr__new__proxy(space, w_subtype, w_obj, w_callable=None):
     raise OperationError(
diff --git a/pypy/module/_weakref/test/test_weakref.py b/pypy/module/_weakref/test/test_weakref.py
--- a/pypy/module/_weakref/test/test_weakref.py
+++ b/pypy/module/_weakref/test/test_weakref.py
@@ -369,6 +369,26 @@
             return A
         raises(TypeError, tryit)
 
+    def test_proxy_to_dead_object(self):
+        import _weakref, gc
+        class A(object):
+            pass
+        p = _weakref.proxy(A())
+        gc.collect()
+        raises(ReferenceError, "p + 1")
+
+    def test_proxy_with_callback(self):
+        import _weakref, gc
+        class A(object):
+            pass
+        a2 = A()
+        def callback(proxy):
+            a2.seen = proxy
+        p = _weakref.proxy(A(), callback)
+        gc.collect()
+        raises(ReferenceError, "p + 1")
+        assert a2.seen is p
+
     def test_repr(self):
         import _weakref, gc
         for kind in ('ref', 'proxy'):
diff --git a/pypy/module/bz2/interp_bz2.py b/pypy/module/bz2/interp_bz2.py
--- a/pypy/module/bz2/interp_bz2.py
+++ b/pypy/module/bz2/interp_bz2.py
@@ -446,7 +446,9 @@
             result = self.buffer[pos:pos + n]
             self.pos += n
         else:
-            result = self.buffer
+            pos = self.pos
+            assert pos >= 0
+            result = self.buffer[pos:]
             self.pos = 0
             self.buffer = ""
         self.readlength += len(result)
diff --git a/pypy/module/bz2/test/test_bz2_file.py b/pypy/module/bz2/test/test_bz2_file.py
--- a/pypy/module/bz2/test/test_bz2_file.py
+++ b/pypy/module/bz2/test/test_bz2_file.py
@@ -274,14 +274,14 @@
             pass
         del bz2f   # delete from this frame, which is captured in the traceback
 
-    def test_read_chunk10(self):
+    def test_read_chunk9(self):
         from bz2 import BZ2File
         self.create_temp_file()
         
         bz2f = BZ2File(self.temppath)
         text_read = ""
         while True:
-            data = bz2f.read(10)
+            data = bz2f.read(9) # 9 doesn't divide evenly into data length
             if not data:
                 break
             text_read = "%s%s" % (text_read, data)
diff --git a/pypy/module/cpyext/frameobject.py b/pypy/module/cpyext/frameobject.py
--- a/pypy/module/cpyext/frameobject.py
+++ b/pypy/module/cpyext/frameobject.py
@@ -57,7 +57,7 @@
     code = space.interp_w(PyCode, w_code)
     w_globals = from_ref(space, py_frame.c_f_globals)
 
-    frame = space.FrameClass(space, code, w_globals, closure=None)
+    frame = space.FrameClass(space, code, w_globals, outer_func=None)
     frame.f_lineno = py_frame.c_f_lineno
     w_obj = space.wrap(frame)
     track_reference(space, py_obj, w_obj)
diff --git a/pypy/module/marshal/interp_marshal.py b/pypy/module/marshal/interp_marshal.py
--- a/pypy/module/marshal/interp_marshal.py
+++ b/pypy/module/marshal/interp_marshal.py
@@ -40,7 +40,7 @@
         reader = FileReader(space, w_f)
     try:
         u = Unmarshaller(space, reader)
-        return u.load_w_obj(False)
+        return u.load_w_obj()
     finally:
         reader.finished()
 
@@ -49,7 +49,7 @@
 ignored."""
     space.timer.start("marshal loads")
     u = StringUnmarshaller(space, w_str)
-    obj = u.load_w_obj(False)
+    obj = u.load_w_obj()
     space.timer.stop("marshal loads")
     return obj
 
@@ -424,7 +424,7 @@
         lng = self.get_lng()
         return self.get(lng)
 
-    def get_w_obj(self, allow_null):
+    def get_w_obj(self, allow_null=False):
         space = self.space
         w_ret = space.w_None # something not None
         tc = self.get1()
@@ -434,9 +434,9 @@
                 'NULL object in marshal data'))
         return w_ret
 
-    def load_w_obj(self, allow_null):
+    def load_w_obj(self):
         try:
-            return self.get_w_obj(allow_null)
+            return self.get_w_obj()
         except rstackovf.StackOverflow:
             rstackovf.check_stack_overflow()
             self._overflow()
diff --git a/pypy/module/micronumpy/__init__.py b/pypy/module/micronumpy/__init__.py
--- a/pypy/module/micronumpy/__init__.py
+++ b/pypy/module/micronumpy/__init__.py
@@ -1,42 +1,51 @@
-
 from pypy.interpreter.mixedmodule import MixedModule
 
+
 class Module(MixedModule):
-
     applevel_name = 'numpy'
 
     interpleveldefs = {
         'array': 'interp_numarray.SingleDimArray',
         'dtype': 'interp_dtype.W_Dtype',
+        'ufunc': 'interp_ufuncs.W_Ufunc',
 
         'zeros': 'interp_numarray.zeros',
         'empty': 'interp_numarray.zeros',
         'ones': 'interp_numarray.ones',
         'fromstring': 'interp_support.fromstring',
+    }
 
-        # ufuncs
-        'abs': 'interp_ufuncs.absolute',
-        'absolute': 'interp_ufuncs.absolute',
-        'add': 'interp_ufuncs.add',
-        'copysign': 'interp_ufuncs.copysign',
-        'divide': 'interp_ufuncs.divide',
-        'exp': 'interp_ufuncs.exp',
-        'fabs': 'interp_ufuncs.fabs',
-        'floor': 'interp_ufuncs.floor',
-        'maximum': 'interp_ufuncs.maximum',
-        'minimum': 'interp_ufuncs.minimum',
-        'multiply': 'interp_ufuncs.multiply',
-        'negative': 'interp_ufuncs.negative',
-        'reciprocal': 'interp_ufuncs.reciprocal',
-        'sign': 'interp_ufuncs.sign',
-        'subtract': 'interp_ufuncs.subtract',
-        'sin': 'interp_ufuncs.sin',
-        'cos': 'interp_ufuncs.cos',
-        'tan': 'interp_ufuncs.tan',
-        'arcsin': 'interp_ufuncs.arcsin',
-        'arccos': 'interp_ufuncs.arccos',
-        'arctan': 'interp_ufuncs.arctan',
-    }
+    # ufuncs
+    for exposed, impl in [
+        ("abs", "absolute"),
+        ("absolute", "absolute"),
+        ("add", "add"),
+        ("arccos", "arccos"),
+        ("arcsin", "arcsin"),
+        ("arctan", "arctan"),
+        ("copysign", "copysign"),
+        ("cos", "cos"),
+        ("divide", "divide"),
+        ("equal", "equal"),
+        ("exp", "exp"),
+        ("fabs", "fabs"),
+        ("floor", "floor"),
+        ("greater", "greater"),
+        ("greater_equal", "greater_equal"),
+        ("less", "less"),
+        ("less_equal", "less_equal"),
+        ("maximum", "maximum"),
+        ("minimum", "minimum"),
+        ("multiply", "multiply"),
+        ("negative", "negative"),
+        ("not_equal", "not_equal"),
+        ("reciprocal", "reciprocal"),
+        ("sign", "sign"),
+        ("sin", "sin"),
+        ("subtract", "subtract"),
+        ("tan", "tan"),
+    ]:
+        interpleveldefs[exposed] = "interp_ufuncs.get(space).%s" % impl
 
     appleveldefs = {
         'average': 'app_numpy.average',
diff --git a/pypy/module/micronumpy/compile.py b/pypy/module/micronumpy/compile.py
--- a/pypy/module/micronumpy/compile.py
+++ b/pypy/module/micronumpy/compile.py
@@ -20,6 +20,7 @@
 
 class FakeSpace(object):
     w_ValueError = None
+    w_TypeError = None
 
     def __init__(self):
         """NOT_RPYTHON"""
diff --git a/pypy/module/micronumpy/interp_dtype.py b/pypy/module/micronumpy/interp_dtype.py
--- a/pypy/module/micronumpy/interp_dtype.py
+++ b/pypy/module/micronumpy/interp_dtype.py
@@ -53,7 +53,9 @@
 
 VOID_TP = lltype.Ptr(lltype.Array(lltype.Void, hints={'nolength': True, "uncast_on_llgraph": True}))
 
-def create_low_level_dtype(num, kind, name, aliases, applevel_types, T, valtype):
+def create_low_level_dtype(num, kind, name, aliases, applevel_types, T, valtype,
+    expected_size=None):
+
     class Box(BaseBox):
         def __init__(self, val):
             self.val = val
@@ -113,6 +115,8 @@
     W_LowLevelDtype.aliases = aliases
     W_LowLevelDtype.applevel_types = applevel_types
     W_LowLevelDtype.num_bytes = rffi.sizeof(T)
+    if expected_size is not None:
+        assert W_LowLevelDtype.num_bytes == expected_size
     return W_LowLevelDtype
 
 
@@ -125,6 +129,16 @@
         ))
     return impl
 
+def raw_binop(func):
+    # Returns the result unwrapped.
+    @functools.wraps(func)
+    def impl(self, v1, v2):
+        return func(self,
+            self.for_computation(self.unbox(v1)),
+            self.for_computation(self.unbox(v2))
+        )
+    return impl
+
 def unaryop(func):
     @functools.wraps(func)
     def impl(self, v):
@@ -166,8 +180,24 @@
 
     def bool(self, v):
         return bool(self.for_computation(self.unbox(v)))
+    @raw_binop
+    def eq(self, v1, v2):
+        return v1 == v2
+    @raw_binop
     def ne(self, v1, v2):
-        return self.for_computation(self.unbox(v1)) != self.for_computation(self.unbox(v2))
+        return v1 != v2
+    @raw_binop
+    def lt(self, v1, v2):
+        return v1 < v2
+    @raw_binop
+    def le(self, v1, v2):
+        return v1 <= v2
+    @raw_binop
+    def gt(self, v1, v2):
+        return v1 > v2
+    @raw_binop
+    def ge(self, v1, v2):
+        return v1 >= v2
 
 
 class FloatArithmeticDtype(ArithmaticTypeMixin):
@@ -220,7 +250,7 @@
         return math.tan(v)
     @unaryop
     def arcsin(self, v):
-        if v < -1.0 or  v > 1.0:
+        if v < -1.0 or v > 1.0:
             return rfloat.NAN
         return math.asin(v)
     @unaryop
@@ -282,10 +312,21 @@
     applevel_types = [],
     T = rffi.SIGNEDCHAR,
     valtype = rffi.SIGNEDCHAR._type,
+    expected_size = 1,
 )
 class W_Int8Dtype(IntegerArithmeticDtype, W_Int8Dtype):
-    def unwrap(self, space, w_item):
-        return self.adapt_val(space.int_w(space.int(w_item)))
+    pass
+
+W_Int16Dtype = create_low_level_dtype(
+    num = 3, kind = SIGNEDLTR, name = "int16",
+    aliases = ["int16"],
+    applevel_types = [],
+    T = rffi.SHORT,
+    valtype = rffi.SHORT._type,
+    expected_size = 2,
+)
+class W_Int16Dtype(IntegerArithmeticDtype, W_Int16Dtype):
+    pass
 
 W_Int32Dtype = create_low_level_dtype(
     num = 5, kind = SIGNEDLTR, name = "int32",
@@ -293,6 +334,7 @@
     applevel_types = [],
     T = rffi.INT,
     valtype = rffi.INT._type,
+    expected_size = 4,
 )
 class W_Int32Dtype(IntegerArithmeticDtype, W_Int32Dtype):
     pass
@@ -303,6 +345,7 @@
     applevel_types = ["long"],
     T = rffi.LONGLONG,
     valtype = rffi.LONGLONG._type,
+    expected_size = 8,
 )
 class W_Int64Dtype(IntegerArithmeticDtype, W_Int64Dtype):
     pass
@@ -313,6 +356,7 @@
     applevel_types = ["float"],
     T = lltype.Float,
     valtype = float,
+    expected_size = 8,
 )
 class W_Float64Dtype(FloatArithmeticDtype, W_Float64Dtype):
     def unwrap(self, space, w_item):
@@ -323,7 +367,7 @@
 
 ALL_DTYPES = [
     W_BoolDtype,
-    W_Int8Dtype, W_Int32Dtype, W_Int64Dtype,
+    W_Int8Dtype, W_Int16Dtype, W_Int32Dtype, W_Int64Dtype,
     W_Float64Dtype
 ]
 
@@ -353,4 +397,4 @@
     kind = interp_attrproperty("kind", cls=W_Dtype),
     shape = GetSetProperty(W_Dtype.descr_get_shape),
 )
-W_Dtype.typedef.acceptable_as_base_class = False
\ No newline at end of file
+W_Dtype.typedef.acceptable_as_base_class = False
diff --git a/pypy/module/micronumpy/interp_numarray.py b/pypy/module/micronumpy/interp_numarray.py
--- a/pypy/module/micronumpy/interp_numarray.py
+++ b/pypy/module/micronumpy/interp_numarray.py
@@ -53,90 +53,59 @@
             i += 1
         return arr
 
-    def _unaryop_impl(w_ufunc):
+    def _unaryop_impl(ufunc_name):
         def impl(self, space):
-            return w_ufunc(space, self)
-        return func_with_new_name(impl, "unaryop_%s_impl" % w_ufunc.__name__)
+            return getattr(interp_ufuncs.get(space), ufunc_name).call(space, [self])
+        return func_with_new_name(impl, "unaryop_%s_impl" % ufunc_name)
 
-    descr_pos = _unaryop_impl(interp_ufuncs.positive)
-    descr_neg = _unaryop_impl(interp_ufuncs.negative)
-    descr_abs = _unaryop_impl(interp_ufuncs.absolute)
+    descr_pos = _unaryop_impl("positive")
+    descr_neg = _unaryop_impl("negative")
+    descr_abs = _unaryop_impl("absolute")
 
-    def _binop_impl(w_ufunc):
+    def _binop_impl(ufunc_name):
         def impl(self, space, w_other):
-            return w_ufunc(space, self, w_other)
-        return func_with_new_name(impl, "binop_%s_impl" % w_ufunc.__name__)
+            return getattr(interp_ufuncs.get(space), ufunc_name).call(space, [self, w_other])
+        return func_with_new_name(impl, "binop_%s_impl" % ufunc_name)
 
-    descr_add = _binop_impl(interp_ufuncs.add)
-    descr_sub = _binop_impl(interp_ufuncs.subtract)
-    descr_mul = _binop_impl(interp_ufuncs.multiply)
-    descr_div = _binop_impl(interp_ufuncs.divide)
-    descr_pow = _binop_impl(interp_ufuncs.power)
-    descr_mod = _binop_impl(interp_ufuncs.mod)
+    descr_add = _binop_impl("add")
+    descr_sub = _binop_impl("subtract")
+    descr_mul = _binop_impl("multiply")
+    descr_div = _binop_impl("divide")
+    descr_pow = _binop_impl("power")
+    descr_mod = _binop_impl("mod")
 
-    def _binop_right_impl(w_ufunc):
+    descr_eq = _binop_impl("equal")
+    descr_ne = _binop_impl("not_equal")
+    descr_lt = _binop_impl("less")
+    descr_le = _binop_impl("less_equal")
+    descr_gt = _binop_impl("greater")
+    descr_ge = _binop_impl("greater_equal")
+
+    def _binop_right_impl(ufunc_name):
         def impl(self, space, w_other):
             w_other = scalar_w(space,
                 interp_ufuncs.find_dtype_for_scalar(space, w_other, self.find_dtype()),
                 w_other
             )
-            return w_ufunc(space, w_other, self)
-        return func_with_new_name(impl, "binop_right_%s_impl" % w_ufunc.__name__)
+            return getattr(interp_ufuncs.get(space), ufunc_name).call(space, [w_other, self])
+        return func_with_new_name(impl, "binop_right_%s_impl" % ufunc_name)
 
-    descr_radd = _binop_right_impl(interp_ufuncs.add)
-    descr_rsub = _binop_right_impl(interp_ufuncs.subtract)
-    descr_rmul = _binop_right_impl(interp_ufuncs.multiply)
-    descr_rdiv = _binop_right_impl(interp_ufuncs.divide)
-    descr_rpow = _binop_right_impl(interp_ufuncs.power)
-    descr_rmod = _binop_right_impl(interp_ufuncs.mod)
+    descr_radd = _binop_right_impl("add")
+    descr_rsub = _binop_right_impl("subtract")
+    descr_rmul = _binop_right_impl("multiply")
+    descr_rdiv = _binop_right_impl("divide")
+    descr_rpow = _binop_right_impl("power")
+    descr_rmod = _binop_right_impl("mod")
 
-    def _reduce_sum_prod_impl(op_name, init):
-        reduce_driver = jit.JitDriver(greens=['signature'],
-                         reds = ['i', 'size', 'self', 'result', 'res_dtype'])
+    def _reduce_ufunc_impl(ufunc_name):
+        def impl(self, space):
+            return getattr(interp_ufuncs.get(space), ufunc_name).descr_reduce(space, self)
+        return func_with_new_name(impl, "reduce_%s_impl" % ufunc_name)
 
-        def loop(self, res_dtype, result, size):
-            i = 0
-            while i < size:
-                reduce_driver.jit_merge_point(signature=self.signature,
-                                              self=self, res_dtype=res_dtype,
-                                              size=size, i=i, result=result)
-                result = getattr(res_dtype, op_name)(
-                    result,
-                    self.eval(i).convert_to(res_dtype)
-                )
-                i += 1
-            return result
-
-        def impl(self, space):
-            dtype = interp_ufuncs.find_unaryop_result_dtype(
-                space, self.find_dtype(), promote_to_largest=True
-            )
-            result = dtype.adapt_val(init)
-            return loop(self, dtype, result, self.find_size()).wrap(space)
-        return func_with_new_name(impl, "reduce_%s_impl" % op_name)
-
-    def _reduce_max_min_impl(op_name):
-        reduce_driver = jit.JitDriver(greens=['signature'],
-                         reds = ['i', 'size', 'self', 'result', 'dtype'])
-        def loop(self, result, size):
-            i = 1
-            dtype = self.find_dtype()
-            while i < size:
-                reduce_driver.jit_merge_point(signature=self.signature,
-                                              self=self, dtype=dtype,
-                                              size=size, i=i, result=result)
-                result = getattr(dtype, op_name)(result, self.eval(i))
-                i += 1
-            return result
-
-        def impl(self, space):
-            size = self.find_size()
-            if size == 0:
-                raise OperationError(space.w_ValueError,
-                    space.wrap("Can't call %s on zero-size arrays" \
-                            % op_name))
-            return loop(self, self.eval(0), size).wrap(space)
-        return func_with_new_name(impl, "reduce_%s_impl" % op_name)
+    descr_sum = _reduce_ufunc_impl("add")
+    descr_prod = _reduce_ufunc_impl("multiply")
+    descr_max = _reduce_ufunc_impl("maximum")
+    descr_min = _reduce_ufunc_impl("minimum")
 
     def _reduce_argmax_argmin_impl(op_name):
         reduce_driver = jit.JitDriver(greens=['signature'],
@@ -192,10 +161,6 @@
     def descr_any(self, space):
         return space.wrap(self._any())
 
-    descr_sum = _reduce_sum_prod_impl("add", 0)
-    descr_prod = _reduce_sum_prod_impl("mul", 1)
-    descr_max = _reduce_max_min_impl("max")
-    descr_min = _reduce_max_min_impl("min")
     descr_argmax = _reduce_argmax_argmin_impl("max")
     descr_argmin = _reduce_argmax_argmin_impl("min")
 
@@ -248,7 +213,7 @@
         res = "array([" + ", ".join(concrete._getnums(False)) + "]"
         dtype = concrete.find_dtype()
         if (dtype is not space.fromcache(interp_dtype.W_Float64Dtype) and
-            dtype is not space.fromcache(interp_dtype.W_Int64Dtype)):
+            dtype is not space.fromcache(interp_dtype.W_Int64Dtype)) or not self.find_size():
             res += ", dtype=" + dtype.name
         res += ")"
         return space.wrap(res)
@@ -259,7 +224,15 @@
         return space.wrap("[" + " ".join(concrete._getnums(True)) + "]")
 
     def descr_getitem(self, space, w_idx):
-        # TODO: indexing by tuples
+        # TODO: indexing by arrays and lists
+        if space.isinstance_w(w_idx, space.w_tuple):
+            length = space.len_w(w_idx)
+            if length == 0:
+                return space.wrap(self)
+            if length > 1: # only one dimension for now.
+                raise OperationError(space.w_IndexError,
+                                     space.wrap("invalid index"))
+            w_idx = space.getitem(w_idx, space.wrap(0))
         start, stop, step, slice_length = space.decode_index4(w_idx, self.find_size())
         if step == 0:
             # Single index
@@ -273,8 +246,19 @@
             return space.wrap(res)
 
     def descr_setitem(self, space, w_idx, w_value):
-        # TODO: indexing by tuples and lists
+        # TODO: indexing by arrays and lists
         self.invalidated()
+        if space.isinstance_w(w_idx, space.w_tuple):
+            length = space.len_w(w_idx)
+            if length > 1: # only one dimension for now.
+                raise OperationError(space.w_IndexError,
+                                     space.wrap("invalid index"))
+            if length == 0:
+                w_idx = space.newslice(space.wrap(0),
+                                      space.wrap(self.find_size()),
+                                      space.wrap(1))
+            else:
+                w_idx = space.getitem(w_idx, space.wrap(0))
         start, stop, step, slice_length = space.decode_index4(w_idx,
                                                               self.find_size())
         if step == 0:
@@ -427,10 +411,11 @@
     """
     Intermediate class for performing binary operations.
     """
-    def __init__(self, signature, res_dtype, left, right):
+    def __init__(self, signature, calc_dtype, res_dtype, left, right):
         VirtualArray.__init__(self, signature, res_dtype)
         self.left = left
         self.right = right
+        self.calc_dtype = calc_dtype
 
     def _del_sources(self):
         self.left = None
@@ -444,14 +429,14 @@
         return self.right.find_size()
 
     def _eval(self, i):
-        lhs = self.left.eval(i).convert_to(self.res_dtype)
-        rhs = self.right.eval(i).convert_to(self.res_dtype)
+        lhs = self.left.eval(i).convert_to(self.calc_dtype)
+        rhs = self.right.eval(i).convert_to(self.calc_dtype)
 
         sig = jit.promote(self.signature)
         assert isinstance(sig, signature.Signature)
         call_sig = sig.components[0]
         assert isinstance(call_sig, signature.Call2)
-        return call_sig.func(self.res_dtype, lhs, rhs)
+        return call_sig.func(self.calc_dtype, lhs, rhs)
 
 class ViewArray(BaseArray):
     """
@@ -596,18 +581,28 @@
     __pos__ = interp2app(BaseArray.descr_pos),
     __neg__ = interp2app(BaseArray.descr_neg),
     __abs__ = interp2app(BaseArray.descr_abs),
+
     __add__ = interp2app(BaseArray.descr_add),
     __sub__ = interp2app(BaseArray.descr_sub),
     __mul__ = interp2app(BaseArray.descr_mul),
     __div__ = interp2app(BaseArray.descr_div),
     __pow__ = interp2app(BaseArray.descr_pow),
     __mod__ = interp2app(BaseArray.descr_mod),
+
     __radd__ = interp2app(BaseArray.descr_radd),
     __rsub__ = interp2app(BaseArray.descr_rsub),
     __rmul__ = interp2app(BaseArray.descr_rmul),
     __rdiv__ = interp2app(BaseArray.descr_rdiv),
     __rpow__ = interp2app(BaseArray.descr_rpow),
     __rmod__ = interp2app(BaseArray.descr_rmod),
+
+    __eq__ = interp2app(BaseArray.descr_eq),
+    __ne__ = interp2app(BaseArray.descr_ne),
+    __lt__ = interp2app(BaseArray.descr_lt),
+    __le__ = interp2app(BaseArray.descr_le),
+    __gt__ = interp2app(BaseArray.descr_gt),
+    __ge__ = interp2app(BaseArray.descr_ge),
+
     __repr__ = interp2app(BaseArray.descr_repr),
     __str__ = interp2app(BaseArray.descr_str),
 
diff --git a/pypy/module/micronumpy/interp_ufuncs.py b/pypy/module/micronumpy/interp_ufuncs.py
--- a/pypy/module/micronumpy/interp_ufuncs.py
+++ b/pypy/module/micronumpy/interp_ufuncs.py
@@ -1,57 +1,168 @@
+from pypy.interpreter.baseobjspace import Wrappable
+from pypy.interpreter.error import OperationError, operationerrfmt
+from pypy.interpreter.gateway import interp2app
+from pypy.interpreter.typedef import TypeDef, GetSetProperty, interp_attrproperty
 from pypy.module.micronumpy import interp_dtype, signature
+from pypy.rlib import jit
 from pypy.tool.sourcetools import func_with_new_name
 
 
-def ufunc(func=None, promote_to_float=False, promote_bools=False):
-    if func is None:
-        return lambda func: ufunc(func, promote_to_float, promote_bools)
-    call_sig = signature.Call1(func)
-    def impl(space, w_obj):
+reduce_driver = jit.JitDriver(
+    greens = ["signature"],
+    reds = ["i", "size", "self", "dtype", "value", "obj"]
+)
+
+class W_Ufunc(Wrappable):
+    _attrs_ = ["name", "promote_to_float", "promote_bools", "identity"]
+
+    def __init__(self, name, promote_to_float, promote_bools, identity):
+        self.name = name
+        self.promote_to_float = promote_to_float
+        self.promote_bools = promote_bools
+
+        self.identity = identity
+
+    def descr_repr(self, space):
+        return space.wrap("" % self.name)
+
+    def descr_get_identity(self, space):
+        if self.identity is None:
+            return space.w_None
+        return self.identity.wrap(space)
+
+    def descr_call(self, space, __args__):
+        try:
+            args_w = __args__.fixedunpack(self.argcount)
+        except ValueError, e:
+            raise OperationError(space.w_TypeError, space.wrap(str(e)))
+        return self.call(space, args_w)
+
+    def descr_reduce(self, space, w_obj):
+        from pypy.module.micronumpy.interp_numarray import convert_to_array, Scalar
+
+        if self.argcount != 2:
+            raise OperationError(space.w_ValueError, space.wrap("reduce only "
+                "supported for binary functions"))
+
+        assert isinstance(self, W_Ufunc2)
+        obj = convert_to_array(space, w_obj)
+        if isinstance(obj, Scalar):
+            raise OperationError(space.w_TypeError, space.wrap("cannot reduce "
+                "on a scalar"))
+
+        size = obj.find_size()
+        dtype = find_unaryop_result_dtype(
+            space, obj.find_dtype(),
+            promote_to_largest=True
+        )
+        start = 0
+        if self.identity is None:
+            if size == 0:
+                raise operationerrfmt(space.w_ValueError, "zero-size array to "
+                    "%s.reduce without identity", self.name)
+            value = obj.eval(0).convert_to(dtype)
+            start += 1
+        else:
+            value = self.identity.convert_to(dtype)
+        new_sig = signature.Signature.find_sig([
+            self.reduce_signature, obj.signature
+        ])
+        return self.reduce(new_sig, start, value, obj, dtype, size).wrap(space)
+
+    def reduce(self, signature, start, value, obj, dtype, size):
+        i = start
+        while i < size:
+            reduce_driver.jit_merge_point(signature=signature, self=self,
+                                          value=value, obj=obj, i=i,
+                                          dtype=dtype, size=size)
+            value = self.func(dtype, value, obj.eval(i).convert_to(dtype))
+            i += 1
+        return value
+
+class W_Ufunc1(W_Ufunc):
+    argcount = 1
+
+    def __init__(self, func, name, promote_to_float=False, promote_bools=False,
+        identity=None):
+
+        W_Ufunc.__init__(self, name, promote_to_float, promote_bools, identity)
+        self.func = func
+        self.signature = signature.Call1(func)
+
+    def call(self, space, args_w):
         from pypy.module.micronumpy.interp_numarray import (Call1,
             convert_to_array, Scalar)
 
+        [w_obj] = args_w
         w_obj = convert_to_array(space, w_obj)
         res_dtype = find_unaryop_result_dtype(space,
             w_obj.find_dtype(),
-            promote_to_float=promote_to_float,
-            promote_bools=promote_bools,
+            promote_to_float=self.promote_to_float,
+            promote_bools=self.promote_bools,
         )
         if isinstance(w_obj, Scalar):
-            return func(res_dtype, w_obj.value.convert_to(res_dtype)).wrap(space)
+            return self.func(res_dtype, w_obj.value.convert_to(res_dtype)).wrap(space)
 
-        new_sig = signature.Signature.find_sig([call_sig, w_obj.signature])
+        new_sig = signature.Signature.find_sig([self.signature, w_obj.signature])
         w_res = Call1(new_sig, res_dtype, w_obj)
         w_obj.add_invalidates(w_res)
         return w_res
-    return func_with_new_name(impl, "%s_dispatcher" % func.__name__)
 
-def ufunc2(func=None, promote_to_float=False, promote_bools=False):
-    if func is None:
-        return lambda func: ufunc2(func, promote_to_float, promote_bools)
 
-    call_sig = signature.Call2(func)
-    def impl(space, w_lhs, w_rhs):
+class W_Ufunc2(W_Ufunc):
+    argcount = 2
+
+    def __init__(self, func, name, promote_to_float=False, promote_bools=False,
+        identity=None, comparison_func=False):
+
+        W_Ufunc.__init__(self, name, promote_to_float, promote_bools, identity)
+        self.func = func
+        self.comparison_func = comparison_func
+        self.signature = signature.Call2(func)
+        self.reduce_signature = signature.BaseSignature()
+
+    def call(self, space, args_w):
         from pypy.module.micronumpy.interp_numarray import (Call2,
             convert_to_array, Scalar)
 
+        [w_lhs, w_rhs] = args_w
         w_lhs = convert_to_array(space, w_lhs)
         w_rhs = convert_to_array(space, w_rhs)
-        res_dtype = find_binop_result_dtype(space,
+        calc_dtype = find_binop_result_dtype(space,
             w_lhs.find_dtype(), w_rhs.find_dtype(),
-            promote_to_float=promote_to_float,
-            promote_bools=promote_bools,
+            promote_to_float=self.promote_to_float,
+            promote_bools=self.promote_bools,
         )
+        if self.comparison_func:
+            res_dtype = space.fromcache(interp_dtype.W_BoolDtype)
+        else:
+            res_dtype = calc_dtype
         if isinstance(w_lhs, Scalar) and isinstance(w_rhs, Scalar):
-            return func(res_dtype, w_lhs.value, w_rhs.value).wrap(space)
+            return self.func(calc_dtype,
+                w_lhs.value.convert_to(calc_dtype),
+                w_rhs.value.convert_to(calc_dtype)
+            ).wrap(space)
 
         new_sig = signature.Signature.find_sig([
-            call_sig, w_lhs.signature, w_rhs.signature
+            self.signature, w_lhs.signature, w_rhs.signature
         ])
-        w_res = Call2(new_sig, res_dtype, w_lhs, w_rhs)
+        w_res = Call2(new_sig, calc_dtype, res_dtype, w_lhs, w_rhs)
         w_lhs.add_invalidates(w_res)
         w_rhs.add_invalidates(w_res)
         return w_res
-    return func_with_new_name(impl, "%s_dispatcher" % func.__name__)
+
+
+W_Ufunc.typedef = TypeDef("ufunc",
+    __module__ = "numpy",
+
+    __call__ = interp2app(W_Ufunc.descr_call),
+    __repr__ = interp2app(W_Ufunc.descr_repr),
+
+    identity = GetSetProperty(W_Ufunc.descr_get_identity),
+    nin = interp_attrproperty("argcount", cls=W_Ufunc),
+
+    reduce = interp2app(W_Ufunc.descr_reduce),
+)
 
 def find_binop_result_dtype(space, dt1, dt2, promote_to_float=False,
     promote_bools=False):
@@ -74,7 +185,7 @@
     assert False
 
 def find_unaryop_result_dtype(space, dt, promote_to_float=False,
-    promote_to_largest=False, promote_bools=False):
+    promote_bools=False, promote_to_largest=False):
     if promote_bools and (dt.kind == interp_dtype.BOOLLTR):
         return space.fromcache(interp_dtype.W_Int8Dtype)
     if promote_to_float:
@@ -106,53 +217,77 @@
     return space.fromcache(interp_dtype.W_Float64Dtype)
 
 
-def ufunc_dtype_caller(ufunc_name, op_name, argcount, **kwargs):
+def ufunc_dtype_caller(space, ufunc_name, op_name, argcount, comparison_func):
     if argcount == 1:
-        @ufunc(**kwargs)
         def impl(res_dtype, value):
             return getattr(res_dtype, op_name)(value)
     elif argcount == 2:
-        @ufunc2(**kwargs)
         def impl(res_dtype, lvalue, rvalue):
-            return getattr(res_dtype, op_name)(lvalue, rvalue)
+            res = getattr(res_dtype, op_name)(lvalue, rvalue)
+            if comparison_func:
+                res = space.fromcache(interp_dtype.W_BoolDtype).box(res)
+            return res
     return func_with_new_name(impl, ufunc_name)
 
-for ufunc_def in [
-    ("add", "add", 2),
-    ("subtract", "sub", 2),
-    ("multiply", "mul", 2),
-    ("divide", "div", 2, {"promote_bools": True}),
-    ("mod", "mod", 2, {"promote_bools": True}),
-    ("power", "pow", 2, {"promote_bools": True}),
+class UfuncState(object):
+    def __init__(self, space):
+        "NOT_RPYTHON"
+        for ufunc_def in [
+            ("add", "add", 2, {"identity": 0}),
+            ("subtract", "sub", 2),
+            ("multiply", "mul", 2, {"identity": 1}),
+            ("divide", "div", 2, {"promote_bools": True}),
+            ("mod", "mod", 2, {"promote_bools": True}),
+            ("power", "pow", 2, {"promote_bools": True}),
 
-    ("maximum", "max", 2),
-    ("minimum", "min", 2),
+            ("equal", "eq", 2, {"comparison_func": True}),
+            ("not_equal", "ne", 2, {"comparison_func": True}),
+            ("less", "lt", 2, {"comparison_func": True}),
+            ("less_equal", "le", 2, {"comparison_func": True}),
+            ("greater", "gt", 2, {"comparison_func": True}),
+            ("greater_equal", "ge", 2, {"comparison_func": True}),
 
-    ("copysign", "copysign", 2, {"promote_to_float": True}),
+            ("maximum", "max", 2),
+            ("minimum", "min", 2),
 
-    ("positive", "pos", 1),
-    ("negative", "neg", 1),
-    ("absolute", "abs", 1),
-    ("sign", "sign", 1, {"promote_bools": True}),
-    ("reciprocal", "reciprocal", 1),
+            ("copysign", "copysign", 2, {"promote_to_float": True}),
 
-    ("fabs", "fabs", 1, {"promote_to_float": True}),
-    ("floor", "floor", 1, {"promote_to_float": True}),
-    ("exp", "exp", 1, {"promote_to_float": True}),
+            ("positive", "pos", 1),
+            ("negative", "neg", 1),
+            ("absolute", "abs", 1),
+            ("sign", "sign", 1, {"promote_bools": True}),
+            ("reciprocal", "reciprocal", 1),
 
-    ("sin", "sin", 1, {"promote_to_float": True}),
-    ("cos", "cos", 1, {"promote_to_float": True}),
-    ("tan", "tan", 1, {"promote_to_float": True}),
-    ("arcsin", "arcsin", 1, {"promote_to_float": True}),
-    ("arccos", "arccos", 1, {"promote_to_float": True}),
-    ("arctan", "arctan", 1, {"promote_to_float": True}),
-]:
-    ufunc_name = ufunc_def[0]
-    op_name = ufunc_def[1]
-    argcount = ufunc_def[2]
-    try:
-        extra_kwargs = ufunc_def[3]
-    except IndexError:
-        extra_kwargs = {}
+            ("fabs", "fabs", 1, {"promote_to_float": True}),
+            ("floor", "floor", 1, {"promote_to_float": True}),
+            ("exp", "exp", 1, {"promote_to_float": True}),
 
-    globals()[ufunc_name] = ufunc_dtype_caller(ufunc_name, op_name, argcount, **extra_kwargs)
+            ("sin", "sin", 1, {"promote_to_float": True}),
+            ("cos", "cos", 1, {"promote_to_float": True}),
+            ("tan", "tan", 1, {"promote_to_float": True}),
+            ("arcsin", "arcsin", 1, {"promote_to_float": True}),
+            ("arccos", "arccos", 1, {"promote_to_float": True}),
+            ("arctan", "arctan", 1, {"promote_to_float": True}),
+        ]:
+            self.add_ufunc(space, *ufunc_def)
+
+    def add_ufunc(self, space, ufunc_name, op_name, argcount, extra_kwargs=None):
+        if extra_kwargs is None:
+            extra_kwargs = {}
+
+        identity = extra_kwargs.get("identity")
+        if identity is not None:
+            identity = space.fromcache(interp_dtype.W_Int64Dtype).adapt_val(identity)
+        extra_kwargs["identity"] = identity
+
+        func = ufunc_dtype_caller(space, ufunc_name, op_name, argcount,
+            comparison_func=extra_kwargs.get("comparison_func", False)
+        )
+        if argcount == 1:
+            ufunc = W_Ufunc1(func, ufunc_name, **extra_kwargs)
+        elif argcount == 2:
+            ufunc = W_Ufunc2(func, ufunc_name, **extra_kwargs)
+        setattr(self, ufunc_name, ufunc)
+
+def get(space):
+    return space.fromcache(UfuncState)
\ No newline at end of file
diff --git a/pypy/module/micronumpy/test/test_dtypes.py b/pypy/module/micronumpy/test/test_dtypes.py
--- a/pypy/module/micronumpy/test/test_dtypes.py
+++ b/pypy/module/micronumpy/test/test_dtypes.py
@@ -82,10 +82,20 @@
             assert a[1] == 1
 
     def test_add_int8(self):
-        from numpy import array
+        from numpy import array, dtype
 
         a = array(range(5), dtype="int8")
         b = a + a
+        assert b.dtype is dtype("int8")
+        for i in range(5):
+            assert b[i] == i * 2
+
+    def test_add_int16(self):
+        from numpy import array, dtype
+
+        a = array(range(5), dtype="int16")
+        b = a + a
+        assert b.dtype is dtype("int16")
         for i in range(5):
             assert b[i] == i * 2
 
@@ -98,4 +108,4 @@
         from numpy import dtype
 
         # You can't subclass dtype
-        raises(TypeError, type, "Foo", (dtype,), {})
\ No newline at end of file
+        raises(TypeError, type, "Foo", (dtype,), {})
diff --git a/pypy/module/micronumpy/test/test_numarray.py b/pypy/module/micronumpy/test/test_numarray.py
--- a/pypy/module/micronumpy/test/test_numarray.py
+++ b/pypy/module/micronumpy/test/test_numarray.py
@@ -52,10 +52,14 @@
         from numpy import array, zeros
         a = array(range(5), float)
         assert repr(a) == "array([0.0, 1.0, 2.0, 3.0, 4.0])"
+        a = array([], float)
+        assert repr(a) == "array([], dtype=float64)"
         a = zeros(1001)
         assert repr(a) == "array([0.0, 0.0, 0.0, ..., 0.0, 0.0, 0.0])"
         a = array(range(5), long)
         assert repr(a) == "array([0, 1, 2, 3, 4])"
+        a = array([], long)
+        assert repr(a) == "array([], dtype=int64)"
         a = array([True, False, True, False], "?")
         assert repr(a) == "array([True, False, True, False], dtype=bool)"
 
@@ -84,6 +88,9 @@
         a = array(range(5), dtype="int8")
         assert str(a) == "[0 1 2 3 4]"
 
+        a = array(range(5), dtype="int16")
+        assert str(a) == "[0 1 2 3 4]"
+
     def test_str_slice(self):
         from numpy import array, zeros
         a = array(range(5), float)
@@ -102,6 +109,16 @@
         assert a[-1] == 8
         raises(IndexError, "a[-6]")
 
+    def test_getitem_tuple(self):
+        from numpy import array
+        a = array(range(5))
+        raises(IndexError, "a[(1,2)]")
+        for i in xrange(5):
+            assert a[(i,)] == i
+        b = a[()]
+        for i in xrange(5):
+            assert a[i] == b[i]
+
     def test_setitem(self):
         from numpy import array
         a = array(range(5))
@@ -110,6 +127,17 @@
         raises(IndexError, "a[5] = 0.0")
         raises(IndexError, "a[-6] = 3.0")
 
+    def test_setitem_tuple(self):
+        from numpy import array
+        a = array(range(5))
+        raises(IndexError, "a[(1,2)] = [0,1]")
+        for i in xrange(5):
+            a[(i,)] = i+1
+            assert a[i] == i+1
+        a[()] = range(5)
+        for i in xrange(5):
+            assert a[i] == i
+
     def test_setslice_array(self):
         from numpy import array
         a = array(range(5))
@@ -529,6 +557,26 @@
         assert array([1.2, 5]).dtype is dtype(float)
         assert array([]).dtype is dtype(float)
 
+    def test_comparison(self):
+        import operator
+        from numpy import array, dtype
+
+        a = array(range(5))
+        b = array(range(5), float)
+        for func in [
+            operator.eq, operator.ne, operator.lt, operator.le, operator.gt,
+            operator.ge
+        ]:
+            c = func(a, 3)
+            assert c.dtype is dtype(bool)
+            for i in xrange(5):
+                assert c[i] == func(a[i], 3)
+
+            c = func(b, 3)
+            assert c.dtype is dtype(bool)
+            for i in xrange(5):
+                assert c[i] == func(b[i], 3)
+
 
 class AppTestSupport(object):
     def setup_class(cls):
@@ -541,4 +589,4 @@
         a = fromstring(self.data)
         for i in range(4):
             assert a[i] == i + 1
-        raises(ValueError, fromstring, "abc")
\ No newline at end of file
+        raises(ValueError, fromstring, "abc")
diff --git a/pypy/module/micronumpy/test/test_ufuncs.py b/pypy/module/micronumpy/test/test_ufuncs.py
--- a/pypy/module/micronumpy/test/test_ufuncs.py
+++ b/pypy/module/micronumpy/test/test_ufuncs.py
@@ -3,6 +3,32 @@
 
 
 class AppTestUfuncs(BaseNumpyAppTest):
+    def test_ufunc_instance(self):
+        from numpy import add, ufunc
+
+        assert isinstance(add, ufunc)
+        assert repr(add) == ""
+        assert repr(ufunc) == ""
+
+    def test_ufunc_attrs(self):
+        from numpy import add, multiply, sin
+
+        assert add.identity == 0
+        assert multiply.identity == 1
+        assert sin.identity is None
+
+        assert add.nin == 2
+        assert multiply.nin == 2
+        assert sin.nin == 1
+
+    def test_wrong_arguments(self):
+        from numpy import add, sin
+
+        raises(TypeError, add, 1)
+        raises(TypeError, add, 1, 2, 3)
+        raises(TypeError, sin, 1, 2)
+        raises(TypeError, sin)
+
     def test_single_item(self):
         from numpy import negative, sign, minimum
 
@@ -272,3 +298,42 @@
         b = arctan(a)
         assert math.isnan(b[0])
 
+    def test_reduce_errors(self):
+        from numpy import sin, add
+
+        raises(ValueError, sin.reduce, [1, 2, 3])
+        raises(TypeError, add.reduce, 1)
+
+    def test_reduce(self):
+        from numpy import add, maximum
+
+        assert add.reduce([1, 2, 3]) == 6
+        assert maximum.reduce([1]) == 1
+        assert maximum.reduce([1, 2, 3]) == 3
+        raises(ValueError, maximum.reduce, [])
+
+    def test_comparisons(self):
+        import operator
+        from numpy import equal, not_equal, less, less_equal, greater, greater_equal
+
+        for ufunc, func in [
+            (equal, operator.eq),
+            (not_equal, operator.ne),
+            (less, operator.lt),
+            (less_equal, operator.le),
+            (greater, operator.gt),
+            (greater_equal, operator.ge),
+        ]:
+            for a, b in [
+                (3, 3),
+                (3, 4),
+                (4, 3),
+                (3.0, 3.0),
+                (3.0, 3.5),
+                (3.5, 3.0),
+                (3.0, 3),
+                (3, 3.0),
+                (3.5, 3),
+                (3, 3.5),
+            ]:
+                assert ufunc(a, b) is func(a, b)
diff --git a/pypy/module/micronumpy/test/test_zjit.py b/pypy/module/micronumpy/test/test_zjit.py
--- a/pypy/module/micronumpy/test/test_zjit.py
+++ b/pypy/module/micronumpy/test/test_zjit.py
@@ -19,7 +19,7 @@
     def test_add(self):
         def f(i):
             ar = SingleDimArray(i, dtype=self.float64_dtype)
-            v = interp_ufuncs.add(self.space, ar, ar)
+            v = interp_ufuncs.get(self.space).add.call(self.space, [ar, ar])
             return v.get_concrete().eval(3).val
 
         result = self.meta_interp(f, [5], listops=True, backendopt=True)
@@ -31,9 +31,10 @@
     def test_floatadd(self):
         def f(i):
             ar = SingleDimArray(i, dtype=self.float64_dtype)
-            v = interp_ufuncs.add(self.space,
-                ar,
-                scalar_w(self.space, self.float64_dtype, self.space.wrap(4.5))
+            v = interp_ufuncs.get(self.space).add.call(self.space, [
+                    ar,
+                    scalar_w(self.space, self.float64_dtype, self.space.wrap(4.5))
+                ],
             )
             assert isinstance(v, BaseArray)
             return v.get_concrete().eval(3).val
@@ -89,14 +90,21 @@
     def test_max(self):
         space = self.space
         float64_dtype = self.float64_dtype
+        int64_dtype = self.int64_dtype
 
         def f(i):
-            ar = SingleDimArray(i, dtype=NonConstant(float64_dtype))
+            if NonConstant(False):
+                dtype = int64_dtype
+            else:
+                dtype = float64_dtype
+            ar = SingleDimArray(i, dtype=dtype)
             j = 0
             while j < i:
                 ar.get_concrete().setitem(j, float64_dtype.box(float(j)))
                 j += 1
-            return ar.descr_add(space, ar).descr_max(space).floatval
+            v = ar.descr_add(space, ar).descr_max(space)
+            assert isinstance(v, FloatObject)
+            return v.floatval
 
         result = self.meta_interp(f, [5], listops=True, backendopt=True)
         self.check_loops({"getarrayitem_raw": 2, "float_add": 1,
@@ -108,14 +116,21 @@
     def test_min(self):
         space = self.space
         float64_dtype = self.float64_dtype
+        int64_dtype = self.int64_dtype
 
         def f(i):
-            ar = SingleDimArray(i, dtype=NonConstant(float64_dtype))
+            if NonConstant(False):
+                dtype = int64_dtype
+            else:
+                dtype = float64_dtype
+            ar = SingleDimArray(i, dtype=dtype)
             j = 0
             while j < i:
                 ar.get_concrete().setitem(j, float64_dtype.box(float(j)))
                 j += 1
-            return ar.descr_add(space, ar).descr_min(space).floatval
+            v = ar.descr_add(space, ar).descr_min(space)
+            assert isinstance(v, FloatObject)
+            return v.floatval
 
         result = self.meta_interp(f, [5], listops=True, backendopt=True)
         self.check_loops({"getarrayitem_raw": 2, "float_add": 1,
@@ -180,9 +195,9 @@
 
         def f(i):
             ar = SingleDimArray(i, dtype=self.float64_dtype)
-            v1 = interp_ufuncs.add(space, ar, scalar_w(space, self.float64_dtype, space.wrap(4.5)))
+            v1 = interp_ufuncs.get(self.space).add.call(space, [ar, scalar_w(space, self.float64_dtype, space.wrap(4.5))])
             assert isinstance(v1, BaseArray)
-            v2 = interp_ufuncs.multiply(space, v1, scalar_w(space, self.float64_dtype, space.wrap(4.5)))
+            v2 = interp_ufuncs.get(self.space).multiply.call(space, [v1, scalar_w(space, self.float64_dtype, space.wrap(4.5))])
             v1.force_if_needed()
             assert isinstance(v2, BaseArray)
             return v2.get_concrete().eval(3).val
@@ -200,8 +215,8 @@
         space = self.space
         def f(i):
             ar = SingleDimArray(i, dtype=self.float64_dtype)
-            v1 = interp_ufuncs.add(space, ar, ar)
-            v2 = interp_ufuncs.negative(space, v1)
+            v1 = interp_ufuncs.get(self.space).add.call(space, [ar, ar])
+            v2 = interp_ufuncs.get(self.space).negative.call(space, [v1])
             return v2.get_concrete().eval(3).val
 
         result = self.meta_interp(f, [5], listops=True, backendopt=True)
@@ -216,13 +231,13 @@
         def f(i):
             ar = SingleDimArray(i, dtype=self.float64_dtype)
 
-            v1 = interp_ufuncs.add(space, ar, ar)
-            v2 = interp_ufuncs.negative(space, v1)
+            v1 = interp_ufuncs.get(self.space).add.call(space, [ar, ar])
+            v2 = interp_ufuncs.get(self.space).negative.call(space, [v1])
             v2.get_concrete()
 
             for i in xrange(5):
-                v1 = interp_ufuncs.multiply(space, ar, ar)
-                v2 = interp_ufuncs.negative(space, v1)
+                v1 = interp_ufuncs.get(self.space).multiply.call(space, [ar, ar])
+                v2 = interp_ufuncs.get(self.space).negative.call(space, [v1])
                 v2.get_concrete()
 
         self.meta_interp(f, [5], listops=True, backendopt=True)
@@ -237,7 +252,7 @@
                 SingleDimSlice.signature, ar.signature
             ])
             s = SingleDimSlice(0, step*i, step, i, ar, new_sig)
-            v = interp_ufuncs.add(self.space, s, s)
+            v = interp_ufuncs.get(self.space).add.call(self.space, [s, s])
             return v.get_concrete().eval(3).val
 
         result = self.meta_interp(f, [5], listops=True, backendopt=True)
@@ -259,7 +274,7 @@
                 SingleDimSlice.signature, s1.signature
             ])
             s2 = SingleDimSlice(0, step2*i, step2, i, ar, new_sig)
-            v = interp_ufuncs.add(self.space, s1, s2)
+            v = interp_ufuncs.get(self.space).add.call(self.space, [s1, s2])
             return v.get_concrete().eval(3).val
 
         result = self.meta_interp(f, [5], listops=True, backendopt=True)
diff --git a/pypy/module/pwd/__init__.py b/pypy/module/pwd/__init__.py
new file mode 100644
--- /dev/null
+++ b/pypy/module/pwd/__init__.py
@@ -0,0 +1,25 @@
+from pypy.interpreter.mixedmodule import MixedModule
+
+class Module(MixedModule):
+    """
+    This module provides access to the Unix password database.
+    It is available on all Unix versions.
+
+    Password database entries are reported as 7-tuples containing the following
+    items from the password database (see `'), in order:
+    pw_name, pw_passwd, pw_uid, pw_gid, pw_gecos, pw_dir, pw_shell.
+    The uid and gid items are integers, all others are strings. An
+    exception is raised if the entry asked for cannot be found.
+    """
+
+    interpleveldefs = {
+        'getpwuid': 'interp_pwd.getpwuid',
+        'getpwnam': 'interp_pwd.getpwnam',
+        'getpwall': 'interp_pwd.getpwall',
+    }
+
+    appleveldefs = {
+        'struct_passwd': 'app_pwd.struct_passwd',
+        'struct_pwent': 'app_pwd.struct_passwd',
+    }
+
diff --git a/pypy/module/pwd/app_pwd.py b/pypy/module/pwd/app_pwd.py
new file mode 100644
--- /dev/null
+++ b/pypy/module/pwd/app_pwd.py
@@ -0,0 +1,20 @@
+from _structseq import structseqtype, structseqfield
+
+class struct_passwd:
+    """
+    pwd.struct_passwd: Results from getpw*() routines.
+
+    This object may be accessed either as a tuple of
+      (pw_name,pw_passwd,pw_uid,pw_gid,pw_gecos,pw_dir,pw_shell)
+    or via the object attributes as named in the above tuple.
+    """
+    __metaclass__ = structseqtype
+    name = "pwd.struct_passwd"
+
+    pw_name   = structseqfield(0, "user name")
+    pw_passwd = structseqfield(1, "password")
+    pw_uid    = structseqfield(2, "user id")
+    pw_gid    = structseqfield(3, "group id")
+    pw_gecos  = structseqfield(4, "real name")
+    pw_dir    = structseqfield(5, "home directory")
+    pw_shell  = structseqfield(6, "shell program")
diff --git a/pypy/module/pwd/interp_pwd.py b/pypy/module/pwd/interp_pwd.py
new file mode 100644
--- /dev/null
+++ b/pypy/module/pwd/interp_pwd.py
@@ -0,0 +1,95 @@
+from pypy.translator.tool.cbuild import ExternalCompilationInfo
+from pypy.rpython.tool import rffi_platform
+from pypy.rpython.lltypesystem import rffi, lltype
+from pypy.interpreter.gateway import interp2app, unwrap_spec
+from pypy.interpreter.error import OperationError, operationerrfmt
+from pypy.rlib.rarithmetic import intmask
+
+eci = ExternalCompilationInfo(
+    includes=['pwd.h']
+    )
+
+class CConfig:
+    _compilation_info_ = eci
+
+    uid_t = rffi_platform.SimpleType("uid_t")
+
+    passwd = rffi_platform.Struct(
+        'struct passwd',
+        [('pw_name', rffi.CCHARP),
+         ('pw_passwd', rffi.CCHARP),
+         ('pw_uid', rffi.INT),
+         ('pw_gid', rffi.INT),
+         ('pw_gecos', rffi.CCHARP),
+         ('pw_dir', rffi.CCHARP),
+         ('pw_shell', rffi.CCHARP),
+         ])
+
+config = rffi_platform.configure(CConfig)
+passwd_p = lltype.Ptr(config['passwd'])
+uid_t = config['uid_t']
+
+def external(name, args, result, **kwargs):
+    return rffi.llexternal(name, args, result, compilation_info=eci, **kwargs)
+
+c_getpwuid = external("getpwuid", [uid_t], passwd_p)
+c_getpwnam = external("getpwnam", [rffi.CCHARP], passwd_p)
+c_setpwent = external("setpwent", [], lltype.Void)
+c_getpwent = external("getpwent", [], passwd_p)
+c_endpwent = external("endpwent", [], lltype.Void)
+
+def make_struct_passwd(space, pw):
+    w_passwd_struct = space.getattr(space.getbuiltinmodule('pwd'),
+                                    space.wrap('struct_passwd'))
+    w_tuple = space.newtuple([
+        space.wrap(rffi.charp2str(pw.c_pw_name)),
+        space.wrap(rffi.charp2str(pw.c_pw_passwd)),
+        space.wrap(intmask(pw.c_pw_uid)),
+        space.wrap(intmask(pw.c_pw_gid)),
+        space.wrap(rffi.charp2str(pw.c_pw_gecos)),
+        space.wrap(rffi.charp2str(pw.c_pw_dir)),
+        space.wrap(rffi.charp2str(pw.c_pw_shell)),
+        ])
+    return space.call_function(w_passwd_struct, w_tuple)
+
+ at unwrap_spec(uid=int)
+def getpwuid(space, uid):
+    """
+    getpwuid(uid) -> (pw_name,pw_passwd,pw_uid,
+                      pw_gid,pw_gecos,pw_dir,pw_shell)
+    Return the password database entry for the given numeric user ID.
+    See pwd.__doc__ for more on password database entries.
+    """
+    pw = c_getpwuid(uid)
+    if not pw:
+        raise operationerrfmt(space.w_KeyError,
+            "getpwuid(): uid not found: %d", uid)
+    return make_struct_passwd(space, pw)
+
+ at unwrap_spec(name=str)
+def getpwnam(space, name):
+    """
+    getpwnam(name) -> (pw_name,pw_passwd,pw_uid,
+                        pw_gid,pw_gecos,pw_dir,pw_shell)
+    Return the password database entry for the given user name.
+    See pwd.__doc__ for more on password database entries.
+    """
+    pw = c_getpwnam(name)
+    if not pw:
+        raise operationerrfmt(space.w_KeyError,
+            "getpwnam(): name not found: %s", name)
+    return make_struct_passwd(space, pw)
+
+def getpwall(space):
+    users_w = []
+    c_setpwent()
+    try:
+        while True:
+            pw = c_getpwent()
+            if not pw:
+                break
+            users_w.append(make_struct_passwd(space, pw))
+    finally:
+        c_endpwent()
+    return space.newlist(users_w)
+    
diff --git a/pypy/module/pwd/test/test_pwd.py b/pypy/module/pwd/test/test_pwd.py
new file mode 100644
--- /dev/null
+++ b/pypy/module/pwd/test/test_pwd.py
@@ -0,0 +1,28 @@
+from pypy.conftest import gettestobjspace
+
+class AppTestPwd:
+    def setup_class(cls):
+        cls.space = gettestobjspace(usemodules=['pwd'])
+
+    def test_getpwuid(self):
+        import pwd
+        raises(KeyError, pwd.getpwuid, -1)
+        pw = pwd.getpwuid(0)
+        assert pw.pw_name == 'root'
+        assert isinstance(pw.pw_passwd, str)
+        assert pw.pw_uid == 0
+        assert pw.pw_gid == 0
+        assert pw.pw_dir == '/root'
+        assert pw.pw_shell.startswith('/')
+        #
+        assert type(pw.pw_uid) is int
+        assert type(pw.pw_gid) is int
+
+    def test_getpwnam(self):
+        import pwd
+        raises(KeyError, pwd.getpwnam, '~invalid~')
+        assert pwd.getpwnam('root').pw_name == 'root'
+
+    def test_getpwall(self):
+        import pwd
+        assert pwd.getpwnam('root') in pwd.getpwall()
diff --git a/pypy/module/pypyjit/interp_jit.py b/pypy/module/pypyjit/interp_jit.py
--- a/pypy/module/pypyjit/interp_jit.py
+++ b/pypy/module/pypyjit/interp_jit.py
@@ -21,6 +21,7 @@
 
 PyFrame._virtualizable2_ = ['last_instr', 'pycode',
                             'valuestackdepth', 'locals_stack_w[*]',
+                            'cells[*]',
                             'last_exception',
                             'lastblock',
                             'is_being_profiled',
diff --git a/pypy/module/pypyjit/policy.py b/pypy/module/pypyjit/policy.py
--- a/pypy/module/pypyjit/policy.py
+++ b/pypy/module/pypyjit/policy.py
@@ -8,7 +8,8 @@
                 modname == '__builtin__.interp_classobj' or
                 modname == '__builtin__.functional' or
                 modname == '__builtin__.descriptor' or
-                modname == 'thread.os_local'):
+                modname == 'thread.os_local' or
+                modname == 'thread.os_thread'):
             return True
         if '.' in modname:
             modname, _ = modname.split('.', 1)
diff --git a/pypy/module/pypyjit/test/test_policy.py b/pypy/module/pypyjit/test/test_policy.py
--- a/pypy/module/pypyjit/test/test_policy.py
+++ b/pypy/module/pypyjit/test/test_policy.py
@@ -34,7 +34,9 @@
 
 def test_thread_local():
     from pypy.module.thread.os_local import Local
+    from pypy.module.thread.os_thread import get_ident
     assert pypypolicy.look_inside_function(Local.getdict.im_func)
+    assert pypypolicy.look_inside_function(get_ident)
 
 def test_pypy_module():
     from pypy.module._collections.interp_deque import W_Deque
diff --git a/pypy/module/pypyjit/test_pypy_c/model.py b/pypy/module/pypyjit/test_pypy_c/model.py
--- a/pypy/module/pypyjit/test_pypy_c/model.py
+++ b/pypy/module/pypyjit/test_pypy_c/model.py
@@ -2,7 +2,10 @@
 import sys
 import re
 import os.path
-from _pytest.assertion import newinterpret
+try:
+    from _pytest.assertion import newinterpret
+except ImportError:   # e.g. Python 2.5
+    newinterpret = None
 from pypy.tool.jitlogparser.parser import SimpleParser, Function, TraceForOpcode
 from pypy.tool.jitlogparser.storage import LoopStorage
 
@@ -196,7 +199,7 @@
                     source = str(source.deindent()).strip()
         except py.error.ENOENT:
             source = None
-        if source and source.startswith('self._assert('):
+        if source and source.startswith('self._assert(') and newinterpret:
             # transform self._assert(x, 'foo') into assert x, 'foo'
             source = source.replace('self._assert(', 'assert ')
             source = source[:-1] # remove the trailing ')'
diff --git a/pypy/module/pypyjit/test_pypy_c/test_00_model.py b/pypy/module/pypyjit/test_pypy_c/test_00_model.py
--- a/pypy/module/pypyjit/test_pypy_c/test_00_model.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_00_model.py
@@ -1,3 +1,4 @@
+from __future__ import with_statement
 import sys
 import types
 import subprocess
diff --git a/pypy/module/pypyjit/test_pypy_c/test_call.py b/pypy/module/pypyjit/test_pypy_c/test_call.py
--- a/pypy/module/pypyjit/test_pypy_c/test_call.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_call.py
@@ -174,7 +174,7 @@
             guard_no_overflow(descr=...)
             i18 = force_token()
             --TICK--
-            jump(p0, p1, p2, p3, p4, i8, p7, i17, p8, i9, i17, p10, p11, p12, descr=)
+            jump(..., descr=)
         """)
 
     def test_default_and_kw(self):
@@ -396,3 +396,70 @@
             --TICK--
             jump(..., descr=)
         """)
+
+    def test_global_closure_has_constant_cells(self):
+        log = self.run("""
+            def make_adder(n):
+                def add(x):
+                    return x + n
+                return add
+            add5 = make_adder(5)
+            def main():
+                i = 0
+                while i < 5000:
+                    i = add5(i) # ID: call
+            """, [])
+        loop, = log.loops_by_id('call', is_entry_bridge=True)
+        assert loop.match("""
+            guard_value(i6, 1, descr=...)
+            guard_nonnull_class(p8, ConstClass(W_IntObject), descr=...)
+            guard_value(i4, 0, descr=...)
+            guard_value(p3, ConstPtr(ptr14), descr=...)
+            i15 = getfield_gc_pure(p8, descr=)
+            i17 = int_lt(i15, 5000)
+            guard_true(i17, descr=...)
+            p18 = getfield_gc(p0, descr=)
+            guard_value(p18, ConstPtr(ptr19), descr=...)
+            p20 = getfield_gc(p18, descr=)
+            guard_value(p20, ConstPtr(ptr21), descr=...)
+            guard_not_invalidated(descr=...)
+            # most importantly, there is no getarrayitem_gc here
+            p23 = call(ConstClass(getexecutioncontext), descr=)
+            p24 = getfield_gc(p23, descr=)
+            i25 = force_token()
+            p26 = getfield_gc(p23, descr=)
+            guard_isnull(p26, descr=...)
+            i27 = getfield_gc(p23, descr=)
+            i28 = int_is_zero(i27)
+            guard_true(i28, descr=...)
+            p30 = getfield_gc(ConstPtr(ptr29), descr=)
+            guard_nonnull_class(p30, ConstClass(W_IntObject), descr=...)
+            i32 = getfield_gc_pure(p30, descr=)
+            i33 = int_add_ovf(i15, i32)
+            guard_no_overflow(descr=...)
+            --TICK--
+            jump(p0, p1, p2, p5, i33, i32, p23, p30, p24, descr=)
+        """)
+
+    def test_local_closure_is_virtual(self):
+        log = self.run("""
+            def main():
+                i = 0
+                while i < 5000:
+                    def add():
+                        return i + 1
+                    i = add() # ID: call
+            """, [])
+        loop, = log.loops_by_id('call')
+        assert loop.match("""
+            i8 = getfield_gc_pure(p6, descr=)
+            i10 = int_lt(i8, 5000)
+            guard_true(i10, descr=...)
+            i11 = force_token()
+            i13 = int_add(i8, 1)
+            --TICK--
+            p22 = new_with_vtable(ConstClass(W_IntObject))
+            setfield_gc(p22, i13, descr=)
+            setfield_gc(p4, p22, descr=)
+            jump(p0, p1, p2, p3, p4, p7, p22, p7, descr=)
+        """)
diff --git a/pypy/module/pypyjit/test_pypy_c/test_globals.py b/pypy/module/pypyjit/test_pypy_c/test_globals.py
--- a/pypy/module/pypyjit/test_pypy_c/test_globals.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_globals.py
@@ -23,6 +23,4 @@
             guard_not_invalidated(descr=...)
             p19 = getfield_gc(ConstPtr(p17), descr=)
             guard_value(p19, ConstPtr(ptr20), descr=...)
-            p22 = getfield_gc(ConstPtr(ptr21), descr=)
-            guard_nonnull(p22, descr=...)
-        """)
+        """)
\ No newline at end of file
diff --git a/pypy/module/pypyjit/test_pypy_c/test_instance.py b/pypy/module/pypyjit/test_pypy_c/test_instance.py
--- a/pypy/module/pypyjit/test_pypy_c/test_instance.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_instance.py
@@ -142,6 +142,7 @@
             i = 0
             b = B(1)
             while i < 100:
+                b.x
                 v = b.x # ID: loadattr
                 i += v
             return i
@@ -150,8 +151,6 @@
         loop, = log.loops_by_filename(self.filepath)
         assert loop.match_by_id('loadattr',
         '''
-        guard_not_invalidated(descr=...)
-        i16 = arraylen_gc(p10, descr=)
         i19 = call(ConstClass(ll_dict_lookup), _, _, _, descr=...)
         guard_no_exception(descr=...)
         i21 = int_and(i19, _)
@@ -181,8 +180,7 @@
         assert loop.match_by_id("contains", """
             guard_not_invalidated(descr=...)
             i11 = force_token()
-            i12 = int_add_ovf(i5, i7)
-            guard_no_overflow(descr=...)
+            i12 = int_add(i5, 1)
         """)
 
     def test_id_compare_optimization(self):
diff --git a/pypy/module/sys/__init__.py b/pypy/module/sys/__init__.py
--- a/pypy/module/sys/__init__.py
+++ b/pypy/module/sys/__init__.py
@@ -47,6 +47,7 @@
         'pypy_initial_path'     : 'state.pypy_initial_path',
 
         '_getframe'             : 'vm._getframe', 
+        '_current_frames'       : 'vm._current_frames', 
         'setrecursionlimit'     : 'vm.setrecursionlimit', 
         'getrecursionlimit'     : 'vm.getrecursionlimit', 
         'setcheckinterval'      : 'vm.setcheckinterval', 
diff --git a/pypy/module/sys/test/test_encoding.py b/pypy/module/sys/test/test_encoding.py
new file mode 100644
--- /dev/null
+++ b/pypy/module/sys/test/test_encoding.py
@@ -0,0 +1,30 @@
+import os, py
+from pypy.rlib import rlocale
+from pypy.module.sys.interp_encoding import _getfilesystemencoding
+from pypy.module.sys.interp_encoding import base_encoding
+
+
+def test__getfilesystemencoding(space):
+    if not (rlocale.HAVE_LANGINFO and rlocale.CODESET):
+        py.test.skip("requires HAVE_LANGINFO and CODESET")
+
+    def clear():
+        for key in os.environ.keys():
+            if key == 'LANG' or key.startswith('LC_'):
+                del os.environ[key]
+
+    def get(**env):
+        original_env = os.environ.copy()
+        try:
+            clear()
+            os.environ.update(env)
+            return _getfilesystemencoding(space)
+        finally:
+            clear()
+            os.environ.update(original_env)
+
+    assert get() in (base_encoding, 'ANSI_X3.4-1968')
+    assert get(LANG='foobar') in (base_encoding, 'ANSI_X3.4-1968')
+    assert get(LANG='en_US.UTF-8') == 'UTF-8'
+    assert get(LC_ALL='en_US.UTF-8') == 'UTF-8'
+    assert get(LC_CTYPE='en_US.UTF-8') == 'UTF-8'
diff --git a/pypy/module/sys/test/test_sysmodule.py b/pypy/module/sys/test/test_sysmodule.py
--- a/pypy/module/sys/test/test_sysmodule.py
+++ b/pypy/module/sys/test/test_sysmodule.py
@@ -1,6 +1,6 @@
 # -*- coding: iso-8859-1 -*-
 import autopath
-from pypy.conftest import option
+from pypy.conftest import option, gettestobjspace
 from py.test import raises
 from pypy.interpreter.gateway import app2interp_temp
 import sys
@@ -524,3 +524,51 @@
         # If this ever actually becomes a compilation option this test should
         # be changed.
         assert sys.float_repr_style == "short"
+
+class AppTestCurrentFrames:
+
+    def test_current_frames(self):
+        try:
+            import thread
+        except ImportError:
+            pass
+        else:
+            skip('This test requires an intepreter without threads')
+        import sys
+
+        def f():
+            return sys._current_frames()
+        frames = f()
+        assert frames.keys() == [0]
+        assert frames[0].f_code.co_name == 'f'
+
+class AppTestCurrentFramesWithThread(AppTestCurrentFrames):
+    def setup_class(cls):
+        cls.space = gettestobjspace(usemodules=('thread',))
+
+    def test_current_frames(self):
+        import sys
+        import time
+        import thread
+
+        thread_id = thread.get_ident()
+        self.ready = False
+        def other_thread():
+            self.ready = True
+            print "thread started"
+            time.sleep(5)
+        thread.start_new_thread(other_thread, ())
+
+        def f():
+            for i in range(100):
+                if self.ready: break
+                time.sleep(0.1)
+            return sys._current_frames()
+        
+        frames = f()
+        thisframe = frames.pop(thread_id)
+        assert thisframe.f_code.co_name == 'f'
+
+        assert len(frames) == 1
+        _, other_frame = frames.popitem()
+        assert other_frame.f_code.co_name == 'other_thread'
diff --git a/pypy/module/sys/version.py b/pypy/module/sys/version.py
--- a/pypy/module/sys/version.py
+++ b/pypy/module/sys/version.py
@@ -14,7 +14,7 @@
 
 if platform.name == 'msvc':
     COMPILER_INFO = 'MSC v.%d 32 bit' % (platform.version * 10 + 600)
-elif platform.cc.startswith('gcc'):
+elif platform.cc is not None and platform.cc.startswith('gcc'):
     out = platform.execute(platform.cc, '--version').out
     match = re.search(' (\d+\.\d+(\.\d+)*)', out)
     if match:
diff --git a/pypy/module/sys/vm.py b/pypy/module/sys/vm.py
--- a/pypy/module/sys/vm.py
+++ b/pypy/module/sys/vm.py
@@ -43,6 +43,23 @@
     f.mark_as_escaped()
     return space.wrap(f)
 
+def _current_frames(space):
+    """_current_frames() -> dictionary
+
+    Return a dictionary mapping each current thread T's thread id to T's
+    current stack frame.
+
+    This function should be used for specialized purposes only."""
+    w_result = space.newdict()
+    ecs = space.threadlocals.getallvalues()
+    for thread_ident, ec in ecs.items():
+        f = ec.gettopframe_nohidden()
+        f.mark_as_escaped()
+        space.setitem(w_result,
+                      space.wrap(thread_ident),
+                      space.wrap(f))
+    return w_result                      
+
 def setrecursionlimit(space, w_new_limit):
     """setrecursionlimit() sets the maximum number of nested calls that
 can occur before a RuntimeError is raised.  On PyPy the limit is
diff --git a/pypy/module/test_lib_pypy/test_distributed/test_distributed.py b/pypy/module/test_lib_pypy/test_distributed/test_distributed.py
--- a/pypy/module/test_lib_pypy/test_distributed/test_distributed.py
+++ b/pypy/module/test_lib_pypy/test_distributed/test_distributed.py
@@ -1,3 +1,4 @@
+import py; py.test.skip("xxx remove")
 
 """ Controllers tests
 """
@@ -8,7 +9,7 @@
 class AppTestDistributed(object):
     def setup_class(cls):
         cls.space = gettestobjspace(**{"objspace.std.withtproxy": True,
-            "usemodules":("_stackless",)})
+            "usemodules":("_continuation",)})
 
     def test_init(self):
         import distributed
@@ -90,14 +91,12 @@
 
 class AppTestDistributedTasklets(object):
     spaceconfig = {"objspace.std.withtproxy": True,
-                   "objspace.usemodules._stackless": True}
+                   "objspace.usemodules._continuation": True}
     reclimit = sys.getrecursionlimit()
 
     def setup_class(cls):
         import py.test
         py.test.importorskip('greenlet')
-        #cls.space = gettestobjspace(**{"objspace.std.withtproxy": True,
-        #    "usemodules":("_stackless",)})
         cls.w_test_env_ = cls.space.appexec([], """():
         from distributed import test_env
         return (test_env,)
diff --git a/pypy/module/test_lib_pypy/test_distributed/test_greensock.py b/pypy/module/test_lib_pypy/test_distributed/test_greensock.py
--- a/pypy/module/test_lib_pypy/test_distributed/test_greensock.py
+++ b/pypy/module/test_lib_pypy/test_distributed/test_greensock.py
@@ -1,5 +1,4 @@
-
-import py
+import py; py.test.skip("xxx remove")
 from pypy.conftest import gettestobjspace, option
 
 def setup_module(mod):
@@ -10,7 +9,7 @@
         if not option.runappdirect:
             py.test.skip("Cannot run this on top of py.py because of PopenGateway")
         cls.space = gettestobjspace(**{"objspace.std.withtproxy": True,
-                                       "usemodules":("_stackless",)})
+                                       "usemodules":("_continuation",)})
         cls.w_remote_side_code = cls.space.appexec([], """():
         import sys
         sys.path.insert(0, '%s')
diff --git a/pypy/module/test_lib_pypy/test_distributed/test_socklayer.py b/pypy/module/test_lib_pypy/test_distributed/test_socklayer.py
--- a/pypy/module/test_lib_pypy/test_distributed/test_socklayer.py
+++ b/pypy/module/test_lib_pypy/test_distributed/test_socklayer.py
@@ -1,4 +1,4 @@
-import py
+import py; py.test.skip("xxx remove")
 from pypy.conftest import gettestobjspace
 
 def setup_module(mod):
@@ -9,7 +9,8 @@
 class AppTestSocklayer:
     def setup_class(cls):
         cls.space = gettestobjspace(**{"objspace.std.withtproxy": True,
-                                       "usemodules":("_stackless","_socket", "select")})
+                                       "usemodules":("_continuation",
+                                                     "_socket", "select")})
     
     def test_socklayer(self):
         class X(object):
diff --git a/pypy/module/test_lib_pypy/test_greenlet.py b/pypy/module/test_lib_pypy/test_greenlet.py
--- a/pypy/module/test_lib_pypy/test_greenlet.py
+++ b/pypy/module/test_lib_pypy/test_greenlet.py
@@ -231,3 +231,13 @@
         assert res == "next step"
         res = g2.switch("goes to f1 instead")
         assert res == "all ok"
+
+    def test_throw_in_not_started_yet(self):
+        from greenlet import greenlet
+        #
+        def f1():
+            never_reached
+        #
+        g1 = greenlet(f1)
+        raises(ValueError, g1.throw, ValueError)
+        assert g1.dead
diff --git a/pypy/module/test_lib_pypy/test_stackless.py b/pypy/module/test_lib_pypy/test_stackless_pickle.py
rename from pypy/module/test_lib_pypy/test_stackless.py
rename to pypy/module/test_lib_pypy/test_stackless_pickle.py
--- a/pypy/module/test_lib_pypy/test_stackless.py
+++ b/pypy/module/test_lib_pypy/test_stackless_pickle.py
@@ -1,3 +1,4 @@
+import py; py.test.skip("XXX port me")
 from pypy.conftest import gettestobjspace, option
 
 class AppTest_Stackless:
diff --git a/pypy/module/thread/threadlocals.py b/pypy/module/thread/threadlocals.py
--- a/pypy/module/thread/threadlocals.py
+++ b/pypy/module/thread/threadlocals.py
@@ -43,6 +43,9 @@
         ident = self._mainthreadident
         return self._valuedict.get(ident, None)
 
+    def getallvalues(self):
+        return self._valuedict
+
     def enter_thread(self, space):
         "Notification that the current thread is just starting."
         ec = space.getexecutioncontext()
diff --git a/pypy/objspace/flow/flowcontext.py b/pypy/objspace/flow/flowcontext.py
--- a/pypy/objspace/flow/flowcontext.py
+++ b/pypy/objspace/flow/flowcontext.py
@@ -184,7 +184,7 @@
 
 class FlowExecutionContext(ExecutionContext):
 
-    def __init__(self, space, code, globals, constargs={}, closure=None,
+    def __init__(self, space, code, globals, constargs={}, outer_func=None,
                  name=None):
         ExecutionContext.__init__(self, space)
         self.code = code
@@ -193,11 +193,11 @@
 
         self.crnt_offset = -1
         self.crnt_frame = None
-        if closure is None:
+        if outer_func and outer_func.closure:
+            self.closure = [nestedscope.Cell(Constant(value))
+                            for value in outer_func.closure]
+        else:
             self.closure = None
-        else:
-            self.closure = [nestedscope.Cell(Constant(value))
-                            for value in closure]
         frame = self.create_frame()
         formalargcount = code.getformalargcount()
         arg_list = [Variable() for i in range(formalargcount)]
@@ -216,7 +216,7 @@
         # while ignoring any operation like the creation of the locals dict
         self.recorder = []
         frame = FlowSpaceFrame(self.space, self.code,
-                               self.w_globals, self.closure)
+                               self.w_globals, self)
         frame.last_instr = 0
         return frame
 
diff --git a/pypy/objspace/flow/objspace.py b/pypy/objspace/flow/objspace.py
--- a/pypy/objspace/flow/objspace.py
+++ b/pypy/objspace/flow/objspace.py
@@ -252,9 +252,9 @@
             raise TypeError("%r is a generator" % (func,))
         code = PyCode._from_code(self, code)
         if func.func_closure is None:
-            closure = None
+            cl = None
         else:
-            closure = [extract_cell_content(c) for c in func.func_closure]
+            cl = [extract_cell_content(c) for c in func.func_closure]
         # CallableFactory.pycall may add class_ to functions that are methods
         name = func.func_name
         class_ = getattr(func, 'class_', None)
@@ -262,8 +262,10 @@
             name = '%s.%s' % (class_.__name__, name)
         for c in "<>&!":
             name = name.replace(c, '_')
+        class outerfunc: # hack
+            closure = cl
         ec = flowcontext.FlowExecutionContext(self, code, func.func_globals,
-                                              constargs, closure, name)
+                                              constargs, outerfunc, name)
         graph = ec.graph
         graph.func = func
         # attach a signature and defaults to the graph
diff --git a/pypy/objspace/std/celldict.py b/pypy/objspace/std/celldict.py
--- a/pypy/objspace/std/celldict.py
+++ b/pypy/objspace/std/celldict.py
@@ -65,6 +65,10 @@
         if isinstance(cell, ModuleCell):
             cell.w_value = w_value
             return
+        # If the new value and the current value are the same, don't create a
+        # level of indirection, or mutate are version.
+        if self.space.is_w(w_value, cell):
+            return
         if cell is not None:
             w_value = ModuleCell(w_value)
         self.mutated()
diff --git a/pypy/objspace/std/fake.py b/pypy/objspace/std/fake.py
--- a/pypy/objspace/std/fake.py
+++ b/pypy/objspace/std/fake.py
@@ -142,7 +142,7 @@
 
     def funcrun(self, func, args):
         frame = func.space.createframe(self, func.w_func_globals,
-                                        func.closure)
+                                       func)
         sig = self.signature()
         scope_w = args.parse_obj(None, func.name, sig, func.defs_w)
         frame.setfastscope(scope_w)
diff --git a/pypy/objspace/std/floatobject.py b/pypy/objspace/std/floatobject.py
--- a/pypy/objspace/std/floatobject.py
+++ b/pypy/objspace/std/floatobject.py
@@ -355,9 +355,13 @@
     y = w_float2.floatval
     if y == 0.0:
         raise FailedToImplementArgs(space.w_ZeroDivisionError, space.wrap("float modulo"))
-    mod = math.fmod(x, y)
-    if (mod and ((y < 0.0) != (mod < 0.0))):
-        mod += y
+    try:
+        mod = math.fmod(x, y)
+    except ValueError:
+        mod = rfloat.NAN
+    else:
+        if (mod and ((y < 0.0) != (mod < 0.0))):
+            mod += y
 
     return W_FloatObject(mod)
 
@@ -366,7 +370,10 @@
     y = w_float2.floatval
     if y == 0.0:
         raise FailedToImplementArgs(space.w_ZeroDivisionError, space.wrap("float modulo"))
-    mod = math.fmod(x, y)
+    try:
+        mod = math.fmod(x, y)
+    except ValueError:
+        return [W_FloatObject(rfloat.NAN), W_FloatObject(rfloat.NAN)]
     # fmod is typically exact, so vx-mod is *mathematically* an
     # exact multiple of wx.  But this is fp arithmetic, and fp
     # vx - mod is an approximation; the result is that div may
diff --git a/pypy/objspace/std/marshal_impl.py b/pypy/objspace/std/marshal_impl.py
--- a/pypy/objspace/std/marshal_impl.py
+++ b/pypy/objspace/std/marshal_impl.py
@@ -325,10 +325,10 @@
     # of building a list of tuples.
     w_dic = space.newdict()
     while 1:
-        w_key = u.get_w_obj(True)
+        w_key = u.get_w_obj(allow_null=True)
         if w_key is None:
             break
-        w_value = u.get_w_obj(False)
+        w_value = u.get_w_obj()
         space.setitem(w_dic, w_key, w_value)
     return w_dic
 register(TYPE_DICT, unmarshal_DictMulti)
@@ -364,7 +364,7 @@
 # so we no longer can handle it in interp_marshal.atom_strlist
 
 def unmarshal_str(u):
-    w_obj = u.get_w_obj(False)
+    w_obj = u.get_w_obj()
     try:
         return u.space.str_w(w_obj)
     except OperationError, e:
diff --git a/pypy/objspace/std/objecttype.py b/pypy/objspace/std/objecttype.py
--- a/pypy/objspace/std/objecttype.py
+++ b/pypy/objspace/std/objecttype.py
@@ -24,7 +24,12 @@
     return w_obj.getrepr(space, '%s object' % (classname,))
 
 def descr__str__(space, w_obj):
-    return space.repr(w_obj)
+    w_type = space.type(w_obj)
+    w_impl = w_type.lookup("__repr__")
+    if w_impl is None:
+        raise OperationError(space.w_TypeError,      # can it really occur?
+                             space.wrap("operand does not support unary str"))
+    return space.get_and_call_function(w_impl, w_obj)
 
 def descr__class__(space, w_obj):
     return space.type(w_obj)
diff --git a/pypy/objspace/std/objspace.py b/pypy/objspace/std/objspace.py
--- a/pypy/objspace/std/objspace.py
+++ b/pypy/objspace/std/objspace.py
@@ -129,12 +129,12 @@
         ec._py_repr = None
         return ec
 
-    def createframe(self, code, w_globals, closure=None):
+    def createframe(self, code, w_globals, outer_func=None):
         from pypy.objspace.std.fake import CPythonFakeCode, CPythonFakeFrame
         if not we_are_translated() and isinstance(code, CPythonFakeCode):
             return CPythonFakeFrame(self, code, w_globals)
         else:
-            return ObjSpace.createframe(self, code, w_globals, closure)
+            return ObjSpace.createframe(self, code, w_globals, outer_func)
 
     def gettypefor(self, cls):
         return self.gettypeobject(cls.typedef)
diff --git a/pypy/objspace/std/stringobject.py b/pypy/objspace/std/stringobject.py
--- a/pypy/objspace/std/stringobject.py
+++ b/pypy/objspace/std/stringobject.py
@@ -913,12 +913,16 @@
 def repr__String(space, w_str):
     s = w_str._value
 
-    buf = StringBuilder(len(s) + 2)
-
     quote = "'"
     if quote in s and '"' not in s:
         quote = '"'
 
+    return space.wrap(string_escape_encode(s, quote))
+
+def string_escape_encode(s, quote):
+
+    buf = StringBuilder(len(s) + 2)
+
     buf.append(quote)
     startslice = 0
 
@@ -959,7 +963,7 @@
 
     buf.append(quote)
 
-    return space.wrap(buf.build())
+    return buf.build()
 
 
 DEFAULT_NOOP_TABLE = ''.join([chr(i) for i in range(256)])
diff --git a/pypy/objspace/std/test/test_celldict.py b/pypy/objspace/std/test/test_celldict.py
--- a/pypy/objspace/std/test/test_celldict.py
+++ b/pypy/objspace/std/test/test_celldict.py
@@ -39,6 +39,20 @@
         assert d.getitem("a") is None
         assert d.strategy.getdictvalue_no_unwrapping(d, "a") is None
 
+    def test_same_key_set_twice(self):
+        strategy = ModuleDictStrategy(space)
+        storage = strategy.get_empty_storage()
+        d = W_DictMultiObject(space, strategy, storage)
+
+        v1 = strategy.version
+        x = object()
+        d.setitem("a", x)
+        v2 = strategy.version
+        assert v1 is not v2
+        d.setitem("a", x)
+        v3 = strategy.version
+        assert v2 is v3
+
 class AppTestModuleDict(object):
     def setup_class(cls):
         cls.space = gettestobjspace(**{"objspace.std.withcelldict": True})
diff --git a/pypy/objspace/std/test/test_floatobject.py b/pypy/objspace/std/test/test_floatobject.py
--- a/pypy/objspace/std/test/test_floatobject.py
+++ b/pypy/objspace/std/test/test_floatobject.py
@@ -767,3 +767,19 @@
 
     def test_invalid(self):
         raises(ValueError, float.fromhex, "0P")
+
+    def test_division_edgecases(self):
+        import math
+
+        # inf
+        inf = float("inf")
+        assert math.isnan(inf % 3)
+        assert math.isnan(inf // 3)
+        x, y = divmod(inf, 3)
+        assert math.isnan(x)
+        assert math.isnan(y)
+
+        # divide by 0
+        raises(ZeroDivisionError, lambda: inf % 0)
+        raises(ZeroDivisionError, lambda: inf // 0)
+        raises(ZeroDivisionError, divmod, inf, 0)
\ No newline at end of file
diff --git a/pypy/objspace/std/test/test_methodcache.py b/pypy/objspace/std/test/test_methodcache.py
--- a/pypy/objspace/std/test/test_methodcache.py
+++ b/pypy/objspace/std/test/test_methodcache.py
@@ -88,30 +88,37 @@
   
     def test_many_names(self):
         import __pypy__
-        class A(object):
-            foo = 5
-            bar = 6
-            baz = 7
-            xyz = 8
-            stuff = 9
-            a = 10
-            foobar = 11
+        for j in range(20):
+            class A(object):
+                foo = 5
+                bar = 6
+                baz = 7
+                xyz = 8
+                stuff = 9
+                a = 10
+                foobar = 11
 
-        a = A()
-        names = [name for name in A.__dict__.keys()
-                      if not name.startswith('_')]
-        names.sort()
-        names_repeated = names * 10
-        result = []
-        __pypy__.reset_method_cache_counter()
-        for name in names_repeated:
-            result.append(getattr(a, name))
-        append_counter = __pypy__.method_cache_counter("append")
-        names_counters = [__pypy__.method_cache_counter(name)
-                          for name in names]
-        assert append_counter[0] >= 5 * len(names)
-        for name, count in zip(names, names_counters):
-            assert count[0] >= 5, str((name, count))
+            a = A()
+            names = [name for name in A.__dict__.keys()
+                          if not name.startswith('_')]
+            names.sort()
+            names_repeated = names * 10
+            result = []
+            __pypy__.reset_method_cache_counter()
+            for name in names_repeated:
+                result.append(getattr(a, name))
+            append_counter = __pypy__.method_cache_counter("append")
+            names_counters = [__pypy__.method_cache_counter(name)
+                              for name in names]
+            try:
+                assert append_counter[0] >= 10 * len(names) - 1
+                for name, count in zip(names, names_counters):
+                    assert count == (9, 1), str((name, count))
+                break
+            except AssertionError:
+                pass
+        else:
+            raise
 
     def test_mutating_bases(self):
         class C(object):
@@ -134,20 +141,24 @@
 
     def test_custom_metaclass(self):
         import __pypy__
-        class MetaA(type):
-            def __getattribute__(self, x):
-                return 1
-        def f(self):
-            return 42
-        A = type.__new__(MetaA, "A", (), {"f": f})
-        l = [type.__getattribute__(A, "__new__")(A)] * 10
-        __pypy__.reset_method_cache_counter()
-        for i, a in enumerate(l):
-            assert a.f() == 42
-        cache_counter = __pypy__.method_cache_counter("f")
-        assert cache_counter[0] >= 5
-        assert cache_counter[1] >= 1 # should be (27, 3)
-        assert sum(cache_counter) == 10
+        for j in range(20):
+            class MetaA(type):
+                def __getattribute__(self, x):
+                    return 1
+            def f(self):
+                return 42
+            A = type.__new__(MetaA, "A", (), {"f": f})
+            l = [type.__getattribute__(A, "__new__")(A)] * 10
+            __pypy__.reset_method_cache_counter()
+            for i, a in enumerate(l):
+                assert a.f() == 42
+            cache_counter = __pypy__.method_cache_counter("f")
+            assert sum(cache_counter) == 10
+            if cache_counter == (9, 1):
+                break
+            #else the moon is misaligned, try again
+        else:
+            raise AssertionError("cache_counter = %r" % (cache_counter,))
 
     def test_mutate_class(self):
         import __pypy__
diff --git a/pypy/objspace/std/test/test_obj.py b/pypy/objspace/std/test/test_obj.py
--- a/pypy/objspace/std/test/test_obj.py
+++ b/pypy/objspace/std/test/test_obj.py
@@ -94,3 +94,11 @@
         #assert len(log) == 1
         #assert log[0].message.args == ("object.__init__() takes no parameters",)
         #assert type(log[0].message) is DeprecationWarning
+
+    def test_object_str(self):
+        # obscure case: __str__() must delegate to __repr__() without adding
+        # type checking on its own
+        class A(object):
+            def __repr__(self):
+                return 123456
+        assert A().__str__() == 123456
diff --git a/pypy/objspace/std/test/test_rangeobject.py b/pypy/objspace/std/test/test_rangeobject.py
--- a/pypy/objspace/std/test/test_rangeobject.py
+++ b/pypy/objspace/std/test/test_rangeobject.py
@@ -89,6 +89,9 @@
         assert not self.not_forced(r)
         r.sort()
         assert r == range(1, 100) + [999]
+        r = range(10)
+        r.sort(key=lambda x: -x)
+        assert r == range(9, -1, -1)
 
     def test_pop(self):
         r = range(10)
diff --git a/pypy/objspace/std/test/test_stringformat.py b/pypy/objspace/std/test/test_stringformat.py
--- a/pypy/objspace/std/test/test_stringformat.py
+++ b/pypy/objspace/std/test/test_stringformat.py
@@ -168,7 +168,7 @@
 
     def test_incomplete_format(self):
         raises(ValueError, '%'.__mod__, ((23,),))
-        raises(ValueError, '%('.__mod__, ({},))
+        raises((ValueError, TypeError), '%('.__mod__, ({},))
 
     def test_format_char(self):
         import sys
diff --git a/pypy/objspace/std/test/test_unicodeobject.py b/pypy/objspace/std/test/test_unicodeobject.py
--- a/pypy/objspace/std/test/test_unicodeobject.py
+++ b/pypy/objspace/std/test/test_unicodeobject.py
@@ -780,8 +780,22 @@
         assert type(s) is unicode
         assert s == u'\u1234'
 
+        # now the same with a new-style class...
+        class A(object):
+            def __init__(self, num):
+                self.num = num
+            def __str__(self):
+                return unichr(self.num)
+
+        s = '%s' % A(111)    # this is ASCII
+        assert type(s) is unicode
+        assert s == chr(111)
+
+        s = '%s' % A(0x1234)    # this is not ASCII
+        assert type(s) is unicode
+        assert s == u'\u1234'
+
     def test_formatting_unicode__str__2(self):
-        skip("this is completely insane")
         class A:
             def __str__(self):
                 return u'baz'
@@ -798,9 +812,22 @@
         s = '%s %s' % (a, b)
         assert s == u'baz bar'
 
+        skip("but this case here is completely insane")
         s = '%s %s' % (b, a)
         assert s == u'foo baz'
 
+    def test_formatting_unicode__str__3(self):
+        # "bah" is all I can say
+        class X(object):
+            def __repr__(self):
+                return u'\u1234'
+        '%s' % X()
+        #
+        class X(object):
+            def __str__(self):
+                return u'\u1234'
+        '%s' % X()
+
     def test_str_subclass(self):
         class Foo9(str):
             def __unicode__(self):
diff --git a/pypy/rlib/clibffi.py b/pypy/rlib/clibffi.py
--- a/pypy/rlib/clibffi.py
+++ b/pypy/rlib/clibffi.py
@@ -286,10 +286,10 @@
 
 FFI_OK = cConfig.FFI_OK
 FFI_BAD_TYPEDEF = cConfig.FFI_BAD_TYPEDEF
-FFI_DEFAULT_ABI = rffi.cast(rffi.USHORT, cConfig.FFI_DEFAULT_ABI)
+FFI_DEFAULT_ABI = cConfig.FFI_DEFAULT_ABI
 if _WIN32:
-    FFI_STDCALL = rffi.cast(rffi.USHORT, cConfig.FFI_STDCALL)
-FFI_TYPE_STRUCT = rffi.cast(rffi.USHORT, cConfig.FFI_TYPE_STRUCT)
+    FFI_STDCALL = cConfig.FFI_STDCALL
+FFI_TYPE_STRUCT = cConfig.FFI_TYPE_STRUCT
 FFI_CIFP = rffi.COpaquePtr('ffi_cif', compilation_info=eci)
 
 FFI_CLOSUREP = lltype.Ptr(cConfig.ffi_closure)
@@ -319,7 +319,7 @@
        which the 'ffistruct' member is a regular FFI_TYPE.
     """
     tpe = lltype.malloc(FFI_STRUCT_P.TO, len(field_types)+1, flavor='raw')
-    tpe.ffistruct.c_type = FFI_TYPE_STRUCT
+    tpe.ffistruct.c_type = rffi.cast(rffi.USHORT, FFI_TYPE_STRUCT)
     tpe.ffistruct.c_size = rffi.cast(rffi.SIZE_T, size)
     tpe.ffistruct.c_alignment = rffi.cast(rffi.USHORT, aligment)
     tpe.ffistruct.c_elements = rffi.cast(FFI_TYPE_PP,
@@ -402,12 +402,20 @@
 
 closureHeap = ClosureHeap()
 
-FUNCFLAG_STDCALL   = 0
-FUNCFLAG_CDECL     = 1  # for WINAPI calls
+FUNCFLAG_STDCALL   = 0    # on Windows: for WINAPI calls
+FUNCFLAG_CDECL     = 1    # on Windows: for __cdecl calls
 FUNCFLAG_PYTHONAPI = 4
 FUNCFLAG_USE_ERRNO = 8
 FUNCFLAG_USE_LASTERROR = 16
 
+def get_call_conv(flags, from_jit):
+    if _WIN32 and (flags & FUNCFLAG_CDECL == 0):
+        return FFI_STDCALL
+    else:
+        return FFI_DEFAULT_ABI
+get_call_conv._annspecialcase_ = 'specialize:arg(1)'     # hack :-/
+
+
 class AbstractFuncPtr(object):
     ll_cif = lltype.nullptr(FFI_CIFP.TO)
     ll_argtypes = lltype.nullptr(FFI_TYPE_PP.TO)
@@ -427,21 +435,17 @@
         self.ll_cif = lltype.malloc(FFI_CIFP.TO, flavor='raw',
                                     track_allocation=False) # freed by the __del__
 
-        if _WIN32 and (flags & FUNCFLAG_CDECL == 0):
-            cc = FFI_STDCALL
-        else:
-            cc = FFI_DEFAULT_ABI
-
         if _MSVC:
             # This little trick works correctly with MSVC.
             # It returns small structures in registers
-            if r_uint(restype.c_type) == FFI_TYPE_STRUCT:
+            if intmask(restype.c_type) == FFI_TYPE_STRUCT:
                 if restype.c_size <= 4:
                     restype = ffi_type_sint32
                 elif restype.c_size <= 8:
                     restype = ffi_type_sint64
 
-        res = c_ffi_prep_cif(self.ll_cif, cc,
+        res = c_ffi_prep_cif(self.ll_cif,
+                             rffi.cast(rffi.USHORT, get_call_conv(flags,False)),
                              rffi.cast(rffi.UINT, argnum), restype,
                              self.ll_argtypes)
         if not res == FFI_OK:
diff --git a/pypy/rlib/libffi.py b/pypy/rlib/libffi.py
--- a/pypy/rlib/libffi.py
+++ b/pypy/rlib/libffi.py
@@ -75,7 +75,7 @@
     @staticmethod
     @jit.elidable
     def is_struct(ffi_type):
-        return intmask(ffi_type.c_type) == intmask(FFI_TYPE_STRUCT)
+        return intmask(ffi_type.c_type) == FFI_TYPE_STRUCT
 
 types._import()
 
@@ -206,6 +206,7 @@
     _immutable_fields_ = ['funcsym']
     argtypes = []
     restype = lltype.nullptr(clibffi.FFI_TYPE_P.TO)
+    flags = 0
     funcsym = lltype.nullptr(rffi.VOIDP.TO)
 
     def __init__(self, name, argtypes, restype, funcsym, flags=FUNCFLAG_CDECL,
diff --git a/pypy/rlib/parsing/tree.py b/pypy/rlib/parsing/tree.py
--- a/pypy/rlib/parsing/tree.py
+++ b/pypy/rlib/parsing/tree.py
@@ -6,9 +6,16 @@
         content = ["digraph G{"]
         content.extend(self.dot())
         content.append("}")
-        p = py.test.ensuretemp("automaton").join("temp.dot")
+        try:
+            p = py.test.ensuretemp("automaton").join("temp.dot")
+            remove = False
+        except AttributeError: # pytest lacks ensuretemp, make a normal one
+            p = py.path.local.mkdtemp().join('automaton.dot')
+            remove = True
         p.write("\n".join(content))
         graphclient.display_dot_file(str(p))
+        if remove:
+            p.dirpath().remove()
 
 class Symbol(Node):
 
diff --git a/pypy/rlib/rcoroutine.py b/pypy/rlib/rcoroutine.py
deleted file mode 100644
--- a/pypy/rlib/rcoroutine.py
+++ /dev/null
@@ -1,357 +0,0 @@
-"""
-Basic Concept:
---------------
-
-All concurrency is expressed by some means of coroutines.
-This is the lowest possible exposable interface.
-
-A coroutine is a structure that controls a sequence
-of continuations in time. It contains a frame object
-that is a restartable stack chain. This frame object
-is updated on every switch.
-
-The frame can be None. Either the coroutine is not yet
-bound, or it is the current coroutine of some costate.
-See below. XXX rewrite a definition of these terms.
-
-There is always a notation of a "current" and a "last"
-coroutine. Current has no frame and represents the
-running program. last is needed to keep track of the
-coroutine that receives a new frame chain after a switch.
-
-A costate object holds last and current.
-There are different coroutine concepts existing in
-parallel, like plain interp-level coroutines and
-app-level structures like coroutines, greenlets and
-tasklets.
-Every concept is associated with its own costate object.
-This allows for peaceful co-existence of many concepts.
-The type of a switch is determined by the target's costate.
-"""
-
-import py; py.test.skip("fixme: rewrite using rlib.rstacklet")
-# XXX ^^^ the reason it is not done is that pypy.rlib.rcoroutine
-# plus pypy/module/_stackless look like faaaaaar too much code
-# to me :-(
-
-from pypy.rlib.rstack import yield_current_frame_to_caller
-from pypy.rlib.objectmodel import we_are_translated
-
-from pypy.interpreter.error import OperationError
-
-try:
-    from greenlet import greenlet
-    main_greenlet = greenlet.getcurrent()
-except (ImportError, ValueError):
-    def greenlet(*args, **kwargs):
-        raise NotImplementedError("need either greenlets or a translated version of pypy")
-
-class FrameChain(object):
-    """Greenlet-based emulation of the primitive rstack 'frames' of RPython"""
-
-    def __init__(self, thunk=None):
-        if thunk:
-            self.greenlet = greenlet(thunk)
-        else:
-            self.greenlet = greenlet.getcurrent()
-
-    def switch(self):
-        last = FrameChain()
-        return self.greenlet.switch(last)
-
-import sys, os
-
-def make_coroutine_classes(baseclass):
-    class BaseCoState(object):
-        def __init__(self):
-            self.current = self.main = None
-
-        def __repr__(self):
-            "NOT_RPYTHON"
-            # for debugging only
-            return '<%s current=%r>' % (self.__class__.__name__, self.current)
-
-        def update(self, new):
-            syncstate.leaving = self.current
-            syncstate.entering = new
-            self.current = new
-            frame, new.frame = new.frame, None
-            return frame
-
-
-    class CoState(BaseCoState):
-        def __init__(self):
-            BaseCoState.__init__(self)
-            self.current = self.main = Coroutine(self)
-
-    class CoroutineDamage(SystemError):
-        pass
-
-
-    class SyncState(object):
-        def __init__(self):
-            self.reset()
-
-        def reset(self):
-            self.default_costate = None
-            self.leaving = None
-            self.entering = None
-            self.things_to_do = False
-            self.temp_exc = None
-            self.to_delete = []
-
-        def switched(self, incoming_frame):
-            left = syncstate.leaving
-            entered = syncstate.entering
-            syncstate.leaving = syncstate.entering = None
-            if left is not None:   # mostly to work around an annotation problem;
-                                   # should not really be None
-                left.frame = incoming_frame
-                left.goodbye()
-            if entered is not None:
-                entered.hello()
-            if self.things_to_do:
-                self._do_things_to_do()
-
-        def push_exception(self, exc):
-            self.things_to_do = True
-            self.temp_exc = exc
-
-        def check_for_zombie(self, obj):
-            return obj in self.to_delete
-
-        def postpone_deletion(self, obj):
-            self.to_delete.append(obj)
-            self.things_to_do = True
-
-        def _do_things_to_do(self):
-            if self.temp_exc is not None:
-                # somebody left an unhandled exception and switched to us.
-                # this both provides default exception handling and the
-                # way to inject an exception, like CoroutineExit.
-                e, self.temp_exc = self.temp_exc, None
-                self.things_to_do = bool(self.to_delete)
-                raise e
-            while self.to_delete:
-                delete, self.to_delete = self.to_delete, []
-                for obj in delete:
-                    obj.parent = obj.costate.current
-                    obj._kill_finally()
-            else:
-                self.things_to_do = False
-
-        def _freeze_(self):
-            self.reset()
-            return False
-
-    syncstate = SyncState()
-
-
-    class CoroutineExit(SystemExit):
-        # XXX SystemExit's __init__ creates problems in bookkeeper.
-        def __init__(self):
-            pass
-
-    class AbstractThunk(object):
-        def call(self):
-            raise NotImplementedError("abstract base class")
-
-
-    class Coroutine(baseclass):
-        def __init__(self, state=None):
-            self.frame = None
-            if state is None:
-                state = self._get_default_costate()
-            self.costate = state
-            self.parent = None
-            self.thunk = None
-            self.coroutine_exit = False
-
-        def __repr__(self):
-            'NOT_RPYTHON'
-            # just for debugging
-            if hasattr(self, '__name__'):
-                return '' % (self.__name__, self.frame, self.thunk is not None)
-            else:
-                return '' % (self.frame, self.thunk is not None)
-
-        def _get_default_costate():
-            state = syncstate.default_costate
-            if state is None:
-                state = syncstate.default_costate = CoState()
-            return state
-        _get_default_costate = staticmethod(_get_default_costate)
-
-        def _get_default_parent(self):
-            return self.costate.current
-
-        def bind(self, thunk):
-            assert isinstance(thunk, AbstractThunk)
-            if self.frame is not None:
-                raise CoroutineDamage
-            if self.parent is None:
-                self.parent = self._get_default_parent()
-            assert self.parent is not None
-            self.thunk = thunk
-            if we_are_translated():
-                self.frame = self._bind()
-            else:
-                self.frame = self._greenlet_bind()
-
-        def _greenlet_bind(self):
-            weak = [self]
-            def _greenlet_execute(incoming_frame):
-                try:
-                    chain2go2next = weak[0]._execute(incoming_frame)
-                except:
-                    # no exception is supposed to get out of _execute()
-                    # better report it directly into the main greenlet then,
-                    # and hidden to prevent catching
-                    main_greenlet.throw(AssertionError(
-                        "unexpected exception out of Coroutine._execute()",
-                        *sys.exc_info()))
-                    assert 0
-                del weak[0]
-                greenlet.getcurrent().parent = chain2go2next.greenlet
-                return None   # as the result of the FrameChain.switch()
-            chain = FrameChain(_greenlet_execute)
-            return chain
-
-        def _bind(self):
-            state = self.costate
-            incoming_frame = yield_current_frame_to_caller()
-            self = state.current
-            return self._execute(incoming_frame)
-
-        def _execute(self, incoming_frame):
-            state = self.costate
-            try:
-                try:
-                    try:
-                        exc = None
-                        thunk = self.thunk
-                        self.thunk = None
-                        syncstate.switched(incoming_frame)
-                        thunk.call()
-                    except Exception, e:
-                        exc = e
-                        raise
-                finally:
-                    # warning! we must reload the 'self' from the costate,
-                    # because after a clone() the 'self' of both copies
-                    # point to the original!
-                    self = state.current
-                    self.finish(exc)
-            except CoroutineExit:
-                pass
-            except Exception, e:
-                if self.coroutine_exit is False:
-                    # redirect all unhandled exceptions to the parent
-                    syncstate.push_exception(e)
-
-            while self.parent is not None and self.parent.frame is None:
-                # greenlet behavior is fine
-                self.parent = self.parent.parent
-            return state.update(self.parent)
-
-        def switch(self):
-            if self.frame is None:
-                # considered a programming error.
-                # greenlets and tasklets have different ideas about this.
-                raise CoroutineDamage
-            state = self.costate
-            incoming_frame = state.update(self).switch()
-            syncstate.switched(incoming_frame)
-
-        def kill(self):
-            self._kill(CoroutineExit())
-
-        def _kill(self, exc):
-            if self.frame is None:
-                return
-            state = self.costate
-            syncstate.push_exception(exc)
-            # careful here - if setting self.parent to state.current would
-            # create a loop, break it.  The assumption is that 'self'
-            # will die, so that state.current's chain of parents can be
-            # modified to skip 'self' without too many people noticing.
-            p = state.current
-            if p is self or self.parent is None:
-                pass  # killing the current of the main - don't change any parent
-            else:
-                while p.parent is not None:
-                    if p.parent is self:
-                        p.parent = self.parent
-                        break
-                    p = p.parent
-                self.parent = state.current
-            self.switch()
-
-        def _kill_finally(self):
-            try:
-                self._userdel()
-            except Exception:
-                pass # maybe print a warning?
-            self.kill()
-
-        __already_postponed = False
-    
-        def __del__(self):
-            # provide the necessary clean-up
-            # note that AppCoroutine has to take care about this
-            # as well, including a check for user-supplied __del__.
-            # Additionally note that in the context of __del__, we are
-            # not in the position to issue a switch.
-            # we defer it completely.
-            
-            # it is necessary to check whether syncstate is None because CPython
-            # sets it to None when it cleans up the modules, which will lead to
-            # very strange effects
-
-            if not we_are_translated():
-                # we need to make sure that we postpone each coroutine only once on
-                # top of CPython, because this resurrects the coroutine and CPython
-                # calls __del__ again, thus postponing and resurrecting the
-                # coroutine once more :-(
-                if self.__already_postponed:
-                    return
-                self.__already_postponed = True
-            if syncstate is not None:
-                syncstate.postpone_deletion(self)
-
-        # coroutines need complete control over their __del__ behaviour. In
-        # particular they need to care about calling space.userdel themselves
-        handle_del_manually = True
-
-        def _userdel(self):
-            # override this for exposed coros
-            pass
-
-        def is_alive(self):
-            return self.frame is not None or self is self.costate.current
-
-        def is_zombie(self):
-            return self.frame is not None and syncstate.check_for_zombie(self)
-
-        def getcurrent():
-            costate = Coroutine._get_default_costate()
-            return costate.current
-        getcurrent = staticmethod(getcurrent)
-
-        def getmain():
-            costate = Coroutine._get_default_costate()
-            return costate.main
-        getmain = staticmethod(getmain)
-
-        def hello(self):
-            "Called when execution is transferred into this coroutine."
-
-        def goodbye(self):
-            "Called just after execution is transferred away from this coroutine."
-
-        def finish(self, exc=None):
-            "stephan forgot me"
-
-    return locals()
-
-# _________________________________________________
diff --git a/pypy/rlib/runicode.py b/pypy/rlib/runicode.py
--- a/pypy/rlib/runicode.py
+++ b/pypy/rlib/runicode.py
@@ -1403,7 +1403,7 @@
                                     s, pos, pos + unicode_bytes)
             result.append(res)
             continue
-        result.append(unichr(t))
+        result.append(UNICHR(t))
         pos += unicode_bytes
     return result.build(), pos
 
diff --git a/pypy/rlib/test/test_rcoroutine.py b/pypy/rlib/test/test_rcoroutine.py
deleted file mode 100644
--- a/pypy/rlib/test/test_rcoroutine.py
+++ /dev/null
@@ -1,348 +0,0 @@
-"""
-testing coroutines at interprepter level
-"""
-import py
-import os
-from pypy import conftest; conftest.translation_test_so_skip_if_appdirect()
-from pypy.rlib.rcoroutine import make_coroutine_classes
-from pypy.translator.c.test.test_stackless import StacklessTest
-from pypy.translator.c import gc
-
-def setup_module(mod):
-    py.test.importorskip('greenlet')
-
-d = make_coroutine_classes(object)
-syncstate = d['syncstate']
-Coroutine = d['Coroutine']
-AbstractThunk = d['AbstractThunk']
-
-def output(stuff):
-    os.write(2, stuff + '\n')
-
-class _TestCoroutine(StacklessTest):
-    backendopt = True
-    Coroutine = Coroutine
-
-    def setup_method(self, method):
-        syncstate.reset()
-
-    def _freeze_(self):    # for 'self.Coroutine'
-        return True
-
-    def test_coroutine1(self):
-
-        def g(lst, coros):
-            coro_f, coro_g, coro_h = coros
-            lst.append(2)
-            output('g appended 2')
-            coro_h.switch()
-            lst.append(5)
-            output('g appended 5')
-
-        def h(lst, coros):
-            coro_f, coro_g, coro_h = coros
-            lst.append(3)
-            output('h appended 3')
-            coro_f.switch()
-            lst.append(7)
-            output('h appended 7')
-
-        class T(AbstractThunk):
-            def __init__(self, func, arg1, arg2):
-                self.func = func
-                self.arg1 = arg1
-                self.arg2 = arg2
-            def call(self):
-                self.func(self.arg1, self.arg2)
-
-        def f():
-            lst = [1]
-            coro_f = Coroutine.getcurrent()
-            coro_g = self.Coroutine()
-            coro_h = self.Coroutine()
-            coros = [coro_f, coro_g, coro_h]
-            thunk_g = T(g, lst, coros)
-            output('binding g after f set 1')
-            coro_g.bind(thunk_g)
-            thunk_h = T(h, lst, coros)
-            output('binding h after f set 1')
-            coro_h.bind(thunk_h)
-            output('switching to g')
-            coro_g.switch()
-            lst.append(4)
-            output('f appended 4')
-            coro_g.switch()
-            lst.append(6)
-            output('f appended 6')
-            coro_h.switch()
-            lst.append(8)
-            output('f appended 8')
-            n = 0
-            for i in lst:
-                n = n*10 + i
-            return n
-
-        data = self.wrap_stackless_function(f)
-        assert data == 12345678
-
-    def test_coroutine2(self):
-
-        class TBase(AbstractThunk):
-            def call(self):
-                pass
-
-        class T(TBase):
-            def __init__(self, func, arg1, arg2):
-                self.func = func
-                self.arg1 = arg1
-                self.arg2 = arg2
-            def call(self):
-                self.res = self.func(self.arg1, self.arg2)
-
-        class T1(TBase):
-            def __init__(self, func, arg1):
-                self.func = func
-                self.arg1 = arg1
-            def call(self):
-                self.res = self.func(self.arg1)
-
-        def g(lst, coros):
-            coro_f1, coro_g, coro_h = coros
-            lst.append(2)
-            output('g appended 2')
-            coro_h.switch()
-            lst.append(5)
-            output('g appended 5')
-            output('exiting g')
-
-        def h(lst, coros):
-            coro_f1, coro_g, coro_h = coros
-            lst.append(3)
-            output('h appended 3')
-            coro_f1.switch()
-            lst.append(7)
-            output('h appended 7')
-            output('exiting h')
-
-        def f1(coro_f1):
-            lst = [1]
-            coro_g = self.Coroutine()
-            coro_g.__name__ = 'coro_g'
-            coro_h = self.Coroutine()
-            coro_h.__name__ = 'coro_h'
-            coros = [coro_f1, coro_g, coro_h]
-            thunk_g = T(g, lst, coros)
-            output('binding g after f1 set 1')
-            coro_g.bind(thunk_g)
-            thunk_h = T(h, lst, coros)
-            output('binding h after f1 set 1')
-            coro_h.bind(thunk_h)
-            output('switching to g')
-            coro_g.switch()
-            lst.append(4)
-            output('f1 appended 4')
-            coro_g.switch()
-            lst.append(6)
-            output('f1 appended 6')
-            coro_h.switch()
-            lst.append(8)
-            output('f1 appended 8')
-            n = 0
-            for i in lst:
-                n = n*10 + i
-            output('exiting f1')
-            return n     
-
-        def f():
-            coro_f = Coroutine.getcurrent()
-            coro_f.__name__ = 'coro_f'
-            coro_f1 = self.Coroutine()
-            coro_f1.__name__ = 'coro_f1'
-            thunk_f1 = T1(f1, coro_f1)
-            output('binding f1 after f set 1')
-            coro_f1.bind(thunk_f1)
-            coro_f1.switch()
-            output('return to main :-(')
-            return thunk_f1.res
-
-        data = self.wrap_stackless_function(f)
-        assert data == 12345678
-
-    def test_kill_raise_del_coro(self):
-        class T(AbstractThunk):
-            def __init__(self, func, arg):
-                self.func = func
-                self.arg = arg
-            def call(self):
-                self.func(self.arg, self)
-
-        def g(nrec, t, count=0):
-            t.count = count
-            if nrec < 0:
-                raise ValueError
-            if nrec:
-                g(nrec-1, t, count+1)
-            Coroutine.getmain().switch()
-
-        def f():
-            assert Coroutine.getmain().frame is None
-            coro_g = self.Coroutine()
-            coro_g.__name__ = 'coro_g'
-            thunk_g = T(g, 42)
-            coro_g.bind(thunk_g)
-            coro_g.switch()
-            res = thunk_g.count
-            res *= 10
-            res |= coro_g.frame is not None
-            # testing kill
-            coro_g.kill()
-            res *= 10
-            res |= coro_g.frame is None
-            coro_g = self.Coroutine()
-            # see what happens if we __del__
-            thunk_g = T(g, -42)
-            coro_g.bind(thunk_g)
-            try:
-                coro_g.switch()
-            except ValueError:
-                res += 500
-            return res
-
-        data = self.wrap_stackless_function(f)
-        assert data == 4711
-
-    def test_tree_compare(self):
-        class Node:
-            def __init__(self, value, left=None, right=None):
-                self.value = value
-                self.left = left
-                self.right = right
-            def __repr__(self):
-                return 'Node(%r, %r, %r)'%(self.value, self.left, self.right)
-
-        tree1 = Node(1, Node(2, Node(3)))
-        tree2 = Node(1, Node(3, Node(2)))
-        tree3 = Node(1, Node(2), Node(3))
-
-        class Producer(AbstractThunk):
-            def __init__(self, tree, objects, consumer):
-                self.tree = tree
-                self.objects = objects
-                self.consumer = consumer
-            def produce(self, t):
-                if t is None:
-                    return
-                self.objects.append(t.value)
-                self.consumer.switch()
-                self.produce(t.left)
-                self.produce(t.right)
-            def call(self):
-                self.produce(self.tree)
-                while 1:
-                    self.consumer.switch()
-        class Consumer(AbstractThunk):
-            def __init__(self, tree, objects, producer):
-                self.tree = tree
-                self.objects = objects
-                self.producer = producer
-            def consume(self, t):
-                if t is None:
-                    return True
-                self.producer.switch()
-                if not self.objects:
-                    return False
-                if self.objects.pop(0) != t.value:
-                    return False
-                if not self.consume(t.left):
-                    return False
-                return self.consume(t.right)
-
-            def call(self):
-                self.result = self.consume(self.tree)
-                Coroutine.getmain().switch()
-
-        def pre_order_eq(t1, t2):
-            objects = []
-            producer = self.Coroutine()
-            consumer = self.Coroutine()
-
-            producer.bind(Producer(t1, objects, consumer))
-            cons = Consumer(t2, objects, producer)
-            consumer.bind(cons)
-
-            consumer.switch()
-
-            return cons.result
-
-        def ep():
-            return int("%d%d%d%d"%(pre_order_eq(tree1, tree2),
-                                   pre_order_eq(tree1, tree1),
-                                   pre_order_eq(tree1, tree3),
-                                   pre_order_eq(tree2, tree1),
-                                   ))
-
-        output = self.wrap_stackless_function(ep)
-        assert output == int('0110')
-
-    def test_hello_goodbye(self):
-
-        class C(Coroutine):
-            n = 2
-            def __init__(self, n):
-                Coroutine.__init__(self)
-                self.n = n
-            def hello(self):
-                costate.hello_goodbye *= 10
-                costate.hello_goodbye += self.n
-            def goodbye(self):
-                costate.hello_goodbye *= 10
-                costate.hello_goodbye += self.n + 1
-
-        class T(AbstractThunk):
-            def call(self):
-                pass
-
-        costate = Coroutine._get_default_costate()
-        costate.current.__class__ = C
-        costate.hello_goodbye = 0
-
-        def ep():
-            syncstate.default_costate = costate
-            costate.hello_goodbye = 0
-            c1 = C(4)
-            c1.bind(T())
-            c1.switch()
-            return costate.hello_goodbye
-
-        output = self.wrap_stackless_function(ep)
-        # expected result:
-        #   goodbye main   3
-        #   hello   c1     4
-        #   goodbye c1     5
-        #   hello   main   2
-        assert output == 3452
-
-    def test_raise_propagate(self):
-        class T(AbstractThunk):
-            def call(self):
-                raise ValueError
-
-        def ep():
-            c = self.Coroutine()
-            c.bind(T())
-            try:
-                c.switch()
-            except ValueError:
-                return 100
-            else:
-                return -5
-
-        output = self.wrap_stackless_function(ep)
-        assert output == 100
-
-
-TestCoroutine = _TestCoroutine # to activate
-class TestCoroutineOnCPython(_TestCoroutine):
-    def wrap_stackless_function(self, func):
-        return func()
-
diff --git a/pypy/rpython/lltypesystem/ll2ctypes.py b/pypy/rpython/lltypesystem/ll2ctypes.py
--- a/pypy/rpython/lltypesystem/ll2ctypes.py
+++ b/pypy/rpython/lltypesystem/ll2ctypes.py
@@ -113,7 +113,7 @@
         rffi.LONGLONG:   ctypes.c_longlong,
         rffi.ULONGLONG:  ctypes.c_ulonglong,
         rffi.SIZE_T:     ctypes.c_size_t,
-        lltype.Bool:     ctypes.c_bool,
+        lltype.Bool:     getattr(ctypes, "c_bool", ctypes.c_long),
         llmemory.Address:  ctypes.c_void_p,
         llmemory.GCREF:    ctypes.c_void_p,
         llmemory.WeakRef:  ctypes.c_void_p, # XXX
@@ -1153,7 +1153,11 @@
         # an OverflowError on the following line.
         cvalue = ctypes.cast(ctypes.c_void_p(cvalue), cresulttype)
     else:
-        cvalue = cresulttype(cvalue).value   # mask high bits off if needed
+        try:
+            cvalue = cresulttype(cvalue).value   # mask high bits off if needed
+        except TypeError:
+            cvalue = int(cvalue)   # float -> int
+            cvalue = cresulttype(cvalue).value   # try again
     return ctypes2lltype(RESTYPE, cvalue)
 
 class ForceCastEntry(ExtRegistryEntry):
diff --git a/pypy/rpython/lltypesystem/lltype.py b/pypy/rpython/lltypesystem/lltype.py
--- a/pypy/rpython/lltypesystem/lltype.py
+++ b/pypy/rpython/lltypesystem/lltype.py
@@ -1283,6 +1283,8 @@
         try:
             return p._obj._hash_cache_
         except AttributeError:
+            assert self._T._gckind == 'gc'
+            assert self      # not for NULL
             result = hash(p._obj)
             if cache:
                 try:
diff --git a/pypy/rpython/lltypesystem/test/test_rffi.py b/pypy/rpython/lltypesystem/test/test_rffi.py
--- a/pypy/rpython/lltypesystem/test/test_rffi.py
+++ b/pypy/rpython/lltypesystem/test/test_rffi.py
@@ -699,7 +699,10 @@
     def test_cast(self):
         res = cast(SIZE_T, -1)
         assert type(res) is r_size_t
-        assert res == r_size_t(-1)    
+        assert res == r_size_t(-1)
+        #
+        res = cast(lltype.Signed, 42.5)
+        assert res == 42
     
     def test_rffi_sizeof(self):
         try:
diff --git a/pypy/rpython/memory/gc/minimark.py b/pypy/rpython/memory/gc/minimark.py
--- a/pypy/rpython/memory/gc/minimark.py
+++ b/pypy/rpython/memory/gc/minimark.py
@@ -1461,6 +1461,7 @@
         # We will fix such references to point to the copy of the young
         # objects when we walk 'old_objects_pointing_to_young'.
         self.old_objects_pointing_to_young.append(newobj)
+    _trace_drag_out._always_inline_ = True
 
     def _visit_young_rawmalloced_object(self, obj):
         # 'obj' points to a young, raw-malloced object.
diff --git a/pypy/rpython/memory/gctypelayout.py b/pypy/rpython/memory/gctypelayout.py
--- a/pypy/rpython/memory/gctypelayout.py
+++ b/pypy/rpython/memory/gctypelayout.py
@@ -459,7 +459,7 @@
             if t._hints.get('immutable'):
                 return
             if 'immutable_fields' in t._hints:
-                skip = t._hints['immutable_fields'].fields
+                skip = t._hints['immutable_fields'].all_immutable_fields()
         for n, t2 in t._flds.iteritems():
             if isinstance(t2, lltype.Ptr) and t2.TO._gckind == 'gc':
                 if n not in skip:
diff --git a/pypy/rpython/memory/test/test_gctypelayout.py b/pypy/rpython/memory/test/test_gctypelayout.py
--- a/pypy/rpython/memory/test/test_gctypelayout.py
+++ b/pypy/rpython/memory/test/test_gctypelayout.py
@@ -4,7 +4,7 @@
 from pypy.rpython.memory.gctypelayout import gc_pointers_inside
 from pypy.rpython.lltypesystem import lltype, llmemory, rclass
 from pypy.rpython.test.test_llinterp import get_interpreter
-from pypy.rpython.rclass import IR_IMMUTABLE
+from pypy.rpython.rclass import IR_IMMUTABLE, IR_QUASIIMMUTABLE
 from pypy.objspace.flow.model import Constant
 
 class FakeGC:
@@ -102,7 +102,7 @@
     accessor = rclass.FieldListAccessor()
     S3 = lltype.GcStruct('S', ('x', PT), ('y', PT),
                          hints={'immutable_fields': accessor})
-    accessor.initialize(S3, {'x': IR_IMMUTABLE})
+    accessor.initialize(S3, {'x': IR_IMMUTABLE, 'y': IR_QUASIIMMUTABLE})
     #
     s1 = lltype.malloc(S1)
     adr = llmemory.cast_ptr_to_adr(s1)
diff --git a/pypy/rpython/module/ll_os_stat.py b/pypy/rpython/module/ll_os_stat.py
--- a/pypy/rpython/module/ll_os_stat.py
+++ b/pypy/rpython/module/ll_os_stat.py
@@ -49,19 +49,8 @@
     ]
 N_INDEXABLE_FIELDS = 10
 
-# for now, check the host Python to know which st_xxx fields exist
-STAT_FIELDS = [(_name, _TYPE) for (_name, _TYPE) in ALL_STAT_FIELDS
-                              if hasattr(os.stat_result, _name)]
-
-STAT_FIELD_TYPES = dict(STAT_FIELDS)      # {'st_xxx': TYPE}
-
-STAT_FIELD_NAMES = [_name for (_name, _TYPE) in ALL_STAT_FIELDS
-                          if _name in STAT_FIELD_TYPES]
-
-del _name, _TYPE
-
 # For OO backends, expose only the portable fields (the first 10).
-PORTABLE_STAT_FIELDS = STAT_FIELDS[:N_INDEXABLE_FIELDS]
+PORTABLE_STAT_FIELDS = ALL_STAT_FIELDS[:N_INDEXABLE_FIELDS]
 
 # ____________________________________________________________
 #
@@ -142,17 +131,22 @@
     includes = INCLUDES
 )
 
-if sys.platform != 'win32':
+if TIMESPEC is not None:
+    class CConfig_for_timespec:
+        _compilation_info_ = compilation_info
+        TIMESPEC = TIMESPEC
+    TIMESPEC = lltype.Ptr(
+        platform.configure(CConfig_for_timespec)['TIMESPEC'])
+
+
+def posix_declaration(try_to_add=None):
+    global STAT_STRUCT
 
     LL_STAT_FIELDS = STAT_FIELDS[:]
+    if try_to_add:
+        LL_STAT_FIELDS.append(try_to_add)
 
     if TIMESPEC is not None:
-        class CConfig_for_timespec:
-            _compilation_info_ = compilation_info
-            TIMESPEC = TIMESPEC
-
-        TIMESPEC = lltype.Ptr(
-            platform.configure(CConfig_for_timespec)['TIMESPEC'])
 
         def _expand(lst, originalname, timespecname):
             for i, (_name, _TYPE) in enumerate(lst):
@@ -178,9 +172,34 @@
     class CConfig:
         _compilation_info_ = compilation_info
         STAT_STRUCT = platform.Struct('struct %s' % _name_struct_stat, LL_STAT_FIELDS)
-    config = platform.configure(CConfig)
+    try:
+        config = platform.configure(CConfig)
+    except platform.CompilationError:
+        if try_to_add:
+            return    # failed to add this field, give up
+        raise
 
     STAT_STRUCT = lltype.Ptr(config['STAT_STRUCT'])
+    if try_to_add:
+        STAT_FIELDS.append(try_to_add)
+
+
+# This lists only the fields that have been found on the underlying platform.
+# Initially only the PORTABLE_STAT_FIELDS, but more may be added by the
+# following loop.
+STAT_FIELDS = PORTABLE_STAT_FIELDS[:]
+
+if sys.platform != 'win32':
+    posix_declaration()
+    for _i in range(len(PORTABLE_STAT_FIELDS), len(ALL_STAT_FIELDS)):
+        posix_declaration(ALL_STAT_FIELDS[_i])
+    del _i
+
+# these two global vars only list the fields defined in the underlying platform
+STAT_FIELD_TYPES = dict(STAT_FIELDS)      # {'st_xxx': TYPE}
+STAT_FIELD_NAMES = [_name for (_name, _TYPE) in STAT_FIELDS]
+del _name, _TYPE
+
 
 def build_stat_result(st):
     # only for LL backends
diff --git a/pypy/rpython/module/test/test_ll_os_stat.py b/pypy/rpython/module/test/test_ll_os_stat.py
--- a/pypy/rpython/module/test/test_ll_os_stat.py
+++ b/pypy/rpython/module/test/test_ll_os_stat.py
@@ -2,6 +2,16 @@
 import sys, os
 import py
 
+
+class TestLinuxImplementation:
+    def setup_class(cls):
+        if not sys.platform.startswith('linux'):
+            py.test.skip("linux specific tests")
+
+    def test_has_all_fields(self):
+        assert ll_os_stat.STAT_FIELDS == ll_os_stat.ALL_STAT_FIELDS[:13]
+
+
 class TestWin32Implementation:
     def setup_class(cls):
         if sys.platform != 'win32':
diff --git a/pypy/rpython/rclass.py b/pypy/rpython/rclass.py
--- a/pypy/rpython/rclass.py
+++ b/pypy/rpython/rclass.py
@@ -16,6 +16,13 @@
         for x in fields.itervalues():
             assert isinstance(x, ImmutableRanking)
 
+    def all_immutable_fields(self):
+        result = set()
+        for key, value in self.fields.iteritems():
+            if value in (IR_IMMUTABLE, IR_IMMUTABLE_ARRAY):
+                result.add(key)
+        return result
+
     def __repr__(self):
         return '' % getattr(self, 'TYPE', '?')
 
diff --git a/pypy/tool/jitlogparser/parser.py b/pypy/tool/jitlogparser/parser.py
--- a/pypy/tool/jitlogparser/parser.py
+++ b/pypy/tool/jitlogparser/parser.py
@@ -8,6 +8,7 @@
     bridge = None
     offset = None
     asm = None
+    failargs = ()
 
     def __init__(self, name, args, res, descr):
         self.name = name
@@ -18,8 +19,8 @@
         if self._is_guard:
             self.guard_no = int(self.descr[len('> sys.stderr, "skipping", path
+        return
+    for fn in content:
+        filename = os.path.join(path, fn)
+        st = os.lstat(filename)
+        if stat.S_ISDIR(st.st_mode):
+            clean(filename)
+            if fn == '__pycache__':
+                try:
+                    os.rmdir(filename)
+                except OSError:
+                    pass
+        elif fn.endswith('.pyc') or fn.endswith('.pyo'):
+            os.unlink(filename)
+            count += 1
 
 count = 0
 
 for arg in sys.argv[1:] or ['.']:
-    path = py.path.local(arg)
-    print "cleaning path", path, "of .pyc files"
-    for x in path.visit(shouldremove, lambda x: x.check(dotfile=0, link=0)):
-        x.remove()
-        count += 1
+    print "cleaning path", arg, "of .pyc/.pyo/__pycache__ files"
+    clean(arg)
 
 print "%d files removed" % (count,)
diff --git a/pypy/tool/release/package.py b/pypy/tool/release/package.py
--- a/pypy/tool/release/package.py
+++ b/pypy/tool/release/package.py
@@ -52,9 +52,14 @@
             pypy_c_dir = basedir.join('pypy', 'translator', 'goal')
         pypy_c = pypy_c_dir.join('pypy-c.exe')
         libpypy_c = pypy_c_dir.join('libpypy-c.dll')
+        libexpat = pypy_c_dir.join('libexpat.dll')
+        if not libexpat.check():
+            libexpat = py.path.local.sysfind('libexpat.dll')
+            assert libexpat, "libexpat.dll not found"
+            print "Picking %s" % libexpat
         binaries = [(pypy_c, pypy_c.basename),
                     (libpypy_c, libpypy_c.basename),
-                    (pypy_c_dir.join('libexpat.dll'), 'libexpat.dll')]
+                    (libexpat, libexpat.basename)]
     else:
         basename = 'pypy-c'
         if override_pypy_c is None:
diff --git a/pypy/translator/c/src/stacklet/stacklet.c b/pypy/translator/c/src/stacklet/stacklet.c
--- a/pypy/translator/c/src/stacklet/stacklet.c
+++ b/pypy/translator/c/src/stacklet/stacklet.c
@@ -319,10 +319,11 @@
 
 char **_stacklet_translate_pointer(stacklet_handle context, char **ptr)
 {
+  char *p = (char *)ptr;
+  long delta;
   if (context == NULL)
     return ptr;
-  char *p = (char *)ptr;
-  long delta = p - context->stack_start;
+  delta = p - context->stack_start;
   if (((unsigned long)delta) < ((unsigned long)context->stack_saved)) {
       /* a pointer to a saved away word */
       char *c = (char *)(context + 1);
diff --git a/pypy/translator/goal/app_main.py b/pypy/translator/goal/app_main.py
--- a/pypy/translator/goal/app_main.py
+++ b/pypy/translator/goal/app_main.py
@@ -260,6 +260,8 @@
     try:
         import _file
     except ImportError:
+        if sys.version_info < (2, 7):
+            return
         import ctypes # HACK: while running on top of CPython
         set_file_encoding = ctypes.pythonapi.PyFile_SetEncodingAndErrors
         set_file_encoding.argtypes = [ctypes.py_object, ctypes.c_char_p, ctypes.c_char_p]
@@ -479,7 +481,8 @@
             print >> sys.stderr, "'import site' failed"
 
     readenv = not ignore_environment
-    io_encoding = readenv and os.getenv("PYTHONIOENCODING")
+    io_encoding = ((readenv and os.getenv("PYTHONIOENCODING"))
+                   or sys.getfilesystemencoding())
     if io_encoding:
         set_io_encoding(io_encoding)
 
diff --git a/pypy/translator/goal/targetpreimportedpypy.py b/pypy/translator/goal/targetpreimportedpypy.py
deleted file mode 100644
--- a/pypy/translator/goal/targetpreimportedpypy.py
+++ /dev/null
@@ -1,239 +0,0 @@
-import py
-
-import os, sys
-sys.setrecursionlimit(17000)
-
-from pypy.interpreter import gateway
-from pypy.interpreter.error import OperationError
-from pypy.translator.goal.ann_override import PyPyAnnotatorPolicy
-from pypy.config.config import Config, to_optparse, make_dict, SUPPRESS_USAGE
-from pypy.config.config import ConflictConfigError
-from pypy.tool.option import make_objspace
-from pypy.translator.goal.nanos import setup_nanos
-
-EXTRA_MODULES = [
-    #"os",
-    #"decimal",
-    #"difflib",
-    #"tarfile",
-    #"cookielib",
-    #"optparse",
-    "inspect",
-    "random",
-]
-
-thisdir = py.path.local(__file__).dirpath()
-
-try:
-    this_dir = os.path.dirname(__file__)
-except NameError:
-    this_dir = os.path.dirname(sys.argv[0])
-
-def debug(msg):
-    os.write(2, "debug: " + msg + '\n')
-
-# __________  Entry point  __________
-
-def create_entry_point(space, w_dict):
-    w_entry_point = space.getitem(w_dict, space.wrap('entry_point'))
-    w_run_toplevel = space.getitem(w_dict, space.wrap('run_toplevel'))
-    w_call_finish_gateway = space.wrap(gateway.interp2app(call_finish))
-    w_call_startup_gateway = space.wrap(gateway.interp2app(call_startup))
-    w_os = setup_nanos(space)
-
-    def entry_point(argv):
-        space.timer.start("Entrypoint")
-        #debug("entry point starting") 
-        #for arg in argv: 
-        #    debug(" argv -> " + arg)
-        if len(argv) > 2 and argv[1] == '--heapsize':
-            # Undocumented option, handled at interp-level.
-            # It has silently no effect with some GCs.
-            # It works in Boehm and in the semispace or generational GCs
-            # (but see comments in semispace.py:set_max_heap_size()).
-            # At the moment this option exists mainly to support sandboxing.
-            from pypy.rlib import rgc
-            rgc.set_max_heap_size(int(argv[2]))
-            argv = argv[:1] + argv[3:]
-        try:
-            try:
-                space.timer.start("space.startup")
-                space.call_function(w_run_toplevel, w_call_startup_gateway)
-                space.timer.stop("space.startup")
-                w_executable = space.wrap(argv[0])
-                w_argv = space.newlist([space.wrap(s) for s in argv[1:]])
-                space.timer.start("w_entry_point")
-                w_exitcode = space.call_function(w_entry_point, w_executable, w_argv, w_os)
-                space.timer.stop("w_entry_point")
-                exitcode = space.int_w(w_exitcode)
-                # try to pull it all in
-            ##    from pypy.interpreter import main, interactive, error
-            ##    con = interactive.PyPyConsole(space)
-            ##    con.interact()
-            except OperationError, e:
-                debug("OperationError:")
-                debug(" operror-type: " + e.w_type.getname(space))
-                debug(" operror-value: " + space.str_w(space.str(e.get_w_value(space))))
-                return 1
-        finally:
-            try:
-                space.timer.start("space.finish")
-                space.call_function(w_run_toplevel, w_call_finish_gateway)
-                space.timer.stop("space.finish")
-            except OperationError, e:
-                debug("OperationError:")
-                debug(" operror-type: " + e.w_type.getname(space))
-                debug(" operror-value: " + space.str_w(space.str(e.get_w_value(space))))
-                return 1
-        space.timer.stop("Entrypoint")
-        space.timer.dump()
-        return exitcode
-    return entry_point
-
-def call_finish(space):

From noreply at buildbot.pypy.org  Thu Sep  8 05:37:33 2011
From: noreply at buildbot.pypy.org (wlav)
Date: Thu,  8 Sep 2011 05:37:33 +0200 (CEST)
Subject: [pypy-commit] pypy reflex-support: turn type handles into
 rffi.LONGs to make sure they're opaque
Message-ID: <20110908033733.6B1FF82213@wyvern.cs.uni-duesseldorf.de>

Author: Wim Lavrijsen 
Branch: reflex-support
Changeset: r47155:532fbe4e0dd4
Date: 2011-09-07 20:38 -0700
http://bitbucket.org/pypy/pypy/changeset/532fbe4e0dd4/

Log:	turn type handles into rffi.LONGs to make sure they're opaque

diff --git a/pypy/module/cppyy/capi/__init__.py b/pypy/module/cppyy/capi/__init__.py
--- a/pypy/module/cppyy/capi/__init__.py
+++ b/pypy/module/cppyy/capi/__init__.py
@@ -5,7 +5,7 @@
 #import cint_capi as backend
 
 
-C_TYPEHANDLE = rffi.VOIDP
+C_TYPEHANDLE = rffi.LONG
 C_OBJECT = rffi.VOIDP
 
 C_METHPTRGETTER = lltype.FuncType([C_OBJECT], rffi.VOIDP)
diff --git a/pypy/module/cppyy/include/capi.h b/pypy/module/cppyy/include/capi.h
--- a/pypy/module/cppyy/include/capi.h
+++ b/pypy/module/cppyy/include/capi.h
@@ -6,7 +6,7 @@
 #ifdef __cplusplus
 extern "C" {
 #endif // ifdef __cplusplus
-    typedef void* cppyy_typehandle_t;
+    typedef long cppyy_typehandle_t;
     typedef void* cppyy_object_t;
     typedef void* (*cppyy_methptrgetter_t)(cppyy_object_t);
 
diff --git a/pypy/module/cppyy/interp_cppyy.py b/pypy/module/cppyy/interp_cppyy.py
--- a/pypy/module/cppyy/interp_cppyy.py
+++ b/pypy/module/cppyy/interp_cppyy.py
@@ -31,7 +31,8 @@
 
 class State(object):
     def __init__(self, space):
-        self.cpptype_cache = { "void" : W_CPPType(space, "void", NULL_VOIDP) }
+        self.cpptype_cache = {
+            "void" : W_CPPType(space, "void", rffi.cast(capi.C_TYPEHANDLE, NULL_VOIDP)) }
         self.cpptemplatetype_cache = {}
 
 @unwrap_spec(name=str)
@@ -110,7 +111,7 @@
 
     @jit.unroll_safe
     def call(self, cppthis, w_type, args_w):
-        assert lltype.typeOf(cppthis) == rffi.VOIDP
+        assert lltype.typeOf(cppthis) == capi.C_OBJECT
         if self.executor is None:
             raise OperationError(self.space.w_TypeError,
                                  self.space.wrap("return type not handled"))
@@ -220,7 +221,7 @@
 
     def call(self, cppthis, w_type, args_w):
         newthis = capi.c_allocate(self.cpptype.handle)
-        assert lltype.typeOf(newthis) == rffi.VOIDP
+        assert lltype.typeOf(newthis) == capi.C_OBJECT
         try:
             CPPMethod.call(self, newthis, None, args_w)
         except Exception, e:
@@ -234,7 +235,7 @@
 
     def __init__(self, space, scope_handle, func_name, functions):
         self.space = space
-        assert lltype.typeOf(scope_handle) == rffi.VOIDP
+        assert lltype.typeOf(scope_handle) == capi.C_TYPEHANDLE
         self.scope_handle = scope_handle
         self.func_name = func_name
         self.functions = debug.make_sure_not_resized(functions)
@@ -260,7 +261,7 @@
     def call(self, w_cppinstance, w_type, args_w):
         cppinstance = self.space.interp_w(W_CPPInstance, w_cppinstance, can_be_None=True)
         cppthis = self._get_cppthis(cppinstance)
-        assert lltype.typeOf(cppthis) == rffi.VOIDP
+        assert lltype.typeOf(cppthis) == capi.C_OBJECT
 
         space = self.space
         errmsg = 'None of the overloads matched:'
@@ -299,7 +300,7 @@
 
     def __init__(self, space, scope_handle, type_name, offset, is_static):
         self.space = space
-        assert lltype.typeOf(scope_handle) == rffi.VOIDP
+        assert lltype.typeOf(scope_handle) == capi.C_TYPEHANDLE
         self.scope_handle = scope_handle
         self.converter = converter.get_converter(self.space, type_name)
         self.offset = offset
@@ -344,10 +345,12 @@
 class W_CPPScope(Wrappable):
     _immutable_fields_ = ["name", "handle"]
 
+    kind = "scope"
+
     def __init__(self, space, name, handle):
         self.space = space
         self.name = name
-        assert lltype.typeOf(handle) == rffi.VOIDP
+        assert lltype.typeOf(handle) == capi.C_TYPEHANDLE
         self.handle = handle
         self.methods = {}
         # Do not call "self._find_methods()" here, so that a distinction can
@@ -512,7 +515,7 @@
     def __init__(self, space, name, handle):
         self.space = space
         self.name = name
-        assert lltype.typeOf(handle) == rffi.VOIDP
+        assert lltype.typeOf(handle) == capi.C_TYPEHANDLE
         self.handle = handle
 
     def __call__(self, args_w):
@@ -533,7 +536,7 @@
     def __init__(self, space, cppclass, rawobject, python_owns):
         self.space = space
         self.cppclass = cppclass
-        assert lltype.typeOf(rawobject) == rffi.VOIDP
+        assert lltype.typeOf(rawobject) == capi.C_OBJECT
         self.rawobject = rawobject
         self.python_owns = python_owns
 
diff --git a/pypy/module/cppyy/src/reflexcwrapper.cxx b/pypy/module/cppyy/src/reflexcwrapper.cxx
--- a/pypy/module/cppyy/src/reflexcwrapper.cxx
+++ b/pypy/module/cppyy/src/reflexcwrapper.cxx
@@ -47,12 +47,12 @@
 /* name to handle --------------------------------------------------------- */
 cppyy_typehandle_t cppyy_get_typehandle(const char* class_name) {
     Reflex::Scope s = Reflex::Scope::ByName(class_name);
-    return s.Id();
+    return (cppyy_typehandle_t)s.Id();
 }
 
 cppyy_typehandle_t cppyy_get_templatehandle(const char* template_name) {
    Reflex::TypeTemplate tt = Reflex::TypeTemplate::ByName(template_name);
-   return tt.Id();
+   return (cppyy_typehandle_t)tt.Id();
 }
 
 

From noreply at buildbot.pypy.org  Thu Sep  8 09:19:28 2011
From: noreply at buildbot.pypy.org (fijal)
Date: Thu,  8 Sep 2011 09:19:28 +0200 (CEST)
Subject: [pypy-commit] pypy space-iterator-improvements: backout
 b02477f065b2, reinstantiane newlist hint. this time we want to use it
Message-ID: <20110908071928.5D0608203C@wyvern.cs.uni-duesseldorf.de>

Author: Maciej Fijalkowski 
Branch: space-iterator-improvements
Changeset: r47156:6dfe74cae2b2
Date: 2011-09-08 09:18 +0200
http://bitbucket.org/pypy/pypy/changeset/6dfe74cae2b2/

Log:	backout b02477f065b2, reinstantiane newlist hint. this time we want
	to use it

diff --git a/pypy/rlib/objectmodel.py b/pypy/rlib/objectmodel.py
--- a/pypy/rlib/objectmodel.py
+++ b/pypy/rlib/objectmodel.py
@@ -19,6 +19,8 @@
 # def f(...
 #
 
+from pypy.rpython.extregistry import ExtRegistryEntry
+
 class _Specialize(object):
     def memo(self):
         """ Specialize functions based on argument values. All arguments has
@@ -177,6 +179,33 @@
     obj.__class__ = FREED_OBJECT
 
 # ____________________________________________________________
+
+def newlist(sizehint=0):
+    """ Create a new list, but pass a hint how big the size should be
+    preallocated
+    """
+    return []
+
+class Entry(ExtRegistryEntry):
+    _about_ = newlist
+
+    def compute_result_annotation(self, s_sizehint):
+        from pypy.annotation.model import SomeInteger
+        
+        assert isinstance(s_sizehint, SomeInteger)
+        return self.bookkeeper.newlist()
+
+    def specialize_call(self, orig_hop, i_sizehint=None):
+        from pypy.rpython.rlist import rtype_newlist
+        # fish a bit hop
+        hop = orig_hop.copy()
+        v = hop.args_v[0]
+        r, s = hop.r_s_popfirstarg()
+        if s.is_constant():
+            v = hop.inputconst(r, s.const)
+        return rtype_newlist(hop, v_sizehint=v)
+
+# ____________________________________________________________
 #
 # id-like functions.  The idea is that calling hash() or id() is not
 # allowed in RPython.  You have to call one of the following more
@@ -301,8 +330,6 @@
 
 # ----------
 
-from pypy.rpython.extregistry import ExtRegistryEntry
-
 class Entry(ExtRegistryEntry):
     _about_ = compute_hash
 
diff --git a/pypy/rlib/test/test_objectmodel.py b/pypy/rlib/test/test_objectmodel.py
--- a/pypy/rlib/test/test_objectmodel.py
+++ b/pypy/rlib/test/test_objectmodel.py
@@ -424,3 +424,32 @@
     if option.view:
         graph.show()
     return graph
+
+
+def test_newlist():
+    from pypy.annotation.model import SomeInteger
+    def f(z):
+        x = newlist(sizehint=38)
+        if z < 0:
+            x.append(1)
+        return len(x)
+
+    graph = getgraph(f, [SomeInteger()])
+    for llop in graph.startblock.operations:
+        if llop.opname == 'malloc_varsize':
+            break
+    assert llop.args[2].value == 38
+
+def test_newlist_nonconst():
+    from pypy.annotation.model import SomeInteger
+    def f(z):
+        x = newlist(sizehint=z)
+        return len(x)
+
+    graph = getgraph(f, [SomeInteger()])
+    for llop in graph.startblock.operations:
+        if llop.opname == 'malloc_varsize':
+            break
+    assert llop.args[2] is graph.startblock.inputargs[0]
+
+    
diff --git a/pypy/rpython/lltypesystem/rlist.py b/pypy/rpython/lltypesystem/rlist.py
--- a/pypy/rpython/lltypesystem/rlist.py
+++ b/pypy/rpython/lltypesystem/rlist.py
@@ -1,21 +1,14 @@
 from pypy.tool.pairtype import pairtype, pair
-from pypy.annotation import model as annmodel
-from pypy.rpython.error import TyperError
-from pypy.rpython.rmodel import Repr, IntegerRepr, inputconst
+from pypy.rpython.rmodel import Repr, inputconst
 from pypy.rpython.rmodel import externalvsinternal
 from pypy.rpython.rlist import AbstractBaseListRepr, AbstractListRepr, \
-        AbstractFixedSizeListRepr, AbstractListIteratorRepr, rtype_newlist, \
-        rtype_alloc_and_set, ll_setitem_nonneg, ADTIList, ADTIFixedList
-from pypy.rpython.rlist import dum_nocheck, dum_checkidx
-from pypy.rpython.lltypesystem.lltype import \
-     GcForwardReference, Ptr, GcArray, GcStruct, \
-     Void, Signed, malloc, typeOf, Primitive, \
-     Bool, nullptr, typeMethod
+        AbstractFixedSizeListRepr, AbstractListIteratorRepr, \
+        ll_setitem_nonneg, ADTIList, ADTIFixedList
+from pypy.rpython.rlist import dum_nocheck
+from pypy.rpython.lltypesystem.lltype import GcForwardReference, Ptr, GcArray,\
+     GcStruct, Void, Signed, malloc, typeOf, nullptr, typeMethod
 from pypy.rpython.lltypesystem import rstr
-from pypy.rpython import robject
 from pypy.rlib.debug import ll_assert
-from pypy.rpython.lltypesystem import rffi
-from pypy.rpython.lltypesystem.lloperation import llop
 from pypy.rlib import rgc
 
 # ____________________________________________________________
@@ -67,6 +60,7 @@
         ITEMARRAY = GcArray(ITEM,
                             adtmeths = ADTIFixedList({
                                  "ll_newlist": ll_fixed_newlist,
+                                 "ll_newlist_hint": ll_fixed_newlist,
                                  "ll_newemptylist": ll_fixed_newemptylist,
                                  "ll_length": ll_fixed_length,
                                  "ll_items": ll_fixed_items,
@@ -100,6 +94,7 @@
                                               ("items", Ptr(ITEMARRAY)),
                                       adtmeths = ADTIList({
                                           "ll_newlist": ll_newlist,
+                                          "ll_newlist_hint": ll_newlist_hint,
                                           "ll_newemptylist": ll_newemptylist,
                                           "ll_length": ll_length,
                                           "ll_items": ll_items,
@@ -267,6 +262,15 @@
 ll_newlist = typeMethod(ll_newlist)
 ll_newlist.oopspec = 'newlist(length)'
 
+def ll_newlist_hint(LIST, lengthhint):
+    ll_assert(lengthhint >= 0, "negative list length")
+    l = malloc(LIST)
+    l.length = 0
+    l.items = malloc(LIST.items.TO, lengthhint)
+    return l
+ll_newlist_hint = typeMethod(ll_newlist_hint)
+ll_newlist_hint.oopspec = 'newlist(length)'
+
 # should empty lists start with no allocated memory, or with a preallocated
 # minimal number of entries?  XXX compare memory usage versus speed, and
 # check how many always-empty lists there are in a typical pypy-c run...
@@ -337,11 +341,15 @@
     l[index] = item
 ll_fixed_setitem_fast.oopspec = 'list.setitem(l, index, item)'
 
-def newlist(llops, r_list, items_v):
+def newlist(llops, r_list, items_v, v_sizehint=None):
     LIST = r_list.LIST
     if len(items_v) == 0:
-        v_result = llops.gendirectcall(LIST.ll_newemptylist)
+        if v_sizehint is None:
+            v_result = llops.gendirectcall(LIST.ll_newemptylist)
+        else:
+            v_result = llops.gendirectcall(LIST.ll_newlist_hint, v_sizehint)
     else:
+        assert v_sizehint is None
         cno = inputconst(Signed, len(items_v))
         v_result = llops.gendirectcall(LIST.ll_newlist, cno)
     v_func = inputconst(Void, dum_nocheck)
diff --git a/pypy/rpython/ootypesystem/rlist.py b/pypy/rpython/ootypesystem/rlist.py
--- a/pypy/rpython/ootypesystem/rlist.py
+++ b/pypy/rpython/ootypesystem/rlist.py
@@ -124,7 +124,7 @@
         else:
             return ootype.List()
 
-    def _generate_newlist(self, llops, items_v):
+    def _generate_newlist(self, llops, items_v, v_sizehint):
         c_list = inputconst(ootype.Void, self.lowleveltype)
         v_result = llops.genop("new", [c_list], resulttype=self.lowleveltype)
         c_resize = inputconst(ootype.Void, "_ll_resize")
@@ -150,8 +150,8 @@
 
 
 
-def newlist(llops, r_list, items_v):
-    v_result = r_list._generate_newlist(llops, items_v)
+def newlist(llops, r_list, items_v, v_sizehint=None):
+    v_result = r_list._generate_newlist(llops, items_v, v_sizehint)
 
     c_setitem = inputconst(ootype.Void, "ll_setitem_fast")
     for i, v_item in enumerate(items_v):
@@ -224,7 +224,7 @@
     def make_iterator_repr(self):
         return ListIteratorRepr(self)
 
-    def _generate_newlist(self, llops, items_v):
+    def _generate_newlist(self, llops, items_v, v_sizehint):
         c_array = inputconst(ootype.Void, self.lowleveltype)
         c_length = inputconst(ootype.Signed, len(items_v))
         v_result = llops.genop("oonewarray", [c_array, c_length], resulttype=self.lowleveltype)
diff --git a/pypy/rpython/rlist.py b/pypy/rpython/rlist.py
--- a/pypy/rpython/rlist.py
+++ b/pypy/rpython/rlist.py
@@ -2,7 +2,7 @@
 from pypy.objspace.flow.model import Constant
 from pypy.annotation import model as annmodel
 from pypy.rpython.error import TyperError
-from pypy.rpython.rmodel import Repr, IteratorRepr, IntegerRepr, inputconst
+from pypy.rpython.rmodel import Repr, IteratorRepr, IntegerRepr
 from pypy.rpython.rstr import AbstractStringRepr, AbstractCharRepr
 from pypy.rpython.lltypesystem.lltype import typeOf, Ptr, Void, Signed, Bool
 from pypy.rpython.lltypesystem.lltype import nullptr, Char, UniChar, Number
@@ -344,7 +344,7 @@
         return hop.genop('bool_not', [flag], resulttype=Bool)
 
 
-def rtype_newlist(hop):
+def rtype_newlist(hop, v_sizehint=None):
     nb_args = hop.nb_args
     r_list = hop.r_result
     if r_list == robject.pyobj_repr: # special case: SomeObject lists!
@@ -358,7 +358,8 @@
         return v_result
     r_listitem = r_list.item_repr
     items_v = [hop.inputarg(r_listitem, arg=i) for i in range(nb_args)]
-    return hop.rtyper.type_system.rlist.newlist(hop.llops, r_list, items_v)
+    return hop.rtyper.type_system.rlist.newlist(hop.llops, r_list, items_v,
+                                                v_sizehint=v_sizehint)
 
 def rtype_alloc_and_set(hop):
     r_list = hop.r_result
diff --git a/pypy/rpython/test/test_rlist.py b/pypy/rpython/test/test_rlist.py
--- a/pypy/rpython/test/test_rlist.py
+++ b/pypy/rpython/test/test_rlist.py
@@ -1360,6 +1360,19 @@
             assert ('foldable' in func.func_name) == \
                    ("y[*]" in immutable_fields)
 
+    def test_hints(self):
+        from pypy.rlib.objectmodel import newlist
+        from pypy.rpython.annlowlevel import hlstr
+        
+        def f(z):
+            z = hlstr(z)
+            x = newlist(sizehint=13)
+            x += z
+            return ''.join(x)
+
+        res = self.interpret(f, [self.string_to_ll('abc')])
+        assert self.ll_to_string(res) == 'abc'
+
 class TestLLtype(BaseTestRlist, LLRtypeMixin):
     type_system = 'lltype'
     rlist = ll_rlist

From noreply at buildbot.pypy.org  Thu Sep  8 09:24:21 2011
From: noreply at buildbot.pypy.org (fijal)
Date: Thu,  8 Sep 2011 09:24:21 +0200 (CEST)
Subject: [pypy-commit] pypy space-iterator-improvements: use length estimate
 for unpackiterable
Message-ID: <20110908072421.F329D8203C@wyvern.cs.uni-duesseldorf.de>

Author: Maciej Fijalkowski 
Branch: space-iterator-improvements
Changeset: r47157:81a9a624253f
Date: 2011-09-08 09:23 +0200
http://bitbucket.org/pypy/pypy/changeset/81a9a624253f/

Log:	use length estimate for unpackiterable

diff --git a/pypy/interpreter/baseobjspace.py b/pypy/interpreter/baseobjspace.py
--- a/pypy/interpreter/baseobjspace.py
+++ b/pypy/interpreter/baseobjspace.py
@@ -8,13 +8,13 @@
 from pypy.interpreter.miscutils import ThreadLocals
 from pypy.tool.cache import Cache
 from pypy.tool.uid import HUGEVAL_BYTES
-from pypy.rlib.objectmodel import we_are_translated
+from pypy.rlib.objectmodel import we_are_translated, newlist
 from pypy.rlib.debug import make_sure_not_resized
 from pypy.rlib.timer import DummyTimer, Timer
 from pypy.rlib.rarithmetic import r_uint
 from pypy.rlib import jit
 from pypy.tool.sourcetools import func_with_new_name
-import os, sys, py
+import os, sys
 
 __all__ = ['ObjSpace', 'OperationError', 'Wrappable', 'W_Root']
 
@@ -757,7 +757,15 @@
         w_iterator = self.iter(w_iterable)
         # If we know the expected length we can preallocate.
         if expected_length == -1:
-            items = []
+            try:
+                lgt_estimate = self.int_w(self.len(w_iterable))
+            except OperationError:
+                items = []
+            else:
+                try:
+                    items = newlist(lgt_estimate)
+                except MemoryError:
+                    items = [] # it might have lied
         else:
             items = [None] * expected_length
         idx = 0

From noreply at buildbot.pypy.org  Thu Sep  8 09:26:51 2011
From: noreply at buildbot.pypy.org (fijal)
Date: Thu,  8 Sep 2011 09:26:51 +0200 (CEST)
Subject: [pypy-commit] pypy space-iterator-improvements: a minor
	simplification
Message-ID: <20110908072651.880AD8203C@wyvern.cs.uni-duesseldorf.de>

Author: Maciej Fijalkowski 
Branch: space-iterator-improvements
Changeset: r47158:31445f5be38e
Date: 2011-09-08 09:26 +0200
http://bitbucket.org/pypy/pypy/changeset/31445f5be38e/

Log:	a minor simplification

diff --git a/pypy/interpreter/baseobjspace.py b/pypy/interpreter/baseobjspace.py
--- a/pypy/interpreter/baseobjspace.py
+++ b/pypy/interpreter/baseobjspace.py
@@ -758,7 +758,7 @@
         # If we know the expected length we can preallocate.
         if expected_length == -1:
             try:
-                lgt_estimate = self.int_w(self.len(w_iterable))
+                lgt_estimate = self.len_w(w_iterable)
             except OperationError:
                 items = []
             else:

From noreply at buildbot.pypy.org  Thu Sep  8 09:50:54 2011
From: noreply at buildbot.pypy.org (fijal)
Date: Thu,  8 Sep 2011 09:50:54 +0200 (CEST)
Subject: [pypy-commit] pypy space-iterator-improvements: A questionable test,
 but matches cpython behavior. The change itself makes
Message-ID: <20110908075054.881408203C@wyvern.cs.uni-duesseldorf.de>

Author: Maciej Fijalkowski 
Branch: space-iterator-improvements
Changeset: r47159:b4bff794fea0
Date: 2011-09-08 09:50 +0200
http://bitbucket.org/pypy/pypy/changeset/b4bff794fea0/

Log:	A questionable test, but matches cpython behavior. The change itself
	makes sense however.

diff --git a/pypy/interpreter/baseobjspace.py b/pypy/interpreter/baseobjspace.py
--- a/pypy/interpreter/baseobjspace.py
+++ b/pypy/interpreter/baseobjspace.py
@@ -759,7 +759,10 @@
         if expected_length == -1:
             try:
                 lgt_estimate = self.len_w(w_iterable)
-            except OperationError:
+            except OperationError, o:
+                if (not o.match(self, self.w_AttributeError) and
+                    not o.match(self, self.w_TypeError)):
+                    raise
                 items = []
             else:
                 try:
diff --git a/pypy/interpreter/test/test_objspace.py b/pypy/interpreter/test/test_objspace.py
--- a/pypy/interpreter/test/test_objspace.py
+++ b/pypy/interpreter/test/test_objspace.py
@@ -71,6 +71,23 @@
         assert err.value.match(space, space.w_ValueError)
         err = raises(OperationError, space.unpackiterable, w_l, 5)
         assert err.value.match(space, space.w_ValueError)
+        w_a = space.appexec((), """():
+        class A(object):
+            def __iter__(self):
+                return self
+            def next(self):
+                raise StopIteration
+            def __len__(self):
+                1/0
+        return A()
+        """)
+        try:
+            space.unpackiterable(w_a)
+        except OperationError, o:
+            if not o.match(space, space.w_ZeroDivisionError):
+                raise Exception("DID NOT RAISE")
+        else:
+            raise Exception("DID NOT RAISE")
 
     def test_fixedview(self):
         space = self.space

From noreply at buildbot.pypy.org  Thu Sep  8 10:41:50 2011
From: noreply at buildbot.pypy.org (lac)
Date: Thu,  8 Sep 2011 10:41:50 +0200 (CEST)
Subject: [pypy-commit] pypy.org extradoc: add Samuele,
	who has also promised us a photo.
Message-ID: <20110908084150.6E68A8203C@wyvern.cs.uni-duesseldorf.de>

Author: Laura Creighton 
Branch: extradoc
Changeset: r254:ffde8342de9d
Date: 2011-09-08 10:41 +0200
http://bitbucket.org/pypy/pypy.org/changeset/ffde8342de9d/

Log:	add Samuele, who has also promised us a photo.

diff --git a/source/people.txt b/source/people.txt
--- a/source/people.txt
+++ b/source/people.txt
@@ -1,6 +1,6 @@
 ---
 layout: page
-title: Current people of PyPy
+title: People of PyPy
 ---
 
 Armin Rigo
@@ -129,6 +129,18 @@
 .. _`tox`: http://codespeak.net/tox
 .. _`execnet`: http://codespeak.net/execnet
 
+Samuele Pedroni
+===============
+
+Samuele Pedroni got involved with PyPy almost at its inception in the
+spring of 2003. One of the design contributors to PyPy, his help has
+ranged from infrastructure and processes, through building out
+RPython... optimizing the Python interpreter, to compressing resume
+data in the last incarnation of the JIT compiler. Tempted away into the
+application side of the software equation, these days he contributes
+some words and wisdom to PyPy's paper writing.
+
+
 Many more people
 ================
 

From noreply at buildbot.pypy.org  Thu Sep  8 11:33:07 2011
From: noreply at buildbot.pypy.org (fijal)
Date: Thu,  8 Sep 2011 11:33:07 +0200 (CEST)
Subject: [pypy-commit] pypy.org extradoc: remove people from the list,
 it generates more problems than gain and also gets one slot
Message-ID: <20110908093307.DDC268203C@wyvern.cs.uni-duesseldorf.de>

Author: Maciej Fijalkowski 
Branch: extradoc
Changeset: r255:451af0702e2d
Date: 2011-09-08 11:32 +0200
http://bitbucket.org/pypy/pypy.org/changeset/451af0702e2d/

Log:	remove people from the list, it generates more problems than gain
	and also gets one slot

diff --git a/source/_layouts/site.genshi b/source/_layouts/site.genshi
--- a/source/_layouts/site.genshi
+++ b/source/_layouts/site.genshi
@@ -11,7 +11,6 @@
       ('Performance', 'http://speed.pypy.org'),
       ('Dev Documentation', 'http://doc.pypy.org'),
       ('Blog', 'http://morepypy.blogspot.com'),
-      ('People', 'people.html'),
       ('Contact', 'contact.html'),
       ],
     }

From noreply at buildbot.pypy.org  Thu Sep  8 11:36:46 2011
From: noreply at buildbot.pypy.org (fijal)
Date: Thu,  8 Sep 2011 11:36:46 +0200 (CEST)
Subject: [pypy-commit] pypy.org extradoc: Backed out changeset 451af0702e2d
Message-ID: <20110908093646.671428203C@wyvern.cs.uni-duesseldorf.de>

Author: Maciej Fijalkowski 
Branch: extradoc
Changeset: r256:dd354a153c5c
Date: 2011-09-08 11:36 +0200
http://bitbucket.org/pypy/pypy.org/changeset/dd354a153c5c/

Log:	Backed out changeset 451af0702e2d

diff --git a/source/_layouts/site.genshi b/source/_layouts/site.genshi
--- a/source/_layouts/site.genshi
+++ b/source/_layouts/site.genshi
@@ -11,6 +11,7 @@
       ('Performance', 'http://speed.pypy.org'),
       ('Dev Documentation', 'http://doc.pypy.org'),
       ('Blog', 'http://morepypy.blogspot.com'),
+      ('People', 'people.html'),
       ('Contact', 'contact.html'),
       ],
     }

From noreply at buildbot.pypy.org  Thu Sep  8 14:29:56 2011
From: noreply at buildbot.pypy.org (hakanardo)
Date: Thu,  8 Sep 2011 14:29:56 +0200 (CEST)
Subject: [pypy-commit] pypy default: uint8 support
Message-ID: <20110908122956.1F51A8203C@wyvern.cs.uni-duesseldorf.de>

Author: Hakan Ardo 
Branch: 
Changeset: r47160:12d91e2900e9
Date: 2011-09-08 14:29 +0200
http://bitbucket.org/pypy/pypy/changeset/12d91e2900e9/

Log:	uint8 support

diff --git a/pypy/module/micronumpy/interp_dtype.py b/pypy/module/micronumpy/interp_dtype.py
--- a/pypy/module/micronumpy/interp_dtype.py
+++ b/pypy/module/micronumpy/interp_dtype.py
@@ -317,6 +317,17 @@
 class W_Int8Dtype(IntegerArithmeticDtype, W_Int8Dtype):
     pass
 
+W_UInt8Dtype = create_low_level_dtype(
+    num = 1, kind = SIGNEDLTR, name = "uint8",
+    aliases = ["uint8"],
+    applevel_types = [],
+    T = rffi.UCHAR,
+    valtype = rffi.UCHAR._type,
+    expected_size = 1,
+)
+class W_UInt8Dtype(IntegerArithmeticDtype, W_UInt8Dtype):
+    pass
+
 W_Int16Dtype = create_low_level_dtype(
     num = 3, kind = SIGNEDLTR, name = "int16",
     aliases = ["int16"],
@@ -368,6 +379,7 @@
 ALL_DTYPES = [
     W_BoolDtype,
     W_Int8Dtype, W_Int16Dtype, W_Int32Dtype, W_Int64Dtype,
+    W_UInt8Dtype,
     W_Float64Dtype
 ]
 
diff --git a/pypy/module/micronumpy/test/test_dtypes.py b/pypy/module/micronumpy/test/test_dtypes.py
--- a/pypy/module/micronumpy/test/test_dtypes.py
+++ b/pypy/module/micronumpy/test/test_dtypes.py
@@ -12,6 +12,7 @@
         assert dtype(d) is d
         assert dtype(None) is dtype(float)
         raises(TypeError, dtype, 1042)
+        assert dtype('uint8').num == 1
 
     def test_dtype_with_types(self):
         from numpy import dtype
@@ -90,6 +91,15 @@
         for i in range(5):
             assert b[i] == i * 2
 
+    def test_add_uint8(self):
+        from numpy import array, dtype
+
+        a = array(range(5), dtype="uint8")
+        b = a + a
+        assert b.dtype is dtype("uint8")
+        for i in range(5):
+            assert b[i] == i * 2
+
     def test_add_int16(self):
         from numpy import array, dtype
 
@@ -109,3 +119,15 @@
 
         # You can't subclass dtype
         raises(TypeError, type, "Foo", (dtype,), {})
+
+    def test_int_ranges(self):
+        from numpy import array
+        for dtype, minval, maxval in [("int8", -128, 127),
+                                      ("uint8", 0, 255),
+                                      ("int16", -32768, 32767)]:
+            a = array([minval, maxval, minval-1, maxval+1], dtype)
+            assert a[0] == minval
+            assert a[1] == maxval
+            assert a[2] == maxval
+            assert a[3] == minval
+            

From noreply at buildbot.pypy.org  Thu Sep  8 14:52:11 2011
From: noreply at buildbot.pypy.org (l.diekmann)
Date: Thu,  8 Sep 2011 14:52:11 +0200 (CEST)
Subject: [pypy-commit] pypy default: prepare listobject and timsort for
 liststrategies that will be merged later
Message-ID: <20110908125211.BF5898203C@wyvern.cs.uni-duesseldorf.de>

Author: Lukas Diekmann 
Branch: 
Changeset: r47161:1d6e48b23004
Date: 2011-09-08 14:48 +0200
http://bitbucket.org/pypy/pypy/changeset/1d6e48b23004/

Log:	prepare listobject and timsort for liststrategies that will be
	merged later

diff --git a/pypy/objspace/std/listobject.py b/pypy/objspace/std/listobject.py
--- a/pypy/objspace/std/listobject.py
+++ b/pypy/objspace/std/listobject.py
@@ -8,7 +8,7 @@
 
 from pypy.objspace.std import slicetype
 from pypy.interpreter import gateway, baseobjspace
-from pypy.rlib.listsort import TimSort
+from pypy.rlib.listsort import make_timsort_class
 from pypy.interpreter.argument import Signature
 
 class W_ListObject(W_Object):
@@ -445,6 +445,7 @@
         self.w_key = w_key
         self.w_item = w_item
 
+TimSort = make_timsort_class()
 # NOTE: all the subclasses of TimSort should inherit from a common subclass,
 #       so make sure that only SimpleSort inherits directly from TimSort.
 #       This is necessary to hide the parent method TimSort.lt() from the
diff --git a/pypy/rlib/listsort.py b/pypy/rlib/listsort.py
--- a/pypy/rlib/listsort.py
+++ b/pypy/rlib/listsort.py
@@ -7,588 +7,589 @@
 ## ------------------------------------------------------------------------
 ##         Adapted from CPython, original code and algorithms by Tim Peters
 
-## CAREFUL:
-## this class has to be used carefully, because all the lists that are
-## sorted will be unified
+def make_timsort_class():
 
-class TimSort:
-    """TimSort(list).sort()
+    class TimSort:
+        """TimSort(list).sort()
 
-    Sorts the list in-place, using the overridable method lt() for comparison.
-    """
+        Sorts the list in-place, using the overridable method lt() for comparison.
+        """
 
-    def __init__(self, list, listlength=None):
-        self.list = list
-        if listlength is None:
-            listlength = len(list)
-        self.listlength = listlength
+        def __init__(self, list, listlength=None):
+            self.list = list
+            if listlength is None:
+                listlength = len(list)
+            self.listlength = listlength
 
-    def lt(self, a, b):
-        return a < b
+        def lt(self, a, b):
+            return a < b
 
-    def le(self, a, b):
-        return not self.lt(b, a)   # always use self.lt() as the primitive
+        def le(self, a, b):
+            return not self.lt(b, a)   # always use self.lt() as the primitive
 
-    # binarysort is the best method for sorting small arrays: it does
-    # few compares, but can do data movement quadratic in the number of
-    # elements.
-    # "a" is a contiguous slice of a list, and is sorted via binary insertion.
-    # This sort is stable.
-    # On entry, the first "sorted" elements are already sorted.
-    # Even in case of error, the output slice will be some permutation of
-    # the input (nothing is lost or duplicated).
+        # binarysort is the best method for sorting small arrays: it does
+        # few compares, but can do data movement quadratic in the number of
+        # elements.
+        # "a" is a contiguous slice of a list, and is sorted via binary insertion.
+        # This sort is stable.
+        # On entry, the first "sorted" elements are already sorted.
+        # Even in case of error, the output slice will be some permutation of
+        # the input (nothing is lost or duplicated).
 
-    def binarysort(self, a, sorted=1):
-        for start in xrange(a.base + sorted, a.base + a.len):
-            # set l to where list[start] belongs
-            l = a.base
-            r = start
-            pivot = a.list[r]
-            # Invariants:
-            # pivot >= all in [base, l).
-            # pivot  < all in [r, start).
-            # The second is vacuously true at the start.
-            while l < r:
-                p = l + ((r - l) >> 1)
-                if self.lt(pivot, a.list[p]):
-                    r = p
+        def binarysort(self, a, sorted=1):
+            for start in xrange(a.base + sorted, a.base + a.len):
+                # set l to where list[start] belongs
+                l = a.base
+                r = start
+                pivot = a.list[r]
+                # Invariants:
+                # pivot >= all in [base, l).
+                # pivot  < all in [r, start).
+                # The second is vacuously true at the start.
+                while l < r:
+                    p = l + ((r - l) >> 1)
+                    if self.lt(pivot, a.list[p]):
+                        r = p
+                    else:
+                        l = p+1
+                assert l == r
+                # The invariants still hold, so pivot >= all in [base, l) and
+                # pivot < all in [l, start), so pivot belongs at l.  Note
+                # that if there are elements equal to pivot, l points to the
+                # first slot after them -- that's why this sort is stable.
+                # Slide over to make room.
+                for p in xrange(start, l, -1):
+                    a.list[p] = a.list[p-1]
+                a.list[l] = pivot
+
+        # Compute the length of the run in the slice "a".
+        # "A run" is the longest ascending sequence, with
+        #
+        #     a[0] <= a[1] <= a[2] <= ...
+        #
+        # or the longest descending sequence, with
+        #
+        #     a[0] > a[1] > a[2] > ...
+        #
+        # Return (run, descending) where descending is False in the former case,
+        # or True in the latter.
+        # For its intended use in a stable mergesort, the strictness of the defn of
+        # "descending" is needed so that the caller can safely reverse a descending
+        # sequence without violating stability (strict > ensures there are no equal
+        # elements to get out of order).
+
+        def count_run(self, a):
+            if a.len <= 1:
+                n = a.len
+                descending = False
+            else:
+                n = 2
+                if self.lt(a.list[a.base + 1], a.list[a.base]):
+                    descending = True
+                    for p in xrange(a.base + 2, a.base + a.len):
+                        if self.lt(a.list[p], a.list[p-1]):
+                            n += 1
+                        else:
+                            break
                 else:
-                    l = p+1
-            assert l == r
-            # The invariants still hold, so pivot >= all in [base, l) and
-            # pivot < all in [l, start), so pivot belongs at l.  Note
-            # that if there are elements equal to pivot, l points to the
-            # first slot after them -- that's why this sort is stable.
-            # Slide over to make room.
-            for p in xrange(start, l, -1):
-                a.list[p] = a.list[p-1]
-            a.list[l] = pivot
+                    descending = False
+                    for p in xrange(a.base + 2, a.base + a.len):
+                        if self.lt(a.list[p], a.list[p-1]):
+                            break
+                        else:
+                            n += 1
+            return ListSlice(a.list, a.base, n), descending
 
-    # Compute the length of the run in the slice "a".
-    # "A run" is the longest ascending sequence, with
-    #
-    #     a[0] <= a[1] <= a[2] <= ...
-    #
-    # or the longest descending sequence, with
-    #
-    #     a[0] > a[1] > a[2] > ...
-    #
-    # Return (run, descending) where descending is False in the former case,
-    # or True in the latter.
-    # For its intended use in a stable mergesort, the strictness of the defn of
-    # "descending" is needed so that the caller can safely reverse a descending
-    # sequence without violating stability (strict > ensures there are no equal
-    # elements to get out of order).
+        # Locate the proper position of key in a sorted vector; if the vector
+        # contains an element equal to key, return the position immediately to the
+        # left of the leftmost equal element -- or to the right of the rightmost
+        # equal element if the flag "rightmost" is set.
+        #
+        # "hint" is an index at which to begin the search, 0 <= hint < a.len.
+        # The closer hint is to the final result, the faster this runs.
+        #
+        # The return value is the index 0 <= k <= a.len such that
+        #
+        #     a[k-1] < key <= a[k]      (if rightmost is False)
+        #     a[k-1] <= key < a[k]      (if rightmost is True)
+        #
+        # as long as the indices are in bound.  IOW, key belongs at index k;
+        # or, IOW, the first k elements of a should precede key, and the last
+        # n-k should follow key.
 
-    def count_run(self, a):
-        if a.len <= 1:
-            n = a.len
-            descending = False
-        else:
-            n = 2
-            if self.lt(a.list[a.base + 1], a.list[a.base]):
-                descending = True
-                for p in xrange(a.base + 2, a.base + a.len):
-                    if self.lt(a.list[p], a.list[p-1]):
-                        n += 1
-                    else:
+        def gallop(self, key, a, hint, rightmost):
+            assert 0 <= hint < a.len
+            if rightmost:
+                lower = self.le   # search for the largest k for which a[k] <= key
+            else:
+                lower = self.lt   # search for the largest k for which a[k] < key
+
+            p = a.base + hint
+            lastofs = 0
+            ofs = 1
+            if lower(a.list[p], key):
+                # a[hint] < key -- gallop right, until
+                #     a[hint + lastofs] < key <= a[hint + ofs]
+
+                maxofs = a.len - hint     # a[a.len-1] is highest
+                while ofs < maxofs:
+                    if lower(a.list[p + ofs], key):
+                        lastofs = ofs
+                        try:
+                            ofs = ovfcheck_lshift(ofs, 1)
+                        except OverflowError:
+                            ofs = maxofs
+                        else:
+                            ofs = ofs + 1
+                    else:  # key <= a[hint + ofs]
                         break
+
+                if ofs > maxofs:
+                    ofs = maxofs
+                # Translate back to offsets relative to a.
+                lastofs += hint
+                ofs += hint
+
             else:
-                descending = False
-                for p in xrange(a.base + 2, a.base + a.len):
-                    if self.lt(a.list[p], a.list[p-1]):
+                # key <= a[hint] -- gallop left, until
+                #     a[hint - ofs] < key <= a[hint - lastofs]
+                maxofs = hint + 1   # a[0] is lowest
+                while ofs < maxofs:
+                    if lower(a.list[p - ofs], key):
                         break
                     else:
-                        n += 1
-        return ListSlice(a.list, a.base, n), descending
+                        # key <= a[hint - ofs]
+                        lastofs = ofs
+                        try:
+                            ofs = ovfcheck_lshift(ofs, 1)
+                        except OverflowError:
+                            ofs = maxofs
+                        else:
+                            ofs = ofs + 1
+                if ofs > maxofs:
+                    ofs = maxofs
+                # Translate back to positive offsets relative to a.
+                lastofs, ofs = hint-ofs, hint-lastofs
 
-    # Locate the proper position of key in a sorted vector; if the vector
-    # contains an element equal to key, return the position immediately to the
-    # left of the leftmost equal element -- or to the right of the rightmost
-    # equal element if the flag "rightmost" is set.
-    #
-    # "hint" is an index at which to begin the search, 0 <= hint < a.len.
-    # The closer hint is to the final result, the faster this runs.
-    #
-    # The return value is the index 0 <= k <= a.len such that
-    #
-    #     a[k-1] < key <= a[k]      (if rightmost is False)
-    #     a[k-1] <= key < a[k]      (if rightmost is True)
-    #
-    # as long as the indices are in bound.  IOW, key belongs at index k;
-    # or, IOW, the first k elements of a should precede key, and the last
-    # n-k should follow key.
+            assert -1 <= lastofs < ofs <= a.len
 
-    def gallop(self, key, a, hint, rightmost):
-        assert 0 <= hint < a.len
-        if rightmost:
-            lower = self.le   # search for the largest k for which a[k] <= key
-        else:
-            lower = self.lt   # search for the largest k for which a[k] < key
+            # Now a[lastofs] < key <= a[ofs], so key belongs somewhere to the
+            # right of lastofs but no farther right than ofs.  Do a binary
+            # search, with invariant a[lastofs-1] < key <= a[ofs].
 
-        p = a.base + hint
-        lastofs = 0
-        ofs = 1
-        if lower(a.list[p], key):
-            # a[hint] < key -- gallop right, until
-            #     a[hint + lastofs] < key <= a[hint + ofs]
+            lastofs += 1
+            while lastofs < ofs:
+                m = lastofs + ((ofs - lastofs) >> 1)
+                if lower(a.list[a.base + m], key):
+                    lastofs = m+1   # a[m] < key
+                else:
+                    ofs = m         # key <= a[m]
 
-            maxofs = a.len - hint     # a[a.len-1] is highest
-            while ofs < maxofs:
-                if lower(a.list[p + ofs], key):
-                    lastofs = ofs
-                    try:
-                        ofs = ovfcheck_lshift(ofs, 1)
-                    except OverflowError:
-                        ofs = maxofs
-                    else:
-                        ofs = ofs + 1
-                else:  # key <= a[hint + ofs]
-                    break
+            assert lastofs == ofs         # so a[ofs-1] < key <= a[ofs]
+            return ofs
 
-            if ofs > maxofs:
-                ofs = maxofs
-            # Translate back to offsets relative to a.
-            lastofs += hint
-            ofs += hint
+        # hint for the annotator: the argument 'rightmost' is always passed in as
+        # a constant (either True or False), so we can specialize the function for
+        # the two cases.  (This is actually needed for technical reasons: the
+        # variable 'lower' must contain a known method, which is the case in each
+        # specialized version but not in the unspecialized one.)
+        gallop._annspecialcase_ = "specialize:arg(4)"
 
-        else:
-            # key <= a[hint] -- gallop left, until
-            #     a[hint - ofs] < key <= a[hint - lastofs]
-            maxofs = hint + 1   # a[0] is lowest
-            while ofs < maxofs:
-                if lower(a.list[p - ofs], key):
-                    break
-                else:
-                    # key <= a[hint - ofs]
-                    lastofs = ofs
-                    try:
-                        ofs = ovfcheck_lshift(ofs, 1)
-                    except OverflowError:
-                        ofs = maxofs
-                    else:
-                        ofs = ofs + 1
-            if ofs > maxofs:
-                ofs = maxofs
-            # Translate back to positive offsets relative to a.
-            lastofs, ofs = hint-ofs, hint-lastofs
+        # ____________________________________________________________
 
-        assert -1 <= lastofs < ofs <= a.len
+        # When we get into galloping mode, we stay there until both runs win less
+        # often than MIN_GALLOP consecutive times.  See listsort.txt for more info.
+        MIN_GALLOP = 7
 
-        # Now a[lastofs] < key <= a[ofs], so key belongs somewhere to the
-        # right of lastofs but no farther right than ofs.  Do a binary
-        # search, with invariant a[lastofs-1] < key <= a[ofs].
-        
-        lastofs += 1
-        while lastofs < ofs:
-            m = lastofs + ((ofs - lastofs) >> 1)
-            if lower(a.list[a.base + m], key):
-                lastofs = m+1   # a[m] < key
-            else:
-                ofs = m         # key <= a[m]
+        def merge_init(self):
+            # This controls when we get *into* galloping mode.  It's initialized
+            # to MIN_GALLOP.  merge_lo and merge_hi tend to nudge it higher for
+            # random data, and lower for highly structured data.
+            self.min_gallop = self.MIN_GALLOP
 
-        assert lastofs == ofs         # so a[ofs-1] < key <= a[ofs]
-        return ofs
+            # A stack of n pending runs yet to be merged.  Run #i starts at
+            # address pending[i].base and extends for pending[i].len elements.
+            # It's always true (so long as the indices are in bounds) that
+            #
+            #     pending[i].base + pending[i].len == pending[i+1].base
+            #
+            # so we could cut the storage for this, but it's a minor amount,
+            # and keeping all the info explicit simplifies the code.
+            self.pending = []
 
-    # hint for the annotator: the argument 'rightmost' is always passed in as
-    # a constant (either True or False), so we can specialize the function for
-    # the two cases.  (This is actually needed for technical reasons: the
-    # variable 'lower' must contain a known method, which is the case in each
-    # specialized version but not in the unspecialized one.)
-    gallop._annspecialcase_ = "specialize:arg(4)"
+        # Merge the slice "a" with the slice "b" in a stable way, in-place.
+        # a.len and b.len must be > 0, and a.base + a.len == b.base.
+        # Must also have that b.list[b.base] < a.list[a.base], that
+        # a.list[a.base+a.len-1] belongs at the end of the merge, and should have
+        # a.len <= b.len.  See listsort.txt for more info.
 
-    # ____________________________________________________________
+        def merge_lo(self, a, b):
+            assert a.len > 0 and b.len > 0 and a.base + a.len == b.base
+            min_gallop = self.min_gallop
+            dest = a.base
+            a = a.copyitems()
 
-    # When we get into galloping mode, we stay there until both runs win less
-    # often than MIN_GALLOP consecutive times.  See listsort.txt for more info.
-    MIN_GALLOP = 7
+            # Invariant: elements in "a" are waiting to be reinserted into the list
+            # at "dest".  They should be merged with the elements of "b".
+            # b.base == dest + a.len.
+            # We use a finally block to ensure that the elements remaining in
+            # the copy "a" are reinserted back into self.list in all cases.
+            try:
+                self.list[dest] = b.popleft()
+                dest += 1
+                if a.len == 1 or b.len == 0:
+                    return
 
-    def merge_init(self):
-        # This controls when we get *into* galloping mode.  It's initialized
-        # to MIN_GALLOP.  merge_lo and merge_hi tend to nudge it higher for
-        # random data, and lower for highly structured data.
-        self.min_gallop = self.MIN_GALLOP
+                while True:
+                    acount = 0   # number of times A won in a row
+                    bcount = 0   # number of times B won in a row
 
-        # A stack of n pending runs yet to be merged.  Run #i starts at
-        # address pending[i].base and extends for pending[i].len elements.
-        # It's always true (so long as the indices are in bounds) that
-        #
-        #     pending[i].base + pending[i].len == pending[i+1].base
-        #
-        # so we could cut the storage for this, but it's a minor amount,
-        # and keeping all the info explicit simplifies the code.
-        self.pending = []
+                    # Do the straightforward thing until (if ever) one run
+                    # appears to win consistently.
+                    while True:
+                        if self.lt(b.list[b.base], a.list[a.base]):
+                            self.list[dest] = b.popleft()
+                            dest += 1
+                            if b.len == 0:
+                                return
+                            bcount += 1
+                            acount = 0
+                            if bcount >= min_gallop:
+                                break
+                        else:
+                            self.list[dest] = a.popleft()
+                            dest += 1
+                            if a.len == 1:
+                                return
+                            acount += 1
+                            bcount = 0
+                            if acount >= min_gallop:
+                                break
 
-    # Merge the slice "a" with the slice "b" in a stable way, in-place.
-    # a.len and b.len must be > 0, and a.base + a.len == b.base.
-    # Must also have that b.list[b.base] < a.list[a.base], that
-    # a.list[a.base+a.len-1] belongs at the end of the merge, and should have
-    # a.len <= b.len.  See listsort.txt for more info.
+                    # One run is winning so consistently that galloping may
+                    # be a huge win.  So try that, and continue galloping until
+                    # (if ever) neither run appears to be winning consistently
+                    # anymore.
+                    min_gallop += 1
 
-    def merge_lo(self, a, b):
-        assert a.len > 0 and b.len > 0 and a.base + a.len == b.base
-        min_gallop = self.min_gallop
-        dest = a.base
-        a = a.copyitems()
+                    while True:
+                        min_gallop -= min_gallop > 1
+                        self.min_gallop = min_gallop
 
-        # Invariant: elements in "a" are waiting to be reinserted into the list
-        # at "dest".  They should be merged with the elements of "b".
-        # b.base == dest + a.len.
-        # We use a finally block to ensure that the elements remaining in
-        # the copy "a" are reinserted back into self.list in all cases.
-        try:
-            self.list[dest] = b.popleft()
-            dest += 1
-            if a.len == 1 or b.len == 0:
-                return
+                        acount = self.gallop(b.list[b.base], a, hint=0,
+                                             rightmost=True)
+                        for p in xrange(a.base, a.base + acount):
+                            self.list[dest] = a.list[p]
+                            dest += 1
+                        a.advance(acount)
+                        # a.len==0 is impossible now if the comparison
+                        # function is consistent, but we can't assume
+                        # that it is.
+                        if a.len <= 1:
+                            return
 
-            while True:
-                acount = 0   # number of times A won in a row
-                bcount = 0   # number of times B won in a row
-
-                # Do the straightforward thing until (if ever) one run
-                # appears to win consistently.
-                while True:
-                    if self.lt(b.list[b.base], a.list[a.base]):
                         self.list[dest] = b.popleft()
                         dest += 1
                         if b.len == 0:
                             return
-                        bcount += 1
-                        acount = 0
-                        if bcount >= min_gallop:
-                            break
-                    else:
+
+                        bcount = self.gallop(a.list[a.base], b, hint=0,
+                                             rightmost=False)
+                        for p in xrange(b.base, b.base + bcount):
+                            self.list[dest] = b.list[p]
+                            dest += 1
+                        b.advance(bcount)
+                        if b.len == 0:
+                            return
+
                         self.list[dest] = a.popleft()
                         dest += 1
                         if a.len == 1:
                             return
-                        acount += 1
-                        bcount = 0
-                        if acount >= min_gallop:
+
+                        if acount < self.MIN_GALLOP and bcount < self.MIN_GALLOP:
                             break
 
-                # One run is winning so consistently that galloping may
-                # be a huge win.  So try that, and continue galloping until
-                # (if ever) neither run appears to be winning consistently
-                # anymore.
-                min_gallop += 1
+                    min_gallop += 1  # penalize it for leaving galloping mode
+                    self.min_gallop = min_gallop
+
+            finally:
+                # The last element of a belongs at the end of the merge, so we copy
+                # the remaining elements of b before the remaining elements of a.
+                assert a.len >= 0 and b.len >= 0
+                for p in xrange(b.base, b.base + b.len):
+                    self.list[dest] = b.list[p]
+                    dest += 1
+                for p in xrange(a.base, a.base + a.len):
+                    self.list[dest] = a.list[p]
+                    dest += 1
+
+        # Same as merge_lo(), but should have a.len >= b.len.
+
+        def merge_hi(self, a, b):
+            assert a.len > 0 and b.len > 0 and a.base + a.len == b.base
+            min_gallop = self.min_gallop
+            dest = b.base + b.len
+            b = b.copyitems()
+
+            # Invariant: elements in "b" are waiting to be reinserted into the list
+            # before "dest".  They should be merged with the elements of "a".
+            # a.base + a.len == dest - b.len.
+            # We use a finally block to ensure that the elements remaining in
+            # the copy "b" are reinserted back into self.list in all cases.
+            try:
+                dest -= 1
+                self.list[dest] = a.popright()
+                if a.len == 0 or b.len == 1:
+                    return
 
                 while True:
-                    min_gallop -= min_gallop > 1
+                    acount = 0   # number of times A won in a row
+                    bcount = 0   # number of times B won in a row
+
+                    # Do the straightforward thing until (if ever) one run
+                    # appears to win consistently.
+                    while True:
+                        nexta = a.list[a.base + a.len - 1]
+                        nextb = b.list[b.base + b.len - 1]
+                        if self.lt(nextb, nexta):
+                            dest -= 1
+                            self.list[dest] = nexta
+                            a.len -= 1
+                            if a.len == 0:
+                                return
+                            acount += 1
+                            bcount = 0
+                            if acount >= min_gallop:
+                                break
+                        else:
+                            dest -= 1
+                            self.list[dest] = nextb
+                            b.len -= 1
+                            if b.len == 1:
+                                return
+                            bcount += 1
+                            acount = 0
+                            if bcount >= min_gallop:
+                                break
+
+                    # One run is winning so consistently that galloping may
+                    # be a huge win.  So try that, and continue galloping until
+                    # (if ever) neither run appears to be winning consistently
+                    # anymore.
+                    min_gallop += 1
+
+                    while True:
+                        min_gallop -= min_gallop > 1
+                        self.min_gallop = min_gallop
+
+                        nextb = b.list[b.base + b.len - 1]
+                        k = self.gallop(nextb, a, hint=a.len-1, rightmost=True)
+                        acount = a.len - k
+                        for p in xrange(a.base + a.len - 1, a.base + k - 1, -1):
+                            dest -= 1
+                            self.list[dest] = a.list[p]
+                        a.len -= acount
+                        if a.len == 0:
+                            return
+
+                        dest -= 1
+                        self.list[dest] = b.popright()
+                        if b.len == 1:
+                            return
+
+                        nexta = a.list[a.base + a.len - 1]
+                        k = self.gallop(nexta, b, hint=b.len-1, rightmost=False)
+                        bcount = b.len - k
+                        for p in xrange(b.base + b.len - 1, b.base + k - 1, -1):
+                            dest -= 1
+                            self.list[dest] = b.list[p]
+                        b.len -= bcount
+                        # b.len==0 is impossible now if the comparison
+                        # function is consistent, but we can't assume
+                        # that it is.
+                        if b.len <= 1:
+                            return
+
+                        dest -= 1
+                        self.list[dest] = a.popright()
+                        if a.len == 0:
+                            return
+
+                        if acount < self.MIN_GALLOP and bcount < self.MIN_GALLOP:
+                            break
+
+                    min_gallop += 1  # penalize it for leaving galloping mode
                     self.min_gallop = min_gallop
 
-                    acount = self.gallop(b.list[b.base], a, hint=0,
-                                         rightmost=True)
-                    for p in xrange(a.base, a.base + acount):
-                        self.list[dest] = a.list[p]
-                        dest += 1
-                    a.advance(acount)
-                    # a.len==0 is impossible now if the comparison
-                    # function is consistent, but we can't assume
-                    # that it is.
-                    if a.len <= 1:
-                        return
+            finally:
+                # The last element of a belongs at the end of the merge, so we copy
+                # the remaining elements of a and then the remaining elements of b.
+                assert a.len >= 0 and b.len >= 0
+                for p in xrange(a.base + a.len - 1, a.base - 1, -1):
+                    dest -= 1
+                    self.list[dest] = a.list[p]
+                for p in xrange(b.base + b.len - 1, b.base - 1, -1):
+                    dest -= 1
+                    self.list[dest] = b.list[p]
 
-                    self.list[dest] = b.popleft()
-                    dest += 1
-                    if b.len == 0:
-                        return
+        # Merge the two runs at stack indices i and i+1.
 
-                    bcount = self.gallop(a.list[a.base], b, hint=0,
-                                         rightmost=False)
-                    for p in xrange(b.base, b.base + bcount):
-                        self.list[dest] = b.list[p]
-                        dest += 1
-                    b.advance(bcount)
-                    if b.len == 0:
-                        return
+        def merge_at(self, i):
+            a = self.pending[i]
+            b = self.pending[i+1]
+            assert a.len > 0 and b.len > 0
+            assert a.base + a.len == b.base
 
-                    self.list[dest] = a.popleft()
-                    dest += 1
-                    if a.len == 1:
-                        return
+            # Record the length of the combined runs and remove the run b
+            self.pending[i] = ListSlice(self.list, a.base, a.len + b.len)
+            del self.pending[i+1]
 
-                    if acount < self.MIN_GALLOP and bcount < self.MIN_GALLOP:
-                        break
-
-                min_gallop += 1  # penalize it for leaving galloping mode
-                self.min_gallop = min_gallop
-
-        finally:
-            # The last element of a belongs at the end of the merge, so we copy
-            # the remaining elements of b before the remaining elements of a.
-            assert a.len >= 0 and b.len >= 0
-            for p in xrange(b.base, b.base + b.len):
-                self.list[dest] = b.list[p]
-                dest += 1
-            for p in xrange(a.base, a.base + a.len):
-                self.list[dest] = a.list[p]
-                dest += 1
-
-    # Same as merge_lo(), but should have a.len >= b.len.
-
-    def merge_hi(self, a, b):
-        assert a.len > 0 and b.len > 0 and a.base + a.len == b.base
-        min_gallop = self.min_gallop
-        dest = b.base + b.len
-        b = b.copyitems()
-
-        # Invariant: elements in "b" are waiting to be reinserted into the list
-        # before "dest".  They should be merged with the elements of "a".
-        # a.base + a.len == dest - b.len.
-        # We use a finally block to ensure that the elements remaining in
-        # the copy "b" are reinserted back into self.list in all cases.
-        try:
-            dest -= 1
-            self.list[dest] = a.popright()
-            if a.len == 0 or b.len == 1:
+            # Where does b start in a?  Elements in a before that can be
+            # ignored (already in place).
+            k = self.gallop(b.list[b.base], a, hint=0, rightmost=True)
+            a.advance(k)
+            if a.len == 0:
                 return
 
-            while True:
-                acount = 0   # number of times A won in a row
-                bcount = 0   # number of times B won in a row
+            # Where does a end in b?  Elements in b after that can be
+            # ignored (already in place).
+            b.len = self.gallop(a.list[a.base+a.len-1], b, hint=b.len-1,
+                                rightmost=False)
+            if b.len == 0:
+                return
 
-                # Do the straightforward thing until (if ever) one run
-                # appears to win consistently.
-                while True:
-                    nexta = a.list[a.base + a.len - 1]
-                    nextb = b.list[b.base + b.len - 1]
-                    if self.lt(nextb, nexta):
-                        dest -= 1
-                        self.list[dest] = nexta
-                        a.len -= 1
-                        if a.len == 0:
-                            return
-                        acount += 1
-                        bcount = 0
-                        if acount >= min_gallop:
-                            break
+            # Merge what remains of the runs.  The direction is chosen to
+            # minimize the temporary storage needed.
+            if a.len <= b.len:
+                self.merge_lo(a, b)
+            else:
+                self.merge_hi(a, b)
+
+        # Examine the stack of runs waiting to be merged, merging adjacent runs
+        # until the stack invariants are re-established:
+        #
+        # 1. len[-3] > len[-2] + len[-1]
+        # 2. len[-2] > len[-1]
+        #
+        # See listsort.txt for more info.
+
+        def merge_collapse(self):
+            p = self.pending
+            while len(p) > 1:
+                if len(p) >= 3 and p[-3].len <= p[-2].len + p[-1].len:
+                    if p[-3].len < p[-1].len:
+                        self.merge_at(-3)
                     else:
-                        dest -= 1
-                        self.list[dest] = nextb
-                        b.len -= 1
-                        if b.len == 1:
-                            return
-                        bcount += 1
-                        acount = 0
-                        if bcount >= min_gallop:
-                            break
+                        self.merge_at(-2)
+                elif p[-2].len <= p[-1].len:
+                    self.merge_at(-2)
+                else:
+                    break
 
-                # One run is winning so consistently that galloping may
-                # be a huge win.  So try that, and continue galloping until
-                # (if ever) neither run appears to be winning consistently
-                # anymore.
-                min_gallop += 1
+        # Regardless of invariants, merge all runs on the stack until only one
+        # remains.  This is used at the end of the mergesort.
 
-                while True:
-                    min_gallop -= min_gallop > 1
-                    self.min_gallop = min_gallop
-
-                    nextb = b.list[b.base + b.len - 1]
-                    k = self.gallop(nextb, a, hint=a.len-1, rightmost=True)
-                    acount = a.len - k
-                    for p in xrange(a.base + a.len - 1, a.base + k - 1, -1):
-                        dest -= 1
-                        self.list[dest] = a.list[p]
-                    a.len -= acount
-                    if a.len == 0:
-                        return
-
-                    dest -= 1
-                    self.list[dest] = b.popright()
-                    if b.len == 1:
-                        return
-
-                    nexta = a.list[a.base + a.len - 1]
-                    k = self.gallop(nexta, b, hint=b.len-1, rightmost=False)
-                    bcount = b.len - k
-                    for p in xrange(b.base + b.len - 1, b.base + k - 1, -1):
-                        dest -= 1
-                        self.list[dest] = b.list[p]
-                    b.len -= bcount
-                    # b.len==0 is impossible now if the comparison
-                    # function is consistent, but we can't assume
-                    # that it is.
-                    if b.len <= 1:
-                        return
-
-                    dest -= 1
-                    self.list[dest] = a.popright()
-                    if a.len == 0:
-                        return
-
-                    if acount < self.MIN_GALLOP and bcount < self.MIN_GALLOP:
-                        break
-
-                min_gallop += 1  # penalize it for leaving galloping mode
-                self.min_gallop = min_gallop
-
-        finally:
-            # The last element of a belongs at the end of the merge, so we copy
-            # the remaining elements of a and then the remaining elements of b.
-            assert a.len >= 0 and b.len >= 0
-            for p in xrange(a.base + a.len - 1, a.base - 1, -1):
-                dest -= 1
-                self.list[dest] = a.list[p]
-            for p in xrange(b.base + b.len - 1, b.base - 1, -1):
-                dest -= 1
-                self.list[dest] = b.list[p]
-
-    # Merge the two runs at stack indices i and i+1.
-
-    def merge_at(self, i):
-        a = self.pending[i]
-        b = self.pending[i+1]
-        assert a.len > 0 and b.len > 0
-        assert a.base + a.len == b.base
-
-        # Record the length of the combined runs and remove the run b
-        self.pending[i] = ListSlice(self.list, a.base, a.len + b.len)
-        del self.pending[i+1]
-
-        # Where does b start in a?  Elements in a before that can be
-        # ignored (already in place).
-        k = self.gallop(b.list[b.base], a, hint=0, rightmost=True)
-        a.advance(k)
-        if a.len == 0:
-            return
-
-        # Where does a end in b?  Elements in b after that can be
-        # ignored (already in place).
-        b.len = self.gallop(a.list[a.base+a.len-1], b, hint=b.len-1,
-                            rightmost=False)
-        if b.len == 0:
-            return
-
-        # Merge what remains of the runs.  The direction is chosen to
-        # minimize the temporary storage needed.
-        if a.len <= b.len:
-            self.merge_lo(a, b)
-        else:
-            self.merge_hi(a, b)
-
-    # Examine the stack of runs waiting to be merged, merging adjacent runs
-    # until the stack invariants are re-established:
-    #
-    # 1. len[-3] > len[-2] + len[-1]
-    # 2. len[-2] > len[-1]
-    #
-    # See listsort.txt for more info.
-
-    def merge_collapse(self):
-        p = self.pending
-        while len(p) > 1:
-            if len(p) >= 3 and p[-3].len <= p[-2].len + p[-1].len:
-                if p[-3].len < p[-1].len:
+        def merge_force_collapse(self):
+            p = self.pending
+            while len(p) > 1:
+                if len(p) >= 3 and p[-3].len < p[-1].len:
                     self.merge_at(-3)
                 else:
                     self.merge_at(-2)
-            elif p[-2].len <= p[-1].len:
-                self.merge_at(-2)
-            else:
-                break
 
-    # Regardless of invariants, merge all runs on the stack until only one
-    # remains.  This is used at the end of the mergesort.
+        # Compute a good value for the minimum run length; natural runs shorter
+        # than this are boosted artificially via binary insertion.
+        #
+        # If n < 64, return n (it's too small to bother with fancy stuff).
+        # Else if n is an exact power of 2, return 32.
+        # Else return an int k, 32 <= k <= 64, such that n/k is close to, but
+        # strictly less than, an exact power of 2.
+        #
+        # See listsort.txt for more info.
 
-    def merge_force_collapse(self):
-        p = self.pending
-        while len(p) > 1:
-            if len(p) >= 3 and p[-3].len < p[-1].len:
-                self.merge_at(-3)
-            else:
-                self.merge_at(-2)
+        def merge_compute_minrun(self, n):
+            r = 0    # becomes 1 if any 1 bits are shifted off
+            while n >= 64:
+                r |= n & 1
+                n >>= 1
+            return n + r
 
-    # Compute a good value for the minimum run length; natural runs shorter
-    # than this are boosted artificially via binary insertion.
-    #
-    # If n < 64, return n (it's too small to bother with fancy stuff).
-    # Else if n is an exact power of 2, return 32.
-    # Else return an int k, 32 <= k <= 64, such that n/k is close to, but
-    # strictly less than, an exact power of 2.
-    #
-    # See listsort.txt for more info.
+        # ____________________________________________________________
+        # Entry point.
 
-    def merge_compute_minrun(self, n):
-        r = 0    # becomes 1 if any 1 bits are shifted off
-        while n >= 64:
-            r |= n & 1
-            n >>= 1
-        return n + r
+        def sort(self):
+            remaining = ListSlice(self.list, 0, self.listlength)
+            if remaining.len < 2:
+                return
 
-    # ____________________________________________________________
-    # Entry point.
+            # March over the array once, left to right, finding natural runs,
+            # and extending short natural runs to minrun elements.
+            self.merge_init()
+            minrun = self.merge_compute_minrun(remaining.len)
 
-    def sort(self):
-        remaining = ListSlice(self.list, 0, self.listlength)
-        if remaining.len < 2:
-            return
+            while remaining.len > 0:
+                # Identify next run.
+                run, descending = self.count_run(remaining)
+                if descending:
+                    run.reverse()
+                # If short, extend to min(minrun, nremaining).
+                if run.len < minrun:
+                    sorted = run.len
+                    run.len = min(minrun, remaining.len)
+                    self.binarysort(run, sorted)
+                # Advance remaining past this run.
+                remaining.advance(run.len)
+                # Push run onto pending-runs stack, and maybe merge.
+                self.pending.append(run)
+                self.merge_collapse()
 
-        # March over the array once, left to right, finding natural runs,
-        # and extending short natural runs to minrun elements.
-        self.merge_init()
-        minrun = self.merge_compute_minrun(remaining.len)
+            assert remaining.base == self.listlength
 
-        while remaining.len > 0:
-            # Identify next run.
-            run, descending = self.count_run(remaining)
-            if descending:
-                run.reverse()
-            # If short, extend to min(minrun, nremaining).
-            if run.len < minrun:
-                sorted = run.len
-                run.len = min(minrun, remaining.len)
-                self.binarysort(run, sorted)
-            # Advance remaining past this run.
-            remaining.advance(run.len)
-            # Push run onto pending-runs stack, and maybe merge.
-            self.pending.append(run)
-            self.merge_collapse()
+            self.merge_force_collapse()
+            assert len(self.pending) == 1
+            assert self.pending[0].base == 0
+            assert self.pending[0].len == self.listlength
 
-        assert remaining.base == self.listlength
 
-        self.merge_force_collapse()
-        assert len(self.pending) == 1
-        assert self.pending[0].base == 0
-        assert self.pending[0].len == self.listlength
+    class ListSlice:
+        "A sublist of a list."
 
+        def __init__(self, list, base, len):
+            self.list = list
+            self.base = base
+            self.len  = len
 
-class ListSlice:
-    "A sublist of a list."
+        def copyitems(self):
+            "Make a copy of the slice of the original list."
+            start = self.base
+            stop  = self.base + self.len
+            assert 0 <= start <= stop     # annotator hint
+            return ListSlice(self.list[start:stop], 0, self.len)
 
-    def __init__(self, list, base, len):
-        self.list = list
-        self.base = base
-        self.len  = len
+        def advance(self, n):
+            self.base += n
+            self.len -= n
 
-    def copyitems(self):
-        "Make a copy of the slice of the original list."
-        start = self.base
-        stop  = self.base + self.len
-        assert 0 <= start <= stop     # annotator hint
-        return ListSlice(self.list[start:stop], 0, self.len)
+        def popleft(self):
+            result = self.list[self.base]
+            self.base += 1
+            self.len -= 1
+            return result
 
-    def advance(self, n):
-        self.base += n
-        self.len -= n
+        def popright(self):
+            self.len -= 1
+            return self.list[self.base + self.len]
 
-    def popleft(self):
-        result = self.list[self.base]
-        self.base += 1
-        self.len -= 1
-        return result
+        def reverse(self):
+            "Reverse the slice in-place."
+            list = self.list
+            lo = self.base
+            hi = lo + self.len - 1
+            while lo < hi:
+                list[lo], list[hi] = list[hi], list[lo]
+                lo += 1
+                hi -= 1
+    return TimSort
 
-    def popright(self):
-        self.len -= 1
-        return self.list[self.base + self.len]
-
-    def reverse(self):
-        "Reverse the slice in-place."
-        list = self.list
-        lo = self.base
-        hi = lo + self.len - 1
-        while lo < hi:
-            list[lo], list[hi] = list[hi], list[lo]
-            lo += 1
-            hi -= 1
+TimSort = make_timsort_class() #backward compatible interface

From noreply at buildbot.pypy.org  Thu Sep  8 14:52:12 2011
From: noreply at buildbot.pypy.org (l.diekmann)
Date: Thu,  8 Sep 2011 14:52:12 +0200 (CEST)
Subject: [pypy-commit] pypy default: merge
Message-ID: <20110908125212.EF38E8203C@wyvern.cs.uni-duesseldorf.de>

Author: Lukas Diekmann 
Branch: 
Changeset: r47162:e6ecaadbcaf8
Date: 2011-09-08 14:51 +0200
http://bitbucket.org/pypy/pypy/changeset/e6ecaadbcaf8/

Log:	merge

diff --git a/pypy/module/micronumpy/interp_dtype.py b/pypy/module/micronumpy/interp_dtype.py
--- a/pypy/module/micronumpy/interp_dtype.py
+++ b/pypy/module/micronumpy/interp_dtype.py
@@ -317,6 +317,17 @@
 class W_Int8Dtype(IntegerArithmeticDtype, W_Int8Dtype):
     pass
 
+W_UInt8Dtype = create_low_level_dtype(
+    num = 1, kind = SIGNEDLTR, name = "uint8",
+    aliases = ["uint8"],
+    applevel_types = [],
+    T = rffi.UCHAR,
+    valtype = rffi.UCHAR._type,
+    expected_size = 1,
+)
+class W_UInt8Dtype(IntegerArithmeticDtype, W_UInt8Dtype):
+    pass
+
 W_Int16Dtype = create_low_level_dtype(
     num = 3, kind = SIGNEDLTR, name = "int16",
     aliases = ["int16"],
@@ -368,6 +379,7 @@
 ALL_DTYPES = [
     W_BoolDtype,
     W_Int8Dtype, W_Int16Dtype, W_Int32Dtype, W_Int64Dtype,
+    W_UInt8Dtype,
     W_Float64Dtype
 ]
 
diff --git a/pypy/module/micronumpy/test/test_dtypes.py b/pypy/module/micronumpy/test/test_dtypes.py
--- a/pypy/module/micronumpy/test/test_dtypes.py
+++ b/pypy/module/micronumpy/test/test_dtypes.py
@@ -12,6 +12,7 @@
         assert dtype(d) is d
         assert dtype(None) is dtype(float)
         raises(TypeError, dtype, 1042)
+        assert dtype('uint8').num == 1
 
     def test_dtype_with_types(self):
         from numpy import dtype
@@ -90,6 +91,15 @@
         for i in range(5):
             assert b[i] == i * 2
 
+    def test_add_uint8(self):
+        from numpy import array, dtype
+
+        a = array(range(5), dtype="uint8")
+        b = a + a
+        assert b.dtype is dtype("uint8")
+        for i in range(5):
+            assert b[i] == i * 2
+
     def test_add_int16(self):
         from numpy import array, dtype
 
@@ -109,3 +119,15 @@
 
         # You can't subclass dtype
         raises(TypeError, type, "Foo", (dtype,), {})
+
+    def test_int_ranges(self):
+        from numpy import array
+        for dtype, minval, maxval in [("int8", -128, 127),
+                                      ("uint8", 0, 255),
+                                      ("int16", -32768, 32767)]:
+            a = array([minval, maxval, minval-1, maxval+1], dtype)
+            assert a[0] == minval
+            assert a[1] == maxval
+            assert a[2] == maxval
+            assert a[3] == minval
+            

From noreply at buildbot.pypy.org  Thu Sep  8 18:39:42 2011
From: noreply at buildbot.pypy.org (hager)
Date: Thu,  8 Sep 2011 18:39:42 +0200 (CEST)
Subject: [pypy-commit] pypy ppc-jit-backend: Added a basic test for register
	allocation.
Message-ID: <20110908163942.333BE8203C@wyvern.cs.uni-duesseldorf.de>

Author: hager 
Branch: ppc-jit-backend
Changeset: r47163:5dbba4ebe9c8
Date: 2011-09-08 15:22 +0200
http://bitbucket.org/pypy/pypy/changeset/5dbba4ebe9c8/

Log:	Added a basic test for register allocation.

diff --git a/pypy/jit/backend/test/runner_test.py b/pypy/jit/backend/test/runner_test.py
--- a/pypy/jit/backend/test/runner_test.py
+++ b/pypy/jit/backend/test/runner_test.py
@@ -113,6 +113,25 @@
         assert res == 3
         assert fail.identifier == 1
 
+    def test_basic_register_allocator(self):
+        i0 = BoxInt()
+        operations = []
+        prev_box = i0
+        for _ in range(500):
+            next_box = BoxInt()
+            operations.append(
+                ResOperation(rop.INT_ADD, [prev_box, ConstInt(1)], next_box))
+            prev_box = next_box
+        operations.append(ResOperation(rop.FINISH, [prev_box], None, descr=BasicFailDescr(1)))
+        inputargs = [i0]
+        looptoken = LoopToken()
+        self.cpu.compile_loop(inputargs, operations, looptoken)
+        self.cpu.set_future_value_int(0, 20)
+        fail = self.cpu.execute_token(looptoken)
+        res = self.cpu.get_latest_value_int(0)
+        assert res == 520
+        assert fail.identifier == 1
+
     def test_compile_linear_float_loop(self):
         if not self.cpu.supports_floats:
             py.test.skip("floats not supported")

From noreply at buildbot.pypy.org  Thu Sep  8 18:39:43 2011
From: noreply at buildbot.pypy.org (hager)
Date: Thu,  8 Sep 2011 18:39:43 +0200 (CEST)
Subject: [pypy-commit] pypy ppc-jit-backend: (arigo, hager):
Message-ID: <20110908163943.6BEED8203C@wyvern.cs.uni-duesseldorf.de>

Author: hager 
Branch: ppc-jit-backend
Changeset: r47164:f1da4802e995
Date: 2011-09-08 15:29 +0200
http://bitbucket.org/pypy/pypy/changeset/f1da4802e995/

Log:	(arigo, hager): Started refactoring of the PPC cpu and assembler.
	The goal is to have code which is similar to the ARM backend. Most
	important: Use register allocator.

diff --git a/pypy/jit/backend/llsupport/regalloc.py b/pypy/jit/backend/llsupport/regalloc.py
--- a/pypy/jit/backend/llsupport/regalloc.py
+++ b/pypy/jit/backend/llsupport/regalloc.py
@@ -1,4 +1,3 @@
-
 from pypy.jit.metainterp.history import Const, Box, REF
 from pypy.rlib.objectmodel import we_are_translated
 from pypy.jit.metainterp.resoperation import rop
diff --git a/pypy/jit/backend/ppc/ppcgen/arch.py b/pypy/jit/backend/ppc/ppcgen/arch.py
--- a/pypy/jit/backend/ppc/ppcgen/arch.py
+++ b/pypy/jit/backend/ppc/ppcgen/arch.py
@@ -10,5 +10,10 @@
     IS_PPC_32 = False
     IS_PPC_64 = True
 
+ALL_REGS        = range(32)
 NONVOLATILES    = [2] + range(13, 32)
 VOLATILES       = [0] + range(3, 13)
+
+MY_COPY_OF_REGS = 0
+
+GPR_SAVE_AREA   = len(NONVOLATILES) * WORD
diff --git a/pypy/jit/backend/ppc/ppcgen/locations.py b/pypy/jit/backend/ppc/ppcgen/locations.py
new file mode 100644
--- /dev/null
+++ b/pypy/jit/backend/ppc/ppcgen/locations.py
@@ -0,0 +1,62 @@
+from pypy.jit.metainterp.history import INT, FLOAT, REF
+from pypy.jit.backend.arm.arch import WORD
+class AssemblerLocation(object):
+    _immutable_ = True
+    type = INT
+
+    def is_imm(self):
+        return False
+
+    def is_stack(self):
+        return False
+
+    def is_reg(self):
+        return False
+
+    def is_vfp_reg(self):
+        return False
+
+    def is_imm_float(self):
+        return False
+
+    def as_key(self):
+        raise NotImplementedError
+
+class RegisterLocation(AssemblerLocation):
+    _immutable_ = True
+    width = WORD
+
+    def __init__(self, value):
+        self.value = value
+
+    def __repr__(self):
+        return 'r%d' % self.value
+
+    def is_reg(self):
+        return True
+
+    def as_key(self):
+        return self.value
+
+class ImmLocation(AssemblerLocation):
+    _immutable_ = True
+    width = WORD
+
+
+    def __init__(self, value):
+        self.value = value
+
+    def getint(self):
+        return self.value
+
+    def __repr__(self):
+        return "imm(%d)" % (self.value)
+
+    def is_imm(self):
+        return True
+
+    def as_key(self):
+        return self.value + 40
+
+def imm(val):
+    return ImmLocation(val)
diff --git a/pypy/jit/backend/ppc/ppcgen/ppc_assembler.py b/pypy/jit/backend/ppc/ppcgen/ppc_assembler.py
--- a/pypy/jit/backend/ppc/ppcgen/ppc_assembler.py
+++ b/pypy/jit/backend/ppc/ppcgen/ppc_assembler.py
@@ -2,16 +2,25 @@
 import struct
 from pypy.jit.backend.ppc.ppcgen.ppc_form import PPCForm as Form
 from pypy.jit.backend.ppc.ppcgen.ppc_field import ppc_fields
+from pypy.jit.backend.ppc.ppcgen.regalloc import (TempInt, PPCFrameManager,
+                                                  Regalloc)
 from pypy.jit.backend.ppc.ppcgen.assembler import Assembler
 from pypy.jit.backend.ppc.ppcgen.symbol_lookup import lookup
-from pypy.jit.backend.ppc.ppcgen.arch import IS_PPC_32, WORD, NONVOLATILES
+from pypy.jit.backend.ppc.ppcgen.arch import (IS_PPC_32, WORD, NONVOLATILES,
+                                              GPR_SAVE_AREA)
+import pypy.jit.backend.ppc.ppcgen.register as r
 from pypy.jit.metainterp.history import Const, ConstPtr
 from pypy.jit.backend.llsupport.asmmemmgr import BlockBuilderMixin
 from pypy.jit.backend.llsupport.asmmemmgr import AsmMemoryManager
+from pypy.jit.backend.llsupport.regalloc import (RegisterManager, 
+                                                 compute_vars_longevity)
 from pypy.jit.backend.llsupport import symbolic
+from pypy.jit.backend.model import CompiledLoopToken
 from pypy.rpython.lltypesystem import lltype, rffi, rstr
 from pypy.jit.metainterp.resoperation import rop
-from pypy.jit.metainterp.history import BoxInt, ConstInt, Box
+from pypy.jit.metainterp.history import (BoxInt, ConstInt, ConstPtr,
+                                         ConstFloat, Box, INT, REF, FLOAT)
+from pypy.jit.backend.x86.support import values_array
 
 A = Form("frD", "frA", "frB", "XO3", "Rc")
 A1 = Form("frD", "frB", "XO3", "Rc")
@@ -906,10 +915,13 @@
     return (w >> 16) & 0x0000FFFF
 
 class PPCBuilder(PPCAssembler):
-    def __init__(self):
+    def __init__(self, cpu, failargs_limit=1000):
         PPCAssembler.__init__(self)
+        self.cpu = cpu
+        self.fail_boxes_int = values_array(lltype.Signed, failargs_limit)
 
-    def load_word(self, rD, word):
+    def load_imm(self, rD, word):
+        rD = rD.as_key()
         if word <= 32767 and word >= -32768:
             self.li(rD, word)
         elif IS_PPC_32 or (word <= 2147483647 and word >= -2147483648):
@@ -923,7 +935,7 @@
             self.oris(rD, rD, high(word))
             self.ori(rD, rD, lo(word))
 
-    def load_from(self, rD, addr):
+    def load_from_addr(self, rD, addr):
         if IS_PPC_32:
             self.addis(rD, 0, ha(addr))
             self.lwz(rD, rD, la(addr))
@@ -932,26 +944,78 @@
             self.ld(rD, rD, 0)
 
     def store_reg(self, source_reg, addr):
-        self.load_word(0, addr)
+        self.load_imm(r.r0, addr)
         if IS_PPC_32:
-            self.stwx(source_reg, 0, 0)
+            self.stwx(source_reg.value, 0, 0)
         else:
-            self.stdx(source_reg, 0, 0)
+            # ? 
+            self.std(source_reg.value, 0, 10)
 
-    def save_nonvolatiles(self, framesize):
+    def _save_nonvolatiles(self):
         for i, reg in enumerate(NONVOLATILES):
-            if IS_PPC_32:
-                self.stw(reg, 1, framesize - WORD * i)
+            self.stw(reg, 1, self.framesize - 4 * i)
+
+    def _restore_nonvolatiles(self):
+        for i, reg in enumerate(NONVOLATILES):
+            self.lwz(reg, 1, self.framesize - i * 4)
+
+    def _make_prologue(self):
+        self.stwu(1, 1, -self.framesize)
+        self.mflr(0)
+        self.stw(0, 1, self.framesize + 4)
+        self._save_nonvolatiles()
+
+    def _make_epilogue(self):
+        self._restore_nonvolatiles()
+
+    def gen_bootstrap_code(self, nonfloatlocs, inputargs):
+        for i in range(len(nonfloatlocs)):
+            loc = nonfloatlocs[i]
+            arg = inputargs[i]
+            assert arg.type != FLOAT
+            if arg.type == INT:
+                addr = self.fail_boxes_int.get_addr_for_num(i)
+            elif args.type == REF:
+                addr = self.fail_boxes_ptr.get_addr_for_num(i)
             else:
-                self.std(reg, 1, framesize - WORD * i)
+                assert 0, "%s not supported" % arg.type
+            if loc.is_reg():
+                reg = loc
+            else:
+                assert 0, "FIX LATER"
+            self.load_from_addr(reg.value, addr)
 
-    def restore_nonvolatiles(self, framesize):
-        for i, reg in enumerate(NONVOLATILES):
-            if IS_PPC_32:
-                self.lwz(reg, 1, framesize - WORD * i)
+    def assemble_loop(self, inputargs, operations, looptoken, log):
+        self.framesize = 256 + GPR_SAVE_AREA
+        clt = CompiledLoopToken(self.cpu, looptoken.number)
+        looptoken.compiled_loop_token = clt
+
+        longevity = compute_vars_longevity(inputargs, operations)
+        regalloc = Regalloc(longevity, assembler=self,
+                            frame_manager=PPCFrameManager())
+
+        self._make_prologue()
+        nonfloatlocs = regalloc.prepare_loop(inputargs, operations, looptoken)
+        self.gen_bootstrap_code(nonfloatlocs, inputargs)
+        self._walk_operations(operations, regalloc)
+        looptoken.ppc_code = self.assemble(True)
+
+    def _walk_operations(self, operations, regalloc):
+        while regalloc.position() < len(operations) - 1:
+            regalloc.next_instruction()
+            pos = regalloc.position()
+            op = operations[pos]
+            opnum = op.getopnum()
+            if op.has_no_side_effect() and op.result not in regalloc.longevity:
+                regalloc.possibly_free_vars_for_op(op)
             else:
-                self.ld(reg, 1, framesize - WORD * i)
-        
+                arglocs = regalloc.operations[opnum](regalloc, op)
+                if arglocs is not None:
+                    self.operations[opnum](self, op, arglocs, regalloc)
+            if op.result:
+                regalloc.possibly_free_var(op.result)
+            regalloc.possibly_free_vars_for_op(op)
+            regalloc._check_invariants()
 
     # translate a trace operation to corresponding machine code
     def build_op(self, trace_op, cpu):
@@ -994,14 +1058,23 @@
         if isinstance(arg0, Box):
             reg0 = cpu.reg_map[arg0]
         else:
-            reg0 = cpu.get_next_register()
+            #reg0 = cpu.get_next_register()
+            box = TempInt()
+            reg0 = cpu.rm.force_allocate_reg(box)
             self.load_word(reg0, arg0.value)
         if isinstance(arg1, Box):
             reg1 = cpu.reg_map[arg1]
         else:
-            reg1 = cpu.get_next_register()
+            #reg1 = cpu.get_next_register()
+            #reg1 = cpu.rm.force_allocate_reg(arg1)
+            box = TempInt()
+            reg1 = cpu.rm.force_allocate_reg(box)
+            boxed = cpu.rm.make_sure_var_in_reg(box)
             self.load_word(reg1, arg1.value)
-        free_reg = cpu.next_free_register
+            import pdb; pdb.set_trace()
+        #free_reg = cpu.next_free_register
+        free_reg = cpu.rm.force_allocate_reg(op.result)
+
         return free_reg, reg0, reg1
 
     def _int_op_epilog(self, op, cpu, result_reg):
@@ -1045,8 +1118,14 @@
     #             CODE GENERATION             #
     # --------------------------------------- #
 
-    def emit_int_add(self, op, cpu, reg0, reg1, free_reg):
-        self.add(free_reg, reg0, reg1)
+    def emit_int_add(self, op, arglocs, regalloc):
+        l0, l1, res = arglocs
+        if l0.is_imm():
+            self.addi(res.value, l1.value, l0.value)
+        elif l1.is_imm():
+            self.addi(res.value, l0.value, l1.value)
+        else:
+            self.add(res.value, l0.value, l1.value)
 
     def emit_int_add_ovf(self, op, cpu, reg0, reg1, free_reg):
         self.addo(free_reg, reg0, reg1)
@@ -1419,7 +1498,10 @@
         arg_reg = 3
         for arg in args:
             if isinstance(arg, Box):
-                self.mr(arg_reg, cpu.reg_map[arg])
+                try:
+                    self.mr(arg_reg, cpu.reg_map[arg])
+                except KeyError:
+                    self.lwz(arg_reg, 1, cpu.mem_map[arg])
             elif isinstance(arg, Const):
                 self.load_word(arg_reg, arg.value)
             else:
@@ -1435,16 +1517,15 @@
             for i, arg in enumerate(remaining_args):
                 if isinstance(arg, Box):
                     #self.mr(0, cpu.reg_map[arg])
-                    if IS_PPC_32:
+                    try:
                         self.stw(cpu.reg_map[arg], 1, 8 + WORD * i)
-                    else:
-                        self.std(cpu.reg_map[arg], 1, 8 + WORD * i)
+                    except KeyError:
+                        self.load_word(0, cpu.mem_map[arg])
+                        self.lwzx(0, 1, 0)
+                        self.stw(0, 1, 8 + WORD * i)
                 elif isinstance(arg, Const):
                     self.load_word(0, arg.value)
-                    if IS_PPC_32:
-                        self.stw(0, 1, 8 + WORD * i)
-                    else:
-                        self.std(0, 1, 8 + WORD * i)
+                    self.stw(0, 1, 8 + WORD * i)
                 else:
                     assert 0, "%s not supported yet" % arg
 
@@ -1584,34 +1665,23 @@
 
     #_____________________________________
 
-    def emit_finish(self, op, cpu):
+    def emit_finish(self, op, arglocs, regalloc):
         descr = op.getdescr()
-        identifier = self._get_identifier_from_descr(descr, cpu)
-        cpu.saved_descr[identifier] = descr
+        identifier = self._get_identifier_from_descr(descr, self.cpu)
+        self.cpu.saved_descr[identifier] = descr
         args = op.getarglist()
-        for index, arg in enumerate(args):
-            if isinstance(arg, Box):
-                regnum = cpu.reg_map[arg]
-                addr = cpu.fail_boxes_int.get_addr_for_num(index)
-                self.store_reg(regnum, addr)
-            elif isinstance(arg, ConstInt):
-                addr = cpu.fail_boxes_int.get_addr_for_num(index)
-                self.load_word(cpu.next_free_register, arg.value)
-                self.store_reg(cpu.next_free_register, addr)
-            else:
-                assert 0, "arg type not suported"
+        for index, arg in enumerate(arglocs):
+            addr = self.fail_boxes_int.get_addr_for_num(index)
+            self.store_reg(arg, addr)
 
-        framesize = 16 * WORD + 20 * WORD
+        framesize = 256 + GPR_SAVE_AREA
 
-        self.restore_nonvolatiles(framesize)
+        self._restore_nonvolatiles()
 
-        if IS_PPC_32:
-            self.lwz(0, 1, framesize + WORD) # 36
-        else:
-            self.ld(0, 1, framesize + WORD) # 36
+        self.lwz(0, 1, framesize + 4) # 36
         self.mtlr(0)
         self.addi(1, 1, framesize)
-        self.load_word(3, identifier)
+        self.load_imm(r.r3, identifier)
         self.blr()
 
     def emit_jump(self, op, cpu):
@@ -1694,7 +1764,7 @@
             oplist[val] = not_implemented
     return oplist
 
-PPCBuilder.oplist = make_operations()
+PPCBuilder.operations = make_operations()
 
 if __name__ == '__main__':
     main()
diff --git a/pypy/jit/backend/ppc/ppcgen/regalloc.py b/pypy/jit/backend/ppc/ppcgen/regalloc.py
new file mode 100644
--- /dev/null
+++ b/pypy/jit/backend/ppc/ppcgen/regalloc.py
@@ -0,0 +1,218 @@
+from pypy.jit.backend.llsupport.regalloc import (RegisterManager, FrameManager,
+                                                 TempBox, compute_vars_longevity,
+                                                 compute_loop_consts)
+from pypy.jit.backend.ppc.ppcgen.arch import (WORD, MY_COPY_OF_REGS)
+from pypy.jit.metainterp.history import INT, REF, Const, ConstInt, ConstPtr
+from pypy.jit.metainterp.resoperation import rop
+from pypy.jit.backend.ppc.ppcgen import locations
+from pypy.rpython.lltypesystem import rffi, lltype
+import pypy.jit.backend.ppc.ppcgen.register as r
+
+class TempInt(TempBox):
+    type = INT
+
+    def __repr__(self):
+        return "" % (id(self),)
+
+class TempPtr(TempBox):
+    type = REF
+
+    def __repr__(self):
+        return "" % (id(self),)
+
+class PPCRegisterManager(RegisterManager):
+    all_regs              = r.ALL_REGS
+    box_types             = None       # or a list of acceptable types
+    no_lower_byte_regs    = all_regs
+    save_around_call_regs = r.VOLATILES
+
+    REGLOC_TO_COPY_AREA_OFS = {
+        r.r0:   MY_COPY_OF_REGS + 0 * WORD,
+        r.r2:   MY_COPY_OF_REGS + 1 * WORD,
+        r.r3:   MY_COPY_OF_REGS + 2 * WORD,
+        r.r4:   MY_COPY_OF_REGS + 3 * WORD,
+        r.r5:   MY_COPY_OF_REGS + 4 * WORD,
+        r.r6:   MY_COPY_OF_REGS + 5 * WORD,
+        r.r7:   MY_COPY_OF_REGS + 6 * WORD,
+        r.r8:   MY_COPY_OF_REGS + 7 * WORD,
+        r.r9:   MY_COPY_OF_REGS + 8 * WORD,
+        r.r10:  MY_COPY_OF_REGS + 9 * WORD,
+        r.r11:  MY_COPY_OF_REGS + 10 * WORD,
+        r.r12:  MY_COPY_OF_REGS + 11 * WORD,
+        r.r13:  MY_COPY_OF_REGS + 12 * WORD,
+        r.r14:  MY_COPY_OF_REGS + 13 * WORD,
+        r.r15:  MY_COPY_OF_REGS + 14 * WORD,
+        r.r16:  MY_COPY_OF_REGS + 15 * WORD,
+        r.r17:  MY_COPY_OF_REGS + 16 * WORD,
+        r.r18:  MY_COPY_OF_REGS + 17 * WORD,
+        r.r19:  MY_COPY_OF_REGS + 18 * WORD,
+        r.r20:  MY_COPY_OF_REGS + 19 * WORD,
+        r.r21:  MY_COPY_OF_REGS + 20 * WORD,
+        r.r22:  MY_COPY_OF_REGS + 21 * WORD,
+        r.r23:  MY_COPY_OF_REGS + 22 * WORD,
+        r.r24:  MY_COPY_OF_REGS + 23 * WORD,
+        r.r25:  MY_COPY_OF_REGS + 24 * WORD,
+        r.r26:  MY_COPY_OF_REGS + 25 * WORD,
+        r.r27:  MY_COPY_OF_REGS + 26 * WORD,
+        r.r28:  MY_COPY_OF_REGS + 27 * WORD,
+        r.r29:  MY_COPY_OF_REGS + 28 * WORD,
+        r.r30:  MY_COPY_OF_REGS + 29 * WORD,
+        r.r31:  MY_COPY_OF_REGS + 30 * WORD,
+    }
+
+    def __init__(self, longevity, frame_manager=None, assembler=None):
+        RegisterManager.__init__(self, longevity, frame_manager, assembler)
+
+    def call_result_location(self, v):
+        return r.r3
+
+    def convert_to_imm(self, c):
+        if isinstance(c, ConstInt):
+            return locations.ImmLocation(c.value)
+        else:
+            assert isinstance(c, ConstPtr)
+            return locations.ImmLocation(rffi.cast(lltype.Signed, c.value))
+
+class PPCFrameManager(FrameManager):
+    def __init__(self):
+        FrameManager.__init__(self)
+        self.frame_depth = 1
+
+class Regalloc(object):
+    def __init__(self, longevity, frame_manager=None, assembler=None):
+        self.cpu = assembler.cpu
+        self.longevity = longevity
+        self.frame_manager = frame_manager
+        self.assembler = assembler
+        self.rm = PPCRegisterManager(longevity, frame_manager, assembler)
+
+    def prepare_loop(self, inputargs, operations, looptoken):
+        loop_consts = compute_loop_consts(inputargs, operations[-1], looptoken)
+        inputlen = len(inputargs)
+        nonfloatlocs = [None] * len(inputargs)
+        for i in range(inputlen):
+            arg = inputargs[i]
+            assert not isinstance(arg, Const)
+            if arg not in loop_consts and self.longevity[arg][1] > -1:
+                self.try_allocate_reg(arg)
+            loc = self.loc(arg)
+            nonfloatlocs[i] = loc
+        self.possibly_free_vars(inputargs)
+        return nonfloatlocs
+
+    def possibly_free_var(self, var):
+        self.rm.possibly_free_var(var)
+
+    def possibly_free_vars(self, vars):
+        for var in vars:
+            self.possibly_free_var(var)
+
+    def possibly_free_vars_for_op(self, op):
+        for i in range(op.numargs()):
+            var = op.getarg(i)
+            if var is not None:
+                self.possibly_free_var(var)
+
+    def try_allocate_reg(self, v, selected_reg=None, need_lower_byte=False):
+        return self.rm.try_allocate_reg(v, selected_reg, need_lower_byte)
+
+    def force_allocate_reg(self, var, forbidden_vars=[], selected_reg=None, 
+            need_lower_byte=False):
+        return self.rm.force_allocate_reg(var, forbidden_vars, selected_reg,
+                need_lower_byte)
+
+    def _check_invariants(self):
+        self.rm._check_invariants()
+
+    def loc(self, var):
+        return self.rm.loc(var)
+
+    def position(self):
+        return self.rm.position
+
+    def next_instruction(self):
+        self.rm.next_instruction()
+
+    def _check_imm_arg(self, arg):
+        return isinstance(arg, ConstInt)
+
+    def _ensure_value_is_boxed(self, thing, forbidden_vars=[]):
+        box = None
+        loc = None
+        if isinstance(thing, Const):
+            if isinstance(thing, ConstPtr):
+                box = TempPtr()
+            else:
+                box = TempInt()
+            loc = self.force_allocate_reg(box, forbidden_vars=forbidden_vars)
+            imm = self.rm.convert_to_imm(thing)
+            self.assembler.load_word(loc, imm)
+        else:
+            loc = self.make_sure_var_in_reg(thing,
+                    forbidden_vars=forbidden_vars)
+            box = thing
+        return loc, box
+
+    def make_sure_var_in_reg(self, var, forbidden_vars=[],
+                             selected_reg=None, need_lower_byte=False):
+        return self.rm.make_sure_var_in_reg(var, forbidden_vars,
+                selected_reg, need_lower_byte)
+
+    # ******************************************************
+    # *         P R E P A R E  O P E R A T I O N S         * 
+    # ******************************************************
+
+    def prepare_int_add(self, op):
+        boxes = op.getarglist()
+        b0, b1 = boxes
+        imm_b0 = self._check_imm_arg(b0)
+        imm_b1 = self._check_imm_arg(b1)
+        if not imm_b0 and imm_b1:
+            l0, box = self._ensure_value_is_boxed(b0)
+            l1 = self.make_sure_var_in_reg(b1, [b0])
+            boxes.append(box)
+        elif imm_b0 and not imm_b1:
+            l0 = self.make_sure_var_in_reg(b0)
+            l1, box = self._ensure_value_is_boxed(b1, [b0])
+            boxes.append(box)
+        else:
+            l0, box = self._ensure_value_is_boxed(b0)
+            boxes.append(box)
+            l1, box = self._ensure_value_is_boxed(b1, [box])
+            boxes.append(box)
+        #return [l0, l1], boxes
+        locs = [l0, l1]
+        self.possibly_free_vars(boxes)
+        res = self.force_allocate_reg(op.result)
+        return locs + [res]
+
+    def prepare_finish(self, op):
+        #args = [locations.imm(self.frame_manager.frame_depth)]
+        args = []
+        for i in range(op.numargs()):
+            arg = op.getarg(i)
+            if arg:
+                args.append(self.loc(arg))
+                self.possibly_free_var(arg)
+            else:
+                args.append(None)
+        return args
+
+def make_operation_list():
+    def not_implemented(self, op, *args):
+        raise NotImplementedError, op
+
+    operations = [None] * (rop._LAST + 1)
+    for key, val in rop.__dict__.items():
+        key = key.lower()
+        if key.startswith("_"):
+            continue
+        methname = "prepare_%s" % key
+        if hasattr(Regalloc, methname):
+            func = getattr(Regalloc, methname).im_func
+        else:
+            func = not_implemented
+        operations[val] = func
+    return operations
+
+Regalloc.operations = make_operation_list()
diff --git a/pypy/jit/backend/ppc/ppcgen/register.py b/pypy/jit/backend/ppc/ppcgen/register.py
new file mode 100644
--- /dev/null
+++ b/pypy/jit/backend/ppc/ppcgen/register.py
@@ -0,0 +1,11 @@
+from pypy.jit.backend.ppc.ppcgen.locations import RegisterLocation
+
+ALL_REGS = [RegisterLocation(i) for i in range(32)]
+
+r0, r1, r2, r3, r4, r5, r6, r7, r8, r9, r10, r11, r12, r13, r14, r15, r16,\
+    r17, r18, r19, r20, r21, r22, r23, r24, r25, r26, r27, r28, r29, r30, r31\
+    = ALL_REGS
+
+NONVOLATILES    = [r2, r13, r14, r15, r16, r17, r18, r19, r20, r21, r22, r23,
+                    r24, r25, r26, r27, r28, r29, r30, r31]
+VOLATILES       = [r0, r3, r4, r5, r6, r7, r8, r9, r10, r11, r12]
diff --git a/pypy/jit/backend/ppc/runner.py b/pypy/jit/backend/ppc/runner.py
--- a/pypy/jit/backend/ppc/runner.py
+++ b/pypy/jit/backend/ppc/runner.py
@@ -12,7 +12,8 @@
 from pypy.jit.backend.x86 import regloc
 from pypy.jit.backend.x86.support import values_array
 from pypy.jit.backend.ppc.ppcgen.ppc_assembler import PPCBuilder
-from pypy.jit.backend.ppc.ppcgen.arch import IS_PPC_32, NONVOLATILES
+from pypy.jit.backend.ppc.ppcgen.arch import NONVOLATILES, GPR_SAVE_AREA, WORD
+from pypy.jit.backend.ppc.ppcgen.regalloc import PPCRegisterManager, PPCFrameManager
 import sys
 
 from pypy.tool.ansi_print import ansi_log
@@ -28,48 +29,15 @@
         AbstractLLCPU.__init__(self, rtyper, stats, opts,
                                translate_support_code, gcdescr)
 
-        # pointer to an array of ints
-        # XXX length of the integer array is 1000 for now
-        self.fail_boxes_int = values_array(lltype.Signed, 1000)
-
         # floats are not supported yet
         self.supports_floats = False
         self.total_compiled_loops = 0
         self.total_compiled_bridges = 0
+        self.asm = PPCBuilder(self)
 
-    # compile a given trace
-    def compile_loop(self, inputargs, operations, looptoken, log=True):
+    def compile_loop(self, inputargs, operations, looptoken, log=False):
         self.saved_descr = {}
-        self.patch_list = []
-        self.reg_map = {}
-        self.fail_box_count = 0
-
-        codebuilder = PPCBuilder()
-        
-        # function prologue
-        self._make_prologue(codebuilder)
-
-        # initialize registers from memory
-        self.next_free_register = 3
-        for index, arg in enumerate(inputargs):
-            self.reg_map[arg] = self.next_free_register
-            addr = self.fail_boxes_int.get_addr_for_num(index)
-            codebuilder.load_from(self.next_free_register, addr)
-            self.next_free_register += 1
-        
-        self.startpos = codebuilder.get_relative_pos()
-
-        # generate code for operations
-        self._walk_trace_ops(codebuilder, operations)
-
-        # function epilogue
-        self._make_epilogue(codebuilder)
-
-        f = codebuilder.assemble()
-        looptoken.ppc_code = f
-        looptoken.codebuilder = codebuilder
-        self.total_compiled_loops += 1
-        self.teardown()
+        self.asm.assemble_loop(inputargs, operations, looptoken, log)
 
     def compile_bridge(self, descr, inputargs, operations, looptoken):
         self.saved_descr = {}
@@ -110,18 +78,7 @@
         self.next_free_register += 1
         return reg
 
-    def _make_prologue(self, codebuilder):
-        framesize = 16 * WORD + 20 * WORD
-        if IS_PPC_32:
-            codebuilder.stwu(1, 1, -framesize)
-            codebuilder.mflr(0)
-            codebuilder.stw(0, 1, framesize + WORD)
-        else:
-            codebuilder.stdu(1, 1, -framesize)
-            codebuilder.mflr(0)
-            codebuilder.std(0, 1, framesize + WORD)
-        codebuilder.save_nonvolatiles(framesize)
-
+    # XXX not used by now, move to ppc_assembler
     def _make_epilogue(self, codebuilder):
         for op_index, fail_index, guard, reglist in self.patch_list:
             curpos = codebuilder.get_relative_pos()
@@ -147,22 +104,17 @@
             descr.patch_pos = patch_pos
             descr.used_mem_indices = used_mem_indices
 
-            framesize = 16 * WORD + 20 * WORD
-            codebuilder.restore_nonvolatiles(framesize)
+            codebuilder.restore_nonvolatiles(self.framesize)
 
-            if IS_PPC_32:
-                codebuilder.lwz(0, 1, framesize + WORD) # 36
-            else:
-                codebuilder.ld(0, 1, framesize + WORD) # 36
+            codebuilder.lwz(0, 1, self.framesize + 4)
             codebuilder.mtlr(0)
-            codebuilder.addi(1, 1, framesize)
-
+            codebuilder.addi(1, 1, self.framesize)
             codebuilder.li(3, fail_index)            
             codebuilder.blr()
 
     # set value in fail_boxes_int
     def set_future_value_int(self, index, value_int):
-        self.fail_boxes_int.setitem(index, value_int)
+        self.asm.fail_boxes_int.setitem(index, value_int)
 
     def set_future_value_ref(self, index, pointer):
         sign_ptr = rffi.cast(lltype.Signed, pointer)
@@ -174,8 +126,9 @@
 
     # executes the stored machine code in the token
     def execute_token(self, looptoken):   
-        descr_index = looptoken.ppc_code()
-        return self.saved_descr[descr_index]
+        addr = looptoken.ppc_code
+        fail_index = addr()
+        return self.saved_descr[fail_index]
 
     # return the number of values that can be returned
     def get_latest_value_count(self):
@@ -183,7 +136,7 @@
 
     # fetch the result of the computation and return it
     def get_latest_value_int(self, index):
-        value = self.fail_boxes_int.getitem(index)
+        value = self.asm.fail_boxes_int.getitem(index)
         return value
 
     def get_latest_value_ref(self, index):

From noreply at buildbot.pypy.org  Thu Sep  8 18:39:44 2011
From: noreply at buildbot.pypy.org (hager)
Date: Thu,  8 Sep 2011 18:39:44 +0200 (CEST)
Subject: [pypy-commit] pypy ppc-jit-backend: merge
Message-ID: <20110908163944.9AA1C8203C@wyvern.cs.uni-duesseldorf.de>

Author: hager 
Branch: ppc-jit-backend
Changeset: r47165:1965c8f56c96
Date: 2011-09-08 15:48 +0200
http://bitbucket.org/pypy/pypy/changeset/1965c8f56c96/

Log:	merge

diff --git a/pypy/jit/backend/ppc/runner.py b/pypy/jit/backend/ppc/runner.py
--- a/pypy/jit/backend/ppc/runner.py
+++ b/pypy/jit/backend/ppc/runner.py
@@ -107,6 +107,10 @@
             codebuilder.restore_nonvolatiles(self.framesize)
 
             codebuilder.lwz(0, 1, self.framesize + 4)
+            if IS_PPC_32:
+                codebuilder.lwz(0, 1, framesize + WORD) # 36
+            else:
+                codebuilder.ld(0, 1, framesize + WORD) # 36
             codebuilder.mtlr(0)
             codebuilder.addi(1, 1, self.framesize)
             codebuilder.li(3, fail_index)            

From noreply at buildbot.pypy.org  Thu Sep  8 18:40:42 2011
From: noreply at buildbot.pypy.org (antocuni)
Date: Thu,  8 Sep 2011 18:40:42 +0200 (CEST)
Subject: [pypy-commit] pypy.org extradoc: (vak) add some docs about creating
	a tarball
Message-ID: <20110908164042.CB7878203C@wyvern.cs.uni-duesseldorf.de>

Author: Antonio Cuni 
Branch: extradoc
Changeset: r257:426ba9681185
Date: 2011-09-08 16:23 +0200
http://bitbucket.org/pypy/pypy.org/changeset/426ba9681185/

Log:	(vak) add some docs about creating a tarball

diff --git a/download.html b/download.html
--- a/download.html
+++ b/download.html
@@ -57,6 +57,7 @@
 

Packaging

Checksums

Current people of PyPy

People of PyPy

Armin Rigo

Samuele Pedroni

Many more people

(UNRELEASED DRAFT) Call for donations - PyPy to support Python3!

Planned stages of work

About estimates and costs

Step 1: core language

Step 2: extension modules

Step 3: cpyext

Benefits of This Work to the Python Community and the General Public

Planned stages of work

About estimates and costs

${Markup(title)}

Meta

Meta

Meta

(UNRELEASED DRAFT) Call for donations - PyPy to support Python3!

Call for donations - PyPy to support Python3!

Error response

Directory listing for %s

Error response

Directory listing for %s

Hello, ${'world'}!

${document.title}