[pypy-commit] pypy py3.5: hg merge default
rlamy
pypy.commits at gmail.com
Fri Sep 15 11:38:12 EDT 2017
Author: Ronan Lamy <ronan.lamy at gmail.com>
Branch: py3.5
Changeset: r92403:3d6ac44c6fff
Date: 2017-09-15 16:37 +0100
http://bitbucket.org/pypy/pypy/changeset/3d6ac44c6fff/
Log: hg merge default
diff --git a/pypy/doc/whatsnew-head.rst b/pypy/doc/whatsnew-head.rst
--- a/pypy/doc/whatsnew-head.rst
+++ b/pypy/doc/whatsnew-head.rst
@@ -81,3 +81,7 @@
.. branch: pycheck-macros
Convert many Py*_Check cpyext functions into macros, like CPython.
+
+.. branch: py_ssize_t
+
+Explicitly use Py_ssize_t as the Signed type in pypy c-api
diff --git a/pypy/doc/windows.rst b/pypy/doc/windows.rst
--- a/pypy/doc/windows.rst
+++ b/pypy/doc/windows.rst
@@ -114,12 +114,15 @@
INCLUDE, LIB and PATH (for DLLs) environment variables appropriately.
-Abridged method (for -Ojit builds using Visual Studio 2008)
------------------------------------------------------------
+Abridged method (using Visual Studio 2008)
+------------------------------------------
Download the versions of all the external packages from
+https://bitbucket.org/pypy/pypy/downloads/local_59.zip
+(for post-5.8 builds) with sha256 checksum
+``0f96c045db1f5f73ad0fae7857caa69c261324bd8e51f6d2ad1fa842c4a5f26f``
https://bitbucket.org/pypy/pypy/downloads/local_5.8.zip
-(for post-5.7.1 builds) with sha256 checksum
+(to reproduce 5.8 builds) with sha256 checksum
``fbe769bf3a4ab6f5a8b0a05b61930fc7f37da2a9a85a8f609cf5a9bad06e2554`` or
https://bitbucket.org/pypy/pypy/downloads/local_2.4.zip
(for 2.4 release and later) or
@@ -135,8 +138,8 @@
Now you should be good to go. If you choose this method, you do not need
to download/build anything else.
-Nonabrided method (building from scratch)
------------------------------------------
+Nonabridged method (building from scratch)
+------------------------------------------
If you want to, you can rebuild everything from scratch by continuing.
@@ -209,17 +212,85 @@
The expat XML parser
~~~~~~~~~~~~~~~~~~~~
-Download the source code of expat on sourceforge:
-https://github.com/libexpat/libexpat/releases and extract it in the base directory.
-Version 2.1.1 is known to pass tests. Then open the project file ``expat.dsw``
-with Visual Studio; follow the instruction for converting the project files,
-switch to the "Release" configuration, use the ``expat_static`` project,
-reconfigure the runtime for Multi-threaded DLL (/MD) and build. Do the same for
-the ``expat`` project to build the ``expat.dll`` (for tests via ll2ctypes)
+CPython compiles expat from source as part of the build. PyPy uses the same
+code base, but expects to link to a static lib of expat. Here are instructions
+to reproduce the static lib in version 2.2.4.
-Then, copy the file ``win32\bin\release\libexpat.lib`` into
-LIB, and both ``lib\expat.h`` and ``lib\expat_external.h`` in
-INCLUDE, and ``win32\bin\release\libexpat.dll`` into PATH.
+Download the source code of expat: https://github.com/libexpat/libexpat.
+``git checkout`` the proper tag, in this case ``R_2_2_4``. Run
+``vcvars.bat`` to set up the visual compiler tools, and CD into the source
+directory. Create a file ``stdbool.h`` with the content
+
+.. code-block:: c
+
+ #pragma once
+
+ #define false 0
+ #define true 1
+
+ #define bool int
+
+and put it in a place on the ``INCLUDE`` path, or create it in the local
+directory and add ``.`` to the ``INCLUDE`` path::
+
+ SET INCLUDE=%INCLUDE%;.
+
+Then compile all the ``*.c`` file into ``*.obj``::
+
+ cl.exe /nologo /MD /O2 *c /c
+ rem for debug
+ cl.exe /nologo /MD /O0 /Ob0 /Zi *c /c
+
+You may need to move some variable declarations to the beginning of the
+function, to be compliant with C89 standard. Here is the diff for version 2.2.4
+
+.. code-block:: diff
+
+ diff --git a/expat/lib/xmltok.c b/expat/lib/xmltok.c
+ index 007aed0..a2dcaad 100644
+ --- a/expat/lib/xmltok.c
+ +++ b/expat/lib/xmltok.c
+ @@ -399,19 +399,21 @@ utf8_toUtf8(const ENCODING *UNUSED_P(enc),
+ /* Avoid copying partial characters (due to limited space). */
+ const ptrdiff_t bytesAvailable = fromLim - *fromP;
+ const ptrdiff_t bytesStorable = toLim - *toP;
+ + const char * fromLimBefore;
+ + ptrdiff_t bytesToCopy;
+ if (bytesAvailable > bytesStorable) {
+ fromLim = *fromP + bytesStorable;
+ output_exhausted = true;
+ }
+
+ /* Avoid copying partial characters (from incomplete input). */
+ - const char * const fromLimBefore = fromLim;
+ + fromLimBefore = fromLim;
+ align_limit_to_full_utf8_characters(*fromP, &fromLim);
+ if (fromLim < fromLimBefore) {
+ input_incomplete = true;
+ }
+
+ - const ptrdiff_t bytesToCopy = fromLim - *fromP;
+ + bytesToCopy = fromLim - *fromP;
+ memcpy((void *)*toP, (const void *)*fromP, (size_t)bytesToCopy);
+ *fromP += bytesToCopy;
+ *toP += bytesToCopy;
+
+
+Create ``libexpat.lib`` (for translation) and ``libexpat.dll`` (for tests)::
+
+ cl /LD *.obj libexpat.def /Felibexpat.dll
+ rem for debug
+ rem cl /LDd /Zi *.obj libexpat.def /Felibexpat.dll
+
+ rem this will override the export library created in the step above
+ rem but tests do not need the export library, they load the dll dynamically
+ lib *.obj /out:libexpat.lib
+
+Then, copy
+
+- ``libexpat.lib`` into LIB
+- both ``lib\expat.h`` and ``lib\expat_external.h`` in INCLUDE
+- ``libexpat.dll`` into PATH
The OpenSSL library
@@ -363,7 +434,7 @@
It is probably not too much work if the goal is only to get a translated
PyPy executable, and to run all tests before translation. But you need
to start somewhere, and you should start with some tests in
-rpython/translator/c/test/, like ``test_standalone.py`` and
+``rpython/translator/c/test/``, like ``test_standalone.py`` and
``test_newgc.py``: try to have them pass on top of CPython64/64.
Keep in mind that this runs small translations, and some details may go
@@ -373,7 +444,7 @@
should be something like ``long long``.
What is more generally needed is to review all the C files in
-rpython/translator/c/src for the word ``long``, because this means a
+``rpython/translator/c/src`` for the word ``long``, because this means a
32-bit integer even on Win64. Replace it with ``Signed`` most of the
times. You can replace one with the other without breaking anything on
any other platform, so feel free to.
diff --git a/pypy/module/_csv/interp_csv.py b/pypy/module/_csv/interp_csv.py
--- a/pypy/module/_csv/interp_csv.py
+++ b/pypy/module/_csv/interp_csv.py
@@ -36,7 +36,7 @@
return space.int_w(w_src)
except OperationError as e:
if e.match(space, space.w_TypeError):
- raise oefmt(space.w_TypeError, '"%s" must be a string', attrname)
+ raise oefmt(space.w_TypeError, '"%s" must be an int', attrname)
raise
def _get_str(space, w_src, default, attrname):
diff --git a/pypy/module/_socket/test/test_sock_app.py b/pypy/module/_socket/test/test_sock_app.py
--- a/pypy/module/_socket/test/test_sock_app.py
+++ b/pypy/module/_socket/test/test_sock_app.py
@@ -521,7 +521,12 @@
import _socket
s = _socket.socket()
assert s.getsockopt(_socket.IPPROTO_TCP, _socket.TCP_NODELAY, 0) == 0
- assert s.getsockopt(_socket.IPPROTO_TCP, _socket.TCP_NODELAY, 2) == b'\x00\x00'
+ ret = s.getsockopt(_socket.IPPROTO_TCP, _socket.TCP_NODELAY, 2)
+ if len(ret) == 1:
+ # win32 returns a byte-as-bool
+ assert ret == b'\x00'
+ else:
+ assert ret == b'\x00\x00'
s.setsockopt(_socket.IPPROTO_TCP, _socket.TCP_NODELAY, True)
assert s.getsockopt(_socket.IPPROTO_TCP, _socket.TCP_NODELAY, 0) == 1
s.setsockopt(_socket.IPPROTO_TCP, _socket.TCP_NODELAY, 1)
@@ -531,7 +536,11 @@
import _socket
s = _socket.socket()
buf = s.getsockopt(_socket.IPPROTO_TCP, _socket.TCP_NODELAY, 1024)
- assert buf == b'\x00' * 4
+ if len(buf) == 1:
+ # win32 returns a byte-as-bool
+ assert buf == b'\x00'
+ else:
+ assert buf == b'\x00' * 4
raises(_socket.error, s.getsockopt,
_socket.IPPROTO_TCP, _socket.TCP_NODELAY, 1025)
raises(_socket.error, s.getsockopt,
diff --git a/pypy/module/cpyext/api.py b/pypy/module/cpyext/api.py
--- a/pypy/module/cpyext/api.py
+++ b/pypy/module/cpyext/api.py
@@ -1360,6 +1360,7 @@
source_dir / "mysnprintf.c",
source_dir / "pythonrun.c",
source_dir / "sysmodule.c",
+ source_dir / "complexobject.c",
source_dir / "cobject.c",
source_dir / "structseq.c",
source_dir / "capsule.c",
@@ -1368,7 +1369,6 @@
source_dir / "missing.c",
source_dir / "pymem.c",
source_dir / "bytesobject.c",
- source_dir / "complexobject.c",
source_dir / "import.c",
source_dir / "_warnings.c",
source_dir / "pylifecycle.c",
diff --git a/pypy/module/cpyext/methodobject.py b/pypy/module/cpyext/methodobject.py
--- a/pypy/module/cpyext/methodobject.py
+++ b/pypy/module/cpyext/methodobject.py
@@ -102,9 +102,27 @@
return self.space.unwrap(self.descr_method_repr())
def descr_method_repr(self):
- return self.getrepr(
- self.space, u"built-in method '%s' of '%s' object" %
- (self.name.decode('utf-8'), self.w_objclass.getname(self.space)))
+ return self.space.newtext("<method '%s' of '%s' objects>" % (
+ self.name, self.w_objclass.getname(self.space).encode('utf-8')))
+
+ def descr_call(self, space, __args__):
+ args_w, kw_w = __args__.unpack()
+ if len(args_w) < 1:
+ raise oefmt(space.w_TypeError,
+ "descriptor '%8' of '%N' object needs an argument",
+ self.name, self.w_objclass)
+ w_instance = args_w[0]
+ # XXX: needs a stricter test
+ if not space.isinstance_w(w_instance, self.w_objclass):
+ raise oefmt(space.w_TypeError,
+ "descriptor '%8' requires a '%N' object but received a '%T'",
+ self.name, self.w_objclass, w_instance)
+ w_args = space.newtuple(args_w[1:])
+ w_kw = space.newdict()
+ for key, w_obj in kw_w.items():
+ space.setitem(w_kw, space.newtext(key), w_obj)
+ ret = self.call(space, w_instance, w_args, w_kw)
+ return ret
@cpython_api([PyObject], rffi.INT_real, error=CANNOT_FAIL)
def PyCFunction_Check(space, w_obj):
@@ -200,7 +218,7 @@
ret = self.call(space, None, w_args, w_kw)
return ret
-def cmethod_descr_call(space, w_self, __args__):
+def cclassmethod_descr_call(space, w_self, __args__):
self = space.interp_w(W_PyCFunctionObject, w_self)
args_w, kw_w = __args__.unpack()
if len(args_w) < 1:
@@ -238,9 +256,9 @@
W_PyCFunctionObject.typedef.acceptable_as_base_class = False
W_PyCMethodObject.typedef = TypeDef(
- 'method',
+ 'method_descriptor',
__get__ = interp2app(cmethod_descr_get),
- __call__ = interp2app(cmethod_descr_call),
+ __call__ = interp2app(W_PyCMethodObject.descr_call),
__name__ = interp_attrproperty('name', cls=W_PyCMethodObject,
wrapfn="newtext_or_none"),
__objclass__ = interp_attrproperty_w('w_objclass', cls=W_PyCMethodObject),
@@ -251,7 +269,7 @@
W_PyCClassMethodObject.typedef = TypeDef(
'classmethod',
__get__ = interp2app(cclassmethod_descr_get),
- __call__ = interp2app(cmethod_descr_call),
+ __call__ = interp2app(cclassmethod_descr_call),
__name__ = interp_attrproperty('name', cls=W_PyCClassMethodObject,
wrapfn="newtext_or_none"),
__objclass__ = interp_attrproperty_w('w_objclass',
diff --git a/pypy/module/cpyext/test/issue2482.c b/pypy/module/cpyext/test/issue2482.c
--- a/pypy/module/cpyext/test/issue2482.c
+++ b/pypy/module/cpyext/test/issue2482.c
@@ -18,9 +18,9 @@
PyObject *make_object_base_type(void) {
PyHeapTypeObject *heap_type = (PyHeapTypeObject *) PyType_Type.tp_alloc(&PyType_Type, 0);
- if (!heap_type) return NULL;
PyTypeObject *type = &heap_type->ht_type;
+ if (!heap_type) return NULL;
type->tp_name = name;
#ifdef ISSUE_2482
type->tp_base = &PyBaseObject_Type; /*fails */
@@ -85,16 +85,19 @@
#else
PyObject *module = Py_InitModule("issue2482", issue2482_functions);
#endif
+ PyHeapTypeObject *heap_type;
+ PyTypeObject *type;
+ PyObject * base;
if (module == NULL)
INITERROR;
- PyHeapTypeObject *heap_type = (PyHeapTypeObject *) PyType_Type.tp_alloc(&PyType_Type, 0);
+ heap_type = (PyHeapTypeObject *) PyType_Type.tp_alloc(&PyType_Type, 0);
if (!heap_type) INITERROR;
- PyTypeObject *type = &heap_type->ht_type;
+ type = &heap_type->ht_type;
type->tp_name = name;
- PyObject *base = make_object_base_type();
+ base = make_object_base_type();
if (! base) INITERROR;
Py_INCREF(base);
type->tp_base = (PyTypeObject *) base;
diff --git a/pypy/module/cpyext/test/test_eval.py b/pypy/module/cpyext/test/test_eval.py
--- a/pypy/module/cpyext/test/test_eval.py
+++ b/pypy/module/cpyext/test/test_eval.py
@@ -348,21 +348,26 @@
module = self.import_extension('foo', [
("call_recursive", "METH_NOARGS",
"""
- int res = 0;
- int recurse(void) {
- if (Py_EnterRecursiveCall(" while calling recurse"))
- return -1;
- res ++;
- return recurse();
- }
- int oldlimit = Py_GetRecursionLimit();
+ int oldlimit;
+ int recurse(void);
+ res = 0;
+ oldlimit = Py_GetRecursionLimit();
Py_SetRecursionLimit(200);
res = recurse();
Py_SetRecursionLimit(oldlimit);
if (PyErr_Occurred())
return NULL;
return PyLong_FromLong(res);
- """),], prologue= ''' int recurse(void); '''
+ """),], prologue= '''
+ int res;
+ int recurse(void) {
+ if (Py_EnterRecursiveCall(" while calling recurse")) {
+ return -1;
+ }
+ res ++;
+ return recurse();
+ };
+ '''
)
excinfo = raises(RecursionError, module.call_recursive)
assert 'while calling recurse' in str(excinfo.value)
diff --git a/pypy/module/cpyext/test/test_methodobject.py b/pypy/module/cpyext/test/test_methodobject.py
--- a/pypy/module/cpyext/test/test_methodobject.py
+++ b/pypy/module/cpyext/test/test_methodobject.py
@@ -100,26 +100,3 @@
assert mod.check(A) == 0
assert mod.check(A.meth) == 0
assert mod.check(A.stat) == 0
-
-
-class TestPyCMethodObject(BaseApiTest):
- def test_repr(self, space, api):
- """
- W_PyCMethodObject has a repr string which describes it as a method
- and gives its name and the name of its class.
- """
- def func(space, w_self, w_args):
- return space.w_None
- c_func = ApiFunction([PyObject, PyObject], PyObject, func)
- func.api_func = c_func
- ml = lltype.malloc(PyMethodDef, flavor='raw', zero=True)
- namebuf = rffi.cast(rffi.CONST_CCHARP, rffi.str2charp('func'))
- ml.c_ml_name = namebuf
- ml.c_ml_meth = rffi.cast(PyCFunction, c_func.get_llhelper(space))
-
- method = api.PyDescr_NewMethod(space.w_unicode, ml)
- assert repr(method).startswith(
- "<built-in method 'func' of 'str' object ")
-
- rffi.free_charp(namebuf)
- lltype.free(ml, flavor='raw')
diff --git a/pypy/module/cpyext/test/test_typeobject.py b/pypy/module/cpyext/test/test_typeobject.py
--- a/pypy/module/cpyext/test/test_typeobject.py
+++ b/pypy/module/cpyext/test/test_typeobject.py
@@ -125,6 +125,16 @@
obj = module.fooType.classmeth()
assert obj is module.fooType
+ def test_methoddescr(self):
+ module = self.import_module(name='foo')
+ descr = module.fooType.copy
+ assert type(descr).__name__ == 'method_descriptor'
+ assert str(descr) in ("<method 'copy' of 'foo.foo' objects>",
+ "<method 'copy' of 'foo' objects>")
+ assert repr(descr) in ("<method 'copy' of 'foo.foo' objects>",
+ "<method 'copy' of 'foo' objects>")
+ raises(TypeError, descr, None)
+
def test_new(self):
# XXX cpython segfaults but if run singly (with -k test_new) this passes
module = self.import_module(name='foo')
@@ -1341,9 +1351,9 @@
module = self.import_extension('foo', [
("test_flags", "METH_VARARGS",
'''
- long in_flag, my_flag;
+ long long in_flag, my_flag;
PyObject * obj;
- if (!PyArg_ParseTuple(args, "Ol", &obj, &in_flag))
+ if (!PyArg_ParseTuple(args, "OL", &obj, &in_flag))
return NULL;
if (!PyType_Check(obj))
{
diff --git a/pypy/module/cpyext/test/test_userslots.py b/pypy/module/cpyext/test/test_userslots.py
--- a/pypy/module/cpyext/test/test_userslots.py
+++ b/pypy/module/cpyext/test/test_userslots.py
@@ -190,9 +190,10 @@
0, /* tp_basicsize*/
0 /* tp_itemsize */
};
+ PyObject * mod1;
+ PyObject * dt;
''', more_init='''
- PyObject * mod1 = PyImport_ImportModule("datetime");
- PyObject * dt;
+ mod1 = PyImport_ImportModule("datetime");
if (mod1 == NULL) INITERROR;
dt = PyUnicode_FromString("datetime");
datetime_cls = (PyTypeObject*)PyObject_GetAttr(mod1, dt);
diff --git a/pypy/module/select/test/test_select.py b/pypy/module/select/test/test_select.py
--- a/pypy/module/select/test/test_select.py
+++ b/pypy/module/select/test/test_select.py
@@ -330,7 +330,7 @@
}
import os
- if os.uname()[4] == 's390x':
+ if hasattr(os, 'uname') and os.uname()[4] == 's390x':
py.test.skip("build bot for s390x cannot open sockets")
def w_make_server(self):
diff --git a/rpython/memory/gc/base.py b/rpython/memory/gc/base.py
--- a/rpython/memory/gc/base.py
+++ b/rpython/memory/gc/base.py
@@ -22,6 +22,7 @@
can_usually_pin_objects = False
object_minimal_size = 0
gcflag_extra = 0 # or a real GC flag that is always 0 when not collecting
+ _totalroots_rpy = 0 # for inspector.py
def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE,
translated_to_c=True):
@@ -335,6 +336,7 @@
callback2, attrname = _convert_callback_formats(callback) # :-/
setattr(self, attrname, arg)
self.root_walker.walk_roots(callback2, callback2, callback2)
+ self.enum_live_with_finalizers(callback, arg)
self.enum_pending_finalizers(callback, arg)
enumerate_all_roots._annspecialcase_ = 'specialize:arg(1)'
@@ -347,6 +349,12 @@
i += 1
enum_pending_finalizers._annspecialcase_ = 'specialize:arg(1)'
+ def enum_live_with_finalizers(self, callback, arg):
+ # as far as possible, enumerates the live objects with finalizers,
+ # even if they have not been detected as unreachable yet (but may be)
+ pass
+ enum_live_with_finalizers._annspecialcase_ = 'specialize:arg(1)'
+
def _copy_pending_finalizers_deque(self, deque, copy_fn):
tmp = self.AddressDeque()
while deque.non_empty():
diff --git a/rpython/memory/gc/incminimark.py b/rpython/memory/gc/incminimark.py
--- a/rpython/memory/gc/incminimark.py
+++ b/rpython/memory/gc/incminimark.py
@@ -1897,8 +1897,15 @@
if cardbyte & 1:
if interval_stop > length:
interval_stop = length
- ll_assert(cardbyte <= 1 and bytes == 0,
- "premature end of object")
+ #--- the sanity check below almost always
+ #--- passes, except in situations like
+ #--- test_writebarrier_before_copy_manually\
+ # _copy_card_bits
+ #ll_assert(cardbyte <= 1 and bytes == 0,
+ # "premature end of object")
+ ll_assert(bytes == 0, "premature end of object")
+ if interval_stop <= interval_start:
+ break
self.trace_and_drag_out_of_nursery_partial(
obj, interval_start, interval_stop)
#
@@ -2515,6 +2522,11 @@
MovingGCBase.enumerate_all_roots(self, callback, arg)
enumerate_all_roots._annspecialcase_ = 'specialize:arg(1)'
+ def enum_live_with_finalizers(self, callback, arg):
+ self.probably_young_objects_with_finalizers.foreach(callback, arg, 2)
+ self.old_objects_with_finalizers.foreach(callback, arg, 2)
+ enum_live_with_finalizers._annspecialcase_ = 'specialize:arg(1)'
+
def _collect_obj(self, obj, ignored):
# Ignore pinned objects, which are the ones still in the nursery here.
# Cache effects: don't read any flag out of 'obj' at this point.
diff --git a/rpython/memory/gc/inspector.py b/rpython/memory/gc/inspector.py
--- a/rpython/memory/gc/inspector.py
+++ b/rpython/memory/gc/inspector.py
@@ -10,73 +10,65 @@
# ---------- implementation of rpython.rlib.rgc.get_rpy_roots() ----------
-def _counting_rpy_root(obj, gc):
- gc._count_rpy += 1
-
-def _do_count_rpy_roots(gc):
- gc._count_rpy = 0
- gc.enumerate_all_roots(_counting_rpy_root, gc)
- return gc._count_rpy
-
def _append_rpy_root(obj, gc):
# Can use the gc list, but should not allocate!
# It is essential that the list is not resizable!
lst = gc._list_rpy
index = gc._count_rpy
- if index >= len(lst):
- raise ValueError
gc._count_rpy = index + 1
- lst[index] = llmemory.cast_adr_to_ptr(obj, llmemory.GCREF)
+ if index < len(lst):
+ lst[index] = llmemory.cast_adr_to_ptr(obj, llmemory.GCREF)
+ #else:
+ # too many items. This situation is detected in the 'while' loop below
def _do_append_rpy_roots(gc, lst):
gc._count_rpy = 0
gc._list_rpy = lst
gc.enumerate_all_roots(_append_rpy_root, gc)
gc._list_rpy = None
+ return gc._count_rpy
def get_rpy_roots(gc):
- count = _do_count_rpy_roots(gc)
- extra = 16
+ # returns a list that may end with some NULLs
while True:
- result = [lltype.nullptr(llmemory.GCREF.TO)] * (count + extra)
- try:
- _do_append_rpy_roots(gc, result)
- except ValueError:
- extra *= 3
- else:
+ result = [lltype.nullptr(llmemory.GCREF.TO)] * gc._totalroots_rpy
+ count = _do_append_rpy_roots(gc, result)
+ if count <= len(result): # 'count' fits inside the list
return result
+ count += (count // 8)
+ gc._totalroots_rpy = count + 10
# ---------- implementation of rpython.rlib.rgc.get_rpy_referents() ----------
-def _count_rpy_referent(pointer, gc):
- gc._count_rpy += 1
-
-def _do_count_rpy_referents(gc, gcref):
- gc._count_rpy = 0
- gc.trace(llmemory.cast_ptr_to_adr(gcref), _count_rpy_referent, gc)
- return gc._count_rpy
-
def _append_rpy_referent(pointer, gc):
# Can use the gc list, but should not allocate!
# It is essential that the list is not resizable!
lst = gc._list_rpy
index = gc._count_rpy
- if index >= len(lst):
- raise ValueError
gc._count_rpy = index + 1
- lst[index] = llmemory.cast_adr_to_ptr(pointer.address[0],
- llmemory.GCREF)
+ if index < len(lst):
+ lst[index] = llmemory.cast_adr_to_ptr(pointer.address[0],
+ llmemory.GCREF)
+ #else:
+ # too many items. This situation is detected in the 'while' loop below
def _do_append_rpy_referents(gc, gcref, lst):
gc._count_rpy = 0
gc._list_rpy = lst
gc.trace(llmemory.cast_ptr_to_adr(gcref), _append_rpy_referent, gc)
+ gc._list_rpy = None
+ return gc._count_rpy
def get_rpy_referents(gc, gcref):
- count = _do_count_rpy_referents(gc, gcref)
- result = [lltype.nullptr(llmemory.GCREF.TO)] * count
- _do_append_rpy_referents(gc, gcref, result)
- return result
+ # returns a list with no NULLs
+ result = []
+ while True:
+ count = _do_append_rpy_referents(gc, gcref, result)
+ if count <= len(result): # 'count' fits inside the list
+ if count < len(result):
+ result = result[:count]
+ return result
+ result = [lltype.nullptr(llmemory.GCREF.TO)] * count
# ----------
diff --git a/rpython/memory/gc/minimark.py b/rpython/memory/gc/minimark.py
--- a/rpython/memory/gc/minimark.py
+++ b/rpython/memory/gc/minimark.py
@@ -1399,8 +1399,15 @@
if cardbyte & 1:
if interval_stop > length:
interval_stop = length
- ll_assert(cardbyte <= 1 and bytes == 0,
- "premature end of object")
+ #--- the sanity check below almost always
+ #--- passes, except in situations like
+ #--- test_writebarrier_before_copy_manually\
+ # _copy_card_bits
+ #ll_assert(cardbyte <= 1 and bytes == 0,
+ # "premature end of object")
+ ll_assert(bytes == 0, "premature end of object")
+ if interval_stop <= interval_start:
+ break
self.trace_and_drag_out_of_nursery_partial(
obj, interval_start, interval_stop)
#
@@ -1780,6 +1787,11 @@
MovingGCBase.enumerate_all_roots(self, callback, arg)
enumerate_all_roots._annspecialcase_ = 'specialize:arg(1)'
+ def enum_live_with_finalizers(self, callback, arg):
+ self.probably_young_objects_with_finalizers.foreach(callback, arg, 2)
+ self.old_objects_with_finalizers.foreach(callback, arg, 2)
+ enum_live_with_finalizers._annspecialcase_ = 'specialize:arg(1)'
+
@staticmethod
def _collect_obj(obj, objects_to_trace):
objects_to_trace.append(obj)
diff --git a/rpython/memory/support.py b/rpython/memory/support.py
--- a/rpython/memory/support.py
+++ b/rpython/memory/support.py
@@ -269,22 +269,23 @@
self.index_in_oldest = index + 1
return result
- def foreach(self, callback, arg):
+ def foreach(self, callback, arg, step=1):
"""Invoke 'callback(address, arg)' for all addresses in the deque.
Typically, 'callback' is a bound method and 'arg' can be None.
+ If step > 1, only calls it for addresses multiple of 'step'.
"""
chunk = self.oldest_chunk
index = self.index_in_oldest
while chunk is not self.newest_chunk:
while index < chunk_size:
callback(chunk.items[index], arg)
- index += 1
+ index += step
chunk = chunk.next
- index = 0
+ index -= chunk_size
limit = self.index_in_newest
while index < limit:
callback(chunk.items[index], arg)
- index += 1
+ index += step
foreach._annspecialcase_ = 'specialize:arg(1)'
def delete(self):
diff --git a/rpython/memory/test/gc_test_base.py b/rpython/memory/test/gc_test_base.py
--- a/rpython/memory/test/gc_test_base.py
+++ b/rpython/memory/test/gc_test_base.py
@@ -995,6 +995,31 @@
self.interpret(fn, [])
+ def test_writebarrier_before_copy_manually_copy_card_bits(self):
+ S = lltype.GcStruct('S', ('x', lltype.Char))
+ TP = lltype.GcArray(lltype.Ptr(S))
+ def fn():
+ l1 = lltype.malloc(TP, 65)
+ l2 = lltype.malloc(TP, 33)
+ for i in range(65):
+ l1[i] = lltype.malloc(S)
+ l = lltype.malloc(TP, 100)
+ i = 0
+ while i < 65:
+ l[i] = l1[i]
+ i += 1
+ rgc.ll_arraycopy(l, l2, 0, 0, 33)
+ x = []
+ # force minor collect
+ t = (1, lltype.malloc(S))
+ for i in range(20):
+ x.append(t)
+ for i in range(33):
+ assert l2[i] == l[i]
+ return 0
+
+ self.interpret(fn, [])
+
def test_stringbuilder(self):
def fn():
s = StringBuilder(4)
diff --git a/rpython/memory/test/test_support.py b/rpython/memory/test/test_support.py
--- a/rpython/memory/test/test_support.py
+++ b/rpython/memory/test/test_support.py
@@ -160,6 +160,10 @@
ll.foreach(callback, 42)
assert seen == addrs
+ seen = []
+ ll.foreach(callback, 42, step=2)
+ assert seen == addrs[::2]
+
for a in addrs:
b = ll.popleft()
assert a == b
diff --git a/rpython/rlib/rsocket.py b/rpython/rlib/rsocket.py
--- a/rpython/rlib/rsocket.py
+++ b/rpython/rlib/rsocket.py
@@ -845,6 +845,9 @@
@jit.dont_look_inside
def getsockopt_int(self, level, option):
flag_p = lltype.malloc(rffi.INTP.TO, 1, flavor='raw')
+ # some win32 calls use only a byte to represent a bool
+ # zero out so the result is correct anyway
+ flag_p[0] = rffi.cast(rffi.INT, 0)
try:
flagsize_p = lltype.malloc(_c.socklen_t_ptr.TO, flavor='raw')
try:
diff --git a/rpython/translator/platform/windows.py b/rpython/translator/platform/windows.py
--- a/rpython/translator/platform/windows.py
+++ b/rpython/translator/platform/windows.py
@@ -488,19 +488,19 @@
deps.append('icon.res')
wdeps.append('icon.res')
m.rule('$(DEFAULT_TARGET)', ['$(TARGET)'] + deps,
- ['$(CC_LINK) /nologo /debug %s ' % (' '.join(deps),) + \
+ ['$(CC_LINK) /nologo /debug /LARGEADDRESSAWARE %s ' % (' '.join(deps),) + \
'$(SHARED_IMPORT_LIB) /out:$@ ' + \
'/MANIFEST /MANIFESTFILE:$*.manifest',
'mt.exe -nologo -manifest $*.manifest -outputresource:$@;1',
])
m.rule('$(WTARGET)', ['$(TARGET)'] + wdeps,
- ['$(CC_LINK) /nologo /debug /SUBSYSTEM:WINDOWS %s ' % (' '.join(wdeps),) + \
- '$(SHARED_IMPORT_LIB) /out:$@ ' + \
+ ['$(CC_LINK) /nologo /debug /LARGEADDRESSAWARE /SUBSYSTEM:WINDOWS %s ' % (
+ ' '.join(wdeps),) + '$(SHARED_IMPORT_LIB) /out:$@ ' + \
'/MANIFEST /MANIFESTFILE:$*.manifest',
'mt.exe -nologo -manifest $*.manifest -outputresource:$@;1',
])
m.rule('debugmode_$(DEFAULT_TARGET)', ['debugmode_$(TARGET)']+deps,
- ['$(CC_LINK) /nologo /DEBUG %s ' % (' '.join(deps),) + \
+ ['$(CC_LINK) /nologo /DEBUG /LARGEADDRESSAWARE %s ' % (' '.join(deps),) + \
'debugmode_$(SHARED_IMPORT_LIB) /out:$@',
])
More information about the pypy-commit
mailing list