[pypy-commit] pypy gc-hooks: hg merge default
antocuni
pypy.commits at gmail.com
Thu Apr 12 12:04:04 EDT 2018
Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: gc-hooks
Changeset: r94312:f157561218b4
Date: 2018-04-12 18:03 +0200
http://bitbucket.org/pypy/pypy/changeset/f157561218b4/
Log: hg merge default
diff --git a/LICENSE b/LICENSE
--- a/LICENSE
+++ b/LICENSE
@@ -6,36 +6,36 @@
Except when otherwise stated (look for LICENSE files in directories or
information at the beginning of each file) all software and documentation in
the 'rpython', 'pypy', 'ctype_configure', 'dotviewer', 'demo', 'lib_pypy',
-'py', and '_pytest' directories is licensed as follows:
+'py', and '_pytest' directories is licensed as follows:
The MIT License
- Permission is hereby granted, free of charge, to any person
- obtaining a copy of this software and associated documentation
- files (the "Software"), to deal in the Software without
- restriction, including without limitation the rights to use,
- copy, modify, merge, publish, distribute, sublicense, and/or
- sell copies of the Software, and to permit persons to whom the
+ Permission is hereby granted, free of charge, to any person
+ obtaining a copy of this software and associated documentation
+ files (the "Software"), to deal in the Software without
+ restriction, including without limitation the rights to use,
+ copy, modify, merge, publish, distribute, sublicense, and/or
+ sell copies of the Software, and to permit persons to whom the
Software is furnished to do so, subject to the following conditions:
- The above copyright notice and this permission notice shall be included
+ The above copyright notice and this permission notice shall be included
in all copies or substantial portions of the Software.
- THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
- OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
- THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
- LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
- FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
+ OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
PyPy Copyright holders 2003-2018
------------------------------------
+--------------------------------
Except when otherwise stated (look for LICENSE files or information at
the beginning of each file) the files in the 'pypy' directory are each
-copyrighted by one or more of the following people and organizations:
+copyrighted by one or more of the following people and organizations:
Armin Rigo
Maciej Fijalkowski
@@ -89,13 +89,13 @@
Niko Matsakis
Alexander Hesse
Ludovic Aubry
+ stian
Jacob Hallen
Jason Creighton
Mark Young
Alex Martelli
Spenser Bauman
Michal Bendowski
- stian
Jan de Mooij
Tyler Wade
Vincent Legoll
@@ -123,10 +123,10 @@
Wenzhu Man
Konstantin Lopuhin
John Witulski
+ Jeremy Thurgood
Greg Price
Ivan Sichmann Freitas
Dario Bertini
- Jeremy Thurgood
Mark Pearse
Simon Cross
Tobias Pape
@@ -145,18 +145,19 @@
Adrian Kuhn
tav
Georg Brandl
+ Joannah Nanjekye
Bert Freudenberg
Stian Andreassen
Wanja Saatkamp
Mike Blume
- Joannah Nanjekye
Gerald Klix
Oscar Nierstrasz
Rami Chowdhury
Stefan H. Muller
+ Dodan Mihai
Tim Felgentreff
Eugene Oden
- Dodan Mihai
+ Colin Valliant
Jeff Terrace
Henry Mason
Vasily Kuznetsov
@@ -225,12 +226,14 @@
Vaibhav Sood
Reuben Cummings
Attila Gobi
+ Floris Bruynooghe
Christopher Pope
Tristan Arthur
Christian Tismer
Dan Stromberg
Carl Meyer
Florin Papa
+ Arianna Avanzini
Jens-Uwe Mager
Valentina Mukhamedzhanova
Stefano Parmesan
@@ -250,9 +253,11 @@
Alejandro J. Cura
Vladimir Kryachko
Gabriel
+ Thomas Hisch
Mark Williams
Kunal Grover
Nathan Taylor
+ Barry Hart
Travis Francis Athougies
Yasir Suhail
Sergey Kishchenko
@@ -260,6 +265,7 @@
Lutz Paelike
Ian Foote
Philipp Rustemeuer
+ Logan Chien
Catalin Gabriel Manciu
Jacob Oscarson
Ryan Gonzalez
@@ -295,7 +301,6 @@
Akira Li
Gustavo Niemeyer
Rafał Gałczyński
- Logan Chien
Lucas Stadler
roberto at goyle
Matt Bogosian
@@ -308,6 +313,7 @@
Anna Katrina Dominguez
Kim Jin Su
Amber Brown
+ Miro Hrončok
Anthony Sottile
Nate Bragg
Ben Darnell
@@ -315,7 +321,6 @@
Godefroid Chappelle
Julian Berman
Michael Hudson-Doyle
- Floris Bruynooghe
Stephan Busemann
Dan Colish
timo
@@ -357,6 +362,7 @@
Michael Chermside
Anna Ravencroft
remarkablerocket
+ Pauli Virtanen
Petre Vijiac
Berker Peksag
Christian Muirhead
@@ -381,6 +387,7 @@
Graham Markall
Dan Loewenherz
werat
+ Andrew Stepanov
Niclas Olofsson
Chris Pressey
Tobias Diaz
@@ -395,14 +402,14 @@
m at funkyhat.org
Stefan Marr
- Heinrich-Heine University, Germany
+ Heinrich-Heine University, Germany
Open End AB (formerly AB Strakt), Sweden
- merlinux GmbH, Germany
- tismerysoft GmbH, Germany
- Logilab Paris, France
- DFKI GmbH, Germany
+ merlinux GmbH, Germany
+ tismerysoft GmbH, Germany
+ Logilab Paris, France
+ DFKI GmbH, Germany
Impara, Germany
- Change Maker, Sweden
+ Change Maker, Sweden
University of California Berkeley, USA
Google Inc.
King's College London
@@ -410,14 +417,14 @@
The PyPy Logo as used by http://speed.pypy.org and others was created
by Samuel Reis and is distributed on terms of Creative Commons Share Alike
License.
-
-License for 'lib-python/2.7'
-============================
+
+License for 'lib-python/2.7, lib-python/3'
+==========================================
Except when otherwise stated (look for LICENSE files or copyright/license
-information at the beginning of each file) the files in the 'lib-python/2.7'
+information at the beginning of each file) the files in the 'lib-python'
directory are all copyrighted by the Python Software Foundation and licensed
-under the terms that you can find here: https://docs.python.org/2/license.html
+under the terms that you can find here: https://docs.python.org/3/license.html
License for 'pypy/module/unicodedata/'
======================================
@@ -441,9 +448,9 @@
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
-
+
http://www.apache.org/licenses/LICENSE-2.0
-
+
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
diff --git a/lib-python/2.7/test/test_eof.py b/lib-python/2.7/test/test_eof.py
--- a/lib-python/2.7/test/test_eof.py
+++ b/lib-python/2.7/test/test_eof.py
@@ -5,7 +5,7 @@
class EOFTestCase(unittest.TestCase):
def test_EOFC(self):
- expect = "EOL while scanning string literal (<string>, line 1)"
+ expect = "end of line (EOL) while scanning string literal (<string>, line 1)"
try:
eval("""'this is a test\
""")
@@ -15,7 +15,7 @@
raise test_support.TestFailed
def test_EOFS(self):
- expect = ("EOF while scanning triple-quoted string literal "
+ expect = ("end of file (EOF) while scanning triple-quoted string literal "
"(<string>, line 1)")
try:
eval("""'''this is a test""")
diff --git a/lib-python/2.7/test/test_generators.py b/lib-python/2.7/test/test_generators.py
--- a/lib-python/2.7/test/test_generators.py
+++ b/lib-python/2.7/test/test_generators.py
@@ -398,7 +398,10 @@
0
>>> type(i.gi_frame)
<type 'frame'>
->>> i.gi_running = 42
+
+PyPy prints "readonly attribute 'gi_running'" so ignore the exception detail
+
+>>> i.gi_running = 42 # doctest: +IGNORE_EXCEPTION_DETAIL
Traceback (most recent call last):
...
TypeError: readonly attribute
diff --git a/lib-python/2.7/test/test_traceback.py b/lib-python/2.7/test/test_traceback.py
--- a/lib-python/2.7/test/test_traceback.py
+++ b/lib-python/2.7/test/test_traceback.py
@@ -123,10 +123,7 @@
self.assertEqual(len(err), 4)
self.assertEqual(err[1].strip(), "print(2)")
self.assertIn("^", err[2])
- if check_impl_detail():
- self.assertEqual(err[1].find("p"), err[2].find("^"))
- if check_impl_detail(pypy=True):
- self.assertEqual(err[1].find("2)") + 1, err[2].find("^"))
+ self.assertEqual(err[1].find("p"), err[2].find("^"))
def test_base_exception(self):
# Test that exceptions derived from BaseException are formatted right
diff --git a/lib-python/2.7/threading.py b/lib-python/2.7/threading.py
--- a/lib-python/2.7/threading.py
+++ b/lib-python/2.7/threading.py
@@ -351,6 +351,21 @@
# forward-compatibility reasons we do the same.
waiter.acquire()
gotit = True
+ except AttributeError:
+ # someone patched the 'waiter' class, probably.
+ # Fall back to the standard CPython logic.
+ # See the CPython lib for the comments about it...
+ endtime = _time() + timeout
+ delay = 0.0005 # 500 us -> initial delay of 1 ms
+ while True:
+ gotit = waiter.acquire(0)
+ if gotit:
+ break
+ remaining = endtime - _time()
+ if remaining <= 0:
+ break
+ delay = min(delay * 2, remaining, .05)
+ _sleep(delay)
else:
gotit = waiter.acquire(False)
if not gotit:
diff --git a/pypy/doc/contributor.rst b/pypy/doc/contributor.rst
--- a/pypy/doc/contributor.rst
+++ b/pypy/doc/contributor.rst
@@ -56,13 +56,13 @@
Niko Matsakis
Alexander Hesse
Ludovic Aubry
+ stian
Jacob Hallen
Jason Creighton
Mark Young
Alex Martelli
Spenser Bauman
Michal Bendowski
- stian
Jan de Mooij
Tyler Wade
Vincent Legoll
@@ -90,10 +90,10 @@
Wenzhu Man
Konstantin Lopuhin
John Witulski
+ Jeremy Thurgood
Greg Price
Ivan Sichmann Freitas
Dario Bertini
- Jeremy Thurgood
Mark Pearse
Simon Cross
Tobias Pape
@@ -112,18 +112,19 @@
Adrian Kuhn
tav
Georg Brandl
+ Joannah Nanjekye
Bert Freudenberg
Stian Andreassen
Wanja Saatkamp
Mike Blume
- Joannah Nanjekye
Gerald Klix
Oscar Nierstrasz
Rami Chowdhury
Stefan H. Muller
+ Dodan Mihai
Tim Felgentreff
Eugene Oden
- Dodan Mihai
+ Colin Valliant
Jeff Terrace
Henry Mason
Vasily Kuznetsov
@@ -192,12 +193,14 @@
Vaibhav Sood
Reuben Cummings
Attila Gobi
+ Floris Bruynooghe
Christopher Pope
Tristan Arthur
Christian Tismer
Dan Stromberg
Carl Meyer
Florin Papa
+ Arianna Avanzini
Jens-Uwe Mager
Valentina Mukhamedzhanova
Stefano Parmesan
@@ -221,6 +224,7 @@
Mark Williams
Kunal Grover
Nathan Taylor
+ Barry Hart
Travis Francis Athougies
Yasir Suhail
Sergey Kishchenko
@@ -228,6 +232,7 @@
Lutz Paelike
Ian Foote
Philipp Rustemeuer
+ Logan Chien
Catalin Gabriel Manciu
Jacob Oscarson
Ryan Gonzalez
@@ -263,7 +268,6 @@
Akira Li
Gustavo Niemeyer
Rafał Gałczyński
- Logan Chien
Lucas Stadler
roberto at goyle
Matt Bogosian
@@ -276,6 +280,7 @@
Anna Katrina Dominguez
Kim Jin Su
Amber Brown
+ Miro Hrončok
Anthony Sottile
Nate Bragg
Ben Darnell
@@ -283,7 +288,6 @@
Godefroid Chappelle
Julian Berman
Michael Hudson-Doyle
- Floris Bruynooghe
Stephan Busemann
Dan Colish
timo
@@ -325,6 +329,7 @@
Michael Chermside
Anna Ravencroft
remarkablerocket
+ Pauli Virtanen
Petre Vijiac
Berker Peksag
Christian Muirhead
@@ -349,6 +354,7 @@
Graham Markall
Dan Loewenherz
werat
+ Andrew Stepanov
Niclas Olofsson
Chris Pressey
Tobias Diaz
diff --git a/pypy/doc/cpython_differences.rst b/pypy/doc/cpython_differences.rst
--- a/pypy/doc/cpython_differences.rst
+++ b/pypy/doc/cpython_differences.rst
@@ -10,89 +10,6 @@
PyPy.
-.. _extension-modules:
-
-Extension modules
------------------
-
-List of extension modules that we support:
-
-* Supported as built-in modules (in :source:`pypy/module/`):
-
- __builtin__
- :doc:`__pypy__ <__pypy__-module>`
- _ast
- _codecs
- _collections
- :doc:`_continuation <stackless>`
- :doc:`_ffi <discussion/ctypes-implementation>`
- _hashlib
- _io
- _locale
- _lsprof
- _md5
- :doc:`_minimal_curses <config/objspace.usemodules._minimal_curses>`
- _multiprocessing
- _random
- :doc:`_rawffi <discussion/ctypes-implementation>`
- _sha
- _socket
- _sre
- _ssl
- _warnings
- _weakref
- _winreg
- array
- binascii
- bz2
- cStringIO
- cmath
- `cpyext`_
- crypt
- errno
- exceptions
- fcntl
- gc
- imp
- itertools
- marshal
- math
- mmap
- operator
- parser
- posix
- pyexpat
- select
- signal
- struct
- symbol
- sys
- termios
- thread
- time
- token
- unicodedata
- zipimport
- zlib
-
- When translated on Windows, a few Unix-only modules are skipped,
- and the following module is built instead:
-
- _winreg
-
-* Supported by being rewritten in pure Python (possibly using ``cffi``):
- see the :source:`lib_pypy/` directory. Examples of modules that we
- support this way: ``ctypes``, ``cPickle``, ``cmath``, ``dbm``, ``datetime``...
- Note that some modules are both in there and in the list above;
- by default, the built-in module is used (but can be disabled
- at translation time).
-
-The extension modules (i.e. modules written in C, in the standard CPython)
-that are neither mentioned above nor in :source:`lib_pypy/` are not available in PyPy.
-(You may have a chance to use them anyway with `cpyext`_.)
-
-.. _cpyext: http://morepypy.blogspot.com/2010/04/using-cpython-extension-modules-with.html
-
Differences related to garbage collection strategies
----------------------------------------------------
@@ -559,7 +476,96 @@
environment variable. CPython searches for ``vcvarsall.bat`` somewhere **above**
that value.
+* SyntaxError_ s try harder to give details about the cause of the failure, so
+ the error messages are not the same as in CPython
+
+
+.. _extension-modules:
+
+Extension modules
+-----------------
+
+List of extension modules that we support:
+
+* Supported as built-in modules (in :source:`pypy/module/`):
+
+ __builtin__
+ :doc:`__pypy__ <__pypy__-module>`
+ _ast
+ _codecs
+ _collections
+ :doc:`_continuation <stackless>`
+ :doc:`_ffi <discussion/ctypes-implementation>`
+ _hashlib
+ _io
+ _locale
+ _lsprof
+ _md5
+ :doc:`_minimal_curses <config/objspace.usemodules._minimal_curses>`
+ _multiprocessing
+ _random
+ :doc:`_rawffi <discussion/ctypes-implementation>`
+ _sha
+ _socket
+ _sre
+ _ssl
+ _warnings
+ _weakref
+ _winreg
+ array
+ binascii
+ bz2
+ cStringIO
+ cmath
+ `cpyext`_
+ crypt
+ errno
+ exceptions
+ fcntl
+ gc
+ imp
+ itertools
+ marshal
+ math
+ mmap
+ operator
+ parser
+ posix
+ pyexpat
+ select
+ signal
+ struct
+ symbol
+ sys
+ termios
+ thread
+ time
+ token
+ unicodedata
+ zipimport
+ zlib
+
+ When translated on Windows, a few Unix-only modules are skipped,
+ and the following module is built instead:
+
+ _winreg
+
+* Supported by being rewritten in pure Python (possibly using ``cffi``):
+ see the :source:`lib_pypy/` directory. Examples of modules that we
+ support this way: ``ctypes``, ``cPickle``, ``cmath``, ``dbm``, ``datetime``...
+ Note that some modules are both in there and in the list above;
+ by default, the built-in module is used (but can be disabled
+ at translation time).
+
+The extension modules (i.e. modules written in C, in the standard CPython)
+that are neither mentioned above nor in :source:`lib_pypy/` are not available in PyPy.
+(You may have a chance to use them anyway with `cpyext`_.)
+
+.. _cpyext: http://morepypy.blogspot.com/2010/04/using-cpython-extension-modules-with.html
+
+
.. _`is ignored in PyPy`: http://bugs.python.org/issue14621
.. _`little point`: http://events.ccc.de/congress/2012/Fahrplan/events/5152.en.html
.. _`#2072`: https://bitbucket.org/pypy/pypy/issue/2072/
.. _`issue #2653`: https://bitbucket.org/pypy/pypy/issues/2653/
+.. _SyntaxError: https://morepypy.blogspot.co.il/2018/04/improving-syntaxerror-in-pypy.html
diff --git a/pypy/doc/index-of-release-notes.rst b/pypy/doc/index-of-release-notes.rst
--- a/pypy/doc/index-of-release-notes.rst
+++ b/pypy/doc/index-of-release-notes.rst
@@ -6,6 +6,7 @@
.. toctree::
+ release-v6.0.0.rst
release-v5.10.1.rst
release-v5.10.0.rst
release-v5.9.0.rst
diff --git a/pypy/doc/index-of-whatsnew.rst b/pypy/doc/index-of-whatsnew.rst
--- a/pypy/doc/index-of-whatsnew.rst
+++ b/pypy/doc/index-of-whatsnew.rst
@@ -7,6 +7,8 @@
.. toctree::
whatsnew-head.rst
+ whatsnew-pypy2-6.0.0.rst
+ whatsnew-pypy2-5.10.0.rst
whatsnew-pypy2-5.10.0.rst
whatsnew-pypy2-5.9.0.rst
whatsnew-pypy2-5.8.0.rst
diff --git a/pypy/doc/release-v6.0.0.rst b/pypy/doc/release-v6.0.0.rst
new file mode 100644
--- /dev/null
+++ b/pypy/doc/release-v6.0.0.rst
@@ -0,0 +1,109 @@
+======================================
+PyPy2.7 and PyPy3.5 v6.0 dual release
+======================================
+
+The PyPy team is proud to release both PyPy2.7 v6.0 (an interpreter supporting
+Python 2.7 syntax), and a PyPy3.5 v6.0 (an interpreter supporting Python
+3.5 syntax). The two releases are both based on much the same codebase, thus
+the dual release.
+
+This release is a feature release following our previous 5.10 incremental
+release in late December 2017. Our C-API compatability layer ``cpyext`` is
+now much faster (see the `blog post`_) as well as more complete. We have made
+many other improvements in speed and CPython compatibility. Since the changes
+affect the included python development header files, all c-extension modules must
+be recompiled for this version.
+
+First-time python users are often stumped by silly typos and emissions when
+getting started writing code. We have improved our parser to emit more friendly
+`syntax errors`_, making PyPy not only faster but more friendly.
+
+The Windows PyPy3.5 release is still considered beta-quality. There are open
+issues with unicode handling especially around system calls and c-extensions.
+
+The Matplotlib TkAgg backend now works with PyPy, as do pygame and pygobject_.
+
+As always, this release is 100% compatible with the previous one and fixed
+several issues and bugs raised by the growing community of PyPy users.
+We strongly recommend updating.
+
+We updated the cffi module included in PyPy to version 1.11.5
+
+The utf8 branch that changes internal representation of unicode to utf8 did not
+make it into the release, so there is still more goodness coming. We also
+began working on a Python3.6 implementation, help is welcome.
+
+You can download the v6.0 releases here:
+
+ http://pypy.org/download.html
+
+We would like to thank our donors for the continued support of the PyPy
+project. If PyPy is not quite good enough for your needs, we are available for
+direct consulting work.
+
+We would also like to thank our contributors and encourage new people to join
+the project. PyPy has many layers and we need help with all of them: `PyPy`_
+and `RPython`_ documentation improvements, tweaking popular `modules`_ to run
+on pypy, or general `help`_ with making RPython's JIT even better.
+
+.. _`PyPy`: index.html
+.. _`RPython`: https://rpython.readthedocs.org
+.. _`modules`: project-ideas.html#make-more-python-modules-pypy-friendly
+.. _`help`: project-ideas.html
+.. _`blog post`: https://morepypy.blogspot.it/2017/10/cape-of-good-hope-for-pypy-hello-from.html
+.. _pygobject: https://lazka.github.io/posts/2018-04_pypy-pygobject/index.html
+.. _`syntax errors`: https://morepypy.blogspot.com/2018/04/improving-syntaxerror-in-pypy.html
+
+What is PyPy?
+=============
+
+PyPy is a very compliant Python interpreter, almost a drop-in replacement for
+CPython 2.7 and CPython 3.5. It's fast (`PyPy and CPython 2.7.x`_ performance comparison)
+due to its integrated tracing JIT compiler.
+
+We also welcome developers of other `dynamic languages`_ to see what RPython
+can do for them.
+
+The PyPy release supports:
+
+ * **x86** machines on most common operating systems
+ (Linux 32/64 bits, Mac OS X 64 bits, Windows 32 bits, OpenBSD, FreeBSD)
+
+ * newer **ARM** hardware (ARMv6 or ARMv7, with VFPv3) running Linux,
+
+ * big- and little-endian variants of **PPC64** running Linux,
+
+ * **s390x** running Linux
+
+.. _`PyPy and CPython 2.7.x`: http://speed.pypy.org
+.. _`dynamic languages`: http://rpython.readthedocs.io/en/latest/examples.html
+
+Changelog
+=========
+
+* Speed up C-API method calls, and make most Py*_Check calls C macros
+* Speed up C-API slot method calls
+* Enable TkAgg backend support for matplotlib
+* support ``hastzinfo`` and ``tzinfo`` in the C-API ``PyDateTime*`` structures
+* datetime.h is now more similar to CPython
+* We now support ``PyUnicode_AsUTF{16,32}String``, ``_PyLong_AsByteArray``,
+ ``_PyLong_AsByteArrayO``,
+* PyPy3.5 on Windows is compiled with the Microsoft Visual Compiler v14, like
+ CPython
+* Fix performance of attribute lookup when more than 80 attributes are used
+* Improve performance on passing built-in types to C-API C code
+* Improve the performance of datetime and timedelta by skipping the consistency
+ checks of the datetime values (they are correct by construction)
+* Improve handling of ``bigint`` s, including fixing ``int_divmod``
+* Improve reporting of GC statistics
+* Accept unicode filenames in ``dbm.open()``
+* Improve RPython support for half-floats
+* Added missing attributes to C-API ``instancemethod`` on pypy3
+* Store error state in thread-local storage for C-API.
+* Fix JIT bugs exposed in the sre module
+* Improve speed of Python parser, improve ParseError messages slightly
+* Handle JIT hooks more efficiently
+
+We also refactored many parts of the JIT bridge optimizations, as well as cpyext
+internals, and together with new contributors fixed issues, added new
+documentation, and cleaned up the codebase.
diff --git a/pypy/doc/whatsnew-head.rst b/pypy/doc/whatsnew-head.rst
--- a/pypy/doc/whatsnew-head.rst
+++ b/pypy/doc/whatsnew-head.rst
@@ -1,99 +1,18 @@
-===========================
-What's new in PyPy2.7 5.10+
-===========================
+==========================
+What's new in PyPy2.7 6.0+
+==========================
-.. this is a revision shortly after release-pypy2.7-v5.10.0
-.. startrev: 6b024edd9d12
+.. this is a revision shortly after release-pypy-6.0.0
+.. startrev: 2e04adf1b89f
-.. branch: cpyext-avoid-roundtrip
+.. branch: cpyext-subclass-setattr
-Big refactoring of some cpyext code, which avoids a lot of nonsense when
-calling C from Python and vice-versa: the result is a big speedup in
-function/method calls, up to 6 times faster.
+Fix for python-level classes that inherit from C-API types, previously the
+`w_obj` was not necessarily preserved throughout the lifetime of the `pyobj`
+which led to cases where instance attributes were lost. Fixes issue #2793
-.. branch: cpyext-datetime2
-Support ``tzinfo`` field on C-API datetime objects, fixes latest pandas HEAD
+.. branch: pyparser-improvements-2
-
-.. branch: mapdict-size-limit
-
-Fix a corner case of mapdict: When an instance is used like a dict (using
-``setattr`` and ``getattr``, or ``.__dict__``) and a lot of attributes are
-added, then the performance using mapdict is linear in the number of
-attributes. This is now fixed (by switching to a regular dict after 80
-attributes).
-
-
-.. branch: cpyext-faster-arg-passing
-
-When using cpyext, improve the speed of passing certain objects from PyPy to C
-code, most notably None, True, False, types, all instances of C-defined types.
-Before, a dict lookup was needed every time such an object crossed over, now it
-is just a field read.
-
-
-.. branch: 2634_datetime_timedelta_performance
-
-Improve datetime + timedelta performance.
-
-.. branch: memory-accounting
-
-Improve way to describe memory
-
-.. branch: msvc14
-
-Allow compilaiton with Visual Studio 2017 compiler suite on windows
-
-.. branch: refactor-slots
-
-Refactor cpyext slots.
-
-
-.. branch: call-loopinvariant-into-bridges
-
-Speed up branchy code that does a lot of function inlining by saving one call
-to read the TLS in most bridges.
-
-.. branch: rpython-sprint
-
-Refactor in rpython signatures
-
-.. branch: cpyext-tls-operror2
-
-Store error state thread-locally in executioncontext, fixes issue #2764
-
-.. branch: cpyext-fast-typecheck
-
-Optimize `Py*_Check` for `Bool`, `Float`, `Set`. Also refactor and simplify
-`W_PyCWrapperObject` which is used to call slots from the C-API, greatly
-improving microbenchmarks in https://github.com/antocuni/cpyext-benchmarks
-
-
-.. branch: fix-sre-problems
-
-Fix two (unrelated) JIT bugs manifesting in the re module:
-
-- green fields are broken and were thus disabled, plus their usage removed from
- the _sre implementation
-
-- in rare "trace is too long" situations, the JIT could break behaviour
- arbitrarily.
-
-.. branch: jit-hooks-can-be-disabled
-
-Be more efficient about JIT hooks. Make it possible for the frontend to declare
-that jit hooks are currently not enabled at all. in that case, the list of ops
-does not have to be created in the case of the on_abort hook (which is
-expensive).
-
-
-.. branch: pyparser-improvements
-
-Improve speed of Python parser, improve ParseError messages slightly.
-
-.. branch: ioctl-arg-size
-
-Work around possible bugs in upstream ioctl users, like CPython allocate at
-least 1024 bytes for the arg in calls to ``ioctl(fd, request, arg)``. Fixes
-issue #2776
+Improve line offsets that are reported by SyntaxError. Improve error messages
+for a few situations, including mismatched parenthesis.
diff --git a/pypy/doc/whatsnew-pypy2-6.0.0.rst b/pypy/doc/whatsnew-pypy2-6.0.0.rst
new file mode 100644
--- /dev/null
+++ b/pypy/doc/whatsnew-pypy2-6.0.0.rst
@@ -0,0 +1,99 @@
+===========================
+What's new in PyPy2.7 5.10+
+===========================
+
+.. this is a revision shortly after release-pypy2.7-v5.10.0
+.. startrev: 6b024edd9d12
+
+.. branch: cpyext-avoid-roundtrip
+
+Big refactoring of some cpyext code, which avoids a lot of nonsense when
+calling C from Python and vice-versa: the result is a big speedup in
+function/method calls, up to 6 times faster.
+
+.. branch: cpyext-datetime2
+
+Support ``tzinfo`` field on C-API datetime objects, fixes latest pandas HEAD
+
+
+.. branch: mapdict-size-limit
+
+Fix a corner case of mapdict: When an instance is used like a dict (using
+``setattr`` and ``getattr``, or ``.__dict__``) and a lot of attributes are
+added, then the performance using mapdict is linear in the number of
+attributes. This is now fixed (by switching to a regular dict after 80
+attributes).
+
+
+.. branch: cpyext-faster-arg-passing
+
+When using cpyext, improve the speed of passing certain objects from PyPy to C
+code, most notably None, True, False, types, all instances of C-defined types.
+Before, a dict lookup was needed every time such an object crossed over, now it
+is just a field read.
+
+
+.. branch: 2634_datetime_timedelta_performance
+
+Improve datetime + timedelta performance.
+
+.. branch: memory-accounting
+
+Improve way to describe memory
+
+.. branch: msvc14
+
+Allow compilaiton with Visual Studio 2017 compiler suite on windows
+
+.. branch: refactor-slots
+
+Refactor cpyext slots.
+
+
+.. branch: call-loopinvariant-into-bridges
+
+Speed up branchy code that does a lot of function inlining by saving one call
+to read the TLS in most bridges.
+
+.. branch: rpython-sprint
+
+Refactor in rpython signatures
+
+.. branch: cpyext-tls-operror2
+
+Store error state thread-locally in executioncontext, fixes issue #2764
+
+.. branch: cpyext-fast-typecheck
+
+Optimize `Py*_Check` for `Bool`, `Float`, `Set`. Also refactor and simplify
+`W_PyCWrapperObject` which is used to call slots from the C-API, greatly
+improving microbenchmarks in https://github.com/antocuni/cpyext-benchmarks
+
+
+.. branch: fix-sre-problems
+
+Fix two (unrelated) JIT bugs manifesting in the re module:
+
+- green fields are broken and were thus disabled, plus their usage removed from
+ the _sre implementation
+
+- in rare "trace is too long" situations, the JIT could break behaviour
+ arbitrarily.
+
+.. branch: jit-hooks-can-be-disabled
+
+Be more efficient about JIT hooks. Make it possible for the frontend to declare
+that jit hooks are currently not enabled at all. in that case, the list of ops
+does not have to be created in the case of the on_abort hook (which is
+expensive).
+
+
+.. branch: pyparser-improvements
+
+Improve speed of Python parser, improve ParseError messages slightly.
+
+.. branch: ioctl-arg-size
+
+Work around possible bugs in upstream ioctl users, like CPython allocate at
+least 1024 bytes for the arg in calls to ``ioctl(fd, request, arg)``. Fixes
+issue #2776
diff --git a/pypy/interpreter/pyparser/error.py b/pypy/interpreter/pyparser/error.py
--- a/pypy/interpreter/pyparser/error.py
+++ b/pypy/interpreter/pyparser/error.py
@@ -6,6 +6,7 @@
lastlineno=0):
self.msg = msg
self.lineno = lineno
+ # NB: offset is a 1-based index!
self.offset = offset
self.text = text
self.filename = filename
diff --git a/pypy/interpreter/pyparser/parser.py b/pypy/interpreter/pyparser/parser.py
--- a/pypy/interpreter/pyparser/parser.py
+++ b/pypy/interpreter/pyparser/parser.py
@@ -199,6 +199,7 @@
self.token_type = token_type
self.value = value
self.lineno = lineno
+ # this is a 0-based index
self.column = column
self.line = line
self.expected = expected
diff --git a/pypy/interpreter/pyparser/pyparse.py b/pypy/interpreter/pyparser/pyparse.py
--- a/pypy/interpreter/pyparser/pyparse.py
+++ b/pypy/interpreter/pyparser/pyparse.py
@@ -188,7 +188,9 @@
if e.expected_str is not None:
msg += " (expected '%s')" % e.expected_str
- raise new_err(msg, e.lineno, e.column, e.line,
+ # parser.ParseError(...).column is 0-based, but the offsets in the
+ # exceptions in the error module are 1-based, hence the '+ 1'
+ raise new_err(msg, e.lineno, e.column + 1, e.line,
compile_info.filename)
else:
tree = self.root
diff --git a/pypy/interpreter/pyparser/pytokenizer.py b/pypy/interpreter/pyparser/pytokenizer.py
--- a/pypy/interpreter/pyparser/pytokenizer.py
+++ b/pypy/interpreter/pyparser/pytokenizer.py
@@ -73,14 +73,14 @@
logical line; continuation lines are included.
"""
token_list = []
- lnum = parenlev = continued = 0
+ lnum = continued = 0
namechars = NAMECHARS
numchars = NUMCHARS
contstr, needcont = '', 0
contline = None
indents = [0]
last_comment = ''
- parenlevstart = (0, 0, "")
+ parenstack = []
# make the annotator happy
endDFA = DUMMY_DFA
@@ -97,7 +97,7 @@
if contstr:
if not line:
raise TokenError(
- "EOF while scanning triple-quoted string literal",
+ "end of file (EOF) while scanning triple-quoted string literal",
strstart[2], strstart[0], strstart[1]+1,
token_list, lnum-1)
endmatch = endDFA.recognize(line)
@@ -123,7 +123,7 @@
contline = contline + line
continue
- elif parenlev == 0 and not continued: # new statement
+ elif not parenstack and not continued: # new statement
if not line: break
column = 0
while pos < max: # measure leading whitespace
@@ -143,21 +143,21 @@
token_list.append((tokens.INDENT, line[:pos], lnum, 0, line))
last_comment = ''
while column < indents[-1]:
- indents = indents[:-1]
+ indents.pop()
token_list.append((tokens.DEDENT, '', lnum, pos, line))
last_comment = ''
if column != indents[-1]:
err = "unindent does not match any outer indentation level"
- raise TokenIndentationError(err, line, lnum, 0, token_list)
+ raise TokenIndentationError(err, line, lnum, column+1, token_list)
else: # continued statement
if not line:
- if parenlev > 0:
- lnum1, start1, line1 = parenlevstart
+ if parenstack:
+ _, lnum1, start1, line1 = parenstack[0]
raise TokenError("parenthesis is never closed", line1,
lnum1, start1 + 1, token_list, lnum)
- raise TokenError("EOF in multi-line statement", line,
- lnum, 0, token_list)
+ raise TokenError("end of file (EOF) in multi-line statement", line,
+ lnum, 0, token_list) # XXX why is the offset 0 here?
continued = 0
while pos < max:
@@ -180,7 +180,7 @@
token_list.append((tokens.NUMBER, token, lnum, start, line))
last_comment = ''
elif initial in '\r\n':
- if parenlev <= 0:
+ if not parenstack:
tok = (tokens.NEWLINE, last_comment, lnum, start, line)
token_list.append(tok)
last_comment = ''
@@ -222,14 +222,22 @@
continued = 1
else:
if initial in '([{':
- if parenlev == 0:
- parenlevstart = (lnum, start, line)
- parenlev = parenlev + 1
+ parenstack.append((initial, lnum, start, line))
elif initial in ')]}':
- parenlev = parenlev - 1
- if parenlev < 0:
+ if not parenstack:
raise TokenError("unmatched '%s'" % initial, line,
lnum, start + 1, token_list)
+ opening, lnum1, start1, line1 = parenstack.pop()
+ if not ((opening == "(" and initial == ")") or
+ (opening == "[" and initial == "]") or
+ (opening == "{" and initial == "}")):
+ msg = "closing parenthesis '%s' does not match opening parenthesis '%s'" % (
+ initial, opening)
+
+ if lnum1 != lnum:
+ msg += " on line " + str(lnum1)
+ raise TokenError(
+ msg, line, lnum, start + 1, token_list)
if token in python_opmap:
punct = python_opmap[token]
else:
@@ -241,7 +249,7 @@
if start < 0:
start = pos
if start<max and line[start] in single_quoted:
- raise TokenError("EOL while scanning string literal",
+ raise TokenError("end of line (EOL) while scanning string literal",
line, lnum, start+1, token_list)
tok = (tokens.ERRORTOKEN, line[pos], lnum, pos, line)
token_list.append(tok)
diff --git a/pypy/interpreter/pyparser/test/targetparse.py b/pypy/interpreter/pyparser/test/targetparse.py
--- a/pypy/interpreter/pyparser/test/targetparse.py
+++ b/pypy/interpreter/pyparser/test/targetparse.py
@@ -8,25 +8,36 @@
-with file("../../../rpython/rlib/unicodedata/unicodedb_5_2_0.py") as f:
- s = f.read()
-
class FakeSpace(object):
pass
fakespace = FakeSpace()
-def bench(title):
+def bench(fn, s):
a = time.clock()
info = pyparse.CompileInfo("<string>", "exec")
parser = pyparse.PythonParser(fakespace)
tree = parser._parse(s, info)
b = time.clock()
- print title, (b-a)
+ print fn, (b-a)
def entry_point(argv):
- bench("foo")
+ if len(argv) == 2:
+ fn = argv[1]
+ else:
+ fn = "../../../../rpython/rlib/unicodedata/unicodedb_5_2_0.py"
+ fd = os.open(fn, os.O_RDONLY, 0777)
+ res = []
+ while True:
+ s = os.read(fd, 4096)
+ if not s:
+ break
+ res.append(s)
+ os.close(fd)
+ s = "".join(res)
+ print len(s)
+ bench(fn, s)
return 0
diff --git a/pypy/interpreter/pyparser/test/test_pyparse.py b/pypy/interpreter/pyparser/test/test_pyparse.py
--- a/pypy/interpreter/pyparser/test/test_pyparse.py
+++ b/pypy/interpreter/pyparser/test/test_pyparse.py
@@ -76,14 +76,14 @@
exc = py.test.raises(SyntaxError, parse, "name another for").value
assert exc.msg == "invalid syntax"
assert exc.lineno == 1
- assert exc.offset == 5
+ assert exc.offset == 6
assert exc.text.startswith("name another for")
exc = py.test.raises(SyntaxError, parse, "x = \"blah\n\n\n").value
- assert exc.msg == "EOL while scanning string literal"
+ assert exc.msg == "end of line (EOL) while scanning string literal"
assert exc.lineno == 1
assert exc.offset == 5
exc = py.test.raises(SyntaxError, parse, "x = '''\n\n\n").value
- assert exc.msg == "EOF while scanning triple-quoted string literal"
+ assert exc.msg == "end of file (EOF) while scanning triple-quoted string literal"
assert exc.lineno == 1
assert exc.offset == 5
assert exc.lastlineno == 3
@@ -112,7 +112,7 @@
assert exc.msg == "expected an indented block"
assert exc.lineno == 3
assert exc.text.startswith("pass")
- assert exc.offset == 0
+ assert exc.offset == 1
input = "hi\n indented"
exc = py.test.raises(IndentationError, parse, input).value
assert exc.msg == "unexpected indent"
@@ -120,6 +120,7 @@
exc = py.test.raises(IndentationError, parse, input).value
assert exc.msg == "unindent does not match any outer indentation level"
assert exc.lineno == 3
+ assert exc.offset == 3
def test_mac_newline(self):
self.parse("this_is\ra_mac\rfile")
diff --git a/pypy/interpreter/pyparser/test/test_pytokenizer.py b/pypy/interpreter/pyparser/test/test_pytokenizer.py
new file mode 100644
--- /dev/null
+++ b/pypy/interpreter/pyparser/test/test_pytokenizer.py
@@ -0,0 +1,66 @@
+import pytest
+from pypy.interpreter.pyparser import pytokenizer
+from pypy.interpreter.pyparser.pygram import tokens
+from pypy.interpreter.pyparser.error import TokenError
+
+def tokenize(s):
+ return pytokenizer.generate_tokens(s.splitlines(True) + ["\n"], 0)
+
+def check_token_error(s, msg=None, pos=-1, line=-1):
+ error = pytest.raises(TokenError, tokenize, s)
+ if msg is not None:
+ assert error.value.msg == msg
+ if pos != -1:
+ assert error.value.offset == pos
+ if line != -1:
+ assert error.value.lineno == line
+
+
+class TestTokenizer(object):
+
+ def test_simple(self):
+ line = "a+1"
+ tks = tokenize(line)
+ assert tks == [
+ (tokens.NAME, 'a', 1, 0, line),
+ (tokens.PLUS, '+', 1, 1, line),
+ (tokens.NUMBER, '1', 1, 2, line),
+ (tokens.NEWLINE, '', 2, 0, '\n'),
+ (tokens.NEWLINE, '', 2, 0, '\n'),
+ (tokens.ENDMARKER, '', 2, 0, ''),
+ ]
+
+ def test_error_parenthesis(self):
+ for paren in "([{":
+ check_token_error(paren + "1 + 2",
+ "parenthesis is never closed",
+ 1)
+
+ for paren in ")]}":
+ check_token_error("1 + 2" + paren,
+ "unmatched '%s'" % (paren, ),
+ 6)
+
+ for i, opening in enumerate("([{"):
+ for j, closing in enumerate(")]}"):
+ if i == j:
+ continue
+ check_token_error(opening + "1\n" + closing,
+ "closing parenthesis '%s' does not match opening parenthesis '%s' on line 1" % (closing, opening),
+ pos=1, line=2)
+ check_token_error(opening + "1" + closing,
+ "closing parenthesis '%s' does not match opening parenthesis '%s'" % (closing, opening),
+ pos=3, line=1)
+ check_token_error(opening + closing,
+ "closing parenthesis '%s' does not match opening parenthesis '%s'" % (closing, opening),
+ pos=2, line=1)
+
+
+ def test_unknown_char(self):
+ check_token_error("?", "Unknown character", 1)
+
+ def test_eol_string(self):
+ check_token_error("x = 'a", pos=5, line=1)
+
+ def test_eof_triple_quoted(self):
+ check_token_error("'''", pos=1, line=1)
diff --git a/pypy/interpreter/test/test_compiler.py b/pypy/interpreter/test/test_compiler.py
--- a/pypy/interpreter/test/test_compiler.py
+++ b/pypy/interpreter/test/test_compiler.py
@@ -77,7 +77,7 @@
""")
assert self.space.unwrap(w_args) == (
'unindent does not match any outer indentation level',
- ('<string>', 3, 0, ' y\n'))
+ ('<string>', 3, 2, ' y\n'))
def test_getcodeflags(self):
code = self.compiler.compile('from __future__ import division\n',
diff --git a/pypy/interpreter/test/test_syntax.py b/pypy/interpreter/test/test_syntax.py
--- a/pypy/interpreter/test/test_syntax.py
+++ b/pypy/interpreter/test/test_syntax.py
@@ -750,7 +750,7 @@
except SyntaxError as e:
assert e.lineno == 4
assert e.text.endswith('a b c d e\n')
- assert e.offset == e.text.index('b')
+ assert e.offset == e.text.index('b') + 1 # offset is 1-based
else:
raise Exception("no SyntaxError??")
diff --git a/pypy/interpreter/test/test_typedef.py b/pypy/interpreter/test/test_typedef.py
--- a/pypy/interpreter/test/test_typedef.py
+++ b/pypy/interpreter/test/test_typedef.py
@@ -423,3 +423,10 @@
def test_get_with_none_arg(self):
raises(TypeError, type.__dict__['__mro__'].__get__, None)
raises(TypeError, type.__dict__['__mro__'].__get__, None, None)
+
+ def test_builtin_readonly_property(self):
+ import sys
+ x = lambda: 5
+ e = raises(TypeError, 'x.func_globals = {}')
+ if '__pypy__' in sys.builtin_module_names:
+ assert str(e.value) == "readonly attribute 'func_globals'"
diff --git a/pypy/interpreter/typedef.py b/pypy/interpreter/typedef.py
--- a/pypy/interpreter/typedef.py
+++ b/pypy/interpreter/typedef.py
@@ -309,12 +309,18 @@
self.reqcls, Arguments(space, [w_obj,
space.newtext(self.name)]))
+ def readonly_attribute(self, space): # overwritten in cpyext
+ if self.name == '<generic property>':
+ raise oefmt(space.w_TypeError, "readonly attribute")
+ else:
+ raise oefmt(space.w_TypeError, "readonly attribute '%s'", self.name)
+
def descr_property_set(self, space, w_obj, w_value):
"""property.__set__(obj, value)
Change the value of the property of the given obj."""
fset = self.fset
if fset is None:
- raise oefmt(space.w_TypeError, "readonly attribute")
+ raise self.readonly_attribute(space)
try:
fset(self, space, w_obj, w_value)
except DescrMismatch:
diff --git a/pypy/module/_rawffi/alt/test/test_struct.py b/pypy/module/_rawffi/alt/test/test_struct.py
--- a/pypy/module/_rawffi/alt/test/test_struct.py
+++ b/pypy/module/_rawffi/alt/test/test_struct.py
@@ -43,7 +43,11 @@
def setup_class(cls):
BaseAppTestFFI.setup_class.im_func(cls)
- @unwrap_spec(addr=int, typename='text', length=int)
+ from rpython.rlib import clibffi
+ from rpython.rlib.rarithmetic import r_uint
+ from rpython.rtyper.lltypesystem import lltype, rffi
+
+ @unwrap_spec(addr=r_uint, typename='text', length=int)
def read_raw_mem(space, addr, typename, length):
import ctypes
addr = ctypes.cast(addr, ctypes.c_void_p)
@@ -58,9 +62,6 @@
else:
cls.w_read_raw_mem = cls.space.wrap(interp2app(read_raw_mem))
#
- from rpython.rlib import clibffi
- from rpython.rlib.rarithmetic import r_uint
- from rpython.rtyper.lltypesystem import lltype, rffi
dummy_type = lltype.malloc(clibffi.FFI_TYPE_P.TO, flavor='raw')
dummy_type.c_size = r_uint(123)
dummy_type.c_alignment = rffi.cast(rffi.USHORT, 0)
diff --git a/pypy/module/cpyext/api.py b/pypy/module/cpyext/api.py
--- a/pypy/module/cpyext/api.py
+++ b/pypy/module/cpyext/api.py
@@ -640,7 +640,7 @@
'Py_DivisionWarningFlag', 'Py_DontWriteBytecodeFlag', 'Py_NoUserSiteDirectory',
'_Py_QnewFlag', 'Py_Py3kWarningFlag', 'Py_HashRandomizationFlag', '_Py_PackageContext',
'_PyTraceMalloc_Track', '_PyTraceMalloc_Untrack', 'PyMem_Malloc',
- 'Py_IncRef', 'Py_DecRef', 'PyObject_Free', 'PyObject_GC_Del', 'PyType_GenericAlloc',
+ 'PyObject_Free', 'PyObject_GC_Del', 'PyType_GenericAlloc',
'_PyObject_New', '_PyObject_NewVar',
'_PyObject_GC_New', '_PyObject_GC_NewVar',
'PyObject_Init', 'PyObject_InitVar', 'PyInt_FromLong',
diff --git a/pypy/module/cpyext/include/patchlevel.h b/pypy/module/cpyext/include/patchlevel.h
--- a/pypy/module/cpyext/include/patchlevel.h
+++ b/pypy/module/cpyext/include/patchlevel.h
@@ -29,8 +29,8 @@
#define PY_VERSION "2.7.13"
/* PyPy version as a string */
-#define PYPY_VERSION "5.11.0-alpha0"
-#define PYPY_VERSION_NUM 0x050B0000
+#define PYPY_VERSION "6.1.0-alpha0"
+#define PYPY_VERSION_NUM 0x06010000
/* Defined to mean a PyPy where cpyext holds more regular references
to PyObjects, e.g. staying alive as long as the internal PyPy object
diff --git a/pypy/module/cpyext/pyobject.py b/pypy/module/cpyext/pyobject.py
--- a/pypy/module/cpyext/pyobject.py
+++ b/pypy/module/cpyext/pyobject.py
@@ -60,10 +60,10 @@
def _cpyext_attach_pyobj(self, space, py_obj):
self._cpy_ref = py_obj
- rawrefcount.create_link_pyobj(self, py_obj)
+ rawrefcount.create_link_pypy(self, py_obj)
cls._cpyext_attach_pyobj = _cpyext_attach_pyobj
-add_direct_pyobj_storage(W_BaseCPyObject)
+add_direct_pyobj_storage(W_BaseCPyObject)
add_direct_pyobj_storage(W_TypeObject)
add_direct_pyobj_storage(W_NoneObject)
add_direct_pyobj_storage(W_BoolObject)
@@ -414,3 +414,14 @@
@cpython_api([rffi.VOIDP], lltype.Signed, error=CANNOT_FAIL)
def _Py_HashPointer(space, ptr):
return rffi.cast(lltype.Signed, ptr)
+
+ at cpython_api([PyObject], lltype.Void)
+def Py_IncRef(space, obj):
+ # used only ifdef PYPY_DEBUG_REFCOUNT
+ if obj:
+ incref(space, obj)
+
+ at cpython_api([PyObject], lltype.Void)
+def Py_DecRef(space, obj):
+ # used only ifdef PYPY_DEBUG_REFCOUNT
+ decref(space, obj)
diff --git a/pypy/module/cpyext/setobject.py b/pypy/module/cpyext/setobject.py
--- a/pypy/module/cpyext/setobject.py
+++ b/pypy/module/cpyext/setobject.py
@@ -1,4 +1,4 @@
-from pypy.interpreter.error import oefmt
+from pypy.interpreter.error import OperationError, oefmt
from rpython.rtyper.lltypesystem import rffi, lltype
from pypy.module.cpyext.api import (cpython_api, Py_ssize_t, CANNOT_FAIL,
build_type_checkers)
@@ -64,8 +64,13 @@
instance of set or its subtype."""
if not PySet_Check(space, w_s):
PyErr_BadInternalCall(space)
- space.call_method(space.w_set, 'discard', w_s, w_obj)
- return 0
+ try:
+ space.call_method(space.w_set, 'remove', w_s, w_obj)
+ except OperationError as e:
+ if e.match(space, space.w_KeyError):
+ return 0
+ raise
+ return 1
@cpython_api([PyObject], PyObject)
diff --git a/pypy/module/cpyext/src/object.c b/pypy/module/cpyext/src/object.c
--- a/pypy/module/cpyext/src/object.c
+++ b/pypy/module/cpyext/src/object.c
@@ -5,18 +5,6 @@
extern void _PyPy_Free(void *ptr);
extern void *_PyPy_Malloc(Py_ssize_t size);
-void
-Py_IncRef(PyObject *o)
-{
- Py_XINCREF(o);
-}
-
-void
-Py_DecRef(PyObject *o)
-{
- Py_XDECREF(o);
-}
-
/*
* The actual value of this variable will be the address of
* pyobject.w_marker_deallocating, and will be set by
diff --git a/pypy/module/cpyext/test/array.c b/pypy/module/cpyext/test/array.c
--- a/pypy/module/cpyext/test/array.c
+++ b/pypy/module/cpyext/test/array.c
@@ -2492,6 +2492,87 @@
return PyLong_FromLong(obj1->ob_type->tp_dealloc == obj2->ob_type->tp_dealloc);
}
+static PyObject *
+subclass_with_attribute(PyObject *self, PyObject* args) {
+ /* what happens when we use tp_alloc to create the subclass, then
+ * assign to the w_obj via python, then get the GC to collect?
+ * The w_obj should not be collected!!
+ */
+ PyObject * obj, *sub, *attrib, *funcname, *attribname, *collect, *res, *tup;
+ PyTypeObject * subtype;
+ int i;
+ if (!PyArg_ParseTuple(args, "OOOO", &obj, &funcname, &attribname, &collect)) {
+ return NULL;
+ }
+ if (!PyType_Check(obj)) {
+ PyErr_SetString(PyExc_TypeError,
+ "expected type object");
+ return NULL;
+ }
+ subtype = (PyTypeObject*)obj;
+ sub = subtype->tp_alloc(subtype, 0);
+ if (!sub) {
+ return NULL;
+ }
+ attrib = PyObject_GetAttr(sub, funcname);
+ if (!attrib || (attrib == Py_None) ) {
+ PyErr_SetString(PyExc_ValueError,
+ "could not find function to call");
+ Py_XDECREF(attrib);
+ Py_DECREF(sub);
+ return NULL;
+ }
+ tup = PyTuple_New(0);
+ /*
+ #ifdef PYPY_VERSION
+ printf("calling addattrib pypylink %lu \n", sub->ob_pypy_link);
+ #endif
+ */
+ res = PyObject_Call(attrib, tup, NULL);
+ /*
+ #ifdef PYPY_VERSION
+ printf("after addattrib pypylink %lu \n", sub->ob_pypy_link);
+ #endif
+ */
+ Py_DECREF(attrib);
+ if (res == NULL) {
+ Py_DECREF(tup);
+ Py_DECREF(sub);
+ return NULL;
+ }
+ Py_DECREF(res);
+ for(i=0; i<10; i++) {
+ /*
+ #ifdef PYPY_VERSION
+ printf("starting loop iteration %d refcnt %lu pypylink %lu \n", i,
+ sub->ob_refcnt, sub->ob_pypy_link);
+ #else
+ printf("starting loop iteration %d refcnt %lu\n", i, sub->ob_refcnt);
+ #endif
+ */
+ attrib = PyObject_GetAttr(sub, attribname);
+ if (!attrib || (attrib == Py_None)) {
+ PyErr_SetString(PyExc_ValueError,
+ "could not find attrib on object");
+ Py_XDECREF(attrib);
+ Py_DECREF(tup);
+ Py_DECREF(sub);
+ return NULL;
+ }
+ Py_XDECREF(attrib);
+ res = PyObject_Call(collect, tup, NULL);
+ if (res == NULL) {
+ Py_DECREF(tup);
+ Py_DECREF(sub);
+ return NULL;
+ }
+ Py_DECREF(res);
+ }
+ Py_DECREF(tup);
+ Py_DECREF(sub);
+ Py_RETURN_NONE;
+}
+
/*********************** Install Module **************************/
static PyMethodDef a_methods[] = {
@@ -2502,6 +2583,7 @@
{"create_and_release_buffer", (PyCFunction)create_and_release_buffer, METH_O, NULL},
{"same_dealloc", (PyCFunction)same_dealloc, METH_VARARGS, NULL},
{"getitem", (PyCFunction)getitem, METH_VARARGS, NULL},
+ {"subclass_with_attribute", (PyCFunction)subclass_with_attribute, METH_VARARGS, NULL},
{NULL, NULL, 0, NULL} /* Sentinel */
};
diff --git a/pypy/module/cpyext/test/test_arraymodule.py b/pypy/module/cpyext/test/test_arraymodule.py
--- a/pypy/module/cpyext/test/test_arraymodule.py
+++ b/pypy/module/cpyext/test/test_arraymodule.py
@@ -186,3 +186,15 @@
# array_subscr does)
raises(IndexError, module.getitem, a, -5)
+ def test_subclass_with_attribute(self):
+ module = self.import_module(name='array')
+ class Sub(module.array):
+ def addattrib(self):
+ print('called addattrib')
+ self.attrib = True
+ import gc
+ module.subclass_with_attribute(Sub, "addattrib", "attrib", gc.collect)
+ if self.runappdirect:
+ assert Sub.__module__ == 'pypy.module.cpyext.test.test_arraymodule'
+ assert str(Sub) == "<class 'pypy.module.cpyext.test.test_arraymodule.Sub'>"
+
diff --git a/pypy/module/cpyext/test/test_setobject.py b/pypy/module/cpyext/test/test_setobject.py
--- a/pypy/module/cpyext/test/test_setobject.py
+++ b/pypy/module/cpyext/test/test_setobject.py
@@ -28,7 +28,11 @@
assert api.PySet_Size(w_set) == 4
api.PySet_Add(w_set, space.wrap(6))
assert api.PySet_Size(w_set) == 5
- api.PySet_Discard(w_set, space.wrap(6))
+ res = api.PySet_Discard(w_set, space.wrap(6))
+ assert res == 1
+ assert api.PySet_Size(w_set) == 4
+ res = api.PySet_Discard(w_set, space.wrap(6))
+ assert res == 0
assert api.PySet_Size(w_set) == 4
def test_set_contains(self, space, api):
diff --git a/pypy/module/cpyext/test/test_tupleobject.py b/pypy/module/cpyext/test/test_tupleobject.py
--- a/pypy/module/cpyext/test/test_tupleobject.py
+++ b/pypy/module/cpyext/test/test_tupleobject.py
@@ -67,6 +67,18 @@
assert space.int_w(space.getitem(w_tuple, space.wrap(i))) == 42 + i
decref(space, ar[0])
+ py_tuple = state.ccall("PyTuple_New", 1)
+ ar[0] = py_tuple
+ api._PyTuple_Resize(ar, 1)
+ assert api.PyTuple_Size(ar[0]) == 1
+ decref(space, ar[0])
+
+ py_tuple = state.ccall("PyTuple_New", 1)
+ ar[0] = py_tuple
+ api._PyTuple_Resize(ar, 5)
+ assert api.PyTuple_Size(ar[0]) == 5
+ decref(space, ar[0])
+
lltype.free(ar, flavor='raw')
def test_setitem(self, space, api):
diff --git a/pypy/module/cpyext/test/test_typeobject.py b/pypy/module/cpyext/test/test_typeobject.py
--- a/pypy/module/cpyext/test/test_typeobject.py
+++ b/pypy/module/cpyext/test/test_typeobject.py
@@ -5,7 +5,7 @@
from pypy.module.cpyext.test.test_api import BaseApiTest
from pypy.module.cpyext.api import generic_cpy_call
from pypy.module.cpyext.pyobject import make_ref, from_ref, decref, as_pyobj
-from pypy.module.cpyext.typeobject import PyTypeObjectPtr
+from pypy.module.cpyext.typeobject import PyTypeObjectPtr, W_PyCTypeObject
class AppTestTypeObject(AppTestCpythonExtensionBase):
@@ -412,33 +412,42 @@
def test_type_dict(self):
foo = self.import_module("foo")
module = self.import_extension('test', [
- ("hack_tp_dict", "METH_O",
+ ("hack_tp_dict", "METH_VARARGS",
'''
- PyTypeObject *type = args->ob_type;
+ PyTypeObject *type, *obj;
PyObject *a1 = PyLong_FromLong(1);
PyObject *a2 = PyLong_FromLong(2);
PyObject *value;
+ PyObject * key;
+ if (!PyArg_ParseTuple(args, "OO", &obj, &key))
+ return NULL;
+ type = obj->ob_type;
- if (PyDict_SetItemString(type->tp_dict, "a",
+ if (PyDict_SetItem(type->tp_dict, key,
a1) < 0)
return NULL;
Py_DECREF(a1);
PyType_Modified(type);
- value = PyObject_GetAttrString((PyObject *)type, "a");
+ value = PyObject_GetAttr((PyObject *)type, key);
Py_DECREF(value);
- if (PyDict_SetItemString(type->tp_dict, "a",
+ if (PyDict_SetItem(type->tp_dict, key,
a2) < 0)
return NULL;
Py_DECREF(a2);
PyType_Modified(type);
- value = PyObject_GetAttrString((PyObject *)type, "a");
+ value = PyObject_GetAttr((PyObject *)type, key);
return value;
'''
)
])
obj = foo.new()
- assert module.hack_tp_dict(obj) == 2
+ assert module.hack_tp_dict(obj, "a") == 2
+ class Sub(foo.fooType):
+ pass
+ obj = Sub()
+ assert module.hack_tp_dict(obj, "b") == 2
+
def test_tp_descr_get(self):
module = self.import_extension('foo', [
@@ -560,6 +569,23 @@
assert w_obj is None
assert api.PyErr_Occurred() is None
+ def test_subclass_not_PyCTypeObject(self, space, api):
+ pyobj = make_ref(space, api.PyLong_Type)
+ py_type = rffi.cast(PyTypeObjectPtr, pyobj)
+ w_pyclass = W_PyCTypeObject(space, py_type)
+ w_class = space.appexec([w_pyclass], """(base):
+ class Sub(base):
+ def addattrib(self, value):
+ self.attrib = value
+ return Sub
+ """)
+ assert w_pyclass in w_class.mro_w
+ assert isinstance(w_pyclass, W_PyCTypeObject)
+ assert not isinstance(w_class, W_PyCTypeObject)
+ assert w_pyclass.is_cpytype()
+ # XXX document the current status, not clear if this is desirable
+ assert w_class.is_cpytype()
+
class AppTestSlots(AppTestCpythonExtensionBase):
def setup_class(cls):
@@ -1600,6 +1626,65 @@
pass
C(42) # assert is not aborting
+ def test_getset(self):
+ module = self.import_extension('foo', [
+ ("get_instance", "METH_NOARGS",
+ '''
+ return PyObject_New(PyObject, &Foo_Type);
+ '''
+ ), ("get_number", "METH_NOARGS",
+ '''
+ return PyInt_FromLong(my_global_number);
+ '''
+ )], prologue='''
+ #if PY_MAJOR_VERSION > 2
+ #define PyInt_FromLong PyLong_FromLong
+ #define PyInt_AsLong PyLong_AsLong
+ #endif
+ static long my_global_number;
+ static PyTypeObject Foo_Type = {
+ PyVarObject_HEAD_INIT(NULL, 0)
+ "foo.foo",
+ };
+ static PyObject *bar_get(PyObject *foo, void *closure)
+ {
+ return PyInt_FromLong(1000 + (long)closure);
+ }
+ static PyObject *baz_get(PyObject *foo, void *closure)
+ {
+ return PyInt_FromLong(2000 + (long)closure);
+ }
+ static int baz_set(PyObject *foo, PyObject *x, void *closure)
+ {
+ if (x != NULL)
+ my_global_number = 3000 + (long)closure + PyInt_AsLong(x);
+ else
+ my_global_number = 4000 + (long)closure;
+ return 0;
+ }
+ static PyGetSetDef foo_getset[] = {
+ { "bar", bar_get, NULL, "mybardoc", (void *)42 },
+ { "baz", baz_get, baz_set, "mybazdoc", (void *)43 },
+ { NULL }
+ };
+ ''', more_init = '''
+ Foo_Type.tp_getset = foo_getset;
+ Foo_Type.tp_flags = Py_TPFLAGS_DEFAULT;
+ if (PyType_Ready(&Foo_Type) < 0) INITERROR;
+ ''')
+ foo = module.get_instance()
+ assert foo.bar == 1042
+ assert foo.bar == 1042
+ assert foo.baz == 2043
+ foo.baz = 50000
+ assert module.get_number() == 53043
+ e = raises(AttributeError, "foo.bar = 0")
+ assert str(e.value).startswith("attribute 'bar' of '")
+ assert str(e.value).endswith("foo' objects is not writable")
+ del foo.baz
+ assert module.get_number() == 4043
+ raises(AttributeError, "del foo.bar")
+
class AppTestHashable(AppTestCpythonExtensionBase):
def test_unhashable(self):
diff --git a/pypy/module/cpyext/tupleobject.py b/pypy/module/cpyext/tupleobject.py
--- a/pypy/module/cpyext/tupleobject.py
+++ b/pypy/module/cpyext/tupleobject.py
@@ -187,6 +187,8 @@
PyErr_BadInternalCall(space)
oldref = rffi.cast(PyTupleObject, ref)
oldsize = oldref.c_ob_size
+ if oldsize == newsize:
+ return 0
ptup = state.ccall("PyTuple_New", newsize)
if not ptup:
state.check_and_raise_exception(always=True)
@@ -199,8 +201,9 @@
to_cp = newsize
for i in range(to_cp):
ob = oldref.c_ob_item[i]
- incref(space, ob)
- newref.c_ob_item[i] = ob
+ if ob:
+ incref(space, ob)
+ newref.c_ob_item[i] = ob
except:
decref(space, p_ref[0])
p_ref[0] = lltype.nullptr(PyObject.TO)
diff --git a/pypy/module/cpyext/typeobject.py b/pypy/module/cpyext/typeobject.py
--- a/pypy/module/cpyext/typeobject.py
+++ b/pypy/module/cpyext/typeobject.py
@@ -1,6 +1,6 @@
import os
-from rpython.rlib import jit
+from rpython.rlib import jit, rawrefcount
from rpython.rlib.objectmodel import specialize, we_are_translated
from rpython.rtyper.lltypesystem import rffi, lltype
@@ -54,19 +54,26 @@
class W_GetSetPropertyEx(GetSetProperty):
def __init__(self, getset, w_type):
self.getset = getset
- self.name = rffi.charp2str(getset.c_name)
self.w_type = w_type
- doc = set = get = None
+ doc = fset = fget = fdel = None
if doc:
# XXX dead code?
doc = rffi.charp2str(getset.c_doc)
if getset.c_get:
- get = GettersAndSetters.getter.im_func
+ fget = GettersAndSetters.getter.im_func
if getset.c_set:
- set = GettersAndSetters.setter.im_func
- GetSetProperty.__init__(self, get, set, None, doc,
+ fset = GettersAndSetters.setter.im_func
+ fdel = GettersAndSetters.deleter.im_func
+ GetSetProperty.__init__(self, fget, fset, fdel, doc,
cls=None, use_closure=True,
tag="cpyext_1")
+ self.name = rffi.charp2str(getset.c_name)
+
+ def readonly_attribute(self, space): # overwritten
+ raise oefmt(space.w_AttributeError,
+ "attribute '%s' of '%N' objects is not writable",
+ self.name, self.w_type)
+
def PyDescr_NewGetSet(space, getset, w_type):
return W_GetSetPropertyEx(getset, w_type)
@@ -454,6 +461,16 @@
state = space.fromcache(State)
state.check_and_raise_exception()
+ def deleter(self, space, w_self):
+ assert isinstance(self, W_GetSetPropertyEx)
+ check_descr(space, w_self, self.w_type)
+ res = generic_cpy_call(
+ space, self.getset.c_set, w_self, None,
+ self.getset.c_closure)
+ if rffi.cast(lltype.Signed, res) < 0:
+ state = space.fromcache(State)
+ state.check_and_raise_exception()
+
def member_getter(self, space, w_self):
assert isinstance(self, W_MemberDescr)
check_descr(space, w_self, self.w_type)
@@ -517,6 +534,10 @@
self.w_doc = space.newtext(
rffi.charp2str(cts.cast('char*', pto.c_tp_doc)))
+ def _cpyext_attach_pyobj(self, space, py_obj):
+ self._cpy_ref = py_obj
+ rawrefcount.create_link_pyobj(self, py_obj)
+
@bootstrap_function
def init_typeobject(space):
make_typedescr(space.w_type.layout.typedef,
@@ -777,7 +798,6 @@
try:
w_obj = _type_realize(space, py_obj)
finally:
- name = rffi.charp2str(cts.cast('char*', pto.c_tp_name))
pto.c_tp_flags &= ~Py_TPFLAGS_READYING
pto.c_tp_flags |= Py_TPFLAGS_READY
return w_obj
@@ -884,7 +904,6 @@
base = pto.c_tp_base
base_pyo = rffi.cast(PyObject, pto.c_tp_base)
if base and not base.c_tp_flags & Py_TPFLAGS_READY:
- name = rffi.charp2str(cts.cast('char*', base.c_tp_name))
type_realize(space, base_pyo)
if base and not pto.c_ob_type: # will be filled later
pto.c_ob_type = base.c_ob_type
diff --git a/pypy/module/sys/version.py b/pypy/module/sys/version.py
--- a/pypy/module/sys/version.py
+++ b/pypy/module/sys/version.py
@@ -10,7 +10,7 @@
#XXX # sync CPYTHON_VERSION with patchlevel.h, package.py
CPYTHON_API_VERSION = 1013 #XXX # sync with include/modsupport.h
-PYPY_VERSION = (5, 11, 0, "alpha", 0) #XXX # sync patchlevel.h
+PYPY_VERSION = (6, 1, 0, "alpha", 0) #XXX # sync patchlevel.h
import pypy
diff --git a/pypy/tool/release/repackage.sh b/pypy/tool/release/repackage.sh
--- a/pypy/tool/release/repackage.sh
+++ b/pypy/tool/release/repackage.sh
@@ -1,8 +1,8 @@
# Edit these appropriately before running this script
pmaj=2 # python main version
pmin=7 # python minor version
-maj=5
-min=8
+maj=6
+min=0
rev=0
branchname=release-pypy$pmaj.$pmin-$maj.x # ==OR== release-$maj.x # ==OR== release-$maj.$min.x
tagname=release-pypy$pmaj.$pmin-v$maj.$min.$rev # ==OR== release-$maj.$min
diff --git a/rpython/rlib/rgc.py b/rpython/rlib/rgc.py
--- a/rpython/rlib/rgc.py
+++ b/rpython/rlib/rgc.py
@@ -644,7 +644,10 @@
def get_rpy_type_index(gcref):
from rpython.rlib.rarithmetic import intmask
Class = gcref._x.__class__
- return intmask(id(Class))
+ i = intmask(id(Class))
+ if i < 0:
+ i = ~i # always return a positive number, at least
+ return i
def cast_gcref_to_int(gcref):
# This is meant to be used on cast_instance_to_gcref results.
More information about the pypy-commit
mailing list