[pypy-commit] pypy default: merged upstream
alex_gaynor
noreply at buildbot.pypy.org
Tue Apr 29 04:59:21 CEST 2014
Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch:
Changeset: r71058:1f55d73ad8a0
Date: 2014-04-28 19:58 -0700
http://bitbucket.org/pypy/pypy/changeset/1f55d73ad8a0/
Log: merged upstream
diff --git a/pypy/doc/getting-started.rst b/pypy/doc/getting-started.rst
--- a/pypy/doc/getting-started.rst
+++ b/pypy/doc/getting-started.rst
@@ -76,7 +76,7 @@
.. code-block:: console
$ curl -O http://python-distribute.org/distribute_setup.py
- $ curl -O https://raw.github.com/pypa/pip/master/contrib/get-pip.py
+ $ curl -O https://raw.githubusercontent.com/pypa/pip/master/contrib/get-pip.py
$ ./pypy-2.1/bin/pypy distribute_setup.py
$ ./pypy-2.1/bin/pypy get-pip.py
$ ./pypy-2.1/bin/pip install pygments # for example
diff --git a/pypy/doc/release-2.3.0.rst b/pypy/doc/release-2.3.0.rst
--- a/pypy/doc/release-2.3.0.rst
+++ b/pypy/doc/release-2.3.0.rst
@@ -1,11 +1,17 @@
=======================================
-PyPy 2.3 - XXXX TODO
+PyPy 2.3 - Easier Than Ever
=======================================
We're pleased to announce PyPy 2.3, which targets version 2.7.6 of the Python
language. This release updates the stdlib from 2.7.3, jumping directly to 2.7.6.
-This release also contains several bugfixes and performance improvements.
+This release also contains several bugfixes and performance improvements,
+many generated by real users finding corner cases our `TDD`_ methods missed.
+`CFFI`_ has made it easier than ever to use existing C code with both cpython
+and PyPy, easing the transition for packages like `cryptography`_, `Pillow`_
+(Python Imaging Library [Fork]), a basic port of `pygame-cffi`_, and others.
+
+PyPy can now be embedded in a hosting application, for instance inside `uWSGI`_
You can download the PyPy 2.3 release here:
@@ -17,72 +23,112 @@
Please consider donating more so we can finish those projects! The three
projects are:
-* Py3k (supporting Python 3.x): the release PyPy3 2.2 is imminent.
+* `Py3k`_ (supporting Python 3.x): the release PyPy3 2.2 is imminent.
-* STM (software transactional memory): a preview will be released very soon,
- as soon as we fix a few bugs
+* `STM`_ (software transactional memory): a preview will be released very soon,
+ once we fix a few bugs
-* NumPy: the work done is included in the PyPy 2.2 release. More details below.
+* `NumPy`_ the work done is included in the PyPy 2.2 release. More details below.
-.. _`Raspberry Pi Foundation`: http://www.raspberrypi.org
+.. _`Py3k`: http://pypy.org/py3donate.html
+.. _`STM`: http://pypy.org/tmdonate2.html
+.. _ `Numpy`: http://pypy.org/numpydonate.html
+.. _`TDD`: http://doc.pypy.org/en/latest/how-to-contribute.html
+.. _`CFFI`: http://cffi.readthedocs.org
+.. _`cryptography`: https://cryptography.io
+.. _`Pillow`: https://pypi.python.org/pypi/Pillow/2.4.0
+.. _`pygame-cffi`: https://github.com/CTPUG/pygame_cffi
+.. _`uWSGI`: http://uwsgi-docs.readthedocs.org/en/latest/PyPy.html
What is PyPy?
=============
PyPy is a very compliant Python interpreter, almost a drop-in replacement for
-CPython 2.7. It's fast (`pypy 2.2 and cpython 2.7.2`_ performance comparison)
+CPython 2.7. It's fast (`pypy 2.2 and cpython 2.7.2`_ performance comparison;
+note that the latest cpython is not faster than cpython 2.7.2)
due to its integrated tracing JIT compiler.
-This release supports x86 machines running Linux 32/64, Mac OS X 64, Windows
-32, or ARM (ARMv6 or ARMv7, with VFPv3).
+This release supports x86 machines running Linux 32/64, Mac OS X 64, Windows,
+and OpenBSD,
+as well as newer ARM hardware (ARMv6 or ARMv7, with VFPv3) running Linux.
-Work on the native Windows 64 is still stalling, we would welcome a volunteer
-to handle that.
+While we support 32 bit python on Windows, work on the native Windows 64
+bit python is still stalling, we would welcome a volunteer
+to `handle that`_.
.. _`pypy 2.2 and cpython 2.7.2`: http://speed.pypy.org
+.. _`handle that`: http://doc.pypy.org/en/latest/windows.html#what-is-missing-for-a-full-64-bit-translation
Highlights
==========
-* Our Garbage Collector is now "incremental". It should avoid almost
- all pauses due to a major collection taking place. Previously, it
- would pause the program (rarely) to walk all live objects, which
- could take arbitrarily long if your process is using a whole lot of
- RAM. Now the same work is done in steps. This should make PyPy
- more responsive, e.g. in games. There are still other pauses, from
- the GC and the JIT, but they should be on the order of 5
- milliseconds each.
+Bugfixes
+--------
-* The JIT counters for hot code were never reset, which meant that a
- process running for long enough would eventually JIT-compile more
- and more rarely executed code. Not only is it useless to compile
- such code, but as more compiled code means more memory used, this
- gives the impression of a memory leak. This has been tentatively
- fixed by decreasing the counters from time to time.
+Many issues were cleaned up after being reported by users to https://bugs.pypy.org (ignore the bad SSL certificate) or on IRC at #pypy. Note that we consider
+performance slowdowns as bugs.
-* NumPy has been split: now PyPy only contains the core module, called
- ``_numpypy``. The ``numpy`` module itself has been moved to
- ``https://bitbucket.org/pypy/numpy`` and ``numpypy`` disappeared.
- You need to install NumPy separately with a virtualenv:
+* The ARM port no longer crashes on unaligned memory access to floats and doubles,
+ and singlefloats are supported in the JIT.
+
+* Generators are faster since they now skip unecessary cleanup
+
+* A first time contributor simplified JIT traces by adding integer bound
+ propagation in indexing and logical operations.
+
+* Optimize consecutive dictionary lookups of the same key in a chain
+
+* Our extensive pre-translation test suite now runs nightly on more platforms
+
+* Fix issues with reimporting builtin modules
+
+* Fix a rpython bug with loop-unrolling that appeared in the `HippyVM`_ PHP port
+
+.. _`HippyVM`: http://www.hippyvm.com
+
+New Platforms and Features
+--------------------------
+
+* Support for OpenBSD
+
+* Code cleanup: we continue to prune out old and unused code, and to refactor
+ large parts of the codebase. We have sepearated rpython from the PyPy python
+ interpreter, and rpython is seeing use in other dynamic language projects.
+
+* Support for precompiled headers in the build process for MSVC
+
+* Support for objects with __int__ and __float__ methods
+
+* Tweak support of errno in cpyext (the PyPy implemenation of the capi)
+
+
+Numpy
+-----
+Numpy support has been split into a builtin ``_numpy`` module and a
+fork of the numpy code base adapted to pypy at
+ ``https://bitbucket.org/pypy/numpy``.
+You need to install NumPy separately with a virtualenv:
``pip install git+https://bitbucket.org/pypy/numpy.git``;
or directly:
``git clone https://bitbucket.org/pypy/numpy.git``;
``cd numpy``; ``pypy setup.py install``.
-* non-inlined calls have less overhead
+* NumPy support has been improved, many failures in indexing, dtypes,
+ and scalars were corrected. We are slowly approaching our goal of passing
+ the numpy test suite. We still do not support object or unicode ndarrays.
-* Things that use ``sys.set_trace`` are now JITted (like coverage)
+* speed of iteration in dot() is now within 1.5x of the numpy c
+ implementation (without BLAS acceleration). Since the same array
+ iterator is used throughout the ``_numpy`` module, speed increases should
+ be apparent in all Numpy functionality.
-* JSON decoding is now very fast (JSON encoding was already very fast)
+* Most of the core functionality of nditer has been implemented.
-* various buffer copying methods experience speedups (like list-of-ints to
- ``int[]`` buffer from cffi)
+* A cffi-based ``numpy.random`` module is available as a branch in the numpy
+ repository, it will be merged soon after this release.
-* We finally wrote (hopefully) all the missing ``os.xxx()`` functions,
- including ``os.startfile()`` on Windows and a handful of rare ones
- on Posix.
+* enhancements to the PyPy JIT were made to support virtualizing the raw_store/raw_load memory operations used in numpy arrays. Further work remains here in virtualizing the alloc_raw_storage when possible. This will allow scalars to have storages but still be virtualized when possible in loops.
-* numpy has a rudimentary C API that cooperates with ``cpyext``
+Cheers
+The PyPy Team
-Cheers,
-Armin Rigo and Maciej Fijalkowski
diff --git a/pypy/module/sys/version.py b/pypy/module/sys/version.py
--- a/pypy/module/sys/version.py
+++ b/pypy/module/sys/version.py
@@ -14,7 +14,8 @@
if platform.name == 'msvc':
COMPILER_INFO = 'MSC v.%d 32 bit' % (platform.version * 10 + 600)
-elif platform.cc is not None and platform.cc.startswith(('gcc', 'clang')):
+elif platform.cc is not None and \
+ os.path.basename(platform.cc).startswith(('gcc', 'clang')):
from rpython.rtyper.tool import rffi_platform
COMPILER_INFO = 'GCC ' + rffi_platform.getdefinedstring('__VERSION__', '')
else:
diff --git a/pypy/module/zipimport/test/test_undocumented.py b/pypy/module/zipimport/test/test_undocumented.py
--- a/pypy/module/zipimport/test/test_undocumented.py
+++ b/pypy/module/zipimport/test/test_undocumented.py
@@ -135,8 +135,7 @@
importer = zipimport.zipimporter(os.path.join(zip_path, '_pkg'))
assert zip_path in zipimport._zip_directory_cache
file_set = set(zipimport._zip_directory_cache[zip_path].iterkeys())
- compare_set = set(path.replace(os.path.sep, '/') + '.py'
- for path in self.created_paths)
+ compare_set = set(path + '.py' for path in self.created_paths)
assert file_set == compare_set
finally:
self.cleanup_zipfile(self.created_paths)
diff --git a/pypy/tool/release/force-builds.py b/pypy/tool/release/force-builds.py
--- a/pypy/tool/release/force-builds.py
+++ b/pypy/tool/release/force-builds.py
@@ -9,7 +9,7 @@
modified by PyPy team
"""
-import os, sys, pwd, urllib
+import os, sys, urllib
from twisted.internet import reactor, defer
from twisted.python import log
@@ -34,6 +34,13 @@
'build-pypy-c-jit-linux-armel',
]
+def get_user():
+ if sys.platform == 'win32':
+ return os.environ['USERNAME']
+ else:
+ import pwd
+ return pwd.getpwuid(os.getuid())[0]
+
def main():
#XXX: handle release tags
#XXX: handle validity checks
@@ -49,7 +56,7 @@
print 'Forcing', builder, '...'
url = "http://buildbot.pypy.org/builders/" + builder + "/force"
args = [
- ('username', pwd.getpwuid(os.getuid())[0]),
+ ('username', get_user()),
('revision', ''),
('submit', 'Force Build'),
('branch', branch),
diff --git a/rpython/rlib/parsing/lexer.py b/rpython/rlib/parsing/lexer.py
--- a/rpython/rlib/parsing/lexer.py
+++ b/rpython/rlib/parsing/lexer.py
@@ -107,7 +107,7 @@
self.matcher = matcher
self.lineno = 0
self.columnno = 0
-
+
def find_next_token(self):
while 1:
self.state = 0
@@ -126,8 +126,8 @@
i = ~i
stop = self.last_matched_index + 1
assert stop >= 0
- if start == stop:
- source_pos = SourcePos(i - 1, self.lineno, self.columnno)
+ if start == stop:
+ source_pos = self.token_position_class(i - 1, self.lineno, self.columnno)
raise deterministic.LexerError(self.text, self.state,
source_pos)
source = self.text[start:stop]
@@ -147,7 +147,7 @@
else:
raise StopIteration
return result
- source_pos = SourcePos(i - 1, self.lineno, self.columnno)
+ source_pos = self.token_position_class(i - 1, self.lineno, self.columnno)
raise deterministic.LexerError(self.text, self.state, source_pos)
def adjust_position(self, token):
@@ -158,7 +158,7 @@
self.columnno += len(token)
else:
self.columnno = token.rfind("\n")
-
+
# def inner_loop(self, i):
# while i < len(self.text):
# char = self.text[i]
@@ -186,10 +186,15 @@
class LexingDFARunner(AbstractLexingDFARunner):
def __init__(self, matcher, automaton, text, ignore, eof=False,
token_class=None):
- if token_class is None:
+
+ if not token_class:
self.token_class = Token
+ self.token_position_class = SourcePos
+
else:
self.token_class = token_class
+ self.token_position_class = token_class.source_position_class
+
AbstractLexingDFARunner.__init__(self, matcher, automaton, text, eof)
self.ignore = ignore
@@ -198,8 +203,10 @@
def make_token(self, index, state, text, eof=False):
assert (eof and state == -1) or 0 <= state < len(self.automaton.names)
- source_pos = SourcePos(index, self.lineno, self.columnno)
+
+ source_pos = self.token_position_class(index, self.lineno, self.columnno)
if eof:
return self.token_class("EOF", "EOF", source_pos)
+
return self.token_class(self.automaton.names[self.last_matched_state],
text, source_pos)
diff --git a/rpython/rlib/rsocket.py b/rpython/rlib/rsocket.py
--- a/rpython/rlib/rsocket.py
+++ b/rpython/rlib/rsocket.py
@@ -1146,9 +1146,9 @@
address_to_fill=None):
# port_or_service is a string, not an int (but try str(port_number)).
assert port_or_service is None or isinstance(port_or_service, str)
- if _c._MACOSX:
- if port_or_service is None or port_or_service == '0':
- port_or_service = '00'
+ if _c._MACOSX and flags & AI_NUMERICSERV and \
+ (port_or_service is None or port_or_service == '0'):
+ port_or_service = '00'
hints = lltype.malloc(_c.addrinfo, flavor='raw', zero=True)
rffi.setintfield(hints, 'c_ai_family', family)
rffi.setintfield(hints, 'c_ai_socktype', socktype)
diff --git a/rpython/rlib/test/test_streamio.py b/rpython/rlib/test/test_streamio.py
--- a/rpython/rlib/test/test_streamio.py
+++ b/rpython/rlib/test/test_streamio.py
@@ -1104,6 +1104,21 @@
finally:
signal(SIGALRM, SIG_DFL)
+ def test_append_mode(self):
+ tfn = str(udir.join('streamio-append-mode'))
+ fo = streamio.open_file_as_stream # shorthand
+ x = fo(tfn, 'w')
+ x.write('abc123')
+ x.close()
+
+ x = fo(tfn, 'a')
+ x.seek(0, 0)
+ x.write('456')
+ x.close()
+ x = fo(tfn, 'r')
+ assert x.read() == 'abc123456'
+ x.close()
+
# Speed test
diff --git a/rpython/rtyper/module/ll_os_path.py b/rpython/rtyper/module/ll_os_path.py
--- a/rpython/rtyper/module/ll_os_path.py
+++ b/rpython/rtyper/module/ll_os_path.py
@@ -6,26 +6,24 @@
import stat
import os
-from rpython.tool.staticmethods import ClassMethods
# Does a path exist?
# This is false for dangling symbolic links.
-class BaseOsPath:
- __metaclass__ = ClassMethods
-
+class BaseOsPath(object):
+ @classmethod
def ll_os_path_exists(cls, path):
"""Test whether a path exists"""
try:
- st = os.stat(cls.from_rstr_nonnull(path))
+ os.stat(cls.from_rstr_nonnull(path))
except OSError:
return False
return True
+ @classmethod
def ll_os_path_isdir(cls, path):
try:
st = os.stat(cls.from_rstr_nonnull(path))
except OSError:
return False
return stat.S_ISDIR(st[0])
-
diff --git a/rpython/tool/identity_dict.py b/rpython/tool/identity_dict.py
--- a/rpython/tool/identity_dict.py
+++ b/rpython/tool/identity_dict.py
@@ -3,15 +3,15 @@
except ImportError:
idict = None
-from UserDict import DictMixin
+from collections import MutableMapping
-class IdentityDictPurePython(object, DictMixin):
+class IdentityDictPurePython(MutableMapping):
__slots__ = "_dict _keys".split()
def __init__(self):
self._dict = {}
- self._keys = {} # id(obj) -> obj
+ self._keys = {} # id(obj) -> obj
def __getitem__(self, arg):
return self._dict[id(arg)]
@@ -24,8 +24,11 @@
del self._keys[id(arg)]
del self._dict[id(arg)]
- def keys(self):
- return self._keys.values()
+ def __iter__(self):
+ return self._keys.itervalues()
+
+ def __len__(self):
+ return len(self._keys)
def __contains__(self, arg):
return id(arg) in self._dict
@@ -37,8 +40,7 @@
return d
-class IdentityDictPyPy(object, DictMixin):
- __slots__ = ["_dict"]
+class IdentityDictPyPy(MutableMapping):
def __init__(self):
self._dict = idict()
@@ -52,8 +54,11 @@
def __delitem__(self, arg):
del self._dict[arg]
- def keys(self):
- return self._dict.keys()
+ def __iter__(self):
+ return iter(self._dict.keys())
+
+ def __len__(self):
+ return len(self._dict)
def __contains__(self, arg):
return arg in self._dict
@@ -64,8 +69,10 @@
assert len(d) == len(self)
return d
+ def __nonzero__(self):
+ return bool(self._dict)
+
if idict is None:
identity_dict = IdentityDictPurePython
else:
identity_dict = IdentityDictPyPy
-
diff --git a/rpython/tool/staticmethods.py b/rpython/tool/staticmethods.py
--- a/rpython/tool/staticmethods.py
+++ b/rpython/tool/staticmethods.py
@@ -10,11 +10,5 @@
class StaticMethods(AbstractMethods):
"""
Metaclass that turns plain methods into staticmethods.
- """
+ """
decorator = staticmethod
-
-class ClassMethods(AbstractMethods):
- """
- Metaclass that turns plain methods into classmethods.
- """
- decorator = classmethod
More information about the pypy-commit
mailing list