[pypy-commit] pypy ndarray-subtype: merge default into branch
mattip
noreply at buildbot.pypy.org
Mon Jul 15 18:44:40 CEST 2013
Author: Matti Picus <matti.picus at gmail.com>
Branch: ndarray-subtype
Changeset: r65403:f050a9218874
Date: 2013-07-14 20:15 +0300
http://bitbucket.org/pypy/pypy/changeset/f050a9218874/
Log: merge default into branch
diff too long, truncating to 2000 out of 8063 lines
diff --git a/LICENSE b/LICENSE
--- a/LICENSE
+++ b/LICENSE
@@ -2,9 +2,9 @@
=======
Except when otherwise stated (look for LICENSE files in directories or
-information at the beginning of each file) all software and
-documentation in the 'pypy', 'ctype_configure', 'dotviewer', 'demo',
-and 'lib_pypy' directories is licensed as follows:
+information at the beginning of each file) all software and documentation in
+the 'rpython', 'pypy', 'ctype_configure', 'dotviewer', 'demo', and 'lib_pypy'
+directories is licensed as follows:
The MIT License
@@ -38,176 +38,239 @@
Armin Rigo
Maciej Fijalkowski
Carl Friedrich Bolz
+ Antonio Cuni
Amaury Forgeot d'Arc
- Antonio Cuni
Samuele Pedroni
+ Alex Gaynor
Michael Hudson
+ David Schneider
Holger Krekel
- Alex Gaynor
Christian Tismer
Hakan Ardo
Benjamin Peterson
- David Schneider
+ Matti Picus
+ Philip Jenvey
+ Anders Chrigstrom
+ Brian Kearns
Eric van Riet Paap
- Anders Chrigstrom
Richard Emslie
+ Alexander Schremmer
+ Wim Lavrijsen
Dan Villiom Podlaski Christiansen
- Alexander Schremmer
+ Manuel Jacob
Lukas Diekmann
+ Sven Hager
+ Anders Lehmann
Aurelien Campeas
- Anders Lehmann
+ Niklaus Haldimann
+ Ronan Lamy
Camillo Bruni
- Niklaus Haldimann
- Sven Hager
+ Laura Creighton
+ Toon Verwaest
Leonardo Santagada
- Toon Verwaest
Seo Sanghyeon
Justin Peel
+ Ronny Pfannschmidt
+ David Edelsohn
+ Anders Hammarquist
+ Jakub Gustak
+ Guido Wesdorp
Lawrence Oluyede
Bartosz Skowron
- Jakub Gustak
- Guido Wesdorp
Daniel Roberts
- Laura Creighton
+ Niko Matsakis
Adrien Di Mascio
Ludovic Aubry
- Niko Matsakis
- Wim Lavrijsen
- Matti Picus
+ Alexander Hesse
+ Jacob Hallen
+ Romain Guillebert
Jason Creighton
- Jacob Hallen
Alex Martelli
- Anders Hammarquist
+ Michal Bendowski
Jan de Mooij
+ Michael Foord
Stephan Diehl
- Michael Foord
Stefan Schwarzer
+ Valentino Volonghi
Tomek Meka
Patrick Maupin
+ stian
Bob Ippolito
Bruno Gola
+ Jean-Paul Calderone
+ Timo Paulssen
Alexandre Fayolle
+ Simon Burton
Marius Gedminas
- Simon Burton
- David Edelsohn
- Jean-Paul Calderone
John Witulski
- Timo Paulssen
- holger krekel
+ Greg Price
Dario Bertini
Mark Pearse
+ Simon Cross
+ Konstantin Lopuhin
Andreas Stührk
Jean-Philippe St. Pierre
Guido van Rossum
Pavel Vinogradov
- Valentino Volonghi
Paul deGrandis
Ilya Osadchiy
- Ronny Pfannschmidt
Adrian Kuhn
+ Boris Feigin
tav
Georg Brandl
- Philip Jenvey
+ Bert Freudenberg
+ Stian Andreassen
+ Stefano Rivera
+ Wanja Saatkamp
Gerald Klix
- Wanja Saatkamp
- Boris Feigin
+ Mike Blume
+ Taavi Burns
Oscar Nierstrasz
David Malcolm
Eugene Oden
Henry Mason
+ Preston Timmons
Jeff Terrace
+ David Ripton
+ Dusty Phillips
Lukas Renggli
Guenter Jantzen
+ Tobias Oberstein
+ Remi Meier
Ned Batchelder
- Bert Freudenberg
Amit Regmi
Ben Young
Nicolas Chauvat
Andrew Durdin
Michael Schneider
Nicholas Riley
+ Jason Chu
+ Igor Trindade Oliveira
+ Jeremy Thurgood
Rocco Moretti
Gintautas Miliauskas
Michael Twomey
- Igor Trindade Oliveira
Lucian Branescu Mihaila
+ Tim Felgentreff
+ Tyler Wade
+ Gabriel Lavoie
Olivier Dormond
Jared Grubb
Karl Bartel
- Gabriel Lavoie
+ Brian Dorsey
Victor Stinner
- Brian Dorsey
Stuart Williams
+ Jasper Schulz
Toby Watson
Antoine Pitrou
+ Aaron Iles
+ Michael Cheng
Justas Sadzevicius
+ Gasper Zejn
Neil Shepperd
Mikael Schönenberg
- Gasper Zejn
+ Elmo Mäntynen
+ Tobias Pape
Jonathan David Riehl
- Elmo Mäntynen
+ Stanislaw Halik
Anders Qvist
+ Chirag Jadwani
Beatrice During
+ Alex Perry
+ Vincent Legoll
+ Alan McIntyre
Alexander Sedov
Corbin Simpson
- Vincent Legoll
- Romain Guillebert
- Alan McIntyre
- Alex Perry
+ Christopher Pope
+ Laurence Tratt
+ Guillebert Romain
+ Christian Tismer
+ Dan Stromberg
+ Stefano Parmesan
+ Christian Hudon
+ Alexis Daboville
Jens-Uwe Mager
- Simon Cross
- Dan Stromberg
- Guillebert Romain
Carl Meyer
+ Karl Ramm
Pieter Zieschang
+ Gabriel
+ Paweł Piotr Przeradowski
+ Andrew Dalke
+ Sylvain Thenault
+ Nathan Taylor
+ Vladimir Kryachko
+ Jacek Generowicz
Alejandro J. Cura
- Sylvain Thenault
- Christoph Gerum
+ Jacob Oscarson
Travis Francis Athougies
+ Kristjan Valur Jonsson
+ Neil Blakey-Milner
+ Lutz Paelike
+ Lucio Torre
+ Lars Wassermann
Henrik Vendelbo
- Lutz Paelike
- Jacob Oscarson
- Martin Blais
- Lucio Torre
- Lene Wagner
+ Dan Buch
Miguel de Val Borro
Artur Lisiecki
- Bruno Gola
+ Sergey Kishchenko
Ignas Mikalajunas
- Stefano Rivera
+ Christoph Gerum
+ Martin Blais
+ Lene Wagner
+ Tomo Cocoa
+ Andrews Medina
+ roberto at goyle
+ William Leslie
+ Bobby Impollonia
+ timo at eistee.fritz.box
+ Andrew Thompson
+ Yusei Tahara
+ Roberto De Ioris
+ Juan Francisco Cantero Hurtado
+ Godefroid Chappelle
Joshua Gilbert
- Godefroid Chappelle
- Yusei Tahara
+ Dan Colish
Christopher Armstrong
+ Michael Hudson-Doyle
+ Anders Sigfridsson
+ Yasir Suhail
+ Floris Bruynooghe
+ Akira Li
+ Gustavo Niemeyer
Stephan Busemann
- Gustavo Niemeyer
- William Leslie
- Akira Li
- Kristjan Valur Jonsson
- Bobby Impollonia
- Michael Hudson-Doyle
- Laurence Tratt
- Yasir Suhail
- Andrew Thompson
- Anders Sigfridsson
- Floris Bruynooghe
- Jacek Generowicz
- Dan Colish
- Zooko Wilcox-O Hearn
- Dan Loewenherz
+ Anna Katrina Dominguez
+ Christian Muirhead
+ James Lan
+ shoma hosaka
+ Daniel Neuhäuser
+ Buck Golemon
+ Konrad Delong
+ Dinu Gherman
Chris Lambacher
- Dinu Gherman
- Brett Cannon
- Daniel Neuhäuser
- Michael Chermside
- Konrad Delong
- Anna Ravencroft
- Greg Price
- Armin Ronacher
- Christian Muirhead
+ coolbutuseless at gmail.com
Jim Baker
Rodrigo Araújo
- Romain Guillebert
+ Armin Ronacher
+ Brett Cannon
+ yrttyr
+ Zooko Wilcox-O Hearn
+ Tomer Chachamu
+ Christopher Groskopf
+ opassembler.py
+ Antony Lee
+ Jim Hunziker
+ Markus Unterwaditzer
+ Even Wiik Thomassen
+ jbs
+ soareschen
+ Flavio Percoco
+ Kristoffer Kleine
+ yasirs
+ Michael Chermside
+ Anna Ravencroft
+ Andrew Chambers
+ Julien Phalip
+ Dan Loewenherz
Heinrich-Heine University, Germany
Open End AB (formerly AB Strakt), Sweden
@@ -219,32 +282,32 @@
Change Maker, Sweden
University of California Berkeley, USA
Google Inc.
+ King's College London
The PyPy Logo as used by http://speed.pypy.org and others was created
by Samuel Reis and is distributed on terms of Creative Commons Share Alike
License.
-License for 'lib-python/2.7.0' and 'lib-python/2.7.0-modified'
-==============================================================
+License for 'lib-python/2.7'
+============================
-Except when otherwise stated (look for LICENSE files or
-copyright/license information at the beginning of each file) the files
-in the 'lib-python/2.7.0' and 'lib-python/2.7.0-modified' directories
-are all copyrighted by the Python Software Foundation and licensed under
-the Python Software License of which you can find a copy here:
+Except when otherwise stated (look for LICENSE files or copyright/license
+information at the beginning of each file) the files in the 'lib-python/2.7'
+directory are all copyrighted by the Python Software Foundation and licensed
+under the Python Software License of which you can find a copy here:
http://www.python.org/doc/Copyright.html
-License for 'pypy/translator/jvm/src/jna.jar'
+License for 'rpython/translator/jvm/src/jna.jar'
=============================================
-The file 'pypy/translator/jvm/src/jna.jar' is licensed under the GNU
+The file 'rpyhton/translator/jvm/src/jna.jar' is licensed under the GNU
Lesser General Public License of which you can find a copy here:
http://www.gnu.org/licenses/lgpl.html
-License for 'pypy/translator/jvm/src/jasmin.jar'
+License for 'rpython/translator/jvm/src/jasmin.jar'
================================================
-The file 'pypy/translator/jvm/src/jasmin.jar' is copyright (c) 1996-2004 Jon Meyer
+The file 'rpyhton/translator/jvm/src/jasmin.jar' is copyright (c) 1996-2004 Jon Meyer
and distributed with permission. The use of Jasmin by PyPy does not imply
that PyPy is endorsed by Jon Meyer nor any of Jasmin's contributors. Furthermore,
the following disclaimer applies to Jasmin:
diff --git a/lib-python/2.7/distutils/sysconfig_pypy.py b/lib-python/2.7/distutils/sysconfig_pypy.py
--- a/lib-python/2.7/distutils/sysconfig_pypy.py
+++ b/lib-python/2.7/distutils/sysconfig_pypy.py
@@ -65,6 +65,11 @@
g['SOABI'] = g['SO'].rsplit('.')[0]
g['LIBDIR'] = os.path.join(sys.prefix, 'lib')
g['CC'] = "gcc -pthread" # -pthread might not be valid on OS/X, check
+ g['OPT'] = ""
+ g['CFLAGS'] = ""
+ g['CPPFLAGS'] = ""
+ g['CCSHARED'] = '-shared -O2 -fPIC -Wimplicit'
+ g['LDSHARED'] = g['CC'] + ' -shared'
global _config_vars
_config_vars = g
@@ -122,13 +127,34 @@
optional C speedup components.
"""
if compiler.compiler_type == "unix":
- compiler.compiler_so.extend(['-O2', '-fPIC', '-Wimplicit'])
+ cc, opt, cflags, ccshared, ldshared = get_config_vars(
+ 'CC', 'OPT', 'CFLAGS', 'CCSHARED', 'LDSHARED')
+
compiler.shared_lib_extension = get_config_var('SO')
- if "CFLAGS" in os.environ:
- cflags = os.environ["CFLAGS"].split()
- compiler.compiler.extend(cflags)
- compiler.compiler_so.extend(cflags)
- compiler.linker_so.extend(cflags)
+
+ if 'LDSHARED' in os.environ:
+ ldshared = os.environ['LDSHARED']
+ if 'CPP' in os.environ:
+ cpp = os.environ['CPP']
+ else:
+ cpp = cc + " -E" # not always
+ if 'LDFLAGS' in os.environ:
+ ldshared = ldshared + ' ' + os.environ['LDFLAGS']
+ if 'CFLAGS' in os.environ:
+ cflags = opt + ' ' + os.environ['CFLAGS']
+ ldshared = ldshared + ' ' + os.environ['CFLAGS']
+ if 'CPPFLAGS' in os.environ:
+ cpp = cpp + ' ' + os.environ['CPPFLAGS']
+ cflags = cflags + ' ' + os.environ['CPPFLAGS']
+ ldshared = ldshared + ' ' + os.environ['CPPFLAGS']
+
+ cc_cmd = cc + ' ' + cflags
+
+ compiler.set_executables(
+ preprocessor=cpp,
+ compiler=cc_cmd,
+ compiler_so=cc_cmd + ' ' + ccshared,
+ linker_so=ldshared)
from sysconfig_cpython import (
diff --git a/lib-python/2.7/json/__init__.py b/lib-python/2.7/json/__init__.py
--- a/lib-python/2.7/json/__init__.py
+++ b/lib-python/2.7/json/__init__.py
@@ -105,6 +105,12 @@
__author__ = 'Bob Ippolito <bob at redivi.com>'
+try:
+ # PyPy speedup, the interface is different than CPython's _json
+ import _pypyjson
+except ImportError:
+ _pypyjson = None
+
from .decoder import JSONDecoder
from .encoder import JSONEncoder
@@ -241,7 +247,6 @@
_default_decoder = JSONDecoder(encoding=None, object_hook=None,
object_pairs_hook=None)
-
def load(fp, encoding=None, cls=None, object_hook=None, parse_float=None,
parse_int=None, parse_constant=None, object_pairs_hook=None, **kw):
"""Deserialize ``fp`` (a ``.read()``-supporting file-like object containing
@@ -323,7 +328,10 @@
if (cls is None and encoding is None and object_hook is None and
parse_int is None and parse_float is None and
parse_constant is None and object_pairs_hook is None and not kw):
- return _default_decoder.decode(s)
+ if _pypyjson and not isinstance(s, unicode):
+ return _pypyjson.loads(s)
+ else:
+ return _default_decoder.decode(s)
if cls is None:
cls = JSONDecoder
if object_hook is not None:
diff --git a/pypy/config/pypyoption.py b/pypy/config/pypyoption.py
--- a/pypy/config/pypyoption.py
+++ b/pypy/config/pypyoption.py
@@ -35,7 +35,7 @@
"thread", "itertools", "pyexpat", "_ssl", "cpyext", "array",
"binascii", "_multiprocessing", '_warnings',
"_collections", "_multibytecodec", "micronumpy", "_ffi",
- "_continuation", "_cffi_backend", "_csv", "cppyy"]
+ "_continuation", "_cffi_backend", "_csv", "cppyy", "_pypyjson"]
))
translation_modules = default_modules.copy()
diff --git a/pypy/doc/coding-guide.rst b/pypy/doc/coding-guide.rst
--- a/pypy/doc/coding-guide.rst
+++ b/pypy/doc/coding-guide.rst
@@ -907,7 +907,7 @@
runs at application level. If you need to use modules
you have to import them within the test function.
-Another possibility to pass in data into the AppTest is to use
+Data can be passed into the AppTest using
the ``setup_class`` method of the AppTest. All wrapped objects that are
attached to the class there and start with ``w_`` can be accessed
via self (but without the ``w_``) in the actual test method. An example::
@@ -922,6 +922,46 @@
.. _`run the tests as usual`:
+Another possibility is to use cls.space.appexec, for example::
+
+ class AppTestSomething(object):
+ def setup_class(cls):
+ arg = 2
+ cls.w_result = cls.space.appexec([cls.space.wrap(arg)], """(arg):
+ return arg ** 6
+ """)
+
+ def test_power(self):
+ assert self.result == 2 ** 6
+
+which executes the code string function with the given arguments at app level.
+Note the use of ``w_result`` in ``setup_class`` but self.result in the test
+Here is how to define an app level class in ``setup_class`` that can be used
+in subsequent tests::
+
+ class AppTestSet(object):
+ def setup_class(cls):
+ w_fakeint = cls.space.appexec([], """():
+ class FakeInt(object):
+ def __init__(self, value):
+ self.value = value
+ def __hash__(self):
+ return hash(self.value)
+
+ def __eq__(self, other):
+ if other == self.value:
+ return True
+ return False
+ return FakeInt
+ """)
+ cls.w_FakeInt = w_fakeint
+
+ def test_fakeint(self):
+ f1 = self.FakeInt(4)
+ assert f1 == 4
+ assert hash(f1) == hash(4)
+
+
Command line tool test_all
--------------------------
diff --git a/pypy/doc/config/objspace.usemodules._pypyjson.txt b/pypy/doc/config/objspace.usemodules._pypyjson.txt
new file mode 100644
--- /dev/null
+++ b/pypy/doc/config/objspace.usemodules._pypyjson.txt
@@ -0,0 +1,1 @@
+RPython speedups for the stdlib json module
diff --git a/pypy/doc/contributor.rst b/pypy/doc/contributor.rst
--- a/pypy/doc/contributor.rst
+++ b/pypy/doc/contributor.rst
@@ -6,171 +6,240 @@
code base, ordered by number of commits (which is certainly not a very
appropriate measure but it's something)::
- Armin Rigo
- Maciej Fijalkowski
- Carl Friedrich Bolz
- Antonio Cuni
- Amaury Forgeot d'Arc
- Samuele Pedroni
- Michael Hudson
- Holger Krekel
- Benjamin Peterson
- Christian Tismer
- Hakan Ardo
- Alex Gaynor
- Eric van Riet Paap
- Anders Chrigstrom
- David Schneider
- Richard Emslie
- Dan Villiom Podlaski Christiansen
- Alexander Schremmer
- Aurelien Campeas
- Anders Lehmann
- Camillo Bruni
- Niklaus Haldimann
- Leonardo Santagada
- Toon Verwaest
- Seo Sanghyeon
- Lawrence Oluyede
- Bartosz Skowron
- Jakub Gustak
- Guido Wesdorp
- Daniel Roberts
- Adrien Di Mascio
- Laura Creighton
- Ludovic Aubry
- Niko Matsakis
- Jason Creighton
- Jacob Hallen
- Alex Martelli
- Anders Hammarquist
- Jan de Mooij
- Wim Lavrijsen
- Stephan Diehl
- Michael Foord
- Stefan Schwarzer
- Tomek Meka
- Patrick Maupin
- Bob Ippolito
- Bruno Gola
- Alexandre Fayolle
- Marius Gedminas
- Simon Burton
- Justin Peel
- Jean-Paul Calderone
- John Witulski
- Lukas Diekmann
- holger krekel
- Wim Lavrijsen
- Dario Bertini
- Andreas Stührk
- Jean-Philippe St. Pierre
- Guido van Rossum
- Pavel Vinogradov
- Valentino Volonghi
- Paul deGrandis
- Adrian Kuhn
- tav
- Georg Brandl
- Gerald Klix
- Wanja Saatkamp
- Ronny Pfannschmidt
- Boris Feigin
- Oscar Nierstrasz
- David Malcolm
- Eugene Oden
- Henry Mason
- Sven Hager
- Lukas Renggli
- Ilya Osadchiy
- Guenter Jantzen
- Bert Freudenberg
- Amit Regmi
- Ben Young
- Nicolas Chauvat
- Andrew Durdin
- Michael Schneider
- Nicholas Riley
- Rocco Moretti
- Gintautas Miliauskas
- Michael Twomey
- Igor Trindade Oliveira
- Lucian Branescu Mihaila
- Olivier Dormond
- Jared Grubb
- Karl Bartel
- Gabriel Lavoie
- Victor Stinner
- Brian Dorsey
- Stuart Williams
- Toby Watson
- Antoine Pitrou
- Justas Sadzevicius
- Neil Shepperd
- Mikael Schönenberg
- Gasper Zejn
- Jonathan David Riehl
- Elmo Mäntynen
- Anders Qvist
- Beatrice During
- Alexander Sedov
- Timo Paulssen
- Corbin Simpson
- Vincent Legoll
- Romain Guillebert
- Alan McIntyre
- Alex Perry
- Jens-Uwe Mager
- Simon Cross
- Dan Stromberg
- Guillebert Romain
- Carl Meyer
- Pieter Zieschang
- Alejandro J. Cura
- Sylvain Thenault
- Christoph Gerum
- Travis Francis Athougies
- Henrik Vendelbo
- Lutz Paelike
- Jacob Oscarson
- Martin Blais
- Lucio Torre
- Lene Wagner
- Miguel de Val Borro
- Ignas Mikalajunas
- Artur Lisiecki
- Philip Jenvey
- Joshua Gilbert
- Godefroid Chappelle
- Yusei Tahara
- Christopher Armstrong
- Stephan Busemann
- Gustavo Niemeyer
- William Leslie
- Akira Li
- Kristjan Valur Jonsson
- Bobby Impollonia
- Michael Hudson-Doyle
- Andrew Thompson
- Anders Sigfridsson
- Floris Bruynooghe
- Jacek Generowicz
- Dan Colish
- Zooko Wilcox-O Hearn
- Dan Villiom Podlaski Christiansen
- Anders Hammarquist
- Chris Lambacher
- Dinu Gherman
- Dan Colish
- Brett Cannon
- Daniel Neuhäuser
- Michael Chermside
- Konrad Delong
- Anna Ravencroft
- Greg Price
- Armin Ronacher
- Christian Muirhead
- Jim Baker
- Rodrigo Araújo
- Romain Guillebert
+ Armin Rigo
+ Maciej Fijalkowski
+ Carl Friedrich Bolz
+ Antonio Cuni
+ Amaury Forgeot d'Arc
+ Samuele Pedroni
+ Alex Gaynor
+ Michael Hudson
+ David Schneider
+ Holger Krekel
+ Christian Tismer
+ Hakan Ardo
+ Benjamin Peterson
+ Matti Picus
+ Philip Jenvey
+ Anders Chrigstrom
+ Brian Kearns
+ Eric van Riet Paap
+ Richard Emslie
+ Alexander Schremmer
+ Wim Lavrijsen
+ Dan Villiom Podlaski Christiansen
+ Manuel Jacob
+ Lukas Diekmann
+ Sven Hager
+ Anders Lehmann
+ Aurelien Campeas
+ Niklaus Haldimann
+ Ronan Lamy
+ Camillo Bruni
+ Laura Creighton
+ Toon Verwaest
+ Leonardo Santagada
+ Seo Sanghyeon
+ Justin Peel
+ Ronny Pfannschmidt
+ David Edelsohn
+ Anders Hammarquist
+ Jakub Gustak
+ Guido Wesdorp
+ Lawrence Oluyede
+ Bartosz Skowron
+ Daniel Roberts
+ Niko Matsakis
+ Adrien Di Mascio
+ Ludovic Aubry
+ Alexander Hesse
+ Jacob Hallen
+ Romain Guillebert
+ Jason Creighton
+ Alex Martelli
+ Michal Bendowski
+ Jan de Mooij
+ Michael Foord
+ Stephan Diehl
+ Stefan Schwarzer
+ Valentino Volonghi
+ Tomek Meka
+ Patrick Maupin
+ stian
+ Bob Ippolito
+ Bruno Gola
+ Jean-Paul Calderone
+ Timo Paulssen
+ Alexandre Fayolle
+ Simon Burton
+ Marius Gedminas
+ John Witulski
+ Greg Price
+ Dario Bertini
+ Mark Pearse
+ Simon Cross
+ Konstantin Lopuhin
+ Andreas Stührk
+ Jean-Philippe St. Pierre
+ Guido van Rossum
+ Pavel Vinogradov
+ Paul deGrandis
+ Ilya Osadchiy
+ Adrian Kuhn
+ Boris Feigin
+ tav
+ Georg Brandl
+ Bert Freudenberg
+ Stian Andreassen
+ Stefano Rivera
+ Wanja Saatkamp
+ Gerald Klix
+ Mike Blume
+ Taavi Burns
+ Oscar Nierstrasz
+ David Malcolm
+ Eugene Oden
+ Henry Mason
+ Preston Timmons
+ Jeff Terrace
+ David Ripton
+ Dusty Phillips
+ Lukas Renggli
+ Guenter Jantzen
+ Tobias Oberstein
+ Remi Meier
+ Ned Batchelder
+ Amit Regmi
+ Ben Young
+ Nicolas Chauvat
+ Andrew Durdin
+ Michael Schneider
+ Nicholas Riley
+ Jason Chu
+ Igor Trindade Oliveira
+ Jeremy Thurgood
+ Rocco Moretti
+ Gintautas Miliauskas
+ Michael Twomey
+ Lucian Branescu Mihaila
+ Tim Felgentreff
+ Tyler Wade
+ Gabriel Lavoie
+ Olivier Dormond
+ Jared Grubb
+ Karl Bartel
+ Brian Dorsey
+ Victor Stinner
+ Stuart Williams
+ Jasper Schulz
+ Toby Watson
+ Antoine Pitrou
+ Aaron Iles
+ Michael Cheng
+ Justas Sadzevicius
+ Gasper Zejn
+ Neil Shepperd
+ Mikael Schönenberg
+ Elmo Mäntynen
+ Tobias Pape
+ Jonathan David Riehl
+ Stanislaw Halik
+ Anders Qvist
+ Chirag Jadwani
+ Beatrice During
+ Alex Perry
+ Vincent Legoll
+ Alan McIntyre
+ Alexander Sedov
+ Corbin Simpson
+ Christopher Pope
+ Laurence Tratt
+ Guillebert Romain
+ Christian Tismer
+ Dan Stromberg
+ Stefano Parmesan
+ Christian Hudon
+ Alexis Daboville
+ Jens-Uwe Mager
+ Carl Meyer
+ Karl Ramm
+ Pieter Zieschang
+ Gabriel
+ Paweł Piotr Przeradowski
+ Andrew Dalke
+ Sylvain Thenault
+ Nathan Taylor
+ Vladimir Kryachko
+ Jacek Generowicz
+ Alejandro J. Cura
+ Jacob Oscarson
+ Travis Francis Athougies
+ Kristjan Valur Jonsson
+ Neil Blakey-Milner
+ Lutz Paelike
+ Lucio Torre
+ Lars Wassermann
+ Henrik Vendelbo
+ Dan Buch
+ Miguel de Val Borro
+ Artur Lisiecki
+ Sergey Kishchenko
+ Ignas Mikalajunas
+ Christoph Gerum
+ Martin Blais
+ Lene Wagner
+ Tomo Cocoa
+ Andrews Medina
+ roberto at goyle
+ William Leslie
+ Bobby Impollonia
+ timo at eistee.fritz.box
+ Andrew Thompson
+ Yusei Tahara
+ Roberto De Ioris
+ Juan Francisco Cantero Hurtado
+ Godefroid Chappelle
+ Joshua Gilbert
+ Dan Colish
+ Christopher Armstrong
+ Michael Hudson-Doyle
+ Anders Sigfridsson
+ Yasir Suhail
+ Floris Bruynooghe
+ Akira Li
+ Gustavo Niemeyer
+ Stephan Busemann
+ Anna Katrina Dominguez
+ Christian Muirhead
+ James Lan
+ shoma hosaka
+ Daniel Neuhäuser
+ Buck Golemon
+ Konrad Delong
+ Dinu Gherman
+ Chris Lambacher
+ coolbutuseless at gmail.com
+ Jim Baker
+ Rodrigo Araújo
+ Armin Ronacher
+ Brett Cannon
+ yrttyr
+ Zooko Wilcox-O Hearn
+ Tomer Chachamu
+ Christopher Groskopf
+ opassembler.py
+ Antony Lee
+ Jim Hunziker
+ Markus Unterwaditzer
+ Even Wiik Thomassen
+ jbs
+ soareschen
+ Flavio Percoco
+ Kristoffer Kleine
+ yasirs
+ Michael Chermside
+ Anna Ravencroft
+ Andrew Chambers
+ Julien Phalip
+ Dan Loewenherz
diff --git a/pypy/doc/faq.rst b/pypy/doc/faq.rst
--- a/pypy/doc/faq.rst
+++ b/pypy/doc/faq.rst
@@ -96,8 +96,21 @@
Does PyPy have a GIL? Why?
-------------------------------------------------
-Yes, PyPy has a GIL. Removing the GIL is very hard. The first problem
-is that our garbage collectors are not re-entrant.
+Yes, PyPy has a GIL. Removing the GIL is very hard. The problems are
+essentially the same as with CPython (including the fact that our
+garbage collectors are not thread-safe so far). Fixing it is possible,
+as shown by Jython and IronPython, but difficult. It would require
+adapting the whole source code of PyPy, including subtle decisions about
+whether some effects are ok or not for the user (i.e. the Python
+programmer).
+
+Instead, since 2012, there is work going on on a still very experimental
+Software Transactional Memory (STM) version of PyPy. This should give
+an alternative PyPy which internally has no GIL, while at the same time
+continuing to give the Python programmer the complete illusion of having
+one. It would in fact push forward *more* GIL-ish behavior, like
+declaring that some sections of the code should run without releasing
+the GIL in the middle (these are called *atomic sections* in STM).
------------------------------------------
How do I write extension modules for PyPy?
@@ -306,7 +319,7 @@
No, and you shouldn't try. First and foremost, RPython is a language
designed for writing interpreters. It is a restricted subset of
-Python. If you program is not an interpreter but tries to do "real
+Python. If your program is not an interpreter but tries to do "real
things", like use *any* part of the standard Python library or *any*
3rd-party library, then it is not RPython to start with. You should
only look at RPython if you try to `write your own interpreter`__.
@@ -322,8 +335,35 @@
Yes, it is possible with enough effort to compile small self-contained
pieces of RPython code doing a few performance-sensitive things. But
this case is not interesting for us. If you needed to rewrite the code
-in RPython, you could as well have rewritten it in C for example. The
-latter is a much more supported, much more documented language `:-)`
+in RPython, you could as well have rewritten it in C or C++ or Java for
+example. These are much more supported, much more documented languages
+`:-)`
+
+ *The above paragraphs are not the whole truth. It* is *true that there
+ are cases where writing a program as RPython gives you substantially
+ better speed than running it on top of PyPy. However, the attitude of
+ the core group of people behind PyPy is to answer: "then report it as a
+ performance bug against PyPy!".*
+
+ *Here is a more diluted way to put it. The "No, don't!" above is a
+ general warning we give to new people. They are likely to need a lot
+ of help from* some *source, because RPython is not so simple nor
+ extensively documented; but at the same time, we, the pypy core group
+ of people, are not willing to invest time in supporting 3rd-party
+ projects that do very different things than interpreters for dynamic
+ languages --- just because we have other interests and there are only
+ so many hours a day. So as a summary I believe it is only fair to
+ attempt to point newcomers at existing alternatives, which are more
+ mainstream and where they will get help from many people.*
+
+ *If anybody seriously wants to promote RPython anyway, he is welcome
+ to: we won't actively resist such a plan. There are a lot of things
+ that could be done to make RPython a better Java-ish language for
+ example, starting with supporting non-GIL-based multithreading, but we
+ don't implement them because they have little relevance to us. This
+ is open source, which means that anybody is free to promote and
+ develop anything; but it also means that you must let us choose* not
+ *to go into that direction ourselves.*
---------------------------------------------------
Which backends are there for the RPython toolchain?
diff --git a/pypy/doc/getting-started-dev.rst b/pypy/doc/getting-started-dev.rst
--- a/pypy/doc/getting-started-dev.rst
+++ b/pypy/doc/getting-started-dev.rst
@@ -4,6 +4,12 @@
.. contents::
+.. warning::
+
+ Please `read this FAQ entry`_ first!
+
+.. _`read this FAQ entry`: http://doc.pypy.org/en/latest/faq.html#do-i-have-to-rewrite-my-programs-in-rpython
+
RPython is a subset of Python that can be statically compiled. The PyPy
interpreter is written mostly in RPython (with pieces in Python), while
the RPython compiler is written in Python. The hard to understand part
diff --git a/pypy/doc/how-to-release.rst b/pypy/doc/how-to-release.rst
--- a/pypy/doc/how-to-release.rst
+++ b/pypy/doc/how-to-release.rst
@@ -32,11 +32,10 @@
* go to pypy/tool/release and run:
force-builds.py /release/<release branch>
* wait for builds to complete, make sure there are no failures
-* run pypy/tool/release/make_release.py, this will build necessary binaries
- and upload them to pypy.org
+* upload binaries to https://bitbucket.org/pypy/pypy/downloads
Following binaries should be built, however, we need more buildbots:
- JIT: windows, linux, os/x
+ JIT: windows, linux, os/x, armhf, armel
no JIT: windows, linux, os/x
sandbox: linux, os/x
diff --git a/pypy/doc/release-2.1.0-beta1.rst b/pypy/doc/release-2.1.0-beta1.rst
new file mode 100644
--- /dev/null
+++ b/pypy/doc/release-2.1.0-beta1.rst
@@ -0,0 +1,72 @@
+===============
+PyPy 2.1 beta 1
+===============
+
+We're pleased to announce the first beta of the upcoming 2.1 release of PyPy.
+This beta contains many bugfixes and improvements, numerous improvements to the
+numpy in pypy effort. The main feature being that the ARM processor support is
+not longer considered alpha level. We would like to thank the `Raspberry Pi
+Foundation`_ for supporting the work to finish PyPy's ARM support.
+
+You can download the PyPy 2.1 beta 1 release here:
+
+ http://pypy.org/download.html
+
+.. _`Raspberry Pi Foundation`: http://www.raspberrypi.org
+
+Highlights
+==========
+
+* Bugfixes to the ARM JIT backend, so that ARM is now an officially
+ supported processor architecture
+
+* Stacklet support on ARM
+
+* Interpreter improvements
+
+* Various numpy improvements
+
+* Bugfixes to cffi and ctypes
+
+* Bugfixes to the stacklet support
+
+* Improved logging performance
+
+* Faster sets for objects
+
+What is PyPy?
+=============
+
+PyPy is a very compliant Python interpreter, almost a drop-in replacement for
+CPython 2.7.3. It's fast due to its integrated tracing JIT compiler.
+
+This release supports x86 machines running Linux 32/64, Mac OS X 64 or Windows
+32. Also this release supports ARM machines running Linux 32bit - anything with
+``ARMv6`` (like the Raspberry Pi) or ``ARMv7`` (like Beagleboard,
+Chromebook, Cubieboard, etc.) that supports ``VFPv3`` should work. Both
+hard-float ``armhf/gnueabihf`` and soft-float ``armel/gnueabi`` builds are
+provided. ``armhf`` builds for Raspbian are created using the Raspberry Pi
+`custom cross-compilation toolchain <https://github.com/raspberrypi>`_
+based on ``gcc-arm-linux-gnueabihf`` and should work on ``ARMv6`` and
+``ARMv7`` devices running Debian or Raspbian. ``armel`` builds are built
+using the ``gcc-arm-linux-gnuebi`` toolchain provided by Ubuntu and
+currently target ``ARMv7``.
+
+Windows 64 work is still stalling, we would welcome a volunteer
+to handle that.
+
+How to use PyPy?
+================
+
+We suggest using PyPy from a `virtualenv`_. Once you have a virtualenv
+installed, you can follow instructions from `pypy documentation`_ on how
+to proceed. This document also covers other `installation schemes`_.
+
+.. _`pypy documentation`: http://doc.pypy.org/en/latest/getting-started.html#installing-using-virtualenv
+.. _`virtualenv`: http://www.virtualenv.org/en/latest/
+.. _`installation schemes`: http://doc.pypy.org/en/latest/getting-started.html#installing-pypy
+.. _`PyPy and pip`: http://doc.pypy.org/en/latest/getting-started.html#installing-pypy
+
+
+Cheers,
+the PyPy team
diff --git a/pypy/doc/whatsnew-head.rst b/pypy/doc/whatsnew-head.rst
--- a/pypy/doc/whatsnew-head.rst
+++ b/pypy/doc/whatsnew-head.rst
@@ -2,77 +2,25 @@
What's new in PyPy 2.1
======================
-.. this is a revision shortly after release-2.0
-.. startrev: a13c07067613
+.. this is a revision shortly after release-2.1-beta
+.. startrev: 4eb52818e7c0
-.. branch: ndarray-ptp
-put and array.put
+.. branch: fastjson
+Fast json decoder written in RPython, about 3-4x faster than the pure Python
+decoder which comes with the stdlib
-.. branch: numpy-pickle
-Pickling of numpy arrays and dtypes (including record dtypes)
+.. branch: improve-str2charp
+Improve the performance of I/O writing up to 15% by using memcpy instead of
+copying char-by-char in str2charp and get_nonmovingbuffer
-.. branch: remove-array-smm
-Remove multimethods in the arraymodule
+.. branch: flowoperators
+Simplify rpython/flowspace/ code by using more metaprogramming. Create
+SpaceOperator class to gather static information about flow graph operations.
-.. branch: callback-stacklet
-Fixed bug when switching stacklets from a C callback
+.. branch: package-tk
+Adapt package.py script to compile CFFI tk extension. Add a --without-tk switch
+to optionally skip it.
-.. branch: remove-set-smm
-Remove multi-methods on sets
-
-.. branch: numpy-subarrays
-Implement subarrays for numpy
-
-.. branch: remove-dict-smm
-Remove multi-methods on dict
-
-.. branch: remove-list-smm-2
-Remove remaining multi-methods on list
-
-.. branch: arm-stacklet
-Stacklet support for ARM, enables _continuation support
-
-.. branch: remove-tuple-smm
-Remove multi-methods on tuple
-
-.. branch: remove-iter-smm
-Remove multi-methods on iterators
-
-.. branch: emit-call-x86
-.. branch: emit-call-arm
-
-.. branch: on-abort-resops
-Added list of resops to the pypyjit on_abort hook.
-
-.. branch: logging-perf
-Speeds up the stdlib logging module
-
-.. branch: operrfmt-NT
-Adds a couple convenient format specifiers to operationerrfmt
-
-.. branch: win32-fixes3
-Skip and fix some non-translated (own) tests for win32 builds
-
-.. branch: ctypes-byref
-Add the '_obj' attribute on ctypes pointer() and byref() objects
-
-.. branch: argsort-segfault
-Fix a segfault in argsort when sorting by chunks on multidim numpypy arrays (mikefc)
-
-.. branch: dtype-isnative
-.. branch: ndarray-round
-
-.. branch: faster-str-of-bigint
-Improve performance of str(long).
-
-.. branch: ndarray-view
-Add view to ndarray and zeroD arrays, not on dtype scalars yet
-
-.. branch: numpypy-segfault
-fix segfault caused by iterating over empty ndarrays
-
-.. branch: identity-set
-Faster sets for objects
-
-.. branch: inline-identityhash
-Inline the fast path of id() and hash()
+.. branch: distutils-cppldflags
+Copy CPython's implementation of customize_compiler, dont call split on
+environment variables, honour CFLAGS, CPPFLAGS, LDSHARED and LDFLAGS on Unices.
diff --git a/pypy/interpreter/astcompiler/test/test_compiler.py b/pypy/interpreter/astcompiler/test/test_compiler.py
--- a/pypy/interpreter/astcompiler/test/test_compiler.py
+++ b/pypy/interpreter/astcompiler/test/test_compiler.py
@@ -881,15 +881,15 @@
assert "0 ('hi')" not in output.getvalue()
def test_print_to(self):
- exec """if 1:
- from StringIO import StringIO
- s = StringIO()
- print >> s, "hi", "lovely!"
- assert s.getvalue() == "hi lovely!\\n"
- s = StringIO()
- print >> s, "hi", "lovely!",
- assert s.getvalue() == "hi lovely!"
- """ in {}
+ exec """if 1:
+ from StringIO import StringIO
+ s = StringIO()
+ print >> s, "hi", "lovely!"
+ assert s.getvalue() == "hi lovely!\\n"
+ s = StringIO()
+ print >> s, "hi", "lovely!",
+ assert s.getvalue() == "hi lovely!"
+ """ in {}
def test_assert_with_tuple_arg(self):
try:
diff --git a/pypy/interpreter/gateway.py b/pypy/interpreter/gateway.py
--- a/pypy/interpreter/gateway.py
+++ b/pypy/interpreter/gateway.py
@@ -347,7 +347,7 @@
(self.use(typ), self.nextarg()))
def visit__ObjSpace(self, el):
- if self.finger != 0:
+ if self.finger > 1:
raise FastFuncNotSupported
self.unwrap.append("space")
@@ -414,21 +414,21 @@
mod = ""
if mod == 'pypy.interpreter.astcompiler.ast':
raise FastFuncNotSupported
- if (not mod.startswith('pypy.module.__builtin__') and
- not mod.startswith('pypy.module.sys') and
- not mod.startswith('pypy.module.math')):
- if not func.__name__.startswith('descr'):
- raise FastFuncNotSupported
+ #if (not mod.startswith('pypy.module.__builtin__') and
+ # not mod.startswith('pypy.module.sys') and
+ # not mod.startswith('pypy.module.math')):
+ # if not func.__name__.startswith('descr'):
+ # raise FastFuncNotSupported
d = {}
unwrap_info.miniglobals['func'] = func
source = """if 1:
def fastfunc_%s_%d(%s):
return func(%s)
- \n""" % (func.__name__, narg,
+ \n""" % (func.__name__.replace('-', '_'), narg,
', '.join(args),
', '.join(unwrap_info.unwrap))
exec compile2(source) in unwrap_info.miniglobals, d
- fastfunc = d['fastfunc_%s_%d' % (func.__name__, narg)]
+ fastfunc = d['fastfunc_%s_%d' % (func.__name__.replace('-', '_'), narg)]
return narg, fastfunc
make_fastfunc = staticmethod(make_fastfunc)
diff --git a/pypy/interpreter/pyframe.py b/pypy/interpreter/pyframe.py
--- a/pypy/interpreter/pyframe.py
+++ b/pypy/interpreter/pyframe.py
@@ -371,7 +371,7 @@
from pypy.module._pickle_support import maker # helper fns
from pypy.interpreter.pycode import PyCode
from pypy.interpreter.module import Module
- args_w = space.unpackiterable(w_args)
+ args_w = space.unpackiterable(w_args, 18)
w_f_back, w_builtin, w_pycode, w_valuestack, w_blockstack, w_exc_value, w_tb,\
w_globals, w_last_instr, w_finished, w_f_lineno, w_fastlocals, w_f_locals, \
w_f_trace, w_instr_lb, w_instr_ub, w_instr_prev_plus_one, w_cells = args_w
diff --git a/pypy/interpreter/pyparser/parsestring.py b/pypy/interpreter/pyparser/parsestring.py
--- a/pypy/interpreter/pyparser/parsestring.py
+++ b/pypy/interpreter/pyparser/parsestring.py
@@ -111,7 +111,7 @@
enc = None
if need_encoding:
- enc = encoding
+ enc = encoding
v = PyString_DecodeEscape(space, substr, enc)
return space.wrap(v)
diff --git a/pypy/interpreter/test/test_function.py b/pypy/interpreter/test/test_function.py
--- a/pypy/interpreter/test/test_function.py
+++ b/pypy/interpreter/test/test_function.py
@@ -636,11 +636,11 @@
assert fn.code.fast_natural_arity == i|PyCode.FLATPYCALL
if i < 5:
- def bomb(*args):
- assert False, "shortcutting should have avoided this"
+ def bomb(*args):
+ assert False, "shortcutting should have avoided this"
- code.funcrun = bomb
- code.funcrun_obj = bomb
+ code.funcrun = bomb
+ code.funcrun_obj = bomb
args_w = map(space.wrap, range(i))
w_res = space.call_function(fn, *args_w)
diff --git a/pypy/interpreter/test/test_gateway.py b/pypy/interpreter/test/test_gateway.py
--- a/pypy/interpreter/test/test_gateway.py
+++ b/pypy/interpreter/test/test_gateway.py
@@ -597,6 +597,32 @@
assert space.is_true(w_res)
assert called == [w_app_f, w_app_f]
+ def test_interp2app_fastcall_method_with_space(self):
+ class W_X(W_Root):
+ def descr_f(self, space, w_x):
+ return w_x
+
+ app_f = gateway.interp2app_temp(W_X.descr_f, unwrap_spec=['self',
+ gateway.ObjSpace, gateway.W_Root])
+
+ w_app_f = self.space.wrap(app_f)
+
+ assert isinstance(w_app_f.code, gateway.BuiltinCode2)
+
+ called = []
+ fastcall_2 = w_app_f.code.fastcall_2
+ def witness_fastcall_2(space, w_func, w_a, w_b):
+ called.append(w_func)
+ return fastcall_2(space, w_func, w_a, w_b)
+
+ w_app_f.code.fastcall_2 = witness_fastcall_2
+ space = self.space
+
+ w_res = space.call_function(w_app_f, W_X(), space.wrap(3))
+
+ assert space.is_true(w_res)
+ assert called == [w_app_f]
+
def test_plain(self):
space = self.space
diff --git a/pypy/interpreter/test/test_nestedscope.py b/pypy/interpreter/test/test_nestedscope.py
--- a/pypy/interpreter/test/test_nestedscope.py
+++ b/pypy/interpreter/test/test_nestedscope.py
@@ -73,7 +73,7 @@
def f():
def f(y):
- return x + y
+ return x + y
return f
x = 1
@@ -85,7 +85,7 @@
if n:
x = 42
def f(y):
- return x + y
+ return x + y
return f
g0 = f(0).func_closure[0]
diff --git a/pypy/interpreter/test/test_zzpickle_and_slow.py b/pypy/interpreter/test/test_zzpickle_and_slow.py
--- a/pypy/interpreter/test/test_zzpickle_and_slow.py
+++ b/pypy/interpreter/test/test_zzpickle_and_slow.py
@@ -226,6 +226,10 @@
restore_top_frame(f1, saved)
f2 = pickle.loads(pckl)
+ def test_frame_setstate_crash(self):
+ import sys
+ raises(ValueError, sys._getframe().__setstate__, [])
+
def test_pickle_traceback(self):
def f():
try:
diff --git a/pypy/module/__builtin__/__init__.py b/pypy/module/__builtin__/__init__.py
--- a/pypy/module/__builtin__/__init__.py
+++ b/pypy/module/__builtin__/__init__.py
@@ -102,26 +102,26 @@
}
def pick_builtin(self, w_globals):
- "Look up the builtin module to use from the __builtins__ global"
- # pick the __builtins__ roughly in the same way CPython does it
- # this is obscure and slow
- space = self.space
- try:
- w_builtin = space.getitem(w_globals, space.wrap('__builtins__'))
- except OperationError, e:
- if not e.match(space, space.w_KeyError):
- raise
- else:
- if w_builtin is space.builtin: # common case
- return space.builtin
- if space.isinstance_w(w_builtin, space.w_dict):
+ "Look up the builtin module to use from the __builtins__ global"
+ # pick the __builtins__ roughly in the same way CPython does it
+ # this is obscure and slow
+ space = self.space
+ try:
+ w_builtin = space.getitem(w_globals, space.wrap('__builtins__'))
+ except OperationError, e:
+ if not e.match(space, space.w_KeyError):
+ raise
+ else:
+ if w_builtin is space.builtin: # common case
+ return space.builtin
+ if space.isinstance_w(w_builtin, space.w_dict):
return module.Module(space, None, w_builtin)
- if isinstance(w_builtin, module.Module):
- return w_builtin
- # no builtin! make a default one. Give them None, at least.
- builtin = module.Module(space, None)
- space.setitem(builtin.w_dict, space.wrap('None'), space.w_None)
- return builtin
+ if isinstance(w_builtin, module.Module):
+ return w_builtin
+ # no builtin! make a default one. Give them None, at least.
+ builtin = module.Module(space, None)
+ space.setitem(builtin.w_dict, space.wrap('None'), space.w_None)
+ return builtin
def setup_after_space_initialization(self):
"""NOT_RPYTHON"""
diff --git a/pypy/module/_minimal_curses/fficurses.py b/pypy/module/_minimal_curses/fficurses.py
--- a/pypy/module/_minimal_curses/fficurses.py
+++ b/pypy/module/_minimal_curses/fficurses.py
@@ -52,7 +52,8 @@
HAS = rffi_platform.Has("setupterm")
if rffi_platform.configure(CConfig)['HAS']:
return eci
- raise ImportError("failed to guess where ncurses is installed")
+ raise ImportError("failed to guess where ncurses is installed. "
+ "You might need to install libncurses5-dev or similar.")
eci = guess_eci()
diff --git a/pypy/module/_pypyjson/__init__.py b/pypy/module/_pypyjson/__init__.py
new file mode 100644
--- /dev/null
+++ b/pypy/module/_pypyjson/__init__.py
@@ -0,0 +1,10 @@
+from pypy.interpreter.mixedmodule import MixedModule
+
+class Module(MixedModule):
+ """fast json implementation"""
+
+ appleveldefs = {}
+
+ interpleveldefs = {
+ 'loads' : 'interp_decoder.loads',
+ }
diff --git a/pypy/module/_pypyjson/interp_decoder.py b/pypy/module/_pypyjson/interp_decoder.py
new file mode 100644
--- /dev/null
+++ b/pypy/module/_pypyjson/interp_decoder.py
@@ -0,0 +1,404 @@
+import sys
+import math
+from rpython.rlib.rstring import StringBuilder
+from rpython.rlib.objectmodel import specialize
+from rpython.rlib import rfloat
+from rpython.rtyper.lltypesystem import lltype, rffi
+from pypy.interpreter.error import OperationError, operationerrfmt
+from pypy.interpreter.gateway import unwrap_spec
+from pypy.interpreter import unicodehelper
+from rpython.rtyper.annlowlevel import llstr, hlunicode
+
+OVF_DIGITS = len(str(sys.maxint))
+
+def is_whitespace(ch):
+ return ch == ' ' or ch == '\t' or ch == '\r' or ch == '\n'
+
+# precomputing negative powers of 10 is MUCH faster than using e.g. math.pow
+# at runtime
+NEG_POW_10 = [10.0**-i for i in range(16)]
+def neg_pow_10(x, exp):
+ if exp >= len(NEG_POW_10):
+ return 0.0
+ return x * NEG_POW_10[exp]
+
+def strslice2unicode_latin1(s, start, end):
+ """
+ Convert s[start:end] to unicode. s is supposed to be an RPython string
+ encoded in latin-1, which means that the numeric value of each char is the
+ same as the corresponding unicode code point.
+
+ Internally it's implemented at the level of low-level helpers, to avoid
+ the extra copy we would need if we take the actual slice first.
+
+ No bound checking is done, use carefully.
+ """
+ from rpython.rtyper.annlowlevel import llstr, hlunicode
+ from rpython.rtyper.lltypesystem.rstr import malloc, UNICODE
+ from rpython.rtyper.lltypesystem.lltype import cast_primitive, UniChar
+ length = end-start
+ ll_s = llstr(s)
+ ll_res = malloc(UNICODE, length)
+ ll_res.hash = 0
+ for i in range(length):
+ ch = ll_s.chars[start+i]
+ ll_res.chars[i] = cast_primitive(UniChar, ch)
+ return hlunicode(ll_res)
+
+TYPE_UNKNOWN = 0
+TYPE_STRING = 1
+class JSONDecoder(object):
+ def __init__(self, space, s):
+ self.space = space
+ self.s = s
+ # we put our string in a raw buffer so:
+ # 1) we automatically get the '\0' sentinel at the end of the string,
+ # which means that we never have to check for the "end of string"
+ # 2) we can pass the buffer directly to strtod
+ self.ll_chars = rffi.str2charp(s)
+ self.end_ptr = lltype.malloc(rffi.CCHARPP.TO, 1, flavor='raw')
+ self.pos = 0
+ self.last_type = TYPE_UNKNOWN
+
+ def close(self):
+ rffi.free_charp(self.ll_chars)
+ lltype.free(self.end_ptr, flavor='raw')
+
+ def getslice(self, start, end):
+ assert start >= 0
+ assert end >= 0
+ return self.s[start:end]
+
+ def skip_whitespace(self, i):
+ while True:
+ ch = self.ll_chars[i]
+ if is_whitespace(ch):
+ i+=1
+ else:
+ break
+ return i
+
+ @specialize.arg(1)
+ def _raise(self, msg, *args):
+ raise operationerrfmt(self.space.w_ValueError, msg, *args)
+
+ def decode_any(self, i):
+ i = self.skip_whitespace(i)
+ ch = self.ll_chars[i]
+ if ch == '"':
+ return self.decode_string(i+1)
+ elif ch == '[':
+ return self.decode_array(i+1)
+ elif ch == '{':
+ return self.decode_object(i+1)
+ elif ch == 'n':
+ return self.decode_null(i+1)
+ elif ch == 't':
+ return self.decode_true(i+1)
+ elif ch == 'f':
+ return self.decode_false(i+1)
+ elif ch == 'I':
+ return self.decode_infinity(i+1)
+ elif ch == 'N':
+ return self.decode_nan(i+1)
+ elif ch == '-':
+ if self.ll_chars[i+1] == 'I':
+ return self.decode_infinity(i+2, sign=-1)
+ return self.decode_numeric(i)
+ elif ch.isdigit():
+ return self.decode_numeric(i)
+ else:
+ self._raise("No JSON object could be decoded: unexpected '%s' at char %d",
+ ch, self.pos)
+
+ def decode_null(self, i):
+ if (self.ll_chars[i] == 'u' and
+ self.ll_chars[i+1] == 'l' and
+ self.ll_chars[i+2] == 'l'):
+ self.pos = i+3
+ return self.space.w_None
+ self._raise("Error when decoding null at char %d", i)
+
+ def decode_true(self, i):
+ if (self.ll_chars[i] == 'r' and
+ self.ll_chars[i+1] == 'u' and
+ self.ll_chars[i+2] == 'e'):
+ self.pos = i+3
+ return self.space.w_True
+ self._raise("Error when decoding true at char %d", i)
+
+ def decode_false(self, i):
+ if (self.ll_chars[i] == 'a' and
+ self.ll_chars[i+1] == 'l' and
+ self.ll_chars[i+2] == 's' and
+ self.ll_chars[i+3] == 'e'):
+ self.pos = i+4
+ return self.space.w_False
+ self._raise("Error when decoding false at char %d", i)
+
+ def decode_infinity(self, i, sign=1):
+ if (self.ll_chars[i] == 'n' and
+ self.ll_chars[i+1] == 'f' and
+ self.ll_chars[i+2] == 'i' and
+ self.ll_chars[i+3] == 'n' and
+ self.ll_chars[i+4] == 'i' and
+ self.ll_chars[i+5] == 't' and
+ self.ll_chars[i+6] == 'y'):
+ self.pos = i+7
+ return self.space.wrap(rfloat.INFINITY * sign)
+ self._raise("Error when decoding Infinity at char %d", i)
+
+ def decode_nan(self, i):
+ if (self.ll_chars[i] == 'a' and
+ self.ll_chars[i+1] == 'N'):
+ self.pos = i+2
+ return self.space.wrap(rfloat.NAN)
+ self._raise("Error when decoding NaN at char %d", i)
+
+ def decode_numeric(self, i):
+ start = i
+ i, ovf_maybe, intval = self.parse_integer(i)
+ #
+ # check for the optional fractional part
+ ch = self.ll_chars[i]
+ if ch == '.':
+ if not self.ll_chars[i+1].isdigit():
+ self._raise("Expected digit at char %d", i+1)
+ return self.decode_float(start)
+ elif ch == 'e' or ch == 'E':
+ return self.decode_float(start)
+ elif ovf_maybe:
+ return self.decode_int_slow(start)
+
+ self.pos = i
+ return self.space.wrap(intval)
+
+ def decode_float(self, i):
+ from rpython.rlib import rdtoa
+ start = rffi.ptradd(self.ll_chars, i)
+ floatval = rdtoa.dg_strtod(start, self.end_ptr)
+ diff = rffi.cast(rffi.LONG, self.end_ptr[0]) - rffi.cast(rffi.LONG, start)
+ self.pos = i + diff
+ return self.space.wrap(floatval)
+
+ def decode_int_slow(self, i):
+ start = i
+ if self.ll_chars[i] == '-':
+ i += 1
+ while self.ll_chars[i].isdigit():
+ i += 1
+ s = self.getslice(start, i)
+ self.pos = i
+ return self.space.call_function(self.space.w_int, self.space.wrap(s))
+
+ def parse_integer(self, i):
+ "Parse a decimal number with an optional minus sign"
+ sign = 1
+ # parse the sign
+ if self.ll_chars[i] == '-':
+ sign = -1
+ i += 1
+ elif self.ll_chars[i] == '+':
+ i += 1
+ #
+ if self.ll_chars[i] == '0':
+ i += 1
+ return i, False, 0
+
+ intval = 0
+ start = i
+ while True:
+ ch = self.ll_chars[i]
+ if ch.isdigit():
+ intval = intval*10 + ord(ch)-ord('0')
+ i += 1
+ else:
+ break
+ count = i - start
+ if count == 0:
+ self._raise("Expected digit at char %d", i)
+ # if the number has more digits than OVF_DIGITS, it might have
+ # overflowed
+ ovf_maybe = (count >= OVF_DIGITS)
+ return i, ovf_maybe, sign * intval
+ parse_integer._always_inline_ = True
+
+ def decode_array(self, i):
+ w_list = self.space.newlist([])
+ start = i
+ count = 0
+ i = self.skip_whitespace(start)
+ if self.ll_chars[i] == ']':
+ self.pos = i+1
+ return w_list
+ #
+ while True:
+ w_item = self.decode_any(i)
+ i = self.pos
+ self.space.call_method(w_list, 'append', w_item)
+ i = self.skip_whitespace(i)
+ ch = self.ll_chars[i]
+ i += 1
+ if ch == ']':
+ self.pos = i
+ return w_list
+ elif ch == ',':
+ pass
+ elif ch == '\0':
+ self._raise("Unterminated array starting at char %d", start)
+ else:
+ self._raise("Unexpected '%s' when decoding array (char %d)",
+ ch, self.pos)
+
+ def decode_object(self, i):
+ start = i
+ w_dict = self.space.newdict()
+ #
+ i = self.skip_whitespace(i)
+ if self.ll_chars[i] == '}':
+ self.pos = i+1
+ return w_dict
+ #
+ while True:
+ # parse a key: value
+ self.last_type = TYPE_UNKNOWN
+ w_name = self.decode_any(i)
+ if self.last_type != TYPE_STRING:
+ self._raise("Key name must be string for object starting at char %d", start)
+ i = self.skip_whitespace(self.pos)
+ ch = self.ll_chars[i]
+ if ch != ':':
+ self._raise("No ':' found at char %d", i)
+ i += 1
+ i = self.skip_whitespace(i)
+ #
+ w_value = self.decode_any(i)
+ self.space.setitem(w_dict, w_name, w_value)
+ i = self.skip_whitespace(self.pos)
+ ch = self.ll_chars[i]
+ i += 1
+ if ch == '}':
+ self.pos = i
+ return w_dict
+ elif ch == ',':
+ pass
+ elif ch == '\0':
+ self._raise("Unterminated object starting at char %d", start)
+ else:
+ self._raise("Unexpected '%s' when decoding object (char %d)",
+ ch, self.pos)
+
+
+ def decode_string(self, i):
+ start = i
+ bits = 0
+ while True:
+ # this loop is a fast path for strings which do not contain escape
+ # characters
+ ch = self.ll_chars[i]
+ i += 1
+ bits |= ord(ch)
+ if ch == '"':
+ if bits & 0x80:
+ # the 8th bit is set, it's an utf8 strnig
+ content_utf8 = self.getslice(start, i-1)
+ content_unicode = unicodehelper.decode_utf8(self.space, content_utf8)
+ else:
+ # ascii only, fast path (ascii is a strict subset of
+ # latin1, and we already checked that all the chars are <
+ # 128)
+ content_unicode = strslice2unicode_latin1(self.s, start, i-1)
+ self.last_type = TYPE_STRING
+ self.pos = i
+ return self.space.wrap(content_unicode)
+ elif ch == '\\':
+ content_so_far = self.getslice(start, i-1)
+ self.pos = i-1
+ return self.decode_string_escaped(start, content_so_far)
+ elif ch == '\0':
+ self._raise("Unterminated string starting at char %d", start)
+
+
+ def decode_string_escaped(self, start, content_so_far):
+ builder = StringBuilder(len(content_so_far)*2) # just an estimate
+ builder.append(content_so_far)
+ i = self.pos
+ while True:
+ ch = self.ll_chars[i]
+ i += 1
+ if ch == '"':
+ content_utf8 = builder.build()
+ content_unicode = unicodehelper.decode_utf8(self.space, content_utf8)
+ self.last_type = TYPE_STRING
+ self.pos = i
+ return self.space.wrap(content_unicode)
+ elif ch == '\\':
+ i = self.decode_escape_sequence(i, builder)
+ elif ch == '\0':
+ self._raise("Unterminated string starting at char %d", start)
+ else:
+ builder.append_multiple_char(ch, 1) # we should implement append_char
+
+ def decode_escape_sequence(self, i, builder):
+ ch = self.ll_chars[i]
+ i += 1
+ put = builder.append_multiple_char
+ if ch == '\\': put('\\', 1)
+ elif ch == '"': put('"' , 1)
+ elif ch == '/': put('/' , 1)
+ elif ch == 'b': put('\b', 1)
+ elif ch == 'f': put('\f', 1)
+ elif ch == 'n': put('\n', 1)
+ elif ch == 'r': put('\r', 1)
+ elif ch == 't': put('\t', 1)
+ elif ch == 'u':
+ return self.decode_escape_sequence_unicode(i, builder)
+ else:
+ self._raise("Invalid \\escape: %s (char %d)", ch, self.pos-1)
+ return i
+
+ def decode_escape_sequence_unicode(self, i, builder):
+ # at this point we are just after the 'u' of the \u1234 sequence.
+ start = i
+ i += 4
+ hexdigits = self.getslice(start, i)
+ try:
+ val = int(hexdigits, 16)
+ if val & 0xfc00 == 0xd800:
+ # surrogate pair
+ val = self.decode_surrogate_pair(i, val)
+ i += 6
+ except ValueError:
+ self._raise("Invalid \uXXXX escape (char %d)", i-1)
+ return # help the annotator to know that we'll never go beyond
+ # this point
+ #
+ uchr = unichr(val)
+ utf8_ch = unicodehelper.encode_utf8(self.space, uchr)
+ builder.append(utf8_ch)
+ return i
+
+ def decode_surrogate_pair(self, i, highsurr):
+ if self.ll_chars[i] != '\\' or self.ll_chars[i+1] != 'u':
+ self._raise("Unpaired high surrogate at char %d", i)
+ i += 2
+ hexdigits = self.getslice(i, i+4)
+ lowsurr = int(hexdigits, 16) # the possible ValueError is caugth by the caller
+ return 0x10000 + (((highsurr - 0xd800) << 10) | (lowsurr - 0xdc00))
+
+def loads(space, w_s):
+ if space.isinstance_w(w_s, space.w_unicode):
+ raise OperationError(space.w_TypeError,
+ space.wrap("Expected utf8-encoded str, got unicode"))
+ s = space.str_w(w_s)
+ decoder = JSONDecoder(space, s)
+ try:
+ w_res = decoder.decode_any(0)
+ i = decoder.skip_whitespace(decoder.pos)
+ if i < len(s):
+ start = i
+ end = len(s) - 1
+ raise operationerrfmt(space.w_ValueError, "Extra data: char %d - %d", start, end)
+ return w_res
+ finally:
+ decoder.close()
diff --git a/pypy/module/_pypyjson/targetjson.py b/pypy/module/_pypyjson/targetjson.py
new file mode 100644
--- /dev/null
+++ b/pypy/module/_pypyjson/targetjson.py
@@ -0,0 +1,143 @@
+import sys
+import py
+ROOT = py.path.local(__file__).dirpath('..', '..', '..')
+sys.path.insert(0, str(ROOT))
+
+import time
+from rpython.rlib.streamio import open_file_as_stream
+from pypy.interpreter.error import OperationError
+from pypy.module._pypyjson.interp_decoder import loads
+
+
+
+## MSG = open('msg.json').read()
+
+class W_Root(object):
+ pass
+
+class W_Dict(W_Root):
+ def __init__(self):
+ self.dictval = {}
+
+class W_Unicode(W_Root):
+ def __init__(self, x):
+ self.unival = x
+
+class W_String(W_Root):
+ def __init__(self, x):
+ self.strval = x
+
+class W_Int(W_Root):
+ def __init__(self, x):
+ self.intval = x
+
+class W_Float(W_Root):
+ def __init__(self, x):
+ self.floatval = x
+
+class W_List(W_Root):
+ def __init__(self):
+ self.listval = []
+
+class W_Singleton(W_Root):
+ def __init__(self, name):
+ self.name = name
+
+class FakeSpace(object):
+
+ w_None = W_Singleton('None')
+ w_True = W_Singleton('True')
+ w_False = W_Singleton('False')
+ w_ValueError = W_Singleton('ValueError')
+ w_UnicodeDecodeError = W_Singleton('UnicodeDecodeError')
+ w_unicode = W_Unicode
+ w_int = W_Int
+ w_float = W_Float
+
+ def newtuple(self, items):
+ return None
+
+ def newdict(self):
+ return W_Dict()
+
+ def newlist(self, items):
+ return W_List()
+
+ def isinstance_w(self, w_x, w_type):
+ return isinstance(w_x, w_type)
+
+ def str_w(self, w_x):
+ assert isinstance(w_x, W_String)
+ return w_x.strval
+
+ def call_method(self, obj, name, arg):
+ assert name == 'append'
+ assert isinstance(obj, W_List)
+ obj.listval.append(arg)
+ call_method._dont_inline_ = True
+
+ def call_function(self, w_func, *args_w):
+ return self.w_None # XXX
+
+ def setitem(self, d, key, value):
+ assert isinstance(d, W_Dict)
+ assert isinstance(key, W_Unicode)
+ d.dictval[key.unival] = value
+
+ def wrapunicode(self, x):
More information about the pypy-commit
mailing list