From asouzaleite at gmx.de  Fri Mar  2 09:13:54 2012
From: asouzaleite at gmx.de (Aroldo Souza-Leite)
Date: Fri, 02 Mar 2012 09:13:54 +0100
Subject: [pypy-dev] pip
In-Reply-To: <4F4EA04A.1040000@gmail.com>
References: <36256051.20120229181544@gmail.com> <4F4E66B1.8020205@gmx.de>
	<CAFRnB2UQyj+=PEowGmOtkj5T8cuq3g0L8p0-eE8JMpKwCnoUyA@mail.gmail.com>
	<4F4EA04A.1040000@gmail.com>
Message-ID: <4F508142.5070101@gmx.de>

Hi list,

the error disappeared in pypy-c-jit-53087-2881f17e1ffc-linuxand and I 
can work on with it.

As there are still warnings, I'm appending the console output below for 
your information.

Thanks a lot again.

Aroldo

-------
(pypy)root at aroldo-laptop:/opt/pypy/bin# which easy_install
/opt/pypy//bin/easy_install
(pypy)root at aroldo-laptop:/opt/pypy/bin# easy_install pip
Searching for pip
Reading http://pypi.python.org/simple/pip/
Reading http://pip.openplans.org
Reading http://www.pip-installer.org
Best match: pip 1.1
Downloading 
http://pypi.python.org/packages/source/p/pip/pip-1.1.tar.gz#md5=62a9
f08dd5dc69d76734568a6c040508
Processing pip-1.1.tar.gz
Running pip-1.1/setup.py -q bdist_egg --dist-dir 
/tmp/easy_install-1Ognh8/pip-1.
1/egg-dist-tmp-5aNrw9
warning: no files found matching '*.html' under directory 'docs'
warning: no previously-included files matching '*.txt' found under 
directory 'do
cs/_build'
no previously-included directories found matching 'docs/_build/_sources'
Adding pip 1.1 to easy-install.pth file
Installing pip script to /opt/pypy/bin
Installing pip-2.7 script to /opt/pypy/bin

Installed 
/opt/pypy-c-jit-53087-2881f17e1ffc-linux/site-packages/pip-1.1-py2.7.e
gg
Processing dependencies for pip
Finished processing dependencies for pip
(pypy)root at aroldo-laptop:/opt/pypy/bin#




Am 29.02.2012 23:01, schrieb Antonio Cuni:
> On 02/29/2012 06:59 PM, Alex Gaynor wrote:
>
>> I ran into this recently and just ignored it, it seemed to work ok.  
>> However,
>> I'm guessing it's a result of d4dee87e47cc.
>
> it's, it's probably because of the autoflush. It should be fixed by 
> f9f3b57f1300 which I just pushed, so tomorrow's nightly should already 
> work well.
>
> ciao,
> anto


From asouzaleite at gmx.de  Fri Mar  2 11:03:53 2012
From: asouzaleite at gmx.de (Aroldo Souza-Leite)
Date: Fri, 02 Mar 2012 11:03:53 +0100
Subject: [pypy-dev] ZODB3
In-Reply-To: <CAMSv6X1UBek-h1TiupYnFTpoZf2ts5mKzHgqCZgKXfw1rXDx-Q@mail.gmail.com>
References: <jhg54k$dte$1@dough.gmane.org> <jhnok5$ve2$1@dough.gmane.org>
	<CAGmFidbutXahNAKbHgaLFOM39uTKZ8C3TkrPLeauB0Jm5CQ-bg@mail.gmail.com>
	<jho9he$c2k$1@dough.gmane.org>
	<CAGmFidZ2DDxnPBi7E5Bo8PTn3iubjfrtxmBv1uEsaombo2ey1Q@mail.gmail.com>
	<jhobh9$ots$1@dough.gmane.org>
	<CAGmFidbEPpbQPrKEsNHkMQ1e3mgsYV1vd0SivdVPvSOYaLSXmA@mail.gmail.com>
	<jie2mh$qub$1@dough.gmane.org> <4F4B5296.6050903@gmx.de>
	<CAK5idxRiqt0ga5JjWz0j1NbkrAbTvJmQvvNSJdxgqJrXs3j0dQ@mail.gmail.com>
	<CAMSv6X1RF6pNuOHFP0OKM7bUpsbtPckazFOShnD_vmQX9m71Vw@mail.gmail.com>
	<4F4C99D7.3050407@gmx.de>
	<CAMSv6X19hg7gPcyiV0bs8jWCWonjcdJkduYNVc6fTWTf7d7WcQ@mail.gmail.com>
	<4F4CA4A3.40100@gmx.de>
	<CAMSv6X1UBek-h1TiupYnFTpoZf2ts5mKzHgqCZgKXfw1rXDx-Q@mail.gmail.com>
Message-ID: <4F509B09.10703@gmx.de>

Hi list,

now ZODB3 and zope.testing can be pip-installed in a PyPy virtual 
environment and the tests can be called. But the tests fail.

The same tests pass in CPython2.7.

Could somebody please make a short comment on this before I eventually 
forward the case to the ZODB department?

The attachment contains the complete bash session in Sphinx format with 
"Test results" as its last subsection. This is  probably the only 
interesting part for developers, so that I'm reproducing it below.

Thanks and cheers.

Aroldo.

-------------------<Test results>

(tmp-env-pypy)aroldo at aroldo-laptop:~$python tmp-env-pypy/site-packages/ZODB/te
sts/testZODB.py


output:

EEEEEEEEEE

======================================================================

ERROR: checkExplicitTransactionManager (__main__.ZODBTests)

----------------------------------------------------------------------

Traceback (most recent call last):
   File "tmp-env-pypy/site-packages/ZODB/tests/testZODB.py", line 159, in checkE
xplicitTransactionManager
   File "/home/aroldo/tmp-env-pypy/lib-python/modified-2.7/UserDict.py", line 50
, in has_key
     def has_key(self, key): return key in self.data
   File "/home/aroldo/tmp-env-pypy/site-packages/persistent/mapping.py", line 30
, in __get__
     return self.func(inst)
   File "/home/aroldo/tmp-env-pypy/site-packages/persistent/mapping.py", line 99
, in data
     data = self.__dict__.pop('_container')
KeyError: '_container'

======================================================================

ERROR: checkExportImport (__main__.ZODBTests)

----------------------------------------------------------------------

Traceback (most recent call last):
   File "tmp-env-pypy/site-packages/ZODB/tests/testZODB.py", line 54, in checkEx
portImport
   File "tmp-env-pypy/site-packages/ZODB/tests/testZODB.py", line 46, in populat
e
   File "/home/aroldo/tmp-env-pypy/site-packages/persistent/mapping.py", line 63
, in __setitem__
     self.__super_setitem(key, v)
   File "/home/aroldo/tmp-env-pypy/lib-python/modified-2.7/UserDict.py", line 29
, in __setitem__
     def __setitem__(self, key, item): self.data[key] = item
   File "/home/aroldo/tmp-env-pypy/site-packages/persistent/mapping.py", line 30
, in __get__
     return self.func(inst)
   File "/home/aroldo/tmp-env-pypy/site-packages/persistent/mapping.py", line 99
, in data
     data = self.__dict__.pop('_container')
KeyError: '_container'

======================================================================

ERROR: checkExportImportAborted (__main__.ZODBTests)

----------------------------------------------------------------------

Traceback (most recent call last):
   File "tmp-env-pypy/site-packages/ZODB/tests/testZODB.py", line 122, in checkE
xportImportAborted
   File "tmp-env-pypy/site-packages/ZODB/tests/testZODB.py", line 54, in checkEx
portImport
   File "tmp-env-pypy/site-packages/ZODB/tests/testZODB.py", line 46, in populat
e
   File "/home/aroldo/tmp-env-pypy/site-packages/persistent/mapping.py", line 63
, in __setitem__
     self.__super_setitem(key, v)
   File "/home/aroldo/tmp-env-pypy/lib-python/modified-2.7/UserDict.py", line 29
, in __setitem__
     def __setitem__(self, key, item): self.data[key] = item
   File "/home/aroldo/tmp-env-pypy/site-packages/persistent/mapping.py", line 30
, in __get__
     return self.func(inst)
   File "/home/aroldo/tmp-env-pypy/site-packages/persistent/mapping.py", line 99
, in data
     data = self.__dict__.pop('_container')
KeyError: '_container'

======================================================================

ERROR: checkFailingCommitSticks (__main__.ZODBTests)

----------------------------------------------------------------------

Traceback (most recent call last):
   File "tmp-env-pypy/site-packages/ZODB/tests/testZODB.py", line 289, in checkF
ailingCommitSticks
   File "/home/aroldo/tmp-env-pypy/site-packages/persistent/mapping.py", line 63
, in __setitem__
     self.__super_setitem(key, v)
   File "/home/aroldo/tmp-env-pypy/lib-python/modified-2.7/UserDict.py", line 29
, in __setitem__
     def __setitem__(self, key, item): self.data[key] = item
   File "/home/aroldo/tmp-env-pypy/site-packages/persistent/mapping.py", line 30
, in __get__
     return self.func(inst)
   File "/home/aroldo/tmp-env-pypy/site-packages/persistent/mapping.py", line 99
, in data
     data = self.__dict__.pop('_container')
KeyError: '_container'

======================================================================

ERROR: checkFailingSavepointSticks (__main__.ZODBTests)

----------------------------------------------------------------------

Traceback (most recent call last):
   File "tmp-env-pypy/site-packages/ZODB/tests/testZODB.py", line 334, in checkF
ailingSavepointSticks
   File "/home/aroldo/tmp-env-pypy/site-packages/persistent/mapping.py", line 63
, in __setitem__
     self.__super_setitem(key, v)
   File "/home/aroldo/tmp-env-pypy/lib-python/modified-2.7/UserDict.py", line 29
, in __setitem__
     def __setitem__(self, key, item): self.data[key] = item
   File "/home/aroldo/tmp-env-pypy/site-packages/persistent/mapping.py", line 30
, in __get__
     return self.func(inst)
   File "/home/aroldo/tmp-env-pypy/site-packages/persistent/mapping.py", line 99
, in data
     data = self.__dict__.pop('_container')
KeyError: '_container'

======================================================================

ERROR: checkMultipleUndoInOneTransaction (__main__.ZODBTests)

----------------------------------------------------------------------

Traceback (most recent call last):
   File "tmp-env-pypy/site-packages/ZODB/tests/testZODB.py", line 403, in checkM
ultipleUndoInOneTransaction
   File "/home/aroldo/tmp-env-pypy/site-packages/persistent/mapping.py", line 63
, in __setitem__
     self.__super_setitem(key, v)
   File "/home/aroldo/tmp-env-pypy/lib-python/modified-2.7/UserDict.py", line 29
, in __setitem__
     def __setitem__(self, key, item): self.data[key] = item
   File "/home/aroldo/tmp-env-pypy/site-packages/persistent/mapping.py", line 30
, in __get__
     return self.func(inst)
   File "/home/aroldo/tmp-env-pypy/site-packages/persistent/mapping.py", line 99
, in data
     data = self.__dict__.pop('_container')
KeyError: '_container'

======================================================================

ERROR: checkResetCache (__main__.ZODBTests)

----------------------------------------------------------------------

Traceback (most recent call last):
   File "tmp-env-pypy/site-packages/ZODB/tests/testZODB.py", line 129, in checkR
esetCache
   File "tmp-env-pypy/site-packages/ZODB/tests/testZODB.py", line 46, in populat
e
   File "/home/aroldo/tmp-env-pypy/site-packages/persistent/mapping.py", line 63
, in __setitem__
     self.__super_setitem(key, v)
   File "/home/aroldo/tmp-env-pypy/lib-python/modified-2.7/UserDict.py", line 29
, in __setitem__
     def __setitem__(self, key, item): self.data[key] = item
   File "/home/aroldo/tmp-env-pypy/site-packages/persistent/mapping.py", line 30
, in __get__
     return self.func(inst)
   File "/home/aroldo/tmp-env-pypy/site-packages/persistent/mapping.py", line 99
, in data
     data = self.__dict__.pop('_container')
KeyError: '_container'

======================================================================

ERROR: checkResetCachesAPI (__main__.ZODBTests)

----------------------------------------------------------------------

Traceback (most recent call last):
   File "tmp-env-pypy/site-packages/ZODB/tests/testZODB.py", line 139, in checkR
esetCachesAPI
   File "tmp-env-pypy/site-packages/ZODB/tests/testZODB.py", line 46, in populat
e
   File "/home/aroldo/tmp-env-pypy/site-packages/persistent/mapping.py", line 63
, in __setitem__
     self.__super_setitem(key, v)
   File "/home/aroldo/tmp-env-pypy/lib-python/modified-2.7/UserDict.py", line 29
, in __setitem__
     def __setitem__(self, key, item): self.data[key] = item
   File "/home/aroldo/tmp-env-pypy/site-packages/persistent/mapping.py", line 30
, in __get__
     return self.func(inst)
   File "/home/aroldo/tmp-env-pypy/site-packages/persistent/mapping.py", line 99
, in data
     data = self.__dict__.pop('_container')
KeyError: '_container'

======================================================================

ERROR: checkSavepointDoesntGetInvalidations (__main__.ZODBTests)

----------------------------------------------------------------------

Traceback (most recent call last):
   File "tmp-env-pypy/site-packages/ZODB/tests/testZODB.py", line 204, in checkS
avepointDoesntGetInvalidations
   File "/home/aroldo/tmp-env-pypy/site-packages/persistent/mapping.py", line 63
, in __setitem__
     self.__super_setitem(key, v)
   File "/home/aroldo/tmp-env-pypy/lib-python/modified-2.7/UserDict.py", line 29
, in __setitem__
     def __setitem__(self, key, item): self.data[key] = item
   File "/home/aroldo/tmp-env-pypy/site-packages/persistent/mapping.py", line 30
, in __get__
     return self.func(inst)
   File "/home/aroldo/tmp-env-pypy/site-packages/persistent/mapping.py", line 99
, in data
     data = self.__dict__.pop('_container')
KeyError: '_container'

======================================================================

ERROR: checkTxnBeginImpliesAbort (__main__.ZODBTests)

----------------------------------------------------------------------

Traceback (most recent call last):
   File "tmp-env-pypy/site-packages/ZODB/tests/testZODB.py", line 257, in checkT
xnBeginImpliesAbort
   File "/home/aroldo/tmp-env-pypy/site-packages/persistent/mapping.py", line 63
, in __setitem__
     self.__super_setitem(key, v)
   File "/home/aroldo/tmp-env-pypy/lib-python/modified-2.7/UserDict.py", line 29
, in __setitem__
     def __setitem__(self, key, item): self.data[key] = item
   File "/home/aroldo/tmp-env-pypy/site-packages/persistent/mapping.py", line 30
, in __get__
     return self.func(inst)
   File "/home/aroldo/tmp-env-pypy/site-packages/persistent/mapping.py", line 99
, in data
     data = self.__dict__.pop('_container')
KeyError: '_container'

----------------------------------------------------------------------

Ran 10 tests in 0.439s

FAILED (errors=10)


-----</Test results>




-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: pypy-zodb3.txt
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120302/0edbb629/attachment-0001.txt>

From eh2406 at wayne.edu  Sun Mar  4 03:12:07 2012
From: eh2406 at wayne.edu (Jacob Finkelman)
Date: Sat, 3 Mar 2012 21:12:07 -0500
Subject: [pypy-dev] How to get PIL to install?
Message-ID: <CAGBU8t6MDthpmuY2arM6_UM6Xm8DXiWkZ4mc61a_C+DfRx6=CA@mail.gmail.com>

Hi,

I have been lurking on this list for quite a while. I always feel
smarter for having tried to follow the conversations on this list. I'd
especially like to thank the people working on getting Cython working.
The progress on that project has been fascinating to read about.

I have been trying to get PIL to work on my Windows 7 32 bit laptop.

I install Pypy. Currently 1.8
Install distribute and PIP.
Install "Microsoft Visual C++ 2008 Express Edition"
run "cmd.exe"
cd c:\pypy-1.8\bin
pip install pil

and I get a long error the end of which is """

tails.
            C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\io.h(322) :
see declaration of 'write'
    C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe /c /nologo /Ox /M
D /W3 /GS- /DNDEBUG -IlibImaging -IC:\pypy-1.8\include /Tcmap.c /Fobuild\temp.wi
n32-2.7\Release\map.obj
    map.c
    C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(76) : warnin
g C4114: same type qualifier used more than once
    C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(76) : error
C2632: 'char' followed by 'char' is illegal
    C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(76) : error
C2059: syntax error : ','
    C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(77) : error
C2632: 'short' followed by 'short' is illegal
    C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(77) : error
C2059: syntax error : ','
    C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(80) : warnin
g C4114: same type qualifier used more than once
    C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(80) : error
C2632: 'char' followed by 'char' is illegal
    C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(80) : error
C2059: syntax error : ','
    C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(81) : warnin
g C4114: same type qualifier used more than once
    C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(81) : error
C2632: 'short' followed by 'short' is illegal
    C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(81) : error
C2059: syntax error : ','
    error: command 'C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe'
failed with exit status 2
    Complete output from command C:\pypy-1.8\pypy.exe -c "import setuptools;__fi
le__='c:\\pypy-1.8\\bin\\build\\pil\\setup.py';exec(compile(open(__file__).read(
).replace('\r\n', '\n'), __file__, 'exec'))" install --single-version-externally
-managed --record c:\users\jlap\appdata\local\temp\pip-y_8euy-record\install-rec
ord.txt:
    WARNING: '' not a valid package name; please use only.-separated package nam
es in setup.py

running install

running build

running build_py

copying PIL\ArgImagePlugin.py -> build\lib.win32-2.7

copying PIL\BdfFontFile.py -> build\lib.win32-2.7

copying PIL\BmpImagePlugin.py -> build\lib.win32-2.7

copying PIL\BufrStubImagePlugin.py -> build\lib.win32-2.7

copying PIL\ContainerIO.py -> build\lib.win32-2.7

copying PIL\CurImagePlugin.py -> build\lib.win32-2.7

copying PIL\DcxImagePlugin.py -> build\lib.win32-2.7

copying PIL\EpsImagePlugin.py -> build\lib.win32-2.7

copying PIL\ExifTags.py -> build\lib.win32-2.7

copying PIL\FitsStubImagePlugin.py -> build\lib.win32-2.7

copying PIL\FliImagePlugin.py -> build\lib.win32-2.7

copying PIL\FontFile.py -> build\lib.win32-2.7

copying PIL\FpxImagePlugin.py -> build\lib.win32-2.7

copying PIL\GbrImagePlugin.py -> build\lib.win32-2.7

copying PIL\GdImageFile.py -> build\lib.win32-2.7

copying PIL\GifImagePlugin.py -> build\lib.win32-2.7

copying PIL\GimpGradientFile.py -> build\lib.win32-2.7

copying PIL\GimpPaletteFile.py -> build\lib.win32-2.7

copying PIL\GribStubImagePlugin.py -> build\lib.win32-2.7

copying PIL\Hdf5StubImagePlugin.py -> build\lib.win32-2.7

copying PIL\IcnsImagePlugin.py -> build\lib.win32-2.7

copying PIL\IcoImagePlugin.py -> build\lib.win32-2.7

copying PIL\Image.py -> build\lib.win32-2.7

copying PIL\ImageChops.py -> build\lib.win32-2.7

copying PIL\ImageCms.py -> build\lib.win32-2.7

copying PIL\ImageColor.py -> build\lib.win32-2.7

copying PIL\ImageDraw.py -> build\lib.win32-2.7

copying PIL\ImageDraw2.py -> build\lib.win32-2.7

copying PIL\ImageEnhance.py -> build\lib.win32-2.7

copying PIL\ImageFile.py -> build\lib.win32-2.7

copying PIL\ImageFileIO.py -> build\lib.win32-2.7

copying PIL\ImageFilter.py -> build\lib.win32-2.7

copying PIL\ImageFont.py -> build\lib.win32-2.7

copying PIL\ImageGL.py -> build\lib.win32-2.7

copying PIL\ImageGrab.py -> build\lib.win32-2.7

copying PIL\ImageMath.py -> build\lib.win32-2.7

copying PIL\ImageMode.py -> build\lib.win32-2.7

copying PIL\ImageOps.py -> build\lib.win32-2.7

copying PIL\ImagePalette.py -> build\lib.win32-2.7

copying PIL\ImagePath.py -> build\lib.win32-2.7

copying PIL\ImageQt.py -> build\lib.win32-2.7

copying PIL\ImageSequence.py -> build\lib.win32-2.7

copying PIL\ImageShow.py -> build\lib.win32-2.7

copying PIL\ImageStat.py -> build\lib.win32-2.7

copying PIL\ImageTk.py -> build\lib.win32-2.7

copying PIL\ImageTransform.py -> build\lib.win32-2.7

copying PIL\ImageWin.py -> build\lib.win32-2.7

copying PIL\ImImagePlugin.py -> build\lib.win32-2.7

copying PIL\ImtImagePlugin.py -> build\lib.win32-2.7

copying PIL\IptcImagePlugin.py -> build\lib.win32-2.7

copying PIL\JpegImagePlugin.py -> build\lib.win32-2.7

copying PIL\McIdasImagePlugin.py -> build\lib.win32-2.7

copying PIL\MicImagePlugin.py -> build\lib.win32-2.7

copying PIL\MpegImagePlugin.py -> build\lib.win32-2.7

copying PIL\MspImagePlugin.py -> build\lib.win32-2.7

copying PIL\OleFileIO.py -> build\lib.win32-2.7

copying PIL\PaletteFile.py -> build\lib.win32-2.7

copying PIL\PalmImagePlugin.py -> build\lib.win32-2.7

copying PIL\PcdImagePlugin.py -> build\lib.win32-2.7

copying PIL\PcfFontFile.py -> build\lib.win32-2.7

copying PIL\PcxImagePlugin.py -> build\lib.win32-2.7

copying PIL\PdfImagePlugin.py -> build\lib.win32-2.7

copying PIL\PixarImagePlugin.py -> build\lib.win32-2.7

copying PIL\PngImagePlugin.py -> build\lib.win32-2.7

copying PIL\PpmImagePlugin.py -> build\lib.win32-2.7

copying PIL\PsdImagePlugin.py -> build\lib.win32-2.7

copying PIL\PSDraw.py -> build\lib.win32-2.7

copying PIL\SgiImagePlugin.py -> build\lib.win32-2.7

copying PIL\SpiderImagePlugin.py -> build\lib.win32-2.7

copying PIL\SunImagePlugin.py -> build\lib.win32-2.7

copying PIL\TarIO.py -> build\lib.win32-2.7

copying PIL\TgaImagePlugin.py -> build\lib.win32-2.7

copying PIL\TiffImagePlugin.py -> build\lib.win32-2.7

copying PIL\TiffTags.py -> build\lib.win32-2.7

copying PIL\WalImageFile.py -> build\lib.win32-2.7

copying PIL\WmfImagePlugin.py -> build\lib.win32-2.7

copying PIL\XbmImagePlugin.py -> build\lib.win32-2.7

copying PIL\XpmImagePlugin.py -> build\lib.win32-2.7

copying PIL\XVThumbImagePlugin.py -> build\lib.win32-2.7

copying PIL\__init__.py -> build\lib.win32-2.7

running build_ext

building '_imaging' extension

C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe /c /nologo /Ox /MD /W
3 /GS- /DNDEBUG -IlibImaging -IC:\pypy-1.8\include /Tc_imaging.c /Fobuild\temp.w
in32-2.7\Release\_imaging.obj

_imaging.c

_imaging.c(1230) : warning C4133: 'function' : incompatible types - from 'PyStri
ngObject *' to 'PyObject *'

_imaging.c(1384) : warning C4996: 'strcpy': This function or variable may be uns
afe. Consider using strcpy_s instead. To disable deprecation, use _CRT_SECURE_NO
_WARNINGS. See online help for details.

_imaging.c(1412) : warning C4996: 'strcpy': This function or variable may be uns
afe. Consider using strcpy_s instead. To disable deprecation, use _CRT_SECURE_NO
_WARNINGS. See online help for details.

_imaging.c(1550) : warning C4996: 'strcpy': This function or variable may be uns
afe. Consider using strcpy_s instead. To disable deprecation, use _CRT_SECURE_NO
_WARNINGS. See online help for details.

C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe /c /nologo /Ox /MD /W
3 /GS- /DNDEBUG -IlibImaging -IC:\pypy-1.8\include /Tcdecode.c /Fobuild\temp.win
32-2.7\Release\decode.obj

decode.c

C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe /c /nologo /Ox /MD /W
3 /GS- /DNDEBUG -IlibImaging -IC:\pypy-1.8\include /Tcencode.c /Fobuild\temp.win
32-2.7\Release\encode.obj

encode.c

encode.c(161) : warning C4996: 'write': The POSIX name for this item is deprecat
ed. Instead, use the ISO C++ conformant name: _write. See online help for detail
s.

        C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\io.h(322) : see
declaration of 'write'

C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe /c /nologo /Ox /MD /W
3 /GS- /DNDEBUG -IlibImaging -IC:\pypy-1.8\include /Tcmap.c /Fobuild\temp.win32-
2.7\Release\map.obj

map.c

C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(76) : warning C4
114: same type qualifier used more than once

C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(76) : error C263
2: 'char' followed by 'char' is illegal

C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(76) : error C205
9: syntax error : ','

C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(77) : error C263
2: 'short' followed by 'short' is illegal

C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(77) : error C205
9: syntax error : ','

C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(80) : warning C4
114: same type qualifier used more than once

C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(80) : error C263
2: 'char' followed by 'char' is illegal

C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(80) : error C205
9: syntax error : ','

C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(81) : warning C4
114: same type qualifier used more than once

C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(81) : error C263
2: 'short' followed by 'short' is illegal

C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(81) : error C205
9: syntax error : ','

error: command 'C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe' fail
ed with exit status 2

----------------------------------------
Command C:\pypy-1.8\pypy.exe -c "import setuptools;__file__='c:\\pypy-1.8\\bin\\
build\\pil\\setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'),
__file__, 'exec'))" install --single-version-externally-managed --record c:\user
s\jlap\appdata\local\temp\pip-y_8euy-record\install-record.txt failed with error
 code 1
Storing complete log in C:\Users\Jlap\AppData\Roaming\pip\pip.log

c:\pypy-1.8\bin>
"""

What should I try next?
What am I doing wrong?

Jacob

From santagada at gmail.com  Sun Mar  4 06:18:23 2012
From: santagada at gmail.com (Leonardo Santagada)
Date: Sun, 4 Mar 2012 02:18:23 -0300
Subject: [pypy-dev] How to get PIL to install?
In-Reply-To: <CAGBU8t6MDthpmuY2arM6_UM6Xm8DXiWkZ4mc61a_C+DfRx6=CA@mail.gmail.com>
References: <CAGBU8t6MDthpmuY2arM6_UM6Xm8DXiWkZ4mc61a_C+DfRx6=CA@mail.gmail.com>
Message-ID: <CA+HFc87PcWm=dKhpqsyJvVo_3NJvzU+PQifiDMoL-NOsc4SiuQ@mail.gmail.com>

try "pip install pillow". Pillow is a fork to make pil easier to be
pip installed (so mainly a better setup.py).

http://pypi.python.org/pypi/Pillow

On Sat, Mar 3, 2012 at 11:12 PM, Jacob Finkelman <eh2406 at wayne.edu> wrote:
> Hi,
>
> I have been lurking on this list for quite a while. I always feel
> smarter for having tried to follow the conversations on this list. I'd
> especially like to thank the people working on getting Cython working.
> The progress on that project has been fascinating to read about.
>
> I have been trying to get PIL to work on my Windows 7 32 bit laptop.
>
> I install Pypy. Currently 1.8
> Install distribute and PIP.
> Install "Microsoft Visual C++ 2008 Express Edition"
> run "cmd.exe"
> cd c:\pypy-1.8\bin
> pip install pil
>
> and I get a long error the end of which is """
>
> tails.
> ? ? ? ? ? ?C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\io.h(322) :
> see declaration of 'write'
> ? ?C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe /c /nologo /Ox /M
> D /W3 /GS- /DNDEBUG -IlibImaging -IC:\pypy-1.8\include /Tcmap.c /Fobuild\temp.wi
> n32-2.7\Release\map.obj
> ? ?map.c
> ? ?C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(76) : warnin
> g C4114: same type qualifier used more than once
> ? ?C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(76) : error
> C2632: 'char' followed by 'char' is illegal
> ? ?C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(76) : error
> C2059: syntax error : ','
> ? ?C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(77) : error
> C2632: 'short' followed by 'short' is illegal
> ? ?C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(77) : error
> C2059: syntax error : ','
> ? ?C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(80) : warnin
> g C4114: same type qualifier used more than once
> ? ?C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(80) : error
> C2632: 'char' followed by 'char' is illegal
> ? ?C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(80) : error
> C2059: syntax error : ','
> ? ?C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(81) : warnin
> g C4114: same type qualifier used more than once
> ? ?C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(81) : error
> C2632: 'short' followed by 'short' is illegal
> ? ?C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(81) : error
> C2059: syntax error : ','
> ? ?error: command 'C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe'
> failed with exit status 2
> ? ?Complete output from command C:\pypy-1.8\pypy.exe -c "import setuptools;__fi
> le__='c:\\pypy-1.8\\bin\\build\\pil\\setup.py';exec(compile(open(__file__).read(
> ).replace('\r\n', '\n'), __file__, 'exec'))" install --single-version-externally
> -managed --record c:\users\jlap\appdata\local\temp\pip-y_8euy-record\install-rec
> ord.txt:
> ? ?WARNING: '' not a valid package name; please use only.-separated package nam
> es in setup.py
>
> running install
>
> running build
>
> running build_py
>
> copying PIL\ArgImagePlugin.py -> build\lib.win32-2.7
>
> copying PIL\BdfFontFile.py -> build\lib.win32-2.7
>
> copying PIL\BmpImagePlugin.py -> build\lib.win32-2.7
>
> copying PIL\BufrStubImagePlugin.py -> build\lib.win32-2.7
>
> copying PIL\ContainerIO.py -> build\lib.win32-2.7
>
> copying PIL\CurImagePlugin.py -> build\lib.win32-2.7
>
> copying PIL\DcxImagePlugin.py -> build\lib.win32-2.7
>
> copying PIL\EpsImagePlugin.py -> build\lib.win32-2.7
>
> copying PIL\ExifTags.py -> build\lib.win32-2.7
>
> copying PIL\FitsStubImagePlugin.py -> build\lib.win32-2.7
>
> copying PIL\FliImagePlugin.py -> build\lib.win32-2.7
>
> copying PIL\FontFile.py -> build\lib.win32-2.7
>
> copying PIL\FpxImagePlugin.py -> build\lib.win32-2.7
>
> copying PIL\GbrImagePlugin.py -> build\lib.win32-2.7
>
> copying PIL\GdImageFile.py -> build\lib.win32-2.7
>
> copying PIL\GifImagePlugin.py -> build\lib.win32-2.7
>
> copying PIL\GimpGradientFile.py -> build\lib.win32-2.7
>
> copying PIL\GimpPaletteFile.py -> build\lib.win32-2.7
>
> copying PIL\GribStubImagePlugin.py -> build\lib.win32-2.7
>
> copying PIL\Hdf5StubImagePlugin.py -> build\lib.win32-2.7
>
> copying PIL\IcnsImagePlugin.py -> build\lib.win32-2.7
>
> copying PIL\IcoImagePlugin.py -> build\lib.win32-2.7
>
> copying PIL\Image.py -> build\lib.win32-2.7
>
> copying PIL\ImageChops.py -> build\lib.win32-2.7
>
> copying PIL\ImageCms.py -> build\lib.win32-2.7
>
> copying PIL\ImageColor.py -> build\lib.win32-2.7
>
> copying PIL\ImageDraw.py -> build\lib.win32-2.7
>
> copying PIL\ImageDraw2.py -> build\lib.win32-2.7
>
> copying PIL\ImageEnhance.py -> build\lib.win32-2.7
>
> copying PIL\ImageFile.py -> build\lib.win32-2.7
>
> copying PIL\ImageFileIO.py -> build\lib.win32-2.7
>
> copying PIL\ImageFilter.py -> build\lib.win32-2.7
>
> copying PIL\ImageFont.py -> build\lib.win32-2.7
>
> copying PIL\ImageGL.py -> build\lib.win32-2.7
>
> copying PIL\ImageGrab.py -> build\lib.win32-2.7
>
> copying PIL\ImageMath.py -> build\lib.win32-2.7
>
> copying PIL\ImageMode.py -> build\lib.win32-2.7
>
> copying PIL\ImageOps.py -> build\lib.win32-2.7
>
> copying PIL\ImagePalette.py -> build\lib.win32-2.7
>
> copying PIL\ImagePath.py -> build\lib.win32-2.7
>
> copying PIL\ImageQt.py -> build\lib.win32-2.7
>
> copying PIL\ImageSequence.py -> build\lib.win32-2.7
>
> copying PIL\ImageShow.py -> build\lib.win32-2.7
>
> copying PIL\ImageStat.py -> build\lib.win32-2.7
>
> copying PIL\ImageTk.py -> build\lib.win32-2.7
>
> copying PIL\ImageTransform.py -> build\lib.win32-2.7
>
> copying PIL\ImageWin.py -> build\lib.win32-2.7
>
> copying PIL\ImImagePlugin.py -> build\lib.win32-2.7
>
> copying PIL\ImtImagePlugin.py -> build\lib.win32-2.7
>
> copying PIL\IptcImagePlugin.py -> build\lib.win32-2.7
>
> copying PIL\JpegImagePlugin.py -> build\lib.win32-2.7
>
> copying PIL\McIdasImagePlugin.py -> build\lib.win32-2.7
>
> copying PIL\MicImagePlugin.py -> build\lib.win32-2.7
>
> copying PIL\MpegImagePlugin.py -> build\lib.win32-2.7
>
> copying PIL\MspImagePlugin.py -> build\lib.win32-2.7
>
> copying PIL\OleFileIO.py -> build\lib.win32-2.7
>
> copying PIL\PaletteFile.py -> build\lib.win32-2.7
>
> copying PIL\PalmImagePlugin.py -> build\lib.win32-2.7
>
> copying PIL\PcdImagePlugin.py -> build\lib.win32-2.7
>
> copying PIL\PcfFontFile.py -> build\lib.win32-2.7
>
> copying PIL\PcxImagePlugin.py -> build\lib.win32-2.7
>
> copying PIL\PdfImagePlugin.py -> build\lib.win32-2.7
>
> copying PIL\PixarImagePlugin.py -> build\lib.win32-2.7
>
> copying PIL\PngImagePlugin.py -> build\lib.win32-2.7
>
> copying PIL\PpmImagePlugin.py -> build\lib.win32-2.7
>
> copying PIL\PsdImagePlugin.py -> build\lib.win32-2.7
>
> copying PIL\PSDraw.py -> build\lib.win32-2.7
>
> copying PIL\SgiImagePlugin.py -> build\lib.win32-2.7
>
> copying PIL\SpiderImagePlugin.py -> build\lib.win32-2.7
>
> copying PIL\SunImagePlugin.py -> build\lib.win32-2.7
>
> copying PIL\TarIO.py -> build\lib.win32-2.7
>
> copying PIL\TgaImagePlugin.py -> build\lib.win32-2.7
>
> copying PIL\TiffImagePlugin.py -> build\lib.win32-2.7
>
> copying PIL\TiffTags.py -> build\lib.win32-2.7
>
> copying PIL\WalImageFile.py -> build\lib.win32-2.7
>
> copying PIL\WmfImagePlugin.py -> build\lib.win32-2.7
>
> copying PIL\XbmImagePlugin.py -> build\lib.win32-2.7
>
> copying PIL\XpmImagePlugin.py -> build\lib.win32-2.7
>
> copying PIL\XVThumbImagePlugin.py -> build\lib.win32-2.7
>
> copying PIL\__init__.py -> build\lib.win32-2.7
>
> running build_ext
>
> building '_imaging' extension
>
> C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe /c /nologo /Ox /MD /W
> 3 /GS- /DNDEBUG -IlibImaging -IC:\pypy-1.8\include /Tc_imaging.c /Fobuild\temp.w
> in32-2.7\Release\_imaging.obj
>
> _imaging.c
>
> _imaging.c(1230) : warning C4133: 'function' : incompatible types - from 'PyStri
> ngObject *' to 'PyObject *'
>
> _imaging.c(1384) : warning C4996: 'strcpy': This function or variable may be uns
> afe. Consider using strcpy_s instead. To disable deprecation, use _CRT_SECURE_NO
> _WARNINGS. See online help for details.
>
> _imaging.c(1412) : warning C4996: 'strcpy': This function or variable may be uns
> afe. Consider using strcpy_s instead. To disable deprecation, use _CRT_SECURE_NO
> _WARNINGS. See online help for details.
>
> _imaging.c(1550) : warning C4996: 'strcpy': This function or variable may be uns
> afe. Consider using strcpy_s instead. To disable deprecation, use _CRT_SECURE_NO
> _WARNINGS. See online help for details.
>
> C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe /c /nologo /Ox /MD /W
> 3 /GS- /DNDEBUG -IlibImaging -IC:\pypy-1.8\include /Tcdecode.c /Fobuild\temp.win
> 32-2.7\Release\decode.obj
>
> decode.c
>
> C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe /c /nologo /Ox /MD /W
> 3 /GS- /DNDEBUG -IlibImaging -IC:\pypy-1.8\include /Tcencode.c /Fobuild\temp.win
> 32-2.7\Release\encode.obj
>
> encode.c
>
> encode.c(161) : warning C4996: 'write': The POSIX name for this item is deprecat
> ed. Instead, use the ISO C++ conformant name: _write. See online help for detail
> s.
>
> ? ? ? ?C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\io.h(322) : see
> declaration of 'write'
>
> C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe /c /nologo /Ox /MD /W
> 3 /GS- /DNDEBUG -IlibImaging -IC:\pypy-1.8\include /Tcmap.c /Fobuild\temp.win32-
> 2.7\Release\map.obj
>
> map.c
>
> C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(76) : warning C4
> 114: same type qualifier used more than once
>
> C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(76) : error C263
> 2: 'char' followed by 'char' is illegal
>
> C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(76) : error C205
> 9: syntax error : ','
>
> C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(77) : error C263
> 2: 'short' followed by 'short' is illegal
>
> C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(77) : error C205
> 9: syntax error : ','
>
> C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(80) : warning C4
> 114: same type qualifier used more than once
>
> C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(80) : error C263
> 2: 'char' followed by 'char' is illegal
>
> C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(80) : error C205
> 9: syntax error : ','
>
> C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(81) : warning C4
> 114: same type qualifier used more than once
>
> C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(81) : error C263
> 2: 'short' followed by 'short' is illegal
>
> C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(81) : error C205
> 9: syntax error : ','
>
> error: command 'C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe' fail
> ed with exit status 2
>
> ----------------------------------------
> Command C:\pypy-1.8\pypy.exe -c "import setuptools;__file__='c:\\pypy-1.8\\bin\\
> build\\pil\\setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'),
> __file__, 'exec'))" install --single-version-externally-managed --record c:\user
> s\jlap\appdata\local\temp\pip-y_8euy-record\install-record.txt failed with error
> ?code 1
> Storing complete log in C:\Users\Jlap\AppData\Roaming\pip\pip.log
>
> c:\pypy-1.8\bin>
> """
>
> What should I try next?
> What am I doing wrong?
>
> Jacob
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev



-- 

Leonardo Santagada

From eh2406 at wayne.edu  Sun Mar  4 06:29:54 2012
From: eh2406 at wayne.edu (Jacob Finkelman)
Date: Sun, 4 Mar 2012 00:29:54 -0500
Subject: [pypy-dev] How to get PIL to install?
In-Reply-To: <CA+HFc87PcWm=dKhpqsyJvVo_3NJvzU+PQifiDMoL-NOsc4SiuQ@mail.gmail.com>
References: <CAGBU8t6MDthpmuY2arM6_UM6Xm8DXiWkZ4mc61a_C+DfRx6=CA@mail.gmail.com>
	<CA+HFc87PcWm=dKhpqsyJvVo_3NJvzU+PQifiDMoL-NOsc4SiuQ@mail.gmail.com>
Message-ID: <CAGBU8t59rdOr45HUy9Jeyw1Ag71f4zHaN-Mj6cQuQuKQ5aqy7A@mail.gmail.com>

Thanks,
That was a little better,

Still getting a similar error message?


    C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(76) : error
C2632: 'char' followed by 'char' is illegal
    C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(76) : error
C2059: syntax error : ','
    C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(77) : error
C2632: 'short' followed by 'short' is illegal
    C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(77) : error
C2059: syntax error : ','
    C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(80) : warnin
g C4114: same type qualifier used more than once
    C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(80) : error
C2632: 'char' followed by 'char' is illegal
    C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(80) : error
C2059: syntax error : ','
    C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(81) : warnin
g C4114: same type qualifier used more than once
    C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(81) : error
C2632: 'short' followed by 'short' is illegal
    C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(81) : error
C2059: syntax error : ','
    error: command 'C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe'
failed with exit status 2
    Complete output from command C:\pypy-1.8\pypy.exe -c "import setuptools;__fi
le__='c:\\pypy-1.8\\bin\\build\\pillow\\setup.py';exec(compile(open(__file__).re
ad().replace('\r\n', '\n'), __file__, 'exec'))" install --single-version-externa
lly-managed --record c:\users\jlap\appdata\local\temp\pip-1sqkcz-record\install-
record.txt:
    running install

running build

running build_py

creating build

creating build\lib.win32-2.7

creating build\lib.win32-2.7\PIL

copying PIL\ArgImagePlugin.py -> build\lib.win32-2.7\PIL

copying PIL\BdfFontFile.py -> build\lib.win32-2.7\PIL

copying PIL\BmpImagePlugin.py -> build\lib.win32-2.7\PIL

copying PIL\BufrStubImagePlugin.py -> build\lib.win32-2.7\PIL

copying PIL\ContainerIO.py -> build\lib.win32-2.7\PIL

copying PIL\CurImagePlugin.py -> build\lib.win32-2.7\PIL

copying PIL\DcxImagePlugin.py -> build\lib.win32-2.7\PIL

copying PIL\EpsImagePlugin.py -> build\lib.win32-2.7\PIL

copying PIL\ExifTags.py -> build\lib.win32-2.7\PIL

copying PIL\FitsStubImagePlugin.py -> build\lib.win32-2.7\PIL

copying PIL\FliImagePlugin.py -> build\lib.win32-2.7\PIL

copying PIL\FontFile.py -> build\lib.win32-2.7\PIL

copying PIL\FpxImagePlugin.py -> build\lib.win32-2.7\PIL

copying PIL\GbrImagePlugin.py -> build\lib.win32-2.7\PIL

copying PIL\GdImageFile.py -> build\lib.win32-2.7\PIL

copying PIL\GifImagePlugin.py -> build\lib.win32-2.7\PIL

copying PIL\GimpGradientFile.py -> build\lib.win32-2.7\PIL

copying PIL\GimpPaletteFile.py -> build\lib.win32-2.7\PIL

copying PIL\GribStubImagePlugin.py -> build\lib.win32-2.7\PIL

copying PIL\Hdf5StubImagePlugin.py -> build\lib.win32-2.7\PIL

copying PIL\IcnsImagePlugin.py -> build\lib.win32-2.7\PIL

copying PIL\IcoImagePlugin.py -> build\lib.win32-2.7\PIL

copying PIL\Image.py -> build\lib.win32-2.7\PIL

copying PIL\ImageChops.py -> build\lib.win32-2.7\PIL

copying PIL\ImageCms.py -> build\lib.win32-2.7\PIL

copying PIL\ImageColor.py -> build\lib.win32-2.7\PIL

copying PIL\ImageDraw.py -> build\lib.win32-2.7\PIL

copying PIL\ImageDraw2.py -> build\lib.win32-2.7\PIL

copying PIL\ImageEnhance.py -> build\lib.win32-2.7\PIL

copying PIL\ImageFile.py -> build\lib.win32-2.7\PIL

copying PIL\ImageFileIO.py -> build\lib.win32-2.7\PIL

copying PIL\ImageFilter.py -> build\lib.win32-2.7\PIL

copying PIL\ImageFont.py -> build\lib.win32-2.7\PIL

copying PIL\ImageGL.py -> build\lib.win32-2.7\PIL

copying PIL\ImageGrab.py -> build\lib.win32-2.7\PIL

copying PIL\ImageMath.py -> build\lib.win32-2.7\PIL

copying PIL\ImageMode.py -> build\lib.win32-2.7\PIL

copying PIL\ImageOps.py -> build\lib.win32-2.7\PIL

copying PIL\ImagePalette.py -> build\lib.win32-2.7\PIL

copying PIL\ImagePath.py -> build\lib.win32-2.7\PIL

copying PIL\ImageQt.py -> build\lib.win32-2.7\PIL

copying PIL\ImageSequence.py -> build\lib.win32-2.7\PIL

copying PIL\ImageShow.py -> build\lib.win32-2.7\PIL

copying PIL\ImageStat.py -> build\lib.win32-2.7\PIL

copying PIL\ImageTk.py -> build\lib.win32-2.7\PIL

copying PIL\ImageTransform.py -> build\lib.win32-2.7\PIL

copying PIL\ImageWin.py -> build\lib.win32-2.7\PIL

copying PIL\ImImagePlugin.py -> build\lib.win32-2.7\PIL

copying PIL\ImtImagePlugin.py -> build\lib.win32-2.7\PIL

copying PIL\IptcImagePlugin.py -> build\lib.win32-2.7\PIL

copying PIL\JpegImagePlugin.py -> build\lib.win32-2.7\PIL

copying PIL\McIdasImagePlugin.py -> build\lib.win32-2.7\PIL

copying PIL\MicImagePlugin.py -> build\lib.win32-2.7\PIL

copying PIL\MpegImagePlugin.py -> build\lib.win32-2.7\PIL

copying PIL\MspImagePlugin.py -> build\lib.win32-2.7\PIL

copying PIL\OleFileIO.py -> build\lib.win32-2.7\PIL

copying PIL\PaletteFile.py -> build\lib.win32-2.7\PIL

copying PIL\PalmImagePlugin.py -> build\lib.win32-2.7\PIL

copying PIL\PcdImagePlugin.py -> build\lib.win32-2.7\PIL

copying PIL\PcfFontFile.py -> build\lib.win32-2.7\PIL

copying PIL\PcxImagePlugin.py -> build\lib.win32-2.7\PIL

copying PIL\PdfImagePlugin.py -> build\lib.win32-2.7\PIL

copying PIL\PixarImagePlugin.py -> build\lib.win32-2.7\PIL

copying PIL\PngImagePlugin.py -> build\lib.win32-2.7\PIL

copying PIL\PpmImagePlugin.py -> build\lib.win32-2.7\PIL

copying PIL\PsdImagePlugin.py -> build\lib.win32-2.7\PIL

copying PIL\PSDraw.py -> build\lib.win32-2.7\PIL

copying PIL\SgiImagePlugin.py -> build\lib.win32-2.7\PIL

copying PIL\SpiderImagePlugin.py -> build\lib.win32-2.7\PIL

copying PIL\SunImagePlugin.py -> build\lib.win32-2.7\PIL

copying PIL\TarIO.py -> build\lib.win32-2.7\PIL

copying PIL\TgaImagePlugin.py -> build\lib.win32-2.7\PIL

copying PIL\TiffImagePlugin.py -> build\lib.win32-2.7\PIL

copying PIL\TiffTags.py -> build\lib.win32-2.7\PIL

copying PIL\WalImageFile.py -> build\lib.win32-2.7\PIL

copying PIL\WmfImagePlugin.py -> build\lib.win32-2.7\PIL

copying PIL\XbmImagePlugin.py -> build\lib.win32-2.7\PIL

copying PIL\XpmImagePlugin.py -> build\lib.win32-2.7\PIL

copying PIL\XVThumbImagePlugin.py -> build\lib.win32-2.7\PIL

copying PIL\__init__.py -> build\lib.win32-2.7\PIL

running build_ext

building '_imaging' extension

creating build\temp.win32-2.7

creating build\temp.win32-2.7\Release

creating build\temp.win32-2.7\Release\libImaging

C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe /c /nologo /Ox /MD /W
3 /GS- /DNDEBUG -IlibImaging -IC:\pypy-1.8\include /Tc_imaging.c /Fobuild\temp.w
in32-2.7\Release\_imaging.obj

_imaging.c

_imaging.c(1230) : warning C4133: 'function' : incompatible types - from 'PyStri
ngObject *' to 'PyObject *'

_imaging.c(1384) : warning C4996: 'strcpy': This function or variable may be uns
afe. Consider using strcpy_s instead. To disable deprecation, use _CRT_SECURE_NO
_WARNINGS. See online help for details.

_imaging.c(1412) : warning C4996: 'strcpy': This function or variable may be uns
afe. Consider using strcpy_s instead. To disable deprecation, use _CRT_SECURE_NO
_WARNINGS. See online help for details.

_imaging.c(1550) : warning C4996: 'strcpy': This function or variable may be uns
afe. Consider using strcpy_s instead. To disable deprecation, use _CRT_SECURE_NO
_WARNINGS. See online help for details.

C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe /c /nologo /Ox /MD /W
3 /GS- /DNDEBUG -IlibImaging -IC:\pypy-1.8\include /Tcdecode.c /Fobuild\temp.win
32-2.7\Release\decode.obj

decode.c

C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe /c /nologo /Ox /MD /W
3 /GS- /DNDEBUG -IlibImaging -IC:\pypy-1.8\include /Tcencode.c /Fobuild\temp.win
32-2.7\Release\encode.obj

encode.c

encode.c(161) : warning C4996: 'write': The POSIX name for this item is deprecat
ed. Instead, use the ISO C++ conformant name: _write. See online help for detail
s.

        C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\io.h(322) : see
declaration of 'write'

C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe /c /nologo /Ox /MD /W
3 /GS- /DNDEBUG -IlibImaging -IC:\pypy-1.8\include /Tcmap.c /Fobuild\temp.win32-
2.7\Release\map.obj

map.c

C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(76) : warning C4
114: same type qualifier used more than once

C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(76) : error C263
2: 'char' followed by 'char' is illegal

C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(76) : error C205
9: syntax error : ','

C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(77) : error C263
2: 'short' followed by 'short' is illegal

C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(77) : error C205
9: syntax error : ','

C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(80) : warning C4
114: same type qualifier used more than once

C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(80) : error C263
2: 'char' followed by 'char' is illegal

C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(80) : error C205
9: syntax error : ','

C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(81) : warning C4
114: same type qualifier used more than once

C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(81) : error C263
2: 'short' followed by 'short' is illegal

C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(81) : error C205
9: syntax error : ','

error: command 'C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe' fail
ed with exit status 2

----------------------------------------
Command C:\pypy-1.8\pypy.exe -c "import setuptools;__file__='c:\\pypy-1.8\\bin\\
build\\pillow\\setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'
), __file__, 'exec'))" install --single-version-externally-managed --record c:\u
sers\jlap\appdata\local\temp\pip-1sqkcz-record\install-record.txt failed with er
ror code 1
Storing complete log in C:\Users\Jlap\AppData\Roaming\pip\pip.log

c:\pypy-1.8\bin>

On Sun, Mar 4, 2012 at 12:18 AM, Leonardo Santagada <santagada at gmail.com> wrote:
> try "pip install pillow". Pillow is a fork to make pil easier to be
> pip installed (so mainly a better setup.py).
>
> http://pypi.python.org/pypi/Pillow
>
> On Sat, Mar 3, 2012 at 11:12 PM, Jacob Finkelman <eh2406 at wayne.edu> wrote:
>> Hi,
>>
>> I have been lurking on this list for quite a while. I always feel
>> smarter for having tried to follow the conversations on this list. I'd
>> especially like to thank the people working on getting Cython working.
>> The progress on that project has been fascinating to read about.
>>
>> I have been trying to get PIL to work on my Windows 7 32 bit laptop.
>>
>> I install Pypy. Currently 1.8
>> Install distribute and PIP.
>> Install "Microsoft Visual C++ 2008 Express Edition"
>> run "cmd.exe"
>> cd c:\pypy-1.8\bin
>> pip install pil
>>
>> and I get a long error the end of which is """
>>
>> tails.
>> ? ? ? ? ? ?C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\io.h(322) :
>> see declaration of 'write'
>> ? ?C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe /c /nologo /Ox /M
>> D /W3 /GS- /DNDEBUG -IlibImaging -IC:\pypy-1.8\include /Tcmap.c /Fobuild\temp.wi
>> n32-2.7\Release\map.obj
>> ? ?map.c
>> ? ?C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(76) : warnin
>> g C4114: same type qualifier used more than once
>> ? ?C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(76) : error
>> C2632: 'char' followed by 'char' is illegal
>> ? ?C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(76) : error
>> C2059: syntax error : ','
>> ? ?C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(77) : error
>> C2632: 'short' followed by 'short' is illegal
>> ? ?C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(77) : error
>> C2059: syntax error : ','
>> ? ?C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(80) : warnin
>> g C4114: same type qualifier used more than once
>> ? ?C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(80) : error
>> C2632: 'char' followed by 'char' is illegal
>> ? ?C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(80) : error
>> C2059: syntax error : ','
>> ? ?C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(81) : warnin
>> g C4114: same type qualifier used more than once
>> ? ?C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(81) : error
>> C2632: 'short' followed by 'short' is illegal
>> ? ?C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(81) : error
>> C2059: syntax error : ','
>> ? ?error: command 'C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe'
>> failed with exit status 2
>> ? ?Complete output from command C:\pypy-1.8\pypy.exe -c "import setuptools;__fi
>> le__='c:\\pypy-1.8\\bin\\build\\pil\\setup.py';exec(compile(open(__file__).read(
>> ).replace('\r\n', '\n'), __file__, 'exec'))" install --single-version-externally
>> -managed --record c:\users\jlap\appdata\local\temp\pip-y_8euy-record\install-rec
>> ord.txt:
>> ? ?WARNING: '' not a valid package name; please use only.-separated package nam
>> es in setup.py
>>
>> running install
>>
>> running build
>>
>> running build_py
>>
>> copying PIL\ArgImagePlugin.py -> build\lib.win32-2.7
>>
>> copying PIL\BdfFontFile.py -> build\lib.win32-2.7
>>
>> copying PIL\BmpImagePlugin.py -> build\lib.win32-2.7
>>
>> copying PIL\BufrStubImagePlugin.py -> build\lib.win32-2.7
>>
>> copying PIL\ContainerIO.py -> build\lib.win32-2.7
>>
>> copying PIL\CurImagePlugin.py -> build\lib.win32-2.7
>>
>> copying PIL\DcxImagePlugin.py -> build\lib.win32-2.7
>>
>> copying PIL\EpsImagePlugin.py -> build\lib.win32-2.7
>>
>> copying PIL\ExifTags.py -> build\lib.win32-2.7
>>
>> copying PIL\FitsStubImagePlugin.py -> build\lib.win32-2.7
>>
>> copying PIL\FliImagePlugin.py -> build\lib.win32-2.7
>>
>> copying PIL\FontFile.py -> build\lib.win32-2.7
>>
>> copying PIL\FpxImagePlugin.py -> build\lib.win32-2.7
>>
>> copying PIL\GbrImagePlugin.py -> build\lib.win32-2.7
>>
>> copying PIL\GdImageFile.py -> build\lib.win32-2.7
>>
>> copying PIL\GifImagePlugin.py -> build\lib.win32-2.7
>>
>> copying PIL\GimpGradientFile.py -> build\lib.win32-2.7
>>
>> copying PIL\GimpPaletteFile.py -> build\lib.win32-2.7
>>
>> copying PIL\GribStubImagePlugin.py -> build\lib.win32-2.7
>>
>> copying PIL\Hdf5StubImagePlugin.py -> build\lib.win32-2.7
>>
>> copying PIL\IcnsImagePlugin.py -> build\lib.win32-2.7
>>
>> copying PIL\IcoImagePlugin.py -> build\lib.win32-2.7
>>
>> copying PIL\Image.py -> build\lib.win32-2.7
>>
>> copying PIL\ImageChops.py -> build\lib.win32-2.7
>>
>> copying PIL\ImageCms.py -> build\lib.win32-2.7
>>
>> copying PIL\ImageColor.py -> build\lib.win32-2.7
>>
>> copying PIL\ImageDraw.py -> build\lib.win32-2.7
>>
>> copying PIL\ImageDraw2.py -> build\lib.win32-2.7
>>
>> copying PIL\ImageEnhance.py -> build\lib.win32-2.7
>>
>> copying PIL\ImageFile.py -> build\lib.win32-2.7
>>
>> copying PIL\ImageFileIO.py -> build\lib.win32-2.7
>>
>> copying PIL\ImageFilter.py -> build\lib.win32-2.7
>>
>> copying PIL\ImageFont.py -> build\lib.win32-2.7
>>
>> copying PIL\ImageGL.py -> build\lib.win32-2.7
>>
>> copying PIL\ImageGrab.py -> build\lib.win32-2.7
>>
>> copying PIL\ImageMath.py -> build\lib.win32-2.7
>>
>> copying PIL\ImageMode.py -> build\lib.win32-2.7
>>
>> copying PIL\ImageOps.py -> build\lib.win32-2.7
>>
>> copying PIL\ImagePalette.py -> build\lib.win32-2.7
>>
>> copying PIL\ImagePath.py -> build\lib.win32-2.7
>>
>> copying PIL\ImageQt.py -> build\lib.win32-2.7
>>
>> copying PIL\ImageSequence.py -> build\lib.win32-2.7
>>
>> copying PIL\ImageShow.py -> build\lib.win32-2.7
>>
>> copying PIL\ImageStat.py -> build\lib.win32-2.7
>>
>> copying PIL\ImageTk.py -> build\lib.win32-2.7
>>
>> copying PIL\ImageTransform.py -> build\lib.win32-2.7
>>
>> copying PIL\ImageWin.py -> build\lib.win32-2.7
>>
>> copying PIL\ImImagePlugin.py -> build\lib.win32-2.7
>>
>> copying PIL\ImtImagePlugin.py -> build\lib.win32-2.7
>>
>> copying PIL\IptcImagePlugin.py -> build\lib.win32-2.7
>>
>> copying PIL\JpegImagePlugin.py -> build\lib.win32-2.7
>>
>> copying PIL\McIdasImagePlugin.py -> build\lib.win32-2.7
>>
>> copying PIL\MicImagePlugin.py -> build\lib.win32-2.7
>>
>> copying PIL\MpegImagePlugin.py -> build\lib.win32-2.7
>>
>> copying PIL\MspImagePlugin.py -> build\lib.win32-2.7
>>
>> copying PIL\OleFileIO.py -> build\lib.win32-2.7
>>
>> copying PIL\PaletteFile.py -> build\lib.win32-2.7
>>
>> copying PIL\PalmImagePlugin.py -> build\lib.win32-2.7
>>
>> copying PIL\PcdImagePlugin.py -> build\lib.win32-2.7
>>
>> copying PIL\PcfFontFile.py -> build\lib.win32-2.7
>>
>> copying PIL\PcxImagePlugin.py -> build\lib.win32-2.7
>>
>> copying PIL\PdfImagePlugin.py -> build\lib.win32-2.7
>>
>> copying PIL\PixarImagePlugin.py -> build\lib.win32-2.7
>>
>> copying PIL\PngImagePlugin.py -> build\lib.win32-2.7
>>
>> copying PIL\PpmImagePlugin.py -> build\lib.win32-2.7
>>
>> copying PIL\PsdImagePlugin.py -> build\lib.win32-2.7
>>
>> copying PIL\PSDraw.py -> build\lib.win32-2.7
>>
>> copying PIL\SgiImagePlugin.py -> build\lib.win32-2.7
>>
>> copying PIL\SpiderImagePlugin.py -> build\lib.win32-2.7
>>
>> copying PIL\SunImagePlugin.py -> build\lib.win32-2.7
>>
>> copying PIL\TarIO.py -> build\lib.win32-2.7
>>
>> copying PIL\TgaImagePlugin.py -> build\lib.win32-2.7
>>
>> copying PIL\TiffImagePlugin.py -> build\lib.win32-2.7
>>
>> copying PIL\TiffTags.py -> build\lib.win32-2.7
>>
>> copying PIL\WalImageFile.py -> build\lib.win32-2.7
>>
>> copying PIL\WmfImagePlugin.py -> build\lib.win32-2.7
>>
>> copying PIL\XbmImagePlugin.py -> build\lib.win32-2.7
>>
>> copying PIL\XpmImagePlugin.py -> build\lib.win32-2.7
>>
>> copying PIL\XVThumbImagePlugin.py -> build\lib.win32-2.7
>>
>> copying PIL\__init__.py -> build\lib.win32-2.7
>>
>> running build_ext
>>
>> building '_imaging' extension
>>
>> C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe /c /nologo /Ox /MD /W
>> 3 /GS- /DNDEBUG -IlibImaging -IC:\pypy-1.8\include /Tc_imaging.c /Fobuild\temp.w
>> in32-2.7\Release\_imaging.obj
>>
>> _imaging.c
>>
>> _imaging.c(1230) : warning C4133: 'function' : incompatible types - from 'PyStri
>> ngObject *' to 'PyObject *'
>>
>> _imaging.c(1384) : warning C4996: 'strcpy': This function or variable may be uns
>> afe. Consider using strcpy_s instead. To disable deprecation, use _CRT_SECURE_NO
>> _WARNINGS. See online help for details.
>>
>> _imaging.c(1412) : warning C4996: 'strcpy': This function or variable may be uns
>> afe. Consider using strcpy_s instead. To disable deprecation, use _CRT_SECURE_NO
>> _WARNINGS. See online help for details.
>>
>> _imaging.c(1550) : warning C4996: 'strcpy': This function or variable may be uns
>> afe. Consider using strcpy_s instead. To disable deprecation, use _CRT_SECURE_NO
>> _WARNINGS. See online help for details.
>>
>> C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe /c /nologo /Ox /MD /W
>> 3 /GS- /DNDEBUG -IlibImaging -IC:\pypy-1.8\include /Tcdecode.c /Fobuild\temp.win
>> 32-2.7\Release\decode.obj
>>
>> decode.c
>>
>> C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe /c /nologo /Ox /MD /W
>> 3 /GS- /DNDEBUG -IlibImaging -IC:\pypy-1.8\include /Tcencode.c /Fobuild\temp.win
>> 32-2.7\Release\encode.obj
>>
>> encode.c
>>
>> encode.c(161) : warning C4996: 'write': The POSIX name for this item is deprecat
>> ed. Instead, use the ISO C++ conformant name: _write. See online help for detail
>> s.
>>
>> ? ? ? ?C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\io.h(322) : see
>> declaration of 'write'
>>
>> C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe /c /nologo /Ox /MD /W
>> 3 /GS- /DNDEBUG -IlibImaging -IC:\pypy-1.8\include /Tcmap.c /Fobuild\temp.win32-
>> 2.7\Release\map.obj
>>
>> map.c
>>
>> C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(76) : warning C4
>> 114: same type qualifier used more than once
>>
>> C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(76) : error C263
>> 2: 'char' followed by 'char' is illegal
>>
>> C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(76) : error C205
>> 9: syntax error : ','
>>
>> C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(77) : error C263
>> 2: 'short' followed by 'short' is illegal
>>
>> C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(77) : error C205
>> 9: syntax error : ','
>>
>> C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(80) : warning C4
>> 114: same type qualifier used more than once
>>
>> C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(80) : error C263
>> 2: 'char' followed by 'char' is illegal
>>
>> C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(80) : error C205
>> 9: syntax error : ','
>>
>> C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(81) : warning C4
>> 114: same type qualifier used more than once
>>
>> C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(81) : error C263
>> 2: 'short' followed by 'short' is illegal
>>
>> C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(81) : error C205
>> 9: syntax error : ','
>>
>> error: command 'C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe' fail
>> ed with exit status 2
>>
>> ----------------------------------------
>> Command C:\pypy-1.8\pypy.exe -c "import setuptools;__file__='c:\\pypy-1.8\\bin\\
>> build\\pil\\setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'),
>> __file__, 'exec'))" install --single-version-externally-managed --record c:\user
>> s\jlap\appdata\local\temp\pip-y_8euy-record\install-record.txt failed with error
>> ?code 1
>> Storing complete log in C:\Users\Jlap\AppData\Roaming\pip\pip.log
>>
>> c:\pypy-1.8\bin>
>> """
>>
>> What should I try next?
>> What am I doing wrong?
>>
>> Jacob
>> _______________________________________________
>> pypy-dev mailing list
>> pypy-dev at python.org
>> http://mail.python.org/mailman/listinfo/pypy-dev
>
>
>
> --
>
> Leonardo Santagada
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pip.log
Type: application/octet-stream
Size: 49052 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120304/3ab1460c/attachment-0001.obj>

From arigo at tunes.org  Sun Mar  4 14:58:30 2012
From: arigo at tunes.org (Armin Rigo)
Date: Sun, 4 Mar 2012 14:58:30 +0100
Subject: [pypy-dev] How to get PIL to install?
In-Reply-To: <CAGBU8t6MDthpmuY2arM6_UM6Xm8DXiWkZ4mc61a_C+DfRx6=CA@mail.gmail.com>
References: <CAGBU8t6MDthpmuY2arM6_UM6Xm8DXiWkZ4mc61a_C+DfRx6=CA@mail.gmail.com>
Message-ID: <CAMSv6X0uZRKfd2ov-jXoTH788uL8osLZ0BSbOPSfq4wQ+=6D+Q@mail.gmail.com>

Hi Jacob,

Googling around, I found this thread:

http://social.msdn.microsoft.com/Forums/en-SG/vcgeneral/thread/6ca96d26-1765-49d8-85fd-d455795fbb7e

It seems related.  The basic issue is that "INT8" is already defined,
possibly as a #define macro, from somewhere inside PIL; then when it
#includes the standard header, crash.  Or maybe it's "UINT8" or
"SINT8", which are typedefs from
pypy/translator/c/src/libffi_msvc/ffi_common.h.  In the first case,
the version of PIL you're using wouldn't compile at all on VC++2008
even on CPython, so it looks unlikely (except in some case like
Python.h happens not to #include this standard header in CPython, but
doing so in PyPy).  To try to fix the second case, try renaming UINT8
and SINT8, which are defined in
./translator/c/src/libffi_msvc/ffi_common.h and apparently used only
once in ./translator/c/src/libffi_msvc/ffi.c.  (I think I'll rename it
anyway to avoid such issues.)


A bient?t,

Armin.

From eh2406 at wayne.edu  Sun Mar  4 15:27:58 2012
From: eh2406 at wayne.edu (Jacob Finkelman)
Date: Sun, 4 Mar 2012 09:27:58 -0500
Subject: [pypy-dev] How to get PIL to install?
In-Reply-To: <CAMSv6X0uZRKfd2ov-jXoTH788uL8osLZ0BSbOPSfq4wQ+=6D+Q@mail.gmail.com>
References: <CAGBU8t6MDthpmuY2arM6_UM6Xm8DXiWkZ4mc61a_C+DfRx6=CA@mail.gmail.com>
	<CAMSv6X0uZRKfd2ov-jXoTH788uL8osLZ0BSbOPSfq4wQ+=6D+Q@mail.gmail.com>
Message-ID: <CAGBU8t4XALkU4OYiFYqa1NP3iDFkVgeFTp8NYB=E9g+Zah-J=g@mail.gmail.com>

Thanks.

I feel really dumb for having to ask you this? Where do I find
.../ffi_common.h? It does not appear to be in my PYPY-1.8 folder.

Jacob

On Sun, Mar 4, 2012 at 8:58 AM, Armin Rigo <arigo at tunes.org> wrote:
> Hi Jacob,
>
> Googling around, I found this thread:
>
> http://social.msdn.microsoft.com/Forums/en-SG/vcgeneral/thread/6ca96d26-1765-49d8-85fd-d455795fbb7e
>
> It seems related. ?The basic issue is that "INT8" is already defined,
> possibly as a #define macro, from somewhere inside PIL; then when it
> #includes the standard header, crash. ?Or maybe it's "UINT8" or
> "SINT8", which are typedefs from
> pypy/translator/c/src/libffi_msvc/ffi_common.h. ?In the first case,
> the version of PIL you're using wouldn't compile at all on VC++2008
> even on CPython, so it looks unlikely (except in some case like
> Python.h happens not to #include this standard header in CPython, but
> doing so in PyPy). ?To try to fix the second case, try renaming UINT8
> and SINT8, which are defined in
> ./translator/c/src/libffi_msvc/ffi_common.h and apparently used only
> once in ./translator/c/src/libffi_msvc/ffi.c. ?(I think I'll rename it
> anyway to avoid such issues.)
>
>
> A bient?t,
>
> Armin.

From arigo at tunes.org  Sun Mar  4 20:26:04 2012
From: arigo at tunes.org (Armin Rigo)
Date: Sun, 4 Mar 2012 20:26:04 +0100
Subject: [pypy-dev] pypy2exe
In-Reply-To: <CANeLV3Zpo8PFYd96xUSrmiBOeOxJco+bWovHvASLfPVKYxK4zg@mail.gmail.com>
References: <CANeLV3Zpo8PFYd96xUSrmiBOeOxJco+bWovHvASLfPVKYxK4zg@mail.gmail.com>
Message-ID: <CAMSv6X1OtFD5xMS=T+j4e4ZDvV0Z7VYENg+N6k_d1heieYAU3w@mail.gmail.com>

Hi Paolo,

On Wed, Feb 29, 2012 at 09:57, Paolo Basso <pa.basso at gmail.com> wrote:
> May be rpythonic (http://code.google.com/p/rpythonic/) useful?

"rpythonic" is an unrelated project.

> Is there or will be there something like a pypy2exe or?already exists a step
> by step guide I missed?

No, there is no specific pypy2exe project right now.  The issue is
caused in part from the fact that we, the core developers, are not
Windows people.  PyPy works "as expected" on Linux and to some extent
on OS/X in the sense that it is becoming more and more integrated in
the normal distributions.  The same is not true on Windows, where we
only offer a .zip of the latest release.  We need someone to seriously
care about Windows and adapt py2exe or other tools.


A bient?t,

Armin.

From eh2406 at wayne.edu  Mon Mar  5 14:17:38 2012
From: eh2406 at wayne.edu (Jacob Finkelman)
Date: Mon, 5 Mar 2012 08:17:38 -0500
Subject: [pypy-dev] How to get PIL to install?
In-Reply-To: <CAGBU8t4XALkU4OYiFYqa1NP3iDFkVgeFTp8NYB=E9g+Zah-J=g@mail.gmail.com>
References: <CAGBU8t6MDthpmuY2arM6_UM6Xm8DXiWkZ4mc61a_C+DfRx6=CA@mail.gmail.com>
	<CAMSv6X0uZRKfd2ov-jXoTH788uL8osLZ0BSbOPSfq4wQ+=6D+Q@mail.gmail.com>
	<CAGBU8t4XALkU4OYiFYqa1NP3iDFkVgeFTp8NYB=E9g+Zah-J=g@mail.gmail.com>
Message-ID: <CAGBU8t6J7DpKECL18-T6QMxrfsONio+h3wtm0d-E+FDFO-vJeQ@mail.gmail.com>

If it involves translating pypy, then I am stuck. This laptop does not have
the resources to translate pypy.

The website has said that pil is " known to work " for several releases
now. I thought that meant it would be pretty trivial to install.

Jacob
On Mar 4, 2012 9:27 AM, "Jacob Finkelman" <eh2406 at wayne.edu> wrote:

> Thanks.
>
> I feel really dumb for having to ask you this? Where do I find
> .../ffi_common.h? It does not appear to be in my PYPY-1.8 folder.
>
> Jacob
>
> On Sun, Mar 4, 2012 at 8:58 AM, Armin Rigo <arigo at tunes.org> wrote:
> > Hi Jacob,
> >
> > Googling around, I found this thread:
> >
> >
> http://social.msdn.microsoft.com/Forums/en-SG/vcgeneral/thread/6ca96d26-1765-49d8-85fd-d455795fbb7e
> >
> > It seems related.  The basic issue is that "INT8" is already defined,
> > possibly as a #define macro, from somewhere inside PIL; then when it
> > #includes the standard header, crash.  Or maybe it's "UINT8" or
> > "SINT8", which are typedefs from
> > pypy/translator/c/src/libffi_msvc/ffi_common.h.  In the first case,
> > the version of PIL you're using wouldn't compile at all on VC++2008
> > even on CPython, so it looks unlikely (except in some case like
> > Python.h happens not to #include this standard header in CPython, but
> > doing so in PyPy).  To try to fix the second case, try renaming UINT8
> > and SINT8, which are defined in
> > ./translator/c/src/libffi_msvc/ffi_common.h and apparently used only
> > once in ./translator/c/src/libffi_msvc/ffi.c.  (I think I'll rename it
> > anyway to avoid such issues.)
> >
> >
> > A bient?t,
> >
> > Armin.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120305/0e0caa44/attachment.html>

From matti.picus at gmail.com  Mon Mar  5 17:57:04 2012
From: matti.picus at gmail.com (Matti Picus)
Date: Mon, 05 Mar 2012 18:57:04 +0200
Subject: [pypy-dev] pypy2exe
Message-ID: <4F54F060.8040102@gmail.com>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120305/c9f13fa4/attachment.html>

From fuzzyman at gmail.com  Mon Mar  5 18:11:07 2012
From: fuzzyman at gmail.com (Michael Foord)
Date: Mon, 5 Mar 2012 17:11:07 +0000
Subject: [pypy-dev] pypy2exe
In-Reply-To: <4F54F060.8040102@gmail.com>
References: <4F54F060.8040102@gmail.com>
Message-ID: <CAKCKLWyJtzD6xacd7D5stJn0JdVSHegJwzkeF5Li6ULiZt1hhQ@mail.gmail.com>

On 5 March 2012 16:57, Matti Picus <matti.picus at gmail.com> wrote:

>  Trying to install in pypy I get
>
> error C2065: '_Py_PackageContext' : undeclared identifier
>

What are you trying to install? It isn't py2exe is it? That hasn't got a
chance of working with pypy because it *patches* the CPython dll (which is
totally different from the pypy binary).

All the best,

Michael


> Matti
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev
>
>


-- 

http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120305/6c17a57e/attachment.html>

From dynamicgl at gmail.com  Tue Mar  6 04:16:20 2012
From: dynamicgl at gmail.com (gelin yan)
Date: Tue, 6 Mar 2012 11:16:20 +0800
Subject: [pypy-dev] pypy2exe
In-Reply-To: <CAKCKLWyJtzD6xacd7D5stJn0JdVSHegJwzkeF5Li6ULiZt1hhQ@mail.gmail.com>
References: <4F54F060.8040102@gmail.com>
	<CAKCKLWyJtzD6xacd7D5stJn0JdVSHegJwzkeF5Li6ULiZt1hhQ@mail.gmail.com>
Message-ID: <CABkOF6TmneT5T5Osaxw6eM2XpVsKWdVRuEUahmg_E7F7zX51zg@mail.gmail.com>

On Tue, Mar 6, 2012 at 1:11 AM, Michael Foord <fuzzyman at gmail.com> wrote:

>
>
> On 5 March 2012 16:57, Matti Picus <matti.picus at gmail.com> wrote:
>
>>  Trying to install in pypy I get
>>
>> error C2065: '_Py_PackageContext' : undeclared identifier
>>
>
> What are you trying to install? It isn't py2exe is it? That hasn't got a
> chance of working with pypy because it *patches* the CPython dll (which is
> totally different from the pypy binary).
>
> All the best,
>
> Michael
>
>
>>  Matti
>>
>> _______________________________________________
>> pypy-dev mailing list
>> pypy-dev at python.org
>> http://mail.python.org/mailman/listinfo/pypy-dev
>>
>>
>
>
> --
>
> http://www.voidspace.org.uk/
>
> May you do good and not evil
> May you find forgiveness for yourself and forgive others
>
> May you share freely, never taking more than you give.
> -- the sqlite blessing http://www.sqlite.org/different.html
>
>
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev
>

If pypy is able to work with cx_Freeze, it will be a great help. Source
distribution is not always convenient.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120306/16853ce3/attachment.html>

From ivenvd at gmail.com  Tue Mar  6 07:05:39 2012
From: ivenvd at gmail.com (Iven Hsu)
Date: Tue, 6 Mar 2012 14:05:39 +0800
Subject: [pypy-dev] Ask for a point to contribute to pypy
Message-ID: <CA+y=efvecSvAX6DEUQb1PfauiFzdbFUzhgELxfUY-U7P=VeBcQ@mail.gmail.com>

Hi,

I'm a student from China, and I've chosen pypy as my thesis subject. I
wonder which part should I participate in?

My supervisor suggest me to do some performance improvements, so I could
list some data and diagrams when I'm doing the thesis defense. And the
thesis should have a topic, so all work are around it.

I have about half a year time to do a project to impove pypy in some
aspects, and write the thesis, although I have to work as an intern in
weekdays.

I'm familiar with Python and I have a little knowledge on compilers. I've
read some code and documents of pypy and get the main idea of it.

I've read the <Potential project list> in the wiki, but not sure if they
are up-to-date. I hope you have some good suggestions.


Thanks,
Iven
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120306/06aa7b5f/attachment-0001.html>

From pa.basso at gmail.com  Tue Mar  6 16:07:18 2012
From: pa.basso at gmail.com (Paolo Basso)
Date: Tue, 6 Mar 2012 16:07:18 +0100
Subject: [pypy-dev] pypy2exe
In-Reply-To: <CAMSv6X1OtFD5xMS=T+j4e4ZDvV0Z7VYENg+N6k_d1heieYAU3w@mail.gmail.com>
References: <CANeLV3Zpo8PFYd96xUSrmiBOeOxJco+bWovHvASLfPVKYxK4zg@mail.gmail.com>
	<CAMSv6X1OtFD5xMS=T+j4e4ZDvV0Z7VYENg+N6k_d1heieYAU3w@mail.gmail.com>
Message-ID: <CANeLV3YfiEt=BH47MM5E8e2cFUX6M0fWMSw8vfbZiGtpE+6tog@mail.gmail.com>

Thanks Armin for the clarification,

I hope a pypy2exe project will take place soon in the future then!

I guess that there is nor the possibility to compile pypy code into .so or
.dll libraries, right?
I mean, I know I'm probably going to say something which is not completely
in the spirit of an open source project but, no way to hide the code? This
would be (to me) a missing feature also in the OS/X and Linux context. Not
that I'm looking for a high level of encryption, even a .pyc file would
work quite well....

Cheers,

Paolo






2012/3/4 Armin Rigo <arigo at tunes.org>

> Hi Paolo,
>
> On Wed, Feb 29, 2012 at 09:57, Paolo Basso <pa.basso at gmail.com> wrote:
> > May be rpythonic (http://code.google.com/p/rpythonic/) useful?
>
> "rpythonic" is an unrelated project.
>
> > Is there or will be there something like a pypy2exe or already exists a
> step
> > by step guide I missed?
>
> No, there is no specific pypy2exe project right now.  The issue is
> caused in part from the fact that we, the core developers, are not
> Windows people.  PyPy works "as expected" on Linux and to some extent
> on OS/X in the sense that it is becoming more and more integrated in
> the normal distributions.  The same is not true on Windows, where we
> only offer a .zip of the latest release.  We need someone to seriously
> care about Windows and adapt py2exe or other tools.
>
>
> A bient?t,
>
> Armin.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120306/3044633b/attachment.html>

From workitharder at gmail.com  Wed Mar  7 17:20:34 2012
From: workitharder at gmail.com (Buck Golemon)
Date: Wed, 7 Mar 2012 08:20:34 -0800
Subject: [pypy-dev] translation crash on 64-bit
Message-ID: <CAPQx2veB+WvWjLcv7JA9+8Pd2PTRcquK6ojmSdXT51ARFucscA@mail.gmail.com>

I put the error message here:
http://pastebin.com/N1j49Xwy

-- 
Buck Golemon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120307/6f26dc70/attachment.html>

From fijall at gmail.com  Wed Mar  7 18:16:26 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 7 Mar 2012 09:16:26 -0800
Subject: [pypy-dev] translation crash on 64-bit
In-Reply-To: <CAPQx2veB+WvWjLcv7JA9+8Pd2PTRcquK6ojmSdXT51ARFucscA@mail.gmail.com>
References: <CAPQx2veB+WvWjLcv7JA9+8Pd2PTRcquK6ojmSdXT51ARFucscA@mail.gmail.com>
Message-ID: <CAK5idxRvYJ7aBWR+RqbbenGuJEzGzdG_E0NpQd-+6AHQyYKdyw@mail.gmail.com>

On Wed, Mar 7, 2012 at 8:20 AM, Buck Golemon <workitharder at gmail.com> wrote:
> I put the error message here:
> http://pastebin.com/N1j49Xwy

Hey.

That looks really obscure. Can you paste relevant piece of C code?
Also would be cool if you can use some pastebin that does not load
videos when I go there, respect poor bandwidth-restricted africans.
paste.pocoo.org is a good example

>
> --
> Buck Golemon
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev
>

From fijall at gmail.com  Wed Mar  7 20:12:15 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 7 Mar 2012 11:12:15 -0800
Subject: [pypy-dev] [pypy-commit] pypy py3k: save the source of applevel
 direct tests in a temporary file: this way, we get nicer tracebacks
In-Reply-To: <20120209151208.A21BE82B1E@wyvern.cs.uni-duesseldorf.de>
References: <20120209151208.A21BE82B1E@wyvern.cs.uni-duesseldorf.de>
Message-ID: <CAK5idxRg-dsGuMvANkvEowJJH3_zrAyiZ9OMENoVhzhQ3LpBoA@mail.gmail.com>

Can we have this on default as well please?

On Thu, Feb 9, 2012 at 7:12 AM, antocuni <noreply at buildbot.pypy.org> wrote:
> Author: Antonio Cuni <anto.cuni at gmail.com>
> Branch: py3k
> Changeset: r52297:5c66b92f0617
> Date: 2012-02-09 15:35 +0100
> http://bitbucket.org/pypy/pypy/changeset/5c66b92f0617/
>
> Log: ? ?save the source of applevel direct tests in a temporary file: this
> ? ? ? ?way, we get nicer tracebacks
>
> diff --git a/pypy/conftest.py b/pypy/conftest.py
> --- a/pypy/conftest.py
> +++ b/pypy/conftest.py
> @@ -213,8 +213,10 @@
> ? ? ? ? ? ? raise AssertionError("DID NOT RAISE")
> ?"""
> ? ? source = py.code.Source(target)[1:].deindent()
> + ? ?pyfile = udir.join('src.py')
> + ? ?pyfile.write(helpers + str(source))
> ? ? res, stdout, stderr = runsubprocess.run_subprocess(
> - ? ? ? ?python, ["-c", helpers + str(source)])
> + ? ? ? ?python, [str(pyfile)])
> ? ? print source
> ? ? print >> sys.stdout, stdout
> ? ? print >> sys.stderr, stderr
> _______________________________________________
> pypy-commit mailing list
> pypy-commit at python.org
> http://mail.python.org/mailman/listinfo/pypy-commit

From workitharder at gmail.com  Wed Mar  7 20:16:29 2012
From: workitharder at gmail.com (Buck Golemon)
Date: Wed, 7 Mar 2012 11:16:29 -0800
Subject: [pypy-dev] translation crash on 64-bit
In-Reply-To: <CAK5idxRvYJ7aBWR+RqbbenGuJEzGzdG_E0NpQd-+6AHQyYKdyw@mail.gmail.com>
References: <CAPQx2veB+WvWjLcv7JA9+8Pd2PTRcquK6ojmSdXT51ARFucscA@mail.gmail.com>
	<CAK5idxRvYJ7aBWR+RqbbenGuJEzGzdG_E0NpQd-+6AHQyYKdyw@mail.gmail.com>
Message-ID: <CAPQx2vdE6oQgFc-3D3YBC9tAHerzV+P6xPWLYfcBO2cFvCOFsQ@mail.gmail.com>

Understood. I'll use pocoo.

Poor bandwidth-restricted Africans should use
AdBlock<http://code.google.com/p/adblockforchrome/>as a
matter-of-course.

The C code is just pypy HEAD.



On Wed, Mar 7, 2012 at 9:16 AM, Maciej Fijalkowski <fijall at gmail.com> wrote:

> On Wed, Mar 7, 2012 at 8:20 AM, Buck Golemon <workitharder at gmail.com>
> wrote:
> > I put the error message here:
> > http://pastebin.com/N1j49Xwy
>
> Hey.
>
> That looks really obscure. Can you paste relevant piece of C code?
> Also would be cool if you can use some pastebin that does not load
> videos when I go there, respect poor bandwidth-restricted africans.
> paste.pocoo.org is a good example
>
> >
> > --
> > Buck Golemon
> >
> > _______________________________________________
> > pypy-dev mailing list
> > pypy-dev at python.org
> > http://mail.python.org/mailman/listinfo/pypy-dev
> >
>



-- 
Buck Golemon
Cell: (408) 202-3901
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120307/c5da746d/attachment.html>

From fijall at gmail.com  Wed Mar  7 20:19:29 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 7 Mar 2012 11:19:29 -0800
Subject: [pypy-dev] translation crash on 64-bit
In-Reply-To: <CAPQx2vdE6oQgFc-3D3YBC9tAHerzV+P6xPWLYfcBO2cFvCOFsQ@mail.gmail.com>
References: <CAPQx2veB+WvWjLcv7JA9+8Pd2PTRcquK6ojmSdXT51ARFucscA@mail.gmail.com>
	<CAK5idxRvYJ7aBWR+RqbbenGuJEzGzdG_E0NpQd-+6AHQyYKdyw@mail.gmail.com>
	<CAPQx2vdE6oQgFc-3D3YBC9tAHerzV+P6xPWLYfcBO2cFvCOFsQ@mail.gmail.com>
Message-ID: <CAK5idxTCsm5Hzdk3-xewpyZ_YebF-CVDpJ+1VmB_n4UxdnRxYA@mail.gmail.com>

On Wed, Mar 7, 2012 at 11:16 AM, Buck Golemon <workitharder at gmail.com> wrote:
> Understood. I'll use pocoo.
>
> Poor bandwidth-restricted Africans should use AdBlock as a matter-of-course.

Somehow did not catch it.

>
> The C code is just pypy HEAD.

The C code got compiled and is in your tmp. I get C code that is
compiling somehow.

From russel at winder.org.uk  Wed Mar  7 20:16:58 2012
From: russel at winder.org.uk (Russel Winder)
Date: Wed, 07 Mar 2012 19:16:58 +0000
Subject: [pypy-dev] Seeking a bit of guidance
In-Reply-To: <1328725976.9949.64.camel@anglides.russel.org.uk>
References: <1328722847.9949.31.camel@anglides.russel.org.uk>
	<CAK5idxS+KTJaL_snQzgJSoON5Zjqy7yWdVB+BR=9Rj-V4T-SMQ@mail.gmail.com>
	<1328725471.9949.60.camel@anglides.russel.org.uk>
	<1328725976.9949.64.camel@anglides.russel.org.uk>
Message-ID: <1331147818.19498.11.camel@anglides.russel.org.uk>

On Wed, 2012-02-08 at 18:32 +0000, Russel Winder wrote:
[...]
> |> pypy /home/Checkouts/Mercurial/SCons/bootstrap.py
> /home/users/russel/bin.Linux.x86_64/pypy /home/Checkouts/Mercurial/SCons/bootstrap/src/script/scons.py
> scons: Reading SConscript files ...
> ImportError: No module named uno:
[...]

Is there any chance of getting PyPy back in the Debian distribution? If
it can get in with CPython then all modules such as uno would just
appear in the PyPy tree as part of installation via the Debian packaging
system.

Thanks.

-- 
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder at ekiga.net
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel at winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120307/5a6ee625/attachment.pgp>

From fijall at gmail.com  Wed Mar  7 20:27:38 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 7 Mar 2012 11:27:38 -0800
Subject: [pypy-dev] Seeking a bit of guidance
In-Reply-To: <1331147818.19498.11.camel@anglides.russel.org.uk>
References: <1328722847.9949.31.camel@anglides.russel.org.uk>
	<CAK5idxS+KTJaL_snQzgJSoON5Zjqy7yWdVB+BR=9Rj-V4T-SMQ@mail.gmail.com>
	<1328725471.9949.60.camel@anglides.russel.org.uk>
	<1328725976.9949.64.camel@anglides.russel.org.uk>
	<1331147818.19498.11.camel@anglides.russel.org.uk>
Message-ID: <CAK5idxQPd=Z6=6RnWahM8n_V=ntKKiDFTxX6kxSFuO5gDTi_RA@mail.gmail.com>

On Wed, Mar 7, 2012 at 11:16 AM, Russel Winder <russel at winder.org.uk> wrote:
> On Wed, 2012-02-08 at 18:32 +0000, Russel Winder wrote:
> [...]
>> |> pypy /home/Checkouts/Mercurial/SCons/bootstrap.py
>> /home/users/russel/bin.Linux.x86_64/pypy /home/Checkouts/Mercurial/SCons/bootstrap/src/script/scons.py
>> scons: Reading SConscript files ...
>> ImportError: No module named uno:
> [...]
>
> Is there any chance of getting PyPy back in the Debian distribution? If
> it can get in with CPython then all modules such as uno would just
> appear in the PyPy tree as part of installation via the Debian packaging
> system.

PyPy is in debian (experimental I think), but it will *not*
automatically pick up whatever is installed on CPython, having
separate package directories.

Cheers,
fijal

From workitharder at gmail.com  Wed Mar  7 20:36:08 2012
From: workitharder at gmail.com (Buck Golemon)
Date: Wed, 7 Mar 2012 11:36:08 -0800
Subject: [pypy-dev] translation crash on 64-bit
In-Reply-To: <CAK5idxTCsm5Hzdk3-xewpyZ_YebF-CVDpJ+1VmB_n4UxdnRxYA@mail.gmail.com>
References: <CAPQx2veB+WvWjLcv7JA9+8Pd2PTRcquK6ojmSdXT51ARFucscA@mail.gmail.com>
	<CAK5idxRvYJ7aBWR+RqbbenGuJEzGzdG_E0NpQd-+6AHQyYKdyw@mail.gmail.com>
	<CAPQx2vdE6oQgFc-3D3YBC9tAHerzV+P6xPWLYfcBO2cFvCOFsQ@mail.gmail.com>
	<CAK5idxTCsm5Hzdk3-xewpyZ_YebF-CVDpJ+1VmB_n4UxdnRxYA@mail.gmail.com>
Message-ID: <CAPQx2vexfcv0wuC8ejn6jwch-0E0CTm2O6Ug4z7hXKDGeGhrQg@mail.gmail.com>

On Wed, Mar 7, 2012 at 11:19 AM, Maciej Fijalkowski <fijall at gmail.com>wrote:

> On Wed, Mar 7, 2012 at 11:16 AM, Buck Golemon <workitharder at gmail.com>
> wrote:
> > Understood. I'll use pocoo.
> >
> > Poor bandwidth-restricted Africans should use AdBlock as a
> matter-of-course.
>
> Somehow did not catch it.
>
> >
> > The C code is just pypy HEAD.
>
> The C code got compiled and is in your tmp. I get C code that is
> compiling somehow.
>


This is at home. It will be some hours before I can get you that data.
For clarity, this is the only "real" error, correct?

    [translation:ERROR]     interpreter_pyparser_pytokenizer.c:2181:2:
error: ?l_v49695? undeclared (first use in this function)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120307/63d2a372/attachment.html>

From russel at winder.org.uk  Wed Mar  7 21:42:38 2012
From: russel at winder.org.uk (Russel Winder)
Date: Wed, 07 Mar 2012 20:42:38 +0000
Subject: [pypy-dev] Seeking a bit of guidance
In-Reply-To: <CAK5idxQPd=Z6=6RnWahM8n_V=ntKKiDFTxX6kxSFuO5gDTi_RA@mail.gmail.com>
References: <1328722847.9949.31.camel@anglides.russel.org.uk>
	<CAK5idxS+KTJaL_snQzgJSoON5Zjqy7yWdVB+BR=9Rj-V4T-SMQ@mail.gmail.com>
	<1328725471.9949.60.camel@anglides.russel.org.uk>
	<1328725976.9949.64.camel@anglides.russel.org.uk>
	<1331147818.19498.11.camel@anglides.russel.org.uk>
	<CAK5idxQPd=Z6=6RnWahM8n_V=ntKKiDFTxX6kxSFuO5gDTi_RA@mail.gmail.com>
Message-ID: <1331152958.10836.6.camel@launcelot.russel.org.uk>

On Wed, 2012-03-07 at 11:27 -0800, Maciej Fijalkowski wrote:
[...]
> PyPy is in debian (experimental I think), but it will *not*
> automatically pick up whatever is installed on CPython, having
> separate package directories.

I'll see if I can install from there.

When you install a shared package such as python-uno, the install checks
for all installed Pythons and does an install for each -- so it gets
installed for Python 2.6 and Python 2.7 for me.  Also some will install
for Python 3.2 as well.  So you are right in saying that a PyPy
execution will not automatically search for stuff in the CPython trees,
but what I am saying is that shared package sources get multiply
installed, once to each appropriate tree.

-- 
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder at ekiga.net
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel at winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120307/f35bd1a1/attachment.pgp>

From russel at winder.org.uk  Wed Mar  7 22:28:48 2012
From: russel at winder.org.uk (Russel Winder)
Date: Wed, 07 Mar 2012 21:28:48 +0000
Subject: [pypy-dev] Seeking a bit of guidance
In-Reply-To: <1331152958.10836.6.camel@launcelot.russel.org.uk>
References: <1328722847.9949.31.camel@anglides.russel.org.uk>
	<CAK5idxS+KTJaL_snQzgJSoON5Zjqy7yWdVB+BR=9Rj-V4T-SMQ@mail.gmail.com>
	<1328725471.9949.60.camel@anglides.russel.org.uk>
	<1328725976.9949.64.camel@anglides.russel.org.uk>
	<1331147818.19498.11.camel@anglides.russel.org.uk>
	<CAK5idxQPd=Z6=6RnWahM8n_V=ntKKiDFTxX6kxSFuO5gDTi_RA@mail.gmail.com>
	<1331152958.10836.6.camel@launcelot.russel.org.uk>
Message-ID: <1331155728.10836.9.camel@launcelot.russel.org.uk>

On Wed, 2012-03-07 at 20:42 +0000, Russel Winder wrote:
[...]
> I'll see if I can install from there.

pypy packages install find from experimental.  Sadly the python-uno
package assumes it is only going to install for CPython 2.7 :-((((

I'll tinker more and see what I can do.

Now to start the campaign to get PyPy moved from experimental to
unstable...

-- 
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder at ekiga.net
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel at winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120307/91f11d8e/attachment.pgp>

From stefan_ml at behnel.de  Thu Mar  8 14:31:38 2012
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 08 Mar 2012 14:31:38 +0100
Subject: [pypy-dev] Py_DecRef() in cpyext) -> GSoC 2012?
In-Reply-To: <CAMSv6X33V=qqL36FkrWoUnVHQQkzbkveQH+VPP=wrJ4p857pTQ@mail.gmail.com>
References: <jicro3$ok5$1@dough.gmane.org>
	<CAMSv6X33V=qqL36FkrWoUnVHQQkzbkveQH+VPP=wrJ4p857pTQ@mail.gmail.com>
Message-ID: <jjacbq$5p0$1@dough.gmane.org>

Armin Rigo, 26.02.2012 11:09:
> On Sun, Feb 26, 2012 at 09:50, Stefan Behnel wrote:
>> Looking at Py_DecRef(), however, left me somewhat baffled. I would have
>> expected this to be the most intensively tuned function in all of cpyext,
>> but it even started with this comment: (...)
> 
> Indeed, it's an obvious starting place if we want to optimize cpyext
> (which did not occur at all so far).  You are welcome to try.

Wouldn't optimising and improving cpyext be a perfect topic for a GSoC student?

Stefan


From kracethekingmaker at gmail.com  Thu Mar  8 19:19:08 2012
From: kracethekingmaker at gmail.com (kracethekingmaker)
Date: Thu, 08 Mar 2012 23:49:08 +0530
Subject: [pypy-dev] Online Python interpreter on top of PyPy
Message-ID: <4F58F81C.2040906@gmail.com>

Hi PyPyians

   Thanks to your support for making this happen. Especially to fijal, 
who was patient enough to answer my question.

www.pylive.codespeaks.in, an online python interpreter(strictly pypy 
interpreter) on top pypy sandboxed version 1.7. Source code of the site 
is located @ www.github.com/kracekumar/pylive, just feel free to try and 
post your feedback, in case pypy community wants any analysis or what 
kind of code ppl try, I can share with the group, please feel free to 
comment.

-- 
"Talk is cheap, show me the code" -- Linus Torvalds
Regards
Kracekumar.R
www.kracekumar.com


From fijall at gmail.com  Thu Mar  8 19:52:54 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Thu, 8 Mar 2012 10:52:54 -0800
Subject: [pypy-dev] Online Python interpreter on top of PyPy
In-Reply-To: <4F58F81C.2040906@gmail.com>
References: <4F58F81C.2040906@gmail.com>
Message-ID: <CAK5idxSx4kjeeC0F+rUuaH6TppqjFZ1zwm5wE=tkZbPqeg2PZw@mail.gmail.com>

On Thu, Mar 8, 2012 at 10:19 AM, kracethekingmaker
<kracethekingmaker at gmail.com> wrote:
> Hi PyPyians
>
> ?Thanks to your support for making this happen. Especially to fijal, who was
> patient enough to answer my question.
>
> www.pylive.codespeaks.in, an online python interpreter(strictly pypy
> interpreter) on top pypy sandboxed version 1.7. Source code of the site is
> located @ www.github.com/kracekumar/pylive, just feel free to try and post
> your feedback, in case pypy community wants any analysis or what kind of
> code ppl try, I can share with the group, please feel free to comment.
>
> --
> "Talk is cheap, show me the code" -- Linus Torvalds
> Regards
> Kracekumar.R
> www.kracekumar.com

502 bad gateway?

Cheers,
fijal

From piotr.skamruk at gmail.com  Thu Mar  8 19:57:51 2012
From: piotr.skamruk at gmail.com (Piotr Skamruk)
Date: Thu, 8 Mar 2012 19:57:51 +0100
Subject: [pypy-dev] Online Python interpreter on top of PyPy
In-Reply-To: <4F58F81C.2040906@gmail.com>
References: <4F58F81C.2040906@gmail.com>
Message-ID: <CAO_YSZMoYvzt2u=z17zav+CxubCDaTjA=8ZTVCrQTzPaRFxTDA@mail.gmail.com>

2012/3/8 kracethekingmaker <kracethekingmaker at gmail.com>:
> Hi PyPyians
>
> ?Thanks to your support for making this happen. Especially to fijal, who was
> patient enough to answer my question.
>
> www.pylive.codespeaks.in, an online python interpreter(strictly pypy
looks like application had bad times...
502 from webserver in front of it...

> interpreter) on top pypy sandboxed version 1.7. Source code of the site is
> located @ www.github.com/kracekumar/pylive, just feel free to try and post
> your feedback, in case pypy community wants any analysis or what kind of
> code ppl try, I can share with the group, please feel free to comment.
>
> --
> "Talk is cheap, show me the code" -- Linus Torvalds
> Regards
> Kracekumar.R
> www.kracekumar.com
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev

From kracethekingmaker at gmail.com  Thu Mar  8 20:01:39 2012
From: kracethekingmaker at gmail.com (kracethekingmaker)
Date: Fri, 09 Mar 2012 00:31:39 +0530
Subject: [pypy-dev] Online Python interpreter on top of PyPy
In-Reply-To: <CAO_YSZMoYvzt2u=z17zav+CxubCDaTjA=8ZTVCrQTzPaRFxTDA@mail.gmail.com>
References: <4F58F81C.2040906@gmail.com>
	<CAO_YSZMoYvzt2u=z17zav+CxubCDaTjA=8ZTVCrQTzPaRFxTDA@mail.gmail.com>
Message-ID: <4F590213.80400@gmail.com>

On 03/09/2012 12:27 AM, Piotr Skamruk wrote:
> 2012/3/8 kracethekingmaker<kracethekingmaker at gmail.com>:
>> Hi PyPyians
>>
>>   Thanks to your support for making this happen. Especially to fijal, who was
>> patient enough to answer my question.
>>
>> www.pylive.codespeaks.in, an online python interpreter(strictly pypy
> looks like application had bad times...
> 502 from webserver in front of it...

Sorry, I did some mistake with git stash pop, I apologise. so whole 
system was corrupted, Thanks for your time. Now system is up. Kindly try.
>> interpreter) on top pypy sandboxed version 1.7. Source code of the site is
>> located @ www.github.com/kracekumar/pylive, just feel free to try and post
>> your feedback, in case pypy community wants any analysis or what kind of
>> code ppl try, I can share with the group, please feel free to comment.
>>
>> --
>> "Talk is cheap, show me the code" -- Linus Torvalds
>> Regards
>> Kracekumar.R
>> www.kracekumar.com
>>
>> _______________________________________________
>> pypy-dev mailing list
>> pypy-dev at python.org
>> http://mail.python.org/mailman/listinfo/pypy-dev


-- 
"Talk is cheap, show me the code" -- Linus Torvalds
Regards
Kracekumar.R
www.kracekumar.com


From haael at interia.pl  Thu Mar  8 21:26:29 2012
From: haael at interia.pl (haael)
Date: Thu, 08 Mar 2012 21:26:29 +0100
Subject: [pypy-dev] Hello, guys
Message-ID: <4F5915F5.40101@interia.pl>


I'm new to PyPy. I'm mostly interested in ARM architecture. My ultimate goal is 
to run Python interpreter on a bare CPU without an operating system, but for 
now I just want to compile PyPy using arm-eabi-gcc. It does not have to work, 
much less support anything, but my question is: what is the status of ARM 
target? There's nothing about it in the documentation.

haael



From s.parmesan at gmail.com  Thu Mar  8 23:12:14 2012
From: s.parmesan at gmail.com (Stefano Parmesan)
Date: Thu, 8 Mar 2012 23:12:14 +0100
Subject: [pypy-dev] Hello, guys
In-Reply-To: <4F5915F5.40101@interia.pl>
References: <4F5915F5.40101@interia.pl>
Message-ID: <CACtfbMR+4Gmvytni=-Ff9Y=C_=XJm_cUmzzDiWWrkW16odQLLw@mail.gmail.com>

Take a look at morepypy and enjoy :-)

http://morepypy.blogspot.com/search?q=arm

--
Dott. Stefano Parmesan
PhD Student ~ University of Trento
<ahref Foundation - Trento - Italy
Tel: 0461-235794 ext. 5544



On 8 March 2012 21:26, haael <haael at interia.pl> wrote:

>
> I'm new to PyPy. I'm mostly interested in ARM architecture. My ultimate
> goal is to run Python interpreter on a bare CPU without an operating
> system, but for now I just want to compile PyPy using arm-eabi-gcc. It does
> not have to work, much less support anything, but my question is: what is
> the status of ARM target? There's nothing about it in the documentation.
>
> haael
>
>
> ______________________________**_________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/**mailman/listinfo/pypy-dev<http://mail.python.org/mailman/listinfo/pypy-dev>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120308/e703b534/attachment-0001.html>

From fijall at gmail.com  Fri Mar  9 01:12:20 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Thu, 8 Mar 2012 16:12:20 -0800
Subject: [pypy-dev] Online Python interpreter on top of PyPy
In-Reply-To: <4F590213.80400@gmail.com>
References: <4F58F81C.2040906@gmail.com>
	<CAO_YSZMoYvzt2u=z17zav+CxubCDaTjA=8ZTVCrQTzPaRFxTDA@mail.gmail.com>
	<4F590213.80400@gmail.com>
Message-ID: <CAK5idxSDyQbqpPDt7avv1eMGwN8GxF4WMAoykgYtXCD99VnHGQ@mail.gmail.com>

On Thu, Mar 8, 2012 at 11:01 AM, kracethekingmaker
<kracethekingmaker at gmail.com> wrote:
> On 03/09/2012 12:27 AM, Piotr Skamruk wrote:
>>
>> 2012/3/8 kracethekingmaker<kracethekingmaker at gmail.com>:
>>>
>>> Hi PyPyians
>>>
>>> ?Thanks to your support for making this happen. Especially to fijal, who
>>> was
>>> patient enough to answer my question.
>>>
>>> www.pylive.codespeaks.in, an online python interpreter(strictly pypy
>>
>> looks like application had bad times...
>> 502 from webserver in front of it...
>
>
> Sorry, I did some mistake with git stash pop, I apologise. so whole system
> was corrupted, Thanks for your time. Now system is up. Kindly try.
>>>
>>> interpreter) on top pypy sandboxed version 1.7. Source code of the site
>>> is
>>>
>>> located @ www.github.com/kracekumar/pylive, just feel free to try and
>>> post
>>> your feedback, in case pypy community wants any analysis or what kind of
>>> code ppl try, I can share with the group, please feel free to comment.
>>>
>>> --
>>> "Talk is cheap, show me the code" -- Linus Torvalds
>>> Regards
>>> Kracekumar.R
>>> www.kracekumar.com
>>>
>>> _______________________________________________
>>> pypy-dev mailing list
>>> pypy-dev at python.org
>>> http://mail.python.org/mailman/listinfo/pypy-dev
>
>
>
> --
> "Talk is cheap, show me the code" -- Linus Torvalds
> Regards
> Kracekumar.R
> www.kracekumar.com
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev

Hey

you don't correctly escape the output. So a print '<h1>' would do
exactly that ;-)

Cheers,
fijal

From arigo at tunes.org  Fri Mar  9 17:40:10 2012
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 9 Mar 2012 08:40:10 -0800
Subject: [pypy-dev] Online Python interpreter on top of PyPy
In-Reply-To: <CAK5idxSDyQbqpPDt7avv1eMGwN8GxF4WMAoykgYtXCD99VnHGQ@mail.gmail.com>
References: <4F58F81C.2040906@gmail.com>
	<CAO_YSZMoYvzt2u=z17zav+CxubCDaTjA=8ZTVCrQTzPaRFxTDA@mail.gmail.com>
	<4F590213.80400@gmail.com>
	<CAK5idxSDyQbqpPDt7avv1eMGwN8GxF4WMAoykgYtXCD99VnHGQ@mail.gmail.com>
Message-ID: <CAMSv6X23QQS53LF+C-hMPuqwomJA1tLysA6zs9HQR6dgDxmBnQ@mail.gmail.com>

Hi,

On Thu, Mar 8, 2012 at 16:12, Maciej Fijalkowski <fijall at gmail.com> wrote:
>> Sorry, I did some mistake with git stash pop, I apologise. so whole system
>> was corrupted, Thanks for your time. Now system is up. Kindly try.

It looks like a nice demo :-)  The system has limits of 200MB or 60
seconds.  For a pypy-sandbox, the only thing that needs to be done
carefully is what occurs if a lot of requests come at the same time.
Does your demo run only one of them and queue or discard the other
ones?

> you don't correctly escape the output. So a print '<h1>' would do
> exactly that ;-)

Indeed, it's annoying when you try to print e.g. 'sys.modules'.  But
it's not really a security bug.


A bient?t,

Armin.

From kracethekingmaker at gmail.com  Fri Mar  9 18:32:32 2012
From: kracethekingmaker at gmail.com (kracethekingmaker)
Date: Fri, 09 Mar 2012 23:02:32 +0530
Subject: [pypy-dev] Online Python interpreter on top of PyPy
In-Reply-To: <CAMSv6X23QQS53LF+C-hMPuqwomJA1tLysA6zs9HQR6dgDxmBnQ@mail.gmail.com>
References: <4F58F81C.2040906@gmail.com>
	<CAO_YSZMoYvzt2u=z17zav+CxubCDaTjA=8ZTVCrQTzPaRFxTDA@mail.gmail.com>
	<4F590213.80400@gmail.com>
	<CAK5idxSDyQbqpPDt7avv1eMGwN8GxF4WMAoykgYtXCD99VnHGQ@mail.gmail.com>
	<CAMSv6X23QQS53LF+C-hMPuqwomJA1tLysA6zs9HQR6dgDxmBnQ@mail.gmail.com>
Message-ID: <4F5A3EB0.80508@gmail.com>

Hi
> Hi,
>
> On Thu, Mar 8, 2012 at 16:12, Maciej Fijalkowski<fijall at gmail.com>  wrote:
>>> Sorry, I did some mistake with git stash pop, I apologise. so whole system
>>> was corrupted, Thanks for your time. Now system is up. Kindly try.
> It looks like a nice demo :-)  The system has limits of 200MB or 60
> seconds.  For a pypy-sandbox, the only thing that needs to be done
> carefully is what occurs if a lot of requests come at the same time.
> Does your demo run only one of them and queue or discard the other
> ones?
Thanks :-) . System has 175MB and 60 seconds timeout as well. I run the 
flask site using gunicorn, the command goes
gunicorn pylive:app --workers=2 --access-logfile 
/path/to/pylive/app_log.log --error-logfile 
/path/to/pylive/error_log.log --log-level debug  --timeout=120 --daemon 
--bind=127.0.0.1:46793 --pid=/path/to/pylive/.pid

Currently total worker space is around 50MB for 2 workers, 175 MB for 
single pypy-sandbox , so 50 + 250MB(300MB), so if server receives more 
than 2 concurrent request, it should be handled by gunicorn(I might be 
wrong). Yes there are chances request will get lost, I am working on 
this area, to separate app and sandbox to different system and introduce 
queuing system, I will be increasing no of workers as well as memory.
>> you don't correctly escape the output. So a print '<h1>' would do
>> exactly that ;-)
> Indeed, it's annoying when you try to print e.g. 'sys.modules'.  But
> it's not really a security bug.
>
I have fixed parsing html tags.

Yes its not security bug since its impossible to write to particular 
location in filesystem and read.

The below code also fails
with open('a.txt', 'w') as f:
     f.write('I am hacker')

with open('a.txt', 'r') as f:
     print f.readlines()



> A bient?t,
>
> Armin.


-- 
"Talk is cheap, show me the code" -- Linus Torvalds
Regards
Kracekumar.R
www.kracekumar.com


From salexandre.bm at gmail.com  Sat Mar 10 17:55:49 2012
From: salexandre.bm at gmail.com (Alexandre Bm)
Date: Sat, 10 Mar 2012 17:55:49 +0100
Subject: [pypy-dev] HTMLParser, patch
Message-ID: <CAFEoyoUei4v10Bj_F2A=cKL-avEP=7KJMF0sVMgg+K6Tn-YzVg@mail.gmail.com>

Hello everyone, i want to submit a patch to HTMLParser.py, is there any one
who know where can i submit it ?
Thanks !

-- 
Nobless Oblige !
Alexandre BM
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120310/f4fe6353/attachment.html>

From amauryfa at gmail.com  Sat Mar 10 18:08:03 2012
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Sat, 10 Mar 2012 18:08:03 +0100
Subject: [pypy-dev] HTMLParser, patch
In-Reply-To: <CAFEoyoUei4v10Bj_F2A=cKL-avEP=7KJMF0sVMgg+K6Tn-YzVg@mail.gmail.com>
References: <CAFEoyoUei4v10Bj_F2A=cKL-avEP=7KJMF0sVMgg+K6Tn-YzVg@mail.gmail.com>
Message-ID: <CAGmFidYC7RmaQyUaqCnkWvrrUgjMj2t=xQoR=kF7Ag2=k45SRA@mail.gmail.com>

Hello Alexandre,

2012/3/10 Alexandre Bm <salexandre.bm at gmail.com>:
> Hello everyone, i want to submit a patch to HTMLParser.py, is there any one
> who know where can i submit it ?
> Thanks !

The simplest is probably to file a new issue in bugs.pypy.org.
Alternatively, you can also fork the mercurial repository, and submit
a Pull Request.

This said, HTMLParser.py is part of the Python standard library,
would your patch also apply to the "official" CPython implementation?

Cheers,

-- 
Amaury Forgeot d'Arc

From fijall at gmail.com  Sat Mar 10 18:09:34 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sat, 10 Mar 2012 09:09:34 -0800
Subject: [pypy-dev] HTMLParser, patch
In-Reply-To: <CAFEoyoUei4v10Bj_F2A=cKL-avEP=7KJMF0sVMgg+K6Tn-YzVg@mail.gmail.com>
References: <CAFEoyoUei4v10Bj_F2A=cKL-avEP=7KJMF0sVMgg+K6Tn-YzVg@mail.gmail.com>
Message-ID: <CAK5idxT1=o+VKJ8R=JJUiZWLkbOUBGGPJ1wtHt63TAOcVd=kbA@mail.gmail.com>

On Sat, Mar 10, 2012 at 8:55 AM, Alexandre Bm <salexandre.bm at gmail.com> wrote:
> Hello everyone, i want to submit a patch to HTMLParser.py, is there any one
> who know where can i submit it ?
> Thanks !

Bug tracker, bugs.pypy.org preferably

From bphillipsen at baden.ch  Sat Mar 10 20:48:25 2012
From: bphillipsen at baden.ch (Bent Phillipsen)
Date: Sat, 10 Mar 2012 20:48:25 +0100
Subject: [pypy-dev] Excellent work!
Message-ID: <56532.1331408905@baden.ch>

Thank you for making pypy available. It is really excellent work!

I tried pypy on some numerical analysis algorithms and compered to fortran and cpython:

On a loop over romberg-integrations I found pypy 2.3 times faster than gfortran without optimization and 2.1 times faster than gfortran with 
optimization (yes, pypy faster than fortran!). Compared to cpython, pypy is about 25 times faster. The results of the computations with pypy and 
cpython are identical.

On a loop over akima-interpolations pypy is 5 times slower than gfortran, but about 20 times faster than cpython.

These results are really impressive. With pypy numerical analysis can be performed efficiently in pure python using lists as arrays (i.e. without 
using numpy). 

By the way: I have observed, that the assert statement is not fully implemented in pypy (which possibly is well known). In cpython with the -O 
option no bytecode is generated for the assert statement. In pypy the same bytecode is generated with and without the -O option. 

Best regards

Bent Phillipsen  










From benjamin at python.org  Sat Mar 10 21:17:08 2012
From: benjamin at python.org (Benjamin Peterson)
Date: Sat, 10 Mar 2012 12:17:08 -0800
Subject: [pypy-dev] Excellent work!
In-Reply-To: <56532.1331408905@baden.ch>
References: <56532.1331408905@baden.ch>
Message-ID: <CAPZV6o8jM=H_X_j8iPVwV-0QBkhtbrm5h5xvU5vJqVa-PuDDFA@mail.gmail.com>

2012/3/10 Bent Phillipsen <bphillipsen at baden.ch>:
> These results are really impressive. With pypy numerical analysis can be performed efficiently in pure python using lists as arrays (i.e. without
> using numpy).

That's great to hear!

>
> By the way: I have observed, that the assert statement is not fully implemented in pypy (which possibly is well known). In cpython with the -O
> option no bytecode is generated for the assert statement. In pypy the same bytecode is generated with and without the -O option.

We don't really do much with the -O option.

-- 
Regards,
Benjamin

From matti.picus at gmail.com  Sat Mar 10 21:40:45 2012
From: matti.picus at gmail.com (Matti Picus)
Date: Sat, 10 Mar 2012 22:40:45 +0200
Subject: [pypy-dev] How to get PIL to install?
Message-ID: <4F5BBC4D.8000106@gmail.com>

I created a fork of Pillow at git://github.com/mattip/Pillow.git for 
win32 Using a pypy nightly and the github code should allow you to 
"setup.py install"  Pillow for win32. Note you may need some external 
libraries like libjpeg and zlib to get full functionality.

I also fixed a bug in pypy so that "setup.py bdist_wininst" works, maybe 
when 1.8.1 is released we can think about how to make binary packages 
publicly available?

mattip


From arigo at tunes.org  Sat Mar 10 22:52:53 2012
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 10 Mar 2012 13:52:53 -0800
Subject: [pypy-dev] Excellent work!
In-Reply-To: <56532.1331408905@baden.ch>
References: <56532.1331408905@baden.ch>
Message-ID: <CAMSv6X1UawgsszYsLLi_X7+sqf3fYfsL4jxQGK6Gh3RF9WsHmA@mail.gmail.com>

Hi Bent,

On Sat, Mar 10, 2012 at 11:48, Bent Phillipsen <bphillipsen at baden.ch> wrote:
> Thank you for making pypy available. It is really excellent work!

Thanks!  :-)

> By the way: I have observed, that the assert statement is not fully implemented in pypy (which possibly is well known). In cpython with the -O
> option no bytecode is generated for the assert statement. In pypy the same bytecode is generated with and without the -O option.

Indeed, "pypy --help" says "-O: dummy optimization flag for
compatibility with C Python".  Maybe it's a difference that we need to
fix at some point.  We never use "-O" ourselves, if only because you
can't translate pypy with this option --- the asserts in PyPy's own
source code are essential :-)


A bient?t,

Armin.

From stefan_ml at behnel.de  Sun Mar 11 08:43:02 2012
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sun, 11 Mar 2012 08:43:02 +0100
Subject: [pypy-dev] HTMLParser, patch
In-Reply-To: <CAK5idxT1=o+VKJ8R=JJUiZWLkbOUBGGPJ1wtHt63TAOcVd=kbA@mail.gmail.com>
References: <CAFEoyoUei4v10Bj_F2A=cKL-avEP=7KJMF0sVMgg+K6Tn-YzVg@mail.gmail.com>
	<CAK5idxT1=o+VKJ8R=JJUiZWLkbOUBGGPJ1wtHt63TAOcVd=kbA@mail.gmail.com>
Message-ID: <jjhl26$e1g$2@dough.gmane.org>

Maciej Fijalkowski, 10.03.2012 18:09:
> On Sat, Mar 10, 2012 at 8:55 AM, Alexandre Bm wrote:
>> Hello everyone, i want to submit a patch to HTMLParser.py, is there any one
>> who know where can i submit it ?
>> Thanks !
> 
> Bug tracker, bugs.pypy.org preferably

More likely bugs.python.org.

Stefan


From fijall at gmail.com  Sun Mar 11 09:00:55 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sun, 11 Mar 2012 00:00:55 -0800
Subject: [pypy-dev] HTMLParser, patch
In-Reply-To: <jjhl26$e1g$2@dough.gmane.org>
References: <CAFEoyoUei4v10Bj_F2A=cKL-avEP=7KJMF0sVMgg+K6Tn-YzVg@mail.gmail.com>
	<CAK5idxT1=o+VKJ8R=JJUiZWLkbOUBGGPJ1wtHt63TAOcVd=kbA@mail.gmail.com>
	<jjhl26$e1g$2@dough.gmane.org>
Message-ID: <CAK5idxQFiusqRRd9d84yFy5QVfc7pmSMXitA0QdBYQRxPkbuyQ@mail.gmail.com>

On Sat, Mar 10, 2012 at 11:43 PM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Maciej Fijalkowski, 10.03.2012 18:09:
>> On Sat, Mar 10, 2012 at 8:55 AM, Alexandre Bm wrote:
>>> Hello everyone, i want to submit a patch to HTMLParser.py, is there any one
>>> who know where can i submit it ?
>>> Thanks !
>>
>> Bug tracker, bugs.pypy.org preferably
>
> More likely bugs.python.org.
>
> Stefan

Yes indeed.

Too fried on the conference, sorry :/

From haael at interia.pl  Sun Mar 11 11:50:42 2012
From: haael at interia.pl (haael)
Date: Sun, 11 Mar 2012 11:50:42 +0100
Subject: [pypy-dev] Hello, guys
In-Reply-To: <CACtfbMR+4Gmvytni=-Ff9Y=C_=XJm_cUmzzDiWWrkW16odQLLw@mail.gmail.com>
References: <4F5915F5.40101@interia.pl>
	<CACtfbMR+4Gmvytni=-Ff9Y=C_=XJm_cUmzzDiWWrkW16odQLLw@mail.gmail.com>
Message-ID: <4F5C8382.6060707@interia.pl>


Cool :). But is there anything to download to play with?

haael

> Take a look at morepypy and enjoy :-)
>
> http://morepypy.blogspot.com/search?q=arm
>
> --
> Dott. Stefano Parmesan
> PhD Student ~ University of Trento
> <ahref Foundation - Trento - Italy
> Tel: 0461-235794 ext. 5544
>
>
>
> On 8 March 2012 21:26, haael<haael at interia.pl>  wrote:
>
>>
>> I'm new to PyPy. I'm mostly interested in ARM architecture. My ultimate
>> goal is to run Python interpreter on a bare CPU without an operating
>> system, but for now I just want to compile PyPy using arm-eabi-gcc. It does
>> not have to work, much less support anything, but my question is: what is
>> the status of ARM target? There's nothing about it in the documentation.
>>
>> haael
>>
>>
>> ______________________________**_________________
>> pypy-dev mailing list
>> pypy-dev at python.org
>> http://mail.python.org/**mailman/listinfo/pypy-dev<http://mail.python.org/mailman/listinfo/pypy-dev>
>>
>


From stefan_ml at behnel.de  Sun Mar 11 17:27:31 2012
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sun, 11 Mar 2012 17:27:31 +0100
Subject: [pypy-dev] "import signal" fails when running PyPy interpreted
Message-ID: <jjijpk$frl$1@dough.gmane.org>

Hi,

I get an ImportError when trying to import the "signal" module in an
interpreted PyPy (latest source checkout):

"""
python pypy/bin/py.py  -c 'import signal'

[platform:execute] gcc -c -O3 -pthread -fomit-frame-pointer -Wall
-Wno-unused -march=core2 -O3 -msse4.1 -pipe -fPIC
/tmp/usession-default-4/platcheck_2.c -o /tmp/usession-default-4/platcheck_2.o
[platform:execute] gcc /tmp/usession-default-4/platcheck_2.o -pthread
-lintl -lrt -o /tmp/usession-default-4/platcheck_2
[platform:execute] gcc -c -O3 -pthread -fomit-frame-pointer -Wall
-Wno-unused -march=core2 -O3 -msse4.1 -pipe -fPIC
/tmp/usession-default-4/platcheck_7.c -o /tmp/usession-default-4/platcheck_7.o
[platform:execute] gcc -c -O3 -pthread -fomit-frame-pointer -Wall
-Wno-unused -march=core2 -O3 -msse4.1 -pipe -fPIC
/tmp/usession-default-4/platcheck_8.c -o /tmp/usession-default-4/platcheck_8.o
[platform:execute] gcc -c -O3 -pthread -fomit-frame-pointer -Wall
-Wno-unused -march=core2 -O3 -msse4.1 -pipe -fPIC
/tmp/usession-default-4/platcheck_14.c -o
/tmp/usession-default-4/platcheck_14.o
[platform:execute] gcc -c -O3 -pthread -fomit-frame-pointer -Wall
-Wno-unused -march=core2 -O3 -msse4.1 -pipe -fPIC
-I/ablage/software/Python/pypy-src/pypy/translator/c
/tmp/usession-default-4/platcheck_18.c -o
/tmp/usession-default-4/platcheck_18.o
[platform:execute] gcc -c -O3 -pthread -fomit-frame-pointer -Wall
-Wno-unused -march=core2 -O3 -msse4.1 -pipe -fPIC
-I/ablage/software/Python/pypy-src/pypy/translator/c
/tmp/usession-default-4/module_cache/module_0.c -o
/tmp/usession-default-4/module_cache/module_0.o
[platform:execute] gcc /tmp/usession-default-4/platcheck_18.o
/tmp/usession-default-4/module_cache/module_0.o -pthread
-Wl,--export-dynamic,--version-script=/tmp/usession-default-4/dynamic-symbols-0
-lrt -o /tmp/usession-default-4/platcheck_18
faking <type 'module'>
faking <type 'member_descriptor'>
Traceback (application-level):
  File "<string>", line 1 in <module>
ImportError: No module named signal
"""

Any ideas what's broken here?

I noticed this when trying to run a test from Cython's test suite in an
interpreted PyPy, and unittest failed to "import signal".

Stefan


From fijall at gmail.com  Sun Mar 11 17:30:45 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sun, 11 Mar 2012 09:30:45 -0700
Subject: [pypy-dev] "import signal" fails when running PyPy interpreted
In-Reply-To: <jjijpk$frl$1@dough.gmane.org>
References: <jjijpk$frl$1@dough.gmane.org>
Message-ID: <CAK5idxR6u5wT1ufLe-_H6pmYC_RdoysUJMXb1kdoOkPi1g+eMw@mail.gmail.com>

On Sun, Mar 11, 2012 at 9:27 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Hi,
>
> I get an ImportError when trying to import the "signal" module in an
> interpreted PyPy (latest source checkout):
>
> """
> python pypy/bin/py.py ?-c 'import signal'
>
> [platform:execute] gcc -c -O3 -pthread -fomit-frame-pointer -Wall
> -Wno-unused -march=core2 -O3 -msse4.1 -pipe -fPIC
> /tmp/usession-default-4/platcheck_2.c -o /tmp/usession-default-4/platcheck_2.o
> [platform:execute] gcc /tmp/usession-default-4/platcheck_2.o -pthread
> -lintl -lrt -o /tmp/usession-default-4/platcheck_2
> [platform:execute] gcc -c -O3 -pthread -fomit-frame-pointer -Wall
> -Wno-unused -march=core2 -O3 -msse4.1 -pipe -fPIC
> /tmp/usession-default-4/platcheck_7.c -o /tmp/usession-default-4/platcheck_7.o
> [platform:execute] gcc -c -O3 -pthread -fomit-frame-pointer -Wall
> -Wno-unused -march=core2 -O3 -msse4.1 -pipe -fPIC
> /tmp/usession-default-4/platcheck_8.c -o /tmp/usession-default-4/platcheck_8.o
> [platform:execute] gcc -c -O3 -pthread -fomit-frame-pointer -Wall
> -Wno-unused -march=core2 -O3 -msse4.1 -pipe -fPIC
> /tmp/usession-default-4/platcheck_14.c -o
> /tmp/usession-default-4/platcheck_14.o
> [platform:execute] gcc -c -O3 -pthread -fomit-frame-pointer -Wall
> -Wno-unused -march=core2 -O3 -msse4.1 -pipe -fPIC
> -I/ablage/software/Python/pypy-src/pypy/translator/c
> /tmp/usession-default-4/platcheck_18.c -o
> /tmp/usession-default-4/platcheck_18.o
> [platform:execute] gcc -c -O3 -pthread -fomit-frame-pointer -Wall
> -Wno-unused -march=core2 -O3 -msse4.1 -pipe -fPIC
> -I/ablage/software/Python/pypy-src/pypy/translator/c
> /tmp/usession-default-4/module_cache/module_0.c -o
> /tmp/usession-default-4/module_cache/module_0.o
> [platform:execute] gcc /tmp/usession-default-4/platcheck_18.o
> /tmp/usession-default-4/module_cache/module_0.o -pthread
> -Wl,--export-dynamic,--version-script=/tmp/usession-default-4/dynamic-symbols-0
> -lrt -o /tmp/usession-default-4/platcheck_18
> faking <type 'module'>
> faking <type 'member_descriptor'>
> Traceback (application-level):
> ?File "<string>", line 1 in <module>
> ImportError: No module named signal
> """
>
> Any ideas what's broken here?
>
> I noticed this when trying to run a test from Cython's test suite in an
> interpreted PyPy, and unittest failed to "import signal".
>
> Stefan
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev

run with --allworkingmodules or with --withmod-signal, otherwise it
does not come with any modules.

From stefan_ml at behnel.de  Sun Mar 11 17:36:37 2012
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sun, 11 Mar 2012 17:36:37 +0100
Subject: [pypy-dev] "import signal" fails when running PyPy interpreted
In-Reply-To: <CAK5idxR6u5wT1ufLe-_H6pmYC_RdoysUJMXb1kdoOkPi1g+eMw@mail.gmail.com>
References: <jjijpk$frl$1@dough.gmane.org>
	<CAK5idxR6u5wT1ufLe-_H6pmYC_RdoysUJMXb1kdoOkPi1g+eMw@mail.gmail.com>
Message-ID: <jjikal$jgn$1@dough.gmane.org>

Maciej Fijalkowski, 11.03.2012 17:30:
> On Sun, Mar 11, 2012 at 9:27 AM, Stefan Behnel wrote:
>> I get an ImportError when trying to import the "signal" module in an
>> interpreted PyPy (latest source checkout):
>>
>> """
>> python pypy/bin/py.py  -c 'import signal'
>> [...]
>> Traceback (application-level):
>>  File "<string>", line 1 in <module>
>> ImportError: No module named signal
>> """
>>
>> Any ideas what's broken here?
>>
>> I noticed this when trying to run a test from Cython's test suite in an
>> interpreted PyPy, and unittest failed to "import signal".
> 
> run with --allworkingmodules or with --withmod-signal, otherwise it
> does not come with any modules.

Interesting. Would be worth adding a comment on that to this doc section:

http://readthedocs.org/docs/pypy/en/latest/getting-started-python.html#running-the-python-interpreter-without-translation

Stefan


From stefan_ml at behnel.de  Sun Mar 11 18:34:39 2012
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sun, 11 Mar 2012 18:34:39 +0100
Subject: [pypy-dev] "import signal" fails when running PyPy interpreted
In-Reply-To: <CAK5idxR6u5wT1ufLe-_H6pmYC_RdoysUJMXb1kdoOkPi1g+eMw@mail.gmail.com>
References: <jjijpk$frl$1@dough.gmane.org>
	<CAK5idxR6u5wT1ufLe-_H6pmYC_RdoysUJMXb1kdoOkPi1g+eMw@mail.gmail.com>
Message-ID: <jjinnf$bih$1@dough.gmane.org>

Maciej Fijalkowski, 11.03.2012 17:30:
> On Sun, Mar 11, 2012 at 9:27 AM, Stefan Behnel wrote:
>> I get an ImportError when trying to import the "signal" module in an
>> interpreted PyPy (latest source checkout):
>>
>> """
>> python pypy/bin/py.py  -c 'import signal'
>> [...]
>> Traceback (application-level):
>>  File "<string>", line 1 in <module>
>> ImportError: No module named signal
>> """
>>
>> Any ideas what's broken here?
>>
>> I noticed this when trying to run a test from Cython's test suite in an
>> interpreted PyPy, and unittest failed to "import signal".
>
> run with --allworkingmodules or with --withmod-signal, otherwise it
> does not come with any modules.

Ok, that got me past the error above. The next one (after a couple of
minutes) is this:

Traceback (application-level):
  File "runtests.py", line 8 in <module>
    import shutil
RuntimeError: not holding the import lock

It keeps running after that, but then the test runs fail with the same
error. I tried running it in CPython and PyPy, same error in both, which
hints at a PyPy bug.

Anyway, given that it takes more than 10 minutes to get the test runner to
the point that it prints what tests it is going to run, and then another
lunchtime to actually run even one of them, I find it totally infeasible to
debug anything with this setup. Guess I'll just resign and let others do
that job.

Stefan


From arigo at tunes.org  Sun Mar 11 19:40:22 2012
From: arigo at tunes.org (Armin Rigo)
Date: Sun, 11 Mar 2012 11:40:22 -0700
Subject: [pypy-dev] "import signal" fails when running PyPy interpreted
In-Reply-To: <jjinnf$bih$1@dough.gmane.org>
References: <jjijpk$frl$1@dough.gmane.org>
	<CAK5idxR6u5wT1ufLe-_H6pmYC_RdoysUJMXb1kdoOkPi1g+eMw@mail.gmail.com>
	<jjinnf$bih$1@dough.gmane.org>
Message-ID: <CAMSv6X0wuNq3XbN7+1PseV=AhK6MDcSY81rwsV2yxsMu1ft-Mg@mail.gmail.com>

Hi Stefan,

Care to explain what you are really trying to do?  We never actually
*use* py.py --- the fact that its options' defaults don't include any
modules should be a hint --- apart for demos and "just because you
can".

If you manage to isolate a failing test, then it can be written in a
particular way so that you get essentially py.py running only the
failing 5 lines.  Additionally, with sharing the object space across
multiple tests, we reduce the run time to a small fraction of second
per test.  Look for "class AppTestXxx" in the majority of the test
files in interpreter/test, module/*/test, or objspace/std/test.

It's tersely documented in pypy/doc/coding_guide.rst.  There are also
several other options for running tests of various lengths at various
levels.  The longest-running test in pypy is maybe a JIT test that
takes more than 5 minutes, but that's considered extreme.


A bient?t,

Armin.

From arigo at tunes.org  Sun Mar 11 19:46:28 2012
From: arigo at tunes.org (Armin Rigo)
Date: Sun, 11 Mar 2012 11:46:28 -0700
Subject: [pypy-dev] Hello, guys
In-Reply-To: <4F5C8382.6060707@interia.pl>
References: <4F5915F5.40101@interia.pl>
	<CACtfbMR+4Gmvytni=-Ff9Y=C_=XJm_cUmzzDiWWrkW16odQLLw@mail.gmail.com>
	<4F5C8382.6060707@interia.pl>
Message-ID: <CAMSv6X2HRVnMsbkB1zKwUUQOkFHeg6cc9RNde_UDCaWmbWKS3g@mail.gmail.com>

Hi Haael,

It is available in the source code repository in the "arm-backend-2"
branch.  If you want a precompiled pypy binary, I see one in
http://buildbot.pypy.org/nightly/arm-backend-2/ , but it might be a
bit outdated.


A bient?t,

Armin.

From stefan_ml at behnel.de  Sun Mar 11 20:15:39 2012
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sun, 11 Mar 2012 20:15:39 +0100
Subject: [pypy-dev] "import signal" fails when running PyPy interpreted
In-Reply-To: <CAMSv6X0wuNq3XbN7+1PseV=AhK6MDcSY81rwsV2yxsMu1ft-Mg@mail.gmail.com>
References: <jjijpk$frl$1@dough.gmane.org>
	<CAK5idxR6u5wT1ufLe-_H6pmYC_RdoysUJMXb1kdoOkPi1g+eMw@mail.gmail.com>
	<jjinnf$bih$1@dough.gmane.org>
	<CAMSv6X0wuNq3XbN7+1PseV=AhK6MDcSY81rwsV2yxsMu1ft-Mg@mail.gmail.com>
Message-ID: <jjitks$k09$1@dough.gmane.org>

Armin Rigo, 11.03.2012 19:40:
> Care to explain what you are really trying to do?

I'm trying to debug the set of crash bugs that Cython's test suite finds in
cpyext. I was surprised enough to find out that the nightly builds don't
have debugging symbols enabled, and Amaury suggested to run PyPy in
interpreted mode for debugging. So I tried and it didn't turn out to be an
option.


> We never actually
> *use* py.py --- the fact that its options' defaults don't include any
> modules should be a hint --- apart for demos and "just because you
> can".

You should put up a "beginners guide to debugging bugs in PyPy" somewhere.


> If you manage to isolate a failing test, then it can be written in a
> particular way so that you get essentially py.py running only the
> failing 5 lines.

I'd be happy with having all of cpyext run in "debug mode" already. Our
test runner runs in Python space to search and build the test modules. Each
test module has doctests which call into its C code. So there's usually not
much C code involved for a test and C-API interaction is thus limited. That
means that any slow-down in cpyext would be acceptable (because that's what
is being debugged), but avoiding any slow-downs in Python space would be
helpful.

Is such a setup possible?


> Additionally, with sharing the object space across
> multiple tests, we reduce the run time to a small fraction of second
> per test.  Look for "class AppTestXxx" in the majority of the test
> files in interpreter/test, module/*/test, or objspace/std/test.

Can't see the magic in those from a quick look.


> It's tersely documented in pypy/doc/coding_guide.rst.

I'm not interested in details. In Cython, I'm used to source level
debugging, be it for Python, Cython or C code, although I tend to work at
the C level most of the time, because that's where most bugs are. I don't
care much about which level I get in PyPy, as long as it's reasonably fast
and jumps into a debugger when an error occurs so that I can look at what's
been going on.

Stefan


From arigo at tunes.org  Sun Mar 11 20:50:53 2012
From: arigo at tunes.org (Armin Rigo)
Date: Sun, 11 Mar 2012 12:50:53 -0700
Subject: [pypy-dev] "import signal" fails when running PyPy interpreted
In-Reply-To: <jjitks$k09$1@dough.gmane.org>
References: <jjijpk$frl$1@dough.gmane.org>
	<CAK5idxR6u5wT1ufLe-_H6pmYC_RdoysUJMXb1kdoOkPi1g+eMw@mail.gmail.com>
	<jjinnf$bih$1@dough.gmane.org>
	<CAMSv6X0wuNq3XbN7+1PseV=AhK6MDcSY81rwsV2yxsMu1ft-Mg@mail.gmail.com>
	<jjitks$k09$1@dough.gmane.org>
Message-ID: <CAMSv6X2R+zMo5UHvHQkkR7EcfdGJHETDCrpn_z-+P7k1VSD6EA@mail.gmail.com>

Hi Stefan,

Did you realize that you said in the *same* e-mail:

> You should put up a "beginners guide to debugging bugs in PyPy" somewhere.

and

>> It's tersely documented in pypy/doc/coding_guide.rst.
> I'm not interested in details.

Sorry if this was not your intention but it sounds to me a lot like
"you need to write docs, but I'm not going to read it anyway".

Armin

From stefan_ml at behnel.de  Sun Mar 11 22:33:23 2012
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sun, 11 Mar 2012 22:33:23 +0100
Subject: [pypy-dev] "import signal" fails when running PyPy interpreted
In-Reply-To: <CAMSv6X2R+zMo5UHvHQkkR7EcfdGJHETDCrpn_z-+P7k1VSD6EA@mail.gmail.com>
References: <jjijpk$frl$1@dough.gmane.org>
	<CAK5idxR6u5wT1ufLe-_H6pmYC_RdoysUJMXb1kdoOkPi1g+eMw@mail.gmail.com>
	<jjinnf$bih$1@dough.gmane.org>
	<CAMSv6X0wuNq3XbN7+1PseV=AhK6MDcSY81rwsV2yxsMu1ft-Mg@mail.gmail.com>
	<jjitks$k09$1@dough.gmane.org>
	<CAMSv6X2R+zMo5UHvHQkkR7EcfdGJHETDCrpn_z-+P7k1VSD6EA@mail.gmail.com>
Message-ID: <jjj5n3$9o4$1@dough.gmane.org>

Armin Rigo, 11.03.2012 20:50:
> Did you realize that you said in the *same* e-mail:
> 
>> You should put up a "beginners guide to debugging bugs in PyPy" somewhere.
> 
> and
> 
>>> It's tersely documented in pypy/doc/coding_guide.rst.
>> I'm not interested in details.
> 
> Sorry if this was not your intention but it sounds to me a lot like
> "you need to write docs, but I'm not going to read it anyway".

"details" is different from a "beginners guide to debugging", at least the
way I understand it. What I'd like to have is a short step by step guide
saying what I need to do, and how to set up a working environment, in order
to efficiently debug into a bug in PyPy, specifically its cpyext module. I
tried two ways so far (nightly build in gdb and interpreted source) and
they both proved to be dead ends.

Here's the trial and error guide for Cython, as an example of what I mean:

http://wiki.cython.org/HackerGuide#Gettingstarted

To be clear, I happily don't care how PyPy works internally and I don't
want to waste my time learning it. I just want to find out what's wrong in
its cpyext layer by analysing the bugs that currently block it from running
Cython generated code.

OTOH, if you prefer to debug and fix everything yourself, I'm fine with
that also. You can look at the guide above, it will tell you how to run the
test suite, and that will quickly show you the crashes, so you can debug
them. Here's a list of test failures:

https://sage.math.washington.edu:8091/hudson/view/dev-scoder/job/cython-scoder-pypy-nightly/lastCompletedBuild/testReport/

and here are the latest builds, the red ones are eventual crashes (using
jitted PyPy up to build 48):

https://sage.math.washington.edu:8091/hudson/view/dev-scoder/job/cython-scoder-pypy-nightly/

A couple of crashers are actually disabled in the PyPy runs, you'll notice
them when you run the tests yourself. The list is written into the build log.

Stefan


From vadius at gmail.com  Mon Mar 12 00:34:04 2012
From: vadius at gmail.com (Christopher Swenson)
Date: Sun, 11 Mar 2012 16:34:04 -0700
Subject: [pypy-dev] Why isn't PyPy compiling itself not the same as CPython?
Message-ID: <CACBmJOVR4hs+ZdK4bh=psEO8m8uOfxo0G8TAWkJugW_802oCpQ@mail.gmail.com>

This may be a known issue, but it's a little worrying.

I compiled PyPy with CPython, and I got a different binary than when I
compiled it with itself:

-rwxrwxr-x 1 swenson swenson 59314173 2012-03-11 17:45 pypy-c*
-rwxrwxr-x 1 swenson swenson 59297884 2012-03-11 18:52 pypy-c2

I know that the final check with GCC is that it compiles itself with
itself, and checks that it gets itself.

Is there some randomness in translate.py?

--Christopher
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120311/cde1caa8/attachment.html>

From fijall at gmail.com  Mon Mar 12 00:38:44 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sun, 11 Mar 2012 16:38:44 -0700
Subject: [pypy-dev] Why isn't PyPy compiling itself not the same as
	CPython?
In-Reply-To: <CACBmJOVR4hs+ZdK4bh=psEO8m8uOfxo0G8TAWkJugW_802oCpQ@mail.gmail.com>
References: <CACBmJOVR4hs+ZdK4bh=psEO8m8uOfxo0G8TAWkJugW_802oCpQ@mail.gmail.com>
Message-ID: <CAK5idxT3v8wsq2zhYbGC433AiPB+1Wism+=st3yTTVsHdSo9xw@mail.gmail.com>

On Sun, Mar 11, 2012 at 4:34 PM, Christopher Swenson <vadius at gmail.com> wrote:
> This may be a known issue, but it's a little worrying.
>
> I compiled PyPy with CPython, and I got a different binary than when I
> compiled it with itself:
>
> -rwxrwxr-x 1 swenson swenson 59314173 2012-03-11 17:45 pypy-c*
> -rwxrwxr-x 1 swenson swenson 59297884 2012-03-11 18:52 pypy-c2
>
> I know that the final check with GCC is that it compiles itself with itself,
> and checks that it gets itself.
>
> Is there some randomness in translate.py?

The answer is "yes". You would get a slightly different result if you
compile it twice with the same binary even.

From vadius at gmail.com  Mon Mar 12 00:42:00 2012
From: vadius at gmail.com (Christopher Swenson)
Date: Sun, 11 Mar 2012 16:42:00 -0700
Subject: [pypy-dev] Why isn't PyPy compiling itself not the same as
	CPython?
In-Reply-To: <CAK5idxT3v8wsq2zhYbGC433AiPB+1Wism+=st3yTTVsHdSo9xw@mail.gmail.com>
References: <CACBmJOVR4hs+ZdK4bh=psEO8m8uOfxo0G8TAWkJugW_802oCpQ@mail.gmail.com>
	<CAK5idxT3v8wsq2zhYbGC433AiPB+1Wism+=st3yTTVsHdSo9xw@mail.gmail.com>
Message-ID: <CACBmJOWhBrEqKRLkNOsFzPVzkFf=rdTExz9C3UKnRUCk_jX35A@mail.gmail.com>

Alright. That's too bad.

Thanks,
--Christopher

On Sun, Mar 11, 2012 at 16:38, Maciej Fijalkowski <fijall at gmail.com> wrote:

> On Sun, Mar 11, 2012 at 4:34 PM, Christopher Swenson <vadius at gmail.com>
> wrote:
> > This may be a known issue, but it's a little worrying.
> >
> > I compiled PyPy with CPython, and I got a different binary than when I
> > compiled it with itself:
> >
> > -rwxrwxr-x 1 swenson swenson 59314173 2012-03-11 17:45 pypy-c*
> > -rwxrwxr-x 1 swenson swenson 59297884 2012-03-11 18:52 pypy-c2
> >
> > I know that the final check with GCC is that it compiles itself with
> itself,
> > and checks that it gets itself.
> >
> > Is there some randomness in translate.py?
>
> The answer is "yes". You would get a slightly different result if you
> compile it twice with the same binary even.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120311/33e10704/attachment-0001.html>

From wlavrijsen at lbl.gov  Mon Mar 12 00:56:28 2012
From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov)
Date: Sun, 11 Mar 2012 16:56:28 -0700 (PDT)
Subject: [pypy-dev] Why isn't PyPy compiling itself not the same as
 CPython?
In-Reply-To: <CACBmJOVR4hs+ZdK4bh=psEO8m8uOfxo0G8TAWkJugW_802oCpQ@mail.gmail.com>
References: <CACBmJOVR4hs+ZdK4bh=psEO8m8uOfxo0G8TAWkJugW_802oCpQ@mail.gmail.com>
Message-ID: <alpine.LNX.2.00.1203111638520.19740@pc3-wlav.lbl.gov>

Christopher,

On Sun, 11 Mar 2012, Christopher Swenson wrote:
> I compiled PyPy with CPython, and I got a different binary than when I
> compiled it with itself:
>
> -rwxrwxr-x 1 swenson swenson 59314173 2012-03-11 17:45 pypy-c*
> -rwxrwxr-x 1 swenson swenson 59297884 2012-03-11 18:52 pypy-c2

willing to venture a guess, based on what I've experienced to my own
detriment: the annotator will actually follow, annotate, and make available
for the translation anything it can get its hands on during the translation,
even if those are otherwise run-time objects.

For example, if a (non-rpython) module in PyPy creates module-level objects,
then the execution of creating those objects is traced. If that creation uses
standard python modules, then in the CPython case that is just more python,
but it may lead to loading of rpython code (as part of the standard module's
internals) in the PyPy case. That code then enters into the translation and
will show up in the final .c files.

> Is there some randomness in translate.py?

Because of the above, I've had random differences, but that was a bug. :)

Best regards,
            Wim
-- 
WLavrijsen at lbl.gov    --    +1 (510) 486 6411    --    www.lavrijsen.net

From stefan_ml at behnel.de  Mon Mar 12 08:42:16 2012
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Mon, 12 Mar 2012 08:42:16 +0100
Subject: [pypy-dev] "import signal" fails when running PyPy interpreted
In-Reply-To: <CAMSv6X0wuNq3XbN7+1PseV=AhK6MDcSY81rwsV2yxsMu1ft-Mg@mail.gmail.com>
References: <jjijpk$frl$1@dough.gmane.org>
	<CAK5idxR6u5wT1ufLe-_H6pmYC_RdoysUJMXb1kdoOkPi1g+eMw@mail.gmail.com>
	<jjinnf$bih$1@dough.gmane.org>
	<CAMSv6X0wuNq3XbN7+1PseV=AhK6MDcSY81rwsV2yxsMu1ft-Mg@mail.gmail.com>
Message-ID: <jjk9co$pge$1@dough.gmane.org>

Armin Rigo, 11.03.2012 19:40:
> Care to explain what you are really trying to do?  We never actually
> *use* py.py --- the fact that its options' defaults don't include any
> modules should be a hint --- apart for demos and "just because you
> can".
> 
> If you manage to isolate a failing test, then it can be written in a
> particular way so that you get essentially py.py running only the
> failing 5 lines.  Additionally, with sharing the object space across
> multiple tests, we reduce the run time to a small fraction of second
> per test.  Look for "class AppTestXxx" in the majority of the test
> files in interpreter/test, module/*/test, or objspace/std/test.
> 
> It's tersely documented in pypy/doc/coding_guide.rst.

I read this a little closer now and it told me more or less what I expected
when I skipped over it for the first time. It tells me that "PyPy can be
debugged on top of CPython", which is (in friendly words) only true if you
are prepared to fire it up before going to bed to get the results the next
morning. I thought we had to 60's for that.

It also tells me (as you did above) that test code can be split into
RPython code and application level code. Fine - just not my use case,
because I don't have application level code to debug, just RPython code.

So, simple question: how do I debug cpyext without slowing down the whole
machinery to a crawl?

Stefan


From amauryfa at gmail.com  Mon Mar 12 09:37:19 2012
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Mon, 12 Mar 2012 09:37:19 +0100
Subject: [pypy-dev] "import signal" fails when running PyPy interpreted
In-Reply-To: <jjk9co$pge$1@dough.gmane.org>
References: <jjijpk$frl$1@dough.gmane.org>
	<CAK5idxR6u5wT1ufLe-_H6pmYC_RdoysUJMXb1kdoOkPi1g+eMw@mail.gmail.com>
	<jjinnf$bih$1@dough.gmane.org>
	<CAMSv6X0wuNq3XbN7+1PseV=AhK6MDcSY81rwsV2yxsMu1ft-Mg@mail.gmail.com>
	<jjk9co$pge$1@dough.gmane.org>
Message-ID: <CAGmFidZ0XXfF=hejEU48Rz_r84=ycZcPTpePksLyUS8ii9rbSQ@mail.gmail.com>

Hi,

2012/3/12 Stefan Behnel <stefan_ml at behnel.de>:
>> It's tersely documented in pypy/doc/coding_guide.rst.
>
> I read this a little closer now and it told me more or less what I expected
> when I skipped over it for the first time. It tells me that "PyPy can be
> debugged on top of CPython", which is (in friendly words) only true if you
> are prepared to fire it up before going to bed to get the results the next
> morning. I thought we had to 60's for that.

Did we tell you that cpyext was slow? :-)
In this case, I don't try to run unittest or doctest.
I write a simple script, and run py.py on it.

In my experience, the slowest parts of py.py are when it imports a large
cpyext extension module. But even "import lxml" takes less than
5 minutes, in a py.py running on top of a debug build of CPython.

-- 
Amaury Forgeot d'Arc

From david.schneider at picle.org  Mon Mar 12 10:08:52 2012
From: david.schneider at picle.org (David Schneider)
Date: Mon, 12 Mar 2012 10:08:52 +0100
Subject: [pypy-dev] Hello, guys
In-Reply-To: <4F5C8382.6060707@interia.pl>
References: <4F5915F5.40101@interia.pl>
	<CACtfbMR+4Gmvytni=-Ff9Y=C_=XJm_cUmzzDiWWrkW16odQLLw@mail.gmail.com>
	<4F5C8382.6060707@interia.pl>
Message-ID: <4F5DBD24.70605@picle.org>

Hello Haael,

As Armin wrote, you can find the source code for ARM in the
arm-backend-2 branch. There are some instructions on hot to
cross-translate PyPy in the doc folder in the repository or on Bitbucket
at https://bitbucket.org/pypy/pypy/src/arm-backend-2/pypy/doc/arm.rst

David

haael wrote:
> 
> Cool :). But is there anything to download to play with?
> 
> haael
> 
>> Take a look at morepypy and enjoy :-)
>>
>> http://morepypy.blogspot.com/search?q=arm
>>
>> -- 
>> Dott. Stefano Parmesan
>> PhD Student ~ University of Trento
>> <ahref Foundation - Trento - Italy
>> Tel: 0461-235794 ext. 5544
>>
>>
>>
>> On 8 March 2012 21:26, haael<haael at interia.pl>  wrote:
>>
>>>
>>> I'm new to PyPy. I'm mostly interested in ARM architecture. My ultimate
>>> goal is to run Python interpreter on a bare CPU without an operating
>>> system, but for now I just want to compile PyPy using arm-eabi-gcc.
>>> It does
>>> not have to work, much less support anything, but my question is:
>>> what is
>>> the status of ARM target? There's nothing about it in the documentation.
>>>
>>> haael
>>>
>>>
>>> ______________________________**_________________
>>> pypy-dev mailing list
>>> pypy-dev at python.org
>>> http://mail.python.org/**mailman/listinfo/pypy-dev<http://mail.python.org/mailman/listinfo/pypy-dev>
>>>
>>>
>>
> 
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev


From stefan_ml at behnel.de  Mon Mar 12 10:21:14 2012
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Mon, 12 Mar 2012 10:21:14 +0100
Subject: [pypy-dev] "import signal" fails when running PyPy interpreted
In-Reply-To: <CAGmFidZ0XXfF=hejEU48Rz_r84=ycZcPTpePksLyUS8ii9rbSQ@mail.gmail.com>
References: <jjijpk$frl$1@dough.gmane.org>
	<CAK5idxR6u5wT1ufLe-_H6pmYC_RdoysUJMXb1kdoOkPi1g+eMw@mail.gmail.com>
	<jjinnf$bih$1@dough.gmane.org>
	<CAMSv6X0wuNq3XbN7+1PseV=AhK6MDcSY81rwsV2yxsMu1ft-Mg@mail.gmail.com>
	<jjk9co$pge$1@dough.gmane.org>
	<CAGmFidZ0XXfF=hejEU48Rz_r84=ycZcPTpePksLyUS8ii9rbSQ@mail.gmail.com>
Message-ID: <jjkf6a$1q5$1@dough.gmane.org>

Amaury Forgeot d'Arc, 12.03.2012 09:37:
> 2012/3/12 Stefan Behnel:
>>> It's tersely documented in pypy/doc/coding_guide.rst.
>>
>> I read this a little closer now and it told me more or less what I expected
>> when I skipped over it for the first time. It tells me that "PyPy can be
>> debugged on top of CPython", which is (in friendly words) only true if you
>> are prepared to fire it up before going to bed to get the results the next
>> morning. I thought we had to 60's for that.
> 
> Did we tell you that cpyext was slow? :-)

Oh, believe me, I'd be happy if everything was running in C and only using
cpyext. As I said in another mail, however, the C code parts here are
totally negligible - they are just short tests. It's the test runner that
takes ages to get started.

Is there a way to manually compile a C module against a PyPy-in-sources
that allows importing it afterwards? Then I could just compile the test
normally (and quickly) in CPython and then recompile the C code manually
for PyPy. I tried, but it didn't work so far. What I tried was:

1) run gcc directly - fails because the header files were not in place

2) copy header files from pypy/modules/cpyext/include/ to plain include/
and compile - still fails because pypy_decl.h and pypy_macros.h are missing

3) copy header files from pre-built PyPy - compiles now, but doesn't get
found - PyPy looks for "modname.pypy-18i.so" only and ignores "modname.so"

4) rename .so name to .pypy-18i.so - module is found but still fails to
import with

"""
ImportError: unable to load extension module
'.../cascadedassignment.pypy-18i.so':
"""

Would be helpful to see a reason after the ":".


> In this case, I don't try to run unittest or doctest.

That explains the huge difference.

Stefan


From amauryfa at gmail.com  Mon Mar 12 10:33:11 2012
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Mon, 12 Mar 2012 10:33:11 +0100
Subject: [pypy-dev] "import signal" fails when running PyPy interpreted
In-Reply-To: <jjkf6a$1q5$1@dough.gmane.org>
References: <jjijpk$frl$1@dough.gmane.org>
	<CAK5idxR6u5wT1ufLe-_H6pmYC_RdoysUJMXb1kdoOkPi1g+eMw@mail.gmail.com>
	<jjinnf$bih$1@dough.gmane.org>
	<CAMSv6X0wuNq3XbN7+1PseV=AhK6MDcSY81rwsV2yxsMu1ft-Mg@mail.gmail.com>
	<jjk9co$pge$1@dough.gmane.org>
	<CAGmFidZ0XXfF=hejEU48Rz_r84=ycZcPTpePksLyUS8ii9rbSQ@mail.gmail.com>
	<jjkf6a$1q5$1@dough.gmane.org>
Message-ID: <CAGmFidaCKPHCz=GPOMrv3BNhG=Wuqry-S=SzLkJ40TkLSHRtpw@mail.gmail.com>

2012/3/12 Stefan Behnel <stefan_ml at behnel.de>:
> Is there a way to manually compile a C module against a PyPy-in-sources
> that allows importing it afterwards? Then I could just compile the test
> normally (and quickly) in CPython and then recompile the C code manually
> for PyPy. I tried, but it didn't work so far. What I tried was:

Keep in mind that "manual" ways won't work because of symbols collision:
you want PyString_FromString() call some PyPy function, not the one
exposed by the
host CPython you are running py.py with.

There is a way to do it though, for distutils-based projects: a
"presetup.py" will tweak
distutils to compile .pypy-18i.so files:
   python /path/to/pypy/module/cpyext/presetup.py setup.py build_ext
The "i" in ".pypy-18i.so" stands for "interpreted", i.e. not
translated pypy. All symbols
are renamed (PyPyString_FromString), and the resulting files can be
imported in py.py.
This is how I debugged lxml, for example.

-- 
Amaury Forgeot d'Arc

From stefan_ml at behnel.de  Mon Mar 12 10:35:22 2012
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Mon, 12 Mar 2012 10:35:22 +0100
Subject: [pypy-dev] "import signal" fails when running PyPy interpreted
In-Reply-To: <jjkf6a$1q5$1@dough.gmane.org>
References: <jjijpk$frl$1@dough.gmane.org>
	<CAK5idxR6u5wT1ufLe-_H6pmYC_RdoysUJMXb1kdoOkPi1g+eMw@mail.gmail.com>
	<jjinnf$bih$1@dough.gmane.org>
	<CAMSv6X0wuNq3XbN7+1PseV=AhK6MDcSY81rwsV2yxsMu1ft-Mg@mail.gmail.com>
	<jjk9co$pge$1@dough.gmane.org>
	<CAGmFidZ0XXfF=hejEU48Rz_r84=ycZcPTpePksLyUS8ii9rbSQ@mail.gmail.com>
	<jjkf6a$1q5$1@dough.gmane.org>
Message-ID: <jjkg0q$6rn$1@dough.gmane.org>

Stefan Behnel, 12.03.2012 10:21:
> Amaury Forgeot d'Arc, 12.03.2012 09:37:
>> 2012/3/12 Stefan Behnel:
>>>> It's tersely documented in pypy/doc/coding_guide.rst.
>>>
>>> I read this a little closer now and it told me more or less what I expected
>>> when I skipped over it for the first time. It tells me that "PyPy can be
>>> debugged on top of CPython", which is (in friendly words) only true if you
>>> are prepared to fire it up before going to bed to get the results the next
>>> morning. I thought we had to 60's for that.
>>
>> Did we tell you that cpyext was slow? :-)
> 
> Oh, believe me, I'd be happy if everything was running in C and only using
> cpyext.

Rethinking this some more - I'd be *really* happy if everything was just C.
Armin noted that the RPython-to-C translation is basically a 1:1 mapping
(plus type specialisation). So, if the generated C code is any usable and
if the nightly builds had debugging symbols in them, I could just run gdb
and callgrind over them and they would pinpoint the obvious problems
immediately.

Would that work? And if so, could someone enable that?

Stefan


From stefan_ml at behnel.de  Mon Mar 12 10:45:50 2012
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Mon, 12 Mar 2012 10:45:50 +0100
Subject: [pypy-dev] "import signal" fails when running PyPy interpreted
In-Reply-To: <CAGmFidaCKPHCz=GPOMrv3BNhG=Wuqry-S=SzLkJ40TkLSHRtpw@mail.gmail.com>
References: <jjijpk$frl$1@dough.gmane.org>
	<CAK5idxR6u5wT1ufLe-_H6pmYC_RdoysUJMXb1kdoOkPi1g+eMw@mail.gmail.com>
	<jjinnf$bih$1@dough.gmane.org>
	<CAMSv6X0wuNq3XbN7+1PseV=AhK6MDcSY81rwsV2yxsMu1ft-Mg@mail.gmail.com>
	<jjk9co$pge$1@dough.gmane.org>
	<CAGmFidZ0XXfF=hejEU48Rz_r84=ycZcPTpePksLyUS8ii9rbSQ@mail.gmail.com>
	<jjkf6a$1q5$1@dough.gmane.org>
	<CAGmFidaCKPHCz=GPOMrv3BNhG=Wuqry-S=SzLkJ40TkLSHRtpw@mail.gmail.com>
Message-ID: <jjkgke$b91$1@dough.gmane.org>

Amaury Forgeot d'Arc, 12.03.2012 10:33:
> 2012/3/12 Stefan Behnel:
>> Is there a way to manually compile a C module against a PyPy-in-sources
>> that allows importing it afterwards? Then I could just compile the test
>> normally (and quickly) in CPython and then recompile the C code manually
>> for PyPy. I tried, but it didn't work so far. What I tried was:
> 
> Keep in mind that "manual" ways won't work because of symbols collision:
> you want PyString_FromString() call some PyPy function, not the one
> exposed by the host CPython you are running py.py with.

Right, sure.


> There is a way to do it though, for distutils-based projects: a
> "presetup.py" will tweak distutils to compile .pypy-18i.so files:
>    python /path/to/pypy/module/cpyext/presetup.py setup.py build_ext
> The "i" in ".pypy-18i.so" stands for "interpreted", i.e. not
> translated pypy. All symbols
> are renamed (PyPyString_FromString), and the resulting files can be
> imported in py.py.
> This is how I debugged lxml, for example.

Hmm, interesting. The test runner uses distutils. That means that it might
even work to let the test runner run normally in CPython and only resort to
PyPy for (1) building the module and (2) running the test. Might be some
work to set up, but could be worth it.

Thanks!

Stefan


From amauryfa at gmail.com  Mon Mar 12 10:47:54 2012
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Mon, 12 Mar 2012 10:47:54 +0100
Subject: [pypy-dev] "import signal" fails when running PyPy interpreted
In-Reply-To: <jjkg0q$6rn$1@dough.gmane.org>
References: <jjijpk$frl$1@dough.gmane.org>
	<CAK5idxR6u5wT1ufLe-_H6pmYC_RdoysUJMXb1kdoOkPi1g+eMw@mail.gmail.com>
	<jjinnf$bih$1@dough.gmane.org>
	<CAMSv6X0wuNq3XbN7+1PseV=AhK6MDcSY81rwsV2yxsMu1ft-Mg@mail.gmail.com>
	<jjk9co$pge$1@dough.gmane.org>
	<CAGmFidZ0XXfF=hejEU48Rz_r84=ycZcPTpePksLyUS8ii9rbSQ@mail.gmail.com>
	<jjkf6a$1q5$1@dough.gmane.org> <jjkg0q$6rn$1@dough.gmane.org>
Message-ID: <CAGmFidYW56aLVo2QbrAAWLA0t7+-w233FjHh78tOxVfGkCwAzg@mail.gmail.com>

2012/3/12 Stefan Behnel <stefan_ml at behnel.de>:
> Rethinking this some more - I'd be *really* happy if everything was just C.
> Armin noted that the RPython-to-C translation is basically a 1:1 mapping
> (plus type specialisation). So, if the generated C code is any usable and
> if the nightly builds had debugging symbols in them, I could just run gdb
> and callgrind over them and they would pinpoint the obvious problems
> immediately.
>
> Would that work? And if so, could someone enable that?

Yes, it works, but the result is not so easy to debug and understand.

This said, compliing pypy is not so difficult on a Linux machine; and you only
need to translate with the -O2 option, the JIT is not useful to debug cpyext.

-- 
Amaury Forgeot d'Arc

From p.j.a.cock at googlemail.com  Mon Mar 12 19:43:21 2012
From: p.j.a.cock at googlemail.com (Peter Cock)
Date: Mon, 12 Mar 2012 18:43:21 +0000
Subject: [pypy-dev] Speeding up zlib in standard library
Message-ID: <CAKVJ-_5D4OUBSkavYqW7usBkaZMxu-EjEtb9NaKU-iCMA=XZQg@mail.gmail.com>

Hi all,

I am looking at some benchmark numbers from some of my code,
and one area where PyPy is slower than 'C' Python (perhaps twice
as slow) is code making heavy use of the zlib standard library.

I had a look at this page about how C libraries are called from PyPy,
it seems sketchy and perhaps somewhat out of date too. I'd find it
helpful if it commented on what PyPy was using for the standard
library C modules: http://doc.pypy.org/en/latest/extending.html

Looking at the source of the zlib library in pypy,
https://bitbucket.org/pypy/pypy/src/default/pypy/module/zlib/__init__.py

I see it starts with these imports:

from pypy.interpreter.mixedmodule import MixedModule
from pypy.rlib import rzlib

Which in turn calls the restricted code,
https://bitbucket.org/pypy/pypy/src/default/pypy/rlib/rzlib.py

Given http://doc.pypy.org/en/latest/extending.html describes this
mixed module approach as the "most advanced and powerful way"
of calling C code, does that mean in this particular case there are
no low handing fruit to speed up using zlib in PyPy?

Thanks,

Peter

From WWatts at vtrIT.com  Mon Mar 12 23:24:40 2012
From: WWatts at vtrIT.com (Watts, Wendy)
Date: Mon, 12 Mar 2012 15:24:40 -0700
Subject: [pypy-dev] Job
Message-ID: <0E23478CC3BCA649B6E08B0A78691B9C5CC12D@CA100EX5.west.vis.com>

Hello, my name is Wendy; I am a IT recruiter for vtrIT which is a
division of Volt Workforce Technical Solutions located in San Francisco.
I have an urgent Senior and Junior Python Engineer positions open for a
client located in CA. I am reaching out to you in finding out your
status of new opportunity, and if we can speak verbal? 

We will pay travel expenses.  

Please job description for your review? 

Specific's of the positions are as follows:

*         Job Title - one Sr. Python Engineer and 2 (mid-level) Python
Engineers

*         Location:  It must be 100% onsite at client location in
Mountain View, California.

*         Duration: six-month  contract

 

If interested please forward me your resume a word attachment and I will
call you ASAP. Email: wwatts at vtrit.com

 

 

 

 

 

Wendy Watts 

IT Recruiter 

 

VTRIT

100 First Street, Suite 200 I San Francisco, CA 55120 

t: 415.536.5844  I f: 415.536.2845

wwatts at vtrit <mailto:mnielson at vtrIT.com> .com | vtrit.com

 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120312/e63416fa/attachment.html>

From cfbolz at gmx.de  Tue Mar 13 11:14:30 2012
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Tue, 13 Mar 2012 11:14:30 +0100
Subject: [pypy-dev] [pypy-commit] pypy default: add a few unrolls to
	tuple
In-Reply-To: <20120312204651.A8D7B822B8@wyvern.cs.uni-duesseldorf.de>
References: <20120312204651.A8D7B822B8@wyvern.cs.uni-duesseldorf.de>
Message-ID: <4F5F1E06.4070301@gmx.de>

Hi Maciek,

I think this commit should come with corresponding pypy_c tests.

Cheers,

Carl Friedrich

On 03/12/2012 09:46 PM, fijal wrote:
> Author: Maciej Fijalkowski<fijall at gmail.com>
> Branch:
> Changeset: r53340:12f4497fa28c
> Date: 2012-03-12 13:46 -0700
> http://bitbucket.org/pypy/pypy/changeset/12f4497fa28c/
>
> Log:	add a few unrolls to tuple
>
> diff --git a/pypy/objspace/std/tupleobject.py b/pypy/objspace/std/tupleobject.py
> --- a/pypy/objspace/std/tupleobject.py
> +++ b/pypy/objspace/std/tupleobject.py
> @@ -6,8 +6,8 @@
>   from pypy.rlib.rarithmetic import intmask
>   from pypy.objspace.std.sliceobject import W_SliceObject, normalize_simple_slice
>   from pypy.objspace.std import slicetype
> -from pypy.interpreter import gateway
>   from pypy.rlib.debug import make_sure_not_resized
> +from pypy.rlib import jit
>
>   class W_AbstractTupleObject(W_Object):
>       __slots__ = ()
> @@ -114,6 +114,9 @@
>   def mul__ANY_Tuple(space, w_times, w_tuple):
>       return mul_tuple_times(space, w_tuple, w_times)
>
> + at jit.look_inside_iff(lambda space, w_tuple1, w_tuple2:
> +                     jit.is_constant(len(w_tuple1.wrappeditems)) and
> +                     jit.is_constant(len(w_tuple2.wrappeditems)))
>   def eq__Tuple_Tuple(space, w_tuple1, w_tuple2):
>       items1 = w_tuple1.wrappeditems
>       items2 = w_tuple2.wrappeditems
> @@ -126,6 +129,9 @@
>               return space.w_False
>       return space.w_True
>
> + at jit.look_inside_iff(lambda space, w_tuple1, w_tuple2:
> +                     jit.is_constant(len(w_tuple1.wrappeditems)) and
> +                     jit.is_constant(len(w_tuple2.wrappeditems)))
>   def lt__Tuple_Tuple(space, w_tuple1, w_tuple2):
>       items1 = w_tuple1.wrappeditems
>       items2 = w_tuple2.wrappeditems
> @@ -137,6 +143,9 @@
>       # No more items to compare -- compare sizes
>       return space.newbool(len(items1)<  len(items2))
>
> + at jit.look_inside_iff(lambda space, w_tuple1, w_tuple2:
> +                     jit.is_constant(len(w_tuple1.wrappeditems)) and
> +                     jit.is_constant(len(w_tuple2.wrappeditems)))
>   def gt__Tuple_Tuple(space, w_tuple1, w_tuple2):
>       items1 = w_tuple1.wrappeditems
>       items2 = w_tuple2.wrappeditems
> @@ -161,6 +170,8 @@
>   def hash__Tuple(space, w_tuple):
>       return space.wrap(hash_tuple(space, w_tuple.wrappeditems))
>
> + at jit.look_inside_iff(lambda space, wrappeditems:
> +                     jit.is_constant(len(wrappeditems)))
>   def hash_tuple(space, wrappeditems):
>       # this is the CPython 2.4 algorithm (changed from 2.3)
>       mult = 1000003
> _______________________________________________
> pypy-commit mailing list
> pypy-commit at python.org
> http://mail.python.org/mailman/listinfo/pypy-commit


From amauryfa at gmail.com  Tue Mar 13 13:25:13 2012
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Tue, 13 Mar 2012 13:25:13 +0100
Subject: [pypy-dev] [pypy-commit] pypy default: relaxing int vs long
Message-ID: <CAGmFidbQrmiXmNoq6T9cWZLryZeQ==Phiixy6f-htayHkx7yig@mail.gmail.com>

Hi Christian,

The changeset 8abd5883fa0c looks wrong to me:
http://mail.python.org/pipermail/pypy-commit/2012-March/060785.html

diff --git a/pypy/interpreter/test/test_compiler.py
b/pypy/interpreter/test/test_compiler.py
--- a/pypy/interpreter/test/test_compiler.py
+++ b/pypy/interpreter/test/test_compiler.py
@@ -794,7 +794,7 @@
     def test_tuple_constants(self):
         ns = {}
         exec "x = (1, 0); y = (1L, 0L)" in ns
-        assert isinstance(ns["x"][0], int)
+        assert isinstance(ns["x"][0], (int, long))
         assert isinstance(ns["y"][0], long)

Why would x[0] be a long? isn't it an int constant?

-- 
Amaury Forgeot d'Arc

From fijall at gmail.com  Tue Mar 13 19:20:36 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Tue, 13 Mar 2012 11:20:36 -0700
Subject: [pypy-dev] [pypy-commit] pypy default: add a few unrolls to
	tuple
In-Reply-To: <4F5F1E06.4070301@gmx.de>
References: <20120312204651.A8D7B822B8@wyvern.cs.uni-duesseldorf.de>
	<4F5F1E06.4070301@gmx.de>
Message-ID: <CAK5idxT2j0G3C_7Ykg-36UBsFgB=yz4d0GCXn+yzYLEQFfUKfw@mail.gmail.com>

On Tue, Mar 13, 2012 at 3:14 AM, Carl Friedrich Bolz <cfbolz at gmx.de> wrote:
> Hi Maciek,
>
> I think this commit should come with corresponding pypy_c tests.
>
> Cheers,
>
> Carl Friedrich

Totally.

We just waited for translation, they're on their way

>
>
> On 03/12/2012 09:46 PM, fijal wrote:
>>
>> Author: Maciej Fijalkowski<fijall at gmail.com>
>> Branch:
>> Changeset: r53340:12f4497fa28c
>> Date: 2012-03-12 13:46 -0700
>> http://bitbucket.org/pypy/pypy/changeset/12f4497fa28c/
>>
>> Log: ? ?add a few unrolls to tuple
>>
>> diff --git a/pypy/objspace/std/tupleobject.py
>> b/pypy/objspace/std/tupleobject.py
>> --- a/pypy/objspace/std/tupleobject.py
>> +++ b/pypy/objspace/std/tupleobject.py
>> @@ -6,8 +6,8 @@
>> ?from pypy.rlib.rarithmetic import intmask
>> ?from pypy.objspace.std.sliceobject import W_SliceObject,
>> normalize_simple_slice
>> ?from pypy.objspace.std import slicetype
>> -from pypy.interpreter import gateway
>> ?from pypy.rlib.debug import make_sure_not_resized
>> +from pypy.rlib import jit
>>
>> ?class W_AbstractTupleObject(W_Object):
>> ? ? ?__slots__ = ()
>> @@ -114,6 +114,9 @@
>> ?def mul__ANY_Tuple(space, w_times, w_tuple):
>> ? ? ?return mul_tuple_times(space, w_tuple, w_times)
>>
>> + at jit.look_inside_iff(lambda space, w_tuple1, w_tuple2:
>> + ? ? ? ? ? ? ? ? ? ? jit.is_constant(len(w_tuple1.wrappeditems)) and
>> + ? ? ? ? ? ? ? ? ? ? jit.is_constant(len(w_tuple2.wrappeditems)))
>> ?def eq__Tuple_Tuple(space, w_tuple1, w_tuple2):
>> ? ? ?items1 = w_tuple1.wrappeditems
>> ? ? ?items2 = w_tuple2.wrappeditems
>> @@ -126,6 +129,9 @@
>> ? ? ? ? ? ? ?return space.w_False
>> ? ? ?return space.w_True
>>
>> + at jit.look_inside_iff(lambda space, w_tuple1, w_tuple2:
>> + ? ? ? ? ? ? ? ? ? ? jit.is_constant(len(w_tuple1.wrappeditems)) and
>> + ? ? ? ? ? ? ? ? ? ? jit.is_constant(len(w_tuple2.wrappeditems)))
>> ?def lt__Tuple_Tuple(space, w_tuple1, w_tuple2):
>> ? ? ?items1 = w_tuple1.wrappeditems
>> ? ? ?items2 = w_tuple2.wrappeditems
>> @@ -137,6 +143,9 @@
>> ? ? ?# No more items to compare -- compare sizes
>> ? ? ?return space.newbool(len(items1)< ?len(items2))
>>
>> + at jit.look_inside_iff(lambda space, w_tuple1, w_tuple2:
>> + ? ? ? ? ? ? ? ? ? ? jit.is_constant(len(w_tuple1.wrappeditems)) and
>> + ? ? ? ? ? ? ? ? ? ? jit.is_constant(len(w_tuple2.wrappeditems)))
>> ?def gt__Tuple_Tuple(space, w_tuple1, w_tuple2):
>> ? ? ?items1 = w_tuple1.wrappeditems
>> ? ? ?items2 = w_tuple2.wrappeditems
>> @@ -161,6 +170,8 @@
>> ?def hash__Tuple(space, w_tuple):
>> ? ? ?return space.wrap(hash_tuple(space, w_tuple.wrappeditems))
>>
>> + at jit.look_inside_iff(lambda space, wrappeditems:
>> + ? ? ? ? ? ? ? ? ? ? jit.is_constant(len(wrappeditems)))
>> ?def hash_tuple(space, wrappeditems):
>> ? ? ?# this is the CPython 2.4 algorithm (changed from 2.3)
>> ? ? ?mult = 1000003
>> _______________________________________________
>> pypy-commit mailing list
>> pypy-commit at python.org
>> http://mail.python.org/mailman/listinfo/pypy-commit
>
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev

From tismer at stackless.com  Tue Mar 13 18:55:40 2012
From: tismer at stackless.com (Christian Tismer)
Date: Tue, 13 Mar 2012 10:55:40 -0700
Subject: [pypy-dev] [pypy-commit] pypy default: relaxing int vs long
In-Reply-To: <CAGmFidbQrmiXmNoq6T9cWZLryZeQ==Phiixy6f-htayHkx7yig@mail.gmail.com>
References: <CAGmFidbQrmiXmNoq6T9cWZLryZeQ==Phiixy6f-htayHkx7yig@mail.gmail.com>
Message-ID: <E8684119-9AA1-420A-82B6-9A7239DAF656@stackless.com>

Hi Amaury,

This is debatable. In this particular case it makes in fact no sense, but in general, we need to stop distinguishing int and long. Instead, a range check is needed to determine an int. 

Sent from my Ei4Steve

On Mar 13, 2012, at 5:25, "Amaury Forgeot d'Arc" <amauryfa at gmail.com> wrote:

> Hi Christian,
> 
> The changeset 8abd5883fa0c looks wrong to me:
> http://mail.python.org/pipermail/pypy-commit/2012-March/060785.html
> 
> diff --git a/pypy/interpreter/test/test_compiler.py
> b/pypy/interpreter/test/test_compiler.py
> --- a/pypy/interpreter/test/test_compiler.py
> +++ b/pypy/interpreter/test/test_compiler.py
> @@ -794,7 +794,7 @@
>     def test_tuple_constants(self):
>         ns = {}
>         exec "x = (1, 0); y = (1L, 0L)" in ns
> -        assert isinstance(ns["x"][0], int)
> +        assert isinstance(ns["x"][0], (int, long))
>         assert isinstance(ns["y"][0], long)
> 
> Why would x[0] be a long? isn't it an int constant?
> 
> -- 
> Amaury Forgeot d'Arc

From benjamin at python.org  Tue Mar 13 22:51:12 2012
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 13 Mar 2012 16:51:12 -0500
Subject: [pypy-dev] Speeding up zlib in standard library
In-Reply-To: <CAKVJ-_5D4OUBSkavYqW7usBkaZMxu-EjEtb9NaKU-iCMA=XZQg@mail.gmail.com>
References: <CAKVJ-_5D4OUBSkavYqW7usBkaZMxu-EjEtb9NaKU-iCMA=XZQg@mail.gmail.com>
Message-ID: <CAPZV6o8fpWjG2d8L6ig4SQds=EAUnb=sypeor9HzazdmXyZy=g@mail.gmail.com>

2012/3/12 Peter Cock <p.j.a.cock at googlemail.com>:
> Given http://doc.pypy.org/en/latest/extending.html describes this
> mixed module approach as the "most advanced and powerful way"
> of calling C code, does that mean in this particular case there are
> no low handing fruit to speed up using zlib in PyPy?

Do you know the slowness comes from the binding overhead, though? It
could be from buffering, for example.



-- 
Regards,
Benjamin

From p.j.a.cock at googlemail.com  Wed Mar 14 11:19:52 2012
From: p.j.a.cock at googlemail.com (Peter Cock)
Date: Wed, 14 Mar 2012 10:19:52 +0000
Subject: [pypy-dev] Speeding up zlib in standard library
In-Reply-To: <CAPZV6o8fpWjG2d8L6ig4SQds=EAUnb=sypeor9HzazdmXyZy=g@mail.gmail.com>
References: <CAKVJ-_5D4OUBSkavYqW7usBkaZMxu-EjEtb9NaKU-iCMA=XZQg@mail.gmail.com>
	<CAPZV6o8fpWjG2d8L6ig4SQds=EAUnb=sypeor9HzazdmXyZy=g@mail.gmail.com>
Message-ID: <CAKVJ-_4zry6WMzg7Gfu2zUj=t-fpce_givh4TFj57aHiagwQ+Q@mail.gmail.com>

On Tue, Mar 13, 2012 at 9:51 PM, Benjamin Peterson <benjamin at python.org> wrote:
> 2012/3/12 Peter Cock <p.j.a.cock at googlemail.com>:
>> Given http://doc.pypy.org/en/latest/extending.html describes this
>> mixed module approach as the "most advanced and powerful way"
>> of calling C code, does that mean in this particular case there are
>> no low handing fruit to speed up using zlib in PyPy?
>
> Do you know the slowness comes from the binding overhead, though?
> It could be from buffering, for example.

I don't know - I was assuming any buffering would be the same
comparing PyPy 1.8 against Python 2.6 (and 3.2). That was one
reason for my email - is binding to C relatively slow (compared to
the rest of PyPy running pure Python)?

Is there any PyPy specific advice on profiling (and how this differs
to under 'C' Python)?

Thanks,

Peter

From anto.cuni at gmail.com  Wed Mar 14 11:29:45 2012
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Wed, 14 Mar 2012 11:29:45 +0100
Subject: [pypy-dev] [pypy-commit] pypy py3k: save the source of applevel
 direct tests in a temporary file: this way, we get nicer tracebacks
In-Reply-To: <CAK5idxRg-dsGuMvANkvEowJJH3_zrAyiZ9OMENoVhzhQ3LpBoA@mail.gmail.com>
References: <20120209151208.A21BE82B1E@wyvern.cs.uni-duesseldorf.de>
	<CAK5idxRg-dsGuMvANkvEowJJH3_zrAyiZ9OMENoVhzhQ3LpBoA@mail.gmail.com>
Message-ID: <4F607319.9080404@gmail.com>

On 03/07/2012 08:12 PM, Maciej Fijalkowski wrote:
> Can we have this on default as well please?

no, because it's something py3k specific: on default, applevel tests with -A
are run directly on the hosting python, while on py3k they are run in a
separate python3 process. Before this checkin, the code was passed using -c,
while now it's written in a tmp file.

ciao,
Anto

From Ronny.Pfannschmidt at gmx.de  Wed Mar 14 15:53:36 2012
From: Ronny.Pfannschmidt at gmx.de (Ronny Pfannschmidt)
Date: Wed, 14 Mar 2012 15:53:36 +0100
Subject: [pypy-dev] mechanism for keeping track of the latest revision on
 default that is known to translate
Message-ID: <4F60B0F0.3010300@gmx.de>

since sprints regularly cause situations where default doesn't translate,
it seems helpful to have a known latest revision that translates

the proposed mechanism would be to have a bookmark
that tracks that version for default,

a good name could be something like "last-known-translatable"

that version would be a good bet for starting new branches,
since its known to translate

i'm not too sure yet how to integrate in the buildbot

regards,
Ronny

From anto.cuni at gmail.com  Wed Mar 14 16:02:49 2012
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Wed, 14 Mar 2012 16:02:49 +0100
Subject: [pypy-dev] mechanism for keeping track of the latest revision
 on default that is known to translate
In-Reply-To: <4F60B0F0.3010300@gmx.de>
References: <4F60B0F0.3010300@gmx.de>
Message-ID: <4F60B319.5010707@gmail.com>

Hi Ronny,

On 03/14/2012 03:53 PM, Ronny Pfannschmidt wrote:
> since sprints regularly cause situations where default doesn't translate,
> it seems helpful to have a known latest revision that translates
> 
> the proposed mechanism would be to have a bookmark
> that tracks that version for default,

I like the idea. However, from what I read on the docs, it seems that the
bookmark automatically moves while I commit.

E.g., suppose that I update to a revision 100 that has the bookmark, then do
some work, then commit revision 101. Will the bookmark be at revision 100 or
101? In the latter case, it's a bit unusable for our purpose, because we want
it to be updated only by buildbot.

> a good name could be something like "last-known-translatable"
> 
> that version would be a good bet for starting new branches,
> since its known to translate
> 
> i'm not too sure yet how to integrate in the buildbot

I suppose we would need the buildslaves to have pushing right to bitbucket.
However, we don't need all of them to push, I suppose that e.g. only linux-32
or linux-64 are enough.

ciao,
Anto

From wlavrijsen at lbl.gov  Wed Mar 14 17:53:30 2012
From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov)
Date: Wed, 14 Mar 2012 09:53:30 -0700 (PDT)
Subject: [pypy-dev] base class hierarchy for string objects breaks
	translation
Message-ID: <alpine.LNX.2.00.1203140950200.21278@pc3-wlav.lbl.gov>

Hi,

the new "mixin" base classes for the new W_StringObject hierarchy are
missing a "_mixin_=True" statement, leading to the error message below.

I added those statements (also needed in the mixin base class), but that
still was not enough to get the translation much further.

Ideas, fix?

Thanks,
      Wim

[translation:ERROR]    File "/home/wlav/pypy-dev/pypy/pypy/annotation/description.py", line 422, in __init__
[translation:ERROR]     "with _mixin_: %r" % (cls,))
[translation:ERROR]  AssertionError: multiple inheritance only supported with _mixin_: <class 'pypy.objspace.std.stringobject.W_StringObject'>
[translation:ERROR] Processing block:
[translation:ERROR]  codeless block is a <class 'pypy.objspace.flow.flowcontext.SpamBlock'>
[translation:ERROR]  in (<ann_override.wrap>:1)wrap__str_bad_argument_type_for_built_in_operation
[translation:ERROR]  containing the following operations:
[translation:ERROR]  --end--

-- 
WLavrijsen at lbl.gov    --    +1 (510) 486 6411    --    www.lavrijsen.net

From anto.cuni at gmail.com  Wed Mar 14 18:03:41 2012
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Wed, 14 Mar 2012 18:03:41 +0100
Subject: [pypy-dev] base class hierarchy for string objects breaks
	translation
In-Reply-To: <alpine.LNX.2.00.1203140950200.21278@pc3-wlav.lbl.gov>
References: <alpine.LNX.2.00.1203140950200.21278@pc3-wlav.lbl.gov>
Message-ID: <4F60CF6D.4090003@gmail.com>

Hi Wim,

On 03/14/2012 05:53 PM, wlavrijsen at lbl.gov wrote:
> Hi,
> 
> the new "mixin" base classes for the new W_StringObject hierarchy are
> missing a "_mixin_=True" statement, leading to the error message below.
> 
> I added those statements (also needed in the mixin base class), but that
> still was not enough to get the translation much further.
> 
> Ideas, fix?

those checkins should have never gone into default, the belonged to a branch.
As an additional annoyance, there were two heads on default, one "wrong" with
the W_StringObject changes and one "right".

I closed the "wrong" one and merged it into the bytearray-refactor branch, so
now the situation should be fixed on default. Most probably, it's enough to
merge default again into your branch and things will work again :-).

ciao,
Anto

From wlavrijsen at lbl.gov  Wed Mar 14 18:11:24 2012
From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov)
Date: Wed, 14 Mar 2012 10:11:24 -0700 (PDT)
Subject: [pypy-dev] base class hierarchy for string objects breaks
 translation
In-Reply-To: <4F60CF6D.4090003@gmail.com>
References: <alpine.LNX.2.00.1203140950200.21278@pc3-wlav.lbl.gov>
	<4F60CF6D.4090003@gmail.com>
Message-ID: <alpine.LNX.2.00.1203141010270.21278@pc3-wlav.lbl.gov>

Hi Anto,

thanks, default at least seems to work (so far), other than a couple:

[annrpython:WARNING] <FunctionGraph of (pypy.objspace.std.objspace:148)wrap__str at 0x43e7eb0> op=0/ isinstance(., long) is not RPython

warnings. I'll merge again and see where that gets me. :)

Best regards,
            Wim
-- 
WLavrijsen at lbl.gov    --    +1 (510) 486 6411    --    www.lavrijsen.net

From anto.cuni at gmail.com  Wed Mar 14 18:28:54 2012
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Wed, 14 Mar 2012 18:28:54 +0100
Subject: [pypy-dev] base class hierarchy for string objects breaks
	translation
In-Reply-To: <alpine.LNX.2.00.1203141010270.21278@pc3-wlav.lbl.gov>
References: <alpine.LNX.2.00.1203140950200.21278@pc3-wlav.lbl.gov>
	<4F60CF6D.4090003@gmail.com>
	<alpine.LNX.2.00.1203141010270.21278@pc3-wlav.lbl.gov>
Message-ID: <4F60D556.2010009@gmail.com>

On 03/14/2012 06:11 PM, wlavrijsen at lbl.gov wrote:
> Hi Anto,
> 
> thanks, default at least seems to work (so far), other than a couple:
> 
> [annrpython:WARNING] <FunctionGraph of
> (pypy.objspace.std.objspace:148)wrap__str at 0x43e7eb0> op=0/ isinstance(.,
> long) is not RPython
> 
> warnings. I'll merge again and see where that gets me. :)

uhm, it's possible that these are related to the recent changes by Christian
about int/long on win64.
If they break the translation, we should revert them.

ciao,
Anto

From wlavrijsen at lbl.gov  Wed Mar 14 18:47:51 2012
From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov)
Date: Wed, 14 Mar 2012 10:47:51 -0700 (PDT)
Subject: [pypy-dev] base class hierarchy for string objects breaks
 translation
In-Reply-To: <4F60D556.2010009@gmail.com>
References: <alpine.LNX.2.00.1203140950200.21278@pc3-wlav.lbl.gov>
	<4F60CF6D.4090003@gmail.com>
	<alpine.LNX.2.00.1203141010270.21278@pc3-wlav.lbl.gov>
	<4F60D556.2010009@gmail.com>
Message-ID: <alpine.LNX.2.00.1203141046050.23704@pc3-wlav.lbl.gov>

Hi Anto,

> uhm, it's possible that these are related to the recent changes by Christian
> about int/long on win64.
> If they break the translation, we should revert them.

I was first thinking that it's just a warning, but further downstream, the
translation still fails for me (translation of default, not the branch). See
the error message below.

There's a party I have to go to, so I'll leave it for a bit. I'll check the
push stream on bitbucket and I'll see a fix appear. ;)

Best regards,
            Wim

[translation:ERROR]    File "/home/wlav/pypy-dev/pypy/pypy/annotation/unaryop.py", line 678, in simple_call
[translation:ERROR]     return bltn.analyser(*args)
[translation:ERROR]    File "/home/wlav/pypy-dev/pypy/pypy/annotation/builtin.py", line 156, in builtin_isinstance
[translation:ERROR]     if issubclass(typ, pypy.rlib.rarithmetic.base_int):
[translation:ERROR]  TypeError': issubclass() arg 1 must be a class
[translation:ERROR]     .. v2364 = simple_call((builtin_function_or_method isinstance), r_0, ((<type 'int'>, <type 'long'>)))
[translation:ERROR]     .. '(pypy.rlib.rarithmetic:139)is_valid_int'
[translation:ERROR] Processing block:
[translation:ERROR]  block at -1 is a <class 'pypy.objspace.flow.flowcontext.SpamBlock'>
[translation:ERROR]  in (pypy.rlib.rarithmetic:139)is_valid_int
[translation:ERROR]  containing the following operations:
[translation:ERROR]        v2364 = simple_call((builtin_function_or_method isinstance), r_0, ((<type 'int'>, <type 'long'>)))
[translation:ERROR]        v2365 = is_true(v2364)
[translation:ERROR]  --end--

-- 
WLavrijsen at lbl.gov    --    +1 (510) 486 6411    --    www.lavrijsen.net

From amauryfa at gmail.com  Wed Mar 14 19:00:41 2012
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Wed, 14 Mar 2012 19:00:41 +0100
Subject: [pypy-dev] base class hierarchy for string objects breaks
	translation
In-Reply-To: <alpine.LNX.2.00.1203141046050.23704@pc3-wlav.lbl.gov>
References: <alpine.LNX.2.00.1203140950200.21278@pc3-wlav.lbl.gov>
	<4F60CF6D.4090003@gmail.com>
	<alpine.LNX.2.00.1203141010270.21278@pc3-wlav.lbl.gov>
	<4F60D556.2010009@gmail.com>
	<alpine.LNX.2.00.1203141046050.23704@pc3-wlav.lbl.gov>
Message-ID: <CAGmFidYnExMCjY+SQsyCV3aSr=NmVWr3AXrLxGtfqQ_gRJ9H3g@mail.gmail.com>

2012/3/14  <wlavrijsen at lbl.gov>:
> [translation:ERROR] ?TypeError': issubclass() arg 1 must be a class
> [translation:ERROR] ? ? .. v2364 = simple_call((builtin_function_or_method
> isinstance), r_0, ((<type 'int'>, <type 'long'>)))
> [translation:ERROR] ? ? .. '(pypy.rlib.rarithmetic:139)is_valid_int'
> [translation:ERROR] Processing block:
> [translation:ERROR] ?block at -1 is a <class
> 'pypy.objspace.flow.flowcontext.SpamBlock'>
> [translation:ERROR] ?in (pypy.rlib.rarithmetic:139)is_valid_int
>
> [translation:ERROR] ?containing the following operations:
> [translation:ERROR] ? ? ? ?v2364 = simple_call((builtin_function_or_method
> isinstance), r_0, ((<type 'int'>, <type 'long'>)))
> [translation:ERROR] ? ? ? ?v2365 = is_true(v2364)
> [translation:ERROR] ?--end--

Bah, isinstance(x, (int,long)) is not RPython.
  isinstance(x, int) or isinstance(x, long)
should be used instead.


-- 
Amaury Forgeot d'Arc

From stefan_ml at behnel.de  Wed Mar 14 21:04:21 2012
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 14 Mar 2012 21:04:21 +0100
Subject: [pypy-dev] GIL in cpyext extensions
Message-ID: <jjqtk5$gvf$1@dough.gmane.org>

Hi,

I stumbled over the following paragraph in

https://bugs.pypy.org/issue1069

"""
PyEval_SaveThread() always return NULL in pypy.  But I don't know if there
is a better value, since the GIL is always released in C code.
"""

Why should the GIL always be released? There are many C extensions that use
it in one way or another to protect their internal state, to implement fast
(optimistic) locking, or whatever. In order to be compatible with them, the
GIL should be held unless explicitly released, just like CPython does it.

Granted, in many cases, it wouldn't have to be the GIL, it could just be a
separate "C code lock". But it's impossible to take this decision in general.

Stefan


From wlavrijsen at lbl.gov  Thu Mar 15 05:47:32 2012
From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov)
Date: Wed, 14 Mar 2012 21:47:32 -0700 (PDT)
Subject: [pypy-dev] base class hierarchy for string objects breaks
 translation
In-Reply-To: <CAGmFidYnExMCjY+SQsyCV3aSr=NmVWr3AXrLxGtfqQ_gRJ9H3g@mail.gmail.com>
References: <alpine.LNX.2.00.1203140950200.21278@pc3-wlav.lbl.gov>
	<4F60CF6D.4090003@gmail.com>
	<alpine.LNX.2.00.1203141010270.21278@pc3-wlav.lbl.gov>
	<4F60D556.2010009@gmail.com>
	<alpine.LNX.2.00.1203141046050.23704@pc3-wlav.lbl.gov>
	<CAGmFidYnExMCjY+SQsyCV3aSr=NmVWr3AXrLxGtfqQ_gRJ9H3g@mail.gmail.com>
Message-ID: <alpine.LNX.2.00.1203142146170.23704@pc3-wlav.lbl.gov>

Hi Amaury,

> Bah, isinstance(x, (int,long)) is not RPython.
>  isinstance(x, int) or isinstance(x, long)
> should be used instead.

yes, that got me further. There was one more fix needed, done by Armin.
Back in business now, thanks all!

Best regards,
            Wim
-- 
WLavrijsen at lbl.gov    --    +1 (510) 486 6411    --    www.lavrijsen.net

From cfbolz at gmx.de  Thu Mar 15 11:26:47 2012
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Thu, 15 Mar 2012 11:26:47 +0100
Subject: [pypy-dev] Call for Papers: Dynamic Languages Symposium 2012
Message-ID: <4F61C3E7.3010404@gmx.de>

Dynamic Languages Symposium 2012

October 22, 2012

Co-located with SPLASH 2012
In association with ACM SIGPLAN
Tucson, Arizona, USA

http://www.dynamic-languages-symposium.org/dls-12/

**** Call for Papers ****

The 8th Dynamic Languages Symposium (DLS) at SPLASH 2012 is a forum for 
the discussion of dynamic languages, their implementation and 
application. While mature dynamic languages including Smalltalk, Lisp, 
Scheme, Self, Prolog, and APL continue to grow and inspire new converts, 
a new generation of dynamic scripting languages such as JavaScript, 
Python, Ruby, PHP, Tcl, Lua, and Clojure are successful in a wide range 
of applications. DLS provides a place for researchers and practitioners 
to come together and share their knowledge, experience, and ideas for 
future research and development.

DLS 2012 invites high quality papers reporting original research, 
innovative contributions or experience related to dynamic languages, 
their implementation and application. Accepted Papers will be published 
in the ACM Digital Library.

Areas of interest include but are not limited to:

  * Innovative language features and implementation techniques
  * Development and platform support, tools
  * Interesting applications
  * Domain-oriented programming
  * Very late binding, dynamic composition, and runtime adaptation
  * Reflection and meta-programming
  * Software evolution
  * Language symbiosis and multi-paradigm languages
  * Dynamic optimization
  * Hardware support
  * Experience reports and case studies
  * Educational approaches and perspectives
  * Object-oriented, aspect-oriented, and context-oriented programming

=== Submissions and proceedings ===

We invite original contributions that neither have been published 
previously nor are under review by other refereed events or 
publications. Research papers should describe work that advances the 
current state of the art. Experience papers should be of broad interest 
and should describe insights gained from substantive practical 
applications. The program committee will evaluate each contributed paper 
based on its relevance, significance, clarity, length, and originality.

Papers should be of a length appropriate to their content: a shorter 
paper may be sufficient to describe a smaller but still significant 
result, and no paper will be rated poorly solely based on its length.

Accepted papers will be published in the ACM Digital Library.

Papers are to be submitted electronically at 
http://www.easychair.org/conferences?conf=dls12 in PDF format. 
Submissions should not exceed 12 pages and need to use the ACM format, 
templates for which can be found at 
http://www.sigplan.org/authorInformation.htm.

=== Important dates ===

Submission of papers: July 11, 2012 (hard deadline)
Author notification: August 1, 2012
Final versions due: August 15, 2012
DLS 2012: October 22, 2012
SPLASH 2012: October 19-26, 2012

=== Program chair ===

Alessandro Warth, Google, USA
		
=== Program committee ===

  * Henry Baker, USA
  * Carl Friedrich Bolz, Heinrich-Heine-Universit?t D?sseldorf, Germany
  * Gilad Bracha, Google, USA
  * William R. Cook, University of Texas at Austin, USA
  * Pascal Costanza, Intel, Belgium
  * Tom Van Cutsem, Vrije Universiteit Brussel, Belgium
  * Marcus Denker, Inria, France
  * Matthew Flatt, University of Utah, USA
  * Charlotte Herzeel, Vrije Universiteit Brussel, Belgium
  * Robert Hirschfeld, Hasso-Plattner-Institut Potsdam, Germany
  * Sam Tobin-Hochstadt, Northeastern University, USA
  * Mark Miller, Google, USA
  * Todd Millstein, University of California, Los Angeles, USA
  * James Noble, Victoria University of Wellington, New Zealand
  * Laurence Tratt, King's College London, UK
  * Alessandro Warth, Google, USA (chair)


From alex0player at gmail.com  Thu Mar 15 11:46:50 2012
From: alex0player at gmail.com (Alexander Sedov)
Date: Thu, 15 Mar 2012 14:46:50 +0400
Subject: [pypy-dev] Finite automata
Message-ID: <CAL7ARbRozG78hr=_AJuZB28POp6rDCdrBx4d2qYjYfCgME0mmA@mail.gmail.com>

Hello PyPy community,
Looks like I have a patch that somehow simplifies automatas generated
by pypy.interpreter.pyparser.{pylexer,genpytokenize}. It's rather
obvious patch that enables:
* Comparing labels in right order;
* Checking for loops;
* Checking for mutual links.
It is mathematically correct, although it does NOT produce minimal
automata in some cases. It reduces number of resulting states, though.
I'm having troubles with pytest (no such option: --assertmode) so I
could not test it.
If someone could test it and commit it, I'd be happy.
If unit tests are required, I'll write 'em.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: automata.patch
Type: text/x-patch
Size: 9259 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120315/f87cd77d/attachment-0001.bin>

From Ronny.Pfannschmidt at gmx.de  Thu Mar 15 14:23:09 2012
From: Ronny.Pfannschmidt at gmx.de (Ronny Pfannschmidt)
Date: Thu, 15 Mar 2012 14:23:09 +0100
Subject: [pypy-dev] Finite automata
In-Reply-To: <CAL7ARbRozG78hr=_AJuZB28POp6rDCdrBx4d2qYjYfCgME0mmA@mail.gmail.com>
References: <CAL7ARbRozG78hr=_AJuZB28POp6rDCdrBx4d2qYjYfCgME0mmA@mail.gmail.com>
Message-ID: <4F61ED3D.30600@gmx.de>

Hello Alexander,


for testing,
please use the shipped pytest by using pytest.py

-- Ronny

On 03/15/2012 11:46 AM, Alexander Sedov wrote:
> Hello PyPy community,
> Looks like I have a patch that somehow simplifies automatas generated
> by pypy.interpreter.pyparser.{pylexer,genpytokenize}. It's rather
> obvious patch that enables:
> * Comparing labels in right order;
> * Checking for loops;
> * Checking for mutual links.
> It is mathematically correct, although it does NOT produce minimal
> automata in some cases. It reduces number of resulting states, though.
> I'm having troubles with pytest (no such option: --assertmode) so I
> could not test it.
> If someone could test it and commit it, I'd be happy.
> If unit tests are required, I'll write 'em.
>
>
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev


From alex0player at gmail.com  Thu Mar 15 17:09:17 2012
From: alex0player at gmail.com (Alexander Sedov)
Date: Thu, 15 Mar 2012 20:09:17 +0400
Subject: [pypy-dev] Finite automata
In-Reply-To: <4F61ED3D.30600@gmx.de>
References: <CAL7ARbRozG78hr=_AJuZB28POp6rDCdrBx4d2qYjYfCgME0mmA@mail.gmail.com>
	<4F61ED3D.30600@gmx.de>
Message-ID: <CAL7ARbSHrMAA_qOV0f_2Af+Wev-m5hu+BkL307VWL4Zo+8FHUg@mail.gmail.com>

2012/3/15 Ronny Pfannschmidt <Ronny.Pfannschmidt at gmx.de>:
> Hello Alexander,
>
>
> for testing,
> please use the shipped pytest by using pytest.py
Fails for me with multiple:
E       AttributeError: TestParsetring instance has no attribute 'space'

From benjamin at python.org  Thu Mar 15 17:15:25 2012
From: benjamin at python.org (Benjamin Peterson)
Date: Thu, 15 Mar 2012 11:15:25 -0500
Subject: [pypy-dev] Finite automata
In-Reply-To: <CAL7ARbSHrMAA_qOV0f_2Af+Wev-m5hu+BkL307VWL4Zo+8FHUg@mail.gmail.com>
References: <CAL7ARbRozG78hr=_AJuZB28POp6rDCdrBx4d2qYjYfCgME0mmA@mail.gmail.com>
	<4F61ED3D.30600@gmx.de>
	<CAL7ARbSHrMAA_qOV0f_2Af+Wev-m5hu+BkL307VWL4Zo+8FHUg@mail.gmail.com>
Message-ID: <CAPZV6o-EysVxg1XuZ9v0VW1mmB9Wy=KfqV-mJ_f-2pN=T5WBoQ@mail.gmail.com>

Try running it from the top of the pypy source tree.

2012/3/15 Alexander Sedov <alex0player at gmail.com>:
> 2012/3/15 Ronny Pfannschmidt <Ronny.Pfannschmidt at gmx.de>:
>> Hello Alexander,
>>
>>
>> for testing,
>> please use the shipped pytest by using pytest.py
> Fails for me with multiple:
> E ? ? ? AttributeError: TestParsetring instance has no attribute 'space'
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev



-- 
Regards,
Benjamin

From alex0player at gmail.com  Thu Mar 15 17:50:37 2012
From: alex0player at gmail.com (Alexander Sedov)
Date: Thu, 15 Mar 2012 20:50:37 +0400
Subject: [pypy-dev] Finite automata
In-Reply-To: <CAPZV6o-EysVxg1XuZ9v0VW1mmB9Wy=KfqV-mJ_f-2pN=T5WBoQ@mail.gmail.com>
References: <CAL7ARbRozG78hr=_AJuZB28POp6rDCdrBx4d2qYjYfCgME0mmA@mail.gmail.com>
	<4F61ED3D.30600@gmx.de>
	<CAL7ARbSHrMAA_qOV0f_2Af+Wev-m5hu+BkL307VWL4Zo+8FHUg@mail.gmail.com>
	<CAPZV6o-EysVxg1XuZ9v0VW1mmB9Wy=KfqV-mJ_f-2pN=T5WBoQ@mail.gmail.com>
Message-ID: <CAL7ARbQRyCdQP5vUh_GK_dNkcur1QHXRBZyudDq55evXdNuMhQ@mail.gmail.com>

2012/3/15 Benjamin Peterson <benjamin at python.org>:
> Try running it from the top of the pypy source tree.
I did.

From arigo at tunes.org  Thu Mar 15 18:56:06 2012
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 15 Mar 2012 10:56:06 -0700
Subject: [pypy-dev] Finite automata
In-Reply-To: <CAL7ARbSHrMAA_qOV0f_2Af+Wev-m5hu+BkL307VWL4Zo+8FHUg@mail.gmail.com>
References: <CAL7ARbRozG78hr=_AJuZB28POp6rDCdrBx4d2qYjYfCgME0mmA@mail.gmail.com>
	<4F61ED3D.30600@gmx.de>
	<CAL7ARbSHrMAA_qOV0f_2Af+Wev-m5hu+BkL307VWL4Zo+8FHUg@mail.gmail.com>
Message-ID: <CAMSv6X2BWwBAFS0njWmM1-SX-n8m9MRjug7L14Us=ogzS+8tCg@mail.gmail.com>

Hi Alexander,

On Thu, Mar 15, 2012 at 09:09, Alexander Sedov <alex0player at gmail.com> wrote:
> Fails for me with multiple:
> E ? ? ? AttributeError: TestParsetring instance has no attribute 'space'

Then either the tests or the code has a bug.  Compare it with running
the same pytest.py on the unmodified "default" branch.  Feel free to
join #pypy on irc.freenode.net for more immediate information :-)


A bient?t,

Armin.

From arigo at tunes.org  Thu Mar 15 19:06:53 2012
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 15 Mar 2012 11:06:53 -0700
Subject: [pypy-dev] Speeding up zlib in standard library
In-Reply-To: <CAKVJ-_4zry6WMzg7Gfu2zUj=t-fpce_givh4TFj57aHiagwQ+Q@mail.gmail.com>
References: <CAKVJ-_5D4OUBSkavYqW7usBkaZMxu-EjEtb9NaKU-iCMA=XZQg@mail.gmail.com>
	<CAPZV6o8fpWjG2d8L6ig4SQds=EAUnb=sypeor9HzazdmXyZy=g@mail.gmail.com>
	<CAKVJ-_4zry6WMzg7Gfu2zUj=t-fpce_givh4TFj57aHiagwQ+Q@mail.gmail.com>
Message-ID: <CAMSv6X1vXnT6jRs2MHvwa613nfZfrqgt9sEVbB0tk6tX5+Ggew@mail.gmail.com>

Hi,

On Wed, Mar 14, 2012 at 03:19, Peter Cock <p.j.a.cock at googlemail.com> wrote:
> I don't know - I was assuming any buffering would be the same
> comparing PyPy 1.8 against Python 2.6 (and 3.2). That was one
> reason for my email - is binding to C relatively slow (compared to
> the rest of PyPy running pure Python)?

Not necessarily.  You get direct C calls, both from the translated
pypy and from JITted assembler code.  There are performance hits when
e.g. the C library relies on macros, but I don't think that's the case
of zlib.

Passing big strings around, on the other hand, is typically slower on
PyPy because they need to be copied between GC-managed areas and
non-GC-managed areas.  There are vague ideas on how to improve but
nothing I can summarize in two words.

At this level, for profiling, you can use valgrind.  You'll see the
time spent in zlib itself, the time spent copying big strings around,
and the time spent actually executing the JIT-generated assembler
(this ends up in "functions" with no name, just an address).


A bient?t,

Armin.

From p.j.a.cock at googlemail.com  Thu Mar 15 19:30:10 2012
From: p.j.a.cock at googlemail.com (Peter Cock)
Date: Thu, 15 Mar 2012 18:30:10 +0000
Subject: [pypy-dev] Speeding up zlib in standard library
In-Reply-To: <CAMSv6X1vXnT6jRs2MHvwa613nfZfrqgt9sEVbB0tk6tX5+Ggew@mail.gmail.com>
References: <CAKVJ-_5D4OUBSkavYqW7usBkaZMxu-EjEtb9NaKU-iCMA=XZQg@mail.gmail.com>
	<CAPZV6o8fpWjG2d8L6ig4SQds=EAUnb=sypeor9HzazdmXyZy=g@mail.gmail.com>
	<CAKVJ-_4zry6WMzg7Gfu2zUj=t-fpce_givh4TFj57aHiagwQ+Q@mail.gmail.com>
	<CAMSv6X1vXnT6jRs2MHvwa613nfZfrqgt9sEVbB0tk6tX5+Ggew@mail.gmail.com>
Message-ID: <CAKVJ-_5jtHOxXpcnUVojV-a070BWV8fybZ99EKaZqjT9+DoLJA@mail.gmail.com>

On Thursday, March 15, 2012, Armin Rigo <arigo at tunes.org> wrote:
> Hi,
>
> On Wed, Mar 14, 2012 at 03:19, Peter Cock <p.j.a.cock at googlemail.com>
wrote:
>> I don't know - I was assuming any buffering would be the same
>> comparing PyPy 1.8 against Python 2.6 (and 3.2). That was one
>> reason for my email - is binding to C relatively slow (compared to
>> the rest of PyPy running pure Python)?
>
> Not necessarily.  You get direct C calls, both from the translated
> pypy and from JITted assembler code.  There are performance hits when
> e.g. the C library relies on macros, but I don't think that's the case
> of zlib.
>
> Passing big strings around, on the other hand, is typically slower on
> PyPy because they need to be copied between GC-managed areas and
> non-GC-managed areas.  There are vague ideas on how to improve but
> nothing I can summarize in two words.
>
> At this level, for profiling, you can use valgrind.  You'll see the
> time spent in zlib itself, the time spent copying big strings around,
> and the time spent actually executing the JIT-generated assembler
> (this ends up in "functions" with no name, just an address).

I think in my case it could be this "big string" issue then, rather
than the interface with zlib  itself. I'm dealing with 64kb chunks
of data which are zlib compressed, and I'm using (bytes) strings
to hold these in Python.

I've used valgrind before, but never with PyPy - hopefully I can
find some time to dig into this a bit further.

Thanks,

Peter

P.S. This is for blocked gzip format (BGZF) if you're curious,
http://blastedbio.blogspot.com/2011/11/bgzf-blocked-bigger-better-gzip.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120315/c3b75dce/attachment-0001.html>

From muranosrl at ciudad.com.ar  Thu Mar 15 19:42:09 2012
From: muranosrl at ciudad.com.ar (=?utf-8?B?5YyX5Lqs54us6LWE5ZWG6LS45YWs5Y+4?=)
Date: Fri, 16 Mar 2012 02:42:09 +0800
Subject: [pypy-dev] =?utf-8?b?5YWs5Y+45paw6KeELTAwODA3Njjlj7c=?=
Message-ID: <A7245B76C91158EC15C01D867BF199CD@fgrbne>


Sales elite two days crazy night training, please consult
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120316/662498a5/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 3????????8312.xls
Type: application/vnd.ms-excel
Size: 28160 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120316/662498a5/attachment-0001.xls>

From timo at wakelift.de  Thu Mar 15 22:35:59 2012
From: timo at wakelift.de (Timo Paulssen)
Date: Thu, 15 Mar 2012 22:35:59 +0100
Subject: [pypy-dev] towards more parallelization in the tracer/optimizer
Message-ID: <4F6260BF.1010205@wakelift.de>

Hello,

I just watched Benjamin Petersons talk on the JIT and one comment that seemed 
like just a sidethought caught my attention:

"we are limited by speed. we don't want to spend three seconds optimizing it as 
fast as gcc possibly could, because noone wants the program to stop for three 
seconds."

If the raw trace - or a minimally optimized version of it - can be run directly, 
why not run it once or thrice while waiting for the optimizer to finish 
optimizing the trace fully? - with clever register allocation and whatnot.

I guess that, in most cases, the tracer + optimizer will kick in for a loop in 
"the middle" of many iterations, meaning that after the trace is optimized, it 
will be run a few more times, maybe hundreds.

Now consider this: If the raw or "minimally optimized" trace can be generated 
very quickly and run maybe a couple of times while the full optimizer runs 
completely in parallel, then even if the optimizer takes longer than the one 
pypy currently uses (because of speed limitations), there will still be a win, 
because a few more iterations have already been finished in parallel.

The win will be even more amazing if the tracer kicks in at the very last 
iteration of a loop, meaning that the program - that would have stopped for the 
optimization previously - can now return from the function or go do some 
completely unrelated work, but the next time be very much faster.

So the benefits of this are two-fold: One, pypys jit can afford more expensive 
optimizations and Two, scripts run in pypy will experience less stuttering if 
there are many different loops hitting the threshold for being considered hot.


Another related idea is to see if there are optimizations that can be run in 
parallel on an incomplete trace, so that when the trace finishes, the 
optimization is already half-done.


Since Pypy still has the GIL (and the STM fork doesn't have a JIT yet), I 
believe it's beneficial to use a second core for miscellaneous tasks, such as 
offloading the optimizations. Combined with a more efficient parallel garbage 
collector, this could lead to smoother speed changes in pypy, as before you'd 
have first slow -> standstill -> blazing fast. In such a parallelized version 
you might end up with slow -> unnoticable pause -> faster -> blazing fast.

Thank you :)
   - Timo

PS: there's also an incomplete branch that would allow pypy to use weaker 
optimizations on loops that have not run as often as the current trace limit. 
The approach here would IMO still be better, because when the trace limit is 
hit, there is no longer a second pause.

From alex0player at gmail.com  Sat Mar 17 22:17:32 2012
From: alex0player at gmail.com (Alexander Sedov)
Date: Sun, 18 Mar 2012 01:17:32 +0400
Subject: [pypy-dev] Finite automata
In-Reply-To: <CAMSv6X2BWwBAFS0njWmM1-SX-n8m9MRjug7L14Us=ogzS+8tCg@mail.gmail.com>
References: <CAL7ARbRozG78hr=_AJuZB28POp6rDCdrBx4d2qYjYfCgME0mmA@mail.gmail.com>
	<4F61ED3D.30600@gmx.de>
	<CAL7ARbSHrMAA_qOV0f_2Af+Wev-m5hu+BkL307VWL4Zo+8FHUg@mail.gmail.com>
	<CAMSv6X2BWwBAFS0njWmM1-SX-n8m9MRjug7L14Us=ogzS+8tCg@mail.gmail.com>
Message-ID: <CAL7ARbTfhWwNsf+5A37ZpB4VyAB-1L5FqE_vpUafsNn_57v3pg@mail.gmail.com>

2012/3/15 Armin Rigo <arigo at tunes.org>:
> Hi Alexander,
>
> On Thu, Mar 15, 2012 at 09:09, Alexander Sedov <alex0player at gmail.com> wrote:
>> Fails for me with multiple:
>> E ? ? ? AttributeError: TestParsetring instance has no attribute 'space'
>
> Then either the tests or the code has a bug. ?Compare it with running
> the same pytest.py on the unmodified "default" branch. ?Feel free to
> join #pypy on irc.freenode.net for more immediate information :-)
Thanks  for inviting me, I used to be there, but now I'm kinda behind firewall.
So mailing list is the only way.
If I was given a permission to commit, I won't be fooling around with
these patches. Actually, I was, but on SVN. I even wrote big part of
mcronumpy. (Now all my code is gone, of course). If you add me to
developers, it'd be wonderful.
As I wrote you, problems are not connected with my code, as I have
reverted everything before running.
Anyway, after updating errors are replaced with other ones. I'll have
time to deal with it.

Still asking somebody to try my patch.

From n.c.alday at gmail.com  Mon Mar 19 03:49:08 2012
From: n.c.alday at gmail.com (Nathan Alday)
Date: Sun, 18 Mar 2012 21:49:08 -0500
Subject: [pypy-dev] String replace() method causing error on translate
Message-ID: <CAA+AE1B=DB9UtP0iR-G6Om30+2oeaFyVjwOb6sUSLfkTKym+jg@mail.gmail.com>

I'm trying to see if I can get Peter Norvig's lis.py (from here:
http://norvig.com/lispy.html) to translate using translate.py, but I get
the following error:

AnnotatorError:
[translation:ERROR]
-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
[translation:ERROR] Blocked block -- operation cannot succeed
[translation:ERROR]  v0 = getattr(s_0, ('replace'))
[translation:ERROR] In <FunctionGraph of (lis:83)tokenize at 0x8f646e4>:
[translation:ERROR] Happened at file lispy/lis.py line 85
[translation:ERROR]
[translation:ERROR] ==>     output = s.replace('(',' ( ')
[translation:ERROR]         output = output.replace( ')', ' ) ')
[translation:ERROR]         return output.split()
[translation:ERROR]
[translation:ERROR] Known variable annotations:
[translation:ERROR]  s_0 = SomePBC(can_be_None=True, const=None,
subset_of=None)



caused by:

def tokenize(s):
    "Convert a string into a list of tokens."
    output = s.replace('(',' ( ')
    output = output.replace( ')', ' ) ')
    return output.split()


It's not clear to me what the issue is. Is there another type that has a
replace method that is confusing the type inference system?

Thanks,
--Nathan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120318/176a513e/attachment.html>

From benjamin at python.org  Mon Mar 19 04:12:49 2012
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 18 Mar 2012 22:12:49 -0500
Subject: [pypy-dev] String replace() method causing error on translate
In-Reply-To: <CAA+AE1B=DB9UtP0iR-G6Om30+2oeaFyVjwOb6sUSLfkTKym+jg@mail.gmail.com>
References: <CAA+AE1B=DB9UtP0iR-G6Om30+2oeaFyVjwOb6sUSLfkTKym+jg@mail.gmail.com>
Message-ID: <CAPZV6o_dL41-EaroGJNqRhAX+3X05LuqV9ii2k-wH7AEOyy8SQ@mail.gmail.com>

2012/3/18 Nathan Alday <n.c.alday at gmail.com>:\
> It's not clear to me what the issue is. Is there another type that has a
> replace method that is confusing the type inference system?

It appears the annotator sees someone passing None into tokenize.


-- 
Regards,
Benjamin

From stefan_ml at behnel.de  Mon Mar 19 07:44:12 2012
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Mon, 19 Mar 2012 07:44:12 +0100
Subject: [pypy-dev] Bringing Cython and PyPy closer together
In-Reply-To: <jie2mh$qub$1@dough.gmane.org>
References: <jhg54k$dte$1@dough.gmane.org> <jhnok5$ve2$1@dough.gmane.org>
	<CAGmFidbutXahNAKbHgaLFOM39uTKZ8C3TkrPLeauB0Jm5CQ-bg@mail.gmail.com>
	<jho9he$c2k$1@dough.gmane.org>
	<CAGmFidZ2DDxnPBi7E5Bo8PTn3iubjfrtxmBv1uEsaombo2ey1Q@mail.gmail.com>
	<jhobh9$ots$1@dough.gmane.org>
	<CAGmFidbEPpbQPrKEsNHkMQ1e3mgsYV1vd0SivdVPvSOYaLSXmA@mail.gmail.com>
	<jie2mh$qub$1@dough.gmane.org>
Message-ID: <jk6kjs$840$1@dough.gmane.org>

Stefan Behnel, 26.02.2012 20:54:
> Amaury Forgeot d'Arc, 18.02.2012 15:41:
>> 2012/2/18 Stefan Behnel
>>> Here's an example.
>>>
>>> Python code:
>>>
>>>  def print_excinfo():
>>>      print(sys.exc_info())
>>>
>>> Cython code:
>>>
>>>  from stuff import print_excinfo
>>>
>>>  try:
>>>      raise TypeError
>>>  except TypeError:
>>>      print_excinfo()
>>>
>>> With the code removed, Cython will not store the TypeError in
>>> sys.exc_info(), so the Python code cannot see it. This may seem like an
>>> unimportant use case (who uses sys.exc_info() anyway, right?), but this
>>> becomes very visible when the code that uses sys.exc_info() is not user
>>> code but CPython itself, e.g. when raising another exception or when
>>> inspecting frames. Things grow really bad here, especially in Python 3.
>>
>> I think I understand now, thanks for your example.
>> Things are a bit simpler in PyPy because these exceptions are
>> stored in the frame that is currently handling it. At least better than
>> CPython
>> which stores it in one place, and has to somehow save the state of the
>> previous frames.
>> Did you consider adding such a function to CPython?
>> "PyErr_SetCurrentFrameExceptionInfo"?
>>
>> For the record, pypy could implement it as:
>> space.getexecutioncontext().gettopframe_nohidden().last_exception =
>> operationerr
>> i.e. the thing returned by sys.exc_info().
> 
> I've dropped a patch for CPython in the corresponding tracker ticket:
> 
> http://bugs.python.org/issue14098
> 
> The (trivial) implementation of the two functions is at the end of this file:
> 
> http://bugs.python.org/file24613/exc_info_capi.patch
> 
> Could you add them to PyPy?

Anyone?

Stefan


From fijall at gmail.com  Mon Mar 19 09:11:13 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Mon, 19 Mar 2012 10:11:13 +0200
Subject: [pypy-dev] Bringing Cython and PyPy closer together
In-Reply-To: <jk6kjs$840$1@dough.gmane.org>
References: <jhg54k$dte$1@dough.gmane.org> <jhnok5$ve2$1@dough.gmane.org>
	<CAGmFidbutXahNAKbHgaLFOM39uTKZ8C3TkrPLeauB0Jm5CQ-bg@mail.gmail.com>
	<jho9he$c2k$1@dough.gmane.org>
	<CAGmFidZ2DDxnPBi7E5Bo8PTn3iubjfrtxmBv1uEsaombo2ey1Q@mail.gmail.com>
	<jhobh9$ots$1@dough.gmane.org>
	<CAGmFidbEPpbQPrKEsNHkMQ1e3mgsYV1vd0SivdVPvSOYaLSXmA@mail.gmail.com>
	<jie2mh$qub$1@dough.gmane.org> <jk6kjs$840$1@dough.gmane.org>
Message-ID: <CAK5idxThPMvpoFGODum85_pnd6ny9MZ81i7961uYvSkWODrqKg@mail.gmail.com>

On Mon, Mar 19, 2012 at 8:44 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Stefan Behnel, 26.02.2012 20:54:
>> Amaury Forgeot d'Arc, 18.02.2012 15:41:
>>> 2012/2/18 Stefan Behnel
>>>> Here's an example.
>>>>
>>>> Python code:
>>>>
>>>> ?def print_excinfo():
>>>> ? ? ?print(sys.exc_info())
>>>>
>>>> Cython code:
>>>>
>>>> ?from stuff import print_excinfo
>>>>
>>>> ?try:
>>>> ? ? ?raise TypeError
>>>> ?except TypeError:
>>>> ? ? ?print_excinfo()
>>>>
>>>> With the code removed, Cython will not store the TypeError in
>>>> sys.exc_info(), so the Python code cannot see it. This may seem like an
>>>> unimportant use case (who uses sys.exc_info() anyway, right?), but this
>>>> becomes very visible when the code that uses sys.exc_info() is not user
>>>> code but CPython itself, e.g. when raising another exception or when
>>>> inspecting frames. Things grow really bad here, especially in Python 3.
>>>
>>> I think I understand now, thanks for your example.
>>> Things are a bit simpler in PyPy because these exceptions are
>>> stored in the frame that is currently handling it. At least better than
>>> CPython
>>> which stores it in one place, and has to somehow save the state of the
>>> previous frames.
>>> Did you consider adding such a function to CPython?
>>> "PyErr_SetCurrentFrameExceptionInfo"?
>>>
>>> For the record, pypy could implement it as:
>>> space.getexecutioncontext().gettopframe_nohidden().last_exception =
>>> operationerr
>>> i.e. the thing returned by sys.exc_info().
>>
>> I've dropped a patch for CPython in the corresponding tracker ticket:
>>
>> http://bugs.python.org/issue14098
>>
>> The (trivial) implementation of the two functions is at the end of this file:
>>
>> http://bugs.python.org/file24613/exc_info_capi.patch
>>
>> Could you add them to PyPy?
>
> Anyone?
>
> Stefan

Hi Stefan.

A lot of people have been completely busy at pycon. As far as I know
they're trying to recover from jetlag/lots of conference. Give them a
bit of breath ;-)

From fijall at gmail.com  Mon Mar 19 09:12:09 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Mon, 19 Mar 2012 10:12:09 +0200
Subject: [pypy-dev] String replace() method causing error on translate
In-Reply-To: <CAPZV6o_dL41-EaroGJNqRhAX+3X05LuqV9ii2k-wH7AEOyy8SQ@mail.gmail.com>
References: <CAA+AE1B=DB9UtP0iR-G6Om30+2oeaFyVjwOb6sUSLfkTKym+jg@mail.gmail.com>
	<CAPZV6o_dL41-EaroGJNqRhAX+3X05LuqV9ii2k-wH7AEOyy8SQ@mail.gmail.com>
Message-ID: <CAK5idxTo1yWQ622qryi57E7kv3AzjdYMs+DFbWyn+HQToOMtZg@mail.gmail.com>

On Mon, Mar 19, 2012 at 5:12 AM, Benjamin Peterson <benjamin at python.org> wrote:
> 2012/3/18 Nathan Alday <n.c.alday at gmail.com>:\
>> It's not clear to me what the issue is. Is there another type that has a
>> replace method that is confusing the type inference system?
>
> It appears the annotator sees someone passing None into tokenize.

Not only that, but noone ever passes anything else than None. And
obviously None has no replace

From p.j.a.cock at googlemail.com  Mon Mar 19 16:49:13 2012
From: p.j.a.cock at googlemail.com (Peter Cock)
Date: Mon, 19 Mar 2012 15:49:13 +0000
Subject: [pypy-dev] Speeding up zlib in standard library
In-Reply-To: <CA+=c-xbzg8k3M_d6N=JsV1aaHRHv-2P1Cm-cSSPUdUh6jYJsnA@mail.gmail.com>
References: <CAKVJ-_5D4OUBSkavYqW7usBkaZMxu-EjEtb9NaKU-iCMA=XZQg@mail.gmail.com>
	<CAPZV6o8fpWjG2d8L6ig4SQds=EAUnb=sypeor9HzazdmXyZy=g@mail.gmail.com>
	<CAKVJ-_4zry6WMzg7Gfu2zUj=t-fpce_givh4TFj57aHiagwQ+Q@mail.gmail.com>
	<CAMSv6X1vXnT6jRs2MHvwa613nfZfrqgt9sEVbB0tk6tX5+Ggew@mail.gmail.com>
	<CAKVJ-_5jtHOxXpcnUVojV-a070BWV8fybZ99EKaZqjT9+DoLJA@mail.gmail.com>
	<CA+=c-xbzg8k3M_d6N=JsV1aaHRHv-2P1Cm-cSSPUdUh6jYJsnA@mail.gmail.com>
Message-ID: <CAKVJ-_6Ov+MwjHginrmpyga2TLWZkxP9mFSaSZcUYToNDRkm0g@mail.gmail.com>

(Replying on list - assuming Justin went off list my mistake)

On Fri, Mar 16, 2012 at 4:54 AM, Justin Peel <peelpy at gmail.com> wrote:
> Two things to mention. First, if you are going to use valgrind, you
> will need to build your own pypy because, as far as I know, the
> buildbot ones do not have the debug info so you won't have any helpful
> function names in your profile. If I remember correctly, most of the
> time used is in _operate in rzlib.py.
>
> Also, in my opinion, the next thing to try in speeding up this code is
> to do a faster copy than a char by char copy for copying to the input
> buffer that is sent to the external C function. I'm not sure if the
> copying from the output buffer to the string builder is char by char
> or not.

Looking at the code for rzlib.py, _operate does seem to be the
core function, and so likely the hot spot.
https://bitbucket.org/pypy/pypy/src/default/pypy/rlib/rzlib.py

It uses the StringBuilder class from rstring via the append_charpsize
method, defined as follows:
https://bitbucket.org/pypy/pypy/src/default/pypy/rlib/rstring.py

    def append_charpsize(self, s, size):
        l = []
        for i in xrange(size):
            l.append(s[i])
        self.l.append(self.tp("").join(l))
        self._grow(size)

So it is indeed doing a char by char copy of the string from Python
to zlib (in my case to decompress a long chunk of data). I don't
know enough about PyPy's internals to say if something naive
like this would work faster (guessing looking at the append_slice
method):

    def append_charpsize(self, s, size):
        assert 0 <= size
        self.l.append(s[0:size])
        self._grow(size)

Presumably to try this idea out I'm going to first need to get
PyPy to build locally?

Thanks,

Peter

From blume.mike at gmail.com  Mon Mar 19 18:10:53 2012
From: blume.mike at gmail.com (Michael Blume)
Date: Mon, 19 Mar 2012 10:10:53 -0700
Subject: [pypy-dev] Does anyone have a git-mapfile?
Message-ID: <CAO2U3Qg4Ordnwh=pE+gf6EYdii_beWuaHOHEd03m=uzX2uHzaQ@mail.gmail.com>

I know there's a github mirror at github.com/pypy/pypy, but in order to
shuffle commits between the two repos, I need to either perform the
translation myself (and I let an EC2 instance run on that for 9 hours...)
or get a git-mapfile with a list of corresponding commits. If you have a
bi-directional repository, it'd be found at pypy/.hg/git-mapfile.

Thanks!

-Mike
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120319/5b08acea/attachment.html>

From Ronny.Pfannschmidt at gmx.de  Mon Mar 19 18:53:55 2012
From: Ronny.Pfannschmidt at gmx.de (Ronny Pfannschmidt)
Date: Mon, 19 Mar 2012 18:53:55 +0100
Subject: [pypy-dev] Does anyone have a git-mapfile?
In-Reply-To: <CAO2U3Qg4Ordnwh=pE+gf6EYdii_beWuaHOHEd03m=uzX2uHzaQ@mail.gmail.com>
References: <CAO2U3Qg4Ordnwh=pE+gf6EYdii_beWuaHOHEd03m=uzX2uHzaQ@mail.gmail.com>
Message-ID: <4F6772B3.8020507@gmx.de>

i started a hg gexport background task on my laptop,
should finish in a hour or so

however i strongly suggest not to develop with git,
you are likely to mess up branching since git
has no idea about named branches

On 03/19/2012 06:10 PM, Michael Blume wrote:
> I know there's a github mirror at github.com/pypy/pypy, but in order to
> shuffle commits between the two repos, I need to either perform the
> translation myself (and I let an EC2 instance run on that for 9 hours...)
> or get a git-mapfile with a list of corresponding commits. If you have a
> bi-directional repository, it'd be found at pypy/.hg/git-mapfile.
>
> Thanks!
>
> -Mike
>
>
>
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev


From anto.cuni at gmail.com  Mon Mar 19 19:01:15 2012
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Mon, 19 Mar 2012 19:01:15 +0100
Subject: [pypy-dev] Does anyone have a git-mapfile?
In-Reply-To: <4F6772B3.8020507@gmx.de>
References: <CAO2U3Qg4Ordnwh=pE+gf6EYdii_beWuaHOHEd03m=uzX2uHzaQ@mail.gmail.com>
	<4F6772B3.8020507@gmx.de>
Message-ID: <4F67746B.7070505@gmail.com>

On 03/19/2012 06:53 PM, Ronny Pfannschmidt wrote:
> i started a hg gexport background task on my laptop,
> should finish in a hour or so
> 
> however i strongly suggest not to develop with git,
> you are likely to mess up branching since git
> has no idea about named branches

I don't know much about git, but I have to add that we use named branches a
lot in our workflow. In particular, if you want to issue a pull request, most
of the time we require it to be in a branch (because it's easier for us to
tell buildbot to test it before merging into default).

ciao,
Anto

From blume.mike at gmail.com  Mon Mar 19 19:05:41 2012
From: blume.mike at gmail.com (Michael Blume)
Date: Mon, 19 Mar 2012 11:05:41 -0700
Subject: [pypy-dev] Does anyone have a git-mapfile?
In-Reply-To: <4F6772B3.8020507@gmx.de>
References: <CAO2U3Qg4Ordnwh=pE+gf6EYdii_beWuaHOHEd03m=uzX2uHzaQ@mail.gmail.com>
	<4F6772B3.8020507@gmx.de>
Message-ID: <CAO2U3Qi-7t7HsTgZwg9=VCqtdbJFnxyhoKBL8Fr=J6oF9Ac83A@mail.gmail.com>

Thanks! =)

I'll either add?--HG-- branch?tags as I go or just come through and
add them when I'm about to push. Better (for me) than trying to use
Mercurial =)

On Mon, Mar 19, 2012 at 10:53 AM, Ronny Pfannschmidt
<Ronny.Pfannschmidt at gmx.de> wrote:
>
> i started a hg gexport background task on my laptop,
> should finish in a hour or so
>
> however i strongly suggest not to develop with git,
> you are likely to mess up branching since git
> has no idea about named branches
>
>
> On 03/19/2012 06:10 PM, Michael Blume wrote:
>>
>> I know there's a github mirror at github.com/pypy/pypy, but in order to
>> shuffle commits between the two repos, I need to either perform the
>> translation myself (and I let an EC2 instance run on that for 9 hours...)
>> or get a git-mapfile with a list of corresponding commits. If you have a
>> bi-directional repository, it'd be found at pypy/.hg/git-mapfile.
>>
>> Thanks!
>>
>> -Mike
>>
>>
>>
>>
>> _______________________________________________
>> pypy-dev mailing list
>> pypy-dev at python.org
>> http://mail.python.org/mailman/listinfo/pypy-dev
>
>

From fijall at gmail.com  Mon Mar 19 19:36:43 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Mon, 19 Mar 2012 20:36:43 +0200
Subject: [pypy-dev] Speeding up zlib in standard library
In-Reply-To: <CAKVJ-_6Ov+MwjHginrmpyga2TLWZkxP9mFSaSZcUYToNDRkm0g@mail.gmail.com>
References: <CAKVJ-_5D4OUBSkavYqW7usBkaZMxu-EjEtb9NaKU-iCMA=XZQg@mail.gmail.com>
	<CAPZV6o8fpWjG2d8L6ig4SQds=EAUnb=sypeor9HzazdmXyZy=g@mail.gmail.com>
	<CAKVJ-_4zry6WMzg7Gfu2zUj=t-fpce_givh4TFj57aHiagwQ+Q@mail.gmail.com>
	<CAMSv6X1vXnT6jRs2MHvwa613nfZfrqgt9sEVbB0tk6tX5+Ggew@mail.gmail.com>
	<CAKVJ-_5jtHOxXpcnUVojV-a070BWV8fybZ99EKaZqjT9+DoLJA@mail.gmail.com>
	<CA+=c-xbzg8k3M_d6N=JsV1aaHRHv-2P1Cm-cSSPUdUh6jYJsnA@mail.gmail.com>
	<CAKVJ-_6Ov+MwjHginrmpyga2TLWZkxP9mFSaSZcUYToNDRkm0g@mail.gmail.com>
Message-ID: <CAK5idxTbhQi9X7O+M2eJy-nFY6BWiFmTdghhfohjNv0y8_Yd1A@mail.gmail.com>

On Mon, Mar 19, 2012 at 5:49 PM, Peter Cock <p.j.a.cock at googlemail.com> wrote:
> (Replying on list - assuming Justin went off list my mistake)
>
> On Fri, Mar 16, 2012 at 4:54 AM, Justin Peel <peelpy at gmail.com> wrote:
>> Two things to mention. First, if you are going to use valgrind, you
>> will need to build your own pypy because, as far as I know, the
>> buildbot ones do not have the debug info so you won't have any helpful
>> function names in your profile. If I remember correctly, most of the
>> time used is in _operate in rzlib.py.
>>
>> Also, in my opinion, the next thing to try in speeding up this code is
>> to do a faster copy than a char by char copy for copying to the input
>> buffer that is sent to the external C function. I'm not sure if the
>> copying from the output buffer to the string builder is char by char
>> or not.
>
> Looking at the code for rzlib.py, _operate does seem to be the
> core function, and so likely the hot spot.
> https://bitbucket.org/pypy/pypy/src/default/pypy/rlib/rzlib.py
>
> It uses the StringBuilder class from rstring via the append_charpsize
> method, defined as follows:
> https://bitbucket.org/pypy/pypy/src/default/pypy/rlib/rstring.py
>
> ? ?def append_charpsize(self, s, size):
> ? ? ? ?l = []
> ? ? ? ?for i in xrange(size):
> ? ? ? ? ? ?l.append(s[i])
> ? ? ? ?self.l.append(self.tp("").join(l))
> ? ? ? ?self._grow(size)
>
> So it is indeed doing a char by char copy of the string from Python
> to zlib (in my case to decompress a long chunk of data). I don't
> know enough about PyPy's internals to say if something naive
> like this would work faster (guessing looking at the append_slice
> method):
>
> ? ?def append_charpsize(self, s, size):
> ? ? ? ?assert 0 <= size
> ? ? ? ?self.l.append(s[0:size])
> ? ? ? ?self._grow(size)
>
> Presumably to try this idea out I'm going to first need to get
> PyPy to build locally?
>
> Thanks,
>
> Peter
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev

append_charpsize is special - it's not the *actual* implementation,
the actual implementation is buried somewhere in
rpython/lltypesystem/rbuilder.py, with the one you're mentioning being
just fake implementation for tests. StringBuilder is special in a
sense that it has some special GC support (which we can probably
improve upon).

Cheers,
fijal

From p.j.a.cock at googlemail.com  Mon Mar 19 20:15:16 2012
From: p.j.a.cock at googlemail.com (Peter Cock)
Date: Mon, 19 Mar 2012 19:15:16 +0000
Subject: [pypy-dev] Speeding up zlib in standard library
In-Reply-To: <CAK5idxTbhQi9X7O+M2eJy-nFY6BWiFmTdghhfohjNv0y8_Yd1A@mail.gmail.com>
References: <CAKVJ-_5D4OUBSkavYqW7usBkaZMxu-EjEtb9NaKU-iCMA=XZQg@mail.gmail.com>
	<CAPZV6o8fpWjG2d8L6ig4SQds=EAUnb=sypeor9HzazdmXyZy=g@mail.gmail.com>
	<CAKVJ-_4zry6WMzg7Gfu2zUj=t-fpce_givh4TFj57aHiagwQ+Q@mail.gmail.com>
	<CAMSv6X1vXnT6jRs2MHvwa613nfZfrqgt9sEVbB0tk6tX5+Ggew@mail.gmail.com>
	<CAKVJ-_5jtHOxXpcnUVojV-a070BWV8fybZ99EKaZqjT9+DoLJA@mail.gmail.com>
	<CA+=c-xbzg8k3M_d6N=JsV1aaHRHv-2P1Cm-cSSPUdUh6jYJsnA@mail.gmail.com>
	<CAKVJ-_6Ov+MwjHginrmpyga2TLWZkxP9mFSaSZcUYToNDRkm0g@mail.gmail.com>
	<CAK5idxTbhQi9X7O+M2eJy-nFY6BWiFmTdghhfohjNv0y8_Yd1A@mail.gmail.com>
Message-ID: <CAKVJ-_5Zd_BXHa1ycDGNr3SS8txKF1wzKgZ7QydYkpQeqEXt3w@mail.gmail.com>

On Mon, Mar 19, 2012 at 6:36 PM, Maciej Fijalkowski <fijall at gmail.com> wrote:
>> http://mail.python.org/mailman/listinfo/pypy-dev
>
> append_charpsize is special - it's not the *actual* implementation,
> the actual implementation is buried somewhere in
> rpython/lltypesystem/rbuilder.py, with the one you're mentioning being
> just fake implementation for tests. StringBuilder is special in a
> sense that it has some special GC support (which we can probably
> improve upon).
>
> Cheers,
> fijal

I guess you are referring to the copy_string_contents function here:
https://bitbucket.org/pypy/pypy/src/default/pypy/rpython/lltypesystem/rstr.py

However, methods ll_append_multiple_char not ll_append_charpsize
defined in rbuilder seem to use this - they both use a for loop char-by-char,
https://bitbucket.org/pypy/pypy/src/default/pypy/rpython/lltypesystem/rbuilder.py

My hunch would be to replace this:

    @staticmethod
    def ll_append_charpsize(ll_builder, charp, size):
        used = ll_builder.used
        if used + size > ll_builder.allocated:
            ll_builder.grow(ll_builder, size)
        for i in xrange(size):
            ll_builder.buf.chars[used] = charp[i]
            used += 1
        ll_builder.used = used

with this:

    @staticmethod
    def ll_append_charpsize(ll_builder, charp, size):
        used = ll_builder.used
        if used + size > ll_builder.allocated:
            ll_builder.grow(ll_builder, size)
        assert size >= 0
        ll_str.copy_contents(charp, ll_builder.buf, 0, used, size)
        ll_builder.used += size

(and similarly for ll_append_multiple_char above it)

Like an onion - more and more layers ;) I'm beginning to suspect
speeding up append_charpsize in order to make passing strings
to/from C code faster is a bit too ambitious for a first contribution
to PyPy! [*]

Peter

[*] Especially as after three hours it is still building from source:
$ python translate.py --opt=jit targetpypystandalone.py

From fijall at gmail.com  Mon Mar 19 20:32:41 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Mon, 19 Mar 2012 21:32:41 +0200
Subject: [pypy-dev] Speeding up zlib in standard library
In-Reply-To: <CAKVJ-_5Zd_BXHa1ycDGNr3SS8txKF1wzKgZ7QydYkpQeqEXt3w@mail.gmail.com>
References: <CAKVJ-_5D4OUBSkavYqW7usBkaZMxu-EjEtb9NaKU-iCMA=XZQg@mail.gmail.com>
	<CAPZV6o8fpWjG2d8L6ig4SQds=EAUnb=sypeor9HzazdmXyZy=g@mail.gmail.com>
	<CAKVJ-_4zry6WMzg7Gfu2zUj=t-fpce_givh4TFj57aHiagwQ+Q@mail.gmail.com>
	<CAMSv6X1vXnT6jRs2MHvwa613nfZfrqgt9sEVbB0tk6tX5+Ggew@mail.gmail.com>
	<CAKVJ-_5jtHOxXpcnUVojV-a070BWV8fybZ99EKaZqjT9+DoLJA@mail.gmail.com>
	<CA+=c-xbzg8k3M_d6N=JsV1aaHRHv-2P1Cm-cSSPUdUh6jYJsnA@mail.gmail.com>
	<CAKVJ-_6Ov+MwjHginrmpyga2TLWZkxP9mFSaSZcUYToNDRkm0g@mail.gmail.com>
	<CAK5idxTbhQi9X7O+M2eJy-nFY6BWiFmTdghhfohjNv0y8_Yd1A@mail.gmail.com>
	<CAKVJ-_5Zd_BXHa1ycDGNr3SS8txKF1wzKgZ7QydYkpQeqEXt3w@mail.gmail.com>
Message-ID: <CAK5idxRrznx6ifCPD3Y1p8XHKYOwO+RmqoA=rdTKeJkhvpuSsA@mail.gmail.com>

On Mon, Mar 19, 2012 at 9:15 PM, Peter Cock <p.j.a.cock at googlemail.com> wrote:
> On Mon, Mar 19, 2012 at 6:36 PM, Maciej Fijalkowski <fijall at gmail.com> wrote:
>>> http://mail.python.org/mailman/listinfo/pypy-dev
>>
>> append_charpsize is special - it's not the *actual* implementation,
>> the actual implementation is buried somewhere in
>> rpython/lltypesystem/rbuilder.py, with the one you're mentioning being
>> just fake implementation for tests. StringBuilder is special in a
>> sense that it has some special GC support (which we can probably
>> improve upon).
>>
>> Cheers,
>> fijal
>
> I guess you are referring to the copy_string_contents function here:
> https://bitbucket.org/pypy/pypy/src/default/pypy/rpython/lltypesystem/rstr.py
>
> However, methods ll_append_multiple_char not ll_append_charpsize
> defined in rbuilder seem to use this - they both use a for loop char-by-char,
> https://bitbucket.org/pypy/pypy/src/default/pypy/rpython/lltypesystem/rbuilder.py
>
> My hunch would be to replace this:
>
> ? ?@staticmethod
> ? ?def ll_append_charpsize(ll_builder, charp, size):
> ? ? ? ?used = ll_builder.used
> ? ? ? ?if used + size > ll_builder.allocated:
> ? ? ? ? ? ?ll_builder.grow(ll_builder, size)
> ? ? ? ?for i in xrange(size):
> ? ? ? ? ? ?ll_builder.buf.chars[used] = charp[i]
> ? ? ? ? ? ?used += 1
> ? ? ? ?ll_builder.used = used
>
> with this:
>
> ? ?@staticmethod
> ? ?def ll_append_charpsize(ll_builder, charp, size):
> ? ? ? ?used = ll_builder.used
> ? ? ? ?if used + size > ll_builder.allocated:
> ? ? ? ? ? ?ll_builder.grow(ll_builder, size)
> ? ? ? ?assert size >= 0
> ? ? ? ?ll_str.copy_contents(charp, ll_builder.buf, 0, used, size)
> ? ? ? ?ll_builder.used += size
>
> (and similarly for ll_append_multiple_char above it)
>
> Like an onion - more and more layers ;) I'm beginning to suspect
> speeding up append_charpsize in order to make passing strings
> to/from C code faster is a bit too ambitious for a first contribution
> to PyPy! [*]
>
> Peter
>
> [*] Especially as after three hours it is still building from source:
> $ python translate.py --opt=jit targetpypystandalone.py

ok, so let me reply a bit more :)

First of all, you don't have to translate pypy to see changes. We
mostly run tests to see if they work. You can also write a very small
rpython program in translator/goal (look at targetnopstandalone.py) if
you want to just test the performance of single function.

I suppose your code is indeed a bit faster, but my bet would be it's
not too much faster (feel free to prove me wrong, especially on older
GCCs, they might not figure out that a loop is vectorizable for
example).

The main source of why passing strings to C is slow is however copying
the string from the GC area to non-moving one, raw malloced in C.
There are various strategies how to approach this, one of those would
be pinning, so the GC structures don't move and you can pass a pointer
to C. This is however definitely not a good  first patch to pypy ;-)

What I would suggest:

* Your patch looks good to me, although I'm not sure if
copy-string-contents would accept a raw memory. Check if tests pass.
* If you want to benchmark, write a small test for passing such
strings in translator/goal and see if it works.

We're usually available for help on IRC and thanks for tackling this problem!

Cheers,
fijal

From alex.gaynor at gmail.com  Mon Mar 19 20:36:46 2012
From: alex.gaynor at gmail.com (Alex Gaynor)
Date: Mon, 19 Mar 2012 14:36:46 -0500
Subject: [pypy-dev] Speeding up zlib in standard library
In-Reply-To: <CAK5idxRrznx6ifCPD3Y1p8XHKYOwO+RmqoA=rdTKeJkhvpuSsA@mail.gmail.com>
References: <CAKVJ-_5D4OUBSkavYqW7usBkaZMxu-EjEtb9NaKU-iCMA=XZQg@mail.gmail.com>
	<CAPZV6o8fpWjG2d8L6ig4SQds=EAUnb=sypeor9HzazdmXyZy=g@mail.gmail.com>
	<CAKVJ-_4zry6WMzg7Gfu2zUj=t-fpce_givh4TFj57aHiagwQ+Q@mail.gmail.com>
	<CAMSv6X1vXnT6jRs2MHvwa613nfZfrqgt9sEVbB0tk6tX5+Ggew@mail.gmail.com>
	<CAKVJ-_5jtHOxXpcnUVojV-a070BWV8fybZ99EKaZqjT9+DoLJA@mail.gmail.com>
	<CA+=c-xbzg8k3M_d6N=JsV1aaHRHv-2P1Cm-cSSPUdUh6jYJsnA@mail.gmail.com>
	<CAKVJ-_6Ov+MwjHginrmpyga2TLWZkxP9mFSaSZcUYToNDRkm0g@mail.gmail.com>
	<CAK5idxTbhQi9X7O+M2eJy-nFY6BWiFmTdghhfohjNv0y8_Yd1A@mail.gmail.com>
	<CAKVJ-_5Zd_BXHa1ycDGNr3SS8txKF1wzKgZ7QydYkpQeqEXt3w@mail.gmail.com>
	<CAK5idxRrznx6ifCPD3Y1p8XHKYOwO+RmqoA=rdTKeJkhvpuSsA@mail.gmail.com>
Message-ID: <CAFRnB2WoLUuZp5Ve9YYB34qC2afJ-Exn=pFM31P2qx82JdC0mA@mail.gmail.com>

On Mon, Mar 19, 2012 at 2:32 PM, Maciej Fijalkowski <fijall at gmail.com>wrote:

> On Mon, Mar 19, 2012 at 9:15 PM, Peter Cock <p.j.a.cock at googlemail.com>
> wrote:
> > On Mon, Mar 19, 2012 at 6:36 PM, Maciej Fijalkowski <fijall at gmail.com>
> wrote:
> >>> http://mail.python.org/mailman/listinfo/pypy-dev
> >>
> >> append_charpsize is special - it's not the *actual* implementation,
> >> the actual implementation is buried somewhere in
> >> rpython/lltypesystem/rbuilder.py, with the one you're mentioning being
> >> just fake implementation for tests. StringBuilder is special in a
> >> sense that it has some special GC support (which we can probably
> >> improve upon).
> >>
> >> Cheers,
> >> fijal
> >
> > I guess you are referring to the copy_string_contents function here:
> >
> https://bitbucket.org/pypy/pypy/src/default/pypy/rpython/lltypesystem/rstr.py
> >
> > However, methods ll_append_multiple_char not ll_append_charpsize
> > defined in rbuilder seem to use this - they both use a for loop
> char-by-char,
> >
> https://bitbucket.org/pypy/pypy/src/default/pypy/rpython/lltypesystem/rbuilder.py
> >
> > My hunch would be to replace this:
> >
> >    @staticmethod
> >    def ll_append_charpsize(ll_builder, charp, size):
> >        used = ll_builder.used
> >        if used + size > ll_builder.allocated:
> >            ll_builder.grow(ll_builder, size)
> >        for i in xrange(size):
> >            ll_builder.buf.chars[used] = charp[i]
> >            used += 1
> >        ll_builder.used = used
> >
> > with this:
> >
> >    @staticmethod
> >    def ll_append_charpsize(ll_builder, charp, size):
> >        used = ll_builder.used
> >        if used + size > ll_builder.allocated:
> >            ll_builder.grow(ll_builder, size)
> >        assert size >= 0
> >        ll_str.copy_contents(charp, ll_builder.buf, 0, used, size)
> >        ll_builder.used += size
> >
> > (and similarly for ll_append_multiple_char above it)
> >
> > Like an onion - more and more layers ;) I'm beginning to suspect
> > speeding up append_charpsize in order to make passing strings
> > to/from C code faster is a bit too ambitious for a first contribution
> > to PyPy! [*]
> >
> > Peter
> >
> > [*] Especially as after three hours it is still building from source:
> > $ python translate.py --opt=jit targetpypystandalone.py
>
> ok, so let me reply a bit more :)
>
> First of all, you don't have to translate pypy to see changes. We
> mostly run tests to see if they work. You can also write a very small
> rpython program in translator/goal (look at targetnopstandalone.py) if
> you want to just test the performance of single function.
>
> I suppose your code is indeed a bit faster, but my bet would be it's
> not too much faster (feel free to prove me wrong, especially on older
> GCCs, they might not figure out that a loop is vectorizable for
> example).
>
> The main source of why passing strings to C is slow is however copying
> the string from the GC area to non-moving one, raw malloced in C.
> There are various strategies how to approach this, one of those would
> be pinning, so the GC structures don't move and you can pass a pointer
> to C. This is however definitely not a good  first patch to pypy ;-)
>
> What I would suggest:
>
> * Your patch looks good to me, although I'm not sure if
> copy-string-contents would accept a raw memory. Check if tests pass.
> * If you want to benchmark, write a small test for passing such
> strings in translator/goal and see if it works.
>
> We're usually available for help on IRC and thanks for tackling this
> problem!
>
> Cheers,
> fijal
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev
>

FWIW, copy_string_contents definitely doesn't take raw memory, it takes an
rstr.

Alex

-- 
"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120319/ee9a517c/attachment-0001.html>

From arigo at tunes.org  Tue Mar 20 00:59:16 2012
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 20 Mar 2012 00:59:16 +0100
Subject: [pypy-dev] Finite automata
In-Reply-To: <CAL7ARbTfhWwNsf+5A37ZpB4VyAB-1L5FqE_vpUafsNn_57v3pg@mail.gmail.com>
References: <CAL7ARbRozG78hr=_AJuZB28POp6rDCdrBx4d2qYjYfCgME0mmA@mail.gmail.com>
	<4F61ED3D.30600@gmx.de>
	<CAL7ARbSHrMAA_qOV0f_2Af+Wev-m5hu+BkL307VWL4Zo+8FHUg@mail.gmail.com>
	<CAMSv6X2BWwBAFS0njWmM1-SX-n8m9MRjug7L14Us=ogzS+8tCg@mail.gmail.com>
	<CAL7ARbTfhWwNsf+5A37ZpB4VyAB-1L5FqE_vpUafsNn_57v3pg@mail.gmail.com>
Message-ID: <CAMSv6X1cRrRw+T4AXZ5QA3K+d6c1xi9chmHLBoWx5jr9U9z=Vw@mail.gmail.com>

Hi Alexander,

On Sat, Mar 17, 2012 at 22:17, Alexander Sedov <alex0player at gmail.com> wrote:
> If I was given a permission to commit, I won't be fooling around with
> these patches. Actually, I was, but on SVN. I even wrote big part of
> mcronumpy. (Now all my code is gone, of course). If you add me to
> developers, it'd be wonderful.

The easiest way for a newcomer to contribute code is to go to
https://bitbucket.org/ , create yourself an account, and then ask
bitbucket to clone the pypy repository (click on "fork" on the page
https://bitbucket.org/pypy/pypy ).  This way, you get your own copy of
the repository.  Then use Mercurial as usual to check in and push
changes to the clone (hg clone https://bitbucket.org/YOURNAME/pypy ;
hg ci ; hg push ).  This lets us view what you are doing, and comment
on it, and later merge it to the main repository.  (Mercurial is
better than Subversion in that it is very easy to merge back two
repositories.)  If this is successful we can later add you as a
committer directly to the main repository, but we'd like to see and
review some code first.


A bient?t,

Armin.

From arigo at tunes.org  Tue Mar 20 01:22:21 2012
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 20 Mar 2012 01:22:21 +0100
Subject: [pypy-dev] Finite automata
In-Reply-To: <CAL7ARbTfhWwNsf+5A37ZpB4VyAB-1L5FqE_vpUafsNn_57v3pg@mail.gmail.com>
References: <CAL7ARbRozG78hr=_AJuZB28POp6rDCdrBx4d2qYjYfCgME0mmA@mail.gmail.com>
	<4F61ED3D.30600@gmx.de>
	<CAL7ARbSHrMAA_qOV0f_2Af+Wev-m5hu+BkL307VWL4Zo+8FHUg@mail.gmail.com>
	<CAMSv6X2BWwBAFS0njWmM1-SX-n8m9MRjug7L14Us=ogzS+8tCg@mail.gmail.com>
	<CAL7ARbTfhWwNsf+5A37ZpB4VyAB-1L5FqE_vpUafsNn_57v3pg@mail.gmail.com>
Message-ID: <CAMSv6X3svG8=N64ifMDpo7NztYZ60V4+rjUie_vcbm+96Gg07Q@mail.gmail.com>

Re-hi,

On Sat, Mar 17, 2012 at 22:17, Alexander Sedov <alex0player at gmail.com> wrote:
> Thanks  for inviting me, I used to be there, but now I'm kinda behind firewall.
> So mailing list is the only way.

Note that there is also http://webchat.freenode.net/ .

> Still asking somebody to try my patch.

I tried the patch, and the tests still pass.  I don't know the details
of pypy/interpreter/pyparser, but it seems to me that you changed
something in pylexer.py and re-ran genpytokenize.py to regenerate the
central part of pytokenize.py.  The changes are probably fine, but
should be reviewed by someone that knows what this code really does
:-/


A bient?t,

Armin.

From simohayha.bobo at gmail.com  Tue Mar 20 08:15:23 2012
From: simohayha.bobo at gmail.com (Simon Liu)
Date: Tue, 20 Mar 2012 15:15:23 +0800
Subject: [pypy-dev] how embed pypy into c?
Message-ID: <CAB6pCs_68=F_zBNowfQfbxcqDQjExP6tAHjjY8vkJgDiEzOesw@mail.gmail.com>

Hi all.

I want to embed pypy  into c code, but in pypy's directory, i don't find
any pypy's c api, except in cypext directory.

And I want to use stackless' c api too,. but i still have not found any
stackless api(example stackless_api.h)  in pypy.

Thanks.

-- 
do not fear to be eccentric in opinion, for every opinion now accepted was
once eccentric.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120320/98585cbf/attachment.html>

From roberto at unbit.it  Tue Mar 20 08:36:03 2012
From: roberto at unbit.it (Roberto De Ioris)
Date: Tue, 20 Mar 2012 08:36:03 +0100
Subject: [pypy-dev] how embed pypy into c?
Message-ID: <9a48d9898d7d1d879842ee73abbb6f32.squirrel@manage.unbit.it>



> Hi all.
>
> I want to embed pypy  into c code, but in pypy's directory, i don't find
any pypy's c api, except in cypext directory.
>
> And I want to use stackless' c api too,. but i still have not found any
stackless api(example stackless_api.h)  in pypy.
>
> Thanks.
>
> --
>

Currently you are on your own.

This is the step i follow:

1) patch genc.py to export RPython_StartupCode in addition to
pypy_main_startup

export_symbols=["pypy_main_startup", "RPython_StartupCode"]))

and run:

pypy translate.py -Ojit --shared

to build libpypy

2) add this declaration on top of your code

#ifdef UWSGI_PYPY
char *RPython_StartupCode(void);
#define pypy_asm_stack_bottom()  asm volatile ("/* GC_STACK_BOTTOM */" : :
: \
                                               "memory")

#endif

3) instead of Py_Initialize() call

pypy_asm_stack_bottom();
RPython_StartupCode();

After that you have an almost working pypy environment.

A bunch of python c api functions are not implemented (but you will find
it is pretty easy to add them, even as noop) so be prepared to some
compromise.

What kills me is the amount of time taken to rebuild the whole library, so
i can work on it only on spare time (i am porting both mod_wsgi and uWSGI)
:)


-- 
Roberto De Ioris
http://unbit.it

From fijall at gmail.com  Tue Mar 20 08:50:14 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Tue, 20 Mar 2012 09:50:14 +0200
Subject: [pypy-dev] how embed pypy into c?
In-Reply-To: <9a48d9898d7d1d879842ee73abbb6f32.squirrel@manage.unbit.it>
References: <9a48d9898d7d1d879842ee73abbb6f32.squirrel@manage.unbit.it>
Message-ID: <CAK5idxTiv8=FGYmHfZbPmeY4Fs7t+uS5UKW0xw3SmaxGNLdcyQ@mail.gmail.com>

> What kills me is the amount of time taken to rebuild the whole library, so
> i can work on it only on spare time (i am porting both mod_wsgi and uWSGI)
> :)

That's really cool! Didn't know someone is working on it. Can I direct
people with questions about that somewhere?

Cheers,
fijal

From simohayha.bobo at gmail.com  Tue Mar 20 09:45:20 2012
From: simohayha.bobo at gmail.com (Simon Liu)
Date: Tue, 20 Mar 2012 16:45:20 +0800
Subject: [pypy-dev] how embed pypy into c?
In-Reply-To: <9a48d9898d7d1d879842ee73abbb6f32.squirrel@manage.unbit.it>
References: <9a48d9898d7d1d879842ee73abbb6f32.squirrel@manage.unbit.it>
Message-ID: <CAB6pCs9zC67EciWgoU8Vke9uL5R7qKYUDEBznaSFU6euHwY4yQ@mail.gmail.com>

Hi.

On Tue, Mar 20, 2012 at 3:36 PM, Roberto De Ioris <roberto at unbit.it> wrote:

>
>
> > Hi all.
> >
> > I want to embed pypy  into c code, but in pypy's directory, i don't find
> any pypy's c api, except in cypext directory.
> >
> > And I want to use stackless' c api too,. but i still have not found any
> stackless api(example stackless_api.h)  in pypy.
> >
> > Thanks.
> >
> > --
> >
>
> Currently you are on your own.
>
> This is the step i follow:
>
> 1) patch genc.py to export RPython_StartupCode in addition to
> pypy_main_startup
>
> export_symbols=["pypy_main_startup", "RPython_StartupCode"]))
>
> and run:
>
> pypy translate.py -Ojit --shared
>
> to build libpypy
>
> 2) add this declaration on top of your code
>
> #ifdef UWSGI_PYPY
> char *RPython_StartupCode(void);
> #define pypy_asm_stack_bottom()  asm volatile ("/* GC_STACK_BOTTOM */" : :
> : \
>                                               "memory")
>
> #endif
>
> 3) instead of Py_Initialize() call
>
> pypy_asm_stack_bottom();
> RPython_StartupCode();
>
> After that you have an almost working pypy environment.
>
> A bunch of python c api functions are not implemented (but you will find
> it is pretty easy to add them, even as noop) so be prepared to some
> compromise.
>
> What kills me is the amount of time taken to rebuild the whole library, so
> i can work on it only on spare time (i am porting both mod_wsgi and uWSGI)
> :)
>
>
> --
> Roberto De Ioris
> http://unbit.it
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev
>


I have several problem(because compile pypy is slow ):

1 add "export_symbols=["pypy_main_startup", "RPython_StartupCode"]))" to
 genc.py in  which line? or top line?

2 this will export stackless c api?

Thanks.
-- 
do not fear to be eccentric in opinion, for every opinion now accepted was
once eccentric.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120320/9e1ecdc1/attachment-0001.html>

From Ronny.Pfannschmidt at gmx.de  Tue Mar 20 12:10:57 2012
From: Ronny.Pfannschmidt at gmx.de (Ronny Pfannschmidt)
Date: Tue, 20 Mar 2012 12:10:57 +0100
Subject: [pypy-dev] Does anyone have a git-mapfile?
In-Reply-To: <CAO2U3Qi-7t7HsTgZwg9=VCqtdbJFnxyhoKBL8Fr=J6oF9Ac83A@mail.gmail.com>
References: <CAO2U3Qg4Ordnwh=pE+gf6EYdii_beWuaHOHEd03m=uzX2uHzaQ@mail.gmail.com>
	<4F6772B3.8020507@gmx.de>
	<CAO2U3Qi-7t7HsTgZwg9=VCqtdbJFnxyhoKBL8Fr=J6oF9Ac83A@mail.gmail.com>
Message-ID: <4F6865C1.8060204@gmx.de>

Hi,

turns out the git repo is not entirely compatible with recent hg-git

so there is a divergence of dozens of commits being replicated,
also there are some direct defects on metadata

i cant recommend working that way,

a possible mode of operation could be hacking in git,
then importing patches to the branch you want to push to



On 03/19/2012 07:05 PM, Michael Blume wrote:
> Thanks! =)
>
> I'll either add --HG-- branch tags as I go or just come through and
> add them when I'm about to push. Better (for me) than trying to use
> Mercurial =)
>
> On Mon, Mar 19, 2012 at 10:53 AM, Ronny Pfannschmidt
> <Ronny.Pfannschmidt at gmx.de>  wrote:
>>
>> i started a hg gexport background task on my laptop,
>> should finish in a hour or so
>>
>> however i strongly suggest not to develop with git,
>> you are likely to mess up branching since git
>> has no idea about named branches
>>
>>
>> On 03/19/2012 06:10 PM, Michael Blume wrote:
>>>
>>> I know there's a github mirror at github.com/pypy/pypy, but in order to
>>> shuffle commits between the two repos, I need to either perform the
>>> translation myself (and I let an EC2 instance run on that for 9 hours...)
>>> or get a git-mapfile with a list of corresponding commits. If you have a
>>> bi-directional repository, it'd be found at pypy/.hg/git-mapfile.
>>>
>>> Thanks!
>>>
>>> -Mike
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> pypy-dev mailing list
>>> pypy-dev at python.org
>>> http://mail.python.org/mailman/listinfo/pypy-dev
>>
>>


From p.j.a.cock at googlemail.com  Tue Mar 20 12:51:41 2012
From: p.j.a.cock at googlemail.com (Peter Cock)
Date: Tue, 20 Mar 2012 11:51:41 +0000
Subject: [pypy-dev] targetreallynopstandalone.py => TypeError: string vs
	LocalPath
Message-ID: <CAKVJ-_6=MKCgPx4kcRqWiQc6wzB-fuYmWSarp2+nRJJSVxTy5Q@mail.gmail.com>

Hi all,

After a teething error with PyPy calling GCC with unescaped paths,
which I solved by moving to a directory without spaces in the full
path, I made some progress.

First, get PyPy from hg - perhaps checking out a known good
revision:

$ hg clone http://bitbucket.org/pypy/pypy pypy
$ cd pypy
$ hg checkout release-1.8

Next, using the Richards example (short but non-trivial),

$ cd pypy/translator/goal
$ python translate.py --run targetrichards.py
...
[translation:info] Running compiled c source...
Translated:
Richards benchmark (Python) starting...
[<pypy.translator.c.genc.Wrapper instance at 0x102df1170>]
finished.
Total time for 500 iterations: 0.58 secs
Average time per iteration: 1.16 ms
CPython:
Richards benchmark (Python) starting... [<function entry_point at 0x1005acf50>]
finished.
Total time for 5 iterations: 1.49 secs
Average time per iteration: 298.09 ms
[Timer] Timings:
[Timer] annotate                       ---  2.7 s
[Timer] rtype_lltype                   ---  0.9 s
[Timer] backendopt_lltype              ---  0.4 s
[Timer] stackcheckinsertion_lltype     ---  0.0 s
[Timer] database_c                     --- 13.7 s
[Timer] source_c                       ---  3.0 s
[Timer] compile_c                      ---  4.8 s
[Timer] run_c                          ---  2.1 s
[Timer] =========================================
[Timer] Total:                         --- 27.6 s

And even simpler example, in principle:

$ python translate.py --run targetreallynopstandalone.py
...
[translation:info] Running compiled c source...
[Timer] Timings:
[Timer] annotate                       ---  1.5 s
[Timer] rtype_lltype                   ---  0.2 s
[Timer] backendopt_lltype              ---  0.1 s
[Timer] stackcheckinsertion_lltype     ---  0.0 s
[Timer] database_c                     --- 13.9 s
[Timer] source_c                       ---  2.3 s
[Timer] compile_c                      ---  1.3 s
[Timer] run_c                          ---  0.0 s
[Timer] =========================================
[Timer] Total:                         --- 19.4 s
[translation:ERROR] Error:
[translation:ERROR]  Traceback (most recent call last):
[translation:ERROR]    File "translate.py", line 309, in main
[translation:ERROR]     drv.proceed(goals)
[translation:ERROR]    File
"/Users/pjcock/pypy-hg/pypy/translator/driver.py", line 810, in
proceed
[translation:ERROR]     return self._execute(goals, task_skip =
self._maybe_skip())
[translation:ERROR]    File
"/Users/pjcock/pypy-hg/pypy/translator/tool/taskengine.py", line 116,
in _execute
[translation:ERROR]     res = self._do(goal, taskcallable, *args, **kwds)
[translation:ERROR]    File
"/Users/pjcock/pypy-hg/pypy/translator/driver.py", line 287, in _do
[translation:ERROR]     res = func()
[translation:ERROR]    File
"/Users/pjcock/pypy-hg/pypy/translator/driver.py", line 594, in
task_run_c
[translation:ERROR]     self.backend_run('c')
[translation:ERROR]    File
"/Users/pjcock/pypy-hg/pypy/translator/driver.py", line 588, in
backend_run
[translation:ERROR]     os.system(c_entryp)
[translation:ERROR]  TypeError: system() argument 1 must be string,
not LocalPath
[translation] start debugger...
> /Users/pjcock/pypy-hg/pypy/translator/driver.py(588)backend_run()
-> os.system(c_entryp)
(Pdb+)

This looks like a string versus path object error. The same happens
with the default branch, 53817:62902925695c

What's going wrong? Am I invoking translate.py incorrectly?

Thanks,

Peter

From p.j.a.cock at googlemail.com  Tue Mar 20 12:54:54 2012
From: p.j.a.cock at googlemail.com (Peter Cock)
Date: Tue, 20 Mar 2012 11:54:54 +0000
Subject: [pypy-dev] targetreallynopstandalone.py => TypeError: string vs
	LocalPath
In-Reply-To: <CAKVJ-_6=MKCgPx4kcRqWiQc6wzB-fuYmWSarp2+nRJJSVxTy5Q@mail.gmail.com>
References: <CAKVJ-_6=MKCgPx4kcRqWiQc6wzB-fuYmWSarp2+nRJJSVxTy5Q@mail.gmail.com>
Message-ID: <CAKVJ-_4yafBcmBbCM6CTQ4mHFR-490ePKD5cxNnzWHGfVGoF_g@mail.gmail.com>

On Tue, Mar 20, 2012 at 11:51 AM, Peter Cock <p.j.a.cock at googlemail.com> wrote:
> Hi all,
>
> After a teething error with PyPy calling GCC with unescaped paths,
> which I solved by moving to a directory without spaces in the full
> path, I made some progress.
>
> ...
>
> $ python translate.py --run targetreallynopstandalone.py
> ...
> [translation:info] Running compiled c source...
> [Timer] Timings:
> [Timer] annotate ? ? ? ? ? ? ? ? ? ? ? --- ?1.5 s
> [Timer] rtype_lltype ? ? ? ? ? ? ? ? ? --- ?0.2 s
> [Timer] backendopt_lltype ? ? ? ? ? ? ?--- ?0.1 s
> [Timer] stackcheckinsertion_lltype ? ? --- ?0.0 s
> [Timer] database_c ? ? ? ? ? ? ? ? ? ? --- 13.9 s
> [Timer] source_c ? ? ? ? ? ? ? ? ? ? ? --- ?2.3 s
> [Timer] compile_c ? ? ? ? ? ? ? ? ? ? ?--- ?1.3 s
> [Timer] run_c ? ? ? ? ? ? ? ? ? ? ? ? ?--- ?0.0 s
> [Timer] =========================================
> [Timer] Total: ? ? ? ? ? ? ? ? ? ? ? ? --- 19.4 s
> [translation:ERROR] Error:
> [translation:ERROR] ?Traceback (most recent call last):
> [translation:ERROR] ? ?File "translate.py", line 309, in main
> [translation:ERROR] ? ? drv.proceed(goals)
> [translation:ERROR] ? ?File
> "/Users/pjcock/pypy-hg/pypy/translator/driver.py", line 810, in
> proceed
> [translation:ERROR] ? ? return self._execute(goals, task_skip =
> self._maybe_skip())
> [translation:ERROR] ? ?File
> "/Users/pjcock/pypy-hg/pypy/translator/tool/taskengine.py", line 116,
> in _execute
> [translation:ERROR] ? ? res = self._do(goal, taskcallable, *args, **kwds)
> [translation:ERROR] ? ?File
> "/Users/pjcock/pypy-hg/pypy/translator/driver.py", line 287, in _do
> [translation:ERROR] ? ? res = func()
> [translation:ERROR] ? ?File
> "/Users/pjcock/pypy-hg/pypy/translator/driver.py", line 594, in
> task_run_c
> [translation:ERROR] ? ? self.backend_run('c')
> [translation:ERROR] ? ?File
> "/Users/pjcock/pypy-hg/pypy/translator/driver.py", line 588, in
> backend_run
> [translation:ERROR] ? ? os.system(c_entryp)
> [translation:ERROR] ?TypeError: system() argument 1 must be string,
> not LocalPath
> [translation] start debugger...
>> /Users/pjcock/pypy-hg/pypy/translator/driver.py(588)backend_run()
> -> os.system(c_entryp)
> (Pdb+)
>
> This looks like a string versus path object error. The same happens
> with the default branch, 53817:62902925695c

Sorry - I forgot to include the OS details, this is under Mac OS X 10.6.8
(Intel 64bit CPU) using the Apple provided Python 2.6,

$ python
Python 2.6.1 (r261:67515, Jun 24 2010, 21:47:49)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>

If I explicitly call Python 2.5, the same error occurs:

$ python2.5 translate.py --run targetreallynopstandalone.py

Peter

From anto.cuni at gmail.com  Tue Mar 20 13:40:41 2012
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Tue, 20 Mar 2012 13:40:41 +0100
Subject: [pypy-dev] targetreallynopstandalone.py => TypeError: string vs
 LocalPath
In-Reply-To: <CAKVJ-_6=MKCgPx4kcRqWiQc6wzB-fuYmWSarp2+nRJJSVxTy5Q@mail.gmail.com>
References: <CAKVJ-_6=MKCgPx4kcRqWiQc6wzB-fuYmWSarp2+nRJJSVxTy5Q@mail.gmail.com>
Message-ID: <4F687AC9.4040205@gmail.com>

On 03/20/2012 12:51 PM, Peter Cock wrote:
> Hi all,
> 
> After a teething error with PyPy calling GCC with unescaped paths,
> which I solved by moving to a directory without spaces in the full
> path, I made some progress.
> 
> First, get PyPy from hg - perhaps checking out a known good
> revision:
> 
> $ hg clone http://bitbucket.org/pypy/pypy pypy
> $ cd pypy
> $ hg checkout release-1.8
> 
> Next, using the Richards example (short but non-trivial),
> 
> $ cd pypy/translator/goal
> $ python translate.py --run targetrichards.py

I never heard of the --run option before :-)
You don't need it: just run translate.py targetrichards.py and you'll get a
targetrichards-c executable.

ciao,
Anto


From p.j.a.cock at googlemail.com  Tue Mar 20 15:17:19 2012
From: p.j.a.cock at googlemail.com (Peter Cock)
Date: Tue, 20 Mar 2012 14:17:19 +0000
Subject: [pypy-dev] targetreallynopstandalone.py => TypeError: string vs
	LocalPath
In-Reply-To: <4F687AC9.4040205@gmail.com>
References: <CAKVJ-_6=MKCgPx4kcRqWiQc6wzB-fuYmWSarp2+nRJJSVxTy5Q@mail.gmail.com>
	<4F687AC9.4040205@gmail.com>
Message-ID: <CAKVJ-_6dmRYRcsz7-dj81McQDW8i3rKq50i+fLnxrtydo6zscQ@mail.gmail.com>

On Tue, Mar 20, 2012 at 12:40 PM, Antonio Cuni <anto.cuni at gmail.com> wrote:
> On 03/20/2012 12:51 PM, Peter Cock wrote:
>>
>> Next, using the Richards example (short but non-trivial),
>>
>> $ cd pypy/translator/goal
>> $ python translate.py --run targetrichards.py
>
> I never heard of the --run option before :-)

It seemed helpful - giving the PyPy vs Python numbers right
there in one go.

> You don't need it: just run translate.py targetrichards.py
> and you'll get a targetrichards-c executable.

Doing that I do get a testing_1.dylib file (Mach-O 64-bit
dynamically linked shared library x86_64) under a temp
folder, but no sign of a targetrichards-c executable.

Interestingly trying some of the other examples the binary
is produced in the current directory:

$ python2.6 translate.py targetreallynopstandalone.py
...
[translation:info] created:
/Users/pjcock/pypy-hg/pypy/translator/goal/targetreallynopstandalone-c
[Timer] Timings:
[Timer] annotate                       ---  2.0 s
[Timer] rtype_lltype                   ---  0.2 s
[Timer] backendopt_lltype              ---  0.0 s
[Timer] stackcheckinsertion_lltype     ---  0.0 s
[Timer] database_c                     --- 14.4 s
[Timer] source_c                       ---  2.3 s
[Timer] compile_c                      ---  1.3 s
[Timer] =========================================
[Timer] Total:                         --- 20.2 s

Peter

From p.j.a.cock at googlemail.com  Tue Mar 20 15:40:49 2012
From: p.j.a.cock at googlemail.com (Peter Cock)
Date: Tue, 20 Mar 2012 14:40:49 +0000
Subject: [pypy-dev] Speeding up zlib in standard library
In-Reply-To: <CAK5idxRrznx6ifCPD3Y1p8XHKYOwO+RmqoA=rdTKeJkhvpuSsA@mail.gmail.com>
References: <CAKVJ-_5D4OUBSkavYqW7usBkaZMxu-EjEtb9NaKU-iCMA=XZQg@mail.gmail.com>
	<CAPZV6o8fpWjG2d8L6ig4SQds=EAUnb=sypeor9HzazdmXyZy=g@mail.gmail.com>
	<CAKVJ-_4zry6WMzg7Gfu2zUj=t-fpce_givh4TFj57aHiagwQ+Q@mail.gmail.com>
	<CAMSv6X1vXnT6jRs2MHvwa613nfZfrqgt9sEVbB0tk6tX5+Ggew@mail.gmail.com>
	<CAKVJ-_5jtHOxXpcnUVojV-a070BWV8fybZ99EKaZqjT9+DoLJA@mail.gmail.com>
	<CA+=c-xbzg8k3M_d6N=JsV1aaHRHv-2P1Cm-cSSPUdUh6jYJsnA@mail.gmail.com>
	<CAKVJ-_6Ov+MwjHginrmpyga2TLWZkxP9mFSaSZcUYToNDRkm0g@mail.gmail.com>
	<CAK5idxTbhQi9X7O+M2eJy-nFY6BWiFmTdghhfohjNv0y8_Yd1A@mail.gmail.com>
	<CAKVJ-_5Zd_BXHa1ycDGNr3SS8txKF1wzKgZ7QydYkpQeqEXt3w@mail.gmail.com>
	<CAK5idxRrznx6ifCPD3Y1p8XHKYOwO+RmqoA=rdTKeJkhvpuSsA@mail.gmail.com>
Message-ID: <CAKVJ-_4=xyR3qGS-ZdcCwGXfxphLwD5Ezgq48f=7ydzDp1A4+A@mail.gmail.com>

Hi again,

Based on the structure of pypy/translator/goal/richards.py and
pypy/translator/goal/targetrichards.py I have tried to make a
simple ZLIB based benchmark based on what I have been using.

$ python2.6 green_bottles.py
Green bottles ZLIB benchmark starting... [<function entry_point at 0x1004c7500>]
Trying ZLIB on 176683 bytes of text 10000 times
finished.
Total time for 10000 iterations: 34.25 secs
Average time per iteration: 3.42 ms

$ pypy green_bottles.py
Green bottles ZLIB benchmark starting... [<function entry_point at
0x0000000102f3b3d0>]
Trying ZLIB on 176683 bytes of text 10000 times
finished.
Total time for 10000 iterations: 45.12 secs
Average time per iteration: 4.51 ms

Here PyPy v1.8 is about 1.3 times as slow as C Python 2.6, not
as bad as the factor of almost two I was seeing in some cases,
but enough to tackle as a measurable target?

Files here: https://gist.github.com/2136181

However, I have a problem with using translate.py on this
(trying the PyPy release-1.8 tag or the latest from bitbucket):

$ python2.6 translate.py --run targetzlib.py
[platform:msg] Setting platform to 'host' cc=None
[translation:info] Translating target as defined by targetzlib
[platform:execute] gcc ...
[translation] translate.py configuration:
[translation] [translate]
[translation]     [goal_options]
[translation]         run = True
[translation]     targetspec = targetzlib
[translation] translation configuration:
[translation] [translation]
[translation]     gc = minimark
[translation]     gctransformer = framework
[translation]     list_comprehension_operations = True
[translation]     withsmallfuncsets = 5
[translation:info] Annotating&simplifying...
[translation:info] with policy: pypy.annotation.policy.AnnotatorPolicy
[platform:execute] gcc ...
[Timer] Timings:
[Timer] annotate                       --- 1.6 s
[Timer] ========================================
[Timer] Total:                         --- 1.6 s
[translation:ERROR] Error:
[translation:ERROR]  Traceback (most recent call last):
[translation:ERROR]    File "translate.py", line 309, in main
[translation:ERROR]     drv.proceed(goals)
[translation:ERROR]    File
"/Users/pjcock/pypy-hg/pypy/translator/driver.py", line 810, in
proceed
[translation:ERROR]     return self._execute(goals, task_skip =
self._maybe_skip())
[translation:ERROR]    File
"/Users/pjcock/pypy-hg/pypy/translator/tool/taskengine.py", line 116,
in _execute
[translation:ERROR]     res = self._do(goal, taskcallable, *args, **kwds)
[translation:ERROR]    File
"/Users/pjcock/pypy-hg/pypy/translator/driver.py", line 287, in _do
[translation:ERROR]     res = func()
[translation:ERROR]    File
"/Users/pjcock/pypy-hg/pypy/translator/driver.py", line 324, in
task_annotate
[translation:ERROR]     s = annotator.build_types(self.entry_point,
self.inputtypes)
[translation:ERROR]    File
"/Users/pjcock/pypy-hg/pypy/annotation/annrpython.py", line 107, in
build_types
[translation:ERROR]     return self.build_graph_types(flowgraph,
inputcells, complete_now=complete_now)
[translation:ERROR]    File
"/Users/pjcock/pypy-hg/pypy/annotation/annrpython.py", line 198, in
build_graph_types
[translation:ERROR]     self.complete()
[translation:ERROR]    File
"/Users/pjcock/pypy-hg/pypy/annotation/annrpython.py", line 254, in
complete
[translation:ERROR]     self.processblock(graph, block)
[translation:ERROR]    File
"/Users/pjcock/pypy-hg/pypy/annotation/annrpython.py", line 452, in
processblock
[translation:ERROR]     self.flowin(graph, block)
[translation:ERROR]    File
"/Users/pjcock/pypy-hg/pypy/annotation/annrpython.py", line 512, in
flowin
[translation:ERROR]     self.consider_op(block.operations[i])
[translation:ERROR]    File
"/Users/pjcock/pypy-hg/pypy/annotation/annrpython.py", line 695, in
consider_op
[translation:ERROR]     argcells = [self.binding(a) for a in op.args]
[translation:ERROR]    File
"/Users/pjcock/pypy-hg/pypy/annotation/annrpython.py", line 297, in
binding
[translation:ERROR]     return self.bookkeeper.immutableconstant(arg)
[translation:ERROR]    File
"/Users/pjcock/pypy-hg/pypy/annotation/bookkeeper.py", line 318, in
immutableconstant
[translation:ERROR]     return self.immutablevalue(const.value)
[translation:ERROR]    File
"/Users/pjcock/pypy-hg/pypy/annotation/bookkeeper.py", line 469, in
immutablevalue
[translation:ERROR]     result = SomePBC([self.getdesc(x)])
[translation:ERROR]    File
"/Users/pjcock/pypy-hg/pypy/annotation/bookkeeper.py", line 535, in
getdesc
[translation:ERROR]     pyobj,))
[translation:ERROR]  Exception: unexpected prebuilt constant:
<built-in function compressobj>
[translation:ERROR] Processing block:
[translation:ERROR]  block at 27 is a <class
'pypy.objspace.flow.flowcontext.SpamBlock'>
[translation:ERROR]  in (pypy.translator.goal.green_bottles:10)compress
[translation:ERROR]  containing the following operations:
[translation:ERROR]        v0 =
simple_call((builtin_function_or_method compressobj), (6), (8), (-15),
(8), (0))
[translation:ERROR]        v1 = getattr(v0, ('compress'))
[translation:ERROR]        v2 = simple_call(v1, orig_data_0)
[translation:ERROR]        v3 = getattr(v0, ('flush'))
[translation:ERROR]        v4 = simple_call(v3)
[translation:ERROR]        v5 = add(v2, v4)
[translation:ERROR]        v6 =
simple_call((builtin_function_or_method crc32), orig_data_0)
[translation:ERROR]        v7 = newtuple(v5, v6)
[translation:ERROR]  --end--
[translation] start debugger...
> /Users/pjcock/pypy-hg/pypy/annotation/bookkeeper.py(535)getdesc()
-> pyobj,))
(Pdb+) quit

Calling it without the --run gives the same:

$ python2.6 translate.py targetzlib.py
...

Should my benchmark Python code be RPython?

Thanks,

Peter

From fijall at gmail.com  Tue Mar 20 15:52:42 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Tue, 20 Mar 2012 16:52:42 +0200
Subject: [pypy-dev] Speeding up zlib in standard library
In-Reply-To: <CAKVJ-_4=xyR3qGS-ZdcCwGXfxphLwD5Ezgq48f=7ydzDp1A4+A@mail.gmail.com>
References: <CAKVJ-_5D4OUBSkavYqW7usBkaZMxu-EjEtb9NaKU-iCMA=XZQg@mail.gmail.com>
	<CAPZV6o8fpWjG2d8L6ig4SQds=EAUnb=sypeor9HzazdmXyZy=g@mail.gmail.com>
	<CAKVJ-_4zry6WMzg7Gfu2zUj=t-fpce_givh4TFj57aHiagwQ+Q@mail.gmail.com>
	<CAMSv6X1vXnT6jRs2MHvwa613nfZfrqgt9sEVbB0tk6tX5+Ggew@mail.gmail.com>
	<CAKVJ-_5jtHOxXpcnUVojV-a070BWV8fybZ99EKaZqjT9+DoLJA@mail.gmail.com>
	<CA+=c-xbzg8k3M_d6N=JsV1aaHRHv-2P1Cm-cSSPUdUh6jYJsnA@mail.gmail.com>
	<CAKVJ-_6Ov+MwjHginrmpyga2TLWZkxP9mFSaSZcUYToNDRkm0g@mail.gmail.com>
	<CAK5idxTbhQi9X7O+M2eJy-nFY6BWiFmTdghhfohjNv0y8_Yd1A@mail.gmail.com>
	<CAKVJ-_5Zd_BXHa1ycDGNr3SS8txKF1wzKgZ7QydYkpQeqEXt3w@mail.gmail.com>
	<CAK5idxRrznx6ifCPD3Y1p8XHKYOwO+RmqoA=rdTKeJkhvpuSsA@mail.gmail.com>
	<CAKVJ-_4=xyR3qGS-ZdcCwGXfxphLwD5Ezgq48f=7ydzDp1A4+A@mail.gmail.com>
Message-ID: <CAK5idxQpU61njri3PES+1cCeM1ddLFf=B1vFozBmKh7Pxh=HDg@mail.gmail.com>

On Tue, Mar 20, 2012 at 4:40 PM, Peter Cock <p.j.a.cock at googlemail.com> wrote:
> Hi again,
>
> Based on the structure of pypy/translator/goal/richards.py and
> pypy/translator/goal/targetrichards.py I have tried to make a
> simple ZLIB based benchmark based on what I have been using.
>
> $ python2.6 green_bottles.py
> Green bottles ZLIB benchmark starting... [<function entry_point at 0x1004c7500>]
> Trying ZLIB on 176683 bytes of text 10000 times
> finished.
> Total time for 10000 iterations: 34.25 secs
> Average time per iteration: 3.42 ms
>
> $ pypy green_bottles.py
> Green bottles ZLIB benchmark starting... [<function entry_point at
> 0x0000000102f3b3d0>]
> Trying ZLIB on 176683 bytes of text 10000 times
> finished.
> Total time for 10000 iterations: 45.12 secs
> Average time per iteration: 4.51 ms
>
> Here PyPy v1.8 is about 1.3 times as slow as C Python 2.6, not
> as bad as the factor of almost two I was seeing in some cases,
> but enough to tackle as a measurable target?
>
> Files here: https://gist.github.com/2136181
>
> However, I have a problem with using translate.py on this
> (trying the PyPy release-1.8 tag or the latest from bitbucket):
>
> $ python2.6 translate.py --run targetzlib.py
> [platform:msg] Setting platform to 'host' cc=None
> [translation:info] Translating target as defined by targetzlib
> [platform:execute] gcc ...
> [translation] translate.py configuration:
> [translation] [translate]
> [translation] ? ? [goal_options]
> [translation] ? ? ? ? run = True
> [translation] ? ? targetspec = targetzlib
> [translation] translation configuration:
> [translation] [translation]
> [translation] ? ? gc = minimark
> [translation] ? ? gctransformer = framework
> [translation] ? ? list_comprehension_operations = True
> [translation] ? ? withsmallfuncsets = 5
> [translation:info] Annotating&simplifying...
> [translation:info] with policy: pypy.annotation.policy.AnnotatorPolicy
> [platform:execute] gcc ...
> [Timer] Timings:
> [Timer] annotate ? ? ? ? ? ? ? ? ? ? ? --- 1.6 s
> [Timer] ========================================
> [Timer] Total: ? ? ? ? ? ? ? ? ? ? ? ? --- 1.6 s
> [translation:ERROR] Error:
> [translation:ERROR] ?Traceback (most recent call last):
> [translation:ERROR] ? ?File "translate.py", line 309, in main
> [translation:ERROR] ? ? drv.proceed(goals)
> [translation:ERROR] ? ?File
> "/Users/pjcock/pypy-hg/pypy/translator/driver.py", line 810, in
> proceed
> [translation:ERROR] ? ? return self._execute(goals, task_skip =
> self._maybe_skip())
> [translation:ERROR] ? ?File
> "/Users/pjcock/pypy-hg/pypy/translator/tool/taskengine.py", line 116,
> in _execute
> [translation:ERROR] ? ? res = self._do(goal, taskcallable, *args, **kwds)
> [translation:ERROR] ? ?File
> "/Users/pjcock/pypy-hg/pypy/translator/driver.py", line 287, in _do
> [translation:ERROR] ? ? res = func()
> [translation:ERROR] ? ?File
> "/Users/pjcock/pypy-hg/pypy/translator/driver.py", line 324, in
> task_annotate
> [translation:ERROR] ? ? s = annotator.build_types(self.entry_point,
> self.inputtypes)
> [translation:ERROR] ? ?File
> "/Users/pjcock/pypy-hg/pypy/annotation/annrpython.py", line 107, in
> build_types
> [translation:ERROR] ? ? return self.build_graph_types(flowgraph,
> inputcells, complete_now=complete_now)
> [translation:ERROR] ? ?File
> "/Users/pjcock/pypy-hg/pypy/annotation/annrpython.py", line 198, in
> build_graph_types
> [translation:ERROR] ? ? self.complete()
> [translation:ERROR] ? ?File
> "/Users/pjcock/pypy-hg/pypy/annotation/annrpython.py", line 254, in
> complete
> [translation:ERROR] ? ? self.processblock(graph, block)
> [translation:ERROR] ? ?File
> "/Users/pjcock/pypy-hg/pypy/annotation/annrpython.py", line 452, in
> processblock
> [translation:ERROR] ? ? self.flowin(graph, block)
> [translation:ERROR] ? ?File
> "/Users/pjcock/pypy-hg/pypy/annotation/annrpython.py", line 512, in
> flowin
> [translation:ERROR] ? ? self.consider_op(block.operations[i])
> [translation:ERROR] ? ?File
> "/Users/pjcock/pypy-hg/pypy/annotation/annrpython.py", line 695, in
> consider_op
> [translation:ERROR] ? ? argcells = [self.binding(a) for a in op.args]
> [translation:ERROR] ? ?File
> "/Users/pjcock/pypy-hg/pypy/annotation/annrpython.py", line 297, in
> binding
> [translation:ERROR] ? ? return self.bookkeeper.immutableconstant(arg)
> [translation:ERROR] ? ?File
> "/Users/pjcock/pypy-hg/pypy/annotation/bookkeeper.py", line 318, in
> immutableconstant
> [translation:ERROR] ? ? return self.immutablevalue(const.value)
> [translation:ERROR] ? ?File
> "/Users/pjcock/pypy-hg/pypy/annotation/bookkeeper.py", line 469, in
> immutablevalue
> [translation:ERROR] ? ? result = SomePBC([self.getdesc(x)])
> [translation:ERROR] ? ?File
> "/Users/pjcock/pypy-hg/pypy/annotation/bookkeeper.py", line 535, in
> getdesc
> [translation:ERROR] ? ? pyobj,))
> [translation:ERROR] ?Exception: unexpected prebuilt constant:
> <built-in function compressobj>
> [translation:ERROR] Processing block:
> [translation:ERROR] ?block at 27 is a <class
> 'pypy.objspace.flow.flowcontext.SpamBlock'>
> [translation:ERROR] ?in (pypy.translator.goal.green_bottles:10)compress
> [translation:ERROR] ?containing the following operations:
> [translation:ERROR] ? ? ? ?v0 =
> simple_call((builtin_function_or_method compressobj), (6), (8), (-15),
> (8), (0))
> [translation:ERROR] ? ? ? ?v1 = getattr(v0, ('compress'))
> [translation:ERROR] ? ? ? ?v2 = simple_call(v1, orig_data_0)
> [translation:ERROR] ? ? ? ?v3 = getattr(v0, ('flush'))
> [translation:ERROR] ? ? ? ?v4 = simple_call(v3)
> [translation:ERROR] ? ? ? ?v5 = add(v2, v4)
> [translation:ERROR] ? ? ? ?v6 =
> simple_call((builtin_function_or_method crc32), orig_data_0)
> [translation:ERROR] ? ? ? ?v7 = newtuple(v5, v6)
> [translation:ERROR] ?--end--
> [translation] start debugger...
>> /Users/pjcock/pypy-hg/pypy/annotation/bookkeeper.py(535)getdesc()
> -> pyobj,))
> (Pdb+) quit
>
> Calling it without the --run gives the same:
>
> $ python2.6 translate.py targetzlib.py
> ...
>
> Should my benchmark Python code be RPython?
>
> Thanks,
>
> Peter
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev

Hey

You're confusing levels. Your benchmark code cannot call zlib. I
suggested you benchmark not zlib, but your changes to StringBuilder
that you proposed. That does not (and cannot) require zlib. You'll be
testing something smaller that you're trying to change instead. Then
obviously a comparison with CPython is meaningless.

Cheers,
fijal

From p.j.a.cock at googlemail.com  Tue Mar 20 18:56:54 2012
From: p.j.a.cock at googlemail.com (Peter Cock)
Date: Tue, 20 Mar 2012 17:56:54 +0000
Subject: [pypy-dev] targetreallynopstandalone.py => TypeError: string vs
	LocalPath
In-Reply-To: <4F687AC9.4040205@gmail.com>
References: <CAKVJ-_6=MKCgPx4kcRqWiQc6wzB-fuYmWSarp2+nRJJSVxTy5Q@mail.gmail.com>
	<4F687AC9.4040205@gmail.com>
Message-ID: <CAKVJ-_7VPA0LZWEgrd69oGR5az16uVk2yueNmZT7WwpwTKWQjA@mail.gmail.com>

On Tue, Mar 20, 2012 at 12:40 PM, Antonio Cuni <anto.cuni at gmail.com> wrote:
> On 03/20/2012 12:51 PM, Peter Cock wrote:
>>
>> Next, using the Richards example (short but non-trivial),
>>
>> $ cd pypy/translator/goal
>> $ python translate.py --run targetrichards.py
>
> I never heard of the --run option before :-)

It seems helpful - giving the PyPy vs Python numbers right
there in one go.

> You don't need it: just run translate.py targetrichards.py
> and you'll get a targetrichards-c executable.

Doing that all do get a testing_1.dylib file (Mach-O 64-bit
dynamically linked shared library x86_64) under a temp
folder. Where should this targetrichards-c executable be?

Interestingly trying some of the other examples the binary
is produced in the current directory:

$ python2.6 translate.py targetreallynopstandalone.py
...
[translation:info] created:
/Users/pjcock/pypy-hg/pypy/translator/goal/targetreallynopstandalone-c
[Timer] Timings:
[Timer] annotate                       ---  2.0 s
[Timer] rtype_lltype                   ---  0.2 s
[Timer] backendopt_lltype              ---  0.0 s
[Timer] stackcheckinsertion_lltype     ---  0.0 s
[Timer] database_c                     --- 14.4 s
[Timer] source_c                       ---  2.3 s
[Timer] compile_c                      ---  1.3 s
[Timer] =========================================
[Timer] Total:                         --- 20.2 s

Peter

From arigo at tunes.org  Tue Mar 20 18:57:10 2012
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 20 Mar 2012 18:57:10 +0100
Subject: [pypy-dev] targetreallynopstandalone.py => TypeError: string vs
	LocalPath
In-Reply-To: <CAKVJ-_6dmRYRcsz7-dj81McQDW8i3rKq50i+fLnxrtydo6zscQ@mail.gmail.com>
References: <CAKVJ-_6=MKCgPx4kcRqWiQc6wzB-fuYmWSarp2+nRJJSVxTy5Q@mail.gmail.com>
	<4F687AC9.4040205@gmail.com>
	<CAKVJ-_6dmRYRcsz7-dj81McQDW8i3rKq50i+fLnxrtydo6zscQ@mail.gmail.com>
Message-ID: <CAMSv6X3s-eYMoiwsyDNkhKQXtSu40kKvm4PkOpcm3NuRU+-bqA@mail.gmail.com>

Hi Peter,

On Tue, Mar 20, 2012 at 15:17, Peter Cock <p.j.a.cock at googlemail.com> wrote:
>> You don't need it: just run translate.py targetrichards.py
>> and you'll get a targetrichards-c executable.
>
> Doing that I do get a testing_1.dylib file (Mach-O 64-bit
> dynamically linked shared library x86_64) under a temp
> folder, but no sign of a targetrichards-c executable.

Ah, I see.  That's because targetrichards.py doesn't create a
stand-alone executable.  The --run option is maybe useful in this
case, but that case itself is kind of deprecated.  You probably want
targetrpystonedalone, which creates a stand-alone executable, with all
optimizations; it runs rpystone by default, or richards if executed
with the command-line argument "r".  (Yes, I know it's obscure.)

For now the --run option has been removed in 0e5b140cf620.


A bient?t,

Armin.

From stefan at drees.name  Wed Mar 21 09:26:33 2012
From: stefan at drees.name (Stefan Drees)
Date: Wed, 21 Mar 2012 09:26:33 +0100
Subject: [pypy-dev] targetreallynopstandalone.py => TypeError: string vs
 LocalPath
In-Reply-To: <CAMSv6X3s-eYMoiwsyDNkhKQXtSu40kKvm4PkOpcm3NuRU+-bqA@mail.gmail.com>
References: <CAKVJ-_6=MKCgPx4kcRqWiQc6wzB-fuYmWSarp2+nRJJSVxTy5Q@mail.gmail.com>
	<4F687AC9.4040205@gmail.com>
	<CAKVJ-_6dmRYRcsz7-dj81McQDW8i3rKq50i+fLnxrtydo6zscQ@mail.gmail.com>
	<CAMSv6X3s-eYMoiwsyDNkhKQXtSu40kKvm4PkOpcm3NuRU+-bqA@mail.gmail.com>
Message-ID: <4F6990B9.6020200@drees.name>

Thanks a lot Armin,

Am 20.03.12 18:57, schrieb Armin Rigo:
> Hi Peter,
>
> On Tue, Mar 20, 2012 at 15:17, Peter Cock<p.j.a.cock at googlemail.com>  wrote:
>>> You don't need it: just run translate.py targetrichards.py
>>> and you'll get a targetrichards-c executable.
>>
>> Doing that I do get a testing_1.dylib file (Mach-O 64-bit
>> dynamically linked shared library x86_64) under a temp
>> folder, but no sign of a targetrichards-c executable.
>
> Ah, I see.  That's because targetrichards.py doesn't create a
> stand-alone executable.  The --run option is maybe useful in this
> case, but that case itself is kind of deprecated.  You probably want
> targetrpystonedalone, which creates a stand-alone executable, with all
> optimizations; it runs rpystone by default, or richards if executed
> with the command-line argument "r".  (Yes, I know it's obscure.)
> ...

I also searched for a non-existant targetrichards-c.
And thanks for the hint on the "r" argument for targetrpystonedalone-c 
reading the source helps a lot, as "rich" and "pyst" also work as 
arguments ;-)

All the best,
Stefan.



-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2310 bytes
Desc: S/MIME Kryptografische Unterschrift
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120321/7a68a7f4/attachment-0001.bin>

From arigo at tunes.org  Wed Mar 21 17:55:25 2012
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 21 Mar 2012 17:55:25 +0100
Subject: [pypy-dev] Bringing Cython and PyPy closer together
In-Reply-To: <CAK5idxThPMvpoFGODum85_pnd6ny9MZ81i7961uYvSkWODrqKg@mail.gmail.com>
References: <jhg54k$dte$1@dough.gmane.org> <jhnok5$ve2$1@dough.gmane.org>
	<CAGmFidbutXahNAKbHgaLFOM39uTKZ8C3TkrPLeauB0Jm5CQ-bg@mail.gmail.com>
	<jho9he$c2k$1@dough.gmane.org>
	<CAGmFidZ2DDxnPBi7E5Bo8PTn3iubjfrtxmBv1uEsaombo2ey1Q@mail.gmail.com>
	<jhobh9$ots$1@dough.gmane.org>
	<CAGmFidbEPpbQPrKEsNHkMQ1e3mgsYV1vd0SivdVPvSOYaLSXmA@mail.gmail.com>
	<jie2mh$qub$1@dough.gmane.org> <jk6kjs$840$1@dough.gmane.org>
	<CAK5idxThPMvpoFGODum85_pnd6ny9MZ81i7961uYvSkWODrqKg@mail.gmail.com>
Message-ID: <CAMSv6X0pJMveNQSVWdh6A7-ke6W0rvi_CahYmVLX+xdX9tgUBQ@mail.gmail.com>

Hi,

On Mon, Mar 19, 2012 at 09:11, Maciej Fijalkowski <fijall at gmail.com> wrote:
>>> The (trivial) implementation of the two functions is at the end of this file:
>>>
>>> http://bugs.python.org/file24613/exc_info_capi.patch
>>>
>>> Could you add them to PyPy?

The new functions make sense to me.  Anyone with cpyext knowledge can
add them to PyPy.  I can probably do it if needed, although I don't
know cpyext very well.  (Alternatively, coming up yourself with a
patch would be the best way to move things forward :-)


A bient?t,

Armin.

From p.j.a.cock at googlemail.com  Wed Mar 21 18:10:35 2012
From: p.j.a.cock at googlemail.com (Peter Cock)
Date: Wed, 21 Mar 2012 17:10:35 +0000
Subject: [pypy-dev] Speeding up zlib in standard library
In-Reply-To: <CAK5idxQpU61njri3PES+1cCeM1ddLFf=B1vFozBmKh7Pxh=HDg@mail.gmail.com>
References: <CAKVJ-_5D4OUBSkavYqW7usBkaZMxu-EjEtb9NaKU-iCMA=XZQg@mail.gmail.com>
	<CAPZV6o8fpWjG2d8L6ig4SQds=EAUnb=sypeor9HzazdmXyZy=g@mail.gmail.com>
	<CAKVJ-_4zry6WMzg7Gfu2zUj=t-fpce_givh4TFj57aHiagwQ+Q@mail.gmail.com>
	<CAMSv6X1vXnT6jRs2MHvwa613nfZfrqgt9sEVbB0tk6tX5+Ggew@mail.gmail.com>
	<CAKVJ-_5jtHOxXpcnUVojV-a070BWV8fybZ99EKaZqjT9+DoLJA@mail.gmail.com>
	<CA+=c-xbzg8k3M_d6N=JsV1aaHRHv-2P1Cm-cSSPUdUh6jYJsnA@mail.gmail.com>
	<CAKVJ-_6Ov+MwjHginrmpyga2TLWZkxP9mFSaSZcUYToNDRkm0g@mail.gmail.com>
	<CAK5idxTbhQi9X7O+M2eJy-nFY6BWiFmTdghhfohjNv0y8_Yd1A@mail.gmail.com>
	<CAKVJ-_5Zd_BXHa1ycDGNr3SS8txKF1wzKgZ7QydYkpQeqEXt3w@mail.gmail.com>
	<CAK5idxRrznx6ifCPD3Y1p8XHKYOwO+RmqoA=rdTKeJkhvpuSsA@mail.gmail.com>
	<CAKVJ-_4=xyR3qGS-ZdcCwGXfxphLwD5Ezgq48f=7ydzDp1A4+A@mail.gmail.com>
	<CAK5idxQpU61njri3PES+1cCeM1ddLFf=B1vFozBmKh7Pxh=HDg@mail.gmail.com>
Message-ID: <CAKVJ-_5-CmM492A-qaMOD2r5Sz6cHhV2Gqz64o_gk2qExyEcng@mail.gmail.com>

On Tue, Mar 20, 2012 at 2:52 PM, Maciej Fijalkowski <fijall at gmail.com> wrote:
>
> Hey
>
> You're confusing levels. Your benchmark code cannot call zlib.

PyPy development has quite a learning curve.

> I suggested you benchmark not zlib, but your changes to StringBuilder
> that you proposed. That does not (and cannot) require zlib. You'll be
> testing something smaller that you're trying to change instead. Then
> obviously a comparison with CPython is meaningless.

i.e. I would create the new StringBuilder benchmark as the file
pypy/translator/targetStringBuilder.py and do this:

$ cd pypy/translator/goal/
$ python translate.py targetStringBuilder.py
$ ./targetStringBuilder-c

Then modify the pypy/rpython/lltypesystem/rbuilder.py (or whatever)
and repeat this. And that should compare the benchmark translation
by the virgin PyPy against the translation by my modified PyPy?

That doesn't sound quite so intimidating... and if I've understood this
now it does seem like the basis of a good development FAQ entry:
http://doc.pypy.org/en/latest/faq.html#development

Then if the benchmark results are encouraging, in principle I could
then recompile the whole of pypy (which is slow), and then go back
to run the patched pypy on my real script to see what difference it
makes, if any.

Right?

Thanks for bearing with me,

Peter

From arigo at tunes.org  Wed Mar 21 18:38:47 2012
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 21 Mar 2012 18:38:47 +0100
Subject: [pypy-dev] ZODB3
In-Reply-To: <4F509B09.10703@gmx.de>
References: <jhg54k$dte$1@dough.gmane.org> <jhnok5$ve2$1@dough.gmane.org>
	<CAGmFidbutXahNAKbHgaLFOM39uTKZ8C3TkrPLeauB0Jm5CQ-bg@mail.gmail.com>
	<jho9he$c2k$1@dough.gmane.org>
	<CAGmFidZ2DDxnPBi7E5Bo8PTn3iubjfrtxmBv1uEsaombo2ey1Q@mail.gmail.com>
	<jhobh9$ots$1@dough.gmane.org>
	<CAGmFidbEPpbQPrKEsNHkMQ1e3mgsYV1vd0SivdVPvSOYaLSXmA@mail.gmail.com>
	<jie2mh$qub$1@dough.gmane.org> <4F4B5296.6050903@gmx.de>
	<CAK5idxRiqt0ga5JjWz0j1NbkrAbTvJmQvvNSJdxgqJrXs3j0dQ@mail.gmail.com>
	<CAMSv6X1RF6pNuOHFP0OKM7bUpsbtPckazFOShnD_vmQX9m71Vw@mail.gmail.com>
	<4F4C99D7.3050407@gmx.de>
	<CAMSv6X19hg7gPcyiV0bs8jWCWonjcdJkduYNVc6fTWTf7d7WcQ@mail.gmail.com>
	<4F4CA4A3.40100@gmx.de>
	<CAMSv6X1UBek-h1TiupYnFTpoZf2ts5mKzHgqCZgKXfw1rXDx-Q@mail.gmail.com>
	<4F509B09.10703@gmx.de>
Message-ID: <CAMSv6X0x0iqpzVZsUs26uR9=W3Dwd-Kx8xuuLJgVg6sVU0+cag@mail.gmail.com>

Hi,

On Fri, Mar 2, 2012 at 11:03, Aroldo Souza-Leite <asouzaleite at gmx.de> wrote:
> ? ?data = self.__dict__.pop('_container')
> KeyError: '_container'

Last I heard, the Persistent base class, written in C, uses old tricks
that are kind of deprecated; if I remember correctly, before Python
2.2, it was known as the place that introduced the trick called
"extension classes" to Python, which later became "new-style classes"
in Python 2.2.

So I would not be surprized if that causes the user-visible __dict__
of its instances to miss an attribute like "_container" or "data", and
so be generally unsupported by PyPy's cpyext.

If you really care about it, you may want to rewrite at least the base
class, persistent.Persistent, in pure Python.


A bient?t,

Armin.

From arigo at tunes.org  Wed Mar 21 18:41:57 2012
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 21 Mar 2012 18:41:57 +0100
Subject: [pypy-dev] How to get PIL to install?
In-Reply-To: <CAGBU8t6J7DpKECL18-T6QMxrfsONio+h3wtm0d-E+FDFO-vJeQ@mail.gmail.com>
References: <CAGBU8t6MDthpmuY2arM6_UM6Xm8DXiWkZ4mc61a_C+DfRx6=CA@mail.gmail.com>
	<CAMSv6X0uZRKfd2ov-jXoTH788uL8osLZ0BSbOPSfq4wQ+=6D+Q@mail.gmail.com>
	<CAGBU8t4XALkU4OYiFYqa1NP3iDFkVgeFTp8NYB=E9g+Zah-J=g@mail.gmail.com>
	<CAGBU8t6J7DpKECL18-T6QMxrfsONio+h3wtm0d-E+FDFO-vJeQ@mail.gmail.com>
Message-ID: <CAMSv6X3DSoBwpK3EXND=LK3-+gGEnWNmt+LzJf+yMBMVOEB05g@mail.gmail.com>

Hi Jacob,

On Mon, Mar 5, 2012 at 14:17, Jacob Finkelman <eh2406 at wayne.edu> wrote:
> If it involves translating pypy, then I am stuck. This laptop does not have
> the resources to translate pypy.

Yes, but I've made the change in the official source of PyPy too,
which means that you can simply upgrade to a more recent version from
http://buildbot.pypy.org/nightly/trunk/ and try again.


A bient?t,

Armin.

From fijall at gmail.com  Wed Mar 21 18:49:37 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 21 Mar 2012 19:49:37 +0200
Subject: [pypy-dev] Speeding up zlib in standard library
In-Reply-To: <CAKVJ-_5-CmM492A-qaMOD2r5Sz6cHhV2Gqz64o_gk2qExyEcng@mail.gmail.com>
References: <CAKVJ-_5D4OUBSkavYqW7usBkaZMxu-EjEtb9NaKU-iCMA=XZQg@mail.gmail.com>
	<CAPZV6o8fpWjG2d8L6ig4SQds=EAUnb=sypeor9HzazdmXyZy=g@mail.gmail.com>
	<CAKVJ-_4zry6WMzg7Gfu2zUj=t-fpce_givh4TFj57aHiagwQ+Q@mail.gmail.com>
	<CAMSv6X1vXnT6jRs2MHvwa613nfZfrqgt9sEVbB0tk6tX5+Ggew@mail.gmail.com>
	<CAKVJ-_5jtHOxXpcnUVojV-a070BWV8fybZ99EKaZqjT9+DoLJA@mail.gmail.com>
	<CA+=c-xbzg8k3M_d6N=JsV1aaHRHv-2P1Cm-cSSPUdUh6jYJsnA@mail.gmail.com>
	<CAKVJ-_6Ov+MwjHginrmpyga2TLWZkxP9mFSaSZcUYToNDRkm0g@mail.gmail.com>
	<CAK5idxTbhQi9X7O+M2eJy-nFY6BWiFmTdghhfohjNv0y8_Yd1A@mail.gmail.com>
	<CAKVJ-_5Zd_BXHa1ycDGNr3SS8txKF1wzKgZ7QydYkpQeqEXt3w@mail.gmail.com>
	<CAK5idxRrznx6ifCPD3Y1p8XHKYOwO+RmqoA=rdTKeJkhvpuSsA@mail.gmail.com>
	<CAKVJ-_4=xyR3qGS-ZdcCwGXfxphLwD5Ezgq48f=7ydzDp1A4+A@mail.gmail.com>
	<CAK5idxQpU61njri3PES+1cCeM1ddLFf=B1vFozBmKh7Pxh=HDg@mail.gmail.com>
	<CAKVJ-_5-CmM492A-qaMOD2r5Sz6cHhV2Gqz64o_gk2qExyEcng@mail.gmail.com>
Message-ID: <CAK5idxS7xO=jQoW27fTBwSdn=2HjX_oHOVQHfNgaOZsk=Ado3w@mail.gmail.com>

On Wed, Mar 21, 2012 at 7:10 PM, Peter Cock <p.j.a.cock at googlemail.com> wrote:
> On Tue, Mar 20, 2012 at 2:52 PM, Maciej Fijalkowski <fijall at gmail.com> wrote:
>>
>> Hey
>>
>> You're confusing levels. Your benchmark code cannot call zlib.
>
> PyPy development has quite a learning curve.
>
>> I suggested you benchmark not zlib, but your changes to StringBuilder
>> that you proposed. That does not (and cannot) require zlib. You'll be
>> testing something smaller that you're trying to change instead. Then
>> obviously a comparison with CPython is meaningless.
>
> i.e. I would create the new StringBuilder benchmark as the file
> pypy/translator/targetStringBuilder.py and do this:
>
> $ cd pypy/translator/goal/
> $ python translate.py targetStringBuilder.py
> $ ./targetStringBuilder-c
>
> Then modify the pypy/rpython/lltypesystem/rbuilder.py (or whatever)
> and repeat this. And that should compare the benchmark translation
> by the virgin PyPy against the translation by my modified PyPy?
>
> That doesn't sound quite so intimidating... and if I've understood this
> now it does seem like the basis of a good development FAQ entry:
> http://doc.pypy.org/en/latest/faq.html#development
>
> Then if the benchmark results are encouraging, in principle I could
> then recompile the whole of pypy (which is slow), and then go back
> to run the patched pypy on my real script to see what difference it
> makes, if any.
>
> Right?
>
> Thanks for bearing with me,
>
> Peter

Right, you got it perfect :)

Can you suggest the wording for a FAQ entry?

Cheers,
fijal

From arigo at tunes.org  Wed Mar 21 19:03:23 2012
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 21 Mar 2012 19:03:23 +0100
Subject: [pypy-dev] pypy2exe
In-Reply-To: <CANeLV3YfiEt=BH47MM5E8e2cFUX6M0fWMSw8vfbZiGtpE+6tog@mail.gmail.com>
References: <CANeLV3Zpo8PFYd96xUSrmiBOeOxJco+bWovHvASLfPVKYxK4zg@mail.gmail.com>
	<CAMSv6X1OtFD5xMS=T+j4e4ZDvV0Z7VYENg+N6k_d1heieYAU3w@mail.gmail.com>
	<CANeLV3YfiEt=BH47MM5E8e2cFUX6M0fWMSw8vfbZiGtpE+6tog@mail.gmail.com>
Message-ID: <CAMSv6X0BU3ZP3aRowbr8yA6L2LXKQbttcpc9gOff0+fwaHkRsA@mail.gmail.com>

Hi Paolo,

On Tue, Mar 6, 2012 at 16:07, Paolo Basso <pa.basso at gmail.com> wrote:
> I mean, I know I'm probably going to say something which is not completely
> in the spirit of an open source project but,?no way to hide the code?

There is no official support for that, but you can try to play around.
 You can edit the file pypy/translator/goal/app_main.py: this file is
"pre-imported" at translation time.  So far the only global import is
"import sys".  But if you add more imports, the corresponding files
will also be pre-imported when you start "translate.py".  This means
that in this way you create a "pypy-c" binary that contains a frozen
live image of your modules, already present in sys.modules when you
start this particular "pypy-c" binary.  In this way you can distribute
a "bloated pypy-c" that can run without even looking for some or all
the .py or .pyc files.


A bient?t,

Armin.

From amauryfa at gmail.com  Wed Mar 21 20:46:41 2012
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Wed, 21 Mar 2012 20:46:41 +0100
Subject: [pypy-dev] ZODB3
In-Reply-To: <CAMSv6X0x0iqpzVZsUs26uR9=W3Dwd-Kx8xuuLJgVg6sVU0+cag@mail.gmail.com>
References: <jhg54k$dte$1@dough.gmane.org> <jhnok5$ve2$1@dough.gmane.org>
	<CAGmFidbutXahNAKbHgaLFOM39uTKZ8C3TkrPLeauB0Jm5CQ-bg@mail.gmail.com>
	<jho9he$c2k$1@dough.gmane.org>
	<CAGmFidZ2DDxnPBi7E5Bo8PTn3iubjfrtxmBv1uEsaombo2ey1Q@mail.gmail.com>
	<jhobh9$ots$1@dough.gmane.org>
	<CAGmFidbEPpbQPrKEsNHkMQ1e3mgsYV1vd0SivdVPvSOYaLSXmA@mail.gmail.com>
	<jie2mh$qub$1@dough.gmane.org> <4F4B5296.6050903@gmx.de>
	<CAK5idxRiqt0ga5JjWz0j1NbkrAbTvJmQvvNSJdxgqJrXs3j0dQ@mail.gmail.com>
	<CAMSv6X1RF6pNuOHFP0OKM7bUpsbtPckazFOShnD_vmQX9m71Vw@mail.gmail.com>
	<4F4C99D7.3050407@gmx.de>
	<CAMSv6X19hg7gPcyiV0bs8jWCWonjcdJkduYNVc6fTWTf7d7WcQ@mail.gmail.com>
	<4F4CA4A3.40100@gmx.de>
	<CAMSv6X1UBek-h1TiupYnFTpoZf2ts5mKzHgqCZgKXfw1rXDx-Q@mail.gmail.com>
	<4F509B09.10703@gmx.de>
	<CAMSv6X0x0iqpzVZsUs26uR9=W3Dwd-Kx8xuuLJgVg6sVU0+cag@mail.gmail.com>
Message-ID: <CAGmFidbv_xHVe0VVz8gsnkAb-58QVnP-b1i3vRoiUnSRXR4viQ@mail.gmail.com>

2012/3/21 Armin Rigo <arigo at tunes.org>:
> On Fri, Mar 2, 2012 at 11:03, Aroldo Souza-Leite <asouzaleite at gmx.de> wrote:
>> ? ?data = self.__dict__.pop('_container')
>> KeyError: '_container'
>
> Last I heard, the Persistent base class, written in C, uses old tricks
> that are kind of deprecated; if I remember correctly, before Python
> 2.2, it was known as the place that introduced the trick called
> "extension classes" to Python, which later became "new-style classes"
> in Python 2.2.

The answer is a bit simpler:
Persistent.__getstate__ uses _PyObject_GetDictPtr(), which always
returns NULL in cpyext.
So the state is None and data is lost.

This function is near to impossible to implement with cpyext.
In the same file (cPersistence.c) you have:
   PyObject **dict = _PyObject_GetDictPtr(self);
   if (!*dict) *dict = PyDict_New();
   PyDict_Update(*dict, state);

This code should probably be rewritten with more conventional API:
PyObject_GetAttrString(self, "__dict__") for example.
Or even with pure Python code.

-- 
Amaury Forgeot d'Arc

From arigo at tunes.org  Thu Mar 22 02:12:14 2012
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 22 Mar 2012 02:12:14 +0100
Subject: [pypy-dev] ZODB3
In-Reply-To: <CAGmFidbv_xHVe0VVz8gsnkAb-58QVnP-b1i3vRoiUnSRXR4viQ@mail.gmail.com>
References: <jhg54k$dte$1@dough.gmane.org> <jhnok5$ve2$1@dough.gmane.org>
	<CAGmFidbutXahNAKbHgaLFOM39uTKZ8C3TkrPLeauB0Jm5CQ-bg@mail.gmail.com>
	<jho9he$c2k$1@dough.gmane.org>
	<CAGmFidZ2DDxnPBi7E5Bo8PTn3iubjfrtxmBv1uEsaombo2ey1Q@mail.gmail.com>
	<jhobh9$ots$1@dough.gmane.org>
	<CAGmFidbEPpbQPrKEsNHkMQ1e3mgsYV1vd0SivdVPvSOYaLSXmA@mail.gmail.com>
	<jie2mh$qub$1@dough.gmane.org> <4F4B5296.6050903@gmx.de>
	<CAK5idxRiqt0ga5JjWz0j1NbkrAbTvJmQvvNSJdxgqJrXs3j0dQ@mail.gmail.com>
	<CAMSv6X1RF6pNuOHFP0OKM7bUpsbtPckazFOShnD_vmQX9m71Vw@mail.gmail.com>
	<4F4C99D7.3050407@gmx.de>
	<CAMSv6X19hg7gPcyiV0bs8jWCWonjcdJkduYNVc6fTWTf7d7WcQ@mail.gmail.com>
	<4F4CA4A3.40100@gmx.de>
	<CAMSv6X1UBek-h1TiupYnFTpoZf2ts5mKzHgqCZgKXfw1rXDx-Q@mail.gmail.com>
	<4F509B09.10703@gmx.de>
	<CAMSv6X0x0iqpzVZsUs26uR9=W3Dwd-Kx8xuuLJgVg6sVU0+cag@mail.gmail.com>
	<CAGmFidbv_xHVe0VVz8gsnkAb-58QVnP-b1i3vRoiUnSRXR4viQ@mail.gmail.com>
Message-ID: <CAMSv6X2Vqh=5pzkpaZTGndMUAt6YzgbPTR4aF32b00fTiGvbpw@mail.gmail.com>

Hi,

On Wed, Mar 21, 2012 at 20:46, Amaury Forgeot d'Arc <amauryfa at gmail.com> wrote:
> Persistent.__getstate__ uses _PyObject_GetDictPtr(), which always
> returns NULL in cpyext.

Does it make sense?  Shouldn't it be unimplemented, or raise a
warning, or something?


A bient?t,

Armin.

From eh2406 at wayne.edu  Thu Mar 22 03:57:13 2012
From: eh2406 at wayne.edu (Jacob Finkelman)
Date: Wed, 21 Mar 2012 22:57:13 -0400
Subject: [pypy-dev] How to get PIL to install?
In-Reply-To: <CAMSv6X3DSoBwpK3EXND=LK3-+gGEnWNmt+LzJf+yMBMVOEB05g@mail.gmail.com>
References: <CAGBU8t6MDthpmuY2arM6_UM6Xm8DXiWkZ4mc61a_C+DfRx6=CA@mail.gmail.com>
	<CAMSv6X0uZRKfd2ov-jXoTH788uL8osLZ0BSbOPSfq4wQ+=6D+Q@mail.gmail.com>
	<CAGBU8t4XALkU4OYiFYqa1NP3iDFkVgeFTp8NYB=E9g+Zah-J=g@mail.gmail.com>
	<CAGBU8t6J7DpKECL18-T6QMxrfsONio+h3wtm0d-E+FDFO-vJeQ@mail.gmail.com>
	<CAMSv6X3DSoBwpK3EXND=LK3-+gGEnWNmt+LzJf+yMBMVOEB05g@mail.gmail.com>
Message-ID: <CAGBU8t5Vd_DX+38opPi7vmvO0cdOQvOqYDDb34a7QsXBwCxvUA@mail.gmail.com>

Thanks

I instaled
pypy-c-jit-latest-win32.zip 17M 2012-03-13
then Install distribute and PIP.
run "cmd.exe"
cd c:\...
pip install pillow
and
...
??? C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(76) : error
C2632: 'char' followed by 'char' is illegal
??? C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(76) : error
C2059: syntax error : ','
??? C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(77) : error
C2632: 'short' followed by 'short' is illegal
??? C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(77) : error
C2059: syntax error : ','
??? C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(80) : warnin
g C4114: same type qualifier used more than once
??? C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(80) : error
C2632: 'char' followed by 'char' is illegal
??? C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(80) : error
C2059: syntax error : ','
??? C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(81) : warnin
g C4114: same type qualifier used more than once
??? C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(81) : error
C2632: 'short' followed by 'short' is illegal
??? C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(81) : error
C2059: syntax error : ','
??? error: command 'c:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe'
failed with exit status 2
??? Complete output from command C:\pypy\pypy.exe -c "import setuptools;__file__
='c:\\pypy\\bin\\build\\pillow\\setup.py';exec(compile(open(__file__).read().rep
lace('\r\n', '\n'), __file__, 'exec'))" install --single-version-externally-mana
ged --record c:\users\jlap\appdata\local\temp\pip-g_jlod-record\install-record.t
xt:
??? running install
running build
running build_py
creating build
creating build\lib.win32-2.7
creating build\lib.win32-2.7\PIL
copying PIL\ArgImagePlugin.py -> build\lib.win32-2.7\PIL
copying PIL\BdfFontFile.py -> build\lib.win32-2.7\PIL
copying PIL\BmpImagePlugin.py -> build\lib.win32-2.7\PIL
copying PIL\BufrStubImagePlugin.py -> build\lib.win32-2.7\PIL
copying PIL\ContainerIO.py -> build\lib.win32-2.7\PIL
copying PIL\CurImagePlugin.py -> build\lib.win32-2.7\PIL
copying PIL\DcxImagePlugin.py -> build\lib.win32-2.7\PIL
copying PIL\EpsImagePlugin.py -> build\lib.win32-2.7\PIL
copying PIL\ExifTags.py -> build\lib.win32-2.7\PIL
copying PIL\FitsStubImagePlugin.py -> build\lib.win32-2.7\PIL
copying PIL\FliImagePlugin.py -> build\lib.win32-2.7\PIL
copying PIL\FontFile.py -> build\lib.win32-2.7\PIL
copying PIL\FpxImagePlugin.py -> build\lib.win32-2.7\PIL
copying PIL\GbrImagePlugin.py -> build\lib.win32-2.7\PIL
copying PIL\GdImageFile.py -> build\lib.win32-2.7\PIL
copying PIL\GifImagePlugin.py -> build\lib.win32-2.7\PIL
copying PIL\GimpGradientFile.py -> build\lib.win32-2.7\PIL
copying PIL\GimpPaletteFile.py -> build\lib.win32-2.7\PIL
copying PIL\GribStubImagePlugin.py -> build\lib.win32-2.7\PIL
copying PIL\Hdf5StubImagePlugin.py -> build\lib.win32-2.7\PIL
copying PIL\IcnsImagePlugin.py -> build\lib.win32-2.7\PIL
copying PIL\IcoImagePlugin.py -> build\lib.win32-2.7\PIL
copying PIL\Image.py -> build\lib.win32-2.7\PIL
copying PIL\ImageChops.py -> build\lib.win32-2.7\PIL
copying PIL\ImageCms.py -> build\lib.win32-2.7\PIL
copying PIL\ImageColor.py -> build\lib.win32-2.7\PIL
copying PIL\ImageDraw.py -> build\lib.win32-2.7\PIL
copying PIL\ImageDraw2.py -> build\lib.win32-2.7\PIL
copying PIL\ImageEnhance.py -> build\lib.win32-2.7\PIL
copying PIL\ImageFile.py -> build\lib.win32-2.7\PIL
copying PIL\ImageFileIO.py -> build\lib.win32-2.7\PIL
copying PIL\ImageFilter.py -> build\lib.win32-2.7\PIL
copying PIL\ImageFont.py -> build\lib.win32-2.7\PIL
copying PIL\ImageGL.py -> build\lib.win32-2.7\PIL
copying PIL\ImageGrab.py -> build\lib.win32-2.7\PIL
copying PIL\ImageMath.py -> build\lib.win32-2.7\PIL
copying PIL\ImageMode.py -> build\lib.win32-2.7\PIL
copying PIL\ImageOps.py -> build\lib.win32-2.7\PIL
copying PIL\ImagePalette.py -> build\lib.win32-2.7\PIL
copying PIL\ImagePath.py -> build\lib.win32-2.7\PIL
copying PIL\ImageQt.py -> build\lib.win32-2.7\PIL
copying PIL\ImageSequence.py -> build\lib.win32-2.7\PIL
copying PIL\ImageShow.py -> build\lib.win32-2.7\PIL
copying PIL\ImageStat.py -> build\lib.win32-2.7\PIL
copying PIL\ImageTk.py -> build\lib.win32-2.7\PIL
copying PIL\ImageTransform.py -> build\lib.win32-2.7\PIL
copying PIL\ImageWin.py -> build\lib.win32-2.7\PIL
copying PIL\ImImagePlugin.py -> build\lib.win32-2.7\PIL
copying PIL\ImtImagePlugin.py -> build\lib.win32-2.7\PIL
copying PIL\IptcImagePlugin.py -> build\lib.win32-2.7\PIL
copying PIL\JpegImagePlugin.py -> build\lib.win32-2.7\PIL
copying PIL\McIdasImagePlugin.py -> build\lib.win32-2.7\PIL
copying PIL\MicImagePlugin.py -> build\lib.win32-2.7\PIL
copying PIL\MpegImagePlugin.py -> build\lib.win32-2.7\PIL
copying PIL\MspImagePlugin.py -> build\lib.win32-2.7\PIL
copying PIL\OleFileIO.py -> build\lib.win32-2.7\PIL
copying PIL\PaletteFile.py -> build\lib.win32-2.7\PIL
copying PIL\PalmImagePlugin.py -> build\lib.win32-2.7\PIL
copying PIL\PcdImagePlugin.py -> build\lib.win32-2.7\PIL
copying PIL\PcfFontFile.py -> build\lib.win32-2.7\PIL
copying PIL\PcxImagePlugin.py -> build\lib.win32-2.7\PIL
copying PIL\PdfImagePlugin.py -> build\lib.win32-2.7\PIL
copying PIL\PixarImagePlugin.py -> build\lib.win32-2.7\PIL
copying PIL\PngImagePlugin.py -> build\lib.win32-2.7\PIL
copying PIL\PpmImagePlugin.py -> build\lib.win32-2.7\PIL
copying PIL\PsdImagePlugin.py -> build\lib.win32-2.7\PIL
copying PIL\PSDraw.py -> build\lib.win32-2.7\PIL
copying PIL\SgiImagePlugin.py -> build\lib.win32-2.7\PIL
copying PIL\SpiderImagePlugin.py -> build\lib.win32-2.7\PIL
copying PIL\SunImagePlugin.py -> build\lib.win32-2.7\PIL
copying PIL\TarIO.py -> build\lib.win32-2.7\PIL
copying PIL\TgaImagePlugin.py -> build\lib.win32-2.7\PIL
copying PIL\TiffImagePlugin.py -> build\lib.win32-2.7\PIL
copying PIL\TiffTags.py -> build\lib.win32-2.7\PIL
copying PIL\WalImageFile.py -> build\lib.win32-2.7\PIL
copying PIL\WmfImagePlugin.py -> build\lib.win32-2.7\PIL
copying PIL\XbmImagePlugin.py -> build\lib.win32-2.7\PIL
copying PIL\XpmImagePlugin.py -> build\lib.win32-2.7\PIL
copying PIL\XVThumbImagePlugin.py -> build\lib.win32-2.7\PIL
copying PIL\__init__.py -> build\lib.win32-2.7\PIL
running build_ext
building '_imaging' extension
creating build\temp.win32-2.7
creating build\temp.win32-2.7\Release
creating build\temp.win32-2.7\Release\libImaging
c:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe /c /nologo /Ox /MD /W
3 /GS- /DNDEBUG -IlibImaging -IC:\pypy\include /Tc_imaging.c /Fobuild\temp.win32
-2.7\Release\_imaging.obj
_imaging.c
_imaging.c(1230) : warning C4133: 'function' : incompatible types - from 'PyStri
ngObject *' to 'PyObject *'
_imaging.c(1384) : warning C4996: 'strcpy': This function or variable may be uns
afe. Consider using strcpy_s instead. To disable deprecation, use _CRT_SECURE_NO
_WARNINGS. See online help for details.
_imaging.c(1412) : warning C4996: 'strcpy': This function or variable may be uns
afe. Consider using strcpy_s instead. To disable deprecation, use _CRT_SECURE_NO
_WARNINGS. See online help for details.
_imaging.c(1550) : warning C4996: 'strcpy': This function or variable may be uns
afe. Consider using strcpy_s instead. To disable deprecation, use _CRT_SECURE_NO
_WARNINGS. See online help for details.
c:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe /c /nologo /Ox /MD /W
3 /GS- /DNDEBUG -IlibImaging -IC:\pypy\include /Tcdecode.c /Fobuild\temp.win32-2
.7\Release\decode.obj
decode.c
c:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe /c /nologo /Ox /MD /W
3 /GS- /DNDEBUG -IlibImaging -IC:\pypy\include /Tcencode.c /Fobuild\temp.win32-2
.7\Release\encode.obj
encode.c
encode.c(161) : warning C4996: 'write': The POSIX name for this item is deprecat
ed. Instead, use the ISO C++ conformant name: _write. See online help for detail
s.
??????? c:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\io.h(322) : see
declaration of 'write'
c:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe /c /nologo /Ox /MD /W
3 /GS- /DNDEBUG -IlibImaging -IC:\pypy\include /Tcmap.c /Fobuild\temp.win32-2.7\
Release\map.obj
map.c
C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(76) : warning C4
114: same type qualifier used more than once
C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(76) : error C263
2: 'char' followed by 'char' is illegal
C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(76) : error C205
9: syntax error : ','
C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(77) : error C263
2: 'short' followed by 'short' is illegal
C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(77) : error C205
9: syntax error : ','
C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(80) : warning C4
114: same type qualifier used more than once
C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(80) : error C263
2: 'char' followed by 'char' is illegal
C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(80) : error C205
9: syntax error : ','
C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(81) : warning C4
114: same type qualifier used more than once
C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(81) : error C263
2: 'short' followed by 'short' is illegal
C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(81) : error C205
9: syntax error : ','
error: command 'c:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe' fail
ed with exit status 2
----------------------------------------
Command C:\pypy\pypy.exe -c "import setuptools;__file__='c:\\pypy\\bin\\build\\p
illow\\setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __fil
e__, 'exec'))" install --single-version-externally-managed --record c:\users\jla
p\appdata\local\temp\pip-g_jlod-record\install-record.txt failed with error code
?1 in c:\pypy\bin\build\pillow
Storing complete log in C:\Users\Jlap\AppData\Roaming\pip\pip.log
c:\pypy\bin>

On Wed, Mar 21, 2012 at 1:41 PM, Armin Rigo <arigo at tunes.org> wrote:
>
> Hi Jacob,
>
> On Mon, Mar 5, 2012 at 14:17, Jacob Finkelman <eh2406 at wayne.edu> wrote:
> > If it involves translating pypy, then I am stuck. This laptop does not have
> > the resources to translate pypy.
>
> Yes, but I've made the change in the official source of PyPy too,
> which means that you can simply upgrade to a more recent version from
> http://buildbot.pypy.org/nightly/trunk/ and try again.
>
>
> A bient?t,
>
> Armin.
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev

From amauryfa at gmail.com  Thu Mar 22 09:33:34 2012
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Thu, 22 Mar 2012 09:33:34 +0100
Subject: [pypy-dev] ZODB3
In-Reply-To: <CAMSv6X2Vqh=5pzkpaZTGndMUAt6YzgbPTR4aF32b00fTiGvbpw@mail.gmail.com>
References: <jhg54k$dte$1@dough.gmane.org> <jhnok5$ve2$1@dough.gmane.org>
	<CAGmFidbutXahNAKbHgaLFOM39uTKZ8C3TkrPLeauB0Jm5CQ-bg@mail.gmail.com>
	<jho9he$c2k$1@dough.gmane.org>
	<CAGmFidZ2DDxnPBi7E5Bo8PTn3iubjfrtxmBv1uEsaombo2ey1Q@mail.gmail.com>
	<jhobh9$ots$1@dough.gmane.org>
	<CAGmFidbEPpbQPrKEsNHkMQ1e3mgsYV1vd0SivdVPvSOYaLSXmA@mail.gmail.com>
	<jie2mh$qub$1@dough.gmane.org> <4F4B5296.6050903@gmx.de>
	<CAK5idxRiqt0ga5JjWz0j1NbkrAbTvJmQvvNSJdxgqJrXs3j0dQ@mail.gmail.com>
	<CAMSv6X1RF6pNuOHFP0OKM7bUpsbtPckazFOShnD_vmQX9m71Vw@mail.gmail.com>
	<4F4C99D7.3050407@gmx.de>
	<CAMSv6X19hg7gPcyiV0bs8jWCWonjcdJkduYNVc6fTWTf7d7WcQ@mail.gmail.com>
	<4F4CA4A3.40100@gmx.de>
	<CAMSv6X1UBek-h1TiupYnFTpoZf2ts5mKzHgqCZgKXfw1rXDx-Q@mail.gmail.com>
	<4F509B09.10703@gmx.de>
	<CAMSv6X0x0iqpzVZsUs26uR9=W3Dwd-Kx8xuuLJgVg6sVU0+cag@mail.gmail.com>
	<CAGmFidbv_xHVe0VVz8gsnkAb-58QVnP-b1i3vRoiUnSRXR4viQ@mail.gmail.com>
	<CAMSv6X2Vqh=5pzkpaZTGndMUAt6YzgbPTR4aF32b00fTiGvbpw@mail.gmail.com>
Message-ID: <CAGmFidanKjf-ByCo+0hmE_qHJopUei47OXzvJth250yG7cxzMQ@mail.gmail.com>

2012/3/22 Armin Rigo <arigo at tunes.org>:
> On Wed, Mar 21, 2012 at 20:46, Amaury Forgeot d'Arc <amauryfa at gmail.com> wrote:
>> Persistent.__getstate__ uses _PyObject_GetDictPtr(), which always
>> returns NULL in cpyext.
>
> Does it make sense? ?Shouldn't it be unimplemented, or raise a
> warning, or something?

The first time I encountered this function it was in SWIG generated code:
https://github.com/klickverbot/swig/blob/244c758f0d32d19856d1e69011aa79f5bf323e0b/Lib/python/pyrun.swg#L1100
which gracefully falls back to a regular PyObject_GetAttr() if the
dict pointer is not provided.

-- 
Amaury Forgeot d'Arc

From arigo at tunes.org  Thu Mar 22 11:08:18 2012
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 22 Mar 2012 11:08:18 +0100
Subject: [pypy-dev] How to get PIL to install?
In-Reply-To: <CAGBU8t5Vd_DX+38opPi7vmvO0cdOQvOqYDDb34a7QsXBwCxvUA@mail.gmail.com>
References: <CAGBU8t6MDthpmuY2arM6_UM6Xm8DXiWkZ4mc61a_C+DfRx6=CA@mail.gmail.com>
	<CAMSv6X0uZRKfd2ov-jXoTH788uL8osLZ0BSbOPSfq4wQ+=6D+Q@mail.gmail.com>
	<CAGBU8t4XALkU4OYiFYqa1NP3iDFkVgeFTp8NYB=E9g+Zah-J=g@mail.gmail.com>
	<CAGBU8t6J7DpKECL18-T6QMxrfsONio+h3wtm0d-E+FDFO-vJeQ@mail.gmail.com>
	<CAMSv6X3DSoBwpK3EXND=LK3-+gGEnWNmt+LzJf+yMBMVOEB05g@mail.gmail.com>
	<CAGBU8t5Vd_DX+38opPi7vmvO0cdOQvOqYDDb34a7QsXBwCxvUA@mail.gmail.com>
Message-ID: <CAMSv6X1z-BS42Hykr_rDWc7KJHcv-TewNTx3AdGJi0ciYF6nUA@mail.gmail.com>

Hi Jacob,

On Thu, Mar 22, 2012 at 03:57, Jacob Finkelman <eh2406 at wayne.edu> wrote:
> ??? C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(76) : error
> C2632: 'char' followed by 'char' is illegal

Sorry, I'm out of guesses.  You need to figure yourself what is wrong.
 Note that anyone with access to a Windows machine and knowledge of C
should be able to help you (which might include me, as I have
occasional access to a Windows machine, but don't count on it...).


A bient?t,

Armin.

From arigo at tunes.org  Thu Mar 22 11:18:26 2012
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 22 Mar 2012 11:18:26 +0100
Subject: [pypy-dev] Ask for a point to contribute to pypy
In-Reply-To: <CA+y=efvecSvAX6DEUQb1PfauiFzdbFUzhgELxfUY-U7P=VeBcQ@mail.gmail.com>
References: <CA+y=efvecSvAX6DEUQb1PfauiFzdbFUzhgELxfUY-U7P=VeBcQ@mail.gmail.com>
Message-ID: <CAMSv6X2dd9DYpPe8bdCSoKGRmzvLq9-AXQfiZ9nY_3K5J0-C2w@mail.gmail.com>

Hi Iven,

Sorry for the delay: PyCon U.S. happened.

On Tue, Mar 6, 2012 at 07:05, Iven Hsu <ivenvd at gmail.com> wrote:
> I'm familiar with Python and I have a little knowledge on compilers. I've
> read some code and documents of pypy and get the main idea of it.
>
> I've read the <Potential project list> in the wiki, but not sure if they are
> up-to-date. I hope you have some good suggestions.

I am bad at giving people suggestions for what to work on :-(
Moreover, nowadays it's a bit like asking "I want to do something in
Python, what do you suggest?".  What area of PyPy are you more
precisely interested in?  The Python interpreter (e.g. the object
model); or the port to Python 3; or compatibility with external
generators like Cython or SWIG; or tweaks to the JIT's optimizer or
the JIT's backend; or to the GC; or supporting Continulets in the JIT;
or contributing to the GCC plugin to replace the shadowstack; etc
etc... (sorry if I miss important topics).


A bient?t,

Armin.

From arigo at tunes.org  Thu Mar 22 11:22:29 2012
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 22 Mar 2012 11:22:29 +0100
Subject: [pypy-dev] translation crash on 64-bit
In-Reply-To: <CAPQx2vexfcv0wuC8ejn6jwch-0E0CTm2O6Ug4z7hXKDGeGhrQg@mail.gmail.com>
References: <CAPQx2veB+WvWjLcv7JA9+8Pd2PTRcquK6ojmSdXT51ARFucscA@mail.gmail.com>
	<CAK5idxRvYJ7aBWR+RqbbenGuJEzGzdG_E0NpQd-+6AHQyYKdyw@mail.gmail.com>
	<CAPQx2vdE6oQgFc-3D3YBC9tAHerzV+P6xPWLYfcBO2cFvCOFsQ@mail.gmail.com>
	<CAK5idxTCsm5Hzdk3-xewpyZ_YebF-CVDpJ+1VmB_n4UxdnRxYA@mail.gmail.com>
	<CAPQx2vexfcv0wuC8ejn6jwch-0E0CTm2O6Ug4z7hXKDGeGhrQg@mail.gmail.com>
Message-ID: <CAMSv6X1q3oDdryU7_BJRFY+_6A0TW9oAcUfmsx_MqM4KuysNag@mail.gmail.com>

Hi Buck,

Sorry, PyCon U.S. happened.

Can you tell us if the error is still relevant by trying to translate
a current trunk?


Armin

From amauryfa at gmail.com  Thu Mar 22 12:08:57 2012
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Thu, 22 Mar 2012 12:08:57 +0100
Subject: [pypy-dev] How to get PIL to install?
In-Reply-To: <CAMSv6X1z-BS42Hykr_rDWc7KJHcv-TewNTx3AdGJi0ciYF6nUA@mail.gmail.com>
References: <CAGBU8t6MDthpmuY2arM6_UM6Xm8DXiWkZ4mc61a_C+DfRx6=CA@mail.gmail.com>
	<CAMSv6X0uZRKfd2ov-jXoTH788uL8osLZ0BSbOPSfq4wQ+=6D+Q@mail.gmail.com>
	<CAGBU8t4XALkU4OYiFYqa1NP3iDFkVgeFTp8NYB=E9g+Zah-J=g@mail.gmail.com>
	<CAGBU8t6J7DpKECL18-T6QMxrfsONio+h3wtm0d-E+FDFO-vJeQ@mail.gmail.com>
	<CAMSv6X3DSoBwpK3EXND=LK3-+gGEnWNmt+LzJf+yMBMVOEB05g@mail.gmail.com>
	<CAGBU8t5Vd_DX+38opPi7vmvO0cdOQvOqYDDb34a7QsXBwCxvUA@mail.gmail.com>
	<CAMSv6X1z-BS42Hykr_rDWc7KJHcv-TewNTx3AdGJi0ciYF6nUA@mail.gmail.com>
Message-ID: <CAGmFidbydjOmQQJUaovVZZv1MVTbfEYPEY6R-SKRVnsfnxjwjg@mail.gmail.com>

Hi

2012/3/22 Armin Rigo <arigo at tunes.org>:
> On Thu, Mar 22, 2012 at 03:57, Jacob Finkelman <eh2406 at wayne.edu> wrote:
>> ??? C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(76) : error
>> C2632: 'char' followed by 'char' is illegal
>
> Sorry, I'm out of guesses. ?You need to figure yourself what is wrong.
> ?Note that anyone with access to a Windows machine and knowledge of C
> should be able to help you (which might include me, as I have
> occasional access to a Windows machine, but don't count on it...).

There seems to be an incompatibility between PIL and newest Microsoft SDKs.
Can you try to compile PIL with CPython?
I'm sure you'll get similar errors.

-- 
Amaury Forgeot d'Arc

From fijall at gmail.com  Thu Mar 22 14:38:55 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Thu, 22 Mar 2012 15:38:55 +0200
Subject: [pypy-dev] Ask for a point to contribute to pypy
In-Reply-To: <CAMSv6X2dd9DYpPe8bdCSoKGRmzvLq9-AXQfiZ9nY_3K5J0-C2w@mail.gmail.com>
References: <CA+y=efvecSvAX6DEUQb1PfauiFzdbFUzhgELxfUY-U7P=VeBcQ@mail.gmail.com>
	<CAMSv6X2dd9DYpPe8bdCSoKGRmzvLq9-AXQfiZ9nY_3K5J0-C2w@mail.gmail.com>
Message-ID: <CAK5idxTdyur=EAF3J_=7YMP54e4HhjuNGkRbC39Af11kzhnt4g@mail.gmail.com>

On Thu, Mar 22, 2012 at 12:18 PM, Armin Rigo <arigo at tunes.org> wrote:
> Hi Iven,
>
> Sorry for the delay: PyCon U.S. happened.
>
> On Tue, Mar 6, 2012 at 07:05, Iven Hsu <ivenvd at gmail.com> wrote:
>> I'm familiar with Python and I have a little knowledge on compilers. I've
>> read some code and documents of pypy and get the main idea of it.
>>
>> I've read the <Potential project list> in the wiki, but not sure if they are
>> up-to-date. I hope you have some good suggestions.
>
> I am bad at giving people suggestions for what to work on :-(
> Moreover, nowadays it's a bit like asking "I want to do something in
> Python, what do you suggest?". ?What area of PyPy are you more
> precisely interested in? ?The Python interpreter (e.g. the object
> model); or the port to Python 3; or compatibility with external
> generators like Cython or SWIG; or tweaks to the JIT's optimizer or
> the JIT's backend; or to the GC; or supporting Continulets in the JIT;
> or contributing to the GCC plugin to replace the shadowstack; etc
> etc... (sorry if I miss important topics).
>
>
> A bient?t,
>
> Armin.

Hey Armin

Since I hear this question again and again - would you be interested
in writing down the document about this? "you want to help with pypy,
here are the ares, here is more docs, here you ask"

Cheers,
fijal

From arigo at tunes.org  Thu Mar 22 16:59:20 2012
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 22 Mar 2012 16:59:20 +0100
Subject: [pypy-dev] Ask for a point to contribute to pypy
In-Reply-To: <CAK5idxTdyur=EAF3J_=7YMP54e4HhjuNGkRbC39Af11kzhnt4g@mail.gmail.com>
References: <CA+y=efvecSvAX6DEUQb1PfauiFzdbFUzhgELxfUY-U7P=VeBcQ@mail.gmail.com>
	<CAMSv6X2dd9DYpPe8bdCSoKGRmzvLq9-AXQfiZ9nY_3K5J0-C2w@mail.gmail.com>
	<CAK5idxTdyur=EAF3J_=7YMP54e4HhjuNGkRbC39Af11kzhnt4g@mail.gmail.com>
Message-ID: <CAMSv6X2PgbL72DnLHnxmRNuL7QevhUJX9-ciMqAXQOwqKGO9Mg@mail.gmail.com>

Hi Iven,

Note also that there is http://doc.pypy.org/en/latest/project-ideas.html .


A bient?t,

Armin.

From matti.picus at gmail.com  Thu Mar 22 17:00:53 2012
From: matti.picus at gmail.com (Matti Picus)
Date: Thu, 22 Mar 2012 18:00:53 +0200
Subject: [pypy-dev] How to get PIL to install?
In-Reply-To: <mailman.25.1332414002.12164.pypy-dev@python.org>
References: <mailman.25.1332414002.12164.pypy-dev@python.org>
Message-ID: <4F6B4CB5.4060503@gmail.com>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120322/f482f88c/attachment.html>

From chef at ghum.de  Thu Mar 22 17:06:36 2012
From: chef at ghum.de (Massa, Harald Armin)
Date: Thu, 22 Mar 2012 17:06:36 +0100
Subject: [pypy-dev] pypy on Windows
Message-ID: <CAMSP2L4hf8s4KC6gzJmTe+0toOj5gFu0H=F3oKcPZ2dbyeAxpw@mail.gmail.com>

Hello pypy-ers,

as a lurker on this mailing list I often read information as in "I do not
have a Windows development system" or similiar.

I know that Microsoft a) is a PSF member and b) was giving
MSDN-Subscriptions to Python Developers and c) also has its own cloud
called azure  d) has an rather active internal Python community

So, was Microsoft allready asked along the lines of "would you be so kind
to provide us with a cloudycloud running Visual C compilers, can pypy has
cheesburger ?"

Or is that be a good thing that is still to do?

Harald
-- 

GHUM GmbH
Harald Armin Massa
Spielberger Stra?e 49
70435 Stuttgart
0173/9409607

Amtsgericht Stuttgart, HRB 734971
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120322/9a8c452b/attachment.html>

From fijall at gmail.com  Thu Mar 22 17:11:34 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Thu, 22 Mar 2012 18:11:34 +0200
Subject: [pypy-dev] How to get PIL to install?
In-Reply-To: <4F6B4CB5.4060503@gmail.com>
References: <mailman.25.1332414002.12164.pypy-dev@python.org>
	<4F6B4CB5.4060503@gmail.com>
Message-ID: <CAK5idxRPcjCah_pQfMVSxeRF-e6wP78q0xTjzXH7GEVyNh+TzQ@mail.gmail.com>

On Thu, Mar 22, 2012 at 6:00 PM, Matti Picus <matti.picus at gmail.com> wrote:
> On 22/03/2012 1:00 PM, pypy-dev-request at python.org wrote:
>
> Date: Thu, 22 Mar 2012 11:08:18 +0100
> From: Armin Rigo <arigo at tunes.org>
> To: Jacob Finkelman <eh2406 at wayne.edu>
> Cc: pypy-dev at python.org
> Subject: Re: [pypy-dev] How to get PIL to install?
> Message-ID:
> 	<CAMSv6X1z-BS42Hykr_rDWc7KJHcv-TewNTx3AdGJi0ciYF6nUA at mail.gmail.com>
> Content-Type: text/plain; charset=ISO-8859-1
>
> Hi Jacob,
>
> On Thu, Mar 22, 2012 at 03:57, Jacob Finkelman <eh2406 at wayne.edu> wrote:
>
> ??? C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\basetsd.h(76) :
> error
> C2632: 'char' followed by 'char' is illegal
>
>
> Hi. Sorry for continuing an off-topic thread but my typing fingers are not
> obeying my brain.
>
> I submitted this changeset to the pillow folks, hopefully they will merge
> and release a new version soonish.
>
> https://github.com/mattip/Pillow/commit/2ba3bf681dc2f7c476a4b674f01e499ec654f68c
>
> Feel free to contact me directly for any further build problems.
>
> Matti

Thanks matti!

From fijall at gmail.com  Thu Mar 22 17:14:23 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Thu, 22 Mar 2012 18:14:23 +0200
Subject: [pypy-dev] pypy on Windows
In-Reply-To: <CAMSP2L4hf8s4KC6gzJmTe+0toOj5gFu0H=F3oKcPZ2dbyeAxpw@mail.gmail.com>
References: <CAMSP2L4hf8s4KC6gzJmTe+0toOj5gFu0H=F3oKcPZ2dbyeAxpw@mail.gmail.com>
Message-ID: <CAK5idxTpNXLoA6yaWEtk25EPUtZ2r3v87oheRw032MoXrQuiHA@mail.gmail.com>

On Thu, Mar 22, 2012 at 6:06 PM, Massa, Harald Armin <chef at ghum.de> wrote:
> Hello pypy-ers,
>
> as a lurker on this mailing list I often read information as in "I do not
> have a Windows development system" or similiar.
>
> I know that Microsoft a) is a PSF member and b) was giving
> MSDN-Subscriptions to Python Developers and c) also has its own cloud called
> azure ?d) has an rather active internal Python community
>
> So, was Microsoft allready asked along the lines of "would you be so kind to
> provide us with a cloudycloud running Visual C compilers, can pypy has
> cheesburger ?"
>
> Or is that be a good thing that is still to do?
>
> Harald

That's useful, but not enough :-) They're not willing to donate a
windows developer as far as I remember.

Cheers,
fijal

From fijall at gmail.com  Thu Mar 22 17:27:40 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Thu, 22 Mar 2012 18:27:40 +0200
Subject: [pypy-dev] pypy on Windows
In-Reply-To: <1332433385.6279.14.camel@launcelot.winder.org.uk>
References: <CAMSP2L4hf8s4KC6gzJmTe+0toOj5gFu0H=F3oKcPZ2dbyeAxpw@mail.gmail.com>
	<CAK5idxTpNXLoA6yaWEtk25EPUtZ2r3v87oheRw032MoXrQuiHA@mail.gmail.com>
	<1332433385.6279.14.camel@launcelot.winder.org.uk>
Message-ID: <CAK5idxR+vGqFQLWCJ1CTPFpk+3CKrqBYfih0zYWkW9Qs4Bhbuw@mail.gmail.com>

On Thu, Mar 22, 2012 at 6:23 PM, Russel Winder <russel at winder.org.uk> wrote:
> On Thu, 2012-03-22 at 18:14 +0200, Maciej Fijalkowski wrote:
> [...]
>> That's useful, but not enough :-) They're not willing to donate a
>> windows developer as far as I remember.
>
> Any one of the many investment banks in London currently tooling up with
> Python could easily afford ?50,000 to fund the work to ensure Windows
> compatibility. ?The issue is getting them to see PyPy as strategic in
> replacement of CPython.
>
> --
> Russel.

As far as I know, that's relatively hard (convincing, not implementing).

Cheers,
fijal

From russel at winder.org.uk  Thu Mar 22 17:23:05 2012
From: russel at winder.org.uk (Russel Winder)
Date: Thu, 22 Mar 2012 16:23:05 +0000
Subject: [pypy-dev] pypy on Windows
In-Reply-To: <CAK5idxTpNXLoA6yaWEtk25EPUtZ2r3v87oheRw032MoXrQuiHA@mail.gmail.com>
References: <CAMSP2L4hf8s4KC6gzJmTe+0toOj5gFu0H=F3oKcPZ2dbyeAxpw@mail.gmail.com>
	<CAK5idxTpNXLoA6yaWEtk25EPUtZ2r3v87oheRw032MoXrQuiHA@mail.gmail.com>
Message-ID: <1332433385.6279.14.camel@launcelot.winder.org.uk>

On Thu, 2012-03-22 at 18:14 +0200, Maciej Fijalkowski wrote:
[...]
> That's useful, but not enough :-) They're not willing to donate a
> windows developer as far as I remember.

Any one of the many investment banks in London currently tooling up with
Python could easily afford ?50,000 to fund the work to ensure Windows
compatibility.  The issue is getting them to see PyPy as strategic in
replacement of CPython.

-- 
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder at ekiga.net
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel at winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120322/874feb30/attachment.pgp>

From workitharder at gmail.com  Thu Mar 22 18:21:45 2012
From: workitharder at gmail.com (Buck Golemon)
Date: Thu, 22 Mar 2012 10:21:45 -0700
Subject: [pypy-dev] translation crash on 64-bit
In-Reply-To: <CAMSv6X1q3oDdryU7_BJRFY+_6A0TW9oAcUfmsx_MqM4KuysNag@mail.gmail.com>
References: <CAPQx2veB+WvWjLcv7JA9+8Pd2PTRcquK6ojmSdXT51ARFucscA@mail.gmail.com>
	<CAK5idxRvYJ7aBWR+RqbbenGuJEzGzdG_E0NpQd-+6AHQyYKdyw@mail.gmail.com>
	<CAPQx2vdE6oQgFc-3D3YBC9tAHerzV+P6xPWLYfcBO2cFvCOFsQ@mail.gmail.com>
	<CAK5idxTCsm5Hzdk3-xewpyZ_YebF-CVDpJ+1VmB_n4UxdnRxYA@mail.gmail.com>
	<CAPQx2vexfcv0wuC8ejn6jwch-0E0CTm2O6Ug4z7hXKDGeGhrQg@mail.gmail.com>
	<CAMSv6X1q3oDdryU7_BJRFY+_6A0TW9oAcUfmsx_MqM4KuysNag@mail.gmail.com>
Message-ID: <CAPQx2veKAJ9VjojO+CHkkTxw4uvVR+6QcCUvWZPC2COAxbgE+A@mail.gmail.com>

I'm not able to reproduce it anymore.
Thanks.

On Thu, Mar 22, 2012 at 3:22 AM, Armin Rigo <arigo at tunes.org> wrote:

> Hi Buck,
>
> Sorry, PyCon U.S. happened.
>
> Can you tell us if the error is still relevant by trying to translate
> a current trunk?
>
>
> Armin
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120322/ba40e135/attachment.html>

From brian at python.org  Thu Mar 22 19:01:28 2012
From: brian at python.org (Brian Curtin)
Date: Thu, 22 Mar 2012 13:01:28 -0500
Subject: [pypy-dev] pypy on Windows
In-Reply-To: <CAMSP2L4hf8s4KC6gzJmTe+0toOj5gFu0H=F3oKcPZ2dbyeAxpw@mail.gmail.com>
References: <CAMSP2L4hf8s4KC6gzJmTe+0toOj5gFu0H=F3oKcPZ2dbyeAxpw@mail.gmail.com>
Message-ID: <CAD+XWwqeNeiBAt+-k_N0pwU7mKfvfLiEu3cM=G0yXfJAm7G_cw@mail.gmail.com>

On Thu, Mar 22, 2012 at 11:06, Massa, Harald Armin <chef at ghum.de> wrote:
> Hello pypy-ers,
>
> as a lurker on this mailing list I often read information as in "I do not
> have a Windows development system" or similiar.
>
> I know that Microsoft a) is a PSF member and b) was giving
> MSDN-Subscriptions to Python Developers and c) also has its own cloud called
> azure ?d) has an rather active internal Python community
>
> So, was Microsoft allready asked along the lines of "would you be so kind to
> provide us with a cloudycloud running Visual C compilers, can pypy has
> cheesburger ?"
>
> Or is that be a good thing that is still to do?

It's really more of an issue of personnel, as Maciej points out. I
have a contact at Microsoft (through CPython) who is willing to help
us get what we need to support Python on Windows, whether it's tools
or help solving problems on the platform. If you solve the problem of
attracting more Windows developers, I could help facilitate discussion
with Microsoft to get something like Azure setup.

From fijall at gmail.com  Thu Mar 22 19:22:51 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Thu, 22 Mar 2012 20:22:51 +0200
Subject: [pypy-dev] pypy on Windows
In-Reply-To: <CAD+XWwqeNeiBAt+-k_N0pwU7mKfvfLiEu3cM=G0yXfJAm7G_cw@mail.gmail.com>
References: <CAMSP2L4hf8s4KC6gzJmTe+0toOj5gFu0H=F3oKcPZ2dbyeAxpw@mail.gmail.com>
	<CAD+XWwqeNeiBAt+-k_N0pwU7mKfvfLiEu3cM=G0yXfJAm7G_cw@mail.gmail.com>
Message-ID: <CAK5idxT=VN-RFMjT-mfd37z9CzVyhs-rNAJa=HGKfiuM1B0wpA@mail.gmail.com>

On Thu, Mar 22, 2012 at 8:01 PM, Brian Curtin <brian at python.org> wrote:
> On Thu, Mar 22, 2012 at 11:06, Massa, Harald Armin <chef at ghum.de> wrote:
>> Hello pypy-ers,
>>
>> as a lurker on this mailing list I often read information as in "I do not
>> have a Windows development system" or similiar.
>>
>> I know that Microsoft a) is a PSF member and b) was giving
>> MSDN-Subscriptions to Python Developers and c) also has its own cloud called
>> azure ?d) has an rather active internal Python community
>>
>> So, was Microsoft allready asked along the lines of "would you be so kind to
>> provide us with a cloudycloud running Visual C compilers, can pypy has
>> cheesburger ?"
>>
>> Or is that be a good thing that is still to do?
>
> It's really more of an issue of personnel, as Maciej points out. I
> have a contact at Microsoft (through CPython) who is willing to help
> us get what we need to support Python on Windows, whether it's tools
> or help solving problems on the platform. If you solve the problem of
> attracting more Windows developers, I could help facilitate discussion
> with Microsoft to get something like Azure setup.

Well, it's about the volunteer personnel. I think we can find people
who would be willing to do it for money.

Cheers,
fijal

From lwallace890 at gmail.com  Fri Mar 23 00:13:06 2012
From: lwallace890 at gmail.com (Lynne Wallace)
Date: Thu, 22 Mar 2012 23:13:06 +0000
Subject: [pypy-dev] GSOC Ideas
Message-ID: <C8ABB3EB-738C-422C-ACE3-F856AA487265@gmail.com>

Hey everyone,

I'm a 3rd year student at ed.ac.uk, interested in getting involved in pypy through google summer of code.

I've been chatting to Alex Gaynor over irc - his primary suggestion was improving the register allocator.
He also suggested browsing through https://bitbucket.org/pypy/extradoc/src/tip/planning/jit.txt to get 
an idea for projects, but it is a little hard to judge the size and complexity of some of the tasks without
a little more internal knowledge.

I was wondering if anyone else had ideas or suggestions for gsoc projects?

Thanks
Lynne
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120322/b3f589c7/attachment-0001.html>

From ivenvd at gmail.com  Fri Mar 23 03:33:17 2012
From: ivenvd at gmail.com (Iven Hsu)
Date: Fri, 23 Mar 2012 10:33:17 +0800
Subject: [pypy-dev] Ask for a point to contribute to pypy
In-Reply-To: <CAMSv6X2PgbL72DnLHnxmRNuL7QevhUJX9-ciMqAXQOwqKGO9Mg@mail.gmail.com>
References: <CA+y=efvecSvAX6DEUQb1PfauiFzdbFUzhgELxfUY-U7P=VeBcQ@mail.gmail.com>
	<CAMSv6X2dd9DYpPe8bdCSoKGRmzvLq9-AXQfiZ9nY_3K5J0-C2w@mail.gmail.com>
	<CAK5idxTdyur=EAF3J_=7YMP54e4HhjuNGkRbC39Af11kzhnt4g@mail.gmail.com>
	<CAMSv6X2PgbL72DnLHnxmRNuL7QevhUJX9-ciMqAXQOwqKGO9Mg@mail.gmail.com>
Message-ID: <CA+y=efuZq1sGMAUJibvBFhje8X+mZDYeuCfR8LJ4JMQiqCiy0A@mail.gmail.com>

Hi Armin,

Thanks for your replying. I've decided to try Embedding PyPy, and I'll
connect you if I meet any problems.

Thanks,
Iven

? 2012?3?22? ??11:59?Armin Rigo <arigo at tunes.org>???

> Hi Iven,
>
> Note also that there is http://doc.pypy.org/en/latest/project-ideas.html .
>
>
> A bient?t,
>
> Armin.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120323/34e8f062/attachment.html>

From arigo at tunes.org  Fri Mar 23 07:55:17 2012
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 23 Mar 2012 07:55:17 +0100
Subject: [pypy-dev] GSOC Ideas
In-Reply-To: <C8ABB3EB-738C-422C-ACE3-F856AA487265@gmail.com>
References: <C8ABB3EB-738C-422C-ACE3-F856AA487265@gmail.com>
Message-ID: <CAMSv6X2GCPAbQjSL11BaGN8dbQP4BuMrQZmDQvYpLALThiFrAw@mail.gmail.com>

Hi Lynne,

On Fri, Mar 23, 2012 at 00:13, Lynne Wallace <lwallace890 at gmail.com> wrote:
> I was wondering if?anyone else had ideas or suggestions for gsoc projects?

We have a list of projects (which might or might not fit a GSoC size):
http://doc.pypy.org/en/latest/project-ideas.html


A bient?t,

Armin.

From arigo at tunes.org  Fri Mar 23 08:15:49 2012
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 23 Mar 2012 08:15:49 +0100
Subject: [pypy-dev] towards more parallelization in the tracer/optimizer
In-Reply-To: <4F6260BF.1010205@wakelift.de>
References: <4F6260BF.1010205@wakelift.de>
Message-ID: <CAMSv6X1eeexcZjKhMG48SF-WTgCk9-bM1HF3UxL1bHo4hEzM2A@mail.gmail.com>

Hi Timo,

On Thu, Mar 15, 2012 at 22:35, Timo Paulssen <timo at wakelift.de> wrote:
> If the raw trace - or a minimally optimized version of it - can be run
> directly, why not run it once or thrice while waiting for the optimizer to
> finish optimizing the trace fully?

Yes, it's something we already thought of, but there are a lot of
issues along the way.  One is that tracing itself takes time too (not
just optimizing) and that cannot be done in parallel.  Also, there are
harder issues about tweaking the GC: right now it "parallelizes"
trivially because it assumes the GIL, but that's no longer true if we
want to run the optimizer-and-backend part truly in parallel.

Note that the speed differences are larger than you seem to assume: in
very very rough orders of magnitude, if the interpreter takes 1 unit
of time to run one iteration of a loop, and the JITted trace takes
0.1, then I think that tracing takes 100 or 1000; and optimizing too.

But maybe more importantly, there is the following issue.  Remember
that typically, in real-life cases, we need several tracings to make
one loop performant, each one tracing a different path, until all
common paths are covered.  As long as not all common paths are traced,
running the partial trace is really slow: every time it hits a
not-yet-compiled branch, we need to fall back to the interpreter.  So
it's unclear that we would get an overall speed-up over today's
situation.


A bient?t,

Armin.

From fijall at gmail.com  Fri Mar 23 09:17:28 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Fri, 23 Mar 2012 10:17:28 +0200
Subject: [pypy-dev] Ask for a point to contribute to pypy
In-Reply-To: <CA+y=efuZq1sGMAUJibvBFhje8X+mZDYeuCfR8LJ4JMQiqCiy0A@mail.gmail.com>
References: <CA+y=efvecSvAX6DEUQb1PfauiFzdbFUzhgELxfUY-U7P=VeBcQ@mail.gmail.com>
	<CAMSv6X2dd9DYpPe8bdCSoKGRmzvLq9-AXQfiZ9nY_3K5J0-C2w@mail.gmail.com>
	<CAK5idxTdyur=EAF3J_=7YMP54e4HhjuNGkRbC39Af11kzhnt4g@mail.gmail.com>
	<CAMSv6X2PgbL72DnLHnxmRNuL7QevhUJX9-ciMqAXQOwqKGO9Mg@mail.gmail.com>
	<CA+y=efuZq1sGMAUJibvBFhje8X+mZDYeuCfR8LJ4JMQiqCiy0A@mail.gmail.com>
Message-ID: <CAK5idxTmTGFtd7oQ=BV4FfbkWL=vKWUKOyDKBnwbJLzEU0B4CA@mail.gmail.com>

On Fri, Mar 23, 2012 at 4:33 AM, Iven Hsu <ivenvd at gmail.com> wrote:
> Hi Armin,
>
> Thanks for your replying. I've decided to try Embedding PyPy, and I'll
> connect you if I meet any problems.
>
> Thanks,
> Iven

We're generally very IRC oriented, so feel free to hang out on #pypy
on freenode and ask questions.

Cheers,
fijal

From arigo at tunes.org  Mon Mar 26 12:53:27 2012
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 26 Mar 2012 12:53:27 +0200
Subject: [pypy-dev] STM proposal funding
Message-ID: <CAMSv6X2GF=1HaZ1wq7OfS_c-DiLO-zC1Sm0hUWKuSG4QXhTw+w@mail.gmail.com>

Hi all,

For those of you who missed it: the proposal on Software Transactional
Memory is launched ("remove the GIL but give a better way to write
multi-threaded applications").  It is available at
http://pypy.org/tmdonate.html .

I plan to ask the PSF for some participation, following the push
towards doing so at the PyCon PSF meeting.  See
http://mail.python.org/pipermail/python-dev/2012-March/117680.html ,
"Funding from the Python Software Foundation".


A bient?t,

Armin.

From andrewfr_ice at yahoo.com  Mon Mar 26 17:28:58 2012
From: andrewfr_ice at yahoo.com (Andrew Francis)
Date: Mon, 26 Mar 2012 08:28:58 -0700 (PDT)
Subject: [pypy-dev] The Work Plan Re:  STM proposal funding
In-Reply-To: <CAMSv6X2GF=1HaZ1wq7OfS_c-DiLO-zC1Sm0hUWKuSG4QXhTw+w@mail.gmail.com>
References: <CAMSv6X2GF=1HaZ1wq7OfS_c-DiLO-zC1Sm0hUWKuSG4QXhTw+w@mail.gmail.com>
Message-ID: <1332775738.67346.YahooMailNeo@web120702.mail.ne1.yahoo.com>

Hi Armin:

I looked over the work plan. Cool! Some questions and comments.

1. rstm library:

I have looked at the rstm library. Wrote the "hello world" of transactional programming: the bank account example. . I also played with your STM stuff a few months ago.? A little while back, I started to go through your code to see what strategies you are using and the general architecture (hence the last batch of questions). I didn't get far but I want to resume real soon. If folks can tolerate scanned hand drawings (I am terrible with illustration software), I could post what I do. And the references I use will looking at the code ( i.e., "Software Transactional Memory 2nd by Tim Harris et el).? I don't know if this is helpful?


User Interface:

I am familar with the AME papers. AME literature seems to taper off. Have you tried talking to the various authors? Or get your hands on an implementation. For join papers, I talked to the Microsoft researcher involved and he was helpful.

To further understand what you are doing and get a feel for? STM would look like, I wanted to Python versions of the STAMP examples. Perhaps this would be a good way to craft an API in parallel with other work?


Half-Baked Ideas:

The reason I encountered STM literature is because I used stackless.py to prototype join patterns. One of these days, I'll finish that work and post it. After talking to the authors of "Scalable Join Patterns," I was referred to the work of John Reppy on Concurrent and Parallel ML.
Polyphonic C# has something that operates like channels. And Concurrent ML channels are very much like Stackless Python channels in the sense they are synchronous.

The tie-in to STM is both those implementations under the hood use a combination of STM and lock-free algorithms for efficient implementation. 

Now what makes this attractive for say,? a future Stackless Python is that message passing model usually lends itself well to a style where the programmer naturally knows how to partition their data amongst coroutines.

STM is used under the hood to implement the channels and other associated constructs. Hopefully this would 1) lead to a smaller transactional footprint. 2) Totally hide STM from the programmer. 3) Use an already existing API.

Thoughts?

Cheers,
Andrew



















________________________________
 From: Armin Rigo <arigo at tunes.org>
To: PyPy Developer Mailing List <pypy-dev at python.org> 
Sent: Monday, March 26, 2012 6:53 AM
Subject: [pypy-dev] STM proposal funding
 
Hi all,

For those of you who missed it: the proposal on Software Transactional
Memory is launched ("remove the GIL but give a better way to write
multi-threaded applications").? It is available at
http://pypy.org/tmdonate.html .

I plan to ask the PSF for some participation, following the push
towards doing so at the PyCon PSF meeting.? See
http://mail.python.org/pipermail/python-dev/2012-March/117680.html ,
"Funding from the Python Software Foundation".


A bient?t,

Armin.
_______________________________________________
pypy-dev mailing list
pypy-dev at python.org
http://mail.python.org/mailman/listinfo/pypy-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120326/737bbdec/attachment.html>

From pauljeffers14 at gmail.com  Mon Mar 26 18:46:23 2012
From: pauljeffers14 at gmail.com (Paul Jeffers)
Date: Mon, 26 Mar 2012 18:46:23 +0200 (CEST)
Subject: [pypy-dev] Job Opening !
Message-ID: <20120326164729.4CB49A264F7@chicago155.server4you.de>


Would you like to make an extra income legitimately as a Feedback Specialist? No Application Fee Required. Contact us for more details. Many thanks,




From roberto at unbit.it  Tue Mar 27 14:16:18 2012
From: roberto at unbit.it (Roberto De Ioris)
Date: Tue, 27 Mar 2012 14:16:18 +0200
Subject: [pypy-dev] update (+patch) on embedding pypy
Message-ID: <0255399bee51bd1c3df0a091a91f8cf3.squirrel@manage.unbit.it>

Hi everyone, i have finally managed to have a pypy plugin into uWSGI via
libpypy-c.

The first step was exporting a bunch of symbols in libpypy-c:

"RPython_StartupCode", "pypy_g_ObjSpace_startup", "pypy_g_call_startup",
"pypy_g_State_startup", "pypy_g_Module_startup",
"pypy_g_pypy_module_cpyext_state_State"

pypy_g_ObjSpace_startup and pypy_g_Module_startup are not strictly
required, i have used them only for debugging some very specific areas.

Attached there is a patch for latest tip.

After that, i have defined a Py_Initialize() replacement:

http://projects.unbit.it/uwsgi/browser/plugins/python/python_plugin.c#L8

and placed a bunch of #ifdef for unsupported functions
(like PyFile_FromFile)

Finally i have built libpypy with

pypy translate.py -Ojit --shared

and the uWSGI plugin with

LDFLAGS="-L." pypy-c uwsgiconfig.py --plugin plugins/pypy core

(LDFLAGS is used as i have libpypy-c in the current dir)

Run uWSGI with the welcome app (and a bunch of processes):

uwsgi --http-socket :8080 --plugin pypy --module welcome --master
--processes 8

Incredibly the whole uwsgi api (that it is pretty complex, involving tons
of c code) works flawlessly as well as the custom wsgi.input object (the
whole upload test suite passes).

Running python threads from the WSGI app works too, but mapping python
threads to already available pthreads is a no go (all explodes, i will
investigate on that)

Importing applications as modules works normally, but importing via file
requires implementing a bunch of c-api in pypy (i will address that soon)

virtualenv, sys.argv and setting programname is not supported (no c api
available) i can get rid of this too in the next few days.

Important thing: i have tested it only on OSX

-- 
Roberto De Ioris
http://unbit.it
-------------- next part --------------
A non-text attachment was scrubbed...
Name: libpypy_export.patch
Type: application/octet-stream
Size: 2181 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120327/0a1dd436/attachment.obj>

From p.j.a.cock at googlemail.com  Tue Mar 27 15:27:41 2012
From: p.j.a.cock at googlemail.com (Peter Cock)
Date: Tue, 27 Mar 2012 14:27:41 +0100
Subject: [pypy-dev] Speeding up zlib in standard library
In-Reply-To: <CAK5idxS7xO=jQoW27fTBwSdn=2HjX_oHOVQHfNgaOZsk=Ado3w@mail.gmail.com>
References: <CAKVJ-_5D4OUBSkavYqW7usBkaZMxu-EjEtb9NaKU-iCMA=XZQg@mail.gmail.com>
	<CAPZV6o8fpWjG2d8L6ig4SQds=EAUnb=sypeor9HzazdmXyZy=g@mail.gmail.com>
	<CAKVJ-_4zry6WMzg7Gfu2zUj=t-fpce_givh4TFj57aHiagwQ+Q@mail.gmail.com>
	<CAMSv6X1vXnT6jRs2MHvwa613nfZfrqgt9sEVbB0tk6tX5+Ggew@mail.gmail.com>
	<CAKVJ-_5jtHOxXpcnUVojV-a070BWV8fybZ99EKaZqjT9+DoLJA@mail.gmail.com>
	<CA+=c-xbzg8k3M_d6N=JsV1aaHRHv-2P1Cm-cSSPUdUh6jYJsnA@mail.gmail.com>
	<CAKVJ-_6Ov+MwjHginrmpyga2TLWZkxP9mFSaSZcUYToNDRkm0g@mail.gmail.com>
	<CAK5idxTbhQi9X7O+M2eJy-nFY6BWiFmTdghhfohjNv0y8_Yd1A@mail.gmail.com>
	<CAKVJ-_5Zd_BXHa1ycDGNr3SS8txKF1wzKgZ7QydYkpQeqEXt3w@mail.gmail.com>
	<CAK5idxRrznx6ifCPD3Y1p8XHKYOwO+RmqoA=rdTKeJkhvpuSsA@mail.gmail.com>
	<CAKVJ-_4=xyR3qGS-ZdcCwGXfxphLwD5Ezgq48f=7ydzDp1A4+A@mail.gmail.com>
	<CAK5idxQpU61njri3PES+1cCeM1ddLFf=B1vFozBmKh7Pxh=HDg@mail.gmail.com>
	<CAKVJ-_5-CmM492A-qaMOD2r5Sz6cHhV2Gqz64o_gk2qExyEcng@mail.gmail.com>
	<CAK5idxS7xO=jQoW27fTBwSdn=2HjX_oHOVQHfNgaOZsk=Ado3w@mail.gmail.com>
Message-ID: <CAKVJ-_4wZo37MY_AO_fEWWXyg=RYm6eEJ-QunVE9wKKq1hAVqQ@mail.gmail.com>

On Wed, Mar 21, 2012 at 5:49 PM, Maciej Fijalkowski <fijall at gmail.com> wrote:
> On Wed, Mar 21, 2012 at 7:10 PM, Peter Cock <p.j.a.cock at googlemail.com> wrote:
>> That doesn't sound quite so intimidating... and if I've understood this
>> now it does seem like the basis of a good development FAQ entry:
>> http://doc.pypy.org/en/latest/faq.html#development
>>
>> Then if the benchmark results are encouraging, in principle I could
>> then recompile the whole of pypy (which is slow), and then go back
>> to run the patched pypy on my real script to see what difference it
>> makes, if any.
>>
>> Right?
>>
>> Thanks for bearing with me,
>>
>> Peter
>
> Right, you got it perfect :)
>
> Can you suggest the wording for a FAQ entry?
>
> Cheers,
> fijal

Something like this? Peter.

Q: How can I test and benchmark a modification to PyPy, for
example in a StringBuilder method?

A: Based on the existing examples, create a new StringBuilder
benchmark as the file pypy/translator/targetStringBuilder.py which
will verify the functionality and/or time it, and do this:

$ cd pypy/translator/goal/
$ python translate.py targetStringBuilder.py
$ ./targetStringBuilder-c

Then make your modifications to pypy/rpython/lltypesystem/rbuilder.py
(or whatever you are working on) and repeat this. That should compare
the benchmark translation by the virgin PyPy against the translation
by your modified PyPy. Once you have a potentially useful improvement,
you should run the full test suite to ensure there are no side effects.

Q: How do I run PyPy's unit tests?

A: ...

From fijall at gmail.com  Tue Mar 27 15:30:53 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Tue, 27 Mar 2012 15:30:53 +0200
Subject: [pypy-dev] Speeding up zlib in standard library
In-Reply-To: <CAKVJ-_4wZo37MY_AO_fEWWXyg=RYm6eEJ-QunVE9wKKq1hAVqQ@mail.gmail.com>
References: <CAKVJ-_5D4OUBSkavYqW7usBkaZMxu-EjEtb9NaKU-iCMA=XZQg@mail.gmail.com>
	<CAPZV6o8fpWjG2d8L6ig4SQds=EAUnb=sypeor9HzazdmXyZy=g@mail.gmail.com>
	<CAKVJ-_4zry6WMzg7Gfu2zUj=t-fpce_givh4TFj57aHiagwQ+Q@mail.gmail.com>
	<CAMSv6X1vXnT6jRs2MHvwa613nfZfrqgt9sEVbB0tk6tX5+Ggew@mail.gmail.com>
	<CAKVJ-_5jtHOxXpcnUVojV-a070BWV8fybZ99EKaZqjT9+DoLJA@mail.gmail.com>
	<CA+=c-xbzg8k3M_d6N=JsV1aaHRHv-2P1Cm-cSSPUdUh6jYJsnA@mail.gmail.com>
	<CAKVJ-_6Ov+MwjHginrmpyga2TLWZkxP9mFSaSZcUYToNDRkm0g@mail.gmail.com>
	<CAK5idxTbhQi9X7O+M2eJy-nFY6BWiFmTdghhfohjNv0y8_Yd1A@mail.gmail.com>
	<CAKVJ-_5Zd_BXHa1ycDGNr3SS8txKF1wzKgZ7QydYkpQeqEXt3w@mail.gmail.com>
	<CAK5idxRrznx6ifCPD3Y1p8XHKYOwO+RmqoA=rdTKeJkhvpuSsA@mail.gmail.com>
	<CAKVJ-_4=xyR3qGS-ZdcCwGXfxphLwD5Ezgq48f=7ydzDp1A4+A@mail.gmail.com>
	<CAK5idxQpU61njri3PES+1cCeM1ddLFf=B1vFozBmKh7Pxh=HDg@mail.gmail.com>
	<CAKVJ-_5-CmM492A-qaMOD2r5Sz6cHhV2Gqz64o_gk2qExyEcng@mail.gmail.com>
	<CAK5idxS7xO=jQoW27fTBwSdn=2HjX_oHOVQHfNgaOZsk=Ado3w@mail.gmail.com>
	<CAKVJ-_4wZo37MY_AO_fEWWXyg=RYm6eEJ-QunVE9wKKq1hAVqQ@mail.gmail.com>
Message-ID: <CAK5idxRr=0=VrHUK39MvftGvZvP+hUhS+Ei-zY3xWT7rpS+Kpw@mail.gmail.com>

On Tue, Mar 27, 2012 at 3:27 PM, Peter Cock <p.j.a.cock at googlemail.com> wrote:
> On Wed, Mar 21, 2012 at 5:49 PM, Maciej Fijalkowski <fijall at gmail.com> wrote:
>> On Wed, Mar 21, 2012 at 7:10 PM, Peter Cock <p.j.a.cock at googlemail.com> wrote:
>>> That doesn't sound quite so intimidating... and if I've understood this
>>> now it does seem like the basis of a good development FAQ entry:
>>> http://doc.pypy.org/en/latest/faq.html#development
>>>
>>> Then if the benchmark results are encouraging, in principle I could
>>> then recompile the whole of pypy (which is slow), and then go back
>>> to run the patched pypy on my real script to see what difference it
>>> makes, if any.
>>>
>>> Right?
>>>
>>> Thanks for bearing with me,
>>>
>>> Peter
>>
>> Right, you got it perfect :)
>>
>> Can you suggest the wording for a FAQ entry?
>>
>> Cheers,
>> fijal
>
> Something like this? Peter.
>
> Q: How can I test and benchmark a modification to PyPy, for
> example in a StringBuilder method?
>
> A: Based on the existing examples, create a new StringBuilder
> benchmark as the file pypy/translator/targetStringBuilder.py which
> will verify the functionality and/or time it, and do this:
>
> $ cd pypy/translator/goal/
> $ python translate.py targetStringBuilder.py
> $ ./targetStringBuilder-c
>
> Then make your modifications to pypy/rpython/lltypesystem/rbuilder.py
> (or whatever you are working on) and repeat this. That should compare
> the benchmark translation by the virgin PyPy against the translation
> by your modified PyPy. Once you have a potentially useful improvement,
> you should run the full test suite to ensure there are no side effects.
>
> Q: How do I run PyPy's unit tests?
>
> A: ...

This sounds overly specific to me. What do others think?

From anto.cuni at gmail.com  Tue Mar 27 21:36:26 2012
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Tue, 27 Mar 2012 21:36:26 +0200
Subject: [pypy-dev] [pypy-commit] pypy default: Python 2.5 compat
In-Reply-To: <20120326113150.07C14820D9@wyvern.cs.uni-duesseldorf.de>
References: <20120326113150.07C14820D9@wyvern.cs.uni-duesseldorf.de>
Message-ID: <4F7216BA.1030805@gmail.com>

Hi Armin,

On 03/26/2012 01:31 PM, arigo wrote:
> Author: Armin Rigo <arigo at tunes.org>
> Branch: 
> Changeset: r53982:db33a634cb09
> Date: 2012-03-26 13:16 +0200
> http://bitbucket.org/pypy/pypy/changeset/db33a634cb09/
> 
> Log:	Python 2.5 compat

do we really want to keep python 2.5 compat? It's not tested, and "what is not
tested is broken"...

ciao,
Anto

From fijall at gmail.com  Tue Mar 27 21:55:31 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Tue, 27 Mar 2012 21:55:31 +0200
Subject: [pypy-dev] [pypy-commit] pypy default: Python 2.5 compat
In-Reply-To: <4F7216BA.1030805@gmail.com>
References: <20120326113150.07C14820D9@wyvern.cs.uni-duesseldorf.de>
	<4F7216BA.1030805@gmail.com>
Message-ID: <CAK5idxRAWMUuDcdcLhd5sTWHx5nag_rhn2J7tnpAHsZUuyTskw@mail.gmail.com>

On Tue, Mar 27, 2012 at 9:36 PM, Antonio Cuni <anto.cuni at gmail.com> wrote:
> Hi Armin,
>
> On 03/26/2012 01:31 PM, arigo wrote:
>> Author: Armin Rigo <arigo at tunes.org>
>> Branch:
>> Changeset: r53982:db33a634cb09
>> Date: 2012-03-26 13:16 +0200
>> http://bitbucket.org/pypy/pypy/changeset/db33a634cb09/
>>
>> Log: ?Python 2.5 compat
>
> do we really want to keep python 2.5 compat? It's not tested, and "what is not
> tested is broken"...
>
> ciao,
> Anto
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev

There is no reason not to.

From benjamin at python.org  Tue Mar 27 22:05:06 2012
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 27 Mar 2012 16:05:06 -0400
Subject: [pypy-dev] [pypy-commit] pypy default: Python 2.5 compat
In-Reply-To: <CAK5idxRAWMUuDcdcLhd5sTWHx5nag_rhn2J7tnpAHsZUuyTskw@mail.gmail.com>
References: <20120326113150.07C14820D9@wyvern.cs.uni-duesseldorf.de>
	<4F7216BA.1030805@gmail.com>
	<CAK5idxRAWMUuDcdcLhd5sTWHx5nag_rhn2J7tnpAHsZUuyTskw@mail.gmail.com>
Message-ID: <CAPZV6o_TM=HigJth949TuBm4vtYP2ihFwCjQijBg9=VGqDASSA@mail.gmail.com>

2012/3/27 Maciej Fijalkowski <fijall at gmail.com>:
> There is no reason not to.

Is it useful for something?


-- 
Regards,
Benjamin

From alex.gaynor at gmail.com  Tue Mar 27 22:06:35 2012
From: alex.gaynor at gmail.com (Alex Gaynor)
Date: Tue, 27 Mar 2012 16:06:35 -0400
Subject: [pypy-dev] [pypy-commit] pypy default: Python 2.5 compat
In-Reply-To: <CAK5idxRAWMUuDcdcLhd5sTWHx5nag_rhn2J7tnpAHsZUuyTskw@mail.gmail.com>
References: <20120326113150.07C14820D9@wyvern.cs.uni-duesseldorf.de>
	<4F7216BA.1030805@gmail.com>
	<CAK5idxRAWMUuDcdcLhd5sTWHx5nag_rhn2J7tnpAHsZUuyTskw@mail.gmail.com>
Message-ID: <CAFRnB2WnvXAQkLkOYMiGkueUgZvG_ppyusGafqMF7KBAhrJvnA@mail.gmail.com>

Well the reason not is that this has come up several times where someone
forgets a future import or something else.

Alex

On Tue, Mar 27, 2012 at 3:55 PM, Maciej Fijalkowski <fijall at gmail.com>wrote:

> On Tue, Mar 27, 2012 at 9:36 PM, Antonio Cuni <anto.cuni at gmail.com> wrote:
> > Hi Armin,
> >
> > On 03/26/2012 01:31 PM, arigo wrote:
> >> Author: Armin Rigo <arigo at tunes.org>
> >> Branch:
> >> Changeset: r53982:db33a634cb09
> >> Date: 2012-03-26 13:16 +0200
> >> http://bitbucket.org/pypy/pypy/changeset/db33a634cb09/
> >>
> >> Log:  Python 2.5 compat
> >
> > do we really want to keep python 2.5 compat? It's not tested, and "what
> is not
> > tested is broken"...
> >
> > ciao,
> > Anto
> > _______________________________________________
> > pypy-dev mailing list
> > pypy-dev at python.org
> > http://mail.python.org/mailman/listinfo/pypy-dev
>
> There is no reason not to.
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev
>



-- 
"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120327/5a7eab63/attachment.html>

From fijall at gmail.com  Tue Mar 27 22:11:33 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Tue, 27 Mar 2012 22:11:33 +0200
Subject: [pypy-dev] GSOC Ideas
In-Reply-To: <C8ABB3EB-738C-422C-ACE3-F856AA487265@gmail.com>
References: <C8ABB3EB-738C-422C-ACE3-F856AA487265@gmail.com>
Message-ID: <CAK5idxQpg5WTGiPNzDbg=uM4SHvwyp5Xbx5KFABWq9x96C-g2w@mail.gmail.com>

On Fri, Mar 23, 2012 at 1:13 AM, Lynne Wallace <lwallace890 at gmail.com> wrote:
> Hey everyone,
>
> I'm a 3rd year student?at ed.ac.uk,?interested in getting involved in pypy
> through google summer of code.
>
> I've been chatting to Alex Gaynor over irc - his primary suggestion was
> improving the register allocator.
> He also suggested browsing
> through?https://bitbucket.org/pypy/extradoc/src/tip/planning/jit.txt?to get
> an idea for projects, but it is a little hard to judge the size and
> complexity of some of the tasks without
> a little more internal knowledge.
>
> I was wondering if?anyone else had ideas or suggestions for gsoc projects?
>
> Thanks
> Lynne

Hi Lynne

Did you find any of the projects interesting? Do you need more ideas?

Cheers,
fijal

From sdouche at gmail.com  Tue Mar 27 22:37:03 2012
From: sdouche at gmail.com (Sebastien Douche)
Date: Tue, 27 Mar 2012 22:37:03 +0200
Subject: [pypy-dev] [pypy-commit] pypy default: Python 2.5 compat
In-Reply-To: <CAK5idxRAWMUuDcdcLhd5sTWHx5nag_rhn2J7tnpAHsZUuyTskw@mail.gmail.com>
References: <20120326113150.07C14820D9@wyvern.cs.uni-duesseldorf.de>
	<4F7216BA.1030805@gmail.com>
	<CAK5idxRAWMUuDcdcLhd5sTWHx5nag_rhn2J7tnpAHsZUuyTskw@mail.gmail.com>
Message-ID: <CAAGHeXFFJUnc4o5fSwrY6guM_erQxMoNKrDfoUWyjr47Xb6kUw@mail.gmail.com>

On Tue, Mar 27, 2012 at 21:55, Maciej Fijalkowski <fijall at gmail.com> wrote:

Hi Maciej

>> do we really want to keep python 2.5 compat? It's not tested, and "what is not
>> tested is broken"...
>
> There is no reason not to.

This version has reached end-of-life: no new version, even for major bugs.


-- 
Sebastien Douche <sdouche at gmail.com>
Twitter: @sdouche / G+: +sdouche

From stefan_ml at behnel.de  Wed Mar 28 09:33:08 2012
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 28 Mar 2012 09:33:08 +0200
Subject: [pypy-dev] GSOC Ideas
In-Reply-To: <CAMSv6X2GCPAbQjSL11BaGN8dbQP4BuMrQZmDQvYpLALThiFrAw@mail.gmail.com>
References: <C8ABB3EB-738C-422C-ACE3-F856AA487265@gmail.com>
	<CAMSv6X2GCPAbQjSL11BaGN8dbQP4BuMrQZmDQvYpLALThiFrAw@mail.gmail.com>
Message-ID: <jkuerk$4kr$1@dough.gmane.org>

Armin Rigo, 23.03.2012 07:55:
> On Fri, Mar 23, 2012 at 00:13, Lynne Wallace wrote:
>> I was wondering if anyone else had ideas or suggestions for gsoc projects?
> 
> We have a list of projects (which might or might not fit a GSoC size):
> http://doc.pypy.org/en/latest/project-ideas.html

I'm missing the project to fix up and optimise cpyext from that list. I
think that would be one of the most widely rewarding in the short run. And
it has the advantage for a GSoC that optimisation is generally open-ended,
so it can stop at any time with a positive result.

Stefan


From fijall at gmail.com  Wed Mar 28 10:28:09 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 28 Mar 2012 10:28:09 +0200
Subject: [pypy-dev] GSOC Ideas
In-Reply-To: <jkuerk$4kr$1@dough.gmane.org>
References: <C8ABB3EB-738C-422C-ACE3-F856AA487265@gmail.com>
	<CAMSv6X2GCPAbQjSL11BaGN8dbQP4BuMrQZmDQvYpLALThiFrAw@mail.gmail.com>
	<jkuerk$4kr$1@dough.gmane.org>
Message-ID: <CAK5idxR=GWHZb_CQSrW4g0tf+XOA2B8DK5T20dMkbRzx-ipWBQ@mail.gmail.com>

On Wed, Mar 28, 2012 at 9:33 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Armin Rigo, 23.03.2012 07:55:
>> On Fri, Mar 23, 2012 at 00:13, Lynne Wallace wrote:
>>> I was wondering if anyone else had ideas or suggestions for gsoc projects?
>>
>> We have a list of projects (which might or might not fit a GSoC size):
>> http://doc.pypy.org/en/latest/project-ideas.html
>
> I'm missing the project to fix up and optimise cpyext from that list. I
> think that would be one of the most widely rewarding in the short run. And
> it has the advantage for a GSoC that optimisation is generally open-ended,
> so it can stop at any time with a positive result.
>
> Stefan

Indeed.

Can you write down the necessary paragraph? You can just commit it
(it's pypy/doc/project-ideas.rst) or I'll commit it if you don't have
commit privs.

Cheers,
fijal

From stefan_ml at behnel.de  Wed Mar 28 10:47:57 2012
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 28 Mar 2012 10:47:57 +0200
Subject: [pypy-dev] GSOC Ideas
In-Reply-To: <CAK5idxR=GWHZb_CQSrW4g0tf+XOA2B8DK5T20dMkbRzx-ipWBQ@mail.gmail.com>
References: <C8ABB3EB-738C-422C-ACE3-F856AA487265@gmail.com>
	<CAMSv6X2GCPAbQjSL11BaGN8dbQP4BuMrQZmDQvYpLALThiFrAw@mail.gmail.com>
	<jkuerk$4kr$1@dough.gmane.org>
	<CAK5idxR=GWHZb_CQSrW4g0tf+XOA2B8DK5T20dMkbRzx-ipWBQ@mail.gmail.com>
Message-ID: <jkuj7t$84v$1@dough.gmane.org>

Maciej Fijalkowski, 28.03.2012 10:28:
> On Wed, Mar 28, 2012 at 9:33 AM, Stefan Behnel wrote:
>> Armin Rigo, 23.03.2012 07:55:
>>> On Fri, Mar 23, 2012 at 00:13, Lynne Wallace wrote:
>>>> I was wondering if anyone else had ideas or suggestions for gsoc projects?
>>>
>>> We have a list of projects (which might or might not fit a GSoC size):
>>> http://doc.pypy.org/en/latest/project-ideas.html
>>
>> I'm missing the project to fix up and optimise cpyext from that list. I
>> think that would be one of the most widely rewarding in the short run. And
>> it has the advantage for a GSoC that optimisation is generally open-ended,
>> so it can stop at any time with a positive result.
> 
> Indeed.
> 
> Can you write down the necessary paragraph? You can just commit it
> (it's pypy/doc/project-ideas.rst) or I'll commit it if you don't have
> commit privs.

I don't, so here's some plain text:

"""
Optimising cpyext (CPython C-API compatibility layer)

A lot of work has gone into PyPy's implementation of CPython's C-API over
the last years to let it reach a practical level of compatibility, so that
C extensions for CPython work on PyPy without major rewrites. However,
there are still many edges and corner cases where it misbehaves, and it has
not received any substantial optimisation so far.

The objective of this project is to fix bugs in cpyext and to optimise
several performance critical parts of it, such as the reference counting
support and other heavily used C-API functions. The net result would be to
have CPython extensions run much faster on PyPy than they currently do, or
to make them work at all if they currently don't. A part of this work would
be to get cpyext into a shape where it supports running Cython generated
extensions.
"""

Stefan


From fijall at gmail.com  Wed Mar 28 10:57:34 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 28 Mar 2012 10:57:34 +0200
Subject: [pypy-dev] GSOC Ideas
In-Reply-To: <jkuj7t$84v$1@dough.gmane.org>
References: <C8ABB3EB-738C-422C-ACE3-F856AA487265@gmail.com>
	<CAMSv6X2GCPAbQjSL11BaGN8dbQP4BuMrQZmDQvYpLALThiFrAw@mail.gmail.com>
	<jkuerk$4kr$1@dough.gmane.org>
	<CAK5idxR=GWHZb_CQSrW4g0tf+XOA2B8DK5T20dMkbRzx-ipWBQ@mail.gmail.com>
	<jkuj7t$84v$1@dough.gmane.org>
Message-ID: <CAK5idxQAaooxwg1FYRhChjsw-Fm_H4=7KHkBEy8vQTJtoXO0=A@mail.gmail.com>

On Wed, Mar 28, 2012 at 10:47 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Maciej Fijalkowski, 28.03.2012 10:28:
>> On Wed, Mar 28, 2012 at 9:33 AM, Stefan Behnel wrote:
>>> Armin Rigo, 23.03.2012 07:55:
>>>> On Fri, Mar 23, 2012 at 00:13, Lynne Wallace wrote:
>>>>> I was wondering if anyone else had ideas or suggestions for gsoc projects?
>>>>
>>>> We have a list of projects (which might or might not fit a GSoC size):
>>>> http://doc.pypy.org/en/latest/project-ideas.html
>>>
>>> I'm missing the project to fix up and optimise cpyext from that list. I
>>> think that would be one of the most widely rewarding in the short run. And
>>> it has the advantage for a GSoC that optimisation is generally open-ended,
>>> so it can stop at any time with a positive result.
>>
>> Indeed.
>>
>> Can you write down the necessary paragraph? You can just commit it
>> (it's pypy/doc/project-ideas.rst) or I'll commit it if you don't have
>> commit privs.
>
> I don't, so here's some plain text:
>
> """
> Optimising cpyext (CPython C-API compatibility layer)
>
> A lot of work has gone into PyPy's implementation of CPython's C-API over
> the last years to let it reach a practical level of compatibility, so that
> C extensions for CPython work on PyPy without major rewrites. However,
> there are still many edges and corner cases where it misbehaves, and it has
> not received any substantial optimisation so far.
>
> The objective of this project is to fix bugs in cpyext and to optimise
> several performance critical parts of it, such as the reference counting
> support and other heavily used C-API functions. The net result would be to
> have CPython extensions run much faster on PyPy than they currently do, or
> to make them work at all if they currently don't. A part of this work would
> be to get cpyext into a shape where it supports running Cython generated
> extensions.
> """
>
> Stefan
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev

Done.

I would not like to do that as a SoC myself, because that sounds like
a mess, but who am I to judge :)

From amauryfa at gmail.com  Wed Mar 28 11:08:19 2012
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Wed, 28 Mar 2012 11:08:19 +0200
Subject: [pypy-dev] GSOC Ideas
In-Reply-To: <jkuj7t$84v$1@dough.gmane.org>
References: <C8ABB3EB-738C-422C-ACE3-F856AA487265@gmail.com>
	<CAMSv6X2GCPAbQjSL11BaGN8dbQP4BuMrQZmDQvYpLALThiFrAw@mail.gmail.com>
	<jkuerk$4kr$1@dough.gmane.org>
	<CAK5idxR=GWHZb_CQSrW4g0tf+XOA2B8DK5T20dMkbRzx-ipWBQ@mail.gmail.com>
	<jkuj7t$84v$1@dough.gmane.org>
Message-ID: <CAGmFida2jjShxvMsdfTJ+AO4OQV5ZzXq3=MfPyrNZfpv_2jQXQ@mail.gmail.com>

2012/3/28 Stefan Behnel <stefan_ml at behnel.de>:
>> Can you write down the necessary paragraph? You can just commit it
>> (it's pypy/doc/project-ideas.rst) or I'll commit it if you don't have
>> commit privs.
>
> I don't, so here's some plain text:
>
> """
> Optimising cpyext (CPython C-API compatibility layer)
>
> A lot of work has gone into PyPy's implementation of CPython's C-API over
> the last years to let it reach a practical level of compatibility, so that
> C extensions for CPython work on PyPy without major rewrites. However,
> there are still many edges and corner cases where it misbehaves, and it has
> not received any substantial optimisation so far.
>
> The objective of this project is to fix bugs in cpyext and to optimise
> several performance critical parts of it, such as the reference counting
> support and other heavily used C-API functions. The net result would be to
> have CPython extensions run much faster on PyPy than they currently do, or
> to make them work at all if they currently don't. A part of this work would
> be to get cpyext into a shape where it supports running Cython generated
> extensions.
> """

I'd also add a task to support embedding PyPy; there are already
interesting results
http://mail.python.org/pipermail/pypy-dev/2012-March/009661.html

-- 
Amaury Forgeot d'Arc

From arigo at tunes.org  Wed Mar 28 11:08:50 2012
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 28 Mar 2012 11:08:50 +0200
Subject: [pypy-dev] The Work Plan Re:  STM proposal funding
In-Reply-To: <1332775738.67346.YahooMailNeo@web120702.mail.ne1.yahoo.com>
References: <CAMSv6X2GF=1HaZ1wq7OfS_c-DiLO-zC1Sm0hUWKuSG4QXhTw+w@mail.gmail.com>
	<1332775738.67346.YahooMailNeo@web120702.mail.ne1.yahoo.com>
Message-ID: <CAMSv6X2wv-jKbtaQt2VFtFp1jf72-xkdwZXYsJfpfhUnPVGaow@mail.gmail.com>

Hi Andrew,

On Mon, Mar 26, 2012 at 17:28, Andrew Francis <andrewfr_ice at yahoo.com> wrote:
> I am familar with the AME papers. AME literature seems to taper
> off. Have you tried talking to the various authors? Or get your
> hands on an implementation.

Indeed, and it was around 2007, so I expect the authors to have been
involved in completely different things for quite some time now...
But I could try to contact them anyway.

> STM is used under the hood to implement the channels and other
> associated constructs. Hopefully this would 1) lead to a smaller
> transactional footprint. 2) Totally hide STM from the programmer. 3)
> Use an already existing API.

Yes for 2) and 3): that's precisely where I'm going.  Right now I'm
focusing on Twisted instead of Stackless (with help from J.P.
Calderone), but the idea is the same: hide it totally from the
end programmer by using the existing API.  Only the one tweaking the
Twisted internals needs to know about it.  The same would apply to
Stackless.


A bient?t,

Armin.

From arigo at tunes.org  Wed Mar 28 11:38:19 2012
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 28 Mar 2012 11:38:19 +0200
Subject: [pypy-dev] [pypy-commit] pypy default: Python 2.5 compat
In-Reply-To: <4F7216BA.1030805@gmail.com>
References: <20120326113150.07C14820D9@wyvern.cs.uni-duesseldorf.de>
	<4F7216BA.1030805@gmail.com>
Message-ID: <CAMSv6X1V52e_oprTok3zqkqi-P-yOOV6_8_BNNphUfjWPTgdnA@mail.gmail.com>

Hi,

On Tue, Mar 27, 2012 at 21:36, Antonio Cuni <anto.cuni at gmail.com> wrote:
> do we really want to keep python 2.5 compat? It's not tested, and "what is not
> tested is broken"...

As long as I'm running python 2.5 and hitting the occasional "with"
statement I see no reason against adding the "from future" import at
the top of the file.  It doesn't hurt anybody.  I don't think we
should actively try to support 2.5 any more, though.


A bient?t,

Armin.

From tbaldridge at gmail.com  Wed Mar 28 16:35:23 2012
From: tbaldridge at gmail.com (Timothy Baldridge)
Date: Wed, 28 Mar 2012 09:35:23 -0500
Subject: [pypy-dev] Profiling pypy code
Message-ID: <CAL36E+vWtY0g0jBtTPh89v6N96km3bV_LeaFBhVPM8ZCFhHqbQ@mail.gmail.com>

I've been trying to find ways to optimize my clojure-on-python
project. And I've been running through my code in jitviewer. However
it seems that jitviewer only displays the number of times a given loop
was run. What should I look into for benchmarking actual times spent
in each loop? It seems that my optimization efforts would be best
spent on areas of code that have a high hit count AND have a long
execution time. Any ideas?

Thanks,

Timothy

-- 
?One of the main causes of the fall of the Roman Empire was
that?lacking zero?they had no way to indicate successful termination
of their C programs.?
(Robert Firth)

From andrewfr_ice at yahoo.com  Wed Mar 28 18:23:21 2012
From: andrewfr_ice at yahoo.com (Andrew Francis)
Date: Wed, 28 Mar 2012 09:23:21 -0700 (PDT)
Subject: [pypy-dev] The Work Plan Re:  STM proposal funding
In-Reply-To: <CAMSv6X2wv-jKbtaQt2VFtFp1jf72-xkdwZXYsJfpfhUnPVGaow@mail.gmail.com>
References: <CAMSv6X2GF=1HaZ1wq7OfS_c-DiLO-zC1Sm0hUWKuSG4QXhTw+w@mail.gmail.com>
	<1332775738.67346.YahooMailNeo@web120702.mail.ne1.yahoo.com>
	<CAMSv6X2wv-jKbtaQt2VFtFp1jf72-xkdwZXYsJfpfhUnPVGaow@mail.gmail.com>
Message-ID: <1332951801.7099.YahooMailNeo@web120701.mail.ne1.yahoo.com>

Hi Armin:


________________________________
 From: Armin Rigo <arigo at tunes.org>
To: Andrew Francis <andrewfr_ice at yahoo.com> 
Cc: PyPy Developer Mailing List <pypy-dev at python.org> 
Sent: Wednesday, March 28, 2012 5:08 AM
Subject: Re: The Work Plan Re: [pypy-dev] STM proposal funding
 
Hi Andrew,

On Mon, Mar 26, 2012 at 17:28, Andrew Francis <andrewfr_ice at yahoo.com> wrote:


AF> I am familar with the AME papers. AME literature seems to taper
AF> off. Have you tried talking to the various authors? Or get your
AF> hands on an implementation.

>Indeed, and it was around 2007, so I expect the authors to have been
>involved in completely different things for quite some time now...
>But I could try to contact them anyway.


Communications is good :-)?

AF> STM is used under the hood to implement the channels and other
AF> associated constructs. Hopefully this would 1) lead to a smaller
AF> transactional footprint. 2) Totally hide STM from the programmer. 3)
AF> Use an already existing API.

>Yes for 2) and 3): that's precisely where I'm going.? Right now I'm
>focusing on Twisted instead of Stackless (with help from J.P.
>Calderone),?

Cool. I use Twisted!

>but the idea is the same: hide it totally from the
>end programmer by using the existing API.? Only the one tweaking the
>Twisted internals needs to know about it.? The same would apply to
>Stackless.


I don't know if Christian is following this conversation....

My PyPy knowledge is still?sketchy but I am changing that. ?I do understand the Twisted reactor model
?(thanks to my 2008 Pycon Talk) so I could follow discussions in that area. Is this discussed on IRC?

I would *love* to engage in a similar effort on the Stackless side. At first things would be slow on my
side. One of the tasks I would need to do is better examine how?threads interact with the Stackless?
scheduler (an area I seldom look at but follow the?discussions in the Stackless Mailing List. I may have
misconceptions that could readily be cleared up).

The low hanging fruit would be 1) rewriting some of the STAMP applications to use Stackless.?
2) exposing some of the lock-free mechanisms - this would get me deeper in the FFI.?

On the wild idea front, a while back, I took a graduate research course in Distributed Algorithms. There
I learnt about ?Nancy Lynch's I/O Automaton model and encountered synchronisers (not that I am an expert).
A modified version?of the?I/O automaton model is used to develop proofs for transactional algorithms.?
For this discussion, ascheduler and a?transaction manager can be thought of as a synchroniser. I have a gut?
feeling that somehow the scheduler?and a transaction manager can be unified. ?I need to revisit the literature.

Cheers,
Andrew
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120328/e506189f/attachment.html>

From SparkVieww at gmail.com  Wed Mar 28 16:05:24 2012
From: SparkVieww at gmail.com (Spark View Inc)
Date: Wed, 28 Mar 2012 16:05:24 +0200
Subject: [pypy-dev] *****SECRET EVALUATOR  POSITION*****
Message-ID: <201203281429.q2SETsVx007044@smtpweb.clb.oleane.net>

Hello ,


Spark View Inc is a market research company that uses Secret Evaluator to measure quality of retail service or gather specific information about products and services. We work with some of the largest, reputable businesses in the world;from Fast Food Petroleum,Technology or Fashion Retailers and more.
Retail Survey Concept is presently recruiting Secret Evaluator in all states of the USA/CANADA .

POSITION TITLE: Secret Evaluator

AVAILABILITY: Open

START DATE: Immediate

COMMISSION: $300 Per Assignment (2 Assignments weekly).

JOB DESCRIPTION & RESPONSIBILITIES:

You will be hired to conduct an all expenses paid surveys and evaluation exercises on behalf of RETAIL SURVEY CONCEPT?. Your job will be to evaluate and measure quality of retail service or gather
specific information about products and services. As our Secret Evaluator posing as normal customers, you will expected to perform specific tasks?such as purchasing a product or using a service.

We presently have a couple of outstanding contracts, which means you will be directed to outlets, firms and corporate bodies to report on your experience by forwarding us a mail.

PLEASE NOTE:You will be provided funds in advance for any upcoming survey via Checks and Wire Transactions to cover expenses.


You do not require any special skills for this opening, however in order to Apply for this job you need to ensure you have access to your e-mails two( 2) times a day and will reply to all notifications promptly.

Interested Candidates should simply fill the form below to make an Official Application:

Full Names:
Address Line 1:
Address Line 2:
City:
State:
Zip Code:
Age:
Phone Number:
Cell / Mobile Phone Number:
Sex:
Current Occupation:
Have you had any work of this nature before:
Email Address:



Thanks,
Xavi Mendez,
Evaluation Manager.
?2006-2012 Spark View Inc? Corporate Headquarters




From stefan_ml at behnel.de  Wed Mar 28 23:34:15 2012
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 28 Mar 2012 23:34:15 +0200
Subject: [pypy-dev] Bringing Cython and PyPy closer together
In-Reply-To: <CAMSv6X0pJMveNQSVWdh6A7-ke6W0rvi_CahYmVLX+xdX9tgUBQ@mail.gmail.com>
References: <jhg54k$dte$1@dough.gmane.org> <jhnok5$ve2$1@dough.gmane.org>
	<CAGmFidbutXahNAKbHgaLFOM39uTKZ8C3TkrPLeauB0Jm5CQ-bg@mail.gmail.com>
	<jho9he$c2k$1@dough.gmane.org>
	<CAGmFidZ2DDxnPBi7E5Bo8PTn3iubjfrtxmBv1uEsaombo2ey1Q@mail.gmail.com>
	<jhobh9$ots$1@dough.gmane.org>
	<CAGmFidbEPpbQPrKEsNHkMQ1e3mgsYV1vd0SivdVPvSOYaLSXmA@mail.gmail.com>
	<jie2mh$qub$1@dough.gmane.org> <jk6kjs$840$1@dough.gmane.org>
	<CAK5idxThPMvpoFGODum85_pnd6ny9MZ81i7961uYvSkWODrqKg@mail.gmail.com>
	<CAMSv6X0pJMveNQSVWdh6A7-ke6W0rvi_CahYmVLX+xdX9tgUBQ@mail.gmail.com>
Message-ID: <jl004o$sej$1@dough.gmane.org>

Armin Rigo, 21.03.2012 17:55:
> On Mon, Mar 19, 2012 at 09:11, Maciej Fijalkowski wrote:
>>>> The (trivial) implementation of the two functions is at the end of this file:
>>>>
>>>> http://bugs.python.org/file24613/exc_info_capi.patch
>>>>
>>>> Could you add them to PyPy?
> 
> The new functions make sense to me.  Anyone with cpyext knowledge can
> add them to PyPy.  I can probably do it if needed, although I don't
> know cpyext very well.  (Alternatively, coming up yourself with a
> patch would be the best way to move things forward :-)

Yes, apparently so. However, we're currently busy with getting Cython 0.16
out of the door and I didn't find the time so far to set up a PyPy
development environment to do anything useful with it.

I'd be glad if this could get finished soon, because then I could finally
merge the Cython side of this into mainline. This has been a major blocker
for a while now, and exception handling or generators aren't exactly rare
features in user code.

Stefan


From hakan at debian.org  Fri Mar 30 07:58:42 2012
From: hakan at debian.org (Hakan Ardo)
Date: Fri, 30 Mar 2012 07:58:42 +0200
Subject: [pypy-dev] [pypy-commit] pypy dynamic-specialized-tuple: kill
 an assert that doesn't hold now.
In-Reply-To: <20120330020007.DFA468236A@wyvern.cs.uni-duesseldorf.de>
References: <20120330020007.DFA468236A@wyvern.cs.uni-duesseldorf.de>
Message-ID: <CAEkL+Rj0=vdjXDVfj6E92LmaeF9++O+0dZ=qOHKOYYCt75iNWg@mail.gmail.com>

Hi,
if the descr can change and assumptions are made about it you'll need
to add it to virtualstate.NotVirtualStateInfo and check it in
generalization_of and _generate_guards. Otherwise if a peeled loop
becomes specialized to some specific descr and that descr is changed
on a bridge, the bridge will jump to the peeled loop, but it has to
jump to the preamble.

On Fri, Mar 30, 2012 at 4:00 AM, alex_gaynor <noreply at buildbot.pypy.org> wrote:
> Author: Alex Gaynor <alex.gaynor at gmail.com>
> Branch: dynamic-specialized-tuple
> Changeset: r54088:5b813268023d
> Date: 2012-03-29 21:59 -0400
> http://bitbucket.org/pypy/pypy/changeset/5b813268023d/
>
> Log: ? ?kill an assert that doesn't hold now.
>
> diff --git a/pypy/jit/metainterp/optimizeopt/optimizer.py b/pypy/jit/metainterp/optimizeopt/optimizer.py
> --- a/pypy/jit/metainterp/optimizeopt/optimizer.py
> +++ b/pypy/jit/metainterp/optimizeopt/optimizer.py
> @@ -59,7 +59,6 @@
> ? ? def make_len_gt(self, mode, descr, val):
> ? ? ? ? if self.lenbound:
> ? ? ? ? ? ? assert self.lenbound.mode == mode
> - ? ? ? ? ? ?assert self.lenbound.descr == descr
> ? ? ? ? ? ? self.lenbound.bound.make_gt(IntBound(val, val))
> ? ? ? ? else:
> ? ? ? ? ? ? self.lenbound = LenBound(mode, descr, IntLowerBound(val + 1))
> _______________________________________________
> pypy-commit mailing list
> pypy-commit at python.org
> http://mail.python.org/mailman/listinfo/pypy-commit



-- 
H?kan Ard?

From alex.gaynor at gmail.com  Fri Mar 30 14:49:19 2012
From: alex.gaynor at gmail.com (Alex Gaynor)
Date: Fri, 30 Mar 2012 08:49:19 -0400
Subject: [pypy-dev] [pypy-commit] pypy dynamic-specialized-tuple: kill
 an assert that doesn't hold now.
In-Reply-To: <CAEkL+Rj0=vdjXDVfj6E92LmaeF9++O+0dZ=qOHKOYYCt75iNWg@mail.gmail.com>
References: <20120330020007.DFA468236A@wyvern.cs.uni-duesseldorf.de>
	<CAEkL+Rj0=vdjXDVfj6E92LmaeF9++O+0dZ=qOHKOYYCt75iNWg@mail.gmail.com>
Message-ID: <CAFRnB2VaFT9oHh4GhCs5egbnFSGShbBG83AMe8iF7natKqTZYA@mail.gmail.com>

On Fri, Mar 30, 2012 at 1:58 AM, Hakan Ardo <hakan at debian.org> wrote:

> Hi,
> if the descr can change and assumptions are made about it you'll need
> to add it to virtualstate.NotVirtualStateInfo and check it in
> generalization_of and _generate_guards. Otherwise if a peeled loop
> becomes specialized to some specific descr and that descr is changed
> on a bridge, the bridge will jump to the peeled loop, but it has to
> jump to the preamble.
>
> On Fri, Mar 30, 2012 at 4:00 AM, alex_gaynor <noreply at buildbot.pypy.org>
> wrote:
> > Author: Alex Gaynor <alex.gaynor at gmail.com>
> > Branch: dynamic-specialized-tuple
> > Changeset: r54088:5b813268023d
> > Date: 2012-03-29 21:59 -0400
> > http://bitbucket.org/pypy/pypy/changeset/5b813268023d/
> >
> > Log:    kill an assert that doesn't hold now.
> >
> > diff --git a/pypy/jit/metainterp/optimizeopt/optimizer.py
> b/pypy/jit/metainterp/optimizeopt/optimizer.py
> > --- a/pypy/jit/metainterp/optimizeopt/optimizer.py
> > +++ b/pypy/jit/metainterp/optimizeopt/optimizer.py
> > @@ -59,7 +59,6 @@
> >     def make_len_gt(self, mode, descr, val):
> >         if self.lenbound:
> >             assert self.lenbound.mode == mode
> > -            assert self.lenbound.descr == descr
> >             self.lenbound.bound.make_gt(IntBound(val, val))
> >         else:
> >             self.lenbound = LenBound(mode, descr, IntLowerBound(val + 1))
> > _______________________________________________
> > pypy-commit mailing list
> > pypy-commit at python.org
> > http://mail.python.org/mailman/listinfo/pypy-commit
>
>
>
> --
> H?kan Ard?
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev
>

I'm a bit concerned that this branch may be breaking too many assumptions
in the JIT with the current approach.  The idea of this branch is to
dynamically specialize the shape of a tuple's memory, to store unboxed
objects.  Based on the current approach (storing all data in an array of
integers) you get traces that look like:

p0 = new(<untyped storage>)
setfield_gc(p0, ConstPtr("io"), <FieldDescr shape>)
setarrayitem_gc(p0, 0, i0, <ArrayDescr int>)
setarrayitem_gc(p0, 1, p1, <ArrayDescr ptr>)
finish(p0)

This is a trace for allocating a tuple like (int1, ptr1) and returning it.
 Do you think a trace like this breaks too many assumptions in the JIT, and
if so, how can we go about making the JIT work with this.

Alex
-- 
"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120330/cb56853f/attachment.html>

From hakan at debian.org  Fri Mar 30 15:15:05 2012
From: hakan at debian.org (Hakan Ardo)
Date: Fri, 30 Mar 2012 15:15:05 +0200
Subject: [pypy-dev] [pypy-commit] pypy dynamic-specialized-tuple: kill
 an assert that doesn't hold now.
In-Reply-To: <CAFRnB2VaFT9oHh4GhCs5egbnFSGShbBG83AMe8iF7natKqTZYA@mail.gmail.com>
References: <20120330020007.DFA468236A@wyvern.cs.uni-duesseldorf.de>
	<CAEkL+Rj0=vdjXDVfj6E92LmaeF9++O+0dZ=qOHKOYYCt75iNWg@mail.gmail.com>
	<CAFRnB2VaFT9oHh4GhCs5egbnFSGShbBG83AMe8iF7natKqTZYA@mail.gmail.com>
Message-ID: <CAEkL+Rh9MM3mAUKb9hdC=2+E8oKR1jyW-f+fr_5L+z6k6a1asg@mail.gmail.com>

On Fri, Mar 30, 2012 at 2:49 PM, Alex Gaynor <alex.gaynor at gmail.com> wrote:
> I'm a bit concerned that this branch may be breaking too many assumptions in
> the JIT with the current approach. ?The idea of this branch is to
> dynamically specialize the shape of a tuple's memory, to store unboxed
> objects. ?Based on the current approach (storing all data in an array of
> integers) you get traces that look like:
>
> p0 = new(<untyped storage>)
> setfield_gc(p0, ConstPtr("io"), <FieldDescr shape>)
> setarrayitem_gc(p0, 0, i0, <ArrayDescr int>)
> setarrayitem_gc(p0, 1, p1, <ArrayDescr ptr>)
> finish(p0)
>
> This is a trace for allocating a tuple like (int1, ptr1) and returning it.
> ?Do you think a trace like this breaks too many assumptions in the JIT, and
> if so, how can we go about making the JIT work with this.

That part might be fine. In this case the descr depends is not realy
changing it just depends on the constant index. The assert could
probably be relaxed to allow that. But what would traces reading
values from the tuple look like. In cases where the tuple may conatin
different types or if the index is not constant? Say something like
this:

sa = 0
a = (1, 2.0)
while sa < 3000:
    sa += a[0] + a[1]
    if sa > 1500:
        a = (a[1], a[0])


-- 
H?kan Ard?

From alex.gaynor at gmail.com  Fri Mar 30 15:18:31 2012
From: alex.gaynor at gmail.com (Alex Gaynor)
Date: Fri, 30 Mar 2012 09:18:31 -0400
Subject: [pypy-dev] [pypy-commit] pypy dynamic-specialized-tuple: kill
 an assert that doesn't hold now.
In-Reply-To: <CAEkL+Rh9MM3mAUKb9hdC=2+E8oKR1jyW-f+fr_5L+z6k6a1asg@mail.gmail.com>
References: <20120330020007.DFA468236A@wyvern.cs.uni-duesseldorf.de>
	<CAEkL+Rj0=vdjXDVfj6E92LmaeF9++O+0dZ=qOHKOYYCt75iNWg@mail.gmail.com>
	<CAFRnB2VaFT9oHh4GhCs5egbnFSGShbBG83AMe8iF7natKqTZYA@mail.gmail.com>
	<CAEkL+Rh9MM3mAUKb9hdC=2+E8oKR1jyW-f+fr_5L+z6k6a1asg@mail.gmail.com>
Message-ID: <CAFRnB2XYN_7WDWB+smwd8KAznCaT0m+VCNc6mLetQsPfu0w5Rw@mail.gmail.com>

On Fri, Mar 30, 2012 at 9:15 AM, Hakan Ardo <hakan at debian.org> wrote:

> On Fri, Mar 30, 2012 at 2:49 PM, Alex Gaynor <alex.gaynor at gmail.com>
> wrote:
> > I'm a bit concerned that this branch may be breaking too many
> assumptions in
> > the JIT with the current approach.  The idea of this branch is to
> > dynamically specialize the shape of a tuple's memory, to store unboxed
> > objects.  Based on the current approach (storing all data in an array of
> > integers) you get traces that look like:
> >
> > p0 = new(<untyped storage>)
> > setfield_gc(p0, ConstPtr("io"), <FieldDescr shape>)
> > setarrayitem_gc(p0, 0, i0, <ArrayDescr int>)
> > setarrayitem_gc(p0, 1, p1, <ArrayDescr ptr>)
> > finish(p0)
> >
> > This is a trace for allocating a tuple like (int1, ptr1) and returning
> it.
> >  Do you think a trace like this breaks too many assumptions in the JIT,
> and
> > if so, how can we go about making the JIT work with this.
>
> That part might be fine. In this case the descr depends is not realy
> changing it just depends on the constant index. The assert could
> probably be relaxed to allow that. But what would traces reading
> values from the tuple look like. In cases where the tuple may conatin
> different types or if the index is not constant? Say something like
> this:
>
> sa = 0
> a = (1, 2.0)
> while sa < 3000:
>    sa += a[0] + a[1]
>    if sa > 1500:
>        a = (a[1], a[0])
>
>
> --
> H?kan Ard?
>

My idea is approximately you promote the shape whenever the index of a read
is constant.  Otherwise you produce a read on the shape, and a guard about
the type.

Alex

-- 
"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120330/c1c7f006/attachment.html>

From hakan at debian.org  Fri Mar 30 15:44:13 2012
From: hakan at debian.org (Hakan Ardo)
Date: Fri, 30 Mar 2012 15:44:13 +0200
Subject: [pypy-dev] [pypy-commit] pypy dynamic-specialized-tuple: kill
 an assert that doesn't hold now.
In-Reply-To: <CAFRnB2XYN_7WDWB+smwd8KAznCaT0m+VCNc6mLetQsPfu0w5Rw@mail.gmail.com>
References: <20120330020007.DFA468236A@wyvern.cs.uni-duesseldorf.de>
	<CAEkL+Rj0=vdjXDVfj6E92LmaeF9++O+0dZ=qOHKOYYCt75iNWg@mail.gmail.com>
	<CAFRnB2VaFT9oHh4GhCs5egbnFSGShbBG83AMe8iF7natKqTZYA@mail.gmail.com>
	<CAEkL+Rh9MM3mAUKb9hdC=2+E8oKR1jyW-f+fr_5L+z6k6a1asg@mail.gmail.com>
	<CAFRnB2XYN_7WDWB+smwd8KAznCaT0m+VCNc6mLetQsPfu0w5Rw@mail.gmail.com>
Message-ID: <CAEkL+RjxidRe4J4Pxuj9=5XXy7GdspiOhR7Lmxgp7CAxDPzd5g@mail.gmail.com>

On Fri, Mar 30, 2012 at 3:18 PM, Alex Gaynor <alex.gaynor at gmail.com> wrote:
>>
>> sa = 0
>> a = (1, 2.0)
>> while sa < 3000:
>> ? ?sa += a[0] + a[1]
>> ? ?if sa > 1500:
>> ? ? ? ?a = (a[1], a[0])
>>
>
> My idea is approximately you promote the shape whenever the index of a read
> is constant.

But you would still need a guard about the type, right?

>?Otherwise you produce a read on the shape, and a guard about
> the type.

These guards we'll need to be added to the virtualstate in the same
way as guards regarding the length of an array is treated. If those
guards are always explicit guards that should be simple. But if they
are implicit it might be tricky. By implicit I mean things like after
getarrayitem_gc(p0, 7) we will assume that the length of p0 is at
least 8.

The alternative is to not allow this kind of information to propagate
across loop boundaries, like we do with unerased pointers. In that
case the "guards" about the types are implied by the surrounding
rpython code and there is no way for the jit to extract the
information.

>
> Alex
>
> --
> "I disapprove of what you say, but I will defend to the death your right to
> say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
> "The people's good is the highest law." -- Cicero
>



-- 
H?kan Ard?

From bookaa at rorsoft.com  Sat Mar 31 04:43:15 2012
From: bookaa at rorsoft.com (bookaa)
Date: Sat, 31 Mar 2012 10:43:15 +0800
Subject: [pypy-dev] dotviewer bug on Windows system
Message-ID: <ED2A5787B55148779CD9B20D9CFA9794@vSHliutaotao>

On Windows system, simply run:
    pypy-pypy-2346207d9946\dotviewer>dotviewer.py ..\pypy\doc\image\JIT.dot
will get a error:
    graphparse.PlainParseError: bad 'node'
Finally I fix this problem by add 3 line in parse_plain function of 
dotviewer\graphparse.py:
        if lines[i].endswith('\\\n'):   # line ending in '\'
            lines[i] = lines[i][:-2] + lines[i+1]
            del lines[i+1]
        elif lines[i].endswith('\\\r\n'):   # line ending in '\'
            lines[i] = lines[i][:-3] + lines[i+1]
            del lines[i+1]

bookaa 


From arigo at tunes.org  Sat Mar 31 12:31:37 2012
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 31 Mar 2012 12:31:37 +0200
Subject: [pypy-dev] dotviewer bug on Windows system
In-Reply-To: <ED2A5787B55148779CD9B20D9CFA9794@vSHliutaotao>
References: <ED2A5787B55148779CD9B20D9CFA9794@vSHliutaotao>
Message-ID: <CAMSv6X0+rDWUd4v0Qjk-9MbYDrJYKVS_CThhe0KySbVxEusAEg@mail.gmail.com>

Hi Bookaa,

On Sat, Mar 31, 2012 at 04:43, bookaa <bookaa at rorsoft.com> wrote:
> Finally I fix this problem by add 3 line in parse_plain function of
> dotviewer\graphparse.py:

Thank you!  I checked in a (slightly more general) fix in
39ba7e60ebca.  Can you tell us if it seems to work fine?


A bient?t,

Armin.