From cfbolz at gmx.de  Wed Jul  1 11:38:34 2015
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Wed, 01 Jul 2015 11:38:34 +0200
Subject: [pypy-dev] A simple file reading is 2x slow wrt CPython
In-Reply-To: <201506292210.t5TMAfUb014770@fido.openend.se>
References: <CAFub=KShXZXs--A1xi85TCN0uNCiKV=ZQDkCiZKMg3vn9THdcw@mail.gmail.com>
 <CAHVvXxSjpehmNDpvLb2ou5kZYorQrng21r4JZAgpN9HnHXBgfQ@mail.gmail.com>
 <CAHVvXxSo8Wo9iv=ObBGJ2apTKry3B5K2yiHtrOWT3NQ7wBKDxQ@mail.gmail.com>
 <CAFub=KQzSQv_WEH=XWfGh6p8zm0zv0xxe6xORMr=K7hpAzh_JA@mail.gmail.com>
 <CAK5idxS0eUvWiJYPcHyzUdNo4BQLDuQn6E+Q5binxNZhKitzbA@mail.gmail.com>
 <20150629144302.GA8918@domone>
 <CAK5idxT3wwCeUKMr1xa=uW0Z8=iMVKfU0YVwy6BXu5owJHYXHA@mail.gmail.com>
 <CAFub=KQamgGcGhCtaQ58LfUA2aHB4XDDGR5M0hBEkb6Up3aH5Q@mail.gmail.com>
 <CAHVvXxQXfv_ceRC2vGZcwujF-GK3KvcojrZ8HCmhuDRM5cvv3Q@mail.gmail.com>
 <ozancag@gmail.com>
 <CAFub=KSOkNL6pbQt=X7+BcTRQD9A56NR9sJjmuzHwN6M5rmPJQ@mail.gmail.com>
 <201506292026.t5TKQsWo012048@fido.openend.se><CAFub=KSBVj51HO8E5cNiQd2wkmt137=+rTCqdeckY0XnzdjhrA@mail.gmail.com>
 <201506292210.t5TMAfUb014770@fido.openend.se>
Message-ID: <5593B51A.9030200@gmx.de>

On 30/06/15 00:10, Laura Creighton wrote:
> In a message of Mon, 29 Jun 2015 23:53:21 +0200, Ozan ?a?layan writes:
>> Well I tried again but cant reproduce it. BTW cut down the whole code to 4
>> seconds with PyPy vs 28 seconds on CPython. The original C++ code might be
>> slower than this, i'll check it.
>>
>> Thanks for fast replies, contributions, kindness :)
>
> You are welcome for the kindness.  Try to encourage it wherever you go.
> So _very_ many things go better with kindness, and so many people think
> that it is in some way beneath them to be kind, to our great sorrow.

:-)

Getting a bit off-topic, but this is a related blog post I read recently:

http://boz.com/articles/be-kind.html

Cheers,

Carl Friedrich


From andy at pistuffing.co.uk  Wed Jul  1 10:58:41 2015
From: andy at pistuffing.co.uk (Andy Baker)
Date: Wed, 1 Jul 2015 09:58:41 +0100
Subject: [pypy-dev] pypy 2.2.1 performance on Raspian
Message-ID: <D60E225F181B4DFB8227F12665FEE5E7@Home>

I am trying pypy as a performance increase over CPython.  My code uses
standard python libraries (not numpy), plus 2 'C' libraries accessing
Raspberry Pi GPIO hardware with python wrappers.  I have compiled both with
pypy (sudo pypy setup.py install).  The pypy version runs with no problems
but is 2.5 times slower than CPython.  From the performance stats, I believe
2.2.1 should be on average 6 times faster.  I presume it's my 'C' python
libraries which are the cause of the reduced performance.  What do I need to
do to make these private libraries pypy compatible?
 
Thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20150701/0565859d/attachment.html>

From cory at lukasa.co.uk  Wed Jul  1 14:48:23 2015
From: cory at lukasa.co.uk (Cory Benfield)
Date: Wed, 1 Jul 2015 13:48:23 +0100
Subject: [pypy-dev] pypy 2.2.1 performance on Raspian
In-Reply-To: <D60E225F181B4DFB8227F12665FEE5E7@Home>
References: <D60E225F181B4DFB8227F12665FEE5E7@Home>
Message-ID: <0D40E7F9-8441-45AB-A815-EDEDD3281753@lukasa.co.uk>


> On 1 Jul 2015, at 09:58, Andy Baker <andy at pistuffing.co.uk> wrote:
> 
> I am trying pypy as a performance increase over CPython.  My code uses standard python libraries (not numpy), plus 2 'C' libraries accessing Raspberry Pi GPIO hardware with python wrappers.  I have compiled both with pypy (sudo pypy setup.py install).  The pypy version runs with no problems but is 2.5 times slower than CPython.  From the performance stats, I believe 2.2.1 should be on average 6 times faster.  I presume it's my 'C' python libraries which are the cause of the reduced performance.  What do I need to do to make these private libraries pypy compatible?
> 

When you say you have Python wrappers, how have you written them? Generally PyPy works best with Python interfaces written using CFFI[0].

Cory

[0]: https://cffi.readthedocs.org/en/latest/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20150701/7c42dfdf/attachment.sig>

From andy at pistuffing.co.uk  Wed Jul  1 15:15:06 2015
From: andy at pistuffing.co.uk (Andy Baker)
Date: Wed, 1 Jul 2015 14:15:06 +0100
Subject: [pypy-dev] pypy 2.2.1 performance on Raspian
In-Reply-To: <0D40E7F9-8441-45AB-A815-EDEDD3281753@lukasa.co.uk>
References: <D60E225F181B4DFB8227F12665FEE5E7@Home>
 <0D40E7F9-8441-45AB-A815-EDEDD3281753@lukasa.co.uk>
Message-ID: <75E051BE11EE42A8A1A8689C5097054F@Home>

Hi Cory,

I didn't write the 'C' code or the Python wrappers myself but I have
modified the 'C' source code and added new python wrapper function - they
are for the Raspberry Pi RPiGPIO and RPIO hardware integration code.  The
"wrappers" are very thin layers defining to the Cpython interpreter (or pypy
JIT) how to map the Python call to the 'C' equvalent call along with
exceptions and control of the GIL.  There's no state stored within this
'wrapper' code - the 'C' code blocks returning OK, or an error which raises
a Python exception.  I'll look at CFFI to see if that sheds any light.

Andy

-----Original Message-----
From: Cory Benfield [mailto:cory at lukasa.co.uk] 
Sent: 01 July 2015 13:48
To: Andy Baker
Cc: pypy-dev at python.org
Subject: Re: [pypy-dev] pypy 2.2.1 performance on Raspian


> On 1 Jul 2015, at 09:58, Andy Baker <andy at pistuffing.co.uk> wrote:
> 
> I am trying pypy as a performance increase over CPython.  My code uses
standard python libraries (not numpy), plus 2 'C' libraries accessing
Raspberry Pi GPIO hardware with python wrappers.  I have compiled both with
pypy (sudo pypy setup.py install).  The pypy version runs with no problems
but is 2.5 times slower than CPython.  From the performance stats, I believe
2.2.1 should be on average 6 times faster.  I presume it's my 'C' python
libraries which are the cause of the reduced performance.  What do I need to
do to make these private libraries pypy compatible?
> 

When you say you have Python wrappers, how have you written them? Generally
PyPy works best with Python interfaces written using CFFI[0].

Cory

[0]: https://cffi.readthedocs.org/en/latest/


From andy at pistuffing.co.uk  Wed Jul  1 16:00:02 2015
From: andy at pistuffing.co.uk (Andy Baker)
Date: Wed, 1 Jul 2015 15:00:02 +0100
Subject: [pypy-dev] pypy 2.2.1 performance on Raspian
References: <D60E225F181B4DFB8227F12665FEE5E7@Home>
 <0D40E7F9-8441-45AB-A815-EDEDD3281753@lukasa.co.uk> 
Message-ID: <9A23DD79E1594D85BF18CAC181FA0451@Home>

To clarify, these are python libraries, imported into the main python
script.  It's these libraries that are just python library API wrappers for
some underlyinig synchronous C code.  Had a quick browse a CFFI, but this
seems more related to call 'C' directly from a Python scripts rather than
'wrapping' it up as a python library.  Since these libraries are not mine,
I'd prefer to stay with the current model rather than discard the library
wrapper and use CFFI instead.

-----Original Message-----
From: Andy Baker [mailto:andy at pistuffing.co.uk] 
Sent: 01 July 2015 14:15
To: 'Cory Benfield'
Cc: 'pypy-dev at python.org'
Subject: RE: [pypy-dev] pypy 2.2.1 performance on Raspian

Hi Cory,

I didn't write the 'C' code or the Python wrappers myself but I have
modified the 'C' source code and added new python wrapper function - they
are for the Raspberry Pi RPiGPIO and RPIO hardware integration code.  The
"wrappers" are very thin layers defining to the Cpython interpreter (or pypy
JIT) how to map the Python call to the 'C' equvalent call along with
exceptions and control of the GIL.  There's no state stored within this
'wrapper' code - the 'C' code blocks returning OK, or an error which raises
a Python exception.  I'll look at CFFI to see if that sheds any light.

Andy

-----Original Message-----
From: Cory Benfield [mailto:cory at lukasa.co.uk]
Sent: 01 July 2015 13:48
To: Andy Baker
Cc: pypy-dev at python.org
Subject: Re: [pypy-dev] pypy 2.2.1 performance on Raspian


> On 1 Jul 2015, at 09:58, Andy Baker <andy at pistuffing.co.uk> wrote:
> 
> I am trying pypy as a performance increase over CPython.  My code uses
standard python libraries (not numpy), plus 2 'C' libraries accessing
Raspberry Pi GPIO hardware with python wrappers.  I have compiled both with
pypy (sudo pypy setup.py install).  The pypy version runs with no problems
but is 2.5 times slower than CPython.  From the performance stats, I believe
2.2.1 should be on average 6 times faster.  I presume it's my 'C' python
libraries which are the cause of the reduced performance.  What do I need to
do to make these private libraries pypy compatible?
> 

When you say you have Python wrappers, how have you written them? Generally
PyPy works best with Python interfaces written using CFFI[0].

Cory

[0]: https://cffi.readthedocs.org/en/latest/


From fijall at gmail.com  Wed Jul  1 16:02:08 2015
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 1 Jul 2015 16:02:08 +0200
Subject: [pypy-dev] pypy 2.2.1 performance on Raspian
In-Reply-To: <9A23DD79E1594D85BF18CAC181FA0451@Home>
References: <D60E225F181B4DFB8227F12665FEE5E7@Home>
 <0D40E7F9-8441-45AB-A815-EDEDD3281753@lukasa.co.uk>
 <9A23DD79E1594D85BF18CAC181FA0451@Home>
Message-ID: <CAK5idxQCngkqc6ByqkpVt8QNgXm4VRWXZk41OSLavKLRCEh6bw@mail.gmail.com>

do you have a link to those wrappers? Generally speaking using CPython
C API from PyPy is bound to be slow and there is not much we can do
about it (it's a miracle if it works at all)

On Wed, Jul 1, 2015 at 4:00 PM, Andy Baker <andy at pistuffing.co.uk> wrote:
> To clarify, these are python libraries, imported into the main python
> script.  It's these libraries that are just python library API wrappers for
> some underlyinig synchronous C code.  Had a quick browse a CFFI, but this
> seems more related to call 'C' directly from a Python scripts rather than
> 'wrapping' it up as a python library.  Since these libraries are not mine,
> I'd prefer to stay with the current model rather than discard the library
> wrapper and use CFFI instead.
>
> -----Original Message-----
> From: Andy Baker [mailto:andy at pistuffing.co.uk]
> Sent: 01 July 2015 14:15
> To: 'Cory Benfield'
> Cc: 'pypy-dev at python.org'
> Subject: RE: [pypy-dev] pypy 2.2.1 performance on Raspian
>
> Hi Cory,
>
> I didn't write the 'C' code or the Python wrappers myself but I have
> modified the 'C' source code and added new python wrapper function - they
> are for the Raspberry Pi RPiGPIO and RPIO hardware integration code.  The
> "wrappers" are very thin layers defining to the Cpython interpreter (or pypy
> JIT) how to map the Python call to the 'C' equvalent call along with
> exceptions and control of the GIL.  There's no state stored within this
> 'wrapper' code - the 'C' code blocks returning OK, or an error which raises
> a Python exception.  I'll look at CFFI to see if that sheds any light.
>
> Andy
>
> -----Original Message-----
> From: Cory Benfield [mailto:cory at lukasa.co.uk]
> Sent: 01 July 2015 13:48
> To: Andy Baker
> Cc: pypy-dev at python.org
> Subject: Re: [pypy-dev] pypy 2.2.1 performance on Raspian
>
>
>> On 1 Jul 2015, at 09:58, Andy Baker <andy at pistuffing.co.uk> wrote:
>>
>> I am trying pypy as a performance increase over CPython.  My code uses
> standard python libraries (not numpy), plus 2 'C' libraries accessing
> Raspberry Pi GPIO hardware with python wrappers.  I have compiled both with
> pypy (sudo pypy setup.py install).  The pypy version runs with no problems
> but is 2.5 times slower than CPython.  From the performance stats, I believe
> 2.2.1 should be on average 6 times faster.  I presume it's my 'C' python
> libraries which are the cause of the reduced performance.  What do I need to
> do to make these private libraries pypy compatible?
>>
>
> When you say you have Python wrappers, how have you written them? Generally
> PyPy works best with Python interfaces written using CFFI[0].
>
> Cory
>
> [0]: https://cffi.readthedocs.org/en/latest/
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev

From andy at pistuffing.co.uk  Wed Jul  1 16:32:30 2015
From: andy at pistuffing.co.uk (Andy Baker)
Date: Wed, 1 Jul 2015 15:32:30 +0100
Subject: [pypy-dev] pypy 2.2.1 performance on Raspian
In-Reply-To: <CAK5idxQCngkqc6ByqkpVt8QNgXm4VRWXZk41OSLavKLRCEh6bw@mail.gmail.com>
References: <D60E225F181B4DFB8227F12665FEE5E7@Home>
 <0D40E7F9-8441-45AB-A815-EDEDD3281753@lukasa.co.uk>
 <9A23DD79E1594D85BF18CAC181FA0451@Home>
 <CAK5idxQCngkqc6ByqkpVt8QNgXm4VRWXZk41OSLavKLRCEh6bw@mail.gmail.com>
Message-ID: <10D1DBD001724B7B93AD6951D4BB1953@Home>

Take a look at GPIO.tgz (174kB) here:
https://github.com/PiStuffing/Quadcopter/blob/master/GPIO.tgz.  Once
unpacked, the 'wrappers' are in source/py_*.

Thanks 
-----Original Message-----
From: Maciej Fijalkowski [mailto:fijall at gmail.com] 
Sent: 01 July 2015 15:02
To: Andy Baker
Cc: Cory Benfield; PyPy Developer Mailing List
Subject: Re: [pypy-dev] pypy 2.2.1 performance on Raspian

do you have a link to those wrappers? Generally speaking using CPython C API
from PyPy is bound to be slow and there is not much we can do about it (it's
a miracle if it works at all)

On Wed, Jul 1, 2015 at 4:00 PM, Andy Baker <andy at pistuffing.co.uk> wrote:
> To clarify, these are python libraries, imported into the main python 
> script.  It's these libraries that are just python library API 
> wrappers for some underlyinig synchronous C code.  Had a quick browse 
> a CFFI, but this seems more related to call 'C' directly from a Python 
> scripts rather than 'wrapping' it up as a python library.  Since these 
> libraries are not mine, I'd prefer to stay with the current model 
> rather than discard the library wrapper and use CFFI instead.
>
> -----Original Message-----
> From: Andy Baker [mailto:andy at pistuffing.co.uk]
> Sent: 01 July 2015 14:15
> To: 'Cory Benfield'
> Cc: 'pypy-dev at python.org'
> Subject: RE: [pypy-dev] pypy 2.2.1 performance on Raspian
>
> Hi Cory,
>
> I didn't write the 'C' code or the Python wrappers myself but I have 
> modified the 'C' source code and added new python wrapper function - 
> they are for the Raspberry Pi RPiGPIO and RPIO hardware integration 
> code.  The "wrappers" are very thin layers defining to the Cpython 
> interpreter (or pypy
> JIT) how to map the Python call to the 'C' equvalent call along with 
> exceptions and control of the GIL.  There's no state stored within 
> this 'wrapper' code - the 'C' code blocks returning OK, or an error 
> which raises a Python exception.  I'll look at CFFI to see if that sheds
any light.
>
> Andy
>
> -----Original Message-----
> From: Cory Benfield [mailto:cory at lukasa.co.uk]
> Sent: 01 July 2015 13:48
> To: Andy Baker
> Cc: pypy-dev at python.org
> Subject: Re: [pypy-dev] pypy 2.2.1 performance on Raspian
>
>
>> On 1 Jul 2015, at 09:58, Andy Baker <andy at pistuffing.co.uk> wrote:
>>
>> I am trying pypy as a performance increase over CPython.  My code 
>> uses
> standard python libraries (not numpy), plus 2 'C' libraries accessing 
> Raspberry Pi GPIO hardware with python wrappers.  I have compiled both 
> with pypy (sudo pypy setup.py install).  The pypy version runs with no 
> problems but is 2.5 times slower than CPython.  From the performance 
> stats, I believe
> 2.2.1 should be on average 6 times faster.  I presume it's my 'C' 
> python libraries which are the cause of the reduced performance.  What 
> do I need to do to make these private libraries pypy compatible?
>>
>
> When you say you have Python wrappers, how have you written them? 
> Generally PyPy works best with Python interfaces written using CFFI[0].
>
> Cory
>
> [0]: https://cffi.readthedocs.org/en/latest/
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev


From arigo at tunes.org  Wed Jul  1 16:33:03 2015
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 1 Jul 2015 16:33:03 +0200
Subject: [pypy-dev] pypy 2.2.1 performance on Raspian
In-Reply-To: <9A23DD79E1594D85BF18CAC181FA0451@Home>
References: <D60E225F181B4DFB8227F12665FEE5E7@Home>
 <0D40E7F9-8441-45AB-A815-EDEDD3281753@lukasa.co.uk>
 <9A23DD79E1594D85BF18CAC181FA0451@Home>
Message-ID: <CAMSv6X0V8CTbPh4Kw7nh2gH+1eja0Xw7NKDOLZqAAguU6rMX6A@mail.gmail.com>

Hi Andy,

Using the CPython C API is very slow on PyPy.  If your program is
mostly spending its time issuing calls via the CPython C API, then
getting "only" 2.5x slower is already good :-/  CFFI is much, much
faster on PyPy.  If you don't want to rewrite your CPython C API
modules (even though you said they are very thin layer, which means it
shouldn't be much work), then you're better off with CPython.

Additionally, make sure to read
http://pypy.readthedocs.org/en/latest/faq.html#how-fast-is-pypy and to
multiply the time estimates given there by 5 or 10: Raspberry Pi is
much slower than desktop PCs.


A bient?t,

Armin.

From andy at pistuffing.co.uk  Wed Jul  1 17:38:59 2015
From: andy at pistuffing.co.uk (Andy Baker)
Date: Wed, 1 Jul 2015 16:38:59 +0100
Subject: [pypy-dev] pypy 2.2.1 performance on Raspian
In-Reply-To: <CAMSv6X0V8CTbPh4Kw7nh2gH+1eja0Xw7NKDOLZqAAguU6rMX6A@mail.gmail.com>
References: <D60E225F181B4DFB8227F12665FEE5E7@Home>
 <0D40E7F9-8441-45AB-A815-EDEDD3281753@lukasa.co.uk>
 <9A23DD79E1594D85BF18CAC181FA0451@Home>
 <CAMSv6X0V8CTbPh4Kw7nh2gH+1eja0Xw7NKDOLZqAAguU6rMX6A@mail.gmail.com>
Message-ID: <2579D9F5197B42A7AAEEEE8E6C12C6CF@Home>

Hi Armin,

Thanks for the insight.  I'll have a look in more detail to port the 'C'
modules away from the CPython C API to CFFI.  The performance tests I was
doing were about 10000 loops of reading sensors (the Cpython C API
libraries) along with 1000 loops of arithmatic / trigonometry processing the
sensor data.  I guess the "only 2.5 times slower" is the balance between the
improved math(s) performance and the reduced sensor performance.

Thanks,

Andy 

-----Original Message-----
From: armin.rigo at gmail.com [mailto:armin.rigo at gmail.com] On Behalf Of Armin
Rigo
Sent: 01 July 2015 15:33
To: Andy Baker
Cc: Cory Benfield; PyPy Developer Mailing List
Subject: Re: [pypy-dev] pypy 2.2.1 performance on Raspian

Hi Andy,

Using the CPython C API is very slow on PyPy.  If your program is mostly
spending its time issuing calls via the CPython C API, then getting "only"
2.5x slower is already good :-/  CFFI is much, much faster on PyPy.  If you
don't want to rewrite your CPython C API modules (even though you said they
are very thin layer, which means it shouldn't be much work), then you're
better off with CPython.

Additionally, make sure to read
http://pypy.readthedocs.org/en/latest/faq.html#how-fast-is-pypy and to
multiply the time estimates given there by 5 or 10: Raspberry Pi is much
slower than desktop PCs.


A bient?t,

Armin.


From admalledd at gmail.com  Wed Jul  1 18:15:49 2015
From: admalledd at gmail.com (Eric Driggers)
Date: Wed, 1 Jul 2015 09:15:49 -0700
Subject: [pypy-dev] pypy 2.2.1 performance on Raspian
In-Reply-To: <2579D9F5197B42A7AAEEEE8E6C12C6CF@Home>
References: <D60E225F181B4DFB8227F12665FEE5E7@Home>
 <0D40E7F9-8441-45AB-A815-EDEDD3281753@lukasa.co.uk>
 <9A23DD79E1594D85BF18CAC181FA0451@Home>
 <CAMSv6X0V8CTbPh4Kw7nh2gH+1eja0Xw7NKDOLZqAAguU6rMX6A@mail.gmail.com>
 <2579D9F5197B42A7AAEEEE8E6C12C6CF@Home>
Message-ID: <CAK-ecOgexj6p7vWVpL+1_f8GR0LF4Y8Y+OBHMrNQ0UUrV99kzw@mail.gmail.com>

Unless I am missunderstanding, isn't pypy 2.2.1 rather old? Have you
tried a more up-to-date version first? (I seem to see current stable
is 2.6.0 and nightly is 2.7.0-alpha0) My memory is that between 2.3
and 2.4 there was some armhf work done, and that between 2.3 and 2.5 a
whole pile of stuff was done that improved pypy -> C (mostly to help
cffi, but helped CAPI and ctypes a bit too IIRC)

It is still probably a good idea to see about using cffi (and you can
use cffi on cpython as well!) but low hanging fruit and all that :D

On Wed, Jul 1, 2015 at 8:38 AM, Andy Baker <andy at pistuffing.co.uk> wrote:
> Hi Armin,
>
> Thanks for the insight.  I'll have a look in more detail to port the 'C'
> modules away from the CPython C API to CFFI.  The performance tests I was
> doing were about 10000 loops of reading sensors (the Cpython C API
> libraries) along with 1000 loops of arithmatic / trigonometry processing the
> sensor data.  I guess the "only 2.5 times slower" is the balance between the
> improved math(s) performance and the reduced sensor performance.
>
> Thanks,
>
> Andy
>
> -----Original Message-----
> From: armin.rigo at gmail.com [mailto:armin.rigo at gmail.com] On Behalf Of Armin
> Rigo
> Sent: 01 July 2015 15:33
> To: Andy Baker
> Cc: Cory Benfield; PyPy Developer Mailing List
> Subject: Re: [pypy-dev] pypy 2.2.1 performance on Raspian
>
> Hi Andy,
>
> Using the CPython C API is very slow on PyPy.  If your program is mostly
> spending its time issuing calls via the CPython C API, then getting "only"
> 2.5x slower is already good :-/  CFFI is much, much faster on PyPy.  If you
> don't want to rewrite your CPython C API modules (even though you said they
> are very thin layer, which means it shouldn't be much work), then you're
> better off with CPython.
>
> Additionally, make sure to read
> http://pypy.readthedocs.org/en/latest/faq.html#how-fast-is-pypy and to
> multiply the time estimates given there by 5 or 10: Raspberry Pi is much
> slower than desktop PCs.
>
>
> A bient?t,
>
> Armin.
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev

From andy at pistuffing.co.uk  Wed Jul  1 18:55:57 2015
From: andy at pistuffing.co.uk (Andy Baker)
Date: Wed, 1 Jul 2015 17:55:57 +0100
Subject: [pypy-dev] pypy 2.2.1 performance on Raspian
In-Reply-To: <CAK-ecOgexj6p7vWVpL+1_f8GR0LF4Y8Y+OBHMrNQ0UUrV99kzw@mail.gmail.com>
References: <D60E225F181B4DFB8227F12665FEE5E7@Home>
 <0D40E7F9-8441-45AB-A815-EDEDD3281753@lukasa.co.uk>
 <9A23DD79E1594D85BF18CAC181FA0451@Home>
 <CAMSv6X0V8CTbPh4Kw7nh2gH+1eja0Xw7NKDOLZqAAguU6rMX6A@mail.gmail.com>
 <2579D9F5197B42A7AAEEEE8E6C12C6CF@Home>
 <CAK-ecOgexj6p7vWVpL+1_f8GR0LF4Y8Y+OBHMrNQ0UUrV99kzw@mail.gmail.com>
Message-ID: <1EC33FF6A15B4DB193E8277671841E83@Home>

Hi Eric,

You're right - 2.2.1 is long-in-the-tooth but it's the version shipped in
the Raspian distribution.  I'll see if I can get the RPIF to upgrade the
distro to PyPy 2.6 and I'll give it a go myself too.

Andy  

-----Original Message-----
From: Eric Driggers [mailto:admalledd at gmail.com] 
Sent: 01 July 2015 17:16
To: Andy Baker
Cc: PyPy Developer Mailing List
Subject: Re: [pypy-dev] pypy 2.2.1 performance on Raspian

Unless I am missunderstanding, isn't pypy 2.2.1 rather old? Have you tried a
more up-to-date version first? (I seem to see current stable is 2.6.0 and
nightly is 2.7.0-alpha0) My memory is that between 2.3 and 2.4 there was
some armhf work done, and that between 2.3 and 2.5 a whole pile of stuff was
done that improved pypy -> C (mostly to help cffi, but helped CAPI and
ctypes a bit too IIRC)

It is still probably a good idea to see about using cffi (and you can use
cffi on cpython as well!) but low hanging fruit and all that :D

On Wed, Jul 1, 2015 at 8:38 AM, Andy Baker <andy at pistuffing.co.uk> wrote:
> Hi Armin,
>
> Thanks for the insight.  I'll have a look in more detail to port the 'C'
> modules away from the CPython C API to CFFI.  The performance tests I 
> was doing were about 10000 loops of reading sensors (the Cpython C API
> libraries) along with 1000 loops of arithmatic / trigonometry 
> processing the sensor data.  I guess the "only 2.5 times slower" is 
> the balance between the improved math(s) performance and the reduced
sensor performance.
>
> Thanks,
>
> Andy
>
> -----Original Message-----
> From: armin.rigo at gmail.com [mailto:armin.rigo at gmail.com] On Behalf Of 
> Armin Rigo
> Sent: 01 July 2015 15:33
> To: Andy Baker
> Cc: Cory Benfield; PyPy Developer Mailing List
> Subject: Re: [pypy-dev] pypy 2.2.1 performance on Raspian
>
> Hi Andy,
>
> Using the CPython C API is very slow on PyPy.  If your program is 
> mostly spending its time issuing calls via the CPython C API, then getting
"only"
> 2.5x slower is already good :-/  CFFI is much, much faster on PyPy.  
> If you don't want to rewrite your CPython C API modules (even though 
> you said they are very thin layer, which means it shouldn't be much 
> work), then you're better off with CPython.
>
> Additionally, make sure to read
> http://pypy.readthedocs.org/en/latest/faq.html#how-fast-is-pypy and to 
> multiply the time estimates given there by 5 or 10: Raspberry Pi is 
> much slower than desktop PCs.
>
>
> A bient?t,
>
> Armin.
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev


From fijall at gmail.com  Thu Jul  2 09:22:25 2015
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Thu, 2 Jul 2015 09:22:25 +0200
Subject: [pypy-dev] pypy 2.2.1 performance on Raspian
In-Reply-To: <1EC33FF6A15B4DB193E8277671841E83@Home>
References: <D60E225F181B4DFB8227F12665FEE5E7@Home>
 <0D40E7F9-8441-45AB-A815-EDEDD3281753@lukasa.co.uk>
 <9A23DD79E1594D85BF18CAC181FA0451@Home>
 <CAMSv6X0V8CTbPh4Kw7nh2gH+1eja0Xw7NKDOLZqAAguU6rMX6A@mail.gmail.com>
 <2579D9F5197B42A7AAEEEE8E6C12C6CF@Home>
 <CAK-ecOgexj6p7vWVpL+1_f8GR0LF4Y8Y+OBHMrNQ0UUrV99kzw@mail.gmail.com>
 <1EC33FF6A15B4DB193E8277671841E83@Home>
Message-ID: <CAK5idxQe1gaAhvPm4AAbLCt53fz=tLm+wuYz7UF_G38=hovr2w@mail.gmail.com>

for what is worth, you not gonna get better performance from C
extensions between PyPy 2.2 and 2.6

On Wed, Jul 1, 2015 at 6:55 PM, Andy Baker <andy at pistuffing.co.uk> wrote:
> Hi Eric,
>
> You're right - 2.2.1 is long-in-the-tooth but it's the version shipped in
> the Raspian distribution.  I'll see if I can get the RPIF to upgrade the
> distro to PyPy 2.6 and I'll give it a go myself too.
>
> Andy
>
> -----Original Message-----
> From: Eric Driggers [mailto:admalledd at gmail.com]
> Sent: 01 July 2015 17:16
> To: Andy Baker
> Cc: PyPy Developer Mailing List
> Subject: Re: [pypy-dev] pypy 2.2.1 performance on Raspian
>
> Unless I am missunderstanding, isn't pypy 2.2.1 rather old? Have you tried a
> more up-to-date version first? (I seem to see current stable is 2.6.0 and
> nightly is 2.7.0-alpha0) My memory is that between 2.3 and 2.4 there was
> some armhf work done, and that between 2.3 and 2.5 a whole pile of stuff was
> done that improved pypy -> C (mostly to help cffi, but helped CAPI and
> ctypes a bit too IIRC)
>
> It is still probably a good idea to see about using cffi (and you can use
> cffi on cpython as well!) but low hanging fruit and all that :D
>
> On Wed, Jul 1, 2015 at 8:38 AM, Andy Baker <andy at pistuffing.co.uk> wrote:
>> Hi Armin,
>>
>> Thanks for the insight.  I'll have a look in more detail to port the 'C'
>> modules away from the CPython C API to CFFI.  The performance tests I
>> was doing were about 10000 loops of reading sensors (the Cpython C API
>> libraries) along with 1000 loops of arithmatic / trigonometry
>> processing the sensor data.  I guess the "only 2.5 times slower" is
>> the balance between the improved math(s) performance and the reduced
> sensor performance.
>>
>> Thanks,
>>
>> Andy
>>
>> -----Original Message-----
>> From: armin.rigo at gmail.com [mailto:armin.rigo at gmail.com] On Behalf Of
>> Armin Rigo
>> Sent: 01 July 2015 15:33
>> To: Andy Baker
>> Cc: Cory Benfield; PyPy Developer Mailing List
>> Subject: Re: [pypy-dev] pypy 2.2.1 performance on Raspian
>>
>> Hi Andy,
>>
>> Using the CPython C API is very slow on PyPy.  If your program is
>> mostly spending its time issuing calls via the CPython C API, then getting
> "only"
>> 2.5x slower is already good :-/  CFFI is much, much faster on PyPy.
>> If you don't want to rewrite your CPython C API modules (even though
>> you said they are very thin layer, which means it shouldn't be much
>> work), then you're better off with CPython.
>>
>> Additionally, make sure to read
>> http://pypy.readthedocs.org/en/latest/faq.html#how-fast-is-pypy and to
>> multiply the time estimates given there by 5 or 10: Raspberry Pi is
>> much slower than desktop PCs.
>>
>>
>> A bient?t,
>>
>> Armin.
>>
>> _______________________________________________
>> pypy-dev mailing list
>> pypy-dev at python.org
>> https://mail.python.org/mailman/listinfo/pypy-dev
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev

From andy at pistuffing.co.uk  Thu Jul  2 10:30:06 2015
From: andy at pistuffing.co.uk (Andy Baker)
Date: Thu, 2 Jul 2015 09:30:06 +0100
Subject: [pypy-dev] pypy 2.2.1 performance on Raspian
In-Reply-To: <CAK5idxQe1gaAhvPm4AAbLCt53fz=tLm+wuYz7UF_G38=hovr2w@mail.gmail.com>
References: <D60E225F181B4DFB8227F12665FEE5E7@Home>
 <0D40E7F9-8441-45AB-A815-EDEDD3281753@lukasa.co.uk>
 <9A23DD79E1594D85BF18CAC181FA0451@Home>
 <CAMSv6X0V8CTbPh4Kw7nh2gH+1eja0Xw7NKDOLZqAAguU6rMX6A@mail.gmail.com>
 <2579D9F5197B42A7AAEEEE8E6C12C6CF@Home>
 <CAK-ecOgexj6p7vWVpL+1_f8GR0LF4Y8Y+OBHMrNQ0UUrV99kzw@mail.gmail.com>
 <1EC33FF6A15B4DB193E8277671841E83@Home>
 <CAK5idxQe1gaAhvPm4AAbLCt53fz=tLm+wuYz7UF_G38=hovr2w@mail.gmail.com>
Message-ID: <A8D4EDA2FA4040DA96A2DA4E35C167EC@Home>

OK, thanks - I'm going to try the Raspian CFFI smbus for pypy (already in
the main distro), and if that improves matters, then I'll convert my
libraries from the Cpython 'C' API to CFFI.  Thanks all.

-----Original Message-----
From: Maciej Fijalkowski [mailto:fijall at gmail.com] 
Sent: 02 July 2015 08:22
To: Andy Baker
Cc: Eric Driggers; PyPy Developer Mailing List
Subject: Re: [pypy-dev] pypy 2.2.1 performance on Raspian

for what is worth, you not gonna get better performance from C extensions
between PyPy 2.2 and 2.6

On Wed, Jul 1, 2015 at 6:55 PM, Andy Baker <andy at pistuffing.co.uk> wrote:
> Hi Eric,
>
> You're right - 2.2.1 is long-in-the-tooth but it's the version shipped 
> in the Raspian distribution.  I'll see if I can get the RPIF to 
> upgrade the distro to PyPy 2.6 and I'll give it a go myself too.
>
> Andy
>
> -----Original Message-----
> From: Eric Driggers [mailto:admalledd at gmail.com]
> Sent: 01 July 2015 17:16
> To: Andy Baker
> Cc: PyPy Developer Mailing List
> Subject: Re: [pypy-dev] pypy 2.2.1 performance on Raspian
>
> Unless I am missunderstanding, isn't pypy 2.2.1 rather old? Have you 
> tried a more up-to-date version first? (I seem to see current stable 
> is 2.6.0 and nightly is 2.7.0-alpha0) My memory is that between 2.3 
> and 2.4 there was some armhf work done, and that between 2.3 and 2.5 a 
> whole pile of stuff was done that improved pypy -> C (mostly to help 
> cffi, but helped CAPI and ctypes a bit too IIRC)
>
> It is still probably a good idea to see about using cffi (and you can 
> use cffi on cpython as well!) but low hanging fruit and all that :D
>
> On Wed, Jul 1, 2015 at 8:38 AM, Andy Baker <andy at pistuffing.co.uk> wrote:
>> Hi Armin,
>>
>> Thanks for the insight.  I'll have a look in more detail to port the 'C'
>> modules away from the CPython C API to CFFI.  The performance tests I 
>> was doing were about 10000 loops of reading sensors (the Cpython C 
>> API
>> libraries) along with 1000 loops of arithmatic / trigonometry 
>> processing the sensor data.  I guess the "only 2.5 times slower" is 
>> the balance between the improved math(s) performance and the reduced
> sensor performance.
>>
>> Thanks,
>>
>> Andy
>>
>> -----Original Message-----
>> From: armin.rigo at gmail.com [mailto:armin.rigo at gmail.com] On Behalf Of 
>> Armin Rigo
>> Sent: 01 July 2015 15:33
>> To: Andy Baker
>> Cc: Cory Benfield; PyPy Developer Mailing List
>> Subject: Re: [pypy-dev] pypy 2.2.1 performance on Raspian
>>
>> Hi Andy,
>>
>> Using the CPython C API is very slow on PyPy.  If your program is 
>> mostly spending its time issuing calls via the CPython C API, then 
>> getting
> "only"
>> 2.5x slower is already good :-/  CFFI is much, much faster on PyPy.
>> If you don't want to rewrite your CPython C API modules (even though 
>> you said they are very thin layer, which means it shouldn't be much 
>> work), then you're better off with CPython.
>>
>> Additionally, make sure to read
>> http://pypy.readthedocs.org/en/latest/faq.html#how-fast-is-pypy and 
>> to multiply the time estimates given there by 5 or 10: Raspberry Pi 
>> is much slower than desktop PCs.
>>
>>
>> A bient?t,
>>
>> Armin.
>>
>> _______________________________________________
>> pypy-dev mailing list
>> pypy-dev at python.org
>> https://mail.python.org/mailman/listinfo/pypy-dev
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev


From germanocarella.list at gmail.com  Sun Jul  5 13:44:05 2015
From: germanocarella.list at gmail.com (germano carella)
Date: Sun, 05 Jul 2015 13:44:05 +0200
Subject: [pypy-dev] problems with pypy and pip
Message-ID: <55991885.9030407@gmail.com>

Hi,
I downloaded pypy 2.6.0 and I ran pypy -m ensurepip. All worked correctly.
Now, in some cases, when i do pip install somepackage, it say could not 
find any download that satisfy requirements.
This even if packages apear in pip search command.
I tried wxPython, but i'm unable to install it...
In other cases, pip says that windows ssdk is not found. But I have 
visual studio 2013 installed and windows sdk is on my pc.
There are variables Can I set?
Thanks!

From matti.picus at gmail.com  Tue Jul  7 19:20:04 2015
From: matti.picus at gmail.com (Matti Picus)
Date: Tue, 07 Jul 2015 13:20:04 -0400
Subject: [pypy-dev] problems with pypy and pip
In-Reply-To: <55991885.9030407@gmail.com>
References: <55991885.9030407@gmail.com>
Message-ID: <559C0A44.7090505@gmail.com>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20150707/b2fa3ae3/attachment.html>

From hengha.mao at gmail.com  Wed Jul  8 11:09:37 2015
From: hengha.mao at gmail.com (Yicong Huang)
Date: Wed, 8 Jul 2015 17:09:37 +0800
Subject: [pypy-dev] cffi dlopen error: undefined symbol
Message-ID: <CAOEzsGB+SFJhZK_+VgcSxEEYneV9iEe2hTSEU8DaSc_DFAAeDg@mail.gmail.com>

Hi,

We tried to use cffi dlopen to load a C shared library.
However, we met the below error message:
Traceback (most recent call last):
  File "<builtin>/app_main.py", line 75, in run_toplevel
  File "distcache.py", line 19, in <module>
    _distcache = ffi.dlopen("./_distcache.so")
  File "/usr/ali/odps-pypy/lib_pypy/cffi/api.py", line 126, in dlopen
    lib, function_cache = _make_ffi_library(self, name, flags)
  File "/usr/ali/odps-pypy/lib_pypy/cffi/api.py", line 498, in
_make_ffi_library
    backendlib = _load_backend_lib(backend, libname, flags)
  File "/usr/ali/odps-pypy/lib_pypy/cffi/api.py", line 487, in
_load_backend_lib
    return backend.load_library(name, flags)
OSError: Cannot load library ./_distcache.so: ./_distcache.so: undefined
symbol: _ZTIN6apsara13ExceptionBaseE

Checking the library, there are not speical dependent libraries:
ldd _distcache.so
        linux-vdso.so.1 =>  (0x00007fff16fe5000)
        libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00007f7b60ba7000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f7b60924000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f7b60715000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f7b603bd000)
        /lib64/ld-linux-x86-64.so.2 (0x0000003fe9800000)

What might be the cause?

Thanks!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20150708/1cb44cfe/attachment.html>

From fijall at gmail.com  Wed Jul  8 11:14:38 2015
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 8 Jul 2015 11:14:38 +0200
Subject: [pypy-dev] cffi dlopen error: undefined symbol
In-Reply-To: <CAOEzsGB+SFJhZK_+VgcSxEEYneV9iEe2hTSEU8DaSc_DFAAeDg@mail.gmail.com>
References: <CAOEzsGB+SFJhZK_+VgcSxEEYneV9iEe2hTSEU8DaSc_DFAAeDg@mail.gmail.com>
Message-ID: <CAK5idxSs7UQxanTqoK-i_tCWFT9-1asJT9NziDcgnBWwBx8prQ@mail.gmail.com>

you should probably use verify instead of dlopen. This symbol is
likely defined in libstdc++, which is not loaded in pypy (you might
try to load it by hand, but I'm way out of my depth here)

On Wed, Jul 8, 2015 at 11:09 AM, Yicong Huang <hengha.mao at gmail.com> wrote:
> Hi,
>
> We tried to use cffi dlopen to load a C shared library.
> However, we met the below error message:
> Traceback (most recent call last):
>   File "<builtin>/app_main.py", line 75, in run_toplevel
>   File "distcache.py", line 19, in <module>
>     _distcache = ffi.dlopen("./_distcache.so")
>   File "/usr/ali/odps-pypy/lib_pypy/cffi/api.py", line 126, in dlopen
>     lib, function_cache = _make_ffi_library(self, name, flags)
>   File "/usr/ali/odps-pypy/lib_pypy/cffi/api.py", line 498, in
> _make_ffi_library
>     backendlib = _load_backend_lib(backend, libname, flags)
>   File "/usr/ali/odps-pypy/lib_pypy/cffi/api.py", line 487, in
> _load_backend_lib
>     return backend.load_library(name, flags)
> OSError: Cannot load library ./_distcache.so: ./_distcache.so: undefined
> symbol: _ZTIN6apsara13ExceptionBaseE
>
> Checking the library, there are not speical dependent libraries:
> ldd _distcache.so
>         linux-vdso.so.1 =>  (0x00007fff16fe5000)
>         libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00007f7b60ba7000)
>         libm.so.6 => /lib64/libm.so.6 (0x00007f7b60924000)
>         libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f7b60715000)
>         libc.so.6 => /lib64/libc.so.6 (0x00007f7b603bd000)
>         /lib64/ld-linux-x86-64.so.2 (0x0000003fe9800000)
>
> What might be the cause?
>
> Thanks!
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>

From arigo at tunes.org  Wed Jul  8 11:18:34 2015
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 8 Jul 2015 11:18:34 +0200
Subject: [pypy-dev] cffi dlopen error: undefined symbol
In-Reply-To: <CAK5idxSs7UQxanTqoK-i_tCWFT9-1asJT9NziDcgnBWwBx8prQ@mail.gmail.com>
References: <CAOEzsGB+SFJhZK_+VgcSxEEYneV9iEe2hTSEU8DaSc_DFAAeDg@mail.gmail.com>
 <CAK5idxSs7UQxanTqoK-i_tCWFT9-1asJT9NziDcgnBWwBx8prQ@mail.gmail.com>
Message-ID: <CAMSv6X1bYO+hDJxEGY=L6Utzrtttb6E2PQyEcvfYwi_ynmmXWg@mail.gmail.com>

Hi Maciej,

On 8 July 2015 at 11:14, Maciej Fijalkowski <fijall at gmail.com> wrote:
> you should probably use verify instead of dlopen. This symbol is
> likely defined in libstdc++, which is not loaded in pypy (you might
> try to load it by hand, but I'm way out of my depth here)

http://demangler.com/ says the symbol is apsara::ExceptionBase, which
doesn't look like it comes from libstdc++.  That's about all I can say
though.  The point is that "./_distcache.so" looks like it is missing
this symbol and can't be loaded (independently of cffi --- try to load
it for example in a small C program that calls dlopen()).

Obviously I agree with Maciej's comment and you should use verify()
instead of dlopen(), particularly if _distcache.so is full of C++
names, because these can't be cleanly accessed with dlopen().


Armin

From amauryfa at gmail.com  Wed Jul  8 11:20:02 2015
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Wed, 8 Jul 2015 11:20:02 +0200
Subject: [pypy-dev] problems with pypy and pip
In-Reply-To: <559C0A44.7090505@gmail.com>
References: <55991885.9030407@gmail.com>
	<559C0A44.7090505@gmail.com>
Message-ID: <CAGmFidb9bnnx1KWjqSp7NO4zeFcJKHUnKgR60LH2ENt9MdDL0w@mail.gmail.com>

2015-07-07 19:20 GMT+02:00 Matti Picus <matti.picus at gmail.com>:

>  This is all documented in our FAQ
> http://pypy.readthedocs.org/en/latest/faq.html#do-cpython-extension-modules-work-with-pypy
>
> Some modules, like wxPython, that use the python c API or ctypes will
> either not work with PyPy or will be much slower than cPython. We highly
> recommend cffi as an alternative foreign function interface, since it is
> nicer than ctypes and is fast on PyPy, but it may be a while until module
> authors see the light of the one true way.
>

Old versions of wxPython ("classic") used SWIG to wrap the C++ libraries,
and could be compiled with PyPy. It worked, but it was a long time ago, was
the result was quite slow...
http://morepypy.blogspot.fr/2010/05/running-wxpython-on-top-of-pypy.html
(unfortunately the screenshots are lost)

Newer versions of wxPython ("Phoenix") use SIP to wrap the C++ libraries,
and there is no way it can be supported directly by PyPy (SIP uses
metaclasses with custom tp_alloc slots to subclass int... I tried).

There was an attempt to port have SIP file generate a cffi interface, it
did not go to completion but looks very promising.
There is a nice story here: http://waedt.blogspot.fr/
It seems that all basic infrastructure are there, but many widgets extend
SIP wrappers and use the C API: they need to be rewritten in a
cffi-friendly fashion.

Good luck with wxPython.


>
> The wiki page https://bitbucket.org/pypy/compatibility/wiki/Home may have
> some hints about module compatibility, note that your mileage may vary and
> updates are welcome.
>
> Also, you should be using visual studio 2008 (for both PyPy 2.6.0 and
> cPython 2.7), using this link should get you started
> https://www.microsoft.com/en-us/download/details.aspx?id=44266
> Matti
>
>
>
> On 05/07/15 07:44, germano carella wrote:
>
> Hi,
> I downloaded pypy 2.6.0 and I ran pypy -m ensurepip. All worked correctly.
> Now, in some cases, when i do pip install somepackage, it say could not
> find any download that satisfy requirements.
> This even if packages apear in pip search command.
> I tried wxPython, but i'm unable to install it...
> In other cases, pip says that windows ssdk is not found. But I have visual
> studio 2013 installed and windows sdk is on my pc.
> There are variables Can I set?
> Thanks!
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>
>
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>
>


-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20150708/32b84a5f/attachment-0001.html>

From hengha.mao at gmail.com  Wed Jul  8 12:03:36 2015
From: hengha.mao at gmail.com (Yicong Huang)
Date: Wed, 8 Jul 2015 18:03:36 +0800
Subject: [pypy-dev] cffi dlopen error: undefined symbol
In-Reply-To: <CAMSv6X1bYO+hDJxEGY=L6Utzrtttb6E2PQyEcvfYwi_ynmmXWg@mail.gmail.com>
References: <CAOEzsGB+SFJhZK_+VgcSxEEYneV9iEe2hTSEU8DaSc_DFAAeDg@mail.gmail.com>
 <CAK5idxSs7UQxanTqoK-i_tCWFT9-1asJT9NziDcgnBWwBx8prQ@mail.gmail.com>
 <CAMSv6X1bYO+hDJxEGY=L6Utzrtttb6E2PQyEcvfYwi_ynmmXWg@mail.gmail.com>
Message-ID: <CAOEzsGAFZ2XzYgOACiucfAYFTeghVO8WNWx+CR36oGZkxWnzqg@mail.gmail.com>

How to use ffi.verfiy() to open a shared library?
I tried a C code by using dlopen, but met the same errors.

void *handle;
handle = dlopen("./_distcache.so", RTLD_LAZY);
    if (!handle) {
        fprintf(stderr, "%s\n", dlerror());
        exit(EXIT_FAILURE);
}

With C code, it also reported that " undefined symbol:
_ZTIN6apsara13ExceptionBaseE".


On Wed, Jul 8, 2015 at 5:18 PM, Armin Rigo <arigo at tunes.org> wrote:

> Hi Maciej,
>
> On 8 July 2015 at 11:14, Maciej Fijalkowski <fijall at gmail.com> wrote:
> > you should probably use verify instead of dlopen. This symbol is
> > likely defined in libstdc++, which is not loaded in pypy (you might
> > try to load it by hand, but I'm way out of my depth here)
>
> http://demangler.com/ says the symbol is apsara::ExceptionBase, which
> doesn't look like it comes from libstdc++.  That's about all I can say
> though.  The point is that "./_distcache.so" looks like it is missing
> this symbol and can't be loaded (independently of cffi --- try to load
> it for example in a small C program that calls dlopen()).
>
> Obviously I agree with Maciej's comment and you should use verify()
> instead of dlopen(), particularly if _distcache.so is full of C++
> names, because these can't be cleanly accessed with dlopen().
>
>
> Armin
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20150708/67c95748/attachment.html>

From arigo at tunes.org  Wed Jul  8 17:25:41 2015
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 8 Jul 2015 17:25:41 +0200
Subject: [pypy-dev] cffi dlopen error: undefined symbol
In-Reply-To: <CAOEzsGAFZ2XzYgOACiucfAYFTeghVO8WNWx+CR36oGZkxWnzqg@mail.gmail.com>
References: <CAOEzsGB+SFJhZK_+VgcSxEEYneV9iEe2hTSEU8DaSc_DFAAeDg@mail.gmail.com>
 <CAK5idxSs7UQxanTqoK-i_tCWFT9-1asJT9NziDcgnBWwBx8prQ@mail.gmail.com>
 <CAMSv6X1bYO+hDJxEGY=L6Utzrtttb6E2PQyEcvfYwi_ynmmXWg@mail.gmail.com>
 <CAOEzsGAFZ2XzYgOACiucfAYFTeghVO8WNWx+CR36oGZkxWnzqg@mail.gmail.com>
Message-ID: <CAMSv6X2iuUn5_21MBYOP1qB3Lz13dU8-tvnf_uzmRm9sd=iEhQ@mail.gmail.com>

Hi Yicong,

On 8 July 2015 at 12:03, Yicong Huang <hengha.mao at gmail.com> wrote:
> With C code, it also reported that " undefined symbol:
> _ZTIN6apsara13ExceptionBaseE".

It seems that "_distcache" cannot be used as you compiled it.  The
problem is not related to cffi, nor PyPy nor Python.  You should look
elsewhere for help about that.  If you didn't make "_distcache"
yourself, ask whoever made it.  If you did make it yourself, ask for
example on stackoverflow.com by describing the problem you get with
dlopen().  (You may need to give much more details to get meaningful
answers.)


A bient?t,

Armin.

From matti.picus at gmail.com  Wed Jul  8 21:17:27 2015
From: matti.picus at gmail.com (Matti Picus)
Date: Wed, 08 Jul 2015 22:17:27 +0300
Subject: [pypy-dev] win32 buildslave
Message-ID: <559D7747.4000300@gmail.com>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20150708/190abcf6/attachment.html>

From yury at shurup.com  Wed Jul  8 21:42:41 2015
From: yury at shurup.com (Yury V. Zaytsev)
Date: Wed, 08 Jul 2015 21:42:41 +0200
Subject: [pypy-dev] win32 buildslave
In-Reply-To: <559D7747.4000300@gmail.com>
References: <559D7747.4000300@gmail.com>
Message-ID: <1436384561.3043.391.camel@newpride>

On Wed, 2015-07-08 at 22:17 +0300, Matti Picus wrote:
> 
> Hi Yury. The win32 buildslave anubis64 seems down, is there a chance
> you could get it working again? 

Hi,

Whoops, it seems that there was a power outage due to a storm on Monday
night of which I wasn't notified, and since the relevant virtual
machines are configured to *not* start automatically on boot, the build
slave didn't come up after the power returned to the building...

I hope that everything should be fine again now, and I'm sorry for the
inconvenience. Please do ping me if you notice that something weird is
happening to the build slave in the future. Unfortunately, I'm not able
to monitor it 24/7 (and not even daily), but if I have access to email
at all, I'm always trying to give such problems priority ;-)

-- 
Sincerely yours,
Yury V. Zaytsev



From yury at shurup.com  Wed Jul  8 21:53:49 2015
From: yury at shurup.com (Yury V. Zaytsev)
Date: Wed, 08 Jul 2015 21:53:49 +0200
Subject: [pypy-dev] cffi dlopen error: undefined symbol
In-Reply-To: <CAOEzsGAFZ2XzYgOACiucfAYFTeghVO8WNWx+CR36oGZkxWnzqg@mail.gmail.com>
References: <CAOEzsGB+SFJhZK_+VgcSxEEYneV9iEe2hTSEU8DaSc_DFAAeDg@mail.gmail.com>
 <CAK5idxSs7UQxanTqoK-i_tCWFT9-1asJT9NziDcgnBWwBx8prQ@mail.gmail.com>
 <CAMSv6X1bYO+hDJxEGY=L6Utzrtttb6E2PQyEcvfYwi_ynmmXWg@mail.gmail.com>
 <CAOEzsGAFZ2XzYgOACiucfAYFTeghVO8WNWx+CR36oGZkxWnzqg@mail.gmail.com>
Message-ID: <1436385229.3043.395.camel@newpride>

On Wed, 2015-07-08 at 18:03 +0800, Yicong Huang wrote:
> 
> With C code, it also reported that " undefined symbol:
> _ZTIN6apsara13ExceptionBaseE". 

You should link distcache.so to the aspara library if you use any of its
symbols. For the details, you have to talk to your local specialist on
Alibaba internal stuff ;-)

-- 
Sincerely yours,
Yury V. Zaytsev



From hengha.mao at gmail.com  Thu Jul  9 06:15:40 2015
From: hengha.mao at gmail.com (Yicong Huang)
Date: Thu, 9 Jul 2015 12:15:40 +0800
Subject: [pypy-dev] cffi dlopen error: undefined symbol
In-Reply-To: <1436385229.3043.395.camel@newpride>
References: <CAOEzsGB+SFJhZK_+VgcSxEEYneV9iEe2hTSEU8DaSc_DFAAeDg@mail.gmail.com>
 <CAK5idxSs7UQxanTqoK-i_tCWFT9-1asJT9NziDcgnBWwBx8prQ@mail.gmail.com>
 <CAMSv6X1bYO+hDJxEGY=L6Utzrtttb6E2PQyEcvfYwi_ynmmXWg@mail.gmail.com>
 <CAOEzsGAFZ2XzYgOACiucfAYFTeghVO8WNWx+CR36oGZkxWnzqg@mail.gmail.com>
 <1436385229.3043.395.camel@newpride>
Message-ID: <CAOEzsGCLjKedL9Bjc=P70VLcHFdmjgwMCyK4R4gbQcrc2ncGmA@mail.gmail.com>

Great thanks for all your help!
We found out the lib was lack of linking with some necessary libs.
By adding those depdent libs, dlopen is able to open the lib.

On Thu, Jul 9, 2015 at 3:53 AM, Yury V. Zaytsev <yury at shurup.com> wrote:

> On Wed, 2015-07-08 at 18:03 +0800, Yicong Huang wrote:
> >
> > With C code, it also reported that " undefined symbol:
> > _ZTIN6apsara13ExceptionBaseE".
>
> You should link distcache.so to the aspara library if you use any of its
> symbols. For the details, you have to talk to your local specialist on
> Alibaba internal stuff ;-)
>
> --
> Sincerely yours,
> Yury V. Zaytsev
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20150709/22ff086a/attachment-0001.html>

From fijall at gmail.com  Sat Jul 11 20:52:03 2015
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sat, 11 Jul 2015 20:52:03 +0200
Subject: [pypy-dev] Docker images for "try pypy"
Message-ID: <CAK5idxT_BMP223ud19tKe74Gsp4H+sX_DCAxktCc-Q+7uWG+8A@mail.gmail.com>

Hi

Would anyone be willing to make some docker images that can be used to
"try pypy"?

Cheers,
fijal

From romain.py at gmail.com  Sat Jul 11 21:15:06 2015
From: romain.py at gmail.com (Romain Guillebert)
Date: Sat, 11 Jul 2015 20:15:06 +0100
Subject: [pypy-dev] Docker images for "try pypy"
In-Reply-To: <CAK5idxT_BMP223ud19tKe74Gsp4H+sX_DCAxktCc-Q+7uWG+8A@mail.gmail.com>
References: <CAK5idxT_BMP223ud19tKe74Gsp4H+sX_DCAxktCc-Q+7uWG+8A@mail.gmail.com>
Message-ID: <CANaNaZXZurniBnHAL-t8C+Rf-k9GPXi3Vk1g5XwJFrJ8bN4xTw@mail.gmail.com>

Hi Maciej

Isn't this : https://registry.hub.docker.com/_/pypy/ what you're looking
for?

Cheers
Romain
On 11 Jul 2015 19:52, "Maciej Fijalkowski" <fijall at gmail.com> wrote:

> Hi
>
> Would anyone be willing to make some docker images that can be used to
> "try pypy"?
>
> Cheers,
> fijal
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20150711/5e253326/attachment.html>

From fijall at gmail.com  Sat Jul 11 22:51:23 2015
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sat, 11 Jul 2015 22:51:23 +0200
Subject: [pypy-dev] Docker images for "try pypy"
In-Reply-To: <CANaNaZXZurniBnHAL-t8C+Rf-k9GPXi3Vk1g5XwJFrJ8bN4xTw@mail.gmail.com>
References: <CAK5idxT_BMP223ud19tKe74Gsp4H+sX_DCAxktCc-Q+7uWG+8A@mail.gmail.com>
 <CANaNaZXZurniBnHAL-t8C+Rf-k9GPXi3Vk1g5XwJFrJ8bN4xTw@mail.gmail.com>
Message-ID: <CAK5idxSTvZv3yU8i2TT6m=JDzRkVm42rxWDhup4OCrp8SQRWBQ@mail.gmail.com>

I don't know, maybe it is :-)

/me investigates

On Sat, Jul 11, 2015 at 9:15 PM, Romain Guillebert <romain.py at gmail.com> wrote:
> Hi Maciej
>
> Isn't this : https://registry.hub.docker.com/_/pypy/ what you're looking
> for?
>
> Cheers
> Romain
>
> On 11 Jul 2015 19:52, "Maciej Fijalkowski" <fijall at gmail.com> wrote:
>>
>> Hi
>>
>> Would anyone be willing to make some docker images that can be used to
>> "try pypy"?
>>
>> Cheers,
>> fijal
>> _______________________________________________
>> pypy-dev mailing list
>> pypy-dev at python.org
>> https://mail.python.org/mailman/listinfo/pypy-dev

From hengha.mao at gmail.com  Wed Jul 15 09:59:17 2015
From: hengha.mao at gmail.com (Yicong Huang)
Date: Wed, 15 Jul 2015 15:59:17 +0800
Subject: [pypy-dev] Are there any methods to view pypy gc log?
Message-ID: <CAOEzsGA5jtj5wXFUf4ps06atfqpyzQa9fRsCE+5xgs1E1rO3yw@mail.gmail.com>

>From the document, we saw there are some GC tuning parameters.
But we've no ideas how to evaluate the tuning.
As for java, it is able to output friendly gc log.
Are there any similar output avaiable in pypy?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20150715/3367e936/attachment.html>

From fijall at gmail.com  Wed Jul 15 10:03:19 2015
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 15 Jul 2015 10:03:19 +0200
Subject: [pypy-dev] Are there any methods to view pypy gc log?
In-Reply-To: <CAOEzsGA5jtj5wXFUf4ps06atfqpyzQa9fRsCE+5xgs1E1rO3yw@mail.gmail.com>
References: <CAOEzsGA5jtj5wXFUf4ps06atfqpyzQa9fRsCE+5xgs1E1rO3yw@mail.gmail.com>
Message-ID: <CAK5idxTr+w29uyG+ccWwHz_=C=n4SuT95+Qyota6LLhku_0L=A@mail.gmail.com>

PYPYLOG=gc:- would give you some idea, but nothing that can help you
tune the GC. We usually just measure total time with various GC
parameters

On Wed, Jul 15, 2015 at 9:59 AM, Yicong Huang <hengha.mao at gmail.com> wrote:
> From the document, we saw there are some GC tuning parameters.
> But we've no ideas how to evaluate the tuning.
> As for java, it is able to output friendly gc log.
> Are there any similar output avaiable in pypy?
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>

From hengha.mao at gmail.com  Wed Jul 15 10:42:17 2015
From: hengha.mao at gmail.com (Yicong Huang)
Date: Wed, 15 Jul 2015 16:42:17 +0800
Subject: [pypy-dev] Are there any methods to view pypy gc log?
In-Reply-To: <CAK5idxTr+w29uyG+ccWwHz_=C=n4SuT95+Qyota6LLhku_0L=A@mail.gmail.com>
References: <CAOEzsGA5jtj5wXFUf4ps06atfqpyzQa9fRsCE+5xgs1E1rO3yw@mail.gmail.com>
 <CAK5idxTr+w29uyG+ccWwHz_=C=n4SuT95+Qyota6LLhku_0L=A@mail.gmail.com>
Message-ID: <CAOEzsGAYuaD88gMrp=d2evTF+ssoMYpa1_3Vhyy_nGrViR6j8w@mail.gmail.com>

Great thanks!
We tried gc log, and got a piece of below output.
However, it is hard to read.
Is it possible to get some common metrics, e.g. timestamp of GC occourred,
gc paused time, gc count?
We found there is a tool gcanalyze.py might help. But the tool depends on
rpython module.
How to run the tool?

starting gc state:  SCANNING
stopping, now in gc state:  MARKING
[1cca56e7e5d61f] gc-collect-step}
[1cca56e7e63793] {gc-collect-step
starting gc state:  MARKING
number of objects to mark 158 plus 59
stopping, now in gc state:  SWEEPING
[1cca56e820f113] gc-collect-step}
[1cca56e8218863] {gc-collect-step
starting gc state:  SWEEPING


On Wed, Jul 15, 2015 at 4:03 PM, Maciej Fijalkowski <fijall at gmail.com>
wrote:

> PYPYLOG=gc:- would give you some idea, but nothing that can help you
> tune the GC. We usually just measure total time with various GC
> parameters
>
> On Wed, Jul 15, 2015 at 9:59 AM, Yicong Huang <hengha.mao at gmail.com>
> wrote:
> > From the document, we saw there are some GC tuning parameters.
> > But we've no ideas how to evaluate the tuning.
> > As for java, it is able to output friendly gc log.
> > Are there any similar output avaiable in pypy?
> >
> > _______________________________________________
> > pypy-dev mailing list
> > pypy-dev at python.org
> > https://mail.python.org/mailman/listinfo/pypy-dev
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20150715/2672dd79/attachment.html>

From fijall at gmail.com  Wed Jul 15 10:47:56 2015
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 15 Jul 2015 10:47:56 +0200
Subject: [pypy-dev] Are there any methods to view pypy gc log?
In-Reply-To: <CAOEzsGAYuaD88gMrp=d2evTF+ssoMYpa1_3Vhyy_nGrViR6j8w@mail.gmail.com>
References: <CAOEzsGA5jtj5wXFUf4ps06atfqpyzQa9fRsCE+5xgs1E1rO3yw@mail.gmail.com>
 <CAK5idxTr+w29uyG+ccWwHz_=C=n4SuT95+Qyota6LLhku_0L=A@mail.gmail.com>
 <CAOEzsGAYuaD88gMrp=d2evTF+ssoMYpa1_3Vhyy_nGrViR6j8w@mail.gmail.com>
Message-ID: <CAK5idxQpqk-1jtB_xVW8i1G-cy+3c6pMzHriPoTc721E62+9pQ@mail.gmail.com>

so in the part you pasted, the hex numbers are timestamps (I think
they're hex-encoded cycle counts), the numbers you have to fish from
text, but that's enough to say when it started and when it ended. if
you run PYPYLOG=log, you can use logparser to display total time spent
in GC (they all require rpython, you need a pypy checkout and
PYTHONPATH)

On Wed, Jul 15, 2015 at 10:42 AM, Yicong Huang <hengha.mao at gmail.com> wrote:
> Great thanks!
> We tried gc log, and got a piece of below output.
> However, it is hard to read.
> Is it possible to get some common metrics, e.g. timestamp of GC occourred,
> gc paused time, gc count?
> We found there is a tool gcanalyze.py might help. But the tool depends on
> rpython module.
> How to run the tool?
>
> starting gc state:  SCANNING
> stopping, now in gc state:  MARKING
> [1cca56e7e5d61f] gc-collect-step}
> [1cca56e7e63793] {gc-collect-step
> starting gc state:  MARKING
> number of objects to mark 158 plus 59
> stopping, now in gc state:  SWEEPING
> [1cca56e820f113] gc-collect-step}
> [1cca56e8218863] {gc-collect-step
> starting gc state:  SWEEPING
>
>
> On Wed, Jul 15, 2015 at 4:03 PM, Maciej Fijalkowski <fijall at gmail.com>
> wrote:
>>
>> PYPYLOG=gc:- would give you some idea, but nothing that can help you
>> tune the GC. We usually just measure total time with various GC
>> parameters
>>
>> On Wed, Jul 15, 2015 at 9:59 AM, Yicong Huang <hengha.mao at gmail.com>
>> wrote:
>> > From the document, we saw there are some GC tuning parameters.
>> > But we've no ideas how to evaluate the tuning.
>> > As for java, it is able to output friendly gc log.
>> > Are there any similar output avaiable in pypy?
>> >
>> > _______________________________________________
>> > pypy-dev mailing list
>> > pypy-dev at python.org
>> > https://mail.python.org/mailman/listinfo/pypy-dev
>> >
>
>

From rymg19 at gmail.com  Wed Jul 15 16:04:07 2015
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Wed, 15 Jul 2015 09:04:07 -0500
Subject: [pypy-dev] Are there any methods to view pypy gc log?
In-Reply-To: <CAOEzsGAYuaD88gMrp=d2evTF+ssoMYpa1_3Vhyy_nGrViR6j8w@mail.gmail.com>
References: <CAOEzsGA5jtj5wXFUf4ps06atfqpyzQa9fRsCE+5xgs1E1rO3yw@mail.gmail.com>
 <CAK5idxTr+w29uyG+ccWwHz_=C=n4SuT95+Qyota6LLhku_0L=A@mail.gmail.com>
 <CAOEzsGAYuaD88gMrp=d2evTF+ssoMYpa1_3Vhyy_nGrViR6j8w@mail.gmail.com>
Message-ID: <5E0A2C0C-B9E5-473B-8951-71529A76122F@gmail.com>



On July 15, 2015 3:42:17 AM CDT, Yicong Huang <hengha.mao at gmail.com> wrote:
>Great thanks!
>We tried gc log, and got a piece of below output.
>However, it is hard to read.
>Is it possible to get some common metrics, e.g. timestamp of GC
>occourred,
>gc paused time, gc count?
>We found there is a tool gcanalyze.py might help. But the tool depends
>on
>rpython module.
>How to run the tool?

Grab the PyPy source. The rpython module is inside it.

>
>starting gc state:  SCANNING
>stopping, now in gc state:  MARKING
>[1cca56e7e5d61f] gc-collect-step}
>[1cca56e7e63793] {gc-collect-step
>starting gc state:  MARKING
>number of objects to mark 158 plus 59
>stopping, now in gc state:  SWEEPING
>[1cca56e820f113] gc-collect-step}
>[1cca56e8218863] {gc-collect-step
>starting gc state:  SWEEPING
>
>
>On Wed, Jul 15, 2015 at 4:03 PM, Maciej Fijalkowski <fijall at gmail.com>
>wrote:
>
>> PYPYLOG=gc:- would give you some idea, but nothing that can help you
>> tune the GC. We usually just measure total time with various GC
>> parameters
>>
>> On Wed, Jul 15, 2015 at 9:59 AM, Yicong Huang <hengha.mao at gmail.com>
>> wrote:
>> > From the document, we saw there are some GC tuning parameters.
>> > But we've no ideas how to evaluate the tuning.
>> > As for java, it is able to output friendly gc log.
>> > Are there any similar output avaiable in pypy?
>> >
>> > _______________________________________________
>> > pypy-dev mailing list
>> > pypy-dev at python.org
>> > https://mail.python.org/mailman/listinfo/pypy-dev
>> >
>>
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>pypy-dev mailing list
>pypy-dev at python.org
>https://mail.python.org/mailman/listinfo/pypy-dev

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

From n210241048576 at gmail.com  Thu Jul 16 18:21:26 2015
From: n210241048576 at gmail.com (Robert Grosse)
Date: Thu, 16 Jul 2015 09:21:26 -0700
Subject: [pypy-dev] Using Pypy as a zygote process to avoid jit warmup
Message-ID: <CALJjGis+w-imKjm26BAYd=JHumwrc8gnK9Jy=_dzbDBB4TwqEw@mail.gmail.com>

One of the main problems with a JIT as opposed to AOT compilation is the
warm-up overhead. I thought that you could achieve a similar effect as AOT
by starting up a program in Pypy, running it on a hardcoded workload to
warm it up, and then pause the process. Then when you want to run the
program, you fork the already warmed up process and run it on your desired
input.

Is this something that is possible to do with Pypy?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20150716/8c131d65/attachment.html>

From fijall at gmail.com  Fri Jul 17 10:57:26 2015
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Fri, 17 Jul 2015 10:57:26 +0200
Subject: [pypy-dev] Using Pypy as a zygote process to avoid jit warmup
In-Reply-To: <CALJjGis+w-imKjm26BAYd=JHumwrc8gnK9Jy=_dzbDBB4TwqEw@mail.gmail.com>
References: <CALJjGis+w-imKjm26BAYd=JHumwrc8gnK9Jy=_dzbDBB4TwqEw@mail.gmail.com>
Message-ID: <CAK5idxTGM18Dc+wOGw3MNksmkbmLhb+xbw5MX9N7cJQ7oW3mMQ@mail.gmail.com>

Hi Robert.

Generally keeping pypy as a daemon and then running small workloads
sounds like a good idea. The forking part I don't like because while
it might mitigate some warmup, it's much harder to get right. You
don't quite know how to push enough calls to JIT the correct paths
etc.. However, if you don't fork you're responsible for cleaning up
your own state (so e.g. any imports).

Cheers,
fijal


On Thu, Jul 16, 2015 at 6:21 PM, Robert Grosse <n210241048576 at gmail.com> wrote:
> One of the main problems with a JIT as opposed to AOT compilation is the
> warm-up overhead. I thought that you could achieve a similar effect as AOT
> by starting up a program in Pypy, running it on a hardcoded workload to warm
> it up, and then pause the process. Then when you want to run the program,
> you fork the already warmed up process and run it on your desired input.
>
> Is this something that is possible to do with Pypy?
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>

From fijall at gmail.com  Fri Jul 17 10:59:22 2015
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Fri, 17 Jul 2015 10:59:22 +0200
Subject: [pypy-dev] Increase the default stack limit
Message-ID: <CAK5idxTaJnDhHOX8bV7uKRWsA2i-cFMii3pcs9riy782PV=KwA@mail.gmail.com>

Hi

The default stack limit is (conservatively) set at 700k or so. Can we
push it to ~3M by default? (this is still far below 8M or so which is
the default on linux)

From phyo.arkarlwin at gmail.com  Sat Jul 18 23:20:10 2015
From: phyo.arkarlwin at gmail.com (Phyo Arkar)
Date: Sun, 19 Jul 2015 03:50:10 +0630
Subject: [pypy-dev] Increase the default stack limit
In-Reply-To: <CAK5idxTaJnDhHOX8bV7uKRWsA2i-cFMii3pcs9riy782PV=KwA@mail.gmail.com>
References: <CAK5idxTaJnDhHOX8bV7uKRWsA2i-cFMii3pcs9riy782PV=KwA@mail.gmail.com>
Message-ID: <CA+HjJrgAGsLnQQjkV54u4Xv_cJN9JuHuuwvNvzQcn_9kPnXQbA@mail.gmail.com>

On Fri, Jul 17, 2015 at 3:29 PM, Maciej Fijalkowski <fijall at gmail.com>
wrote:

> stack limit


That will help alot with recursions?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20150719/8e6fd7b5/attachment.html>

From fijall at gmail.com  Sun Jul 19 06:38:22 2015
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sun, 19 Jul 2015 06:38:22 +0200
Subject: [pypy-dev] Increase the default stack limit
In-Reply-To: <CA+HjJrgAGsLnQQjkV54u4Xv_cJN9JuHuuwvNvzQcn_9kPnXQbA@mail.gmail.com>
References: <CAK5idxTaJnDhHOX8bV7uKRWsA2i-cFMii3pcs9riy782PV=KwA@mail.gmail.com>
 <CA+HjJrgAGsLnQQjkV54u4Xv_cJN9JuHuuwvNvzQcn_9kPnXQbA@mail.gmail.com>
Message-ID: <CAK5idxRMssTFZbfeqSd4od=yQsptt06uYnqXWRd6atDRbUiXKQ@mail.gmail.com>

should improve the default limit by +-4x

On Sat, Jul 18, 2015 at 11:20 PM, Phyo Arkar <phyo.arkarlwin at gmail.com> wrote:
>
> On Fri, Jul 17, 2015 at 3:29 PM, Maciej Fijalkowski <fijall at gmail.com>
> wrote:
>>
>> stack limit
>
>
> That will help alot with recursions?

From phyo.arkarlwin at gmail.com  Sun Jul 19 20:27:02 2015
From: phyo.arkarlwin at gmail.com (Phyo Arkar)
Date: Mon, 20 Jul 2015 00:57:02 +0630
Subject: [pypy-dev] Increase the default stack limit
In-Reply-To: <CAK5idxRMssTFZbfeqSd4od=yQsptt06uYnqXWRd6atDRbUiXKQ@mail.gmail.com>
References: <CAK5idxTaJnDhHOX8bV7uKRWsA2i-cFMii3pcs9riy782PV=KwA@mail.gmail.com>
 <CA+HjJrgAGsLnQQjkV54u4Xv_cJN9JuHuuwvNvzQcn_9kPnXQbA@mail.gmail.com>
 <CAK5idxRMssTFZbfeqSd4od=yQsptt06uYnqXWRd6atDRbUiXKQ@mail.gmail.com>
Message-ID: <CA+HjJri1re=07X3RpaZXtVtwhQ_BE2gV-zhFh-w9OTrYc-=+Cg@mail.gmail.com>

I am sure it is fine for most modern linux distros .

On Sun, Jul 19, 2015 at 11:08 AM, Maciej Fijalkowski <fijall at gmail.com>
wrote:

> should improve the default limit by +-4x
>
> On Sat, Jul 18, 2015 at 11:20 PM, Phyo Arkar <phyo.arkarlwin at gmail.com>
> wrote:
> >
> > On Fri, Jul 17, 2015 at 3:29 PM, Maciej Fijalkowski <fijall at gmail.com>
> > wrote:
> >>
> >> stack limit
> >
> >
> > That will help alot with recursions?
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20150720/908df527/attachment.html>

From oscar.j.benjamin at gmail.com  Wed Jul 22 18:58:09 2015
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Wed, 22 Jul 2015 16:58:09 +0000
Subject: [pypy-dev] FFT crash in numpy fork
Message-ID: <CAHVvXxSMM-KxbZ0iAZgTfhYukWthenhA0ox8iLdYsyVNeK0PWw@mail.gmail.com>

Hi all,

What's the current status of numpy in pypy? Previously the interpreter
would suggest to import numpypy which would then provide a subset of numpy.
Is the suggestion now to install this fork

    https://bitbucket.org/pypy/numpy

which installs under the ordinary numpy name? I just installed that and
attempted to use fft and got:

$ pypy
Python 2.7.9 (295ee98b6928, May 31 2015, 07:29:04)
[PyPy 2.6.0 with GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>> from numpy.fft import fft
>>>> fft([1] * 100)
*** glibc detected *** pypy: malloc(): memory corruption:
0x000000000389da80 ***

At this point pypy has to be killed. It's not using any CPU but completely
unresponsive.

Changing it to fft([1] * 4) I get a core dump:

$ pypy test.py
*** glibc detected *** pypy: munmap_chunk(): invalid pointer:
0x000000000106df70 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x7db26)[0x7f5a560d6b26]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x1921459)[0x7f5a57f55459]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x16ad1e1)[0x7f5a57ce11e1]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x18882d4)[0x7f5a57ebc2d4]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x187ee1a)[0x7f5a57eb2e1a]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x18622fd)[0x7f5a57e962fd]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x1e3786e)[0x7f5a5846b86e]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x18b8a76)[0x7f5a57eeca76]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x18bc059)[0x7f5a57ef0059]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x1dc8ffe)[0x7f5a583fcffe]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x22b96c5)[0x7f5a588ed6c5]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x18a02d4)[0x7f5a57ed42d4]
/space/enojb/current/poly/venv/bin/libpypy-c.so(pypy_execute_frame_trampoline+0x6)[0x7f5a58c692f6]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x18ad20e)[0x7f5a57ee120e]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x18b8446)[0x7f5a57eec446]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x18bc059)[0x7f5a57ef0059]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x1dc8ffe)[0x7f5a583fcffe]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x22b96c5)[0x7f5a588ed6c5]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x18a02d4)[0x7f5a57ed42d4]
/space/enojb/current/poly/venv/bin/libpypy-c.so(pypy_execute_frame_trampoline+0x6)[0x7f5a58c692f6]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x18ad20e)[0x7f5a57ee120e]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x18b8446)[0x7f5a57eec446]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x18bc059)[0x7f5a57ef0059]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x1dc8ffe)[0x7f5a583fcffe]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x22b96c5)[0x7f5a588ed6c5]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x18a02d4)[0x7f5a57ed42d4]
/space/enojb/current/poly/venv/bin/libpypy-c.so(pypy_execute_frame_trampoline+0x6)[0x7f5a58c692f6]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x18ad20e)[0x7f5a57ee120e]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x18b8446)[0x7f5a57eec446]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x18bc059)[0x7f5a57ef0059]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x1dc8ffe)[0x7f5a583fcffe]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x22b96c5)[0x7f5a588ed6c5]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x18a02d4)[0x7f5a57ed42d4]
/space/enojb/current/poly/venv/bin/libpypy-c.so(pypy_execute_frame_trampoline+0x6)[0x7f5a58c692f6]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x18adab1)[0x7f5a57ee1ab1]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x18b72a6)[0x7f5a57eeb2a6]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x18bc059)[0x7f5a57ef0059]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x1dc8ffe)[0x7f5a583fcffe]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x22b96c5)[0x7f5a588ed6c5]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x18a02d4)[0x7f5a57ed42d4]
/space/enojb/current/poly/venv/bin/libpypy-c.so(pypy_execute_frame_trampoline+0x6)[0x7f5a58c692f6]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x18ace53)[0x7f5a57ee0e53]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x18b881c)[0x7f5a57eec81c]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x18bc059)[0x7f5a57ef0059]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x1dc8ffe)[0x7f5a583fcffe]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x22b96c5)[0x7f5a588ed6c5]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x18a02d4)[0x7f5a57ed42d4]
/space/enojb/current/poly/venv/bin/libpypy-c.so(pypy_execute_frame_trampoline+0x6)[0x7f5a58c692f6]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x18ace53)[0x7f5a57ee0e53]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x18b8701)[0x7f5a57eec701]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x18bc059)[0x7f5a57ef0059]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x1dc8ffe)[0x7f5a583fcffe]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x22b96c5)[0x7f5a588ed6c5]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x18a02d4)[0x7f5a57ed42d4]
/space/enojb/current/poly/venv/bin/libpypy-c.so(pypy_execute_frame_trampoline+0x6)[0x7f5a58c692f6]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x18ace53)[0x7f5a57ee0e53]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x18b8785)[0x7f5a57eec785]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x18bc059)[0x7f5a57ef0059]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x1dc8ffe)[0x7f5a583fcffe]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x22b96c5)[0x7f5a588ed6c5]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x18a02d4)[0x7f5a57ed42d4]
/space/enojb/current/poly/venv/bin/libpypy-c.so(pypy_execute_frame_trampoline+0x6)[0x7f5a58c692f6]
/space/enojb/current/poly/venv/bin/libpypy-c.so(+0x184ca5e)[0x7f5a57e80a5e]
======= Memory map: ========
00400000-00401000 r-xp 00000000 08:07 1184381
/space/enojb/current/poly/venv/bin/pypy
00600000-00601000 r--p 00000000 08:07 1184381
/space/enojb/current/poly/venv/bin/pypy
00601000-00602000 rw-p 00001000 08:07 1184381
/space/enojb/current/poly/venv/bin/pypy
00a5c000-02832000 rw-p 00000000 00:00 0
[heap]
7f5a50ef6000-7f5a50efc000 r-xp 00000000 08:07 1840435
/space/enojb/current/poly/venv/site-packages/numpy/random/lib_mtrand.so
7f5a50efc000-7f5a510fb000 ---p 00006000 08:07 1840435
/space/enojb/current/poly/venv/site-packages/numpy/random/lib_mtrand.so
7f5a510fb000-7f5a510fc000 r--p 00005000 08:07 1840435
/space/enojb/current/poly/venv/site-packages/numpy/random/lib_mtrand.so
7f5a510fc000-7f5a510fd000 rw-p 00006000 08:07 1840435
/space/enojb/current/poly/venv/site-packages/numpy/random/lib_mtrand.so
7f5a510fd000-7f5a51104000 r-xp 00000000 08:07 1840444
/space/enojb/current/poly/venv/site-packages/numpy/fft/__pycache__/_
cffi__gf1c0eb94x67809c30.pypy-26.so
7f5a51104000-7f5a51303000 ---p 00007000 08:07 1840444
/space/enojb/current/poly/venv/site-packages/numpy/fft/__pycache__/_
cffi__gf1c0eb94x67809c30.pypy-26.so
7f5a51303000-7f5a51304000 r--p 00006000 08:07 1840444
/space/enojb/current/poly/venv/site-packages/numpy/fft/__pycache__/_
cffi__gf1c0eb94x67809c30.pypy-26.so
7f5a51304000-7f5a51305000 rw-p 00007000 08:07 1840444
/space/enojb/current/poly/venv/site-packages/numpy/fft/__pycache__/_
cffi__gf1c0eb94x67809c30.pypy-26.so
7f5a51305000-7f5a51322000 r-xp 00000000 08:07 1840434
/space/enojb/current/poly/venv/site-packages/numpy/linalg/libumath_linalg_cffi.so
7f5a51322000-7f5a51521000 ---p 0001d000 08:07 1840434
/space/enojb/current/poly/venv/site-packages/numpy/linalg/libumath_linalg_cffi.so
7f5a51521000-7f5a51522000 r--p 0001c000 08:07 1840434
/space/enojb/current/poly/venv/site-packages/numpy/linalg/libumath_linalg_cffi.so
7f5a51522000-7f5a51523000 rw-p 0001d000 08:07 1840434
/space/enojb/current/poly/venv/site-packages/numpy/linalg/libumath_linalg_cffi.so
7f5a51523000-7f5a51558000 r-xp 00000000 08:01 665692
/usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0
7f5a51558000-7f5a51757000 ---p 00035000 08:01 665692
/usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0
7f5a51757000-7f5a51758000 r--p 00034000 08:01 665692
/usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0
7f5a51758000-7f5a51759000 rw-p 00035000 08:01 665692
/usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0
7f5a51759000-7f5a5176e000 r-xp 00000000 08:01 267011
/lib/x86_64-linux-gnu/libgcc_s.so.1
7f5a5176e000-7f5a5196d000 ---p 00015000 08:01 267011
/lib/x86_64-linux-gnu/libgcc_s.so.1
7f5a5196d000-7f5a5196e000 r--p 00014000 08:01 267011
/lib/x86_64-linux-gnu/libgcc_s.so.1
7f5a5196e000-7f5a5196f000 rw-p 00015000 08:01 267011
/lib/x86_64-linux-gnu/libgcc_s.so.1
7f5a5196f000-7f5a51a83000 r-xp 00000000 08:01 681056
/usr/lib/x86_64-linux-gnu/libgfortran.so.3.0.0
7f5a51a83000-7f5a51c83000 ---p 00114000 08:01 681056
/usr/lib/x86_64-linux-gnu/libgfortran.so.3.0.0
7f5a51c83000-7f5a51c84000 r--p 00114000 08:01 681056
/usr/lib/x86_64-linux-gnu/libgfortran.so.3.0.0
7f5a51c84000-7f5a51c86000 rw-p 00115000 08:01 681056
/usr/lib/x86_64-linux-gnu/libgfortran.so.3.0.0
7f5a51c86000-7f5a51d1f000 r-xp 00000000 08:01 682100
/usr/lib/libblas/libblas.so.3gf.0
7f5a51d1f000-7f5a51f1e000 ---p 00099000 08:01 682100
/usr/lib/libblas/libblas.so.3gf.0
7f5a51f1e000-7f5a51f1f000 r--p 00098000 08:01 682100
/usr/lib/libblas/libblas.so.3gf.0
7f5a51f1f000-7f5a51f20000 rw-p 00099000 08:01 682100
/usr/lib/libblas/libblas.so.3gf.0
7f5a51f20000-7f5a52805000 r-xp 00000000 08:01 946355
/usr/lib/lapack/liblapack.so.3gf.0
7f5a52805000-7f5a52a04000 ---p 008e5000 08:01 946355
/usr/lib/lapack/liblapack.so.3gf.0
7f5a52a04000-7f5a52a05000 r--p 008e4000 08:01 946355
/usr/lib/lapack/liblapack.so.3gf.0
7f5a52a05000-7f5a52a09000 rw-p 008e5000 08:01 946355
/usr/lib/lapack/liblapack.so.3gf.0
7f5a52a09000-7f5a52b97000 rw-p 00000000 00:00 0
7f5a52c98000-7f5a52d9a000 rw-p 00000000 00:00 0
7f5a52d9a000-7f5a52d9d000 r-xp 00000000 08:01 665639
/usr/lib/x86_64-linux-gnu/libpanel.so.5.9
7f5a52d9d000-7f5a52f9c000 ---p 00003000 08:01 665639
/usr/lib/x86_64-linux-gnu/libpanel.so.5.9
7f5a52f9c000-7f5a52f9d000 r--p 00002000 08:01 665639
/usr/lib/x86_64-linux-gnu/libpanel.so.5.9
7f5a52f9d000-7f5a52f9e000 rw-p 00003000 08:01 665639
/usr/lib/x86_64-linux-gnu/libpanel.so.5.9
7f5a52f9e000-7f5a52fbd000 r-xp 00000000 08:01 267028
/lib/x86_64-linux-gnu/libncurses.so.5.9
7f5a52fbd000-7f5a531bd000 ---p 0001f000 08:01 267028
/lib/x86_64-linux-gnu/libncurses.so.5.9
7f5a531bd000-7f5a531be000 r--p 0001f000 08:01 267028
/lib/x86_64-linux-gnu/libncurses.so.5.9
7f5a531be000-7f5a531bf000 rw-p 00020000 08:01 267028
/lib/x86_64-linux-gnu/libncurses.so.5.9
7f5a531bf000-7f5a531ca000 r-xp 00000000 08:06 143128
/users/enojb/.local/pypy-2.6.0-linux64/lib_pypy/_curses_cffi.pypy-26.so
7f5a531ca000-7f5a533ca000 ---p 0000b000 08:06 143128
/users/enojb/.local/pypy-2.6.0-linux64/lib_pypy/_curses_cffi.pypy-26.so
7f5a533ca000-7f5a533cd000 r--p 0000b000 08:06 143128
/users/enojb/.local/pypy-2.6.0-linux64/lib_pypy/_curses_cffi.pypy-26.so
7f5a533cd000-7f5a533ce000 rw-p 0000e000 08:06 143128
/users/enojb/.local/pypy-2.6.0-linux64/lib_pypy/_curses_cffi.pypy-26.so
7f5a533ce000-7f5a535d2000 rw-p 00000000 00:00 0
7f5a535d2000-7f5a535d4000 r-xp 00000000 08:06 143111
/users/enojb/.local/pypy-2.6.0-linux64/lib_pypy/_pwdgrp_cffi.pypy-26.so
7f5a535d4000-7f5a537d3000 ---p 00002000 08:06 143111
/users/enojb/.local/pypy-2.6.0-linux64/lib_pypy/_pwdgrp_cffi.pypy-26.so
7f5a537d3000-7f5a537d4000 r--p 00001000 08:06 143111
/users/enojb/.local/pypy-2.6.0-linux64/lib_pypy/_pwdgrp_cffi.pypy-26.so
7f5a537d4000-7f5a537d5000 rw-p 00002000 08:06 143111
/users/enojb/.local/pypy-2.6.0-linux64/lib_pypy/_pwdgrp_cffi.pypy-26.so
7f5a537d5000-7f5a53856000 rw-p 00000000 00:00 0
7f5a53856000-7f5a53956000 rwxp 00000000 00:00 0
7f5a53956000-7f5a5403a000 r--p 00000000 08:01 663686
/usr/lib/locale/locale-archive
7f5a5403a000-7f5a5445c000 rw-p 00000000 00:00 0
7f5a5445c000-7f5a5447e000 r-xp 00000000 08:01 267087
/lib/x86_64-linux-gnu/libtinfo.so.5.9
7f5a5447e000-7f5a5467e000 ---p 00022000 08:01 267087
/lib/x86_64-linux-gnu/libtinfo.so.5.9
7f5a5467e000-7f5a54682000 r--p 00022000 08:01 267087
/lib/x86_64-linux-gnu/libtinfo.so.5.9
7f5a54682000-7f5a54683000 rw-p 00026000 08:01 267087
/lib/x86_64-linux-gnu/libtinfo.so.5.9
7f5a54683000-7f5a54685000 r-xp 00000000 08:01 280141
/lib/x86_64-linux-gnu/libutil-2.15.so
7f5a54685000-7f5a54884000 ---p 00002000 08:01 280141
/lib/x86_64-linux-gnu/libutil-2.15.so
7f5a54884000-7f5a54885000 r--p 00001000 08:01 280141
/lib/x86_64-linux-gnu/libutil-2.15.so
7f5a54885000-7f5a54886000 rw-p 00002000 08:01 280141
/lib/x86_64-linux-gnu/libutil-2.15.so
7f5a54886000-7f5a5488d000 r-xp 00000000 08:01 665395
/usr/lib/x86_64-linux-gnu/libffi.so.6.0.0
7f5a5488d000-7f5a54a8c000 ---p 00007000 08:01 665395
/usr/lib/x86_64-linux-gnu/libffi.so.6.0.0
7f5a54a8c000-7f5a54a8d000 r--p 00006000 08:01 665395
/usr/lib/x86_64-linux-gnu/libffi.so.6.0.0
7f5a54a8d000-7f5a54a8e000 rw-p 00007000 08:01 665395
/usr/lib/x86_64-linux-gnu/libffi.so.6.0.0
7f5a54a8e000-7f5a54a95000 r-xp 00000000 08:01 289152
/lib/x86_64-linux-gnu/librt-2.15.so
7f5a54a95000-7f5a54c94000 ---p 00007000 08:01 289152
/lib/x86_64-linux-gnu/librt-2.15.so
7f5a54c94000-7f5a54c95000 r--p 00006000 08:01 289152
/lib/x86_64-linux-gnu/librt-2.15.so
7f5a54c95000-7f5a54c96000 rw-p 00007000 08:01 289152
/lib/x86_64-linux-gnu/librt-2.15.so
7f5a54c96000-7f5a54c9f000 r-xp 00000000 08:01 280140
/lib/x86_64-linux-gnu/libcrypt-2.15.so
7f5a54c9f000-7f5a54e9f000 ---p 00009000 08:01 280140
/lib/x86_64-linux-gnu/libcrypt-2.15.so
7f5a54e9f000-7f5a54ea0000 r--p 00009000 08:01 280140
/lib/x86_64-linux-gnu/libcrypt-2.15.so
7f5a54ea0000-7f5a54ea1000 rw-p 0000a000 08:01 280140
/lib/x86_64-linux-gnu/libcrypt-2.15.so
7f5a54ea1000-7f5a54ecf000 rw-p 00000000 00:00 0
7f5a54ecf000-7f5a54ef6000 r-xp 00000000 08:01 267008
/lib/x86_64-linux-gnu/libexpat.so.1.5.2
7f5a54ef6000-7f5a550f6000 ---p 00027000 08:01 267008
/lib/x86_64-linux-gnu/libexpat.so.1.5.2
7f5a550f6000-7f5a550f8000 r--p 00027000 08:01 267008
/lib/x86_64-linux-gnu/libexpat.so.1.5.2
7f5a550f8000-7f5a550f9000 rw-p 00029000 08:01 267008
/lib/x86_64-linux-gnu/libexpat.so.1.5.2
7f5a550f9000-7f5a552ab000 r-xp 00000000 08:01 266972
/lib/x86_64-linux-gnu/libcrypto.so.1.0.0
7f5a552ab000-7f5a554aa000 ---p 001b2000 08:01 266972
/lib/x86_64-linux-gnu/libcrypto.so.1.0.0
7f5a554aa000-7f5a554c5000 r--p 001b1000 08:01 266972
/lib/x86_64-linux-gnu/libcrypto.so.1.0.0
7f5a554c5000-7f5a554d0000 rw-p 001cc000 08:01 266972
/lib/x86_64-linux-gnu/libcrypto.so.1.0.0
7f5a554d0000-7f5a554d4000 rw-p 00000000 00:00 0
7f5a554d4000-7f5a55529000 r-xp 00000000 08:01 263454
/lib/x86_64-linux-gnu/libssl.so.1.0.0
7f5a55529000-7f5a55728000 ---p 00055000 08:01 263454
/lib/x86_64-linux-gnu/libssl.so.1.0.0
7f5a55728000-7f5a5572b000 r--p 00054000 08:01 263454
/lib/x86_64-linux-gnu/libssl.so.1.0.0
7f5a5572b000-7f5a55732000 rw-p 00057000 08:01 263454
/lib/x86_64-linux-gnu/libssl.so.1.0.0
7f5a55732000-7f5a55748000 r-xp 00000000 08:01 267101
/lib/x86_64-linux-gnu/libz.so.1.2.3.4
7f5a55748000-7f5a55947000 ---p 00016000 08:01 267101
/lib/x86_64-linux-gnu/libz.so.1.2.3.4
7f5a55947000-7f5a55948000 r--p 00015000 08:01 267101
/lib/x86_64-linux-gnu/libz.so.1.2.3.4
7f5a55948000-7f5a55949000 rw-p 00016000 08:01 267101
/lib/x86_64-linux-gnu/libz.so.1.2.3.4
7f5a55949000-7f5a55a44000 r-xp 00000000 08:01 288645
/lib/x86_64-linux-gnu/libm-2.15.so
7f5a55a44000-7f5a55c43000 ---p 000fb000 08:01 288645
/lib/x86_64-linux-gnu/libm-2.15.so
7f5a55c43000-7f5a55c44000 r--p 000fa000 08:01 288645
/lib/x86_64-linux-gnu/libm-2.15.so
7f5a55c44000-7f5a55c45000 rw-p 000fb000 08:01 288645
/lib/x86_64-linux-gnu/libm-2.15.so
7f5a55c45000-7f5a55c54000 r-xp 00000000 08:01 266989
/lib/x86_64-linux-gnu/libbz2.so.1.0.4
7f5a55c54000-7f5a55e53000 ---p 0000f000 08:01 266989
/lib/x86_64-linux-gnu/libbz2.so.1.0.4
7f5a55e53000-7f5a55e54000 r--p 0000e000 08:01 266989
/lib/x86_64-linux-gnu/libbz2.so.1.0.4
7f5a55e54000-7f5a55e55000 rw-p 0000f000 08:01 266989
/lib/x86_64-linux-gnu/libbz2.so.1.0.4
7f5a55e55000-7f5a55e57000 r-xp 00000000 08:01 289529
/lib/x86_64-linux-gnu/libdl-2.15.so
7f5a55e57000-7f5a56057000 ---p 00002000 08:01 289529
/lib/x86_64-linux-gnu/libdl-2.15.so
7f5a56057000-7f5a56058000 r--p 00002000 08:01 289529
/lib/x86_64-linux-gnu/libdl-2.15.so
7f5a56058000-7f5a56059000 rw-p 00003000 08:01 289529
/lib/x86_64-linux-gnu/libdl-2.15.so
7f5a56059000-7f5a5620d000 r-xp 00000000 08:01 289524
/lib/x86_64-linux-gnu/libc-2.15.so
7f5a5620d000-7f5a5640c000 ---p 001b4000 08:01 289524
/lib/x86_64-linux-gnu/libc-2.15.so
7f5a5640c000-7f5a56410000 r--p 001b3000 08:01 289524
/lib/x86_64-linux-gnu/libc-2.15.so
7f5a56410000-7f5a56412000 rw-p 001b7000 08:01 289524
/lib/x86_64-linux-gnu/libc-2.15.so
7f5a56412000-7f5a56417000 rw-p 00000000 00:00 0
7f5a56417000-7f5a5642f000 r-xp 00000000 08:01 288670
/lib/x86_64-linux-gnu/libpthread-2.15.so
7f5a5642f000-7f5a5662e000 ---p 00018000 08:01 288670
/lib/x86_64-linux-gnu/libpthread-2.15.so
7f5a5662e000-7f5a5662f000 r--p 00017000 08:01 288670
/lib/x86_64-linux-gnu/libpthread-2.15.so
7f5a5662f000-7f5a56630000 rw-p 00018000 08:01 288670
/lib/x86_64-linux-gnu/libpthread-2.15.so
7f5a56630000-7f5a56634000 rw-p 00000000 00:00 0
7f5a56634000-7f5a58fc1000 r-xp 00000000 08:06 3471
/users/enojb/.local/pypy-2.6.0-linux64/bin/libpypy-c.so
7f5a58fc1000-7f5a591c0000 ---p 0298d000 08:06 3471
/users/enojb/.local/pypy-2.6.0-linux64/bin/libpypy-c.so
7f5a591c0000-7f5a591dc000 r--p 0298c000 08:06 3471
/users/enojb/.local/pypy-2.6.0-linux64/bin/libpypy-c.so
7f5a591dc000-7f5a5a83f000 rw-p 029a8000 08:06 3471
/users/enojb/.local/pypy-2.6.0-linux64/bin/libpypy-c.so
7f5a5a83f000-7f5a5a866000 rw-p 00000000 00:00 0
7f5a5a866000-7f5a5a888000 r-xp 00000000 08:01 289150
/lib/x86_64-linux-gnu/ld-2.15.so
7f5a5a8e0000-7f5a5aa6b000 rw-p 00000000 00:00 0
7f5a5aa84000-7f5a5aa85000 rw-p 00000000 00:00 0
7f5a5aa85000-7f5a5aa86000 rwxp 00000000 00:00 0
7f5a5aa86000-7f5a5aa88000 rw-p 00000000 00:00 0
7f5a5aa88000-7f5a5aa89000 r--p 00022000 08:01 289150
/lib/x86_64-linux-gnu/ld-2.15.so
7f5a5aa89000-7f5a5aa8b000 rw-p 00023000 08:01 289150
/lib/x86_64-linux-gnu/ld-2.15.so
7ffd2d70f000-7ffd2d72f000 rwxp 00000000 00:00 0
[stack]
7ffd2d72f000-7ffd2d730000 rw-p 00000000 00:00 0
7ffd2d73b000-7ffd2d73d000 r-xp 00000000 00:00 0
[vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0
[vsyscall]
Aborted (core dumped)

--
Oscar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20150722/c81512e9/attachment-0001.html>

From ronan.lamy at gmail.com  Wed Jul 22 19:24:46 2015
From: ronan.lamy at gmail.com (Ronan Lamy)
Date: Wed, 22 Jul 2015 18:24:46 +0100
Subject: [pypy-dev] FFT crash in numpy fork
In-Reply-To: <CAHVvXxSMM-KxbZ0iAZgTfhYukWthenhA0ox8iLdYsyVNeK0PWw@mail.gmail.com>
References: <CAHVvXxSMM-KxbZ0iAZgTfhYukWthenhA0ox8iLdYsyVNeK0PWw@mail.gmail.com>
Message-ID: <55AFD1DE.10002@gmail.com>

Le 22/07/15 17:58, Oscar Benjamin a ?crit :
> Hi all,
>
> What's the current status of numpy in pypy? Previously the interpreter
> would suggest to import numpypy which would then provide a subset of
> numpy. Is the suggestion now to install this fork
>
> https://bitbucket.org/pypy/numpy
>
> which installs under the ordinary numpy name? I just installed that and

Yes, this is the way to install numpy now.

> attempted to use fft and got:
>
> $ pypy
> Python 2.7.9 (295ee98b6928, May 31 2015, 07:29:04)
> [PyPy 2.6.0 with GCC 4.8.2] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>  >>>> from numpy.fft import fft
>  >>>> fft([1] * 100)
> *** glibc detected *** pypy: malloc(): memory corruption:
> 0x000000000389da80 ***
>
> At this point pypy has to be killed. It's not using any CPU but
> completely unresponsive.
>
> Changing it to fft([1] * 4) I get a core dump:

Well, it's a bug. Please report it at https://bitbucket.org/pypy/numpy

From oscar.j.benjamin at gmail.com  Wed Jul 22 20:07:26 2015
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Wed, 22 Jul 2015 18:07:26 +0000
Subject: [pypy-dev] FFT crash in numpy fork
In-Reply-To: <55AFD1DE.10002@gmail.com>
References: <CAHVvXxSMM-KxbZ0iAZgTfhYukWthenhA0ox8iLdYsyVNeK0PWw@mail.gmail.com>
 <55AFD1DE.10002@gmail.com>
Message-ID: <CAHVvXxRafi15f1=4KRKnr2HPAwyomAHYEjaWFs-oJEr=tW5nHw@mail.gmail.com>

On Wed, 22 Jul 2015 at 18:26 Ronan Lamy <ronan.lamy at gmail.com> wrote:

>
> Well, it's a bug. Please report it at https://bitbucket.org/pypy/numpy
>

Done:

https://bitbucket.org/pypy/numpy/issues/39/fft-crashes-when-given-a-list-of-non

 --
Oscar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20150722/27b9b575/attachment.html>

From dimaqq at gmail.com  Thu Jul 23 11:39:58 2015
From: dimaqq at gmail.com (Dima Tisnek)
Date: Thu, 23 Jul 2015 11:39:58 +0200
Subject: [pypy-dev] Increase the default stack limit
In-Reply-To: <CAK5idxTaJnDhHOX8bV7uKRWsA2i-cFMii3pcs9riy782PV=KwA@mail.gmail.com>
References: <CAK5idxTaJnDhHOX8bV7uKRWsA2i-cFMii3pcs9riy782PV=KwA@mail.gmail.com>
Message-ID: <CAGGBzXKETNqO_EGx-CS6H6txcBecUQHhQ1AHi__01VZRSJnbow@mail.gmail.com>

Pardon me if I don't know what I'm talking about:

Can the stack limit not be read from ulimit/getrlimit? Or from failure
to push stack?
That used to work for PHP, at least in the past.
It also gives users the flexibility between highly recursive
computation (very high limit) and insane multithreading (low limit to
conserve memory).

Alternatively, pypi could stick to CPython fixed number of Python
frames limit, and demand as much system stack as pypi requires to
implement that.

my 2c.

On 17 July 2015 at 10:59, Maciej Fijalkowski <fijall at gmail.com> wrote:
> Hi
>
> The default stack limit is (conservatively) set at 700k or so. Can we
> push it to ~3M by default? (this is still far below 8M or so which is
> the default on linux)
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev

From fijall at gmail.com  Thu Jul 23 13:34:19 2015
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Thu, 23 Jul 2015 13:34:19 +0200
Subject: [pypy-dev] Increase the default stack limit
In-Reply-To: <CAGGBzXKETNqO_EGx-CS6H6txcBecUQHhQ1AHi__01VZRSJnbow@mail.gmail.com>
References: <CAK5idxTaJnDhHOX8bV7uKRWsA2i-cFMii3pcs9riy782PV=KwA@mail.gmail.com>
 <CAGGBzXKETNqO_EGx-CS6H6txcBecUQHhQ1AHi__01VZRSJnbow@mail.gmail.com>
Message-ID: <CAK5idxS2H5yJk5gQC7_hpBFQCJYzkB+YJn5JAdEXQsesUYpYaA@mail.gmail.com>

On Thu, Jul 23, 2015 at 11:39 AM, Dima Tisnek <dimaqq at gmail.com> wrote:
> Pardon me if I don't know what I'm talking about:
>
> Can the stack limit not be read from ulimit/getrlimit? Or from failure
> to push stack?

failure is segfault

> That used to work for PHP, at least in the past.
> It also gives users the flexibility between highly recursive
> computation (very high limit) and insane multithreading (low limit to
> conserve memory).
>
> Alternatively, pypi could stick to CPython fixed number of Python
> frames limit, and demand as much system stack as pypi requires to
> implement that.
>
> my 2c.

That's a terrible idea. You can create cases where CPython crashes
with fat frames with the default recursion limit. On the other hand
some JIT code does not consume any stack (so counting frames is
unnecessary computation to start with)

From dimaqq at gmail.com  Thu Jul 23 13:58:05 2015
From: dimaqq at gmail.com (Dima Tisnek)
Date: Thu, 23 Jul 2015 13:58:05 +0200
Subject: [pypy-dev] Docker images for "try pypy"
In-Reply-To: <CAK5idxSTvZv3yU8i2TT6m=JDzRkVm42rxWDhup4OCrp8SQRWBQ@mail.gmail.com>
References: <CAK5idxT_BMP223ud19tKe74Gsp4H+sX_DCAxktCc-Q+7uWG+8A@mail.gmail.com>
 <CANaNaZXZurniBnHAL-t8C+Rf-k9GPXi3Vk1g5XwJFrJ8bN4xTw@mail.gmail.com>
 <CAK5idxSTvZv3yU8i2TT6m=JDzRkVm42rxWDhup4OCrp8SQRWBQ@mail.gmail.com>
Message-ID: <CAGGBzXJFsLEtiKazDaeY=vaz5d=6=oBefPipWRxs_UPaeXb1BQ@mail.gmail.com>

It does work:

# start docker daemon (depends on your OS)
docker pull pypy  # I think it's optional
docker run -a stdin -a stdout -a stderr -i -t  pypy
Python 3.2.5 (b2091e973da6, Oct 19 2014, 18:29:55)
[PyPy 2.4.0 with GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>>



On 11 July 2015 at 22:51, Maciej Fijalkowski <fijall at gmail.com> wrote:
> I don't know, maybe it is :-)
>
> /me investigates
>
> On Sat, Jul 11, 2015 at 9:15 PM, Romain Guillebert <romain.py at gmail.com> wrote:
>> Hi Maciej
>>
>> Isn't this : https://registry.hub.docker.com/_/pypy/ what you're looking
>> for?
>>
>> Cheers
>> Romain
>>
>> On 11 Jul 2015 19:52, "Maciej Fijalkowski" <fijall at gmail.com> wrote:
>>>
>>> Hi
>>>
>>> Would anyone be willing to make some docker images that can be used to
>>> "try pypy"?
>>>
>>> Cheers,
>>> fijal
>>> _______________________________________________
>>> pypy-dev mailing list
>>> pypy-dev at python.org
>>> https://mail.python.org/mailman/listinfo/pypy-dev
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev

From dimaqq at gmail.com  Thu Jul 23 14:03:48 2015
From: dimaqq at gmail.com (Dima Tisnek)
Date: Thu, 23 Jul 2015 14:03:48 +0200
Subject: [pypy-dev] Increase the default stack limit
In-Reply-To: <CAK5idxS2H5yJk5gQC7_hpBFQCJYzkB+YJn5JAdEXQsesUYpYaA@mail.gmail.com>
References: <CAK5idxTaJnDhHOX8bV7uKRWsA2i-cFMii3pcs9riy782PV=KwA@mail.gmail.com>
 <CAGGBzXKETNqO_EGx-CS6H6txcBecUQHhQ1AHi__01VZRSJnbow@mail.gmail.com>
 <CAK5idxS2H5yJk5gQC7_hpBFQCJYzkB+YJn5JAdEXQsesUYpYaA@mail.gmail.com>
Message-ID: <CAGGBzX+giENf-7omySrdEA3i5dYsh5vJ1UfEv2tTshNVFhzVWQ@mail.gmail.com>

>> Can the stack limit not be read from ulimit/getrlimit? Or from failure
>> to push stack?
>
> failure is segfault

good point, I doubt stack probe is worth the overhead
(settings/resetting signals and/or segv handler that allows probe
only)

how about stack limit based on getrlimit though?

From anto.cuni at gmail.com  Fri Jul 24 00:40:54 2015
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Fri, 24 Jul 2015 00:40:54 +0200
Subject: [pypy-dev] syntax sugar for stm TransactionQueue
Message-ID: <CAHPg78wQjwv1rBL_d_Wwjv647FBTjeH8x_14M7UWd=sOU+9Eaw@mail.gmail.com>

hi Armin,
following the discussion we had today, that TransactionQueue could be
easier to understand for people if you explain it as "a for loop in which
you don't know the order of the iteration", I figured out that we might
even introduce some syntactic sugar for it; not sure if it makes things
simpler or more complicated, though :).
Anyway, I'm thinking of something like this:

def parallel(iterable):
    def decorator(f):
        tr = TransactionQueue()
        for item in iterable:
            tr.add(f, item)
        tr.run()
    return decorator


to be used in this way:

mylist = [1, 2, 3]

@parallel(mylist)
def for_(item):
    # do something with item
    pass


ciao,
Anto
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20150724/4e3d1ec4/attachment.html>

From dimaqq at gmail.com  Fri Jul 24 09:26:21 2015
From: dimaqq at gmail.com (Dima Tisnek)
Date: Fri, 24 Jul 2015 09:26:21 +0200
Subject: [pypy-dev] syntax sugar for stm TransactionQueue
In-Reply-To: <CAHPg78wQjwv1rBL_d_Wwjv647FBTjeH8x_14M7UWd=sOU+9Eaw@mail.gmail.com>
References: <CAHPg78wQjwv1rBL_d_Wwjv647FBTjeH8x_14M7UWd=sOU+9Eaw@mail.gmail.com>
Message-ID: <CAGGBzXJzovyZJ-j7V27s6SRFUv3+01EgSem1Ey75EKGf7QyCtg@mail.gmail.com>

Supplying argument list at decoration time is hardly ideal.
Typical call site would pass argument sequence at invocation time.
Thus you probably want to rewrite this decorator and allow application
without argument list:

@parallel
def foo(item)
    return item ** 123

results = foo(range(9))


Perhaps OT:
My immediate reaction when I saw TransactionQueue in the docs was:
1. does add() have set semantics, that is, can I add() same item several times?
2. is queue synchronous or asynchronous, that is, can I do the following:

tq = TransactionQueue()
tq.start()
for item in some_expensive_stream():
    tq.add(func, item)
tq.wait()

Or is item execution already implied during .add() before .run()?

Perhaps it's just something that needs to be clarified in the docs; or
it may influence your sugar.

On 24 July 2015 at 00:40, Antonio Cuni <anto.cuni at gmail.com> wrote:
> hi Armin,
> following the discussion we had today, that TransactionQueue could be easier
> to understand for people if you explain it as "a for loop in which you don't
> know the order of the iteration", I figured out that we might even introduce
> some syntactic sugar for it; not sure if it makes things simpler or more
> complicated, though :).
> Anyway, I'm thinking of something like this:
>
> def parallel(iterable):
>     def decorator(f):
>         tr = TransactionQueue()
>         for item in iterable:
>             tr.add(f, item)
>         tr.run()
>     return decorator
>
>
> to be used in this way:
>
> mylist = [1, 2, 3]
>
> @parallel(mylist)
> def for_(item):
>     # do something with item
>     pass
>
>
> ciao,
> Anto
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>

From sinha.piyush0609 at gmail.com  Mon Jul 27 23:41:31 2015
From: sinha.piyush0609 at gmail.com (Piyush Sinha)
Date: Tue, 28 Jul 2015 03:11:31 +0530
Subject: [pypy-dev] guidance about working with pypy
Message-ID: <CAKGzi31nojVZoOxccv57xWeMZpGW6LhWch==P7HZur2T0--kXw@mail.gmail.com>

Hi, I'm a student developer interested in getting some open source
experience and I was wondering if anyone could suggest some bugs that are
suitable for a beginner in this project?"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20150728/ac4cb715/attachment.html>

From rich at pasra.at  Tue Jul 28 09:32:03 2015
From: rich at pasra.at (Richard Plangger)
Date: Tue, 28 Jul 2015 09:32:03 +0200
Subject: [pypy-dev] guidance about working with pypy
In-Reply-To: <CAKGzi31nojVZoOxccv57xWeMZpGW6LhWch==P7HZur2T0--kXw@mail.gmail.com>
References: <CAKGzi31nojVZoOxccv57xWeMZpGW6LhWch==P7HZur2T0--kXw@mail.gmail.com>
Message-ID: <55B72FF3.2040907@pasra.at>

Hi,

the answer to your question very much depends on your python experience.
A good place to start is the bug tracker [1] and (potentially much more
interesting) the documentation[2][3].

Most of the time smaller tasks include adding features in the python 3
branch of pypy (py3k or py3.3).

I guess no one will tell you, that you ``should fix bug X''. It would be
better (my opinion) if you start on the documentation and see if you
feel conformable with the way how to develop in pypy. Then find some
tests at build bot [4] and see how you can fix them.

[1] https://bitbucket.org/pypy/pypy/issues
[2] http://rpython.readthedocs.org/en/latest/
[3] http://pypy.readthedocs.org/en/latest/
[4] http://buildbot.pypy.org/summary?branch=%3Ctrunk%3E

cheers,
richard

On 07/27/2015 11:41 PM, Piyush Sinha wrote:
> Hi, I'm a student developer interested in getting some open source
> experience and I was wondering if anyone could suggest some bugs that
> are suitable for a beginner in this project?"
> 
> 
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20150728/093e5bb6/attachment.sig>

From matti.picus at gmail.com  Wed Jul 29 05:26:55 2015
From: matti.picus at gmail.com (Matti Picus)
Date: Wed, 29 Jul 2015 06:26:55 +0300
Subject: [pypy-dev] prepping for pypy 2.6.1 release cycle
Message-ID: <55B847FF.7080709@gmail.com>

It seems the time has come to release pypy 2.6.1
Does anyone have work not-yet-merged that they would like in the 
release, or any big issues blocking a release?
Matti

From alex.gaynor at gmail.com  Wed Jul 29 05:28:24 2015
From: alex.gaynor at gmail.com (Alex Gaynor)
Date: Tue, 28 Jul 2015 23:28:24 -0400
Subject: [pypy-dev] prepping for pypy 2.6.1 release cycle
In-Reply-To: <55B847FF.7080709@gmail.com>
References: <55B847FF.7080709@gmail.com>
Message-ID: <CAFRnB2U_GCgtdP-YsPz1kWDe3Ge0uNrr7nBGGCWrh1U_wYpzdg@mail.gmail.com>

I believe at the EuroPython sprints, Armin was working on support for `#if`
in cffi. If that's happening, it'd be good to get it into a cffi release at
the same time.

Cheers,
Alex

On Tue, Jul 28, 2015 at 11:26 PM, Matti Picus <matti.picus at gmail.com> wrote:

> It seems the time has come to release pypy 2.6.1
> Does anyone have work not-yet-merged that they would like in the release,
> or any big issues blocking a release?
> Matti
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>



-- 
"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero
GPG Key fingerprint: 125F 5C67 DFE9 4084
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20150728/695f5b9a/attachment.html>

From arigo at tunes.org  Wed Jul 29 11:17:12 2015
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 29 Jul 2015 11:17:12 +0200
Subject: [pypy-dev] prepping for pypy 2.6.1 release cycle
In-Reply-To: <CAFRnB2U_GCgtdP-YsPz1kWDe3Ge0uNrr7nBGGCWrh1U_wYpzdg@mail.gmail.com>
References: <55B847FF.7080709@gmail.com>
 <CAFRnB2U_GCgtdP-YsPz1kWDe3Ge0uNrr7nBGGCWrh1U_wYpzdg@mail.gmail.com>
Message-ID: <CAMSv6X2jbNnnRJHM0fJ1RTbRcH9et9jrfy1+9QJSC_LxyShu1g@mail.gmail.com>

Hi Matti,

On 29 July 2015 at 05:28, Alex Gaynor <alex.gaynor at gmail.com> wrote:
> On Tue, Jul 28, 2015 at 11:26 PM, Matti Picus <matti.picus at gmail.com> wrote:
>>
>> It seems the time has come to release pypy 2.6.1
>> Does anyone have work not-yet-merged that they would like in the release,
>> or any big issues blocking a release?

Cool!  Does it make sense to target August 14th as the release date?
It's not immediately now (a few of us are still in post-EuroPython
"off" mode), and it is at the end of another small sprint.

> I believe at the EuroPython sprints, Armin was working on support for `#if`
> in cffi. If that's happening, it'd be good to get it into a cffi release at
> the same time.

Yes, it would be nice if that feature was finished and part of cffi
release 1.2.

In PyPy itself, there is the branch "fix-strbuf" that should be either
confirmed to work and merged (by fijal or myself), or dropped again.


A bient?t,

Armin.

From rymg19 at gmail.com  Wed Jul 29 16:12:29 2015
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Wed, 29 Jul 2015 09:12:29 -0500
Subject: [pypy-dev] prepping for pypy 2.6.1 release cycle
In-Reply-To: <55B847FF.7080709@gmail.com>
References: <55B847FF.7080709@gmail.com>
Message-ID: <84C2167C-3ED0-4E2C-A3A9-4BE9E937B864@gmail.com>

ARE YOU SERIOUS???

The Chocolatey mods haven't even approved the 2.6 release yet!


On July 28, 2015 10:26:55 PM CDT, Matti Picus <matti.picus at gmail.com> wrote:
>It seems the time has come to release pypy 2.6.1
>Does anyone have work not-yet-merged that they would like in the 
>release, or any big issues blocking a release?
>Matti
>_______________________________________________
>pypy-dev mailing list
>pypy-dev at python.org
>https://mail.python.org/mailman/listinfo/pypy-dev

-- 
Sent from my Nexus 5 with K-9 Mail. Please excuse my brevity.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20150729/625154ef/attachment.html>

From arigo at tunes.org  Wed Jul 29 20:06:11 2015
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 29 Jul 2015 20:06:11 +0200
Subject: [pypy-dev] guidance about working with pypy
In-Reply-To: <55B72FF3.2040907@pasra.at>
References: <CAKGzi31nojVZoOxccv57xWeMZpGW6LhWch==P7HZur2T0--kXw@mail.gmail.com>
 <55B72FF3.2040907@pasra.at>
Message-ID: <CAMSv6X2K0Ujb6x6shOiPiEf4ZXD8A0DAYPJ80cHPwPBEXz-2Xg@mail.gmail.com>

Hi,

On 28 July 2015 at 09:32, Richard Plangger <rich at pasra.at> wrote:
> the answer to your question very much depends on your python experience.
> A good place to start is the bug tracker [1] and (potentially much more
> interesting) the documentation[2][3].
>
> Most of the time smaller tasks include adding features in the python 3
> branch of pypy (py3k or py3.3).
>
> I guess no one will tell you, that you ``should fix bug X''. It would be
> better (my opinion) if you start on the documentation and see if you
> feel conformable with the way how to develop in pypy. Then find some
> tests at build bot [4] and see how you can fix them.
>
> [1] https://bitbucket.org/pypy/pypy/issues
> [2] http://rpython.readthedocs.org/en/latest/
> [3] http://pypy.readthedocs.org/en/latest/
> [4] http://buildbot.pypy.org/summary?branch=%3Ctrunk%3E
>
> cheers,
> richard

In addition, you can find here the failing tests on the py3.3 branch,
some of which are probably easy fixes (click on the red "F"s):

    http://buildbot.pypy.org/summary?branch=py3.3


Armin

From arigo at tunes.org  Wed Jul 29 20:40:34 2015
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 29 Jul 2015 20:40:34 +0200
Subject: [pypy-dev] syntax sugar for stm TransactionQueue
In-Reply-To: <CAGGBzXJzovyZJ-j7V27s6SRFUv3+01EgSem1Ey75EKGf7QyCtg@mail.gmail.com>
References: <CAHPg78wQjwv1rBL_d_Wwjv647FBTjeH8x_14M7UWd=sOU+9Eaw@mail.gmail.com>
 <CAGGBzXJzovyZJ-j7V27s6SRFUv3+01EgSem1Ey75EKGf7QyCtg@mail.gmail.com>
Message-ID: <CAMSv6X060YyqhWUV5xWM3_PoFX7mC5udeSNJVb4KY=1UuUCSkQ@mail.gmail.com>

Hi,

> On 24 July 2015 at 00:40, Antonio Cuni <anto.cuni at gmail.com> wrote:
>> hi Armin,
>> following the discussion we had today, that TransactionQueue could be easier
>> to understand for people if you explain it as "a for loop in which you don't
>> know the order of the iteration"

On 24 July 2015 at 09:26, Dima Tisnek <dimaqq at gmail.com> wrote:
> Supplying argument list at decoration time is hardly ideal.
> Typical call site would pass argument sequence at invocation time.
> Thus you probably want to rewrite this decorator and allow application
> without argument list:

I'm open to suggestions.  However, TransactionQueue is also about
being Queue-like.  More items can be added dynamically by running
transactions, like this:

def foo(n):
    ...
    tr.add(foo, 2 * n)
    tr.add(foo, 2 * n + 1)

tr = TransactionQueue()
tr.add(foo, 0)
tr.run()

In both cases, the add()ed functions are not called immediately, but
only after the current "block of code" finishes. In one case it means
after foo() returned.  In the other case it is when tr.run() is
invoked.

Dima's suggestion in particular:

> @parallel
> def foo(item)
>     return item ** 123
>
> results = foo(range(9))

can be written without a magic decorator but with a simple map-like function:

   def foo(item):
       return item ** 123

   results = transaction.map(foo, range(9))

This can be added easily (and probably should).  However, if we go
down this path, I'd like to stop before we copy half of the API of the
multiprocessing module.  I'd prefer to apply my own feelings about the
recent CPython APIs (like async stuff).  My feeling is that instead of
adding tons and tons of new APIs, we should rather add only the core
language feature in a given (CPython/PyPy) release, and let 3rd-party
libraries be developed on top of it.  Optionally, a few releases
later, if there is one clear winner among these libraries, we can
bring it into the core.


A bient?t,

Armin.

From phyo.arkarlwin at gmail.com  Thu Jul 30 10:21:33 2015
From: phyo.arkarlwin at gmail.com (Phyo Arkar)
Date: Thu, 30 Jul 2015 14:51:33 +0630
Subject: [pypy-dev] Can we announce What we've build using pypy on this list?
Message-ID: <CA+HjJriSHTj7vUDd0xEwzBsS0sJXVcHF8FD_2FbyEKvSYNbvbA@mail.gmail.com>

We have built an addictive Team communication platform using python and
server side is purely running on pypy!

Can I announce on this mailing list?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20150730/4a0feaac/attachment.html>

From cfbolz at gmx.de  Thu Jul 30 10:25:12 2015
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Thu, 30 Jul 2015 10:25:12 +0200
Subject: [pypy-dev] Can we announce What we've build using pypy on this
	list?
In-Reply-To: <CA+HjJriSHTj7vUDd0xEwzBsS0sJXVcHF8FD_2FbyEKvSYNbvbA@mail.gmail.com>
References: <CA+HjJriSHTj7vUDd0xEwzBsS0sJXVcHF8FD_2FbyEKvSYNbvbA@mail.gmail.com>
Message-ID: <bbb1f9b6-e87b-45fb-af67-2733f811366e@email.android.com>

Hey, 

If it's not too often and too spammy, please feel free to post to the list. 

If you want to, you can write an experience report on the PyPy blog. Please get in touch with me or Maciek via a private mail in that case. 

Cheers, 

Carl Friedrich

On July 30, 2015 10:21:33 AM GMT+02:00, Phyo Arkar <phyo.arkarlwin at gmail.com> wrote:
>We have built an addictive Team communication platform using python and
>server side is purely running on pypy!
>
>Can I announce on this mailing list?
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>pypy-dev mailing list
>pypy-dev at python.org
>https://mail.python.org/mailman/listinfo/pypy-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20150730/c8436ac4/attachment.html>

From arigo at tunes.org  Thu Jul 30 12:03:39 2015
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 30 Jul 2015 12:03:39 +0200
Subject: [pypy-dev] Properties in RPython
Message-ID: <CAMSv6X3DcYqW8hX1Qq5hqVpn4C1-iougN1rT1gkG81HQ_huF_g@mail.gmail.com>

Hi all,

There are two tests in rpython/annotator/test/test_annrpython.py that
show that the property support in RPython is fragile.  The tests
sometimes pass and sometimes fail.  These particular tests fail more
systematically when run on "pypy py.test", but not exclusively, as
shown by http://buildbot.pypy.org/summary/longrepr?testname=TestAnnotateTestCase.%28%29.test_property_union_2&builder=own-linux-x86-64&build=3952&mod=annotator.test.test_annrpython
.

I've been trying unsuccessfully to fix them, twice now.   Could
someone else look?  As far as I'm concerned they mean that property
support should not be used, and should be disabled from rpython.  I'm
sure that someone would be unhappy with that, though.  I think the
only reasonable conclusion is to disable support with a global
variable set to False, and if someone really wants it, he can manually
set the global variable to True...


A bient?t,

Armin.

From lockhart at csl.cornell.edu  Thu Jul 30 18:07:03 2015
From: lockhart at csl.cornell.edu (Derek Lockhart)
Date: Thu, 30 Jul 2015 09:07:03 -0700
Subject: [pypy-dev] Properties in RPython
In-Reply-To: <CAMSv6X3DcYqW8hX1Qq5hqVpn4C1-iougN1rT1gkG81HQ_huF_g@mail.gmail.com>
References: <CAMSv6X3DcYqW8hX1Qq5hqVpn4C1-iougN1rT1gkG81HQ_huF_g@mail.gmail.com>
Message-ID: <CADEfmBtFDcYDbXkNCN_ZF0had+r8uDO8cm0HVFi=YMXh0jDuTA@mail.gmail.com>

This is an emphatic vote to **not** get rid of property support in
RPython.  We use this heavily to expose a clean API for describing ISAs in
 Pydgin: https://github.com/cornell-brg/pydgin

I'd be happy to take a look when I have more time next week; I'll pop into
IRC around then to chat.  Currently have my hands completely full trying to
get the final version of my thesis in by the Monday deadline!

Derek

On Thu, Jul 30, 2015 at 3:03 AM, Armin Rigo <arigo at tunes.org> wrote:

> Hi all,
>
> There are two tests in rpython/annotator/test/test_annrpython.py that
> show that the property support in RPython is fragile.  The tests
> sometimes pass and sometimes fail.  These particular tests fail more
> systematically when run on "pypy py.test", but not exclusively, as
> shown by
> http://buildbot.pypy.org/summary/longrepr?testname=TestAnnotateTestCase.%28%29.test_property_union_2&builder=own-linux-x86-64&build=3952&mod=annotator.test.test_annrpython
> .
>
> I've been trying unsuccessfully to fix them, twice now.   Could
> someone else look?  As far as I'm concerned they mean that property
> support should not be used, and should be disabled from rpython.  I'm
> sure that someone would be unhappy with that, though.  I think the
> only reasonable conclusion is to disable support with a global
> variable set to False, and if someone really wants it, he can manually
> set the global variable to True...
>
>
> A bient?t,
>
> Armin.
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20150730/6589853a/attachment.html>

From fijall at gmail.com  Fri Jul 31 13:24:52 2015
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Fri, 31 Jul 2015 13:24:52 +0200
Subject: [pypy-dev] Can we announce What we've build using pypy on this
	list?
In-Reply-To: <CA+HjJriSHTj7vUDd0xEwzBsS0sJXVcHF8FD_2FbyEKvSYNbvbA@mail.gmail.com>
References: <CA+HjJriSHTj7vUDd0xEwzBsS0sJXVcHF8FD_2FbyEKvSYNbvbA@mail.gmail.com>
Message-ID: <CAK5idxQmcdVmkjg-wL=GBt8Q-urx8pbyLgN8RbtMBGLAV3q1HA@mail.gmail.com>

Hi Phyo.

Mailing list is great, but indeed a blog post would have been even
better. Feel free to get in touch with me privately (although I will
be out of sight till Mon/Tue)

On Thu, Jul 30, 2015 at 10:21 AM, Phyo Arkar <phyo.arkarlwin at gmail.com> wrote:
> We have built an addictive Team communication platform using python and
> server side is purely running on pypy!
>
> Can I announce on this mailing list?
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>

From jimhorng at gmail.com  Fri Jul 31 11:21:44 2015
From: jimhorng at gmail.com (jim horng)
Date: Fri, 31 Jul 2015 17:21:44 +0800
Subject: [pypy-dev] pypy-2.3.1 memory profiling strategies
Message-ID: <CALZWULo13nxTrq2AAkjj5CrCEn486MLtCmc_Vny92adLNj3i0A@mail.gmail.com>

Hi all,
Not sure if it's appropriate to post here since there's no pypy-user group
:)
My question is as title, we got memory leak on pypy process and process
will down when out of memory, only on production site.
Our simplified environment as below:
* OS: Centos 6
* pypy-2.3.1

objgraph is seems the only profiling library we can use in this env, and
only with its partial function of printing all current objects in memory
instead of any further info such as references (.getrefcount not
implemented).
It turns out we can only see lots of "int", "str", "list" objects seems
leaking rather than knowing who are using them or whom they are using. :(

Our constraint are
* it's hard to change production python runtime since it might affect to
our users
* we cannot reproduce on other environment

"pmap" produced data only shows memory growing in a [anon] block.

Please advise if there's other tool/methodologies to attack this problem,
thanks a lot in advance :)

Best Regards,
Jim(???)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20150731/4ae2b1f8/attachment.html>

From arigo at tunes.org  Fri Jul 31 17:22:54 2015
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 31 Jul 2015 17:22:54 +0200
Subject: [pypy-dev] Increase the default stack limit
In-Reply-To: <CAK5idxTaJnDhHOX8bV7uKRWsA2i-cFMii3pcs9riy782PV=KwA@mail.gmail.com>
References: <CAK5idxTaJnDhHOX8bV7uKRWsA2i-cFMii3pcs9riy782PV=KwA@mail.gmail.com>
Message-ID: <CAMSv6X2qZo-MQ==Vuaq0kG=pk22DpLavO6kGsOGf6BVxGFopRA@mail.gmail.com>

Hi Fijal,

On 17 July 2015 at 10:59, Maciej Fijalkowski <fijall at gmail.com> wrote:
> The default stack limit is (conservatively) set at 700k or so. Can we
> push it to ~3M by default? (this is still far below 8M or so which is
> the default on linux)

Sadly, on Windows (I think it is XP, at least cl.exe is version 15.*
and linker is version 9.*; translation of that to a MSVC version is
left as an exercise for the reader) the default limit is still 1 MB.
That's why the limit was set to 768KB in the first place.  If you want
to increase it, you'll need some #ifdefs...


A bient?t,

Armin.