From rymg19 at gmail.com  Fri Nov  1 00:20:04 2013
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Thu, 31 Oct 2013 18:20:04 -0500
Subject: [pypy-dev] RPython Environment Variables
Message-ID: <CAO41-mM+hv8up1fTkeYmqGdEhrENzK7rETPMtXM3AfOQX12NQg@mail.gmail.com>

I am writing a tool in RPython that needs to access environment variables.
Now, all has been going well, but I noticed something interesting. The
RPython translator sees os.environ as constant. But, will it still be
constant on other platforms?

What I'm saying is, are the environment variables permanently embedded into
the application? i.e., if I run it on another PC, the values will be the
same? Or is there an RLib equivalent? Because I really don't feel like
writing a C wrapper...

-- 
Ryan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131031/72201260/attachment.html>

From benjamin at python.org  Fri Nov  1 00:23:43 2013
From: benjamin at python.org (Benjamin Peterson)
Date: Thu, 31 Oct 2013 19:23:43 -0400
Subject: [pypy-dev] RPython Environment Variables
In-Reply-To: <CAO41-mM+hv8up1fTkeYmqGdEhrENzK7rETPMtXM3AfOQX12NQg@mail.gmail.com>
References: <CAO41-mM+hv8up1fTkeYmqGdEhrENzK7rETPMtXM3AfOQX12NQg@mail.gmail.com>
Message-ID: <CAPZV6o_yhKBRnJqX8NAAXzbNo-We7ipxZspQpj7C0EEvNk376A@mail.gmail.com>

Presumably, you can just use get/setenv()

2013/10/31 Ryan Gonzalez <rymg19 at gmail.com>:
> I am writing a tool in RPython that needs to access environment variables.
> Now, all has been going well, but I noticed something interesting. The
> RPython translator sees os.environ as constant. But, will it still be
> constant on other platforms?
>
> What I'm saying is, are the environment variables permanently embedded into
> the application? i.e., if I run it on another PC, the values will be the
> same? Or is there an RLib equivalent? Because I really don't feel like
> writing a C wrapper...
>
> --
> Ryan
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>



-- 
Regards,
Benjamin

From amauryfa at gmail.com  Fri Nov  1 01:16:01 2013
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Fri, 1 Nov 2013 01:16:01 +0100
Subject: [pypy-dev] RPython Environment Variables
In-Reply-To: <CAO41-mM+hv8up1fTkeYmqGdEhrENzK7rETPMtXM3AfOQX12NQg@mail.gmail.com>
References: <CAO41-mM+hv8up1fTkeYmqGdEhrENzK7rETPMtXM3AfOQX12NQg@mail.gmail.com>
Message-ID: <CAGmFidapjXz_pw7Vma_BZGC5LsWH4LUrm3N6+67EQOPjitGFiA@mail.gmail.com>

2013/11/1 Ryan Gonzalez <rymg19 at gmail.com>

> I am writing a tool in RPython that needs to access environment variables.
> Now, all has been going well, but I noticed something interesting. The
> RPython translator sees os.environ as constant. But, will it still be
> constant on other platforms?
>
> What I'm saying is, are the environment variables permanently embedded
> into the application? i.e., if I run it on another PC, the values will be
> the same? Or is there an RLib equivalent? Because I really don't feel like
> writing a C wrapper...
>

RPython magic...

In RPython, os.environ is not a dictionary, but a clever wrapper around
getenv() and putenv().
See the implementation in ll_os_environ.py:
https://bitbucket.org/pypy/pypy/src/default/rpython/rtyper/module/ll_os_environ.py
- as an object, os.environ is equivalent to None: nothing is embedded in
the application.
- only some methods are implemented, and redirect to some implementation
that use the C primitives.

Yes, I also found this "Controller" crazy the first time I saw it. But very
clever and powerful.

-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131101/ef6f5a9c/attachment.html>

From roadhome at gmail.com  Fri Nov  1 02:07:33 2013
From: roadhome at gmail.com (Ricky Park)
Date: Fri, 1 Nov 2013 10:07:33 +0900
Subject: [pypy-dev] how to get current max_heap_size value of minimark in
	pypy 2.x
Message-ID: <CAM7TyLXY7ZNd4ifgP3Uo7_CYsw_qBLPYb+oXsopo_KsEY8quLQ@mail.gmail.com>

Hello,

I read some articles about setting PYPY_GC_MAX environment variable.
But I can't find how to get current max_heap_size value of minimark.

Please let me know how-to :)

Thanks,
Ricky
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131101/d4256edd/attachment.html>

From mail2shine at qq.com  Fri Nov  1 04:07:36 2013
From: mail2shine at qq.com (=?ISO-8859-1?B?S2FTaGluaW5n?=)
Date: Fri, 1 Nov 2013 11:07:36 +0800
Subject: [pypy-dev] pip install rbtree fail in pypy env!
In-Reply-To: <CAGGBd_ofMP07gxBD55UXJWkQ_eE-4=tmOc3=TaX5eGKqMAkNcA@mail.gmail.com>
References: <tencent_4E3B56BE6174923D5FEAF3E9@qq.com>
 <CAMSv6X0XdHCAJmjocbSQ=KRF9DL2-7Xrc_Ewau51P+uT-euDWg@mail.gmail.com>
 <CAGGBd_ofMP07gxBD55UXJWkQ_eE-4=tmOc3=TaX5eGKqMAkNcA@mail.gmail.com>
Message-ID: <tencent_4E2B300B0757C47E4FF2C139@qq.com>

tks,but 

>>>>test-red_black_dict_mod.py under pypy-2.1. consume :3.89574193954
import sys
import red_black_dict_mod
import time
import random


t = red_black_dict_mod.RedBlackTree()

begin = time.time()
for i in range(100000):
        k = random.randint(0, 10000000)
        t[k] = k
end = time.time()

print end - begin



while

>>>>test-rbtree.py under python2.7.3 .consume:0.35432600975
import sys
import rbtree
import time
import random


t = rbtree.rbtree()

begin = time.time()
for i in range(100000):
        k = random.randint(0, 10000000)
        t[k] = k
end = time.time()

print end - begin

------------------ Original ------------------
From:  "Dan Stromberg";<drsalists at gmail.com>;
Date:  Fri, Nov 1, 2013 06:29 AM
To:  "Armin Rigo"<arigo at tunes.org>; 
Cc:  "KaShining"<mail2shine at qq.com>; "pypy-dev"<pypy-dev at python.org>; 
Subject:  Re: [pypy-dev] pip install rbtree fail in pypy env!




On Thu, Oct 31, 2013 at 1:53 AM, Armin Rigo <arigo at tunes.org> wrote:
 Hi KaShining,
 
 On Thu, Oct 31, 2013 at 9:32 AM, KaShining <mail2shine at qq.com> wrote:
 >     src/rbtree.c: In function '__pyx_f_6rbtree_6rbtree_byOffset':
 
 This is using Cython.  Compiling Cython modules with PyPy
 kind-of-works but is shaky.
 

Here's a pure python red black tree:
https://pypi.python.org/pypi/red-black-tree-mod
 



Full disclosure: I put it on Pypi after getting it passing pylint and such.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131101/de984cd7/attachment.html>

From drsalists at gmail.com  Fri Nov  1 05:04:00 2013
From: drsalists at gmail.com (Dan Stromberg)
Date: Thu, 31 Oct 2013 21:04:00 -0700
Subject: [pypy-dev] pip install rbtree fail in pypy env!
In-Reply-To: <tencent_4E2B300B0757C47E4FF2C139@qq.com>
References: <tencent_4E3B56BE6174923D5FEAF3E9@qq.com>
 <CAMSv6X0XdHCAJmjocbSQ=KRF9DL2-7Xrc_Ewau51P+uT-euDWg@mail.gmail.com>
 <CAGGBd_ofMP07gxBD55UXJWkQ_eE-4=tmOc3=TaX5eGKqMAkNcA@mail.gmail.com>
 <tencent_4E2B300B0757C47E4FF2C139@qq.com>
Message-ID: <CAGGBd_pVG-D+cM-fMWt-_gp=TC77x2832MR2sGatEyMPhxxNmw@mail.gmail.com>

On Thu, Oct 31, 2013 at 8:07 PM, KaShining <mail2shine at qq.com> wrote:

>
> tks,but
>
> >>>>test-red_black_dict_mod.py under pypy-2.1. consume :3.89574193954
> >>>>test-rbtree.py under python2.7.3 .consume:0.35432600975s at gmail.com<drsalists at gmail.com>
> >;
>

Interesting.

Do you need good performance?

You might consider a treap:
https://pypi.python.org/pypi/treap

It has a cython version, as well as pure python.  Supposedly treaps
outperform red black trees on average, but give a larger standard deviation
in operation times.  I've used this treap code on CPython 2.x, CPython 3.x,
Pypy, Pypy3 (beta) and Jython.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131031/19a79692/attachment-0001.html>

From arigo at tunes.org  Fri Nov  1 10:06:57 2013
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 1 Nov 2013 10:06:57 +0100
Subject: [pypy-dev] pip install rbtree fail in pypy env!
In-Reply-To: <tencent_4E2B300B0757C47E4FF2C139@qq.com>
References: <tencent_4E3B56BE6174923D5FEAF3E9@qq.com>
 <CAMSv6X0XdHCAJmjocbSQ=KRF9DL2-7Xrc_Ewau51P+uT-euDWg@mail.gmail.com>
 <CAGGBd_ofMP07gxBD55UXJWkQ_eE-4=tmOc3=TaX5eGKqMAkNcA@mail.gmail.com>
 <tencent_4E2B300B0757C47E4FF2C139@qq.com>
Message-ID: <CAMSv6X0_kFc_Yi91RgQV+ekqT_8C-M07N_ioxGUb22zV_0rxMw@mail.gmail.com>

Hi KaShining,

Yes, the issue then is that pure Python equivalents are often slower
than their C version --- for example, this red_black_tree_mod file is
huge for the job (more than 1000 lines), written in a Java-ish style
with tons of small overridden methods, but full of properties,
recursive function invocations, and so on.  Getting 10 times slower
than the C version is not good, but I guess that the C version was
more optimized.  Often we can hope to get at most a few times slower.

PyPy is not faster at everything.  Usually there is no way it can be
faster than a program that exercices only a CPython C extension
module.  In some cases like this one, the solutions you have with PyPy
are all a lot slower --- both some pure Python version, and the cpyext
version.  The pure Python version could be rewritten with
performance-on-PyPy in mind (just like the C version was written with
performance-on-CPython in mind).

Note that the point of red_black_tree_mod is very unclear to me.  It
seems to replace plain dicts.  Why can't you use plain dicts?


A bient?t,

Armin.

From arigo at tunes.org  Fri Nov  1 10:10:49 2013
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 1 Nov 2013 10:10:49 +0100
Subject: [pypy-dev] how to get current max_heap_size value of minimark
 in pypy 2.x
In-Reply-To: <CAM7TyLXY7ZNd4ifgP3Uo7_CYsw_qBLPYb+oXsopo_KsEY8quLQ@mail.gmail.com>
References: <CAM7TyLXY7ZNd4ifgP3Uo7_CYsw_qBLPYb+oXsopo_KsEY8quLQ@mail.gmail.com>
Message-ID: <CAMSv6X2OExPicvCp9cQ+v7zSABw_98_z1NVpY+=w=wh+7E6tAA@mail.gmail.com>

Hi Ricky,

On Fri, Nov 1, 2013 at 2:07 AM, Ricky Park <roadhome at gmail.com> wrote:
> I read some articles about setting PYPY_GC_MAX environment variable.
> But I can't find how to get current max_heap_size value of minimark.

os.environ['PYPY_GC_MAX']?  There is no direct way to get or set the
maximum from the Python program itself.


A bient?t,

Armin.

From davewats at cisco.com  Fri Nov  1 15:17:17 2013
From: davewats at cisco.com (Dave Watson (davewats))
Date: Fri, 1 Nov 2013 14:17:17 +0000
Subject: [pypy-dev] Question on installing pypy and the source code ..
In-Reply-To: <1383257144.2618.16.camel@newpride>
References: <7265C53BF726F640BF92D1F05619FDE7201819D4@xmb-aln-x11.cisco.com>
 <1383237092.2861.122.camel@newpride>
 <7265C53BF726F640BF92D1F05619FDE720181B72@xmb-aln-x11.cisco.com>
 <1383257144.2618.16.camel@newpride>
Message-ID: <7265C53BF726F640BF92D1F05619FDE720182E5F@xmb-aln-x11.cisco.com>

Hi Yury ..

Thanks for the info .. I will try to see I can add the appropriate includes/libs in via the CFLAGS and LFLAGS ..

Thanks again for your help .. 

Cheers

Dave W. 

-----Original Message-----
From: Yury V. Zaytsev [mailto:yury at shurup.com] 
Sent: Thursday, October 31, 2013 6:06 PM
To: Dave Watson (davewats)
Cc: pypy-dev at python.org; Steve Marsh (stevmars); Gilbert Ramirez (gilramir)
Subject: RE: [pypy-dev] Question on installing pypy and the source code ..

On Thu, 2013-10-31 at 19:10 +0000, Dave Watson (davewats) wrote:
> 
> I was wondering I you or someone else might know of a good place that 
> would allow the required alternative library location to be added to 
> the configuration ..

I think that the easiest way to go is to manipulate CFLAGS and LDFLAGS directly. Just export them and they should be picked up automatically.

If you have a specific problem, please post the raw error messages, otherwise, it's only possible to offer very generic suggestions.

--
Sincerely yours,
Yury V. Zaytsev



From mail2shine at qq.com  Sat Nov  2 10:44:59 2013
From: mail2shine at qq.com (=?utf-8?B?S2FTaGluaW5n?=)
Date: Sat, 2 Nov 2013 17:44:59 +0800
Subject: [pypy-dev] pip install rbtree fail in pypy env!
In-Reply-To: <CAMSv6X0_kFc_Yi91RgQV+ekqT_8C-M07N_ioxGUb22zV_0rxMw@mail.gmail.com>
References: <tencent_4E3B56BE6174923D5FEAF3E9@qq.com>
 <CAMSv6X0XdHCAJmjocbSQ=KRF9DL2-7Xrc_Ewau51P+uT-euDWg@mail.gmail.com>
 <CAGGBd_ofMP07gxBD55UXJWkQ_eE-4=tmOc3=TaX5eGKqMAkNcA@mail.gmail.com>
 <tencent_4E2B300B0757C47E4FF2C139@qq.com>
 <CAMSv6X0_kFc_Yi91RgQV+ekqT_8C-M07N_ioxGUb22zV_0rxMw@mail.gmail.com>
Message-ID: <tencent_79F126087602F8325627044F@qq.com>

I found bintrees-fastrbtree is the best choice?

?some test code?
def test_dict(tree, msg):
    begin = time.time()
    for i in range(100000):
        k = random.randint(0, 10000000)
        tree[k] = k
    end = time.time()

    print msg,end - begin
?

bintrees_rbtree under cpython    0.393133878708
bintrees_fast_rbtree under pypy 0.357424020767


------------------ Original ------------------
From:  "Armin Rigo";<arigo at tunes.org>;
Date:  Fri, Nov 1, 2013 05:06 PM
To:  "KaShining"<mail2shine at qq.com>; 
Cc:  "Dan Stromberg"<drsalists at gmail.com>; "pypy-dev"<pypy-dev at python.org>; 
Subject:  Re: [pypy-dev] pip install rbtree fail in pypy env!



Hi KaShining,

Yes, the issue then is that pure Python equivalents are often slower
than their C version --- for example, this red_black_tree_mod file is
huge for the job (more than 1000 lines), written in a Java-ish style
with tons of small overridden methods, but full of properties,
recursive function invocations, and so on.  Getting 10 times slower
than the C version is not good, but I guess that the C version was
more optimized.  Often we can hope to get at most a few times slower.

PyPy is not faster at everything.  Usually there is no way it can be
faster than a program that exercices only a CPython C extension
module.  In some cases like this one, the solutions you have with PyPy
are all a lot slower --- both some pure Python version, and the cpyext
version.  The pure Python version could be rewritten with
performance-on-PyPy in mind (just like the C version was written with
performance-on-CPython in mind).

Note that the point of red_black_tree_mod is very unclear to me.  It
seems to replace plain dicts.  Why can't you use plain dicts?


A bient?t,

Armin.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131102/260980b6/attachment.html>

From rymg19 at gmail.com  Mon Nov  4 00:07:56 2013
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Sun, 3 Nov 2013 17:07:56 -0600
Subject: [pypy-dev] Using PyPy's GC
Message-ID: <CAO41-mOVP_OP-F3zVMJTUnSiBe_8Ta7+N0KErpMvCknuk-i8ng@mail.gmail.com>

If I'm implementing my own language in RPython, can I use the same garbage
collector that PyPy uses? Or do I have to write my own?

-- 
Ryan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131103/c84d318e/attachment.html>

From fijall at gmail.com  Mon Nov  4 00:57:26 2013
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Mon, 4 Nov 2013 01:57:26 +0200
Subject: [pypy-dev] Using PyPy's GC
In-Reply-To: <CAO41-mOVP_OP-F3zVMJTUnSiBe_8Ta7+N0KErpMvCknuk-i8ng@mail.gmail.com>
References: <CAO41-mOVP_OP-F3zVMJTUnSiBe_8Ta7+N0KErpMvCknuk-i8ng@mail.gmail.com>
Message-ID: <CAK5idxT9W7F82gbEYNtNWtcShQ8cjLJjyORvheJ+1PV-h=R52w@mail.gmail.com>

On Mon, Nov 4, 2013 at 1:07 AM, Ryan Gonzalez <rymg19 at gmail.com> wrote:
> If I'm implementing my own language in RPython, can I use the same garbage
> collector that PyPy uses? Or do I have to write my own?
>
> --
> Ryan
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>

Yes, you can use the same GC. RPython is a garbage collected language.
In fact you can use the same JIT.

From mail2shine at qq.com  Mon Nov  4 08:47:12 2013
From: mail2shine at qq.com (=?ISO-8859-1?B?S2FTaGluaW5n?=)
Date: Mon, 4 Nov 2013 15:47:12 +0800
Subject: [pypy-dev] pypy about c-extension
Message-ID: <tencent_331F26CC25D78A4F1D107EFE@qq.com>

I use this method to enhance protobuf
http://yz.mit.edu/wp/fast-native-c-protocol-buffers-from-python/


but after i gen the podpbpypy.so
 then import podpbpypy:

Traceback (most recent call last):
  File "app_main.py", line 72, in run_toplevel
  File "t2.py", line 4, in <module>
    import podpbpypy
ImportError: No module named podpbpypy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131104/9ec0fde9/attachment.html>

From arigo at tunes.org  Mon Nov  4 08:59:49 2013
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 4 Nov 2013 08:59:49 +0100
Subject: [pypy-dev] Using PyPy's GC
In-Reply-To: <CAK5idxT9W7F82gbEYNtNWtcShQ8cjLJjyORvheJ+1PV-h=R52w@mail.gmail.com>
References: <CAO41-mOVP_OP-F3zVMJTUnSiBe_8Ta7+N0KErpMvCknuk-i8ng@mail.gmail.com>
 <CAK5idxT9W7F82gbEYNtNWtcShQ8cjLJjyORvheJ+1PV-h=R52w@mail.gmail.com>
Message-ID: <CAMSv6X0y_oETq+oC24CDEZcaMJhCm+T4C8JxPnFvbceaoJxBNA@mail.gmail.com>

Hi,

On Mon, Nov 4, 2013 at 12:57 AM, Maciej Fijalkowski <fijall at gmail.com> wrote:
>> If I'm implementing my own language in RPython, can I use the same garbage
>> collector that PyPy uses? Or do I have to write my own?
>
> Yes, you can use the same GC.

More precisely, you *have* to use the same GC (or possibly a different
one that is written inside RPython too).  You cannot use RPython
without also using the GC.


A bient?t,

Armin.

From arigo at tunes.org  Mon Nov  4 09:02:38 2013
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 4 Nov 2013 09:02:38 +0100
Subject: [pypy-dev] pypy about c-extension
In-Reply-To: <tencent_331F26CC25D78A4F1D107EFE@qq.com>
References: <tencent_331F26CC25D78A4F1D107EFE@qq.com>
Message-ID: <CAMSv6X0iFEbQDcej1N3W=d1duzUiny_fwfWeAdxOFg9M6VHjQQ@mail.gmail.com>

Hi,

On Mon, Nov 4, 2013 at 8:47 AM, KaShining <mail2shine at qq.com> wrote:
> but after i gen the podpbpypy.so

How?


Armin

From mail2shine at qq.com  Mon Nov  4 09:04:54 2013
From: mail2shine at qq.com (=?ISO-8859-1?B?S2FTaGluaW5n?=)
Date: Mon, 4 Nov 2013 16:04:54 +0800
Subject: [pypy-dev] pypy about c-extension
In-Reply-To: <CAMSv6X0iFEbQDcej1N3W=d1duzUiny_fwfWeAdxOFg9M6VHjQQ@mail.gmail.com>
References: <tencent_331F26CC25D78A4F1D107EFE@qq.com>
 <CAMSv6X0iFEbQDcej1N3W=d1duzUiny_fwfWeAdxOFg9M6VHjQQ@mail.gmail.com>
Message-ID: <tencent_2C25FC213F3C3CCD29DC463D@qq.com>

import podpbpypy
ImportError: No module named podpbpypy

seen can't find podpbpypy.so




------------------ Original ------------------
From:  "Armin Rigo";<arigo at tunes.org>;
Date:  Mon, Nov 4, 2013 04:02 PM
To:  "KaShining"<mail2shine at qq.com>; 
Cc:  "pypy-dev"<pypy-dev at python.org>; 
Subject:  Re: [pypy-dev] pypy about c-extension



Hi,

On Mon, Nov 4, 2013 at 8:47 AM, KaShining <mail2shine at qq.com> wrote:
> but after i gen the podpbpypy.so

How?


Armin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131104/8a6b65e1/attachment.html>

From arigo at tunes.org  Mon Nov  4 09:08:14 2013
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 4 Nov 2013 09:08:14 +0100
Subject: [pypy-dev] pypy about c-extension
In-Reply-To: <tencent_2C25FC213F3C3CCD29DC463D@qq.com>
References: <tencent_331F26CC25D78A4F1D107EFE@qq.com>
 <CAMSv6X0iFEbQDcej1N3W=d1duzUiny_fwfWeAdxOFg9M6VHjQQ@mail.gmail.com>
 <tencent_2C25FC213F3C3CCD29DC463D@qq.com>
Message-ID: <CAMSv6X1oeJh9p1MF5fboFsMwt5OURggE1mODZmaGYQA4XGKKFA@mail.gmail.com>

Hi,

On Mon, Nov 4, 2013 at 9:04 AM, KaShining <mail2shine at qq.com> wrote:
> import podpbpypy
> ImportError: No module named podpbpypy
>
> seen can't find podpbpypy.so

There is no point repeating what you already said.  I'm asking, how
did you build the .so in the first place.


Armin

From william.leslie.ttg at gmail.com  Mon Nov  4 09:12:56 2013
From: william.leslie.ttg at gmail.com (William ML Leslie)
Date: Mon, 4 Nov 2013 19:12:56 +1100
Subject: [pypy-dev] pypy about c-extension
In-Reply-To: <CAMSv6X1oeJh9p1MF5fboFsMwt5OURggE1mODZmaGYQA4XGKKFA@mail.gmail.com>
References: <tencent_331F26CC25D78A4F1D107EFE@qq.com>
 <CAMSv6X0iFEbQDcej1N3W=d1duzUiny_fwfWeAdxOFg9M6VHjQQ@mail.gmail.com>
 <tencent_2C25FC213F3C3CCD29DC463D@qq.com>
 <CAMSv6X1oeJh9p1MF5fboFsMwt5OURggE1mODZmaGYQA4XGKKFA@mail.gmail.com>
Message-ID: <CAHgd1hHajUYnVVsLx-iioawVsVCXh_ZTUV6Hm9n9S2_kvtG-oA@mail.gmail.com>

The blog post describes how to produce a cpython module .so for your
schema. Needless to say that is pointless on pypy.
On 04/11/2013 7:09 PM, "Armin Rigo" <arigo at tunes.org> wrote:

> Hi,
>
> On Mon, Nov 4, 2013 at 9:04 AM, KaShining <mail2shine at qq.com> wrote:
> > import podpbpypy
> > ImportError: No module named podpbpypy
> >
> > seen can't find podpbpypy.so
>
> There is no point repeating what you already said.  I'm asking, how
> did you build the .so in the first place.
>
>
> Armin
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131104/63b32eb9/attachment.html>

From mail2shine at qq.com  Mon Nov  4 09:13:41 2013
From: mail2shine at qq.com (=?gb18030?B?S2FTaGluaW5n?=)
Date: Mon, 4 Nov 2013 16:13:41 +0800
Subject: [pypy-dev] pypy about c-extension
In-Reply-To: <CAMSv6X1oeJh9p1MF5fboFsMwt5OURggE1mODZmaGYQA4XGKKFA@mail.gmail.com>
References: <tencent_331F26CC25D78A4F1D107EFE@qq.com>
 <CAMSv6X0iFEbQDcej1N3W=d1duzUiny_fwfWeAdxOFg9M6VHjQQ@mail.gmail.com>
 <tencent_2C25FC213F3C3CCD29DC463D@qq.com>
 <CAMSv6X1oeJh9p1MF5fboFsMwt5OURggE1mODZmaGYQA4XGKKFA@mail.gmail.com>
Message-ID: <tencent_7FE6D83B27BB1ABC2D291C71@qq.com>

+++++++++++++++++++
addressbook.proto
package tutorial;

message Person {
  required string name = 1;
  required int32 id = 2;
  optional string email = 3;

  enum PhoneType {
    MOBILE = 0;
    HOME = 1;
    WORK = 2;
  }

  message PhoneNumber {
    required string number = 1;
    optional PhoneType type = 2 [default = HOME];
  }

  repeated PhoneNumber phone = 4;
}

message AddressBook {
  repeated Person person = 1;
}

 /usr/local/bin/protoc -I=./ --cpp_out=./ ./addressbook.proto
 gen  addressboo.pb.h and addressboo.pb.cc

 
addressbook.c?
 #include <Python.h>

static PyMethodDef podMethods[] = {
   {NULL, NULL, 0, NULL}        /* Sentinel */
 };

PyMODINIT_FUNC
 initpodpb(void)
 {
   PyObject *m;

  m = Py_InitModule("podpb", podMethods);
   if (m == NULL)
     return;
 }

 

setup.py
 
import os
 import platform

from setuptools import setup, find_packages
 from distutils.core import setup, Extension

setup(
     ext_modules=[Extension('podpb',
 sources=['./addressbook.c','./addressbook.pb.cc'], libraries=['protobuf'])]
     )

 

 python setup.py build
gen build/lib.linux-x86_64-2.7/podpbpypy.so

then in my test.py
import podpbpypy

throw err:

ImportError: No module named podpbpypy

 

 




------------------ Original ------------------
From:  "Armin Rigo";<arigo at tunes.org>;
Date:  Mon, Nov 4, 2013 04:08 PM
To:  "KaShining"<mail2shine at qq.com>; 
Cc:  "pypy-dev"<pypy-dev at python.org>; 
Subject:  Re: [pypy-dev] pypy about c-extension



Hi,

On Mon, Nov 4, 2013 at 9:04 AM, KaShining <mail2shine at qq.com> wrote:
> import podpbpypy
> ImportError: No module named podpbpypy
>
> seen can't find podpbpypy.so

There is no point repeating what you already said.  I'm asking, how
did you build the .so in the first place.


Armin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131104/bd6304dc/attachment.html>

From mail2shine at qq.com  Mon Nov  4 09:24:39 2013
From: mail2shine at qq.com (=?gb18030?B?S2FTaGluaW5n?=)
Date: Mon, 4 Nov 2013 16:24:39 +0800
Subject: [pypy-dev] =?gb18030?b?u9i4tKO6ICBweXB5IGFib3V0IGMtZXh0ZW5zaW9u?=
In-Reply-To: <tencent_7FE6D83B27BB1ABC2D291C71@qq.com>
References: <tencent_331F26CC25D78A4F1D107EFE@qq.com>
 <CAMSv6X0iFEbQDcej1N3W=d1duzUiny_fwfWeAdxOFg9M6VHjQQ@mail.gmail.com>
 <tencent_2C25FC213F3C3CCD29DC463D@qq.com>
 <CAMSv6X1oeJh9p1MF5fboFsMwt5OURggE1mODZmaGYQA4XGKKFA@mail.gmail.com>
 <tencent_7FE6D83B27BB1ABC2D291C71@qq.com>
Message-ID: <tencent_66692B4624FFD9B450ED90FA@qq.com>

------------------ ???? ------------------
???: "KaShining";<mail2shine at qq.com>;
????: 2013?11?4?(???) ??4:13
???: "Armin Rigo"<arigo at tunes.org>; 
??: "pypy-dev"<pypy-dev at python.org>; 
??: Re: [pypy-dev] pypy about c-extension



+++++++++++++++++++
addressbook.proto
package tutorial;

message Person {
  required string name = 1;
  required int32 id = 2;
  optional string email = 3;

  enum PhoneType {
    MOBILE = 0;
    HOME = 1;
    WORK = 2;
  }

  message PhoneNumber {
    required string number = 1;
    optional PhoneType type = 2 [default = HOME];
  }

  repeated PhoneNumber phone = 4;
}

message AddressBook {
  repeated Person person = 1;
}

 /usr/local/bin/protoc -I=./ --cpp_out=./ ./addressbook.proto
 gen  addressboo.pb.h and addressboo.pb.cc

 
addressbook.c?
 #include <Python.h>

static PyMethodDef podMethods[] = {
   {NULL, NULL, 0, NULL}        /* Sentinel */
 };

PyMODINIT_FUNC
 initpodpb(void)
 {
   PyObject *m;

  m = Py_InitModule("podpb", podMethods);
   if (m == NULL)
     return;
 }

 

setup.py
 
import os
 import platform

from setuptools import setup, find_packages
 from distutils.core import setup, Extension

setup(
     ext_modules=[Extension('podpb',
 sources=['./addressbook.c','./addressbook.pb.cc'], libraries=['protobuf'])]
     )

 

 python setup.py build
gen build/lib.linux-x86_64-2.7/podpbpypy.so

then in my test.py
import podpbpypy

throw err:

ImportError: No module named podpbpypy

 

 




------------------ Original ------------------
From:  "Armin Rigo";<arigo at tunes.org>;
Date:  Mon, Nov 4, 2013 04:08 PM
To:  "KaShining"<mail2shine at qq.com>; 
Cc:  "pypy-dev"<pypy-dev at python.org>; 
Subject:  Re: [pypy-dev] pypy about c-extension



Hi,

On Mon, Nov 4, 2013 at 9:04 AM, KaShining <mail2shine at qq.com> wrote:
> import podpbpypy
> ImportError: No module named podpbpypy
>
> seen can't find podpbpypy.so

There is no point repeating what you already said.  I'm asking, how
did you build the .so in the first place.


Armin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131104/210071aa/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: protobuf.pypy.tar.gz
Type: application/octet-stream
Size: 8878 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131104/210071aa/attachment-0001.obj>

From arigo at tunes.org  Mon Nov  4 09:25:52 2013
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 4 Nov 2013 09:25:52 +0100
Subject: [pypy-dev] pypy about c-extension
In-Reply-To: <tencent_7FE6D83B27BB1ABC2D291C71@qq.com>
References: <tencent_331F26CC25D78A4F1D107EFE@qq.com>
 <CAMSv6X0iFEbQDcej1N3W=d1duzUiny_fwfWeAdxOFg9M6VHjQQ@mail.gmail.com>
 <tencent_2C25FC213F3C3CCD29DC463D@qq.com>
 <CAMSv6X1oeJh9p1MF5fboFsMwt5OURggE1mODZmaGYQA4XGKKFA@mail.gmail.com>
 <tencent_7FE6D83B27BB1ABC2D291C71@qq.com>
Message-ID: <CAMSv6X39GrM7-eXPbzipuCbRTAPi_6CJ-6fv_RRUGcsTajn6kg@mail.gmail.com>

Hi KaShining,

You seem to be mixing CPython and PyPy.  If you use "python setup.py",
meaning CPython, then you make a .so that cannot be imported on PyPy.
You need "pypy setup.py".

But please also consider William's comment.

Armin

From mail2shine at qq.com  Mon Nov  4 09:31:33 2013
From: mail2shine at qq.com (=?ISO-8859-1?B?S2FTaGluaW5n?=)
Date: Mon, 4 Nov 2013 16:31:33 +0800
Subject: [pypy-dev] pypy about c-extension
In-Reply-To: <CAMSv6X39GrM7-eXPbzipuCbRTAPi_6CJ-6fv_RRUGcsTajn6kg@mail.gmail.com>
References: <tencent_331F26CC25D78A4F1D107EFE@qq.com>
 <CAMSv6X0iFEbQDcej1N3W=d1duzUiny_fwfWeAdxOFg9M6VHjQQ@mail.gmail.com>
 <tencent_2C25FC213F3C3CCD29DC463D@qq.com>
 <CAMSv6X1oeJh9p1MF5fboFsMwt5OURggE1mODZmaGYQA4XGKKFA@mail.gmail.com>
 <tencent_7FE6D83B27BB1ABC2D291C71@qq.com>
 <CAMSv6X39GrM7-eXPbzipuCbRTAPi_6CJ-6fv_RRUGcsTajn6kg@mail.gmail.com>
Message-ID: <tencent_1179C3E022FCE2C21423BF59@qq.com>

NO.
I gen the .so under pypy-virtualenv.




------------------ Original ------------------
From:  "Armin Rigo";<arigo at tunes.org>;
Date:  Mon, Nov 4, 2013 04:25 PM
To:  "KaShining"<mail2shine at qq.com>; 
Cc:  "pypy-dev"<pypy-dev at python.org>; 
Subject:  Re: [pypy-dev] pypy about c-extension



Hi KaShining,

You seem to be mixing CPython and PyPy.  If you use "python setup.py",
meaning CPython, then you make a .so that cannot be imported on PyPy.
You need "pypy setup.py".

But please also consider William's comment.

Armin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131104/391b7550/attachment.html>

From arigo at tunes.org  Mon Nov  4 09:36:46 2013
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 4 Nov 2013 09:36:46 +0100
Subject: [pypy-dev] pypy about c-extension
In-Reply-To: <tencent_1179C3E022FCE2C21423BF59@qq.com>
References: <tencent_331F26CC25D78A4F1D107EFE@qq.com>
 <CAMSv6X0iFEbQDcej1N3W=d1duzUiny_fwfWeAdxOFg9M6VHjQQ@mail.gmail.com>
 <tencent_2C25FC213F3C3CCD29DC463D@qq.com>
 <CAMSv6X1oeJh9p1MF5fboFsMwt5OURggE1mODZmaGYQA4XGKKFA@mail.gmail.com>
 <tencent_7FE6D83B27BB1ABC2D291C71@qq.com>
 <CAMSv6X39GrM7-eXPbzipuCbRTAPi_6CJ-6fv_RRUGcsTajn6kg@mail.gmail.com>
 <tencent_1179C3E022FCE2C21423BF59@qq.com>
Message-ID: <CAMSv6X0CgjH=Kr0H=DvVm4TYOpo=UGPqUg+oU+4fxMads_mELw@mail.gmail.com>

Hi,

Then maybe this?  The .so is produced by "setup.py build" in
build/lib.linux-x86_64-2.7/podpbpypy.so.  It can't be imported from
there.  You need "setup.py install" too.


Armin

From amauryfa at gmail.com  Mon Nov  4 10:16:33 2013
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Mon, 4 Nov 2013 10:16:33 +0100
Subject: [pypy-dev] pypy about c-extension
In-Reply-To: <tencent_331F26CC25D78A4F1D107EFE@qq.com>
References: <tencent_331F26CC25D78A4F1D107EFE@qq.com>
Message-ID: <CAGmFidaNcR9mkZXzpsCsCROAKJE5xvuZpCf80abTnFk80BmTNg@mail.gmail.com>

Hi,

2013/11/4 KaShining <mail2shine at qq.com>

> I use this method to enhance protobuf
> http://yz.mit.edu/wp/fast-native-c-protocol-buffers-from-python/


Don't.
With PyPy, protobuf are much faster with the pure-python version.
The C version is actually much slower on PyPy.

The Encode and Decode functions may look faster,
but accessing values or building a message in Python code will be much
slower.



>
> but after i gen the podpbpypy.so
> then import podpbpypy:
>
> Traceback (most recent call last):
> File "app_main.py", line 72, in run_toplevel
> File "t2.py", line 4, in <module>
> import podpbpypy
> ImportError: No module named podpbpypy
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>
>


-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131104/4ba27b81/attachment.html>

From matti.picus at gmail.com  Mon Nov  4 12:16:02 2013
From: matti.picus at gmail.com (Matti Picus)
Date: Mon, 04 Nov 2013 13:16:02 +0200
Subject: [pypy-dev] May we delete the pypy/tkinter repository?
Message-ID: <527781F2.3050508@gmail.com>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131104/70857ff9/attachment.html>

From fijall at gmail.com  Mon Nov  4 12:19:07 2013
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Mon, 4 Nov 2013 13:19:07 +0200
Subject: [pypy-dev] May we delete the pypy/tkinter repository?
In-Reply-To: <527781F2.3050508@gmail.com>
References: <527781F2.3050508@gmail.com>
Message-ID: <CAK5idxQXGbGmMksdFktBi0fdgfTjwm3OLZJ7Se+b_i7Abj0VJw@mail.gmail.com>

do we have any alternative?

On Mon, Nov 4, 2013 at 1:16 PM, Matti Picus <matti.picus at gmail.com> wrote:
> Someone recently tried to use this repository
> https://bitbucket.org/pypy/tkinter
> and failed, it would seem we would be better off without it.
> May I delete it?
> Matti
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>

From amauryfa at gmail.com  Mon Nov  4 12:29:01 2013
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Mon, 4 Nov 2013 12:29:01 +0100
Subject: [pypy-dev] May we delete the pypy/tkinter repository?
In-Reply-To: <CAK5idxQXGbGmMksdFktBi0fdgfTjwm3OLZJ7Se+b_i7Abj0VJw@mail.gmail.com>
References: <527781F2.3050508@gmail.com>
 <CAK5idxQXGbGmMksdFktBi0fdgfTjwm3OLZJ7Se+b_i7Abj0VJw@mail.gmail.com>
Message-ID: <CAGmFidYF65NuFvjquycx7=MDacwR6ZVpkJA4mtZYj6XhbkqSAA@mail.gmail.com>

2013/11/4 Maciej Fijalkowski <fijall at gmail.com>

> do we have any alternative?
>

Sure. _tkinter has a cffi implementation, in lib_pypy since 2.1.


>
> On Mon, Nov 4, 2013 at 1:16 PM, Matti Picus <matti.picus at gmail.com> wrote:
> > Someone recently tried to use this repository
> > https://bitbucket.org/pypy/tkinter
> > and failed, it would seem we would be better off without it.
> > May I delete it?
> > Matti
> >
> > _______________________________________________
> > pypy-dev mailing list
> > pypy-dev at python.org
> > https://mail.python.org/mailman/listinfo/pypy-dev
> >
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>



-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131104/3e020d79/attachment.html>

From fijall at gmail.com  Mon Nov  4 12:36:01 2013
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Mon, 4 Nov 2013 13:36:01 +0200
Subject: [pypy-dev] May we delete the pypy/tkinter repository?
In-Reply-To: <CAGmFidYF65NuFvjquycx7=MDacwR6ZVpkJA4mtZYj6XhbkqSAA@mail.gmail.com>
References: <527781F2.3050508@gmail.com>
 <CAK5idxQXGbGmMksdFktBi0fdgfTjwm3OLZJ7Se+b_i7Abj0VJw@mail.gmail.com>
 <CAGmFidYF65NuFvjquycx7=MDacwR6ZVpkJA4mtZYj6XhbkqSAA@mail.gmail.com>
Message-ID: <CAK5idxSPvg4cFJsJ2RevgkVbMYC+Zjhaa4BD-7yEzGbsrmxXUg@mail.gmail.com>

On Mon, Nov 4, 2013 at 1:29 PM, Amaury Forgeot d'Arc <amauryfa at gmail.com> wrote:
> 2013/11/4 Maciej Fijalkowski <fijall at gmail.com>
>>
>> do we have any alternative?
>
>
> Sure. _tkinter has a cffi implementation, in lib_pypy since 2.1.

then yes please :)

>
>>
>>
>> On Mon, Nov 4, 2013 at 1:16 PM, Matti Picus <matti.picus at gmail.com> wrote:
>> > Someone recently tried to use this repository
>> > https://bitbucket.org/pypy/tkinter
>> > and failed, it would seem we would be better off without it.
>> > May I delete it?
>> > Matti
>> >
>> > _______________________________________________
>> > pypy-dev mailing list
>> > pypy-dev at python.org
>> > https://mail.python.org/mailman/listinfo/pypy-dev
>> >
>> _______________________________________________
>> pypy-dev mailing list
>> pypy-dev at python.org
>> https://mail.python.org/mailman/listinfo/pypy-dev
>
>
>
>
> --
> Amaury Forgeot d'Arc

From matti.picus at gmail.com  Mon Nov  4 13:12:48 2013
From: matti.picus at gmail.com (Matti Picus)
Date: Mon, 04 Nov 2013 14:12:48 +0200
Subject: [pypy-dev] May we delete the pypy/tkinter repository?
In-Reply-To: <CAK5idxSPvg4cFJsJ2RevgkVbMYC+Zjhaa4BD-7yEzGbsrmxXUg@mail.gmail.com>
References: <527781F2.3050508@gmail.com>
 <CAK5idxQXGbGmMksdFktBi0fdgfTjwm3OLZJ7Se+b_i7Abj0VJw@mail.gmail.com>
 <CAGmFidYF65NuFvjquycx7=MDacwR6ZVpkJA4mtZYj6XhbkqSAA@mail.gmail.com>
 <CAK5idxSPvg4cFJsJ2RevgkVbMYC+Zjhaa4BD-7yEzGbsrmxXUg@mail.gmail.com>
Message-ID: <52778F40.7030701@gmail.com>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131104/f0724eb0/attachment.html>

From mail2shine at qq.com  Wed Nov  6 04:46:21 2013
From: mail2shine at qq.com (=?ISO-8859-1?B?S2FTaGluaW5n?=)
Date: Wed, 6 Nov 2013 11:46:21 +0800
Subject: [pypy-dev] error of gunicorn+gevent under pypy
Message-ID: <tencent_2D093D1C7692A98309DFBF36@qq.com>

set up gevent by
http://smetj.net/installing-pypy-with-gevent-and-virtualenv-on-fedora.html

and run my test.py by gunicorn+gevent, err below:

Dose anything else I need to do??
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131106/d3d92f1f/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 31F7F80A at C9D92540.8DBB7952
Type: application/octet-stream
Size: 20635 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131106/d3d92f1f/attachment-0001.obj>

From mail2shine at qq.com  Wed Nov  6 11:16:43 2013
From: mail2shine at qq.com (=?ISO-8859-1?B?S2FTaGluaW5n?=)
Date: Wed, 6 Nov 2013 18:16:43 +0800
Subject: [pypy-dev] error of gunicorn+gevent under pypy
In-Reply-To: <tencent_2D093D1C7692A98309DFBF36@qq.com>
References: <tencent_2D093D1C7692A98309DFBF36@qq.com>
Message-ID: <tencent_1B5061A428B8FD5C0CA93FBB@qq.com>

Fixed!




------------------ Original ------------------
From:  "KaShining";<mail2shine at qq.com>;
Date:  Wed, Nov 6, 2013 11:46 AM
To:  "pypy-dev"<pypy-dev at python.org>; 

Subject:  error of gunicorn+gevent under pypy



set up gevent by
http://smetj.net/installing-pypy-with-gevent-and-virtualenv-on-fedora.html

and run my test.py by gunicorn+gevent, err below:

Dose anything else I need to do??
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131106/5a594653/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 03CF66AC at FB48A93C.0B177A52.8DBB7952
Type: application/octet-stream
Size: 20635 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131106/5a594653/attachment-0001.obj>

From laurie at tratt.net  Thu Nov  7 23:18:00 2013
From: laurie at tratt.net (Laurence Tratt)
Date: Thu, 7 Nov 2013 22:18:00 +0000
Subject: [pypy-dev] More strategies
Message-ID: <20131107221800.GB3251@overdrive.tratt.net>

I've been fiddling with extending some of the strategies in PyPy. My first
port of call has been to provide fast paths in IntegerListStrategy.find,
which is the basis of "x in l" and "l.index(x)". Previously this was very
fast if you looked up an integer, but rather slow if you tried up looking
anything else. Since we know exactly what the type of every element in such a
list is, there seem to be quite a few opportunities for optimisation. I have
left the code for the common case (looking up an int in an int list) as
before; but have provided various fast paths. Looking up floats and longs is
now much faster; for many other types (e.g. strings or user objects whose
class does not override __eq__) these return immediately. I have modified
FloatListStrategy similarly.

I attach the simple benchmark I used to test the changes. Before the patch
here are the results:

  ===> Integer lists
  1 in l: 6.935
  's' in l: 10.585
  0 in l: 0.002
  1.0 in l: 22.527
  1L in l: 70.244
  object() in l: 10.587
  t1() in l: 14.433
  t2() in l: 181.644
  ===> Float lists
  1 in l: 17.131
  's' in l: 44.034
  0.8 in l: 0.002
  1.0 in l: 9.372
  1L in l: 96.680
  object() in l: 70.232
  t1() in l: 165.662
  t2() in l: 181.558

After applying the patch:

  ===> Integer lists
  1 in l: 6.928
  's' in l: 0.001
  0 in l: 0.001
  1.0 in l: 6.929
  1L in l: 6.928
  object() in l: 0.002
  t1() in l: 17.411
  t2() in l: 139.246
  ===> Float lists
  1 in l: 9.387
  's' in l: 0.001
  0.8 in l: 0.001
  1.0 in l: 9.388
  1L in l: 9.388
  object() in l: 0.002
  t1() in l: 129.031
  t2() in l: 139.265

There is one slowdown ("t1() in l" for integer lists), which I can't explain,
but overall the pattern of speed-ups is clear. As you might expect, this
patch doesn't make much difference to the speed.pypy.org benchmarks.

The patch itself is here:

  https://bitbucket.org/pypy/pypy/commits/599ed4285a6de348c7e7e732e303336d3160ce78

I welcome any comments on this. I don't pretend to know PyPy's internals
inside out, and I may have made one or more stupid changes in doing this. If
I have made a mistake, I'd love to know before I press on with other possible
changes!


Laurie

From alex.gaynor at gmail.com  Thu Nov  7 23:30:21 2013
From: alex.gaynor at gmail.com (Alex Gaynor)
Date: Thu, 7 Nov 2013 14:30:21 -0800
Subject: [pypy-dev] More strategies
In-Reply-To: <20131107221800.GB3251@overdrive.tratt.net>
References: <20131107221800.GB3251@overdrive.tratt.net>
Message-ID: <CAFRnB2UaQB7-1EEsuaPm1ZmYoTCcR6APr5tjDpGw8mar4gMjeg@mail.gmail.com>

What is t1 in this context? That's a pretty dramatic slow-down, so I'd like
to understand it better.

Alex


On Thu, Nov 7, 2013 at 2:18 PM, Laurence Tratt <laurie at tratt.net> wrote:

> I've been fiddling with extending some of the strategies in PyPy. My first
> port of call has been to provide fast paths in IntegerListStrategy.find,
> which is the basis of "x in l" and "l.index(x)". Previously this was very
> fast if you looked up an integer, but rather slow if you tried up looking
> anything else. Since we know exactly what the type of every element in
> such a
> list is, there seem to be quite a few opportunities for optimisation. I
> have
> left the code for the common case (looking up an int in an int list) as
> before; but have provided various fast paths. Looking up floats and longs
> is
> now much faster; for many other types (e.g. strings or user objects whose
> class does not override __eq__) these return immediately. I have modified
> FloatListStrategy similarly.
>
> I attach the simple benchmark I used to test the changes. Before the patch
> here are the results:
>
>   ===> Integer lists
>   1 in l: 6.935
>   's' in l: 10.585
>   0 in l: 0.002
>   1.0 in l: 22.527
>   1L in l: 70.244
>   object() in l: 10.587
>   t1() in l: 14.433
>   t2() in l: 181.644
>   ===> Float lists
>   1 in l: 17.131
>   's' in l: 44.034
>   0.8 in l: 0.002
>   1.0 in l: 9.372
>   1L in l: 96.680
>   object() in l: 70.232
>   t1() in l: 165.662
>   t2() in l: 181.558
>
> After applying the patch:
>
>   ===> Integer lists
>   1 in l: 6.928
>   's' in l: 0.001
>   0 in l: 0.001
>   1.0 in l: 6.929
>   1L in l: 6.928
>   object() in l: 0.002
>   t1() in l: 17.411
>   t2() in l: 139.246
>   ===> Float lists
>   1 in l: 9.387
>   's' in l: 0.001
>   0.8 in l: 0.001
>   1.0 in l: 9.388
>   1L in l: 9.388
>   object() in l: 0.002
>   t1() in l: 129.031
>   t2() in l: 139.265
>
> There is one slowdown ("t1() in l" for integer lists), which I can't
> explain,
> but overall the pattern of speed-ups is clear. As you might expect, this
> patch doesn't make much difference to the speed.pypy.org benchmarks.
>
> The patch itself is here:
>
>
> https://bitbucket.org/pypy/pypy/commits/599ed4285a6de348c7e7e732e303336d3160ce78
>
> I welcome any comments on this. I don't pretend to know PyPy's internals
> inside out, and I may have made one or more stupid changes in doing this.
> If
> I have made a mistake, I'd love to know before I press on with other
> possible
> changes!
>
>
> Laurie
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>



-- 
"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero
GPG Key fingerprint: 125F 5C67 DFE9 4084
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131107/6496bdb0/attachment.html>

From laurie at tratt.net  Thu Nov  7 23:33:25 2013
From: laurie at tratt.net (Laurence Tratt)
Date: Thu, 7 Nov 2013 22:33:25 +0000
Subject: [pypy-dev] More strategies
In-Reply-To: <CAFRnB2UaQB7-1EEsuaPm1ZmYoTCcR6APr5tjDpGw8mar4gMjeg@mail.gmail.com>
References: <20131107221800.GB3251@overdrive.tratt.net>
 <CAFRnB2UaQB7-1EEsuaPm1ZmYoTCcR6APr5tjDpGw8mar4gMjeg@mail.gmail.com>
Message-ID: <20131107223325.GC3251@overdrive.tratt.net>

On Thu, Nov 07, 2013 at 02:30:21PM -0800, Alex Gaynor wrote:

> What is t1 in this context? That's a pretty dramatic slow-down, so I'd like
> to understand it better.

And here's the point where I realise I didn't attach the benchmarks. Mea
culpa. Attached this time (with luck).


Laurie
-------------- next part --------------
import time

class t1(object):
    def __eq__(self, o):
        return o == 1

class t2(object):
    def __eq__(self, o):
        return o == 1.0

def bench(l, exp):
    f = eval("lambda: " + exp)
    t = time.time()
    for i in range(1000):
        f()
    print "%s: %.3f" % (exp, time.time() - t)


print "===> Integer lists"
l = []
for i in range(10000000):
    l.append(0)
l.append(1)
bench(l, "1 in l")
bench(l, "'s' in l")
bench(l, "0 in l")
bench(l, "1.0 in l")
bench(l, "1L in l")
bench(l, "object() in l")
bench(l, "t1() in l")
bench(l, "t2() in l")


print "===> Float lists"
l = []
for i in range(10000000):
    l.append(0.8)
l.append(1.0)
bench(l, "1 in l")
bench(l, "'s' in l")
bench(l, "0.8 in l")
bench(l, "1.0 in l")
bench(l, "1L in l")
bench(l, "object() in l")
bench(l, "t1() in l")
bench(l, "t2() in l")

From amauryfa at gmail.com  Thu Nov  7 23:42:47 2013
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Thu, 7 Nov 2013 23:42:47 +0100
Subject: [pypy-dev] More strategies
In-Reply-To: <20131107221800.GB3251@overdrive.tratt.net>
References: <20131107221800.GB3251@overdrive.tratt.net>
Message-ID: <CAGmFidYJDhCigGkzYHPpHvq2yxJWmg6Y-DOmw3jk_hKZy9xN-A@mail.gmail.com>

2013/11/7 Laurence Tratt <laurie at tratt.net>

> The patch itself is here:
>
>
> https://bitbucket.org/pypy/pypy/commits/599ed4285a6de348c7e7e732e303336d3160ce78
>

Hum, [1,2,3].__contains__(1.9)?


-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131107/9e308abd/attachment.html>

From laurie at tratt.net  Thu Nov  7 23:51:08 2013
From: laurie at tratt.net (Laurence Tratt)
Date: Thu, 7 Nov 2013 22:51:08 +0000
Subject: [pypy-dev] More strategies
In-Reply-To: <CAGmFidYJDhCigGkzYHPpHvq2yxJWmg6Y-DOmw3jk_hKZy9xN-A@mail.gmail.com>
References: <20131107221800.GB3251@overdrive.tratt.net>
 <CAGmFidYJDhCigGkzYHPpHvq2yxJWmg6Y-DOmw3jk_hKZy9xN-A@mail.gmail.com>
Message-ID: <20131107225108.GD3251@overdrive.tratt.net>

On Thu, Nov 07, 2013 at 11:42:47PM +0100, Amaury Forgeot d'Arc wrote:

> Hum, [1,2,3].__contains__(1.9)?

Good point. This is something I clearly got wrong.

Is there a practical way to tell if a float could ever compare True to an
integer? I suppose I could convert it to an int, and do a comparison back to
the float?


Laurie

From alex.gaynor at gmail.com  Thu Nov  7 23:52:38 2013
From: alex.gaynor at gmail.com (Alex Gaynor)
Date: Thu, 7 Nov 2013 14:52:38 -0800
Subject: [pypy-dev] More strategies
In-Reply-To: <20131107225108.GD3251@overdrive.tratt.net>
References: <20131107221800.GB3251@overdrive.tratt.net>
 <CAGmFidYJDhCigGkzYHPpHvq2yxJWmg6Y-DOmw3jk_hKZy9xN-A@mail.gmail.com>
 <20131107225108.GD3251@overdrive.tratt.net>
Message-ID: <CAFRnB2VeuUYs-Ke-ZG68di0os=EbHjW5Or8oAKR575MDqV9hsg@mail.gmail.com>

int(f) == f should be correct.

Alex


On Thu, Nov 7, 2013 at 2:51 PM, Laurence Tratt <laurie at tratt.net> wrote:

> On Thu, Nov 07, 2013 at 11:42:47PM +0100, Amaury Forgeot d'Arc wrote:
>
> > Hum, [1,2,3].__contains__(1.9)?
>
> Good point. This is something I clearly got wrong.
>
> Is there a practical way to tell if a float could ever compare True to an
> integer? I suppose I could convert it to an int, and do a comparison back
> to
> the float?
>
>
> Laurie
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>



-- 
"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero
GPG Key fingerprint: 125F 5C67 DFE9 4084
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131107/11401646/attachment-0001.html>

From william.leslie.ttg at gmail.com  Fri Nov  8 00:14:30 2013
From: william.leslie.ttg at gmail.com (William ML Leslie)
Date: Fri, 8 Nov 2013 10:14:30 +1100
Subject: [pypy-dev] More strategies
In-Reply-To: <20131107221800.GB3251@overdrive.tratt.net>
References: <20131107221800.GB3251@overdrive.tratt.net>
Message-ID: <CAHgd1hEwgSmPwjKceGBo-vaSMfS2LOxK9iZyf_FRXfuvTRG5Lw@mail.gmail.com>

On 8 November 2013 09:18, Laurence Tratt <laurie at tratt.net> wrote:
> Looking up floats and longs is
> now much faster; for many other types (e.g. strings or user objects whose
> class does not override __eq__) these return immediately.

I wonder a bit if it is worth introducing additional fast paths for
clearly nonsense (eg, string) cases, when it's the sensible case
(user-provided types that provide __eq__) that we should be optimising
for.  Did you menchbark the code without such cases?

-- 
William Leslie

Notice:
Likely much of this email is, by the nature of copyright, covered
under copyright law.  You absolutely may reproduce any part of it in
accordance with the copyright law of the nation you are reading this
in.  Any attempt to deny you those rights would be illegal without
prior contractual agreement.

From mail2shine at qq.com  Fri Nov  8 09:02:32 2013
From: mail2shine at qq.com (=?ISO-8859-1?B?S2FTaGluaW5n?=)
Date: Fri, 8 Nov 2013 16:02:32 +0800
Subject: [pypy-dev] fail about resumable hashlib code under pypy
Message-ID: <tencent_1BA95B727873D2BA4B4C6278@qq.com>

Here is my test.py about resumable-hashlib:

from ctypes import *
import hashlib

PyObject_HEAD = [
    ('ob_refcnt', c_size_t),
    ('ob_type', c_void_p),
]

class EVP_MD(Structure):
    _fields_ = [
        ('type', c_int),
        ('pkey_type', c_int),
        ('md_size', c_int),
        ('flags', c_ulong),
        ('init', c_void_p),
        ('update', c_void_p),
        ('final', c_void_p),
        ('copy', c_void_p),
        ('cleanup', c_void_p),
        ('sign', c_void_p),
        ('verify', c_void_p),
        ('required_pkey_type', c_int*5),
        ('block_size', c_int),
        ('ctx_size', c_int),
    ]

class EVP_MD_CTX(Structure):
    _fields_ = [
        ('digest', POINTER(EVP_MD)),
        ('engine', c_void_p),
        ('flags', c_ulong),
        ('md_data', POINTER(c_char)),
    ]

class EVPobject(Structure):
    _fields_ = PyObject_HEAD + [
        ('name', py_object),
        ('ctx', EVP_MD_CTX),
    ]

#import string

#    str4096 = random_str(4096)
str4096 = 'sssss'

hash2 = hashlib.sha1()
c_evp_obj = cast(c_void_p(id(hash2)), POINTER(EVPobject)).contents
ctx = c_evp_obj.ctx
digest = ctx.digest.contents
state = ctx.md_data[:digest.ctx_size]
#print digest.ctx_size

Pass in cpython while fail in pypy:

Traceback (most recent call last):
  File "app_main.py", line 72, in run_toplevel
  File "t.py", line 49, in <module>
    digest = ctx.digest.contents
  File "/opt/pypy2.1/lib_pypy/_ctypes/pointer.py", line 83, in getcontents
    raise ValueError("NULL pointer access")
ValueError: NULL pointer access
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131108/a608ef7a/attachment.html>

From fijall at gmail.com  Fri Nov  8 09:47:29 2013
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Fri, 8 Nov 2013 10:47:29 +0200
Subject: [pypy-dev] fail about resumable hashlib code under pypy
In-Reply-To: <tencent_1BA95B727873D2BA4B4C6278@qq.com>
References: <tencent_1BA95B727873D2BA4B4C6278@qq.com>
Message-ID: <CAK5idxSqnDX6L+WK2J9s+Vwt8y0-TO9eqaoOY5WVgESAk24hVg@mail.gmail.com>

PyPy objects are not implemented like this. This is an evil and
atrocious hack to get the refcount and pyobjects stuff from python
level using ctypes. works only on cpython.

On Fri, Nov 8, 2013 at 10:02 AM, KaShining <mail2shine at qq.com> wrote:
> Here is my test.py about resumable-hashlib:
>
> from ctypes import *
> import hashlib
>
> PyObject_HEAD = [
>     ('ob_refcnt', c_size_t),
>     ('ob_type', c_void_p),
> ]
>
> class EVP_MD(Structure):
>     _fields_ = [
>         ('type', c_int),
>         ('pkey_type', c_int),
>         ('md_size', c_int),
>         ('flags', c_ulong),
>         ('init', c_void_p),
>         ('update', c_void_p),
>         ('final', c_void_p),
>         ('copy', c_void_p),
>         ('cleanup', c_void_p),
>         ('sign', c_void_p),
>         ('verify', c_void_p),
>         ('required_pkey_type', c_int*5),
>         ('block_size', c_int),
>         ('ctx_size', c_int),
>     ]
>
> class EVP_MD_CTX(Structure):
>     _fields_ = [
>         ('digest', POINTER(EVP_MD)),
>         ('engine', c_void_p),
>         ('flags', c_ulong),
>         ('md_data', POINTER(c_char)),
>     ]
>
> class EVPobject(Structure):
>     _fields_ = PyObject_HEAD + [
>         ('name', py_object),
>         ('ctx', EVP_MD_CTX),
>     ]
>
> #import string
>
> #    str4096 = random_str(4096)
> str4096 = 'sssss'
>
> hash2 = hashlib.sha1()
> c_evp_obj = cast(c_void_p(id(hash2)), POINTER(EVPobject)).contents
> ctx = c_evp_obj.ctx
> digest = ctx.digest.contents
> state = ctx.md_data[:digest.ctx_size]
> #print digest.ctx_size
>
> Pass in cpython while fail in pypy:
>
> Traceback (most recent call last):
>   File "app_main.py", line 72, in run_toplevel
>   File "t.py", line 49, in <module>
>     digest = ctx.digest.contents
>   File "/opt/pypy2.1/lib_pypy/_ctypes/pointer.py", line 83, in getcontents
>     raise ValueError("NULL pointer access")
> ValueError: NULL pointer access
>
>
>
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>

From mail2shine at qq.com  Fri Nov  8 09:49:26 2013
From: mail2shine at qq.com (=?gb18030?B?S2FTaGluaW5n?=)
Date: Fri, 8 Nov 2013 16:49:26 +0800
Subject: [pypy-dev] =?gb18030?b?u9i4tKO6ICBmYWlsIGFib3V0IHJlc3VtYWJsZSBo?=
 =?gb18030?q?ashlib_code_under_pypy?=
In-Reply-To: <CAK5idxSqnDX6L+WK2J9s+Vwt8y0-TO9eqaoOY5WVgESAk24hVg@mail.gmail.com>
References: <tencent_1BA95B727873D2BA4B4C6278@qq.com>
 <CAK5idxSqnDX6L+WK2J9s+Vwt8y0-TO9eqaoOY5WVgESAk24hVg@mail.gmail.com>
Message-ID: <tencent_2559BBAD5DD13187066F5853@qq.com>

Maybe this way can work:
https://bitbucket.org/pypy/pypy/src/cli-jit/pypy/lib/_hashlib.py?at=cli-jit




------------------ ???? ------------------
???: "Maciej Fijalkowski";<fijall at gmail.com>;
????: 2013?11?8?(???) ??4:47
???: "KaShining"<mail2shine at qq.com>; 
??: "pypy-dev"<pypy-dev at python.org>; 
??: Re: [pypy-dev] fail about resumable hashlib code under pypy



PyPy objects are not implemented like this. This is an evil and
atrocious hack to get the refcount and pyobjects stuff from python
level using ctypes. works only on cpython.

On Fri, Nov 8, 2013 at 10:02 AM, KaShining <mail2shine at qq.com> wrote:
> Here is my test.py about resumable-hashlib:
>
> from ctypes import *
> import hashlib
>
> PyObject_HEAD = [
>     ('ob_refcnt', c_size_t),
>     ('ob_type', c_void_p),
> ]
>
> class EVP_MD(Structure):
>     _fields_ = [
>         ('type', c_int),
>         ('pkey_type', c_int),
>         ('md_size', c_int),
>         ('flags', c_ulong),
>         ('init', c_void_p),
>         ('update', c_void_p),
>         ('final', c_void_p),
>         ('copy', c_void_p),
>         ('cleanup', c_void_p),
>         ('sign', c_void_p),
>         ('verify', c_void_p),
>         ('required_pkey_type', c_int*5),
>         ('block_size', c_int),
>         ('ctx_size', c_int),
>     ]
>
> class EVP_MD_CTX(Structure):
>     _fields_ = [
>         ('digest', POINTER(EVP_MD)),
>         ('engine', c_void_p),
>         ('flags', c_ulong),
>         ('md_data', POINTER(c_char)),
>     ]
>
> class EVPobject(Structure):
>     _fields_ = PyObject_HEAD + [
>         ('name', py_object),
>         ('ctx', EVP_MD_CTX),
>     ]
>
> #import string
>
> #    str4096 = random_str(4096)
> str4096 = 'sssss'
>
> hash2 = hashlib.sha1()
> c_evp_obj = cast(c_void_p(id(hash2)), POINTER(EVPobject)).contents
> ctx = c_evp_obj.ctx
> digest = ctx.digest.contents
> state = ctx.md_data[:digest.ctx_size]
> #print digest.ctx_size
>
> Pass in cpython while fail in pypy:
>
> Traceback (most recent call last):
>   File "app_main.py", line 72, in run_toplevel
>   File "t.py", line 49, in <module>
>     digest = ctx.digest.contents
>   File "/opt/pypy2.1/lib_pypy/_ctypes/pointer.py", line 83, in getcontents
>     raise ValueError("NULL pointer access")
> ValueError: NULL pointer access
>
>
>
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131108/461fe435/attachment-0001.html>

From fijall at gmail.com  Fri Nov  8 09:51:43 2013
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Fri, 8 Nov 2013 10:51:43 +0200
Subject: [pypy-dev]
	=?utf-8?b?5Zue5aSN77yaICBmYWlsIGFib3V0IHJlc3VtYWJsZSBo?=
	=?utf-8?q?ashlib_code_under_pypy?=
In-Reply-To: <tencent_2559BBAD5DD13187066F5853@qq.com>
References: <tencent_1BA95B727873D2BA4B4C6278@qq.com>
 <CAK5idxSqnDX6L+WK2J9s+Vwt8y0-TO9eqaoOY5WVgESAk24hVg@mail.gmail.com>
 <tencent_2559BBAD5DD13187066F5853@qq.com>
Message-ID: <CAK5idxQghdQ16wfcyR6mYN4Cwzr78-e188+n74TJ6uTp+OXQMg@mail.gmail.com>

this can work, but these days we have implemented hashlib differently., why?

On Fri, Nov 8, 2013 at 10:49 AM, KaShining <mail2shine at qq.com> wrote:
> Maybe this way can work:
> https://bitbucket.org/pypy/pypy/src/cli-jit/pypy/lib/_hashlib.py?at=cli-jit
>
>
> ------------------ ???? ------------------
> ???: "Maciej Fijalkowski";<fijall at gmail.com>;
> ????: 2013?11?8?(???) ??4:47
> ???: "KaShining"<mail2shine at qq.com>;
> ??: "pypy-dev"<pypy-dev at python.org>;
> ??: Re: [pypy-dev] fail about resumable hashlib code under pypy
>
> PyPy objects are not implemented like this. This is an evil and
> atrocious hack to get the refcount and pyobjects stuff from python
> level using ctypes. works only on cpython.
>
> On Fri, Nov 8, 2013 at 10:02 AM, KaShining <mail2shine at qq.com> wrote:
>> Here is my test.py about resumable-hashlib:
>>
>> from ctypes import *
>> import hashlib
>>
>> PyObject_HEAD = [
>>     ('ob_refcnt', c_size_t),
>>     ('ob_type', c_void_p),
>> ]
>>
>> class EVP_MD(Structure):
>>     _fields_ = [
>>         ('type', c_int),
>>         ('pkey_type', c_int),
>>         ('md_size', c_int),
>>         ('flags', c_ulong),
>>         ('init', c_void_p),
>>         ('update', c_void_p),
>>         ('final', c_void_p),
>>         ('copy', c_void_p),
>>         ('cleanup', c_void_p),
>>         ('sign', c_void_p),
>>         ('verify', c_void_p),
>>         ('required_pkey_type', c_int*5),
>>         ('block_size', c_int),
>>         ('ctx_size', c_int),
>>     ]
>>
>> class EVP_MD_CTX(Structure):
>>     _fields_ = [
>>         ('digest', POINTER(EVP_MD)),
>>         ('engine', c_void_p),
>>         ('flags', c_ulong),
>>         ('md_data', POINTER(c_char)),
>>     ]
>>
>> class EVPobject(Structure):
>>     _fields_ = PyObject_HEAD + [
>>         ('name', py_object),
>>         ('ctx', EVP_MD_CTX),
>>     ]
>>
>> #import string
>>
>> #    str4096 = random_str(4096)
>> str4096 = 'sssss'
>>
>> hash2 = hashlib.sha1()
>> c_evp_obj = cast(c_void_p(id(hash2)), POINTER(EVPobject)).contents
>> ctx = c_evp_obj.ctx
>> digest = ctx.digest.contents
>> state = ctx.md_data[:digest.ctx_size]
>> #print digest.ctx_size
>>
>> Pass in cpython while fail in pypy:
>>
>> Traceback (most recent call last):
>>   File "app_main.py", line 72, in run_toplevel
>>   File "t.py", line 49, in <module>
>>     digest = ctx.digest.contents
>>   File "/opt/pypy2.1/lib_pypy/_ctypes/pointer.py", line 83, in getcontents
>>     raise ValueError("NULL pointer access")
>> ValueError: NULL pointer access
>>
>>
>>
>>
>> _______________________________________________
>> pypy-dev mailing list
>> pypy-dev at python.org
>> https://mail.python.org/mailman/listinfo/pypy-dev
>>

From mail2shine at qq.com  Fri Nov  8 09:55:41 2013
From: mail2shine at qq.com (=?gb18030?B?S2FTaGluaW5n?=)
Date: Fri, 8 Nov 2013 16:55:41 +0800
Subject: [pypy-dev] =?gb18030?b?u9i4tKO6ILvYuLSjuiAgZmFpbCBhYm91dCByZXN1?=
 =?gb18030?q?mable_hashlib_code_under_pypy?=
In-Reply-To: <CAK5idxQghdQ16wfcyR6mYN4Cwzr78-e188+n74TJ6uTp+OXQMg@mail.gmail.com>
References: <tencent_1BA95B727873D2BA4B4C6278@qq.com>
 <CAK5idxSqnDX6L+WK2J9s+Vwt8y0-TO9eqaoOY5WVgESAk24hVg@mail.gmail.com>
 <tencent_2559BBAD5DD13187066F5853@qq.com>
 <CAK5idxQghdQ16wfcyR6mYN4Cwzr78-e188+n74TJ6uTp+OXQMg@mail.gmail.com>
Message-ID: <tencent_27BE2DAE10FD1A2D6EF298FE@qq.com>

ok.
Can i see any demo code about hashlib-pypy ??




------------------ ???? ------------------
???: "Maciej Fijalkowski";<fijall at gmail.com>;
????: 2013?11?8?(???) ??4:51
???: "KaShining"<mail2shine at qq.com>; 
??: "pypy-dev"<pypy-dev at python.org>; 
??: Re: ??? [pypy-dev] fail about resumable hashlib code under pypy



this can work, but these days we have implemented hashlib differently., why?

On Fri, Nov 8, 2013 at 10:49 AM, KaShining <mail2shine at qq.com> wrote:
> Maybe this way can work:
> https://bitbucket.org/pypy/pypy/src/cli-jit/pypy/lib/_hashlib.py?at=cli-jit
>
>
> ------------------ ???? ------------------
> ???: "Maciej Fijalkowski";<fijall at gmail.com>;
> ????: 2013?11?8?(???) ??4:47
> ???: "KaShining"<mail2shine at qq.com>;
> ??: "pypy-dev"<pypy-dev at python.org>;
> ??: Re: [pypy-dev] fail about resumable hashlib code under pypy
>
> PyPy objects are not implemented like this. This is an evil and
> atrocious hack to get the refcount and pyobjects stuff from python
> level using ctypes. works only on cpython.
>
> On Fri, Nov 8, 2013 at 10:02 AM, KaShining <mail2shine at qq.com> wrote:
>> Here is my test.py about resumable-hashlib:
>>
>> from ctypes import *
>> import hashlib
>>
>> PyObject_HEAD = [
>>     ('ob_refcnt', c_size_t),
>>     ('ob_type', c_void_p),
>> ]
>>
>> class EVP_MD(Structure):
>>     _fields_ = [
>>         ('type', c_int),
>>         ('pkey_type', c_int),
>>         ('md_size', c_int),
>>         ('flags', c_ulong),
>>         ('init', c_void_p),
>>         ('update', c_void_p),
>>         ('final', c_void_p),
>>         ('copy', c_void_p),
>>         ('cleanup', c_void_p),
>>         ('sign', c_void_p),
>>         ('verify', c_void_p),
>>         ('required_pkey_type', c_int*5),
>>         ('block_size', c_int),
>>         ('ctx_size', c_int),
>>     ]
>>
>> class EVP_MD_CTX(Structure):
>>     _fields_ = [
>>         ('digest', POINTER(EVP_MD)),
>>         ('engine', c_void_p),
>>         ('flags', c_ulong),
>>         ('md_data', POINTER(c_char)),
>>     ]
>>
>> class EVPobject(Structure):
>>     _fields_ = PyObject_HEAD + [
>>         ('name', py_object),
>>         ('ctx', EVP_MD_CTX),
>>     ]
>>
>> #import string
>>
>> #    str4096 = random_str(4096)
>> str4096 = 'sssss'
>>
>> hash2 = hashlib.sha1()
>> c_evp_obj = cast(c_void_p(id(hash2)), POINTER(EVPobject)).contents
>> ctx = c_evp_obj.ctx
>> digest = ctx.digest.contents
>> state = ctx.md_data[:digest.ctx_size]
>> #print digest.ctx_size
>>
>> Pass in cpython while fail in pypy:
>>
>> Traceback (most recent call last):
>>   File "app_main.py", line 72, in run_toplevel
>>   File "t.py", line 49, in <module>
>>     digest = ctx.digest.contents
>>   File "/opt/pypy2.1/lib_pypy/_ctypes/pointer.py", line 83, in getcontents
>>     raise ValueError("NULL pointer access")
>> ValueError: NULL pointer access
>>
>>
>>
>>
>> _______________________________________________
>> pypy-dev mailing list
>> pypy-dev at python.org
>> https://mail.python.org/mailman/listinfo/pypy-dev
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131108/dfa96922/attachment.html>

From fijall at gmail.com  Fri Nov  8 10:08:53 2013
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Fri, 8 Nov 2013 11:08:53 +0200
Subject: [pypy-dev]
	=?utf-8?b?5Zue5aSN77yaIOWbnuWkje+8miAgZmFpbCBhYm91dCBy?=
	=?utf-8?q?esumable_hashlib_code_under_pypy?=
In-Reply-To: <tencent_27BE2DAE10FD1A2D6EF298FE@qq.com>
References: <tencent_1BA95B727873D2BA4B4C6278@qq.com>
 <CAK5idxSqnDX6L+WK2J9s+Vwt8y0-TO9eqaoOY5WVgESAk24hVg@mail.gmail.com>
 <tencent_2559BBAD5DD13187066F5853@qq.com>
 <CAK5idxQghdQ16wfcyR6mYN4Cwzr78-e188+n74TJ6uTp+OXQMg@mail.gmail.com>
 <tencent_27BE2DAE10FD1A2D6EF298FE@qq.com>
Message-ID: <CAK5idxRssg8W2BzykYv1MLXOuOpG+amg5yTLo+rpmPaJtz08hA@mail.gmail.com>

pypy comes with normal hashlib. it just works like cpython one.

On Fri, Nov 8, 2013 at 10:55 AM, KaShining <mail2shine at qq.com> wrote:
> ok.
> Can i see any demo code about hashlib-pypy ??
>
>
> ------------------ ???? ------------------
> ???: "Maciej Fijalkowski";<fijall at gmail.com>;
> ????: 2013?11?8?(???) ??4:51
> ???: "KaShining"<mail2shine at qq.com>;
> ??: "pypy-dev"<pypy-dev at python.org>;
> ??: Re: ??? [pypy-dev] fail about resumable hashlib code under pypy
>
> this can work, but these days we have implemented hashlib differently., why?
>
> On Fri, Nov 8, 2013 at 10:49 AM, KaShining <mail2shine at qq.com> wrote:
>> Maybe this way can work:
>>
>> https://bitbucket.org/pypy/pypy/src/cli-jit/pypy/lib/_hashlib.py?at=cli-jit
>>
>>
>> ------------------ ???? ------------------
>> ???: "Maciej Fijalkowski";<fijall at gmail.com>;
>> ????: 2013?11?8?(???) ??4:47
>> ???: "KaShining"<mail2shine at qq.com>;
>> ??: "pypy-dev"<pypy-dev at python.org>;
>> ??: Re: [pypy-dev] fail about resumable hashlib code under pypy
>>
>> PyPy objects are not implemented like this. This is an evil and
>> atrocious hack to get the refcount and pyobjects stuff from python
>> level using ctypes. works only on cpython.
>>
>> On Fri, Nov 8, 2013 at 10:02 AM, KaShining <mail2shine at qq.com> wrote:
>>> Here is my test.py about resumable-hashlib:
>>>
>>> from ctypes import *
>>> import hashlib
>>>
>>> PyObject_HEAD = [
>>>     ('ob_refcnt', c_size_t),
>>>     ('ob_type', c_void_p),
>>> ]
>>>
>>> class EVP_MD(Structure):
>>>     _fields_ = [
>>>         ('type', c_int),
>>>         ('pkey_type', c_int),
>>>         ('md_size', c_int),
>>>         ('flags', c_ulong),
>>>         ('init', c_void_p),
>>>         ('update', c_void_p),
>>>         ('final', c_void_p),
>>>         ('copy', c_void_p),
>>>         ('cleanup', c_void_p),
>>>         ('sign', c_void_p),
>>>         ('verify', c_void_p),
>>>         ('required_pkey_type', c_int*5),
>>>         ('block_size', c_int),
>>>         ('ctx_size', c_int),
>>>     ]
>>>
>>> class EVP_MD_CTX(Structure):
>>>     _fields_ = [
>>>         ('digest', POINTER(EVP_MD)),
>>>         ('engine', c_void_p),
>>>         ('flags', c_ulong),
>>>         ('md_data', POINTER(c_char)),
>>>     ]
>>>
>>> class EVPobject(Structure):
>>>     _fields_ = PyObject_HEAD + [
>>>         ('name', py_object),
>>>         ('ctx', EVP_MD_CTX),
>>>     ]
>>>
>>> #import string
>>>
>>> #    str4096 = random_str(4096)
>>> str4096 = 'sssss'
>>>
>>> hash2 = hashlib.sha1()
>>> c_evp_obj = cast(c_void_p(id(hash2)), POINTER(EVPobject)).contents
>>> ctx = c_evp_obj.ctx
>>> digest = ctx.digest.contents
>>> state = ctx.md_data[:digest.ctx_size]
>>> #print digest.ctx_size
>>>
>>> Pass in cpython while fail in pypy:
>>>
>>> Traceback (most recent call last):
>>>   File "app_main.py", line 72, in run_toplevel
>>>   File "t.py", line 49, in <module>
>>>     digest = ctx.digest.contents
>>>   File "/opt/pypy2.1/lib_pypy/_ctypes/pointer.py", line 83, in
>>> getcontents
>>>     raise ValueError("NULL pointer access")
>>> ValueError: NULL pointer access
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> pypy-dev mailing list
>>> pypy-dev at python.org
>>> https://mail.python.org/mailman/listinfo/pypy-dev
>>>

From arigo at tunes.org  Fri Nov  8 10:19:30 2013
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 8 Nov 2013 10:19:30 +0100
Subject: [pypy-dev] More strategies
In-Reply-To: <CAHgd1hEwgSmPwjKceGBo-vaSMfS2LOxK9iZyf_FRXfuvTRG5Lw@mail.gmail.com>
References: <20131107221800.GB3251@overdrive.tratt.net>
 <CAHgd1hEwgSmPwjKceGBo-vaSMfS2LOxK9iZyf_FRXfuvTRG5Lw@mail.gmail.com>
Message-ID: <CAMSv6X3e9u=h7oO9qBdOOzNtxBx3dnBjfbvUGwWQvZ=Xc2fkSg@mail.gmail.com>

Hi,

On Fri, Nov 8, 2013 at 12:14 AM, William ML Leslie
<william.leslie.ttg at gmail.com> wrote:
> I wonder a bit if it is worth introducing additional fast paths for
> clearly nonsense (eg, string) cases, when it's the sensible case
> (user-provided types that provide __eq__) that we should be optimising
> for.  Did you menchbark the code without such cases?

I fear a bit that *all* cases are on the clearly nonsense side of
things.  I don't particularly think there is code out there that would
see any speed-ups using these cases.  As Amaury pointed out this
introduces delicate correctness issues, and for maybe nothing... :-/
I can be convinced of the countrary though.


A bient?t,

Armin.

From njh at njhurst.com  Fri Nov  8 14:59:47 2013
From: njh at njhurst.com (Nathan Hurst)
Date: Sat, 9 Nov 2013 00:59:47 +1100
Subject: [pypy-dev] cffi python long
Message-ID: <20131108135947.GA30918@ajhurst.org>

Is there a nicer way to pass python long ints (bigint) into C
efficiently?  I'm currently cutting the value up into 64 bit chunks in
python and passing in as an unsigned long*:

cdef("int bigInt(int n, unsigned long* x);")

x = sum(1 << i for i in [100,200,123])
xs = []
while x > 0:
    xs.append(x & ((1 << 64) - 1))
    x >>= 64
print lib.bigInt(len(xs), xs)

but this is quite slow, and it seems like this data must already be
lurking somewhere in exactly the right form.

njh

From amauryfa at gmail.com  Fri Nov  8 16:35:16 2013
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Fri, 8 Nov 2013 16:35:16 +0100
Subject: [pypy-dev] cffi python long
In-Reply-To: <20131108135947.GA30918@ajhurst.org>
References: <20131108135947.GA30918@ajhurst.org>
Message-ID: <CAGmFidZYFzdKZvn+iDJVsuvekJBoQrZjZaVDc-rSYH0yjwJ+KA@mail.gmail.com>

2013/11/8 Nathan Hurst <njh at njhurst.com>

> Is there a nicer way to pass python long ints (bigint) into C
> efficiently?  I'm currently cutting the value up into 64 bit chunks in
> python and passing in as an unsigned long*:
>
> cdef("int bigInt(int n, unsigned long* x);")
>
> x = sum(1 << i for i in [100,200,123])
> xs = []
> while x > 0:
>     xs.append(x & ((1 << 64) - 1))
>     x >>= 64
> print lib.bigInt(len(xs), xs)
>
> but this is quite slow, and it seems like this data must already be
> lurking somewhere in exactly the right form.
>

In Python3 you could use the to_bytes() method of ints,
but I could not find any way to have the same in python2.

Or just use hex(). You might have endianness issues, though.


-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131108/357ee08b/attachment.html>

From arigo at tunes.org  Fri Nov  8 16:59:57 2013
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 8 Nov 2013 16:59:57 +0100
Subject: [pypy-dev] cffi python long
In-Reply-To: <CAGmFidZYFzdKZvn+iDJVsuvekJBoQrZjZaVDc-rSYH0yjwJ+KA@mail.gmail.com>
References: <20131108135947.GA30918@ajhurst.org>
 <CAGmFidZYFzdKZvn+iDJVsuvekJBoQrZjZaVDc-rSYH0yjwJ+KA@mail.gmail.com>
Message-ID: <CAMSv6X3qR5dUnBcO8j1jp_HaCtRsCpJZ7-LaP_V5JKfEoGo-WQ@mail.gmail.com>

Hi Nathan,

On Fri, Nov 8, 2013 at 4:35 PM, Amaury Forgeot d'Arc <amauryfa at gmail.com> wrote:
>> Is there a nicer way to pass python long ints (bigint) into C
>> efficiently?

It depends what the C code wants to do with it.  If it's just for
passing around, you can use ffi.from_handle().  If the C code expects
to read the value, then I fear you need to decompose it into some
format, either manually or using hex() or marshal.  The internal
format of longs in PyPy is not directly accessible: it is not using
64-bit integers, but 63-bit if the C compiler supports __int128_t and
31-bit otherwise.


A bient?t,

Armin.

From laurie at tratt.net  Fri Nov  8 17:28:55 2013
From: laurie at tratt.net (Laurence Tratt)
Date: Fri, 8 Nov 2013 16:28:55 +0000
Subject: [pypy-dev] More strategies
In-Reply-To: <CAMSv6X3e9u=h7oO9qBdOOzNtxBx3dnBjfbvUGwWQvZ=Xc2fkSg@mail.gmail.com>
References: <20131107221800.GB3251@overdrive.tratt.net>
 <CAHgd1hEwgSmPwjKceGBo-vaSMfS2LOxK9iZyf_FRXfuvTRG5Lw@mail.gmail.com>
 <CAMSv6X3e9u=h7oO9qBdOOzNtxBx3dnBjfbvUGwWQvZ=Xc2fkSg@mail.gmail.com>
Message-ID: <20131108162855.GD25802@overdrive.tratt.net>

On Fri, Nov 08, 2013 at 10:19:30AM +0100, Armin Rigo wrote:

> I fear a bit that *all* cases are on the clearly nonsense side of things.
> I don't particularly think there is code out there that would see any
> speed-ups using these cases.  As Amaury pointed out this introduces
> delicate correctness issues, and for maybe nothing... :-/ I can be
> convinced of the countrary though.

The fairly unlikely ones are cheap and easy to verify, so it's arguable as to
whether they're worth worrying about. We can drop some or all if we wish.

The ones that I'm much more sure are a win are checking for one type of
number in a list of another type. In essence, this patch makes the
performance of intermingling number types considerably less surprising than
before. For example, the patch doubles the speed of checking for an int in a
list of floats and makes looking up a long in a list of ints 10x faster. In
my experience, Python programs that nominally use floats actually end up
slinging lots of integers around since very few people can be bothered to
write "x = 1.0" instead of "x = 1". I expect the other types are similarly
intermingled in many programs.

The original benchmark I created wasn't as clear as it could have been about
this, so I've redone the figures with performance differences so you can
easily see how performance is affected. I've also tweaked the benchmark to be
sure that there was no possibility of the JIT warming up half-way through
(which doesn't seem to have happened, but better safe than sorry).

Pre-patch:

  ===> Integer lists
  1 in l: 13.998
  's' in l: 21.191
  0 in l: 0.001
  1.0 in l: 45.065
  1L in l: 140.420
  object() in l: 21.169
  t1() in l: 28.889
  t2() in l: 373.511
  ===> Float lists
  1 in l: 34.217
  's' in l: 93.441
  0.8 in l: 0.000
  1.0 in l: 18.741
  1L in l: 188.214
  object() in l: 145.564
  t1() in l: 336.450
  t2() in l: 373.866

Post-patch (with approximate performance differences):

  ===> Integer lists
  1 in l: 14.020 [1x]
  's' in l: 0.001 [near inf x]
  0 in l: 0.001 [no difference]
  1.0 in l: 14.033 [3.2x]
  1L in l: 14.036 [10x]
  object() in l: 0.002 [near inf x]
  t1() in l: 34.778 [0.83x]
  t2() in l: 278.776 [1.33x]
  ===> Float lists
  1 in l: 18.768 [1.8x]
  's' in l: 0.000 [near inf x]
  0.8 in l: 0.000 [no difference]
  1.0 in l: 18.773 [1x]
  1L in l: 18.779 [10x]
  object() in l: 0.000 [near inf x]
  t1() in l: 246.722 [1.36x]
  t2() in l: 278.489 [1.34x]

One interesting thing that this shows is the apparent lack of performance of
symmetry between comparing ints and floats vs. comparing floats and ints.


Laurie
-------------- next part --------------
import time

class t1(object):
    def __eq__(self, o):
        return o == 1

class t2(object):
    def __eq__(self, o):
        return o == 1.0

def bench(l, exp, f, i):
    t = time.time()
    for _ in range(2000):
        f()
    if i == 1:
        print "%s: %.3f" % (exp, time.time() - t)


print "===> Integer lists"
for i in range(2):
    l = []
    if i == 0:
        s = 2000
    else:
        s = 10000000
    for j in range(s):
        l.append(0)
    l.append(1)
    bench(l, "1 in l", lambda: 1 in l, i)
    bench(l, "'s' in l", lambda: 's' in l, i)
    bench(l, "0 in l", lambda: 0 in l, i)
    bench(l, "1.0 in l", lambda: 1.0 in l, i)
    bench(l, "1L in l", lambda: 1L in l, i)
    bench(l, "object() in l", lambda: object() in l, i)
    bench(l, "t1() in l", lambda: t1() in l, i)
    bench(l, "t2() in l", lambda: t2() in l, i)


print "===> Float lists"
for i in range(2):
    l = []
    if i == 0:
        s = 2000
    else:
        s = 10000000
    for j in range(s):
        l.append(0.8)
    l.append(1.0)
    bench(l, "1 in l", lambda: 1 in l, i)
    bench(l, "'s' in l", lambda: 's' in l, i)
    bench(l, "0.8 in l", lambda: 0.8 in l, i)
    bench(l, "1.0 in l", lambda: 1.0 in l, i)
    bench(l, "1L in l", lambda: 1L in l, i)
    bench(l, "object() in l", lambda: object() in l, i)
    bench(l, "t1() in l", lambda: t1() in l, i)
    bench(l, "t2() in l", lambda: t2() in l, i)


From rymg19 at gmail.com  Sun Nov 10 02:25:55 2013
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Sat, 9 Nov 2013 19:25:55 -0600
Subject: [pypy-dev] PyPy acting slow
Message-ID: <CAO41-mPXv_xTB=EUWFXn7MbTydWWBAtoTja-uwu_ki0FB3MUiA@mail.gmail.com>

For some reason, my PyPy installation seems very slow; both the version
from the Ubuntu repos and a custom build I did(WITH the JIT enabled). I ran
example2.py from the RPython tutorial using CPython and PyPy. Even after
the JIT(visibly) kicked in, it was still much slower than CPython. While
CPython took 0.5 seconds, PyPy took 10.8! Why is this happening?

-- 
Ryan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131109/ebc6a8ed/attachment.html>

From fijall at gmail.com  Sun Nov 10 06:28:36 2013
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sun, 10 Nov 2013 07:28:36 +0200
Subject: [pypy-dev] PyPy acting slow
In-Reply-To: <CAO41-mPXv_xTB=EUWFXn7MbTydWWBAtoTja-uwu_ki0FB3MUiA@mail.gmail.com>
References: <CAO41-mPXv_xTB=EUWFXn7MbTydWWBAtoTja-uwu_ki0FB3MUiA@mail.gmail.com>
Message-ID: <CAK5idxQkMtrxO6M0KqL7ikZrteBGjtgqkxV1QffTBZReRhLfrw@mail.gmail.com>

Hi Ryan.

Which example are you talking about?

On Sun, Nov 10, 2013 at 3:25 AM, Ryan Gonzalez <rymg19 at gmail.com> wrote:
> For some reason, my PyPy installation seems very slow; both the version from
> the Ubuntu repos and a custom build I did(WITH the JIT enabled). I ran
> example2.py from the RPython tutorial using CPython and PyPy. Even after the
> JIT(visibly) kicked in, it was still much slower than CPython. While CPython
> took 0.5 seconds, PyPy took 10.8! Why is this happening?
>
> --
> Ryan
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>

From matti.picus at gmail.com  Sun Nov 10 21:24:48 2013
From: matti.picus at gmail.com (Matti Picus)
Date: Sun, 10 Nov 2013 22:24:48 +0200
Subject: [pypy-dev] can we get stdlib-2.7.5 merged?
Message-ID: <527FEB90.8010507@gmail.com>

There are only a handful (ok - 10, two others are from the nightly 
default) of failures on the stdlib-2.7.5 branch:
http://buildbot.pypy.org/summary?branch=stdlib-2.7.5
It seems the major one is handling eintr signal on file reading, which 
AFAICT would fix more or less 3 of the failures. The others seem to be 
unrelated to one-another. Would anyone like to take on some of this work?

Matti

From arigo at tunes.org  Mon Nov 11 09:13:45 2013
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 11 Nov 2013 09:13:45 +0100
Subject: [pypy-dev] can we get stdlib-2.7.5 merged?
In-Reply-To: <527FEB90.8010507@gmail.com>
References: <527FEB90.8010507@gmail.com>
Message-ID: <CAMSv6X364ztF70t8mxqLJWM8Jr6Kugb=URQdjawE5XAaa6ryAg@mail.gmail.com>

Hi Matti,

On Sun, Nov 10, 2013 at 9:24 PM, Matti Picus <matti.picus at gmail.com> wrote:
> There are only a handful (ok - 10, two others are from the nightly default)
> of failures on the stdlib-2.7.5 branch:
> http://buildbot.pypy.org/summary?branch=stdlib-2.7.5
> It seems the major one is handling eintr signal on file reading, which
> AFAICT would fix more or less 3 of the failures. The others seem to be
> unrelated to one-another. Would anyone like to take on some of this work?

I fear the branch contains a few delicate issues.  Without someone
ready to take this work, I'd rather like to do the next release now.
I've been fixing a lot of small issues already on "default", so I
wouldn't mind if we didn't at the same time merge a branch that
probably adds its own lot of small issues.


A bient?t,

Armin.

From arigo at tunes.org  Mon Nov 11 12:53:45 2013
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 11 Nov 2013 12:53:45 +0100
Subject: [pypy-dev] Preparing for the release of PyPy 2.2
Message-ID: <CAMSv6X1nF-i0_5GkWTPM3SFt_Lj18NKo7R=ZozPGU2PePuL2ew@mail.gmail.com>

Hi all,

Preparing for the next release.  The Linux binaries of PyPy 2.2,
release candidate, are here:

http://buildbot.pypy.org/nightly/release-2.2.x/

This corresponds to the "release-2.2.x" branch in the repo.

There are a few bug reports that will likely *not* get fixed for lack
of manpower: we would need someone involved with Windows, OS/X, or the
BSDs.

https://bugs.pypy.org/issue1626 - Tkinter on Windows
https://bugs.pypy.org/issue1166 - kqueue on OSX/BSD
https://bugs.pypy.org/issue1441 - version of sqlite3.dll on Win.

Also, we need again someone to make the OS/X binary (Alex?); we can't
use our buildbot because its old compiler builds slightly bogus
binaries.


A bient?t,

Armin.

From arigo at tunes.org  Mon Nov 11 22:54:28 2013
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 11 Nov 2013 22:54:28 +0100
Subject: [pypy-dev] Preparing for the release of PyPy 2.2
In-Reply-To: <CAMSv6X1nF-i0_5GkWTPM3SFt_Lj18NKo7R=ZozPGU2PePuL2ew@mail.gmail.com>
References: <CAMSv6X1nF-i0_5GkWTPM3SFt_Lj18NKo7R=ZozPGU2PePuL2ew@mail.gmail.com>
Message-ID: <CAMSv6X1QQqAg5-reLkALgj0Vb9LBubVL+9j06F5bCU+AHM1=ig@mail.gmail.com>

Re-hi,

The OS/X release candidate (thanks Alex!):
http://cobra.cs.uni-duesseldorf.de/~buildmaster/misc/pypy-2.2-osx.tar.bz2


Armin

From tobias.oberstein at tavendo.de  Tue Nov 12 02:36:33 2013
From: tobias.oberstein at tavendo.de (Tobias Oberstein)
Date: Mon, 11 Nov 2013 17:36:33 -0800
Subject: [pypy-dev] Preparing for the release of PyPy 2.2
In-Reply-To: <CAMSv6X1nF-i0_5GkWTPM3SFt_Lj18NKo7R=ZozPGU2PePuL2ew@mail.gmail.com>
References: <CAMSv6X1nF-i0_5GkWTPM3SFt_Lj18NKo7R=ZozPGU2PePuL2ew@mail.gmail.com>
Message-ID: <634914A010D0B943A035D226786325D4446A0D92A6@EXVMBX020-12.exch020.serverdata.net>

Hi Armin,

> manpower: we would need someone involved with Windows, OS/X, or the BSDs.

Firstly, I could offer to operate a FreeBSD buildslave for PyPy.

The current one fails ("libssl" missing) and seems very old (FreeBSD 7).

We (Tavendo) operate 2 slaves for the Twisted project (buildbot based).

I have already asked there, since ideally I'd like to operate 1 slave (box) for both Twisted _and_ PyPy.

That would allow running Twisted trial against nightly PyPy's - which might come handy fixing stuff.

Twisted would welcome running 3 Twisted bots: CPy (supported), PyPy/stable (supported) and PyPy/nightly (unsupported).

I am lobbying for having PyPy 2.2 (not less) built from vanilla release sources as the "stable/supported".

The buildslave would need to run behind NAT .. as this is an office box running the slave inside a VM.

So would that be welcome?

/Tobias

PS: Rgd the kqueue "issue": I could address that. "address the issue" == FreeBSD. I don't care OSX .
But honestly, I don't think it's serious ... see my comments on the issue. So it'd be low prio in
my overfull queue;)

From arigo at tunes.org  Tue Nov 12 09:43:01 2013
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 12 Nov 2013 09:43:01 +0100
Subject: [pypy-dev] Preparing for the release of PyPy 2.2
In-Reply-To: <634914A010D0B943A035D226786325D4446A0D92A6@EXVMBX020-12.exch020.serverdata.net>
References: <CAMSv6X1nF-i0_5GkWTPM3SFt_Lj18NKo7R=ZozPGU2PePuL2ew@mail.gmail.com>
 <634914A010D0B943A035D226786325D4446A0D92A6@EXVMBX020-12.exch020.serverdata.net>
Message-ID: <CAMSv6X3tFfObwxOXEfD_aGCzDye4wGqhoVy_+C2jG91D5zQNrg@mail.gmail.com>

Hi Tobias,

On Tue, Nov 12, 2013 at 2:36 AM, Tobias Oberstein
<tobias.oberstein at tavendo.de> wrote:
> Firstly, I could offer to operate a FreeBSD buildslave for PyPy.

That would be nice, if it also comes with occasional fixes (having a
buildslave that nobody looks at is a bit pointless).  Can we do this
after the release?


A bient?t,

Armin.

From arigo at tunes.org  Tue Nov 12 09:48:23 2013
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 12 Nov 2013 09:48:23 +0100
Subject: [pypy-dev] More strategies
In-Reply-To: <20131108162855.GD25802@overdrive.tratt.net>
References: <20131107221800.GB3251@overdrive.tratt.net>
 <CAHgd1hEwgSmPwjKceGBo-vaSMfS2LOxK9iZyf_FRXfuvTRG5Lw@mail.gmail.com>
 <CAMSv6X3e9u=h7oO9qBdOOzNtxBx3dnBjfbvUGwWQvZ=Xc2fkSg@mail.gmail.com>
 <20131108162855.GD25802@overdrive.tratt.net>
Message-ID: <CAMSv6X1NW8k-BB_OaMcLb7fENk2vEWyJRmZjU-MfnH-j4-ingQ@mail.gmail.com>

Hi Laurence,

On Fri, Nov 8, 2013 at 5:28 PM, Laurence Tratt <laurie at tratt.net> wrote:
> my experience, Python programs that nominally use floats actually end up
> slinging lots of integers around since very few people can be bothered to
> write "x = 1.0" instead of "x = 1". I expect the other types are similarly
> intermingled in many programs.

Ok, but then it seems that the first problem we hit down this road,
before arriving at the __contains__ performance, is actually mixing
ints and floats in the same list.  We need to do something to store
such lists efficiently.  I'm suggesting using a list of floats with a
custom encoding of N-bit integers into NaN values (for N==32 or maybe
a higher number that still fits).  This requires a bit of tweaking
here and there (maybe all the way to the JIT) but shouldn't be too
hard.


A bient?t,

Armin.

From tobias.oberstein at tavendo.de  Tue Nov 12 10:57:07 2013
From: tobias.oberstein at tavendo.de (Tobias Oberstein)
Date: Tue, 12 Nov 2013 01:57:07 -0800
Subject: [pypy-dev] Preparing for the release of PyPy 2.2
In-Reply-To: <CAMSv6X3tFfObwxOXEfD_aGCzDye4wGqhoVy_+C2jG91D5zQNrg@mail.gmail.com>
References: <CAMSv6X1nF-i0_5GkWTPM3SFt_Lj18NKo7R=ZozPGU2PePuL2ew@mail.gmail.com>
 <634914A010D0B943A035D226786325D4446A0D92A6@EXVMBX020-12.exch020.serverdata.net>
 <CAMSv6X3tFfObwxOXEfD_aGCzDye4wGqhoVy_+C2jG91D5zQNrg@mail.gmail.com>
Message-ID: <634914A010D0B943A035D226786325D4446A0D92B8@EXVMBX020-12.exch020.serverdata.net>

> Hi Tobias,
> 
> On Tue, Nov 12, 2013 at 2:36 AM, Tobias Oberstein
> <tobias.oberstein at tavendo.de> wrote:
> > Firstly, I could offer to operate a FreeBSD buildslave for PyPy.
> 
> That would be nice, if it also comes with occasional fixes (having a buildslave
> that nobody looks at is a bit pointless).  Can we do this after the release?

Sure. And yes: we'll be maintaining that slave. We do that for Twisted also ..

/Tobias


From arigo at tunes.org  Tue Nov 12 15:32:46 2013
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 12 Nov 2013 15:32:46 +0100
Subject: [pypy-dev] Preparing for the release of PyPy 2.2
In-Reply-To: <634914A010D0B943A035D226786325D4446A0D92B8@EXVMBX020-12.exch020.serverdata.net>
References: <CAMSv6X1nF-i0_5GkWTPM3SFt_Lj18NKo7R=ZozPGU2PePuL2ew@mail.gmail.com>
 <634914A010D0B943A035D226786325D4446A0D92A6@EXVMBX020-12.exch020.serverdata.net>
 <CAMSv6X3tFfObwxOXEfD_aGCzDye4wGqhoVy_+C2jG91D5zQNrg@mail.gmail.com>
 <634914A010D0B943A035D226786325D4446A0D92B8@EXVMBX020-12.exch020.serverdata.net>
Message-ID: <CAMSv6X3OejTPX2QB+r21SJ8zNGinJiUifscqDMVBN5JgL6eGKA@mail.gmail.com>

Hi Tobias,

On Tue, Nov 12, 2013 at 10:57 AM, Tobias Oberstein
<tobias.oberstein at tavendo.de> wrote:
> Sure. And yes: we'll be maintaining that slave. We do that for Twisted also ..

I meant not the slave, but the project itself: it's a bit useless to
know that 42 tests are failing on *BSD is no-one bothers to do
anything about them.


A bient?t,

Armin.

From arigo at tunes.org  Tue Nov 12 17:26:07 2013
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 12 Nov 2013 17:26:07 +0100
Subject: [pypy-dev] Preparing for the release of PyPy 2.2
In-Reply-To: <CAMSv6X1QQqAg5-reLkALgj0Vb9LBubVL+9j06F5bCU+AHM1=ig@mail.gmail.com>
References: <CAMSv6X1nF-i0_5GkWTPM3SFt_Lj18NKo7R=ZozPGU2PePuL2ew@mail.gmail.com>
 <CAMSv6X1QQqAg5-reLkALgj0Vb9LBubVL+9j06F5bCU+AHM1=ig@mail.gmail.com>
Message-ID: <CAMSv6X2RWtt-oJhuhOA_Unysi2j=_gqhwmw0QqgcOGEoibCg+g@mail.gmail.com>

Hi David,

Any reason why the buildbot page
http://buildbot.pypy.org/builders/build-pypy-c-jit-linux-armhf-raring
is the only one missing the section "Force build"?  It prevents us
from asking for a build on the release-2.2.x branch.


A bient?t,

Armin.

From tobias.oberstein at tavendo.de  Tue Nov 12 18:20:32 2013
From: tobias.oberstein at tavendo.de (Tobias Oberstein)
Date: Tue, 12 Nov 2013 09:20:32 -0800
Subject: [pypy-dev] Preparing for the release of PyPy 2.2
In-Reply-To: <CAMSv6X3OejTPX2QB+r21SJ8zNGinJiUifscqDMVBN5JgL6eGKA@mail.gmail.com>
References: <CAMSv6X1nF-i0_5GkWTPM3SFt_Lj18NKo7R=ZozPGU2PePuL2ew@mail.gmail.com>
 <634914A010D0B943A035D226786325D4446A0D92A6@EXVMBX020-12.exch020.serverdata.net>
 <CAMSv6X3tFfObwxOXEfD_aGCzDye4wGqhoVy_+C2jG91D5zQNrg@mail.gmail.com>
 <634914A010D0B943A035D226786325D4446A0D92B8@EXVMBX020-12.exch020.serverdata.net>
 <CAMSv6X3OejTPX2QB+r21SJ8zNGinJiUifscqDMVBN5JgL6eGKA@mail.gmail.com>
Message-ID: <634914A010D0B943A035D226786325D4446A0D94D9@EXVMBX020-12.exch020.serverdata.net>

> On Tue, Nov 12, 2013 at 10:57 AM, Tobias Oberstein
> <tobias.oberstein at tavendo.de> wrote:
> > Sure. And yes: we'll be maintaining that slave. We do that for Twisted also ..
> 
> I meant not the slave, but the project itself: it's a bit useless to know that 42
> tests are failing on *BSD is no-one bothers to do anything about them.

Sure, but we don't even know that today, since the buildslave is broken. At least I don't know about 42 issues.

And I hope you wouldn't require a commitment by me to fix all issues as a prerequisite for being allowed to contribute a buildslave;)

Well, I don't think so, and, FWIW, here is my motivation: help making Twisted/PyPy/FreeBSD a first class supported combo. It's my fav. combo. and we use that commercially. So I am serious I guess.

To support that commitment: I already contributed kqueue/FreeBSD both to Twisted and PyPy (for completeness, in both cases, there was old non-working/incomplete code there) and helped in general making FreeBSD an officially supported platform for Twisted (CPython). And I will contribute more code if time permits.

Anyway, I am looking forward to the 2.2 release, and after that will come again asking for buildbot credentials if thats fine ..

/Tobias


From david.schneider at picle.org  Tue Nov 12 18:29:44 2013
From: david.schneider at picle.org (David Schneider)
Date: Tue, 12 Nov 2013 18:29:44 +0100
Subject: [pypy-dev] Preparing for the release of PyPy 2.2
In-Reply-To: <CAMSv6X2RWtt-oJhuhOA_Unysi2j=_gqhwmw0QqgcOGEoibCg+g@mail.gmail.com>
References: <CAMSv6X1nF-i0_5GkWTPM3SFt_Lj18NKo7R=ZozPGU2PePuL2ew@mail.gmail.com>
 <CAMSv6X1QQqAg5-reLkALgj0Vb9LBubVL+9j06F5bCU+AHM1=ig@mail.gmail.com>
 <CAMSv6X2RWtt-oJhuhOA_Unysi2j=_gqhwmw0QqgcOGEoibCg+g@mail.gmail.com>
Message-ID: <E042E403-CFE2-47E4-AA47-34CB25A72D78@picle.org>

On 12.11.2013, at 17:26, Armin Rigo <arigo at tunes.org> wrote:

> Hi David,
> 
> Any reason why the buildbot page
> http://buildbot.pypy.org/builders/build-pypy-c-jit-linux-armhf-raring
> is the only one missing the section "Force build"?  It prevents us
> from asking for a build on the release-2.2.x branch.
> 
> 
> A bient?t,
> 
> Armin.


Hi Armin,

No reason in particular, I probably just missed it during the update. I added it in 2c32efb0fb5d. After a restart/reload of the buildmaster it should be there.

Cheers

David

From rymg19 at gmail.com  Tue Nov 12 20:06:05 2013
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Tue, 12 Nov 2013 13:06:05 -0600
Subject: [pypy-dev] Preparing for the release of PyPy 2.2
In-Reply-To: <CAMSv6X1nF-i0_5GkWTPM3SFt_Lj18NKo7R=ZozPGU2PePuL2ew@mail.gmail.com>
References: <CAMSv6X1nF-i0_5GkWTPM3SFt_Lj18NKo7R=ZozPGU2PePuL2ew@mail.gmail.com>
Message-ID: <CAO41-mOKroZscWkLY7WKO-4TvKQ9vT0QSvLZsGqiOd_tK4i4FQ@mail.gmail.com>

I could probably do the Windows DLL fixes, if someone told me how to upload
the changes...


On Mon, Nov 11, 2013 at 5:53 AM, Armin Rigo <arigo at tunes.org> wrote:

> Hi all,
>
> Preparing for the next release.  The Linux binaries of PyPy 2.2,
> release candidate, are here:
>
> http://buildbot.pypy.org/nightly/release-2.2.x/
>
> This corresponds to the "release-2.2.x" branch in the repo.
>
> There are a few bug reports that will likely *not* get fixed for lack
> of manpower: we would need someone involved with Windows, OS/X, or the
> BSDs.
>
> https://bugs.pypy.org/issue1626 - Tkinter on Windows
> https://bugs.pypy.org/issue1166 - kqueue on OSX/BSD
> https://bugs.pypy.org/issue1441 - version of sqlite3.dll on Win.
>
> Also, we need again someone to make the OS/X binary (Alex?); we can't
> use our buildbot because its old compiler builds slightly bogus
> binaries.
>
>
> A bient?t,
>
> Armin.
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>



-- 
Ryan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131112/977506f4/attachment.html>

From matti.picus at gmail.com  Tue Nov 12 20:42:40 2013
From: matti.picus at gmail.com (Matti Picus)
Date: Tue, 12 Nov 2013 21:42:40 +0200
Subject: [pypy-dev] Preparing for the release of PyPy 2.2
In-Reply-To: <CAO41-mOKroZscWkLY7WKO-4TvKQ9vT0QSvLZsGqiOd_tK4i4FQ@mail.gmail.com>
References: <CAMSv6X1nF-i0_5GkWTPM3SFt_Lj18NKo7R=ZozPGU2PePuL2ew@mail.gmail.com>
 <CAO41-mOKroZscWkLY7WKO-4TvKQ9vT0QSvLZsGqiOd_tK4i4FQ@mail.gmail.com>
Message-ID: <528284B0.9090209@gmail.com>

I have been commenting in the bug reports. The general strategy we 
recommend on the windows build page
http://doc.pypy.org/en/latest/windows.html#abridged-method-for-ojit-builds-using-visual-studio-2008
is to download a zip file with all the recommended dependencies, and 
build after adding the appropriate enviroment variables to INCLUDE, LIB, 
and PATH.
So I have been finding all the pieces to add/update/document the needed 
DLLs. Opinions on a desired version of sqlite3.dll would be appreciated, 
see comments to issue 1441.
Once this is all resolved, we can update the buildbot to use the zip 
file contents and package the needed dlls
Matti

On 12/11/2013 9:06 PM, Ryan Gonzalez wrote:
> I could probably do the Windows DLL fixes, if someone told me how to 
> upload the changes...
>
>
> On Mon, Nov 11, 2013 at 5:53 AM, Armin Rigo <arigo at tunes.org 
> <mailto:arigo at tunes.org>> wrote:
>
>     Hi all,
>
>     Preparing for the next release.  The Linux binaries of PyPy 2.2,
>     release candidate, are here:
>
>     http://buildbot.pypy.org/nightly/release-2.2.x/
>
>     This corresponds to the "release-2.2.x" branch in the repo.
>
>     There are a few bug reports that will likely *not* get fixed for lack
>     of manpower: we would need someone involved with Windows, OS/X, or the
>     BSDs.
>
>     https://bugs.pypy.org/issue1626 - Tkinter on Windows
>     https://bugs.pypy.org/issue1166 - kqueue on OSX/BSD
>     https://bugs.pypy.org/issue1441 - version of sqlite3.dll on Win.
>
>     Also, we need again someone to make the OS/X binary (Alex?); we can't
>     use our buildbot because its old compiler builds slightly bogus
>     binaries.
>
>
>     A bient?t,
>
>     Armin.
>     _______________________________________________
>     pypy-dev mailing list
>     pypy-dev at python.org <mailto:pypy-dev at python.org>
>     https://mail.python.org/mailman/listinfo/pypy-dev
>
>
>
>
> -- 
> Ryan
>
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev


From tobias.oberstein at tavendo.de  Tue Nov 12 20:48:04 2013
From: tobias.oberstein at tavendo.de (Tobias Oberstein)
Date: Tue, 12 Nov 2013 11:48:04 -0800
Subject: [pypy-dev] FreeBSD buildslaves: change proposal
Message-ID: <634914A010D0B943A035D226786325D4446A0D962E@EXVMBX020-12.exch020.serverdata.net>

Hi,

we (Tavendo) currently operate 2 Twisted/FreeBSD buildslaves:

a) FreeBSD 8.2, i386 with 1 builder (CPy 2.7)
=> part of the list of "officially supported" builders

b) FreeBSD 9.1, amd64 with 2 builders (CPy 2.7 and PyPy)
=> only listed under "unsupported"

We would like to change that into the following:

c) one FreeBSD amd64 host with 3 builders

c1) CPy 2.7 (supported)
c2) PyPy 2.2 (supported)
c3) PyPy Nightly (unsupported)

Here are some reasons:

- FreeBSD 8.2 is very old (read, unsupported) now, FreeBSD 10 is around the corner, and 9.2 the current official stable release
- i386 is .. getting obscure. Does _anybody_ run FreeBSD i386 anymore?
- there are ongoing workarounds to support that thing .. time that could be spent better
- we want to run a PyPy slave for the PyPy project on that same host (this allows to produce the PyPy/Nightlies .. on a daily basis .. and immediately run Twisted trials against that)
- someone needs to run a PyPy slave anyway .. the current one is broken

- the current Twisted 13.2 has run 100% clean on above b) (CPy) : http://buildbot.twistedmatrix.com/builders/freebsd-9.1-amd64-python2.7/builds/198

So, I'd like to ask Twisted developers if above makes sense, and you would support this way forward.

I've been in contact with PyPy developers, and there seems to be clear interest in improving support for both Twisted and FreeBSD (hence the cross-post).

In general, Tavendo is committed to contribute time & material making Twisted + CPy|PyPy + FreeBSD a combination fully (even better) supported and actually first-class in terms of features, compatibility and performance.

We think < Twisted + PyPy + FreeBSD > in particular has great potential and can provide an extremely powerful stack:

"The _complete_ power to serve."

.. stealing from FreeBSD's motto here - heh, and sorry the the marketing blurb .. I just couldn't resist;)
Thanks!

/Tobias

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131112/28f5334f/attachment.html>

From rymg19 at gmail.com  Tue Nov 12 22:16:15 2013
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Tue, 12 Nov 2013 15:16:15 -0600
Subject: [pypy-dev] Preparing for the release of PyPy 2.2
In-Reply-To: <528284B0.9090209@gmail.com>
References: <CAMSv6X1nF-i0_5GkWTPM3SFt_Lj18NKo7R=ZozPGU2PePuL2ew@mail.gmail.com>
 <CAO41-mOKroZscWkLY7WKO-4TvKQ9vT0QSvLZsGqiOd_tK4i4FQ@mail.gmail.com>
 <528284B0.9090209@gmail.com>
Message-ID: <CAO41-mP6SS4zK8mjeMjd5eh4r_ddo5sQp7nCnz4hPBn_9H2j9w@mail.gmail.com>

Ok, just to make sure, this is what is needed:

-Include a prebuilt TkInter???
-Fix the erratic sqlite versions

Correct me if I'm wrong. Two more questions:

-What version of sqlite is needed
-How the heck do I upload this stuff or something?


On Tue, Nov 12, 2013 at 1:42 PM, Matti Picus <matti.picus at gmail.com> wrote:

> I have been commenting in the bug reports. The general strategy we
> recommend on the windows build page
> http://doc.pypy.org/en/latest/windows.html#abridged-method-
> for-ojit-builds-using-visual-studio-2008
> is to download a zip file with all the recommended dependencies, and build
> after adding the appropriate enviroment variables to INCLUDE, LIB, and PATH.
> So I have been finding all the pieces to add/update/document the needed
> DLLs. Opinions on a desired version of sqlite3.dll would be appreciated,
> see comments to issue 1441.
> Once this is all resolved, we can update the buildbot to use the zip file
> contents and package the needed dlls
> Matti
>
>
> On 12/11/2013 9:06 PM, Ryan Gonzalez wrote:
>
>> I could probably do the Windows DLL fixes, if someone told me how to
>> upload the changes...
>>
>>
>> On Mon, Nov 11, 2013 at 5:53 AM, Armin Rigo <arigo at tunes.org <mailto:
>> arigo at tunes.org>> wrote:
>>
>>     Hi all,
>>
>>     Preparing for the next release.  The Linux binaries of PyPy 2.2,
>>     release candidate, are here:
>>
>>     http://buildbot.pypy.org/nightly/release-2.2.x/
>>
>>     This corresponds to the "release-2.2.x" branch in the repo.
>>
>>     There are a few bug reports that will likely *not* get fixed for lack
>>     of manpower: we would need someone involved with Windows, OS/X, or the
>>     BSDs.
>>
>>     https://bugs.pypy.org/issue1626 - Tkinter on Windows
>>     https://bugs.pypy.org/issue1166 - kqueue on OSX/BSD
>>     https://bugs.pypy.org/issue1441 - version of sqlite3.dll on Win.
>>
>>     Also, we need again someone to make the OS/X binary (Alex?); we can't
>>     use our buildbot because its old compiler builds slightly bogus
>>     binaries.
>>
>>
>>     A bient?t,
>>
>>     Armin.
>>     _______________________________________________
>>     pypy-dev mailing list
>>     pypy-dev at python.org <mailto:pypy-dev at python.org>
>>
>>     https://mail.python.org/mailman/listinfo/pypy-dev
>>
>>
>>
>>
>> --
>> Ryan
>>
>>
>> _______________________________________________
>> pypy-dev mailing list
>> pypy-dev at python.org
>> https://mail.python.org/mailman/listinfo/pypy-dev
>>
>
>


-- 
Ryan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131112/f273a1a8/attachment.html>

From arigo at tunes.org  Tue Nov 12 22:21:27 2013
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 12 Nov 2013 22:21:27 +0100
Subject: [pypy-dev] Preparing for the release of PyPy 2.2
In-Reply-To: <634914A010D0B943A035D226786325D4446A0D94D9@EXVMBX020-12.exch020.serverdata.net>
References: <CAMSv6X1nF-i0_5GkWTPM3SFt_Lj18NKo7R=ZozPGU2PePuL2ew@mail.gmail.com>
 <634914A010D0B943A035D226786325D4446A0D92A6@EXVMBX020-12.exch020.serverdata.net>
 <CAMSv6X3tFfObwxOXEfD_aGCzDye4wGqhoVy_+C2jG91D5zQNrg@mail.gmail.com>
 <634914A010D0B943A035D226786325D4446A0D92B8@EXVMBX020-12.exch020.serverdata.net>
 <CAMSv6X3OejTPX2QB+r21SJ8zNGinJiUifscqDMVBN5JgL6eGKA@mail.gmail.com>
 <634914A010D0B943A035D226786325D4446A0D94D9@EXVMBX020-12.exch020.serverdata.net>
Message-ID: <CAMSv6X0Czf+zcEfJzpfOYS60o5CnmtFrB4dVpSXN0f+r1=OP8w@mail.gmail.com>

Hi Tobias,

On Tue, Nov 12, 2013 at 6:20 PM, Tobias Oberstein
<tobias.oberstein at tavendo.de> wrote:
> And I hope you wouldn't require a commitment by me to fix all issues as a prerequisite for being allowed to contribute a buildslave;)

Sure :-)  No worry.  I'm not doubting (any more) that you have a
chance of actually helping :-)

> Anyway, I am looking forward to the 2.2 release, and after that will come again asking for buildbot credentials if thats fine ..

Yes, perfect.


A bient?t,

Armin.

From amauryfa at gmail.com  Tue Nov 12 22:56:34 2013
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Tue, 12 Nov 2013 22:56:34 +0100
Subject: [pypy-dev] Preparing for the release of PyPy 2.2
In-Reply-To: <CAO41-mP6SS4zK8mjeMjd5eh4r_ddo5sQp7nCnz4hPBn_9H2j9w@mail.gmail.com>
References: <CAMSv6X1nF-i0_5GkWTPM3SFt_Lj18NKo7R=ZozPGU2PePuL2ew@mail.gmail.com>
 <CAO41-mOKroZscWkLY7WKO-4TvKQ9vT0QSvLZsGqiOd_tK4i4FQ@mail.gmail.com>
 <528284B0.9090209@gmail.com>
 <CAO41-mP6SS4zK8mjeMjd5eh4r_ddo5sQp7nCnz4hPBn_9H2j9w@mail.gmail.com>
Message-ID: <CAGmFida5ZT5ZmOCqUCn_6=4TQd5rmtzSMVWkxxrqyORbqAM4Rg@mail.gmail.com>

2013/11/12 Ryan Gonzalez <rymg19 at gmail.com>

> Ok, just to make sure, this is what is needed:
>
> -Include a prebuilt TkInter???
>

Not only DLLs, but also header files and .lib.
cffi need a development environment.
The instructions in
http://hg.python.org/cpython/file/v2.7.5/Tools/buildbot/external-common.bat
and
http://hg.python.org/cpython/file/v2.7.5/Tools/buildbot/external.bat<http://hg.python.org/cpython/file/v2.7.5/Tools/buildbot/external-common.bat>
could be reused.


> -Fix the erratic sqlite versions
>
> Correct me if I'm wrong. Two more questions:
>
> -What version of sqlite is needed
>

According to the file above, and to match CPython 2.7.5: sqlite-3.6.21


> -How the heck do I upload this stuff or something?
>
>
> On Tue, Nov 12, 2013 at 1:42 PM, Matti Picus <matti.picus at gmail.com>wrote:
>
>> I have been commenting in the bug reports. The general strategy we
>> recommend on the windows build page
>> http://doc.pypy.org/en/latest/windows.html#abridged-method-
>> for-ojit-builds-using-visual-studio-2008
>> is to download a zip file with all the recommended dependencies, and
>> build after adding the appropriate enviroment variables to INCLUDE, LIB,
>> and PATH.
>> So I have been finding all the pieces to add/update/document the needed
>> DLLs. Opinions on a desired version of sqlite3.dll would be appreciated,
>> see comments to issue 1441.
>> Once this is all resolved, we can update the buildbot to use the zip file
>> contents and package the needed dlls
>> Matti
>>
>>
>> On 12/11/2013 9:06 PM, Ryan Gonzalez wrote:
>>
>>> I could probably do the Windows DLL fixes, if someone told me how to
>>> upload the changes...
>>>
>>>
>>> On Mon, Nov 11, 2013 at 5:53 AM, Armin Rigo <arigo at tunes.org <mailto:
>>> arigo at tunes.org>> wrote:
>>>
>>>     Hi all,
>>>
>>>     Preparing for the next release.  The Linux binaries of PyPy 2.2,
>>>     release candidate, are here:
>>>
>>>     http://buildbot.pypy.org/nightly/release-2.2.x/
>>>
>>>     This corresponds to the "release-2.2.x" branch in the repo.
>>>
>>>     There are a few bug reports that will likely *not* get fixed for lack
>>>     of manpower: we would need someone involved with Windows, OS/X, or
>>> the
>>>     BSDs.
>>>
>>>     https://bugs.pypy.org/issue1626 - Tkinter on Windows
>>>     https://bugs.pypy.org/issue1166 - kqueue on OSX/BSD
>>>     https://bugs.pypy.org/issue1441 - version of sqlite3.dll on Win.
>>>
>>>     Also, we need again someone to make the OS/X binary (Alex?); we can't
>>>     use our buildbot because its old compiler builds slightly bogus
>>>     binaries.
>>>
>>>
>>>     A bient?t,
>>>
>>>     Armin.
>>>     _______________________________________________
>>>     pypy-dev mailing list
>>>     pypy-dev at python.org <mailto:pypy-dev at python.org>
>>>
>>>     https://mail.python.org/mailman/listinfo/pypy-dev
>>>
>>>
>>>
>>>
>>> --
>>> Ryan
>>>
>>>
>>> _______________________________________________
>>> pypy-dev mailing list
>>> pypy-dev at python.org
>>> https://mail.python.org/mailman/listinfo/pypy-dev
>>>
>>
>>
>
>
> --
> Ryan
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>
>


-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131112/59792890/attachment-0001.html>

From laurie at tratt.net  Wed Nov 13 01:01:29 2013
From: laurie at tratt.net (Laurence Tratt)
Date: Wed, 13 Nov 2013 00:01:29 +0000
Subject: [pypy-dev] More strategies
In-Reply-To: <CAMSv6X1NW8k-BB_OaMcLb7fENk2vEWyJRmZjU-MfnH-j4-ingQ@mail.gmail.com>
References: <20131107221800.GB3251@overdrive.tratt.net>
 <CAHgd1hEwgSmPwjKceGBo-vaSMfS2LOxK9iZyf_FRXfuvTRG5Lw@mail.gmail.com>
 <CAMSv6X3e9u=h7oO9qBdOOzNtxBx3dnBjfbvUGwWQvZ=Xc2fkSg@mail.gmail.com>
 <20131108162855.GD25802@overdrive.tratt.net>
 <CAMSv6X1NW8k-BB_OaMcLb7fENk2vEWyJRmZjU-MfnH-j4-ingQ@mail.gmail.com>
Message-ID: <20131113000129.GT17882@phase.tratt.net>

On Tue, Nov 12, 2013 at 09:48:23AM +0100, Armin Rigo wrote:

Hi Armin,

> Ok, but then it seems that the first problem we hit down this road, before
> arriving at the __contains__ performance, is actually mixing ints and
> floats in the same list.  We need to do something to store such lists
> efficiently.  I'm suggesting using a list of floats with a custom encoding
> of N-bit integers into NaN values (for N==32 or maybe a higher number that
> still fits).  This requires a bit of tweaking here and there (maybe all the
> way to the JIT) but shouldn't be too hard.

This is something I have been thinking about too. I can't pretend that I've
thought of every in and out, but my current thinking is that it needs to be
alongside an IntegerListStrategy. If we stored every list with numbers as
floats, lists which only store ints (which we know are very common [1]) would
double in storage size. The JIT would also then be disadvantaged, as
currently it can trivially prove that every item coming out of an integer
list is an integer, allowing it to remove many type checks; in an int+float
list, it will have to add a type check for every item that is extracted
(maybe changes to the JIT can reduce the overhead of this). So, if we decide
to do this, I think we would still want IntegerListStrategy -- with some
variant of the patch to 'find' I proposed -- alongside an
IntegerFloatListStrategy.


Laurie

[1] See table 3:
  http://tratt.net/laurie/research/pubs/html/bolz_diekmann_tratt__storage_strategies_for_collections_in_dynamically_typed_languages/

From matti.picus at gmail.com  Wed Nov 13 01:01:59 2013
From: matti.picus at gmail.com (Matti Picus)
Date: Wed, 13 Nov 2013 02:01:59 +0200
Subject: [pypy-dev] Preparing for the release of PyPy 2.2
In-Reply-To: <CAGmFida5ZT5ZmOCqUCn_6=4TQd5rmtzSMVWkxxrqyORbqAM4Rg@mail.gmail.com>
References: <CAMSv6X1nF-i0_5GkWTPM3SFt_Lj18NKo7R=ZozPGU2PePuL2ew@mail.gmail.com>	<CAO41-mOKroZscWkLY7WKO-4TvKQ9vT0QSvLZsGqiOd_tK4i4FQ@mail.gmail.com>	<528284B0.9090209@gmail.com>	<CAO41-mP6SS4zK8mjeMjd5eh4r_ddo5sQp7nCnz4hPBn_9H2j9w@mail.gmail.com>
 <CAGmFida5ZT5ZmOCqUCn_6=4TQd5rmtzSMVWkxxrqyORbqAM4Rg@mail.gmail.com>
Message-ID: <5282C177.5080300@gmail.com>

The sqlite3 issue was easy, we can simply replace the one on the 
buildbot with the 3.6.21 version.
I have asked the buildbot manager to do the replacement.
I have opened a branch for the tkinter changes, since they require a bit 
more work. Those scripts are definitely helpful, thanks.
It seems we also need to ship the entire tk/tcl runtime framework, in 
cpython this is handled by the msi or inno installer, we will need to 
mimic this in package.py
Matti

On 12/11/2013 11:56 PM, Amaury Forgeot d'Arc wrote:
> 2013/11/12 Ryan Gonzalez <rymg19 at gmail.com <mailto:rymg19 at gmail.com>>
>
>     Ok, just to make sure, this is what is needed:
>
>     -Include a prebuilt TkInter???
>
>
> Not only DLLs, but also header files and .lib.
> cffi need a development environment.
> The instructions in
> http://hg.python.org/cpython/file/v2.7.5/Tools/buildbot/external-common.bat
> and
> http://hg.python.org/cpython/file/v2.7.5/Tools/buildbot/external.bat 
> <http://hg.python.org/cpython/file/v2.7.5/Tools/buildbot/external-common.bat>
> could be reused.
>
>     -Fix the erratic sqlite versions
>
>     Correct me if I'm wrong. Two more questions:
>
>     -What version of sqlite is needed
>
>
> According to the file above, and to match CPython 2.7.5: sqlite-3.6.21
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131113/0b34b944/attachment.html>

From fijall at gmail.com  Wed Nov 13 08:35:00 2013
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 13 Nov 2013 09:35:00 +0200
Subject: [pypy-dev] More strategies
In-Reply-To: <20131113000129.GT17882@phase.tratt.net>
References: <20131107221800.GB3251@overdrive.tratt.net>
 <CAHgd1hEwgSmPwjKceGBo-vaSMfS2LOxK9iZyf_FRXfuvTRG5Lw@mail.gmail.com>
 <CAMSv6X3e9u=h7oO9qBdOOzNtxBx3dnBjfbvUGwWQvZ=Xc2fkSg@mail.gmail.com>
 <20131108162855.GD25802@overdrive.tratt.net>
 <CAMSv6X1NW8k-BB_OaMcLb7fENk2vEWyJRmZjU-MfnH-j4-ingQ@mail.gmail.com>
 <20131113000129.GT17882@phase.tratt.net>
Message-ID: <CAK5idxSQo5vPVzwsx7pfnDQ+-BE=isbLA_3HyFJ9u-GOn_gUmQ@mail.gmail.com>

On Wed, Nov 13, 2013 at 2:01 AM, Laurence Tratt <laurie at tratt.net> wrote:
> On Tue, Nov 12, 2013 at 09:48:23AM +0100, Armin Rigo wrote:
>
> Hi Armin,
>
>> Ok, but then it seems that the first problem we hit down this road, before
>> arriving at the __contains__ performance, is actually mixing ints and
>> floats in the same list.  We need to do something to store such lists
>> efficiently.  I'm suggesting using a list of floats with a custom encoding
>> of N-bit integers into NaN values (for N==32 or maybe a higher number that
>> still fits).  This requires a bit of tweaking here and there (maybe all the
>> way to the JIT) but shouldn't be too hard.
>
> This is something I have been thinking about too. I can't pretend that I've
> thought of every in and out, but my current thinking is that it needs to be
> alongside an IntegerListStrategy. If we stored every list with numbers as
> floats, lists which only store ints (which we know are very common [1]) would
> double in storage size. The JIT would also then be disadvantaged, as
> currently it can trivially prove that every item coming out of an integer
> list is an integer, allowing it to remove many type checks; in an int+float
> list, it will have to add a type check for every item that is extracted
> (maybe changes to the JIT can reduce the overhead of this). So, if we decide
> to do this, I think we would still want IntegerListStrategy -- with some
> variant of the patch to 'find' I proposed -- alongside an
> IntegerFloatListStrategy.
>
>
> Laurie
>
> [1] See table 3:
>   http://tratt.net/laurie/research/pubs/html/bolz_diekmann_tratt__storage_strategies_for_collections_in_dynamically_typed_languages/

Yes, we want both.

From arigo at tunes.org  Wed Nov 13 09:27:04 2013
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 13 Nov 2013 09:27:04 +0100
Subject: [pypy-dev] Preparing for the release of PyPy 2.2
In-Reply-To: <5282C177.5080300@gmail.com>
References: <CAMSv6X1nF-i0_5GkWTPM3SFt_Lj18NKo7R=ZozPGU2PePuL2ew@mail.gmail.com>
 <CAO41-mOKroZscWkLY7WKO-4TvKQ9vT0QSvLZsGqiOd_tK4i4FQ@mail.gmail.com>
 <528284B0.9090209@gmail.com>
 <CAO41-mP6SS4zK8mjeMjd5eh4r_ddo5sQp7nCnz4hPBn_9H2j9w@mail.gmail.com>
 <CAGmFida5ZT5ZmOCqUCn_6=4TQd5rmtzSMVWkxxrqyORbqAM4Rg@mail.gmail.com>
 <5282C177.5080300@gmail.com>
Message-ID: <CAMSv6X0kSCTyhe5WpgL2VZyq9exuxmYmrejjmt=6GJozYE_DKg@mail.gmail.com>

Hi Matti,

Thanks for doing all this work!  Fwiw I plan to do the release
tomorrow, but being a few days late on Windows wouldn't be new.


Armin

From arigo at tunes.org  Thu Nov 14 08:58:54 2013
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 14 Nov 2013 08:58:54 +0100
Subject: [pypy-dev] Preparing for the release of PyPy 2.2
In-Reply-To: <5282C177.5080300@gmail.com>
References: <CAMSv6X1nF-i0_5GkWTPM3SFt_Lj18NKo7R=ZozPGU2PePuL2ew@mail.gmail.com>
 <CAO41-mOKroZscWkLY7WKO-4TvKQ9vT0QSvLZsGqiOd_tK4i4FQ@mail.gmail.com>
 <528284B0.9090209@gmail.com>
 <CAO41-mP6SS4zK8mjeMjd5eh4r_ddo5sQp7nCnz4hPBn_9H2j9w@mail.gmail.com>
 <CAGmFida5ZT5ZmOCqUCn_6=4TQd5rmtzSMVWkxxrqyORbqAM4Rg@mail.gmail.com>
 <5282C177.5080300@gmail.com>
Message-ID: <CAMSv6X1r=JS25Sir7Ssc2p33PwGz55zYDe9_ZPeqhbrnkufc2w@mail.gmail.com>

Hi Matti,

On Wed, Nov 13, 2013 at 1:01 AM, Matti Picus <matti.picus at gmail.com> wrote:
> The sqlite3 issue was easy, we can simply replace the one on the buildbot
> with the 3.6.21 version.

I saw the fix.  Thanks!

> I have opened a branch for the tkinter changes, since they require a bit
> more work.

What is the status of this 'windows-packaging' branch?


Armin.

From arigo at tunes.org  Thu Nov 14 09:30:21 2013
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 14 Nov 2013 09:30:21 +0100
Subject: [pypy-dev] More strategies
In-Reply-To: <CAK5idxSQo5vPVzwsx7pfnDQ+-BE=isbLA_3HyFJ9u-GOn_gUmQ@mail.gmail.com>
References: <20131107221800.GB3251@overdrive.tratt.net>
 <CAHgd1hEwgSmPwjKceGBo-vaSMfS2LOxK9iZyf_FRXfuvTRG5Lw@mail.gmail.com>
 <CAMSv6X3e9u=h7oO9qBdOOzNtxBx3dnBjfbvUGwWQvZ=Xc2fkSg@mail.gmail.com>
 <20131108162855.GD25802@overdrive.tratt.net>
 <CAMSv6X1NW8k-BB_OaMcLb7fENk2vEWyJRmZjU-MfnH-j4-ingQ@mail.gmail.com>
 <20131113000129.GT17882@phase.tratt.net>
 <CAK5idxSQo5vPVzwsx7pfnDQ+-BE=isbLA_3HyFJ9u-GOn_gUmQ@mail.gmail.com>
Message-ID: <CAMSv6X3Gc4V1WyDchBub1qt=dYrTteQNR4_xSwtgWTpVTKo7ig@mail.gmail.com>

Hi,

On Wed, Nov 13, 2013 at 8:35 AM, Maciej Fijalkowski <fijall at gmail.com> wrote:
>> So, if we decide
>> to do this, I think we would still want IntegerListStrategy
>
> Yes, we want both.

Yes, but the question is whether we want both FloatIntegerListStrategy
and FloatListStrategy.  The same arguments apply about the JIT not
needing to check the value of a float grabbed out of the list in the
FloatListStrategy case, but it's less clear...


A bient?t,

Armin.

From arigo at tunes.org  Thu Nov 14 10:34:52 2013
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 14 Nov 2013 10:34:52 +0100
Subject: [pypy-dev] PyPy 2.2
Message-ID: <CAMSv6X2_jcrQmpTQ1PfoT7-kDdBORtBXoANhCcwuqjj6xUJcMQ@mail.gmail.com>

=======================================
PyPy 2.2 - Incrementalism
=======================================

We're pleased to announce PyPy 2.2, which targets version 2.7.3 of the Python
language. This release main highlight is the introduction of the incremental
garbage collector, sponsored by the `Raspberry Pi Foundation`_.

This release also contains several bugfixes and performance improvements.

You can download the PyPy 2.2 release here:

    http://pypy.org/download.html

Note that the Win32 download is delayed as we fix a few more things.

We would like to thank our donors for the continued support of the PyPy
project. We showed quite a bit of progress on all three projects (see below)
and we're slowly running out of funds.
Please consider donating more so we can finish those projects!  The three
projects are:

* Py3k (supporting Python 3.x): the release PyPy3 2.2 is imminent.

* STM (software transactional memory): a preview will be released very soon,
  as soon as we fix a few bugs

* NumPy: the work done is included in the PyPy 2.2 release. More details below.

.. _`Raspberry Pi Foundation`: http://www.raspberrypi.org

What is PyPy?
=============

PyPy is a very compliant Python interpreter, almost a drop-in replacement for
CPython 2.7. It's fast (`pypy 2.2 and cpython 2.7.2`_ performance comparison)
due to its integrated tracing JIT compiler.

This release supports x86 machines running Linux 32/64, Mac OS X 64, Windows
32, or ARM (ARMv6 or ARMv7, with VFPv3).

Work on the native Windows 64 is still stalling, we would welcome a volunteer
to handle that.

.. _`pypy 2.2 and cpython 2.7.2`: http://speed.pypy.org

Highlights
==========

* Our Garbage Collector is now "incremental".  It should avoid almost
  all pauses due to a major collection taking place.  Previously, it
  would pause the program (rarely) to walk all live objects, which
  could take arbitrarily long if your process is using a whole lot of
  RAM.  Now the same work is done in steps.  This should make PyPy
  more responsive, e.g. in games.  There are still other pauses, from
  the GC and the JIT, but they should be on the order of 5
  milliseconds each.

* The JIT counters for hot code were never reset, which meant that a
  process running for long enough would eventually JIT-compile more
  and more rarely executed code.  Not only is it useless to compile
  such code, but as more compiled code means more memory used, this
  gives the impression of a memory leak.  This has been tentatively
  fixed by decreasing the counters from time to time.

* NumPy has been split: now PyPy only contains the core module, called
  ``_numpypy``.  The ``numpy`` module itself has been moved to
  ``https://bitbucket.org/pypy/numpy`` and ``numpypy`` disappeared.
  You need to install NumPy separately with a virtualenv:
  ``pip install git+https://bitbucket.org/pypy/numpy.git``;
  or by directly doing
  ``git clone https://bitbucket.org/pypy/numpy.git``,
  ``cd numpy``, ``python setup.py install``.

* non-inlined calls have less overhead

* Things that use ``sys.set_trace`` are now JITted (like coverage)

* JSON encoding used to be very fast, now decoding is as well

* various buffer copying methods experience speedups (like list-of-ints to
  ``int[]`` buffer from cffi)

* We finally wrote (hopefully) all the missing ``os.xxx()`` functions,
  including ``os.startfile()`` on Windows and a handful of rare ones
  on Posix.

* numpy has a rudimentary C API that cooperates with ``cpyext``

Cheers,
Armin Rigo and Maciej Fijalkowski

From laurie at tratt.net  Thu Nov 14 10:47:26 2013
From: laurie at tratt.net (Laurence Tratt)
Date: Thu, 14 Nov 2013 09:47:26 +0000
Subject: [pypy-dev] More strategies
In-Reply-To: <CAMSv6X3Gc4V1WyDchBub1qt=dYrTteQNR4_xSwtgWTpVTKo7ig@mail.gmail.com>
References: <20131107221800.GB3251@overdrive.tratt.net>
 <CAHgd1hEwgSmPwjKceGBo-vaSMfS2LOxK9iZyf_FRXfuvTRG5Lw@mail.gmail.com>
 <CAMSv6X3e9u=h7oO9qBdOOzNtxBx3dnBjfbvUGwWQvZ=Xc2fkSg@mail.gmail.com>
 <20131108162855.GD25802@overdrive.tratt.net>
 <CAMSv6X1NW8k-BB_OaMcLb7fENk2vEWyJRmZjU-MfnH-j4-ingQ@mail.gmail.com>
 <20131113000129.GT17882@phase.tratt.net>
 <CAK5idxSQo5vPVzwsx7pfnDQ+-BE=isbLA_3HyFJ9u-GOn_gUmQ@mail.gmail.com>
 <CAMSv6X3Gc4V1WyDchBub1qt=dYrTteQNR4_xSwtgWTpVTKo7ig@mail.gmail.com>
Message-ID: <20131114094726.GS15430@phase.tratt.net>

On Thu, Nov 14, 2013 at 09:30:21AM +0100, Armin Rigo wrote:

Hi Armin,

> Yes, but the question is whether we want both FloatIntegerListStrategy and
> FloatListStrategy.  The same arguments apply about the JIT not needing to
> check the value of a float grabbed out of the list in the FloatListStrategy
> case, but it's less clear...

I think that question can only really be answered by trying it out and seeing
what the benchmarks say. My guess is that you still want both, but if people
mix ints and floats even more often than I think they do, it may be the case
that FloatListStrategy kicks in pretty rarely.


Laurie
-- 
Personal                                             http://tratt.net/laurie/
Software Development Team                                http://soft-dev.org/
   https://github.com/ltratt              http://twitter.com/laurencetratt


From arigo at tunes.org  Thu Nov 14 10:57:36 2013
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 14 Nov 2013 10:57:36 +0100
Subject: [pypy-dev] More strategies
In-Reply-To: <20131114094726.GS15430@phase.tratt.net>
References: <20131107221800.GB3251@overdrive.tratt.net>
 <CAHgd1hEwgSmPwjKceGBo-vaSMfS2LOxK9iZyf_FRXfuvTRG5Lw@mail.gmail.com>
 <CAMSv6X3e9u=h7oO9qBdOOzNtxBx3dnBjfbvUGwWQvZ=Xc2fkSg@mail.gmail.com>
 <20131108162855.GD25802@overdrive.tratt.net>
 <CAMSv6X1NW8k-BB_OaMcLb7fENk2vEWyJRmZjU-MfnH-j4-ingQ@mail.gmail.com>
 <20131113000129.GT17882@phase.tratt.net>
 <CAK5idxSQo5vPVzwsx7pfnDQ+-BE=isbLA_3HyFJ9u-GOn_gUmQ@mail.gmail.com>
 <CAMSv6X3Gc4V1WyDchBub1qt=dYrTteQNR4_xSwtgWTpVTKo7ig@mail.gmail.com>
 <20131114094726.GS15430@phase.tratt.net>
Message-ID: <CAMSv6X1r9g1xcZ7xNd-6S_po1-+j_PO1X16qFm2gbD8=eap73Q@mail.gmail.com>

Hi Laurence,

On Thu, Nov 14, 2013 at 10:47 AM, Laurence Tratt <laurie at tratt.net> wrote:
> I think that question can only really be answered by trying it out and seeing
> what the benchmarks say. My guess is that you still want both, but if people
> mix ints and floats even more often than I think they do, it may be the case
> that FloatListStrategy kicks in pretty rarely.

Bah, there is another issue.  The following code happens to work right now:

    x, = struct.unpack("d", "ABCDxx\xff\x7f")
    y = struct.pack("d", x)
    assert y == "ABCDxx\xff\x7f"

This works even though x happens to be a NaN; its bit pattern is
preserved.  Such an x could not be stored into a
FloatIntegerListStrategy: if it has the wrong bit pattern, we'd get
the nonsensical result that storing it in a list and reading it back
gives us suddenly an integer object with a random value...

Unsure what to do about that.


A bient?t,

Armin.

From matti.picus at gmail.com  Thu Nov 14 11:40:47 2013
From: matti.picus at gmail.com (matti picus)
Date: Thu, 14 Nov 2013 12:40:47 +0200
Subject: [pypy-dev] Preparing for the release of PyPy 2.2
In-Reply-To: <CAMSv6X1r=JS25Sir7Ssc2p33PwGz55zYDe9_ZPeqhbrnkufc2w@mail.gmail.com>
References: <CAMSv6X1nF-i0_5GkWTPM3SFt_Lj18NKo7R=ZozPGU2PePuL2ew@mail.gmail.com>
 <CAO41-mOKroZscWkLY7WKO-4TvKQ9vT0QSvLZsGqiOd_tK4i4FQ@mail.gmail.com>
 <528284B0.9090209@gmail.com>
 <CAO41-mP6SS4zK8mjeMjd5eh4r_ddo5sQp7nCnz4hPBn_9H2j9w@mail.gmail.com>
 <CAGmFida5ZT5ZmOCqUCn_6=4TQd5rmtzSMVWkxxrqyORbqAM4Rg@mail.gmail.com>
 <5282C177.5080300@gmail.com>
 <CAMSv6X1r=JS25Sir7Ssc2p33PwGz55zYDe9_ZPeqhbrnkufc2w@mail.gmail.com>
Message-ID: <CAO8bCghzspq_PXrpO9ZkUNmy=+cPfPp_8YwoYZe=8doGEYg6gg@mail.gmail.com>

The main problem with Tkinter on windows is that it requires shipping the
tk/tcl runtime It ends up in the <python_install>/tcl directory. So I need
to figure out a reliable way to do this in package.py, like python does in
http://hg.python.org/cpython/file/c27237d57231/Tools/msi/msi.py (lines 1024
or so) . I would suggest that we ship pypy2.2 without a functioning Tkinter
module, to be consistent with all previous pypy versions :)
Matti


On Thu, Nov 14, 2013 at 9:58 AM, Armin Rigo <arigo at tunes.org> wrote:

> Hi Matti,
>
> On Wed, Nov 13, 2013 at 1:01 AM, Matti Picus <matti.picus at gmail.com>
> wrote:
> > The sqlite3 issue was easy, we can simply replace the one on the buildbot
> > with the 3.6.21 version.
>
> I saw the fix.  Thanks!
>
> > I have opened a branch for the tkinter changes, since they require a bit
> > more work.
>
> What is the status of this 'windows-packaging' branch?
>
>
> Armin.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131114/93948d28/attachment-0001.html>

From arigo at tunes.org  Thu Nov 14 11:50:45 2013
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 14 Nov 2013 11:50:45 +0100
Subject: [pypy-dev] Preparing for the release of PyPy 2.2
In-Reply-To: <CAO8bCghzspq_PXrpO9ZkUNmy=+cPfPp_8YwoYZe=8doGEYg6gg@mail.gmail.com>
References: <CAMSv6X1nF-i0_5GkWTPM3SFt_Lj18NKo7R=ZozPGU2PePuL2ew@mail.gmail.com>
 <CAO41-mOKroZscWkLY7WKO-4TvKQ9vT0QSvLZsGqiOd_tK4i4FQ@mail.gmail.com>
 <528284B0.9090209@gmail.com>
 <CAO41-mP6SS4zK8mjeMjd5eh4r_ddo5sQp7nCnz4hPBn_9H2j9w@mail.gmail.com>
 <CAGmFida5ZT5ZmOCqUCn_6=4TQd5rmtzSMVWkxxrqyORbqAM4Rg@mail.gmail.com>
 <5282C177.5080300@gmail.com>
 <CAMSv6X1r=JS25Sir7Ssc2p33PwGz55zYDe9_ZPeqhbrnkufc2w@mail.gmail.com>
 <CAO8bCghzspq_PXrpO9ZkUNmy=+cPfPp_8YwoYZe=8doGEYg6gg@mail.gmail.com>
Message-ID: <CAMSv6X1C3yCEOAUVi66JNRYVLmLn3F5EBbKmTqfwtstUXv0KqA@mail.gmail.com>

Hi Matti.

On Thu, Nov 14, 2013 at 11:40 AM, matti picus <matti.picus at gmail.com> wrote:
> I would suggest that we ship pypy2.2 without a functioning Tkinter
> module, to be consistent with all previous pypy versions :)

Ok, so I believe I can take this:

http://buildbot.pypy.org/nightly/release-2.2.x/pypy-c-jit-67978-2cec7296d7fb-win32.zip

decompress it, replace only the file sqlite3.dll with the newer one,
and rezip it --- right?


A bient?t,

Armin.

From anto.cuni at gmail.com  Thu Nov 14 14:35:03 2013
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Thu, 14 Nov 2013 14:35:03 +0100
Subject: [pypy-dev] More strategies
In-Reply-To: <CAMSv6X1r9g1xcZ7xNd-6S_po1-+j_PO1X16qFm2gbD8=eap73Q@mail.gmail.com>
References: <20131107221800.GB3251@overdrive.tratt.net>
 <CAHgd1hEwgSmPwjKceGBo-vaSMfS2LOxK9iZyf_FRXfuvTRG5Lw@mail.gmail.com>
 <CAMSv6X3e9u=h7oO9qBdOOzNtxBx3dnBjfbvUGwWQvZ=Xc2fkSg@mail.gmail.com>
 <20131108162855.GD25802@overdrive.tratt.net>
 <CAMSv6X1NW8k-BB_OaMcLb7fENk2vEWyJRmZjU-MfnH-j4-ingQ@mail.gmail.com>
 <20131113000129.GT17882@phase.tratt.net>
 <CAK5idxSQo5vPVzwsx7pfnDQ+-BE=isbLA_3HyFJ9u-GOn_gUmQ@mail.gmail.com>
 <CAMSv6X3Gc4V1WyDchBub1qt=dYrTteQNR4_xSwtgWTpVTKo7ig@mail.gmail.com>
 <20131114094726.GS15430@phase.tratt.net>
 <CAMSv6X1r9g1xcZ7xNd-6S_po1-+j_PO1X16qFm2gbD8=eap73Q@mail.gmail.com>
Message-ID: <5284D187.6060505@gmail.com>

On 14/11/13 10:57, Armin Rigo wrote:

> Bah, there is another issue.  The following code happens to work right now:
>
>      x, = struct.unpack("d", "ABCDxx\xff\x7f")
>      y = struct.pack("d", x)
>      assert y == "ABCDxx\xff\x7f"
>
> This works even though x happens to be a NaN; its bit pattern is
> preserved.  Such an x could not be stored into a
> FloatIntegerListStrategy: if it has the wrong bit pattern, we'd get
> the nonsensical result that storing it in a list and reading it back
> gives us suddenly an integer object with a random value...
>
> Unsure what to do about that.

I'm tempted to say that this is an implementation detail, although for all I 
know there might be some code relying on this.

If we REALLY want to support this case, we can always have a 
W_FloatObjectPreservingTheBits which cannot be put in a FloatIntegerListStrategy.
W_FloatObjectPresevingTheBits will be created only by operations like 
struct.unpack, cffi.cast, etc.

Not sure if it's worth the pain.



From wickedgrey at gmail.com  Thu Nov 14 21:31:05 2013
From: wickedgrey at gmail.com (Eli Stevens (Gmail))
Date: Thu, 14 Nov 2013 12:31:05 -0800
Subject: [pypy-dev] More strategies
In-Reply-To: <5284D187.6060505@gmail.com>
References: <20131107221800.GB3251@overdrive.tratt.net>
 <CAHgd1hEwgSmPwjKceGBo-vaSMfS2LOxK9iZyf_FRXfuvTRG5Lw@mail.gmail.com>
 <CAMSv6X3e9u=h7oO9qBdOOzNtxBx3dnBjfbvUGwWQvZ=Xc2fkSg@mail.gmail.com>
 <20131108162855.GD25802@overdrive.tratt.net>
 <CAMSv6X1NW8k-BB_OaMcLb7fENk2vEWyJRmZjU-MfnH-j4-ingQ@mail.gmail.com>
 <20131113000129.GT17882@phase.tratt.net>
 <CAK5idxSQo5vPVzwsx7pfnDQ+-BE=isbLA_3HyFJ9u-GOn_gUmQ@mail.gmail.com>
 <CAMSv6X3Gc4V1WyDchBub1qt=dYrTteQNR4_xSwtgWTpVTKo7ig@mail.gmail.com>
 <20131114094726.GS15430@phase.tratt.net>
 <CAMSv6X1r9g1xcZ7xNd-6S_po1-+j_PO1X16qFm2gbD8=eap73Q@mail.gmail.com>
 <5284D187.6060505@gmail.com>
Message-ID: <CADa34LC=nHyPtKbjEy_EJV_qPC-WmVmz1ARWB=4KYqDrrgsqSg@mail.gmail.com>

>From http://en.wikipedia.org/wiki/NaN#Encoding :

"The state/value of the remaining bits (i.e. other than the ones used
to identify a NaN as NaN, including the quiet/signaled bits) are not
defined by the standard except that they must not be all zero. This
value is called the 'payload' of the NaN. If an operation has a single
NaN input and propagates it to the output, the result NaN's payload
should be that of the input NaN. If there are multiple NaN inputs, the
result NaN's payload should be from one of the input NaNs; the
standard does not specify which."

Speaking from the perspective of a potential user of pypy, I would
like to advocate strongly against any solution which trades
correctness for speed in a way that isn't easily and explicitly
controllable by the end user at runtime (I'd point to the CUDA fast
math options as examples of this done well:
http://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/index.html#options-for-steering-cuda-compilation).

Running into a situation where custom NaN payloads are in use and
having pypy not handle them properly would be incredibly frustrating
(and hard to debug, especially since it would be turning something
exceptional into something ordinary-looking but incorrect).

I would also like to point out that anyone who mixes ints and floats
in a list (or anywhere, really) under python 2.7 is asking for
trouble.

>>> [a / b for a, b in itertools.izip([1, 1, 1.0, 1.0], [2, 2.0, 2, 2.0])]
[0, 0.5, 0.5, 0.5]

IMHO, It's not a use case worth optimizing for.

Thanks for all the hard work!

Cheers,
Eli

On Thu, Nov 14, 2013 at 5:35 AM, Antonio Cuni <anto.cuni at gmail.com> wrote:
> On 14/11/13 10:57, Armin Rigo wrote:
>
>> Bah, there is another issue.  The following code happens to work right
>> now:
>>
>>      x, = struct.unpack("d", "ABCDxx\xff\x7f")
>>      y = struct.pack("d", x)
>>      assert y == "ABCDxx\xff\x7f"
>>
>> This works even though x happens to be a NaN; its bit pattern is
>> preserved.  Such an x could not be stored into a
>> FloatIntegerListStrategy: if it has the wrong bit pattern, we'd get
>> the nonsensical result that storing it in a list and reading it back
>> gives us suddenly an integer object with a random value...
>>
>> Unsure what to do about that.
>
>
> I'm tempted to say that this is an implementation detail, although for all I
> know there might be some code relying on this.
>
> If we REALLY want to support this case, we can always have a
> W_FloatObjectPreservingTheBits which cannot be put in a
> FloatIntegerListStrategy.
> W_FloatObjectPresevingTheBits will be created only by operations like
> struct.unpack, cffi.cast, etc.
>
> Not sure if it's worth the pain.
>
>
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev

From arigo at tunes.org  Thu Nov 14 21:55:23 2013
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 14 Nov 2013 21:55:23 +0100
Subject: [pypy-dev] More strategies
In-Reply-To: <CADa34LC=nHyPtKbjEy_EJV_qPC-WmVmz1ARWB=4KYqDrrgsqSg@mail.gmail.com>
References: <20131107221800.GB3251@overdrive.tratt.net>
 <CAHgd1hEwgSmPwjKceGBo-vaSMfS2LOxK9iZyf_FRXfuvTRG5Lw@mail.gmail.com>
 <CAMSv6X3e9u=h7oO9qBdOOzNtxBx3dnBjfbvUGwWQvZ=Xc2fkSg@mail.gmail.com>
 <20131108162855.GD25802@overdrive.tratt.net>
 <CAMSv6X1NW8k-BB_OaMcLb7fENk2vEWyJRmZjU-MfnH-j4-ingQ@mail.gmail.com>
 <20131113000129.GT17882@phase.tratt.net>
 <CAK5idxSQo5vPVzwsx7pfnDQ+-BE=isbLA_3HyFJ9u-GOn_gUmQ@mail.gmail.com>
 <CAMSv6X3Gc4V1WyDchBub1qt=dYrTteQNR4_xSwtgWTpVTKo7ig@mail.gmail.com>
 <20131114094726.GS15430@phase.tratt.net>
 <CAMSv6X1r9g1xcZ7xNd-6S_po1-+j_PO1X16qFm2gbD8=eap73Q@mail.gmail.com>
 <5284D187.6060505@gmail.com>
 <CADa34LC=nHyPtKbjEy_EJV_qPC-WmVmz1ARWB=4KYqDrrgsqSg@mail.gmail.com>
Message-ID: <CAMSv6X0=a0WbuvOsOxU_QYPZTfKGjz+ioUX1Mr+ipXTPxpeTqA@mail.gmail.com>

Hi Eli,

On Thu, Nov 14, 2013 at 9:31 PM, Eli Stevens (Gmail)
<wickedgrey at gmail.com> wrote:
> From http://en.wikipedia.org/wiki/NaN#Encoding :
>
> "The state/value of the remaining bits (i.e. other than the ones used
> to identify a NaN as NaN, including the quiet/signaled bits) are not
> defined by the standard except that they must not be all zero. This
> value is called the 'payload' of the NaN. If an operation has a single
> NaN input and propagates it to the output, the result NaN's payload
> should be that of the input NaN. If there are multiple NaN inputs, the
> result NaN's payload should be from one of the input NaNs; the
> standard does not specify which."

Thanks!  That's enough to convince me that we need to be extremely
careful about NaN bits.  For what it's worth, that's not enough to
make us give up on the idea of lists-of-floats-or-ints :-)


A bient?t,

Armin.

From arigo at tunes.org  Thu Nov 14 22:07:51 2013
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 14 Nov 2013 22:07:51 +0100
Subject: [pypy-dev] More strategies
In-Reply-To: <5284D187.6060505@gmail.com>
References: <20131107221800.GB3251@overdrive.tratt.net>
 <CAHgd1hEwgSmPwjKceGBo-vaSMfS2LOxK9iZyf_FRXfuvTRG5Lw@mail.gmail.com>
 <CAMSv6X3e9u=h7oO9qBdOOzNtxBx3dnBjfbvUGwWQvZ=Xc2fkSg@mail.gmail.com>
 <20131108162855.GD25802@overdrive.tratt.net>
 <CAMSv6X1NW8k-BB_OaMcLb7fENk2vEWyJRmZjU-MfnH-j4-ingQ@mail.gmail.com>
 <20131113000129.GT17882@phase.tratt.net>
 <CAK5idxSQo5vPVzwsx7pfnDQ+-BE=isbLA_3HyFJ9u-GOn_gUmQ@mail.gmail.com>
 <CAMSv6X3Gc4V1WyDchBub1qt=dYrTteQNR4_xSwtgWTpVTKo7ig@mail.gmail.com>
 <20131114094726.GS15430@phase.tratt.net>
 <CAMSv6X1r9g1xcZ7xNd-6S_po1-+j_PO1X16qFm2gbD8=eap73Q@mail.gmail.com>
 <5284D187.6060505@gmail.com>
Message-ID: <CAMSv6X2OTxkP=yC3zMRU95dUTrpfvaG=7+mVQXJgzjVONjTg3g@mail.gmail.com>

Hi Antonio,

On Thu, Nov 14, 2013 at 2:35 PM, Antonio Cuni <anto.cuni at gmail.com> wrote:
> W_FloatObjectPresevingTheBits will be created only by operations like
> struct.unpack, cffi.cast, etc.

That's not enough: if you read one such float into a variable and then
append that variable into another list, then the other list also needs
to record the fact that it contains special NaNs.

It seems we could pick the following solution instead: keep
FloatStrategy, storing a RPython list of floats --- including possibly
special NaNs; and add FloatIntStrategy, which cannot store special
NaNs.  We check for the special NaNs when we add into a
FloatIntStrategy list, and when converting from FloatStrategy to
FloatIntStrategy.  For the latter case we do indeed need to check all
items, which sounds a bit pointless, but (1) this is already a good
progress over the current situation, which is that we need to allocate
a W_FloatObject per item and a new RPython list to hold them; and (2)
doing the check over all items upon conversion is actually the same
total amount of work as it would be if we checked each item as it was
added to the FloatStrategy list.

(Fwiw, I'm also fond of the idea that it should actually be a
"FloatIntNoneStrategy"; it would improve the situation even for lists
of int-or-None.)


A bient?t,

Armin.

From fijall at gmail.com  Fri Nov 15 08:07:20 2013
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Fri, 15 Nov 2013 09:07:20 +0200
Subject: [pypy-dev] More strategies
In-Reply-To: <CAMSv6X2OTxkP=yC3zMRU95dUTrpfvaG=7+mVQXJgzjVONjTg3g@mail.gmail.com>
References: <20131107221800.GB3251@overdrive.tratt.net>
 <CAHgd1hEwgSmPwjKceGBo-vaSMfS2LOxK9iZyf_FRXfuvTRG5Lw@mail.gmail.com>
 <CAMSv6X3e9u=h7oO9qBdOOzNtxBx3dnBjfbvUGwWQvZ=Xc2fkSg@mail.gmail.com>
 <20131108162855.GD25802@overdrive.tratt.net>
 <CAMSv6X1NW8k-BB_OaMcLb7fENk2vEWyJRmZjU-MfnH-j4-ingQ@mail.gmail.com>
 <20131113000129.GT17882@phase.tratt.net>
 <CAK5idxSQo5vPVzwsx7pfnDQ+-BE=isbLA_3HyFJ9u-GOn_gUmQ@mail.gmail.com>
 <CAMSv6X3Gc4V1WyDchBub1qt=dYrTteQNR4_xSwtgWTpVTKo7ig@mail.gmail.com>
 <20131114094726.GS15430@phase.tratt.net>
 <CAMSv6X1r9g1xcZ7xNd-6S_po1-+j_PO1X16qFm2gbD8=eap73Q@mail.gmail.com>
 <5284D187.6060505@gmail.com>
 <CAMSv6X2OTxkP=yC3zMRU95dUTrpfvaG=7+mVQXJgzjVONjTg3g@mail.gmail.com>
Message-ID: <CAK5idxQaHG7BbwhUG9pTmuXxr=YrgcLcqe8CxZ09nBnuNfd=eQ@mail.gmail.com>

On Thu, Nov 14, 2013 at 11:07 PM, Armin Rigo <arigo at tunes.org> wrote:
> Hi Antonio,
>
> On Thu, Nov 14, 2013 at 2:35 PM, Antonio Cuni <anto.cuni at gmail.com> wrote:
>> W_FloatObjectPresevingTheBits will be created only by operations like
>> struct.unpack, cffi.cast, etc.
>
> That's not enough: if you read one such float into a variable and then
> append that variable into another list, then the other list also needs
> to record the fact that it contains special NaNs.
>
> It seems we could pick the following solution instead: keep
> FloatStrategy, storing a RPython list of floats --- including possibly
> special NaNs; and add FloatIntStrategy, which cannot store special
> NaNs.  We check for the special NaNs when we add into a
> FloatIntStrategy list, and when converting from FloatStrategy to
> FloatIntStrategy.  For the latter case we do indeed need to check all
> items, which sounds a bit pointless, but (1) this is already a good
> progress over the current situation, which is that we need to allocate
> a W_FloatObject per item and a new RPython list to hold them; and (2)
> doing the check over all items upon conversion is actually the same
> total amount of work as it would be if we checked each item as it was
> added to the FloatStrategy list.
>
> (Fwiw, I'm also fond of the idea that it should actually be a
> "FloatIntNoneStrategy"; it would improve the situation even for lists
> of int-or-None.)
>
>
> A bient?t,
>
> Armin.
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev

+1 for IntFloatNoneButNotStrangeNans strategy

From arigo at tunes.org  Fri Nov 15 11:29:44 2013
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 15 Nov 2013 11:29:44 +0100
Subject: [pypy-dev] Preparing for the release of PyPy 2.2
In-Reply-To: <CAGmFida5ZT5ZmOCqUCn_6=4TQd5rmtzSMVWkxxrqyORbqAM4Rg@mail.gmail.com>
References: <CAMSv6X1nF-i0_5GkWTPM3SFt_Lj18NKo7R=ZozPGU2PePuL2ew@mail.gmail.com>
 <CAO41-mOKroZscWkLY7WKO-4TvKQ9vT0QSvLZsGqiOd_tK4i4FQ@mail.gmail.com>
 <528284B0.9090209@gmail.com>
 <CAO41-mP6SS4zK8mjeMjd5eh4r_ddo5sQp7nCnz4hPBn_9H2j9w@mail.gmail.com>
 <CAGmFida5ZT5ZmOCqUCn_6=4TQd5rmtzSMVWkxxrqyORbqAM4Rg@mail.gmail.com>
Message-ID: <CAMSv6X1RAa+fmKrM5+itMhhhf8Ddqi3CzqkrzgKMb4igZQGjeQ@mail.gmail.com>

Hi Amaury,

On Tue, Nov 12, 2013 at 10:56 PM, Amaury Forgeot d'Arc
<amauryfa at gmail.com> wrote:
>> -Include a prebuilt TkInter???
>
> Not only DLLs, but also header files and .lib.
> cffi need a development environment.

Actually, why?  Does tkinter really need more than sqlite3?  It seems
that in both cases the buildbot needs a development environment, but
the final released binary only needs the DLLs.  Can't we just do the
same thing for both and be done?


A bient?t,

Armin.

From techtonik at gmail.com  Fri Nov 15 12:04:30 2013
From: techtonik at gmail.com (anatoly techtonik)
Date: Fri, 15 Nov 2013 14:04:30 +0300
Subject: [pypy-dev] List of finished NumPy functions // Progressbar
Message-ID: <CAPkN8xLNjCDdfzn=MebbAY1NQX5aoE5p99NygcrqB-_zvZpcGw@mail.gmail.com>

Hi,

Please CC.

I am looking for that table of supported and not supported NumPy
functions in PyPy. I expected to find it here, but it is missing -
http://pypy.org/numpydonate.html

It is also interesting to see NumPy status as a progress bar that
consists from the area of all highlighted blocks with function names
that are scattered over the page. It will be possible to add some
complexity indicators to these blocks and they will directly affect
which area of the progress bar is covered by this block. The biggest
blocks can then be decomposed into smaller ones, which may not carry
only function names, but rather task/research steps to be done to make
the stuff operational.

If this visual decomposition is implemented, it will enable focused
parallel development, and include more people, as more people will be
able to see where can they help.
--
anatoly t.

From yeomanyaacov at gmail.com  Fri Nov 15 15:50:34 2013
From: yeomanyaacov at gmail.com (Yaacov Finkelman)
Date: Fri, 15 Nov 2013 09:50:34 -0500
Subject: [pypy-dev] List of finished NumPy functions // Progressbar
In-Reply-To: <CAPkN8xLNjCDdfzn=MebbAY1NQX5aoE5p99NygcrqB-_zvZpcGw@mail.gmail.com>
References: <CAPkN8xLNjCDdfzn=MebbAY1NQX5aoE5p99NygcrqB-_zvZpcGw@mail.gmail.com>
Message-ID: <CAGBU8t4hwoB5fuQSXdhaTP9du9UudQT5VN=DH6O9UrP1sdG61A@mail.gmail.com>

Is this roughly what you are looking for?
http://buildbot.pypy.org/numpy-status/latest.html
It was linked to on http://morepypy.blogspot.com/ a while ago.

On Fri, Nov 15, 2013 at 6:04 AM, anatoly techtonik <techtonik at gmail.com> wrote:
> Hi,
>
> Please CC.
>
> I am looking for that table of supported and not supported NumPy
> functions in PyPy. I expected to find it here, but it is missing -
> http://pypy.org/numpydonate.html
>
> It is also interesting to see NumPy status as a progress bar that
> consists from the area of all highlighted blocks with function names
> that are scattered over the page. It will be possible to add some
> complexity indicators to these blocks and they will directly affect
> which area of the progress bar is covered by this block. The biggest
> blocks can then be decomposed into smaller ones, which may not carry
> only function names, but rather task/research steps to be done to make
> the stuff operational.
>
> If this visual decomposition is implemented, it will enable focused
> parallel development, and include more people, as more people will be
> able to see where can they help.
> --
> anatoly t.
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev

From mvyskocil at suse.cz  Fri Nov 15 15:56:40 2013
From: mvyskocil at suse.cz (Michal Vyskocil)
Date: Fri, 15 Nov 2013 15:56:40 +0100
Subject: [pypy-dev] Missing PyUnicode_Compat
Message-ID: <20131115145640.GA18596@linux-xtv2.site>

Hi all,

during a build of pypy-numpy from
https://bitbucket.org/pypy/numpy.git

I've got a following error

cc: numpy/core/src/dummymodule.c
In file included from numpy/core/src/private/npy_pycompat.h:4:0,
                 from numpy/core/src/dummymodule.c:11:
numpy/core/include/numpy/npy_3kcompat.h: In function 'PyUnicode_ConcatAndDel':
numpy/core/include/numpy/npy_3kcompat.h:125:5: warning: implicit declaration of function 'PyUnicode_Concat' [-Wimplicit-function-declaration]
     newobj = PyUnicode_Concat(*left, right);

So first of all, is dummymodule.c imporant or errors can be ignored there?
Guessing from its name, I'd not say so.

But during a grepping I found that even if it is somehow declared in
cpyext/stubs.py, the symbol is missing from pypy itself, where some others
PyUnicode_ things are in there. Is that correct and cpyext/stub.py is enough,
or not?

Regards
Michal Vyskocil
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131115/6acbf93d/attachment.sig>

From rymg19 at gmail.com  Fri Nov 15 16:28:21 2013
From: rymg19 at gmail.com (Ryan)
Date: Fri, 15 Nov 2013 09:28:21 -0600
Subject: [pypy-dev] Missing PyUnicode_Compat
In-Reply-To: <20131115145640.GA18596@linux-xtv2.site>
References: <20131115145640.GA18596@linux-xtv2.site>
Message-ID: <a7eb527e-585d-401d-bfcb-6f691f621ed5@email.android.com>

I don't know much about this, but it doesn't matter as long as the linker sees a symbol somewhere. So, I wouldn't say that those are important regardless.

Michal Vyskocil <mvyskocil at suse.cz> wrote:
>Hi all,
>
>during a build of pypy-numpy from
>https://bitbucket.org/pypy/numpy.git
>
>I've got a following error
>
>cc: numpy/core/src/dummymodule.c
>In file included from numpy/core/src/private/npy_pycompat.h:4:0,
>                 from numpy/core/src/dummymodule.c:11:
>numpy/core/include/numpy/npy_3kcompat.h: In function
>'PyUnicode_ConcatAndDel':
>numpy/core/include/numpy/npy_3kcompat.h:125:5: warning: implicit
>declaration of function 'PyUnicode_Concat'
>[-Wimplicit-function-declaration]
>     newobj = PyUnicode_Concat(*left, right);
>
>So first of all, is dummymodule.c imporant or errors can be ignored
>there?
>Guessing from its name, I'd not say so.
>
>But during a grepping I found that even if it is somehow declared in
>cpyext/stubs.py, the symbol is missing from pypy itself, where some
>others
>PyUnicode_ things are in there. Is that correct and cpyext/stub.py is
>enough,
>or not?
>
>Regards
>Michal Vyskocil
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>pypy-dev mailing list
>pypy-dev at python.org
>https://mail.python.org/mailman/listinfo/pypy-dev

-- 
Sent from my Android phone with K-9 Mail. Please excuse my brevity.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131115/4a17e0c3/attachment.html>

From techtonik at gmail.com  Sat Nov 16 08:20:41 2013
From: techtonik at gmail.com (anatoly techtonik)
Date: Sat, 16 Nov 2013 10:20:41 +0300
Subject: [pypy-dev] List of finished NumPy functions // Progressbar
In-Reply-To: <CAGBU8t4hwoB5fuQSXdhaTP9du9UudQT5VN=DH6O9UrP1sdG61A@mail.gmail.com>
References: <CAPkN8xLNjCDdfzn=MebbAY1NQX5aoE5p99NygcrqB-_zvZpcGw@mail.gmail.com>
 <CAGBU8t4hwoB5fuQSXdhaTP9du9UudQT5VN=DH6O9UrP1sdG61A@mail.gmail.com>
Message-ID: <CAPkN8xK2xsav51j3xPGLCrfEqnmHFk_zgfSAyTYUXpmp3E8PGA@mail.gmail.com>

Thanks. It is exactly I was looking for. I have found it two hours
later, but it will be convenient for people donating to NumPy cause to
have direct reference to the page. If you have some metrics - donation
plot over the whole period, I'll be extremely interested to see what
will be an effect after placing that link in appropriate place, and
how do blog progress posts rely on donation income. Also, how an
ability to see the progress of any particular feature and donate to
specific features can increase interest and the flow.

About progress bar. This one is basically it:
Overall: 508/558 names 138/161 ndarray attributes, 34/48 dtype
attributes, 75/134 generic attributes, 28/32 flatiter attributes,
19/28 ufunc attributes
But it ideally should be drawn as a horizontal completion line (with
achievement milestones..?, historical period marks..?, future
achievement goals..?).
--
anatoly t.


On Fri, Nov 15, 2013 at 5:50 PM, Yaacov Finkelman
<yeomanyaacov at gmail.com> wrote:
> Is this roughly what you are looking for?
> http://buildbot.pypy.org/numpy-status/latest.html
> It was linked to on http://morepypy.blogspot.com/ a while ago.
>
> On Fri, Nov 15, 2013 at 6:04 AM, anatoly techtonik <techtonik at gmail.com> wrote:
>> Hi,
>>
>> Please CC.
>>
>> I am looking for that table of supported and not supported NumPy
>> functions in PyPy. I expected to find it here, but it is missing -
>> http://pypy.org/numpydonate.html
>>
>> It is also interesting to see NumPy status as a progress bar that
>> consists from the area of all highlighted blocks with function names
>> that are scattered over the page. It will be possible to add some
>> complexity indicators to these blocks and they will directly affect
>> which area of the progress bar is covered by this block. The biggest
>> blocks can then be decomposed into smaller ones, which may not carry
>> only function names, but rather task/research steps to be done to make
>> the stuff operational.
>>
>> If this visual decomposition is implemented, it will enable focused
>> parallel development, and include more people, as more people will be
>> able to see where can they help.
>> --
>> anatoly t.
>> _______________________________________________
>> pypy-dev mailing list
>> pypy-dev at python.org
>> https://mail.python.org/mailman/listinfo/pypy-dev

From arigo at tunes.org  Sat Nov 16 11:13:55 2013
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 16 Nov 2013 11:13:55 +0100
Subject: [pypy-dev] List of finished NumPy functions // Progressbar
In-Reply-To: <CAPkN8xK2xsav51j3xPGLCrfEqnmHFk_zgfSAyTYUXpmp3E8PGA@mail.gmail.com>
References: <CAPkN8xLNjCDdfzn=MebbAY1NQX5aoE5p99NygcrqB-_zvZpcGw@mail.gmail.com>
 <CAGBU8t4hwoB5fuQSXdhaTP9du9UudQT5VN=DH6O9UrP1sdG61A@mail.gmail.com>
 <CAPkN8xK2xsav51j3xPGLCrfEqnmHFk_zgfSAyTYUXpmp3E8PGA@mail.gmail.com>
Message-ID: <CAMSv6X3ndsNBBMNbtsar0cTSd_h7hBBhVSjTCZdRjkKAv4M34A@mail.gmail.com>

Hi Anatoly,

On Sat, Nov 16, 2013 at 8:20 AM, anatoly techtonik <techtonik at gmail.com> wrote:
> (...)
> But it ideally should be drawn as a horizontal completion line (with
> achievement milestones..?, historical period marks..?, future
> achievement goals..?).

I suppose this answer is not going to help, but how about helping?  We
welcome any contribution, and certainly more the ones in the form of
working code (like most open source projects).  The html page above is
generated by the following command in a virtualenv where jinja2 was
installed:

pypy pypy/module/micronumpy/tool/numready/ pypy numpy-compat.html


A bient?t,

Armin.

From anto.cuni at gmail.com  Sat Nov 16 13:25:38 2013
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Sat, 16 Nov 2013 13:25:38 +0100
Subject: [pypy-dev] More strategies
In-Reply-To: <CAK5idxQaHG7BbwhUG9pTmuXxr=YrgcLcqe8CxZ09nBnuNfd=eQ@mail.gmail.com>
References: <20131107221800.GB3251@overdrive.tratt.net>
 <CAHgd1hEwgSmPwjKceGBo-vaSMfS2LOxK9iZyf_FRXfuvTRG5Lw@mail.gmail.com>
 <CAMSv6X3e9u=h7oO9qBdOOzNtxBx3dnBjfbvUGwWQvZ=Xc2fkSg@mail.gmail.com>
 <20131108162855.GD25802@overdrive.tratt.net>
 <CAMSv6X1NW8k-BB_OaMcLb7fENk2vEWyJRmZjU-MfnH-j4-ingQ@mail.gmail.com>
 <20131113000129.GT17882@phase.tratt.net>
 <CAK5idxSQo5vPVzwsx7pfnDQ+-BE=isbLA_3HyFJ9u-GOn_gUmQ@mail.gmail.com>
 <CAMSv6X3Gc4V1WyDchBub1qt=dYrTteQNR4_xSwtgWTpVTKo7ig@mail.gmail.com>
 <20131114094726.GS15430@phase.tratt.net>
 <CAMSv6X1r9g1xcZ7xNd-6S_po1-+j_PO1X16qFm2gbD8=eap73Q@mail.gmail.com>
 <5284D187.6060505@gmail.com>
 <CAMSv6X2OTxkP=yC3zMRU95dUTrpfvaG=7+mVQXJgzjVONjTg3g@mail.gmail.com>
 <CAK5idxQaHG7BbwhUG9pTmuXxr=YrgcLcqe8CxZ09nBnuNfd=eQ@mail.gmail.com>
Message-ID: <CAHPg78wjwkzTu1HRvxvu9BkERiDh+BLYEueS3RoDp+r=jb66Rg@mail.gmail.com>

yes, +1 for IntFloatNoneButNotStrangeNans as well. It seems like the best
we can do, and it probably covers 99.9% of the real world use cases


On Fri, Nov 15, 2013 at 8:07 AM, Maciej Fijalkowski <fijall at gmail.com>wrote:

> On Thu, Nov 14, 2013 at 11:07 PM, Armin Rigo <arigo at tunes.org> wrote:
> > Hi Antonio,
> >
> > On Thu, Nov 14, 2013 at 2:35 PM, Antonio Cuni <anto.cuni at gmail.com>
> wrote:
> >> W_FloatObjectPresevingTheBits will be created only by operations like
> >> struct.unpack, cffi.cast, etc.
> >
> > That's not enough: if you read one such float into a variable and then
> > append that variable into another list, then the other list also needs
> > to record the fact that it contains special NaNs.
> >
> > It seems we could pick the following solution instead: keep
> > FloatStrategy, storing a RPython list of floats --- including possibly
> > special NaNs; and add FloatIntStrategy, which cannot store special
> > NaNs.  We check for the special NaNs when we add into a
> > FloatIntStrategy list, and when converting from FloatStrategy to
> > FloatIntStrategy.  For the latter case we do indeed need to check all
> > items, which sounds a bit pointless, but (1) this is already a good
> > progress over the current situation, which is that we need to allocate
> > a W_FloatObject per item and a new RPython list to hold them; and (2)
> > doing the check over all items upon conversion is actually the same
> > total amount of work as it would be if we checked each item as it was
> > added to the FloatStrategy list.
> >
> > (Fwiw, I'm also fond of the idea that it should actually be a
> > "FloatIntNoneStrategy"; it would improve the situation even for lists
> > of int-or-None.)
> >
> >
> > A bient?t,
> >
> > Armin.
> > _______________________________________________
> > pypy-dev mailing list
> > pypy-dev at python.org
> > https://mail.python.org/mailman/listinfo/pypy-dev
>
> +1 for IntFloatNoneButNotStrangeNans strategy
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131116/415da647/attachment.html>

From bookaa at rorsoft.com  Sun Nov 17 00:48:50 2013
From: bookaa at rorsoft.com (bookaa)
Date: Sun, 17 Nov 2013 07:48:50 +0800
Subject: [pypy-dev] I find a bug
Message-ID: <28DC485DEA914AE0B01C68A123D168B0@vSHliutaotao>

--- a/pypy/interpreter/test/test_app_main.py    Thu Nov 14 03:50:49 
2013 -0500
+++ b/pypy/interpreter/test/test_app_main.py    Sun Nov 17 07:46:26 2013 
+0800
@@ -942,7 +942,8 @@

         self.w_tmp_dir = self.space.wrap(tmp_dir)

-        foo_py = prefix.join('foo.py').write("pass")
+        foo_py = prefix.join('foo.py')
+        foo_py.write("pass")
         self.w_foo_py = self.space.wrap(str(foo_py))

     def test_setup_bootstrap_path(self): 


From arigo at tunes.org  Sun Nov 17 09:49:56 2013
From: arigo at tunes.org (Armin Rigo)
Date: Sun, 17 Nov 2013 09:49:56 +0100
Subject: [pypy-dev] I find a bug
In-Reply-To: <28DC485DEA914AE0B01C68A123D168B0@vSHliutaotao>
References: <28DC485DEA914AE0B01C68A123D168B0@vSHliutaotao>
Message-ID: <CAMSv6X0L8+3fCe1DBgu5d1A8uZ5ZE6TiL2iGG0iXd9EgA25tbw@mail.gmail.com>

Hi Bookaa,

On Sun, Nov 17, 2013 at 12:48 AM, bookaa <bookaa at rorsoft.com> wrote:
> -        foo_py = prefix.join('foo.py').write("pass")
> +        foo_py = prefix.join('foo.py')
> +        foo_py.write("pass")

Thanks !

Armin

From tobias.oberstein at tavendo.de  Sun Nov 17 17:22:52 2013
From: tobias.oberstein at tavendo.de (Tobias Oberstein)
Date: Sun, 17 Nov 2013 08:22:52 -0800
Subject: [pypy-dev] FreeBSD 9.2 buildslave: first run
Message-ID: <634914A010D0B943A035D226786325D4446A18BDD4@EXVMBX020-12.exch020.serverdata.net>

Hi,

the new FreeBSD buildslave has run the first time:

http://buildbot.pypy.org/builders/pypy-c-jit-freebsd-9-x86-64/builds/6

PyPy was built successfully, but there are a couple of issues:

https://bugs.pypy.org/issue1637
https://bugs.pypy.org/issue1638
https://bugs.pypy.org/issue1639

I'll address those.

There is another issue that bugs me .. look at the built times:

[Timer] Timings:
[Timer] annotate                       ---  576.0 s
[Timer] rtype_lltype                   --- 1323.7 s
[Timer] pyjitpl_lltype                 --- 1098.6 s
[Timer] backendopt_lltype              ---  314.2 s
[Timer] stackcheckinsertion_lltype     ---  161.5 s
[Timer] database_c                     ---  451.3 s
[Timer] source_c                       ---  668.5 s
[Timer] compile_c                      --- 2403.8 s
[Timer] ===========================================
[Timer] Total:                         --- 6997.6 s

and compare with "Linux 64 on allegro64":

[Timer] Timings:
[Timer] annotate                       ---  641.5 s
[Timer] rtype_lltype                   --- 1528.5 s
[Timer] pyjitpl_lltype                 ---  938.7 s
[Timer] backendopt_lltype              ---  242.2 s
[Timer] stackcheckinsertion_lltype     ---  206.8 s
[Timer] database_c                     ---  325.3 s
[Timer] source_c                       ---  357.7 s
[Timer] compile_c                      ---  246.4 s
[Timer] ===========================================
[Timer] Total:                         --- 4487.1 s

*****

Why the heck does "compile_c" take 10x the time on FreeBSD?

Note that FreeBSD builder uses Clang .. could that be a reason?

Any hints on that are welcome!

Cheers,
Tobias




From fijall at gmail.com  Sun Nov 17 18:09:00 2013
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sun, 17 Nov 2013 19:09:00 +0200
Subject: [pypy-dev] FreeBSD 9.2 buildslave: first run
In-Reply-To: <634914A010D0B943A035D226786325D4446A18BDD4@EXVMBX020-12.exch020.serverdata.net>
References: <634914A010D0B943A035D226786325D4446A18BDD4@EXVMBX020-12.exch020.serverdata.net>
Message-ID: <CAK5idxR_A+--AbpexGUF+DQKe3aFf0K5wSNy0_62M8X6c40BRw@mail.gmail.com>

On Sun, Nov 17, 2013 at 6:22 PM, Tobias Oberstein
<tobias.oberstein at tavendo.de> wrote:
> Hi,
>
> the new FreeBSD buildslave has run the first time:
>
> http://buildbot.pypy.org/builders/pypy-c-jit-freebsd-9-x86-64/builds/6
>
> PyPy was built successfully, but there are a couple of issues:
>
> https://bugs.pypy.org/issue1637
> https://bugs.pypy.org/issue1638
> https://bugs.pypy.org/issue1639
>
> I'll address those.
>
> There is another issue that bugs me .. look at the built times:
>
> [Timer] Timings:
> [Timer] annotate                       ---  576.0 s
> [Timer] rtype_lltype                   --- 1323.7 s
> [Timer] pyjitpl_lltype                 --- 1098.6 s
> [Timer] backendopt_lltype              ---  314.2 s
> [Timer] stackcheckinsertion_lltype     ---  161.5 s
> [Timer] database_c                     ---  451.3 s
> [Timer] source_c                       ---  668.5 s
> [Timer] compile_c                      --- 2403.8 s
> [Timer] ===========================================
> [Timer] Total:                         --- 6997.6 s
>
> and compare with "Linux 64 on allegro64":
>
> [Timer] Timings:
> [Timer] annotate                       ---  641.5 s
> [Timer] rtype_lltype                   --- 1528.5 s
> [Timer] pyjitpl_lltype                 ---  938.7 s
> [Timer] backendopt_lltype              ---  242.2 s
> [Timer] stackcheckinsertion_lltype     ---  206.8 s
> [Timer] database_c                     ---  325.3 s
> [Timer] source_c                       ---  357.7 s
> [Timer] compile_c                      ---  246.4 s
> [Timer] ===========================================
> [Timer] Total:                         --- 4487.1 s
>
> *****
>
> Why the heck does "compile_c" take 10x the time on FreeBSD?
>
> Note that FreeBSD builder uses Clang .. could that be a reason?
>
> Any hints on that are welcome!
>
> Cheers,
> Tobias
>
>
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev

I would guess because we can't guess the number of processors we don't
parallelize the build correctly.

From arigo at tunes.org  Mon Nov 18 14:23:35 2013
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 18 Nov 2013 14:23:35 +0100
Subject: [pypy-dev] Windows version 2.2.1
Message-ID: <CAMSv6X3gRtaMSxGCqoKb4vkZVKu1SbtVWE=5hSVPgmWKRv15pw@mail.gmail.com>

Hi all,

Here's the release candidate for PyPy 2.2.1 on Windows: it fixes the
json decoding, and adds Tk support (thanks Matti!).  Please tell me if
there is some problem with it.

http://buildbot.pypy.org/nightly/release-2.2.x/pypy-c-jit-68207-1567dba349e6-win32.zip

On other platforms, PyPy 2.2.1 will not be released, as nothing changed.


A bient?t,

Armin.

From mvyskocil at suse.cz  Tue Nov 19 11:21:37 2013
From: mvyskocil at suse.cz (Michal Vyskocil)
Date: Tue, 19 Nov 2013 11:21:37 +0100
Subject: [pypy-dev] pypy's INSTALL_SCHEME
Message-ID: <20131119102137.GB1159@linux-xtv2.site>

Hi,

I found pypy's behavior of pypy setup.py install --prefix=/usr quite
surprising as it installs all files to /usr/site-packages, which is not
the most expected location. But distutils/commands/install.py says so

'pypy': {
    'purelib': '$base/site-packages',
    'platlib': '$base/site-packages',
    'headers': '$base/include',
    'scripts': '$base/bin',
    'data'   : '$base',
},

which does not makes a sense to me as /usr/site-packages is not in
default sys.path

$ pypy -c "import sys; print(sys.path)"
['', '/usr/lib64/pypy-2.2/lib_pypy/__extensions__',
'/usr/lib64/pypy-2.2/lib_pypy', '/usr/lib64/pypy-2.2/lib-python/2.7',
'/usr/lib64/pypy-2.2/lib-python/2.7/lib-tk',
'/usr/lib64/pypy-2.2/lib-python/2.7/plat-linux2',
'/usr/lib64/pypy-2.2/site-packages']

and all distros will refuse to install files into such location. Can't
be it changed to point to pypy's own site-packages directory?

Regards
Michal Vyskocil
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131119/af9a64aa/attachment.sig>

From arigo at tunes.org  Wed Nov 20 10:22:08 2013
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 20 Nov 2013 10:22:08 +0100
Subject: [pypy-dev] pypy's INSTALL_SCHEME
In-Reply-To: <20131119102137.GB1159@linux-xtv2.site>
References: <20131119102137.GB1159@linux-xtv2.site>
Message-ID: <CAMSv6X3ber_c5dO2Wbb8Edwyxx7FsXg=Cc2t3S7tfJ0PDCtVYw@mail.gmail.com>

Hi Michal,

On Tue, Nov 19, 2013 at 11:21 AM, Michal Vyskocil <mvyskocil at suse.cz> wrote:
> I found pypy's behavior of pypy setup.py install --prefix=/usr quite
> surprising as it installs all files to /usr/site-packages, which is not
> the most expected location.

Indeed, that should maybe be worked around.  However:

> which does not makes a sense to me as /usr/site-packages is not in
> default sys.path

We don't have a default install location.  It's a bit like complaining
that on CPython, if you use "python setup.py install
--prefix=/foo/bar" you end up with things in
"/foo/bar/.../site-packages", which is indeed not in sys.path.  The
paths you see, "/usr/lib64/pypy-2.2", are choosen by your
distribution, and they make sense there, but the recommended path to
install the distribution-independent release is usually
"/opt/pypy-2.2".

Do you have a suggestion about how "--prefix=/usr" would end up
meaning "/usr/lib64/pypy-2.2"?  More importantly, why do you need to
specify any "--prefix" at all?


A bient?t,

Armin.

From yury at shurup.com  Wed Nov 20 10:49:21 2013
From: yury at shurup.com (Yury V. Zaytsev)
Date: Wed, 20 Nov 2013 10:49:21 +0100
Subject: [pypy-dev] pypy's INSTALL_SCHEME
In-Reply-To: <CAMSv6X3ber_c5dO2Wbb8Edwyxx7FsXg=Cc2t3S7tfJ0PDCtVYw@mail.gmail.com>
References: <20131119102137.GB1159@linux-xtv2.site>
 <CAMSv6X3ber_c5dO2Wbb8Edwyxx7FsXg=Cc2t3S7tfJ0PDCtVYw@mail.gmail.com>
Message-ID: <1384940961.3090.24.camel@newpride>

Hi Armin,

I'd like to second the complaint by Michal.

On Wed, 2013-11-20 at 10:22 +0100, Armin Rigo wrote:
> 
> Do you have a suggestion about how "--prefix=/usr" would end up
> meaning "/usr/lib64/pypy-2.2"?  

In what concerns your question, please have a look here:

http://docs.python.org/2/install/

For arch-indep modules (purelib), everything is clear, the default
install location should be just

    $prefix/lib/pypyX.Y/site-packages

so it's a matter of

-    'purelib': '$base/site-packages',
+    'purelib': '$base/lib/pypy{}.{}/site-packages'.format(X, Y),

One could argue that every pure-Python module that is compatible with
PyPy declaring compatibility with a given version of Python should be
available for this version of Python as well and so go to pythonX.Y
rather than pypyX.Y, but I'm not sure whether this is a good default.

In my opinion, it's better to leave this up to the distribution
packagers and policy makers.

For binary extensions (platlib), the path depends upon the
implementation of multiarch for every specific distribution. For Red
Hat, for instance, the packages containing binary extensions should go
to

    $prefix/lib64/pypyX.Y/site-packages

on x86_64 systems and to

    $prefix/lib/pypyX.Y/site-packages

on i386 systems. I'm not sure how this is implemented for CPython
though, maybe Michal can comment on this.

I know that in CPython you have platlib and purelib, which are defined
in sysconfig.py and can be overridden from the command line. Probably
the distributions patch sysconfig.py in their packages, because from
what I recall the CPython default is to use lib for both platlib and
purelib.

I guess that PyPy should ideally follow suit...

> More importantly, why do you need to specify any "--prefix" at all?

I hope that Michal will correct me if I'm wrong, but from what I know,
it's a distribution policy in SuSe for Python packages.

I don't quite understand the reasoning behind this choice, but it has
something to do with staged installation, which is the way the packages
are built. Somehow, the PYC-files are not properly generated if the
--prefix is not specified...

Hope that helps,

-- 
Sincerely yours,
Yury V. Zaytsev



From mvyskocil at suse.cz  Wed Nov 20 12:50:09 2013
From: mvyskocil at suse.cz (Michal Vyskocil)
Date: Wed, 20 Nov 2013 12:50:09 +0100
Subject: [pypy-dev] pypy's INSTALL_SCHEME
In-Reply-To: <1384940961.3090.24.camel@newpride>
References: <20131119102137.GB1159@linux-xtv2.site>
 <CAMSv6X3ber_c5dO2Wbb8Edwyxx7FsXg=Cc2t3S7tfJ0PDCtVYw@mail.gmail.com>
 <1384940961.3090.24.camel@newpride>
Message-ID: <20131120115009.GA22574@linux-xtv2.site>

On Wed, Nov 20, 2013 at 10:49:21AM +0100, Yury V. Zaytsev wrote:
> Hi Armin,
> 
> I'd like to second the complaint by Michal.
> 
> On Wed, 2013-11-20 at 10:22 +0100, Armin Rigo wrote:
> > 
> > Do you have a suggestion about how "--prefix=/usr" would end up
> > meaning "/usr/lib64/pypy-2.2"?  
Hi Armin,


> 
> In what concerns your question, please have a look here:
> 
> http://docs.python.org/2/install/

yes, that's the document, thanks Yury! The prefix on nixes usually
defaults to /usr/local/ and distros changed that to /usr/.

> 
> For arch-indep modules (purelib), everything is clear, the default
> install location should be just
> 
>     $prefix/lib/pypyX.Y/site-packages
> 
> so it's a matter of
> 
> -    'purelib': '$base/site-packages',
> +    'purelib': '$base/lib/pypy{}.{}/site-packages'.format(X, Y),

actually sources does have site-packages in
lib-python/2.7/site-packages, but you got the point.
+    'purelib': '$base/lib/pypy{}.{}/lib-python/{}/site-packages'.format(X, Y, sys.version),

> 
> One could argue that every pure-Python module that is compatible with
> PyPy declaring compatibility with a given version of Python should be
> available for this version of Python as well and so go to pythonX.Y
> rather than pypyX.Y, but I'm not sure whether this is a good default.

Actually we do not have any plans for making pypy- packages, except
pypy-numpy. So users are expected to use pypy with virtualenv, rather
with distro packages by default.

BTW: there is still a problem with bytecode-compatibility, which has
been solved quite recently via __pycache__, so CPython3 and PyPy3 might
coexists together better.

> 
> In my opinion, it's better to leave this up to the distribution
> packagers and policy makers.
> 
> For binary extensions (platlib), the path depends upon the
> implementation of multiarch for every specific distribution. For Red
> Hat, for instance, the packages containing binary extensions should go
> to
> 
>     $prefix/lib64/pypyX.Y/site-packages
> 
> on x86_64 systems and to
> 
>     $prefix/lib/pypyX.Y/site-packages
> 
> on i386 systems. I'm not sure how this is implemented for CPython
> though, maybe Michal can comment on this.
> 
> I know that in CPython you have platlib and purelib, which are defined
> in sysconfig.py and can be overridden from the command line. Probably
> the distributions patch sysconfig.py in their packages, because from
> what I recall the CPython default is to use lib for both platlib and
> purelib.
> 
> I guess that PyPy should ideally follow suit...

Yes, I think so - with a few deviations like a correct path to
site-packages, but both CPython and PyPy should have similar defaults.

> 
> > More importantly, why do you need to specify any "--prefix" at all?
> 
> I hope that Michal will correct me if I'm wrong, but from what I know,
> it's a distribution policy in SuSe for Python packages.

I have no idea, so I will ask our python maintainers.

Regards
Michal Vyskocil
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131120/85534120/attachment.sig>

From alex.gaynor at gmail.com  Thu Nov 21 21:32:42 2013
From: alex.gaynor at gmail.com (Alex Gaynor)
Date: Thu, 21 Nov 2013 12:32:42 -0800
Subject: [pypy-dev] Doing a 2.2.1 release
Message-ID: <CAFRnB2UjWBATVQE+CRdcw08YmCZ7yQYPHE3cn86DeFpr1JNqhw@mail.gmail.com>

Hey all,

In the 2.2 release we just did, there's a bug in ssl/socket which makes
simple things like requests.get("https://pypi.python.org") with the
requests module fail. I've landed a fix in 963c6d6d7d6c. What do people
think about doing a release?

Alex

-- 
"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero
GPG Key fingerprint: 125F 5C67 DFE9 4084
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131121/3be74729/attachment.html>

From tobias.oberstein at tavendo.de  Thu Nov 21 23:12:05 2013
From: tobias.oberstein at tavendo.de (Tobias Oberstein)
Date: Thu, 21 Nov 2013 14:12:05 -0800
Subject: [pypy-dev] Doing a 2.2.1 release
In-Reply-To: <CAFRnB2UjWBATVQE+CRdcw08YmCZ7yQYPHE3cn86DeFpr1JNqhw@mail.gmail.com>
References: <CAFRnB2UjWBATVQE+CRdcw08YmCZ7yQYPHE3cn86DeFpr1JNqhw@mail.gmail.com>
Message-ID: <634914A010D0B943A035D226786325D4446A257232@EXVMBX020-12.exch020.serverdata.net>

>In the 2.2 release we just did, there's a bug in ssl/socket which makes simple things like requests.get("https://pypi.python.org") with the requests module fail. I've landed a fix in?963c6d6d7d6c. What do people >think about doing a release?

Absolutely, please do! 2.2 brings a lot of good stuff, and it would be a pity if people would not use it and stay on 2.1 and wait for 2.3.

Will you merge in everything from current head also, or is that fix a backport to 2.2?

/Tobias

From anto.cuni at gmail.com  Fri Nov 22 19:11:04 2013
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Fri, 22 Nov 2013 19:11:04 +0100
Subject: [pypy-dev] PyArray_Type cpyext bug
Message-ID: <528F9E38.7070201@gmail.com>

Hi,

I committed a cpyext+numpy failing test in a3c3b75a7f2b.
In short, the following fails:

     if (!PyArg_ParseTuple(args, "O!", &PyArray_Type, &obj))
         return NULL;

The problem is that PyArray_Type is currently defined in ndarrayobject.c but 
never initialized:

     PyTypeObject PyArray_Type;

not a suprise that we get a segfault when we try to use it.

However, I'm not sure about what is the best way to fix it, so I ask the 
cpyext wizards :)

I think that at the end what we want is an object for which &PyArray_Type is 
equal to the PyObject* that we get when we pass _numpypy.multiarray.ndarray to C.

One possibility is to run the following code in some initialization function:

     static PyObject* _PyArray_Type;
     #define PyArray_Type (*_PyArray_Type)

     PyObject* np = PyImport_ImportModule("numpy");
     if (!np)
         return;

     _PyArray_Type = PyObject_GetAttrString(np, "ndarray");
     if (!My_PyArray_Type)
         return;


I'm sure there is a better way to do this. However, I tried to play with the 
cpyext source for a while and didn't manage to find it. Any suggestion?

ciao,
Anto

From matti.picus at gmail.com  Sat Nov 23 20:55:53 2013
From: matti.picus at gmail.com (Matti Picus)
Date: Sat, 23 Nov 2013 21:55:53 +0200
Subject: [pypy-dev] PyArray_Type cpyext bug
In-Reply-To: <528F9E38.7070201@gmail.com>
References: <528F9E38.7070201@gmail.com>
Message-ID: <52910849.4080801@gmail.com>

Can we just say "don't do that?"
I guess the answer is no...
Going down the initialization route seems to be the way numpy does it, I see
  import_array();
used extensively in numpy c code.
Although making sure it is only called once seems to really complicate 
the header files, with API defines and strange macros.
Matti

On 11/22/2013 08:11 PM, Antonio Cuni wrote:
> Hi,
>
> I committed a cpyext+numpy failing test in a3c3b75a7f2b.
> In short, the following fails:
>
>     if (!PyArg_ParseTuple(args, "O!", &PyArray_Type, &obj))
>         return NULL;
>
> The problem is that PyArray_Type is currently defined in 
> ndarrayobject.c but never initialized:
>
>     PyTypeObject PyArray_Type;
>
> not a suprise that we get a segfault when we try to use it.
>
> However, I'm not sure about what is the best way to fix it, so I ask 
> the cpyext wizards :)
>
> I think that at the end what we want is an object for which 
> &PyArray_Type is equal to the PyObject* that we get when we pass 
> _numpypy.multiarray.ndarray to C.
>
> One possibility is to run the following code in some initialization 
> function:
>
>     static PyObject* _PyArray_Type;
>     #define PyArray_Type (*_PyArray_Type)
>
>     PyObject* np = PyImport_ImportModule("numpy");
>     if (!np)
>         return;
>
>     _PyArray_Type = PyObject_GetAttrString(np, "ndarray");
>     if (!My_PyArray_Type)
>         return;
>
>
> I'm sure there is a better way to do this. However, I tried to play 
> with the cpyext source for a while and didn't manage to find it. Any 
> suggestion?
>
> ciao,
> Anto
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev


From arigo at tunes.org  Sun Nov 24 11:55:22 2013
From: arigo at tunes.org (Armin Rigo)
Date: Sun, 24 Nov 2013 11:55:22 +0100
Subject: [pypy-dev] Doing a 2.2.1 release
In-Reply-To: <634914A010D0B943A035D226786325D4446A257232@EXVMBX020-12.exch020.serverdata.net>
References: <CAFRnB2UjWBATVQE+CRdcw08YmCZ7yQYPHE3cn86DeFpr1JNqhw@mail.gmail.com>
 <634914A010D0B943A035D226786325D4446A257232@EXVMBX020-12.exch020.serverdata.net>
Message-ID: <CAMSv6X294nhqU++XU_6W7rt1+epp=PLOLg1y03ehGKvD+yf_TQ@mail.gmail.com>

Hi all, hi Tobias,

On Thu, Nov 21, 2013 at 11:12 PM, Tobias Oberstein
<tobias.oberstein at tavendo.de> wrote:
>> What do people think about doing a release?
>
> Will you merge in everything from current head also, or is that fix a backport to 2.2?

It's a backport.  I have transplanted the checkins to "default" that
seem relevant: the windows-packaging branch, the pypyjson bug on
platforms where sizeof(wchar_t)==2, the fixes to package.py, the fixes
to lib/_tkinter for FreeBSD, the cpyext/config.h fix for MSVC, and
Alex's (untested) fix to lib-python/2.7/socket.py.

Can people check it out and complain if anything is missing?

Thanks!


Armin.

From matti.picus at gmail.com  Sun Nov 24 16:52:48 2013
From: matti.picus at gmail.com (matti picus)
Date: Sun, 24 Nov 2013 17:52:48 +0200
Subject: [pypy-dev] Doing a 2.2.1 release
In-Reply-To: <CAMSv6X294nhqU++XU_6W7rt1+epp=PLOLg1y03ehGKvD+yf_TQ@mail.gmail.com>
References: <CAFRnB2UjWBATVQE+CRdcw08YmCZ7yQYPHE3cn86DeFpr1JNqhw@mail.gmail.com>
 <634914A010D0B943A035D226786325D4446A257232@EXVMBX020-12.exch020.serverdata.net>
 <CAMSv6X294nhqU++XU_6W7rt1+epp=PLOLg1y03ehGKvD+yf_TQ@mail.gmail.com>
Message-ID: <CAO8bCgi_HQS5XcKCmRfJjGLPk9MrWO-BhxMCuLJsZYcsEMpZ_A@mail.gmail.com>

Armin - can you kick the bots so we can test it?
Matti


On Sun, Nov 24, 2013 at 12:55 PM, Armin Rigo <arigo at tunes.org> wrote:

> Hi all, hi Tobias,
>
> On Thu, Nov 21, 2013 at 11:12 PM, Tobias Oberstein
> <tobias.oberstein at tavendo.de> wrote:
> >> What do people think about doing a release?
> >
> > Will you merge in everything from current head also, or is that fix a
> backport to 2.2?
>
> It's a backport.  I have transplanted the checkins to "default" that
> seem relevant: the windows-packaging branch, the pypyjson bug on
> platforms where sizeof(wchar_t)==2, the fixes to package.py, the fixes
> to lib/_tkinter for FreeBSD, the cpyext/config.h fix for MSVC, and
> Alex's (untested) fix to lib-python/2.7/socket.py.
>
> Can people check it out and complain if anything is missing?
>
> Thanks!
>
>
> Armin.
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131124/a39fbb44/attachment.html>

From fijall at gmail.com  Sun Nov 24 17:03:04 2013
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sun, 24 Nov 2013 17:03:04 +0100
Subject: [pypy-dev] Doing a 2.2.1 release
In-Reply-To: <CAO8bCgi_HQS5XcKCmRfJjGLPk9MrWO-BhxMCuLJsZYcsEMpZ_A@mail.gmail.com>
References: <CAFRnB2UjWBATVQE+CRdcw08YmCZ7yQYPHE3cn86DeFpr1JNqhw@mail.gmail.com>
 <634914A010D0B943A035D226786325D4446A257232@EXVMBX020-12.exch020.serverdata.net>
 <CAMSv6X294nhqU++XU_6W7rt1+epp=PLOLg1y03ehGKvD+yf_TQ@mail.gmail.com>
 <CAO8bCgi_HQS5XcKCmRfJjGLPk9MrWO-BhxMCuLJsZYcsEMpZ_A@mail.gmail.com>
Message-ID: <CAK5idxQvjH-yb3RrAONn6PC-nsQOAnpfDy7DEcHkaDE-x8cJ-g@mail.gmail.com>

On Sun, Nov 24, 2013 at 4:52 PM, matti picus <matti.picus at gmail.com> wrote:
> Armin - can you kick the bots so we can test it?

you can kick the bots too btw (through the web interface)

> Matti
>
>
> On Sun, Nov 24, 2013 at 12:55 PM, Armin Rigo <arigo at tunes.org> wrote:
>>
>> Hi all, hi Tobias,
>>
>> On Thu, Nov 21, 2013 at 11:12 PM, Tobias Oberstein
>> <tobias.oberstein at tavendo.de> wrote:
>> >> What do people think about doing a release?
>> >
>> > Will you merge in everything from current head also, or is that fix a
>> > backport to 2.2?
>>
>> It's a backport.  I have transplanted the checkins to "default" that
>> seem relevant: the windows-packaging branch, the pypyjson bug on
>> platforms where sizeof(wchar_t)==2, the fixes to package.py, the fixes
>> to lib/_tkinter for FreeBSD, the cpyext/config.h fix for MSVC, and
>> Alex's (untested) fix to lib-python/2.7/socket.py.
>>
>> Can people check it out and complain if anything is missing?
>>
>> Thanks!
>>
>>
>> Armin.
>> _______________________________________________
>> pypy-dev mailing list
>> pypy-dev at python.org
>> https://mail.python.org/mailman/listinfo/pypy-dev
>
>
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>

From arigo at tunes.org  Sun Nov 24 17:58:52 2013
From: arigo at tunes.org (Armin Rigo)
Date: Sun, 24 Nov 2013 17:58:52 +0100
Subject: [pypy-dev] Doing a 2.2.1 release
In-Reply-To: <CAO8bCgi_HQS5XcKCmRfJjGLPk9MrWO-BhxMCuLJsZYcsEMpZ_A@mail.gmail.com>
References: <CAFRnB2UjWBATVQE+CRdcw08YmCZ7yQYPHE3cn86DeFpr1JNqhw@mail.gmail.com>
 <634914A010D0B943A035D226786325D4446A257232@EXVMBX020-12.exch020.serverdata.net>
 <CAMSv6X294nhqU++XU_6W7rt1+epp=PLOLg1y03ehGKvD+yf_TQ@mail.gmail.com>
 <CAO8bCgi_HQS5XcKCmRfJjGLPk9MrWO-BhxMCuLJsZYcsEMpZ_A@mail.gmail.com>
Message-ID: <CAMSv6X2PdaE1pqMEmOuT90Z8arSsfxff+Um9PA7KjDqE5yEXNw@mail.gmail.com>

Hi Matti,

Do you have a clue about
http://buildbot.pypy.org/builders/pypy-c-jit-win-x86-32/builds/1207 ?


A bient?t,

Armin.

From matti.picus at gmail.com  Mon Nov 25 00:54:45 2013
From: matti.picus at gmail.com (Matti Picus)
Date: Mon, 25 Nov 2013 01:54:45 +0200
Subject: [pypy-dev] pytest and numpypy status
In-Reply-To: <52910418.7050308@gmail.com>
References: <52910418.7050308@gmail.com>
Message-ID: <529291C5.4050300@gmail.com>

Currently we are running tests via nose for the pypy-compat fork of 
numpy under pypyin a nightly buildbot task. I have changed the test 
runner to pytest in a branch of the buildbot, and it runs and creates a 
summary page, but I fear the page with ~2000 failures will create too 
much of a load on the buildbot.
I tried to sort the failures by frequency here 
http://bpaste.net/show/152720/ , this counts how many times each "E   
<errortxt>" appears in the log file.
The biggest stumbling blocks to passing more tests seem to be
- a lack of support for dtypes based on objects
- datetime dtype support
- unicode dtype support.
- on line 245 of the paste, AssertionError which is followed in the log 
file by a report of differences in results over an acceptable threshold.

On the positive side, we pass more than 50% of the numpy tests:
======= 1854 failed, 2124 passed, 93 skipped, 65 error in 184.72 seconds 
=======
Matti


From holger at merlinux.eu  Mon Nov 25 07:47:04 2013
From: holger at merlinux.eu (Holger Krekel)
Date: Mon, 25 Nov 2013 07:47:04 +0100
Subject: [pypy-dev] pytest and numpypy status
In-Reply-To: <529291C5.4050300@gmail.com>
References: <52910418.7050308@gmail.com> <529291C5.4050300@gmail.com>
Message-ID: <02e9263f-ddec-4462-9e7d-068e132a885e@email.android.com>

Matti Picus <matti.picus at gmail.com> wrote:
>Currently we are running tests via nose for the pypy-compat fork of 
>numpy under pypyin a nightly buildbot task. I have changed the test 
>runner to pytest in a branch of the buildbot, and it runs and creates a
>
>summary page, but I fear the page with ~2000 failures will create too 
>much of a load on the buildbot.
>I tried to sort the failures by frequency here 
>http://bpaste.net/show/152720/ , this counts how many times each "E   
><errortxt>" appears in the log file.
>The biggest stumbling blocks to passing more tests seem to be
>- a lack of support for dtypes based on objects
>- datetime dtype support
>- unicode dtype support.
>- on line 245 of the paste, AssertionError which is followed in the log
>
>file by a report of differences in results over an acceptable
>threshold.
>
>On the positive side, we pass more than 50% of the numpy tests:
>======= 1854 failed, 2124 passed, 93 skipped, 65 error in 184.72
>seconds 
>=======
>Matti
>
>_______________________________________________
>pypy-dev mailing list
>pypy-dev at python.org
>https://mail.python.org/mailman/listinfo/pypy-dev

Hi matti, 
Not sure it helps much but you can use - - tb=native or =line to shorten output and quicken it's creation. 
Cheers, holger
-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

From anto.cuni at gmail.com  Mon Nov 25 14:09:18 2013
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Mon, 25 Nov 2013 14:09:18 +0100
Subject: [pypy-dev] PyArray_Type cpyext bug
In-Reply-To: <52910849.4080801@gmail.com>
References: <528F9E38.7070201@gmail.com> <52910849.4080801@gmail.com>
Message-ID: <52934BFE.7080908@gmail.com>

Hi Matti,

On 23/11/13 20:55, Matti Picus wrote:
> Can we just say "don't do that?"
> I guess the answer is no...
> Going down the initialization route seems to be the way numpy does it, I see
>   import_array();
> used extensively in numpy c code.
> Although making sure it is only called once seems to really complicate the
> header files, with API defines and strange macros.

I'm not sure about what you are saying. Of course we need to support 
PyArray_Type because it's part (a very important part) of the numpy C API. 
"import_array()" is unrelated because its role is to setup the functions to be 
called from C code, while PyArray_Type is part of static data.

Anyway, I fixed it in 7f3a776cc72a, and I think it's the "correct" fix.

ciao,
Anto

From cfbolz at gmx.de  Mon Nov 25 18:05:45 2013
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Mon, 25 Nov 2013 18:05:45 +0100
Subject: [pypy-dev] some stm links
Message-ID: <52938369.7000402@gmx.de>

Hi all,

during OOPSLA I came upon some papers/projects that may be relevant to
the STM effort, that I found interesting (though I haven't read them in
detail yet, just skimmed).

IBM Research Japan is doing a project to try to eliminate the GIL of
Ruby's C VM using HTM extensions of various processors (mostly IBM
stuff, I think):

http://researcher.watson.ibm.com/researcher/view_person_subpage.php?id=4800

CDSChecker is a model checker for concurrent C/C++ code that claims to
put a lot of effort into making the model checker efficient, to be able
to scale to interestingly sized programs:
http://demsky.eecs.uci.edu/c11modelchecker.html

An STM paper from OOPSLA 2003 won the "most influential paper award":
http://www.cl.cam.ac.uk/research/srg/netos/papers/2003-ls4lt.pdf

Just though I'd throw these out there.

Cheers,

Carl Friedrich

From rymg19 at gmail.com  Mon Nov 25 23:51:18 2013
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Mon, 25 Nov 2013 16:51:18 -0600
Subject: [pypy-dev] Print unicode strings in RPython?
Message-ID: <CAO41-mO9b88RFo8ihrmvbVMo1-2A8Kb1bVQ6d_K0Ne_oGPndxw@mail.gmail.com>

Hello,

I'm writing an app in RPython. Whenever I try and print a unicode string,
the compiled application will crash with a UnicodeEncodingError. Here is an
example:

def entry_point(argv):
    print u'\u2502'
    return 0

def target(driver, args):
    return entry_point, None

When running it, I get this:

RPython traceback:
  File "implement.c", line 164, in entry_point
  File "rpython_rtyper_lltypesystem_rstr.c", line 190, in
ll_str__UnicodeR_Ptr_GcStruct_rpy_unic_rpy_unico
Fatal RPython error: UnicodeEncodeError
Aborted (core dumped)

Am I missing something? The app runs fine under CPython.

-- 
Ryan
When your hammer is C++, everything begins to look like a thumb.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131125/f3d69d94/attachment.html>

From ahti at ahti.bluemoon.ee  Tue Nov 26 13:15:57 2013
From: ahti at ahti.bluemoon.ee (Ahti Heinla)
Date: Tue, 26 Nov 2013 14:15:57 +0200
Subject: [pypy-dev] Proposal for sprint in Tallinn, Estonia
Message-ID: <12834110.ecgp6i7Rsd@karoliine>

Hi,

	I am new to PyPy, but very impressed with what you guys have done.
Myself I am best known for having been a founding engineer and Chief Technical
Architect for Skype. Python is my favourite language, and I want to help PyPy
somehow.

	How about I organise a sprint in Tallinn, Estonia (where I live)? I can hopefully
drum up some interest among local developers, perhaps also get some top
ex-Skype engineers to join. The requirement to contribute a week full-time is a
deterrent for many though, so I am not sure yet how many people I can get.

	Myself I have no background in interpreters/compilers, but have written
code for 32 years, often optimised lowlevel code (assembly, C++). I would
need some hand-holding to get started, but I am sure I'd be productive in
a day or two.

Ahti
ahtih on IRC

From fijall at gmail.com  Tue Nov 26 23:22:25 2013
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Tue, 26 Nov 2013 23:22:25 +0100
Subject: [pypy-dev] Print unicode strings in RPython?
In-Reply-To: <CAO41-mO9b88RFo8ihrmvbVMo1-2A8Kb1bVQ6d_K0Ne_oGPndxw@mail.gmail.com>
References: <CAO41-mO9b88RFo8ihrmvbVMo1-2A8Kb1bVQ6d_K0Ne_oGPndxw@mail.gmail.com>
Message-ID: <CAK5idxTox5+bYAKedDckPdG=mG45tyFiXh-hOKQ+8GvU88sLZQ@mail.gmail.com>

On Mon, Nov 25, 2013 at 11:51 PM, Ryan Gonzalez <rymg19 at gmail.com> wrote:
> Hello,
>
> I'm writing an app in RPython. Whenever I try and print a unicode string,
> the compiled application will crash with a UnicodeEncodingError. Here is an
> example:
>
> def entry_point(argv):
>     print u'\u2502'
>     return 0
>
> def target(driver, args):
>     return entry_point, None
>
> When running it, I get this:
>
> RPython traceback:
>   File "implement.c", line 164, in entry_point
>   File "rpython_rtyper_lltypesystem_rstr.c", line 190, in
> ll_str__UnicodeR_Ptr_GcStruct_rpy_unic_rpy_unico
> Fatal RPython error: UnicodeEncodeError
> Aborted (core dumped)
>
> Am I missing something? The app runs fine under CPython.
>
> --
> Ryan
> When your hammer is C++, everything begins to look like a thumb.

The answer is that you can't print unicode strings in RPython. We can
consider lifting this limitation, but right now it tries to encode
them as ascii. Maybe print u"xxx".encode("utf-8") is what you want?

Cheers,
fijal

From andrewfr_ice at yahoo.com  Wed Nov 27 03:05:54 2013
From: andrewfr_ice at yahoo.com (Andrew Francis)
Date: Tue, 26 Nov 2013 18:05:54 -0800 (PST)
Subject: [pypy-dev] some stm links
In-Reply-To: <52938369.7000402@gmx.de>
References: <52938369.7000402@gmx.de>
Message-ID: <1385517954.7784.YahooMailNeo@web140703.mail.bf1.yahoo.com>

Hi Carl:

>An STM paper from OOPSLA 2003 won the "most influential paper award":
>http://www.cl.cam.ac.uk/research/srg/netos/papers/2003-ls4lt.pdf

>Just though I'd throw these out there.


I would recommend the book "Transactional Memory 2nd edition" (2010) by Harris, Larus, and Rajwar.?

Cheers,
Andrew

P.S - once I get the latest pypy stm branch compiled, I would like to give a talk to the Montreal Python User's Group about PyPy stm. Using "Transactional Memory," I would describe the approach. I'll also want to see if I can write a PyPy STM version of the Santa Claus problem a la the Haskell solution in "Beautiful Code."
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131126/5ed051d1/attachment.html>

From alendit at googlemail.com  Wed Nov 27 09:17:07 2013
From: alendit at googlemail.com (Dimitri Vorona)
Date: Wed, 27 Nov 2013 09:17:07 +0100
Subject: [pypy-dev] Pluggable HTM
Message-ID: <CACu5Zfki2gcw2yGpo8-qFcra7a4yo8r381KzLFd4c9XAyrthyQ@mail.gmail.com>

Hi,

the original STM proposal spoke of HTM as of a thing of a far future. Now,
Haswells are out and provide built-in HTM support in form of TSX. In the
near future I expect more and more systems to have it.

Are there plan to make PyPy use HTM if it is available on the system?

Regards,

Dimitri.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131127/2d6b0c2b/attachment.html>

From anto.cuni at gmail.com  Wed Nov 27 11:00:49 2013
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Wed, 27 Nov 2013 11:00:49 +0100
Subject: [pypy-dev] Proposal for sprint in Tallinn, Estonia
In-Reply-To: <12834110.ecgp6i7Rsd@karoliine>
References: <12834110.ecgp6i7Rsd@karoliine>
Message-ID: <CAHPg78z7H+t1+nfxK=vS84e6i7Z1fTeD7xRrtJ=j-wzG5dqFWA@mail.gmail.com>

Hi,

as I already said on IRC, I'd be glad to do a sprint in Tallin, as I've
already been there and I really liked the city :)


On Tue, Nov 26, 2013 at 1:15 PM, Ahti Heinla <ahti at ahti.bluemoon.ee> wrote:

> Hi,
>
>         I am new to PyPy, but very impressed with what you guys have done.
> Myself I am best known for having been a founding engineer and Chief
> Technical
> Architect for Skype. Python is my favourite language, and I want to help
> PyPy
> somehow.
>
>         How about I organise a sprint in Tallinn, Estonia (where I live)?
> I can hopefully
> drum up some interest among local developers, perhaps also get some top
> ex-Skype engineers to join. The requirement to contribute a week full-time
> is a
> deterrent for many though, so I am not sure yet how many people I can get.
>
>         Myself I have no background in interpreters/compilers, but have
> written
> code for 32 years, often optimised lowlevel code (assembly, C++). I would
> need some hand-holding to get started, but I am sure I'd be productive in
> a day or two.
>
> Ahti
> ahtih on IRC
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20131127/192f6eeb/attachment.html>

From arigo at tunes.org  Wed Nov 27 11:20:44 2013
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 27 Nov 2013 11:20:44 +0100
Subject: [pypy-dev] Print unicode strings in RPython?
In-Reply-To: <CAK5idxTox5+bYAKedDckPdG=mG45tyFiXh-hOKQ+8GvU88sLZQ@mail.gmail.com>
References: <CAO41-mO9b88RFo8ihrmvbVMo1-2A8Kb1bVQ6d_K0Ne_oGPndxw@mail.gmail.com>
 <CAK5idxTox5+bYAKedDckPdG=mG45tyFiXh-hOKQ+8GvU88sLZQ@mail.gmail.com>
Message-ID: <CAMSv6X2V6cZMJ4mgGVEFUkJec==pGmQbh5x8_QM38kK5zmTGkA@mail.gmail.com>

Hi Ryan,

Also, the usual warning:

http://doc.pypy.org/en/latest/faq.html#do-i-have-to-rewrite-my-programs-in-rpython

I believe we already pointed you to it, but it doesn't hurt repeating,
because you say "The app runs fine under CPython".  I'm sure it also
works fine under PyPy.


A bient?t,

Armin.

From dimaqq at gmail.com  Wed Nov 27 16:07:54 2013
From: dimaqq at gmail.com (Dima Tisnek)
Date: Wed, 27 Nov 2013 16:07:54 +0100
Subject: [pypy-dev] clarification cffi vs _ffi_ vs _ffi vs libffi vs _rawffi
	vs rffi
Message-ID: <CAGGBzXKxf+3UnuLjBPkuLWxNv1w4YFuPS=X7wuH3Tjq9yVZkwA@mail.gmail.com>

there seems to be some confusion (could be all mine) between N terms
related to FFI:

are there both _ffi_ and _ffi, or is one of those a typo?

extending page mentions libffi and not cffi, is it out of date, or is
there a reason to stay away from cffi, although recommended in general
faq page?

in my own testing, I noticed that equivalent code cffi code is faster
than ctypes, is there an explanation for this?

thanks,
d.

docs in question:
http://doc.pypy.org/en/latest/faq.html
http://doc.pypy.org/en/latest/extending.html
http://doc.pypy.org/en/latest/rffi.html

From arigo at tunes.org  Wed Nov 27 17:24:10 2013
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 27 Nov 2013 17:24:10 +0100
Subject: [pypy-dev] clarification cffi vs _ffi_ vs _ffi vs libffi vs
 _rawffi vs rffi
In-Reply-To: <CAGGBzXKxf+3UnuLjBPkuLWxNv1w4YFuPS=X7wuH3Tjq9yVZkwA@mail.gmail.com>
References: <CAGGBzXKxf+3UnuLjBPkuLWxNv1w4YFuPS=X7wuH3Tjq9yVZkwA@mail.gmail.com>
Message-ID: <CAMSv6X0PRBECCpmNhPSZSOL4S_oak5h+B92Mrk1VDT8Vs5PV6A@mail.gmail.com>

Hi Dima,

On Wed, Nov 27, 2013 at 4:07 PM, Dima Tisnek <dimaqq at gmail.com> wrote:
> there seems to be some confusion (could be all mine) between N terms
> related to FFI:

As a user of PyPy you only need to concern yourself with cffi and
ctypes.  The rest is all a number of more-or-less related internal
sub-parts of PyPy or the RPython translation toolchain that all
attempt to do accessing C-level functionality: rffi, _ffi, _rawffi.
And libffi is the existing, non-PyPy-related, C library that is
(sometimes) used by these other levels.

> docs in question:
> http://doc.pypy.org/en/latest/faq.html
> http://doc.pypy.org/en/latest/extending.html
> http://doc.pypy.org/en/latest/rffi.html

"rffi" is about writing RPython code, e.g. a special extension module
for PyPy; in most cases you don't need that.  The page
http://doc.pypy.org/en/latest/extending.html is a bit outdated :-(
Thanks for pointing it out.

> in my own testing, I noticed that equivalent code cffi code is faster
> than ctypes, is there an explanation for this?

On top of PyPy, yes, definitely.  CFFI is simpler than ctypes; it was
*designed* to be simple.  Although it was not particularly designed
for PyPy and works fine on CPython (and was actually first developed
for CPython), this has the intended benefit that PyPy's JIT can do a
good job on it.


A bient?t,

Armin.

From arigo at tunes.org  Wed Nov 27 18:08:29 2013
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 27 Nov 2013 18:08:29 +0100
Subject: [pypy-dev] clarification cffi vs _ffi_ vs _ffi vs libffi vs
 _rawffi vs rffi
In-Reply-To: <CAMSv6X0PRBECCpmNhPSZSOL4S_oak5h+B92Mrk1VDT8Vs5PV6A@mail.gmail.com>
References: <CAGGBzXKxf+3UnuLjBPkuLWxNv1w4YFuPS=X7wuH3Tjq9yVZkwA@mail.gmail.com>
 <CAMSv6X0PRBECCpmNhPSZSOL4S_oak5h+B92Mrk1VDT8Vs5PV6A@mail.gmail.com>
Message-ID: <CAMSv6X2mqDhVVLGeDNGe5rDmr7wTFEgT8Ggc-ZqcBihqm+oc3A@mail.gmail.com>

Re-Hi,

On Wed, Nov 27, 2013 at 5:24 PM, Armin Rigo <arigo at tunes.org> wrote:
>> http://doc.pypy.org/en/latest/extending.html

I updated that page.


Armin

From arigo at tunes.org  Wed Nov 27 18:23:24 2013
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 27 Nov 2013 18:23:24 +0100
Subject: [pypy-dev] PyPy 2.2.1
Message-ID: <CAMSv6X0khAS0m=XVK1Xi66jRBwG1dtLcXkPWwx-1kO6eOQmuNA@mail.gmail.com>

Hi all !

Here is finally PyPy 2.2.1 (including on non-Windows platforms, as it
includes a few general fixes too).

=======================================
PyPy 2.2.1 - Incrementalism.1
=======================================

We're pleased to announce PyPy 2.2.1, which targets version 2.7.3 of
the Python language. This is a bugfix release over 2.2.

You can download the PyPy 2.2.1 release here:

    http://pypy.org/download.html

What is PyPy?
=============

PyPy is a very compliant Python interpreter, almost a drop-in
replacement for CPython 2.7. It's fast (`pypy 2.2 and cpython 2.7.2`_
performance comparison) due to its integrated tracing JIT compiler.

This release supports x86 machines running Linux 32/64, Mac OS X
64, Windows 32, or ARM (ARMv6 or ARMv7, with VFPv3).

Work on the native Windows 64 is still stalling, we would welcome a
volunteer to handle that.

.. _`pypy 2.2 and cpython 2.7.2`: http://speed.pypy.org

Highlights
==========

This is a bugfix release.  The most important bugs fixed are:

* an issue in sockets' reference counting emulation, showing up
  notably when using the ssl module and calling ``makefile()``.

* Tkinter support on Windows.

* If sys.maxunicode==65535 (on Windows and maybe OS/X), the
  json decoder incorrectly decoded surrogate pairs.

* some FreeBSD fixes.

Note that CFFI 0.8.1 was released.  Both versions 0.8 and 0.8.1 are
compatible with both PyPy 2.2 and 2.2.1.

Cheers,
Armin Rigo & everybody

From arigo at tunes.org  Wed Nov 27 18:25:43 2013
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 27 Nov 2013 18:25:43 +0100
Subject: [pypy-dev] PyPy3 release?
Message-ID: <CAMSv6X2Aam+ZFuhsBjbMgFg=E6A6302OGpSZG6TpiajxHLT=EQ@mail.gmail.com>

Hi all,

Any volunteer to handle the PyPy3 release corresponding to PyPy 2.2?


A bient?t,

Armin.

From chrish at pianocktail.org  Fri Nov 29 02:42:28 2013
From: chrish at pianocktail.org (Christian Hudon)
Date: Thu, 28 Nov 2013 20:42:28 -0500
Subject: [pypy-dev] can we get stdlib-2.7.5 merged?
In-Reply-To: <527FEB90.8010507@gmail.com>
References: <527FEB90.8010507@gmail.com>
Message-ID: <5297F104.8050109@pianocktail.org>

Le 2013-11-10 15:24, Matti Picus a ?crit :
> There are only a handful (ok - 10, two others are from the nightly 
> default) of failures on the stdlib-2.7.5 branch:
> http://buildbot.pypy.org/summary?branch=stdlib-2.7.5
> It seems the major one is handling eintr signal on file reading, which 
> AFAICT would fix more or less 3 of the failures. The others seem to be 
> unrelated to one-another. Would anyone like to take on some of this work?

I'm working (really slowly) on fixing the remaining issues in the 
stdlib-2.7.5 branch. (I think it's a reasonably good task for a beginner 
like me.) I didn't want to pipe in for the 2.2 release, as I'm the wrong 
person to do this kind of work on PyPy quickly (or any kind of work for 
that matter quickly for PyPy). With a bit of luck, I should have it done 
for the 2.3 release, though.

The thing that would be really helpful to me is if someone could take a 
look at the Buildbot for the stdlib-2.7.5 branch and have it rebuild 
automatically on a commit on the branch. I think this used to work when 
the branch was called stdlib-2.7.4. My guess is that the branch was 
renamed but the buildbot's config wasn't updated to track the new 
branch. Could someone who has access to the buildbot's config check this?

Thanks,

   Christian


From fijall at gmail.com  Fri Nov 29 07:18:00 2013
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Fri, 29 Nov 2013 08:18:00 +0200
Subject: [pypy-dev] can we get stdlib-2.7.5 merged?
In-Reply-To: <5297F104.8050109@pianocktail.org>
References: <527FEB90.8010507@gmail.com> <5297F104.8050109@pianocktail.org>
Message-ID: <CAK5idxSeBLRh3U74eC7FSGpSupyy=SJ9cW=mKBM+w+JKfmrxRA@mail.gmail.com>

On Fri, Nov 29, 2013 at 3:42 AM, Christian Hudon <chrish at pianocktail.org> wrote:
> Le 2013-11-10 15:24, Matti Picus a ?crit :
>
>> There are only a handful (ok - 10, two others are from the nightly
>> default) of failures on the stdlib-2.7.5 branch:
>> http://buildbot.pypy.org/summary?branch=stdlib-2.7.5
>> It seems the major one is handling eintr signal on file reading, which
>> AFAICT would fix more or less 3 of the failures. The others seem to be
>> unrelated to one-another. Would anyone like to take on some of this work?
>
>
> I'm working (really slowly) on fixing the remaining issues in the
> stdlib-2.7.5 branch. (I think it's a reasonably good task for a beginner
> like me.) I didn't want to pipe in for the 2.2 release, as I'm the wrong
> person to do this kind of work on PyPy quickly (or any kind of work for that
> matter quickly for PyPy). With a bit of luck, I should have it done for the
> 2.3 release, though.
>
> The thing that would be really helpful to me is if someone could take a look
> at the Buildbot for the stdlib-2.7.5 branch and have it rebuild
> automatically on a commit on the branch. I think this used to work when the
> branch was called stdlib-2.7.4. My guess is that the branch was renamed but
> the buildbot's config wasn't updated to track the new branch. Could someone
> who has access to the buildbot's config check this?
>
> Thanks,
>
>   Christian

Hi Christian

Thanks for tackling this! The buildbot does not automatically build on
a commit (it takes too long and there are too many commits). Instead,
you can click by hand and type in branch name in the field. This is an
example for benchmarks:
http://buildbot.pypy.org/builders/jit-benchmark-linux-x86-64 but you
can pick any builder

From romain.py at gmail.com  Fri Nov 29 18:17:20 2013
From: romain.py at gmail.com (Romain Guillebert)
Date: Fri, 29 Nov 2013 18:17:20 +0100
Subject: [pypy-dev] Proposal for sprint in Tallinn, Estonia
In-Reply-To: <12834110.ecgp6i7Rsd@karoliine>
References: <12834110.ecgp6i7Rsd@karoliine>
Message-ID: <20131129171720.GA27004@Plop>

Hi Ahti

I'm interested in going if this sprint happens, I'd be able to go after
February 2nd (as long as it's not during PyCon).

Cheers
Romain

On 11/26, Ahti Heinla wrote:
> Hi,
> 
> 	I am new to PyPy, but very impressed with what you guys have done.
> Myself I am best known for having been a founding engineer and Chief Technical
> Architect for Skype. Python is my favourite language, and I want to help PyPy
> somehow.
> 
> 	How about I organise a sprint in Tallinn, Estonia (where I live)? I can hopefully
> drum up some interest among local developers, perhaps also get some top
> ex-Skype engineers to join. The requirement to contribute a week full-time is a
> deterrent for many though, so I am not sure yet how many people I can get.
> 
> 	Myself I have no background in interpreters/compilers, but have written
> code for 32 years, often optimised lowlevel code (assembly, C++). I would
> need some hand-holding to get started, but I am sure I'd be productive in
> a day or two.
> 
> Ahti
> ahtih on IRC
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev

From rami.chowdhury at gmail.com  Fri Nov 29 20:20:21 2013
From: rami.chowdhury at gmail.com (Rami Chowdhury)
Date: Fri, 29 Nov 2013 19:20:21 +0000
Subject: [pypy-dev] PyPy3 release?
In-Reply-To: <CAMSv6X2Aam+ZFuhsBjbMgFg=E6A6302OGpSZG6TpiajxHLT=EQ@mail.gmail.com>
References: <CAMSv6X2Aam+ZFuhsBjbMgFg=E6A6302OGpSZG6TpiajxHLT=EQ@mail.gmail.com>
Message-ID: <CAGEKe-3=XXf7KBJmvC88bFSYRsQjK+E=k8=K=2YTY7YU3MiY4Q@mail.gmail.com>

Hi Armin,

What would this involve? I ask not only for my benefit but also for
any other newbies that may be lurking here and thinking about getting
involved :-)

Thanks
Rami

On Wed, Nov 27, 2013 at 5:25 PM, Armin Rigo <arigo at tunes.org> wrote:
> Hi all,
>
> Any volunteer to handle the PyPy3 release corresponding to PyPy 2.2?
>
>
> A bient?t,
>
> Armin.
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev



-- 
Rami Chowdhury
"A mind all logic is like a knife all blade - it makes the hand bleed
that uses it." -- Rabindranath Tagore
+44-7581-430-517 / +1-408-597-7068 / +88-01771-064063

From arigo at tunes.org  Sat Nov 30 09:53:11 2013
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 30 Nov 2013 09:53:11 +0100
Subject: [pypy-dev] Leysin sprint (11-19 January 2014)
Message-ID: <CAMSv6X1cO3KBrHnY-QO-6zjx+0dqAX8y+DiMXEsCp4Msr3Dhcg@mail.gmail.com>

=====================================================================
         PyPy Leysin Winter Sprint (11-19st January 2014)
=====================================================================

The next PyPy sprint will be in Leysin, Switzerland, for the ninth time.
This is a fully public sprint: newcomers and topics other than those
proposed below are welcome.

------------------------------
Goals and topics of the sprint
------------------------------

* Py3k: work towards supporting Python 3 in PyPy

* NumPyPy: work towards supporting the numpy module in PyPy

* STM: work towards supporting Software Transactional Memory

* And as usual, the main side goal is to have fun in winter sports :-)
  We can take a day off for ski.

-----------
Exact times
-----------

For a change, and as an attempt to simplify things, I specified the
dates as 11-19 January 2014, where 11 and 19 are travel days.  We will
work full days between the 12 and the 18.  You are of course allowed to
show up for a part of that time only, too.

-----------------------
Location & Accomodation
-----------------------

Leysin, Switzerland, "same place as before".  Let me refresh your
memory: both the sprint venue and the lodging will be in a very spacious
pair of chalets built specifically for bed & breakfast:
http://www.ermina.ch/.  The place has a good ADSL Internet connexion
with wireless installed.  You can of course arrange your own lodging
anywhere (as long as you are in Leysin, you cannot be more than a 15
minutes walk away from the sprint venue), but I definitely recommend
lodging there too -- you won't find a better view anywhere else (though
you probably won't get much worse ones easily, either :-)

Please *confirm* that you are coming so that we can adjust the
reservations as appropriate.  The rate so far has been around 60 CHF a
night all included in 2-person rooms, with breakfast.  There are larger
rooms too (less expensive per person) and maybe the possibility to get a
single room if you really want to.

Please register by Mercurial::

  https://bitbucket.org/pypy/extradoc/
  https://bitbucket.org/pypy/extradoc/raw/extradoc/sprintinfo/leysin-winter-2014

or on the pypy-dev mailing list if you do not yet have check-in rights:

  http://mail.python.org/mailman/listinfo/pypy-dev

You need a Swiss-to-(insert country here) power adapter.  There will be
some Swiss-to-EU adapters around -- bring a EU-format power strip if you
have one.

Armin Rigo

From arigo at tunes.org  Sat Nov 30 10:02:08 2013
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 30 Nov 2013 10:02:08 +0100
Subject: [pypy-dev] PyPy3 release?
In-Reply-To: <CAGEKe-3=XXf7KBJmvC88bFSYRsQjK+E=k8=K=2YTY7YU3MiY4Q@mail.gmail.com>
References: <CAMSv6X2Aam+ZFuhsBjbMgFg=E6A6302OGpSZG6TpiajxHLT=EQ@mail.gmail.com>
 <CAGEKe-3=XXf7KBJmvC88bFSYRsQjK+E=k8=K=2YTY7YU3MiY4Q@mail.gmail.com>
Message-ID: <CAMSv6X3djv0Q4LJqw9GgLfSZavqSRKD4oaKYYs8+2YmRsqOs0A@mail.gmail.com>

Hi Rami,

On Fri, Nov 29, 2013 at 8:20 PM, Rami Chowdhury
<rami.chowdhury at gmail.com> wrote:
> What would this involve? I ask not only for my benefit but also for
> any other newbies that may be lurking here and thinking about getting
> involved :-)

http://doc.pypy.org/en/latest/how-to-release.html


A bient?t,

Armin.

From matti.picus at gmail.com  Sat Nov 30 20:18:23 2013
From: matti.picus at gmail.com (Matti Picus)
Date: Sat, 30 Nov 2013 21:18:23 +0200
Subject: [pypy-dev] Proposal for sprint in Tallinn, Estonia
In-Reply-To: <20131129171720.GA27004@Plop>
References: <12834110.ecgp6i7Rsd@karoliine> <20131129171720.GA27004@Plop>
Message-ID: <529A39FF.6090807@gmail.com>

I would seriously consider Feb in Tallinn.
Matti

On 29/11/2013 7:17 PM, Romain Guillebert wrote:
> Hi Ahti
>
> I'm interested in going if this sprint happens, I'd be able to go after
> February 2nd (as long as it's not during PyCon).
>
> Cheers
> Romain
>
> On 11/26, Ahti Heinla wrote:
>> Hi,
>>
>> 	I am new to PyPy, but very impressed with what you guys have done.
>> Myself I am best known for having been a founding engineer and Chief Technical
>> Architect for Skype. Python is my favourite language, and I want to help PyPy
>> somehow.
>>
>> 	How about I organise a sprint in Tallinn, Estonia (where I live)? I can hopefully
>> drum up some interest among local developers, perhaps also get some top
>> ex-Skype engineers to join. The requirement to contribute a week full-time is a
>> deterrent for many though, so I am not sure yet how many people I can get.
>>
>> 	Myself I have no background in interpreters/compilers, but have written
>> code for 32 years, often optimised lowlevel code (assembly, C++). I would
>> need some hand-holding to get started, but I am sure I'd be productive in
>> a day or two.
>>
>> Ahti
>> ahtih on IRC
>> _______________________________________________
>> pypy-dev mailing list
>> pypy-dev at python.org
>> https://mail.python.org/mailman/listinfo/pypy-dev
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev