From tuom.larsen at gmail.com Tue Oct 6 19:32:00 2015 From: tuom.larsen at gmail.com (Tuom Larsen) Date: Tue, 6 Oct 2015 19:32:00 +0200 Subject: [pypy-dev] Size limit Message-ID: Hello! I have a piece of auto-generated code and I noticed that if a function has more than certain number of expressions it starts to run much slower. I'm guessing this has something to do with what JIT considers to be still worth of compiling but I just would like to confirm this: is there any limit on how big (in term of operations) a function needs to be in order to get compiled by JIT? Is it possible to increase the limit from user code? Or is there any other reason for the slowdown? Please find below an arficial example. Here it turn in ~0.13 seconds but when I uncomment the first line it drops to 5.3 seconds. Thanks, Tuom Larsen def f(x): a0 = x*0.1; a1 = x*0.1; a2 = x*0.2; a3 = x*0.3; a4 = x*0.4 a5 = x*0.5; a6 = x*0.6; a7 = x*0.7; a8 = x*0.8; a9 = x*0.9 b0 = x*0.1; b1 = x*0.1; b2 = x*0.2; b3 = x*0.3; b4 = x*0.4 b5 = x*0.5; b6 = x*0.6; b7 = x*0.7; b8 = x*0.8; b9 = x*0.9 return sum([ # (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9)]) for i in range(100000): f(0.1) From fijall at gmail.com Wed Oct 7 11:26:41 2015 From: fijall at gmail.com (Maciej Fijalkowski) Date: Wed, 7 Oct 2015 11:26:41 +0200 Subject: [pypy-dev] Size limit In-Reply-To: References: Message-ID: Hi Tuom. It's the limit of number of operations while compiling. It can be (globally) edited using: pypy --jit trace_limit=some-number default is 6000. --jit help will show you some information. We should improve the heuristic to always trace a function that's the only one maybe though On Tue, Oct 6, 2015 at 7:32 PM, Tuom Larsen wrote: > Hello! > > I have a piece of auto-generated code and I noticed that if a function > has more than certain number of expressions it starts to run much > slower. > > I'm guessing this has something to do with what JIT considers to be > still worth of compiling but I just would like to confirm this: is > there any limit on how big (in term of operations) a function needs to > be in order to get compiled by JIT? Is it possible to increase the > limit from user code? Or is there any other reason for the slowdown? > > Please find below an arficial example. Here it turn in ~0.13 seconds > but when I uncomment the first line it drops to 5.3 seconds. > > Thanks, > > Tuom Larsen > > def f(x): > a0 = x*0.1; a1 = x*0.1; a2 = x*0.2; a3 = x*0.3; a4 = x*0.4 > a5 = x*0.5; a6 = x*0.6; a7 = x*0.7; a8 = x*0.8; a9 = x*0.9 > b0 = x*0.1; b1 = x*0.1; b2 = x*0.2; b3 = x*0.3; b4 = x*0.4 > b5 = x*0.5; b6 = x*0.6; b7 = x*0.7; b8 = x*0.8; b9 = x*0.9 > return sum([ > # (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), > (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), > (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), > (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), > (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), > (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), > (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), > (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), > (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), > (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), > (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), > (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), > (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), > (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), > (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), > (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), > (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), > (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), > (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), > (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), > (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), > (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), > (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), > (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), > (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), > (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), > (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), > (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), > (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), > (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), > (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), > (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), > (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), > (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), > (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), > (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), > (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), > (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), > (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), > (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), > (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), > (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9)]) > > for i in range(100000): > f(0.1) > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev From ofirshw at gmail.com Wed Oct 7 10:51:29 2015 From: ofirshw at gmail.com (Ofir Shwartz) Date: Wed, 7 Oct 2015 11:51:29 +0300 Subject: [pypy-dev] Working version of pypy3 that support Python 3.3 Message-ID: Is there any update on that matter? I tried it now and got the same results for the most updated jit and no-jit pypy3.3. Obviously, pypy 3.2 runs perfectly good. Please assist. Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From amauryfa at gmail.com Wed Oct 7 11:51:32 2015 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Wed, 7 Oct 2015 11:51:32 +0200 Subject: [pypy-dev] Working version of pypy3 that support Python 3.3 In-Reply-To: References: Message-ID: 2015-10-07 10:51 GMT+02:00 Ofir Shwartz : > Is there any update on that matter? > I tried it now and got the same results for the most updated jit and > no-jit pypy3.3. > Obviously, pypy 3.2 runs perfectly good. > Please assist. > There are still many features missing from our pypy3.3. Help needed! You can track the progress here: http://buildbot.pypy.org/summary?branch=py3.3 -- Amaury Forgeot d'Arc -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuom.larsen at gmail.com Wed Oct 7 12:17:51 2015 From: tuom.larsen at gmail.com (Tuom Larsen) Date: Wed, 7 Oct 2015 12:17:51 +0200 Subject: [pypy-dev] Size limit In-Reply-To: References: Message-ID: Thanks, even trace_limit=9000 is sufficient in my case! That heuristic would be most awesome so that other people don't have to manually override the limit. I'm guessing it is more complicated than that but simplify lifting the limit to, say, 12000 wouldn't suffice? On Wed, Oct 7, 2015 at 11:26 AM, Maciej Fijalkowski wrote: > Hi Tuom. It's the limit of number of operations while compiling. It > can be (globally) edited using: > > pypy --jit trace_limit=some-number > > default is 6000. --jit help will show you some information. We should > improve the heuristic to always trace a function that's the only one > maybe though > > On Tue, Oct 6, 2015 at 7:32 PM, Tuom Larsen wrote: >> Hello! >> >> I have a piece of auto-generated code and I noticed that if a function >> has more than certain number of expressions it starts to run much >> slower. >> >> I'm guessing this has something to do with what JIT considers to be >> still worth of compiling but I just would like to confirm this: is >> there any limit on how big (in term of operations) a function needs to >> be in order to get compiled by JIT? Is it possible to increase the >> limit from user code? Or is there any other reason for the slowdown? >> >> Please find below an arficial example. Here it turn in ~0.13 seconds >> but when I uncomment the first line it drops to 5.3 seconds. >> >> Thanks, >> >> Tuom Larsen >> >> def f(x): >> a0 = x*0.1; a1 = x*0.1; a2 = x*0.2; a3 = x*0.3; a4 = x*0.4 >> a5 = x*0.5; a6 = x*0.6; a7 = x*0.7; a8 = x*0.8; a9 = x*0.9 >> b0 = x*0.1; b1 = x*0.1; b2 = x*0.2; b3 = x*0.3; b4 = x*0.4 >> b5 = x*0.5; b6 = x*0.6; b7 = x*0.7; b8 = x*0.8; b9 = x*0.9 >> return sum([ >> # (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9)]) >> >> for i in range(100000): >> f(0.1) >> _______________________________________________ >> pypy-dev mailing list >> pypy-dev at python.org >> https://mail.python.org/mailman/listinfo/pypy-dev From fijall at gmail.com Wed Oct 7 12:44:16 2015 From: fijall at gmail.com (Maciej Fijalkowski) Date: Wed, 7 Oct 2015 12:44:16 +0200 Subject: [pypy-dev] Size limit In-Reply-To: References: Message-ID: we found quite a few problems with the current heuristic so we'll put some effort into it some time soon. On Wed, Oct 7, 2015 at 12:17 PM, Tuom Larsen wrote: > Thanks, even trace_limit=9000 is sufficient in my case! > > That heuristic would be most awesome so that other people don't have > to manually override the limit. I'm guessing it is more complicated > than that but simplify lifting the limit to, say, 12000 wouldn't > suffice? > > On Wed, Oct 7, 2015 at 11:26 AM, Maciej Fijalkowski wrote: >> Hi Tuom. It's the limit of number of operations while compiling. It >> can be (globally) edited using: >> >> pypy --jit trace_limit=some-number >> >> default is 6000. --jit help will show you some information. We should >> improve the heuristic to always trace a function that's the only one >> maybe though >> >> On Tue, Oct 6, 2015 at 7:32 PM, Tuom Larsen wrote: >>> Hello! >>> >>> I have a piece of auto-generated code and I noticed that if a function >>> has more than certain number of expressions it starts to run much >>> slower. >>> >>> I'm guessing this has something to do with what JIT considers to be >>> still worth of compiling but I just would like to confirm this: is >>> there any limit on how big (in term of operations) a function needs to >>> be in order to get compiled by JIT? Is it possible to increase the >>> limit from user code? Or is there any other reason for the slowdown? >>> >>> Please find below an arficial example. Here it turn in ~0.13 seconds >>> but when I uncomment the first line it drops to 5.3 seconds. >>> >>> Thanks, >>> >>> Tuom Larsen >>> >>> def f(x): >>> a0 = x*0.1; a1 = x*0.1; a2 = x*0.2; a3 = x*0.3; a4 = x*0.4 >>> a5 = x*0.5; a6 = x*0.6; a7 = x*0.7; a8 = x*0.8; a9 = x*0.9 >>> b0 = x*0.1; b1 = x*0.1; b2 = x*0.2; b3 = x*0.3; b4 = x*0.4 >>> b5 = x*0.5; b6 = x*0.6; b7 = x*0.7; b8 = x*0.8; b9 = x*0.9 >>> return sum([ >>> # (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >>> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >>> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >>> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >>> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >>> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >>> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >>> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >>> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >>> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >>> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >>> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >>> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >>> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >>> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >>> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >>> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >>> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >>> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >>> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >>> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >>> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >>> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >>> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >>> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >>> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >>> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >>> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >>> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >>> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >>> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >>> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >>> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >>> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >>> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >>> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >>> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >>> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >>> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >>> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >>> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9), >>> (a0+a1+a2+a3+a4+a5+a6+a7+a8+a9)*(b0+b1+b2+b3+b4+b5+b6+b7+b8+b9)]) >>> >>> for i in range(100000): >>> f(0.1) >>> _______________________________________________ >>> pypy-dev mailing list >>> pypy-dev at python.org >>> https://mail.python.org/mailman/listinfo/pypy-dev From arigo at tunes.org Wed Oct 7 13:26:28 2015 From: arigo at tunes.org (Armin Rigo) Date: Wed, 7 Oct 2015 13:26:28 +0200 Subject: [pypy-dev] Working version of pypy3 that support Python 3.3 In-Reply-To: References: Message-ID: Hi Ofir, On Wed, Oct 7, 2015 at 10:51 AM, Ofir Shwartz wrote: > Is there any update on that matter? > I tried it now and got the same results for the most updated jit and no-jit > pypy3.3. I looked into it a bit more. (I think it's sad that no-one had a look since June, but I suppose it just shows that our pypy3 involvement is rather thin.) The problem is not that the archive is incomplete; it is complete as far as I can see. The problem is that the binary only works if it runs inside a directory with exactly the same name as it was compiled with. The crash I get is in _getfilesystemencoding, called from pypy/module/sys/__init__py:startup(). It is indeed extremely early, before we have got any chance to set up sys.path correctly. I tried to fix the problem but gave up after hitting several nested issues. The failing check is pushed in 88a921f5ae6b; somebody please fix it :-) A bient?t, Armin. From ofirshw at gmail.com Wed Oct 7 13:35:45 2015 From: ofirshw at gmail.com (Ofir Shwartz) Date: Wed, 7 Oct 2015 14:35:45 +0300 Subject: [pypy-dev] Working version of pypy3 that support Python 3.3 In-Reply-To: References: Message-ID: Thank you for having a look. It's really sad that such a small involvement is put in such a terrific tool (I got great speed-ups compared to python3). Just to be precise, my report is based on pypy-c-jit-79909-4fa19970ddeb-linux64 (currently the latest), and the exact error is: ofir at psl-12core-server:~/pypy-c-jit-79909-4fa19970ddeb-linux64/bin$ ./pypy3.3 debug: OperationError: debug: operror-type: ImportError debug: operror-value: No module named 'encodings' debug: OperationError: debug: operror-type: AttributeError debug: operror-value: stdout (both for pypy3 and pypy3.3 executables) Thanks, Ofir On 7 October 2015 at 14:26, Armin Rigo wrote: > Hi Ofir, > > On Wed, Oct 7, 2015 at 10:51 AM, Ofir Shwartz wrote: > > Is there any update on that matter? > > I tried it now and got the same results for the most updated jit and > no-jit > > pypy3.3. > > I looked into it a bit more. (I think it's sad that no-one had a look > since June, but I suppose it just shows that our pypy3 involvement is > rather thin.) The problem is not that the archive is incomplete; it > is complete as far as I can see. The problem is that the binary only > works if it runs inside a directory with exactly the same name as it > was compiled with. > > The crash I get is in _getfilesystemencoding, called from > pypy/module/sys/__init__py:startup(). It is indeed extremely early, > before we have got any chance to set up sys.path correctly. I tried > to fix the problem but gave up after hitting several nested issues. > The failing check is pushed in 88a921f5ae6b; somebody please fix it > :-) > > > A bient?t, > > Armin. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuom.larsen at gmail.com Wed Oct 7 14:14:23 2015 From: tuom.larsen at gmail.com (Tuom Larsen) Date: Wed, 7 Oct 2015 14:14:23 +0200 Subject: [pypy-dev] Crash in JIT Message-ID: Hello! I have a fairly long code here so I have yet to distill a short test case but I started to get a crash in 2.6.0 (I never saw this kind of error in earlier versions). Is there anything I can do to give you more helpful information besides the following error message? RPython traceback: File "pypy_module_pypyjit_interp_jit.c", line 149, in jump_absolute__AccessDirect_None File "rpython_jit_metainterp_warmstate.c", line 7443, in maybe_compile_and_run__star_5_1 File "rpython_jit_metainterp_warmstate.c", line 15882, in bound_reached__star_5_1 File "rpython_jit_metainterp_pyjitpl.c", line 7116, in compile_and_run_once___rpython_jit_metainterp_ji_5 File "rpython_jit_metainterp_pyjitpl.c", line 4903, in MetaInterp__compile_and_run_once File "rpython_jit_metainterp_pyjitpl.c", line 8771, in MetaInterp_interpret File "rpython_jit_metainterp_pyjitpl.c", line 13684, in MetaInterp__interpret File "rpython_jit_metainterp_pyjitpl.c", line 21400, in MIFrame_run_one_step File "rpython_jit_metainterp_pyjitpl_1.c", line 64946, in MIFrame_opimpl_jit_merge_point File "rpython_jit_metainterp_pyjitpl_2.c", line 50109, in MetaInterp_reached_loop_header File "rpython_jit_metainterp_pyjitpl_3.c", line 22634, in MetaInterp_compile_loop File "rpython_jit_metainterp_compile.c", line 12811, in compile_loop File "rpython_jit_metainterp_optimizeopt___init__.c", line 428, in optimize_trace File "rpython_jit_metainterp_optimizeopt_unroll.c", line 2607, in UnrollOptimizer_propagate_all_forward File "rpython_jit_metainterp_optimizeopt_unroll.c", line 21014, in UnrollOptimizer_close_loop File "rpython_jit_metainterp_optimizeopt_unroll.c", line 28710, in UnrollOptimizer_import_box File "rpython_jit_metainterp_optimizeopt_unroll.c", line 31670, in UnrollOptimizer_add_op_to_short ~~~ Crash in JIT! and `print sys.version`: 2.7.9 (295ee98b6928, May 31 2015, 07:28:49) [PyPy 2.6.0 with GCC 4.2.1 Compatible Apple LLVM 5.1 (clang-503.0.40)] From fijall at gmail.com Wed Oct 7 17:26:45 2015 From: fijall at gmail.com (Maciej Fijalkowski) Date: Wed, 7 Oct 2015 17:26:45 +0200 Subject: [pypy-dev] Crash in JIT In-Reply-To: References: Message-ID: Hi Tuom. This sort of error should be fixed on default (so post 2.6.1 release), any chance you can try the nightly? On Wed, Oct 7, 2015 at 2:14 PM, Tuom Larsen wrote: > Hello! > > I have a fairly long code here so I have yet to distill a short test > case but I started to get a crash in 2.6.0 (I never saw this kind of > error in earlier versions). Is there anything I can do to give you > more helpful information besides the following error message? > > RPython traceback: > File "pypy_module_pypyjit_interp_jit.c", line 149, in > jump_absolute__AccessDirect_None > File "rpython_jit_metainterp_warmstate.c", line 7443, in > maybe_compile_and_run__star_5_1 > File "rpython_jit_metainterp_warmstate.c", line 15882, in > bound_reached__star_5_1 > File "rpython_jit_metainterp_pyjitpl.c", line 7116, in > compile_and_run_once___rpython_jit_metainterp_ji_5 > File "rpython_jit_metainterp_pyjitpl.c", line 4903, in > MetaInterp__compile_and_run_once > File "rpython_jit_metainterp_pyjitpl.c", line 8771, in MetaInterp_interpret > File "rpython_jit_metainterp_pyjitpl.c", line 13684, in MetaInterp__interpret > File "rpython_jit_metainterp_pyjitpl.c", line 21400, in MIFrame_run_one_step > File "rpython_jit_metainterp_pyjitpl_1.c", line 64946, in > MIFrame_opimpl_jit_merge_point > File "rpython_jit_metainterp_pyjitpl_2.c", line 50109, in > MetaInterp_reached_loop_header > File "rpython_jit_metainterp_pyjitpl_3.c", line 22634, in > MetaInterp_compile_loop > File "rpython_jit_metainterp_compile.c", line 12811, in compile_loop > File "rpython_jit_metainterp_optimizeopt___init__.c", line 428, in > optimize_trace > File "rpython_jit_metainterp_optimizeopt_unroll.c", line 2607, in > UnrollOptimizer_propagate_all_forward > File "rpython_jit_metainterp_optimizeopt_unroll.c", line 21014, in > UnrollOptimizer_close_loop > File "rpython_jit_metainterp_optimizeopt_unroll.c", line 28710, in > UnrollOptimizer_import_box > File "rpython_jit_metainterp_optimizeopt_unroll.c", line 31670, in > UnrollOptimizer_add_op_to_short > ~~~ Crash in JIT! > > and `print sys.version`: > 2.7.9 (295ee98b6928, May 31 2015, 07:28:49) > [PyPy 2.6.0 with GCC 4.2.1 Compatible Apple LLVM 5.1 (clang-503.0.40)] > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev From tuom.larsen at gmail.com Wed Oct 7 18:59:50 2015 From: tuom.larsen at gmail.com (Tuom Larsen) Date: Wed, 7 Oct 2015 18:59:50 +0200 Subject: [pypy-dev] Crash in JIT In-Reply-To: References: Message-ID: I tried it with the latest nightly build and it never crashed. So I think it is safe to consider it fixed. Thank you! On Wed, Oct 7, 2015 at 5:26 PM, Maciej Fijalkowski wrote: > Hi Tuom. > > This sort of error should be fixed on default (so post 2.6.1 release), > any chance you can try the nightly? > > On Wed, Oct 7, 2015 at 2:14 PM, Tuom Larsen wrote: >> Hello! >> >> I have a fairly long code here so I have yet to distill a short test >> case but I started to get a crash in 2.6.0 (I never saw this kind of >> error in earlier versions). Is there anything I can do to give you >> more helpful information besides the following error message? >> >> RPython traceback: >> File "pypy_module_pypyjit_interp_jit.c", line 149, in >> jump_absolute__AccessDirect_None >> File "rpython_jit_metainterp_warmstate.c", line 7443, in >> maybe_compile_and_run__star_5_1 >> File "rpython_jit_metainterp_warmstate.c", line 15882, in >> bound_reached__star_5_1 >> File "rpython_jit_metainterp_pyjitpl.c", line 7116, in >> compile_and_run_once___rpython_jit_metainterp_ji_5 >> File "rpython_jit_metainterp_pyjitpl.c", line 4903, in >> MetaInterp__compile_and_run_once >> File "rpython_jit_metainterp_pyjitpl.c", line 8771, in MetaInterp_interpret >> File "rpython_jit_metainterp_pyjitpl.c", line 13684, in MetaInterp__interpret >> File "rpython_jit_metainterp_pyjitpl.c", line 21400, in MIFrame_run_one_step >> File "rpython_jit_metainterp_pyjitpl_1.c", line 64946, in >> MIFrame_opimpl_jit_merge_point >> File "rpython_jit_metainterp_pyjitpl_2.c", line 50109, in >> MetaInterp_reached_loop_header >> File "rpython_jit_metainterp_pyjitpl_3.c", line 22634, in >> MetaInterp_compile_loop >> File "rpython_jit_metainterp_compile.c", line 12811, in compile_loop >> File "rpython_jit_metainterp_optimizeopt___init__.c", line 428, in >> optimize_trace >> File "rpython_jit_metainterp_optimizeopt_unroll.c", line 2607, in >> UnrollOptimizer_propagate_all_forward >> File "rpython_jit_metainterp_optimizeopt_unroll.c", line 21014, in >> UnrollOptimizer_close_loop >> File "rpython_jit_metainterp_optimizeopt_unroll.c", line 28710, in >> UnrollOptimizer_import_box >> File "rpython_jit_metainterp_optimizeopt_unroll.c", line 31670, in >> UnrollOptimizer_add_op_to_short >> ~~~ Crash in JIT! >> >> and `print sys.version`: >> 2.7.9 (295ee98b6928, May 31 2015, 07:28:49) >> [PyPy 2.6.0 with GCC 4.2.1 Compatible Apple LLVM 5.1 (clang-503.0.40)] >> _______________________________________________ >> pypy-dev mailing list >> pypy-dev at python.org >> https://mail.python.org/mailman/listinfo/pypy-dev From tuom.larsen at gmail.com Wed Oct 7 19:01:11 2015 From: tuom.larsen at gmail.com (Tuom Larsen) Date: Wed, 7 Oct 2015 19:01:11 +0200 Subject: [pypy-dev] Copy of list In-Reply-To: References: Message-ID: So I just tried it with the latest nightly build and `[:]` is now even a bit faster than `list()`! Thank you once more! On Tue, Sep 29, 2015 at 12:27 PM, Tuom Larsen wrote: > Hi Armin, > > thanks a lot, both for the explanation and the fix! I will try it soon. > > Have a nice day! > > Tuom > > PS: The speed difference came from larger piece of code, which I tried > to reproduce in "minimal viable test case". Hence that `timeit`, where > it showed up as well. But in any case, thanks a lot once more! > > > On Tue, Sep 29, 2015 at 9:25 AM, Armin Rigo wrote: >> Hi Tuom, >> >> On Tue, Sep 29, 2015 at 7:31 AM, Tuom Larsen wrote: >>> Please, let me rephrase my question: currently I use `[:]` because it >>> is faster in CPython (0.131 usec vs 0.269 usec per loop). I almost >>> don't mind changing it to `list()` because of PyPy but I was wondering >>> what do PyPy developers recommend. I don't understand why is `[:]` >>> twice as slow as `list()` as it seems it should do the same thing >>> (create a list and copy the content). >> >> Looking at the jit logs, it is tripped by a RPython function with a >> loop in its slow-path. Fixed in 4e688540cfe9. >> >> There is still a bit of overhead. For example, lst[:] is equivalent >> to lst[0:9223372036854775807]. The general logic looks like this: >> when doing lst[a:b], if b > len(lst) then replace b with len(lst). >> This means here a check if 9223372036854775807 > len(lst)... It is >> not possible that the length of a list be that huge, but this >> knowledge is not codified explicitly. >> >> Yes, we could improve that in the future. >> But this is really advanced details. You should write 'list()' or >> '[:]' as you feel more natural, or maybe as benefits the speed of >> CPython if it makes an important difference there. Using 'timeit' to >> measure microbenchmarks in PyPy may or may not give a useful result. >> In this case it did only after you stopped using range() and only >> because we don't have more advanced optimizations that realize that >> the resulting list is not needed at all. In general, you should not >> rely on it. >> >> What you should do instead is measure how much time is spent in some >> real loop of your algorithm, and compare it with variants. (Make sure >> every variant is run in its own process, otherwise the JITting of >> similar pieces of code might interfere in unexpected ways.) If you're >> lucky you may be able to find a variant that is overall much faster. >> If you're not, it means that what you're changing is not relevant for >> performance. >> >> >> A bient?t, >> >> Armin. From amauryfa at gmail.com Wed Oct 7 23:25:16 2015 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Wed, 7 Oct 2015 23:25:16 +0200 Subject: [pypy-dev] Working version of pypy3 that support Python 3.3 In-Reply-To: References: Message-ID: 2015-10-07 13:26 GMT+02:00 Armin Rigo : > Hi Ofir, > > On Wed, Oct 7, 2015 at 10:51 AM, Ofir Shwartz wrote: > > Is there any update on that matter? > > I tried it now and got the same results for the most updated jit and > no-jit > > pypy3.3. > > I looked into it a bit more. (I think it's sad that no-one had a look > since June, but I suppose it just shows that our pypy3 involvement is > rather thin.) The problem is not that the archive is incomplete; it > is complete as far as I can see. The problem is that the binary only > works if it runs inside a directory with exactly the same name as it > was compiled with. > > The crash I get is in _getfilesystemencoding, called from > pypy/module/sys/__init__py:startup(). It is indeed extremely early, > before we have got any chance to set up sys.path correctly. I tried > to fix the problem but gave up after hitting several nested issues. > The failing check is pushed in 88a921f5ae6b; somebody please fix it > :-) > Maybe move setup_bootstrap_path() from app_main.py to module/sys/__init__.py? > A bient?t, > > Armin. > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev > -- Amaury Forgeot d'Arc -------------- next part -------------- An HTML attachment was scrubbed... URL: From cherian.rosh at gmail.com Thu Oct 8 18:53:14 2015 From: cherian.rosh at gmail.com (Roshan Cherian) Date: Thu, 8 Oct 2015 09:53:14 -0700 Subject: [pypy-dev] p12 to p7b Message-ID: Hi Team, I have a requirement to convert from p12 to p7b. I am sorry for my little knowledge on this and I am not even sure I can ask this here. I am doing the following: p12 = load_pkcs12(file(self._pkcs12Path, 'rb').read()) pem = dump_certificate(FILETYPE_PEM, p12.get_certificate()) p7 = load_pkcs7_data(FILETYPE_PEM, pem) However I am getting an error: Error: [('asn1 encoding routines', 'ASN1_CHECK_TLEN', 'wrong tag'), ('asn1 encoding routines', 'ASN1_D2I_EX_PRIMITIVE', 'nested asn1 error'), ('asn1 encoding routines', 'ASN1_TEMPLATE_NOEXP_D2I', 'nested asn1 error'), ('PEM routines', 'PEM_ASN1_read_bio', 'ASN1 lib')] I am pretty sure I am doing something wrong, could you help? Thanks, -Roshan -------------- next part -------------- An HTML attachment was scrubbed... URL: From jdoran at lexmachina.com Sun Oct 11 00:28:16 2015 From: jdoran at lexmachina.com (Jeff Doran) Date: Sat, 10 Oct 2015 15:28:16 -0700 Subject: [pypy-dev] Looking for clues to consistent Seg Fault in PyPy 2.6.1 Message-ID: Hi, I've run out of options trying to find a Seg Fault which happens when running lxml under PyPy 2.6.1. This problem only occurs under PyPy as the rest of the code works fine under CPython 2.7. I've been in contact with the lxml dev team and they confirmed my problem, but could not determine where the cause of the Seg Fault lies. They suggested reaching out to you folks for ideas or a possible work around as the stack trace is always in libpypy-c.so and seems to be triggered by eventual garbage collection. Below is the gist of the email I sent to the lxml team earlier. It describes my ability to reproduce this Seg Fault using their supplied test suite. As of this writing, lxml is at version 3.5.0b1. I'd appreciate any feedback you might supply on how to deal with this error. Seg Faults are notoriously hard to find since they inevitably happen upstream for the crash. We are anxious to perform a complete evaluation of transitioning to PyPy as our Python platform of choice and this appears to be the last major blockage. Thanks for your help. - Jeff Doran ---------------- I'm running the lxml test suite under PyPy which I installed from here,https://bitbucket.org/squeaky/portable-pypy/downloads/pypy-2.6.1-linux_x86_64-portable.tar.bz2 . I've downloaded the current lxml src from https://github.com/lxml/lxml. I built lxml using 'python setup.py build --with-cython' *From ..lxml/src/lxml I run: * nosetests -v --pdb --nocapture Comparing with ElementTree 1.3.0 Comparing with cElementTree 1.3.0 TESTED VERSION: 3.5.0.beta1 Python: (major=2, minor=7, micro=10, releaselevel='final', serial=42) lxml.etree: (3, 5, 0, -99) libxml used: (2, 8, 0) libxml compiled: (2, 8, 0) libxslt used: (1, 1, 26) libxslt compiled: (1, 1, 26) lxml.html.tests.test_autolink.test_suite ... ok lxml.html.tests.test_basic.test_suite ... ok test_allow_tags (lxml.html.tests.test_clean.CleanerTest) ... ok test_clean_invalid_root_tag (lxml.html.tests.test_clean.CleanerTest) ... ok test_safe_attrs_excluded (lxml.html.tests.test_clean.CleanerTest) ... ok test_safe_attrs_included (lxml.html.tests.test_clean.CleanerTest) ... ok lxml.html.tests.test_clean.test_suite ... ok lxml.html.tests.test_diff.test_suite ... ok test_body (lxml.html.tests.test_elementsoup.SoupParserTestCase) ... ok test_broken_attribute (lxml.html.tests.test_elementsoup.SoupParserTestCase) ... fish: Job 1, ?nosetests -v --pdb --nocapture? terminated by signal SIGSEGV (Address boundary error) ``` Running these tests under gdb reveals the following partial stack trace: #0 0x00007ffff57fa08a in ?? () from /home/jeff/debian_pypy_env/bin/libpypy-c.so #1 0x00007ffff57fa110 in ?? () from /home/jeff/debian_pypy_env/bin/libpypy-c.so #2 0x00007ffff5776be3 in PyPyWeakref_LockObject () from /home/jeff/debian_pypy_env/bin/libpypy-c.so #3 0x00007ffff0337399 in __pyx_f_4lxml_5etree__isProxyAliveInPypy.isra.265 () from /home/jeff/lxml/src/lxml/etree.pypy-26.so #4 0x00007ffff033fbfb in __pyx_f_4lxml_5etree_attemptDeallocation () from /home/jeff/lxml/src/lxml/etree.pypy-26.so #5 0x00007ffff033fd70 in __pyx_tp_dealloc_4lxml_5etree__Element () from /home/jeff/lxml/src/lxml/etree.pypy-26.so #6 0x00007ffff62186f9 in ?? () from /home/jeff/debian_pypy_env/bin/libpypy-c.so #7 0x00007ffff57aec90 in ?? () from /home/jeff/debian_pypy_env/bin/libpypy-c.so #8 0x00007ffff57fa670 in ?? () from /home/jeff/debian_pypy_env/bin/libpypy-c.so #9 0x00007ffff5fa0da8 in ?? () from /home/jeff/debian_pypy_env/bin/libpypy-c.so #10 0x00007ffff5f89a31 in ?? () -------------- next part -------------- An HTML attachment was scrubbed... URL: From arigo at tunes.org Sun Oct 11 10:04:02 2015 From: arigo at tunes.org (Armin Rigo) Date: Sun, 11 Oct 2015 10:04:02 +0200 Subject: [pypy-dev] Looking for clues to consistent Seg Fault in PyPy 2.6.1 In-Reply-To: References: Message-ID: Hi Jeff, I will have a look, but first let me mention that lxml-cffi is much better supported (and faster) on pypy than lxml. It is unmaintained though, so I guess it is getting out of date with lxml developments. A bient?t, Armin. From lac at openend.se Sun Oct 11 10:59:03 2015 From: lac at openend.se (Laura Creighton) Date: Sun, 11 Oct 2015 10:59:03 +0200 Subject: [pypy-dev] Looking for clues to consistent Seg Fault in PyPy 2.6.1 In-Reply-To: References: Message-ID: <201510110859.t9B8x3dq030165@fido.openend.se> In a message of Sun, 11 Oct 2015 10:04:02 +0200, Armin Rigo writes: >Hi Jeff, > >I will have a look, but first let me mention that lxml-cffi is much >better supported (and faster) on pypy than lxml. It is unmaintained >though, so I guess it is getting out of date with lxml developments. > > >A bient?t, > >Armin. Has https://github.com/amauryfa/lxml/tree/lxml-cffi been renamed to https://github.com/amauryfa/lxml/tree/cffi ? The first one now is 404 on github, despite being the first google hit for lxml-cffi Laura From arigo at tunes.org Sun Oct 11 11:52:14 2015 From: arigo at tunes.org (Armin Rigo) Date: Sun, 11 Oct 2015 11:52:14 +0200 Subject: [pypy-dev] Looking for clues to consistent Seg Fault in PyPy 2.6.1 In-Reply-To: References: Message-ID: Hi, On Sun, Oct 11, 2015 at 12:28 AM, Jeff Doran wrote: > I've run out of options trying to find a Seg Fault which happens when > running lxml under PyPy 2.6.1. This problem only occurs under PyPy as the > rest of the code works fine under CPython 2.7. I've been in contact with > the lxml dev team and they confirmed my problem, but could not determine > where the cause of the Seg Fault lies. After some debugging, it seems that the PyPy-specific code with weakrefs in "proxy.pxi" is to blame. It seems to me that it would also have the same problem if it were compiled on CPython. (I understand why it is there, and indeed it is necessary to do *something* different on PyPy.) The problem is that if you start with two C structures "xmlNode" which form a small tree: XA: xmlNode with child XB XB: xmlNode You have two corresponding Python objects (actually cdef class _Element, but I think it's not important that they are Cython classes here): EA: _Element with _c_code = XA EB: _Element with _c_code = XB The reverse pointing is done differently on PyPy and on CPython. On CPython first: XA._private = (void *)EA XB._private = (void *)EB It's a plain pointer which doesn't hold a reference. The deallocation logic of _Element will reset the '_c_code._private' pointer back to NULL. On PyPy instead, there is an indirection: _private holds a reference to a weakref object. The effect is mostly the same. But the deallocation logic of _Element is subtly different as a result. Let's dig: The deallocation logic of E is: we reset E._c_code._private to NULL, and then if all X's in the tree have _private "set to NULL", then delete the whole tree. The problem is that "set to NULL" is more subtle in the weakref version. It really means "contains a weakref to a dead object". But weakrefs can die *before* the deallocator for their target is called. This is possible in both PyPy and CPython. So what occurs here: * we forget both EA and EB at the same time (for CPython, it can occur if there are in a cycle). * both weakrefs die * we call the deallocator of EA: it thinks the whole tree is dead because all weakrefs are dead, and frees it * we call the deallocator of EB: it still has _c_code pointing to XB, but that is garbage and crashes. That's the problem. I don't have a fix right now :-) A bient?t, Armin. From jdoran at lexmachina.com Sun Oct 11 23:26:16 2015 From: jdoran at lexmachina.com (Jeff Doran) Date: Sun, 11 Oct 2015 14:26:16 -0700 Subject: [pypy-dev] Looking for clues to consistent Seg Fault in PyPy 2.6.1 In-Reply-To: References: Message-ID: First Armin I want to thank you for taking the time to dig into this. It's a wonderful intro for me to the PyPy dev list. Next, while I'm admitedly a noob in the PyPy lower level, I'm curious why this problem hasn't been encountered more often. It's seems that each _Element should be responsible for deallocating it's own weakref and never have that outsourced to any other _Element.. In any case, thanks again and I will await a new PyPy to continue my investigation of this new platform for our production. Please note I will be happy to test any proposed solutions as they occur, nightly or otherwise. Best - Jeff On Sun, Oct 11, 2015 at 2:52 AM, Armin Rigo wrote: > Hi, > > On Sun, Oct 11, 2015 at 12:28 AM, Jeff Doran > wrote: > > I've run out of options trying to find a Seg Fault which happens when > > running lxml under PyPy 2.6.1. This problem only occurs under PyPy as > the > > rest of the code works fine under CPython 2.7. I've been in contact > with > > the lxml dev team and they confirmed my problem, but could not determine > > where the cause of the Seg Fault lies. > > After some debugging, it seems that the PyPy-specific code with > weakrefs in "proxy.pxi" is to blame. It seems to me that it would > also have the same problem if it were compiled on CPython. (I > understand why it is there, and indeed it is necessary to do > *something* different on PyPy.) > > The problem is that if you start with two C structures "xmlNode" which > form a small tree: > > XA: xmlNode with child XB > XB: xmlNode > > You have two corresponding Python objects (actually cdef class > _Element, but I think it's not important that they are Cython classes > here): > > EA: _Element with _c_code = XA > EB: _Element with _c_code = XB > > The reverse pointing is done differently on PyPy and on CPython. On > CPython first: > > XA._private = (void *)EA > XB._private = (void *)EB > > It's a plain pointer which doesn't hold a reference. The deallocation > logic of _Element will reset the '_c_code._private' pointer back to > NULL. > > On PyPy instead, there is an indirection: _private holds a reference > to a weakref object. The effect is mostly the same. But the > deallocation logic of _Element is subtly different as a result. Let's > dig: > > The deallocation logic of E is: we reset E._c_code._private to NULL, > and then if all X's in the tree have _private "set to NULL", then > delete the whole tree. The problem is that "set to NULL" is more > subtle in the weakref version. It really means "contains a weakref to > a dead object". But weakrefs can die *before* the deallocator for > their target is called. This is possible in both PyPy and CPython. > So what occurs here: > > * we forget both EA and EB at the same time (for CPython, it can occur > if there are in a cycle). > > * both weakrefs die > > * we call the deallocator of EA: it thinks the whole tree is dead > because all weakrefs are dead, and frees it > > * we call the deallocator of EB: it still has _c_code pointing to XB, > but that is garbage and crashes. > > That's the problem. I don't have a fix right now :-) > > > A bient?t, > > Armin. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From arigo at tunes.org Mon Oct 12 09:10:42 2015 From: arigo at tunes.org (Armin Rigo) Date: Mon, 12 Oct 2015 09:10:42 +0200 Subject: [pypy-dev] Looking for clues to consistent Seg Fault in PyPy 2.6.1 In-Reply-To: <201510110859.t9B8x3dq030165@fido.openend.se> References: <201510110859.t9B8x3dq030165@fido.openend.se> Message-ID: Hi Laura, On Sun, Oct 11, 2015 at 10:59 AM, Laura Creighton wrote: > Has https://github.com/amauryfa/lxml/tree/lxml-cffi been renamed > to https://github.com/amauryfa/lxml/tree/cffi ? > The first one now is 404 on github, despite being the first > google hit for lxml-cffi Uh, that's very new. I accessed that page about two days ago. Amaury, do you have a comment? A bient?t, Armin. From fijall at gmail.com Mon Oct 12 09:44:39 2015 From: fijall at gmail.com (Maciej Fijalkowski) Date: Mon, 12 Oct 2015 09:44:39 +0200 Subject: [pypy-dev] Looking for clues to consistent Seg Fault in PyPy 2.6.1 In-Reply-To: References: <201510110859.t9B8x3dq030165@fido.openend.se> Message-ID: On Mon, Oct 12, 2015 at 9:10 AM, Armin Rigo wrote: > Hi Laura, > > On Sun, Oct 11, 2015 at 10:59 AM, Laura Creighton wrote: >> Has https://github.com/amauryfa/lxml/tree/lxml-cffi been renamed >> to https://github.com/amauryfa/lxml/tree/cffi ? >> The first one now is 404 on github, despite being the first >> google hit for lxml-cffi > > Uh, that's very new. I accessed that page about two days ago. > Amaury, do you have a comment? > > > A bient?t, > > Armin. Are you sure? I think lxml-cffi link has been giving 404 for a while and the branch of lxml has been the official one. From arigo at tunes.org Mon Oct 12 09:56:36 2015 From: arigo at tunes.org (Armin Rigo) Date: Mon, 12 Oct 2015 09:56:36 +0200 Subject: [pypy-dev] Looking for clues to consistent Seg Fault in PyPy 2.6.1 In-Reply-To: References: <201510110859.t9B8x3dq030165@fido.openend.se> Message-ID: Hi Fijal, On Mon, Oct 12, 2015 at 9:44 AM, Maciej Fijalkowski wrote: > Are you sure? > > I think lxml-cffi link has been giving 404 for a while and the branch > of lxml has been the official one. I remember I typed "lxml-cffi" in Google and clicked on the first link. Maybe the Google results moved around? Right now, it doesn't show https://github.com/amauryfa/lxml/tree/cffi anywhere in the first two pages, but it shows https://bitbucket.org/pypy/compatibility/wiki/lxml which links to it. A bient?t, Armin. From amauryfa at gmail.com Mon Oct 12 10:19:45 2015 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Mon, 12 Oct 2015 10:19:45 +0200 Subject: [pypy-dev] Looking for clues to consistent Seg Fault in PyPy 2.6.1 In-Reply-To: References: <201510110859.t9B8x3dq030165@fido.openend.se> Message-ID: 2015-10-12 9:10 GMT+02:00 Armin Rigo : > Hi Laura, > > On Sun, Oct 11, 2015 at 10:59 AM, Laura Creighton wrote: > > Has https://github.com/amauryfa/lxml/tree/lxml-cffi been renamed > > to https://github.com/amauryfa/lxml/tree/cffi ? > > The first one now is 404 on github, despite being the first > > google hit for lxml-cffi > > Uh, that's very new. I accessed that page about two days ago. > Amaury, do you have a comment? > It's quite possible that early versions of the repo had this branch, and I somehow deleted it; I cannot remember how. But I'm sure nothing has changed in the last months. -- Amaury Forgeot d'Arc -------------- next part -------------- An HTML attachment was scrubbed... URL: From arigo at tunes.org Mon Oct 12 11:10:48 2015 From: arigo at tunes.org (Armin Rigo) Date: Mon, 12 Oct 2015 11:10:48 +0200 Subject: [pypy-dev] Looking for clues to consistent Seg Fault in PyPy 2.6.1 In-Reply-To: References: Message-ID: Hi Jeff, Ah, I think I understand now what is going on. The special-casing for PyPy should not be needed at all, but when I removed it, I get assertion failures. Now I get the reason---yes, it is yet another bug inside cpyext, caused by yet another access pattern of "PyObject *" that is somehow unusual but should be allowed. I can try to fix it inside cpyext, and then the "if IS_PYPY" cases in proxy.pxi should be removed. (I'm unsure if it is easy to change them to "if the version of PyPy is <= 2.6.1"...) A bient?t, Armin. From amauryfa at gmail.com Mon Oct 12 11:24:00 2015 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Mon, 12 Oct 2015 11:24:00 +0200 Subject: [pypy-dev] Looking for clues to consistent Seg Fault in PyPy 2.6.1 In-Reply-To: References: Message-ID: 2015-10-12 11:10 GMT+02:00 Armin Rigo : > Hi Jeff, > > Ah, I think I understand now what is going on. The special-casing for > PyPy should not be needed at all, but when I removed it, I get > assertion failures. Now I get the reason---yes, it is yet another bug > inside cpyext, caused by yet another access pattern of "PyObject *" > that is somehow unusual but should be allowed. > > I can try to fix it inside cpyext, and then the "if IS_PYPY" cases in > proxy.pxi should be removed. (I'm unsure if it is easy to change them > to "if the version of PyPy is <= 2.6.1"...) > We define it as a string (in pypy/module/cpyext/include/patchlevel.h) #define PYPY_VERSION "2.6.0-alpha0" -- Amaury Forgeot d'Arc -------------- next part -------------- An HTML attachment was scrubbed... URL: From arigo at tunes.org Mon Oct 12 11:30:05 2015 From: arigo at tunes.org (Armin Rigo) Date: Mon, 12 Oct 2015 11:30:05 +0200 Subject: [pypy-dev] Looking for clues to consistent Seg Fault in PyPy 2.6.1 In-Reply-To: References: Message-ID: Hi, On Mon, Oct 12, 2015 at 11:24 AM, Amaury Forgeot d'Arc wrote: > We define it as a string (in pypy/module/cpyext/include/patchlevel.h) > #define PYPY_VERSION "2.6.0-alpha0" Yes, but i don't know how to do this check in Cython. Well, it's probably easy. Armin From arigo at tunes.org Mon Oct 12 17:47:40 2015 From: arigo at tunes.org (Armin Rigo) Date: Mon, 12 Oct 2015 17:47:40 +0200 Subject: [pypy-dev] p12 to p7b In-Reply-To: References: Message-ID: Hi Roshan, On Thu, Oct 8, 2015 at 6:53 PM, Roshan Cherian wrote: > I have a requirement to convert from p12 to p7b. I am sorry for my little > knowledge on this and I am not even sure I can ask this here. Is this error showing up on PyPy but not on CPython for the same code? Are you using the same version of the libraries in both cases (which library is that, pyopenssl?) A bient?t, Armin. From stefan_ml at behnel.de Tue Oct 13 09:30:40 2015 From: stefan_ml at behnel.de (Stefan Behnel) Date: Tue, 13 Oct 2015 09:30:40 +0200 Subject: [pypy-dev] Looking for clues to consistent Seg Fault in PyPy 2.6.1 In-Reply-To: References: Message-ID: Armin Rigo schrieb am 12.10.2015 um 11:10: > Ah, I think I understand now what is going on. The special-casing for > PyPy should not be needed at all, but when I removed it, I get > assertion failures. Now I get the reason---yes, it is yet another bug > inside cpyext, caused by yet another access pattern of "PyObject *" > that is somehow unusual but should be allowed. > > I can try to fix it inside cpyext, and then the "if IS_PYPY" cases in > proxy.pxi should be removed. (I'm unsure if it is easy to change them > to "if the version of PyPy is <= 2.6.1"...) Armin, thanks for looking into this. I'll happily remove them if you can find a fix. Stefan From lac at openend.se Tue Oct 13 12:56:17 2015 From: lac at openend.se (Laura Creighton) Date: Tue, 13 Oct 2015 12:56:17 +0200 Subject: [pypy-dev] PEP 0484 - the Numeric Tower Message-ID: <201510131056.t9DAuHYn016661@fido.openend.se> Any chance of adding Decimal to the list of things that are also acceptable for things annotated float? Laura From lac at openend.se Tue Oct 13 13:20:55 2015 From: lac at openend.se (Laura Creighton) Date: Tue, 13 Oct 2015 13:20:55 +0200 Subject: [pypy-dev] PEP 0484 - the Numeric Tower In-Reply-To: <201510131056.t9DAuHYn016661@fido.openend.se> References: <201510131056.t9DAuHYn016661@fido.openend.se> Message-ID: <201510131120.t9DBKtMe018517@fido.openend.se> In a message of Tue, 13 Oct 2015 12:56:17 +0200, Laura Creighton writes: >Any chance of adding Decimal to the list of things that are also >acceptable for things annotated float? > >Laura Grrr. Meant to send that to python-dev. Sorry for the noise. Laura From cherian.rosh at gmail.com Wed Oct 14 00:38:09 2015 From: cherian.rosh at gmail.com (Roshan Cherian) Date: Tue, 13 Oct 2015 15:38:09 -0700 Subject: [pypy-dev] p12 to p7b In-Reply-To: References: Message-ID: Hi Armin, I am getting this issue in cpython as well as pypy. I am using pyopenssl 0.12. I am probably not using it the right way to convert to p7b? Thanks, -Roshan On Mon, Oct 12, 2015 at 8:47 AM, Armin Rigo wrote: > Hi Roshan, > > On Thu, Oct 8, 2015 at 6:53 PM, Roshan Cherian > wrote: > > I have a requirement to convert from p12 to p7b. I am sorry for my little > > knowledge on this and I am not even sure I can ask this here. > > Is this error showing up on PyPy but not on CPython for the same code? > Are you using the same version of the libraries in both cases (which > library is that, pyopenssl?) > > > A bient?t, > > Armin. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From arigo at tunes.org Wed Oct 14 08:13:07 2015 From: arigo at tunes.org (Armin Rigo) Date: Wed, 14 Oct 2015 08:13:07 +0200 Subject: [pypy-dev] p12 to p7b In-Reply-To: References: Message-ID: Hi Roshan, On Wed, Oct 14, 2015 at 12:38 AM, Roshan Cherian wrote: > I am getting this issue in cpython as well as pypy. I am using pyopenssl > 0.12. I am probably not using it the right way to convert to p7b? Ah, then you're asking in the wrong list. This list is about the development of PyPy. (As you can guess I have never heard about p7b or p12 myself.) You should ask either in some list related to pyopenssl, or in the general Python user group list , or on StackOverflow. A bient?t, Armin. From vaibhav_sood at persistent.com Wed Oct 14 11:26:29 2015 From: vaibhav_sood at persistent.com (Vaibhav Sood) Date: Wed, 14 Oct 2015 09:26:29 +0000 Subject: [pypy-dev] PPC stacklet switching code Message-ID: <4176FFD2EDA9C24CA9AB011273CFDDC8025C802E@HJ-MBX1.persistent.co.in> Hi, The stack switching code (switch_xxx_gcc.h) for PPC is missing for stacklets under rpython/translator/c/src/stacklet, I am trying to work on adding that code but I am facing an issue when doing that due to the way PPC assembly works and needed some help/advice I added the switch_ppc64_gcc.h file for PPC with the slp_switch function similar to the one's in the other files (eg. switch_x86_gcc.h etc) and I run the test cases in tests.c to test the new code but these fail with segfaults. I gdb'd the failures and find that the segfaults are due to PPC assembly using the calling functions stack to store the LR register but the g_save_state/g_restore_state functions in stacklet.c overwriting the stack contents causing the stored value to be destroyed Specifically, Running test_new(..) in tests.c calls stacklet_new(..) to create a new stacklet which in turn calls g_initialstub. _stacklet_switchstack is called twice here, first to save the old stack contents using g_initial_save_state in the heap memory and then again in the "if" code to restore the contents to the new stack using g_restore_state. The issue is when running the g_restore_state function, at the start the function prologue code saves the Link Register (LR) in the LR save area in the calling function's (slp_switch calls g_restore_state) stack frame as per PPC function calling conventions. This value is restored back into the LR register at the end by the epilogue code to return back to the calling function slp_switch. However in between the calling function's stack content is overwritten by the g_restore_state code (it memcpy's the stored heap memory data to the stack) causing the saved LR value to be destroyed and thus returning from the g_restore_state function (using blr) causes a segfault I am not sure how to get around this problem since it is specific to the way PPC assembly works and the same thing is not an issue on other platforms, any help/advice/comments would be a great help Thanks! Vaibhav DISCLAIMER ========== This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails. -------------- next part -------------- An HTML attachment was scrubbed... URL: From arigo at tunes.org Wed Oct 14 15:59:33 2015 From: arigo at tunes.org (Armin Rigo) Date: Wed, 14 Oct 2015 15:59:33 +0200 Subject: [pypy-dev] PPC stacklet switching code In-Reply-To: <4176FFD2EDA9C24CA9AB011273CFDDC8025C802E@HJ-MBX1.persistent.co.in> References: <4176FFD2EDA9C24CA9AB011273CFDDC8025C802E@HJ-MBX1.persistent.co.in> Message-ID: Hi Vaibhav, On Wed, Oct 14, 2015 at 11:26 AM, Vaibhav Sood wrote: > code to return back to the calling function slp_switch. However in between > the calling function?s stack content is overwritten by the g_restore_state > code (it memcpy?s the stored heap memory data to the stack) causing the > saved LR value to be destroyed and thus returning from the g_restore_state > function (using blr) causes a segfault I would decrement sp around the two calls to the C functions, g_save_state and g_restore_state, to give them their own temporary places to write to without interference. This is similar to how the x86 version gives "rsp" as argument, but then uses CALL, which pushes the return address just below rsp, i.e. out of scope of the saving/restoring. Example: ... mr r3, sp # the 1st argument is the value of sp without the extra space stdu sp, -64(sp) # decrement sp again; 64 should be enough ...call g_save_state... addi sp, sp, 64 # restore sp ... A bient?t, Armin. From arigo at tunes.org Wed Oct 14 16:07:46 2015 From: arigo at tunes.org (Armin Rigo) Date: Wed, 14 Oct 2015 16:07:46 +0200 Subject: [pypy-dev] PPC stacklet switching code In-Reply-To: References: <4176FFD2EDA9C24CA9AB011273CFDDC8025C802E@HJ-MBX1.persistent.co.in> Message-ID: Re-hi, Note another issue: the slp_switch() function, from gcc's point of view, does not call any other function. This means gcc will not produce the needed header/footer code. Look at switch_arm_gcc.h for the lengths we need to go through to make sure to save/restore everything and realign the stack. A bient?t, Armin. From mdione at grulic.org.ar Wed Oct 14 12:58:47 2015 From: mdione at grulic.org.ar (Marcos Dione) Date: Wed, 14 Oct 2015 12:58:47 +0200 Subject: [pypy-dev] [mdione@grulic.org.ar: modifying locals] Message-ID: <20151014105847.GM2798@diablo.grulicueva.local> Forwarding as suggested in python-list... ----- Forwarded message from Marcos Dione ----- Date: Sun, 11 Oct 2015 15:30:05 +0200 From: Marcos Dione Subject: modifying locals To: help at python.org Message-ID: <20151011133005.GC2798 at diablo.grulicueva.local> Content-Type: text/plain; charset=us-ascii First of all, I'm not subscribed to the list, so please CC me. I'm writing a language[1] heavily based on Python. The way I'm implementing it is by having my own parser (based/stolen from pypy), obtaining an AST, modifying it and running the result with Python3's exec(). One of its particular features is that I intend to support semi-transparent remote execution using paramiko. For that, there is a construction that looks like this: [local code 1] with remote(...): [remote code] [local code 2] The idea is that [remote code] is effectively executed in a remote machine specified in remote()'s parameters. This is implemented[3] by remote() being a context manager. Part of the transparency is that globals and locals at the remote() point will be accesible to the remote code (so far only variables), and that modifications to locals will be visible back to [local code 2]. So far I managed to do the first part, but not the second. I managed to transfer the locals back from the remote. My problem is modifying the locals in remote()'s __exit__() method. As I'm running the code using exec(), it's not just a matter of modifying locals()[2]. What I tried was the following: * When I execute the code, I pass an empty dictionary to hold the locals. I save a reference to it, and I try to use it to modify the locals [local code 2] sees, but as locals are related to stack frame, it doesn't work. * This leads to the second attempt: use the stack to modify remote()'s caller's locals by[4]: inception_locals= sys._getframe().f_back.f_locals inception_locals.update (locals) Somehow this doesn't work either, and it's driving me crazy. I run out of ideas, so maybe you could throw some? Thanks in advance. -- [1] if you're interested, check https://github.com/StyXman/ayrton/ [2] isn't it? Just by writing that I got my doubts... [3] https://github.com/StyXman/ayrton/blob/develop/ayrton/functions.py#L112 [4] https://github.com/StyXman/ayrton/blob/develop/ayrton/functions.py#L254 -- (Not so) Random fortune: They called me mad, and I called them mad, and damn them, they outvoted me. -- Nathaniel Lee ----- End forwarded message ----- -- (Not so) Random fortune: 20:45 < perrito666> la_rayis: la esperanza es lo ultimo que se pierde 20:45 * perrito666 se aferra a eso o ya estaria casado From vaibhav_sood at persistent.com Thu Oct 15 06:15:11 2015 From: vaibhav_sood at persistent.com (Vaibhav Sood) Date: Thu, 15 Oct 2015 04:15:11 +0000 Subject: [pypy-dev] PPC stacklet switching code In-Reply-To: References: <4176FFD2EDA9C24CA9AB011273CFDDC8025C802E@HJ-MBX1.persistent.co.in> Message-ID: <4176FFD2EDA9C24CA9AB011273CFDDC8025CE0A1@HJ-MBX1.persistent.co.in> Sure, I have been referring to the ARM file (since it?s the one similar to PPC), I will try and take care of including the necessary header/footer and stack adjusting code, thanks for mentioning this issue! Regards, Vaibhav -----Original Message----- From: armin.rigo at gmail.com [mailto:armin.rigo at gmail.com] On Behalf Of Armin Rigo Sent: Wednesday, October 14, 2015 7:38 PM To: Vaibhav Sood Cc: pypy-dev at python.org Subject: Re: [pypy-dev] PPC stacklet switching code Re-hi, Note another issue: the slp_switch() function, from gcc's point of view, does not call any other function. This means gcc will not produce the needed header/footer code. Look at switch_arm_gcc.h for the lengths we need to go through to make sure to save/restore everything and realign the stack. A bient?t, Armin. DISCLAIMER ========== This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails. From vaibhav_sood at persistent.com Thu Oct 15 06:15:11 2015 From: vaibhav_sood at persistent.com (Vaibhav Sood) Date: Thu, 15 Oct 2015 04:15:11 +0000 Subject: [pypy-dev] PPC stacklet switching code In-Reply-To: References: <4176FFD2EDA9C24CA9AB011273CFDDC8025C802E@HJ-MBX1.persistent.co.in> Message-ID: <4176FFD2EDA9C24CA9AB011273CFDDC8025CD09C@HJ-MBX1.persistent.co.in> Thanks Armin! This helps me a lot, I will try this out Regards, Vaibhav -----Original Message----- From: armin.rigo at gmail.com [mailto:armin.rigo at gmail.com] On Behalf Of Armin Rigo Sent: Wednesday, October 14, 2015 7:30 PM To: Vaibhav Sood Cc: pypy-dev at python.org Subject: Re: [pypy-dev] PPC stacklet switching code Hi Vaibhav, On Wed, Oct 14, 2015 at 11:26 AM, Vaibhav Sood wrote: > code to return back to the calling function slp_switch. However in > between the calling function?s stack content is overwritten by the > g_restore_state code (it memcpy?s the stored heap memory data to the > stack) causing the saved LR value to be destroyed and thus returning > from the g_restore_state function (using blr) causes a segfault I would decrement sp around the two calls to the C functions, g_save_state and g_restore_state, to give them their own temporary places to write to without interference. This is similar to how the x86 version gives "rsp" as argument, but then uses CALL, which pushes the return address just below rsp, i.e. out of scope of the saving/restoring. Example: ... mr r3, sp # the 1st argument is the value of sp without the extra space stdu sp, -64(sp) # decrement sp again; 64 should be enough ...call g_save_state... addi sp, sp, 64 # restore sp ... A bient?t, Armin. DISCLAIMER ========== This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails. From cournape at gmail.com Thu Oct 15 13:45:28 2015 From: cournape at gmail.com (David Cournapeau) Date: Thu, 15 Oct 2015 12:45:28 +0100 Subject: [pypy-dev] Building pypy from sources: partial build support ? Message-ID: Hi there, I am trying to build pypy from sources on Linux, but I get some compilation/linking errors at the end (related to gdbm, tcl/tk, etc...). Fixing those are easy, but running "pypy ../../rpython/bin/rpython -Ojit targetpypystandalone " seems to start a build from scratch, so every small fix has a 1.5 hour cycle. Is there a way to restart the compilation from the last point it failed ? The docs did not seem to mention anything, thanks, David -------------- next part -------------- An HTML attachment was scrubbed... URL: From fijall at gmail.com Thu Oct 15 14:14:37 2015 From: fijall at gmail.com (Maciej Fijalkowski) Date: Thu, 15 Oct 2015 14:14:37 +0200 Subject: [pypy-dev] Building pypy from sources: partial build support ? In-Reply-To: References: Message-ID: you can go to /tmp/usession-default-/testing_1 and type make to finish after the source code got generated. This is the only "incremental" part of the build, unfortunately (there were attempts to fix it, failed so far) On Thu, Oct 15, 2015 at 1:45 PM, David Cournapeau wrote: > Hi there, > > I am trying to build pypy from sources on Linux, but I get some > compilation/linking errors at the end (related to gdbm, tcl/tk, etc...). > Fixing those are easy, but running "pypy ../../rpython/bin/rpython -Ojit > targetpypystandalone " seems to start a build from scratch, so every small > fix has a 1.5 hour cycle. > > Is there a way to restart the compilation from the last point it failed ? > The docs did not seem to mention anything, > > thanks, > David > > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev > From yury at shurup.com Thu Oct 15 14:03:19 2015 From: yury at shurup.com (Yury V. Zaytsev) Date: Thu, 15 Oct 2015 14:03:19 +0200 (CEST) Subject: [pypy-dev] Building pypy from sources: partial build support ? In-Reply-To: References: Message-ID: Hi David, On Thu, 15 Oct 2015, David Cournapeau wrote: > Is there a way to restart the compilation from the last point it failed > ? The docs did not seem to mention anything, I don't think that there is a way to do it via the translation driver, but if the translation itself was succesfull, what you can do is to change into the directory in /tmp where the generated sources were written, edit the files directly and run make to continue the build from the point where it was interrupted. Hope that helps! -- Sincerely yours, Yury V. Zaytsev From arigo at tunes.org Thu Oct 15 16:14:02 2015 From: arigo at tunes.org (Armin Rigo) Date: Thu, 15 Oct 2015 16:14:02 +0200 Subject: [pypy-dev] [mdione@grulic.org.ar: modifying locals] In-Reply-To: <20151014105847.GM2798@diablo.grulicueva.local> References: <20151014105847.GM2798@diablo.grulicueva.local> Message-ID: Hi Marcos, On Wed, Oct 14, 2015 at 12:58 PM, Marcos Dione wrote: > inception_locals= sys._getframe().f_back.f_locals > inception_locals.update (locals) I think this works if you *assign* to f_locals. When you only *read* f_locals, you get a copy of the dict. The assignment is needed to write the new values back into the frame's real ("optimized") locals: inception_locals= sys._getframe().f_back.f_locals inception_locals.update (locals) sys._getframe().f_back.f_locals = inception_locals A bient?t, Armin. From mdione at grulic.org.ar Thu Oct 15 16:47:57 2015 From: mdione at grulic.org.ar (Marcos Dione) Date: Thu, 15 Oct 2015 16:47:57 +0200 Subject: [pypy-dev] [mdione@grulic.org.ar: modifying locals] In-Reply-To: References: <20151014105847.GM2798@diablo.grulicueva.local> Message-ID: <20151015144757.GP2798@diablo.grulicueva.local> On Thu, Oct 15, 2015 at 04:14:02PM +0200, Armin Rigo wrote: > Hi Marcos, > > On Wed, Oct 14, 2015 at 12:58 PM, Marcos Dione wrote: > > inception_locals= sys._getframe().f_back.f_locals > > inception_locals.update (locals) > > I think this works if you *assign* to f_locals. When you only *read* > f_locals, you get a copy of the dict. The assignment is needed to > write the new values back into the frame's real ("optimized") locals: > > inception_locals= sys._getframe().f_back.f_locals > inception_locals.update (locals) > sys._getframe().f_back.f_locals = inception_locals AttributeError: attribute 'f_locals' of 'frame' objects is not writable so no cigar here. on the other hand, this works: http://mike.verdone.ca/media/Dangerously%20Advanced%20Python.pdf [p21] so I'll keep investigating :-] > A bient?t, ?+ :) -- (Not so) Random fortune: Army General: [shouting] You told us that windows 98 would be faster, and more efficient with better access to the internet! Bill Gates: It IS faster! Over five million... [General shoots Bill Gates in the face and everyone cheers] -- South Park From mdione at grulic.org.ar Thu Oct 15 16:50:15 2015 From: mdione at grulic.org.ar (Marcos Dione) Date: Thu, 15 Oct 2015 16:50:15 +0200 Subject: [pypy-dev] [mdione@grulic.org.ar: modifying locals] In-Reply-To: <20151015144757.GP2798@diablo.grulicueva.local> References: <20151014105847.GM2798@diablo.grulicueva.local> <20151015144757.GP2798@diablo.grulicueva.local> Message-ID: <20151015145015.GQ2798@diablo.grulicueva.local> On Thu, Oct 15, 2015 at 04:47:57PM +0200, Marcos Dione wrote: > On Thu, Oct 15, 2015 at 04:14:02PM +0200, Armin Rigo wrote: > > Hi Marcos, > > > > On Wed, Oct 14, 2015 at 12:58 PM, Marcos Dione wrote: > > > inception_locals= sys._getframe().f_back.f_locals > > > inception_locals.update (locals) > > > > I think this works if you *assign* to f_locals. When you only *read* > > f_locals, you get a copy of the dict. The assignment is needed to > > write the new values back into the frame's real ("optimized") locals: > > > > inception_locals= sys._getframe().f_back.f_locals > > inception_locals.update (locals) > > sys._getframe().f_back.f_locals = inception_locals > > AttributeError: attribute 'f_locals' of 'frame' objects is not writable > > so no cigar here. on the other hand, this works: > > http://mike.verdone.ca/media/Dangerously%20Advanced%20Python.pdf [p21] > > so I'll keep investigating :-] maybe context managers add a stack frame/do something with them internally? -- (Not so) Random fortune: 02:46 < daemonkeeper> StyXman: Every good system administrator needs to configure sendmail at least once, a real good one won't do it a second time From yury at shurup.com Thu Oct 15 19:18:50 2015 From: yury at shurup.com (Yury V. Zaytsev) Date: Thu, 15 Oct 2015 19:18:50 +0200 Subject: [pypy-dev] [Fwd: Re: Building pypy from sources: partial build support ?] Message-ID: <1444929530.16386.1648.camel@newpride> -------- Forwarded Message -------- From: David Cournapeau To: Yury V. Zaytsev Subject: Re: [pypy-dev] Building pypy from sources: partial build support ? Date: Thu, 15 Oct 2015 16:37:15 +0100 Thank Yury. It seems my error happens after that stage actually: * _tkinter/tklib_build.py /usr/bin/ld: cannot find -ltcl collect2: ld returned 1 exit status Traceback (most recent call last): File "_tkinter/tklib_build.py", line 51, in library_dirs = libdirs File "/home/vagrant/src/tmp/pypy-2.6.1-src/lib_pypy/cffi/api.py", line 405, in verify lib = self.verifier.load_library() File "/home/vagrant/src/tmp/pypy-2.6.1-src/lib_pypy/cffi/verifier.py", line 100, in load_library self._compile_module() File "/home/vagrant/src/tmp/pypy-2.6.1-src/lib_pypy/cffi/verifier.py", line 196, in _compile_module outputfilename = ffiplatform.compile(tmpdir, self.get_extension()) File "/home/vagrant/src/tmp/pypy-2.6.1-src/lib_pypy/cffi/ffiplatform.py", line 38, in compile outputfilename = _build(tmpdir, ext) File "/home/vagrant/src/tmp/pypy-2.6.1-src/lib_pypy/cffi/ffiplatform.py", line 65, in _build raise VerificationError('%s: %s' % (e.__class__.__name__, e)) VerificationError: LinkError: command 'cc' failed with exit status 1 /home/vagrant/src/tmp/pypy-2.6.1-src/pypy/goal/pypy-c successfully built, but errors while building the above modules will be ignored Is there any way for to fix my build environment and restart the build without starting from scratch there ? David On Thu, Oct 15, 2015 at 1:03 PM, Yury V. Zaytsev wrote: Hi David, On Thu, 15 Oct 2015, David Cournapeau wrote: Is there a way to restart the compilation from the last point it failed ? The docs did not seem to mention anything, I don't think that there is a way to do it via the translation driver, but if the translation itself was succesfull, what you can do is to change into the directory in /tmp where the generated sources were written, edit the files directly and run make to continue the build from the point where it was interrupted. Hope that helps! -- Sincerely yours, Yury V. Zaytsev -- Sincerely yours, Yury V. Zaytsev From yury at shurup.com Thu Oct 15 19:18:36 2015 From: yury at shurup.com (Yury V. Zaytsev) Date: Thu, 15 Oct 2015 19:18:36 +0200 Subject: [pypy-dev] Building pypy from sources: partial build support ? In-Reply-To: References: Message-ID: <1444929516.16386.1647.camel@newpride> On Thu, 2015-10-15 at 16:37 +0100, David Cournapeau wrote: > > Is there any way for to fix my build environment and restart the build > without starting from scratch there ? Hi David, I will forward your original mail to the list, so that someone could correct me if I'm wrong, but I think that in your case since pypy-c was built successfully, you can just try to fix the tcl stuff and run pypy-c .../pypy/tool/build_cffi_imports.py as suggested here: http://pypy.org/download.html#building-from-source . As to your particular error, you need to edit lib_pypy/_tkinter/tklib.py and adjust linklibs accordingly. Most likely your distribution is trying to make it possible to install several versions to TCL/TK in parallel, and so there is no generic libtcl.so and libtk.so shipped, but rather something like libtcl86.so and libtk86.so, so you'd have to add the correct version... -- Sincerely yours, Yury V. Zaytsev From matti.picus at gmail.com Thu Oct 15 19:35:16 2015 From: matti.picus at gmail.com (Matti Picus) Date: Thu, 15 Oct 2015 20:35:16 +0300 Subject: [pypy-dev] [Fwd: Re: Building pypy from sources: partial build support ?] In-Reply-To: <1444929530.16386.1648.camel@newpride> References: <1444929530.16386.1648.camel@newpride> Message-ID: <561FE3D4.9080004@gmail.com> On 15/10/15 20:18, Yury V. Zaytsev wrote: > -------- Forwarded Message -------- > From: David Cournapeau > To: Yury V. Zaytsev > Subject: Re: [pypy-dev] Building pypy from sources: partial build > support ? > Date: Thu, 15 Oct 2015 16:37:15 +0100 > > Thank Yury. > > > It seems my error happens after that stage actually: > ... > /home/vagrant/src/tmp/pypy-2.6.1-src/pypy/goal/pypy-c successfully > built, but errors while building the above modules will be ignored > > > Is there any way for to fix my build environment and restart the build > without starting from scratch there ? > > > David > David your translation (conversion of rpython to pypy-c) succeeded. What failed is a hack to build the cffi import libraries needed for some standard python stdlib support libraries. You can just ignore these and move on to testing your translated pypy. If you actually *need* those stdlib modules, you can rerun the python import scripts at any time as pypy-c lib_pypy/_*_build.py where the * is any of sqlite3, audioop, curses, gdbm, pwdgrp, or syslog. The script in pypy/tool/release/package.py will build or rebuild these import libraries and package everything up into a nice tarball, you can see its documentation via --help Matti From matti.picus at gmail.com Fri Oct 16 07:12:02 2015 From: matti.picus at gmail.com (Matti Picus) Date: Fri, 16 Oct 2015 08:12:02 +0300 Subject: [pypy-dev] PyPy 15.11 release is imminent Message-ID: <56208722.5020507@gmail.com> An HTML attachment was scrubbed... URL: From arigo at tunes.org Fri Oct 16 08:32:36 2015 From: arigo at tunes.org (Armin Rigo) Date: Fri, 16 Oct 2015 08:32:36 +0200 Subject: [pypy-dev] [mdione@grulic.org.ar: modifying locals] In-Reply-To: <20151015145015.GQ2798@diablo.grulicueva.local> References: <20151014105847.GM2798@diablo.grulicueva.local> <20151015144757.GP2798@diablo.grulicueva.local> <20151015145015.GQ2798@diablo.grulicueva.local> Message-ID: Hi Marcos, On Thu, Oct 15, 2015 at 4:50 PM, Marcos Dione wrote: >> so I'll keep investigating :-] > > maybe context managers add a stack frame/do something with them > internally? Sorry, you're right. Now it seems from looking at the CPython source code that there is no way. This is getting quite off-topic for the pypy-dev list, but maybe this CPython-only awful hack: import ctypes ctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(frame), 0) A bient?t, Armin. From mdione at grulic.org.ar Fri Oct 16 09:07:27 2015 From: mdione at grulic.org.ar (Marcos Dione) Date: Fri, 16 Oct 2015 09:07:27 +0200 Subject: [pypy-dev] [mdione@grulic.org.ar: modifying locals] In-Reply-To: References: <20151014105847.GM2798@diablo.grulicueva.local> <20151015144757.GP2798@diablo.grulicueva.local> <20151015145015.GQ2798@diablo.grulicueva.local> Message-ID: <20151016070727.GB32186@diablo.grulicueva.local> On Fri, Oct 16, 2015 at 08:32:36AM +0200, Armin Rigo wrote: > On Thu, Oct 15, 2015 at 4:50 PM, Marcos Dione wrote: > >> so I'll keep investigating :-] > > > > maybe context managers add a stack frame/do something with them > > internally? > > Sorry, you're right. Now it seems from looking at the CPython source > code that there is no way. could you point me where in the code? maybe there? a bug to fix, or at least a reason to understand. > This is getting quite off-topic for the > pypy-dev list, but maybe this CPython-only awful hack: > > import ctypes > ctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(frame), 0) cool, thanks, I'll try that today. -- (Not so) Random fortune: La raz?n por la que la industria de medios es tan amiga del t?rmino ?pirater?a? es que inmediatamente evoca una imagen negativa, de violencia y saqueo, en la que ellos son las v?ctimas que se ven privadas de ?su propiedad?. [...] en una conferencia que diera la abogada de la Federaci?n Internacional de la Industria Fonogr?fica, [...] escuchamos c?mo un abogado se alarmaba del hecho de que ?la sociedad se est? apropiando de la cultura?. -- http://federratas.codigolibre.net/?p=137 From phyo.arkarlwin at gmail.com Fri Oct 16 14:21:54 2015 From: phyo.arkarlwin at gmail.com (Phyo Arkar) Date: Fri, 16 Oct 2015 18:51:54 +0630 Subject: [pypy-dev] PyPy 15.11 release is imminent In-Reply-To: <56208722.5020507@gmail.com> References: <56208722.5020507@gmail.com> Message-ID: So that will work like 15.11 , 15.5 ? something like that? Predefined release cycle ? And Awesome work Guys!@ On Fri, Oct 16, 2015 at 11:42 AM, Matti Picus wrote: > I have started a major release cycle, and consensus was to start a new > numbering scheme, based on yy.mm > > While every release is a major event (yes 2.5.0, you can get a > participation award too) this one really is a biggie. > Warmup and tracing memory improvements, internal refactoring, SIMD > vectorization on x86, and more. > > Please let me know if there are more good things worth waiting for and > help flesh out the release notice > https://bitbucket.org/pypy/pypy/src/default/pypy/doc/release-15.11.0.rst > > Matti > > > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From phyo.arkarlwin at gmail.com Fri Oct 16 14:23:35 2015 From: phyo.arkarlwin at gmail.com (Phyo Arkar) Date: Fri, 16 Oct 2015 18:53:35 +0630 Subject: [pypy-dev] PyPy 15.11 release is imminent In-Reply-To: References: <56208722.5020507@gmail.com> Message-ID: warmup time improvement of 20% or so at the cost of a minor regression in jitted code speed. 20% Warmup is huge! That will skyrocket in bencmarks. But Minor regression? how minor ? On Fri, Oct 16, 2015 at 6:51 PM, Phyo Arkar wrote: > So that will work like 15.11 , 15.5 ? something like that? > Predefined release cycle ? > > And Awesome work Guys!@ > > On Fri, Oct 16, 2015 at 11:42 AM, Matti Picus > wrote: > >> I have started a major release cycle, and consensus was to start a new >> numbering scheme, based on yy.mm >> >> While every release is a major event (yes 2.5.0, you can get a >> participation award too) this one really is a biggie. >> Warmup and tracing memory improvements, internal refactoring, SIMD >> vectorization on x86, and more. >> >> Please let me know if there are more good things worth waiting for and >> help flesh out the release notice >> https://bitbucket.org/pypy/pypy/src/default/pypy/doc/release-15.11.0.rst >> >> Matti >> >> >> _______________________________________________ >> pypy-dev mailing list >> pypy-dev at python.org >> https://mail.python.org/mailman/listinfo/pypy-dev >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From matti.picus at gmail.com Fri Oct 16 14:33:51 2015 From: matti.picus at gmail.com (Matti Picus) Date: Fri, 16 Oct 2015 15:33:51 +0300 Subject: [pypy-dev] PyPy 15.11 release is imminent In-Reply-To: References: <56208722.5020507@gmail.com> Message-ID: <5620EEAF.4070100@gmail.com> On 16/10/15 15:23, Phyo Arkar wrote: > warmup time improvement of 20% or so at the cost of a minor > regression in jitted code speed. > > 20% Warmup is huge! That will skyrocket in bencmarks. > > But Minor regression? how minor ? > speed.pypy.org (which measures only jit performance without warmup) went from 7.10 to 6.90, but of course that is just for those benchmarks Matti From fijall at gmail.com Fri Oct 16 15:31:38 2015 From: fijall at gmail.com (Maciej Fijalkowski) Date: Fri, 16 Oct 2015 15:31:38 +0200 Subject: [pypy-dev] PyPy 15.11 release is imminent In-Reply-To: <5620EEAF.4070100@gmail.com> References: <56208722.5020507@gmail.com> <5620EEAF.4070100@gmail.com> Message-ID: I claim at least a part of the jitted code change is a fluke. A part is due to disabling of retracing, which never worked properly as far as I know anyway, we'll reenable that somehow at some point. I would not put it in the release announcement. On Fri, Oct 16, 2015 at 2:33 PM, Matti Picus wrote: > On 16/10/15 15:23, Phyo Arkar wrote: >> >> warmup time improvement of 20% or so at the cost of a minor regression in >> jitted code speed. >> >> 20% Warmup is huge! That will skyrocket in bencmarks. >> >> But Minor regression? how minor ? >> > > speed.pypy.org (which measures only jit performance without warmup) went > from 7.10 to 6.90, but of course that is just for those benchmarks > > > Matti > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev From cournape at gmail.com Fri Oct 16 17:51:43 2015 From: cournape at gmail.com (David Cournapeau) Date: Fri, 16 Oct 2015 16:51:43 +0100 Subject: [pypy-dev] [Fwd: Re: Building pypy from sources: partial build support ?] In-Reply-To: <561FE3D4.9080004@gmail.com> References: <1444929530.16386.1648.camel@newpride> <561FE3D4.9080004@gmail.com> Message-ID: On Thu, Oct 15, 2015 at 6:35 PM, Matti Picus wrote: > > > On 15/10/15 20:18, Yury V. Zaytsev wrote: > >> -------- Forwarded Message -------- >> From: David Cournapeau >> To: Yury V. Zaytsev >> Subject: Re: [pypy-dev] Building pypy from sources: partial build >> support ? >> Date: Thu, 15 Oct 2015 16:37:15 +0100 >> >> Thank Yury. >> >> >> It seems my error happens after that stage actually: >> ... >> /home/vagrant/src/tmp/pypy-2.6.1-src/pypy/goal/pypy-c successfully >> built, but errors while building the above modules will be ignored >> >> >> Is there any way for to fix my build environment and restart the build >> without starting from scratch there ? >> >> >> David >> >> David your translation (conversion of rpython to pypy-c) succeeded. > What failed is a hack to build the cffi import libraries needed for some > standard python stdlib support libraries. You can just ignore these and > move on to testing your translated pypy. > If you actually *need* those stdlib modules, you can rerun the python > import scripts at any time as > pypy-c lib_pypy/_*_build.py > where the * is any of sqlite3, audioop, curses, gdbm, pwdgrp, or syslog. > The script in pypy/tool/release/package.py will build or rebuild these > import libraries and package everything up into a nice tarball, you can see > its documentation via --help > Indeed, I could make it work with a bit more violence ! Thank you everybody for the help, David > > Matti > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdione at grulic.org.ar Fri Oct 16 20:16:24 2015 From: mdione at grulic.org.ar (Marcos Dione) Date: Fri, 16 Oct 2015 20:16:24 +0200 Subject: [pypy-dev] [mdione@grulic.org.ar: modifying locals] In-Reply-To: <20151016070727.GB32186@diablo.grulicueva.local> References: <20151014105847.GM2798@diablo.grulicueva.local> <20151015144757.GP2798@diablo.grulicueva.local> <20151015145015.GQ2798@diablo.grulicueva.local> <20151016070727.GB32186@diablo.grulicueva.local> Message-ID: <20151016181624.GE32186@diablo.grulicueva.local> On Fri, Oct 16, 2015 at 09:07:27AM +0200, Marcos Dione wrote: > On Fri, Oct 16, 2015 at 08:32:36AM +0200, Armin Rigo wrote: > > On Thu, Oct 15, 2015 at 4:50 PM, Marcos Dione wrote: > > >> so I'll keep investigating :-] > > > > > > maybe context managers add a stack frame/do something with them > > > internally? > > > > Sorry, you're right. Now it seems from looking at the CPython source > > code that there is no way. > > could you point me where in the code? maybe there? a bug to fix, or > at least a reason to understand. here's very good post on the subject: https://mail.python.org/pipermail/python-dev/2005-January/051018.html in short: f_locals is a built-on-the-fly dict that actually has the copy of the original data; it's modifiable, but those modifs don't go back to where that came from. he also suggests using PyFrame_LocalsToFast(), and the only two places where that function is called in in the implementation of from foo import * and some other place I hadn't figure out yet. and this seems to be like this since before py2.1. as for the hack itself, it works! \o/ thanks very much for the pointers :) Cheers, -- Marcos. -- (Not so) Random fortune: windows s'enrichis, le libre t'enrichis From tuom.larsen at gmail.com Fri Oct 16 21:31:32 2015 From: tuom.larsen at gmail.com (Tuom Larsen) Date: Fri, 16 Oct 2015 21:31:32 +0200 Subject: [pypy-dev] "PyPy lets us go where we need to" Message-ID: Dead list, not sure if this is the right place to share but there is a presentation by a Mozilla dev being currently discussed on Reddit where PyPy received quite some praise: https://www.reddit.com/r/golang/comments/3oztwi/from_python_to_go_and_back_again_mozilla_dev/ Cheers! From rymg19 at gmail.com Fri Oct 16 21:44:19 2015 From: rymg19 at gmail.com (Ryan Gonzalez) Date: Fri, 16 Oct 2015 14:44:19 -0500 Subject: [pypy-dev] "PyPy lets us go where we need to" In-Reply-To: References: Message-ID: You know how it goes: everyone overhypes something as "perfect", only to realize it's not actually perfect. See: Node.js, Java, Rails, ... PyPy definitely deserves a bit more use in the real world, so I really like this presentation. On October 16, 2015 2:31:32 PM CDT, Tuom Larsen wrote: >Dead list, > >not sure if this is the right place to share but there is a >presentation by a Mozilla dev being currently discussed on Reddit >where PyPy received quite some praise: > >https://www.reddit.com/r/golang/comments/3oztwi/from_python_to_go_and_back_again_mozilla_dev/ > >Cheers! >_______________________________________________ >pypy-dev mailing list >pypy-dev at python.org >https://mail.python.org/mailman/listinfo/pypy-dev -- Sent from my Nexus 5 with K-9 Mail. Please excuse my brevity. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rymg19 at gmail.com Fri Oct 16 22:07:24 2015 From: rymg19 at gmail.com (Ryan Gonzalez) Date: Fri, 16 Oct 2015 15:07:24 -0500 Subject: [pypy-dev] "PyPy lets us go where we need to" In-Reply-To: References: Message-ID: <7248BFB0-3B90-4D87-BEC0-993F02706E1F@gmail.com> You know how it goes: everyone overhypes something as "perfect", only to realize it's not actually perfect. See: Node.js, Java, Rails, ... PyPy definitely deserves a bit more use in the real world, so I really like this presentation. On October 16, 2015 2:31:32 PM CDT, Tuom Larsen wrote: >Dead list, > >not sure if this is the right place to share but there is a >presentation by a Mozilla dev being currently discussed on Reddit >where PyPy received quite some praise: > >https://www.reddit.com/r/golang/comments/3oztwi/from_python_to_go_and_back_again_mozilla_dev/ > >Cheers! >_______________________________________________ >pypy-dev mailing list >pypy-dev at python.org >https://mail.python.org/mailman/listinfo/pypy-dev -- Sent from my Nexus 5 with K-9 Mail. Please excuse my brevity. -- Sent from my Nexus 5 with K-9 Mail. Please excuse my brevity. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rymg19 at gmail.com Fri Oct 16 22:15:53 2015 From: rymg19 at gmail.com (Ryan Gonzalez) Date: Fri, 16 Oct 2015 15:15:53 -0500 Subject: [pypy-dev] "PyPy lets us go where we need to" In-Reply-To: <7248BFB0-3B90-4D87-BEC0-993F02706E1F@gmail.com> References: <7248BFB0-3B90-4D87-BEC0-993F02706E1F@gmail.com> Message-ID: Crud...I think I accidentally sent that twice... :O I also x-posted this to /r/programming: https://www.reddit.com/r/programming/comments/3p128a/from_python_to_go_and_back_again_xposted_from/. On October 16, 2015 3:07:24 PM CDT, Ryan Gonzalez wrote: >You know how it goes: everyone overhypes something as "perfect", only >to realize it's not actually perfect. > >See: Node.js, Java, Rails, ... > >PyPy definitely deserves a bit more use in the real world, so I really >like this presentation. > >On October 16, 2015 2:31:32 PM CDT, Tuom Larsen >wrote: >>Dead list, >> >>not sure if this is the right place to share but there is a >>presentation by a Mozilla dev being currently discussed on Reddit >>where PyPy received quite some praise: >> >>https://www.reddit.com/r/golang/comments/3oztwi/from_python_to_go_and_back_again_mozilla_dev/ >> >>Cheers! >>_______________________________________________ >>pypy-dev mailing list >>pypy-dev at python.org >>https://mail.python.org/mailman/listinfo/pypy-dev > >-- >Sent from my Nexus 5 with K-9 Mail. Please excuse my brevity. >-- >Sent from my Nexus 5 with K-9 Mail. Please excuse my brevity. -- Sent from my Nexus 5 with K-9 Mail. Please excuse my brevity. -------------- next part -------------- An HTML attachment was scrubbed... URL: From arigo at tunes.org Sat Oct 17 08:40:46 2015 From: arigo at tunes.org (Armin Rigo) Date: Sat, 17 Oct 2015 08:40:46 +0200 Subject: [pypy-dev] [mdione@grulic.org.ar: modifying locals] In-Reply-To: <20151016181624.GE32186@diablo.grulicueva.local> References: <20151014105847.GM2798@diablo.grulicueva.local> <20151015144757.GP2798@diablo.grulicueva.local> <20151015145015.GQ2798@diablo.grulicueva.local> <20151016070727.GB32186@diablo.grulicueva.local> <20151016181624.GE32186@diablo.grulicueva.local> Message-ID: Hi Marcos, On Fri, Oct 16, 2015 at 8:16 PM, Marcos Dione wrote: > as for the hack itself, it works! \o/ thanks very much for the > pointers :) Cheers, No problem. Actually, some Googling finds the very same hack used at various places and explained in blog posts... Armin From arigo at tunes.org Sat Oct 17 10:57:40 2015 From: arigo at tunes.org (Armin Rigo) Date: Sat, 17 Oct 2015 10:57:40 +0200 Subject: [pypy-dev] PyPy 15.11 release is imminent In-Reply-To: References: <56208722.5020507@gmail.com> Message-ID: Hi Phyo, On Fri, Oct 16, 2015 at 2:21 PM, Phyo Arkar wrote: > So that will work like 15.11 , 15.5 ? something like that? > Predefined release cycle ? No, you're jumping to conclusions only because of numbers that look like Ubuntu's. Armin From phyo.arkarlwin at gmail.com Sat Oct 17 14:10:10 2015 From: phyo.arkarlwin at gmail.com (Phyo Arkar) Date: Sat, 17 Oct 2015 18:40:10 +0630 Subject: [pypy-dev] PyPy 15.11 release is imminent In-Reply-To: References: <56208722.5020507@gmail.com> Message-ID: Lol I C. I was thinking that way. How abt also adding code names? I can come up with fun ones. On Oct 17, 2015 3:28 PM, "Armin Rigo" wrote: > Hi Phyo, > > On Fri, Oct 16, 2015 at 2:21 PM, Phyo Arkar > wrote: > > So that will work like 15.11 , 15.5 ? something like that? > > Predefined release cycle ? > > No, you're jumping to conclusions only because of numbers that look > like Ubuntu's. > > > Armin > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fijall at gmail.com Sat Oct 17 15:25:04 2015 From: fijall at gmail.com (Maciej Fijalkowski) Date: Sat, 17 Oct 2015 15:25:04 +0200 Subject: [pypy-dev] PyPy 15.11 release is imminent In-Reply-To: References: <56208722.5020507@gmail.com> Message-ID: We usually have code names, they just don't follow ANY known pattern On Sat, Oct 17, 2015 at 2:10 PM, Phyo Arkar wrote: > Lol I C. I was thinking that way. How abt also adding code names? I can come > up with fun ones. > > On Oct 17, 2015 3:28 PM, "Armin Rigo" wrote: >> >> Hi Phyo, >> >> On Fri, Oct 16, 2015 at 2:21 PM, Phyo Arkar >> wrote: >> > So that will work like 15.11 , 15.5 ? something like that? >> > Predefined release cycle ? >> >> No, you're jumping to conclusions only because of numbers that look >> like Ubuntu's. >> >> >> Armin > > > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev > From matti.picus at gmail.com Sat Oct 17 18:02:51 2015 From: matti.picus at gmail.com (Matti Picus) Date: Sat, 17 Oct 2015 19:02:51 +0300 Subject: [pypy-dev] PyPy 15.11 release is imminent In-Reply-To: References: <56208722.5020507@gmail.com> Message-ID: <5622712B.7000004@gmail.com> On 17/10/15 15:10, Phyo Arkar wrote: > > Lol I C. I was thinking that way. How abt also adding code names? I > can come up with fun ones. > > On Oct 17, 2015 3:28 PM, "Armin Rigo" > wrote: I kind of gave up with code names, it was a bit irrelevant and no-one used it after the release. But if someone has a good one, I am listening Matti From techtonik at gmail.com Sun Oct 18 07:34:43 2015 From: techtonik at gmail.com (anatoly techtonik) Date: Sun, 18 Oct 2015 08:34:43 +0300 Subject: [pypy-dev] PyPy 15.11 release is imminent In-Reply-To: References: <56208722.5020507@gmail.com> Message-ID: On Sat, Oct 17, 2015 at 11:57 AM, Armin Rigo wrote: > Hi Phyo, > > On Fri, Oct 16, 2015 at 2:21 PM, Phyo Arkar > wrote: > > So that will work like 15.11 , 15.5 ? something like that? > > Predefined release cycle ? > > No, you're jumping to conclusions only because of numbers that look > like Ubuntu's. > I don't see the reason then. You can't jump back to plain versions. I used to do this versioning, but then realized that if I do major API break or just commit something huge, nobody will get the idea. -- anatoly t. -------------- next part -------------- An HTML attachment was scrubbed... URL: From naylor.b.david at gmail.com Sun Oct 18 12:15:02 2015 From: naylor.b.david at gmail.com (David Naylor) Date: Sun, 18 Oct 2015 03:15:02 -0700 (PDT) Subject: [pypy-dev] PyPy 15.11 release is imminent In-Reply-To: References: <56208722.5020507@gmail.com> Message-ID: <8354791.5sJ5c96EUL@dragon.local> On Sunday, 18 October 2015 08:34:43 anatoly techtonik wrote: > On Sat, Oct 17, 2015 at 11:57 AM, Armin Rigo wrote: > > Hi Phyo, > > > > On Fri, Oct 16, 2015 at 2:21 PM, Phyo Arkar > > > > wrote: > > > So that will work like 15.11 , 15.5 ? something like that? > > > Predefined release cycle ? > > > > No, you're jumping to conclusions only because of numbers that look > > like Ubuntu's. > > I don't see the reason then. You can't jump back to plain versions. I used > to > do this versioning, but then realized that if I do major API break or just > commit > something huge, nobody will get the idea. I agree, I prefer to use semantic versioning (see http://semver.org/). Not sure if it is too late but has this been considered, and rejected? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 949 bytes Desc: This is a digitally signed message part. URL: From robin at reportlab.com Mon Oct 19 16:00:30 2015 From: robin at reportlab.com (Robin Becker) Date: Mon, 19 Oct 2015 15:00:30 +0100 Subject: [pypy-dev] PyPy 15.11 release is imminent In-Reply-To: <8354791.5sJ5c96EUL@dragon.local> References: <56208722.5020507@gmail.com> <8354791.5sJ5c96EUL@dragon.local> Message-ID: <5624F77E.4050407@chamonix.reportlab.co.uk> .......... > > I agree, I prefer to use semantic versioning (see http://semver.org/). Not > sure if it is too late but has this been considered, and rejected? ...... I think the intention is just year:month, but how does this relate to the python version semantics ie does 15.11 support 2.7 only? -- Robin Becker From techtonik at gmail.com Mon Oct 19 16:07:52 2015 From: techtonik at gmail.com (anatoly techtonik) Date: Mon, 19 Oct 2015 17:07:52 +0300 Subject: [pypy-dev] PyPy 15.11 release is imminent In-Reply-To: <5624F77E.4050407@chamonix.reportlab.co.uk> References: <56208722.5020507@gmail.com> <8354791.5sJ5c96EUL@dragon.local> <5624F77E.4050407@chamonix.reportlab.co.uk> Message-ID: On Mon, Oct 19, 2015 at 5:00 PM, Robin Becker wrote: > .......... > >> >> I agree, I prefer to use semantic versioning (see http://semver.org/). >> Not >> sure if it is too late but has this been considered, and rejected? >> > ...... > I think the intention is just year:month, but how does this relate to the > python version semantics ie does 15.11 support 2.7 only? FWIW, the final format that I chose became 1.YY.M 1 - is the API version or whatever YY.M - is an obvious indicator of the freshness of the package -- anatoly t. -------------- next part -------------- An HTML attachment was scrubbed... URL: From fijall at gmail.com Mon Oct 19 16:38:36 2015 From: fijall at gmail.com (Maciej Fijalkowski) Date: Mon, 19 Oct 2015 16:38:36 +0200 Subject: [pypy-dev] PyPy 15.11 release is imminent In-Reply-To: <5624F77E.4050407@chamonix.reportlab.co.uk> References: <56208722.5020507@gmail.com> <8354791.5sJ5c96EUL@dragon.local> <5624F77E.4050407@chamonix.reportlab.co.uk> Message-ID: Yes, PyPy, whatever version, will support python 2.7 for the forseeable future. PyPy3 whatever version will support 3.x On Mon, Oct 19, 2015 at 4:00 PM, Robin Becker wrote: > .......... >> >> >> I agree, I prefer to use semantic versioning (see http://semver.org/). >> Not >> sure if it is too late but has this been considered, and rejected? > > ...... > I think the intention is just year:month, but how does this relate to the > python version semantics ie does 15.11 support 2.7 only? > -- > Robin Becker > > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev From anto.cuni at gmail.com Tue Oct 20 15:48:01 2015 From: anto.cuni at gmail.com (Antonio Cuni) Date: Tue, 20 Oct 2015 15:48:01 +0200 Subject: [pypy-dev] PyPy 15.11 release is imminent In-Reply-To: <56208722.5020507@gmail.com> References: <56208722.5020507@gmail.com> Message-ID: why using yy.mm instead of just a single increasing int number like Chrome? To avoid confusions, we should probably skip pypy 3 and start releasing from pypy 4. It looks just simpler than 15.11 and friends to me. ciao, Anto On Fri, Oct 16, 2015 at 7:12 AM, Matti Picus wrote: > I have started a major release cycle, and consensus was to start a new > numbering scheme, based on yy.mm > > While every release is a major event (yes 2.5.0, you can get a > participation award too) this one really is a biggie. > Warmup and tracing memory improvements, internal refactoring, SIMD > vectorization on x86, and more. > > Please let me know if there are more good things worth waiting for and > help flesh out the release notice > https://bitbucket.org/pypy/pypy/src/default/pypy/doc/release-15.11.0.rst > > Matti > > > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuom.larsen at gmail.com Tue Oct 20 16:20:19 2015 From: tuom.larsen at gmail.com (Tuom Larsen) Date: Tue, 20 Oct 2015 16:20:19 +0200 Subject: [pypy-dev] Automatic SIMD vectorization Message-ID: Hello all! I just read with great interest the blog post "Automatic SIMD vectorization support in PyPy". Please, I have a few questions: - Does regular Python code benefit from the vectorization? I mean, the article on one hand says "it is not specifically targeted for the NumPy library" but on the other it says "Any interpreter (written in RPython)". - I would like to write a vector class as much suitable for PyPy as possible, what approach should I take in order to implement it? For example, what would suit PyPy JIT the best: class Vector3d: def __init__(a, x, y, z): (a.x, a.y, a.z) = x, y, z def __add__(a, b): return Vector3d(a.x + b.x, a.y+b.y, a.z+b.z) def add1(a, b): (ax, ay, az) = a (bx, by, bz) = b return [ax + bx, ay + by, az + bz] def add2(a, b): (ax, ay, az) = a (bx, by, bz) = b return (ax + bx, ay + by, az + bz) def add3((ax, ay, az), (bx, by, bz)): return (ax + bx, ay + by, az + bz) def add3: ??? - Is NumPyPy going to be included with regular PyPy download/install? Thanks a lot in advance! From phyo.arkarlwin at gmail.com Tue Oct 20 16:37:17 2015 From: phyo.arkarlwin at gmail.com (Phyo Arkar) Date: Tue, 20 Oct 2015 21:07:17 +0630 Subject: [pypy-dev] Any Release incoming for pypy 3.x? In-Reply-To: References: Message-ID: Any progress in it? On Tue, Oct 20, 2015 at 9:02 PM, Phyo Arkar wrote: > I am waiting to switch to Python3 , but i am not switching without pypy3 > moving forward > -------------- next part -------------- An HTML attachment was scrubbed... URL: From phyo.arkarlwin at gmail.com Tue Oct 20 16:32:02 2015 From: phyo.arkarlwin at gmail.com (Phyo Arkar) Date: Tue, 20 Oct 2015 21:02:02 +0630 Subject: [pypy-dev] Any Release incoming for pypy 3.x? Message-ID: I am waiting to switch to Python3 , but i am not switching without pypy3 moving forward -------------- next part -------------- An HTML attachment was scrubbed... URL: From planrichi at gmail.com Tue Oct 20 16:41:41 2015 From: planrichi at gmail.com (Richard Plangger) Date: Tue, 20 Oct 2015 16:41:41 +0200 Subject: [pypy-dev] Automatic SIMD vectorization In-Reply-To: References: Message-ID: <562652A5.70901@gmail.com> Hi, glad you liked the post! See the answers below... On 10/20/2015 04:20 PM, Tuom Larsen wrote: > Hello all! > > I just read with great interest the blog post "Automatic SIMD > vectorization support in PyPy". > > Please, I have a few questions: > > - Does regular Python code benefit from the vectorization? I mean, the > article on one hand says "it is not specifically targeted for the > NumPy library" but on the other it says "Any interpreter (written in > RPython)". Speaking about 'regular' Python code, there is potential !BUT! only if enough time is spent in numeric code. What I meant in the article was: E.g. you have a vector construct in your language (like the one the R language has) you could use the optimization to vectorize operations on the variables that represent. Take a look at my test virtual machine implementing a small subset of R. https://bitbucket.org/plan_rich/vecopt-test-vm > > - I would like to write a vector class as much suitable for PyPy as > possible, what approach should I take in order to implement it? For > example, what would suit PyPy JIT the best: > > class Vector3d: > def __init__(a, x, y, z): > (a.x, a.y, a.z) = x, y, z > def __add__(a, b): > return Vector3d(a.x + b.x, a.y+b.y, a.z+b.z) > > def add1(a, b): > (ax, ay, az) = a > (bx, by, bz) = b > return [ax + bx, ay + by, az + bz] > > def add2(a, b): > (ax, ay, az) = a > (bx, by, bz) = b > return (ax + bx, ay + by, az + bz) > > def add3((ax, ay, az), (bx, by, bz)): > return (ax + bx, ay + by, az + bz) > > def add3: > ??? > I have made some tests with this already. You would need to use the array module. Python lists would also work, but they leave behind some instructions that are not well optimized. It is described in this post: http://pypyvecopt.blogspot.co.at/2015/08/gsoc-vec-little-brother-of-numpy-array.html The missing piece is then the --jit vec_all=1 parameter, that you must specify on the command line. Be aware: 1) vec_all=1 parameter might lead to a crash, I have tested it, but it occurred to me that it is not really ready for production. I'm still working on this, thus it is disabled by default. 2) that with a very low number of vector elements (e.g 3) the overhead to iterate the first iteration is quite significant. So I do not think that there is much you can get just executing 1 vector add in parallel on x86. > - Is NumPyPy going to be included with regular PyPy download/install? > NumPyPy is included in a normal PyPy release version > Thanks a lot in advance! > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev > Cheers, Richard From tuom.larsen at gmail.com Tue Oct 20 17:08:12 2015 From: tuom.larsen at gmail.com (Tuom Larsen) Date: Tue, 20 Oct 2015 17:08:12 +0200 Subject: [pypy-dev] Automatic SIMD vectorization In-Reply-To: <562652A5.70901@gmail.com> References: <562652A5.70901@gmail.com> Message-ID: Awesome, thank you! On Tue, Oct 20, 2015 at 4:41 PM, Richard Plangger wrote: > Hi, > > glad you liked the post! See the answers below... > > On 10/20/2015 04:20 PM, Tuom Larsen wrote: >> Hello all! >> >> I just read with great interest the blog post "Automatic SIMD >> vectorization support in PyPy". >> >> Please, I have a few questions: >> >> - Does regular Python code benefit from the vectorization? I mean, the >> article on one hand says "it is not specifically targeted for the >> NumPy library" but on the other it says "Any interpreter (written in >> RPython)". > > Speaking about 'regular' Python code, there is potential !BUT! only if > enough time is spent in numeric code. What I meant in the article was: > E.g. you have a vector construct in your language (like the one the R > language has) you could use the optimization to vectorize operations on > the variables that represent. > > Take a look at my test virtual machine implementing a small subset of R. > https://bitbucket.org/plan_rich/vecopt-test-vm > >> >> - I would like to write a vector class as much suitable for PyPy as >> possible, what approach should I take in order to implement it? For >> example, what would suit PyPy JIT the best: >> >> class Vector3d: >> def __init__(a, x, y, z): >> (a.x, a.y, a.z) = x, y, z >> def __add__(a, b): >> return Vector3d(a.x + b.x, a.y+b.y, a.z+b.z) >> >> def add1(a, b): >> (ax, ay, az) = a >> (bx, by, bz) = b >> return [ax + bx, ay + by, az + bz] >> >> def add2(a, b): >> (ax, ay, az) = a >> (bx, by, bz) = b >> return (ax + bx, ay + by, az + bz) >> >> def add3((ax, ay, az), (bx, by, bz)): >> return (ax + bx, ay + by, az + bz) >> >> def add3: >> ??? >> > > I have made some tests with this already. You would need to use the > array module. Python lists would also work, but they leave behind some > instructions that are not well optimized. It is described in this post: > > http://pypyvecopt.blogspot.co.at/2015/08/gsoc-vec-little-brother-of-numpy-array.html > > The missing piece is then the --jit vec_all=1 parameter, that you must > specify on the command line. > > Be aware: > > 1) vec_all=1 parameter might lead to a crash, I have tested it, but it > occurred to me that it is not really ready for production. I'm still > working on this, thus it is disabled by default. > > 2) that with a very low number of vector elements (e.g 3) the overhead > to iterate the first iteration is quite significant. So I do not think > that there is much you can get just executing 1 vector add in parallel > on x86. > >> - Is NumPyPy going to be included with regular PyPy download/install? >> > > NumPyPy is included in a normal PyPy release version > >> Thanks a lot in advance! >> _______________________________________________ >> pypy-dev mailing list >> pypy-dev at python.org >> https://mail.python.org/mailman/listinfo/pypy-dev >> > > Cheers, > Richard > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev From planrichi at gmail.com Tue Oct 20 17:11:08 2015 From: planrichi at gmail.com (Richard Plangger) Date: Tue, 20 Oct 2015 17:11:08 +0200 Subject: [pypy-dev] Automatic SIMD vectorization In-Reply-To: References: <562652A5.70901@gmail.com> Message-ID: <5626598C.8020205@gmail.com> You're welcome. I forgot to mention that I'm happy to help if vec_all crashes. So if you give it a shot and get stuck get in touch with me here (or issues on bitbucket)! Cheers, Richard On 10/20/2015 05:08 PM, Tuom Larsen wrote: > Awesome, thank you! > > On Tue, Oct 20, 2015 at 4:41 PM, Richard Plangger wrote: >> Hi, >> >> glad you liked the post! See the answers below... >> >> On 10/20/2015 04:20 PM, Tuom Larsen wrote: >>> Hello all! >>> >>> I just read with great interest the blog post "Automatic SIMD >>> vectorization support in PyPy". >>> >>> Please, I have a few questions: >>> >>> - Does regular Python code benefit from the vectorization? I mean, the >>> article on one hand says "it is not specifically targeted for the >>> NumPy library" but on the other it says "Any interpreter (written in >>> RPython)". >> >> Speaking about 'regular' Python code, there is potential !BUT! only if >> enough time is spent in numeric code. What I meant in the article was: >> E.g. you have a vector construct in your language (like the one the R >> language has) you could use the optimization to vectorize operations on >> the variables that represent. >> >> Take a look at my test virtual machine implementing a small subset of R. >> https://bitbucket.org/plan_rich/vecopt-test-vm >> >>> >>> - I would like to write a vector class as much suitable for PyPy as >>> possible, what approach should I take in order to implement it? For >>> example, what would suit PyPy JIT the best: >>> >>> class Vector3d: >>> def __init__(a, x, y, z): >>> (a.x, a.y, a.z) = x, y, z >>> def __add__(a, b): >>> return Vector3d(a.x + b.x, a.y+b.y, a.z+b.z) >>> >>> def add1(a, b): >>> (ax, ay, az) = a >>> (bx, by, bz) = b >>> return [ax + bx, ay + by, az + bz] >>> >>> def add2(a, b): >>> (ax, ay, az) = a >>> (bx, by, bz) = b >>> return (ax + bx, ay + by, az + bz) >>> >>> def add3((ax, ay, az), (bx, by, bz)): >>> return (ax + bx, ay + by, az + bz) >>> >>> def add3: >>> ??? >>> >> >> I have made some tests with this already. You would need to use the >> array module. Python lists would also work, but they leave behind some >> instructions that are not well optimized. It is described in this post: >> >> http://pypyvecopt.blogspot.co.at/2015/08/gsoc-vec-little-brother-of-numpy-array.html >> >> The missing piece is then the --jit vec_all=1 parameter, that you must >> specify on the command line. >> >> Be aware: >> >> 1) vec_all=1 parameter might lead to a crash, I have tested it, but it >> occurred to me that it is not really ready for production. I'm still >> working on this, thus it is disabled by default. >> >> 2) that with a very low number of vector elements (e.g 3) the overhead >> to iterate the first iteration is quite significant. So I do not think >> that there is much you can get just executing 1 vector add in parallel >> on x86. >> >>> - Is NumPyPy going to be included with regular PyPy download/install? >>> >> >> NumPyPy is included in a normal PyPy release version >> >>> Thanks a lot in advance! >>> _______________________________________________ >>> pypy-dev mailing list >>> pypy-dev at python.org >>> https://mail.python.org/mailman/listinfo/pypy-dev >>> >> >> Cheers, >> Richard >> _______________________________________________ >> pypy-dev mailing list >> pypy-dev at python.org >> https://mail.python.org/mailman/listinfo/pypy-dev > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev > From pjenvey at underboss.org Tue Oct 20 21:16:10 2015 From: pjenvey at underboss.org (Philip Jenvey) Date: Tue, 20 Oct 2015 12:16:10 -0700 Subject: [pypy-dev] Any Release incoming for pypy 3.x? In-Reply-To: References: Message-ID: This year?s GSoC made significant progress towards 3.3 support. There was some talk of releasing that but I?m not sure it?s quite there yet. Are you looking for that specifically or just an updated version of the 3.2 compatible pypy3? It might be time for another release of the 3.2 version with an upgraded cffi among other things from the default branch (which should be less work than finishing 3.3 support). -- Philip Jenvey > On Oct 20, 2015, at 7:37 AM, Phyo Arkar wrote: > > Any progress in it? > > On Tue, Oct 20, 2015 at 9:02 PM, Phyo Arkar > wrote: > I am waiting to switch to Python3 , but i am not switching without pypy3 moving forward > > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From matti.picus at gmail.com Wed Oct 21 01:09:55 2015 From: matti.picus at gmail.com (Matti Picus) Date: Wed, 21 Oct 2015 10:09:55 +1100 Subject: [pypy-dev] PyPy 15.11 release is imminent In-Reply-To: References: <56208722.5020507@gmail.com> Message-ID: <5626C9C3.4030603@gmail.com> On 21/10/15 00:48, Antonio Cuni wrote: > why using yy.mm instead of just a single increasing int > number like Chrome? > To avoid confusions, we should probably skip pypy 3 and start > releasing from pypy 4. It looks just simpler than 15.11 and friends to me. > > ciao, > Anto > After a short discussion on IRC, it turns out this is the popular view, so the next release wll be called PyPy 4.0.0 Matti From yury at shurup.com Wed Oct 21 07:38:39 2015 From: yury at shurup.com (Yury V. Zaytsev) Date: Wed, 21 Oct 2015 07:38:39 +0200 Subject: [pypy-dev] PyPy 15.11 release is imminent In-Reply-To: <5626C9C3.4030603@gmail.com> References: <56208722.5020507@gmail.com> <5626C9C3.4030603@gmail.com> Message-ID: <1445405919.2612.0.camel@newpride> On Wed, 2015-10-21 at 10:09 +1100, Matti Picus wrote: > > After a short discussion on IRC, it turns out this is the popular > view, so the next release wll be called PyPy 4.0.0 Yay! I also think that it's a great(er) idea ;-) -- Sincerely yours, Yury V. Zaytsev From phyo.arkarlwin at gmail.com Wed Oct 21 08:04:41 2015 From: phyo.arkarlwin at gmail.com (Phyo Arkar) Date: Wed, 21 Oct 2015 06:04:41 +0000 Subject: [pypy-dev] PyPy 15.11 release is imminent In-Reply-To: <5626C9C3.4030603@gmail.com> References: <56208722.5020507@gmail.com> <5626C9C3.4030603@gmail.com> Message-ID: Just make a pypy that automatically compatible between python2 nd 3 and name it pyoy5 :P I know it is not feasible. On Wed, Oct 21, 2015, 5:40 AM Matti Picus wrote: > On 21/10/15 00:48, Antonio Cuni wrote: > > why using yy.mm instead of just a single increasing int > > number like Chrome? > > To avoid confusions, we should probably skip pypy 3 and start > > releasing from pypy 4. It looks just simpler than 15.11 and friends to > me. > > > > ciao, > > Anto > > > > After a short discussion on IRC, it turns out this is the popular view, > so the next release wll be called PyPy 4.0.0 > Matti > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From matti at picus.org.il Wed Oct 21 14:46:30 2015 From: matti at picus.org.il (Matti Picus) Date: Wed, 21 Oct 2015 23:46:30 +1100 Subject: [pypy-dev] Please try the PyPy 4.0.0 release candidates Message-ID: <56278926.30907@picus.org.il> The first translation of the upcoming PyPy 4.0.0 release is now available at http://buildbot.pypy.org/nightly/release-4.0.x Also, contributions to the release notice would be appreciated, it can be found at http://pypy.readthedocs.org/en/latest/release-4.0.0.html and the source at https://bitbucket.org/pypy/pypy/src/default/pypy/doc/release-4.0.0.rst Thanks Matti From ndbecker2 at gmail.com Wed Oct 21 16:53:47 2015 From: ndbecker2 at gmail.com (Neal Becker) Date: Wed, 21 Oct 2015 10:53:47 -0400 Subject: [pypy-dev] Please try the PyPy 4.0.0 release candidates References: <56278926.30907@picus.org.il> Message-ID: Matti Picus wrote: > The first translation of the upcoming PyPy 4.0.0 release is now available > at http://buildbot.pypy.org/nightly/release-4.0.x > > Also, contributions to the release notice would be appreciated, it can > be found at > http://pypy.readthedocs.org/en/latest/release-4.0.0.html > and the source at > https://bitbucket.org/pypy/pypy/src/default/pypy/doc/release-4.0.0.rst > > Thanks > Matti Where? Are these in the nightlies? From me at manueljacob.de Wed Oct 21 23:16:20 2015 From: me at manueljacob.de (Manuel Jacob) Date: Wed, 21 Oct 2015 23:16:20 +0200 Subject: [pypy-dev] Any Release incoming for pypy 3.x? In-Reply-To: References: Message-ID: Hi, I planned to do a release of the Python 3.2-compatible version after the PyPy 2.6.1 release but didn't have time to do so until recently. But now that the PyPy 4.0.0 release is imminent, it makes sense to wait a week longer and then do a release of the Python 3.2-compatible version based on PyPy 4.0.0. Python 3.3 support is almost complete feature-wise. However, there are still a lot of failing tests which are often a bit subtle to fix. I hope to be able to invest more time in the next weeks. -Manuel On 2015-10-20 21:16, Philip Jenvey wrote: > This year?s GSoC made significant progress towards 3.3 support. There > was some talk of releasing that but I?m not sure it?s quite there yet. > > Are you looking for that specifically or just an updated version of > the 3.2 compatible pypy3? > > It might be time for another release of the 3.2 version with an > upgraded cffi among other things from the default branch (which should > be less work than finishing 3.3 support). > > -- > Philip Jenvey > >> On Oct 20, 2015, at 7:37 AM, Phyo Arkar >> wrote: >> >> Any progress in it? >> >> On Tue, Oct 20, 2015 at 9:02 PM, Phyo Arkar > > wrote: >> I am waiting to switch to Python3 , but i am not switching without >> pypy3 moving forward >> >> _______________________________________________ >> pypy-dev mailing list >> pypy-dev at python.org >> https://mail.python.org/mailman/listinfo/pypy-dev > > > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev From rymg19 at gmail.com Thu Oct 22 00:47:46 2015 From: rymg19 at gmail.com (Ryan Gonzalez) Date: Wed, 21 Oct 2015 17:47:46 -0500 Subject: [pypy-dev] PyPy doesn't implement the most important module! Message-ID: ryan at DevPC-LX:~$ ipython Python 2.7.6 (default, Mar 22 2014, 22:59:56) Type "copyright", "credits" or "license" for more information. IPython 1.1.0 -- An enhanced Interactive Python. ? -> Introduction and overview of IPython's features. %quickref -> Quick reference. help -> Python's own help system. object? -> Details about 'object', use 'object??' for extra details. In [1]: import __hello__ Hello world... In [2]: Do you really want to exit ([y]/n)? y ryan at DevPC-LX:~$ pypy-ipython Python 2.7.8 (1a994c305af0, Nov 15 2014, 00:43:41) Type "copyright", "credits" or "license" for more information. IPython 4.0.0 -- An enhanced Interactive Python. ? -> Introduction and overview of IPython's features. %quickref -> Quick reference. help -> Python's own help system. object? -> Details about 'object', use 'object??' for extra details. In [1]: import __hello__ --------------------------------------------------------------------------- ImportError Traceback (most recent call last) in () ----> 1 import __hello__ ImportError: No module named __hello__ In [2]: Do you really want to exit ([y]/n)? y ryan at DevPC-LX:~$ -- Ryan [ERROR]: Your autotools build scripts are 200 lines longer than your program. Something?s wrong. http://kirbyfan64.github.io/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From techtonik at gmail.com Thu Oct 22 09:19:23 2015 From: techtonik at gmail.com (anatoly techtonik) Date: Thu, 22 Oct 2015 10:19:23 +0300 Subject: [pypy-dev] PyPy 15.11 release is imminent In-Reply-To: References: <56208722.5020507@gmail.com> <5626C9C3.4030603@gmail.com> Message-ID: On Wed, Oct 21, 2015 at 9:04 AM, Phyo Arkar wrote: > Just make a pypy that automatically compatible between python2 nd 3 and > name it pyoy5 :P > > I know it is not feasible. > Run two in parallel and kill the one that gives error first. -- anatoly t -------------- next part -------------- An HTML attachment was scrubbed... URL: From phyo.arkarlwin at gmail.com Thu Oct 22 09:21:21 2015 From: phyo.arkarlwin at gmail.com (Phyo Arkar) Date: Thu, 22 Oct 2015 13:51:21 +0630 Subject: [pypy-dev] Any Release incoming for pypy 3.x? In-Reply-To: References: Message-ID: Thanks a lot. PyPy3 is lagging behind by a lot so i just pinged. PyPy3 users will be particularly interested in 3.4-3.5x due to async-await , yield from and many other powerful features. As tornado now heading towards Python3.5 async-await , these things become quite interesting. On Thu, Oct 22, 2015 at 3:46 AM, Manuel Jacob wrote: > Hi, > > I planned to do a release of the Python 3.2-compatible version after the > PyPy 2.6.1 release but didn't have time to do so until recently. But now > that the PyPy 4.0.0 release is imminent, it makes sense to wait a week > longer and then do a release of the Python 3.2-compatible version based on > PyPy 4.0.0. > > Python 3.3 support is almost complete feature-wise. However, there are > still a lot of failing tests which are often a bit subtle to fix. I hope > to be able to invest more time in the next weeks. > > -Manuel > > On 2015-10-20 21:16, Philip Jenvey wrote: > >> This year?s GSoC made significant progress towards 3.3 support. There >> was some talk of releasing that but I?m not sure it?s quite there yet. >> >> Are you looking for that specifically or just an updated version of >> the 3.2 compatible pypy3? >> >> It might be time for another release of the 3.2 version with an >> upgraded cffi among other things from the default branch (which should >> be less work than finishing 3.3 support). >> >> -- >> Philip Jenvey >> >> On Oct 20, 2015, at 7:37 AM, Phyo Arkar wrote: >>> >>> Any progress in it? >>> >>> On Tue, Oct 20, 2015 at 9:02 PM, Phyo Arkar >> > wrote: >>> I am waiting to switch to Python3 , but i am not switching without pypy3 >>> moving forward >>> >>> _______________________________________________ >>> pypy-dev mailing list >>> pypy-dev at python.org >>> https://mail.python.org/mailman/listinfo/pypy-dev >>> >> >> >> _______________________________________________ >> pypy-dev mailing list >> pypy-dev at python.org >> https://mail.python.org/mailman/listinfo/pypy-dev >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From techtonik at gmail.com Thu Oct 22 09:24:06 2015 From: techtonik at gmail.com (anatoly techtonik) Date: Thu, 22 Oct 2015 10:24:06 +0300 Subject: [pypy-dev] PyPy doesn't implement the most important module! In-Reply-To: References: Message-ID: On Thu, Oct 22, 2015 at 1:47 AM, Ryan Gonzalez wrote: > ----> 1 import __hello__ > > ImportError: No module named __hello__ > PyPy is shy. -- anatoly t. -------------- next part -------------- An HTML attachment was scrubbed... URL: From fijall at gmail.com Thu Oct 22 10:14:58 2015 From: fijall at gmail.com (Maciej Fijalkowski) Date: Thu, 22 Oct 2015 10:14:58 +0200 Subject: [pypy-dev] PyPy doesn't implement the most important module! In-Reply-To: References: Message-ID: pypy will not automatically pick up packages from site-packages of cpython. you need to install them separately, preferably in a virtualenv. On Thu, Oct 22, 2015 at 12:47 AM, Ryan Gonzalez wrote: > ryan at DevPC-LX:~$ ipython > Python 2.7.6 (default, Mar 22 2014, 22:59:56) > Type "copyright", "credits" or "license" for more information. > > IPython 1.1.0 -- An enhanced Interactive Python. > ? -> Introduction and overview of IPython's features. > %quickref -> Quick reference. > help -> Python's own help system. > object? -> Details about 'object', use 'object??' for extra details. > > In [1]: import __hello__ > Hello world... > > In [2]: > Do you really want to exit ([y]/n)? y > ryan at DevPC-LX:~$ pypy-ipython > Python 2.7.8 (1a994c305af0, Nov 15 2014, 00:43:41) > Type "copyright", "credits" or "license" for more information. > > IPython 4.0.0 -- An enhanced Interactive Python. > ? -> Introduction and overview of IPython's features. > %quickref -> Quick reference. > help -> Python's own help system. > object? -> Details about 'object', use 'object??' for extra details. > > In [1]: import __hello__ > --------------------------------------------------------------------------- > ImportError Traceback (most recent call last) > in () > ----> 1 import __hello__ > > ImportError: No module named __hello__ > > In [2]: > > Do you really want to exit ([y]/n)? y > ryan at DevPC-LX:~$ > > -- > Ryan > [ERROR]: Your autotools build scripts are 200 lines longer than your > program. Something?s wrong. > http://kirbyfan64.github.io/ > > > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev > From william.leslie.ttg at gmail.com Thu Oct 22 10:48:00 2015 From: william.leslie.ttg at gmail.com (William ML Leslie) Date: Thu, 22 Oct 2015 19:48:00 +1100 Subject: [pypy-dev] PyPy doesn't implement the most important module! In-Reply-To: References: Message-ID: On 22 October 2015 at 19:14, Maciej Fijalkowski wrote: > pypy will not automatically pick up packages from site-packages of > cpython. you need to install them separately, preferably in a > virtualenv. It comes with cpython, as part of the 'frozen' module system. https://docs.python.org/2/library/imp.html#imp.PY_FROZEN -- William Leslie Notice: Likely much of this email is, by the nature of copyright, covered under copyright law. You absolutely MAY reproduce any part of it in accordance with the copyright law of the nation you are reading this in. Any attempt to DENY YOU THOSE RIGHTS would be illegal without prior contractual agreement. From arigo at tunes.org Thu Oct 22 14:06:39 2015 From: arigo at tunes.org (Armin Rigo) Date: Thu, 22 Oct 2015 14:06:39 +0200 Subject: [pypy-dev] PyPy doesn't implement the most important module! In-Reply-To: References: Message-ID: Hi, On Thu, Oct 22, 2015 at 10:48 AM, William ML Leslie wrote: > It comes with cpython, as part of the 'frozen' module system. We can add a file ``lib_pypy/__hello__.py`` containing ``print 'Hello world...'``. That seems highly pointless, though, if we don't support frozen modules anyway. A bient?t, Armin. From phyo.arkarlwin at gmail.com Fri Oct 23 06:54:24 2015 From: phyo.arkarlwin at gmail.com (Phyo Arkar) Date: Fri, 23 Oct 2015 17:24:24 +0630 Subject: [pypy-dev] Any Release incoming for pypy 3.x? In-Reply-To: References: Message-ID: Jacob, I think Python 3.5 going to be the version many will be switching , want to know if it is beneficial to jump directly to that? as PyPy3 lacks behind a lot in development vs Python 2 , a skip would be less work? On Thu, Oct 22, 2015 at 3:46 AM, Manuel Jacob wrote: > Hi, > > I planned to do a release of the Python 3.2-compatible version after the > PyPy 2.6.1 release but didn't have time to do so until recently. But now > that the PyPy 4.0.0 release is imminent, it makes sense to wait a week > longer and then do a release of the Python 3.2-compatible version based on > PyPy 4.0.0. > > Python 3.3 support is almost complete feature-wise. However, there are > still a lot of failing tests which are often a bit subtle to fix. I hope > to be able to invest more time in the next weeks. > > -Manuel > > On 2015-10-20 21:16, Philip Jenvey wrote: > >> This year?s GSoC made significant progress towards 3.3 support. There >> was some talk of releasing that but I?m not sure it?s quite there yet. >> >> Are you looking for that specifically or just an updated version of >> the 3.2 compatible pypy3? >> >> It might be time for another release of the 3.2 version with an >> upgraded cffi among other things from the default branch (which should >> be less work than finishing 3.3 support). >> >> -- >> Philip Jenvey >> >> On Oct 20, 2015, at 7:37 AM, Phyo Arkar wrote: >>> >>> Any progress in it? >>> >>> On Tue, Oct 20, 2015 at 9:02 PM, Phyo Arkar >> > wrote: >>> I am waiting to switch to Python3 , but i am not switching without pypy3 >>> moving forward >>> >>> _______________________________________________ >>> pypy-dev mailing list >>> pypy-dev at python.org >>> https://mail.python.org/mailman/listinfo/pypy-dev >>> >> >> >> _______________________________________________ >> pypy-dev mailing list >> pypy-dev at python.org >> https://mail.python.org/mailman/listinfo/pypy-dev >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tritium-list at sdamon.com Sat Oct 24 18:25:40 2015 From: tritium-list at sdamon.com (Alexander Walters) Date: Sat, 24 Oct 2015 18:25:40 -0400 Subject: [pypy-dev] PyPy 15.11 release is imminent In-Reply-To: <5626C9C3.4030603@gmail.com> References: <56208722.5020507@gmail.com> <5626C9C3.4030603@gmail.com> Message-ID: <562C0564.9040307@sdamon.com> On 10/20/2015 19:09, Matti Picus wrote: > After a short discussion on IRC, it turns out this is the popular > view, so the next release wll be called PyPy 4.0.0 > Matti Oh thank goodness. From phyo.arkarlwin at gmail.com Mon Oct 26 02:47:33 2015 From: phyo.arkarlwin at gmail.com (Phyo Arkar) Date: Mon, 26 Oct 2015 13:17:33 +0630 Subject: [pypy-dev] PyPy 15.11 release is imminent In-Reply-To: <562C0564.9040307@sdamon.com> References: <56208722.5020507@gmail.com> <5626C9C3.4030603@gmail.com> <562C0564.9040307@sdamon.com> Message-ID: PyPy 4.0.0 sounds a lot better but how we will name newer versions of PyPy3 ? On Sun, Oct 25, 2015 at 4:55 AM, Alexander Walters wrote: > On 10/20/2015 19:09, Matti Picus wrote: > >> After a short discussion on IRC, it turns out this is the popular view, >> so the next release wll be called PyPy 4.0.0 >> Matti >> > Oh thank goodness. > > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tritium-list at sdamon.com Mon Oct 26 02:48:49 2015 From: tritium-list at sdamon.com (Alexander Walters) Date: Mon, 26 Oct 2015 02:48:49 -0400 Subject: [pypy-dev] PyPy 15.11 release is imminent In-Reply-To: References: <56208722.5020507@gmail.com> <5626C9C3.4030603@gmail.com> <562C0564.9040307@sdamon.com> Message-ID: <562DCCD1.9070707@sdamon.com> PyPy3 4.0.0 On 10/26/2015 02:47, Phyo Arkar wrote: > PyPy 4.0.0 sounds a lot better but how we will name newer versions of > PyPy3 ? > > On Sun, Oct 25, 2015 at 4:55 AM, Alexander Walters > > wrote: > > On 10/20/2015 19:09, Matti Picus wrote: > > After a short discussion on IRC, it turns out this is the > popular view, so the next release wll be called PyPy 4.0.0 > Matti > > Oh thank goodness. > > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From phyo.arkarlwin at gmail.com Mon Oct 26 02:50:34 2015 From: phyo.arkarlwin at gmail.com (Phyo Arkar) Date: Mon, 26 Oct 2015 13:20:34 +0630 Subject: [pypy-dev] PyPy 15.11 release is imminent In-Reply-To: <562DCCD1.9070707@sdamon.com> References: <56208722.5020507@gmail.com> <5626C9C3.4030603@gmail.com> <562C0564.9040307@sdamon.com> <562DCCD1.9070707@sdamon.com> Message-ID: yeah , that sounds weird. On Mon, Oct 26, 2015 at 1:18 PM, Alexander Walters wrote: > PyPy3 4.0.0 > > > On 10/26/2015 02:47, Phyo Arkar wrote: > > PyPy 4.0.0 sounds a lot better but how we will name newer versions of > PyPy3 ? > > On Sun, Oct 25, 2015 at 4:55 AM, Alexander Walters < > tritium-list at sdamon.com> wrote: > >> On 10/20/2015 19:09, Matti Picus wrote: >> >>> After a short discussion on IRC, it turns out this is the popular view, >>> so the next release wll be called PyPy 4.0.0 >>> Matti >>> >> Oh thank goodness. >> >> _______________________________________________ >> pypy-dev mailing list >> pypy-dev at python.org >> https://mail.python.org/mailman/listinfo/pypy-dev >> > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fijall at gmail.com Tue Oct 27 04:25:11 2015 From: fijall at gmail.com (Maciej Fijalkowski) Date: Tue, 27 Oct 2015 09:25:11 +0100 Subject: [pypy-dev] PyPy 15.11 release is imminent In-Reply-To: References: <56208722.5020507@gmail.com> <5626C9C3.4030603@gmail.com> <562C0564.9040307@sdamon.com> <562DCCD1.9070707@sdamon.com> Message-ID: The problem is that everything sounds weird and pypy3 4.0 sounds less weird than other options On Mon, Oct 26, 2015 at 7:50 AM, Phyo Arkar wrote: > yeah , that sounds weird. > > On Mon, Oct 26, 2015 at 1:18 PM, Alexander Walters > wrote: >> >> PyPy3 4.0.0 >> >> >> On 10/26/2015 02:47, Phyo Arkar wrote: >> >> PyPy 4.0.0 sounds a lot better but how we will name newer versions of >> PyPy3 ? >> >> On Sun, Oct 25, 2015 at 4:55 AM, Alexander Walters >> wrote: >>> >>> On 10/20/2015 19:09, Matti Picus wrote: >>>> >>>> After a short discussion on IRC, it turns out this is the popular view, >>>> so the next release wll be called PyPy 4.0.0 >>>> Matti >>> >>> Oh thank goodness. >>> >>> _______________________________________________ >>> pypy-dev mailing list >>> pypy-dev at python.org >>> https://mail.python.org/mailman/listinfo/pypy-dev >> >> >> > > > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev > From g3orge.app at gmail.com Fri Oct 30 13:46:08 2015 From: g3orge.app at gmail.com (George Papanikolaou) Date: Fri, 30 Oct 2015 19:46:08 +0200 Subject: [pypy-dev] Diploma Thesis on Pypy Message-ID: Hello, I'm a CS student at the University of Patras in Greece, and I'm really interested in doing my thesis on programming language theory, compilers JIT optimization etc. I find your project really interesting and I'd like you to advice me if there is any potential for my thesis here. A contribution maybe. Is it feasible? Thanks in advance for all answers and considerations, George -- George 'papanikge' Papanikolaou From planrichi at gmail.com Sat Oct 31 05:57:10 2015 From: planrichi at gmail.com (Richard Plangger) Date: Sat, 31 Oct 2015 10:57:10 +0100 Subject: [pypy-dev] Diploma Thesis on Pypy In-Reply-To: References: Message-ID: <56349076.9090803@gmail.com> Hi there, I finished my thesis recently implementing a vectorizing optimization. So I guess it is feasible :). My advice would be to build the VM and make some baby steps in the test environment. This might sound weird, but the development is very different to what you do at university (at least in austria) and also very different to most projects I know. I think there are many possible ways you could improve PyPy. It is hard to give some advice on exactly what you should implement. After all you need to be motivated until you finish your thesis. It would probably be better to think about a topic (VM X does Y), see if PyPy implemented Y (you can of course ask us, it is hard to get around in such a big project in the beginning), (if not) reflect if it gives any benefit for the VM and then implement it. Here is a list of potential projects (might get you started): http://pypy.readthedocs.org/en/latest/project-ideas.html What really would help PyPy (might be more engineering than a 'researchy' topic) is to implement the Python 3 spec (e.g. 3.3,3.4,3.5). You can also join IRC and we will see if we can help to choose a topic. Cheers, Richard On 10/30/2015 06:46 PM, George Papanikolaou wrote: > Hello, > > I'm a CS student at the University of Patras in Greece, and I'm really > interested in doing my thesis on programming language theory, compilers JIT > optimization etc. I find your project really interesting and I'd like you to > advice me if there is any potential for my thesis here. A contribution maybe. > Is it feasible? > > Thanks in advance for all answers and considerations, > George > From g3orge.app at gmail.com Sat Oct 31 16:37:19 2015 From: g3orge.app at gmail.com (George Papanikolaou) Date: Sat, 31 Oct 2015 22:37:19 +0200 Subject: [pypy-dev] Diploma Thesis on Pypy In-Reply-To: <56349076.9090803@gmail.com> References: <56349076.9090803@gmail.com> Message-ID: Hi, First of all thanks for answering, yes I'll pop on IRC soon. My thesis will be for an undergraduate degree (https://en.wikipedia.org/wiki/Engineer%27s_degree#Greece) so I guess I can go for an engineering project more than a research-y one. How hard would implementing the 3.0 spec be? It sounds tedious but pretty standard. I'll get started right away with your advice in order to get familiar with the VM, the environment and coding nuisances in general. Regards, George On Sat, Oct 31, 2015 at 11:57 AM, Richard Plangger wrote: > Hi there, > > I finished my thesis recently implementing a vectorizing optimization. > So I guess it is feasible :). My advice would be to build the VM and > make some baby steps in the test environment. This might sound weird, > but the development is very different to what you do at university (at > least in austria) and also very different to most projects I know. > > I think there are many possible ways you could improve PyPy. It is hard > to give some advice on exactly what you should implement. After all you > need to be motivated until you finish your thesis. > It would probably be better to think about a topic (VM X does Y), see if > PyPy implemented Y (you can of course ask us, it is hard to get around > in such a big project in the beginning), (if not) reflect if it gives > any benefit for the VM and then implement it. > > Here is a list of potential projects (might get you started): > http://pypy.readthedocs.org/en/latest/project-ideas.html > What really would help PyPy (might be more engineering than a > 'researchy' topic) is to implement the Python 3 spec (e.g. 3.3,3.4,3.5). > You can also join IRC and we will see if we can help to choose a topic. > > Cheers, > Richard > > On 10/30/2015 06:46 PM, George Papanikolaou wrote: >> Hello, >> >> I'm a CS student at the University of Patras in Greece, and I'm really >> interested in doing my thesis on programming language theory, compilers JIT >> optimization etc. I find your project really interesting and I'd like you to >> advice me if there is any potential for my thesis here. A contribution maybe. >> Is it feasible? >> >> Thanks in advance for all answers and considerations, >> George >> > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev -- George 'papanikge' Papanikolaou http://www.5slingshots.com/ From luciano at ramalho.org Sat Oct 31 20:04:08 2015 From: luciano at ramalho.org (Luciano Ramalho) Date: Sat, 31 Oct 2015 22:04:08 -0200 Subject: [pypy-dev] Diploma Thesis on Pypy In-Reply-To: References: <56349076.9090803@gmail.com> Message-ID: On Sat, Oct 31, 2015 at 6:37 PM, George Papanikolaou wrote: > First of all thanks for answering, yes I'll pop on IRC soon. > My thesis will be for an undergraduate degree > (https://en.wikipedia.org/wiki/Engineer%27s_degree#Greece) so I guess I can go > for an engineering project more than a research-y one. How hard would > implementing the 3.0 spec be? It sounds tedious but pretty standard. I'll get > started right away with your advice in order to get familiar with the VM, the > environment and coding nuisances in general. Important stuff todo, and not tedious at all. Python 3.3 introduced the `yield from` construct and other changes in generator semantics. The asyncio library depends on them and many Pypy users would love to use it. Cheers, Luciano > > Regards, > George > > On Sat, Oct 31, 2015 at 11:57 AM, Richard Plangger wrote: >> Hi there, >> >> I finished my thesis recently implementing a vectorizing optimization. >> So I guess it is feasible :). My advice would be to build the VM and >> make some baby steps in the test environment. This might sound weird, >> but the development is very different to what you do at university (at >> least in austria) and also very different to most projects I know. >> >> I think there are many possible ways you could improve PyPy. It is hard >> to give some advice on exactly what you should implement. After all you >> need to be motivated until you finish your thesis. >> It would probably be better to think about a topic (VM X does Y), see if >> PyPy implemented Y (you can of course ask us, it is hard to get around >> in such a big project in the beginning), (if not) reflect if it gives >> any benefit for the VM and then implement it. >> >> Here is a list of potential projects (might get you started): >> http://pypy.readthedocs.org/en/latest/project-ideas.html >> What really would help PyPy (might be more engineering than a >> 'researchy' topic) is to implement the Python 3 spec (e.g. 3.3,3.4,3.5). >> You can also join IRC and we will see if we can help to choose a topic. >> >> Cheers, >> Richard >> >> On 10/30/2015 06:46 PM, George Papanikolaou wrote: >>> Hello, >>> >>> I'm a CS student at the University of Patras in Greece, and I'm really >>> interested in doing my thesis on programming language theory, compilers JIT >>> optimization etc. I find your project really interesting and I'd like you to >>> advice me if there is any potential for my thesis here. A contribution maybe. >>> Is it feasible? >>> >>> Thanks in advance for all answers and considerations, >>> George >>> >> _______________________________________________ >> pypy-dev mailing list >> pypy-dev at python.org >> https://mail.python.org/mailman/listinfo/pypy-dev > > > > -- > George 'papanikge' Papanikolaou > http://www.5slingshots.com/ > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev -- Luciano Ramalho | Author of Fluent Python (O'Reilly, 2015) | http://shop.oreilly.com/product/0636920032519.do | Professor em: http://python.pro.br | Twitter: @ramalhoorg