From tracker at bugs.pypy.org Sat Oct 1 14:55:35 2011 From: tracker at bugs.pypy.org (Pavel) Date: Sat, 01 Oct 2011 12:55:35 +0000 Subject: [pypy-issue] [issue890] bytearray.find is terribly slow In-Reply-To: <1317473735.62.0.641929196398.issue890@bugs.pypy.org> Message-ID: <1317473735.62.0.641929196398.issue890@bugs.pypy.org> New submission from Pavel : [~/Python/dict] $ python2 -mtimeit -s 's = bytearray (b"0" * (1 << 20))' 's.find (b"t", 3, 10)' 1000000 loops, best of 3: 0.32 usec per loop [~/Python/dict] $ python3 -mtimeit -s 's = bytearray (b"0" * (1 << 20))' 's.find (b"t", 3, 10)' 1000000 loops, best of 3: 0.291 usec per loop [~/Python/dict] $ pypy -mtimeit -s 's = bytearray (b"0" * (1 << 20))' 's.find (b"t", 3, 10)' 1000 loops, best of 3: 1.2 msec per loop [~/Python/dict] $ ---------- messages: 3224 nosy: pypy-issue priority: bug status: unread title: bytearray.find is terribly slow ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Mon Oct 3 07:44:43 2011 From: tracker at bugs.pypy.org (Justin Peel) Date: Mon, 03 Oct 2011 05:44:43 +0000 Subject: [pypy-issue] [issue891] reduce and filter aren't jitted. map isn't jitted for built-in functions In-Reply-To: <1317620683.16.0.391298518561.issue891@bugs.pypy.org> Message-ID: <1317620683.16.0.391298518561.issue891@bugs.pypy.org> New submission from Justin Peel : Map is not jitted for built-in functions. $ pypy -m timeit -s "x = range(256)" "map(abs,x)" 10000 loops, best of 3: 27.1 usec per loop $ pypy -m timeit -s "x = range(256); _abs = lambda y: abs(y)" "map(_abs, x)" 100000 loops, best of 3: 10.6 usec per loop $ pypy -m timeit -s "x = range(256)" "[abs(y) for y in x]" 100000 loops, best of 3: 11 usec per loop As you can see, passing non-built-in function that calls a built-in is jitted by map, but not by calling using the built-in function. Filter and reduce are both not jitted at all. $ pypy -m timeit -s "x = range(256); f = lambda x: x > 100" "filter(f, x)" 10000 loops, best of 3: 75.7 usec per loop $ pypy -m timeit -s "x = range(256); f = lambda x: x > 100" "[y for y in x if f(y)]" 100000 loops, best of 3: 6.81 usec per loop $ pypy -m timeit -s "x = range(256)" "reduce(max, x)" 10000 loops, best of 3: 49.9 usec per loop $ python -m timeit -s "x = range(256)" "reduce(max, x)" 10000 loops, best of 3: 30 usec per loop Notice that for filter I compared with a list expression equivalent (or nearly equivalent) while for reduce I just compared with CPython (2.7.1). We have a choice here: do we make these all jitted in RPython or do we want to make them app level? ---------- messages: 3225 nosy: justinpeel, pypy-issue priority: feature status: unread title: reduce and filter aren't jitted. map isn't jitted for built-in functions ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Mon Oct 3 12:58:44 2011 From: tracker at bugs.pypy.org (Vetoshkin Nikita) Date: Mon, 03 Oct 2011 10:58:44 +0000 Subject: [pypy-issue] [issue885] Lack of libffi5 on Ubuntu 11.10 stops PyPy running In-Reply-To: <1317176080.11.0.207082241886.issue885@bugs.pypy.org> Message-ID: <1317639524.3.0.299544501188.issue885@bugs.pypy.org> Vetoshkin Nikita added the comment: e.g. fedora's libffi-dev doesn't have .a file :( ---------- nosy: +nekto0n ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Mon Oct 3 15:42:14 2011 From: tracker at bugs.pypy.org (Vetoshkin Nikita) Date: Mon, 03 Oct 2011 13:42:14 +0000 Subject: [pypy-issue] [issue846] TypeError: type 'basestring' is not an acceptable base class In-Reply-To: <1314298597.06.0.673686395864.issue846@bugs.pypy.org> Message-ID: <1317649334.86.0.0359700635297.issue846@bugs.pypy.org> Vetoshkin Nikita added the comment: Seems like a cpython bug http://docs.python.org/library/functions.html#basestring > This abstract type is the superclass for str and unicode. It cannot be called or instantiated... ---------- nosy: +nekto0n status: unread -> chatting ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Mon Oct 3 15:45:11 2011 From: tracker at bugs.pypy.org (Victor Costan) Date: Mon, 03 Oct 2011 13:45:11 +0000 Subject: [pypy-issue] [issue885] Lack of libffi5 on Ubuntu 11.10 stops PyPy running In-Reply-To: <1317176080.11.0.207082241886.issue885@bugs.pypy.org> Message-ID: <1317649511.04.0.583001135251.issue885@bugs.pypy.org> Victor Costan added the comment: For people who get here because they need pypy on Ubuntu 11.10, here are x86 and x86_64 binaries: http://people.csail.mit.edu/costan/pypy/ I built the binaries from pypy trunk. The 1.6 source doesn't build on my Oneiric box because of compilation errors. ---------- nosy: +pwnall ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Mon Oct 3 15:46:52 2011 From: tracker at bugs.pypy.org (Vetoshkin Nikita) Date: Mon, 03 Oct 2011 13:46:52 +0000 Subject: [pypy-issue] [issue846] TypeError: type 'basestring' is not an acceptable base class In-Reply-To: <1314298597.06.0.673686395864.issue846@bugs.pypy.org> Message-ID: <1317649612.33.0.273895183125.issue846@bugs.pypy.org> Vetoshkin Nikita added the comment: hm.... cannot be called or instantiated, but can be subclassed... ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Mon Oct 3 15:51:02 2011 From: tracker at bugs.pypy.org (Amaury Forgeot d Arc) Date: Mon, 03 Oct 2011 13:51:02 +0000 Subject: [pypy-issue] [issue846] TypeError: type 'basestring' is not an acceptable base class In-Reply-To: <1314298597.06.0.673686395864.issue846@bugs.pypy.org> Message-ID: <1317649862.35.0.0665230050178.issue846@bugs.pypy.org> Amaury Forgeot d Arc added the comment: ...unless you provide a __new__ which returns something else! ---------- nosy: +afa ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Mon Oct 3 16:12:20 2011 From: tracker at bugs.pypy.org (Vetoshkin Nikita) Date: Mon, 03 Oct 2011 14:12:20 +0000 Subject: [pypy-issue] [issue846] TypeError: type 'basestring' is not an acceptable base class In-Reply-To: <1314298597.06.0.673686395864.issue846@bugs.pypy.org> Message-ID: <1317651140.6.0.0449055576626.issue846@bugs.pypy.org> Vetoshkin Nikita added the comment: attached patch, is it correct? ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Mon Oct 3 16:18:39 2011 From: tracker at bugs.pypy.org (Amaury Forgeot d Arc) Date: Mon, 03 Oct 2011 14:18:39 +0000 Subject: [pypy-issue] [issue846] TypeError: type 'basestring' is not an acceptable base class In-Reply-To: <1314298597.06.0.673686395864.issue846@bugs.pypy.org> Message-ID: <1317651519.56.0.171253412301.issue846@bugs.pypy.org> Amaury Forgeot d Arc added the comment: Maybe, please add tests: one that fails when a subclass of basestring has no __new__ one that passes when the subclass defines __new__. ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Mon Oct 3 16:19:46 2011 From: tracker at bugs.pypy.org (Vetoshkin Nikita) Date: Mon, 03 Oct 2011 14:19:46 +0000 Subject: [pypy-issue] [issue846] TypeError: type 'basestring' is not an acceptable base class In-Reply-To: <1314298597.06.0.673686395864.issue846@bugs.pypy.org> Message-ID: <1317651586.38.0.0546636935778.issue846@bugs.pypy.org> Vetoshkin Nikita added the comment: where do I put these tests? ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Mon Oct 3 16:32:57 2011 From: tracker at bugs.pypy.org (Amaury Forgeot d Arc) Date: Mon, 03 Oct 2011 14:32:57 +0000 Subject: [pypy-issue] [issue846] TypeError: type 'basestring' is not an acceptable base class In-Reply-To: <1314298597.06.0.673686395864.issue846@bugs.pypy.org> Message-ID: <1317652377.79.0.190785349965.issue846@bugs.pypy.org> Amaury Forgeot d Arc added the comment: Since you modified pypy/objspace/std/basestring.py, tests should go in a new file pypy/objspace/std/test/test_basestring.py. Look at test_stringobject.py, in AppTestStringObject there is already a test_subclass() which tests a subclass of str. Run the tests with: ./test_all.py objspace/std/test/test_basestring.py ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Mon Oct 3 17:57:03 2011 From: tracker at bugs.pypy.org (Vetoshkin Nikita) Date: Mon, 03 Oct 2011 15:57:03 +0000 Subject: [pypy-issue] [issue846] TypeError: type 'basestring' is not an acceptable base class In-Reply-To: <1314298597.06.0.673686395864.issue846@bugs.pypy.org> Message-ID: <1317657423.62.0.0150679086272.issue846@bugs.pypy.org> Vetoshkin Nikita added the comment: updated patch: - added test ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Mon Oct 3 18:03:35 2011 From: tracker at bugs.pypy.org (Armin Rigo) Date: Mon, 03 Oct 2011 16:03:35 +0000 Subject: [pypy-issue] [issue846] TypeError: type 'basestring' is not an acceptable base class In-Reply-To: <1314298597.06.0.673686395864.issue846@bugs.pypy.org> Message-ID: <1317657815.73.0.182125499203.issue846@bugs.pypy.org> Armin Rigo added the comment: Can we also look in nltk to know why, on earth why, they are subclassing 'basestring' and adding a __new__ that returns something else? It may be the case that there is an obscure reason to do that --- instead of, say, subclassing 'object' and adding a __new__ that returns something else. If there is then we should make sure that we copy that obscure reason's behavior too... If there isn't, then well, I suppose we can just add the patch anyway, just because. ---------- nosy: +arigo ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Mon Oct 3 18:08:43 2011 From: tracker at bugs.pypy.org (Amaury Forgeot d Arc) Date: Mon, 03 Oct 2011 16:08:43 +0000 Subject: [pypy-issue] [issue846] TypeError: type 'basestring' is not an acceptable base class In-Reply-To: <1314298597.06.0.673686395864.issue846@bugs.pypy.org> Message-ID: <1317658123.58.0.156690503007.issue846@bugs.pypy.org> Amaury Forgeot d Arc added the comment: Here is the code: http://code.google.com/p/nltk/source/browse/trunk/nltk/nltk/sourcedstring.py#282 ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Mon Oct 3 18:21:22 2011 From: tracker at bugs.pypy.org (Armin Rigo) Date: Mon, 03 Oct 2011 16:21:22 +0000 Subject: [pypy-issue] [issue884] Diabolical Multicore Thread Performance In-Reply-To: <1317176061.39.0.582537402848.issue884@bugs.pypy.org> Message-ID: <1317658882.38.0.0900985912122.issue884@bugs.pypy.org> Armin Rigo added the comment: Merged on trunk (99233931c2bf). On Linux it gives very good results for your example. On Windows not, but I don't really understand why... ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Mon Oct 3 19:41:55 2011 From: tracker at bugs.pypy.org (Leonardo Santagada) Date: Mon, 03 Oct 2011 17:41:55 +0000 Subject: [pypy-issue] [issue892] Pypy is slower on longs than it should In-Reply-To: <1317663715.18.0.663914446646.issue892@bugs.pypy.org> Message-ID: <1317663715.18.0.663914446646.issue892@bugs.pypy.org> New submission from Leonardo Santagada : In Python Brasil this year we found that this code runs relatively the same on pypy and cpython on linux (and is much worse on osx): fibs = {0: 0, 1: 1} def fib(n): if n in fibs: return fibs[n] if n % 2 == 0: fibs[n] = ((2 * fib((n / 2) - 1)) + fib(n / 2)) * fib(n / 2) else: fibs[n] = (fib((n - 1) / 2) ** 2) + (fib((n + 1) / 2) ** 2) return fibs[n] fib(10000000) original by Henrique Pereira (implementation of a dijkstra algo) the link is here: https://gist.github.com/1256985 Alex Gaynor said that the reason is probably because pypy longs are known to be slower than cpythons. ---------- messages: 3239 nosy: pypy-issue, santagada priority: bug status: unread title: Pypy is slower on longs than it should ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Mon Oct 3 19:45:25 2011 From: tracker at bugs.pypy.org (Justin Peel) Date: Mon, 03 Oct 2011 17:45:25 +0000 Subject: [pypy-issue] [issue892] Pypy is slower on longs than it should In-Reply-To: <1317663715.18.0.663914446646.issue892@bugs.pypy.org> Message-ID: <1317663925.77.0.959017334105.issue892@bugs.pypy.org> Justin Peel added the comment: Yes, ~85% of the time is spent doing multiplications with longs in Pypy and it is known that long multiplication is slower in Pypy for now. ---------- nosy: +justinpeel status: unread -> chatting ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Tue Oct 4 12:32:39 2011 From: tracker at bugs.pypy.org (Vetoshkin Nikita) Date: Tue, 04 Oct 2011 10:32:39 +0000 Subject: [pypy-issue] [issue783] subprocess cannot handle unicode args In-Reply-To: <1309962687.3.0.947652570439.issue783@bugs.pypy.org> Message-ID: <1317724359.24.0.178322495028.issue783@bugs.pypy.org> Vetoshkin Nikita added the comment: About using FileEncoder in os.exec* functions. One can receive a list containing unicode and plain str. Armin suggested changing selfmade encoding function to list of FileEncoder objects and str's and... I think this means moving encode_to_filesystem() function to rposix module and copy-pasting it in every exec() function. ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Tue Oct 4 14:09:22 2011 From: tracker at bugs.pypy.org (Fijal) Date: Tue, 04 Oct 2011 12:09:22 +0000 Subject: [pypy-issue] [issue884] Diabolical Multicore Thread Performance In-Reply-To: <1317176061.39.0.582537402848.issue884@bugs.pypy.org> Message-ID: <1317730162.59.0.402961263723.issue884@bugs.pypy.org> Fijal added the comment: Very good being? ---------- nosy: +fijal ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Tue Oct 4 14:21:18 2011 From: tracker at bugs.pypy.org (David Beazley) Date: Tue, 04 Oct 2011 12:21:18 +0000 Subject: [pypy-issue] [issue884] Diabolical Multicore Thread Performance In-Reply-To: <1317176061.39.0.582537402848.issue884@bugs.pypy.org> Message-ID: <1317730878.82.0.153006190194.issue884@bugs.pypy.org> David Beazley added the comment: I'm trying to test the patch, but haven't been able to successfully run translate.py on trunk (fails after an hour with a broken pipe, Linux). Still fiddling with it. ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Tue Oct 4 14:29:22 2011 From: tracker at bugs.pypy.org (Vetoshkin Nikita) Date: Tue, 04 Oct 2011 12:29:22 +0000 Subject: [pypy-issue] [issue869] Implement ctypes..from_buffer method In-Reply-To: <1315674477.29.0.889845043655.issue869@bugs.pypy.org> Message-ID: <1317731362.11.0.51761852397.issue869@bugs.pypy.org> Vetoshkin Nikita added the comment: So one can implement it, but no one bothered because those tests are skipped and don't show up? ---------- nosy: +nekto0n ________________________________________ PyPy bug tracker ________________________________________ From fijall at gmail.com Wed Oct 5 00:50:30 2011 From: fijall at gmail.com (Maciej Fijalkowski) Date: Wed, 5 Oct 2011 00:50:30 +0200 Subject: [pypy-issue] [issue869] Implement ctypes..from_buffer method In-Reply-To: <1317731362.11.0.51761852397.issue869@bugs.pypy.org> References: <1315674477.29.0.889845043655.issue869@bugs.pypy.org> <1317731362.11.0.51761852397.issue869@bugs.pypy.org> Message-ID: On Tue, Oct 4, 2011 at 2:29 PM, Vetoshkin Nikita wrote: > > Vetoshkin Nikita added the comment: > > So one can implement it, but no one bothered because those tests are skipped and > don't show up? Just noone bothered > > ---------- > nosy: +nekto0n > > ________________________________________ > PyPy bug tracker > > ________________________________________ > _______________________________________________ > pypy-issue mailing list > pypy-issue at python.org > http://mail.python.org/mailman/listinfo/pypy-issue > From tracker at bugs.pypy.org Wed Oct 5 14:07:43 2011 From: tracker at bugs.pypy.org (Carl Friedrich Bolz) Date: Wed, 05 Oct 2011 12:07:43 +0000 Subject: [pypy-issue] [issue866] [PATCH] join is slow about 2x slower than Python In-Reply-To: <1315461531.27.0.58792221009.issue866@bugs.pypy.org> Message-ID: <1317816463.99.0.432683365809.issue866@bugs.pypy.org> Carl Friedrich Bolz added the comment: implemented this idea in 547bc8141987. need to benchmark. ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Thu Oct 6 10:30:50 2011 From: tracker at bugs.pypy.org (Carl Friedrich Bolz) Date: Thu, 06 Oct 2011 08:30:50 +0000 Subject: [pypy-issue] [issue866] [PATCH] join is slow about 2x slower than Python In-Reply-To: <1315461531.27.0.58792221009.issue866@bugs.pypy.org> Message-ID: <1317889850.84.0.21541224701.issue866@bugs.pypy.org> Carl Friedrich Bolz added the comment: Ok, benchmarks: CPython 2.7.1: 0.200s PyPy nightly: 0.215s PyPy on list-strategies branch: 0.156s closing the issue. ---------- status: chatting -> resolved ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Thu Oct 6 21:56:35 2011 From: tracker at bugs.pypy.org (Dustin Sallings) Date: Thu, 06 Oct 2011 19:56:35 +0000 Subject: [pypy-issue] [issue893] PIL compilation problem on OS X Lion In-Reply-To: <1317930995.52.0.397430004492.issue893@bugs.pypy.org> Message-ID: <1317930995.52.0.397430004492.issue893@bugs.pypy.org> New submission from Dustin Sallings : Attempting to use pip and a virtualenv to install PIL under pypy 1.6 fails with the following error: File "/usr/local/Cellar/pypy/1.6.0/lib-python/modified- 2.7/distutils/unixccompiler.py", line 328, in find_library_file m = re.search(r'-isysroot\s+(\S+)', cflags) File "/Users/dustin/stuff/pypy/lib-python/2.7/re.py", line 142, in search return _compile(pattern, flags).search(string) TypeError: unsupported operand type for unary buffer: 'NoneType' This is a brew installed pypy 1.6 with a brand new virtualenv. ---------- files: pip.log messages: 3248 nosy: dustin, pypy-issue priority: bug status: unread title: PIL compilation problem on OS X Lion ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Thu Oct 6 22:14:15 2011 From: tracker at bugs.pypy.org (Alex Gaynor) Date: Thu, 06 Oct 2011 20:14:15 +0000 Subject: [pypy-issue] [issue893] PIL compilation problem on OS X Lion In-Reply-To: <1317930995.52.0.397430004492.issue893@bugs.pypy.org> Message-ID: <1317932055.31.0.805315811002.issue893@bugs.pypy.org> Alex Gaynor added the comment: Looks like this was fixed in 9cc9fd4f2b14, can you grab a nightly and confirm? ---------- nosy: +agaynor status: unread -> chatting ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Thu Oct 6 22:29:33 2011 From: tracker at bugs.pypy.org (Dustin Sallings) Date: Thu, 06 Oct 2011 20:29:33 +0000 Subject: [pypy-issue] [issue893] PIL compilation problem on OS X Lion In-Reply-To: <1317930995.52.0.397430004492.issue893@bugs.pypy.org> Message-ID: <1317932973.47.0.21283773065.issue893@bugs.pypy.org> Dustin Sallings added the comment: Confirmed fixed. Thanks. ---------- status: chatting -> resolved ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Thu Oct 6 22:30:43 2011 From: tracker at bugs.pypy.org (Fijal) Date: Thu, 06 Oct 2011 20:30:43 +0000 Subject: [pypy-issue] [issue783] subprocess cannot handle unicode args In-Reply-To: <1309962687.3.0.947652570439.issue783@bugs.pypy.org> Message-ID: <1317933043.89.0.0156605878922.issue783@bugs.pypy.org> Fijal added the comment: How about just a new unwrap_spec like we have for fileno? ---------- nosy: +fijal ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Thu Oct 6 22:50:14 2011 From: tracker at bugs.pypy.org (Vetoshkin Nikita) Date: Thu, 06 Oct 2011 20:50:14 +0000 Subject: [pypy-issue] [issue783] subprocess cannot handle unicode args In-Reply-To: <1309962687.3.0.947652570439.issue783@bugs.pypy.org> Message-ID: <1317934214.27.0.94894057405.issue783@bugs.pypy.org> Vetoshkin Nikita added the comment: Would it be semantically ok - unwrapping a list of w_unicode and w_str objects into a list of str objects? If it's ok - can you give a hint how to implement that, as I can see only simple objects unwrapping rules are defined (in interpreter/gateway.py)? ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Thu Oct 6 22:53:23 2011 From: tracker at bugs.pypy.org (Fijal) Date: Thu, 06 Oct 2011 20:53:23 +0000 Subject: [pypy-issue] [issue783] subprocess cannot handle unicode args In-Reply-To: <1309962687.3.0.947652570439.issue783@bugs.pypy.org> Message-ID: <1317934403.87.0.552627647779.issue783@bugs.pypy.org> Fijal added the comment: Sorry, this is not a proper forum, can you come to IRC? how does this involve a list in any way? I just meant accepting w_unicode or w_str and using filesystem encoding to cast it correctly. ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Thu Oct 6 23:27:14 2011 From: tracker at bugs.pypy.org (Dustin Sallings) Date: Thu, 06 Oct 2011 21:27:14 +0000 Subject: [pypy-issue] [issue894] Undefined symbol in PIL 1.7 on pypy 1.6 In-Reply-To: <1317936434.53.0.308160585553.issue894@bugs.pypy.org> Message-ID: <1317936434.53.0.308160585553.issue894@bugs.pypy.org> New submission from Dustin Sallings : (Slightly modified to not remove the symbol) ImportError: unable to load extension module '/Users/dustin/stuff/pypy/site- packages/PIL/_imaging.pypy-16.so': dlopen(/Users/dustin/stuff/pypy/site- packages/PIL/_imaging.pypy-16.so, 6): Symbol not found: _PyExc_AttributeError 1.6.1-dev0 (pypy-c-jit-47836-383ca802ba07-osx64.tar.bz2) I see the symbols being requested here: [pypy] dustinnmb:~/stuff/pypy/site-packages/PIL 551% nm _imaging.pypy-16.so | grep PyExc U _PyExc_AttributeError U _PyExc_IOError U _PyExc_IndexError U _PyExc_RuntimeError U _PyExc_SystemError U _PyExc_TypeError U _PyExc_ValueError But it's not clear to me how they'd be resolved. ---------- messages: 3254 nosy: dustin, pypy-issue priority: bug status: unread title: Undefined symbol in PIL 1.7 on pypy 1.6 ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Fri Oct 7 12:05:53 2011 From: tracker at bugs.pypy.org (Armin Rigo) Date: Fri, 07 Oct 2011 10:05:53 +0000 Subject: [pypy-issue] [issue895] Continulets + JIT In-Reply-To: <1317981953.45.0.228294129679.issue895@bugs.pypy.org> Message-ID: <1317981953.45.0.228294129679.issue895@bugs.pypy.org> New submission from Armin Rigo : Stacklets (and continulets, greenlets, etc.) are almost, but not completely, working with the JIT. Stacklets are transparently compatible with the JIT, in the sense that the JIT will trace and compile code corresponding to one stacklet at a time. If this stacklet switches temporarily away while the JIT is tracing, the tracing is suspended and resumes normally when the stacklet resumes. The exception to this rule is virtualizables and virtualrefs: the 'vable_token' field in PyFrames and in JIT_VIRTUAL_REFs contains a stack address, but the stack might be switched away. So if you have code that accesses (directly or indirectly) the frames of another stacklet in a JITted loop, it crashes completely. Note that e.g. in a non-main greenlet, if you just call sys._getframe(n) for a large enough n, then you are reading frames that belong to the main greenlet --- crash. ---------- messages: 3255 nosy: arigo, pypy-issue priority: bug status: unread title: Continulets + JIT ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Fri Oct 7 18:48:54 2011 From: tracker at bugs.pypy.org (Vetoshkin Nikita) Date: Fri, 07 Oct 2011 16:48:54 +0000 Subject: [pypy-issue] [issue783] subprocess cannot handle unicode args In-Reply-To: <1309962687.3.0.947652570439.issue783@bugs.pypy.org> Message-ID: <1318006134.72.0.233459148551.issue783@bugs.pypy.org> Vetoshkin Nikita added the comment: looks like it's fixed in https://bitbucket.org/pypy/pypy/changeset/787a29a88622. closing? ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Fri Oct 7 19:00:21 2011 From: tracker at bugs.pypy.org (Fijal) Date: Fri, 07 Oct 2011 17:00:21 +0000 Subject: [pypy-issue] [issue888] json module in pypy is much more lower than cpython In-Reply-To: <1317284039.94.0.390923902277.issue888@bugs.pypy.org> Message-ID: <1318006821.71.0.1031931107.issue888@bugs.pypy.org> Fijal added the comment: There is work in progress on making json faster. Thanks for pointers ---------- nosy: +fijal status: unread -> chatting ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Fri Oct 7 19:20:47 2011 From: tracker at bugs.pypy.org (Vetoshkin Nikita) Date: Fri, 07 Oct 2011 17:20:47 +0000 Subject: [pypy-issue] [issue872] getpass.unix_getpass fails with "IOError: [Errno 29] Illegal seek: ''" In-Reply-To: <1316259307.04.0.352496531406.issue872@bugs.pypy.org> Message-ID: <1318008047.03.0.937189311027.issue872@bugs.pypy.org> Vetoshkin Nikita added the comment: >I guess I need to look into the source code of glibc this http://sourceware.org/git/? p=glibc.git;a=blob;f=libio/fileops.c;h=a6f7cbad82e4d6d96b0d08db6dedc4cafa1be34b;h b=HEAD#l929 ? ---------- nosy: +nekto0n ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Fri Oct 7 19:21:49 2011 From: tracker at bugs.pypy.org (Vetoshkin Nikita) Date: Fri, 07 Oct 2011 17:21:49 +0000 Subject: [pypy-issue] [issue872] getpass.unix_getpass fails with "IOError: [Errno 29] Illegal seek: ''" In-Reply-To: <1316259307.04.0.352496531406.issue872@bugs.pypy.org> Message-ID: <1318008109.97.0.991803383035.issue872@bugs.pypy.org> Vetoshkin Nikita added the comment: shorter version http://clck.ru/MPAl :) ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Fri Oct 7 19:37:46 2011 From: tracker at bugs.pypy.org (Fijal) Date: Fri, 07 Oct 2011 17:37:46 +0000 Subject: [pypy-issue] [issue882] `import resource` fails In-Reply-To: <1316988288.39.0.556894974857.issue882@bugs.pypy.org> Message-ID: <1318009066.22.0.365920681969.issue882@bugs.pypy.org> Fijal added the comment: thepian: can you provide more details? It also works for me ---------- nosy: +fijal ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Fri Oct 7 19:52:47 2011 From: tracker at bugs.pypy.org (Joao Paulo Farias) Date: Fri, 07 Oct 2011 17:52:47 +0000 Subject: [pypy-issue] [issue896] Bad quotation? In-Reply-To: <1318009967.57.0.315391021514.issue896@bugs.pypy.org> Message-ID: <1318009967.57.0.315391021514.issue896@bugs.pypy.org> New submission from Joao Paulo Farias : This quotation made me think JIT doesn't work on OS/X, might be good to remove it. """ And now for something completely different: ``the JIT doesn't work on OS/X (nov'09)'' """ ---------- messages: 3261 nosy: jpfarias, pypy-issue priority: bug status: unread title: Bad quotation? ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Fri Oct 7 22:26:19 2011 From: tracker at bugs.pypy.org (Fijal) Date: Fri, 07 Oct 2011 20:26:19 +0000 Subject: [pypy-issue] [issue888] json module in pypy is much more lower than cpython In-Reply-To: <1317284039.94.0.390923902277.issue888@bugs.pypy.org> Message-ID: <1318019179.83.0.086446808423.issue888@bugs.pypy.org> Fijal added the comment: Also *never* use default CPython's timeit for comparisons. It disables the cyclic GC. ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Fri Oct 7 23:29:45 2011 From: tracker at bugs.pypy.org (Henrik Vendelbo) Date: Fri, 07 Oct 2011 21:29:45 +0000 Subject: [pypy-issue] [issue882] `import resource` fails In-Reply-To: <1316988288.39.0.556894974857.issue882@bugs.pypy.org> Message-ID: <1318022985.43.0.278371224715.issue882@bugs.pypy.org> Henrik Vendelbo added the comment: I have copied lib_pypy and lib-python and an executable from the release-1.6 branch. Somehow that doesn't seem to be sufficient. It seems there is something about the loading process that I don't understand. ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Fri Oct 7 23:32:15 2011 From: tracker at bugs.pypy.org (Fijal) Date: Fri, 07 Oct 2011 21:32:15 +0000 Subject: [pypy-issue] [issue882] `import resource` fails In-Reply-To: <1316988288.39.0.556894974857.issue882@bugs.pypy.org> Message-ID: <1318023135.64.0.0353877160535.issue882@bugs.pypy.org> Fijal added the comment: They have to be in their respective directories. Look how the .tar.gz is organized. This all follows CPython-style of putting things in places. ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Sat Oct 8 00:09:24 2011 From: tracker at bugs.pypy.org (Henrik Vendelbo) Date: Fri, 07 Oct 2011 22:09:24 +0000 Subject: [pypy-issue] [issue882] `import resource` fails In-Reply-To: <1316988288.39.0.556894974857.issue882@bugs.pypy.org> Message-ID: <1318025364.91.0.794733385489.issue882@bugs.pypy.org> Henrik Vendelbo added the comment: where do the _32_ and _64_ come from. I need to create a single distribution for multiple platforms, so I guess I need to tweak the package.py script ? ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Sat Oct 8 00:10:48 2011 From: tracker at bugs.pypy.org (Fijal) Date: Fri, 07 Oct 2011 22:10:48 +0000 Subject: [pypy-issue] [issue882] `import resource` fails In-Reply-To: <1316988288.39.0.556894974857.issue882@bugs.pypy.org> Message-ID: <1318025448.02.0.478527538913.issue882@bugs.pypy.org> Fijal added the comment: _32_ and _64_ come from translation. If you translate 32bit executable, there is a _32_, if 64bit executable, there is a _64_ file. The reason is that they should not collide ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Sat Oct 8 01:04:47 2011 From: tracker at bugs.pypy.org (Henrik Vendelbo) Date: Fri, 07 Oct 2011 23:04:47 +0000 Subject: [pypy-issue] [issue882] `import resource` fails In-Reply-To: <1316988288.39.0.556894974857.issue882@bugs.pypy.org> Message-ID: <1318028687.29.0.912236470923.issue882@bugs.pypy.org> Henrik Vendelbo added the comment: Ideally I think there should be a way to generate both 64 and 32 versions without doing a full translate run. It obviously isn't a bug though ---------- status: chatting -> resolved ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Sat Oct 8 21:16:41 2011 From: tracker at bugs.pypy.org (Valery) Date: Sat, 08 Oct 2011 19:16:41 +0000 Subject: [pypy-issue] [issue878] too many open files In-Reply-To: <1316725510.98.0.71219979546.issue878@bugs.pypy.org> Message-ID: <1318101401.75.0.0864858241689.issue878@bugs.pypy.org> Valery added the comment: http://bugs.python.org/issue13133 Plus, there are about 10 places in setuptools-0.6c12dev-r88846 where files are open but not closed explicitly. And FD leak will depend on mercy of GC model. ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Sat Oct 8 22:44:55 2011 From: tracker at bugs.pypy.org (Fijal) Date: Sat, 08 Oct 2011 20:44:55 +0000 Subject: [pypy-issue] [issue897] minimark GC does not work with make lldebug In-Reply-To: <1318106695.63.0.0405614409621.issue897@bugs.pypy.org> Message-ID: <1318106695.63.0.0405614409621.issue897@bugs.pypy.org> New submission from Fijal : There is a failing assertion about "wrong type id" because we pass 0 for tid from the JIT in case we call malloc_slowpath and then malloc_fixedsize_clear on the GC. I can fix that probably but I'm fed up with yaks for now. ---------- messages: 3269 nosy: fijal, pypy-issue priority: bug status: unread title: minimark GC does not work with make lldebug ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Sun Oct 9 00:46:32 2011 From: tracker at bugs.pypy.org (Vetoshkin Nikita) Date: Sat, 08 Oct 2011 22:46:32 +0000 Subject: [pypy-issue] [issue747] fcntl.lockf() is broken with (fcntl.LOCK_EX | fcntl.LOCK_NB) In-Reply-To: <1307775802.38.0.717100903574.issue747@bugs.pypy.org> Message-ID: <1318113992.65.0.981013206569.issue747@bugs.pypy.org> Vetoshkin Nikita added the comment: This tests passes on latest checkout under py.py ---------- nosy: +nekto0n ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Sun Oct 9 08:58:40 2011 From: tracker at bugs.pypy.org (Garen Parham) Date: Sun, 09 Oct 2011 06:58:40 +0000 Subject: [pypy-issue] [issue898] reduce() builtin appears to get no benefit from JIT; both jit, nojit versions 5x slower than CPython In-Reply-To: <1318143520.08.0.393790794524.issue898@bugs.pypy.org> Message-ID: <1318143520.08.0.393790794524.issue898@bugs.pypy.org> New submission from Garen Parham : While going through some scripts to test how fast pypy is, I came across this gem: $ cat factorial_functional.py import sys from operator import mul def factorial(n): return reduce(mul, xrange(1,n+1), 1) if __name__ == '__main__': factorial(int(sys.argv[1])) $ time /opt/python27/bin/python factorial_functional.py 100000 real 0m5.678s user 0m5.346s sys 0m0.329s The the large number of 100000 was chosen to help amortize the JIT warmup cost, but when using pypy 1.6 or either of the 2011-10-09 nightlies of pypy-c-jit and pypy-c-nojit, all show about the same (much slower) performance--presumably due to calls to reduce() not being JIT'd: $ time ~/dev/pypy-c-nojit-47882-3f96afe7cdc2-linux/bin/pypy factorial_functional.py 100000 real 0m25.985s user 0m25.904s sys 0m0.048s $ time ~/dev/pypy-c-jit-47882-3f96afe7cdc2-linux/bin/pypy factorial_functional.py 100000 real 0m25.963s user 0m25.827s sys 0m0.118s $ time ~/dev/pypy-1.6_32bit/bin/pypy factorial_functional.py 100000 real 0m26.139s user 0m25.994s sys 0m0.091s ---------- messages: 3271 nosy: Garen, pypy-issue priority: bug status: unread title: reduce() builtin appears to get no benefit from JIT; both jit, nojit versions 5x slower than CPython ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Sun Oct 9 09:27:03 2011 From: tracker at bugs.pypy.org (Garen Parham) Date: Sun, 09 Oct 2011 07:27:03 +0000 Subject: [pypy-issue] [issue899] Simple inner loop with mult-op much slower than CPython with/without JIT In-Reply-To: <1318145223.07.0.0582511175863.issue899@bugs.pypy.org> Message-ID: <1318145223.07.0.0582511175863.issue899@bugs.pypy.org> New submission from Garen Parham : $ cat factorial_iterative.py import sys def factorial(n): result = 1 for i in xrange(1, n+1): result *= i return result if __name__ == '__main__': factorial(int(sys.argv[1])) $ time /opt/python27/bin/python factorial_iterative.py 100000 real 0m5.914s user 0m5.513s sys 0m0.358s $ time ~/dev/pypy-c-jit-47882-3f96afe7cdc2-linux/bin/pypy factorial_iterative.py 100000 real 0m25.916s user 0m25.751s sys 0m0.056s $ time ~/dev/pypy-c-nojit-47882-3f96afe7cdc2-linux/bin/pypy factorial_iterative.py 100000 real 0m25.578s user 0m25.524s sys 0m0.051s ---------- messages: 3272 nosy: Garen, pypy-issue priority: bug status: unread title: Simple inner loop with mult-op much slower than CPython with/without JIT ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Sun Oct 9 10:19:20 2011 From: tracker at bugs.pypy.org (Armin Rigo) Date: Sun, 09 Oct 2011 08:19:20 +0000 Subject: [pypy-issue] [issue899] Simple inner loop with mult-op much slower than CPython with/without JIT In-Reply-To: <1318145223.07.0.0582511175863.issue899@bugs.pypy.org> Message-ID: <1318148360.67.0.373216472571.issue899@bugs.pypy.org> Armin Rigo added the comment: Duplicate of Issue892. ---------- nosy: +arigo status: unread -> duplicate ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Sun Oct 9 10:22:34 2011 From: tracker at bugs.pypy.org (Armin Rigo) Date: Sun, 09 Oct 2011 08:22:34 +0000 Subject: [pypy-issue] [issue897] minimark GC does not work with make lldebug In-Reply-To: <1318106695.63.0.0405614409621.issue897@bugs.pypy.org> Message-ID: <1318148554.79.0.704219075208.issue897@bugs.pypy.org> Armin Rigo added the comment: I'm always using "make lldebug" with the minimark GC. You have to be more precise... ---------- nosy: +arigo status: unread -> chatting ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Sun Oct 9 10:24:18 2011 From: tracker at bugs.pypy.org (Armin Rigo) Date: Sun, 09 Oct 2011 08:24:18 +0000 Subject: [pypy-issue] [issue898] reduce() builtin appears to get no benefit from JIT; both jit, nojit versions 5x slower than CPython In-Reply-To: <1318143520.08.0.393790794524.issue898@bugs.pypy.org> Message-ID: <1318148658.05.0.518211378556.issue898@bugs.pypy.org> Armin Rigo added the comment: The issue is not reduce(), it's again just issue892: multiplication of longs is too slow. ---------- nosy: +arigo status: unread -> duplicate ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Sun Oct 9 10:29:15 2011 From: tracker at bugs.pypy.org (Garen Parham) Date: Sun, 09 Oct 2011 08:29:15 +0000 Subject: [pypy-issue] [issue900] Using cProfile on Windows generates negative tottime values In-Reply-To: <1318148955.37.0.118875417908.issue900@bugs.pypy.org> Message-ID: <1318148955.37.0.118875417908.issue900@bugs.pypy.org> New submission from Garen Parham : PS C:\Users\Garen\dev\pyfun> pypy-c -m cProfile .\primes_inf_gen.py 10000000 10664601 function calls in 20.204 seconds Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) 1 0.000 0.000 20.204 20.204 primes_inf_gen.py:1() 1 1.111 1.111 20.204 20.204 primes_inf_gen.py:21(primes_up_to) 664580 0.263 0.000 0.263 0.000 primes_inf_gen.py:22() 664580 -5015.618 -0.008 18.830 0.000 primes_inf_gen.py:3(primes) 1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects} 9335438 5034.447 0.001 5034.447 0.001 {method 'pop' of 'dict' objects} Issue doesn't occur with just '-m profile': PS C:\Users\Garen\dev\pyfun> pypy-c -m profile .\primes_inf_gen.py 10000000 10664602 function calls in 14.609 seconds Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) 9335438 1.789 0.000 1.789 0.000 :0(pop) 1 0.001 0.001 0.001 0.001 :0(setprofile) 1 0.000 0.000 14.607 14.607 primes_inf_gen.py:1() 1 2.153 2.153 14.607 14.607 primes_inf_gen.py:21(primes_up_to) 664580 0.673 0.000 0.673 0.000 primes_inf_gen.py:22() 664580 9.992 0.000 11.781 0.000 primes_inf_gen.py:3(primes) 1 0.000 0.000 14.609 14.609 profile:0(, file '.\primes_inf_gen.py', line 1>) 0 0.000 0.000 profile:0(profiler) Using Windows 7 x64, FWIW. ---------- files: primes_inf_gen.py messages: 3276 nosy: Garen, pypy-issue priority: bug status: unread title: Using cProfile on Windows generates negative tottime values ________________________________________ PyPy bug tracker ________________________________________ -------------- next part -------------- import sys import itertools def primes(): composites = {} # A mapping from composite numbers to the smallest prime # that is a factor of it (its witness). n = 2 # The current number being considered as a prime. while True: if n not in composites: yield n # Not a composite, therefore prime. composites[n**2] = n # The next unseen composite number is n squared. else: # n is composite. Find the next unseen composite number with the # same witness as n. witness = composites.pop(n) next = n + witness while next in composites: next += witness composites[next] = witness n += 1 def primes_up_to(limit): return list(itertools.takewhile(lambda p: p <= limit, primes())) if __name__ == '__main__': primes_up_to(int(sys.argv[1])) From tracker at bugs.pypy.org Sun Oct 9 10:38:23 2011 From: tracker at bugs.pypy.org (Garen Parham) Date: Sun, 09 Oct 2011 08:38:23 +0000 Subject: [pypy-issue] [issue900] Using cProfile on Windows generates negative tottime values In-Reply-To: <1318148955.37.0.118875417908.issue900@bugs.pypy.org> Message-ID: <1318149503.62.0.781089270011.issue900@bugs.pypy.org> Garen Parham added the comment: Release used was 1.6 (not present in Release drop-down, only goes up to 1.5). ---------- status: unread -> chatting ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Sun Oct 9 20:14:40 2011 From: tracker at bugs.pypy.org (Josh Ayers) Date: Sun, 09 Oct 2011 18:14:40 +0000 Subject: [pypy-issue] [issue900] Using cProfile on Windows generates negative tottime values In-Reply-To: <1318148955.37.0.118875417908.issue900@bugs.pypy.org> Message-ID: <1318184080.52.0.37284366568.issue900@bugs.pypy.org> Josh Ayers added the comment: I was about to submit a separate bug on this issue when I saw this one. FWIW, I attached another script that demonstrates the same problem. I'm also using PyPy1.6 on Windows 7. ---------- nosy: +jayers2003 ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Mon Oct 10 01:30:36 2011 From: tracker at bugs.pypy.org (Garen Parham) Date: Sun, 09 Oct 2011 23:30:36 +0000 Subject: [pypy-issue] [issue840] string-escape encoding should escape single quotes In-Reply-To: <1313963816.06.0.158902991143.issue840@bugs.pypy.org> Message-ID: <1318203036.91.0.20682733588.issue840@bugs.pypy.org> Garen Parham added the comment: Looks like this has been fixed after 1.6: $ ~/dev/pypy-c-nojit-47882-3f96afe7cdc2-linux/bin/pypy Python 2.7.1 (3f96afe7cdc2, Oct 09 2011, 04:21:59) [PyPy 1.6.1-dev0 with GCC 4.4.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. And now for something completely different: ``"The HHU campus is a good Quake level" - Armin'' >>>> "'".encode('string-escape') == "\\'" True >>>> Which agrees with CPython: $ /opt/python27/bin/python Python 2.7.2 (default, Oct 8 2011, 23:08:55) [GCC 4.1.2 20080704 (Red Hat 4.1.2-51)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> "'".encode('string-escape') == "\\'" True >>> ---------- nosy: +Garen status: unread -> chatting ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Mon Oct 10 02:08:39 2011 From: tracker at bugs.pypy.org (Garen Parham) Date: Mon, 10 Oct 2011 00:08:39 +0000 Subject: [pypy-issue] [issue731] Problem handling set operations In-Reply-To: <1306517163.22.0.300457778584.issue731@bugs.pypy.org> Message-ID: <1318205319.31.0.285905099182.issue731@bugs.pypy.org> Garen Parham added the comment: As jamartinh indicates, the download link for the win32 distribution now works, so the OP could confirm with pypy-1.6 if desired. GraphLib.py now works without crashing on Windows 7 (even with that really long range(1000) in the main loop), as well as the snippet in msg2556 from antocuni. ---------- nosy: +Garen status: in-progress -> resolved ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Mon Oct 10 03:48:21 2011 From: tracker at bugs.pypy.org (Garen Parham) Date: Mon, 10 Oct 2011 01:48:21 +0000 Subject: [pypy-issue] [issue701] Pypy can't find msvcr100.dll if launched via symlink In-Reply-To: <1304187738.03.0.397564181695.issue701@> Message-ID: <1318211301.93.0.32429955882.issue701@bugs.pypy.org> Garen Parham added the comment: Reproduced on 1.6. Here's a concrete repro: C:\Users\Garen\dev>mklink pypy.exe .\pypy-1.6\pypy-c.exe symbolic link created for pypy.exe <<===>> .\pypy-1.6\pypy-c.exe C:\Users\Garen\dev>C:\Users\Garen\dev\pypy.exe -V Python 2.7.1 (080f42d5c4b4, Aug 23 2011, 11:41:11) [PyPy 1.6.0 with MSC v.1500 32 bit] Just getting the version works, but firing up the interpreter doesn't: C:\Users\Garen\dev>.\pypy.exe debug: WARNING: Library path not found, using compiled-in sys.path. debug: WARNING: 'sys.prefix' will not be set. debug: WARNING: Make sure the pypy binary is kept inside its tree of files. debug: WARNING: It is ok to create a symlink to it from somewhere else. 'import site' failed Python 2.7.1 (080f42d5c4b4, Aug 23 2011, 11:41:11) [PyPy 1.6.0 with MSC v.1500 32 bit] on win32 Type "help", "copyright", "credits" or "license" for more information. debug: OperationError: debug: operror-type: ImportError debug: operror-value: No module named _pypy_interact ---------- nosy: +Garen release: ??? -> 1.5 ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Mon Oct 10 08:35:41 2011 From: tracker at bugs.pypy.org (Garen Parham) Date: Mon, 10 Oct 2011 06:35:41 +0000 Subject: [pypy-issue] [issue645] twisted.test.test_threadable.SynchronizationTestCase.testThreadedSynchronization hangs In-Reply-To: <1298652832.91.0.285352073761.issue645@> Message-ID: <1318228541.78.0.12343707637.issue645@bugs.pypy.org> Garen Parham added the comment: "This test"? Did you forget to upload a file? ---------- nosy: +Garen ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Mon Oct 10 12:02:28 2011 From: tracker at bugs.pypy.org (Ram Rachum) Date: Mon, 10 Oct 2011 10:02:28 +0000 Subject: [pypy-issue] [issue701] Pypy can't find msvcr100.dll if launched via symlink In-Reply-To: <1304187738.03.0.397564181695.issue701@> Message-ID: <1318240948.25.0.322976691788.issue701@bugs.pypy.org> Ram Rachum added the comment: I also experienced this. ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Mon Oct 10 12:07:23 2011 From: tracker at bugs.pypy.org (Amaury Forgeot d Arc) Date: Mon, 10 Oct 2011 10:07:23 +0000 Subject: [pypy-issue] [issue701] Pypy can't find msvcr100.dll if launched via symlink In-Reply-To: <1304187738.03.0.397564181695.issue701@> Message-ID: <1318241243.47.0.211463211346.issue701@bugs.pypy.org> Amaury Forgeot d Arc added the comment: Yes, pypy on windows has no real support for symlinks. But I suspect it is the same for CPython. Can you test with CPython 2.7, and 3.2 eventually? ---------- nosy: +afa ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Mon Oct 10 12:40:54 2011 From: tracker at bugs.pypy.org (Ram Rachum) Date: Mon, 10 Oct 2011 10:40:54 +0000 Subject: [pypy-issue] [issue701] Pypy can't find msvcr100.dll if launched via symlink In-Reply-To: <1304187738.03.0.397564181695.issue701@> Message-ID: <1318243254.61.0.557299240762.issue701@bugs.pypy.org> Ram Rachum added the comment: I use symlinks for Python versions 2.5, 2.6, 2.7, 3.1 and 3.2 on my Windows XP machine and they work fine. ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Mon Oct 10 13:04:43 2011 From: tracker at bugs.pypy.org (Amaury Forgeot d Arc) Date: Mon, 10 Oct 2011 11:04:43 +0000 Subject: [pypy-issue] [issue701] Pypy can't find msvcr100.dll if launched via symlink In-Reply-To: <1304187738.03.0.397564181695.issue701@> Message-ID: <1318244683.31.0.417523269318.issue701@bugs.pypy.org> Amaury Forgeot d Arc added the comment: I suppose that these versions are *installed* i.e. stored in the registry &co. Could someone please try the following: - copy the whole c:\python27 tree to a new place - start this: c:/new/place/python.exe -c "import os; print os" it should print some path inside c:/new/place - create a symlink to this new python.exe - run it with -c "import os; print os" and check the path again ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Mon Oct 10 20:16:42 2011 From: tracker at bugs.pypy.org (Alexander Milenko) Date: Mon, 10 Oct 2011 18:16:42 +0000 Subject: [pypy-issue] [issue901] *** glibc detected *** pypy: corrupted double-linked list: 0x000000000bd76de0 *** In-Reply-To: <1318270602.56.0.17883229495.issue901@bugs.pypy.org> Message-ID: <1318270602.56.0.17883229495.issue901@bugs.pypy.org> New submission from Alexander Milenko : script contain same strings by same rule... read data from replicated slave server, write to master. soty if file not of rule ---------- files: pypy-traceback.txt messages: 3289 nosy: Alvein, pypy-issue priority: bug status: unread title: *** glibc detected *** pypy: corrupted double-linked list: 0x000000000bd76de0 *** ________________________________________ PyPy bug tracker ________________________________________ -------------- next part -------------- *** glibc detected *** pypy: corrupted double-linked list: 0x000000000bd76de0 *** ======= Backtrace: ========= /lib/libc.so.6(+0x71ad6)[0x7fde403c0ad6] /lib/libc.so.6(+0x73508)[0x7fde403c2508] /lib/libc.so.6(cfree+0x6c)[0x7fde403c584c] pypy(PyObject_Del+0x11)[0xedc361] pypy[0xfc9635] pypy[0xf14c1a] pypy[0xd07976] pypy(PyObject_dealloc+0x13)[0xed2ca3] pypy[0xfc9635] pypy[0xf156e2] pypy[0xf3f88e] pypy[0xf51514] pypy[0xcd9b9e] [0x7fde3f3e4cbb] ======= Memory map: ======== 00400000-01225000 r-xp 00000000 09:02 31392187 /opt/pypy-env/bin/pypy 01424000-01425000 r--p 00e24000 09:02 31392187 /opt/pypy-env/bin/pypy 01425000-02756000 rw-p 00e25000 09:02 31392187 /opt/pypy-env/bin/pypy 02756000-0275e000 rw-p 00000000 00:00 0 03825000-0f4df000 rw-p 00000000 00:00 0 [heap] 7fde28000000-7fde29b0e000 rw-p 00000000 00:00 0 7fde29b0e000-7fde2c000000 ---p 00000000 00:00 0 7fde2fe00000-7fde30000000 rwxp 00000000 00:00 0 7fde30000000-7fde33fff000 rw-p 00000000 00:00 0 7fde33fff000-7fde34000000 ---p 00000000 00:00 0 7fde352ff000-7fde3530a000 r-xp 00000000 09:02 27574290 /lib/libnss_files-2.11.2.so 7fde3530a000-7fde35509000 ---p 0000b000 09:02 27574290 /lib/libnss_files-2.11.2.so 7fde35509000-7fde3550a000 r--p 0000a000 09:02 27574290 /lib/libnss_files-2.11.2.so 7fde3550a000-7fde3550b000 rw-p 0000b000 09:02 27574290 /lib/libnss_files-2.11.2.so 7fde3550b000-7fde35521000 r-xp 00000000 09:02 27575370 /lib/libgcc_s.so.1 7fde35521000-7fde35720000 ---p 00016000 09:02 27575370 /lib/libgcc_s.so.1 7fde35720000-7fde35721000 rw-p 00015000 09:02 27575370 /lib/libgcc_s.so.1 7fde35725000-7fde35726000 ---p 00000000 00:00 0 7fde35726000-7fde35f26000 rw-p 00000000 00:00 0 7fde35f26000-7fde35f27000 ---p 00000000 00:00 0 7fde35f27000-7fde36727000 rw-p 00000000 00:00 0 7fde36727000-7fde36728000 ---p 00000000 00:00 0 7fde36728000-7fde36f28000 rw-p 00000000 00:00 0 7fde36f28000-7fde36f29000 ---p 00000000 00:00 0 7fde36f29000-7fde37729000 rw-p 00000000 00:00 0 7fde37729000-7fde3772a000 ---p 00000000 00:00 0 7fde3772a000-7fde37f2a000 rw-p 00000000 00:00 0 7fde37f2a000-7fde37f2b000 ---p 00000000 00:00 0 7fde37f2b000-7fde3872b000 rw-p 00000000 00:00 0 7fde3872b000-7fde3872c000 ---p 00000000 00:00 0 7fde3872c000-7fde38f2c000 rw-p 00000000 00:00 0 7fde38f2c000-7fde38f2d000 ---p 00000000 00:00 0 7fde38f2d000-7fde3972d000 rw-p 00000000 00:00 0 7fde3972d000-7fde3972e000 ---p 00000000 00:00 0 7fde3972e000-7fde39f2e000 rw-p 00000000 00:00 0 7fde39f2e000-7fde39f2f000 ---p 00000000 00:00 0 7fde39f2f000-7fde3a72f000 rw-p 00000000 00:00 0 7fde3a72f000-7fde3a730000 ---p 00000000 00:00 0 7fde3a730000-7fde3af30000 rw-p 00000000 00:00 0 7fde3af30000-7fde3af31000 ---p 00000000 00:00 0 7fde3af31000-7fde3b731000 rw-p 00000000 00:00 0 7fde3b731000-7fde3b732000 ---p 00000000 00:00 0 7fde3b732000-7fde3bf32000 rw-p 00000000 00:00 0 7fde3bf32000-7fde3bf33000 ---p 00000000 00:00 0 7fde3bf33000-7fde3c733000 rw-p 00000000 00:00 0 7fde3c733000-7fde3c734000 ---p 00000000 00:00 0 7fde3c734000-7fde3cf34000 rw-p 00000000 00:00 0 7fde3cf34000-7fde3cf35000 ---p 00000000 00:00 0 7fde3cf35000-7fde3d735000 rw-p 00000000 00:00 0 7fde3d735000-7fde3d736000 ---p 00000000 00:00 0 7fde3d736000-7fde3e1bb000 rw-p 00000000 00:00 0 7fde3e23c000-7fde3e746000 rw-p 00000000 00:00 0 7fde3e746000-7fde3e75b000 r-xp 00000000 09:02 27574296 /lib/libnsl-2.11.2.so 7fde3e75b000-7fde3e95a000 ---p 00015000 09:02 27574296 /lib/libnsl-2.11.2.so 7fde3e95a000-7fde3e95b000 r--p 00014000 09:02 27574296 /lib/libnsl-2.11.2.so 7fde3e95b000-7fde3e95c000 rw-p 00015000 09:02 27574296 /lib/libnsl-2.11.2.so 7fde3e95c000-7fde3e95e000 rw-p 00000000 00:00 0 7fde3e95e000-7fde3eb2f000 r-xp 00000000 09:02 43394476 /usr/lib/libmysqlclient_r.so.16.0.0 7fde3eb2f000-7fde3ed2e000 ---p 001d1000 09:02 43394476 /usr/lib/libmysqlclient_r.so.16.0.0 7fde3ed2e000-7fde3ed36000 r--p 001d0000 09:02 43394476 /usr/lib/libmysqlclient_r.so.16.0.0 7fde3ed36000-7fde3ed80000 rw-p 001d8000 09:02 43394476 /usr/lib/libmysqlclient_r.so.16.0.0 7fde3ed80000-7fde3ed81000 rw-p 00000000 00:00 0 7fde3ed81000-7fde3ed8c000 r-xp 00000000 09:02 31408207 /opt/idei/egg/MySQL_python-1.2.3-py2.7-linux-x86_64.egg-tmp/_mysql.pypy-16.so 7fde3ed8c000-7fde3ef8b000 ---p 0000b000 09:02 31408207 /opt/idei/egg/MySQL_python-1.2.3-py2.7-linux-x86_64.egg-tmp/_mysql.pypy-16.so 7fde3ef8b000-7fde3ef90000 rw-p 0000a000 09:02 31408207 /opt/idei/egg/MySQL_python-1.2.3-py2.7-linux-x86_64.egg-tmp/_mysql.pypy-16.so 7fde3ef90000-7fde3f092000 rw-p 00000000 00:00 0 7fde3f094000-7fde3f194000 rwxp 00000000 00:00 0 7fde3f194000-7fde3f317000 rw-p 00000000 00:00 0 7fde3f39a000-7fde3f49a000 rwxp 00000000 00:00 0 7fde3f49a000-7fde3f51b000 rw-p 00000000 00:00 0 7fde3f51f000-7fde3f71f000 rwxp 00000000 00:00 0 7fde3f71f000-7fde3f7a0000 rw-p 00000000 00:00 0 7fde3f821000-7fde3fdac000 rw-p 00000000 00:00 0 7fde3fdac000-7fde3feac000 rwxp 00000000 00:00 0 7fde3feac000-7fde4034f000 rw-p 00000000 00:00 0 7fde4034f000-7fde404a7000 r-xp 00000000 09:02 27574302 /lib/libc-2.11.2.so 7fde404a7000-7fde406a6000 ---p 00158000 09:02 27574302 /lib/libc-2.11.2.so 7fde406a6000-7fde406aa000 r--p 00157000 09:02 27574302 /lib/libc-2.11.2.so 7fde406aa000-7fde406ab000 rw-p 0015b000 09:02 27574302 /lib/libc-2.11.2.so 7fde406ab000-7fde406b0000 rw-p 00000000 00:00 0 7fde406b0000-7fde406c7000 r-xp 00000000 09:02 27575371 /lib/libpthread-2.11.2.so 7fde406c7000-7fde408c6000 ---p 00017000 09:02 27575371 /lib/libpthread-2.11.2.so 7fde408c6000-7fde408c7000 r--p 00016000 09:02 27575371 /lib/libpthread-2.11.2.so 7fde408c7000-7fde408c8000 rw-p 00017000 09:02 27575371 /lib/libpthread-2.11.2.so 7fde408c8000-7fde408cc000 rw-p 00000000 00:00 0 7fde408cc000-7fde4090e000 r-xp 00000000 09:02 27575404 /lib/libncurses.so.5.7 7fde4090e000-7fde40b0d000 ---p 00042000 09:02 27575404 /lib/libncurses.so.5.7 7fde40b0d000-7fde40b12000 rw-p 00041000 09:02 27575404 /lib/libncurses.so.5.7 7fde40b12000-7fde40b14000 r-xp 00000000 09:02 27574280 /lib/libutiAborted early with precompiled: > Fatal error in cpyext, CPython compatibility layer, calling PyTuple_SetItem > Either report a bug or consider not using this particular extension > > RPython traceback: > File "module_cpyext_pyobject.c", line 515, in make_ref > File "rpython_lltypesystem_rdict.c", line 425, in ll_dict_getitem__dicttablePtr_objectPtr > Segmentation fault script: # -*- coding: utf8 -*- """ ???????? ?????????????? ???? ?????????????????????? """ import sys import os import time path = os.path.normpath(os.path.join(os.getcwd(), '..')) sys.path.append(path) #from django.core.management import setup_environ #import dev_settings #setup_environ(dev_settings) from django.db.models.query_utils import Q from idea.vasya.models import Idei74Matrix, ProviderMatrix, ProviderMatrixAnalogs, Providers #from vasya.update_signature import BITS BITS = { u'4': 1152921504606846976, u'5': 2305843009213693952, u'1': 144115188075855872, u'0': 72057594037927936, u'3': 576460752303423488, u'??': 268435456, u'??': 134217728, u'??': 1073741824, u'??': 536870912, u'??': 4294967296, u'??': 2147483648, u'??': 17179869184, u'??': 8589934592, u'7': 288230376151711744, u'??': 34359738368, u'??': 137438953472, u'??': 68719476736, u'??': 549755813888, u'??': 274877906944, u'??': 2199023255552, u'??': 1099511627776, u'??': 8796093022208, u'??': 4398046511104, u'??': 35184372088832, u'??': 17592186044416, u'??': 140737488355328, u'??': 70368744177664, u'??': 562949953421312, u'??': 281474976710656, u'??': 2251799813685248, u'??': 1125899906842624, u'6': 144115188075855872, u'??': 4503599627370496, u'??': 36028797018963968, u'??': 9007199254740992, u'9': 1152921504606846976, u'8': 576460752303423488, u'2': 288230376151711744, u'a': 2, u'c': 8, u'b': 4, u'e': 32, u'd': 16, u'g': 128, u'f': 64, u'i': 512, u'h': 256, u'k': 2048, u'j': 1024, u'm': 8192, u'l': 4096, u'o': 32768, u'n': 16384, u'q': 131072, u'p': 65536, u's': 524288, u'r': 262144, u'u': 2097152, u't': 1048576, u'w': 8388608, u'v': 4194304, u'y': 33554432, u'x': 16777216, u'z': 67108864, } from Queue import Queue from threading import Thread class Worker(Thread): def __init__(self, tasks): Thread.__init__(self) self.tasks = tasks self.daemon = True self.start() def __del__(self): self.tasks = None self.daemon = None def run(self): while True: func, args, kwargs = self.tasks.get() try: func(*args, **kwargs) except Exception, e: print "%s %s %s" % (e, args, kwargs) self.tasks.task_done() class ThreadPool: def __init__(self, num_threads): self.tasks = Queue(num_threads) for _ in range(num_threads): Worker(self.tasks) def __del__(self): self.tasks = None def add_task(self, func, *args, **kargs): self.tasks.put((func, args, kargs)) def awaiting_completion(self): self.tasks.join() def start_check(matrix, provider_items): # print matrix.id """?????????????????? ?????? ???? ?????????????? ??????????????????????""" # todo: ?????????????????? ????????????????????! src_pma = ProviderMatrixAnalogs.objects.filter(provider__in=provider_items, idei__id=matrix.id, provider__provider=provider_items[0].provider.pk, bad_variant=True) src_pma = [pma.provider.id for pma in src_pma] if src_pma: provider_items = provider_items.filter(~Q(id__in=src_pma)) matrix_hashes = matrix.hash.split(":::") if matrix.hash else [] matrix_frases = matrix.frase.split(":::") if matrix.frase else [] try: for item in provider_items: # print "item: %s" % item.id find_zero = False # ?????????? ???? ???????????????? hashes = item.hash.split(":::") frases = item.frase.split(":::") for i, hash in enumerate(hashes): if not find_zero: for ii, mh in enumerate(matrix_hashes): if not find_zero and mh and hash: # ?????????????????? ???? ???????? r = int(mh) ^ int(hash) l = 0 for b in BITS: if r & BITS[b]: l += 1 if l > 2: break if l < 3: d = distance(matrix_frases[ii], frases[i]) #if src_pma and src_pma.filter(distance=d, algorythm=('alg%s' % i), provider=item.pk): # continue if d < 3: #print u"%s::%s" % (matrix.name, item.name) pma = ProviderMatrixAnalogs() pma.idei = matrix pma.provider = item pma.distance = d pma.algorythm = 'alg%s' % i if not d: find_zero = True pma.is_analog = True # ???????? ???? ?????????? ???????????? ???????????????????? ???? ?????????????? ??????????????????, ???? ???????????????????? ?????? ?????????????????? #ProviderMatrixAnalogs.objects.filter(idei=matrix.pk, provider__provider=item.provider.pk, is_analog=False).delete() pass pma.save(using='master') except ValueError, e: print "error on matrix item id: %s" % matrix.id print "e: %s" % e print "matrix_hashes: %s" % matrix_hashes print "matrix_frases: %s" % matrix_frases print "hashes: %s" % hashes print "frases: %s" % frases print "-----" def distance(a, b): "Calculates the Levenshtein distance between a and b." n, m = len(a), len(b) if n > m: # Make sure n <= m, to use O(min(n,m)) space a, b = b, a n, m = m, n current_row = range(n + 1) # Keep current and previous row, not entire matrix for i in range(1, m + 1): previous_row, current_row = current_row, [i] + [0] * m for j in range(1, n + 1): add, delete, change = previous_row[j] + 1, current_row[j - 1] + 1, previous_row[j - 1] if a[j - 1] != b[i - 1]: change += 1 current_row[j] = min(add, delete, change) return current_row[n] pool = ThreadPool(16) idea_items = Idei74Matrix.objects.filter(~Q(group=221) & Q(show=True)) providers = Providers.objects.all() #providers = Providers.objects.filter(id__in=[22, 23, 25]) # cool!! providers_id= map(lambda obj: obj.id, providers) for p in providers: print p # ?????? ?????????????? ???????????????????? ?????????????? ???????? ?????????? ?????????????? ???? ???????????? # ?????? ?????????????????????? ?????????????? analogs_items = ProviderMatrixAnalogs.objects.filter(Q(provider__provider__id=p.id) & Q(is_analog=True) & Q(provider__show=True)) # ?????????????? ?????????????? ???????????????? ???????????? ?????????????????? bad_items = ProviderMatrixAnalogs.objects.filter(Q(provider__provider__id=p.id) & Q(bad_variant=True)) # ???????????? ID ?????????????? ???????? ?????????????????????? ???? ?????????????????? ?????? ?????????????? idea_items_new = [obj.idei.id for obj in analogs_items] # ???????????? ID ?????????????? ???????????????????? ?????????????????????? ???? ?????????????????? ?????? ?????????????? provider_items = [obj.provider.id for obj in analogs_items] # ?????????????????? ????????????, ?????????????? ?????? ?????????????? ?? ???????????? ?????????????????? ?? ???????????? idea_items_new = idea_items.filter(~Q(id__in=idea_items_new) & Q(group__providers=p)) # ?????????????????? ????????????, ?????????????? ?????? ?????????????? ?????? ?????????????????????? ?????? ?????????????????? ?????????????? provider_items = ProviderMatrix.objects.filter(Q(provider__id=p.id) & ~Q(id__in=provider_items) & Q(show=True) & Q(hide=False)) if provider_items: for item in idea_items_new: pool.add_task(start_check, item, provider_items) pool.awaiting_completion() From tracker at bugs.pypy.org Mon Oct 10 20:39:29 2011 From: tracker at bugs.pypy.org (Fijal) Date: Mon, 10 Oct 2011 18:39:29 +0000 Subject: [pypy-issue] [issue901] *** glibc detected *** pypy: corrupted double-linked list: 0x000000000bd76de0 *** In-Reply-To: <1318270602.56.0.17883229495.issue901@bugs.pypy.org> Message-ID: <1318271969.84.0.27041097318.issue901@bugs.pypy.org> Fijal added the comment: We need the script I fear ---------- nosy: +fijal status: unread -> chatting ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Mon Oct 10 20:47:16 2011 From: tracker at bugs.pypy.org (Alexander Milenko) Date: Mon, 10 Oct 2011 18:47:16 +0000 Subject: [pypy-issue] [issue901] *** glibc detected *** pypy: corrupted double-linked list: 0x000000000bd76de0 *** In-Reply-To: <1318270602.56.0.17883229495.issue901@bugs.pypy.org> Message-ID: <1318272436.74.0.0485040332359.issue901@bugs.pypy.org> Alexander Milenko added the comment: I do not know how to give information. them more than 5 million records. Before committing an error was 8-hour treatment on the nuclear machine with 8 GB of RAM ________________________________________ PyPy bug tracker ________________________________________ -------------- next part -------------- # -*- coding: utf8 -*- import sys import os import time path = os.path.normpath(os.path.join(os.getcwd(), '..')) sys.path.append(path) from django.db.models.query_utils import Q from idea.vasya.models import Idei74Matrix, ProviderMatrix, ProviderMatrixAnalogs, Providers BITS = { u'4': 1152921504606846976, u'5': 2305843009213693952, u'1': 144115188075855872, u'0': 72057594037927936, u'3': 576460752303423488, u'??': 268435456, u'??': 134217728, u'??': 1073741824, u'??': 536870912, u'??': 4294967296, u'??': 2147483648, u'??': 17179869184, u'??': 8589934592, u'7': 288230376151711744, u'??': 34359738368, u'??': 137438953472, u'??': 68719476736, u'??': 549755813888, u'??': 274877906944, u'??': 2199023255552, u'??': 1099511627776, u'??': 8796093022208, u'??': 4398046511104, u'??': 35184372088832, u'??': 17592186044416, u'??': 140737488355328, u'??': 70368744177664, u'??': 562949953421312, u'??': 281474976710656, u'??': 2251799813685248, u'??': 1125899906842624, u'6': 144115188075855872, u'??': 4503599627370496, u'??': 36028797018963968, u'??': 9007199254740992, u'9': 1152921504606846976, u'8': 576460752303423488, u'2': 288230376151711744, u'a': 2, u'c': 8, u'b': 4, u'e': 32, u'd': 16, u'g': 128, u'f': 64, u'i': 512, u'h': 256, u'k': 2048, u'j': 1024, u'm': 8192, u'l': 4096, u'o': 32768, u'n': 16384, u'q': 131072, u'p': 65536, u's': 524288, u'r': 262144, u'u': 2097152, u't': 1048576, u'w': 8388608, u'v': 4194304, u'y': 33554432, u'x': 16777216, u'z': 67108864, } from Queue import Queue from threading import Thread class Worker(Thread): def __init__(self, tasks): Thread.__init__(self) self.tasks = tasks self.daemon = True self.start() def __del__(self): self.tasks = None self.daemon = None def run(self): while True: func, args, kwargs = self.tasks.get() try: func(*args, **kwargs) except Exception, e: print "%s %s %s" % (e, args, kwargs) self.tasks.task_done() class ThreadPool: def __init__(self, num_threads): self.tasks = Queue(num_threads) for _ in range(num_threads): Worker(self.tasks) def __del__(self): self.tasks = None def add_task(self, func, *args, **kargs): self.tasks.put((func, args, kargs)) def awaiting_completion(self): self.tasks.join() def start_check(matrix, provider_items): src_pma = ProviderMatrixAnalogs.objects.filter(provider__in=provider_items, idei__id=matrix.id, provider__provider=provider_items[0].provider.pk, bad_variant=True) src_pma = [pma.provider.id for pma in src_pma] if src_pma: provider_items = provider_items.filter(~Q(id__in=src_pma)) matrix_hashes = matrix.hash.split(":::") if matrix.hash else [] matrix_frases = matrix.frase.split(":::") if matrix.frase else [] try: for item in provider_items: find_zero = False hashes = item.hash.split(":::") frases = item.frase.split(":::") for i, hash in enumerate(hashes): if not find_zero: for ii, mh in enumerate(matrix_hashes): if not find_zero and mh and hash: r = int(mh) ^ int(hash) l = 0 for b in BITS: if r & BITS[b]: l += 1 if l > 2: break if l < 3: d = distance(matrix_frases[ii], frases[i]) if d < 3: pma = ProviderMatrixAnalogs() pma.idei = matrix pma.provider = item pma.distance = d pma.algorythm = 'alg%s' % i if not d: find_zero = True pma.is_analog = True pma.save(using='master') except ValueError, e: print "error on matrix item id: %s" % matrix.id print "e: %s" % e print "matrix_hashes: %s" % matrix_hashes print "matrix_frases: %s" % matrix_frases print "hashes: %s" % hashes print "frases: %s" % frases print "-----" def distance(a, b): "Calculates the Levenshtein distance between a and b." n, m = len(a), len(b) if n > m: # Make sure n <= m, to use O(min(n,m)) space a, b = b, a n, m = m, n current_row = range(n + 1) # Keep current and previous row, not entire matrix for i in range(1, m + 1): previous_row, current_row = current_row, [i] + [0] * m for j in range(1, n + 1): add, delete, change = previous_row[j] + 1, current_row[j - 1] + 1, previous_row[j - 1] if a[j - 1] != b[i - 1]: change += 1 current_row[j] = min(add, delete, change) return current_row[n] pool = ThreadPool(16) idea_items = Idei74Matrix.objects.filter(~Q(group=221) & Q(show=True)) providers = Providers.objects.all() for p in providers: analogs_items = ProviderMatrixAnalogs.objects.filter(Q(provider__provider__id=p.id) & Q(is_analog=True) & Q(provider__show=True)) bad_items = ProviderMatrixAnalogs.objects.filter(Q(provider__provider__id=p.id) & Q(bad_variant=True)) idea_items_new = [obj.idei.id for obj in analogs_items] provider_items = [obj.provider.id for obj in analogs_items] idea_items_new = idea_items.filter(~Q(id__in=idea_items_new) & Q(group__providers=p)) provider_items = ProviderMatrix.objects.filter(Q(provider__id=p.id) & ~Q(id__in=provider_items) & Q(show=True) & Q(hide=False)) if provider_items: for item in idea_items_new: pool.add_task(start_check, item, provider_items) pool.awaiting_completion() From tracker at bugs.pypy.org Tue Oct 11 21:58:51 2011 From: tracker at bugs.pypy.org (Shawn Walker) Date: Tue, 11 Oct 2011 19:58:51 +0000 Subject: [pypy-issue] [issue902] pypy translation build fails due to non-POSIX time field usage In-Reply-To: <1318363131.14.0.376606025293.issue902@bugs.pypy.org> Message-ID: <1318363131.14.0.376606025293.issue902@bugs.pypy.org> New submission from Shawn Walker : While compiling a current pypy hg clone on Solaris, I encountered the following compilation error: [platform:execute] /usr/gcc/4.5/bin/gcc -c -O3 -pthread -fomit-frame-pointer - Wall -Wno-unused -std=gnu99 -D__EXTENSIONS__ -DSEM_VALUE_MAX=INT_MAX /tmp/usession-default-17/platcheck_45.c -o /tmp/usession-default- 17/platcheck_45.o [platform:Error] /tmp/usession-default-17/platcheck_45.c: In function ?dump_section_tm?: [platform:Error] /tmp/usession-default-17/platcheck_45.c:116:27: error: ?platcheck_t? has no member named ?tm_gmtoff? [platform:Error] /tmp/usession-default-17/platcheck_45.c:117:38: error: ?platcheck_t? has no member named ?tm_gmtoff? [platform:Error] /tmp/usession-default-17/platcheck_45.c:118:3: error: ?platcheck_t? has no member named ?tm_gmtoff? [platform:Error] /tmp/usession-default-17/platcheck_45.c:118:20: error: ?platcheck_t? has no member named ?tm_gmtoff? [platform:Error] /tmp/usession-default-17/platcheck_45.c:118:35: error: ?platcheck_t? has no member named ?tm_gmtoff? [platform:Error] /tmp/usession-default-17/platcheck_45.c:119:33: error: ?platcheck_t? has no member named ?tm_gmtoff? [platform:Error] /tmp/usession-default-17/platcheck_45.c:120:25: error: ?platcheck_t? has no member named ?tm_zone? [platform:Error] /tmp/usession-default-17/platcheck_45.c:121:36: error: ?platcheck_t? has no member named ?tm_zone? [Timer] Timings: [Timer] annotate --- 972.1 s [Timer] ========================================== [Timer] Total: --- 972.1 s [translation:ERROR] Error: [translation:ERROR] Traceback (most recent call last): [translation:ERROR] File "translate.py", line 308, in main [translation:ERROR] drv.proceed(goals) [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/translator/driver.py", line 809, in proceed [translation:ERROR] return self._execute(goals, task_skip = self._maybe_skip()) [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/translator/tool/taskengine.py", line 116, in _execute [translation:ERROR] res = self._do(goal, taskcallable, *args, **kwds) [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/translator/driver.py", line 286, in _do [translation:ERROR] res = func() [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/translator/driver.py", line 323, in task_annotate [translation:ERROR] s = annotator.build_types(self.entry_point, self.inputtypes) [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/annotation/annrpython.py", line 103, in build_types [translation:ERROR] return self.build_graph_types(flowgraph, inputcells, complete_now=complete_now) [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/annotation/annrpython.py", line 194, in build_graph_types [translation:ERROR] self.complete() [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/annotation/annrpython.py", line 250, in complete [translation:ERROR] self.processblock(graph, block) [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/annotation/annrpython.py", line 448, in processblock [translation:ERROR] self.flowin(graph, block) [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/annotation/annrpython.py", line 508, in flowin [translation:ERROR] self.consider_op(block.operations[i]) [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/annotation/annrpython.py", line 710, in consider_op [translation:ERROR] raise_nicer_exception(op, str(graph)) [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/annotation/annrpython.py", line 707, in consider_op [translation:ERROR] resultcell = consider_meth(*argcells) [translation:ERROR] File "<4160-codegen /export/home/swalker/devel/pypy/pypy/annotation/annrpython.py:745>", line 3, in consider_op_getattr [translation:ERROR] return arg.getattr(*args) [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/annotation/unaryop.py", line 687, in getattr [translation:ERROR] return bookkeeper.pbc_getattr(pbc, s_attr) [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/annotation/bookkeeper.py", line 603, in pbc_getattr [translation:ERROR] return first.s_read_attribute(attr) [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/annotation/description.py", line 918, in s_read_attribute [translation:ERROR] return self.bookkeeper.immutablevalue(value) [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/annotation/bookkeeper.py", line 396, in immutablevalue [translation:ERROR] result.dictdef.generalize_value(self.immutablevalue(ev)) [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/annotation/bookkeeper.py", line 469, in immutablevalue [translation:ERROR] elif hasattr(x, '_freeze_') and x._freeze_(): [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/interpreter/mixedmodule.py", line 128, in _freeze_ [translation:ERROR] self.getdict(self.space) [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/interpreter/mixedmodule.py", line 121, in getdict [translation:ERROR] w_value = self.get(name) [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/interpreter/mixedmodule.py", line 74, in get [translation:ERROR] w_value = self.getdictvalue(space, name) [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/interpreter/mixedmodule.py", line 86, in getdictvalue [translation:ERROR] return self._load_lazily(space, name) [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/interpreter/mixedmodule.py", line 96, in _load_lazily [translation:ERROR] w_value = loader(space) [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/interpreter/mixedmodule.py", line 204, in ifileloader [translation:ERROR] d[name] = __import__(pkgroot+'.'+name, None, None, [name]) [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/module/rctime/interp_time.py", line 125, in [translation:ERROR] for k, v in platform.configure(CConfig).items(): [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/rpython/tool/rffi_platform.py", line 203, in configure [translation:ERROR] ignore_errors=ignore_errors)) [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/rpython/tool/rffi_platform.py", line 687, in run_example_code [translation:ERROR] output = build_executable_cache(files, eci, ignore_errors=ignore_errors) [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/tool/gcc_cache.py", line 31, in build_executable_cache [translation:ERROR] result = platform.execute(platform.compile(c_files, eci)) [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/translator/platform/__init__.py", line 51, in compile [translation:ERROR] ofiles = self._compile_o_files(cfiles, eci, standalone) [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/translator/platform/__init__.py", line 60, in _compile_o_files [translation:ERROR] ofiles.append(self._compile_c_file(self.cc, cfile, compile_args)) [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/translator/platform/posix.py", line 38, in _compile_c_file [translation:ERROR] cwd=str(cfile.dirpath())) [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/translator/platform/__init__.py", line 118, in _execute_c_compiler [translation:ERROR] self._handle_error(returncode, stdout, stderr, outname) [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/translator/platform/__init__.py", line 130, in _handle_error [translation:ERROR] raise CompilationError(stdout, stderr) [translation:ERROR] CompilationError': CompilationError(err=""" [translation:ERROR] /tmp/usession-default-17/platcheck_45.c: In function ?dump_section_tm?: [translation:ERROR] /tmp/usession-default-17/platcheck_45.c:116:27: error: ?platcheck_t? has no member named ?tm_gmtoff? [translation:ERROR] /tmp/usession-default-17/platcheck_45.c:117:38: error: ?platcheck_t? has no member named ?tm_gmtoff? [translation:ERROR] /tmp/usession-default-17/platcheck_45.c:118:3: error: ?platcheck_t? has no member named ?tm_gmtoff? [translation:ERROR] /tmp/usession-default-17/platcheck_45.c:118:20: error: ?platcheck_t? has no member named ?tm_gmtoff? [translation:ERROR] /tmp/usession-default-17/platcheck_45.c:118:35: error: ?platcheck_t? has no member named ?tm_gmtoff? [translation:ERROR] /tmp/usession-default-17/platcheck_45.c:119:33: error: ?platcheck_t? has no member named ?tm_gmtoff? [translation:ERROR] /tmp/usession-default-17/platcheck_45.c:120:25: error: ?platcheck_t? has no member named ?tm_zone? [translation:ERROR] /tmp/usession-default-17/platcheck_45.c:121:36: error: ?platcheck_t? has no member named ?tm_zone? [translation:ERROR] """) [translation:ERROR] .. v10 = getattr(space_0, ('builtin_modules')) [translation:ERROR] .. '(pypy.module.imp.interp_imp:140)init_builtin' [translation:ERROR] Processing block: [translation:ERROR] block at 3 is a [translation:ERROR] in (pypy.module.imp.interp_imp:140)init_builtin [translation:ERROR] containing the following operations: [translation:ERROR] v11 = getattr(space_0, ('str_w')) [translation:ERROR] v12 = simple_call(v11, w_name_0) [translation:ERROR] v10 = getattr(space_0, ('builtin_modules')) [translation:ERROR] v13 = contains(v10, v12) [translation:ERROR] v14 = is_true(v13) [translation:ERROR] --end-- tm_gmtoff and tm_zone are BSD/GNU libc extensions and are not part of the POSIX standard or strictly ISO C conforming environments. Solaris does not provide these fields at all. What confuses me is that pypy/config/pypyoption.py seems to account for this: 68 if sys.platform == "sunos5": 69 del working_modules['mmap'] # depend on ctypes, can't get at c-level 'errono' 70 del working_modules['rctime'] # depend on ctypes, missing tm_zone/tm_gmtoff ---------- messages: 3292 nosy: binarycrusader, pypy-issue priority: bug release: 1.6 status: unread title: pypy translation build fails due to non-POSIX time field usage ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Tue Oct 11 23:21:25 2011 From: tracker at bugs.pypy.org (Alex Gaynor) Date: Tue, 11 Oct 2011 21:21:25 +0000 Subject: [pypy-issue] [issue903] [list-strategies] expose __pypy__.newlist() In-Reply-To: <1318368085.8.0.525402249889.issue903@bugs.pypy.org> Message-ID: <1318368085.8.0.525402249889.issue903@bugs.pypy.org> New submission from Alex Gaynor : Now that we've moved map() to app-level we'd like to expose newlist(), however we're waiting for list-strategies because it exposes an issue. How do you pre- allocate a list when you don't know what type it'll have? The proposed solution is for newlist() to create an empty list with a PreallocateStrategy, this strategy promotes as normal to a specialized strategy, but uses rlib.objectmodule.newlist to preallocate tehe list. ---------- messages: 3293 nosy: agaynor, pypy-issue priority: feature status: unread title: [list-strategies] expose __pypy__.newlist() ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Tue Oct 11 23:56:07 2011 From: tracker at bugs.pypy.org (Shawn Walker) Date: Tue, 11 Oct 2011 21:56:07 +0000 Subject: [pypy-issue] [issue904] pypy build translation fails due to use of non-standard ifr_ifindex In-Reply-To: <1318370167.35.0.638855593826.issue904@bugs.pypy.org> Message-ID: <1318370167.35.0.638855593826.issue904@bugs.pypy.org> New submission from Shawn Walker : While compiling an hg clone of pypy trunk, I encountered the following compilation error: $ python2.6 translate.py -Ojit [platform:msg] Setting platform to 'host' cc=None [translation:info] Translating target as defined by targetpypystandalone ... [platform:Error] /tmp/usession-default-21/platcheck_4.c: In function ?dump_section_ifreq?: [platform:Error] /tmp/usession-default-21/platcheck_4.c:4327:29: error: ?platcheck_t? has no member named ?ifr_ifindex? [platform:Error] /tmp/usession-default-21/platcheck_4.c:4328:40: error: ?platcheck_t? has no member named ?ifr_ifindex? [platform:Error] /tmp/usession-default-21/platcheck_4.c:4329:3: error: ?platcheck_t? has no member named ?ifr_ifindex? [platform:Error] /tmp/usession-default-21/platcheck_4.c:4329:22: error: ?platcheck_t? has no member named ?ifr_ifindex? [platform:Error] /tmp/usession-default-21/platcheck_4.c:4329:39: error: ?platcheck_t? has no member named ?ifr_ifindex? [platform:Error] /tmp/usession-default-21/platcheck_4.c:4330:35: error: ?platcheck_t? has no member named ?ifr_ifindex? ... platform:execute] /usr/gcc/4.5/bin/gcc -c -O3 -pthread -fomit-frame-pointer - Wall -Wno-unused -std=gnu99 -D__EXTENSIONS__ -DSEM_VALUE_MAX=INT_MAX /tmp/usession-default-21/platcheck_27.c -o /tmp/usession-default- 21/platcheck_27.o [platform:Error] /tmp/usession-default-21/platcheck_27.c: In function ?dump_section_ifreq?: [platform:Error] /tmp/usession-default-21/platcheck_27.c:4327:29: error: ?platcheck_t? has no member named ?ifr_ifindex? [platform:Error] /tmp/usession-default-21/platcheck_27.c:4328:40: error: ?platcheck_t? has no member named ?ifr_ifindex? [platform:Error] /tmp/usession-default-21/platcheck_27.c:4329:3: error: ?platcheck_t? has no member named ?ifr_ifindex? [platform:Error] /tmp/usession-default-21/platcheck_27.c:4329:22: error: ?platcheck_t? has no member named ?ifr_ifindex? [platform:Error] /tmp/usession-default-21/platcheck_27.c:4329:39: error: ?platcheck_t? has no member named ?ifr_ifindex? [platform:Error] /tmp/usession-default-21/platcheck_27.c:4330:35: error: ?platcheck_t? has no member named ?ifr_ifindex? ... [translation] PyPy config object: [translation] [objspace] [translation] honor__builtins__ = False [translation] nofaking = True [translation] [opcodes] [translation] CALL_METHOD = True [translation] [std] [translation] builtinshortcut = True [translation] getattributeshortcut = True [translation] newshortcut = True [translation] optimized_list_getitem = True [translation] withcelldict = True [translation] withmapdict = True [translation] withmethodcache = True [translation] withprebuiltchar = True [translation] withrangelist = True [translation] withtypeversion = True [translation] [usemodules] [translation] __pypy__ = True [translation] _ast = True [translation] _bisect = True [translation] _codecs = True [translation] _collections = True [translation] _continuation = True [translation] _ffi = True [translation] _io = True [translation] _locale = True [translation] _lsprof = True [translation] _md5 = True [translation] _multibytecodec = True [translation] _multiprocessing = True [translation] _pickle_support = True [translation] _random = True [translation] _rawffi = True [translation] _sha = True [translation] _socket = True [translation] _sre = True [translation] _testing = True [translation] _warnings = True [translation] _weakref = True [translation] array = True [translation] binascii = True [translation] bz2 = True [translation] cStringIO = True [translation] cmath = True [translation] cpyext = True [translation] crypt = True [translation] errno = True [translation] gc = True [translation] imp = True [translation] itertools = True [translation] marshal = True [translation] math = True [translation] micronumpy = True [translation] operator = True [translation] parser = True [translation] posix = True [translation] pwd = True [translation] pyexpat = True [translation] pypyjit = True [translation] rctime = False [translation] select = True [translation] struct = True [translation] symbol = True [translation] thread = True [translation] time = True [translation] token = True [translation] unicodedata = True [translation] zipimport = True [translation] zlib = True [translation:WARNING] The module '_hashlib' is disabled [translation:WARNING] because importing pypy.module._ssl.interp_ssl raised CompilationError [translation:WARNING] CompilationError(err=""" [translation:WARNING] /tmp/usession-default-21/platcheck_4.c: In function ?dump_section_ifreq?: [translation:WARNING] /tmp/usession-default-21/platcheck_4.c:4327:29: error: ?platcheck_t? has no member named ?ifr_ifindex? [translation:WARNING] /tmp/usession-default-21/platcheck_4.c:4328:40: error: ?platcheck_t? has no member named ?ifr_ifindex? [translation:WARNING] /tmp/usession-default-21/platcheck_4.c:4329:3: error: ?platcheck_t? has no member named ?ifr_ifindex? [translation:WARNING] /tmp/usession-default-21/platcheck_4.c:4329:22: error: ?platcheck_t? has no member named ?ifr_ifindex? [translation:WARNING] /tmp/usession-default-21/platcheck_4.c:4329:39: error: ?platcheck_t? has no member named ?ifr_ifindex? [translation:WARNING] /tmp/usession-default-21/platcheck_4.c:4330:35: error: ?platcheck_t? has no member named ?ifr_ifindex? [translation:WARNING] """) [translation:WARNING] The module '_ssl' is disabled [translation:WARNING] because importing pypy.module._ssl.interp_ssl raised CompilationError [translation:WARNING] CompilationError(err=""" [translation:WARNING] /tmp/usession-default-21/platcheck_27.c: In function ?dump_section_ifreq?: [translation:WARNING] /tmp/usession-default-21/platcheck_27.c:4327:29: error: ?platcheck_t? has no member named ?ifr_ifindex? [translation:WARNING] /tmp/usession-default-21/platcheck_27.c:4328:40: error: ?platcheck_t? has no member named ?ifr_ifindex? [translation:WARNING] /tmp/usession-default-21/platcheck_27.c:4329:3: error: ?platcheck_t? has no member named ?ifr_ifindex? [translation:WARNING] /tmp/usession-default-21/platcheck_27.c:4329:22: error: ?platcheck_t? has no member named ?ifr_ifindex? [translation:WARNING] /tmp/usession-default-21/platcheck_27.c:4329:39: error: ?platcheck_t? has no member named ?ifr_ifindex? [translation:WARNING] /tmp/usession-default-21/platcheck_27.c:4330:35: error: ?platcheck_t? has no member named ?ifr_ifindex? [translation:WARNING] """) ... [platform:execute] /usr/gcc/4.5/bin/gcc -c -O3 -pthread -fomit-frame-pointer - Wall -Wno-unused -std=gnu99 -D__EXTENSIONS__ -DSEM_VALUE_MAX=INT_MAX /tmp/usession-default-21/platcheck_31.c -o /tmp/usession-default- 21/platcheck_31.o [platform:Error] /tmp/usession-default-21/platcheck_31.c: In function ?dump_section_ifreq?: [platform:Error] /tmp/usession-default-21/platcheck_31.c:4327:29: error: ?platcheck_t? has no member named ?ifr_ifindex? [platform:Error] /tmp/usession-default-21/platcheck_31.c:4328:40: error: ?platcheck_t? has no member named ?ifr_ifindex? [platform:Error] /tmp/usession-default-21/platcheck_31.c:4329:3: error: ?platcheck_t? has no member named ?ifr_ifindex? [platform:Error] /tmp/usession-default-21/platcheck_31.c:4329:22: error: ?platcheck_t? has no member named ?ifr_ifindex? [platform:Error] /tmp/usession-default-21/platcheck_31.c:4329:39: error: ?platcheck_t? has no member named ?ifr_ifindex? [platform:Error] /tmp/usession-default-21/platcheck_31.c:4330:35: error: ?platcheck_t? has no member named ?ifr_ifindex? [translation:ERROR] Error: [translation:ERROR] Traceback (most recent call last): [translation:ERROR] File "translate.py", line 275, in main [translation:ERROR] default_goal='compile') [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/translator/driver.py", line 828, in from_targetspec [translation:ERROR] spec = target(driver, args) [translation:ERROR] File "targetpypystandalone.py", line 226, in target [translation:ERROR] return self.get_entry_point(config) [translation:ERROR] File "targetpypystandalone.py", line 237, in get_entry_point [translation:ERROR] space = make_objspace(config) [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/tool/option.py", line 48, in make_objspace [translation:ERROR] space = Space(config) [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/interpreter/baseobjspace.py", line 308, in __init__ [translation:ERROR] self.initialize() [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/objspace/std/objspace.py", line 75, in initialize [translation:ERROR] self.make_builtins() [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/interpreter/baseobjspace.py", line 525, in make_builtins [translation:ERROR] self.install_mixedmodule(mixedname, installed_builtin_modules) [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/interpreter/baseobjspace.py", line 564, in install_mixedmodule [translation:ERROR] modname = self.setbuiltinmodule(mixedname) [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/interpreter/baseobjspace.py", line 405, in setbuiltinmodule [translation:ERROR] mod = Module(self, self.wrap(name)) [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/interpreter/mixedmodule.py", line 26, in __init__ [translation:ERROR] self.__class__.buildloaders() [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/module/_socket/__init__.py", line 24, in buildloaders [translation:ERROR] from pypy.rlib import rsocket [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/rlib/rsocket.py", line 18, in [translation:ERROR] from pypy.rlib import _rsocket_rffi as _c [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/rlib/_rsocket_rffi.py", line 369, in [translation:ERROR] cConfig.__dict__.update(platform.configure(CConfig)) [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/rpython/tool/rffi_platform.py", line 203, in configure [translation:ERROR] ignore_errors=ignore_errors)) [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/rpython/tool/rffi_platform.py", line 687, in run_example_code [translation:ERROR] output = build_executable_cache(files, eci, ignore_errors=ignore_errors) [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/tool/gcc_cache.py", line 31, in build_executable_cache [translation:ERROR] result = platform.execute(platform.compile(c_files, eci)) [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/translator/platform/__init__.py", line 51, in compile [translation:ERROR] ofiles = self._compile_o_files(cfiles, eci, standalone) [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/translator/platform/__init__.py", line 60, in _compile_o_files [translation:ERROR] ofiles.append(self._compile_c_file(self.cc, cfile, compile_args)) [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/translator/platform/posix.py", line 38, in _compile_c_file [translation:ERROR] cwd=str(cfile.dirpath())) [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/translator/platform/__init__.py", line 118, in _execute_c_compiler [translation:ERROR] self._handle_error(returncode, stdout, stderr, outname) [translation:ERROR] File "/export/home/swalker/devel/pypy/pypy/translator/platform/__init__.py", line 130, in _handle_error [translation:ERROR] raise CompilationError(stdout, stderr) [translation:ERROR] CompilationError: CompilationError(err=""" [translation:ERROR] /tmp/usession-default-21/platcheck_31.c: In function ?dump_section_ifreq?: [translation:ERROR] /tmp/usession-default-21/platcheck_31.c:4327:29: error: ?platcheck_t? has no member named ?ifr_ifindex? [translation:ERROR] /tmp/usession-default-21/platcheck_31.c:4328:40: error: ?platcheck_t? has no member named ?ifr_ifindex? [translation:ERROR] /tmp/usession-default-21/platcheck_31.c:4329:3: error: ?platcheck_t? has no member named ?ifr_ifindex? [translation:ERROR] /tmp/usession-default-21/platcheck_31.c:4329:22: error: ?platcheck_t? has no member named ?ifr_ifindex? [translation:ERROR] /tmp/usession-default-21/platcheck_31.c:4329:39: error: ?platcheck_t? has no member named ?ifr_ifindex? [translation:ERROR] /tmp/usession-default-21/platcheck_31.c:4330:35: error: ?platcheck_t? has no member named ?ifr_ifindex? [translation:ERROR] """) ... A possibly related Python bug where this was worked around for CPython: http://bugs.python.org/issue8852 ---------- messages: 3294 nosy: pypy-issue priority: bug release: 1.6 status: unread title: pypy build translation fails due to use of non-standard ifr_ifindex ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Wed Oct 12 01:01:27 2011 From: tracker at bugs.pypy.org (David Markey) Date: Tue, 11 Oct 2011 23:01:27 +0000 Subject: [pypy-issue] [issue905] ZipFile.extractall is very slow compared to CPython 2.6 In-Reply-To: <1318374087.9.0.996755989845.issue905@bugs.pypy.org> Message-ID: <1318374087.9.0.996755989845.issue905@bugs.pypy.org> New submission from David Markey : (env)dmarkey at dubsdavidmar02:~$ du -k test_zip.zip 21024 test_zip.zip (env)dmarkey at dubsdavidmar02:~$ date python zipfile_test.py date: extra operand `zipfile_test.py' Try `date --help' for more information. (env)dmarkey at dubsdavidmar02:~$ time python zipfile_test.py real 0m1.834s user 0m1.160s sys 0m0.660s (env)dmarkey at dubsdavidmar02:~$ time Downloads/pypy-c-jit-43780-b590cf6de419- linux64/bin/pypy zipfile_test.py real 0m4.712s user 0m4.030s sys 0m0.630s ---------- files: zipfile_test.py messages: 3295 nosy: dmarkey, pypy-issue priority: bug status: unread title: ZipFile.extractall is very slow compared to CPython 2.6 ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Wed Oct 12 01:11:42 2011 From: tracker at bugs.pypy.org (David Markey) Date: Tue, 11 Oct 2011 23:11:42 +0000 Subject: [pypy-issue] [issue905] ZipFile.extractall is very slow compared to CPython 2.6 In-Reply-To: <1318374087.9.0.996755989845.issue905@bugs.pypy.org> Message-ID: <1318374702.72.0.337323529626.issue905@bugs.pypy.org> David Markey added the comment: This time on 1.6/32bit (env)dmarkey at dubsdavidmar02:~$ time python zipfile_test.py real 0m2.192s user 0m1.430s sys 0m0.730s (env)dmarkey at dubsdavidmar02:~$ time Downloads/pypy-1.6/bin/pypy zipfile_test.py real 0m4.387s user 0m3.760s sys 0m0.580s (env)dmarkey at dubsdavidmar02:~$ Downloads/pypy-1.6/bin/pypy -V Python 2.7.1 (d8ac7d23d3ec, Aug 17 2011, 11:51:18) [PyPy 1.6.0 with GCC 4.4.3] ---------- status: unread -> chatting ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Wed Oct 12 12:11:20 2011 From: tracker at bugs.pypy.org (Valery) Date: Wed, 12 Oct 2011 10:11:20 +0000 Subject: [pypy-issue] [issue846] TypeError: type 'basestring' is not an acceptable base class In-Reply-To: <1314298597.06.0.673686395864.issue846@bugs.pypy.org> Message-ID: <1318414280.61.0.218656134851.issue846@bugs.pypy.org> Valery added the comment: btw, nltk should now be compilable & installable out-of-the-box: http://code.google.com/p/nltk/issues/detail?id=721 Could anyone check please? ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Wed Oct 12 16:23:25 2011 From: tracker at bugs.pypy.org (Valery) Date: Wed, 12 Oct 2011 14:23:25 +0000 Subject: [pypy-issue] [issue906] CTRL-leftarrow and CTRL-rightarrow don't work in PyPy console In-Reply-To: <1318429405.87.0.821520990977.issue906@bugs.pypy.org> Message-ID: <1318429405.87.0.821520990977.issue906@bugs.pypy.org> New submission from Valery : they are supposed to make jumps over the "words" ---------- messages: 3298 nosy: pypy-issue, vak priority: bug status: unread title: CTRL-leftarrow and CTRL-rightarrow don't work in PyPy console ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Thu Oct 13 18:27:44 2011 From: tracker at bugs.pypy.org (Laurence Tratt) Date: Thu, 13 Oct 2011 16:27:44 +0000 Subject: [pypy-issue] [issue907] Stacklets running out of memory on higher optimisation levels In-Reply-To: <1318523264.95.0.361208519335.issue907@bugs.pypy.org> Message-ID: <1318523264.95.0.361208519335.issue907@bugs.pypy.org> New submission from Laurence Tratt : I've finally gotten around to implementing generator support in my prototype Converge VM. For simple examples, this was super-duper easy to get up and running - I was very impressed! However, for more involved examples, the stacklet approach seems to run out of memory on --opt=2, --opt=3, and --opt=jit and then dumps out a 2G core file (which probably doesn't do my SSD much good!). I must admit, I have yet to work out why this is happening, as I'm fairly sure I'm destroying every stacklet that's created as soon as I can reasonably do so, and there doesn't appear to be a problem on --opt=1 or --opt=0. Here's an example error with --opt=3: RPython traceback: File "implement.c", line 310, in entry_point File "implement.c", line 5758, in VM_apply_closure File "implement.c", line 10764, in VM_execute File "implement.c", line 13910, in VM_bc_loop File "implement.c", line 17090, in VM__instr_apply File "implement.c", line 22377, in VM__apply_pump File "rpython_memory_gctransform_shadowstack.c", line 416, in ShadowStackPool_save_current_state_away Fatal RPython error: MemoryError You can see the full VM code here: https://github.com/ltratt/converge/tree/pypyvm/pypyvm Here's an example Converge program which causes things to blow up: func g(): i := 10 while i > 0: yield i i -= 1 fail func main(): i := 5000000 while i > 0: for g() I attach a bytecode version of that file to this bug report so that you can execute things with this command: ./main-c t For me, the blow-up (on --opt=3) happens a few iterations into the while loop executing the "for g()" statement. The switching code is in VM.py - there is only one stacklet creation site, and two switches, so it's reasonably easy to see what's going on, I hope. I will warn you that it's possible I'm doing something very stupid here. However, because there are no errors in --opt=1, but there are in --opt=3, it seems possible that there's something a bit squiffy going on in PyPy, perhaps in the interaction with the garbage collector. ---------- assignedto: arigo files: t messages: 3299 nosy: arigo, ltratt, pypy-issue priority: bug status: unread title: Stacklets running out of memory on higher optimisation levels ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Fri Oct 14 02:54:35 2011 From: tracker at bugs.pypy.org (Alex Gaynor) Date: Fri, 14 Oct 2011 00:54:35 +0000 Subject: [pypy-issue] [issue891] reduce and filter aren't jitted. map isn't jitted for built-in functions In-Reply-To: <1317620683.16.0.391298518561.issue891@bugs.pypy.org> Message-ID: <1318553675.49.0.770783736447.issue891@bugs.pypy.org> Alex Gaynor added the comment: These are all at app-level now. They're also all very fast :) ---------- nosy: +agaynor status: unread -> resolved ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Fri Oct 14 18:22:13 2011 From: tracker at bugs.pypy.org (Alex Gaynor) Date: Fri, 14 Oct 2011 16:22:13 +0000 Subject: [pypy-issue] [issue880] Django/SQLite3 - 'One and only one statement required' Warning In-Reply-To: <1316926620.11.0.498276554023.issue880@bugs.pypy.org> Message-ID: <1318609333.88.0.206064617188.issue880@bugs.pypy.org> Alex Gaynor added the comment: Closing as fixed then. ---------- nosy: +agaynor status: chatting -> resolved ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Fri Oct 14 18:23:23 2011 From: tracker at bugs.pypy.org (Alex Gaynor) Date: Fri, 14 Oct 2011 16:23:23 +0000 Subject: [pypy-issue] [issue908] Django tests failing In-Reply-To: <1318609403.25.0.782175954517.issue908@bugs.pypy.org> Message-ID: <1318609403.25.0.782175954517.issue908@bugs.pypy.org> New submission from Alex Gaynor : With the latest PyPy and trunk of Django I consistantly get: alex at alex-gaynor-laptop:~/projects/django$ PYTHONPATH=`pwd` ~/projects/pypy/pypy-c tests/runtests.py --settings=test_sqlite --failfast Creating test database for alias 'default'... Creating test database for alias 'other'... ................................................................................ ................................................................................ ................................................................................ ................................................................................ ................................................................................ .......................................s........................................ ........................................................s....................... ................................................................................ .....................................s.......................................... ................................................s............................... ................................................................................ ................................................................................ ................................................................................ ................................................................................ ...............................x................................................ ................................................................................ .............................................s.................................. ................................................................................ ......................................................s......................... ................................................................................ ................................................................................ ...................x...........................................s..........s..... ................................................................................ ................................................................................ ................................................................................ ................................................................................ ................s...................................................ss.......... .x.............................................................................. ................................................................................ ............................................s......s...s........E ====================================================================== ERROR: test_booleanfieldlistfilter (regressiontests.admin_filters.tests.ListFiltersTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/alex/projects/django/tests/regressiontests/admin_filters/tests.py", line 345, in test_booleanfieldlistfilter self.verify_booleanfieldlistfilter(modeladmin) File "/home/alex/projects/django/tests/regressiontests/admin_filters/tests.py", line 359, in verify_booleanfieldlistfilter queryset = changelist.get_query_set(request) File "/home/alex/projects/django/django/contrib/admin/views/main.py", line 299, in get_query_set self.filter_specs, self.has_filters = self.get_filters(request, use_distinct) File "/home/alex/projects/django/django/contrib/admin/views/main.py", line 110, in get_filters self.model, self.model_admin, field_path=field_path) File "/home/alex/projects/django/django/contrib/admin/filters.py", line 142, in create model, model_admin, field_path=field_path) File "/home/alex/projects/django/django/contrib/admin/filters.py", line 163, in __init__ self.lookup_choices = field.get_choices(include_blank=False) File "/home/alex/projects/django/django/db/models/fields/__init__.py", line 399, in get_choices self.rel.limit_choices_to)] File "/home/alex/projects/django/django/db/models/query.py", line 117, in _result_iter self._fill_cache() File "/home/alex/projects/django/django/db/models/query.py", line 855, in _fill_cache self._result_cache.append(self._iter.next()) File "/home/alex/projects/django/django/db/models/query.py", line 288, in iterator for row in compiler.results_iter(): File "/home/alex/projects/django/django/db/models/sql/compiler.py", line 701, in results_iter for rows in self.execute_sql(MULTI): File "/home/alex/projects/django/django/db/models/sql/compiler.py", line 756, in execute_sql cursor.execute(sql, params) File "/home/alex/projects/django/django/db/backends/sqlite3/base.py", line 271, in execute return Database.Cursor.execute(self, query, params) File "/home/alex/projects/pypy/lib_pypy/_sqlite3.py", line 737, in execute self.statement = self.connection.statement_cache.get(sql, self, self.row_factory) File "/home/alex/projects/pypy/lib_pypy/_sqlite3.py", line 289, in get stat = Statement(self.connection, sql) File "/home/alex/projects/pypy/lib_pypy/_sqlite3.py", line 930, in __init__ raise self.con._get_exception(ret) DatabaseError: near ""."": syntax error ---------------------------------------------------------------------- Ran 2385 tests in 176.376s FAILED (errors=1, skipped=14, expected failures=3) Destroying test database for alias 'default'... Destroying test database for alias 'other'... Looks to be a ctypes bug of some sort. ---------- messages: 3302 nosy: agaynor, pypy-issue priority: bug status: unread title: Django tests failing ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Sat Oct 15 15:53:20 2011 From: tracker at bugs.pypy.org (Armin Rigo) Date: Sat, 15 Oct 2011 13:53:20 +0000 Subject: [pypy-issue] [issue884] Diabolical Multicore Thread Performance In-Reply-To: <1317176061.39.0.582537402848.issue884@bugs.pypy.org> Message-ID: <1318686800.32.0.937037022229.issue884@bugs.pypy.org> Armin Rigo added the comment: Thanks zseil. I guess the MS compiler gave warnings, but they were lost in the sea. It's mixing a critical section and an event, yes. I don't see how to use just a single mutex to achieve the goal I describe in my long post below. ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Sat Oct 15 15:54:31 2011 From: tracker at bugs.pypy.org (Armin Rigo) Date: Sat, 15 Oct 2011 13:54:31 +0000 Subject: [pypy-issue] [issue884] Diabolical Multicore Thread Performance In-Reply-To: <1317176061.39.0.582537402848.issue884@bugs.pypy.org> Message-ID: <1318686871.41.0.219748815757.issue884@bugs.pypy.org> Armin Rigo added the comment: fijal: "very good" means "it matches the expectations I describe in the previous post". ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Sat Oct 15 21:37:50 2011 From: tracker at bugs.pypy.org (Michal) Date: Sat, 15 Oct 2011 19:37:50 +0000 Subject: [pypy-issue] [issue909] My code is slower under PyPy than under cPython In-Reply-To: <1318707470.31.0.667707050926.issue909@bugs.pypy.org> Message-ID: <1318707470.31.0.667707050926.issue909@bugs.pypy.org> New submission from Michal : I tried to run my code under different python implementations: under pypy is slower then cpython, under psyco it is much slower first time I invoke it and then it is much quicker, compiled with cython is slightly faster. Please see privided example. Just run and it returns ms spent on it. In my machine (pypy ver: Python 2.7.1 (080f42d5c4b4, Aug 23 2011, 11:41:11) [PyPy 1.6.0 with MSC v.1500 32 bit] on win32 ) cpython: about 8 ms pypy: about 18 ms ---------- files: main.py messages: 3305 nosy: mmierzwa, pypy-issue priority: wish status: unread title: My code is slower under PyPy than under cPython ________________________________________ PyPy bug tracker ________________________________________ -------------- next part -------------- ''' Created on 15-10-2011 @author: root ''' from time import clock, time class SW(object): """ It is used to measure time intervals """ #file handler common to all objects of this class log = None def __init__(self, name): #defines target where logs will be stored to self.log_to = {"console": True} self.timer = 0; self.name = name self.alltimer = 0 self.allclock = 0 self.pause_all_timer = 0 self.pause_all_clock = 0 self.pause_timer = 0 self.pause_clock = 0 def start(self): if self.log_to["console"]: print "SW started for: " + self.name self.timer = time() self.clock = clock() def pause(self): self.pause_timer = time() self.pause_clock = clock() def resume(self): self.pause_all_timer += time() - self.pause_timer self.pause_timer = 0 self.pause_all_clock += clock() - self.pause_clock self.pause_clock = 0 def stop(self): if self.pause_timer != 0: self.resume() self.alltimer = time() - self.timer - self.pause_all_timer self.allclock = clock() - self.clock - self.pause_all_clock if self.log_to["console"]: print self.name + " has taken: " + str(round(self.alltimer * 1000)) + "ms, busy: " + str(round(self.allclock *1000)) + "ms." return round(self.alltimer * 1000), round(self.allclock *1000) def get_point_content1(_, _1, _2, _3): return [] ALT_COLORS = ( (-20, "^4", "^b"), #deep water (00, "^6", "^c"), #shallow water (20, "^2", "^g"), (40, "^3", "^y"), (60, "^1", "^r"), (80, "^5", "^m") ) def get_alt_color1(alt): if alt < ALT_COLORS[0][0]: return ALT_COLORS[0][1] if alt < ALT_COLORS[1][0]: return ALT_COLORS[1][1] if alt < ALT_COLORS[2][0]: return ALT_COLORS[2][1] if alt < ALT_COLORS[3][0]: return ALT_COLORS[3][1] if alt < ALT_COLORS[4][0]: return ALT_COLORS[4][1] if alt < ALT_COLORS[5][0]: return ALT_COLORS[5][1] return ALT_COLORS[5][1] def get_alt1(x, y, area): return 0 #Kinds of items ALL = 0 PCS = 1 OBJECTS = 2 WORMHOLE = 3 MOBS = 4 DESC = 5 PARAMS = 6 ALT = 7 COLORS1 = { 'DIR': "^g", OBJECTS: "^w", MOBS: "^m", WORMHOLE: "^y", DESC: "^W"} MAP_COLORS1 = { ALL: "^W", OBJECTS: "^W", MOBS: "^M", WORMHOLE: "^Y"} def size_dist_rel1(size,dist, sight): return dist < size / 3 + sight def get_heights_area(aname): return {} def get_area(aname): return {} BLOCK_VOLATILE = 0 #in ex. parameters, description BLOCK_NONE = 1 #in example knife BLOCK_BLOCKING = 2 #usually mobs BLOCK_ALL = 3 #usually walls blocking_rules = {BLOCK_VOLATILE: [False,False,False,False], BLOCK_NONE: [False,False,False,True], BLOCK_BLOCKING: [False,False,True,True], BLOCK_ALL: [False,True,True,True]} def get_lang(what, _): return what def eval_dir_dist(nx,ny,dist,srange): "Evaluate direction and distance of point" wsp = abs(float(ny) / (nx - 0.001)) _lang = 'pl' if wsp >= 2: if ny > 0: dir2 = "N " else: if ny < 0: dir2 = "S " else: if wsp > 0.5: if ny > 0 and nx > 0: dir2 = "NE " else: if ny > 0 and nx < 0: dir2 = "NW " else: if ny < 0 and nx > 0: dir2 = "SE " else: if ny < 0 and nx < 0: dir2 = "SW " else: if wsp <= 0.5: if nx > 0: dir2 = "E " else: if nx < 0: dir2 = "W " if dist < 3: dir3 = get_lang('close', _lang) + "\t" else: if dist < 5: dir3 = get_lang('farther', _lang) + "\t" else: dir3 = get_lang('far', _lang) + "\t" return dir2, dir3 def create_next_sign(nb): "Creates next sign to display on the map view" if nb < 10: sc = chr(nb + ord('0')) else: if nb < 36: sc = chr(nb - 10 + ord('A')) else: if nb < 62: sc = chr(nb - 36 + ord('a')) else: sc = '*' return sc def NL(): return def rsendNL(what): print what def get_alt(x,y, aname): "Returns ground altitude of that point" #@inlined in mud.pathfinding.cached_passable - UPDATE IT THERE area = get_heights_area(aname) return 0 def get_neighbouring_point(x, y, area): "Returns mobs in point x,y" mylist = [] mylang = 'pl' content = get_point_content1(x, y, area, MOBS) for itm in content: if itm.myname: mylist.append(itm.name[mylang].name + ": " + itm.myname.name + " [" + str(itm.get_id_short()) + "]^~") else: mylist.append(itm.name[mylang].name + " [" + str(itm.get_id_short()) + "]^~") return ", ".join(mylist) def show_neighbourhood(): "Shows mobs in naighbouring points" mx = 0 my = 0 aname = 'space' altarea = get_heights_area(aname) area = get_area(aname) _get_alt_color = get_alt_color1 _get_alt = get_alt1 _gnp = get_neighbouring_point _s = rsendNL alt = _get_alt(mx-1, my+1, altarea) content = _gnp(mx-1, my+1, area) if content: _s(_get_alt_color(alt) + "NW:^~\t" + content) alt = _get_alt(mx, my+1, altarea) content = _gnp(mx, my+1, area) if content: _s(_get_alt_color(alt) + "N:^~\t" + content) alt = _get_alt(mx+1, my+1, altarea) content = _gnp(mx+1, my+1, area) if content: _s(_get_alt_color(alt) + "NE:^~\t" + content) alt = _get_alt(mx-1, my, altarea) content = _gnp(mx-1, my, area) if content: _s(_get_alt_color(alt) + "W:^~\t" + content) alt = _get_alt(mx+1, my, altarea) content = _gnp(mx+1, my, area) if content: _s(_get_alt_color(alt) + "E:^~\t" + content) alt = _get_alt(mx-1, my-1, altarea) content = _gnp(mx-1, my-1, area) if content: _s(_get_alt_color(alt) + "SW:^~\t" + content) alt = _get_alt(mx, my-1, altarea) content = _gnp(mx, my-1, area) if content: _s(_get_alt_color(alt) + "S:^~\t" + content) alt = _get_alt(mx+1, my-1, altarea) content = _gnp(mx+1, my-1, area) if content: _s(_get_alt_color(alt) + "SE:^~\t" + content) def show_here(prm = None): "Display content of point we are standing in and neighbouring mobs" NL() _DESC = DESC _OBJECTS = OBJECTS _WORMHOLE = WORMHOLE _COLORS = COLORS1 _get_point_content = get_point_content1 mylang = 'pl' aname = 'space' area = get_area(aname) mx = 0 my = 0 my_point_alt = get_alt(mx, my, aname) rsendNL( get_alt_color1(my_point_alt) + 'here' + "^~") itms = _get_point_content(mx, my, area, _DESC) if itms: found_descs = [] count = 0 for itm in itms: itm_name = itm.name[mylang].name if itm.desc: found_descs.append(itm) count += 1 rsendNL(_COLORS[_DESC] + itm_name + "[" + str(count) + "]^~") else: rsendNL(_COLORS[_DESC] + itm_name + "^~") buf = {} itms = _get_point_content(mx, my, area, _OBJECTS) for itm in itms: itm_name = itm.name[mylang].name if itm_name in buf: buf[itm_name].append(itm.get_id_short()) else: buf[itm_name] = [itm.get_id_short()] strbuff = "" for name in buf: if strbuff != "": strbuff += ", " if len(buf[name]) > 1: strbuff += "(" + str(len(buf[name])) + ") " + name + " " + str(buf[name]) else: strbuff += name + " " + str(buf[name]) if strbuff: rsendNL(_COLORS[_OBJECTS] + strbuff + "^~") itms = _get_point_content(mx, my, area, _WORMHOLE) for itm in itms: itm_name = itm.name[mylang].name rsendNL(_COLORS[_WORMHOLE] + itm_name + " [" + str(itm.get_id_short()) + "]^~") NL() show_neighbourhood() rsendNL("A: " + aname + "\tX: " + str(mx) + "\tY: " + str(my) + "\tZ: " + str(my_point_alt)) def sight_range(): "Return sight range, that is how big map it can see" return 8 def get_sight(): "Returns sight ability of a mob, normal is 0" return 0; def map_directions(): "Displays map and directions and distances of items (look 1)" sw = SW("Look map directions") sw.start() #shorthands get_point_content = get_point_content1 get_alt_color = get_alt_color1 get_alt = get_alt1 MAP_COLORS = MAP_COLORS1 COLORS = COLORS1 size_dist_rel = size_dist_rel1 _abs = abs aname = 'space' altarea = get_heights_area(aname) area = get_area(aname) srange = sight_range() mx = 0 my = 0 my_ground_alt = get_alt(mx, my, altarea) height = 0 m_alt = my_ground_alt + height NL() lang = 'pl' lmap = {} #counter of unique signs on map nb = 0 #containter of the legend to map legend = [] #iterate over the perimeter for ii in xrange(-srange, srange+1): for side in ((ii,srange), (ii,-srange), (srange,ii), (-srange,ii)): nx = side[0] ny = side[1] #calculate line from 0,0 to nx,ny point if _abs(ny) > _abs(nx): dist = ny else: dist = nx w = ny/(nx + 0.01) if dist >= 0: di = 1 else: di = -1 a = -999 #iterate over points lying on that line for i in xrange(di, dist+di, di): if _abs(nx) >= _abs(ny): ax = i ay = round(ax * (w + 0.01)) else: ay = i ax = round(ay / (w + 0.01)) if (ax,ay) in lmap: if 'a' in lmap[(ax,ay)]: a = lmap[(ax,ay)]['a'] continue #altitude of that point b_alt = g_alt = get_alt(ax+mx, ay+my, altarea) cellcolor = get_alt_color(b_alt) #fetch all cell items #@inline itms = get_point_content(ax+mx, ay+my, area, world.ALL) x_gpc, y_gpc = ax+mx, ay+my itms = [] if (x_gpc,y_gpc) in area: point = area[(x_gpc,y_gpc)] itms.extend(point[MOBS].values()) itms.extend(point[OBJECTS].values()) if WORMHOLE in point: itms.extend(point[WORMHOLE].values()) #@inline end itms = get_point_content(ax+mx, ay+my, area, world.ALL) if itms: for itm in itms: if itm.blocking == BLOCK_ALL: b_alt += itm.height lmap[(ax,ay)] = {} map_cell = lmap[(ax,ay)] fc = cellcolor + " " sc = " " #check whether point lies above line of sight min_visible_height = a * _abs(i) + m_alt if b_alt > min_visible_height: #calculate new line 'a' parameter between 0,0 and that point a = (b_alt - m_alt)/float(_abs(i)) lmap[(ax,ay)]['a'] = a if b_alt > min_visible_height: covered = False else: #this cell lies under the line of sight fc = cellcolor + "^K." covered = True if not itms: lmap[(ax,ay)]['cell'] = fc + sc continue #list of items in that point cell_items = {} #most important kind of items in that point most_important_kind = 0 for itm in itms: #if item too small it will not be visible, depends also of how good sight mob has if not size_dist_rel(itm.size, _abs(i), get_sight()): continue #if point not visible check if item is height enough to be visible if not, we can omit it if covered: if itm.blocking != BLOCK_ALL and g_alt + itm.height < min_visible_height: continue #if itm has it's own fixed sign or even both signs if itm.on_map == itm.MAP_FIXED: #we can set it's color if covered: fc = "^K" + itm.std_char1 else: fc = MAP_COLORS[OBJECTS] + itm.std_char1 #and if it has second char and it is not occupied we can display it either if sc == " " and itm.std_char2: sc = itm.std_char2 continue #we want to know what most important item type is in cell in order to know how to color it if itm.kind > most_important_kind: most_important_kind = itm.kind #if list of items is empty we have to prepare it if not cell_items: #obtain direction and distance name dirg, distg = eval_dir_dist(ax,ay,_abs(i),srange) #we do not have sign for this cell yet sc = create_next_sign(nb) nb += 1 direction = sc + ". " + COLORS['DIR'] + "[" + dirg + distg + "]^~ " itm_name = itm.name[lang].name #if itm is not in list if itm_name not in cell_items: #we have to add it cell_items[itm_name] = [1,itm.kind, [itm.get_id_short()]] else: #otherwise just increase its counter cell_items[itm_name][0] += 1 cell_items[itm_name][2].append(itm.get_id_short()) if covered: map_cell['cell'] = cellcolor + fc + "^K" + sc else: map_cell['cell'] = cellcolor + fc + MAP_COLORS[most_important_kind] + sc str_items = "" first_elm = True for elm in cell_items: if not first_elm: str_items += ", " else: first_elm = False str_items += COLORS[cell_items[elm][1]] if cell_items[elm][0] > 1: str_items += "(" + str(cell_items[elm][0]) + ") " str_items += elm + " " + str(cell_items[elm][2]) + "^~" if str_items: legend.append(direction + str_items) #display map for y in xrange(srange, -srange-1, -1): row = " " for x in xrange(-srange, srange + 1): if x == 0 and y == 0: content = get_point_content(mx, my, area, OBJECTS) fc = " " for itm in content: if itm.on_map == itm.MAP_FIXED: fc = MAP_COLORS[OBJECTS] + itm.std_char1 break row += get_alt_color(my_ground_alt) + fc + "^G@" continue row += lmap[(x,y)]['cell'] rsendNL(row + "^~") #display legend NL() for elm in legend: rsendNL(elm) show_here() sw.stop() if __name__ == '__main__': map_directions() pass From tracker at bugs.pypy.org Sat Oct 15 22:01:47 2011 From: tracker at bugs.pypy.org (Ronny Pfannschmidt) Date: Sat, 15 Oct 2011 20:01:47 +0000 Subject: [pypy-issue] [issue909] My code is slower under PyPy than under cPython In-Reply-To: <1318707470.31.0.667707050926.issue909@bugs.pypy.org> Message-ID: <1318708907.76.0.205755662569.issue909@bugs.pypy.org> Ronny Pfannschmidt added the comment: the example is not running enough iterations to actually benefit from the jit also the code s full of conditionals pypy without the jit benefit is roughly half as fast as cpython, which means your timing results are expected try with a much lager sample - something that takes a least seconds to run on cpython and has a few thousand iterations just to test the basics i did the following after removing some print statements: $ pypy-bin -m timeit -n 2000 -s 'import main' 'main.map_directions()' 2000 loops, best of 3: 1.74 msec per loop $ pypy-bin -m timeit -s 'import main' 'main.map_directions()' 10 loops, best of 3: 3.98 msec per loop $ python -m timeit -s 'import main' 'main.map_directions()' 100 loops, best of 3: 2.82 msec per loop as you can see, pypy gets about twice as fast after hitting the jit treshold to over-stress it i also did $ pypy-bin -m timeit -n 20000 -s 'import main' 'main.map_directions()' 20000 loops, best of 3: 1.71 msec per loop but there doesn't seem to be more improvement (imho 0.03 msec can be considered nose) ---------- nosy: +ronny status: unread -> chatting ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Sun Oct 16 02:19:55 2011 From: tracker at bugs.pypy.org (Andrew Mahone) Date: Sun, 16 Oct 2011 00:19:55 +0000 Subject: [pypy-issue] [issue910] eval(code, globals, locals) slower than cpython In-Reply-To: <1318724395.39.0.821234303302.issue910@bugs.pypy.org> Message-ID: <1318724395.39.0.821234303302.issue910@bugs.pypy.org> New submission from Andrew Mahone : This is about 2x slower than cpython on pypy: python -m timeit -n 250000 -r 5 -s "e = compile('a+'*255+'a','','eval'); a = 1; g = globals(); l = locals()" "eval(e,g,l)" Per discussion on irc, eval may be retrieving the calle.r frame even when it does not need to, causing jit bailout ---------- messages: 3307 nosy: pypy-issue priority: bug status: unread title: eval(code, globals, locals) slower than cpython ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Sun Oct 16 16:56:44 2011 From: tracker at bugs.pypy.org (Armin Rigo) Date: Sun, 16 Oct 2011 14:56:44 +0000 Subject: [pypy-issue] [issue907] Stacklets running out of memory on higher optimisation levels In-Reply-To: <1318523264.95.0.361208519335.issue907@bugs.pypy.org> Message-ID: <1318777004.32.0.298948779873.issue907@bugs.pypy.org> Armin Rigo added the comment: If you add a gc.collect() regularly in the RPython code, does it fix the issue? If it does, then it should be relatively simple: your code creates a lot of stacklets and not much else, and the memory consumed by a stacklet is not taken into account by the GC to decide when to run the next collection. We need to fix this. ---------- status: unread -> chatting ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Sun Oct 16 17:15:25 2011 From: tracker at bugs.pypy.org (Armin Rigo) Date: Sun, 16 Oct 2011 15:15:25 +0000 Subject: [pypy-issue] [issue906] CTRL-leftarrow and CTRL-rightarrow don't work in PyPy console In-Reply-To: <1318429405.87.0.821520990977.issue906@bugs.pypy.org> Message-ID: <1318778125.68.0.668293764199.issue906@bugs.pypy.org> Armin Rigo added the comment: It is an issue of "pyrepl", which is implemented as regular pure Python code present in lib_pypy/pyrepl, copied from https://bitbucket.org/pypy/pyrepl . It's not very easy to fix because there are several levels, but it can be done without former knowledge of PyPy. ---------- nosy: +arigo status: unread -> chatting ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Sun Oct 16 17:23:42 2011 From: tracker at bugs.pypy.org (Armin Rigo) Date: Sun, 16 Oct 2011 15:23:42 +0000 Subject: [pypy-issue] [issue910] eval(code, globals, locals) slower than cpython In-Reply-To: <1318724395.39.0.821234303302.issue910@bugs.pypy.org> Message-ID: <1318778622.69.0.855248002125.issue910@bugs.pypy.org> Armin Rigo added the comment: Does this patch fix it? http://paste.pocoo.org/show/493364/ ---------- nosy: +arigo status: unread -> chatting ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Sun Oct 16 17:34:35 2011 From: tracker at bugs.pypy.org (Armin Rigo) Date: Sun, 16 Oct 2011 15:34:35 +0000 Subject: [pypy-issue] [issue895] Continulets + JIT In-Reply-To: <1317981953.45.0.228294129679.issue895@bugs.pypy.org> Message-ID: <1318779275.48.0.462882891872.issue895@bugs.pypy.org> Armin Rigo added the comment: Workaround checked in: f671d242b164 . Unsatisfactory, because it basically disables the JIT on any loop that contains a switch(), but at least it works (e.g. module/_continuation/test pass on pypy-c-jit -A). ---------- status: unread -> chatting ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Sun Oct 16 17:48:06 2011 From: tracker at bugs.pypy.org (Armin Rigo) Date: Sun, 16 Oct 2011 15:48:06 +0000 Subject: [pypy-issue] [issue885] Lack of libffi5 on Ubuntu 11.10 stops PyPy running In-Reply-To: <1317176080.11.0.207082241886.issue885@bugs.pypy.org> Message-ID: <1318780086.82.0.607967642756.issue885@bugs.pypy.org> Armin Rigo added the comment: Bah, we already have everything done and installed as I described, but it was only used on 32-bit Linux builds, not on 64-bit. Fixed in ad955890a446. ---------- status: chatting -> resolved ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Sun Oct 16 21:35:06 2011 From: tracker at bugs.pypy.org (Armin Rigo) Date: Sun, 16 Oct 2011 19:35:06 +0000 Subject: [pypy-issue] [issue884] Diabolical Multicore Thread Performance In-Reply-To: <1317176061.39.0.582537402848.issue884@bugs.pypy.org> Message-ID: <1318793706.23.0.255284191585.issue884@bugs.pypy.org> Armin Rigo added the comment: Thanks to zseil for finding the source of the problems, now it seems to work on Windows too (but it was minimally tested so far). ---------- status: chatting -> resolved ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Sun Oct 16 21:41:00 2011 From: tracker at bugs.pypy.org (Laurence Tratt) Date: Sun, 16 Oct 2011 19:41:00 +0000 Subject: [pypy-issue] [issue907] Stacklets running out of memory on higher optimisation levels In-Reply-To: <1318523264.95.0.361208519335.issue907@bugs.pypy.org> Message-ID: <1318794060.06.0.0568603591425.issue907@bugs.pypy.org> Laurence Tratt added the comment: Putting gc.collect in the main VM loop is very interesting. On --opt=1, the memory used for my simple test program is a smidgen over 1MB, and doesn't increase from there (it may be that, temporarily it is more than 1MB, but with such frequent garbage collections, running top at 1s intervals shows it at 1MB). With --opt=3, the memory monotonically increases until the task eventually blows up (by garbage collecting so often, this takes several minutes!). ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Sun Oct 16 22:02:29 2011 From: tracker at bugs.pypy.org (Armin Rigo) Date: Sun, 16 Oct 2011 20:02:29 +0000 Subject: [pypy-issue] [issue907] Stacklets running out of memory on higher optimisation levels In-Reply-To: <1318523264.95.0.361208519335.issue907@bugs.pypy.org> Message-ID: <1318795349.49.0.61181819922.issue907@bugs.pypy.org> Armin Rigo added the comment: Indeed, reproduced with the given (boringly simple) Python script to run on a pypy-c. If the continulets are "finished" with the commented-out call to c.switch(), everything works fine. If the continulets are forgotten while in the running state and left behind for the GC to collect, then it leaks. ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Sun Oct 16 22:22:27 2011 From: tracker at bugs.pypy.org (Jean-Paul Calderone) Date: Sun, 16 Oct 2011 20:22:27 +0000 Subject: [pypy-issue] [issue645] twisted.test.test_threadable.SynchronizationTestCase.testThreadedSynchronization hangs In-Reply-To: <1298652832.91.0.285352073761.issue645@> Message-ID: <1318796547.78.0.626412269788.issue645@bugs.pypy.org> Jean-Paul Calderone added the comment: The test that fails is the one mentioned in the issue title. ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Sun Oct 16 22:33:28 2011 From: tracker at bugs.pypy.org (Andrew Mahone) Date: Sun, 16 Oct 2011 20:33:28 +0000 Subject: [pypy-issue] [issue910] eval(code, globals, locals) slower than cpython In-Reply-To: <1318724395.39.0.821234303302.issue910@bugs.pypy.org> Message-ID: <1318797208.39.0.981700892307.issue910@bugs.pypy.org> Andrew Mahone added the comment: Sorry, I'm not seeing an improvement with this change. Is there anything else I can do to produce more useful feedback than "it's still slow"? ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Sun Oct 16 22:56:04 2011 From: tracker at bugs.pypy.org (Armin Rigo) Date: Sun, 16 Oct 2011 20:56:04 +0000 Subject: [pypy-issue] [issue910] eval(code, globals, locals) slower than cpython In-Reply-To: <1318724395.39.0.821234303302.issue910@bugs.pypy.org> Message-ID: <1318798564.29.0.174492981352.issue910@bugs.pypy.org> Armin Rigo added the comment: Closed: the real issue is that it hits the trace limit. It is (a bit) faster than CPython with "pypy --jit trace_limit=12000" for example. All these 'a' are doing lookups in the locals dictionary followed by guard checks for the type, which I suppose takes a lot of operations. (One could improve by converting StringDicts to CellDicts automatically in exec/eval, maybe.) So we end up with "trace too long", and all operations are from the same function --- i.e. the compiled code object --- so that it's not possible to reduce the length of the trace by disabling some inlining... which might be another issue to look at, maybe, although it's unlikely that many functions are so large in practice. ---------- status: chatting -> invalid ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Sun Oct 16 23:12:48 2011 From: tracker at bugs.pypy.org (Armin Rigo) Date: Sun, 16 Oct 2011 21:12:48 +0000 Subject: [pypy-issue] [issue907] Stacklets running out of memory on higher optimisation levels In-Reply-To: <1318523264.95.0.361208519335.issue907@bugs.pypy.org> Message-ID: <1318799568.36.0.262296241797.issue907@bugs.pypy.org> Armin Rigo added the comment: Fixed, but it was module/_continuation that was not calling sthread.destroy(). Now x.py works without a leak. It probably doesn't help you much :-/ ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Sun Oct 16 23:25:17 2011 From: tracker at bugs.pypy.org (Laurence Tratt) Date: Sun, 16 Oct 2011 21:25:17 +0000 Subject: [pypy-issue] [issue907] Stacklets running out of memory on higher optimisation levels In-Reply-To: <1318523264.95.0.361208519335.issue907@bugs.pypy.org> Message-ID: <1318800317.81.0.949118330573.issue907@bugs.pypy.org> Laurence Tratt added the comment: If I stick print statements before calling stacklet.new, and stacklet.destroy, I get a 1:1 match, so I don't think that's my problem (though I might be missing out on something obvious!). I feel sure that there must be some sort of clue in the fact that --opt=1 works for me, but --opt=2 and above don't... What that clue is, I have no idea. ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Sun Oct 16 23:53:16 2011 From: tracker at bugs.pypy.org (Michael Hudson-Doyle) Date: Sun, 16 Oct 2011 21:53:16 +0000 Subject: [pypy-issue] [issue906] CTRL-leftarrow and CTRL-rightarrow don't work in PyPy console In-Reply-To: <1318429405.87.0.821520990977.issue906@bugs.pypy.org> Message-ID: <1318801996.38.0.208415241926.issue906@bugs.pypy.org> Michael Hudson-Doyle added the comment: If I remember correctly, the problem here is that there is no entry in the terminfo database to let pyrepl know what escape sequences correspond to control-. If I'm wrong, this is super easy to fix. If I'm not, I'm not sure what the best thing to do is -- of course, probably basically everyone these days uses something that derives at least in some sense from xterm, so hard coding what that sends (^[[1;5C/^[[1;5D) might well work fine. I wonder what readline does... ---------- nosy: +mwh ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Mon Oct 17 00:16:28 2011 From: tracker at bugs.pypy.org (Michael Hudson-Doyle) Date: Sun, 16 Oct 2011 22:16:28 +0000 Subject: [pypy-issue] [issue906] CTRL-leftarrow and CTRL-rightarrow don't work in PyPy console In-Reply-To: <1318429405.87.0.821520990977.issue906@bugs.pypy.org> Message-ID: <1318803388.99.0.185796805647.issue906@bugs.pypy.org> Michael Hudson-Doyle added the comment: Ah, it seems that on my system, reading /etc/inputrc would suffice as that defines a bunch of possibilities for the control-left/right as backward/forward-word ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Mon Oct 17 09:13:35 2011 From: tracker at bugs.pypy.org (Fijal) Date: Mon, 17 Oct 2011 07:13:35 +0000 Subject: [pypy-issue] [issue911] sobel demo 2x slower on trunk than 1.5 (or 1.6) In-Reply-To: <1318835615.45.0.504205240262.issue911@bugs.pypy.org> Message-ID: <1318835615.45.0.504205240262.issue911@bugs.pypy.org> New submission from Fijal : >From extradoc/talk/iwtc11/... ---------- messages: 3323 nosy: fijal, pypy-issue priority: performance bug status: unread title: sobel demo 2x slower on trunk than 1.5 (or 1.6) ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Mon Oct 17 10:54:20 2011 From: tracker at bugs.pypy.org (Fijal) Date: Mon, 17 Oct 2011 08:54:20 +0000 Subject: [pypy-issue] [issue867] urllib2 on pypy can leak fd's In-Reply-To: <1315669439.97.0.201746613632.issue867@bugs.pypy.org> Message-ID: <1318841660.08.0.875308375864.issue867@bugs.pypy.org> Fijal added the comment: This is fixed, but it broke a test test_urllib2_localnet. Closing the issue ---------- status: chatting -> resolved ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Mon Oct 17 11:49:43 2011 From: tracker at bugs.pypy.org (Armin Rigo) Date: Mon, 17 Oct 2011 09:49:43 +0000 Subject: [pypy-issue] [issue911] sobel demo 2x slower on trunk than 1.5 (or 1.6) In-Reply-To: <1318835615.45.0.504205240262.issue911@bugs.pypy.org> Message-ID: <1318844983.17.0.803696274984.issue911@bugs.pypy.org> Armin Rigo added the comment: May be fixed by 849075adda3f: the issue was that even invalidated loops were considered as potential jump targets for new loops. This creates inefficient code. ---------- nosy: +arigo status: unread -> chatting ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Tue Oct 18 11:38:56 2011 From: tracker at bugs.pypy.org (Tyler Kennedy) Date: Tue, 18 Oct 2011 09:38:56 +0000 Subject: [pypy-issue] [issue912] ZipFile.read() leaks file handles In-Reply-To: <1318930736.76.0.381437213231.issue912@bugs.pypy.org> Message-ID: <1318930736.76.0.381437213231.issue912@bugs.pypy.org> New submission from Tyler Kennedy : PyPy's implementation of ZipFile will constantly leak file handles if the ZipFile was originally opened given a file path instead of a file handle. For example, this will thrown an OSError on ZipFiles with over 1024 files: zp = zipfile.ZipFile('/path/to/zip/with/many/files.zip') self.files = dict((f, zp.read(f)) for f in zp.namelist()) zp.close() The offending bit of code is zipfile.py line 879 to 882: # Only open a new file for instances where we were not # given a file object in the constructor if self._filePassed: zef_file = self.fp else: zef_file = open(self.filename, 'rb') ...in conjunction with: def read(self, name, pwd=None): """Return file bytes (as a string) for name.""" return self.open(name, "r", pwd).read() Because read() never closes the handles, and the handles don't get collected inside of a loop. ---------- messages: 3326 nosy: TkTech, pypy-issue priority: bug release: 1.6 status: unread title: ZipFile.read() leaks file handles ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Tue Oct 18 11:51:31 2011 From: tracker at bugs.pypy.org (Armin Rigo) Date: Tue, 18 Oct 2011 09:51:31 +0000 Subject: [pypy-issue] [issue645] twisted.test.test_threadable.SynchronizationTestCase.testThreadedSynchronization hangs In-Reply-To: <1298652832.91.0.285352073761.issue645@> Message-ID: <1318931491.47.0.465474651493.issue645@bugs.pypy.org> Armin Rigo added the comment: It seems to work nowadays with pypy 1.6. On Ubuntu Linux 64: - copy /usr/lib/python2.7/dist-packages/{twisted,zope} to /tmp/somedir - run PYTHONPATH=/tmp/somedir pypy /usr/bin/trial twisted.test.test_threadable Runs successfully. Should we close this bug, or do you still have issues on non-Linux platforms? ---------- nosy: +arigo ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Tue Oct 18 12:15:37 2011 From: tracker at bugs.pypy.org (Tyler Kennedy) Date: Tue, 18 Oct 2011 10:15:37 +0000 Subject: [pypy-issue] [issue912] ZipFile.read() leaks file handles In-Reply-To: <1318930736.76.0.381437213231.issue912@bugs.pypy.org> Message-ID: <1318932937.59.0.790756105414.issue912@bugs.pypy.org> Tyler Kennedy added the comment: Sorry, I worded that poorly. Even without using read(), you will still eventually get an OSError for too many file handles. zef_file is never closed and leaks every time open() is called if the ZipFile was opened using a path. ---------- status: unread -> chatting ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Tue Oct 18 13:02:24 2011 From: tracker at bugs.pypy.org (Armin Rigo) Date: Tue, 18 Oct 2011 11:02:24 +0000 Subject: [pypy-issue] [issue911] sobel demo 2x slower on trunk than 1.5 (or 1.6) In-Reply-To: <1318835615.45.0.504205240262.issue911@bugs.pypy.org> Message-ID: <1318935744.57.0.562904029445.issue911@bugs.pypy.org> Armin Rigo added the comment: It's already a bit faster, but still not as good as it used to be. ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Tue Oct 18 13:06:33 2011 From: tracker at bugs.pypy.org (Armin Rigo) Date: Tue, 18 Oct 2011 11:06:33 +0000 Subject: [pypy-issue] [issue911] sobel demo 2x slower on trunk than 1.5 (or 1.6) In-Reply-To: <1318835615.45.0.504205240262.issue911@bugs.pypy.org> Message-ID: <1318935993.58.0.494602230324.issue911@bugs.pypy.org> Armin Rigo added the comment: Wrong, it's within 10-20% of the same speed, but there is a different issue: it takes longer to warm up. Unsure how much we want to care right now, so marking this issue as resolved for now. ---------- status: chatting -> resolved ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Tue Oct 18 13:08:08 2011 From: tracker at bugs.pypy.org (Amaury Forgeot d Arc) Date: Tue, 18 Oct 2011 11:08:08 +0000 Subject: [pypy-issue] [issue912] ZipFile.read() leaks file handles In-Reply-To: <1318930736.76.0.381437213231.issue912@bugs.pypy.org> Message-ID: <1318936088.27.0.155359588539.issue912@bugs.pypy.org> Amaury Forgeot d Arc added the comment: Can you try with the attached patch? (it corresponds to changes already done in CPython, but for version 3.x: http://hg.python.org/cpython/rev/260ff379115c http://hg.python.org/cpython/rev/2c370065c1b4 and http://hg.python.org/cpython/rev/19ebcb934f3) ---------- nosy: +afa ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Tue Oct 18 13:15:32 2011 From: tracker at bugs.pypy.org (Armin Rigo) Date: Tue, 18 Oct 2011 11:15:32 +0000 Subject: [pypy-issue] [issue889] Windows binary distribution is missing cpyext header files In-Reply-To: <1317395541.31.0.735873324924.issue889@bugs.pypy.org> Message-ID: <1318936532.27.0.683347331541.issue889@bugs.pypy.org> Armin Rigo added the comment: The cpyext headers are found in the "include" directory in binary distributions, both on Linux and on Windows. What is missing? ---------- nosy: +arigo ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Tue Oct 18 13:38:43 2011 From: tracker at bugs.pypy.org (Tyler Kennedy) Date: Tue, 18 Oct 2011 11:38:43 +0000 Subject: [pypy-issue] [issue912] ZipFile.read() leaks file handles In-Reply-To: <1318930736.76.0.381437213231.issue912@bugs.pypy.org> Message-ID: <1318937923.59.0.801936156444.issue912@bugs.pypy.org> Tyler Kennedy added the comment: Applying the patch breaks it: Traceback (most recent call last): File "app_main.py", line 53, in run_toplevel ... File "/Users/tyler/Github/Helicon/helicon/jarfile.py", line 32, in __init__ self.files = dict((f, zp.read(f)) for f in zp.namelist()) File "/Users/tyler/Github/Helicon/helicon/jarfile.py", line 32, in self.files = dict((f, zp.read(f)) for f in zp.namelist()) File "/usr/local/Cellar/pypy/1.6.0/lib-python/2.7/zipfile.py", line 876, in read return fp.read() File "/usr/local/Cellar/pypy/1.6.0/lib-python/2.7/zipfile.py", line 657, in close super().close() TypeError: __new__() takes at least 2 arguments (1 given) ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Tue Oct 18 14:13:57 2011 From: tracker at bugs.pypy.org (Amaury Forgeot d Arc) Date: Tue, 18 Oct 2011 12:13:57 +0000 Subject: [pypy-issue] [issue912] ZipFile.read() leaks file handles In-Reply-To: <1318930736.76.0.381437213231.issue912@bugs.pypy.org> Message-ID: <1318940037.34.0.53392112016.issue912@bugs.pypy.org> Amaury Forgeot d Arc added the comment: Yes sorry, super() should be replaced by super(ZipExtFile, self) ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Tue Oct 18 14:19:33 2011 From: tracker at bugs.pypy.org (Jean-Paul Calderone) Date: Tue, 18 Oct 2011 12:19:33 +0000 Subject: [pypy-issue] [issue645] twisted.test.test_threadable.SynchronizationTestCase.testThreadedSynchronization hangs In-Reply-To: <1298652832.91.0.285352073761.issue645@> Message-ID: <1318940373.44.0.889617041079.issue645@bugs.pypy.org> Jean-Paul Calderone added the comment: It's passing here now too. Thanks. ---------- status: chatting -> resolved ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Tue Oct 18 14:22:08 2011 From: tracker at bugs.pypy.org (Tyler Kennedy) Date: Tue, 18 Oct 2011 12:22:08 +0000 Subject: [pypy-issue] [issue912] ZipFile.read() leaks file handles In-Reply-To: <1318930736.76.0.381437213231.issue912@bugs.pypy.org> Message-ID: <1318940528.57.0.948266273577.issue912@bugs.pypy.org> Tyler Kennedy added the comment: Patches + the one correction and it's working well, albiet still 5x slower than cpython. Thanks. ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Tue Oct 18 15:47:08 2011 From: tracker at bugs.pypy.org (Armin Rigo) Date: Tue, 18 Oct 2011 13:47:08 +0000 Subject: [pypy-issue] [issue908] Django tests failing In-Reply-To: <1318609403.25.0.782175954517.issue908@bugs.pypy.org> Message-ID: <1318945628.4.0.0704952737348.issue908@bugs.pypy.org> Armin Rigo added the comment: Failed to reproduce. Running http://code.djangoproject.com/svn/django/trunk revision 17009 and pypy 781e9199c50b9+ on linux64. I have a lot of UnicodeEncodeError on non-ascii in file names, and 5 assertion errors from dispatch/tests/test_dispatcher.py, but test_booleanfieldlistfilter passes for me. ---------- nosy: +arigo status: unread -> chatting ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Tue Oct 18 15:53:15 2011 From: tracker at bugs.pypy.org (Alex Gaynor) Date: Tue, 18 Oct 2011 13:53:15 +0000 Subject: [pypy-issue] [issue908] Django tests failing In-Reply-To: <1318609403.25.0.782175954517.issue908@bugs.pypy.org> Message-ID: <1318945995.34.0.798009565556.issue908@bugs.pypy.org> Alex Gaynor added the comment: FWIW I see a different error ATM (when run with --failfast, I believe I'll still see this one if I run the full suite), but it still fails (http://paste.pocoo.org/show/494359/). Are you sure that's the right PyPy revision, I don't see it in the history, my PyPy is 1d42dc6c3971. ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Tue Oct 18 16:03:32 2011 From: tracker at bugs.pypy.org (Peter) Date: Tue, 18 Oct 2011 14:03:32 +0000 Subject: [pypy-issue] [issue913] Reduce-like *functions* for micronumpy missing, e.g. numpy.dot In-Reply-To: <1318946612.94.0.137292123198.issue913@bugs.pypy.org> Message-ID: <1318946612.94.0.137292123198.issue913@bugs.pypy.org> New submission from Peter : I see that under Issue 758 many numpy array reduce *methods* like dot were introduced. However, relevant numpy reduce *functions* are still missing. The following should work on C Python with any version of NumPy, in this case I used NumPy 1.2.1 on Mac OS X. >>> import numpy as np >>> np.dot(np.ones(5), np.ones(5)) 5.0 >>> np.vdot(np.ones(5), np.ones(5)) 5.0 >>> np.sum(np.ones(5)) 5.0 >>> np.power(np.ones(5)*2, 3) array([ 8., 8., 8., 8., 8.]) >>> np.max(np.ones(5)) 5.0 >>> np.min(np.ones(5)) 5.0 >>> np.argmax(np.ones(5)) 0 >>> np.argmin(np.ones(5)) 0 >>> np.all(np.ones(5)) True >>> np.any(np.ones(5)) True Using PyPy 1.6, there is no dot, vdot, sum, power, add, min, max, argmin, argmmax, all, any, ... function: >>>> import numpy as np >>>> np.dot(np.ones(5), np.ones(5)) Traceback (most recent call last): File "", line 1, in AttributeError: 'module' object has no attribute 'dot' >>>> np.ones(5).vdot(np.ones(5)) Traceback (most recent call last): File "", line 1, in AttributeError: 'numarray' object has no attribute 'vdot' >>>> np.sum(no.ones(5)) Traceback (most recent call last): File "", line 1, in AttributeError: 'module' object has no attribute 'sum' >>>> np.power(np.ones(5)*2, 3) Traceback (most recent call last): File "", line 1, in AttributeError: 'module' object has no attribute 'power' >>>> np.max(no.ones(5)) Traceback (most recent call last): File "", line 1, in AttributeError: 'module' object has no attribute 'max' >>>> np.min(no.ones(5)) Traceback (most recent call last): File "", line 1, in AttributeError: 'module' object has no attribute 'min' >>>> np.argmax(no.ones(5)) Traceback (most recent call last): File "", line 1, in AttributeError: 'module' object has no attribute 'argmax' >>>> np.argmin(no.ones(5)) Traceback (most recent call last): File "", line 1, in AttributeError: 'module' object has no attribute 'argmin' >>>> np.all(no.ones(5)) Traceback (most recent call last): File "", line 1, in AttributeError: 'module' object has no attribute 'all' >>>> np.any(no.ones(5)) Traceback (most recent call last): File "", line 1, in AttributeError: 'module' object has no attribute 'any' This is probably just a sublist, but some of these are quite widely used. ---------- messages: 3339 nosy: pypy-issue priority: bug release: 1.6 status: unread title: Reduce-like *functions* for micronumpy missing, e.g. numpy.dot ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Tue Oct 18 16:09:08 2011 From: tracker at bugs.pypy.org (Jean-Paul Calderone) Date: Tue, 18 Oct 2011 14:09:08 +0000 Subject: [pypy-issue] [issue889] Windows binary distribution is missing cpyext header files In-Reply-To: <1317395541.31.0.735873324924.issue889@bugs.pypy.org> Message-ID: <1318946948.78.0.993135921048.issue889@bugs.pypy.org> Jean-Paul Calderone added the comment: Hmm, indeed. Sorry for the poor bug report. I filed this in response to . It seems that it wasn't actually a Windows binary that produced the problem, but a custom build from MacPorts, which probably just didn't have cpyext enabled at all. However, I did also test on Windows and was unable to build Twisted's extensions. The problem was an inability to find libpython27 though. If you'd prefer, feel free to close this ticket and I'll file a new one with details about the build failure I can reproduce. ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Tue Oct 18 16:12:10 2011 From: tracker at bugs.pypy.org (Jean-Paul Calderone) Date: Tue, 18 Oct 2011 14:12:10 +0000 Subject: [pypy-issue] [issue889] Windows binary distribution is missing cpyext header files In-Reply-To: <1317395541.31.0.735873324924.issue889@bugs.pypy.org> Message-ID: <1318947130.3.0.84074858195.issue889@bugs.pypy.org> Jean-Paul Calderone added the comment: (Or I can add the details here, later, when I'm in front of a machine with Windows) ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Tue Oct 18 16:13:39 2011 From: tracker at bugs.pypy.org (Peter) Date: Tue, 18 Oct 2011 14:13:39 +0000 Subject: [pypy-issue] [issue914] Missing xml.parsers.expat.model In-Reply-To: <1318947219.55.0.30396177762.issue914@bugs.pypy.org> Message-ID: <1318947219.55.0.30396177762.issue914@bugs.pypy.org> New submission from Peter : The following should work on C Python 2.0 or later, here using Python 2.6.1 >>> from xml.parsers.expat import model >>> model.XML_CTYPE_ANY 2 >>> help(model) Help on module pyexpat.model in pyexpat: NAME pyexpat.model - Constants used to interpret content model information. FILE (built-in) DATA XML_CQUANT_NONE = 0 XML_CQUANT_OPT = 1 XML_CQUANT_PLUS = 3 XML_CQUANT_REP = 2 XML_CTYPE_ANY = 2 XML_CTYPE_CHOICE = 5 XML_CTYPE_EMPTY = 1 XML_CTYPE_MIXED = 3 XML_CTYPE_NAME = 4 XML_CTYPE_SEQ = 6 This fails on PyPy 1.6, >>>> from xml.parsers.expat import model Traceback (most recent call last): File "", line 1, in ImportError: cannot import name 'model' See also: http://docs.python.org/library/pyexpat.html ---------- messages: 3342 nosy: pypy-issue priority: bug release: 1.6 status: unread title: Missing xml.parsers.expat.model ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Tue Oct 18 16:21:35 2011 From: tracker at bugs.pypy.org (Armin Rigo) Date: Tue, 18 Oct 2011 14:21:35 +0000 Subject: [pypy-issue] [issue908] Django tests failing In-Reply-To: <1318609403.25.0.782175954517.issue908@bugs.pypy.org> Message-ID: <1318947695.64.0.205015621888.issue908@bugs.pypy.org> Armin Rigo added the comment: Sorry, paste error. It was 81e9199c50b9+. I also tried on Linux32 with 759db9b281eb, and while I get a different set of errors, still no test_booleanfieldlistfilter. I still have the window with the 81e9199c50b9+ results around. No test_tabular_non_field_errors in it either... ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Tue Oct 18 16:31:38 2011 From: tracker at bugs.pypy.org (Peter) Date: Tue, 18 Oct 2011 14:31:38 +0000 Subject: [pypy-issue] [issue915] implement numpy.linalg in micronumpy In-Reply-To: <1318948298.71.0.561740137124.issue915@bugs.pypy.org> Message-ID: <1318948298.71.0.561740137124.issue915@bugs.pypy.org> New submission from Peter : PyPy's micronumpy does not include the numpy.linalg module, http://docs.scipy.org/doc/numpy/reference/routines.linalg.html Under NumPy 1.2.1, this includes: >>> import numpy.linalg >>> dir(numpy.linalg) ['LinAlgError', 'Tester', '__builtins__', '__doc__', '__file__', '__name__', '__package__', '__path__', 'bench', 'cholesky', 'cond', 'det', 'eig', 'eigh', 'eigvals', 'eigvalsh', 'info', 'inv', 'lapack_lite', 'linalg', 'lstsq', 'matrix_power', 'norm', 'pinv', 'qr', 'solve', 'svd', 'tensorinv', 'tensorsolve', 'test'] In particular we are using numpy.linalg.solve, numpy.linalg.det and numpy.linalg.svd and for us their performance is not a bottleneck in C Python. ---------- messages: 3344 nosy: pypy-issue priority: wish release: 1.6 status: unread title: implement numpy.linalg in micronumpy ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Tue Oct 18 16:36:36 2011 From: tracker at bugs.pypy.org (Garen Parham) Date: Tue, 18 Oct 2011 14:36:36 +0000 Subject: [pypy-issue] [issue691] pypy-head does not compile under Windows In-Reply-To: <1303737570.22.0.620331113708.issue691@> Message-ID: <1318948596.1.0.0217782612296.issue691@bugs.pypy.org> Garen Parham added the comment: On Windows 7 x64, trying to build trunk with pypy 1.6 crashes fairly early - the Visual Studio debugger throws up an Access Violation to address 0x00000000. Went looking for a more recent build on the nightly builds page for win32, but all the recent win32 builds for sometime show up as 0B: http://buildbot.pypy.org/nightly/trunk/ It doesn't look like there is a win32 build with all tests passing anywhere to try... Trying with the latest 32-bit python 2.7.2, it goes for quite awhile but eventually reaches the 2GB memory limit and crashes with a MemoryError--looks identical to what DragonSA reported in msg2492 on 2011-05-04. With pypy 1.5, it makes more progress, but eventually crashes too: File "implement.c", line 839, in entry_point File "implement_5.c", line 5243, in funccall__star_1 File "implement_5.c", line 5272, in funccall__star_1 File "implement_5.c", line 7266, in MiniMarkGC_collect_and_reserve File "implement_5.c", line 3243, in MiniMarkGC_minor_collection File "implement_6.c", line 36662, in MiniMarkGC_collect_oldrefs_to_nursery File "implement_7.c", line 43655, in trace___trace_drag_out File "implement_7.c", line 43222, in MiniMarkGC__trace_drag_out File "implement_6.c", line 43542, in ArenaCollection_malloc File "implement_7.c", line 54691, in ArenaCollection_allocate_new_page File "implement_13.c", line 43205, in ArenaCollection_allocate_new_arena Fatal RPython error: MemoryError Using pypy 1.5 seems to "work" in that it doesn't crash prematurely, but like CPython runs out of memory: [translation:ERROR] Error: [translation:ERROR] Traceback (most recent call last): [translation:ERROR] File "./translate.py", line 308, in main [translation:ERROR] drv.proceed(goals) [translation:ERROR] File "U:\vcs\hg\pypy\pypy\translator\driver.py", line 809, in proceed [translation:ERROR] return self._execute(goals, task_skip = self._maybe_skip()) [translation:ERROR] File "U:\vcs\hg\pypy\pypy\translator\tool\taskengine.py", line 116, in _execute [translation:ERROR] res = self._do(goal, taskcallable, *args, **kwds) [translation:ERROR] File "U:\vcs\hg\pypy\pypy\translator\driver.py", line 286, in _do [translation:ERROR] res = func() [translation:ERROR] File "U:\vcs\hg\pypy\pypy\translator\driver.py", line 505, in task_database_c [translation:ERROR] database = cbuilder.build_database() [translation:ERROR] File "U:\vcs\hg\pypy\pypy\translator\c\genc.py", line 168, in build_database [translation:ERROR] db.complete() [translation:ERROR] File "U:\vcs\hg\pypy\pypy\translator\c\database.py", line 306, in complete [translation:ERROR] add_dependencies(node.enum_dependencies()) [translation:ERROR] File "U:\vcs\hg\pypy\pypy\translator\c\database.py", line 294, in add_dependencies [translation:ERROR] self.get(value) [translation:ERROR] File "U:\vcs\hg\pypy\pypy\translator\c\database.py", line 230, in get [translation:ERROR] node = self.getcontainernode(container) [translation:ERROR] File "U:\vcs\hg\pypy\pypy\translator\c\database.py", line 164, in getcontainernode [translation:ERROR] node = nodefactory(self, T, container, **buildkwds) [translation:ERROR] File "U:\vcs\hg\pypy\pypy\translator\c\node.py", line 833, in __init__ [translation:ERROR] self.make_funcgens() [translation:ERROR] File "U:\vcs\hg\pypy\pypy\translator\c\node.py", line 841, in make_funcgens [translation:ERROR] self.funcgens = select_function_code_generators(self.obj, self.db, self.name) [translation:ERROR] File "U:\vcs\hg\pypy\pypy\translator\c\node.py", line 958, in select_function_code_generators [translation:ERROR] functionname)] [translation:ERROR] File "U:\vcs\hg\pypy\pypy\translator\c\funcgen.py", line 51, in __init__ [translation:ERROR] self.db.exctransformer.create_exception_handling(self.graph) [translation:ERROR] File "U:\vcs\hg\pypy\pypy\translator\exceptiontransform.py", line 201, in create_exception_handli ng [translation:ERROR] self.raise_analyzer.analyze_direct_call(graph) [translation:ERROR] File "U:\vcs\hg\pypy\pypy\translator\backendopt\graphanalyze.py", line 136, in analyze_direct_cal l [translation:ERROR] self.analyzed_calls[graph] = result [translation:ERROR] MemoryError (Using PYPY_GC_MAX_DELTA=200MB and loop_longevity=300 as suggested via the notes section of http://pypy.org/download.html) Other than using editbin (wish seems kind of fishy) I'm about out of straws and stabs in the dark. :) ---------- nosy: +Garen ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Tue Oct 18 16:50:58 2011 From: tracker at bugs.pypy.org (Armin Rigo) Date: Tue, 18 Oct 2011 14:50:58 +0000 Subject: [pypy-issue] [issue914] Missing xml.parsers.expat.model In-Reply-To: <1318947219.55.0.30396177762.issue914@bugs.pypy.org> Message-ID: <1318949458.32.0.488099060245.issue914@bugs.pypy.org> Armin Rigo added the comment: Thanks. Should be fixed in 595bfa0049db. ---------- nosy: +arigo status: unread -> chatting ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Tue Oct 18 18:13:53 2011 From: tracker at bugs.pypy.org (Fijal) Date: Tue, 18 Oct 2011 16:13:53 +0000 Subject: [pypy-issue] [issue691] pypy-head does not compile under Windows In-Reply-To: <1303737570.22.0.620331113708.issue691@> Message-ID: <1318954433.72.0.951604776476.issue691@bugs.pypy.org> Fijal added the comment: Windows 64 bit is unsupported. Any help would be appreciated. ---------- nosy: +fijal ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Tue Oct 18 23:41:32 2011 From: tracker at bugs.pypy.org (Jean-Paul Calderone) Date: Tue, 18 Oct 2011 21:41:32 +0000 Subject: [pypy-issue] [issue889] Windows binary distribution is missing cpyext header files In-Reply-To: <1317395541.31.0.735873324924.issue889@bugs.pypy.org> Message-ID: <1318974092.66.0.955364847971.issue889@bugs.pypy.org> Jean-Paul Calderone added the comment: C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\link.exe /DLL /nologo /INCREMENTAL:NO /LIBPATH:c:\Documents and Settings\Jean-Paul Calderone\Desktop\pypy-c\ include python27.lib /EXPORT:init_initgroups build\temp.win32-2.7\Release\twisted/python/_initgroups.obj /OUT:C:\Documents and Settings\Jean-Paul Calderone\Desk top\Projects\Twisted\twisted\python\_initgroups.pypy-16.pyd /IMPLIB:build\temp.win32-2.7\Release\twisted/python\_initgroups.pypy-16.lib /MANIFEST /MANIFESTFILE: build\temp.win32-2.7\Release\twisted/python\_initgroups.pypy-16.pyd.manifest LINK : fatal error LNK1181: cannot open input file 'python27.lib' error: command 'C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\link.exe' failed with exit status 1181 ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Tue Oct 18 23:53:46 2011 From: tracker at bugs.pypy.org (Armin Rigo) Date: Tue, 18 Oct 2011 21:53:46 +0000 Subject: [pypy-issue] [issue889] Windows binary distribution is missing cpyext header files In-Reply-To: <1317395541.31.0.735873324924.issue889@bugs.pypy.org> Message-ID: <1318974826.12.0.807804378613.issue889@bugs.pypy.org> Armin Rigo added the comment: No clue about this. So far it seems that twisted's build script is inserting "python27.lib", which makes little sense for pypy. Is it only twisted, or is it more general? ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Wed Oct 19 01:14:18 2011 From: tracker at bugs.pypy.org (Dominik Hannen) Date: Tue, 18 Oct 2011 23:14:18 +0000 Subject: [pypy-issue] [issue916] int(string, 0), if string is not a parseable number In-Reply-To: <1318979658.65.0.761643684658.issue916@bugs.pypy.org> Message-ID: <1318979658.65.0.761643684658.issue916@bugs.pypy.org> New submission from Dominik Hannen : >>>> int("",0) Traceback (most recent call last): File "", line 1, in ValueError: invalid literal for int() with base 10: '' Should be saying ".. base 0: '' .." like CPython does: >>> int("",0) Traceback (most recent call last): File "", line 1, in ValueError: invalid literal for int() with base 0: '' .. the supplied patch adds a test for this into pypy/objspace/std/test/test_strutil.py ---------- files: test_strutil.patch messages: 3350 nosy: dhx, pypy-issue priority: bug status: unread title: int(string, 0), if string is not a parseable number ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Wed Oct 19 01:20:00 2011 From: tracker at bugs.pypy.org (Dominik Hannen) Date: Tue, 18 Oct 2011 23:20:00 +0000 Subject: [pypy-issue] [issue916] int(string, 0), if string is not a parseable number In-Reply-To: <1318979658.65.0.761643684658.issue916@bugs.pypy.org> Message-ID: <1318980000.6.0.326632933085.issue916@bugs.pypy.org> Dominik Hannen added the comment: And here is a possible patch for pypy/objspace/std/strutil.py I am not sure if it is useful, but it fixes the issue. ---------- status: unread -> chatting ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Wed Oct 19 13:17:20 2011 From: tracker at bugs.pypy.org (Armin Rigo) Date: Wed, 19 Oct 2011 11:17:20 +0000 Subject: [pypy-issue] [issue916] int(string, 0), if string is not a parseable number In-Reply-To: <1318979658.65.0.761643684658.issue916@bugs.pypy.org> Message-ID: <1319023040.23.0.402437823128.issue916@bugs.pypy.org> Armin Rigo added the comment: Just for fun: on CPython, >>> int("5", -909) 5 But only if the base is set to -909. PyPy doesn't reproduce this behavior, but I guess this is not a bug :-) ---------- nosy: +arigo ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Wed Oct 19 13:25:54 2011 From: tracker at bugs.pypy.org (Armin Rigo) Date: Wed, 19 Oct 2011 11:25:54 +0000 Subject: [pypy-issue] [issue916] int(string, 0), if string is not a parseable number In-Reply-To: <1318979658.65.0.761643684658.issue916@bugs.pypy.org> Message-ID: <1319023554.94.0.310568549865.issue916@bugs.pypy.org> Armin Rigo added the comment: Accepted, thanks. ff2cfafa5e63 ---------- status: chatting -> resolved ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Wed Oct 19 13:27:03 2011 From: tracker at bugs.pypy.org (Amaury Forgeot d Arc) Date: Wed, 19 Oct 2011 11:27:03 +0000 Subject: [pypy-issue] [issue916] int(string, 0), if string is not a parseable number In-Reply-To: <1318979658.65.0.761643684658.issue916@bugs.pypy.org> Message-ID: <1319023623.63.0.651738278507.issue916@bugs.pypy.org> Amaury Forgeot d Arc added the comment: yes, CPython 2.7 uses -909 for the default. This weird behavior was removed in python3. ---------- nosy: +afa status: resolved -> chatting ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Wed Oct 19 22:58:54 2011 From: tracker at bugs.pypy.org (Dominik Hannen) Date: Wed, 19 Oct 2011 20:58:54 +0000 Subject: [pypy-issue] [issue917] raw_input ignores stdout.softspace In-Reply-To: <1319057934.8.0.920793195061.issue917@bugs.pypy.org> Message-ID: <1319057934.8.0.920793195061.issue917@bugs.pypy.org> New submission from Dominik Hannen : raw_input ignores stdout.softspace when writing its prompt to stdout. Then the softspace is used by the next print statement after raw_input. The bug is reproducible with this script: ############ problem.py: print "Input:", raw_input() print("<< no softspace") print "\nsoftspace after ':'" print "Input:", raw_input("prompt") ############ The output is (PyPy): Input: << no softspace softspace after ':' Input:prompt ############ It should look like: Input: << no softspace softspace after ':' Input: prompt ############ The supplied patchfile contains tests and a patch to correct this. ---------- files: app_io.patch messages: 3355 nosy: dhx, pypy-issue priority: bug status: unread title: raw_input ignores stdout.softspace ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Thu Oct 20 00:08:49 2011 From: tracker at bugs.pypy.org (Armin Rigo) Date: Wed, 19 Oct 2011 22:08:49 +0000 Subject: [pypy-issue] [issue917] raw_input ignores stdout.softspace In-Reply-To: <1319057934.8.0.920793195061.issue917@bugs.pypy.org> Message-ID: <1319062129.63.0.251382133347.issue917@bugs.pypy.org> Armin Rigo added the comment: Thanks. Checked in a slightly different fix (leaving to "print" the detection of the softspace, and being careful about AttributeErrors). Your tests now pass. ---------- nosy: +arigo status: unread -> resolved ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Thu Oct 20 01:35:04 2011 From: tracker at bugs.pypy.org (timo) Date: Wed, 19 Oct 2011 23:35:04 +0000 Subject: [pypy-issue] [issue915] implement numpy.linalg in micronumpy In-Reply-To: <1318948298.71.0.561740137124.issue915@bugs.pypy.org> Message-ID: <1319067304.5.0.135939021488.issue915@bugs.pypy.org> timo added the comment: I suppose most of this is implemented in numpy by calling BLAS. Should we call BLAS, too, to implement this, implement it in pure-py or wait for any kind of solution to the general numpy/micronumpy situation before continuing with this? ---------- nosy: +timo status: unread -> chatting ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Thu Oct 20 10:36:35 2011 From: tracker at bugs.pypy.org (Armin Rigo) Date: Thu, 20 Oct 2011 08:36:35 +0000 Subject: [pypy-issue] [issue904] pypy build translation fails due to use of non-standard ifr_ifindex In-Reply-To: <1318370167.35.0.638855593826.issue904@bugs.pypy.org> Message-ID: <1319099795.95.0.263094245834.issue904@bugs.pypy.org> Armin Rigo added the comment: For completeness: you are likely compiling on Solaris, according to issue902. ---------- nosy: +arigo status: unread -> chatting ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Thu Oct 20 11:00:14 2011 From: tracker at bugs.pypy.org (Armin Rigo) Date: Thu, 20 Oct 2011 09:00:14 +0000 Subject: [pypy-issue] [issue904] pypy build translation fails due to use of non-standard ifr_ifindex In-Reply-To: <1318370167.35.0.638855593826.issue904@bugs.pypy.org> Message-ID: <1319101214.53.0.416818063308.issue904@bugs.pypy.org> Armin Rigo added the comment: Fixed in 82601a1c1f1d by only enabling AF_PACKET on Linux. This follows the comment on bugs.python.org: The AF_PACKET support was original meant for Linux. When Solaris now also supports that socket family, and with a similar interface, it might be interesting to port that support to Solaris. Notably, "similar" means "but not identical". ---------- status: chatting -> resolved ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Thu Oct 20 11:05:46 2011 From: tracker at bugs.pypy.org (Armin Rigo) Date: Thu, 20 Oct 2011 09:05:46 +0000 Subject: [pypy-issue] [issue902] pypy translation build fails due to non-POSIX time field usage In-Reply-To: <1318363131.14.0.376606025293.issue902@bugs.pypy.org> Message-ID: <1319101546.07.0.581750052524.issue902@bugs.pypy.org> Armin Rigo added the comment: Bah, it's probably re-enabled by the following lines in pypyoption.py: module_dependencies = { '_multiprocessing': [('objspace.usemodules.rctime', True), ('objspace.usemodules.thread', True)], } I would suggest that, in order to get a really working pypy on Solaris, you (or someone with interest in Solaris) spends some time fixing at least some of the modules listed in "del working_modules[..]". Some of them are really needed: almost no Python program would work without a "time" module, for example. ---------- nosy: +arigo -binarycrusader status: unread -> chatting ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Thu Oct 20 12:14:50 2011 From: tracker at bugs.pypy.org (Armin Rigo) Date: Thu, 20 Oct 2011 10:14:50 +0000 Subject: [pypy-issue] [issue897] minimark GC does not work with make lldebug In-Reply-To: <1318106695.63.0.0405614409621.issue897@bugs.pypy.org> Message-ID: <1319105690.77.0.260233687629.issue897@bugs.pypy.org> Armin Rigo added the comment: Closing as "works for me". ---------- status: chatting -> invalid ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Thu Oct 20 12:44:14 2011 From: tracker at bugs.pypy.org (Fijal) Date: Thu, 20 Oct 2011 10:44:14 +0000 Subject: [pypy-issue] [issue915] implement numpy.linalg in micronumpy In-Reply-To: <1318948298.71.0.561740137124.issue915@bugs.pypy.org> Message-ID: <1319107454.83.0.080962216283.issue915@bugs.pypy.org> Fijal added the comment: We should totally call BLAS ---------- nosy: +fijal ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Thu Oct 20 12:54:03 2011 From: tracker at bugs.pypy.org (Armin Rigo) Date: Thu, 20 Oct 2011 10:54:03 +0000 Subject: [pypy-issue] [issue892] Pypy is slower on longs than it should In-Reply-To: <1317663715.18.0.663914446646.issue892@bugs.pypy.org> Message-ID: <1319108043.01.0.916648801485.issue892@bugs.pypy.org> Armin Rigo added the comment: What was the reason for which CPython uses its own "long" instead of relying on the faster gmp library? And: does it make sense for us to just use gmp? ---------- nosy: +arigo ________________________________________ PyPy bug tracker ________________________________________ From amauryfa at gmail.com Thu Oct 20 13:31:01 2011 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Thu, 20 Oct 2011 13:31:01 +0200 Subject: [pypy-issue] [issue892] Pypy is slower on longs than it should In-Reply-To: <1319108043.01.0.916648801485.issue892@bugs.pypy.org> References: <1317663715.18.0.663914446646.issue892@bugs.pypy.org> <1319108043.01.0.916648801485.issue892@bugs.pypy.org> Message-ID: 2011/10/20 Armin Rigo > What was the reason for which CPython uses its own "long" instead of > relying on > the faster gmp library? > Because of the LGPL license? -- Amaury Forgeot d'Arc From tracker at bugs.pypy.org Thu Oct 20 14:04:57 2011 From: tracker at bugs.pypy.org (Armin Rigo) Date: Thu, 20 Oct 2011 12:04:57 +0000 Subject: [pypy-issue] [issue892] Pypy is slower on longs than it should In-Reply-To: <1317663715.18.0.663914446646.issue892@bugs.pypy.org> Message-ID: <1319112297.61.0.0547472517987.issue892@bugs.pypy.org> Armin Rigo added the comment: But they have the 'readline' module, which links to a GPL-licensed library, which seems far more restrictive than LGPL. Oh well. I suppose I'll stop trying to understand license issues --- beyond the fact that LGPL is fine as far as I'm concerned --- and just try to implement it. ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Thu Oct 20 15:17:01 2011 From: tracker at bugs.pypy.org (anon) Date: Thu, 20 Oct 2011 13:17:01 +0000 Subject: [pypy-issue] [issue892] Pypy is slower on longs than it should In-Reply-To: <1317663715.18.0.663914446646.issue892@bugs.pypy.org> Message-ID: <1319116621.01.0.217654217349.issue892@bugs.pypy.org> anon added the comment: Using GMP would also fix: https://bugs.pypy.org/issue819 ---------- nosy: +anon ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Thu Oct 20 16:01:37 2011 From: tracker at bugs.pypy.org (Leonardo Santagada) Date: Thu, 20 Oct 2011 14:01:37 +0000 Subject: [pypy-issue] [issue892] Pypy is slower on longs than it should In-Reply-To: <1317663715.18.0.663914446646.issue892@bugs.pypy.org> Message-ID: <1319119297.98.0.453403116219.issue892@bugs.pypy.org> Leonardo Santagada added the comment: the problem is that without gmp you would end up without longs in python, but without readline you still have a functional language(and you can link against lidedit on bsd). If you can make it link to gmp if it is avaliable and just use the slower implementation if not then I don't see a problem. ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Thu Oct 20 17:56:15 2011 From: tracker at bugs.pypy.org (Armin Rigo) Date: Thu, 20 Oct 2011 15:56:15 +0000 Subject: [pypy-issue] [issue892] Pypy is slower on longs than it should In-Reply-To: <1317663715.18.0.663914446646.issue892@bugs.pypy.org> Message-ID: <1319126175.63.0.685987915676.issue892@bugs.pypy.org> Armin Rigo added the comment: Sorry, I give up already on "gmp". It never reports out-of-memory errors. We can change the allocation functions, but out of memory conditions must terminate the program. Even using longjmp() to bail out of the allocator is specified as "don't do that". There is no way this is compatible with Python short of dedicating a separate process in which to run all long operations, but this makes the interface slow as hell. ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Thu Oct 20 18:08:25 2011 From: tracker at bugs.pypy.org (Amaury Forgeot d Arc) Date: Thu, 20 Oct 2011 16:08:25 +0000 Subject: [pypy-issue] [issue892] Pypy is slower on longs than it should In-Reply-To: <1317663715.18.0.663914446646.issue892@bugs.pypy.org> Message-ID: <1319126905.53.0.0444643582981.issue892@bugs.pypy.org> Amaury Forgeot d Arc added the comment: It seems that people use C++ exceptions with some success: http://gmplib.org/list-archives/gmp-discuss/2008-December/003479.html see the last paragraph. ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Thu Oct 20 21:19:43 2011 From: tracker at bugs.pypy.org (Garen Parham) Date: Thu, 20 Oct 2011 19:19:43 +0000 Subject: [pypy-issue] [issue691] pypy-head does not compile under Windows In-Reply-To: <1303737570.22.0.620331113708.issue691@> Message-ID: <1319138383.37.0.127932012318.issue691@bugs.pypy.org> Garen Parham added the comment: I should have probably mentioned that all the programs being run were 32-bit (CPython 2.7, pypy 1.5, 1.6) so the issue isn't that I'm building with 64-bit tools. For good measure, I reproduced all the previously reported problems on Windows 7 x86 (32-bit). After using editbin to set the pypy-c.exe to be large address aware, the build gets most of the way through, but fails to link; the link failure indicates a missing symbol in zlib. The version of zlib used was 1.2.3, linked from the pypy docs here: http://codespeak.net/pypy/dist/pypy/doc/windows.html I tried a pre-built DLL of the current zlib version (1.2.5) from zlib.net, but got the same link error. Strangely, later I noticed the link error message also contained a path to link.exe, which is the wrong path. The shell environment is created by the "Visual Studio 2008 Command Prompt" shortcut, and looking in the PATH reveals no thing 2010 related. Despite that, the link.exe path in the error message was "C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\bin\link.exe" Perhaps there's an issue with folks who have multiple versions installed side- by-side? Smells that way so far. ---------- release: ??? -> 1.6 ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Thu Oct 20 21:22:04 2011 From: tracker at bugs.pypy.org (Garen Parham) Date: Thu, 20 Oct 2011 19:22:04 +0000 Subject: [pypy-issue] [issue691] pypy-head does not compile under Windows In-Reply-To: <1303737570.22.0.620331113708.issue691@> Message-ID: <1319138524.61.0.697834229147.issue691@bugs.pypy.org> Garen Parham added the comment: Should have also mentioned: the pypy-c.exe I set to be "largeaddressaware" was from 1.5; pypy 1.6 still crashes with a null pointer deref issue after about a minute; using a recent nightly build also crashes in the same way (http://buildbot.pypy.org/nightly/trunk/pypy-c-jit-48192-1ae56b2043d4-win32.zip) ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Sat Oct 22 15:43:51 2011 From: tracker at bugs.pypy.org (Laurence Tratt) Date: Sat, 22 Oct 2011 13:43:51 +0000 Subject: [pypy-issue] [issue907] Stacklets running out of memory on higher optimisation levels In-Reply-To: <1318523264.95.0.361208519335.issue907@bugs.pypy.org> Message-ID: <1319291031.72.0.419406007038.issue907@bugs.pypy.org> Laurence Tratt added the comment: To ensure this isn't a platform issue (with OpenBSD), I've now confirmed that the same thing happens on PyPy with OS X. At --opt=2 and above, the memory of the VM grows continuously when stacklets are used. [I wasn't able to get --opt=1 to build because PyPy doesn't seem to pick up the boehm-gc libraries properly on the machine I have access to.] I'm not able to tell whether this problem happens under Windows or Linux as well. ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Sun Oct 23 00:19:34 2011 From: tracker at bugs.pypy.org (Peter) Date: Sat, 22 Oct 2011 22:19:34 +0000 Subject: [pypy-issue] [issue918] implement numpy.matrix in micronumpy In-Reply-To: <1319321974.08.0.127911146223.issue918@bugs.pypy.org> Message-ID: <1319321974.08.0.127911146223.issue918@bugs.pypy.org> New submission from Peter : Following should work on any version of bumpy, here using 1.5.1 on Python 2.6 on Mac OS X, >>> import numpy >>> m = numpy.matrix([[1,0],[0,1]]) >>> m matrix([[1, 0], [0, 1]]) Fails on pypy 1.6, $ pypy Python 2.7.1 (dcae7aed462b, Aug 17 2011, 09:46:15) [PyPy 1.6.0 with GCC 4.0.1] on darwin Type "help", "copyright", "credits" or "license" for more information. And now for something completely different: ``it's not a hack, it's a workaround'' >>>> import numpy >>>> m = numpy.matrix([[1,0],[0,1]]) Traceback (most recent call last): File "", line 1, in AttributeError: 'module' object has no attribute 'matrix' The matrix class is generally similar to a 2 dimensional array in numpy except that the * operator does matrix multiplication rather than element wise operation. This is particularly useful in teaching classes, even if many serious bumpy users prefer to avoid the matrix class and work with raw arrays and the dot method/function. ---------- messages: 3373 nosy: pypy-issue priority: bug release: 1.6 status: unread title: implement numpy.matrix in micronumpy ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Sun Oct 23 00:33:31 2011 From: tracker at bugs.pypy.org (Peter) Date: Sat, 22 Oct 2011 22:33:31 +0000 Subject: [pypy-issue] [issue919] implement multi-dimention arrays in micronumpy In-Reply-To: <1319322811.27.0.133989194674.issue919@bugs.pypy.org> Message-ID: <1319322811.27.0.133989194674.issue919@bugs.pypy.org> New submission from Peter : The following should work in any recent release of numpy, >>> import numpy >>> two_dim = numpy.array([[1,0],[0,1]]) >>> two_dim array([[1, 0], [0, 1]]) >>> three_dim = numpy.array([[[1,0],[0,1]],[[1,2],[0,1]]]) >>> three_dim array([[[1, 0], [0, 1]], [[1, 2], [0, 1]]]) However, while one dimensional arrays seem to work, higher dimensions do not: >>>> two_dim = numpy.array([[1,0],[0,1]]) Traceback (most recent call last): File "", line 1, in >>>> three_dim = numpy.array([[[1,0],[0,1]],[[1,2],[0,1]]]) Traceback (most recent call last): File "", line 1, in TypeError: unsupported operand type for float(): 'list' Note this also blocks implementing the matrix class, see Issue 918. I filed that first because I hit that first (and it didn't occur to me till afterwards to check if arrays worked properly). ---------- messages: 3374 nosy: peterjc, pypy-issue priority: bug release: 1.6 status: unread title: implement multi-dimention arrays in micronumpy ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Sun Oct 23 00:34:08 2011 From: tracker at bugs.pypy.org (Peter) Date: Sat, 22 Oct 2011 22:34:08 +0000 Subject: [pypy-issue] [issue918] implement numpy.matrix in micronumpy In-Reply-To: <1319321974.08.0.127911146223.issue918@bugs.pypy.org> Message-ID: <1319322848.97.0.0416091533096.issue918@bugs.pypy.org> Peter added the comment: Note this will probably require fixing Issue 919 first, multidimensional arrays. ---------- nosy: +peterjc status: unread -> chatting ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Mon Oct 24 20:49:19 2011 From: tracker at bugs.pypy.org (Stefano Rivera) Date: Mon, 24 Oct 2011 18:49:19 +0000 Subject: [pypy-issue] [issue920] Lost data with async I/O In-Reply-To: <1319482159.83.0.0983179975147.issue920@bugs.pypy.org> Message-ID: <1319482159.83.0.0983179975147.issue920@bugs.pypy.org> New submission from Stefano Rivera : The attached file, async_sub.py, is a slightly modified for debugging copy, of a file of the same name in pygame, which is a descendant of an ActiveState recipe. It loses output sometimes, when there are many EAGAINs on reads. This is easy to trigger when running a script containing: #!/bin/sh for i in `seq 10`; do echo "hi" sleep 0.002 # tweak this, anywhere up to 0.1 done This is due to in-method buffering during loops at many layers of the stream stack. The EAGAIN exceptions are unexpected and cause the buffers to be lost. ---------- files: async_sub.py messages: 3376 nosy: pypy-issue, stefanor priority: bug status: unread title: Lost data with async I/O ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Mon Oct 24 20:50:36 2011 From: tracker at bugs.pypy.org (Stefano Rivera) Date: Mon, 24 Oct 2011 18:50:36 +0000 Subject: [pypy-issue] [issue920] Lost data with async I/O In-Reply-To: <1319482159.83.0.0983179975147.issue920@bugs.pypy.org> Message-ID: <1319482236.84.0.533495842144.issue920@bugs.pypy.org> Stefano Rivera added the comment: I have a fix in progress in https://bitbucket.org/stefanor/pypy (see https://bitbucket.org/pypy/pypy/pull-request/12/dont-lose-data-when-doing-non- blocking-i-o ) ---------- status: unread -> chatting ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Tue Oct 25 14:09:15 2011 From: tracker at bugs.pypy.org (Armin Rigo) Date: Tue, 25 Oct 2011 12:09:15 +0000 Subject: [pypy-issue] [issue907] Stacklets running out of memory on higher optimisation levels In-Reply-To: <1318523264.95.0.361208519335.issue907@bugs.pypy.org> Message-ID: <1319544555.31.0.617839040904.issue907@bugs.pypy.org> Armin Rigo added the comment: It seems I need to have Stdlib.cvl and possibly others. How do I build them? ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Tue Oct 25 14:23:24 2011 From: tracker at bugs.pypy.org (Laurence Tratt) Date: Tue, 25 Oct 2011 12:23:24 +0000 Subject: [pypy-issue] [issue907] Stacklets running out of memory on higher optimisation levels In-Reply-To: <1318523264.95.0.361208519335.issue907@bugs.pypy.org> Message-ID: <1319545404.61.0.0178537459838.issue907@bugs.pypy.org> Laurence Tratt added the comment: The best option for you to test this problem will be to roll back to an older commit, as a few other things have changed and expanded in the interim, and they'll needlessly muddy the waters as far as you're concerned. I'd suggest the c56aac83ffe37ace4ef18aa99a3ee98a5d59ef5b commit is a good one for your purposes. ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Tue Oct 25 20:00:28 2011 From: tracker at bugs.pypy.org (Fijal) Date: Tue, 25 Oct 2011 18:00:28 +0000 Subject: [pypy-issue] [issue921] _structseq is a slow mess In-Reply-To: <1319565628.25.0.385940441591.issue921@bugs.pypy.org> Message-ID: <1319565628.25.0.385940441591.issue921@bugs.pypy.org> New submission from Fijal : http://paste.pocoo.org/show/498047/ is 2x slower under pypy than cpython ---------- messages: 3380 nosy: fijal, pypy-issue priority: performance bug status: unread title: _structseq is a slow mess ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Tue Oct 25 20:41:38 2011 From: tracker at bugs.pypy.org (Alex Gaynor) Date: Tue, 25 Oct 2011 18:41:38 +0000 Subject: [pypy-issue] [issue921] _structseq is a slow mess In-Reply-To: <1319565628.25.0.385940441591.issue921@bugs.pypy.org> Message-ID: <1319568098.51.0.429074565594.issue921@bugs.pypy.org> Alex Gaynor added the comment: I guess it'd be nice if this was a more unittest type thing, without needing random IO. Can we write a benchmark that just constructs a structseq class and instantiates it? ---------- nosy: +agaynor status: unread -> chatting ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Tue Oct 25 20:51:44 2011 From: tracker at bugs.pypy.org (Amaury Forgeot d Arc) Date: Tue, 25 Oct 2011 18:51:44 +0000 Subject: [pypy-issue] [issue921] _structseq is a slow mess In-Reply-To: <1319565628.25.0.385940441591.issue921@bugs.pypy.org> Message-ID: <1319568704.58.0.289406047621.issue921@bugs.pypy.org> Amaury Forgeot d Arc added the comment: os.stat_result((None,) * 10) ---------- nosy: +afa ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Wed Oct 26 01:12:53 2011 From: tracker at bugs.pypy.org (awsum) Date: Tue, 25 Oct 2011 23:12:53 +0000 Subject: [pypy-issue] [issue922] 1.6 crashes with time.sleep(negative_number) In-Reply-To: <1319584373.54.0.90788508105.issue922@bugs.pypy.org> Message-ID: <1319584373.54.0.90788508105.issue922@bugs.pypy.org> New submission from awsum : cpython just throws IOError: Invalid argument ---------- messages: 3383 nosy: awsum, pypy-issue priority: bug release: 1.6 status: unread title: 1.6 crashes with time.sleep(negative_number) ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Wed Oct 26 03:04:15 2011 From: tracker at bugs.pypy.org (timo) Date: Wed, 26 Oct 2011 01:04:15 +0000 Subject: [pypy-issue] [issue922] 1.6 crashes with time.sleep(negative_number) In-Reply-To: <1319584373.54.0.90788508105.issue922@bugs.pypy.org> Message-ID: <1319591055.18.0.15809054316.issue922@bugs.pypy.org> timo added the comment: Should be fixed with the last two commits I made to pypy. Unfortunately I missed the buildbots by 5 minutes, so the next nightly will not yet have the fix for you. ---------- nosy: +timo status: in-progress -> testing ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Thu Oct 27 06:54:21 2011 From: tracker at bugs.pypy.org (Bruno Gola) Date: Thu, 27 Oct 2011 04:54:21 +0000 Subject: [pypy-issue] [issue923] Regular expressions with optional look-ahead operations failing to match In-Reply-To: <1319691261.3.0.984459192648.issue923@bugs.pypy.org> Message-ID: <1319691261.3.0.984459192648.issue923@bugs.pypy.org> New submission from Bruno Gola : I'm trying to run moin-2.0 on PyPy and I found this weird behavior that I think is a bug. Whenever you have a regexp that uses the look-ahead operator [ (?=) ] followed by the optional operator [ ? ], the regex fails. For example: CPython 2.7: >>> re.compile('asdf(?=(1))?').match('asdf1a').groups() ('1',) PyPy 1.6: >>>> re.compile('asdf(?=(1))?').match('asdf1a').groups() (None,) I believe this affects the PyPy development version as well as there were almost no changes to the RE stuff in PyPy. ---------- messages: 3385 nosy: bgola, pypy-issue priority: bug release: 1.7 status: unread title: Regular expressions with optional look-ahead operations failing to match ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Thu Oct 27 10:52:30 2011 From: tracker at bugs.pypy.org (Ronny Pfannschmidt) Date: Thu, 27 Oct 2011 08:52:30 +0000 Subject: [pypy-issue] [issue924] stdlib heapq islice missuse In-Reply-To: <1319705550.59.0.693123252564.issue924@bugs.pypy.org> Message-ID: <1319705550.59.0.693123252564.issue924@bugs.pypy.org> New submission from Ronny Pfannschmidt : http://bugs.python.org/issue13274 needs a copy in modified-2.7 and tests ---------- assignedto: ronny messages: 3386 nosy: pypy-issue, ronny priority: bug status: in-progress title: stdlib heapq islice missuse ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Thu Oct 27 16:07:14 2011 From: tracker at bugs.pypy.org (Bruno Gola) Date: Thu, 27 Oct 2011 14:07:14 +0000 Subject: [pypy-issue] [issue923] Regular expressions with optional look-ahead operations failing to match groups In-Reply-To: <1319691261.3.0.984459192648.issue923@bugs.pypy.org> Message-ID: <1319724434.39.0.43634512423.issue923@bugs.pypy.org> Bruno Gola added the comment: More information on this, I simplified the test case and did some debug. The regexps bytecode are the same on both PyPy and CPython. PyPy 1.6: >>>> re.compile("1(?=(1))?").match("11").groups() # Using the '?' operator on the whole assert/look-ahead block fails (None,) >>>> re.compile("1(?=(1)?)").match("11").groups() # Using the '?' operator just on the subpattern ('1',) CPython 2.7 >>> re.compile("1(?=(1))?").match("11").groups() ('1',) >>> re.compile("1(?=(1)?)").match("11").groups() ('1',) ---------- status: unread -> chatting title: Regular expressions with optional look-ahead operations failing to match -> Regular expressions with optional look-ahead operations failing to match groups ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Thu Oct 27 22:19:36 2011 From: tracker at bugs.pypy.org (Laurence Tratt) Date: Thu, 27 Oct 2011 20:19:36 +0000 Subject: [pypy-issue] [issue925] RPython arrays sometimes getting the wrong type? In-Reply-To: <1319746776.9.0.524751907394.issue925@bugs.pypy.org> Message-ID: <1319746776.9.0.524751907394.issue925@bugs.pypy.org> New submission from Laurence Tratt : This patch to the RPython Converge VM was meant to be a simple code tidy-up: https://github.com/ltratt/converge/commit/65cc1dc822c19df0f3478b0ac01a169d51350d93 Under --opt=3 and below it works fine. With --opt=jit, the translation throws up this error: [translation:ERROR] Error: [translation:ERROR] Traceback (most recent call last): [translation:ERROR] File "/home/ltratt/tmp/pypy/pypy/translator/goal/translate.py", line 308, in main [translation:ERROR] drv.proceed(goals) [translation:ERROR] File "/home/ltratt/tmp/pypy/pypy/translator/driver.py", line 809, in proceed [translation:ERROR] return self._execute(goals, task_skip = self._maybe_skip()) [translation:ERROR] File "/home/ltratt/tmp/pypy/pypy/translator/tool/taskengine.py", line 116, in _execute [translation:ERROR] res = self._do(goal, taskcallable, *args, **kwds) [translation:ERROR] File "/home/ltratt/tmp/pypy/pypy/translator/driver.py", line 286, in _do [translation:ERROR] res = func() [translation:ERROR] File "/home/ltratt/tmp/pypy/pypy/translator/driver.py", line 397, in task_pyjitpl_lltype [translation:ERROR] backend_name=self.config.translation.jit_backend, inline=True) [translation:ERROR] File "/home/ltratt/tmp/pypy/pypy/jit/metainterp/warmspot.py", line 43, in apply_jit [translation:ERROR] **kwds) [translation:ERROR] File "/home/ltratt/tmp/pypy/pypy/jit/metainterp/warmspot.py", line 210, in __init__ [translation:ERROR] self.codewriter.make_jitcodes(verbose=verbose) [translation:ERROR] File "/home/ltratt/tmp/pypy/pypy/jit/codewriter/codewriter.py", line 72, in make_jitcodes [translation:ERROR] self.transform_graph_to_jitcode(graph, jitcode, verbose) [translation:ERROR] File "/home/ltratt/tmp/pypy/pypy/jit/codewriter/codewriter.py", line 41, in transform_graph_to_jitcode [translation:ERROR] transform_graph(graph, self.cpu, self.callcontrol, portal_jd) [translation:ERROR] File "/home/ltratt/tmp/pypy/pypy/jit/codewriter/jtransform.py", line 24, in transform_graph [translation:ERROR] t.transform(graph) [translation:ERROR] File "/home/ltratt/tmp/pypy/pypy/jit/codewriter/jtransform.py", line 43, in transform [translation:ERROR] self.optimize_block(block) [translation:ERROR] File "/home/ltratt/tmp/pypy/pypy/jit/codewriter/jtransform.py", line 67, in optimize_block [translation:ERROR] oplist = self.rewrite_operation(op) [translation:ERROR] File "/home/ltratt/tmp/pypy/pypy/jit/codewriter/jtransform.py", line 202, in rewrite_operation [translation:ERROR] return rewrite(self, op) [translation:ERROR] File "/home/ltratt/tmp/pypy/pypy/jit/codewriter/jtransform.py", line 520, in rewrite_op_getarrayitem [translation:ERROR] arraydescr = self.cpu.arraydescrof(ARRAY) [translation:ERROR] File "/home/ltratt/tmp/pypy/pypy/jit/backend/llsupport/llmodel.py", line 236, in arraydescrof [translation:ERROR] return get_array_descr(self.gc_ll_descr, A) [translation:ERROR] File "/home/ltratt/tmp/pypy/pypy/jit/backend/llsupport/descr.py", line 254, in get_array_descr [translation:ERROR] assert isinstance(ARRAY, lltype.GcArray) [translation:ERROR] AssertionError Sticking a print in to see what type ARRAY is gives this: Array of * Func ( * GcStruct VM.VM { super, inst_pypy_config, inst_builtins, inst_cf_stack, inst_mods, inst_spare_ff, inst_st } ) -> Void It turns out that only one tiny little part of the patch causes problems. This patch reverts two lines: https://github.com/ltratt/converge/commit/75498f726fda2056722bac69c1388f42c229935c but then ARRAY has what I assume is the correct type: GcArray of dictentry { key: * GcStruct rpy_string { hash, chars }, value: * GcStruct object { typeptr } } At the very least, everything executes as I would expect with this type! What's weird is that executing one of: Builtins.new_c_con_func_for_mod(vm, "exit", exit, mod) Builtins.new_c_con_func_for_mod(vm, "println", println, mod) on their own is fine (i.e. comment out either one, and things translate fine). It's only in conjunction with each other that they throw a problem. This is all running against PyPy HEAD. I have to be honest, this one has got me completely stumped. ---------- messages: 3388 nosy: ltratt, pypy-issue priority: bug status: unread title: RPython arrays sometimes getting the wrong type? ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Fri Oct 28 01:47:13 2011 From: tracker at bugs.pypy.org (Bruno Gola) Date: Thu, 27 Oct 2011 23:47:13 +0000 Subject: [pypy-issue] [issue923] Regular expressions with optional look-ahead operations failing to match groups In-Reply-To: <1319691261.3.0.984459192648.issue923@bugs.pypy.org> Message-ID: <1319759233.76.0.363074186336.issue923@bugs.pypy.org> Bruno Gola added the comment: I just opened a pull request that fixes this issue. https://bitbucket.org/pypy/pypy/pull-request/13/fixes-issue-923 ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Fri Oct 28 09:41:41 2011 From: tracker at bugs.pypy.org (Fijal) Date: Fri, 28 Oct 2011 07:41:41 +0000 Subject: [pypy-issue] [issue923] Regular expressions with optional look-ahead operations failing to match groups In-Reply-To: <1319691261.3.0.984459192648.issue923@bugs.pypy.org> Message-ID: <1319787701.61.0.66201790108.issue923@bugs.pypy.org> Fijal added the comment: Merged, thanks! ---------- nosy: +fijal status: chatting -> resolved ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Mon Oct 31 08:20:40 2011 From: tracker at bugs.pypy.org (Armin Rigo) Date: Mon, 31 Oct 2011 07:20:40 +0000 Subject: [pypy-issue] [issue925] RPython arrays sometimes getting the wrong type? In-Reply-To: <1319746776.9.0.524751907394.issue925@bugs.pypy.org> Message-ID: <1320045640.55.0.343415663651.issue925@bugs.pypy.org> Armin Rigo added the comment: Fixed (7c12a810b394). ---------- nosy: +arigo status: unread -> resolved ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Mon Oct 31 11:13:30 2011 From: tracker at bugs.pypy.org (Laurence Tratt) Date: Mon, 31 Oct 2011 10:13:30 +0000 Subject: [pypy-issue] [issue925] RPython arrays sometimes getting the wrong type? In-Reply-To: <1319746776.9.0.524751907394.issue925@bugs.pypy.org> Message-ID: <1320056010.72.0.843589801873.issue925@bugs.pypy.org> Laurence Tratt added the comment: This doesn't appear to fix the bug for me. If I revert this patch: https://github.com/ltratt/converge/commit/75498f726fda2056722bac69c1388f42c229935c I still get a similar error (albeit with a line number or two changed): [translation:ERROR] Error: [translation:ERROR] Traceback (most recent call last): [translation:ERROR] File "/home/ltratt/tmp/pypy/pypy/translator/goal/translate.py", line 308, in main [translation:ERROR] drv.proceed(goals) [translation:ERROR] File "/home/ltratt/tmp/pypy/pypy/translator/driver.py", line 809, in proceed [translation:ERROR] return self._execute(goals, task_skip = self._maybe_skip()) [translation:ERROR] File "/home/ltratt/tmp/pypy/pypy/translator/tool/taskengine.py", line 116, in _execute [translation:ERROR] res = self._do(goal, taskcallable, *args, **kwds) [translation:ERROR] File "/home/ltratt/tmp/pypy/pypy/translator/driver.py", line 286, in _do [translation:ERROR] res = func() [translation:ERROR] File "/home/ltratt/tmp/pypy/pypy/translator/driver.py", line 397, in task_pyjitpl_lltype [translation:ERROR] backend_name=self.config.translation.jit_backend, inline=True) [translation:ERROR] File "/home/ltratt/tmp/pypy/pypy/jit/metainterp/warmspot.py", line 43, in apply_jit [translation:ERROR] **kwds) [translation:ERROR] File "/home/ltratt/tmp/pypy/pypy/jit/metainterp/warmspot.py", line 215, in __init__ [translation:ERROR] self.codewriter.make_jitcodes(verbose=verbose) [translation:ERROR] File "/home/ltratt/tmp/pypy/pypy/jit/codewriter/codewriter.py", line 72, in make_jitcodes [translation:ERROR] self.transform_graph_to_jitcode(graph, jitcode, verbose) [translation:ERROR] File "/home/ltratt/tmp/pypy/pypy/jit/codewriter/codewriter.py", line 41, in transform_graph_to_jitcode [translation:ERROR] transform_graph(graph, self.cpu, self.callcontrol, portal_jd) [translation:ERROR] File "/home/ltratt/tmp/pypy/pypy/jit/codewriter/jtransform.py", line 24, in transform_graph [translation:ERROR] t.transform(graph) [translation:ERROR] File "/home/ltratt/tmp/pypy/pypy/jit/codewriter/jtransform.py", line 43, in transform [translation:ERROR] self.optimize_block(block) [translation:ERROR] File "/home/ltratt/tmp/pypy/pypy/jit/codewriter/jtransform.py", line 67, in optimize_block [translation:ERROR] oplist = self.rewrite_operation(op) [translation:ERROR] File "/home/ltratt/tmp/pypy/pypy/jit/codewriter/jtransform.py", line 202, in rewrite_operation [translation:ERROR] return rewrite(self, op) [translation:ERROR] File "/home/ltratt/tmp/pypy/pypy/jit/codewriter/jtransform.py", line 520, in rewrite_op_getarrayitem [translation:ERROR] arraydescr = self.cpu.arraydescrof(ARRAY) [translation:ERROR] File "/home/ltratt/tmp/pypy/pypy/jit/backend/llsupport/llmodel.py", line 236, in arraydescrof [translation:ERROR] return get_array_descr(self.gc_ll_descr, A) [translation:ERROR] File "/home/ltratt/tmp/pypy/pypy/jit/backend/llsupport/descr.py", line 253, in get_array_descr [translation:ERROR] assert isinstance(ARRAY, lltype.GcArray) [translation:ERROR] AssertionError ---------- status: resolved -> unread ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Mon Oct 31 12:08:30 2011 From: tracker at bugs.pypy.org (Armin Rigo) Date: Mon, 31 Oct 2011 11:08:30 +0000 Subject: [pypy-issue] [issue925] RPython arrays sometimes getting the wrong type? In-Reply-To: <1319746776.9.0.524751907394.issue925@bugs.pypy.org> Message-ID: <1320059310.92.0.481292853988.issue925@bugs.pypy.org> Armin Rigo added the comment: If I "git pull" and revert this patch, then it still works for me: I can translate it with "translate.py --continuation -Ojit main". Maybe you didn't update your copy of pypy? ---------- status: unread -> chatting ________________________________________ PyPy bug tracker ________________________________________ From tracker at bugs.pypy.org Mon Oct 31 12:28:10 2011 From: tracker at bugs.pypy.org (Laurence Tratt) Date: Mon, 31 Oct 2011 11:28:10 +0000 Subject: [pypy-issue] [issue925] RPython arrays sometimes getting the wrong type? In-Reply-To: <1319746776.9.0.524751907394.issue925@bugs.pypy.org> Message-ID: <1320060490.33.0.349539826007.issue925@bugs.pypy.org> Laurence Tratt added the comment: Mea culpa! I apologise for the amateur error - apparently I forgot to "hg update" (I have still yet to work out why sometimes I need to do that and sometimes not). I'm really sorry about that. Thanks for fixing this so quickly. I've now been able to revert the patch: https://github.com/ltratt/converge/commit/5c0d819edb7db2423fe5dcb2773a60677dfe87d0 ---------- status: chatting -> unread ________________________________________ PyPy bug tracker ________________________________________