From issues-reply at bitbucket.org Mon Sep 2 14:01:00 2019 From: issues-reply at bitbucket.org (Fabio Zadrozny) Date: Mon, 02 Sep 2019 18:01:00 +0000 (UTC) Subject: [pypy-issue] Issue #3067: Error getting locals for frame from a different thread (pypy/pypy) Message-ID: <20190902180100.24544.41782@celery-worker-111.ash1.bb-inf.net> New issue 3067: Error getting locals for frame from a different thread https://bitbucket.org/pypy/pypy/issues/3067/error-getting-locals-for-frame-from-a Fabio Zadrozny: Pypy info: Python 3.6.1 \(784b254d669919c872a505b807db8462b6140973, May 08 2019, 09:29:32\) \[PyPy 7.1.1-beta0 with GCC 7.3.0\] on linux I?m attaching a test-case showcasing the issue. As a note, I found this error because of a bug-report in the pydevd debugger: [https://github.com/microsoft/ptvsd/issues/1730](https://github.com/microsoft/ptvsd/issues/1730) From issues-reply at bitbucket.org Tue Sep 3 08:46:35 2019 From: issues-reply at bitbucket.org (Fabio Zadrozny) Date: Tue, 03 Sep 2019 12:46:35 +0000 (UTC) Subject: [pypy-issue] Issue #3068: Error setting frame tracing from a different thread (pypy/pypy) Message-ID: <20190903124635.31630.34168@celery-worker-110.ash1.bb-inf.net> New issue 3068: Error setting frame tracing from a different thread https://bitbucket.org/pypy/pypy/issues/3068/error-setting-frame-tracing-from-a Fabio Zadrozny: I?m not sure if this is the same as: [https://bitbucket.org/pypy/pypy/issues/3067/error-getting-locals-for-frame-from-a](https://bitbucket.org/pypy/pypy/issues/3067/error-getting-locals-for-frame-from-a) \(internally it may be, but the way to reproduce is very different and this is much more critical to the debugger, so, I?m doing a new report\). Just to give some background, without being able to set the frame tracing from a different thread, the `pydevd` debugger is not able to change breakpoints \(in `pydevd` the frame is checked to see if it has a breakpoint and if it has it's traced, if it doesn't have a breakpoint it runs untraced -- now, if a user adds a breakpoint later on, `pydevd` will set `frame.f_trace` for the frames that were previously untraced so that the breakpoints are re-evaluated, but this is not working on `pypy` due to this bug\). I guess the debugger could workaround by disabling this optimization, but this is such a huge performance boost in most real-world cases when debugging \(where there are breakpoints only in a few places\) that it'd be really nice if `PyPy` could properly support that. I'm attaching a test-case which shows the issue \(this works in `CPython` but doesn't work in `PyPy`\). From issues-reply at bitbucket.org Thu Sep 5 13:39:15 2019 From: issues-reply at bitbucket.org (Anthony Sottile) Date: Thu, 05 Sep 2019 17:39:15 +0000 (UTC) Subject: [pypy-issue] Issue #3069: pypy3.6 on windows: incorrect line separator (pypy/pypy) Message-ID: <20190905173915.5882.81144@celery-worker-111.ash1.bb-inf.net> New issue 3069: pypy3.6 on windows: incorrect line separator https://bitbucket.org/pypy/pypy/issues/3069/pypy36-on-windows-incorrect-line-separator Anthony Sottile: ``` $ echo 'print("hello world") > t.py $ ./pypy3.exe -c 'import sys, subprocess; print(subprocess.check_output((sys.executable, "t.py")))' b'hello world\n' $ C:/python36-32/python.exe -c 'import sys, subprocess; print(subprocess.check_output((sys.executable, "t.py")))' b'hello world\r\n' ``` ? ``` $ ./pypy3 --version Python 3.6.1 (784b254d6699, Apr 16 2019, 12:10:48) [PyPy 7.1.1-beta0 with MSC v.1910 32 bit] ``` From issues-reply at bitbucket.org Fri Sep 6 07:32:56 2019 From: issues-reply at bitbucket.org (Ronan Lamy) Date: Fri, 06 Sep 2019 11:32:56 +0000 (UTC) Subject: [pypy-issue] Issue #3070: fake_pytest.raises() should support the with: syntax (pypy/pypy) Message-ID: <20190906113256.352.80003@celery-worker-110.ash1.bb-inf.net> New issue 3070: fake_pytest.raises() should support the with: syntax https://bitbucket.org/pypy/pypy/issues/3070/fake_pytestraises-should-support-the-with Ronan Lamy: The recommended syntax for pytest.raises\(\) nowadays is ```python with raises(Exception) as excinfo: ... ``` It should be available in -D and -A tests. From issues-reply at bitbucket.org Sat Sep 7 16:55:47 2019 From: issues-reply at bitbucket.org (AliReza Mosajjal) Date: Sat, 07 Sep 2019 20:55:47 +0000 (UTC) Subject: [pypy-issue] Issue #3071: pypy3 slows down simple reading from stdin (pypy/pypy) Message-ID: <20190907205547.21894.99209@app-137.ash1.bb-inf.net> New issue 3071: pypy3 slows down simple reading from stdin https://bitbucket.org/pypy/pypy/issues/3071/pypy3-slows-down-simple-reading-from-stdin AliReza Mosajjal: Hi, ? I?m writing a wrapper for a C library which has a very fast stdin/stdout line per second. Python 3.7.4 is a good option but I wanted to test my code against pypy3 to see the performance optimizations and I was caught by surprise. pypy2 is outperforming Python \(as expected\) but pypy3 is surprisingly slow. Here?s the simplest code snippet to test: ```python import sys if __name__ == "__main__": for line in sys.stdin: sys.stdout.write(line) ``` I ran this code against 4 different python versions \(python 2.7.15, python 3.7.4, pypy2 7.1.1, pypy3.6 7.1.1\). Here?s the results: ``` $ timeout 10 cat /dev/urandom | base64 | pypy stdin.py > pypy2.list $ timeout 10 cat /dev/urandom | base64 | pypy3 stdin.py > pypy2.list $ timeout 10 cat /dev/urandom | base64 | python stdin.py > python2.list $ timeout 10 cat /dev/urandom | base64 | python3 stdin.py > python3.list $ wc - l *.list 26053216 pypy2.list 6653260 pypy3.list 21715329 python2.list 21327765 python3.list ``` The machine is an Arch Linux with this pypy3: ``` Python 3.6.1 (784b254d669919c872a505b807db8462b6140973, May 09 2019, 13:17:30) [PyPy 7.1.1-beta0 with GCC 8.3.0] on linux ``` so why do you think pypy3 is performing so bad compared to others? ? Cheers, From issues-reply at bitbucket.org Wed Sep 11 10:04:00 2019 From: issues-reply at bitbucket.org (Kirill Smelkov) Date: Wed, 11 Sep 2019 14:04:00 +0000 (UTC) Subject: [pypy-issue] Issue #3072: Race in PyThread_release_lock - can lead to memory corruption and deadlock (pypy/pypy) Message-ID: <20190911140358.29707.37847@celery-worker-112.ash1.bb-inf.net> New issue 3072: Race in PyThread_release_lock - can lead to memory corruption and deadlock https://bitbucket.org/pypy/pypy/issues/3072/race-in-pythread_release_lock-can-lead-to Kirill Smelkov: Hello up there. I recently found concurency bug that applies to CPython2.7, PyPy2 and PyPy3. Basically PyThread\_release\_lock, on systems where it is implemented via mutex \+ condition variable, has a race condition since pthread\_cond\_signal is called outside of mutex-protected region. This race condition can lead to memory corruption and deadlock. I?ve hit this bug for real while testing Pygolang on macOS. Original bug report is here: [https://bugs.python.org/issue38106](https://bugs.python.org/issue38106). Copy of original bug report follows: ----8<---- ### Bug history \( Please see attached pylock\_bug.pyx for the program that triggers the bug for real. \) On Darwin, even though this is considered as POSIX, Python uses mutex\+condition variable to implement its lock, and, as of 20190828, Py2.7 implementation, even though similar issue was fixed for Py3 in 2012, contains synchronization bug: the condition is signalled after mutex unlock while the correct protocol is to signal condition from under mutex: [https://github.com/python/cpython/blob/v2.7.16-127-g0229b56d8c0/Python/thread\_pthread.h#L486-L506](https://github.com/python/cpython/blob/v2.7.16-127-g0229b56d8c0/Python/thread_pthread.h#L486-L506) [https://github.com/python/cpython/commit/187aa545165d](https://github.com/python/cpython/commit/187aa545165d) \(py3 fix\) PyPy has the same bug for both pypy2 and pypy3: [https://bitbucket.org/pypy/pypy/src/578667b3fef9/rpython/translator/c/src/thread\_pthread.c#lines-443:465](https://bitbucket.org/pypy/pypy/src/578667b3fef9/rpython/translator/c/src/thread_pthread.c#lines-443:465) [https://bitbucket.org/pypy/pypy/src/5b42890d48c3/rpython/translator/c/src/thread\_pthread.c#lines-443:465](https://bitbucket.org/pypy/pypy/src/5b42890d48c3/rpython/translator/c/src/thread_pthread.c#lines-443:465) Signalling condition outside of corresponding mutex is considered OK by POSIX, but in Python context it can lead to at least memory corruption if we consider the whole lifetime of python level lock. For example the following logical scenario: ``` T1 T2 sema = Lock() sema.acquire() sema.release() sema.acquire() free(sema) ... ``` can translate to the next C-level calls: ``` T1 T2 # sema = Lock() sema = malloc(...) sema.locked = 0 pthread_mutex_init(&sema.mut) pthread_cond_init (&sema.lock_released) # sema.acquire() pthread_mutex_lock(&sema.mut) # sees sema.locked == 0 sema.locked = 1 pthread_mutex_unlock(&sema.mut) # sema.release() pthread_mutex_lock(&sema.mut) sema.locked = 0 pthread_mutex_unlock(&sema.mut) # OS scheduler gets in and relinquishes control from T2 # to another process ... # second sema.acquire() pthread_mutex_lock(&sema.mut) # sees sema.locked == 0 sema.locked = 1 pthread_mutex_unlock(&sema.mut) # free(sema) pthread_mutex_destroy(&sema.mut) pthread_cond_destroy (&sema.lock_released) free(sema) # ... e.g. malloc() which returns memory where sema was ... # OS scheduler returns control to T2 # sema.release() continues # # BUT sema was already freed and writing to anywhere # inside sema block CORRUPTS MEMORY. In particular if # _another_ python-level lock was allocated where sema # block was, writing into the memory can have effect on # further synchronization correctness and in particular # lead to deadlock on lock that was next allocated. pthread_cond_signal(&sema.lock_released) ``` Note that T2.pthread\_cond\_signal\(&sema.lock\_released\) CORRUPTS MEMORY as it is called when sema memory was already freed and is potentially reallocated for another object. The fix is to move pthread\_cond\_signal to be done under corresponding mutex: ``` # sema.release() pthread_mutex_lock(&sema.mut) sema.locked = 0 pthread_cond_signal(&sema.lock_released) pthread_mutex_unlock(&sema.mut) ``` by cherry-picking commit 187aa545165d \("Signal condition variables with the mutex held. Destroy condition variables before their mutexes"\). ### Bug history The bug was there since 1994 - since at least \[1\]. It was discussed in 2001 with original code author\[2\], but the code was still considered to be race-free. In 2010 the place where pthread\_cond\_signal should be - before or after pthread\_mutex\_unlock - was discussed with the rationale to avoid threads bouncing\[3,4,5\], and in 2012 pthread\_cond\_signal was moved to be called from under mutex, but only for CPython3\[6,7\]. In 2019 the bug was \(re-\)discovered while testing Pygolang\[8\] on macOS with CPython2 and PyPy2 and PyPy3. \[1\] [https://github.com/python/cpython/commit/2c8cb9f3d240](https://github.com/python/cpython/commit/2c8cb9f3d240) \[2\] [https://bugs.python.org/issue433625](https://bugs.python.org/issue433625) \[3\] [https://bugs.python.org/issue8299#msg103224](https://bugs.python.org/issue8299#msg103224) \[4\] [https://bugs.python.org/issue8410#msg103313](https://bugs.python.org/issue8410#msg103313) \[5\] [https://bugs.python.org/issue8411#msg113301](https://bugs.python.org/issue8411#msg113301) \[6\] [https://bugs.python.org/issue15038#msg163187](https://bugs.python.org/issue15038#msg163187) \[7\] [https://github.com/python/cpython/commit/187aa545165d](https://github.com/python/cpython/commit/187aa545165d) \[8\] [https://pypi.org/project/pygolang](https://pypi.org/project/pygolang) From issues-reply at bitbucket.org Wed Sep 11 15:47:15 2019 From: issues-reply at bitbucket.org (Ronan Lamy) Date: Wed, 11 Sep 2019 19:47:15 +0000 (UTC) Subject: [pypy-issue] Issue #3073: Missing stat.S_IFDOOR when running from a virtualenv (pypy/pypy) Message-ID: <20190911194715.25109.95705@app-147.ash1.bb-inf.net> New issue 3073: Missing stat.S_IFDOOR when running from a virtualenv https://bitbucket.org/pypy/pypy/issues/3073/missing-stats_ifdoor-when-running-from-a Ronan Lamy: S\_IFDOOR is defined in \_stat \(in lib-pypy\) but is supposed to be exposed in \(lib-python\) stat. Outside a virtualenv, stat imports S\_IFDOOR from \_stat. With a nightly in a virtualenv, however, `from _stat import *` seems to fail in `stat`, though it's still possible to import `_stat` afterwards. From issues-reply at bitbucket.org Sun Sep 15 08:58:29 2019 From: issues-reply at bitbucket.org (Ned Batchelder) Date: Sun, 15 Sep 2019 12:58:29 +0000 (UTC) Subject: [pypy-issue] Issue #3074: compile() fails on non-ascii filename if LANG=C (pypy/pypy) Message-ID: <20190915125829.5540.80145@app-147.ash1.bb-inf.net> New issue 3074: compile() fails on non-ascii filename if LANG=C https://bitbucket.org/pypy/pypy/issues/3074/compile-fails-on-non-ascii-filename-if Ned Batchelder: Setting LANG=C sets getfilesystemencoding to ?ascii?. In CPython, and PyPy before 7.1.1, compile\(\) could still compile a non-ascii filename. Now in PyPy3 7.1.1, compile\(\) fails in this situation. Reproduction: ``` $ cat weird_file.py import sys print("\n--- {} --------------".format(sys.executable)) print(sys.version) print("sys.getfilesystemencoding(): {!r}".format(sys.getfilesystemencoding())) globs = {} code = "a = 1\n" exec(compile(code, "wut\xe9\xea\xeb\xec\x01\x02.py", 'exec'), globs) print(globs['a']) $ for py in /usr/local/bin/{python,pypy}*; do LANG=C $py weird_file.py; done --- /usr/local/bin/python2.7 -------------- 2.7.14 (default, Oct 4 2017, 09:45:53) [GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.37)] sys.getfilesystemencoding(): 'utf-8' 1 --- /usr/local/bin/python3.1 -------------- 3.1.5 (default, Jun 18 2016, 17:54:07) [GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)] sys.getfilesystemencoding(): 'utf-8' 1 --- /usr/local/bin/python3.2 -------------- 3.2.6 (default, Jun 18 2016, 17:56:34) [GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)] sys.getfilesystemencoding(): 'utf-8' 1 --- /usr/local/bin/python3.3 -------------- 3.3.6 (default, Jun 18 2016, 17:58:30) [GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)] sys.getfilesystemencoding(): 'utf-8' 1 --- /usr/local/bin/python3.4 -------------- 3.4.8 (default, Feb 19 2018, 09:34:36) [GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)] sys.getfilesystemencoding(): 'utf-8' 1 --- /usr/local/bin/python3.5 -------------- 3.5.5 (default, Feb 19 2018, 09:28:14) [GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)] sys.getfilesystemencoding(): 'utf-8' 1 --- /usr/local/bin/python3.6 -------------- 3.6.9 (default, Jul 29 2019, 08:21:55) [GCC 4.2.1 Compatible Apple LLVM 10.0.0 (clang-1000.10.44.4)] sys.getfilesystemencoding(): 'utf-8' 1 --- /usr/local/bin/python3.7 -------------- 3.7.4 (default, Jul 11 2019, 19:53:42) [Clang 10.0.0 (clang-1000.10.44.4)] sys.getfilesystemencoding(): 'utf-8' 1 --- /usr/local/bin/python3.8 -------------- 3.8.0b4 (default, Aug 30 2019, 06:51:19) [Clang 10.0.0 (clang-1000.10.44.4)] sys.getfilesystemencoding(): 'utf-8' 1 --- /usr/local/bin/pypy6.0 -------------- 2.7.13 (ab0b9caf307d, Apr 24 2018, 18:05:02) [PyPy 6.0.0 with GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.1)] sys.getfilesystemencoding(): 'US-ASCII' 1 --- /usr/local/bin/pypy7.0 -------------- 2.7.13 (9112c8071614, Feb 05 2019, 13:50:20) [PyPy 7.0.0 with GCC 4.2.1 Compatible Apple LLVM 10.0.0 (clang-1000.11.45.5)] sys.getfilesystemencoding(): 'US-ASCII' 1 --- /usr/local/bin/pypy7.1 -------------- 2.7.13 (8cdda8b8cdb8, Apr 14 2019, 14:06:58) [PyPy 7.1.1 with GCC 4.2.1 Compatible Apple LLVM 10.0.0 (clang-1000.11.45.5)] sys.getfilesystemencoding(): 'US-ASCII' 1 --- /usr/local/bin/pypy3-6.0 -------------- 3.5.3 (fdd60ed87e94, Apr 24 2018, 06:10:18) [PyPy 6.0.0 with GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.1)] sys.getfilesystemencoding(): 'ascii' 1 --- /usr/local/bin/pypy3-7.0 -------------- 3.5.3 (928a4f70d3de, Feb 08 2019, 10:43:14) [PyPy 7.0.0 with GCC 4.2.1 Compatible Apple LLVM 10.0.0 (clang-1000.11.45.5)] sys.getfilesystemencoding(): 'ascii' 1 --- /usr/local/bin/pypy3-7.1 -------------- 3.6.1 (784b254d6699, Apr 14 2019, 10:22:55) [PyPy 7.1.1-beta0 with GCC 4.2.1 Compatible Apple LLVM 10.0.0 (clang-1000.11.45.5)] sys.getfilesystemencoding(): 'ascii' Traceback (most recent call last): File "weird_file.py", line 7, in exec(compile(code, "wut\xe9\xea\xeb\xec\x01\x02.py", 'exec'), globs) UnicodeEncodeError: 'ascii' codec can't encode characters in position 3-6: ordinal not in range(128) $ ``` ? From issues-reply at bitbucket.org Mon Sep 16 23:08:20 2019 From: issues-reply at bitbucket.org (=?utf-8?q?Alexander_B=C3=B6hn?=) Date: Tue, 17 Sep 2019 03:08:20 +0000 (UTC) Subject: [pypy-issue] =?utf-8?q?Issue_=233075=3A_Class_keywords_are_impro?= =?utf-8?q?perly_passed_in_PyPy3_when_using_ancestors_from_the_=E2=80=9Cab?= =?utf-8?b?Y+KAnSBtb2R1bGUgKHB5cHkvcHlweSk=?= Message-ID: <20190917030820.12003.17916@celery-worker-109.ash1.bb-inf.net> New issue 3075: Class keywords are improperly passed in PyPy3 when using ancestors from the ?abc? module https://bitbucket.org/pypy/pypy/issues/3075/class-keywords-are-improperly-passed-in Alexander B?hn: I produced this fairly minimal example: ? ```python # -*- encoding: utf-8 -*- from __future__ import print_function import abc class Metabase(abc.ABCMeta): def __new__(metacls, name, bases, attributes, **kwargs): # Call up: return super(Metabase, metacls).__new__(metacls, name, bases, attributes, **kwargs) class Metaclass(Metabase): def __new__(metacls, name, bases, attributes, metakeyword="meta", **kwargs): # Stow metaclass keyword in attributes: if 'metakeyword' not in attributes: attributes['metakeyword'] = metakeyword # Remove metaclass? keyword and call up: return super(Metaclass, metacls).__new__(metacls, name, bases, attributes, **kwargs) class Mixin(abc.ABC, metaclass=Metabase): @classmethod def __init_subclass__(cls, mixinkeyword="mixin", **kwargs): # Remove mixins? keyword and call up: super(Mixin, cls).__init_subclass__(**kwargs) # Stow mixin keyword on subclass: cls.mixinkeyword = mixinkeyword class Base(Mixin, metaclass=Metaclass): pass class Derived(Base, mixinkeyword="derived-mixin", metakeyword="derived-meta"): pass def test(): d = Derived() print(d) print(d.mixinkeyword) print(d.metakeyword) if __name__ == '__main__': test() ``` ? ? if this is run with `python3 class-kwargs.py` :grinning: ? if this is run with `pypy3 class-kwargs.py` :sob: ? ```python Traceback (most recent call last): File ".script-bin/class-kwargs.py", line 43, in metakeyword="derived-meta"): File ".script-bin/class-kwargs.py", line 27, in __new__ **kwargs) File ".script-bin/class-kwargs.py", line 14, in __new__ **kwargs) TypeError: __new__() got an unexpected keyword argument 'mixinkeyword' ``` ? ? the output of `python3 --version` is `Python 3.7.4` ? the output of `pypy3 --version` is: ```plaintext Python 3.6.1 (784b254d669919c872a505b807db8462b6140973, Sep 04 2019, 12:08:51) [PyPy 7.1.1-beta0 with GCC 4.2.1 Compatible Apple LLVM 10.0.1 (clang-1001.0.46.4)] ``` There were, as far as I have been able to see looking through the docs, no major changes made to the way `__init_subclass__(?)` or metaclass keyword arguments worked between Python 3.6.1 and Python 3.7. The `class-kwargs.py` script is attached here. From issues-reply at bitbucket.org Thu Sep 19 05:46:57 2019 From: issues-reply at bitbucket.org (Ned Batchelder) Date: Thu, 19 Sep 2019 09:46:57 +0000 (UTC) Subject: [pypy-issue] Issue #3076: First line traced in a nested structure depends on the length of the structure (pypy/pypy) Message-ID: <20190919094656.7498.85502@celery-worker-110.ash1.bb-inf.net> New issue 3076: First line traced in a nested structure depends on the length of the structure https://bitbucket.org/pypy/pypy/issues/3076/first-line-traced-in-a-nested-structure Ned Batchelder: When creating a nested list, the settrace trace function is called with a ?call? event. The list looks like: ``` data = [ [ 0 ], [ 1 ], [ 2 ], [ 3 ], [ 4 ], [ 5 ], [ 6 ], [ 7 ], [ 8 ], [ 9 ], ] ``` With PyPy 3.x, the call event is on line 1. With PyPy 2.x, the location of the call event depends on the length of the list. If the list is short \(10 elements\), the call event is on line 2, or if the list were more deeply nested, it would be whatever line had the first integer. If the list is long \(2000 elements\), the call event is on line 1. It seems wrong that the length of the list would change the call event. Code to demonstrate the problem: ``` $ cat make_file.py import sys num = int(sys.argv[1]) print("data = [") for i in range(num): print(" [ {} ],".format(i)) print("]") print("print(len(data))") $ python make_file.py 10 > short.py $ python make_file.py 2000 > long.py $ cat run.py import os import sys print(sys.version) blacklist = ["frozen", "_structseq", "utf_8"] def trace(frame, event, arg): if event == 'call': fname = os.path.basename(frame.f_code.co_filename) noise = any(bword in fname for bword in blacklist) if not noise: print("Call {}:{}".format(fname, frame.f_lineno)) return trace sys.settrace(trace) print("short:") import short print("long:") import long $ pypy2 run.py 2.7.13 (8cdda8b8cdb8, Apr 14 2019, 14:06:58) [PyPy 7.1.1 with GCC 4.2.1 Compatible Apple LLVM 10.0.0 (clang-1000.11.45.5)] short: Call short.py:2 10 long: Call long.py:1 2000 Call app_main.py:96 $ pypy3 run.py 3.6.1 (784b254d6699, Apr 14 2019, 10:22:55) [PyPy 7.1.1-beta0 with GCC 4.2.1 Compatible Apple LLVM 10.0.0 (clang-1000.11.45.5)] short: Call short.py:1 10 long: Call long.py:1 2000 Call app_main.py:100 $ cat short.py data = [ [ 0 ], [ 1 ], [ 2 ], [ 3 ], [ 4 ], [ 5 ], [ 6 ], [ 7 ], [ 8 ], [ 9 ], ] print(len(data)) $ ``` ? From issues-reply at bitbucket.org Thu Sep 19 06:09:11 2019 From: issues-reply at bitbucket.org (Carl Friedrich Bolz-Tereick) Date: Thu, 19 Sep 2019 10:09:11 +0000 (UTC) Subject: [pypy-issue] Issue #3077: Document difference with CPython in shutdown (pypy/pypy) Message-ID: <20190919100911.865.28212@app-147.ash1.bb-inf.net> New issue 3077: Document difference with CPython in shutdown https://bitbucket.org/pypy/pypy/issues/3077/document-difference-with-cpython-in Carl Friedrich Bolz-Tereick: On shutdown, CPython will carefully do stuff to free memory and call remaining finalizers. PyPy doesn?t, it just flushes buffers and closes files. That?s not documented in `cpython_differences.rst` and it should be. From issues-reply at bitbucket.org Thu Sep 19 17:32:57 2019 From: issues-reply at bitbucket.org (Mukesh D) Date: Thu, 19 Sep 2019 21:32:57 +0000 (UTC) Subject: [pypy-issue] Issue #3078: Memory Error/ Compilation Error/ Multiprocessing error (pypy/pypy) Message-ID: <20190919213257.10620.22718@celery-worker-108.ash1.bb-inf.net> New issue 3078: Memory Error/ Compilation Error/ Multiprocessing error https://bitbucket.org/pypy/pypy/issues/3078/memory-error-compilation-error Mukesh D: P**lease help me solve any of the following cases: Any help would be greatly appreciated.** I?m trying to install pypy3 on Windows\(Preferred\) and Linux. However, getting error in both the cases **Case 1: Windows 64 bit , Python 3.6 , pypy3.5-v7.0.0-win32** ? ```python pypy3 C:\LAA\8.12\LAATranslator.py ``` > _**Error : Traceback \(most recent call last\):**_ > > File "C:\\LAF\\8.12\\[interface.py](http://interface.py)", line 9, in **init** > self.\_\_child, self.\_\_parent = Pipe\(duplex=duplex\) > File "C:\\Users\\\\Downloads\\pypy3.5-v7.0.0-win32\\pypy3.5-v7.0.0-win32\\lib-python\\3\\multiprocessing\\[context.py](http://context.py)", line 61, in Pipe > return Pipe\(duplex\) > File "C:\\Users\\\\Downloads\\pypy3.5-v7.0.0-win32\\pypy3.5-v7.0.0-win32\\lib-python\\3\\multiprocessing\\[connection.py](http://connection.py)", line 552, in Pipe > \_, err = overlapped.GetOverlappedResult\(True\) > File "C:\\Users\\\\Downloads\\pypy3.5-v7.0.0-win32\\pypy3.5-v7.0.0-win32\\lib\_pypy\_winapi.py", line 113, in GetOverlappedResult > if self.completed and self.read\_buffer: > AttributeError: 'Overlapped' object has no attribute 'read\_buffer' ? --- ? **Case 2 Windows 64 bit , Python 3.6 pypy-c-jit-97473-f29a87447d84-win32** Error: > [LogPacketTranslator.py](http://LogPacketTranslator.py)", line 182, in process\_log > parsers = ParserManager\(generator\_interfaces, debug=self.\_\_debug,debug\_internal = self.\_\_debug\_internal, unreliable\_mode= self.\__unreliable\_mode\)_ > _File "C:\\LAF\\8.12\\parsers\_init_.py", line 96, in **init** > self.\_\_asserted = Value\('i', 0\) > File "C:\\Users\\\\Downloads\\pypy-c-jit-97473-f29a87447d84-win32\\pypy-c-jit-97473-f29a87447d84-win32\\lib-python\\3\\multiprocessing\\[context.py](http://context.py)", line 135, in Value > ctx=self.get\_context\(\)\) > File "C:\\Users\\\\Downloads\\pypy-c-jit-97473-f29a87447d84-win32\\pypy-c-jit-97473-f29a87447d84-win32\\lib-python\\3\\multiprocessing\\[sharedctypes.py](http://sharedctypes.py)", line 73, in Value > obj = RawValue\(typecode\_or\_type, \*args\) > File "C:\\Users\\\\Downloads\\pypy-c-jit-97473-f29a87447d84-win32\\pypy-c-jit-97473-f29a87447d84-win32\\lib-python\\3\\multiprocessing\\[sharedctypes.py](http://sharedctypes.py)", line 48, in RawValue > obj = _new\_value\(type_\) > File "C:\\Users\\\\Downloads\\pypy-c-jit-97473-f29a87447d84-win32\\pypy-c-jit-97473-f29a87447d84-win32\\lib-python\\3\\multiprocessing\\[sharedctypes.py](http://sharedctypes.py)", line 40, in \_new\_value > wrapper = heap.BufferWrapper\(size\) > File "C:\\Users\\\\Downloads\\pypy-c-jit-97473-f29a87447d84-win32\\pypy-c-jit-97473-f29a87447d84-win32\\lib-python\\3\\multiprocessing\\[heap.py](http://heap.py)", line 248, in **init** > block = BufferWrapper.\_heap.malloc\(size\) > File "C:\\Users\\\\Downloads\\pypy-c-jit-97473-f29a87447d84-win32\\pypy-c-jit-97473-f29a87447d84-win32\\lib-python\\3\\multiprocessing\\[heap.py](http://heap.py)", line 230, in malloc > \(arena, start, stop\) = self.\_malloc\(size\) > File "C:\\Users\\\\Downloads\\pypy-c-jit-97473-f29a87447d84-win32\\pypy-c-jit-97473-f29a87447d84-win32\\lib-python\\3\\multiprocessing\\[heap.py](http://heap.py)", line 128, in \_malloc > arena = Arena\(length\) > File "C:\\Users\\\\Downloads\\pypy-c-jit-97473-f29a87447d84-win32\\pypy-c-jit-97473-f29a87447d84-win32\\lib-python\\3\\multiprocessing\\[heap.py](http://heap.py)", line 44, in **init** > raise FileExistsError\('Cannot find name for new mmap'\) > FileExistsError: Cannot find name for new mmap ? --- **Case 3 CentOS/Ubuntu - Can not compile the source** Errors on both Pypy 3.6 beta and nightly build then i tried to build from pypy source however getting following error in command which pypy /usr/bin/pypy ``` ? ``` ``` 1:~/pypy/pypy/pypy/goal$ pypy ../../rpython/bin/rpython -Ojit targetpypystandalone [translation:info] 2.7.13 (5.10.0+dfsg-3build2, Feb 06 2018, 18:37:50) [PyPy 5.10.0 with GCC 7.3.0] [platform:msg] Set platform with 'host' cc=None, using cc='gcc', version='Unknown' [translation:info] Translating target as defined by targetpypystandalone ``` > ? > > \[Timer\] Timings: > \[Timer\] annotate --- 376.5 s > \[Timer\] rtype\_lltype --- 568.4 s > \[Timer\] pyjitpl\_lltype --- 340.1 s > \[Timer\] =========================================== > \[Timer\] Total: --- 1285.1 s > \[translation:info\] Error: > File "/home/ubuntu/pypy/pypy/rpython/translator/goal/translate.py", line 318, in main > drv.proceed\(goals\) > File "/home/ubuntu/pypy/pypy/rpython/translator/driver.py", line 554, in proceed > result = self.\_execute\(goals, task\_skip = self.\_maybe\_skip\(\)\) > File "/home/ubuntu/pypy/pypy/rpython/translator/tool/taskengine.py", line 114, in \_execute > res = self.\_do\(goal, taskcallable, \*args, \*\*kwds\) > File "/home/ubuntu/pypy/pypy/rpython/translator/driver.py", line 278, in \_do > res = func\(\) > File "/home/ubuntu/pypy/pypy/rpython/translator/driver.py", line 361, in task\_pyjitpl\_lltype > backend\_name=self.config.translation.jit\_backend, inline=True\) > File "/home/ubuntu/pypy/pypy/rpython/jit/metainterp/warmspot.py", line 72, in apply\_jit > warmrunnerdesc.finish\(\) > File "/home/ubuntu/pypy/pypy/rpython/jit/metainterp/warmspot.py", line 297, in finish > self.annhelper.finish\(\) > File "/home/ubuntu/pypy/pypy/rpython/rtyper/annlowlevel.py", line 218, in finish > self.finish\_annotate\(\) > File "/home/ubuntu/pypy/pypy/rpython/rtyper/annlowlevel.py", line 238, in finish\_annotate > ann.complete\_helpers\(\) > File "/home/ubuntu/pypy/pypy/rpython/annotator/annrpython.py", line 118, in complete\_helpers > self.simplify\(block\_subset=self.added\_blocks\) > File "/home/ubuntu/pypy/pypy/rpython/annotator/annrpython.py", line 360, in simplify > extra\_passes=extra\_passes\) > File "/home/ubuntu/pypy/pypy/rpython/translator/transform.py", line 269, in transform\_graph > transform\_dead\_op\_vars\(ann, block\_subset\) > File "/home/ubuntu/pypy/pypy/rpython/translator/transform.py", line 142, in transform\_dead\_op\_vars > transform\_dead\_op\_vars\_in\_blocks\(block\_subset, self.translator.graphs\) > File "/home/ubuntu/pypy/pypy/rpython/translator/simplify.py", line 480, in transform\_dead\_op\_vars\_in\_blocks > flow\_read\_var\_backward\(set\(read\_vars\)\) > File "/home/ubuntu/pypy/pypy/rpython/translator/simplify.py", line 475, in flow\_read\_var\_backward > for prevvar in dependencies\[var\]: > **\[translation:ERROR\] MemoryError** > **\[translation\] start debugger...** > /home/ubuntu/pypy/pypy/rpytho > ? > n/translator/simplify.py\(475\)flow\_read\_var\_backward\(\) > -> for prevvar in dependencies\[var\]: ? > **for debug purpose Here \[translation\] \[objspace\]** > \[std\] > intshortcut = True > optimized\_list\_getitem = True > withspecialisedtuple = True > \[usemodules\] > **pypy** = True > \_ast = True > \_cffi\_backend = True > \_codecs = True > \_collections = True > \_continuation = True > \_cppyy = True > \_csv = True > \_jitlog = True > \_lsprof = True > \_md5 = True > \_minimal\_curses = True > \_multibytecodec = True > \_multiprocessing = True > \_pickle\_support = True > \_posixsubprocess = True > \_pypyjson = True > \_random = True > \_rawffi = True > \_socket = True > \_sre = True > \_string = True > \_testing = True > \_vmprof = True > \_weakref = True > array = True > atexit = True > binascii = True > bz2 = True > cmath = True > cpyext = True > errno = True > faulthandler = True > fcntl = True > gc = True > imp = True > marshal = True > math = True > mmap = True > parser = True > pwd = True > pyexpat = True > pypyjit = True > select = True > signal = True > struct = True > symbol = True > termios = True > thread = True > time = True > token = True > unicodedata = True > zipimport = True > zlib = True > \[translation\] [translate.py](http://translate.py) configuration: > \[translation\] \[translate\] > opt = jit > targetspec = targetpypystandalone > \[translation\] translation configuration: > \[translation\] \[translation\] > check\_str\_without\_nul = True > continuation = True > gc = incminimark > gcrootfinder = shadowstack > gctransformer = framework > icon = pypy.ico > jit = True > list\_comprehension\_operations = True > secondaryentrypoints = cpyext,main > shared = True > thread = True > type\_system = lltype > withsmallfuncsets = 5 > \[translation:info\] Annotating&simplifying... > > ? ? From issues-reply at bitbucket.org Sat Sep 21 19:15:53 2019 From: issues-reply at bitbucket.org (Ondrej B) Date: Sat, 21 Sep 2019 23:15:53 +0000 (UTC) Subject: [pypy-issue] Issue #3079: Intermittent SystemError in time.strptime (pypy/pypy) Message-ID: <20190921231553.3354.91183@celery-worker-111.ash1.bb-inf.net> New issue 3079: Intermittent SystemError in time.strptime https://bitbucket.org/pypy/pypy/issues/3079/intermittent-systemerror-in-timestrptime Ondrej B: I?m trying to build Pillow on PyPy3.6 using latest nightly `pypy-c-jit-97573-3035e3be6a6d-win32`. I?m running into a SystemError: ``` E SystemError: unexpected internal exception (please report a bug): ; internal traceback was dumped to stderr c:\pypy-c-jit-97573-3035e3be6a6d-win32\lib-python\3\_strptime.py:95: SystemError ---------------------------- Captured stderr call ----------------------------- RPython traceback: File "pypy_interpreter.c", line 46473, in BuiltinCode1_fastcall_1 File "implement_2.c", line 2303, in fastfunc_descr_lower_1_1 File "pypy_objspace_std_7.c", line 38028, in _lower_unicode File "rpython_rlib.c", line 17550, in _nonascii_unichr_as_utf8_append ``` This issue happens every time when running all Pillow tests, but running the failing test individually doesn?t reproduce this issue. Full log: [https://ci.appveyor.com/project/nulano/pillow/builds/27570685/job/3a9a5k889njigrkb#L8823](https://ci.appveyor.com/project/nulano/pillow/builds/27570685/job/3a9a5k889njigrkb#L8823) From issues-reply at bitbucket.org Sun Sep 22 19:05:21 2019 From: issues-reply at bitbucket.org (Mukesh D) Date: Sun, 22 Sep 2019 23:05:21 +0000 (UTC) Subject: [pypy-issue] Issue #3080: SQL Connection Error- cannot commit transaction - SQL statements in progress (pypy/pypy) Message-ID: <20190922230521.13511.34367@celery-worker-108.ash1.bb-inf.net> New issue 3080: SQL Connection Error- cannot commit transaction - SQL statements in progress https://bitbucket.org/pypy/pypy/issues/3080/sql-connection-error-cannot-commit Mukesh D: P**lease help me solve the following cases: Any help would be greatly appreciated.** I?m trying to install pypy3 on Windows and Linux. However, getting SQL error . In normal python run everything runs fine. I?m using /pypy-c-jit-97558-b2dd9b388196-linux64 nightly build since i was getting more time date errors in official release ? ``` UIM;UIM_LS]SQL Error: cannot commit transaction - SQL statements in progress Process Process-1: Traceback (most recent call last): File "/opt/pypy_3.5_src_build/lib-python/3/multiprocessing/process.py", line 249, in _bootstrap self.run() File "/opt/pypy_3.5_src_build/lib-python/3/multiprocessing/process.py", line 93, in run self._target(*self._args, **self._kwargs) File "/home/ubuntu/8.12/log/logging.py", line 46, in sqlite_worker stream.create_index() File "/home/ubuntu/8.12/log/handler.py", line 148, in create_index '''CREATE INDEX index_name ON sql_table(ID)''' File "/opt/pypy_3.5_src_build/lib_pypy/_sqlite3.py", line 770, in wrapper return func(self, *args, **kwargs) File "/opt/pypy_3.5_src_build/lib_pypy/_sqlite3.py", line 949, in execute return self.__execute(False, sql, [params]) File "/opt/pypy_3.5_src_build/lib_pypy/_sqlite3.py", line 890, in __execute self.__connection.commit() File "/opt/pypy_3.5_src_build/lib_pypy/_sqlite3.py", line 475, in commit raise self._get_exception(ret) _sqlite3.OperationalError: cannot commit transaction - SQL statements in progress ``` ? From issues-reply at bitbucket.org Wed Sep 25 02:56:01 2019 From: issues-reply at bitbucket.org (Ondrej B) Date: Wed, 25 Sep 2019 06:56:01 +0000 (UTC) Subject: [pypy-issue] Issue #3081: OperationError: cp65001 is only available on Windows (pypy/pypy) Message-ID: <20190925065601.4825.39071@celery-worker-108.ash1.bb-inf.net> New issue 3081: OperationError: cp65001 is only available on Windows https://bitbucket.org/pypy/pypy/issues/3081/operationerror-cp65001-is-only-available Ondrej B: Running recent nightly of pypy3.6 \(`pypy-c-jit-97573-3035e3be6a6d-win32`\) on Windows with active codepage 65001 gives the following error: ? ``` C:\pypy-c-jit-97573-3035e3be6a6d-win32>chcp 65001 Active code page: 65001 C:\pypy-c-jit-97573-3035e3be6a6d-win32>pypy3 debug: OperationError: debug: operror-type: LookupError debug: operror-value: cp65001 encoding is only available on Windows C:\pypy-c-jit-97573-3035e3be6a6d-win32> ``` Found with Pillow tests on GitHub Actions: https://github.com/nulano/Pillow/runs/234636672#step:16:1340 The same issue was closed in 2013 as ?will fix for pypy3.3?: https://bitbucket.org/pypy/pypy/issues/1613/utf-8-encoding-problem-with-powershell-on Is there any update on this? From issues-reply at bitbucket.org Mon Sep 30 10:26:48 2019 From: issues-reply at bitbucket.org (Pierre Augier) Date: Mon, 30 Sep 2019 14:26:48 +0000 (UTC) Subject: [pypy-issue] Issue #3082: Migration from Bitbucket to ??? (pypy/pypy) Message-ID: <20190930142647.12785.63975@celery-worker-110.ash1.bb-inf.net> New issue 3082: Migration from Bitbucket to ??? https://bitbucket.org/pypy/pypy/issues/3082/migration-from-bitbucket-to Pierre Augier: As explained here [https://bitbucket.org/blog/sunsetting-mercurial-support-in-bitbucket](https://bitbucket.org/blog/sunsetting-mercurial-support-in-bitbucket), **Mercurial features and repositories will be officially removed from Bitbucket and its API on June 1, 2020.** I guess you thought about what to do for PyPy repository and PyPy development. I am also a Mercurial user and I am facing the same problem for some repositories. Thus, I am very interested to know what you plan to do. In particular, I am looking for a good forge for the Transonic project \([https://transonic.readthedocs.io](https://transonic.readthedocs.io)\), which by the way is compatible with PyPy3.6 \(currently only the nightly builds\) and could help a wider adoption of PyPy3 for projects using the Python scientific stack \(just by accelerating the numerical kernels using Numpy and let PyPy accelerate pure Python code\). I follow the Heptapod project [https://heptapod.net/](https://heptapod.net/) \(a friendly fork of GitLab to bring very good Mercurial support\). It seems to me that it starts to be nearly production ready and that it would nicely fit your workflow with Mercurial. Of course, it would be very convenient if a common instance \(free to use for simple users until a certain point\) could be setup \(something somehow similar to Github, but with Mercurial support\). Even for PyPy, it would be good in terms of visibility. I also heard that your repository would have to be fixed to be able to use Heptapod. What is the current status of these issues for PyPy? Have you already decided what to do? ?