From issues-reply at bitbucket.org Sun Jul 2 08:53:39 2017 From: issues-reply at bitbucket.org (Marian Beermann) Date: Sun, 02 Jul 2017 12:53:39 +0000 (UTC) Subject: [pypy-issue] Issue #2598: pwd.getpwnam throws SystemError with Unicode (pypy/pypy) Message-ID: <20170702125339.7639.42610@celery-worker-107.ash1.bb-inf.net> New issue 2598: pwd.getpwnam throws SystemError with Unicode https://bitbucket.org/pypy/pypy/issues/2598/pwdgetpwnam-throws-systemerror-with Marian Beermann: Reproducer: ``` #!python import pwd pwd.getpwnam('?bel') RPython traceback: File "pypy_interpreter_1.c", line 39108, in execute_frame File "rpython_jit_metainterp.c", line 20915, in ll_portal_runner__Unsigned_Bool_pypy_interpreter File "pypy_module_pypyjit.c", line 182, in portal_12 File "pypy_interpreter_1.c", line 37644, in handle__AccessDirect_None_2 File "pypy_interpreter.c", line 25194, in OperationError_normalize_exception File "pypy_interpreter.c", line 55416, in OpErrFmt__compute_value_7 File "rpython_rtyper_lltypesystem.c", line 11057, in ll_str2unicode__rpy_stringPtr SystemError: unexpected internal exception (please report a bug): ; internal traceback was dumped to stderr zsh: exit 1 python ``` Python 3.5.3 (a37ecfe5f142bc971a86d17305cc5d1d70abec64, Jun 08 2017, 19:43:54) [PyPy 5.8.0-beta0 with GCC 6.3.0] on linux From issues-reply at bitbucket.org Sun Jul 2 09:13:14 2017 From: issues-reply at bitbucket.org (Marian Beermann) Date: Sun, 02 Jul 2017 13:13:14 +0000 (UTC) Subject: [pypy-issue] Issue #2599: Py_XINCREF/Py_XDECREF use argument twice (pypy/pypy) Message-ID: <20170702131313.20515.57591@celery-worker-107.ash1.bb-inf.net> New issue 2599: Py_XINCREF/Py_XDECREF use argument twice https://bitbucket.org/pypy/pypy/issues/2599/py_xincref-py_xdecref-use-argument-twice Marian Beermann: This isn't documented, but in CPython these are implemented to never use the macro's argument twice: ``` #define Py_XDECREF(op) \ do { \ PyObject *_py_xdecref_tmp = (PyObject *)(op); \ if (_py_xdecref_tmp != NULL) \ Py_DECREF(_py_xdecref_tmp); \ } while (0) ``` PyPy does: ``` #define Py_XDECREF(op) do { if ((op) == NULL) ; else Py_DECREF(op); } while (0) ``` I didn't notice this in my code, and it's bad style (and I'll fix it in my code ;), but Google suggests that writing `Py_XDECREF(PyObject_CallMethod(...` is somewhat common. From issues-reply at bitbucket.org Sun Jul 2 09:15:13 2017 From: issues-reply at bitbucket.org (Marian Beermann) Date: Sun, 02 Jul 2017 13:15:13 +0000 (UTC) Subject: [pypy-issue] Issue #2600: "cpyext: missing slot wrapper wrap_delarch/m32r/platforms/usrv" (pypy/pypy) Message-ID: <20170702131513.36156.69206@celery-worker-105.ash1.bb-inf.net> New issue 2600: "cpyext: missing slot wrapper wrap_delarch/m32r/platforms/usrv" https://bitbucket.org/pypy/pypy/issues/2600/cpyext-missing-slot-wrapper-wrap_delarch Marian Beermann: Application crashes with this somewhat vague error. What does it mean / how to debug? ``` cpyext: missing slot wrapper wrap_delarch/m32r/platforms/usrv RPython traceback: File "pypy_interpreter.c", line 39639, in BuiltinCodePassThroughArguments1_funcrun_obj File "pypy_module_cpyext_3.c", line 2862, in wrap_del Fatal RPython error: NotImplementedError Fatal Python error: Aborted Stack (most recent call first, approximate line numbers): File "/home/mabe/Projekte/_oss/borg/pypy3.5-5.8-beta-linux_x86_64-portable/lib_pypy/_ctypes/function.py", line 461 in _conv_param zsh: abort (core dumped) ``` I'm not sure if this is a bug in PyPy, in a C extension or in the interaction of both. From issues-reply at bitbucket.org Thu Jul 6 13:26:22 2017 From: issues-reply at bitbucket.org (Ronan Lamy) Date: Thu, 06 Jul 2017 17:26:22 +0000 (UTC) Subject: [pypy-issue] Issue #2601: dictproxy should be recognised as a mapping, not a sequence (pypy/pypy) Message-ID: <20170706172621.33546.99786@celery-worker-105.ash1.bb-inf.net> New issue 2601: dictproxy should be recognised as a mapping, not a sequence https://bitbucket.org/pypy/pypy/issues/2601/dictproxy-should-be-recognised-as-a Ronan Lamy: On CPython: ```python >>> type(int.__dict__) >>> reversed(int.__dict__) Traceback (most recent call last): File "", line 1, in TypeError: argument to reversed() must be a sequence ``` On PyPy: ```python >>>> type(int.__dict__) >>>> reversed(int.__dict__) ``` This was fixed for pypy3 in 78dee6627756, but backporting that commit isn't enough to fix pypy2. From issues-reply at bitbucket.org Fri Jul 14 07:49:38 2017 From: issues-reply at bitbucket.org (Diego Magrini) Date: Fri, 14 Jul 2017 11:49:38 +0000 (UTC) Subject: [pypy-issue] Issue #2602: Unable to install pyzmq on pypy3 (pypy/pypy) Message-ID: <20170714114938.34461.44209@celery-worker-105.ash1.bb-inf.net> New issue 2602: Unable to install pyzmq on pypy3 https://bitbucket.org/pypy/pypy/issues/2602/unable-to-install-pyzmq-on-pypy3 Diego Magrini: While trying to install pyzmq inside pypy3 (version 5.8.0) i got this error: ``` #!bash Undefined symbols for architecture x86_64: "_PyPyModule_Create2", referenced from: _PyInit_libsodium in initlibsodium.o ld: symbol(s) not found for architecture x86_64 clang: error: linker command failed with exit code 1 (use -v to see invocation) error: command 'cc' failed with exit status 1 ``` From issues-reply at bitbucket.org Fri Jul 14 13:50:59 2017 From: issues-reply at bitbucket.org (CTPaHHuK-HEbA NA) Date: Fri, 14 Jul 2017 17:50:59 +0000 (UTC) Subject: [pypy-issue] Issue #2603: Pillow Windows - reduce (pypy/pypy) Message-ID: <20170714175059.16407.16330@celery-worker-107.ash1.bb-inf.net> New issue 2603: Pillow Windows - reduce https://bitbucket.org/pypy/pypy/issues/2603/pillow-windows-reduce CTPaHHuK-HEbA NA: timeit.timeit('PIL.ImageGrab.grab()', 'import PIL', number=100) Python 2.7.13: 4.14 sec PyPy 5.8.0: 6.75 sec -40% From issues-reply at bitbucket.org Fri Jul 14 15:50:10 2017 From: issues-reply at bitbucket.org (Herwig Hochleitner) Date: Fri, 14 Jul 2017 19:50:10 +0000 (UTC) Subject: [pypy-issue] Issue #2604: lib-python/2.7/test/test_os.py::URandomFDTests fails, when os.urandom can use the getrandom syscall (pypy/pypy) Message-ID: <20170714195010.3924.51004@celery-worker-108.ash1.bb-inf.net> New issue 2604: lib-python/2.7/test/test_os.py::URandomFDTests fails, when os.urandom can use the getrandom syscall https://bitbucket.org/pypy/pypy/issues/2604/lib-python-27-test-test_ospy Herwig Hochleitner: This doesn't show up in the portable binary, because that uses `/dev/urandom`. When building on NixOS, pypy uses _getrandom, causing that test to fail. see `rpython/rlib/rurandom.py` From issues-reply at bitbucket.org Sun Jul 16 11:53:00 2017 From: issues-reply at bitbucket.org (Jean-Louis Fuchs) Date: Sun, 16 Jul 2017 15:53:00 +0000 (UTC) Subject: [pypy-issue] Issue #2605: libress defines X509*__MISMATCH constants (pypy/pypy) Message-ID: <20170716155300.25949.61172@celery-worker-105.ash1.bb-inf.net> New issue 2605: libress defines X509*__MISMATCH constants https://bitbucket.org/pypy/pypy/issues/2605/libress-defines-x509-__mismatch-constants Jean-Louis Fuchs: My version of libressl (2.5.4) defines these constants X509_V_ERR_HOSTNAME_MISMATCH X509_V_ERR_EMAIL_MISMATCH X509_V_ERR_IP_ADDRESS_MISMATCH Patch tested against PyPy3.5 v5.8.0 release From issues-reply at bitbucket.org Sun Jul 16 11:58:33 2017 From: issues-reply at bitbucket.org (Jean-Louis Fuchs) Date: Sun, 16 Jul 2017 15:58:33 +0000 (UTC) Subject: [pypy-issue] Issue #2606: tcl version detection with default (pypy/pypy) Message-ID: <20170716155833.27229.3898@celery-worker-106.ash1.bb-inf.net> New issue 2606: tcl version detection with default https://bitbucket.org/pypy/pypy/issues/2606/tcl-version-detection-with-default Jean-Louis Fuchs: On Alpine Linux the tcl version 8.6 but there is no special include dir. The following patch will make version 8.6 default. From issues-reply at bitbucket.org Sun Jul 16 12:03:20 2017 From: issues-reply at bitbucket.org (Jean-Louis Fuchs) Date: Sun, 16 Jul 2017 16:03:20 +0000 (UTC) Subject: [pypy-issue] Issue #2607: Recommended include path for poll.h (pypy/pypy) Message-ID: <20170716160320.10628.27948@celery-worker-107.ash1.bb-inf.net> New issue 2607: Recommended include path for poll.h https://bitbucket.org/pypy/pypy/issues/2607/recommended-include-path-for-pollh Jean-Louis Fuchs: musl libc recommends including poll.h instead of sys/poll.h (usually the do this because it is defined in POSIX) ``` #!bash $> cat /usr/include/sys/poll.h #warning redirecting incorrect #include to #include ``` The patch is tested with PyPy3.5 v5.8.0 From issues-reply at bitbucket.org Sun Jul 16 12:09:50 2017 From: issues-reply at bitbucket.org (Jean-Louis Fuchs) Date: Sun, 16 Jul 2017 16:09:50 +0000 (UTC) Subject: [pypy-issue] Issue #2608: make rfile stdio variable const (pypy/pypy) Message-ID: <20170716160950.24964.41348@celery-worker-101.ash1.bb-inf.net> New issue 2608: make rfile stdio variable const https://bitbucket.org/pypy/pypy/issues/2608/make-rfile-stdio-variable-const Jean-Louis Fuchs: On Alpine Linux build fails without this patch. I'm not sure if the patch will break other builds. If other builds fail: I'm not sure how to make a better patch. Let me know how to improve this patch. Tested with PyPy3.5 v5.8.0 release. From issues-reply at bitbucket.org Wed Jul 19 04:49:27 2017 From: issues-reply at bitbucket.org (Armin Rigo) Date: Wed, 19 Jul 2017 08:49:27 +0000 (UTC) Subject: [pypy-issue] Issue #2609: multiprocessing.queues.SimpleQueue missing close() (pypy/pypy) Message-ID: <20170719084926.3920.34238@celery-worker-106.ash1.bb-inf.net> New issue 2609: multiprocessing.queues.SimpleQueue missing close() https://bitbucket.org/pypy/pypy/issues/2609/multiprocessingqueuessimplequeue-missing Armin Rigo: ``multiprocessing.queues.SimpleQueue`` should have a close() method. This is needed to explicitly release the two file descriptors of the Pipe used internally. I think the following would do: ``` #!diff diff -r 0b72fd1a7641 lib-python/2.7/multiprocessing/queues.py --- a/lib-python/2.7/multiprocessing/queues.py Sun Jul 16 13:41:28 2017 +0200 +++ b/lib-python/2.7/multiprocessing/queues.py Wed Jul 19 10:45:03 2017 +0200 @@ -358,6 +358,11 @@ self._wlock = Lock() self._make_methods() + def close(self): + # PyPy extension: CPython doesn't have this method! + self._reader.close() + self._writer.close() + def empty(self): return not self._reader.poll() ``` We need to decide if it's ok to add this method when CPython doesn't have it. From issues-reply at bitbucket.org Wed Jul 19 05:32:18 2017 From: issues-reply at bitbucket.org (Jaymin Dabhi) Date: Wed, 19 Jul 2017 09:32:18 +0000 (UTC) Subject: [pypy-issue] Issue #2610: Errors while integrating pypy in Yocto build. (pypy/pypy) Message-ID: <20170719093218.12792.93287@celery-worker-101.ash1.bb-inf.net> New issue 2610: Errors while integrating pypy in Yocto build. https://bitbucket.org/pypy/pypy/issues/2610/errors-while-integrating-pypy-in-yocto Jaymin Dabhi: Hi All, I am trying to integrate pypy in my current Yocto (Jethro branch) build for my i.MX6UL development board. I have a "armv7a" (cortexa7) architecture. I am using "meta-pypy" layer from [here](https://github.com/mzakharo/meta-pypy) and inherit the pypy package in my local.conf. But, it gives me error during bitbaking. I am bitbaking the "core-image-base". The error is as below : ``` #! ERROR: Fetcher failure: Unable to find file file://pypy-4.0.1-cortexa7.tar.bz2 anywhere. ``` I think, it can't find the package "pypy-4.0.1-cortexa7.tar.bz2" in pypy directory. What should be the other possible reason? and How it can be solved? Suggestions are welcome. Thanks, From issues-reply at bitbucket.org Thu Jul 20 13:10:37 2017 From: issues-reply at bitbucket.org (Ronan Lamy) Date: Thu, 20 Jul 2017 17:10:37 +0000 (UTC) Subject: [pypy-issue] Issue #2611: Bad test isolation : finalizers running too late in app-tests (pypy/pypy) Message-ID: <20170720171037.25218.30783@celery-worker-107.ash1.bb-inf.net> New issue 2611: Bad test isolation : finalizers running too late in app-tests https://bitbucket.org/pypy/pypy/issues/2611/bad-test-isolation-finalizers-running-too Ronan Lamy: In some circumstances, objects that are created in some app-level test may only get finalized in the next app-level test. For instance, running `py.test pypy/module/cpyext/test/test_module.py -vv -s -k 'functionality or reload' --no-tb` on commit e2b16d8 shows that the finalizer for the `Example` object created in `test_functionality` only runs at the start of `test_reload` (look for the position of `cpyext: missing slot wrapper wrap_del` in the test's output. So far, I only confirmed the issue in cpyext tests on py3.5, but I guess that all app-tests are affected. Responsible: rlamy From issues-reply at bitbucket.org Fri Jul 21 05:51:14 2017 From: issues-reply at bitbucket.org (Daniel Sutcliffe) Date: Fri, 21 Jul 2017 09:51:14 +0000 (UTC) Subject: [pypy-issue] Issue #2612: gc.get_referrers doesn't always find instance of referrer class (pypy/pypy) Message-ID: <20170721095114.17881.90619@celery-worker-109.ash1.bb-inf.net> New issue 2612: gc.get_referrers doesn't always find instance of referrer class https://bitbucket.org/pypy/pypy/issues/2612/gcget_referrers-doesnt-always-find Daniel Sutcliffe: This problem is intermittent in that in can come and go away with code structure but when code is unchanged results are repeatable. Reproduced using official Docker pypy container and using squeaky-pl's portable PyPy with version 5.8.0 build for Python 2.7.13. Actually initially found trying to get Twisted test suite passing under PyPy where the twisted.test.test_rebuild.RebuildTests.testFileRebuild FAILs but included is the smallest bit of code I have managed to get this to fail under. Two files required as problem doesn't seem to occur with class in `__main__` __BC.py__ ``` #!python class B(object): def m(self): return 'b1' ``` __referrers.py__ ``` #!python import BC import gc b = BC.B() # Doesn't find for r in gc.get_referrers(BC.B): if getattr(r, '__class__', None) is BC.B: print('get_referrers finds {0}'.format(r)) # Finds for r in gc.get_objects(): if getattr(r, '__class__', None) is BC.B: print('get_objects finds {0}'.format(r)) ``` The last 4 lines just there to show get_objects always seems to work - used as workaround in my Twisted fork. As stated, I have seen `gc.get_referrers` many times return the correct result, often just prior to including one more line of code before the call so this is obviously something a little weird - hope the issue is not too difficult to reproduce and find with this info, I must admit to having not looked at PyPy gc code Cheers /dan From issues-reply at bitbucket.org Fri Jul 21 09:21:05 2017 From: issues-reply at bitbucket.org (Antonio Cuni) Date: Fri, 21 Jul 2017 13:21:05 +0000 (UTC) Subject: [pypy-issue] Issue #2613: Fix the ABI tag (pypy/pypy) Message-ID: <20170721132105.30434.39980@celery-worker-106.ash1.bb-inf.net> New issue 2613: Fix the ABI tag https://bitbucket.org/pypy/pypy/issues/2613/fix-the-abi-tag Antonio Cuni: currently, the pypy ABI tag is fixed to `pypy-41`, as defined in `pypy/module/imp/importing.py`: ``` #DEFAULT_SOABI = 'pypy-%d%d' % PYPY_VERSION[:2] DEFAULT_SOABI = 'pypy-41' ``` However, this is probably wrong: thanks to the continuous work on cpyext, it is likely that any released pypy is not compatible with the previous ones. I propose to just revert to the old way of making SOABI dependent on the pypy version. Then, when we are stable enough, we can fix it again. As the comment in `importing.py` suggests, we should also decide what to do with cffi's ABI tag: if we keep it the same as cpyext ABI tag, it means that cffi modules will be recompiled needlessly at each pypy version. We might want to use a different (more stable) ABI only for cffi stuff, but we need to check whether the existing tools such as `pip` or `setuptools` are able to handle such a thing. From issues-reply at bitbucket.org Sat Jul 22 16:51:48 2017 From: issues-reply at bitbucket.org (realead) Date: Sat, 22 Jul 2017 20:51:48 +0000 (UTC) Subject: [pypy-issue] Issue #2614: Differences in buffering of stdin between CPython and PyPy (pypy/pypy) Message-ID: <20170722205148.12222.51096@celery-worker-109.ash1.bb-inf.net> New issue 2614: Differences in buffering of stdin between CPython and PyPy https://bitbucket.org/pypy/pypy/issues/2614/differences-in-buffering-of-stdin-between realead: The man pages of CPython say the following about option `-u`: -u Force stdin, stdout and stderr to be totally unbuffered. ... Note that there is internal buffering in xread? lines(), readlines() and file-object iterators ("for line in sys.stdin") which is not influenced by this option. So if we read from `sys.stdin` with help of file-object iterator, we can see/feel the buffering even if python was called with options `-u`. However, for pypy we can see, that if started with option `-u` the reading from `sys.stdin` with help of file-object iterator becomes slower. Admittedly, pypy man pages don't say, that `sys.sdin` stays buffered for `-u`, but this is still a difference from the behavior of CPython. To reproduce: First step: create a large input file by running the following script: ``` #!python #create_data.py for i in xrange(10**6): print 10**9 ``` via `python create_data.py > input.in`. Second step: run the following script to see the difference of running times: ``` #!python #read_it.py import sys ans = 0 for line in sys.stdin: if int(line)%k == 0: ans += 1 print(ans) ``` with `pypy read_it.py < input.in` and `pypy -u read_it.py < input.in` From issues-reply at bitbucket.org Sun Jul 23 02:08:54 2017 From: issues-reply at bitbucket.org (Nathaniel Smith) Date: Sun, 23 Jul 2017 06:08:54 +0000 (UTC) Subject: [pypy-issue] Issue #2615: This looks like a thread-safety bug causing memory corruption in pypy 3 ssl (pypy/pypy) Message-ID: <20170723060853.31661.79608@celery-worker-109.ash1.bb-inf.net> New issue 2615: This looks like a thread-safety bug causing memory corruption in pypy 3 ssl https://bitbucket.org/pypy/pypy/issues/2615/this-looks-like-a-thread-safety-bug Nathaniel Smith: Filing on behalf of @yuvipanda, who's having some trouble with bitbucket... The traceback here: https://github.com/kubernetes-client/python-base/issues/23 seems to indicate that occasionally, pypy 3's `SSLSocket.getpeercert` method is returning a corrupted string like `10.0.0.1\x005.1` for one of the subjectAltName fields, instead of the correct `10.0.0.1`. Given that symptom, and that it's observed to happen occasionally when running a stress-test that uses threads, I would suspect some sort of data race happening inside `lib_pypy/_cffi_ssl/_stdssl/certificate.py:_get_peer_alt_names`. That function seems to use a bunch of OpenSSL functions whose docs say things like "The returned pointer is an internal value which must not be freed up."... From issues-reply at bitbucket.org Sun Jul 23 19:30:14 2017 From: issues-reply at bitbucket.org (Ben Darnell) Date: Sun, 23 Jul 2017 23:30:14 +0000 (UTC) Subject: [pypy-issue] Issue #2616: Incomplete error handling in `_SSLSocket.read` (pypy/pypy) Message-ID: <20170723233013.34914.49720@celery-worker-108.ash1.bb-inf.net> New issue 2616: Incomplete error handling in `_SSLSocket.read` https://bitbucket.org/pypy/pypy/issues/2616/incomplete-error-handling-in Ben Darnell: The error handling block in [`_SSLSocket.read`](https://github.com/pypy/pypy-std-ssl/blob/415df87c6b8ea0300166726d455507cb888492a6/_stdssl/__init__.py#L473-L482) appears to be incomplete: if `SSL_read` returns `SSL_ERR_SYSCALL` (or potentially others; I don't know what is possible here but `SSL_ERR_SYSCALL` definitely is), it assumes success and returns a successful zero-byte read. I discovered this while running Tornado's test suite on pypy3 (version 5.8). The test in question is kind of ugly in the way it triggers the syscall error, but I believe this is a real issue in pypy's ssl module. Possibly related to #2578 From issues-reply at bitbucket.org Mon Jul 24 10:17:55 2017 From: issues-reply at bitbucket.org (Ronan Lamy) Date: Mon, 24 Jul 2017 14:17:55 +0000 (UTC) Subject: [pypy-issue] Issue #4: "bad callback index" (pypy/revdb) Message-ID: <20170724141746.25273.17430@celery-worker-108.ash1.bb-inf.net> New issue 4: "bad callback index" https://bitbucket.org/pypy/revdb/issues/4/bad-callback-index Ronan Lamy: After running `python -m pytest pytest pypy/module/cpyext/test/test_arraymodule.py -k test_subclass` on pypy commit e698bd62fa9d in revdb, I'm getting the following: ``` (pypy-std) ronan at Berder:~/devel/pypy$ python revdb.py log.rdb RevDB: /home/ronan/.virtualenvs/revdb/bin/python -m pytest pypy/module/cpyext/test/test_arraymodule.py -k test_subclass File "/app_main.py", line 824 in setup_bootstrap_path: (1)$ c (83000000...)bad callback index 4ad6 Traceback (most recent call last): File "/home/ronan/.virtualenvs/pypy-std/site-packages/_revdb/interact.py", line 110, in run_command runner(argument) File "/home/ronan/.virtualenvs/pypy-std/site-packages/_revdb/interact.py", line 366, in command_continue self.pgroup.get_current_time()) File "/home/ronan/.virtualenvs/pypy-std/site-packages/_revdb/interact.py", line 216, in move_forward self.pgroup.go_forward(steps) File "/home/ronan/.virtualenvs/pypy-std/site-packages/_revdb/process.py", line 368, in go_forward bkpt = self.active.forward(rel_next_clone, breakpoint_mode) File "/home/ronan/.virtualenvs/pypy-std/site-packages/_revdb/process.py", line 189, in forward bkpt = self._forward(steps, breakpoints_mode) File "/home/ronan/.virtualenvs/pypy-std/site-packages/_revdb/process.py", line 213, in _forward msg = self.recv() File "/home/ronan/.virtualenvs/pypy-std/site-packages/_revdb/process.py", line 118, in recv binary = self._recv_all(struct.calcsize("iIqqq")) File "/home/ronan/.virtualenvs/pypy-std/site-packages/_revdb/process.py", line 106, in _recv_all raise EOFError EOFError Something went wrong. You are now in a pdb; press Ctrl-D to continue. > /home/ronan/.virtualenvs/pypy-std/site-packages/_revdb/process.py(106)_recv_all() -> raise EOFError (Pdb) ``` Any clue ? From issues-reply at bitbucket.org Tue Jul 25 01:05:35 2017 From: issues-reply at bitbucket.org (Nathaniel Smith) Date: Tue, 25 Jul 2017 05:05:35 +0000 (UTC) Subject: [pypy-issue] Issue #2617: pypy binary is linked to too much stuff, which breaks manylinux wheels (pypy/pypy) Message-ID: <20170725050534.973.80363@celery-worker-105.ash1.bb-inf.net> New issue 2617: pypy binary is linked to too much stuff, which breaks manylinux wheels https://bitbucket.org/pypy/pypy/issues/2617/pypy-binary-is-linked-to-too-much-stuff Nathaniel Smith: With pypy 5.8, `ldd pypy` shows that it's linked to `libbz2`, `libcrypto`, `libffi`, `libncurses`, ... Likewise for pypy3 5.8 and `ldd pypy3.5`. You would not think so, but it turns out that this is a big problem for distributing wheels. The issue is that the way ELF works, any libraries that show up in `ldd $TOPLEVELBINARY` effectively get LD_PRELOADed into any extension modules that you load later. So, for example, if some wheel distributes its own version of openssl, then any symbols that show up in both their copy of openssl and pypy's copy of openssl will get shadowed and hello segfaults. The cryptography project recently ran into this with uwsgi: https://github.com/pyca/cryptography/issues/3804#issuecomment-317401627 Fortunately this has not been a big deal so far because, uh... nobody distributes pypy wheels. But in the future maybe this is something that should be supported :-). And while in theory it would be nice if this could be fixed on the wheel side, [this is not trivial](https://github.com/pypa/auditwheel/issues/79). The obvious solution would be to switch things around so that the top-level pypy executable does `dlopen("libpypy-c.so", RTLD_LOCAL)` to start the interpreter, instead of linking against it with `-lpypy-c`. Then the symbols from `libpypy-c.so` and everything it links to would be confined to an ELF local namespace, and would stop polluting the namespace of random extension modules. However... there is a problem, which is that cpyext extension modules need *some* way to get at the C API symbols, and I assume cffi extension modules need access to some pypy symbols as well. This is... tricky, given how rpython wants to mush everything together into one giant .so, and ELF makes it difficult to only expose *some* symbols from a binary like this. Some options: * when using libcrypto or whatever from rpython, use `dlopen("libcrypto", RTLD_LOCAL)` instead of `-lcrypto`. I guess this could be done systematically in rffi? * provide a special `libcpyext` that uses `dlopen` to fetch the symbols from `libpypy-c.so` and then manually re-exports them? From issues-reply at bitbucket.org Wed Jul 26 13:12:46 2017 From: issues-reply at bitbucket.org (Cosimo Lupo) Date: Wed, 26 Jul 2017 17:12:46 +0000 (UTC) Subject: [pypy-issue] Issue #2618: incorrect "surrogatepass" encoding with pypy3.5-5.8.0 (pypy/pypy) Message-ID: <20170726171245.39686.54101@celery-worker-108.ash1.bb-inf.net> New issue 2618: incorrect "surrogatepass" encoding with pypy3.5-5.8.0 https://bitbucket.org/pypy/pypy/issues/2618/incorrect-surrogatepass-encoding-with Cosimo Lupo: Hello, I'm getting different encodings between CPython 3.5.3 and pypy3.5-5.8.0 when the input string contains surrogate escapes. When I roundtrip the string 'Carrot \ud83e\udd55' through "utf_16_be" encoding with errors="surrogatepass", in CPython I correctly get 'Carrot \U0001f955' ``` Python 3.5.3 (default, Jul 18 2017, 13:04:39) [GCC 4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.42)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> 'Carrot \ud83e\udd55'.encode('utf_16_be', errors='surrogatepass') b'\x00C\x00a\x00r\x00r\x00o\x00t\x00 \xd8>\xddU' >>> 'Carrot \ud83e\udd55'.encode('utf_16_be', errors='surrogatepass').decode('utf_16_be') 'Carrot \U0001f955' ``` However, with PyPy3.5 5.8.0, same input and code, I get this: ``` Python 3.5.3 (a37ecfe5f142bc971a86d17305cc5d1d70abec64, Jul 25 2017, 16:48:07) [PyPy 5.8.0-beta0 with GCC 4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.42)] on darwin Type "help", "copyright", "credits" or "license" for more information. And now for something completely different: ``the future has just begun'' >>>> 'Carrot \ud83e\udd55'.encode('utf_16_be', errors='surrogatepass') b'\x00C\x00a\x00r\x00r\x00o\x00t\x00 >\xd8U\xdd' >>>> 'Carrot \ud83e\udd55'.encode('utf_16_be', errors='surrogatepass').decode('utf_16_be') 'Carrot ??' ``` I'm on macOS 10.12.6, I compiled pypy3 from source, using latest GCC 7.1.0 from homebrew. I haven't had the chance to try on Linux yet. Thanks for your help. From issues-reply at bitbucket.org Wed Jul 26 19:42:58 2017 From: issues-reply at bitbucket.org (Daniel Sutcliffe) Date: Wed, 26 Jul 2017 23:42:58 +0000 (UTC) Subject: [pypy-issue] Issue #2619: Difference between variables assigned bound methods in PyPy and CPython (pypy/pypy) Message-ID: <20170726234256.22925.69384@celery-worker-105.ash1.bb-inf.net> New issue 2619: Difference between variables assigned bound methods in PyPy and CPython https://bitbucket.org/pypy/pypy/issues/2619/difference-between-variables-assigned Daniel Sutcliffe: Apologies if this is covered in in the [differences](http://doc.pypy.org/en/latest/cpython_differences.html) I have read through several times and been extra careful in [section about `id` and `is`](http://doc.pypy.org/en/latest/cpython_differences.html#object-identity-of-primitive-values-is-and-id) but I don't think it applies to this situation. A bit of code should easily show where the difference is: ``` #!python class Foo: def method(self): pass foo = Foo() mth = foo.method assert(mth is not foo.method) ``` This works fine in CPython (2.7.13 and 3.5.3) but gives an AssertionError in PyPy 5.8.0 for both the 2.7.13 and 3.5.3 builds. Unbound method assignments seem to work the same in my testing.