From issues-reply at bitbucket.org  Sun Mar 10 14:59:41 2019
From: issues-reply at bitbucket.org (Daniel Hahler)
Date: Sun, 10 Mar 2019 18:59:41 +0000 (UTC)
Subject: [pypy-issue] Issue #2970: pyrepl._minimal_curses: "ImportError:
 curses library not found": should look for ncursesw also (pypy/pypy)
Message-ID: <20190310185941.10345.85986@app-137.ash1.bb-inf.net>

New issue 2970: pyrepl._minimal_curses: "ImportError: curses library not found": should look for ncursesw also
https://bitbucket.org/pypy/pypy/issues/2970/pyrepl_minimal_curses-importerror-curses

Daniel Hahler:

```
Python 3.6.1 (dab365a465140aa79a5f3ba4db784c4af4d5c195, Feb 10 2019, 01:16:39)
[PyPy 7.0.0-alpha0 with GCC 8.2.1 20181127] on linux
Type "help", "copyright", "credits" or "license" for more information.
And now for something completely different: ``all we need is a native Prolog
machine''
>>>> from pyrepl import _minimal_curses
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "?/Vcs/pyrepl-git/pyrepl/_minimal_curses.py", line 27, in <module>
    _clibpath = _find_clib()
  File "?/Vcs/pyrepl-git/pyrepl/_minimal_curses.py", line 25, in _find_clib
    raise ImportError("curses library not found")
ImportError: curses library not found
```

https://bitbucket.org/pypy/pyrepl/issues/19/does-not-find-curses-via-ctypes-on-arch

A patch is available in the apparently obsolete/unmaintained pyrepl repo:

```diff
diff -r 9401662c4e6c pyrepl/_minimal_curses.py
--- a/pyrepl/_minimal_curses.py Fri Dec 04 13:10:13 2015 +0000
+++ b/pyrepl/_minimal_curses.py Wed Aug 31 17:10:45 2016 +0200
@@ -17,7 +17,7 @@


 def _find_clib():
-    trylibs = ['ncurses', 'curses']
+    trylibs = ['ncursesw', 'ncurses', 'curses']

     for lib in trylibs:
         path = ctypes.util.find_library(lib)
```



From issues-reply at bitbucket.org  Sun Mar 10 15:09:56 2019
From: issues-reply at bitbucket.org (Daniel Hahler)
Date: Sun, 10 Mar 2019 19:09:56 +0000 (UTC)
Subject: [pypy-issue] Issue #2971: pyrepl: package for / update on PyPI
 (pypy/pypy)
Message-ID: <20190310190956.22572.62157@celery-worker-112.ash1.bb-inf.net>

New issue 2971: pyrepl: package for / update on PyPI
https://bitbucket.org/pypy/pypy/issues/2971/pyrepl-package-for-update-on-pypi

Daniel Hahler:

I've forked pypy/pyrepl at https://github.com/blueyed/pyrepl, and fixed issues for py3 there - before noticing that it appears to have been integrated into PyPy by now?

I would like to have a fixed package for it on PyPI - what is the recommended way for this?

py2/py3 compatibility is in different PyPy branches, correct?
Is "py3.6" the branch to use for py3 compatibility, i.e. the most recent one?

Currently I am seeing test failures with it though, and wondered if this passes on PyPy's CI after all?!
(where can this be seen?)

This is basically using `pypy/lib_pypy/pyrepl` and
`pypy/extra_tests/test_pyrepl` with the tox.ini from
https://github.com/blueyed/pyrepl.

```
% tox -e pypy3
GLOB sdist-make: ?/Vcs/pyrepl-git/setup.py
pypy3 inst-nodeps: /tmp/tox?/Vcs/pyrepl-git/.tmp/package/1/pyrepl-0.8.4.zip
pypy3 installed: atomicwrites==1.3.0,attrs==19.1.0,cffi==1.12.0,greenlet==0.4.13,more-itertools==6.0.0,pexpect==4.6.0,pluggy==0.9.0,ptyprocess==0.6.0,py==1.8.0,pyrepl==0.8.4,pytest==4.3.0,readline==6.2.4.1,six==1.12.0
pypy3 run-test-pre: PYTHONHASHSEED='3582607578'
pypy3 runtests: commands[0] | pytest
==================================================================================== test session starts =====================================================================================
platform linux -- Python 3.6.1[pypy-7.0.0-alpha], pytest-4.3.0, py-1.8.0, pluggy-0.9.0
cachedir: /tmp/tox?/Vcs/pyrepl-git/pypy3/.pytest_cache
rootdir: ?/Vcs/pyrepl-git, inifile: tox.ini
collected 21 items                                                                                                                                                                           

testing/test_basic.py .............                                                                                                                                                    [ 61%]
testing/test_bugs.py x..                                                                                                                                                               [ 76%]
testing/test_functional.py .                                                                                                                                                           [ 80%]
testing/test_keymap.py .                                                                                                                                                               [ 85%]
testing/test_reader.py .                                                                                                                                                               [ 90%]
testing/test_readline.py F                                                                                                                                                             [ 95%]
testing/test_wishes.py F                                                                                                                                                               [100%]

========================================================================================== FAILURES ==========================================================================================
_______________________________________________________________________________________ test_raw_input _______________________________________________________________________________________

    @pytest.mark.skipif("os.name != 'posix' or 'darwin' in sys.platform or "
                        "'freebsd' in sys.platform")
    def test_raw_input():
        import os
        import pty
        from pyrepl.readline import _ReadlineWrapper
    
        master, slave = pty.openpty()
        readline_wrapper = _ReadlineWrapper(slave, slave)
        os.write(master, b'input\n')
    
        with sane_term():
            result = readline_wrapper.get_reader().readline()
        #result = readline_wrapper.raw_input('prompt:')
>       assert result == 'input'
E       AssertionError: assert b'input' == 'input'

testing/test_readline.py:20: AssertionError
_________________________________________________________________________________ test_quoted_insert_repeat __________________________________________________________________________________

    def test_quoted_insert_repeat():
        read_spec([
            (('digit-arg', '3'),      ['']),
            (('quoted-insert', None), ['']),
            (('self-insert', '\033'), ['^[^[^[']),
>           (('accept', None),        None)])

testing/test_wishes.py:31: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
testing/infrastructure.py:76: in read_spec
    reader.readline()
pyrepl/reader.py:621: in readline
    self.handle1()
pyrepl/reader.py:604: in handle1
    self.do_cmd(cmd)
pyrepl/reader.py:556: in do_cmd
    self.refresh()
testing/infrastructure.py:68: in refresh
    Reader.refresh(self)
pyrepl/reader.py:538: in refresh
    self.console.refresh(screen, self.cxy)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <testing.infrastructure.TestConsole object at 0x00007fe7368dbe18>, screen = ['\\u001b\\u001b\\u001b'], xy = (18, 0)

    def refresh(self, screen, xy):
        if self.next_screen is not None:
                assert screen == self.next_screen, "[ %s != %s after %r ]" % (
>                   screen, self.next_screen, self.last_event_name)
E               AssertionError: [ ['\\u001b\\u001b\\u001b'] != ['^[^[^['] after 'self-insert' ]

testing/infrastructure.py:49: AssertionError
------------------------------------------------------------------------------------ Captured stdout call ------------------------------------------------------------------------------------
event ('digit-arg', '3')
event ('quoted-insert', None)
event ('self-insert', '\x1b')
======================================================================= 2 failed, 18 passed, 1 xfailed in 1.53 seconds =======================================================================
ERROR: InvocationError for command '/tmp/tox?/Vcs/pyrepl-git/pypy3/bin/pytest' (exited with code 1)
__________________________________________________________________________________________ summary ___________________________________________________________________________________________
ERROR:   pypy3: commands failed
```



From issues-reply at bitbucket.org  Mon Mar 11 09:50:58 2019
From: issues-reply at bitbucket.org (Carl Friedrich Bolz-Tereick)
Date: Mon, 11 Mar 2019 13:50:58 +0000 (UTC)
Subject: [pypy-issue] Issue #2972: simplified codecs not accepting error
 arguments (pypy/pypy)
Message-ID: <20190311135058.4533.87891@celery-worker-108.ash1.bb-inf.net>

New issue 2972: simplified codecs not accepting error arguments
https://bitbucket.org/pypy/pypy/issues/2972/simplified-codecs-not-accepting-error

Carl Friedrich Bolz-Tereick:

It seems there is code in the wild that registers codecs that don't accept an error argument at all:

https://github.com/modoboa/modoboa/blob/master/modoboa/lib/imap_utf7.py#L74

(reported here: https://twitter.com/patthemav/status/1102059893438246912 )

The attached file reproduces the problem. CPython seems to have complicated logic to actually not pass a second argument in some cases (see ``Python/codecs.c``).



From issues-reply at bitbucket.org  Thu Mar 14 05:09:27 2019
From: issues-reply at bitbucket.org (El. Plau-t)
Date: Thu, 14 Mar 2019 09:09:27 +0000 (UTC)
Subject: [pypy-issue] Issue #2973: [ubuntu] install old versions from ppa
 (pypy/pypy)
Message-ID: <20190314090926.12873.20027@app-137.ash1.bb-inf.net>

New issue 2973: [ubuntu] install old versions from ppa
https://bitbucket.org/pypy/pypy/issues/2973/ubuntu-install-old-versions-from-ppa

El. Plau-t:

Hi,

I hope this is the right place to report this issue.

I'm trying to install v6.0.0 on ubuntu 16.04 from the ppa, but the [Packages.gz](http://ppa.launchpad.net/pypy/ppa/ubuntu/dists/xenial/main/binary-amd64/Packages.gz) file doesn't expose all the [available package](http://ppa.launchpad.net/pypy/ppa/ubuntu/pool/main/p/pypy/), 6.0.0 in particular.

Many thanks!
Eli



From issues-reply at bitbucket.org  Sat Mar 16 00:06:32 2019
From: issues-reply at bitbucket.org (Alex Kashirin)
Date: Sat, 16 Mar 2019 04:06:32 +0000 (UTC)
Subject: [pypy-issue] =?utf-8?q?Issue_=232974=3A_GC_+_an_api_to_Malloc=27?=
 =?utf-8?q?s_call_to_ReleaseFreeMemory_for_the_=E2=80=9Cunreturned_memory?=
 =?utf-8?b?4oCdIGNhc2UgKHB5cHkvcHlweSk=?=
Message-ID: <20190316040632.1372.53783@celery-worker-109.ash1.bb-inf.net>

New issue 2974: GC + an api to Malloc's call to ReleaseFreeMemory for the ?unreturned memory? case
https://bitbucket.org/pypy/pypy/issues/2974/gc-an-api-to-mallocs-call-to

Alex Kashirin:

In this case, the none GLIBC_MALLOC is TCMALLOC 

A successful build with TCMALLOC goes well with this flag preset to the translation and build,


```
#!bash

LDFLAGS="-DTCMALLOC -ltcmalloc -lunwind \
-fno-builtin-malloc -fno-builtin-calloc -fno-builtin-realloc -fno-builtin-free" \
CFLAGS="$LDFLAGS" \
python ../../rpython/bin/rpython \
--no-shared --thread --make-jobs=8 \
--opt=jit targetpypystandalone.py --allworkingmodules 

```
for ref to full build, https://github.com/kashirin-alex/environments-builder/blob/master/scripts/pypy3.sh#L11

By the attached test,
The result are great, with at peak of 24GB~, the final RSS is 128MB for a steady consumption of 64MB while initial RSS was 56 MB

For the test purpose, the test command was :

```
#!bash

TCMALLOC_RELEASE_RATE=1000 MALLOCSTATS=2 pypy3 mem_chk.py

```
TCMALLOC_RELEASE_RATE, 1000 is a quite high, evaluated to release 100% each time possible. while 100 is around 10% of possible.


What could be a real improvement is an API in the PYPY-GC to Malloc's release mem calls,
In case of TCMALLOC , it is the 

for a compilation with the compiler definers, -DTCMALLOC or -DTCMALLOC_MINIMAL
required to add to the PYPY-GC with a corresponding py-api

```
#!C

#if defined(TCMALLOC) || defined(TCMALLOC_MINIMAL)
#include <gperftools/malloc_extension.h>
void ...pypy..GC.. malloc_release(){
   MallocExtension::instance()->ReleaseFreeMemory();
}
#endif
```

This way the memory can be release as needed and not only at RELEASE_RATE
  

the memtest.py results look as follow:

```
#!bash
Total memory consumed:
    GC used:            15.1MB (peak: 24473.6MB)
       in arenas:            4.7MB
       rawmalloced:          2.4MB
       nursery:              8.0MB
    raw assembler used: 57.1kB
    -----------------------------
    Total:              15.2MB

    Total memory allocated:
    GC allocated:            16.4MB (peak: 24474.1MB)
       in arenas:            5.6MB
       rawmalloced:          24460.1MB
       nursery:              8.0MB
    raw assembler allocated: 1.0MB
    -----------------------------
    Total:                   17.4MB

    Total time spent in GC:  11.225

********************************************************************************
steady consumed duration: 22.838454008102417
steady num tests: 207
test load duration: 54.853511571884155
initial_rss: 0.056 GB
actual consumed: 0.119 GB
********************************************************************************
------------------------------------------------
MALLOC:       19724440 (   18.8 MiB) Bytes in use by application
MALLOC: +     88956928 (   84.8 MiB) Bytes in page heap freelist
MALLOC: +       479624 (    0.5 MiB) Bytes in central cache freelist
MALLOC: +            0 (    0.0 MiB) Bytes in transfer cache freelist
MALLOC: +       849376 (    0.8 MiB) Bytes in thread cache freelists
MALLOC: +     39059456 (   37.2 MiB) Bytes in malloc metadata
MALLOC:   ------------
MALLOC: =    149069824 (  142.2 MiB) Actual memory used (physical + swap)
MALLOC: +  28540633088 (27218.5 MiB) Bytes released to OS (aka unmapped)
MALLOC:   ------------
MALLOC: =  28689702912 (27360.6 MiB) Virtual address space used
MALLOC:
MALLOC:            347              Spans in use
MALLOC:              2              Thread heaps in use
MALLOC:           8192              Tcmalloc page size
------------------------------------------------
Call ReleaseFreeMemory() to release freelist memory to the OS (via madvise()).
Bytes released to the OS take up virtual address space but no physical memory.
```

If you would like to have the pypy3 compilation in a package , let me know, it require some  .



From issues-reply at bitbucket.org  Thu Mar 21 03:28:18 2019
From: issues-reply at bitbucket.org (Andrew Stepanov)
Date: Thu, 21 Mar 2019 07:28:18 +0000 (UTC)
Subject: [pypy-issue] Issue #2975: Joining long bytes or str sequences is
 10-40x slower on PyPy2 and PyPy3 (pypy/pypy)
Message-ID: <20190321072818.37130.63527@app-137.ash1.bb-inf.net>

New issue 2975: Joining long bytes or str sequences is 10-40x slower on PyPy2 and PyPy3
https://bitbucket.org/pypy/pypy/issues/2975/joining-long-bytes-or-str-sequences-is-10

Andrew Stepanov:

I've encountered performance degradation on PyPy2 and PyPy3 with the following code:

```python
import time
import sys

sep = '' if sys.argv[1] == "str" else b''
data = ('0' if sys.argv[1] == "str" else b'0') * int(sys.argv[2])


def workload(num_runs):
    for i in range(num_runs):
        sep.join((data, data))

start = time.time()
workload(200000)
print(time.time() - start)
```

When I try joining bytes or strings in a loop, everything is fine when the sequence is short (100 bytes, for example), PyPy is 3x or 4x times faster.

```bash
$ python3.7 join_test.py str 100
0.04081892967224121
```

```bash
$ python3.7 join_test.py bytes 100
0.03854179382324219
```

```bash
$ pypy3 join_test.py bytes 100
0.010421037673950195
```

```bash
$ pypy3 join_test.py str 100
0.014775991439819336
```
But when I try joining long bytes or strings, PyPy suddenly starts to lag behind:

```bash
$ python3.7 join_test.py bytes 100000
1.3616209030151367
```

```bash
$ python3.7 join_test.py str 100000
1.4790830612182617
```

```bash
$ pypy3 join_test.py bytes 100000
27.6971218585968
```

```bash
$ pypy3 join_test.py str 100000
73.05885100364685
``` 

I've tried adding `--jit off` option to PyPy command line, but it didn't change anything. On Linux machine, the difference is less pronounced (around 6x slower for 100KB version). Same happens on PyPy2. This issue may be related to [this issue](https://bitbucket.org/pypy/pypy/issues/2782/extending-bytearray-is-x30-slower-than). I am using the following versions of PyPy:


```bash
$ pypy -V
Python 2.7.13 (990cef41fe11e5d46b019a46aa956ff46ea1a234, Mar 18 2019, 17:41:49)
[PyPy 7.1.0 with GCC 4.2.1 Compatible Apple LLVM 10.0.0 (clang-1000.11.45.5)]
```

```bash
$ pypy3 -V
Python 3.5.3 (928a4f70d3de, Feb 08 2019, 10:43:14)
[PyPy 7.0.0 with GCC 4.2.1 Compatible Apple LLVM 10.0.0 (clang-1000.11.45.5)]
```



From issues-reply at bitbucket.org  Fri Mar 22 11:39:42 2019
From: issues-reply at bitbucket.org (=?utf-8?q?Ren=C3=A9_Dudfield?=)
Date: Fri, 22 Mar 2019 15:39:42 +0000 (UTC)
Subject: [pypy-issue] Issue #2976: PyUnicode_DecodeLocale is missing from
 cpyext in pypy 3.6 (pypy/pypy)
Message-ID: <20190322153942.14878.63542@celery-worker-112.ash1.bb-inf.net>

New issue 2976: PyUnicode_DecodeLocale is missing from cpyext in pypy 3.6
https://bitbucket.org/pypy/pypy/issues/2976/pyunicode_decodelocale-is-missing-from

Ren? Dudfield:

Here is the cpython docs:
https://docs.python.org/3/c-api/unicode.html#c.PyUnicode_DecodeLocale



From issues-reply at bitbucket.org  Sat Mar 23 15:10:24 2019
From: issues-reply at bitbucket.org (mattip)
Date: Sat, 23 Mar 2019 19:10:24 +0000 (UTC)
Subject: [pypy-issue] Issue #2977: errno.EWOULDBLOCK must be
 errno.WSAEWOULDBLOCK on windows (pypy/pypy)
Message-ID: <20190323191024.2001.2094@celery-worker-111.ash1.bb-inf.net>

New issue 2977: errno.EWOULDBLOCK must be errno.WSAEWOULDBLOCK on windows
https://bitbucket.org/pypy/pypy/issues/2977/errnoewouldblock-must-be

mattip:

msvc 2010 added EWOULDBLOCK, previously it was set to WSAEWOULDBLOCK. We do not override the E* values with the WSAE* values as [cpython does](https://github.com/python/cpython/blob/3.7/Modules/errnomodule.c#L10). This causes [pika](https://github.com/pika/pika/blob/stable/pika/adapters/base_connection.py#L33) to fail. CPython raises a `"BlockingIOError(10035, ...) when the sendbuffer is full. PyPy3 raises BlockingIOError(140, ...) since errno.EWOULDBLOCK is 140 but cpython uses errno.WSAEWOULDBLOCK == 10035



From issues-reply at bitbucket.org  Mon Mar 25 14:51:57 2019
From: issues-reply at bitbucket.org (Stefano Rivera)
Date: Mon, 25 Mar 2019 18:51:57 +0000 (UTC)
Subject: [pypy-issue] Issue #2978: Avoid timeouts - extra dot in graphlib
 (pypy/pypy)
Message-ID: <20190325185157.21231.31987@celery-worker-110.ash1.bb-inf.net>

New issue 2978: Avoid timeouts - extra dot in graphlib
https://bitbucket.org/pypy/pypy/issues/2978/avoid-timeouts-extra-dot-in-graphlib

Stefano Rivera:

Debian has had some issues with pypy builds timing out on some of our slower architectures. [Logs](https://buildd.debian.org/status/logs.php?pkg=pypy).
I brought this up on IRC a while ago (probably at least a year), and planned to go digging for some place to insert `log.dot()` calls. Armin was not very hopeful that I'd find useful places to do this, as we don't want to emit a dot unless the build is actually progressing, and not stuck in a loop.

Anyway, I found 1 place that made a big difference for us. So, I thought I'd share the patch.
 
I'm happy to test variants of this on MIPS64, which was timing out most "reliably".

I'd say the next most frequent cause of timeouts is in the test harness for cpyext., e.g. [log](https://buildd.debian.org/status/fetch.php?pkg=pypy&arch=kfreebsd-i386&ver=7.0.0%2Bdfsg-3&stamp=1553476497&raw=0)



From issues-reply at bitbucket.org  Mon Mar 25 16:20:30 2019
From: issues-reply at bitbucket.org (Oliver Margetts)
Date: Mon, 25 Mar 2019 20:20:30 +0000 (UTC)
Subject: [pypy-issue] Issue #2979: Calling gc.get_objects() in the REPL on
 PyPy 7.1.0 SEGFAULTs (pypy/pypy)
Message-ID: <20190325202030.1577.89455@celery-worker-111.ash1.bb-inf.net>

New issue 2979: Calling gc.get_objects() in the REPL on PyPy 7.1.0 SEGFAULTs
https://bitbucket.org/pypy/pypy/issues/2979/calling-gcget_objects-in-the-repl-on-pypy

Oliver Margetts:

# Issue

Under the PyPy 7.1.0 REPL the following happens:
```bash
$ pypy3
Python 3.6.1 (bb0d05b190b9c579f0c889a368636e14f6205bab, Mar 19 2019, 13:27:00)
[PyPy 7.1.0 with GCC 8.2.1 20181127] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>> import gc
>>>> gc.get_objects()
Segmentation fault (core dumped)
```

This does persist after switching to PyPy2 v7.1.0. This does not happen on PyPy 7.0.0, and nor does it happen if one puts this in a `.py` file and runs it.

# Other notes - currently I'm running Arch Linux
```bash
$ uname -a
Linux 5.0.3-arch1-1-ARCH #1 SMP PREEMPT Tue Mar 19 13:09:13 UTC 2019 x86_64 GNU/Linux
```



From issues-reply at bitbucket.org  Tue Mar 26 02:48:30 2019
From: issues-reply at bitbucket.org (AliReza Mosajjal)
Date: Tue, 26 Mar 2019 06:48:30 +0000 (UTC)
Subject: [pypy-issue] Issue #2980: awful bisect performance in pypy3
 compared to cpython or pypy2 (pypy/pypy)
Message-ID: <20190326064830.22936.98676@celery-worker-112.ash1.bb-inf.net>

New issue 2980: awful bisect performance in pypy3 compared to cpython or pypy2
https://bitbucket.org/pypy/pypy/issues/2980/awful-bisect-performance-in-pypy3-compared

AliReza Mosajjal:

Hi. I created a simple code to search in IPs and filter out which one is from a certain country. Here's the code sample:

https://pastebin.com/NA3jpLei

to run it, I use this command (for testing):
```
shuf -i 16777216-4294967295 -n 1000000 |  python main.py
```
Now take a look at the output with different python versions:
```
shuf -i 16777216-4294967295 -n 1000000 | time pypy3 main.py
11.13user 0.05system 0:11.93elapsed 93%CPU (0avgtext+0avgdata 94676maxresident)k
```
```
shuf -i 16777216-4294967295 -n 1000000 | time pypy main.py
0.52user 0.06system 0:01.42elapsed 40%CPU (0avgtext+0avgdata 79356maxresident)k
```
```
shuf -i 16777216-4294967295 -n 1000000 | time python2 main.py
1.09user 0.00system 0:01.94elapsed 56%CPU (0avgtext+0avgdata 9504maxresident)k
```
```
shuf -i 16777216-4294967295 -n 1000000 | time python3 main.py
1.17user 0.01system 0:02.00elapsed 59%CPU (0avgtext+0avgdata 11380maxresident)k
```

What makes pypy3 slower (by a considerable magnitude) to other implementations?



From issues-reply at bitbucket.org  Wed Mar 27 01:05:07 2019
From: issues-reply at bitbucket.org (wrq)
Date: Wed, 27 Mar 2019 05:05:07 +0000 (UTC)
Subject: [pypy-issue] Issue #2981: script exited with `exit` break from pypy
 interactive mode (pypy/pypy)
Message-ID: <20190327050507.5442.26482@celery-worker-109.ash1.bb-inf.net>

New issue 2981: script exited with `exit` break from pypy interactive mode
https://bitbucket.org/pypy/pypy/issues/2981/script-exited-with-exit-break-from-pypy

wrq:

`pypy -i -c 'exit(-1)'` did not provide an interactive pypy console on exit, and so do any script exited with the obvious exit function; `sys.exit` is unaffected.

I use pypy2, so more versions are not tested, I would suggest that anyone test this on all platforms.

```
[dummy at f2e2157eeb03 project]$ pypy -i -c 'import sys;sys.exit(-1)'
Traceback (most recent call last):
  File "<builtin>/app_main.py", line 76, in handle_sys_exit
SystemExit: -1
>>>> 
[dummy at f2e2157eeb03 project]$ pypy -i -c 'exit(-1)'
Traceback (most recent call last):
  File "<builtin>/app_main.py", line 76, in handle_sys_exit
SystemExit: -1
Traceback (most recent call last):
  File "/usr/lib64/pypy-6.0/lib_pypy/_pypy_interact.py", line 30, in interactive_console
    if not os.isatty(sys.stdin.fileno()):
ValueError: I/O operation on closed file
[dummy at f2e2157eeb03 project]$ python -i -c 'import sys;sys.exit(-1)'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
SystemExit: -1
>>> 
[dummy at f2e2157eeb03 project]$ python -i -c 'exit(-1)'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib64/python2.7/site.py", line 355, in __call__
    raise SystemExit(code)
SystemExit: -1
>>> 
[dummy at f2e2157eeb03 project]$ pypy --version
Python 2.7.13 (1d8462e42c37, Aug 30 2018, 09:58:57)
[PyPy 6.0.0 with GCC 8.2.1 20180801 (Red Hat 8.2.1-2)]

```



From issues-reply at bitbucket.org  Wed Mar 27 20:07:44 2019
From: issues-reply at bitbucket.org (Robert Whitcher)
Date: Thu, 28 Mar 2019 00:07:44 +0000 (UTC)
Subject: [pypy-issue] Issue #2982: Continual Increase In Memory Utilization
 Using PyPy 6.0 (python 2.7) and Pymongo (pypy/pypy)
Message-ID: <20190328000744.38003.67984@celery-worker-108.ash1.bb-inf.net>

New issue 2982: Continual Increase In Memory Utilization Using PyPy 6.0 (python 2.7) and Pymongo
https://bitbucket.org/pypy/pypy/issues/2982/continual-increase-in-memory-utilization

Robert Whitcher:

So I have a process running PyPy that hold a pymongo connection and loops.
It does basically the same thing every loop, which is to query a table in via pymongo and do a few non-saved calculations and then wait and loop again 

The RSS of the process continually increased (the PYPY_GC_MAX is set pretty high).
So I hooked in the GC stats output per: http://doc.pypy.org/en/latest/gc_info.html
I also assured that gc.collect() was called at least every 3 minutes.

What I see is that... The memory while high is fair constant for a long time (output about every minute):
2019-03-27 00:04:10.033-0600 [-] main_thread(29621)log (async_worker_process.py:304): INFO_FLUSH: RSS: 144244736
...
2019-03-27 01:01:46.841-0600 [-] main_thread(29621)log (async_worker_process.py:304): INFO_FLUSH: RSS: 144420864
2019-03-27 01:02:36.943-0600 [-] main_thread(29621)log (async_worker_process.py:304): INFO_FLUSH: RSS: 144269312

Then it decides (and the exact per-loop behavior is the same each time) to chew up much more memory:

2019-03-27 01:04:17.184-0600 [-] main_thread(29621)log (async_worker_process.py:304): INFO_FLUSH: RSS: 145469440
2019-03-27 01:05:07.305-0600 [-] main_thread(29621)log (async_worker_process.py:304): INFO_FLUSH: RSS: 158175232
2019-03-27 01:05:57.401-0600 [-] main_thread(29621)log (async_worker_process.py:304): INFO_FLUSH: RSS: 173191168
2019-03-27 01:06:47.490-0600 [-] main_thread(29621)log (async_worker_process.py:304): INFO_FLUSH: RSS: 196943872
2019-03-27 01:07:37.575-0600 [-] main_thread(29621)log (async_worker_process.py:304): INFO_FLUSH: RSS: 205406208
2019-03-27 01:08:27.659-0600 [-] main_thread(29621)log (async_worker_process.py:304): INFO_FLUSH: RSS: 254562304
2019-03-27 01:09:17.770-0600 [-] main_thread(29621)log (async_worker_process.py:304): INFO_FLUSH: RSS: 256020480
2019-03-27 01:10:07.866-0600 [-] main_thread(29621)log (async_worker_process.py:304): INFO_FLUSH: RSS: 289779712

That's 140 MB .... Where is all that memory going...
What's more is that the PyPy GC stats do not show anything different:

Here are the GC stats from GC-Complete when we were at 144MB:

2019-03-26 23:55:49.127-0600 [-] main_thread(29621)log (async_worker_process.py:304): INFO_FLUSH: RSS: 140632064
2019-03-26 23:55:49.133-0600 [-] main_thread(29621)log (async_worker_process.py:308): DBG0: Total memory consumed:
            GC used:            56.8MB (peak: 69.6MB)
               in arenas:            39.3MB
               rawmalloced:          14.5MB
               nursery:              3.0MB
            raw assembler used: 521.6kB
            -----------------------------
            Total:              57.4MB

            Total memory allocated:
            GC allocated:            63.0MB (peak: 71.2MB)
               in arenas:            43.9MB
               rawmalloced:          22.7MB
               nursery:              3.0MB
            raw assembler allocated: 1.0MB
            -----------------------------
            Total:                   64.0MB

Here are the GC stats from GC-Complete when we are at 285MB:

2019-03-27 01:42:41.751-0600 [-] main_thread(29621)log (async_worker_process.py:304): INFO_FLUSH: RSS: 285147136
2019-03-27 01:42:41.751-0600 [-] main_thread(29621)log (async_worker_process.py:308): DBG0: Total memory consumed:
            GC used:            57.5MB (peak: 69.6MB)
               in arenas:            39.9MB
               rawmalloced:          14.6MB
               nursery:              3.0MB
            raw assembler used: 1.5MB
            -----------------------------
            Total:              58.9MB

            Total memory allocated:
            GC allocated:            63.1MB (peak: 71.2MB)
               in arenas:            43.9MB
               rawmalloced:          22.7MB
               nursery:              3.0MB
            raw assembler allocated: 2.0MB
            -----------------------------
            Total:                   65.1MB

How is this possible?

I am measuring RSS with:

def get_rss_mem_usage():
    '''
    Get the RSS memory usage in bytes
    @return: memory size in bytes; -1 if error occurs
    '''
    try:
        process = psutil.Process(os.getpid())
        return process.get_memory_info().rss
    except:
        return -1

And cross referencing with "ps -orss -p <pid>" and the RSS values reported are correct....

I cannot figure out where to go from here with this as it appears that PyPy is leaking this memory somehow...
And I have no idea how to proceed from here...
I end up having memory problems and getting Memory Warnings for a process that just loops and queries via pymongo
Pymongo version is 3.7.1

This is:

Python 2.7.13 (ab0b9caf307db6592905a80b8faffd69b39005b8, Apr 30 2018, 08:21:35)
[PyPy 6.0.0 with GCC 7.2.0]



From issues-reply at bitbucket.org  Thu Mar 28 14:34:50 2019
From: issues-reply at bitbucket.org (=?utf-8?q?Vilhj=C3=A1lmur_Thorsteinsson?=)
Date: Thu, 28 Mar 2019 18:34:50 +0000 (UTC)
Subject: [pypy-issue] Issue #2983: Regression in 7.2: readline not working
 (pypy/pypy)
Message-ID: <20190328183451.5318.52737@app-147.ash1.bb-inf.net>

New issue 2983: Regression in 7.2: readline not working
https://bitbucket.org/pypy/pypy/issues/2983/regression-in-72-readline-not-working

Vilhj?lmur Thorsteinsson:

In PyPy 7.2 (Python 3.6 beta), I get a UserWarning: readline.read_init_file() not implemented upon startup. This was fine in 7.0. REPL cursor operations subsequently behave funnily, e.g. appending '\u0000' to lines when pressing cursor-up or cursor-down.



From issues-reply at bitbucket.org  Fri Mar 29 10:41:12 2019
From: issues-reply at bitbucket.org (Sven-Hendrik Haase)
Date: Fri, 29 Mar 2019 14:41:12 +0000 (UTC)
Subject: [pypy-issue] Issue #2984: pypy3-7.1.0 doesn't build (pypy/pypy)
Message-ID: <20190329144112.37422.11127@app-137.ash1.bb-inf.net>

New issue 2984: pypy3-7.1.0 doesn't build
https://bitbucket.org/pypy/pypy/issues/2984/pypy3-710-doesnt-build

Sven-Hendrik Haase:

I'm the maintainer for pypy3 in Arch Linux and it doesn't appear to build. The release candidate did build. Log attached.



From issues-reply at bitbucket.org  Sun Mar 31 15:33:59 2019
From: issues-reply at bitbucket.org (Anthony Sottile)
Date: Sun, 31 Mar 2019 19:33:59 +0000 (UTC)
Subject: [pypy-issue] Issue #2985: pypy3.6: `os.replace(...,
 pathlike)`: TypeError: 'PosixPath' does not support the buffer
 interface (pypy/pypy)
Message-ID: <20190331193359.34444.1375@celery-worker-111.ash1.bb-inf.net>

New issue 2985: pypy3.6: `os.replace(..., pathlike)`: TypeError: 'PosixPath' does not support the buffer interface
https://bitbucket.org/pypy/pypy/issues/2985/pypy36-osreplace-pathlike-typeerror

Anthony Sottile:

### sample script

```python
import os
import pathlib

open('a', 'a+').close()
os.replace('a', pathlib.Path('b'))
```

### pypy output

```console
$ ./pypy3.6-v7.1.0-linux64/bin/pypy3 --version
Python 3.6.1 (de061d87e39c, Mar 24 2019, 22:18:07)
[PyPy 7.1.0-beta0 with GCC 6.2.0 20160901]
$ ./pypy3.6-v7.1.0-linux64/bin/pypy3 x.py 
Traceback (most recent call last):
  File "x.py", line 5, in <module>
    os.replace('a', pathlib.Path('b'))
TypeError: 'PosixPath' does not support the buffer interface
```

### cpython output

```console
$ python3.6 --version --version
Python 3.6.7 (default, Oct 22 2018, 11:32:17) 
[GCC 8.2.0]
$ python3.6 x.py 
$
```

### misc

this was noticed while attempting to use https://github.com/ambv/black using pypy3.6

### guess

my guess is this hasn't been updated from the changes in python3.6 as the error message matches exactly what is happening on pypy3.5:

```
$ pypy3 --version
Python 3.5.3 (fdd60ed87e94, Apr 24 2018, 06:10:04)
[PyPy 6.0.0 with GCC 6.2.0 20160901]
$ pypy3 x.py 
Traceback (most recent call last):
  File "x.py", line 5, in <module>
    os.replace('a', pathlib.Path('b'))
TypeError: 'PosixPath' does not support the buffer interface

$ python3.5 --version --version
Python 3.5.7
$ python3.5 x.py 
Traceback (most recent call last):
  File "x.py", line 5, in <module>
    os.replace('a', pathlib.Path('b'))
TypeError: replace: illegal type for dst parameter
```



From issues-reply at bitbucket.org  Sun Mar 31 18:46:13 2019
From: issues-reply at bitbucket.org (Paul G)
Date: Sun, 31 Mar 2019 22:46:13 +0000 (UTC)
Subject: [pypy-issue] Issue #2986: cpyext missing FromTimestamp methods in
 PyDatetime_CAPI (pypy/pypy)
Message-ID: <20190331224612.18044.72129@celery-worker-111.ash1.bb-inf.net>

New issue 2986: cpyext missing FromTimestamp methods in PyDatetime_CAPI
https://bitbucket.org/pypy/pypy/issues/2986/cpyext-missing-fromtimestamp-methods-in

Paul G:

The [current implementation of cpyext](https://bitbucket.org/pypy/pypy/src/3c18b70c56bdf83522557872bd49e1f81dde36a5/pypy/module/cpyext/parse/cpyext_datetime.h?at=default&fileviewer=file-view-default#cpyext_datetime.h-2) is missing two constructors in the `PyDatetime_CAPI` struct that were present [in Python 3.5](https://github.com/python/cpython/blob/3.5/Include/datetime.h#L146):

- `PyDateTime_CAPI->DateTime_FromTimestamp`
- `PyDateTime_CAPI->Date_FromTimestamp`

The [relevant "macro" implementations](https://bitbucket.org/pypy/pypy/src/3c18b70c56bdf83522557872bd49e1f81dde36a5/pypy/module/cpyext/cdatetime.py?at=default#lines-240) are present, they just need to be added to the struct.

This came up [when adding PyPy support to PyO3](https://github.com/PyO3/pyo3/pull/393/files#r268716309).



From issues-reply at bitbucket.org  Sun Mar 31 18:58:31 2019
From: issues-reply at bitbucket.org (Paul G)
Date: Sun, 31 Mar 2019 22:58:31 +0000 (UTC)
Subject: [pypy-issue] Issue #2987: cpyext datetime bindings missing support
 for fold in Python 3.6 (pypy/pypy)
Message-ID: <20190331225831.7239.77129@app-147.ash1.bb-inf.net>

New issue 2987: cpyext datetime bindings missing support for fold in Python 3.6
https://bitbucket.org/pypy/pypy/issues/2987/cpyext-datetime-bindings-missing-support

Paul G:

Part of CPython's [PEP 495](https://www.python.org/dev/peps/pep-0495/) implementation was to add [two new constructors to the `PyDateTime_CAPI` struct](https://github.com/python/cpython/blob/3.6/Include/datetime.h#L169):

- `DateTime_FromDateAndTimeAndFold`
- `Time_FromTimeAndFold`

This also has corresponding macros:

- [`PyDateTime_FromDateAndTimeAndFold`](https://github.com/python/cpython/blob/3.6/Include/datetime.h#L229)
- [`PyTime_FromTimeAndFold`](https://github.com/python/cpython/blob/3.6/Include/datetime.h#L237)

The current implementation of [`cdatetime.py`](https://bitbucket.org/pypy/pypy/src/default/pypy/module/cpyext/cdatetime.py) and [`cpyext_datetime.h`](https://bitbucket.org/pypy/pypy/src/default/pypy/module/cpyext/parse/cpyext_datetime.h) should be updated to include these.

This came up when [adding PyPy support to PyO3](https://github.com/PyO3/pyo3/pull/393#issuecomment-477755187).