From issues-reply at bitbucket.org  Tue Sep  4 06:51:22 2018
From: issues-reply at bitbucket.org (Armin Rigo)
Date: Tue, 04 Sep 2018 10:51:22 +0000 (UTC)
Subject: [pypy-issue] Issue #2881: List strategies: switching to object can
 create more memory pressure than expected (pypy/pypy)
Message-ID: <20180904105121.20344.68808@celery-worker-105.ash1.bb-inf.net>

New issue 2881: List strategies: switching to object can create more memory pressure than expected
https://bitbucket.org/pypy/pypy/issues/2881/list-strategies-switching-to-object-can

Armin Rigo:

Here's a case where memory pressure increases much more than naively expected:

```
#! python

lst = [0] * (10**8)
lst[0] = 'X!'
```

That's because the switch to the object strategy will create 100 million new copies of the object ``0``, whereas on CPython the list contains 100 million references to the same object ``0``.



From issues-reply at bitbucket.org  Tue Sep  4 06:54:20 2018
From: issues-reply at bitbucket.org (Giovanni DG)
Date: Tue, 04 Sep 2018 10:54:20 +0000 (UTC)
Subject: [pypy-issue] Issue #2882: PyPy3+ numpy in a docker container with
 Jupyter, test errors (pypy/pypy)
Message-ID: <20180904105420.21176.25313@celery-worker-110.ash1.bb-inf.net>

New issue 2882: PyPy3+ numpy  in a docker container with Jupyter, test errors
https://bitbucket.org/pypy/pypy/issues/2882/pypy3-numpy-in-a-docker-container-with

Giovanni DG:

I am trying to encapsulate PyPy with numpy ina [docker container]( http://hub.docker.com/r/giodegas/pypy-jupyter).

I run the test with:

    pytest --pyargs --continue-on-collection-errors numpy > numpy_pytest.log
 
which terminates with 

    ======= 9 failed, 4643 passed, 445 skipped, 9 xfailed in 1379.38 seconds =======

I included the numpy_pytest.log



From issues-reply at bitbucket.org  Sun Sep  9 06:50:47 2018
From: issues-reply at bitbucket.org (Vincent Pelletier)
Date: Sun, 09 Sep 2018 10:50:47 +0000 (UTC)
Subject: [pypy-issue] Issue #2883: Bytecode contains
 jump-to-unconditional-jump (pypy/pypy)
Message-ID: <20180909105047.35882.54343@celery-worker-105.ash1.bb-inf.net>

New issue 2883: Bytecode contains jump-to-unconditional-jump
https://bitbucket.org/pypy/pypy/issues/2883/bytecode-contains-jump-to-unconditional

Vincent Pelletier:

Found while comparing cPython 2.7.15 with pypy 6.0.0.
This seems suboptimal, any jump (conditional or not) jumping to an interrupted chain of unconditional jumps should be possible to simplify by updating original instruction to jump to the last element in that chain, modulo absolute/relative addressing considerations. In below example, 19 jump to 29 which is an unconditional jump.

I tried poking at pypy/astcompiler/assemble.py:_resolve_block_targets but could not get good enough results. I am not sure all the code is visible to it (code-less blocks ?).

Also, "<n> JUMP <n+1>" (below, 29 jumps to 32) instructions should be possible to remove entirely (which may require updating anything jumping to that instruction).

$ python -c "import dis, t; dis.dis(t)" > cpython.t.pydis
$ pypy -c "import dis, t; dis.dis(t)" > pypy.t.pydis
$ diff -u cpython.t.pydis pypy.t.pydis
--- cpython.t.pydis     2018-09-09 06:03:51.763765156 +0000
+++ pypy.t.pydis        2018-09-09 06:04:02.987767118 +0000
@@ -16,12 +16,12 @@
  10          12 LOAD_GLOBAL              0 (bar)
              15 CALL_FUNCTION            0
              18 POP_TOP
-             19 JUMP_ABSOLUTE           32
+             19 JUMP_FORWARD             7 (to 29)
 
  12     >>   22 LOAD_GLOBAL              1 (baz)
              25 CALL_FUNCTION            0
              28 POP_TOP
-             29 JUMP_FORWARD             0 (to 32)
+        >>   29 JUMP_FORWARD             0 (to 32)
         >>   32 LOAD_CONST               0 (None)
              35 RETURN_VALUE
 
$ cat t.py
def bar():
    pass
 
def baz():
    pass
 
def foo(a, b):
  if a:
    if b:
        bar()
    else:
        baz()



From issues-reply at bitbucket.org  Mon Sep 10 03:51:01 2018
From: issues-reply at bitbucket.org (Gary Wang)
Date: Mon, 10 Sep 2018 07:51:01 +0000 (UTC)
Subject: [pypy-issue] Issue #2884: [3.6] PythonCodeGenerator fails on
 annotated list comprehensions assignments (pypy/pypy)
Message-ID: <20180910075101.30766.78294@celery-worker-107.ash1.bb-inf.net>

New issue 2884: [3.6] PythonCodeGenerator fails on annotated list comprehensions assignments
https://bitbucket.org/pypy/pypy/issues/2884/36-pythoncodegenerator-fails-on-annotated

Gary Wang:

CPython 3.6:

```
$ python3.6 -c 'a: list = [i for i in range(10)]; print(a)'
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
```

PyPy 3.6 nightly:

```
$ ./pypy3 -c 'a: list = [i for i in range(10)]'
RPython traceback:
  File "pypy_interpreter.c", line 33880, in BuiltinCode_funcrun_obj
  File "pypy_module___builtin__.c", line 24006, in exec_
  File "pypy_interpreter_2.c", line 35793, in PyFrame_exec_
  File "pypy_interpreter.c", line 53550, in PythonAstCompiler__compile_ast
  File "pypy_interpreter_astcompiler.c", line 1544, in compile_ast
  File "pypy_interpreter_astcompiler.c", line 5965, in Module_walkabout
  File "pypy_interpreter_astcompiler.c", line 13374, in PythonCodeGenerator__handle_body
  File "pypy_interpreter_astcompiler_2.c", line 10804, in PythonCodeGenerator_visit_AnnAssign
  File "pypy_interpreter_astcompiler_2.c", line 15234, in PythonCodeGenerator_visit_ListComp
  File "pypy_interpreter_astcompiler_2.c", line 37308, in PythonCodeGenerator__compile_comprehension
  File "pypy_interpreter_astcompiler_2.c", line 35667, in PythonCodeGenerator_sub_scope
  File "pypy_interpreter_astcompiler.c", line 11360, in PythonCodeGenerator___init__
SystemError: unexpected internal exception (please report a bug): <KeyError object at 0x7fcc4f8c65c0>; internal traceback was dumped to stderr
```

Annotated variable assignments without comprehensions are fine:

```
$ ./pypy3 -c 'a: list = [1, 2, 3]; print(a)'
[1, 2, 3]
```



From issues-reply at bitbucket.org  Mon Sep 10 14:37:46 2018
From: issues-reply at bitbucket.org (Ryan Hileman)
Date: Mon, 10 Sep 2018 18:37:46 +0000 (UTC)
Subject: [pypy-issue] Issue #2885: Document pypy3's use of keyed hash
 function. (pypy/pypy)
Message-ID: <20180910183745.37904.86917@celery-worker-111.ash1.bb-inf.net>

New issue 2885: Document pypy3's use of keyed hash function.
https://bitbucket.org/pypy/pypy/issues/2885/document-pypy3s-use-of-keyed-hash-function

Ryan Hileman:

The docs claim PyPy ignores hash randomization:

http://doc.pypy.org/en/latest/cpython_differences.html#miscellaneous

> Hash randomization (-R) is ignored in PyPy. In CPython before 3.4 it has little point.

However, after asking on IRC, it appears pypy3 *does* use SipHash by default (presumably with a randomized key), but still ignores the -R flag (which doesn't matter because it's on by default).

mjacob from IRC:
> we didn't follow CPython in implementing their first (unsuccessful) attempt to fix the problem
> however we followed CPython in implementing the second attempt to fix the problem with a cryptographic hash function
> (see also sys.hash_info)

I think the cpython_differences doc should be updated to say this is not an issue in pypy3 (if this information is correct).



From issues-reply at bitbucket.org  Tue Sep 11 17:57:12 2018
From: issues-reply at bitbucket.org (Gary Wang)
Date: Tue, 11 Sep 2018 21:57:12 +0000 (UTC)
Subject: [pypy-issue] Issue #2886: [3.6] SyntaxError on float literals with
 underscores in the fractional part (pypy/pypy)
Message-ID: <20180911215712.33859.6464@app-147.ash1.bb-inf.net>

New issue 2886: [3.6] SyntaxError on float literals with underscores in the fractional part
https://bitbucket.org/pypy/pypy/issues/2886/36-syntaxerror-on-float-literals-with

Gary Wang:

CPython 3.6:
```
$ python3.6 -c 'print(0.00_2)'
0.002
```

PyPy 3.6 nightly:
```
$ ./pypy3 -c 'print(0.00_2)'
  File "<string>", line 1
    print(0.00_2)
              ^
SyntaxError: invalid syntax (expected ')')
```



From issues-reply at bitbucket.org  Thu Sep 13 04:56:17 2018
From: issues-reply at bitbucket.org (Tinho Lee)
Date: Thu, 13 Sep 2018 08:56:17 +0000 (UTC)
Subject: [pypy-issue] Issue #2887: CFFI call with bool is much slower than
 int (pypy/pypy)
Message-ID: <20180913085617.35808.37858@app-147.ash1.bb-inf.net>

New issue 2887: CFFI call with bool is much slower than int
https://bitbucket.org/pypy/pypy/issues/2887/cffi-call-with-bool-is-much-slower-than

Tinho Lee:

Here is a simple test case, where *x* is a pre-compiled cffi module, and *empty* is just a empty c function with signature *void empty(bool, bool, bool);*.

test_a costs about 0.22s, while test_b 0.17s. It seems passing python bool to c bool_t is slower than passing python int. 

```
#!python

def test_a():
	for n in xrange(10000000):
		x.lib.empty(True, True, True)

def test_b():
	for n in xrange(10000000):
		x.lib.empty(1, 1, 1)

```

I have read the code *convert_from_object* of *W_CTypePrimitiveBool* in ctypeprim.py, which calls *misc.as_unsigned_long* for conversion. 

There is a shortcut for *W_IntObject* inside the method *as_unsigned_long*, however, *W_BoolObject* would be treated as big int for a much slower conversion. It seems some optimization could help to improve the performance of passing python bool.



From issues-reply at bitbucket.org  Thu Sep 13 07:33:37 2018
From: issues-reply at bitbucket.org (Armin Rigo)
Date: Thu, 13 Sep 2018 11:33:37 +0000 (UTC)
Subject: [pypy-issue] Issue #2888: PyPy3: try to turn W_LongObject to
 W_IntObject more often (pypy/pypy)
Message-ID: <20180913113337.33793.54934@celery-worker-108.ash1.bb-inf.net>

New issue 2888: PyPy3: try to turn W_LongObject to W_IntObject more often
https://bitbucket.org/pypy/pypy/issues/2888/pypy3-try-to-turn-w_longobject-to

Armin Rigo:

To save memory, it would be useful to figure out some operations that generally give an integer that fits in a machine word, even if the operation involves large integers.  The most obvious example is ``long % int``.  On PyPy2 this must always give a ``long``, but there is no such condition on PyPy3.



From issues-reply at bitbucket.org  Sat Sep 15 11:04:03 2018
From: issues-reply at bitbucket.org (Tinho Lee)
Date: Sat, 15 Sep 2018 15:04:03 +0000 (UTC)
Subject: [pypy-issue] Issue #2889: Unnecessary tuple creation when passing
 string object with CFFI and jit enabled (pypy/pypy)
Message-ID: <20180915150403.35174.52364@celery-worker-107.ash1.bb-inf.net>

New issue 2889: Unnecessary tuple creation when passing string object with CFFI and jit enabled
https://bitbucket.org/pypy/pypy/issues/2889/unnecessary-tuple-creation-when-passing

Tinho Lee:

It seems some optimization could help to improve the performance of passing python string to c library through cffi.

*get_nonmovingbuffer* is called when raw char pointer is needed. The annotation *always_inline* is set in order to get rid of the returned tuple, however, this function is decorated by *jit.dont_look_inside*, so the pypy_tuple cannot be eliminated when jit is in operation. I have confirmed *get_nonmovingbuffer*, instead of the inline version, is called with linux perf and jit log.

```
#!python

@jit.dont_look_inside
def get_nonmovingbuffer(data):
    lldata = llstrtype(data)
    count = len(data)

    if rgc.must_split_gc_address_space():
        flag = '\x06'    # always make a copy in this case
    elif we_are_translated_to_c() and not rgc.can_move(data):
        flag = '\x04'    # no copy needed
    else:
        if we_are_translated_to_c() and rgc.pin(data):
            flag = '\x05'     # successfully pinned
        else:
            flag = '\x06'     # must still make a copy
    if flag == '\x06':
        buf = lltype.malloc(TYPEP.TO, count + (TYPEP is CCHARP),
                            flavor='raw')
        copy_string_to_raw(lldata, buf, 0, count)
        return buf, '\x06'

    data_start = cast_ptr_to_adr(lldata) + \
        offsetof(STRTYPE, 'chars') + itemoffsetof(STRTYPE.chars, 0)

    return cast(TYPEP, data_start), flag

get_nonmovingbuffer._always_inline_ = 'try' # get rid of the returned tuple
get_nonmovingbuffer._annenforceargs_ = [strtype]

```

I don't think the returned tuple is necessary in this case. Maybe something like change the implementation into two function calls would help to remove the memory allocation of returned value, which decreases the total gc object. 

```
#!python

def get_nonmovingbuffer_flag(data)
def get_nonmovingbuffer_buf(data, flag)
```



From issues-reply at bitbucket.org  Mon Sep 17 10:17:42 2018
From: issues-reply at bitbucket.org (mattip)
Date: Mon, 17 Sep 2018 14:17:42 +0000 (UTC)
Subject: [pypy-issue] Issue #2890: missing os.sched* functionality
 (pypy/pypy)
Message-ID: <20180917141742.39934.949@celery-worker-112.ash1.bb-inf.net>

New issue 2890: missing os.sched* functionality
https://bitbucket.org/pypy/pypy/issues/2890/missing-ossched-functionality

mattip:

Apparently CPython added an interface to the [os scheduler](https://docs.python.org/3/library/os.html#interface-to-the-scheduler) in python 3.3, we lack support for it in posix. Tests in `lib-python/3/test/test_posix.py` are silently skipped, even on linux where it should exist.

This turns out is used in [CI testing](https://github.com/scikit-learn/scikit-learn/pull/12039#discussion_r218074306) to determine how many cpus can be used.

Since this means someone is trying to get PyPy into their test matrix, we should try to encourage them.



From issues-reply at bitbucket.org  Wed Sep 19 04:04:32 2018
From: issues-reply at bitbucket.org (Hong)
Date: Wed, 19 Sep 2018 08:04:32 +0000 (UTC)
Subject: [pypy-issue] Issue #2891: PyEval_GetFrame is unavailable in the C
 API (pypy/pypy)
Message-ID: <20180919080432.38198.31271@celery-worker-105.ash1.bb-inf.net>

New issue 2891: PyEval_GetFrame is unavailable in the C API
https://bitbucket.org/pypy/pypy/issues/2891/pyeval_getframe-is-unavailable-in-the-c

Hong:

While I understand that PyPy is stackless, this function is in the standard Python C API ([here](https://docs.python.org/3.5/c-api/reflection.html)) and may cause compilation errors. Do you think it's a better idea to have this function defined but it always returns NULL?



From issues-reply at bitbucket.org  Wed Sep 19 07:28:24 2018
From: issues-reply at bitbucket.org (Andrew Lawrence)
Date: Wed, 19 Sep 2018 11:28:24 +0000 (UTC)
Subject: [pypy-issue] Issue #2892: Unable to build on windows with multiple
 versions of visual studio installed (pypy/pypy)
Message-ID: <20180919112824.38052.40296@celery-worker-107.ash1.bb-inf.net>

New issue 2892: Unable to build on windows with multiple versions of visual studio installed
https://bitbucket.org/pypy/pypy/issues/2892/unable-to-build-on-windows-with-multiple

Andrew Lawrence:

I have Visual Studio 2010 and 2017 installed on my machine. pypy detects different versions during different stage of the builds. I am under the impression that Visual Studio 2010 is no longer supported and should not be used by the build environment

The get_externals script correctly detects the 2017 version.

When I perform the translation of pypy with the following command the 2010 version is detected.
C:\pypy\pypy\goal>C:\pypy2\pypy2-v6.0.0-win32\pypy2-v6.0.0-win32\pypy.exe ../../
rpython/bin/rpython --opt=jit

```
#!

C:\pypy\pypy\goal>C:\pypy2\pypy2-v6.0.0-win32\pypy2-v6.0.0-win32\pypy.exe ../..
rpython/bin/rpython --opt=jit
Initializing msvc platform
cc is none, finding msvc environment
getting mscvc env
getting mscvc env
getting mscvc env
getting mscvc env
[platform:msg] Updated environment with vsver 100, using x64 False
msvc environment found
[translation:info] 2.7.13 (ab0b9caf307d, Apr 24 2018, 17:52:36)
[PyPy 6.0.0 with MSC v.1500 32 bit]
Initializing msvc platform
cc is none, finding msvc environment
getting mscvc env
getting mscvc env
getting mscvc env
getting mscvc env
[platform:msg] Updated environment with vsver 100, using x64 False
msvc environment found
[platform:msg] Set platform with 'host' cc=None, using cc='cl.exe', version=100
[translation:info] Translating target as defined by targetpypystandalone

```

I have tried to set the version of Visual Studio using the CC environment variable but this does not work due to the pypy rejecting some of the spaces in the name.



From issues-reply at bitbucket.org  Wed Sep 19 09:11:31 2018
From: issues-reply at bitbucket.org (John Aldis)
Date: Wed, 19 Sep 2018 13:11:31 +0000 (UTC)
Subject: [pypy-issue] Issue #2893: CPython getargs.c updated for Python 3.6
 (pypy/pypy)
Message-ID: <20180919131131.13563.66978@celery-worker-107.ash1.bb-inf.net>

New issue 2893: CPython getargs.c updated for Python 3.6
https://bitbucket.org/pypy/pypy/issues/2893/cpython-getargsc-updated-for-python-36

John Aldis:

`pypy`'s `getargs.c` is a copy of the same file from CPython, but this file has been updated for 3.6, which leads to test failures. For example, the occurrences of `PyExc_RuntimeError` have been replaced with `PyExc_SystemError`, `PyExc_TypeError` etc. See https://github.com/python/cpython/blob/3.6/Python/getargs.c and compare with https://bitbucket.org/pypy/pypy/src/default/pypy/module/cpyext/src/getargs.c



From issues-reply at bitbucket.org  Fri Sep 21 13:52:43 2018
From: issues-reply at bitbucket.org (Andrew Lawrence)
Date: Fri, 21 Sep 2018 17:52:43 +0000 (UTC)
Subject: [pypy-issue] Issue #2894: MSI installer on Windows (pypy/pypy)
Message-ID: <20180921175243.27982.68280@app-148.ash1.bb-inf.net>

New issue 2894: MSI installer on Windows
https://bitbucket.org/pypy/pypy/issues/2894/msi-installer-on-windows

Andrew Lawrence:

Would it be possible to have an installer for Windows?



From issues-reply at bitbucket.org  Thu Sep 27 15:06:39 2018
From: issues-reply at bitbucket.org (Creation Elemental)
Date: Thu, 27 Sep 2018 19:06:39 +0000 (UTC)
Subject: [pypy-issue] Issue #2895: Shutil.move Rpython assertion error
 (pypy/pypy)
Message-ID: <20180927190639.7768.7844@celery-worker-109.ash1.bb-inf.net>

New issue 2895: Shutil.move Rpython assertion error
https://bitbucket.org/pypy/pypy/issues/2895/shutilmove-rpython-assertion-error

Creation Elemental:

If you use shutil.move and the resulting path name is too long, it will raise an Rpython error that will crash the entire interpreter instead of raising any kind of exception. Was really annoying to figure out when this happened in a really large script and would not print out any kind of traceback to tell me where the error was happening. It took quite a bit of debugging and testing to determine the line where the issue was occuring, and the fact that it was being caused by a file path being too long. Trying to manually do the copy outside of pypy told me the pathname was too long.



From issues-reply at bitbucket.org  Fri Sep 28 10:44:56 2018
From: issues-reply at bitbucket.org (Peter Cock)
Date: Fri, 28 Sep 2018 14:44:56 +0000 (UTC)
Subject: [pypy-issue] Issue #2896: pypy3 -m ensurepip fails on macOS,
 Symbol not found: _utimensat (pypy/pypy)
Message-ID: <20180928144455.40604.77514@app-147.ash1.bb-inf.net>

New issue 2896: pypy3 -m ensurepip fails on macOS, Symbol not found: _utimensat
https://bitbucket.org/pypy/pypy/issues/2896/pypy3-m-ensurepip-fails-on-macos-symbol

Peter Cock:

Setup - macOS Sierra, version 10.12.6

```
$ cd /tmp
$ curl -O -L https://bitbucket.org/pypy/pypy/downloads/pypy3-v6.0.0-osx64.tar.bz2
$ md5 pypy3-v6.0.0-osx64.tar.bz2 
MD5 (pypy3-v6.0.0-osx64.tar.bz2) = 9dfb0512570c95c247993687111bf980
$ shasum -a 256 pypy3-v6.0.0-osx64.tar.bz2 
938b8034e30f5f5060d2a079070c56c3be5559bc7ae9cc0c8395fe6fc45cfe4c  pypy3-v6.0.0-osx64.tar.bz2
$ tar -zxvf pypy3-v6.0.0-osx64.tar.bz2
```

To reproduce the problem:

```
$ pypy3-v6.0.0-osx64/bin/pypy3 -m ensurepip
Collecting setuptools
Collecting pip
Installing collected packages: setuptools, pip
dyld: lazy symbol binding failed: Symbol not found: _utimensat
  Referenced from: /private/tmp/pypy3-v6.0.0-osx64/bin/libpypy3-c.dylib
  Expected in: flat namespace

dyld: Symbol not found: _utimensat
  Referenced from: /private/tmp/pypy3-v6.0.0-osx64/bin/libpypy3-c.dylib
  Expected in: flat namespace

Abort trap: 6
```

Expected result - ensure that pip is installed on pypy3

(The same procedure works fine with https://bitbucket.org/pypy/pypy/downloads/pypy2-v6.0.0-osx64.tar.bz2 on this machine)



From issues-reply at bitbucket.org  Fri Sep 28 11:44:49 2018
From: issues-reply at bitbucket.org (Juergen Breuel)
Date: Fri, 28 Sep 2018 15:44:49 +0000 (UTC)
Subject: [pypy-issue] Issue #2897: VCRUNTIME140.dll not found system error
 on Windows 10 (pypy/pypy)
Message-ID: <20180928154449.10478.35238@celery-worker-112.ash1.bb-inf.net>

New issue 2897: VCRUNTIME140.dll not found system error on Windows 10
https://bitbucket.org/pypy/pypy/issues/2897/vcruntime140dll-not-found-system-error-on

Juergen Breuel:

Executing pypy3.exe in command line from pypy3-v6.0.0-win32.zip (cmd.exe) on Windows 10 gives a VCRUNTIME140.dll not found system error.![Unbenannt.PNG](https://bitbucket.org/repo/R7AbB/images/934537304-Unbenannt.PNG)