From Anthony at bitbucket.org  Tue Jun  4 14:42:31 2019
From: Anthony at bitbucket.org (Anthony at bitbucket.org)
Date: Tue, 04 Jun 2019 18:42:31 +0000 (UTC)
Subject: [pypy-issue] Issue #3021: `io.open(directory)` leaks a file
 descriptor (pypy/pypy)
Message-ID: <20190604184230.39954.53135@celery-worker-110.ash1.bb-inf.net>

New issue 3021: `io.open(directory)` leaks a file descriptor
https://bitbucket.org/pypy/pypy/issues/3021/ioopen-directory-leaks-a-file-descriptor

Anthony Sottile:

This is the minimal setup that I?ve been able to reproduce this with:

?

```python
import io
import pytest
import warnings

def test():
    for _ in range(15000):
        with pytest.raises(IsADirectoryError):
            with io.open('.'):
                pass


with warnings.catch_warnings(record=True) as wrns:
    warnings.simplefilter('always')
    test()

if len(wrns):
    print('*' * 79)
    print('warnings: {}'.format(len(wrns)))
    print('first warning:')
    print('{}:{}:{}'.format(wrns[0].filename, wrns[0].lineno, wrns[0].message))
    print('*' * 79)
```

?

On my machine the fd limits are pretty low so I reproduced this easier in docker or as root with `ulimit -n ...`

```shell
(venvpp) root at asottile-MacBookPro:/tmp# python  t.py
*******************************************************************************
warnings: 10422
first warning:
t.py:8:unclosed file <_io.FileIO fd=3 mode='rb' closefd=True>
*******************************************************************************
```

?

I?ve tried the following and both have reproduced:

```
# python --version
Python 3.6.1 (784b254d6699, Apr 14 2019, 10:22:42)
[PyPy 7.1.1-beta0 with GCC 6.2.0 20160901]
```

```
# /venv/bin/python --version
Python 3.5.3 (928a4f70d3de, Feb 08 2019, 10:42:58)
[PyPy 7.0.0 with GCC 6.2.0 20160901]
```

These are the packages I?ve been using, though it doesn?t appear to make much difference:

```
# pip freeze -l
atomicwrites==1.3.0
attrs==19.1.0
importlib-metadata==0.17
more-itertools==7.0.0
packaging==19.0
pluggy==0.12.0
py==1.8.0
pyparsing==2.4.0
pytest==4.6.2
six==1.12.0
wcwidth==0.1.7
zipp==0.5.1
```

Original investigation was done here: [https://github.com/nedbat/coveragepy/issues/807](https://github.com/nedbat/coveragepy/issues/807)

Related: [https://github.com/pytest-dev/pytest/issues/5342](https://github.com/pytest-dev/pytest/issues/5342)

I cannot reproduce this with pypy2.x:

```
$ pypy --version
Python 2.7.13 (8cdda8b8cdb8, Apr 14 2019, 14:06:44)
[PyPy 7.1.1 with GCC 6.2.0 20160901]
```



From Jan at bitbucket.org  Tue Jun  4 19:35:24 2019
From: Jan at bitbucket.org (Jan at bitbucket.org)
Date: Tue, 04 Jun 2019 23:35:24 +0000 (UTC)
Subject: [pypy-issue] Issue #3022: pypy3 fails to deepcopy nested ctype
 structures (pypy/pypy)
Message-ID: <20190604233524.15238.29900@celery-worker-110.ash1.bb-inf.net>

New issue 3022: pypy3 fails to deepcopy nested ctype structures
https://bitbucket.org/pypy/pypy/issues/3022/pypy3-fails-to-deepcopy-nested-ctype

Jan Vesely:

```python
import ctypes, copy

class struct_a(ctypes.Structure):
    pass
    
class struct_b(ctypes.Structure):
    pass
    
struct_a._fields_ = [('first',struct_b)]

a = struct_a()
b = copy.deepcopy(a)
```

results in:

```
$ pypy3 ctype-struct.py 
Traceback (most recent call last):
  File "ctype-struct.py", line 12, in <module>
    b = copy.deepcopy(a)
  File "/usr/lib/pypy3.6/lib-python/3/copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/usr/lib/pypy3.6/lib-python/3/copy.py", line 280, in _reconstruct
    state = deepcopy(state, memo)
  File "/usr/lib/pypy3.6/lib-python/3/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/usr/lib/pypy3.6/lib-python/3/copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/lib/pypy3.6/lib-python/3/copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/usr/lib/pypy3.6/lib-python/3/copy.py", line 274, in _reconstruct
    y = func(*args)
  File "/usr/lib/pypy3.6/lib-python/3/copyreg.py", line 88, in __newobj__
    return cls.__new__(cls, *args)
TypeError: object.__new__(StructureInstanceAutoFree) is not safe, use StructureInstanceAutoFree.__new__()
```



From CTimmerman2 at bitbucket.org  Wed Jun  5 07:34:25 2019
From: CTimmerman2 at bitbucket.org (CTimmerman2 at bitbucket.org)
Date: Wed, 05 Jun 2019 11:34:25 +0000 (UTC)
Subject: [pypy-issue] Issue #3023: CPython 3.6.8 faster than PyPy
 3.6.1-7.1.1-beta0 (pypy/pypy)
Message-ID: <20190605113425.37444.10335@app-147.ash1.bb-inf.net>

New issue 3023: CPython 3.6.8 faster than PyPy 3.6.1-7.1.1-beta0
https://bitbucket.org/pypy/pypy/issues/3023/cpython-368-faster-than-pypy-361-711-beta0

Cees Timmerman:

This performance test should be faster in PyPy according to [https://pypy.org/performance.html](https://pypy.org/performance.html) as it runs Python code \(including SymPy\) multiple times and takes many seconds to complete.

```
"""SymPy server by Cees Timmerman, 2019-05-16.
Inspired by https://stackoverflow.com/questions/56155624/speeding-up-python-c-call/
"""
import re

import falcon


whitelist = re.compile("(?:[ \d.,<>\[\]()*\/+-]|for|in|Intersection|solveset|S|x|Min|Max|p|Reals)+")

def eval_sympy(q):
	"""
	Handle stuff like:
	>>> eval_sympy("Intersection(*[solveset(p, x, S.Reals) for p in [(x > 4.0000), (x < 6.0000), (((x) * 4.0000 + 5.0000) > 5.0000)]])")
	Interval.open(4.00000000000000, 6.00000000000000)
	>>> eval_sympy("Intersection(*[solveset(p, x, S.Reals) for p in [(x > 4.0000), (x < 6.0000), ((x * (Min(Max(x, 4.0000), 5.0000))) > 7.0000), ((Min(Max(x, 4.0000), 5.0000)) > 5.0000)]])")
	EmptySet()
	>>> eval_sympy("")
	"Unauthorized string(s): ''"
	>>> eval_sympy("Min(p)")
	"ERR: name 'p' is not defined"
	>>> eval_sympy("1/0")
	'ERR: division by zero'
	>>> eval_sympy("__import__('os').system('clear')")
	'Unauthorized string(s): "__im ort__ \\'os\\' system \\'clear\\' "'
	>>> eval_sympy("().__class__.__bases__[0]")
	"Unauthorized string(s): ' __class__ __bases__ '"
	>>> eval_sympy("Max(evil), Min(good)")
	"Unauthorized string(s): ' evil good '"
	
	https://nedbatchelder.com/blog/201206/eval_really_is_dangerous.html
	"""
	from sympy import Intersection, solveset, S
	from sympy.abc import x
	from sympy.functions.elementary.miscellaneous import Min, Max
	# Whitelist allowed strings.
	if not re.fullmatch(whitelist, q):
		return "Unauthorized string(s): %r" % re.sub(whitelist, " ", q)
	# Blacklist builtins.
	env = {k:v for (k,v) in locals().items() if not '_' in k}
	try:
		return eval(q, env)
	except Exception as err:
		return "ERR: {}".format(err)

def test_performance():
	"""Best of 5:
	Python 3.6.8:
	local sympy imports: 9.31 s
	global sympy imports: 9.24 s
	PyPy 3.6.1-7.1.1-beta0:
	local sympy imports: 22.93 s
	global sympy imports: 23.55 s
	"""
	import timeit
	return timeit.timeit("""eval_sympy("Intersection(*[solveset(p, x, S.Reals) for p in [(x > 4.0000), (x < 6.0000), ((x * (Min(Max(x, 4.0000), 5.0000))) > 7.0000), ((Min(Max(x, 4.0000), 5.0000)) > 5.0000)]])")""", number=10, globals=globals())

class SymPyResource:
	def on_get(self, req, resp):
		"""
		>>> from falcon import testing; client = testing.TestClient(api)
		>>> r = client.simulate_get('/sympy'); r.status_code
		200
		>>> r.json
		{'result': "ERR: No 'q' parameter."}
		>>> client.simulate_get('/sympy?q=Max(1, 2)').json
		{'result': '2'}
		"""
		#q = req.get_param('q', required=True)  # Returns <error><title>Missing parameter</title><description>The "q" parameter is required.</description></error>
		if 'q' in req.params:
			q = str(req.params['q'])
			result = {"result": "%s" % eval_sympy(q)}
		else:
			result = {"result": "ERR: No 'q' parameter."}
		
		resp.media = result

api = falcon.API()
api.add_route('/sympy', SymPyResource())

if __name__ == '__main__':
	import sys
	if "test" in sys.argv:
		print("Testing Python", sys.version)
		import doctest
		result = doctest.testmod(optionflags=doctest.FAIL_FAST if "fast" in sys.argv else 0)  # Optionally stop testing after a single failure.
		print(result)
		if "perf" in sys.argv:
			seconds = test_performance()
			print("%ss" % seconds)
			sys.exit(seconds > 1)
		
		sys.exit(1 if result.failed else 0)
		
	from wsgiref import simple_server
	httpd = simple_server.make_server('127.0.0.1', 8000, api)
	print("Serving on http://%s:%s/sympy" % httpd.socket.getsockname())
	httpd.serve_forever()
```

[https://github.com/CTimmerman/SymPyServer/blob/master/sympy\_server.py](https://github.com/CTimmerman/SymPyServer/blob/master/sympy_server.py)



From Platon at bitbucket.org  Fri Jun  7 11:59:21 2019
From: Platon at bitbucket.org (Platon at bitbucket.org)
Date: Fri, 07 Jun 2019 15:59:21 +0000 (UTC)
Subject: [pypy-issue] Issue #3024: PyPy3 much slower than PyPy2 for decimal
 calculations (pypy/pypy)
Message-ID: <20190607155921.19712.12624@celery-worker-112.ash1.bb-inf.net>

New issue 3024: PyPy3 much slower than PyPy2 for decimal calculations
https://bitbucket.org/pypy/pypy/issues/3024/pypy3-much-slower-than-pypy2-for-decimal

Platon Pronko:

The following code takes much longer when using PyPy3 instead of PyPy2:

```python
from decimal import Decimal

s = Decimal("0")
d = Decimal("0.0000001")
i = 0
while i < 10000000:
    s += d
    i += 1
print(s)
```

Runtimes on my machine:

```
$ time pypy slowdown.py 
1.0000000

real	0m1.899s
user	0m1.850s
sys	0m0.040s
$ time pypy3 slowdown.py 
1.0000000

real	0m24.475s
user	0m24.305s
sys	0m0.054s
```

Versions:

```
$ pypy --version
Python 2.7.13 (8cdda8b8cdb8ff29d9e620cccd6c5edd2f2a23ec, Apr 23 2019, 17:07:51)
[PyPy 7.1.1 with GCC 8.3.0]
$ pypy3 --version
Python 3.6.1 (784b254d669919c872a505b807db8462b6140973, May 09 2019, 13:17:30)
[PyPy 7.1.1-beta0 with GCC 8.3.0]
```
OS: Arch Linux, `Linux 5.1.5-arch1-2-ARCH #1 SMP PREEMPT Mon May 27 03:37:39 UTC 2019 x86_64 GNU/Linux`



From Jan at bitbucket.org  Sun Jun  9 23:42:26 2019
From: Jan at bitbucket.org (Jan at bitbucket.org)
Date: Mon, 10 Jun 2019 03:42:26 +0000 (UTC)
Subject: [pypy-issue] Issue #3025: importing autograd module fails
 (pypy/pypy)
Message-ID: <20190610034226.36904.99888@app-137.ash1.bb-inf.net>

New issue 3025: importing autograd module fails
https://bitbucket.org/pypy/pypy/issues/3025/importing-autograd-module-fails

Jan Vesely:

```
Python 3.6.1 (784b254d669919c872a505b807db8462b6140973, Apr 21 2019, 14:47:54)
[PyPy 7.1.1-beta0 with GCC 8.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
And now for something completely different: ``"messy" is not a judgement, but
just a fact of complicatedness''
>>>> import autograd
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/orome/.local/lib/pypy3.6/site-packages/autograd/__init__.py", line 2, in <module>
    from .differential_operators import (
  File "/home/orome/.local/lib/pypy3.6/site-packages/autograd/differential_operators.py", line 12, in <module>
    import autograd.numpy as np
  File "/home/orome/.local/lib/pypy3.6/site-packages/autograd/numpy/__init__.py", line 5, in <module>
    from . import numpy_vjps
  File "/home/orome/.local/lib/pypy3.6/site-packages/autograd/numpy/numpy_vjps.py", line 130, in <module>
    defvjp(anp.where, None,
AttributeError: module 'autograd.numpy.numpy_wrapper' has no attribute 'where'
```

the function is exported in this way:

```
def wrap_namespace(old, new):
    unchanged_types = {float, int, type(None), type}
    int_types = {_np.int, _np.int8, _np.int16, _np.int32, _np.int64, _np.integer}
    function_types = {_np.ufunc, types.FunctionType, types.BuiltinFunctionType}
    for name, obj in old.items():
        if obj in notrace_functions:
            new[name] = notrace_primitive(obj)
        elif type(obj) in function_types:
            new[name] = primitive(obj)
        elif type(obj) is type and obj in int_types:
            new[name] = wrap_intdtype(obj)
        elif type(obj) in unchanged_types:
            new[name] = obj

wrap_namespace(_np.__dict__, globals())
```



From Aure at bitbucket.org  Mon Jun 10 03:53:25 2019
From: Aure at bitbucket.org (Aure at bitbucket.org)
Date: Mon, 10 Jun 2019 07:53:25 +0000 (UTC)
Subject: [pypy-issue] Issue #3026: struct format does not support space in
 struct.py (pypy/pypy)
Message-ID: <20190610075325.14465.39524@app-137.ash1.bb-inf.net>

New issue 3026: struct format does not support space in struct.py
https://bitbucket.org/pypy/pypy/issues/3026/struct-format-does-not-support-space-in

Aur?lien Lambert:

While trying to import pytz in brython \(which is using pypy pure python implementation of libraries\), I encountered a bug on `struct.py` \([https://bitbucket.org/pypy/pypy/src/18626459a9b2/lib\_pypy/\_struct.py](https://bitbucket.org/pypy/pypy/src/18626459a9b2/lib_pypy/_struct.py)\): spaces are not allowed in the format string. This behavior is inconsistent with python documentation \(?Whitespace characters between formats are ignored;?\) and does not happen with the compiled struct module.

```python
from test.struct import calcsize # local copy of pypy struct.py
calcsize('>4s c 15x 6l') # format from pytz

>>> StructError:   is not a valid format
```

This is due to the functions `getmode` and `getNum` not checking for whitespaces. This implementation of `getNum` gets rid of middle spaces.

```python
def getNum(fmt,i):
    num=None
    cur = fmt[i]
    while cur == ' ':
        i += 1
        cur = fmt[i]
    while ('0'<= cur ) and ( cur <= '9'):
        if num == None:
            num = int(cur)
        else:
            num = 10*num + int(cur)
        i += 1
        cur = fmt[i]
    return num,i
```

Getting rid of end spaces will be little trickier because all the function use `getNum` like this:

```python
    while i<len(fmt):
        num,i = getNum(fmt,i)
        cur = fmt[i]
        [...]
        i += 1
```

I suggest either to systematically use `fmt = fmt.rstrip()`, either to change the behavior of `getNum`

```python
def getNum(fmt,i):
    num=None
    cur = fmt[i]
    while cur == ' ':
        i += 1
        cur = fmt[i]
    while ('0'<= cur ) and ( cur <= '9'):
        if num == None:
            num = int(cur)
        else:
            num = 10*num + int(cur)
        i += 1
        cur = fmt[i]
    i += 1
    while i < len(fmt) and fmt[i] == ' ':
        i += 1
    return num,i,cur
```

And use it like this

```python
    while i<len(fmt):
        num,i,cur = getNum(fmt,i)
        [...]
```



From arigo at bitbucket.org  Mon Jun 17 10:28:20 2019
From: arigo at bitbucket.org (arigo at bitbucket.org)
Date: Mon, 17 Jun 2019 14:28:20 +0000 (UTC)
Subject: [pypy-issue] Issue #3027: list.pop(0) ten times slower than on
 CPython (pypy/pypy)
Message-ID: <20190617142820.5574.65851@celery-worker-111.ash1.bb-inf.net>

New issue 3027: list.pop(0) ten times slower than on CPython
https://bitbucket.org/pypy/pypy/issues/3027/listpop-0-ten-times-slower-than-on-cpython

Armin Rigo:

The following (bad) code runs 10 times slower on PyPy 2 or 3 than on CPython 2.x or 3.x.  Figure out why:

```
#!python

import time
lst = [[i, i] for i in range(62500)]
t = time.time()
while lst:
    lst.pop(0)
t2 = time.time()
print(t2 - t)
```



From jacklovell at bitbucket.org  Wed Jun 19 10:58:59 2019
From: jacklovell at bitbucket.org (jacklovell at bitbucket.org)
Date: Wed, 19 Jun 2019 14:58:59 +0000 (UTC)
Subject: [pypy-issue] Issue #3028: Assignment to element of 2D memoryview
 fails with NotImplementedError (pypy/pypy)
Message-ID: <20190619145859.7317.99166@celery-worker-109.ash1.bb-inf.net>

New issue 3028: Assignment to element of 2D memoryview fails with NotImplementedError
https://bitbucket.org/pypy/pypy/issues/3028/assignment-to-element-of-2d-memoryview

Jack Lovell:

When trying to assign to a single element in a 2D memoryview as in the attached example,  PyPy raises a NotImplementedError. CPython runs the example correctly. Reading the element works correctly in both CPython and PyPy.

Using PyPy 7.1.1-beta0



From fijal at bitbucket.org  Sat Jun 22 14:03:49 2019
From: fijal at bitbucket.org (fijal at bitbucket.org)
Date: Sat, 22 Jun 2019 18:03:49 +0000 (UTC)
Subject: [pypy-issue] Issue #3029: ll2ctypes (and possibly ctypes) does not
 work on pypy when running JIT tests (pypy/pypy)
Message-ID: <20190622180349.14490.50971@app-137.ash1.bb-inf.net>

New issue 3029: ll2ctypes (and possibly ctypes) does not work on pypy when running JIT tests
https://bitbucket.org/pypy/pypy/issues/3029/ll2ctypes-and-possibly-ctypes-does-not

Maciej Fijalkowski:

When running something like:

`pypy py.test rpython/jit/backend/x86/test/test_runner.py`

We get a crash somewhere in Boehm on OS X. I think it does not work on linux either, but did not check. This is a problem, because on a new OS X you can?t attach a debugger to a system binary. Which means pypy becomes a default choice for Python, which in turn ends up with weird crashes. At the very least test\_runner should clearly complain when run on top of PyPy.



From fijal at bitbucket.org  Sat Jun 22 14:05:41 2019
From: fijal at bitbucket.org (fijal at bitbucket.org)
Date: Sat, 22 Jun 2019 18:05:41 +0000 (UTC)
Subject: [pypy-issue] Issue #3030: Assertions in RPython callbacks are
 ignored when running tests (pypy/pypy)
Message-ID: <20190622180541.30244.68968@celery-worker-110.ash1.bb-inf.net>

New issue 3030: Assertions in RPython callbacks are ignored when running tests
https://bitbucket.org/pypy/pypy/issues/3030/assertions-in-rpython-callbacks-are

Maciej Fijalkowski:

An example is in `test_assembler_call` in `runner_test.py` - there is a line asserting value of `get_int_value`, but this would be ignored if tests are run. Hopefully it?ll be caught later that called has not been written to, but maybe not and maybe there are other cases. Can we have a systematic solution that catches all exceptions raised in rpython callbacks and crashes tests?



From issues-reply at bitbucket.org  Thu Jun 27 09:55:31 2019
From: issues-reply at bitbucket.org (Luanna)
Date: Thu, 27 Jun 2019 13:55:31 +0000 (UTC)
Subject: [pypy-issue] Issue #3031: math.gcd much slower than CPython
 (pypy/pypy)
Message-ID: <20190627135531.40880.49931@celery-worker-109.ash1.bb-inf.net>

New issue 3031: math.gcd much slower than CPython
https://bitbucket.org/pypy/pypy/issues/3031/mathgcd-much-slower-than-cpython

Luanna:

CPython uses Lehmer's GCD algorithm which is about ten times faster for sufficiently large values. A test that shows this:

```
from fractions import Fraction
sum(Fraction(1, n) for n in range(1, 10**4))
```

It takes 70 seconds in PyPy3 but only 3 seconds in CPython.

The good news is that Lehmer?s algorithm is easy to implement. CPython?s implementation is in [longobject.c](https://golang.org/src/math/big/int.go). There?s also an easier to read implementation [in Go](https://golang.org/src/math/big/int.go). For efficiency Lehmer?s algorithm needs to operate on machine words, so the implementation needs to be at the RPython level which I?m unfamiliar with. I can provide an app-level implementation if that is useful although it is only faster for very large values.

Please let me know if there?s any way I could help.



From issues-reply at bitbucket.org  Thu Jun 27 10:04:27 2019
From: issues-reply at bitbucket.org (Luanna)
Date: Thu, 27 Jun 2019 14:04:27 +0000 (UTC)
Subject: [pypy-issue] Issue #3032: 64bit Windows Support (pypy/pypy)
Message-ID: <20190627140427.24261.56865@app-137.ash1.bb-inf.net>

New issue 3032: 64bit Windows Support
https://bitbucket.org/pypy/pypy/issues/3032/64bit-windows-support

Luanna:

Are there any plans to eventually support 64 bit Windows? These days 4GB address space isn?t that much!



From issues-reply at bitbucket.org  Fri Jun 28 22:30:40 2019
From: issues-reply at bitbucket.org (Kang Seonghoon)
Date: Sat, 29 Jun 2019 02:30:40 +0000 (UTC)
Subject: [pypy-issue] Issue #3033: `0.5.__round__(None)` raises TypeError
 while CPython doesn't (pypy/pypy)
Message-ID: <20190629023039.8228.75701@celery-worker-109.ash1.bb-inf.net>

New issue 3033: `0.5.__round__(None)` raises TypeError while CPython doesn't
https://bitbucket.org/pypy/pypy/issues/3033/05__round__-none-raises-typeerror-while

Kang Seonghoon:

```python
>>> 0.5.__round__(None)
0

>>>> 0.5.__round__(None)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'NoneType' object cannot be interpreted as an integer
```

I'm not sure if this is a specified behavior of both CPython \(in that ndigits can be None in the `__round__` magic method\) and PyPy \(in that None counts as ?invalid arguments? as commented from [differences from CPython document](https://pypy.readthedocs.io/en/latest/cpython_differences.html)\).