[pypy-issue] Issue #2830: multiprocessing fails with "expected integer, got NoneType object" (pypy/pypy)
Mike Auty
issues-reply at bitbucket.org
Sun May 13 14:04:31 EDT 2018
New issue 2830: multiprocessing fails with "expected integer, got NoneType object"
https://bitbucket.org/pypy/pypy/issues/2830/multiprocessing-fails-with-expected
Mike Auty:
Hi there,
I haven't been able to minimize the test condition, and I can only provide the rough outline of the code, but hopefully the location of the error will help identify it. The exception traceback is:
```
#!python
Traceback (most recent call last):
File "/usr/lib64/pypy3/lib-python/3/multiprocessing/pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "/usr/lib64/pypy3/lib-python/3/multiprocessing/pool.py", line 44, in mapstar
return list(map(*args))
File "/usr/lib64/pypy3/lib_pypy/_functools.py", line 80, in __call__
return self._func(*(self._args + fargs), **fkeywords)
File "/path/to/scan_test.py", line 226, in _scan_chunk
progress.value = chunk_end
File "/usr/lib64/pypy3/lib-python/3/multiprocessing/managers.py", line 1034, in set
return self._callmethod('set', (value,))
File "/usr/lib64/pypy3/lib-python/3/multiprocessing/managers.py", line 716, in _callmethod
conn.send((self._id, methodname, args, kwds))
File "/usr/lib64/pypy3/lib-python/3/multiprocessing/connection.py", line 206, in send
self._send_bytes(ForkingPickler.dumps(obj))
File "/usr/lib64/pypy3/lib-python/3/multiprocessing/connection.py", line 404, in _send_bytes
self._send(header + buf)
File "/usr/lib64/pypy3/lib-python/3/multiprocessing/connection.py", line 368, in _send
n = write(self._handle, buf)
TypeError: expected integer, got NoneType object
```
In this instance, progress.value is `multiprocessing.Manager().Value("Q", 0)` and the multiprocessing code looks like:
```
#!python
with multiprocessing.Pool() as pool:
result = pool.map_async(scan_chunk, scan_iterator())
while not result.ready():
result.wait(0.1)
for value in result.get():
yield from value
```
Some information that might help is that if I insert a print statement in the scan_chunk method, it ends up running fine, suggesting it's a timing condition of some kind. It looks as though the handle's been closed, but I don't know enough about the code to figure out why. Please let me know if you need more information and I'd be happy to provide to provide it if I'm allowed. I'm also happy to test patches against 6.0.0 if that'd help...
Pypy --version:
Python 3.5.3 (fdd60ed87e941677e8ea11acf9f1819466521bf2, Apr 27 2018, 14:54:57)
[PyPy 6.0.0 with GCC 7.3.0]
More information about the pypy-issue
mailing list