[pypy-issue] [issue1484] numpy pickling

[mikefc]

New submission from mikefc <coolbutuseless at gmail.com>:

Trying to use a multiprocessing.Pool() to map a function onto a list of numpy 
arrays fails in latest pypy nightly (OSX).

Code attached. 

In python 2.7.x:
Not MP: [0.0, 0.0, 0.0, 0.0]
MP: [0.0, 0.0, 0.0, 0.0]

In pypy (OSX nightly latest)
Not MP: [0.0, 0.0, 0.0, 0.0]
<Float64 object at 0x101755110>
<Float64 object at 0x101755110>
Process PoolWorker-1:
Traceback (most recent call last):
  File "/Users/mike/python/pypy-latest/lib-python/2.7/multiprocessing/process.py", 
line 258, in _bootstrap
    self.run()
  File "/Users/mike/python/pypy-latest/lib-python/2.7/multiprocessing/process.py", 
line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/mike/python/pypy-latest/lib-python/2.7/multiprocessing/pool.py", 
line 99, in worker

----------
files: bugreport.txt
messages: 5710
nosy: mikefc, pypy-issue
priority: bug
status: unread
title: numpy pickling

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1484>
________________________________________
-------------- next part --------------
Not MP: [0.0, 0.0, 0.0, 0.0]
<Float64 object at 0x101755110>
<Float64 object at 0x101755110>
Process PoolWorker-1:
Traceback (most recent call last):
  File "/Users/mike/python/pypy-latest/lib-python/2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/Users/mike/python/pypy-latest/lib-python/2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/mike/python/pypy-latest/lib-python/2.7/multiprocessing/pool.py", line 99, in worker
Process PoolWorker-2:
Traceback (most recent call last):
  File "/Users/mike/python/pypy-latest/lib-python/2.7/multiprocessing/process.py", line 258, in _bootstrap
    put((job, i, result))
  File "/Users/mike/python/pypy-latest/lib-python/2.7/multiprocessing/queues.py", line 390, in put
    return send(obj)
  File "/Users/mike/python/pypy-latest/lib-python/2.7/pickle.py", line 1417, in dumps
    self.run()
  File "/Users/mike/python/pypy-latest/lib-python/2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/mike/python/pypy-latest/lib-python/2.7/multiprocessing/pool.py", line 99, in worker
    Pickler(file, protocol).dump(obj)
  File "/Users/mike/python/pypy-latest/lib-python/2.7/pickle.py", line 224, in dump
    self.save(obj)
    put((job, i, result))
  File "/Users/mike/python/pypy-latest/lib-python/2.7/pickle.py", line 286, in save
  File "/Users/mike/python/pypy-latest/lib-python/2.7/multiprocessing/queues.py", line 390, in put
    f(self, obj) # Call unbound method with explicit self
  File "/Users/mike/python/pypy-latest/lib-python/2.7/pickle.py", line 548, in save_tuple
    return send(obj)
  File "/Users/mike/python/pypy-latest/lib-python/2.7/pickle.py", line 1417, in dumps
    save(element)
  File "/Users/mike/python/pypy-latest/lib-python/2.7/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/Users/mike/python/pypy-latest/lib-python/2.7/pickle.py", line 548, in save_tuple
    save(element)
  File "/Users/mike/python/pypy-latest/lib-python/2.7/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/Users/mike/python/pypy-latest/lib-python/2.7/pickle.py", line 600, in save_list
    self._batch_appends(iter(obj))
  File "/Users/mike/python/pypy-latest/lib-python/2.7/pickle.py", line 636, in _batch_appends
    save(tmp[0])
  File "/Users/mike/python/pypy-latest/lib-python/2.7/pickle.py", line 306, in save
    rv = reduce(self.proto)
TypeError: unsupported operand type for __getnewargs__: 'float64'
    Pickler(file, protocol).dump(obj)
  File "/Users/mike/python/pypy-latest/lib-python/2.7/pickle.py", line 224, in dump
    self.save(obj)
  File "/Users/mike/python/pypy-latest/lib-python/2.7/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/Users/mike/python/pypy-latest/lib-python/2.7/pickle.py", line 548, in save_tuple
    save(element)
  File "/Users/mike/python/pypy-latest/lib-python/2.7/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/Users/mike/python/pypy-latest/lib-python/2.7/pickle.py", line 548, in save_tuple
    save(element)
  File "/Users/mike/python/pypy-latest/lib-python/2.7/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/Users/mike/python/pypy-latest/lib-python/2.7/pickle.py", line 600, in save_list
    self._batch_appends(iter(obj))
  File "/Users/mike/python/pypy-latest/lib-python/2.7/pickle.py", line 636, in _batch_appends
    save(tmp[0])
  File "/Users/mike/python/pypy-latest/lib-python/2.7/pickle.py", line 306, in save
    rv = reduce(self.proto)
TypeError: unsupported operand type for __getnewargs__: 'float64'
MP:<Float64 object at 0x101755110>
Process PoolWorker-3:
<Float64 object at 0x101755110>
Traceback (most recent call last):
  File "/Users/mike/python/pypy-latest/lib-python/2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/Users/mike/python/pypy-latest/lib-python/2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/mike/python/pypy-latest/lib-python/2.7/multiprocessing/pool.py", line 99, in worker
    put((job, i, result))
  File "/Users/mike/python/pypy-latest/lib-python/2.7/multiprocessing/queues.py", line 390, in put
Process PoolWorker-4:
Traceback (most recent call last):
  File "/Users/mike/python/pypy-latest/lib-python/2.7/multiprocessing/process.py", line 258, in _bootstrap
    return send(obj)
  File "/Users/mike/python/pypy-latest/lib-python/2.7/pickle.py", line 1417, in dumps
    self.run()
  File "/Users/mike/python/pypy-latest/lib-python/2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/mike/python/pypy-latest/lib-python/2.7/multiprocessing/pool.py", line 99, in worker
    Pickler(file, protocol).dump(obj)
  File "/Users/mike/python/pypy-latest/lib-python/2.7/pickle.py", line 224, in dump
    self.save(obj)
  File "/Users/mike/python/pypy-latest/lib-python/2.7/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/Users/mike/python/pypy-latest/lib-python/2.7/pickle.py", line 548, in save_tuple
    save(element)
  File "/Users/mike/python/pypy-latest/lib-python/2.7/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/Users/mike/python/pypy-latest/lib-python/2.7/pickle.py", line 548, in save_tuple
    save(element)
  File "/Users/mike/python/pypy-latest/lib-python/2.7/pickle.py", line 286, in save
    put((job, i, result))
    f(self, obj) # Call unbound method with explicit self
  File "/Users/mike/python/pypy-latest/lib-python/2.7/pickle.py", line 600, in save_list
  File "/Users/mike/python/pypy-latest/lib-python/2.7/multiprocessing/queues.py", line 390, in put
    self._batch_appends(iter(obj))
  File "/Users/mike/python/pypy-latest/lib-python/2.7/pickle.py", line 636, in _batch_appends
    save(tmp[0])
  File "/Users/mike/python/pypy-latest/lib-python/2.7/pickle.py", line 306, in save
    rv = reduce(self.proto)
TypeError: unsupported operand type for __getnewargs__: 'float64'
    return send(obj)
  File "/Users/mike/python/pypy-latest/lib-python/2.7/pickle.py", line 1417, in dumps
    Pickler(file, protocol).dump(obj)
  File "/Users/mike/python/pypy-latest/lib-python/2.7/pickle.py", line 224, in dump
    self.save(obj)
  File "/Users/mike/python/pypy-latest/lib-python/2.7/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/Users/mike/python/pypy-latest/lib-python/2.7/pickle.py", line 548, in save_tuple
    save(element)
  File "/Users/mike/python/pypy-latest/lib-python/2.7/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/Users/mike/python/pypy-latest/lib-python/2.7/pickle.py", line 548, in save_tuple
    save(element)
  File "/Users/mike/python/pypy-latest/lib-python/2.7/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/Users/mike/python/pypy-latest/lib-python/2.7/pickle.py", line 600, in save_list
    self._batch_appends(iter(obj))
  File "/Users/mike/python/pypy-latest/lib-python/2.7/pickle.py", line 636, in _batch_appends
    save(tmp[0])
  File "/Users/mike/python/pypy-latest/lib-python/2.7/pickle.py", line 306, in save
    rv = reduce(self.proto)
TypeError: unsupported operand type for __getnewargs__: 'float64'