Traceback when using multiprocessing, less than helpful?

[Terry Reedy]

On 11/21/2013 12:01 PM, John Ladasky wrote:

This is a case where you need to dig into the code (or maybe docs) a bit

> File ".../evaluate.py", line 81, in evaluate
 >   result = pool.map(evaluate, bundles) File
> "/usr/lib/python3.3/multiprocessing/pool.py", line 228, in map
 >   return self._map_async(func, iterable, mapstar, chunksize).get()

The call to _map_async gets a blank MapResult (a subclass of 
ApplyResult), queues tasks to fill it in, and returns the filled in 
result. This call is designed to always return as task exceptions are 
caught and assigned to MapResult._value in both ApplyResult._set and 
MapResult._set.

result = MapResult(self._cache, chunksize, len(iterable), callback,
                            error_callback=error_callback)
self._taskqueue.put((((result._job, i, mapper, (x,), {})
                            for i, x in enumerate(task_batches)), None))
return result

It is the subsequent call to get() that 'fails', because it raises
the caught exception.

 > File "/usr/lib/python3.3/multiprocessing/pool.py", line 564, in get
 >   raise self._value

ValueError: operands could not be broadcast together with shapes (1,3) (4)

> Notice that no line of numpy appears in the traceback?  Still, there
> are three things that make me think that this error is coming from
> numpy.

It comes from one of your tasks as the 'result', and your tasks use numpy.

> If I could only see the line of numpy code which is generating the
> ValueError, I would have a better chance of spotting the bug in my
> code.

Definitely.

 > So, WHY isn't there any reference to numpy in my traceback?

I suspect that raising the exception may replace its __traceback__ 
attribute.  Anyway, there are three things I might try.

1. Use 3.3.3 or latest 3.4 to see if there is any improvement in output. 
I vaguely remember a tracker issue that might be related.

2. _map_async takes an error_callback arg that defaults to None and 
which is passed on to MapResult. When _value is set to an exception, 
"error_callback(_value)" is called in ._set() before the later .get() 
re-raises it. pool.map does not allow you to set either the (success) 
callback or the error_callback, but pool.map_async does (this is the 
difference between the two methods). So switch to the latter so you can 
pass a function that uses the traceback module to print (or log) the 
traceback attached to _value, assuming that there is one.

3. If that does not work, wrap the current body of your task function in
try: <current suite>
except exception as e:
   <use traceback module to add traceback to message>
   raise e <or a new exception>

-- 
Terry Jan Reedy