[concurrency] map error returning object

[Jay L.]

Richard,

I had not realized that the file descriptor was inherited by the child.  It
looks like PEP443 <http://www.python.org/dev/peps/pep-0433/> seeks to
overcome this with an optional flag in Python 3.3.  I wonder if something
similar does not exist for those of us 'stuck' in 2.x for a bit longer.

If anyone else has any ideas on the specifics of what is occurring, I would
love to understand the lower level details of what the concurrent
seeks/tells are doing to cause an overflow.

Thanks,
Jay

P.S. Many thanks for the traceback code.  This will likely become standard
in a good bit of my test code as debugging can sometimes be a pain.


On Wed, Jun 5, 2013 at 9:07 AM, Richard Oudkerk <shibturn at gmail.com> wrote:

> On 05/06/2013 3:30pm, Jay L. wrote:
>
>> I am working with the multiprocessing module attempting to get parallel
>> reads working.  Currently, if I open the file in each child process and
>> read from an offset I achieve concurrency.  If I attempt to open the
>> file in the parent (via a class) and pass the class object to the
>> children I get an overflow error.
>>
>
> If you inherit the parent process's fd, then (if I remember correctly) all
> the processes will share the same file offset: seeking in one process will
> affect the file offset in *all* processes.  It's not surprising that this
> causes problems, although I can't give a precise explanation of what is
> happening.
>
> BTW, the exception object returned to the main process will not have a
> traceback attached to it.  To see a full traceback you could catch, print
> and reraise the exception in the worker process:
>
>     def get_object(row):
>         try:
>             return file.get(row)
>         except:
>             # print exception with traceback then reraise
>             sys.excepthook(**sys.exc_info(**))
>             raise
> ______________________________**_________________
> concurrency-sig mailing list
> concurrency-sig at python.org
> http://mail.python.org/**mailman/listinfo/concurrency-**sig<http://mail.python.org/mailman/listinfo/concurrency-sig>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/concurrency-sig/attachments/20130605/c0bae455/attachment.html>