multiprocessing+simplehttpserver+windows

[Nabla Kvadrat]

Additional information:
complete traceback:

Serving at http://localhost:8000 using 2 worker processes
To exit press Ctrl-Break
Traceback (most recent call last):
  File "C:\workspaces\workspace\skrserver-multiprocess\src\t.py", line
70, in <module>
    test()
  File "C:\workspaces\workspace\skrserver-multiprocess\src\t.py", line
65, in test
    runpool(ADDRESS, NUMBER_OF_PROCESSES)
  File "C:\workspaces\workspace\skrserver-multiprocess\src\t.py", line
49, in runpool
    Process(target=serve_forever, args=(server,)).start()
  File "C:\Python26\lib\multiprocessing\process.py", line 104, in start
    self._popen = Popen(self)
  File "C:\Python26\lib\multiprocessing\forking.py", line 239, in __init__
    dump(process_obj, to_child, HIGHEST_PROTOCOL)
  File "C:\Python26\lib\multiprocessing\forking.py", line 162, in dump
    ForkingPickler(file, protocol).dump(obj)
  File "C:\Python26\lib\pickle.py", line 224, in dump
    self.save(obj)
  File "C:\Python26\lib\pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "C:\Python26\lib\pickle.py", line 419, in save_reduce
    save(state)
  File "C:\Python26\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python26\lib\pickle.py", line 649, in save_dict
    self._batch_setitems(obj.iteritems())
  File "C:\Python26\lib\pickle.py", line 681, in _batch_setitems
    save(v)
  File "C:\Python26\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python26\lib\pickle.py", line 548, in save_tuple
    save(element)
  File "C:\Python26\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python26\lib\pickle.py", line 725, in save_inst
    save(stuff)
  File "C:\Python26\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python26\lib\pickle.py", line 649, in save_dict
    self._batch_setitems(obj.iteritems())
  File "C:\Python26\lib\pickle.py", line 681, in _batch_setitems
    save(v)
  File "C:\Python26\lib\pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "C:\Python26\lib\pickle.py", line 419, in save_reduce
    save(state)
  File "C:\Python26\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python26\lib\pickle.py", line 649, in save_dict
    self._batch_setitems(obj.iteritems())
  File "C:\Python26\lib\pickle.py", line 681, in _batch_setitems
    save(v)
  File "C:\Python26\lib\pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "C:\Python26\lib\pickle.py", line 419, in save_reduce
    save(state)
  File "C:\Python26\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python26\lib\pickle.py", line 649, in save_dict
    self._batch_setitems(obj.iteritems())
  File "C:\Python26\lib\pickle.py", line 681, in _batch_setitems
    save(v)
  File "C:\Python26\lib\pickle.py", line 313, in save
    (t.__name__, obj))
pickle.PicklingError: Can't pickle 'lock' object: <thread.lock object
at 0x00929370>
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Python26\lib\multiprocessing\forking.py", line 342, in main
    self = load(from_parent)
  File "C:\Python26\lib\pickle.py", line 1370, in load
    return Unpickler(file).load()
  File "C:\Python26\lib\pickle.py", line 858, in load
    dispatch[key](self)
  File "C:\Python26\lib\pickle.py", line 1133, in load_reduce
    value = func(*args)
  File "C:\Python26\lib\multiprocessing\reduction.py", line 167, in
rebuild_socket
    _sock = fromfd(fd, family, type_, proto)
  File "C:\Python26\lib\multiprocessing\reduction.py", line 156, in fromfd
    s = socket.fromfd(fd, family, type_, proto)
AttributeError: 'module' object has no attribute 'fromfd'

--------------
It seems that patch is available http://bugs.python.org/issue1378 but
not yet applied on the Windows latest release. I have no environment
to build it from source with this patch (in addition, I don't even
know how :-))

Thanks for any help.

On Thu, Dec 11, 2008 at 6:55 AM, Nabla Kvadrat <nabla.kvadrat at gmail.com> wrote:
> Hi, can someone please help me with/explain if following problem can
> be somehow resolved?
>
> I have Py2.6.1 win32 and I'm playing around with multiprocessing
> module. Trying examples here:
> http://docs.python.org/library/multiprocessing.html#examples. Example
> with SimpleHTTPServer.HttpServer don't work because of socket.fromfd
> in multiprocessing.reduction. If number of processes is >1 it ends
> with traceback ....... pickle.: 'module' object has no attribute
> 'fromfd'.
> As I understand socket.fromfd on windows is not implemented (?), so
> why the example script is care about platform
> if sys.platform == 'win32':
>    import multiprocessing.reduction
> and finally, is there some easy (no patching and recompiling python
> from source) way to get this example to work on win32 platform?
> Thanks for any help, I'm Python amateur...
> N.
>