[issue12996] multiprocessing.Connection endianness issue

[Charles-François Natali]

Charles-François Natali <neologix at free.fr> added the comment:

> "Since the rewrite in pure Python of multiprocessing.Connection (issue #11743), multiprocessing.Connection sends and receives the length of the data (used as header) in host byte order."
>
> I don't think so, the C code uses also the host endian. This issue is a feature request.
>

No.
http://hg.python.org/cpython/file/5deecc04b7a2/Modules/_multiprocessing/socket_connection.c
In conn_send_string():
"""
     /* The "header" of the message is a 32 bit unsigned number (in
        network order) which specifies the length of the "body".  If
        the message is shorter than about 16kb then it is quicker to
        combine the "header" and the "body" of the message and send
        them at once. */
[...]
         *(UINT32*)message = htonl((UINT32)length);
"""

in conn_recv_string():
"""
     ulength = ntohl(ulength);
"""

> I don't know if anyone uses multiprocessing on different hosts (because it doesn't work currently).
>
> If you would like to support using multiprocessing on different hosts, it should be documented in multiprocessing doc.

It does work, it's even documented ;-)

http://docs.python.org/dev/library/multiprocessing.html#multiprocessing-managers
"""
A manager object returned by Manager() controls a server process which
holds Python objects and allows other processes to manipulate them
using proxies.
[...]
Server process managers are more flexible than using shared memory
objects because they can be made to support arbitrary object types.
Also, a single manager can be shared by processes on different
computers over a network. They are, however, slower than using shared
memory.
"""

Managers use multiprocessing.connection to serialize data and send
them over a socket:
http://hg.python.org/cpython/file/5deecc04b7a2/Lib/multiprocessing/managers.py
"""
#
# Mapping from serializer name to Listener and Client types
#
listener_client = {
     'pickle' : (connection.Listener, connection.Client),
     'xmlrpclib' : (connection.XmlListener, connection.XmlClient)
     }
"""

Yeah, Python's awesome :-)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue12996>
_______________________________________