Pickled objects over the network

[Walker Lindley]

I'm working on a distributed computing program and need to send Python
objects over a TCP socket. Specifically, the objects that I'm working with
subclass the builtin list type (I don't know whether or not that matters),
but also include other data fields. These objects are put into dictionaries
along with some other control information and the dictionaries are pickled
and sent across the network. I'm using the makefile() function in socket to
get a file descriptor for the socket and then calling pickle.dump with the
dictionary object and the socket file descriptor. I was originally using
pickle protocol 0, but then read about the possibility of buffering problems
and realized that protocol 0 was giving me strings about 1500 characters
long. After some testing, I settled on protocol 2 which manages to package
the data into about 400 characters.

In any case, the client side of the program appears to send the data just
fine, at least as far as I can tell. On the server side, however, the data
takes a very long time to get there and then I get an error in the pickle
module:
Traceback (most recent call last):
  File "server.py", line 176, in <module>
    serv.runServer()
  File "server.py", line 99, in runServer
    recvPacket = pickle.load(clientFile)
  File "/usr/lib/python2.5/pickle.py", line 1370, in load
    return Unpickler(file).load()
  File "/usr/lib/python2.5/pickle.py", line 858, in load
    dispatch[key](self)
  File "/usr/lib/python2.5/pickle.py", line 1187, in load_appends
    mark = self.marker()
  File "/usr/lib/python2.5/pickle.py", line 874, in marker
    while stack[k] is not mark: k = k-1
IndexError: list index out of range

Hopefully I'm doing something obviously wrong, but if anyone can help based
on that description or if you need to see the source, please let me know
(it's GPL'd). Thank you so much for any help.



-Walker Lindley

-- 
This e-mail is licensed under the Creative Commons
Attribution-NoDerivs 2.5License. To view a copy of this license, visit
http://creativecommons.org/licenses/by-nd/2.5/ or send a letter to Creative
Commons, 543 Howard Street, 5th Floor, San Francisco, California, 94105,
USA.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20070717/96933c88/attachment.html>