unpickle from URL problem

[Hrvoje Niksic]

Alan Isaac <aisaac at american.edu> writes:

> I upload to a server.
> I try to unpickle from the URL.  No luck.  Try it:
> x1, x2 = pickle.load(urllib.urlopen('http://www.american.edu/econ/notes/hw/example1'))
>
> I change the filetype to unix.  I upload again.
> I try to unpickle from the URL.  Now it works.  Try it:
> x1, x2 = pickle.load(urllib.urlopen('http://www.american.edu/econ/notes/hw/example2'))
>
> Why the difference?

The first upload breaks the file.  You uploaded it in (presumably
FTP's) text mode, which changes \n -> \r\n.  But you download it using
http, which specifies no such conversion in the opposite direction.

It is true that the lowest pickle protocol is textual in nature, but
it just means that you can open a file in text mode, write the pickle
into the file, and reliably read it back on the same platform.  The
newlines will be converted to the platform newline representation
(such as \r\n on Windows or \r on Mac) upon write, and back to the
original single-byte (\n) representation upon read.  But if you feed
the unpickler a string that already contains \r\n, there is nothing to
convert it back to \n.