[ python-Bugs-1439659 ] file to store pickled object should be opened in binary mode

[SourceForge.net]

Bugs item #1439659, was opened at 2006-02-27 14:51
Message generated for change (Comment added) made by gbrandl
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1439659&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: Python 2.4
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Raghuram Devarakonda (draghuram)
Assigned to: Nobody/Anonymous (nobody)
Summary: file to store pickled object should be opened in binary mode

Initial Comment:

Hi,

To unpickle objects on UNIX that were pickled on
windows, the file should be opened in binary mode for
read and write on windows. It will be nice to have this
mentioned in the doc. Something along these lines will
be good enough:

"Please note that if you are writing the pickled object
to a file on windows, make sure it is opened in binary
mode. Otherwise, the file may not be able to be used to
unpickle on other non-windows platforms".

Thanks,
Raghu. 

----------------------------------------------------------------------

>Comment By: Georg Brandl (gbrandl)
Date: 2006-03-07 20:30

Message:
Logged In: YES 
user_id=849994

I recently added a note to the docs about this. I it's not
enough, feel free to reopen.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2006-03-06 19:48

Message:
Logged In: YES 
user_id=31435

Pickle files should be opened in binary mode, regardless of
pickle protocol, regardless of platform, and regardless of
whether reading or writing.  That protocol 0 used to be
called "text mode" was an historic mistake, and was given
that name by a Unix-head who wouldn't know the difference
between text and binary mode if it suck its teeth in their
posterior and tore off an entire bloody buttock :-)

----------------------------------------------------------------------

Comment By: Terry J. Reedy (tjreedy)
Date: 2006-03-06 19:03

Message:
Logged In: YES 
user_id=593130

Any text file written with /r/n in Windows can have 
problems when read on *nix or Mac or ???, so this problem 
is not specific to pickle or even to Python.  I am under 
the impression that the new universal newline support was 
intended to fix such problems (when used).  But as a Win-
only user (currently) I have no experience with it.

----------------------------------------------------------------------

Comment By: Raghuram Devarakonda (draghuram)
Date: 2006-03-06 15:12

Message:
Logged In: YES 
user_id=984087


There seems to be a mistane in my earlier comment. I meant
the unpickling on *UNIX* fails even if ASCII was used on
windows.

Raghu.

----------------------------------------------------------------------

Comment By: Raghuram Devarakonda (draghuram)
Date: 2006-03-06 15:11

Message:
Logged In: YES 
user_id=984087


Hi,

The point is that even if ASCII protocol is used on
windows(as I was), the unpickling process would fail on
windows. So the pickle file should be opened in binary mode
on windows even for ASCII protocol. This may make reading
the file in notepad difficult but then, cross platform
support should take precedence, I suppose. Another option
would be for python to do line end translations for ASCII
pickle files. 

Raghu.


----------------------------------------------------------------------

Comment By: Terry J. Reedy (tjreedy)
Date: 2006-03-06 03:36

Message:
Logged In: YES 
user_id=593130

For ASCII protocol pickles, intended for human 
readability, one should use text mode to be able to read 
the file, for instance, in Notepad.

For either binary protocol, I would think binary mode 
should be best even to reread on Windows.  If anything is 
added, I would put
"On Windows, open files in a mode (text/binary) that 
matches the protocol."
at the bottom of http://docs.python.org/lib/node64.html

Looking forward to 3.0, when file mode might be more 
significant (text/binary corresponding to unicode/bytes), 
the "for windoews' qualifier might be omitted, but this 
might be premature.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1439659&group_id=5470