[Python-Dev] when is binary mode required for pickle?

[Guido van Rossum]

> Unused as I am to programming on Windows, it's not clear to me under what
> circumstances I might need to open a file in binary mode for Pickle's
> consumption.  The reason I ask is that while working on the csv PEP and
> module, we had some input complaining about our requirement that CSV files
> be opened with the 'b' flag (where it matters).  Andrew mentioned that
> Pickle requires files be opened in binary mode.  However, the documentation
> doesn't explicitly state this.  Must, for example, a file be opened with the
> 'b' flag if an ASCII pickle is to be written?  The doc for the Unpickler
> class says the file-like object must support both read(n) and readline()
> methods.  The requirement for readline() suggests the file be opened in text
> mode.

The situation for pickling is complex.  Pickling protocol 0 *allows*
opening the file in text mode, as long as this is done both for
reading and for writing.  Pickling protocol 1 (AKA known as "binary
mode") and protocol 2 (new in Python 2.3) *require* opening the file
in binary mode (but nobody ever checks).

There's no requirement that files are opened in text mode to use
readline() -- but when a file was written in text mode and then
readline() is used in binary mode, all your line endings will look
like "\r\n".

> (Maybe this is just a nit with the Pickle docs and I don't know it?)

Who knows.

--Guido van Rossum (home page: http://www.python.org/~guido/)