[Tutor] How does pickle save files?

[Danny Yoo]

On Wed, 30 Nov 2005, Nathan Pinno wrote:

> In what format does pickle save files as? I mean it is .txt, .dat, or
> some other way?

Hi Nathan,

pickle actually has three formats: there's a textual format that looks
weird, and two binary formats that looks even weirder.  *grin*  You
should probably consider them all as ".dat", as the only consumer of
pickles is supposed to be the pickle module: the output is not
human-friendly.


There's some introductory information on the three formats here:

    http://www.python.org/doc/lib/node64.html

Protocol zero is textual, and the other two are binary.  None of them are
really meant to be read by humans, although Protocol 0's output is
somewhat comprehendable by looking at it.

######
>>> pickle.dumps(['hello', ('world',)], protocol=0)
"(lp0\nS'hello'\np1\na(S'world'\np2\ntp3\na."
>>> print pickle.dumps(['hello', ('world',)], protocol=0)
(lp0
S'hello'
p1
a(S'world'
p2
tp3
a.
######


Protocols one and two are less easy to read; we can't really print them
out since they don't fit ASCII, although we can see their byte
representation:

######
>>> pickle.dumps(['hello', ('world',)], protocol=1)
']q\x00(U\x05helloq\x01(U\x05worldq\x02tq\x03e.'
>>> pickle.dumps(['hello', ('world',)], protocol=2)
'\x80\x02]q\x00(U\x05helloq\x01U\x05worldq\x02\x85q\x03e.'
######


All three protocols can be inspected through the 'pickletools' module and
its disassembly function 'dis()':

    http://www.python.org/doc/lib/module-pickletools.html

We can learn about the gory details of pickle's format by studying the the
source code in 'pickletools.py':

    http://svn.python.org/projects/python/trunk/Lib/pickletools.py

That being said, pickle's format is a very low-level detail, and
understanding it also requires a bit of knowledge on virtual machines; I'm
not sure how well the details will make sense to you.  You probably don't
need to worry how Pickle writes its output unless you are very curious.


Best of wishes to you!